Welcome to the Q&A Forum

If you have the check my links extension in Chrome, it makes it easy to find URLs that 404 on any page.  You can install it here: https://chrome.google.com/webstore/detail/ojkcdipcgfaekbeaelaapakgnjflfglf.

Excel has a feature called advanced filters that I use to filter these out.  See here for more info: http://office.microsoft.com/en-us/excel-help/filter-by-using-advanced-criteria-HP005200178.aspx.  It allows you to filter your data by multiple criteria.  You might have to break up your regex a bit into multiple different filters, but you should be able to get it done with this.  They support * and ?.

If you use Google Analytics, you could view this by setting up an advanced filter with a regex to separate branded vs. non-branded paid search queries.

In Google Webmaster Tools, you can go in, and go to Diagnostics > Fetch as Googlebot.  Once Googlebot fetches the page, you can click the "Submit to index" link.

In Bing Webmaster Central, go to Index > Submit URLs, and you can submit the page to be re-indexed there.

With the AJAX crawlability guide implementation, Google knows they're requesting a different page than the one being shown to users, so it's not quite the same as cloaking.  That being said, you could go black hat and return a completely different page, but Google has their ways of finding these things out.

I think you mean JavaScript and not Java.  What you're suggesting is what Google recommends in their AJAX crawling guide here http://code.google.com/web/ajaxcrawling/.  They want you to create a static HTML page to serve to Googlebot instead of your regular page.

Google is getting better at crawling JavaScript content that's loaded asynchronously, so you might want to dedicate your resources elsewhere.  On one of my sites, Google is indexing text that's loaded asynchronously (Bing isn't yet), and Matt Cutts has said that Google is crawling some comments that are loaded asynchronously, like Facebook comments (see http://www.searchenginejournal.com/google-indexing-facebook-comments/35594/)

They're indexing AJAX comments for sure (e.g. Facebook comments - see http://www.searchenginejournal.com/google-indexing-facebook-comments/35594/), and on one of my sites where the main page content is loaded via AJAX, Google is indexing the text, because when I search for that exact text, it's returning that page. This was nice, because my developers weren't looking forward to having to implement the HTML snapshots as Google suggests in http://code.google.com/web/ajaxcrawling/.

So, Google is already doing it, and presumably they'll only be getting better at it.  That same search that works in Google doesn't work in Bing, so I think Bing is behind with crawling AJAX content.

If the content is too late, you're right, the Googlebot may not grab it. However, Google is getting better and better at indexing AJAX content that's loaded after the fact. On one of the sites I work on, we really didn't want to go through the whole process of serving up an HTML snapshot to Googlebot (outlined http://code.google.com/web/ajaxcrawling/).  About a month ago, I did a search in Google based on the AJAX content, and it returned the page, meaning Google is finding that AJAX content and indexing it!  They're indexing comments now (see http://www.searchenginejournal.com/google-indexing-facebook-comments/35594/) as well, like Disqus and Facebook comments.  What kind of comments widget are you loading that Google can't get at?  Maybe they'll be able to index them soon?

I would guess that Google would devalue

<noscript>text, as almost everyone has JavaScript enabled. Otherwise, everyone would be keyword stuffing in their <noscript> tags.</p> <p style="color: #5e5e5e;">The option you outlined sounds like it could work. If you're just taking the content from JavaScript, and loading it in the HTML if the user doesn't have JavaScript enabled.  Google is actually suggesting in their ajax crawling guide to actually serve the Googlebot a static page instead of the page with AJAX content, which seems much closer to cloaking than the option you're suggesting.</p></noscript>

That's not really necessary unless there URLs or directories you're disallowing after the allow in your robots.txt. Allow is a directive supported by major search engines, but search engines assume they're allowed to crawl everything they find unless you disallow it specifically in your robots.txt.

The following is universally accepted by bots and essentially means the same thing as what I think you're trying to say, allowing bots to crawl everything:

User-agent: *
Disallow:

There's a sample use of the Allow directive on the wikipedia robots.txt page here.

Take a look at this Whiteboard Friday; Cyrus can answer your question better than I can: http://www.seomoz.org/blog/beyond-exact-match-anchor-text-to-next-generation-link-signals-whiteboard-friday

If you meta noindex, follow these pages, they won't appear on search result pages, but they'll still pass link juice.  It won't increase the amount of link juice any of your indexed pages are getting however, so I'm not sure why you'd want to do that unless you want those pages removed from search engine indexes.

The search engines have gotten rid of link juice sculpting.  Your next question might be whether or not you should start nofollowing links to your own pages to try to pass more link juice to important pages.  That won't work either, as that only serves to burn the link juice the links would pass, it doesn't cause the followed links to pass more.

This may be obvious but I'd start with Adwords.  They have a huge reach over their display network, and the most control over your site retargeting campaigns.  You can create different ad groups depending on whether the user visited different pages, or have been away from your site for different durations of time (so you can control the ad text and bids for longer intervals).

We've also used BurstMedia and they've been able to hit our CPA targets.  I'm still looking into trying Adroll, Retargeter, Adbrite, Valueclick, Criteo, and Adify (Cox Digital Solutions now?) to name a few.  Have you tried any of those (to turn your question back on to you)?

If you have any questions about retargeting in Adwords, let me know.

Yeah, that's a problem.  That means if people put links to http://mydomain.co.uk, they won't be passing their link juice on to http://www.mydomain.co.uk if the redirect is a 302 (temporary) redirect.  You should configure this to be a 301 (permanent) redirect.

Reddit's self-service ad platform is pretty easy and intuitive to use.  I like it because users can comment on your ads and you can reply, making it easy to engage with the community.

As far as its paid discovery goes, we didn't have much success with conversions.  It's great for getting people to your site, but since people just stumble from site to site, our traffic from them had a really high bounce rate, and not very many conversions.  I believe they've come out with better targeting options since I've tried it, so I can't speak to those.

Here's a post from someone who had a spike of traffic come from StumbleUpon.  The tl;dr; the author gives there is "From this, I’d sum up that, while StumbleUpon users are just as likely to stay to read as anyone else, they will probably leave after they've read the stumbled page, and are not likely to engage further with the site."

According to this infographic here, Stumbleupon is pretty huge traffic-wise.

Also, Digg is dying a slow death.  Reddit is where it's at.

Eventually, but that's the code Google recommends to return when your site is having downtime, so I would expect them to be more lenient towards not removing things right away.  I wouldn't expect it to be as efficient as returning a 404 or a 410.

The best way to get content de-indexed is to return a page with a meta noindex tag on it, if you're really keen on getting it removed immediately.

Google. Pick a few sentences or phrases in the article, and Google them. If you put quotes around them, Google will only return results with exact phrase matches. If any of the phrases show results, look and see if the article is the same.

Nofollowing links doesn't sculpt link juice, it burns it.  Doing this to internal links is bad, as you're wasting your own link juice!  If I don't want a page on my site on the SERPs, I'd much rather set a meta noindex, follow tag on the page I don't want indexed rather than nofollow links to it.

To explain burning vs. sculpting, here's an example.  Say page A has 6 votes of link juice to pass, and links to 3 other pages, B, C, and D.  If all the links are followed, it'll pass 2 votes to each page.  However, if you nofollow the link to page D, it won't give 3 votes to B and C, it'll give the same 2 votes to B and C, and the 2 votes that should have gone to page D go nowhere.  They're burned.

If instead you set a meta tag on page D to noindex, follow, the link juice will flow in and out of page D, so that would be much preferred.

The ads that rank best may appear at the top of the page.  In the screenshot you've attached, the top result there with the orange background is an Office Depot ad.  Branded searches rank very well, so you'll usually see that companies result up there at the top of the SERPs.

In my experience, bidding on competitors branded terms does not convert well.  People are looking for that site, and not your site.  My experience is not in e-commerce though; if you sell shoes, and you bid on "buy nikes", you're may be doing it right.

First, in answer to your questions:

1. There are no different policies other than what you stated above.  You can not use a companies trademarked name in ad text, but you can use their names in keywords.
2. They don't block them.  By entering the auction, they'll drive up the price for their competitors.  Since their quality scores are higher, they'll win the auctions with cheaper bids.

If your sales are pretty steady, you can run a test to see if running a branded campaign bidding on your own name is worthwhile.  Your organic traffic will go down a bit, but since people are looking for your company, your keyword quality scores will be very high, and you should be winning the auctions with relatively cheap bids. If anyone else is bidding on your brand keywords, it's almost always a good idea for you to be in there as well.  Otherwise, it's worth testing to see if it has a positive ROI.

Your examples are a bit flawed... a query like "Nallys used cars" is going to trigger for all broad and phrase keywords targeting "used cars", which many companies are going to target.  Also "la jolla cosmetic surgery centre" has "cosmetic surgery" which lots of people are bidding on.

Also, I saw ads for all of your examples above which you didn't see ads for (Best Buy, Victoria's Secret, and Office Depot).

Are you in Austrailia?  We have IP based redirection in place to redirect Austrailians to the Aussie version of our site (au.ixl.com) via a 302.  I still don't know why Alexa would be displaying that as the title of our site...