Welcome to the Q&A Forum

WMT is alright, apart from the fact you can't force Google to crawl all your pages. I would doubt that even a majority of the pages were crawled and indexed by Google (though I don't know what the site is).

Plus, as you say, it only deals with internal links and 404s coming in.

Do you know what the upper limit is on how many crawl errors WMT will display?

You might be out of luck on a site that size.

I think WebCEO can do this with their online version but to get 100,000 urls crawled I think it'll cost you a bomb (the sort of money that it'd be cheaper to buy a second PC to run Xenu, lol).

Anyway - http://www.webceo.com/ - I think it may also be possible to install the download version to a server and run it that way.

I've tried to see if I can replicate it with any of my sites, but both the www and non-www site: searches always return the homepage.

Has there been any changes in ranking or traffic/keywords sending you visitors? (Or basically, why is it that this worries you :D)

Do the subpages showing above your homepage have a greater link popularity? Been building deep links or are they popular content?

If I understand correctly I'm going to say no (with a low degree of certainty :D).

So /store/product-old1 you are assuming already goes through /store/index.php?id=product-old1 (or similar) to produce that first url?

So to get to /store/product-new1 from a link to /store/product-old1 you're wondering how many steps your .htaccess puts it through?

I'll have to let somebody with better .htaccess skills than mine confirm, but I think it's all resolved as one, in sequence, if it's on the same server (and set up correctly). I'm maybe talking nonsense though.

However if it was a chain then, for example, somebody going to my WordPress site using mysite.com/page1 would end up at www.mysite.com/redirected-page1/ and after hitting about 4 redirects (www, permalinks, redirect and trailing slash) and I've not seen anything to suggest that it's a problem.

Thought I'd have a look, there's one site with just a link from that page (and the 4 variations of it, PR 5-7) and it has a PR4 itself, so it's almost definitely passing juice along.

Hmm, can't replicate it and just ran your site through Xenu to crawl it and it would appear it was just me navigating using the address bar.

I doubt many people will be doing that, though I often change through sections on ecommerce sites directly.

However it does show up things like http://www.rainchainsdirect.com/collections/all-products/products/large-copper-pails-rain-chain and http://www.rainchainsdirect.com/collections/styled-cup-designs/products/large-copper-pails-rain-chain

So rel=canonical or maybe a redirect on the product pages wouldn't hurt

Are you likely to get a lot more pages/products?

If not having everything accessible via drop downs would be my preference, not least of all as a user because if you make me click to a page to get to another page I'm not going to be sticking around long and search engines do want you to think about the users.

This is replicated in search engines by pages you have to click twice to get to being further down the link juice chain even if it's the next subfolder down in terms of URLs.

Actually looking at it, is is possible to get to the pages using different routes? http://www.rainchainsdirect.com/products/copper-blossoms-rain-chain-patina and http://www.rainchainsdirect.com/collections/floral-designs/products/copper-blossoms-rain-chain-patina would appear to be the same.

There's no canonical tag so actually I'd have a think about my site structure in general (appreciate it can be hard with ecommerce sites). Here's an interesting YouMOZ post - http://www.seomoz.org/ugc/the-effect-of-site-structure-on-organic-traffic-add-categories-ampamp-products.

Also, from a conversion point of view I may make a link to my top level categories of products accessible from all pages.

You've given everything away! I know where the poison is!

Well it is a big PR 7 link, though there are already a lot of links on the page. I think it would grant a level of authority you couldn't otherwise buy but, I've never thought about it really. I think I might depending on what the 'cost' was.

Sponsored links on directories may mean that you get up a category or two from normal (from /cat/sub-cat/location/site to /cat/ say) and as such more likely to get indexed on that site.

Sponsored posts may have the blog they're on devalued or links not pass juice but really shouldn't be a big risk (in so far as getting a penalty) to you, but may end up being a waste of money.

So top nav and sidebar nav duplicates for example.

Hard to tell; generally Google will only count anchor text of the first link to a certain page - http://www.seomoz.org/blog/results-of-google-experimentation-only-the-first-anchor-text-counts - so we could surmise that the second example on a page definitely isn't counted as a full link.

Also here's some info from a recent YouMOZ post - http://www.seomoz.org/ugc/the-nature-of-the-100link-limit-of-google

I also think Google can often differentiate repeated nav links and doesn't value them as much, but I don't have anything to back that up with.

Basically, I think you'll be alright as Google will filter them before deciding which to crawl.

Not cheap but you could maybe get a lot out of - http://www.syntryx.com/

It'll give you IP, other sites that person owns, what vertical they're in, strength of sites and dozens of other things.

I dunno if a mod will stop by but I imagine there will be a lot of requests to 'out' other sites (not saying I've found anything dodgy); are there any guidelines on reporting competitors dodgy tactics as I know mostly seoMoz and other big sites usually avoid public outings and would there be any liability should the site complain?

Just thinking aloud.

Hi Ryan,

not really.

This should explain - http://www.seomoz.org/blog/link-consolidation-the-new-pagerank-sculpting

Basically even if you nofollow the link you still pass the juice through it, except it just evaporates

So you'll be better off working on navigation structure and consolidation than trying to nofollow to pass PR.

However, in terms of when to nofollow, use it as intended! Pages that you don't want indexed or crawled you can still nofollow login pages, user comments and pagination etc.

Do you track any of your rankings? Have they fallen by a uniform amount?

What's actually changed for you though? Have you looked in the analytics to see what pages are no longer bringing traffic, what keywords are no longer bringing you traffic, that kind of thing?

Is it across all pages and all keywords or is it just a few high traffic keywords and pages?

Just because your niche has many link buyers doesn't mean that you're not getting penalised for it

I don't think Google will have known or cared about those millions of pages, I assume none of them have shown up as a landing page for visitors so I would guess they were effectively invisible to Google.

Paid links (and indeed normal links), you may not be getting penalised for them, but if some of your highest value links are themselves being punished or in some way devalued you may be losing out there as well.

I asume no other significant change occured at this time?

Have had some success in the past using this - http://wordpress.org/extend/plugins/seo-title-tag/ - but not sure if it's still supported.

Check out Headspace2 - http://wordpress.org/extend/plugins/headspace2/ -(NB: there are definitely plugin compatibility errors with this and WP3.1 just now) and All in One SEO - http://wordpress.org/extend/plugins/all-in-one-seo-pack/

Any, with a bit of tweaking, should allow you to change your tag post titles to something like "All information about %tag%"

Eh, actually I don't think %tag% is a thing, I'll go look it up

Raven Tools are definitely worth checking out - http://raventools.com/

Edit - To elaborate you can have several link builders log in and 'claim' a site to work with. It also provides a bunch of useful stats about the value of a link etc and can also allow clients to specify costs for links etc. Try the demo, see what you think

I've no experience with them so couldn't say for definite, but when using a link buying agency make sure that it's very clear that you own the links and it's not all links on content networks run by the builders as when you stop paying all your links will be removed.

Had a company in Bulgaria do that though luckily weren't with them for very long.

As a rule of thumb if you can get full disclosure from them (or any agency) and what they're doing it may be worth trying, but any agency that's shady or less than forthcoming about their activities I would avoid.

First thing that came to my mind was email writing but Andrew's covered that already.

That to me goes hand in hand with link evaluation, deciding what's a good link and why; number of outbound links on a page, using mozRank or Raven tools evaluator to give a quick evaluation as to a page's worth then look at what links are going into that page, context, link location, that kind of thing.

But I guess the stuff you're covering is the very basics and the evaluation a bit more advanced and then how to find those links in the first place to evaluate is the hard part

Hmm, I'll probably say one thing and someone will come along with the opposite again, lol.

Well, the only real advantage of forwarding the domain is for type in traffic. If it's a site that people would expect to be live they may just put it straight into the address bar and end up at your site. 301 redirecting that domain means that it won't appear in the search engines. Your new site won't really rank for the exact match domain you're forwarding.

You could also make an index.html/php page and meta refresh them to your main domain and that would allow the site to rank for exact match but still not really going to give you much benefit.

If the URL is awesome, consider putting some content up and sending it to your site via links, but otherwise I'd just forward them at the registrar.