Welcome to the Q&A Forum

"I still want the category pages to distribute their link authority to the individual pages, then I should leave the category pages in the site map and meta noindex them, rather than robots.txt them. Is that correct?"

This will achieve the goal, yes. You would ideally include noindex, follow (as opposed to nofollow) in the meta tag of the page you want to exclude. This means that Google crawls the page in full and allows PageRank to flow from that page to the pages it links to, but doesn't include any of the page's content or its URL in the index.

If you exclude the page via robots.txt, Google never crawls the page at all. You sometimes see URLs whose pages have been excluded via robots.txt showing up in Google's index, because robots.txt doesn't say "don't index this URL"; it simply says "don't crawl it." That's also why excluding a page in robots.txt and putting a noindex meta tag on the page would be redundant - Google would never see the noindex tag because it would never crawl the page.

Hi Cole,

Unfortunately there is a solution for this for international duplication but not national. If we were talking about international locations, the solution is the hreflang tag. I'll link to it here just in case it's of use in the future: https://support.google.com/webmasters/answer/189077?hl=en

Nationally, canonicalisation will remove the non-canonical versions from the index and of course from rankings, as Chris has said.

I've looked at medical queries in the past, and Google is very adept at taking IP into account when returning results, the fact that it thinks my IP is located an hour's drive south of here notwithstanding

I would say that re-written content is your best bet if you can't use one page listing multiple locations (highly unlikely) and truly need separate sites for all 13. There can be a little cross-over / duplication without causing too much worry, but I would be concerned that Google is not good enough at a national level to differentiate between duplicates in the same way it can do this for internationalisation.

Two great answers here already.

Basically, you may have a strong resource to work with but Google is also keen on not allowing brand new site owners from benefitting too much from the authority of newly-purchased websites. Google is a domain registrar for the sole purpose of accessing comprehensive whois information and understanding when websites / domains change hands. If a domain's registration information, hosting and content changes drastically, there is nothing stopping Google from removing the authority that domain once had.

Again, this is not to say that the domain won't be useful, but you will probably need to build a useful site on that domain that reflects the quality of the links that have helped it achieve a PR6 in the past. Simply redirecting it to a new location will probably not be of much use - I seem to remember that Google started stamping out that practice's usefulness five or six years ago.

$100 also seems quite cheap for a PR6 domain, but maybe you lucked out!

Hi Eduardo,

If you're suffering from a Penguin issue, you may see the site's results improve over the next few days or weeks. Google confirmed last night that they are rolling out Penguin 4.0, starting yesterday. Most people have seen more movement in US results so far, but there have certainly been some international changes, and it's no longer 2011 when non-US results took a very long time to update after Google first released Panda.

If nothing changes for the site in the next few weeks, either Penguin is not the issue or you have not done enough to remove the bad links. Have you tried removing them, not just submitting a disavow report? Removal is recommended; if removal turns out to be impossible, that's when you need to use the disavow tool.

The site's issues could stem from on-site, meaning that you'll need to look through the ideas presented here and conduct a full on-site audit.

Hi Thomas,

I think your problem is partially duplicate content:

https://www.google.co.uk/search?q=%22Once+you%E2%80%99re+approved+for+a+ReebokONE+profile+and+you+sign+up+for+the+commission+program%2C+visitors+can+purchase+products+displayed+on+your%22&oq=%22Once+you%E2%80%99re+approved+for+a+ReebokONE+profile+and+you+sign+up+for+the+commission+program%2C+visitors+can+purchase+products+displayed+on+your%22&aqs=chrome..69i57.1682j0j7&sourceid=chrome&es_sm=119&ie=UTF-8#filter=0&q=%22Once+you%E2%80%99re+approved+for+a+ReebokONE+profile+and+you+sign+up+for+the+commission+program%2C+visitors+can+purchase+products+displayed+on+your%22&safe=off

This page is virtually identical to a bunch of others in international subfolders, e.g. http://www.reebok.com/sv-SE/reebokonehome/Landing-Page/, http://www.reebok.com/sv-SE/reebokonehome/ (same page but without /Landing-Page/, http://www.reebok.com/nl-nl/reebokonehome/, etc.

It's highly unlikely that Google sees any of these resources as highly valuable on their own, given their duplicated many times. The solution here is pretty simple (in theory) though: the rel="alternative" tag (also referred to as the href lang tag) is meant for the purpose of telling Google that although these pages / subfolders, etc. are duplicates of each other, Version A is meant for the US, Version B for Sweden, Version C for Finland, etc. You can also create, for example, an English and Spanish version of the content for the United States and say: "these two pages are for a US audience but this one is for Spanish queries and this one for English."

Here are some resources about the tag:

https://support.google.com/webmasters/answer/189077?hl=en

http://moz.com/blog/using-the-correct-hreflang-tag-a-new-generator-tool

Essentially, Google may be refusing to pick this page up because it's basically already seen it many, many times.

Cheers,

Jane

I would say that it depends on why the penalty happened in the first place, but if it's a manual penalty then removing the resource probably won't get rid of the penalty overnight. I'd do a combination of trying to remove the links, disavowing what I couldn't remove, removing the page with a 410 and filing for reconsideration explaining what I did and how I've tried to fix it. I'd also explain that the page was obsolete to begin with and was always destined for the scrap heap.

Hi there,

The rel='prev' you have on the second page points to http://lds.net/forums/topic/35625-april-fools/page-1, which in turn 301 redirects to http://lds.net/forums/topic/35625-april-fools/. I would change this to point to the resolving URL.

For the sake of consistency, you may want to use " instead of ' around prev and next as well, simply as this is standard.

Another good, comprehensive guide to pagination is here: http://www.ayima.com/seo-knowledge/conquering-pagination-guide.html in case you're keen on looking at your range of options.

Hi Karl,

Have the tags been implemented, and have you seen any improvement in the situation?

Not to be the bearer of bad news, but I had a client accidentally canonicalise two entire websites to those website's home pages in 2012. That is, www.site.com/page/product/123.html contained a canonical tag pointing to www.site.com, as did every other page on the website. They put this live across two ecommerce domains.

We kicked up a huge fuss, but it still took their dev team a week to fix (the fix was put into a queue... grr). The site lost all of its rankings very quickly, which wasn't surprising given that its ranking all stemmed from internal pages ranking for products. It took about six weeks for Google to properly re-index and start ranking the internal pages again, and months to return to the previous rankings. Granted, these sites were quite new when this happened so they didn't have much authority to begin with.

The short version is that it took a minute to break, six days to fix and six weeks to re-index. Every single case like this will be different, and I do not think the brand-new status of the two domains helped in this case.

Hi there,

This is horribly frustrating, but the difference is that they're Facebook and YouTube and the rest of us aren't.

That's the short version. The big boys and girls get away with a lot more than normal folk Also, a widget from a huge brand is going to make less of an overall dent in the make-up of the site's backlink profile than it is for a brand that is not enormous.

The longer version is that this method of link acquisition was spammed pretty badly a few years ago. Moz's former CTO Matt Inman (better known as the Oatmeal comic writer) wrote a post about doing this six years ago. He achieved very good results very quickly using widgets / badges. The post is here: http://moz.com/blog/widgetbait-gone-wild

As you can see in the post, Matt's method of gaining links was blindingly effective. He's a good comic writer and a smart guy who knew how to tap into a lot of people's egos and interest for links. This was also in 2008, so long before everyone had seen the Buzzfeed-style "which city should you live in?" quizzes go around Facebook. However, he abused the method (as he readily admits) and he got caught.

Anything that has been abused like this is on Google's radar and they don't like it. Less interesting versions of this sort of link dev include newsletter sign-up forms with "email marketing managed by X" links and similar. It seems very unfair and disingenuous on Google's part that smaller businesses are penalised or at least not helped by these kinds of links, but it's certainly true that a smaller business (and by this, I mean "smaller than Facebook") gaining hundreds or thousands of links in two weeks is a far bigger shift in backlink profile than Facebook gaining hundreds of thousands of links constantly from all sources, including widgets.

Linking internally with optimised anchor text should be fine, but be careful not to overdo this in the same manner that you would not overdo inbound links with optimised anchors.

In the last day, I've read a couple of theories about over-done internal anchor text being a common factor in sites that have recently lost rankings, but do NOT take that as gospel or even particularly important yet, as this was acknowledged by the person who said it as a very preliminary theory that they had not run proper analysis on yet. Just something to keep an eye on - I am sure the person will publish this theory if they establish that it has more merit!

The image re-usage should be fine, although it's best to use entirely unique content in both text and imagery.

Hi Stephen,

Checking in to see if you had checked the points William has raised above. Do you see anything that could have resulted in the drop? Also, are you comfortable sharing the site here? We might be able to have a look too (feel free to PM if you are not comfortable sharing publicly).

Cheers,

Jane

I have direct experience with this.

It looks to be pretty useless to contact the authors of old posts and ask them to insert links to your site, with anchor text or without. This was a very very popular link development tactic three of four years ago. It was pretty easy to scale and it was effective.

For Google, negative the positive effects of this tactic was likely quite easy:

• Cached versions of a page contain no external link to www.yoursite.com
• Page exists for extended period, gains PageRank
• Link suddenly appears on page to www.yoursite.com, often with optimised anchor text
• If you're being really obvious about it, you've paid for this link and set a time period of one year for the link to be up. All of a sudden after a year (or a set period), the link disappears, and www.yoursite.com seems to have a lot of links that disappear from older posts after one year. This is so blindingly obvious and easy enough for amateurs to spot: it's unfeasible that Google can't spot this too.

A natural link is going to appear in a new blog post, not an old one. How many times have you gone back and edited an old post to include a new link? I can think of once or twice in my history online where I have done that with the legitimate intention of adding a good resource to the article. I am not adding links to insurance websites, jewellers, etc.

A new post may or may not acquire authority, but if you are trying to place links based on authority, look at the website the new post will go up on. Do its posts regularly receive a good number of links, decent traffic, social media attention, etc.?

This tactic of placing links on old blog posts was effective in the past, but I would be confident saying that its effectiveness en masse disappeared some time around 2010.

Definitely use the rel="alternative" tag, also referred to as the hreflang tag, if you plan on direct translations and pages that are basically the same thing in either language.

Two good resources: https://support.google.com/webmasters/answer/189077?hl=en and http://moz.com/blog/using-the-correct-hreflang-tag-a-new-generator-tool.

I too would go for the subfolder option over subdomains for similar reasons to those Kevin mentions.

Are you translating the content by hand?

Hi Simon,

Like Andy says, is there a possibility that you can have these windows open in a new tab with a nofollow attribute if necessary? Whether to keep people on site or send them away has been a problem / debate in marketing for a long time (well before we had to worry about penalisation / outbound link issues to go with it). There's a fine line between keeping people on a site they no longer want to be on and accidentally allowing them to forget about where they were before.

You might want to test different ways of sending visitors away to see which results in the best conversion metrics (whatever you choose those metrics to be). As a user, I almost invariably open links in new tabs myself, but different people have different ways of handling this themselves and I'd test what works best for your user base in terms of visitor retention / other goals.

To add to what Cyrus has said, Google will "choose" a version of content it finds duplicated to favour, so has clearly chosen the local. subdomain. The main website would likely fall back into favour if you were to consolidate the subdomain and redirect it to the main site, but I would definitely find out if the third party agency who developed the subdomain has done some promotion of the subdomain that you aren't aware of.

There is no real problem using subdomains or subfolders for specific local rankings / campaigns. I wouldn't say you are gaming or abusing it here - you have a dupe content problem that needs fixing though.

If you were to use subfolders, the duplicate content problem would remain as long as there are multiple versions of the same content, but if you were to consolidate the main site and subdomain's strength, then any subfolders you added to the one remaining site (assuming all content was unique) should inherit authority.

Hi Raymond,

Either canonicalisation to page 1 or noindexing would do the trick here if the content has been duplicated. If you noindex, use noindex, follow so links in and out of that page benefit the website.

Cheers,

Jane

How are the indexed numbers looking in WMT today? I see 3,370 results for a site: search on the domain, but those can be iffy in terms of up to date accuracy: https://www.google.co.uk/search?q=site%3Agoautohub.com&oq=site%3Agoautohub.com&aqs=chrome..69i57j69i58.798j0j4&sourceid=chrome&es_sm=119&ie=UTF-8

Not that this should matter too much if you are submitting a sitemap through WMT but your robots.txt file specifies sitemap.xml. There is a duplciate sitemap on that URL (http://goautohub.com/sitemap.xml) - are you using sitemap.php, which you mention here, in WMT? .php can be used for sitemaps, but I would update the robots.txt file to reflect the correct URL - http://i.imgur.com/uSB1P1g.png, whichever is meant to be right. I am not aware of problems with having duplicate sitemaps, as long as they are identical, but I'd use just one if it were me.

Hi Asif,

The numbers you see in WMT will be fairly accurate: the numbers for site: searches are rarely accurate, so I wouldn't pay them too much attention. You can usually trust what you see in WMT though.

Cheers,

Jane

Exactly as Anirban says. We're seeing a lot of directory SEO questions lately, and it's worrying. Both the exact anchor text and the fact that directory links are more often than not junk is why you should remove them.

If they were directories that were HIGHLY relevant to your business and frequented by people looking for your type of services, they might be fine. For instance, if you found a directory listing gardeners, plumbers, electricians and builders in Los Angeles and you happen to be a gardener in Los Angeles, there is some sense here.

However, the vast majority of directory submissions we see are made-for-SEO and should be removed ASAP.

Hi there,

You are definitely redirecting this properly (screenshot using my former agency's plugin, showing your 301 working correctly): http://i.imgur.com/AbUIpNa.png.

Echoing the question about which tool created the report showing a duplicate home page? Google should totally disregard the non-www version of the site given the correct redirection. Set your preferred domain in WMT as Chris says as well, to be on the safe side.

Cheers,

Jane