How to Get Google to Recognize Your Pages Are Gone

[ABK717]

Here's a quick background of the site and issue. A site lost half of its traffic over 18 months ago and its believed to be a Panda penalty. Many, many items were already taken care of and crossed off the list, but here's something that was recently brought up.

There are 30,000 pages indexed in Google,but there are about 12,000 active products. Many of these pages in their index are out of stock items. A site visitor cannot find them by browsing the site unless he/she had bookmarked and item before, was given the link by a friend, read about it, etc. If they get to an old product because they had a link to it, they will see an out of stock graphic and not allow to make the purchase.

So, efforts have been made about 1 month ago to 301 old products to something similar, if possible, or 410 them. Google has not been removing them from the index. My question is how to make sure Google sees that these pages are no longer there and remove from the index? Some of the items have links to them and this will help Google see them, but what about the items which have 0 external / internal links?

Thanks in advance for your assistance.

In working on a site which has about 10,000 items available for sale. Looking in G

[PatrickDelehanty]

Hi there

Did you make sure to remove those pages from your sitemap.xml? It takes Google a minute sometimes to see that pages are gone. I still have issues with that as well, but it's just the name of the game.

I would also check your internal links to make sure all links that point to those pages are pointing to their new locations. I would also check your backlink profile to see if any good links are out there that point to those old pages. Reach out and have those corrected.

Also check the product descriptions on those new pages to make sure they are robust and unique to the product.

Hope this helps - let me know if you have any more questions.

[renehansen]

I am just stating the obvious, but have you used Google Webmaster Tools and prompted for a reindexation? Do you want to give a visitor a custom 404 or a an image of the sold out product, with newer and on stock alternatives?

That could be an elegant solution, though it could be a technical challenge.

[ABK717]

Hi Patrick,

The sitemap only shows the active products; therefore, the older, out of stock items, are not in there (definitely a good thing to check).

If you try to go to one of these pages, the header does show a 301 or 410, respectively. But, does Google recrawl all of the pages in their index? How will they see that these are gone if there are no links to many of these pages?

All product descriptions are unique, but unfortunately, a large site scraped them for a few years and recently stopped. That's another big piece to the puzzle as Google gave them credit when in fact, it was coped from the penalized site.

[PatrickDelehanty]

Hi again

Okay - thanks for the clarification.

Now, I have never used this tool, but you could try the Remove Outdated Content tool from Google in Webmaster Tools. The reason I put that disclaimer here is because I don't know the timeline in how long it takes the content to get removed and I want you to make sure that this is a step you want to take, especially for thousands of pages.

Otherwise, you best bet is to just hang tight, rerun your sitemap, reupload to WMT, and let the crawls take their course. This may be the best bet. Good luck!

[ABK717]

No problem with stating the obvious...A fetch within GWT was done, but they would start from the homepage and work their way down from what I understand. How would they crawl these 'dead' pages which have been 301'd and 410'd?

[renehansen]

Sure, I can see the issues there. Having a look at the sitemap and submitting that would be my best guess.