How to find all 404 deadlinks - webmaster only allows 1000 to be downloaded...

[CayenneRed89]

Hi Guys

I have a question...I am currently working on a website that was hit by a spam attack.

The website was hacked and 1000's of adult censored pages were created on the wordpress site.

The hosting company cleared all of the dubious files - but this has left 1000's of dead 404 pages.

We want to fix the dead pages but Google webmaster only shows and allows you to download 1000.

There are a lot more than 1000....does any know of any Good tools that allows you to identify all 404 pages?

Thanks, Duncan

[DirkC]

Probably the easiest solution is to buy a licence from Screaming Frog & to crawl your site locally. The tool can do a lot of useful stuff to audit sites and will show you not only the full list of 4xx errors but also the pages that link to them.

There is also a free version but that allows you to crawl only 500 pages - which in your case is probably not sufficient but it would allow you to see how it works.

Hope this helps,

Dirk

[VERBInteractive]

I must second Dirk's suggestion of screaming frog, great tool and I use it daily, a license is well worth the cost. Although spider crawl of the site will only point out 404's that have are links from an existing page, so if the hosting company cleaned up the not all of these 404's will surface.

One approach I would suggest is run the current 1000 404's in GWT through Screaming frog as a manually added list, (do it in 2 batches if you have the free version), start a spreadsheet of the resulting 404's and start working through that list. Once you have the 404's mark those as fixed as GWT tools set a reminder to check back in a few days and after a few days export the new list of 1000 404's and run these through screaming frog adding the resulting list to your spreadsheet. Keep doing this until you get the 404's errors in GWT down a manageable level.

I hope that helps, good luck.

[CleverPhD]

The Moz crawl report will also show 404s.  I sometimes find that different spiders may find different things.  Between the Search Console report, Screaming Frog (great investment) and Moz, you should have a nice collection of things to fix.