Indexed Pages in Google, How do I find Out?

[JohnPeters]

Is there a way to get a list of pages that google has indexed?

Is there some software that can do this?

I do not have access to webmaster tools, so hoping there is another way to do this.

Would be great if I could also see if the indexed page is a 404 or other

Thanks for your help, sorry if its  basic question

[BlueprintMarketing]

Google Webmaster tools should supply you with this information. In addition Seomoz tools will tell you that and more. Run your website through the campaign section of seomoz you will then see any issues with your website.

You may also want to of course use Google Webmaster tools run a test as a Google bot the Google but should show you any issues you are having such is 404's or other fun things that websites do.

If you're running WordPress there are plenty of plug-ins I recommend 404 returned

sincerely,

Thomas

[JohnPeters]

Thanks for the reply.

I do not have access to webmaster tools and the seomoz tools do not show a great deal of the pages on my site for some reason

Majestic shows up to 100 pages. Ahrefs shows some also.

I need to compare what google has indexed and the status of the page

Does screaming frog do thiss?

[BlueprintMarketing]

just thought I might add these links these might help explain it better than I did.

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1352276

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=2409443&topic=2446029&ctx=topic

http://pro.seomoz.org/tools/crawl-test

you should definitely sign up for Google Webmaster tools it is free here is a link all you need to do is add an e-mail address and password

http://support.google.com/webmasters/bin/topic.py?hl=en&topic=1724121

I hope I have been of help to you sincerely,

Thomas

[riplash]

If you don't have access to Webmaster Tools, the most basic way to see which pages Google has indexed is obviously to do a site: search on Google itself - like "site:google.com" - to return pages of SERPs containing the pages from your site which Google has indexed.

Problem is, how do you get the data from those SERPs in a useful format to run through Screaming Frog or similar?

Enter Chris Le's Google Scraper for Google Docs

It will let scrape the first 100 results, then let you offset your search by 100 and get the next 100, etc.. slightly cumbersome, but it will achieve what you want to do.

Then you can crawl the URLs using Screaming Frog or another crawler.

[BlueprintMarketing]

can I get your sites URL? By the way this might be a better way into Google Webmaster tools

https://accounts.google.com/ServiceLogin?service=sitemaps&passive=1209600&continue=https://www.google.com/webmasters/tools/&followup=https://www.google.com/webmasters/tools/

if you have a Gmail account use that if you don't just sign up using your regular e-mail.

Of course using SEOmoz via http://pro.seomoz.org/tools/crawl-test  will give you a full rundown of all of your links and how they're running. Are you not seen all of them?

Another tool I have found very useful. Is website analysis as well as their midsize product from Alexia

http://www.alexa.com/products

I hope I have helped,

Tom

[JohnPeters]

this is what I am looking for Thanks

Strange that there is no tool I can buy to do this in full without the 100 limit

Anyway, i will give that a go

[BlueprintMarketing]

Neil you are correct I agree with screaming frog is excellent they definitely will show you your site. Here is a link from SEOmoz associate that I believe will benefit you

http://www.seomoz.org/q/404-error-but-i-can-t-find-any-broken-links-on-the-referrer-pages

sincerely,

Thomas

[BlueprintMarketing]

What giving you a 100 limit?

try using Raven tools or spider mate they both have excellent free trials and allow you quite a bit of information.

[BlueprintMarketing]

sorry to keep sending you messages but I wanted to make sure that you know SEOmoz does have a fantastic tool for what you are requesting. Please look at this link and then click on the bottom where it should says show more and I believe you will agree it does everything you've asked and more.

http://pro.seomoz.org/tools/crawl-test

Sincerely,

Thomas

does this answer your question?

[SebastianCowie]

Hi John,

Hope I'm not too late to the party! When checking URL's for their cache status I suggest using Scrapebox (with proxies).

Be warned, it was created as a black-hat tool, and as such is frowned upon, but there are a number of excellent white-hat uses for it! Costs $57 one off

[askshopper]

If you want to find all your indexed pages in Google just type:  site:yourdomain.com or .co.uk or other  without the www.