Export Google Search Results

[Evan_Wright]

I would like to export a list of pages indexed by Google and other search engines.  According to our site search with google, we have over 300,000 pages indexed. Anyway to export a list of all of these pages?

Tools like ScreamingFrog (which I own a copy of) can crawl your site but does not tell you the pages indexed by a search engine...  I've tried using tools that will let you export each page of the results. However, this won't work for a 300K page website.

Thanks for your help!

[thmsmrrtt]

This post is deleted!

[thmsmrrtt]

I'm not aware of any tools that would be able to achieve what you need. However, would GWT not be able to help you here?

Search Analytics section would allow you to download all the pages that are indexed.

[LoganRay]

There's no tool that will handle it in that kind of volume. I know of one tool called SERP Scraper that will do 100 URLs, but that's no good when you're evaluating 300k. I'm fairly certain Google makes it impossible for anyone to build this tool, as useful as it would be, it would exist already if this weren't the case.

[Evan_Wright]

Thank you Thomas and Logan! Thomas, GWT puts a cap at 999 pages, but it is also the search terms used to get to our site as opposed to the pages that are indexed.  Logan, it appears your answer is accurate as I have been researching this for awhile with no luck.

[MeganSingley]

Hi Shani! Do these responses help to answer your question or are you looking for more information? If you're good to go, please mark this as answered. Thanks!