Can Google read content/see links on subscription sites?

[CustardOnlineMarketing]

If an article is published on The Times (for example), can Google by-pass the subscription sign-in to read the content and index the links in the article?

Example: http://www.thetimes.co.uk/tto/life/property/overseas/article4245346.ece

In the above article there is a link to the resort's website but you can't see this unless you subscribe. I checked the source code of the page with the subscription prompt present and the link isn't there.

Is there a way that these sites deal with search engines differently to other user agents to allow the content to be crawled and indexed?

[LesleyPaone]

I would say no. The content of the article other than what is seen is not in the source code. They could be showing something different to Google, but if they did it would be against Google's terms of service. https://support.google.com/webmasters/answer/66355?hl=en

[davidangotti]

Great question! Yes, Google has an effective way to deal with this since 2007. The three ways they deal with this include first click free, subscription designation, and then disallowing content. Here is their official support article on it:

https://support.google.com/news/publisher/answer/40543?hl=en

Here is a quote from the help article:
"To summarize, we will crawl and index your site to the extent that you allow Googlebot to access it. In order to provide the best possible user experience and help more users discover your content, we encourage you to try First Click Free. If you prefer to limit access to your site to subscribers only, we will respect your decision and show a “subscription” label next to your links on Google News."

Here is what Matt Cutts said about it in an interview with Search Engine Land:
"First Click Free originated with Google News, but you can use the same way of handling content in web search (show the same page to users and Googlebot, then if the user clicks to read a different article, then you can show them the registration or pay page). Because the same page is presented to users and to Googlebot, it’s not cloaking. So First Click Free is a great way if you have premium content to surface it in Google’s web index without cloaking. Hope that makes sense."

It is possible to allow the Googlebot to access the content and simultaneously NOT provide it for free to non-subscribers. The above help article above should answer all of your questions. Hope this helps!

[CustardOnlineMarketing]

Thanks Dan, it doesn't look like the example article is using first click free. So I guess the answer is no, Google can't read the hidden content in this example?

[davidangotti]

Hey Matt,

The best way to tell what the news organization or site is using is to turn off javascript or view the google cache to determine how Google "sees" the page.

This article is using the second option in the article I mentioned - snippets. Here is what the article has to say about that:
"If you prefer this option, please display a snippet of your article that is at least 80 words long and includes either an excerpt or a summary of the specific article."