URL not indexed but shows in results?

[MickEdwards]

We are working on a site that has a whole section that is not indexed (well a few pages are).  There is also a problem where there are 2 directories that are the same content and it is the incorrect directory with the indexed URLs.

The problem is if I do a search in Google to find a URL - typically location + term then I get the URL (from the wrong directory) up there in the top 5.  However, do a site: for that URL and it is not indexed!  What could be going on here?

There is nothing in robots or the source, and GWT fetch works fine.

[DonnaDuncan]

If you search for the page directly, can you see if a version of it has been cached?

[MickEdwards]

yes, a sample page is cached.  It was cached today, however that URL using site: is not indexed. This URL was not showing as indexed yesterday either!

[DonnaDuncan]

The site command doesn't always show you every page that is indexed. You can:

• look to see if it has been cached (like you just did); or
• execute a specific site:domain.com/pagename.html or site:domain.com/section/ command to see if Google returns an indexed result; or
• look at Google Analytics to see if the page is receiving any search-engine-sourced page entries.

It sounds like your pages might, in fact, be indexed.

As to the wrong directory content getting indexed, I'm assuming you've no indexed one of them or assigned canonical tags indicating your strong preference. Both of these are only "suggestions" to Google. It can ignore you and when that happens, the situation like the one you describe happens.

The other thing to bear in mind is how long ago you noindexed or tagged your pages. It can take Google days, weeks, months and sometimes forever to catch up to your requested changes. You have to be patient and cross your fingers.

[MickEdwards]

The canonical issue is identified.  This is more of a "i've never seen that" day.  Yes the directory Site: search returns all the URLs, but do a site: search for individual URLs and 95% are not showing as indexed.

[DonnaDuncan]

It appears as though they are though. You got what you need then? Your question is answered?

[MickEdwards]

I wouldn't say that the question is answered as such, more an issue identified.  For me it looks like having a directory of URLs having a canonical set to another directory of duplicate URLs messes things up for Google.

I get virtually no visibly indexed single URLs out of around 500 URLs, the directory site: search returns the URLs.  Some URLs were cached in the last day or 2, and plenty throw a 404 Google page when checking for a cached version. Seems flaky all round.

[DonnaDuncan]

If you want to share a set of urls I'd be happy to take a look at it in case anything else jumps out.