Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
Hi Alan, I understand that, but the problem Evan is describing seems to be related to duplicate content and crawl allowance. There's no perfect answer but in my experience the types of pages that Evan is describing aren't often linked to. Taking that into consideration, IMO robots.txt is the correct solution.
Hi Even, this is quite a common problem. There are a couple of things to consider when deciding if Noindex is the solution rather than robots.txt.
Unless there is a reason the pages need to be crawled (like there are pages on the site that are only linked to from those pages) I would use robots.txt. Noindex doesn't stop search engines crawling those pages, only from putting them in the index. So in theory, search engines could spend all there time crawling pages that you don't want to be in the index.
Here's what I'd do:
Decide on a reasonable number of facets, for example, if you're selling TVs people might search for:
But past 3 facets tends to get very little search volume (do keyword research for your own market)
In this case I'd create a rule that appends something to the URL after 3 facets hat would make it easy to block in robots.txt. For example I might make my structure:
But as soon as I add a 4th facet, for example 'colour'- I add in the filter subfolder
I can then easily block all these pages in robots.txt using:
Disallow: /filter/
I hope this helps.
Hi Rox,
I think the problem is because of the Doctype of that page. If you select HTML 5 from the doctype dropdown, you won't get any HTML validation errors related the schema.
In summary, it's not worth worrying about.
I hope this helps,
Craig
Although I agree with what some of the others have said, it's not enough to just disavow all links. Google want to see that you have actually tried to remove the links. So make sure you keep record of the sites you tried to contact as proof that you have tried. Google want to make this is hard for you, otherwise there is no incentive to just do the same again. Don't be surprised if you need to try upwards of 5 reinclusion requests.
Some good resources on the process are here:
http://moz.com/blog/ultimate-guide-to-google-penalty-removal
http://moz.com/ugc/the-anatomy-of-a-successful-reconsideration-request
Good luck
Hi W,
Sorry but I'm not sure I understand your question 100%. Can you share the URL and the problem you're having?
Thanks
Semrush is a good option, another is Searchmetrics which will give you that information and more. Remember though that these are only trends, there is no way to get exact data. Obviously you can just rank check as well if you really want to know. Be wary of going down a rabbit hole though, realistically what are you going to do that improves your business by knowing what your competitors rank for? There are probably better ways to spend your time.
I hope this helps,
Craig
Hi Mike, just to be clear on what Thomas is suggesting, as I think he might be getting mixed up between noindex and robots.txt.
If you simply add noindex,nofollow to a bunch of pages, this could still get you in trouble. Noindex doesn't mean DO NOT CRAWL, it means DO NOT INDEX. There's a big difference.
If something has noindex, Google can still crawl that content but they won't put it in the search results.
The only way to completely make sure that Google won't crawl content is by blocking it in robots.txt or in your case putting it behind a username and password.
So to answer your question, yes it's fine as long as it's behind a login, Google can't punish you for it since they can't see it.
I hope this helps,
Craig
Thanks Thomas for jumping in with such a detailed answer. Let me build on it a little though as I think what Belcaro meant was, does it matter if your URL is:
www.example.com/category/subcategory/page1
or should it be:
Personally I'd rather go for the structured version for a couple of reasons.
1 - I thinks it's good for users, it helps them know where they are on the site if they just landed on that page
2 - From a tracking pooint of view, it means the content drilldown report will work in your Google Analytics
From an SEO point of view I don't think having one or the other will move the needle much so I'd try not to worry about it either way.
I hope this helps,
Craig
That's funny Ryan as I just looked at one of our account that I know has structured data and saw the same thing, a dip then a rise.
Sounds like a bug in Google's side, which was my first guess, I only got more interested when I read the forums and people said they also had a drop in rankings. Having looked into it, it sounds like those who got a drop in traffic are likely being impacted by updates around that time which was nothing to do with the data issue.
I'm going to mark the question as resolved but I'll add any details I find out.
Craig
Hey Ryan,
I'm going to look into this for you, so far I've only seen the same articles that you have but I'll ask around and get back to you as soon as I can, just thought I'd drop you a note to let you know your question is being looked at.
Craig
Hi Richard, It sounds like the best way to answer your question would be to use an example and think about it from a user point of view.
Let's say you own a hotel site with the following URL structure:
www.example.com/hotels/london - (in your case this would be your www.example.com/venue/paris page)
If I was a user and I now want to see all hotels, it would make sense that the page www.example.com/hotels/ would show me that. If it doesn't and I get redirected to a less important page I would find that strange. Forgetting Google for a minute it's also not good to put users through an internal redirect as it's slower and if users see the URL change it can confuse them.
To summarize, I would remove the 301 redirect and create a category page as it's better for your users and can have a chance of ranking for a category keyword.
I hope this helps,
Craig
Hi G,
I wouldn't worry about it from an SEO point of view, changes of that size will have little or no impact. It's strange from a user point of view though to have the main heading of a page below the fold so I might consider doing it anyway.
Hi Prime,
I don't see there being a problem with this at all, as you said it's helpful for users, however, make sure that it makes sense from a grammar point of view as well, for example, "see all TV's" makes sense, but "see all car insurance" does not. Don't try to shoehorn a keyword in just for the sake of it.
I agree with Karl that breadcrumbs are a good idea both from an SEO point of view and a user point of view.
I hope this helps.