Welcome to the Q&A Forum

I guess it really depends on why would you do that. Typically, the H1 tag should appear toward the top of the page since it is the title of the page. Anything other than that would look weird and could be considered spammy, especially if it looks like you are trying to hide keyword rich text somewhere where only robots can see it.

I've not tried what you are suggesting specifically but I have had pages with multiple h1 tags and Google seems to ignore those multiple h1s (to whatever extent h1s influence rankings). As a result, I would suspect something similar would happen if you were trying to hide the h1 in the footer.

All that being said, if as part of the design you have selected to put the h1 page name in a footer but still make that h1 page name a prominent part of the page, then I suppose it wouldn't be too big of a deal where search engine spiders are concerned.

Really though, it depends on intent. Are you trying to hide the h1? If so, don't do that because that is the point at which you cross into black hat territory and risk a penalty.

Yep, all of those tactics will be considered black hat and those tactics will not be likely to work. Google can see hidden text and can easily detect those kinds of tactics.

Here is an article more about the hidden text and other black hat techniques. Don't do any of these!
http://www.beanstalk-inc.com/tactics/black-hat.htm

Hey,

Those two pages are not identical but they are nearly identical. I've seen SEOmoz's tool trigger these types of duplicate content before. For example, both page headers and titles are "Misc. Spare Motor and Pumps", both contain nearly the same equipment description, and the same links to the right of the buttons. That is normal on product sites, but that is probably why they are considered duplicated.

When I've seen this happen on my campaigns it almost always follows that Google has only indexed one of those pages and not the other. Is that happening to you as well? Or, has Google indexed both pages (and ranking both pages and sending traffic to both pages)?

If Google is performing okay with these pages (the pages rank for relevant terms and you get traffic), I wouldn't worry too much about the duplicate content errors in your campaign. However, if Google hasn't indexed either page (or one but not the other) you should find ways to differentiate these pages.

To make these pages unique, formatting changes won't do it because the content will still be the same. You would need to add content that is truly unique. For example, you could add a short paragraph about each product to each page. You could make the title tag and header more unique. Or, you could try adjusting your alt text (right now, the alt text on the main image is the same on both pages).

I hope that helps.

Hey,

Should that second URL be www.sub.domain.com/collection/adresboeken.html?whatever=something If so, then by using /collection/?* you are saying that anything within /collection/ with a query string should not be indexed. If adresboeken.html always has a query string, it may not get indexed.

The other options I'd consider before using robots.txt are telling Google to ignore dir=desc&order=color in Google Webmaster Tools parameter handling. This is the best way to handle query string issues. (Assuming you are trying to influence Google. Clearly Google Webmaster Tools won't affect Bing!)

Another idea is to set a canonical URL on /collection/adresboeken.html referencing /collection/adresboeken.html without the query string. This tells the search engines that the query strings do not make a unique URL. (adresboeken.html?dir=desc&order=color is the same as adresboeken.html?dir=desc&order=price is the same as adresboeken.html?dir=asc&order=color is the same as adresboeken.html, and so on).

I hope that helps. Thanks,
Matthew

Hi,

Here is an article explaining how to do this in robots.txt:
http://sanzon.wordpress.com/2008/04/29/advanced-usage-of-robotstxt-w-querystrings/

Depending on what you are trying to do, it might also be worth investigating parameter handling in Google Webmaster Tools:
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1235687

Thanks,
Matthew

You can do this via RedirectMatch statements. There is an example of different match types you could use here. Basically, you create a pattern and then the code is smart enough to redirect based on that pattern.

http://www.askapache.com/htaccess/301-redirect-with-mod_rewrite-or-redirectmatch.html#301_Redirects_RedirectMatch

Your redirect might look something like this though you might want to adjust the pattern match the directory in between /reviews/ and /item/. Mine is just a basic alphanumeric catch all.

RedirectMatch 301 ^/articles/reviews/([-A-z0-9]+)/item/(.*)$ http://clients.qwconsulting.com/reviews/$1

I hope that helps.

Yep, on two different sites we did thousands of redirects at a time with no issues. In one case it was annual and the other it was quarterly but I don't see any reason monthly would be any different.

Definitely post your findings after implementation or maybe even write a YouMoz post about what you find out!

I guess it depends on what is the intention behind these pages? Are you attempting to get more traffic for search? Or, are these pages simply for people visiting the site? If it is more about people and not search engines, I wouldn't really worry about the duplicate content issues (provided the pages make sense to users). Adding a nofollow to the links referencing these pages (and a noindex,follow on these pages) will prevent Google from crawling unnecessary pages and indexing unnecessary pages.

I've had similar situations come up (it always seems to on large sites). Typically what I've tried to do is identify which pages have the best shot at getting activity within the search results (based on keyword research, link research, etc.). On those pages, I do everything I can to have really unique content that sets those pages apart (I'm talking bigger scale changes, as opposed to geo-targeting changes). For the other pages where there isn't such a great chance with search engines, I use the nofollow tag to prevent Google from accessing those pages. That seems to help keep the focus for SEO on the right pages.

Hey,

In general, I would opt for option 1 as that would be the most scale-able solution. Whenever I've done this, I've not seen any issues with having lots of 301s appear. Given the shorter life span of those product pages you probably won't have lots of links going to those pages (or social, etc.) and I think that helps explain why I've not seen issues redirecting this many pages.

That being said, if you do have lots of links or social signals referencing a certain product page, that is when I'd opt for the custom page listing similar products. I've had success doing this for high-traffic product pages that have been removed as it can help maintain the sale.  In terms of the signal, it really depends. If you are still offering unique content relevant to search queries and links referencing that page, I'd deliver a status 200 (it is still a good page worthy of attention). If the content isn't all that unique, and it is more for people (to maintain the sale) as opposed to search, I would have that page deliver a status 410 (saying it is gone).

I hope that helps!

Matthew

Hi,

The position of (main) text on the page doesn't affect the indexing or crawling of that page. From a user's standpoint, you probably want to position the text higher on the web page to make it easier to read. When it comes to choosing the position, I'd go with what makes the most sense for the overall design of your site.

If you meant the location of the text inside the HTML of the page, I don't think that really matters. Good coding practices would keep the HTML as simplistic as possible and put the content in a logical location. But that is more about coding practices, not SEO; I've seen sites with really sloppy HTML get crawled and indexed.

Hope that helps!

Hey,

You can do cross domain canonicals. Google does support that (http://www.youtube.com/watch?v=zI6L2N4A0hA) and the one time I had to use that, it did seem to help. That being said, I'm not sure if Bing supports that or not.

Hope that helps.

Hi Omar,

The URL structure really won't hurt you. In fact, your product-level URLs are better than most since they use actual keywords in them. Any more, Google and Bing do a good job of understanding all kinds of URLs. If you had a brand new site, I'd say go with the URL you suggested in (a) but for an existing site with thousands of pages, you won't see any noticeable impact by making this change.

With your URLs, you want to make sure your internal linking structure makes it clear what pages are first, second, third, etc. in the hierarchy (how and where you place links). You also want to make sure Google and Bing are indexing all of those pages (meaning, Google and Bing can find all the URLs). So long as those two things are happening, you really do not have a problem.

I hope that helps. Thanks,

Matthew

Hi,

There are a few different ways to answer your question...

First, those pages are technically duplicated. If you go to either URL they return the same content with a status 200. That should be easy enough for your programmers to correct. They need to detect if the URL contains "php"  and, if so, 301 redirect to the ".html" version. Remember though that it isn't just the extension that is different. In the HTML version you have dashes and underscores in the PHP version. However...

Second, the real question for duplicate content isn't "do I have it?" but "where are the multiple URLs?" You want to know how SEOmoz found that ".php" version if you haven't linked to it internally. The way to do that is to download the CSV file in the upper right hand corner of the duplicate content page. Column L (the duplicate page content column) tells you "TRUE" if that page is duplicated and column AM contains the referral. You want to check that referral source to see how they found the link.

Finally, check to see if Google and Bing have indexed these PHP pages. Also, check to see if you've gotten traffic to the PHP pages. If they haven't been indexed and you don't have any traffic, you probably are okay. But, I'd still investigate to find the source and then redirect the PHP version to the HTML version. (Edit: check external links to...is anybody else linking to these ".php" pages?)

I hope that helps. Thanks,
Matthew

This URL contains some advanced tricks to specifically prevent prefetching by Firefox. I've only tried to the htaccess mod_rewrite technique. However, I modified that technique to send prefetch attempts to an empty file instead of the normal 404 page (saving resources): http://www.petefreitag.com/item/312.cfm

I would avoid only showing tags to Googlebot. It does look a little spammy and like cloaking but more than that, adjusting content for Google can be an involved coding process that comes with risks (accidentally showing something else to Google, etc.).

Hey,

I don't think you are safe from an issue with cannibalization just because the one tool says you don't have that. From the situation described, it sounds like you do have issues on your site with overlapping content and that could be hurting your ranking (as could other factors too, obviously). The best way to find out is to remove the content that you think is overlapping or self canabalizing and then see if you regain rankings.

I'm not certain how the cannibalization check happens in SEOmoz's on page tool, but I have seen instances where cannibalization is missed if those pages are not linked together. As I understand it, that on page tool checks only the one page as opposed to an in depth site crawl.

Hope that helps.

Matthew

Hey,

The same basics apply but I've found that links to the domain and authority of the domain matter more than page authority. In terms of fast indexing, make sure you have an RSS feed. Also, if you are trying to get accepted in to Google News, do make sure you have met those requirements: http://support.google.com/news/publisher/bin/answer.py?hl=en&answer=40787. Having a news specific XML sitemap is also helpful: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=74288

Why would you remove the content a day or two after publication? If you have an online newspaper, that content may still be relevant for some time after the original publication date. I've tended to keep news content around years after original publication and still gotten some traffic to those articles.

Given that, I've not has experience removing content that quickly but when I have removed content from news websites, I try to redirect the old url to the most relevant new article (if possible) or news category page (usually easier to do). This preserves link value, which is obviously important. For really old stories that don't have links or traffic, I set the article to return a 410 status on that URL to signal that page is removed permanently. Again though, I'd default to keeping the article around as long as possible.

Hope that helps.

Hey John,

Even if Authority Labs can do that (and I agree, it seems unlikely), my question becomes: who really cares? Seriously, what makes knowing that information useful? Does it actually tell you something new to know you don't show up in 59 million results (beyond the top 50, who cares)? I don't see the point and I think it is silly the other guy keeps bringing it up.

You already know you aren't ranking high enough for that term to matter but you know you need to. That means the real question is, "okay, what do we do about it?" If you can answer that question, you add something new to the conversation.

As others have mentioned, earning those rankings requires link building, authority building, social connections, and writing content that differentiates you from your competition (and you have big competitors).

Hope that perspective helps.

Matthew

Hey,

Brent's advice is spot on. I'd add that you want to verify that all of your key pages are indexed (to answer your first question). Review Webmaster Tools to verify that you have no errors, check SEOmoz's crawl stats, query site:smoothmovesupplies.com to ensure Google is indexing your pages. I would also look at Google Analytics to see if you are getting any traffic right now, perhaps from long tail terms.

Unfortunately, I'm not familiar with Authority Labs so I can't help you there. I agree that seems unrealistic though.

Thanks,
Matthew

Hi,

Let me back up a step. When you say remove duplicate content, do you mean delete the page? For example, pageA.html and pageB.html have the same content. In your example, are you asking do you delete pageA.html or delete pageB.html? I wouldn't delete either page as both pages likely have traffic, links, etc.

Instead of deleting, you want to pick which page is the canonical version of that page. More simply, when somebody Googles the terms that page is targeting, do you want pageA.html to show up or do you pageB.html to show up? If you decide you want pageA.html to show up, then you need to redirect pageB.html to pageA.html. That way, if somebody goes to pageB.html they end up on pageA.html. If you delete pageB.html and somebody goes there, they'll end up on an error page.

Now, on where question...in terms of deciding what version is canonical, I would suggest you look at authority first and foremost. If the domain authority in OSE is higher for pageA.html than it is for pageB.html, keeping pageA.html makes sense. Things like URL structure are only mildly important, especially when making decisions like this.

Hope that helps. Thanks,
Matthew

Ryan's answer is correct. I just wanted to jump in to say that I know from first hand experience that Google and Bing are both able to read the sitemap file even if it is a different extension and even if you can't name it sitemap.xml.