Welcome to the Q&A Forum

I'm getting the same issue.

The problem seems to be a "referer exploit". I found this on a Google Groups forum:

From Google Groups (http://groups.google.com/group/Google_Web_Search_Help-UsingWS/browse_thread/thread/2219b91fc28ecb51?pli=1)

_"This sounds like the "referer exploit" where web-sites are hacked and files, such as .htaccess have been modified.  When trying to access specific sites through a link in search results, the redirect occurs. When typing the URL manually into the address bar, the redirect does not occur. _

_Here is a discussion on this from the Internet Storm Center: _

_http://isc.sans.org/diary.html?storyid=5150 _

_These are discussions from Google Web Search Help: _

_http://groups.google.com/group/Google_Web_Search_Help/search?hl=en&gr... _

_This is a similar discussion in the Webmaster Help Group: _

_http://groups.google.com/group/Google_Webmaster_Help-Indexing/browse_... _

_And here is one from Ask Dave Taylor: _

_http://www.askdavetaylor.com/how_people_hack_apache_web_server_rewrit... _

_Hopefully, these documents will help you resolve the issue with your _
_web-site (and/or provide enough information for the web-hosting _
_provider to do so.) _

_Another user recently reported that it was necessary to scroll down _
_beyond the blank lines that had been inserted at the top of _
the .htaccess files, to find the code that was added. "

Hope this helps.

Hi Mozter,

I tend to agree with Cyrus on this issue, and both answers have already provided a great deal of insight.

But.. I'll throw my own two cents in as well, just in case you're still torn. There are Pros and Cons to each approach, as already mentioned. This dilemma most commonly arises when optimizing eCommerce based sites with multiple categories and subcategories, and a lot of times the best approach depends on the industry in question.

A little bit of keyword research is always important and will likely shed a lot of light on how users search for products in a particular niche. Google's Keyword Tool and SEOMoz's Keyword Difficulty Tool are both great resources.

Cyrus already noted several benefits of a shorter URL structure, and I'll expand on that as it relates to commerce driven platforms.

Like over-optimization, keyword cannibalization can be a major issue when optimizing products within the same category. For example, the URL you mentioned will likely have several similar URLs:

Mybrand.com/software/tablets/ipad/supertrader.html

Mybrand.com/software/tablets/ipad/product2.html

Mybrand.com/software/tablets/ipad/product3.html

and so on...

The result of this structure is that you have multiple URLs that are competing to rank for the same keywords (software, tablets, and ipad) and 9 times out of 10, all of the URLs will rank lower in Google, rather than a single URL ranking higher in the results.

If you were a potential customer, what would you search for? "SuperTrader Software for Ipad Tablet" or "SuperTrader for Ipad"? My guess would be the 2nd.

Assuming a more short-tail keyword search i.e. "Ipad Software", again you'll have multiple URLs containing those keywords, and likely a lower ranking for each.

Also remember, the shorter the URL, the more relevance that is placed on each keyword in that URL.

My suggestion is to use a flat structure that contains more descriptive keywords such as:

Mybrand.com/tablet-software/ (Category Page w/subcategories for each OS)

Mybrand.com/ipad-software/ (Category w/ software products for Ipad)

**Mybrand.com/supertrader-ipad/ **(Product - SuperTrader for iPad)

Mybrand.com/android-sofware/ (Category w/ software products for Android)

Mybrand.com/supertrader-android/ (Product - SuperTrader for Android)

This should allow better optimization for long-tail and short-tail keywords and help to avoid multiple URLs competing for the same keywords. It will be much easier to process for the user as well.

Also "Apps" may be a better choice than "Software" but I'm sure the competition is much fiercer.

Finally, on a side note, if you're using Magento as your CMS, I'd suggest removing the ".html" URL suffix from the URLs and Categories and replacing it with a trailing slash by going to System --> Configuration --> Catalog --> Search Engine Optimization. I'd be glad to help you with enabling the shorter URLs and Rel Canonical Meta Tags to best optimize this structure in Magento as well. If you're running another eCommerce Platform, I got nothin' for ya.

Hope this helps and good luck.

Anthony

Hi FinalFrontier,

To fix this, you'll just need to choose which version of the domain you'd like to use and then implement a 301 redirect from the domain you don't want displayed to the preferred domain.

My personal choice is the "naked domain" (no "www"). Technically speaking, www.mysite.com is a subdomain of mysite.com and you'll notice that almost every major brand advertises their site without the "www".

When's the last time you saw an Apple commercial trying to convince you to go to www.apple.com? Seen www.eharmony.com anywhere lately?

The choice however is up to you... the key thing is make the decision and when you link to your site from another location stick with one or the other.

To implement the 301 redirect, the most common method is to edit the .htaccess file in the root directory of your site. Also, many hosting control panels (like cPanel) have this functionality built in where it can simply be activated by choosing the appropriate option in your server's configuration.

For www to non-www simply add this to your .htaccess file (replace mysite.com with your own domain)

RewriteEngine On

RewriteCond %{HTTP_HOST} ^www.mysite.com [NC]

RewriteRule ^(.*)$ http://mysite.com/$1 [L,R=301]

For the opposite (non-www to www) add this:

RewriteEngine On

RewriteCond %{HTTP_HOST} ^mysite.com [NC]

RewriteRule ^(.*)$ http://www.mysite.com/$1 [L,R=301]

Hope this helps!

Anthony

Hi sftravel,

Good question.

There has to be some kind of keyword cannibalization happening if your custom 404 page is really ranking higher than the page you've optimized for those keywords.

In short... Yes. No-Index your 404 page. Users seeing a 404 error page as a search result doesn't really do anything but harm your site's reputation, as it more than likely makes them think that the site is ill-maintained.

The more important question is this:

**Is the 404 page itself ranking and receiving the traffic or is there a page on your site that has moved and is now yielding the 404 error? **

I'd suggest using the Queries and Landing Pages sections in the Traffic Sources (Search Engine Optimization) menu in Google Analytics  to track down what search queries are leading to your 404 page. Webmaster tools will work too. Im almost positive that it's simply a page that was ranking and has since moved that's causing the problem. If so, a simple 301 redirect to the correct URL should solve the problem without harming the ranking of the page or decreasing organic traffic. It may actually improve, as I'm sure the bounce rate for the clicks from that specific query will decrease.

Hope this helps.

Anthony

Hi Pete,

If you are referring to Meta Keywords and Meta Descriptions, the answer is mixed. Most major search engines have ignored Keyword Meta Information for some time now, so it's best to leave those fields blank. There's no benefit in adding them, and in one respect it can actually be detrimental...

Why share the hard work you've put into keyword analysis with your competitors by advertising your targeted keywords to the world??

As far as the Meta Description, that's a different story. While there's a no inherent boost in ranking provided by keywords in Meta Descriptions, this area can have a huge effect on your page's CTR (Click Through Rate) which can indirectly effect rankings. Remember this description is displayed directly below the Meta Title in SERPs. You should use the Meta Description to accurately describe the page, and to entice the potential visitor to click the link. Some keywords will surely help, as Google will make any words in the Meta Description that match the search query bold, which can draw attention to your page, but it's probably more effective to use this area as a call to action...

Why should I click on your link rather than the other links? Do you offer free shipping? Is your customer service excellent? Are you offering a special promotion? The Meta Description is the place to let your potential visitors know about these things. Be concise, as Google only displays the first 156 characters, and avoid using the same description for every page.

Meta Description - Make 'em count.

Meta Keywords - Leave 'em blank.

Hope this helps.

Thanks,

Anthony

HI Pol,

You can check out the Linkscape Update Schedule here:

http://apiwiki.seomoz.org/w/page/25141119/Linkscape%20Schedule

Looks like April 27th is the day. You're not the only one holding your breath, I promise! But with the next index including over 100 billion URLs for the first time, I'm sure SEOMoz has a few kinks to work out.

Also here's a very in depth explanation of the reason for the delayed update, which was originally planned for April 6th:

http://www.seomoz.org/blog/linkscape-index-delay-explained

Hope this helps.

Thanks,

Anthony