Welcome to the Q&A Forum

Hi again,

Currently you have that product number listed under two different URL's

http://www.vistastores.com/indoorlighting-patiolivingconcepts-20947.html is returning a 301

http://www.vistastores.com/floor-lamps-patio-living-concepts-20947.html is returning a 404.

There are those who would say that it is perfectly fine to return a 404 as long as the 404 page contains proper navigation links and access to other products etc.

I do not disagree with their opinion as far as what should be on the page that is returned (your custom 404 page does this), but in the case of an ecommerce site, I disagree that it is a good idea to just let all expired products return 404 errors.

The nature of ecommerce is such that this will eventually result in search engines seeing an ever-growing list of broken links on your site. With large ecommerce sites, this could result in thousands of 404 errors which the search engine will see no attempt is ever made to correct or reduce.

If you choose to allow searchers to be sent to your custom 404 page, I take a different approach:

• Name the custom 404 page something innocuous like /custom-introduction
• Remove the 404 references from the Title, description and page text - just use a simple explanation that the product is no longer available and invite the visitor to select from the categories available to search for another product, without mentioning the term "404 not found"
• add a catch-all 301 redirect which sends any request for a page that does not exist on the server to /custom-introduction.

By doing this you will avoid creating an ever growing list of 404's, but as I said, there are those who don't worry about that at all.

As to not using 301's because you have put all that work into the pages, if they are disabled in your store the content will no longer be visible to the search engines anyway. As much as we all hate having our hard work seemingly come to nothing like this, it is very much the nature of the business in ecommerce. Products come and go all the time. This is exactly why the 301 redirect is the best option - because it passes almost all of the link value to the new page. So your hard work has not been wasted, but helped to improve the value of another page on the site.

Hope that helps,

Sha

Hi there,

Unfortunately products that go out of stock are one of the biggest frustrations for ecommerce site managers. Where products are sourced from another company you can be very much at the mercy of the way that they manage their inventory. This means you need to decide which method of dealing with expired products will work best for you.

From what you have described it sounds like the product was marked temporary out of stock by the supplier, then more recently changed to "Do Not Reorder" status. So, the product has permanently expired and will not be restocked at any time in the future.

Your best option in this situation is a 301 redirect, which will pass link value from the page to whichever page you decide to redirect to. This is where you need to make decisions.

You have essentially 3 choices when choosing an appropriate page to set your redirect target.

1. If you have a product in your inventory that is very similar, you could redirect to that as the product may be of interest to people who were looking for the original product.
2. If you do not have a similar product, then you can redirect to the higher level category page for that item.
   For example, redirect to http://www.vistastores.com/lighting
3. Finally, since you have an internal search mechanism, you could choose to redirect to a search for a relevant term.
   For example, redirect to search ?q=lighting

The aim is to always send the visitor to the most relevant or helpful page to improve the chance of them converting.

Hope that helps,

Sha

Hi GaB,

I would suggest a completely different solution.

I would not use 301 Redirects for the Phase 1 change. Rather, I would use a Rewrite Rule to serve the file with the new extension. The Rewrite Rule is basically telling the server to serve OURLEX when a request is made for OURL. The nett result of this is that as far as the Search Engine is concerned, nothing changes.

Here is a Code example which will serve the .php when the .aspx is requested:

RewriteEngine on

RewriteBase /
RewriteRule ^(.*).aspx$ $1.php [L]

Then, when Phase 2 is complete, I would implement the 301 redirects using a single Rule once again to permanently redirect all OURL's to NURL's. So, the solution I would recommend is the last one on your list.

Option 6 - Something else I think is better:

• Phase 1: Use Rewrite Rule to satisfy all requests for the OURL by serving the OURLEX
• Phase 2: create one rule that will 301 redirect all the OURL to the NURL

It is best to keep in mind that a 301 redirect should only be used when you intend the redirect to be permanent. Also while Google's stated position is that googlebot will follow multiple 301 redirects, it has also said that "daisy chaining" them is not generally a good practice. There are two reasons for this:

1. Google's Matt Cutts has indicated that "at some point we will stop following them if there are a lot of them"
   Should you happen to have other domains that are already 301'd to the site, the daisy chain effect is amplified.
2. As Ryan mentioned, the small amount of link juice that is lost with a 301 Redirect does accumulate over multiple redirects and can have a negative impact.

Hope that helps,

Sha

Hi Augustos,

Yes, bounce rate is now acknowledged as a ranking factor by major search engines.

I would say you are probably doing exactly what I talked about in my response to Volkinator - are your conversion rates also improved?

Sha

Hi Volkinator,

I would actually offer a slightly different perspective on this.

I agree that there doesn't appear to be any real evidence that it helps rankings, but...

Think about what works for the visitor ... clever use of bold text can actually help conversions, but you have to think carefully about what you are doing because it can also hurt them terribly if misused.

First, highlighting just repetitive keyword terms or long sections of text is likely to hurt in 2 ways:

1. It is likely to annoy people and therefore increase the bounce rate for the page.
2. long passages of bold text reduce comprehension dramatically - if people don't understand the text on the page they are less likely to convert.

However, you can highlight just short phrases in the text to send a direct message to the reader. This will drive your message home more effectively and increase conversions.

This is an artificially constructed version to demonstrate what I am talking about, but in essence, the highlighted phrases when read alone create a shorter, more direct action statement from within the two longer sentences.

I have found this to be quite effective in improving conversions on client sites. One site in particular went from a conversion rate of 2.2% to a conversion rate of 26% after making only this change.

Hope that helps,

Sha

Hi Ryan,

Sounds like the comment from Ricardo Nunez in Cyrus' Whiteboard Friday on Google's Negative Ranking Factors in July this year, but I don't think Ricardo has written for Youmoz.

Sha

Hi Simon,

I would suggest starting with the basics - getting a good grounding in HTML and CSS will make a huge difference when you are not restricted to getting someone else to make every change for you. You will also find it easier to work with your tech people once you know the basics.

Once you get a taste for it I suspect you'll find yourself seeking out other languages to learn.

Hope that helps,

Sha

Hi Ian,

It's great to hear that Q&A is working for you.

I have to confess that a large share of the "warm fuzzies" in my life emanate from this forum. Seeing so many people sharing and working together here every day always puts a smile on my face.

Thank you too for posting this - as EGOL mentioned, it's great to know what has worked and what the outcome was for your site. Feedback like this is as valuable for the rest of us as the answers were to you.

I would most definitely agree - Q&A alone can make the subscription cost worthwhile

Sha

Hi again Greg,

Just one more option that is available to you if you happen to have a Wordpress blog on the site (or have the option of rebuilding the entire site using Wordpress).

You could install the Bad Behavior plugin for Wordpress. The plugin is part of Project Honeypot, which tracks millions of bad ip addresses and gathers information from the plugin and feeds it back to the honeypot. Bad Behavior also works against link spam, email and content harvesters and other malicious sites.

Sha

Hi,

You can certainly export all of the keywords in your campaign in a single csv file, select keywords using the usual "check the box" system to delete and also bulk add keywords to a campaign.

To export all of the keywords loaded in your campaign, go to the Rankings Screen, change the selector at the top right of the Rankings Report from pdf to csv and click export. You can now open the file in Excel and the first column of the report is your entire keyword list.

To bulk add keywords to your campaign, click Manage Keywords from the Campaign Overview OR Click the Add/Manage Keywords button on the Rankings screen.

You can paste a list into the input field. Keywords can be separated by comma, tab or new line. If you have the keywords in an excel list (maybe exported from Analytics or similar), just copy the keywords from the column and paste.

The one that will be most onerous if you have hundreds of terms is deleting, as terms are listed alphabetically, so you need to work through the list and check the boxes for each before clicking the red Delete Keywords button.

You will find the delete option in Manage Keywords also (below the input to add keywords). If by chance the keywords you want to delete happen to have labels assigned to them, you can use the selector at the top of the screen to choose only keywords with those labels, which would make choosing the checkboxes a little easier.

Hope that helps,

Sha

Sorry Alan,

Obviously misread what you were asking.

The answer is No. You cannot change the domain name. The following is from the FAQ on campaign setup:

Q.  Can I change the domain later?
A.  You cannot change the domain of a campaign once it is created. If you need to track content, rankings, etc. on a new subdomain or domain, you must create a new campaign.  Don’t panic – archiving and adding campaigns is easy.

It is also not possible to duplicate the campaign and change details, so you will need to archive the old campaign and create a new one, but that is not so difficult to do.

The most arduous part of re-creating a campaign is adding the keywords, but this is also quite simple. All you need to do is export your keyword list to csv from the old campaign, open it in Excel, select the keywords in the first column and copy to your clipboard.

When you create the new campaign, just add one "placeholder" keyword in the screen that asks for keywords, then continue setup. When the campaign has been created, click "Manage Keywords" link at the top of the Overview page and paste the keywords from your csv file into the input field. Click Add Keywords and you're done.

If you choose a "placeholder" keyword that was not in the original list then you're fine. If you used one that was already in the list, you will just need to delete the duplicate so it does'nt mess up your reports.

Hope that helps,

Sha

Hi Vjay,

The sitemaps protocol was adopted as a standard by the three major search engines back in 2007.

This is not to say that other methods of submitting a sitemap don't work, but if you follow the sitemap protocol as described at sitemaps.org then you know that you are working within the standard recognized by Google, Bing & Yahoo.

Hope that helps,

Sha

Hi Alan,

You can change the campaign name by clicking Settings in the campaign.

Just change the information in the Campaign name field and save.

Hope that helps,

Sha

hmmm...I like to pick my battles.

Scumbags are scumbags and will always find a way to win in the short term.

I like to live by two things my grandma taught me a long time ago...

"What goes around comes around" and "revenge is a dish best served cold"

As to there being an easy way out - you're an SEO Ryan! You know the deal.

Sha

Hi Ryan,

In this case Greg already knows the site has been scraped and duplicated. Blocking the scraper and serving the image via the bot-response php script is simply a "gift" to the duplicate site if they return to update their stolen content as they often do.

It is entirely possible to put the solution in place for well known scrapers such as Pagegrabber etc, but there are thousands of them, the people using them can easily change the name when they have been outed and anyone can write their own.

I understand that everyone wants a "list", but even if you Google "user agent blacklist" and find one, there will be problems. Adding thousands of rules to your .htaccess will eventually cause processing issues, the list will constantly be out of date etc.

As I explained at the outset, the key is to be aware of what is happening on your server and respond where necessary. Unfortunately, this is not a "set and forget" issue. In my experience though, bots will likely be visible in your logs long before they have scraped your entire site.

Sha

The reason is fairly obvious...gaming search engines with social mentions.

This will never go away because there will always be masses of people out there who will think "what have I got to lose?". People recognize that if it's a scam all they lost was a click and if it isn't they'll get something for nothing.

The general public has no idea that a share is "worth" anything for a website and values a click at zero.

Sucks to be them if someone sees it and decides to let the engines know what is going on though eh?

Sha

I guess the use of bot-response.php and bot-response.gif is the gentle internet version of a public shaming campaign.

Sometimes it's a matter of picking your battles, but engineering enough of a win to make your client feel better without launching into an all-out war that could end up costing way more than you're willing to pay.:)

Sha

bot-response.gif

Authorship markup was recently introduced by Google because there was no reliable way to verify ownership. The markup can be added to any page on any site to show Google that you are the original author.

Sha

Hi Ryan,

The major problem is that any experienced programmer can easily write their own script to scrape a site. So there could be thousands of "bad bots" out there that have not been seen before.

There are a few recurring themes that appear amongst suspicious User Agents that are easy to spot - generally anything that has a name including words like grabber, siphon, leach, downloader, extractor, stripper, sucker or any name with a bad connotation like reaper, vampire, widow etc. Some of these guys just can't help themselves!

The most important thing though is to properly identify the ones that are giving you a problem by checking server logs and tracing where they originate from using Roundtrip DNS and WhoIs Lookups.

Matt Cutts wrote a post a long time ago on how to verify googlebot and of course the method applies to other search engines as well. The doublecheck is to then use WhoIs to verify that the IP address you have falls within the IP range assigned to Google (or whichever search engine you are checking).

If you are experienced at reading server logs it becomes fairly easy to spot spikes in hits, bandwidth etc which will alert you to bots. Depending which server stats package you are using, some or all of the bots may already be highlighted for you. Some packages do a much better job than others. Some provide only a limited list.

If you have access to a programmer who is easy to get along with, the best way to get your head around this is to sit down with them for an hour and walk through the process.

Hope that helps,

Sha

PS - I'm starting to think you sleep less than I do!

Hi Atul,

The most reliable way to show Google ownership of your content is to implement Authorship markup.

Matt Cutts and Othar Hansson explain the different methods in this Google Webmaster Help video on Authorship Markup and this alternative method Using URL Paramaters for Authorship Markup if you are unable to edit tags (if for example you are using a cms).

There is also a very good explanation for bloggers from Yoast that is worth reading.

Hope that helps,

Sha