Hi,

htaccess can be a pain and I will admit I usually manage what I am after with a bit of trial and error. Try the following, and if you have problems concentrate on the lines:

RewriteCond %{THE_REQUEST} .html
RewriteRule ^(.*).html$ /$1 [R=301,L]

I have added a redirect for index.html to root, and from non www to www and removed the last .html from the last list of _ to - redirects. Give it a shot, and keep that backup handy just in case. If no go, maybe one of the htaccess experts around can step in and have a look, I am not 100% sure what some of those other rules are doing to be honest!

<ifmodule pagespeed_module="">ModPagespeed on 
ModPagespeedEnableFilters extend_cache,combine_css, collapse_whitespace,move_css_to_head, remove_comments</ifmodule>

<ifmodule mod_headers.c="">Header set Connection keep-alive</ifmodule>

AddCharset UTF-8 .html

<filesmatch ".(js|css|html|htm|php|xml|swf|flv|ashx)$"="">

#SetOutputFilter DEFLATE 
#</filesmatch>

<ifmodule mod_expires.c="">ExpiresActive On 
ExpiresByType image/gif "access plus 1 years"
ExpiresByType image/jpeg "access plus 1 years"
ExpiresByType image/png "access plus 1 years" 
ExpiresByType image/x-icon "access plus 1 years" 
ExpiresByType image/jpg "access plus 1 years" 
ExpiresByType text/css "access 1 years" 
ExpiresByType text/x-javascript "access 1 years" 
ExpiresByType application/javascript "access 1 years" 
ExpiresByType image/x-icon "access 1 years"</ifmodule>

<files 403.shtml="">order allow,deny allow from all</files>

# mod_rewrite On only needed once
RewriteEngine On

301 permanent redirect old underscore.html to new dash urls

redirect 301 /PRODUCTS http://www.timports.co.uk/kiln-dried-logs 
redirect 301 /kindling_firewood.html http://www.timports.co.uk/kindling-firewood
redirect 301 /about_us.html http://www.timports.co.uk/about-us
redirect 301 /log_delivery.html http://www.timports.co.uk/log-delivery
redirect 301 /oak_boards_delivery.html http://www.timports.co.uk/oak-boards-delivery
redirect 301 /un_edged_oak_boards.html http://www.timports.co.uk/un-edged-oak-boards
redirect 301 /wholesale_logs.html http://www.timports.co.uk/wholesale-logs
redirect 301 /privacy_policy.html http://www.timports.co.uk/privacy-policy
redirect 301 /payment_failed.html http://www.timports.co.uk/payment-failed
redirect 301 /payment_info.html http://www.timports.co.uk/payment-info

301 permanent redirect index.html to folder

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)index.html?\ HTTP/
RewriteRule ^(([^/]+/))index.html?$ http://www.timports.co.uk/$1 [R=301,L]

301 permanent redirect non-www to www

RewriteCond %{HTTP_HOST} !^(www.timports.co.uk)?$
RewriteRule (.*) http://www.timports.co.uk/$1 [R=301,L]

301 permanent redirect all .html to non .html

RewriteCond %{THE_REQUEST} .html
RewriteRule ^(.*).html$ /$1 [R=301,L]

Hi,

I think you will only need this bit:

#301 from example.com/page.html to example.com/page
RewriteCond%{THE_REQUEST}^[A-Z]{3,9}\ /..html\ HTTP/
RewriteRule^(.).html$ /$1 [R=301,L]

And you would replace this bit below with the above:

Rewrite valid requests on .html files  RewriteCond %{REQUEST_FILENAME}.html -f

RewriteRule ^ %{REQUEST_URI}.html?rw=1 [L,QSA] 
 #

Return 404 on direct requests against .html files

RewriteCond %{REQUEST_URI} .html$  
RewriteCond %{QUERY_STRING} !rw=1 [NC] 
 RewriteRule ^ - [R=404]

But leave the  at the end of that section.

htaccess files can be a bit picky, so be sure to keep a backup so you can quickly undo something if it is not working!

Hi,

Indeed there seems to be an issue with your redirects since the .html versions are still available on your site. Two things to check in the first instance:

1. The redirect line for the .html to non .html versions:

Rewrite valid requests on .html files  RewriteCond %{REQUEST_FILENAME}.html -f

RewriteRule ^ %{REQUEST_URI}.html?rw=1 [L,QSA]

I am not sure if this will work the way you want it. First of all a # at the beginning of this line means it is a comment and not processed so you seem to have the RewriteCond part of the statement as a comment (maybe this is just the forum formatting it wrong, but good to check).
You can check some other solutions for redirecting .html to non .html here: http://stackoverflow.com/questions/5730092/how-to-remove-html-from-url

2. At the bottom of the file you have a bunch of 301 redirects like this:

redirect 301 /kindling_firewood.html http://www.timports.co.uk/kindling-firewood.html

Which are working as expected redirecting underscored urls to urls with dashes. But they are also redirecting to the .html version which means you will be getting into double redirects which is pointless in your case. Once you have the non .html redirects working as expected you should adjust these 301s to go to the non .html version like so:

redirect 301 /kindling_firewood.html http://www.timports.co.uk/kindling-firewood

Hope that helps!

Hi Jonny,

As the others have said rel=canonical is the way to go, and as you describe is the way to implement. Just put the rel=canonical tag in the head of your index page (or whichever page is in question) and all the urls with your affiliate ids in them will see the same head and rel=canonical tag and you should be good to go!

Hi Daniel,

Since I was looking I put the site through Screaming Frog (http://www.screamingfrog.co.uk/seo-spider/) and there are a bunch of 404 errors being shown.

I think it must be the relative links you have in your footer. Weird thing is on chrome they work ok, but in firefox they return a page not found! Not sure if this is related in any way to your rankings drop, but even if just for user experience worth having a look and maybe even hard coding those footer links.

You can see an example if you go to http://www.scojo.com/mens-reading-glasses.html in firefox and then click on any of the left hand footer links to the other categories.

Good luck!

Hi Daniel,

For the images, if you download your sitemap here: http://www.scojo.com/sitemap/sitemap.xml and then search it for image:imageyou will find them. The first one for example lists this url: http://www.scojo.com/catalog/product/image/size/0x0/c/o/court_1133.jpg.jpg</image:image>

Which is a 404 (not pointing to a 404 page, just returning a 404 not found).

Hi Daniel,

Maybe something else changed in the weeks prior to this that had an effect, like a template or layout change? Is it just you or have other sites fluctuated in the serps also? You have probably gone through this process, but worth mentioning since the robots.txt itself doesn't look very suspicious.

As for the image indexation, I think I have your culprit. In your existing sitemap http://www.scojo.com/sitemap/sitemap.xml there are 582 image urls and they seem to be all going to a 404 page.

Hope that helps!

Hi Daniel,

At first glance I'm not sure a robots.txt issue would be responsible for the rankings drop you describe. What pages are still blocked that you dont want blocked? Maybe something else is going on in regards the rankings?

For issue 2, have you submitted an image sitemap.xml? If so what is the address? I see plenty of your images in google image search, so some of them at least are being indexed.

As David says a lot depends on the niche/keyword/competition scene, but if I may play the devils advocate:

I am not sure how valid that ctr data is with all the ongoing layout changes google has been trying out. Good ctr data seems nearly impossible to find but this report http://www.slingshotseo.com/wp-content/uploads/2011/07/Google-vs-Bing-CTR-Study-2012.pdf makes for an interesting read, and the numbers are substantially lower.

Beyond that, what you are suggesting sounds a bit like playing with fire! You are making 8 grand a month now on 3rd? Surely it would less time consuming to further strengthen that site than to build another one from scratch? How long did it take for you to get to 3rd?

And 301'ing a near exact match domain to an exact match domain in today's landscape.... well that just sounds a bit risky!

I think David hits it on the head with: "The other way is to make a few extra bucks and invest in them in tissues".

Just my two cents.

Hi Bob,

The first thing I looked for when visiting your site was a big "what is nlp" button, I really had no idea! Maybe google is thinking (or has started thinking) a bit like this for the search term nlp also? With the potential for local results etc mixed in it seems to me that the term could be a bit volitile in terms of rankings and your experience possibly hints at that.

I see the sitelinks you mention if I search for nplcalifornia with no space, but am not well aquainted with these expanded sitelinks and their finer points, maybe someone else?

I would still be wary of messing too much with those site wide links!

I don't think changing the link text on those two sites is going to help much to be honest. If anything I would be tempted to try and reduce the amount of sitewide links, there seem to be a few.

A couple of things come to mind.

1. Why should you be ranking for NLP? I am searching from Europe and the first page has some heavy hitters (wikipedia, stanford U), some local(ish) results and a few nlp.something type sites that I guess might be your online competitors?

The term NLP is a bit ambiguous, It could be Neuro-Linguistic Programming, it could be Natural Language Programming (the Standford link) or it could be the NLP Theatre down the road. So if google is going to return results based on what it thinks the user intent is then you have a mix of local and 'what is NLP' type results which seems pretty reasonable.

So with that in mind, I'm not sure your site has enough 'What is NLP' type content to justify ranking on the first page and maybe that is something to consider in terms of adding content.

2. Sitelinks seem to come and go depending on the search term and who knows what else so I wouldn't worry too much if you are not seeing them right now. The good news is that searching for "NLP California" from where I am you are number one, sitelinks and all!

Yes I think microdata would help a lot in this case, and once you have a template setup it really takes no time at all to implement!

Hi James,

That will probably do it. If you want to avoid having to fiddle with the paging code etc check out something like (grabbed the first link I saw) http://www.jquery4u.com/function-demos/ajax/

Loading content with ajax/jquery is pretty straightforward

I will have a stab at it

I am a bit confused over your main problem since you say that individual properties have great unique content and are good for sharing, but then that they have similar descriptions and you would prefer for them not to be emphasized in google.

If I understand right, this seems to be mainly an architecture type issue since you want to put emphasis on location + type of accommodation so that users are landing on the 'category' page such as it is and can then drill down.

If so, I don't think you have any problems if the category/url structure is right so something like
Florida/Miami/Condos gives a pretty good indication of the way you are intending visitors to discover your content. You will want to make sure that each page in that trail has some unique content beyond a simple list of properties. The individual property pages could then be optimized more for the name of the rental than its location (although obviously location would also be included).

I would think that schema markup might be your friend here. Have a look at http://schema.org/LodgingBusiness and particularly the location details and see how you can apply this to the individual pages.You could then replicate the locality information through your url/folder structure.

Using breadcrumbs (and structured markup with them) would also help emphasize this to both users and search engines... and maybe you will get some of those lovely breadcrumb links in your serps!

Hope that makes sense!

As Andy says you could pull the information into a popup/lightbox/div using javascript and source it all from the same url, so should do the trick.

Just thinking out loud now, but it might be worth testing putting the generic texts back into some of the product pages and seeing what happens. 250 word unique texts for each product might be enough to make the moz crawler (and the search engines) see enough value and difference on each page for it not to be an issue anymore. You could test in a single category for example and cross check against analytics and ranking data to see what happens.

Hi

This should do it, you place the robots.txt in the root directory of your site.

User-agent: * Disallow: /product-category/

You can check out some more examples here: http://www.seomoz.org/learn-seo/robotstxt

As for the multiple urls linking to the same pages, you will just need to check all possible variants and make sure you have them covered in the robots.txt file.

Google webmaster tools has a page where you can use to check if the robots.txt file is doing what you expect it to do (under Health -> Blocked Urls).

It might be easier to block the pages with a meta tag as described in the link above if you are running a plugin allowing this, that should take care of all the different url structures also.

Hope that helps!

Indeed that is another question.

I suppose it might if you use it as a second instance of describing 'x with y' in a page's text although the word 'with' is so generic in many instances that I'm not sure it would make a huge difference... I could be wrong though!

Hi Faye,

As Martijn said there is an issue with your main menu, if you turn off javascript in firefox and visit the site... no menu at all!

You can see the result in your sitemap.xml also, only 2 urls are created in it, the homepage and the dentists page which you have a link to in the body text.

Try redoing your menu in css, it should fix the problem. Have a look here:
http://www.cssnewbie.com/easy-css-dropdown-menus/

I just grabbed this example, if you search for css dropdown menus you will find many more.

Hi Matt,

You will want to be careful of duplicate content issues since all product and static blocks etc will be sourced from the same database. But this is something you can edit on a store view by store view basis so as long as you make sure the content is suitable then really from an SEO point of view it is like you have 3 different sites.

It shouldn't effect domain authority.

The multi domain setup in Magento is usually done through the use of virtual hosts on the server side in combination with different base urls for each of the main store views.

Search engines would still see the 3 domains as 3 separate domains and links pointing to the individual domains would still be pointing to separate domains.... so I don't think there would be a problem in regards domain authority.

You have probably seen it already, but here is the official Magento description of the setup procedure: http://www.magentocommerce.com/knowledge-base/entry/tutorial-multi-site-multi-domain-setup