Welcome to the Q&A Forum

Hi,

That line of htaccess says:

If the host is legacytravel.com OR www.legacytravel.com, then lets try this rule below.

One conditions will always be true since the htaccess is on your site. So they cancel each other out, and are not needed.

The rule says if the URI matches the regular expression '^carrollton-travel-agent$' then redirect (r=301) to the target URL and stop processing rules (L flag for last)

Your RewriteRule target doesn't need characters escaping because it is not a regular expression, just a URL to redirect to. You also don't need the quotes, they're useful for grouping arguments with spaces in them but that's not affecting it.

Really this rewriterule doesn't accomplish anything, it redirects /carrollton-travel-agent to /carrollton-travel-agent, forcing the www. no matter which version is accessed. But if that is what you're wishing to achieve then there's a much simpler solution:

RewriteCond %{HTTP_HOST} !^www.legacytravel.com$ [NC]

RewriteRule ^(.*)$ http://www.legacytravel.com/$1 [R=301,L]

Notice we only use the \backslash to escape characters in regex patterns, and not in our target URL. What that rule will do is redirect any visitor where the host is missing the www to the equivalent page with the preceding www.

Hope that helps.

Hi,

Since an iframe is just a window into your site, it cannot harm your site and it doesn’t put you in a spammy neighbourhood as all of the links to your page are from within your own site.

It will be registering views and visits on your analytics too, so you’re still getting some extra traffic. If you feel this Iranian traffic is unwanted, not converting or are simply just annoyed that this site is using your content as their own, you can actually block iframe calls quite easily.

There are a couple of things you could do:

.htaccess

Simply add this line to enable iframe calls from your own domain, but block them from external sites:

Header always append X-Frame-Options SAMEORIGIN

Or disable iframe calls entirely with:

Header set X-Frame-Options DENY

These directives will only work if you have enabled mod_headers in apache.

Another way to deal with it would be to use a javascript redirect when the top url does not match the page URL like this:

if(top != self) { top.location = self.location; }

Hope this helps,

Tom

Hi Jamie,

If you don’t need the http version accessible and want to force the https you could simply redirect all traffic to the secure site with a 301, transferring all your pagerank to the main site.

If you need both versions of the site accessible, for instance if you only needed https for logged in users, and you only want one version to appear in SERPs the best thing would be to use a canonical tag to consolidate all that SEO juice into the version you wish to rank.

If there’s only a few secure pages with links to other non-secure pages then meta robots noindex,follow would work well, since the SEO juice will flow through those noindexed page and into the rest of your site, but if the whole site is duplicated on both versions this could be a big mistake.

No-indexing an entire https version would be a bad move even if you were using noindex,follow since your internal linking will be to the secure pages. Even though pagerank will be passed through those pages it will eventually come to a dead end or leave through an external links. With the canonical tag, any links pointing to your secure version will pass their SEO juice to the non-secure site, rather than be lost in the noindexed site where it has nowhere to go.

Have a little read of this interview with Matt Cutts a few years back for further clarification, it’s got a good quote about how PR flows through noindexed, followed pages: http://www.stonetemple.com/articles/interview-matt-cutts.shtml

Matt Cutts: A NoIndex page can accumulate PageRank, because the links are still followed outwards from a NoIndex page.

Eric Enge: So, it can accumulate and pass PageRank.

Matt Cutts: Right, and it will still accumulate PageRank, but it won't be showing in our Index. So, I wouldn't make a NoIndex page that itself is a dead end. You can make a NoIndex page that has links to lots of other pages.

So it’ll be different depending on your circumstances but if you’re in doubt, the canonical tag is your best bet as you’re only consolidating those pages in googles eyes. If those pages perform well and you noindex them without sending that PR somewhere useful you could be throwing away all that benefit.

Hope that helps,

Tom

Hi there,

Question 1: I'll come back to that...

Question 2: You can use redirect chains but you shouldn't. Although a desktop browser will redirect through the chain quite quickly (providing there are no mistakes), a mobile browser takes 0.6 seconds to get a connection to a page, so each redirect quickly adds up to poor user experience for any mobile users. And although Google has stated it’s crawler does well to deal with 1 or 2 redirects, it can come across problems with longer redirect chains and you could see your final page not getting crawled from the redirect as it should. Both Yahoo and Bing have stated their crawlers do not perform well when it comes to redirect chains. As you make the transition to the new domain you should update your original redirects to send the visitor to the correct page after the first redirect.

Question 3: Best practise for redirects is to specify the more specific rules first, if you're using regex with redirectmatch or rewrite rule, then you'll want to put them after your more specific oldpage.html to newpage.html, so that the more specific rule is given the chance to match before the regular expression is given a more broad chance to match. And finally add an 'if all else fails' rule at the bottom to redirect all requests that were not dealt with by a previous directive.

There's a nice post here on execution order in your htaccess if you'd like to give it a read:http://www.adrianworlddesign.com/Knowledge-Base/Web-Hosting/Apache-Web-Server/Execution-order

Back to Question 1:

If you're choosing not to follow Daniels advice and make the change all at once, you can 301 the new sites backlinks into the existing site, and 301 the old URL structure to the new URLs. But when you do implement the change, you'll want to modify all of your existing redirect to point to the final page the user should end up at, and not force them through a maze of redirects. Then you can then just remove the redirect from the new site, and have those users land on the pages the new links you built are pointing at. Don’t forget Google takes time recrawl, index and ‘trust’ new redirects and attribute all PR and SEO juice to the correct pages.

If at any point you plan on having the same content live on both domains without a redirect in place it would be best practise to use the rel=canonical link attribute in your to signal to google which is the preferred version on content to show in SERPs.

Hope that helps give a bit more information,

Tom

Hi,

Both versions of the URL aren’t appearing in search results but both are accessible from the same URL, depending on which device you attempt to access the site with. But when you search for a page on your site on a mobile phone and hit the link to the page you’re requesting, then you're directed toward the mobile page thanks to your redirect, even though you aren’t hitting a link to the mobile subdomain.

If you wanted to remove the canonical tags from all your pages that would see your mobile pages indexed in SERPs, but you’d be causing yourself a duplicate content issue and forcing both those pages to compete with one another for the same keyword(s).

So when you search for a page on your site in Google after removing the canonical tag, you’d see the full site URL and then the mobile URL, giving the user the choice to click either. When they click to go to the full site they’ll still be redirected to your mobile page.

I can understand your concern that your mobile site isn’t getting indexed and therefore will not be found in Google but you’re misunderstanding the point - these pages are essentially 2 versions of the same page, containing the same content and different styles. Rather than forcing mobile and main site to compete with each other, you can have the best of both worlds with your current (correct) implementation of the canonical tag.

Thanks,

Tom

Hi there,

I’ve taken a quick look at your site, and the reason these pages are not getting indexed is because of the rel=canonical link element in your which indicates to google which version of the page you’d like to be indexed. Note this is just a guideline to Google, but since you’re using it properly they are only indexing the none mobile version of your site.

I can see that you’ve got a redirect in place to send mobile traffic to the mobile page, and there is no need to have both versions of a single page being indexed, and therefore competing against each other in SERPs. As these pages are next to identical in content, googlebot is seeing your rel=canonical and attributing all the SEO juice to your original page.

When someone searches for one of your products they see the result for your desktop page as only one result, not 2 results competing against each other. The user then clicks on the link and is directed toward the mobile site if they’re a mobile user, and your full site if they’re a desktop user.

Take a look at this article on the Google Webmasters Blog:

http://googlewebmastercentral.blogspot.co.uk/2009/02/specify-your-canonical.html

This line is key: ‘With rel="canonical", properties of the two URLs are consolidated in our index and search results display wikia.com's intended version.’

Let me know if there’s something I’ve missed here, but it looks to me as though you are using the rel=canonical tag correctly.

Thanks,

Tom

Hi Rich,

Elaborating on FedeEinhorn’s answer, if the page structure is the same you could just redirect all requests to the same URI on your new domain, as he stated. You could do this very easily with a .htaccess file on the root folder of your old domain (providing you’re running an apache webserver like most people). To redirect using regular expressions and capture groups we can use the RedirectMatch directive, which would look like this:

RedirectMatch 301 ^(.*)$ http://www.newsite.com$1

As simple as that, you’ve redirected all existing pages to the same page on the new domain. If you haven't used this before, here's a brief look at how that works for you:

Firstly, RedirectMatch simply tells apache we’re using the RedirectMatch directive from mod_alias.

301 specifies a 301 SEO friendly redirect which passes all that lovely SEO juice to your new site.

^(.*)$ is our regular expression. It states, from the start of the requested URI (^) start capturing the request (using the brackets to show what we want to capture), capture it all (with . meaning any character or symbol and the * meaning 0 or more of the preceding . , which will lead to everything being caught by our capture group (the brackets). And the $ meaning the end of the requested URI.

The final part of this redirect is specifying the page to redirect to, but as we have captured the request in the previous part, we use $1 to append our first capture (only capture in this distance) to the end of our new domain.

If you have completely changed your site, you may wish to redirect all requests to your homepage or another page, it is as easy as modifying the previous code to redirect without appending our capture to the end of your redirection target, so this would be acceptable:

RedirectMatch 301 ^(.*)$ http://www.newsite.com

But since we don’t need to use anything from the requested URI, we should really remove the brackets (the capture group) for the sake of tidiness, resulting in:

RedirectMatch 301 ^.*$ http://www.newsite.com

You could use a mixture of these 2 code, for instance if your blog posts are all identical but your main site pages have all changed - this code would redirect all pages starting with /blog/ to their double on the new domain, but redirect all other pages to a /we-have-moved/ landing page:

RedirectMatch 301 ^(/blog/.*)$ http://www.newsite.com$1

RedirectMatch 301 ^.*$ http://www.newsite.com/we-have-moved/

Hope that's useful,

Tom

Directories and forum posts as easy backlinks are being devalued more and more by google, so there often isn't a lot of SEO value for these links.

Just have a look at this video on searchenginewatch: http://searchenginewatch.com/article/2326161/Matt-Cutts-Using-Article-Directories-for-Links-Just-No

But that's not to say that forums and directories don't have their place in a natural link profile, but you want to select decent directories and forums based on whether or not they will send the traffic you want to your site. For example, directories called things like "seobacklinkdirectory" or "freeseolinks" that promise followed indexed backlinks probably aren't being used by humans to find websites, but by SEOs wanting free back links. These sorts of links are not going to benefit your business or your site.

But a nice niche directory where your audience will search for businesses like yours could add value, and maybe lead to some conversions. The same applies for forum posting - if you're adding value with your comments and subtly linking back to your site every now and again, you'll drive relevant traffic to your site which has the potential to convert. If you're dropping 100 links into a different forum each day before they ban you, you're probably not going to see any benefit.

Hi Tom,

I agree with David on this - the best way to obtain links to your content is to create linkworthy content.

If the content you create is interesting, value-adding and answers a question that your industry/readers are asking then it should naturally acquire links. You will also find that you pick up links if you have content on your site that is seen as a great industry resource - this might come from being the first person in the niche to create content on a particular topic or event, or it could come from addressing an issue from an unusual standpoint.

You can also increase the link potential of your content by looking at the existing content on the domain you'd like a link from. Are there any gaps in the content - and could you write something that would fill it? This makes a conversation with your desired domain much easier as you can approach them in a friendly tone and manner and show them what you've written, and explain why it's of value to their readers.

Good luck, hope the above helps

Tom

Hi,

If you simply want to stop these pages from being indexed until they have some non-duplicate content on them, you could use the meta robots tag in your

This is how that would look:

<meta name="robots" content="<strong>noindex</strong>"></meta name="robots">

And this will immediately drop these pages from the index when googlebot crawls a page with this directive in the

If you wish to stop these pages being crawled by rogerbot (moz’s crawler) so the duplicate content errors do not show up, you could add these lines to your robots.txt:

User-agent: rogerbot

Disallow: /oopspage1.html

Disallow:/oops/page2.php

User-agent: *
Allow: /

This will still allow googlebot  to crawl these pages, and will keep them in the index, so when you do update the content you won’t have any problems if you forget to remove the meta robots tag on these pages.

Be careful with robots.txt and remember that Disallow:/oops/ will stop crawlers from crawling subpages of the /oops/ folder, so you need to remember your filenames. But in this case, since you’re only setting these rules for rogerbot, it wouldn’t be catastrophic for your site but it’s best to take care either way.

In reality though, duplicate content is a problem when multiple pages are competing with each other for the same terms. Sometimes the wrong version of the page will be displayed above your preferred version. Unless you’re looking to rank for your ‘Oops nothing here’ phrase, these duplicate content problems are nothing to worry about and the notices will soon disappear from your crawl errors when you create some fresh content for them.

Hope this helps,

Tom

No problem Derrick, my pleasure.

Tom

Hi Derrick,

if you wish to use robots.txt you could simply use:

Allow: /service-area/*
Disallow: /service-area/

This will allow access to any child of /service-area/ but not /service-area/.

You could redirect this page to your homepage if you wished, and to stop children of this page being redirected you could use RedirectMatch instead of the Redirect directive and use a simple regular expression to only redirect if the URI ends with /service-area/, like this:

RedirectMatch 301 /service-area/?$ http://www.lvpoolcleaners.com/

The $ sign at the end signs that the apache should only redirect if the URI is ending in that pattern, and the ? after the trailing / allows the redirect to happen with or without the trailing slash.

But perhaps the simplest solution to this problem would be making your /service-area/ link point to '#' if the Joomla menu will allow it. This will append an empty anchor to the url, it will not refresh or redirect the page and anchors in URLs are not counted as duplicate URLs.

For human usability this would be the nicest way to interact with the menu, as you don't want a visitor being interrupted mid-way through their buying cycle by being sent back to the homepage when they didn't ask for it.

Hope that helps!

Hi,

Duplicate content is bad for search because it forces pages on your site to compete with each other for rank. If each tag only contains one post then you are duplicating every post twice - once in the original post and once in the tag which is displaying the same post and only that post.

If you use a lot of the same tags for each post, for example you tag every post 'blog' and 'daily’, then those pages will contain the same posts and therefore be duplicate content.

It may be worth checking your analytics to see if any of these pages are getting entrances from organic search, which will tell you if the 'duplicate' is outranking the original post. But often this is because that page contains a lot more information on the subject than a single blog post. So you may not be able to replicate that success with a smaller single blog post.

As the previous answer stated using a robots.txt indicating Disallow: /tags/whatever2 will tell a spider not to crawl that page, you could do it selectively by disallowing only the tags which are being flagged as duplicates, or disallow all tagged pages from being crawled with Disallow: /tags/*

But every site is different and you will need to decide for yourself if your 'duplicate' content is actually harming your site, you might find that those pages are full of keywords naturally and are attracting all your traffic.

Hope that helps!

A new client has come to me because they have found themselves in hot water with Google having received a manual spam action for unnatural inbound links.

The sad case in this story is that I completely believe that the client has been a victim of an attack by a competitor. They finally made it onto page 1 in their competitive niche, and within a few days random links started appearing on spammy sites (often foreign language sites.) By the time the message came from Google about 2 weeks later several thousand of these links had been built.

The first stage was to get the client to be very honest with me about anything he personally had done that might be considered manipulative. Unfortunately following some bad advice several months ago the client purchased one site-wide link (already in the process of removing it.) The same company that gave him this advice also built just under 100 links to his website (over the course of a couple of days) in early December.

So - we know the client hasn't been whiter than white, and we are going to undo anything that he had responsibility for asap. We are also working to ensure that he is earning really high quality links in the right way (already have some great press coverage in the pipeline and are working on unique content.)

My question is - given some past mistakes made by the client - is there any way that we credibly get across the fact that this recent huge volume of spam is absolutely nothing to do with him in a reconsideration request? Of course we can start work on removing these links and can disavow anything we can't remove - but my expectation is that should this be successful the same competitor is going to continue throwing spam links at my client.

I appreciate that previous actions by the client would in themselves have been worthy of a manual spam action - but it seems far too much of a coincidence given the timing if this penalty was unrelated to the recent attack.

I'd really appreciate any insights from the Moz community and will look forward to sharing our eventual success story as a YouMoz post!

Tom

Hi I'm working on a clients site that has an odd structure with 8000+ 302 temporary redirects. They are related to actions on the site (they have to be there and work this way for the site to function) but they also have a stupid number of perameters. Would it be ok to block them all in the robots.txt file? Would that make any difference?