You shouldn't use rel canonical for pagination - it's main use is to avoid duplicate content issues. It's possible to combine it with rel next/prev but in very specific cases - example can be found here: https://support.google.com/webmasters/answer/1663744?hl=en :

rel="next" and rel="prev" are orthogonal concepts to rel="canonical". You can include both declarations. For example, http://www.example.com/article?story=abc&page=2&sessionid=123 may contain:

=> as you can see the canonical is used to strip the sessionid which could cause duplicate content issues - not to solve the pagination issue

With rel next/previous you indicate to google that the sequence of pages should be considered as one page - which makes sense if you have like 4/5 pages max. If you have a huge number of pages in a pagination this doesn't really make sense. In that case you could just decide to do nothing - or only have the first page indexed - and the other pages have a noindex/follow tag.

Hope this clarifies.

Dirk

Hi,

You're quite right that having clean readable url's are usefull - both for visitors & bots.

There is no technical need to have these 'ugly' urls - as they can always be rewritten to something nicer. You will have to use a combination of URL rewriting & redirects) - you can find some useful links here on how to implement the rewriting (the article is not very recent - but these basics haven't changed). If they use a CMS it could also be useful to check the documentation - almost every decent CMS offers some build-in rewriting functionality.

The second issue with the strange domain name can be solved with a 301 redirect - by adding these lines in the .htaccess file of the "strange domain"

RewriteEngine On
RewriteCond %{HTTP_HOST} ^olddomain.com$ [OR]
RewriteCond %{HTTP_HOST} ^www.olddomain.com$
RewriteRule (.*)$ http://www.newdomain.com/$1 [R=301,L]

(no need to tell that you'll have to replace olddomain & newdomain by the actual domain names)

Apart from the wrong domain the issue with the tracking parameters in

http://website1-px.rtrk.com/?utm_source=ReachLocal&utm_medium=PPC&utm_campaign=AssetManagement&reference_id=15&publisher=yellowpages&placement=ypwebsitemip&action_target=listing_website

could be solved by either a redirect or a canonical url. With the redirect rule above the webwite-px.rtk.com will be redirected to www.yourdomain.com - but this doesn't get rid of the tracking code.

You could put a self referencing canonical url in the head of the pages -

or strip of the parameters using a redirect (you can find an example on how this could be done here

If you use the canonical solution - it could be a good idea to strip off the parameters in Google Analytics

Hope this helps,

Dirk

No - it won't be seen as manipulative, in fact it is the recommended way to syndicate content. Check https://support.google.com/webmasters/answer/139066:

"Addressing syndicated content. If you syndicate your content for publication on other domains, you want to consolidate page ranking to your preferred URL.

To address these issues, we recommend you define a canonical URL for content (or equivalent content) available through multiple URLs"

These emors are the problems the googlebot encounters while crawling your site. A site map can help the googlebot to better crawl your site but isn't strictly necessary .

rgds

Dirk

I agree with Jordan on this - shouldn't cause troubles.

Just make sure that you at least adapt the wording on the site  - we might both speak dutch but not all the words have the same meaning & we don't use the same words to describe the same things. As an example - in Belgium we like "konfituur" - you prefer "jam" - pretty useless to try put a page optimised for "jam" in Belgium as nobody will look for it.

Dirk

If I understand it well it will be something like directory.com/listing.htm links to directory.com/companypage which is then redirected to www.company.com (so users actually never see directory.com/companypage).

I guess this type of link will be considered as a "follow" type link as server side redirects pass link juice to the destination (unless they block directory.com/companypage for indexing with their robots.txt and/or they put a nofollow on all the links to directory.com/companypage)

Dirk

Bob,

It depends on the category & type of product. I remember a Dutch site selling shutters who just put the NL content on a BE domain - problem was that in Belgium we don't use this word when looking for this type of product and hence Google wasn't showing the site (they did rank pos. 1 for shutters in Belgium but probably with 0 traffic)

You don't have to rewrite the content for Google - but it would probably be a good idea to let a Flemish person check the content. If it's just a small word here and there it's no problem - if it's about your main keywords then it's an issue

To reply to your other question - when searching in BE I quite often get NL results if Google doesn't find a good BE result or the NL site is just better. You could just put the content on the be domain - and see if it brings results (even without doing the cross-linking - although I think that would be a useful feature). Belgian backlinks will always help - but it will take time & effort. Take a trial & error approach - there is no risk - if it doesn't work you can always improve later on.

Dirk

While I agree with the concept of Danny's answer (the farther content is from the root the lower the possibility of ranking) - there is no relationship between the depth of an article & the structure of the url.

To put it extreme - a url like mydomain.com/folder/subfolder/subsubfolder/page could be at 1 click from the homepage & a url like mydomain.com/page could be at 5 clicks from the homepage.

I would migrate to the structure you propose:domain.com/blog/post.htm, mainly because it's easier for reporting purposes. You can find an interesting article from Bruce Clay here on why it's good to have a well structured url: http://www.bruceclay.com/blog/structured-urls/

rgds,

Dirk

It's probably a good thing - I would keep them blocked.

Check https://www.mattcutts.com/blog/search-results-in-search-results/ - quote "Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines."

Dirk

Hi,

You might want to read this article on faceted navigation on the google webmaster blog which gives some good advice on how to handle the situation. What to use depends a bit on your actual situation.
Options include using a nofollow links / use a separate subdomain or block in robots.txt (using a separate folder).

On Moz there is this article (the part of faceting) - its mainly about listing sites - but the core problem is more or less similar.

Hope this helps,

Dirk

Not really an expert in robots.txt - but you could always try to simulate Rogerbot using a tool like Screaming Frog (the tool is free if you crawl less than 500 pages).

You can enter a custom user agent. Rogerbot seems to be rogerbot/1.0 (http://moz.com/help/pro/what-is-rogerbot-, rogerbot-crawler+shiny@moz.com) and try crawling the pages on your site. In default mode, the tool will only crawl pages that are allowed by the robots.txt. If the tool crawls your pages, then something is wrong with the robots.txt - if not, than you should contact Moz staff to check with them.

If you look at the results on Google fr - I find it more surprising that apart from the first result - all the other results that are shown are coming from the .com version rather than the .fr version. If I search for Revolve cloathing on google.pt - I only get the US results & instagram.

You seem to use a system of ip detection - if you visit the French site from an American ip address you are redirect to the .com version (at least for the desktop version) - check this screenshot from the French site taken with a American ip address: http://www.webpagetest.org/screen_shot.php?test=150930_BN_1DSQ&run=1&cached=0 => this is clearly the US version. Remember that the main googlebot is surfing from a Californian ip - so he will mainly see the US version - there are bots that visit with other ip's but they don't guarantee that these visit with the same frequency & same depth (https://support.google.com/webmasters/answer/6144055?hl=en). This could be the reason of your problem.

On top of that - your HTML is huge - the example page you mention has 13038 lines of HTML code and takes ages to load ( 16sec - http://www.webpagetest.org/result/150930_VJ_1KRP/ ). Size is a whopping 6000KB. Speed score  for Google : 39%. You might want to look to that.

Hope this helps,

Dirk

Hi Jamie,

The link in your profile looks ok (it's not a "nofollow") - I guess that the issue is more related to the data delays of Moz OSE.

rgds

Dirk

Hi,

For questions about redirect Google & Stackoverflow are your best friends:

http://stackoverflow.com/questions/18998608/redirect-folder-to-another-with-htaccess

Put this code in your htacess (if you already have rewrite rules you just have to add the rewrite rules (bold) before or after the existing ones and not the rest of the code - difficult to say if it needs to be before or after - it depends on the rules that you already have)

Options +FollowSymLinks -MultiViews

Turn mod_rewrite on

RewriteEngine On
RewriteBase /

RewriteRule ^label/(.*)$ /tags/$1 [L,NC,R=301]

rgds,

Dirk

You can't force Moz to crawl all backlinks or submit backlinks. While the Moz bot is crawling a huge number of sites/pages (check this page for the stats). Even when these figures are impressive - it's still only a fraction of the total web and what Googlebot is crawling/indexing. So it's quite possible that some backlinks were not (yet) discovered by Moz. Check also the FAQ on how the backlinks are discovered.There just was an update - so you'll have to check again after Nov. 15th.

As each supplier is using a different method to crawl the web - it's a good idea to check multiple resources to get your backlink profile (raven tools, afhref,...)

Dirk

The biggest problem with duplicate content (in most cases) is that you leave it up to Google (or search engines in general) to decide to which version they are going to send the traffic.

I assume that if you have a site in https you would want all visits to go to the https version. Rather than using a canonical url (which is merely a friendly request) I would 301 the http to the https version.

Dirk

Hi Rick,

Don't fully agree on that - having almost identical content on different TLD's isn't necessary considered duplicate content - as indicated in this video: https://www.youtube.com/watch?gl=BE&v=Ets7nHOV1Yo. The sites also use the hreflang tag to indicate that they are related & which version should be shown to which user.

I do agree that it would be better to take into account country specifics like currency & spelling

rgds,

Dirk

Hi

You didn't answer the first part of Zoe's question - are you sure that your site is crawlable and that there are no issues with the robots.txt / noindex tags, ip detection  systems, canonicals on all pages pointing to the home and so on. It's not because you can see all pages of your site in a browser that they are accessible/crawlable/indexable by google.

Try a crawl with Screaming Frog and user agent Googlebot to see if your pages can be crawled and indexed.

Backlinks are needed to have your site ranked for keywords - but it's not a prerequisite to have your site crawled. (noticed that a few times when a dev site was indexed by accident)

Without the actual url it's impossible to give a more detailed answer.

Dirk

Hi Justin,

It's quite common to have links between different versions of site if they are targeting different countries. It could also be considered good for the user experience to offer the user the choice to choose the site which corresponds to his country.

You just have to make sure that the links between these different versions are voluntary not accidental links. If a user is browsing your site and suddenly switches from to the .com to the .co.nz because the internal links on your site are not properly configured this could be considered as bad practice.

As usual - Screaming Frog is your best friend for this. Crawl the sites, check the pages with outgoing links to the other versions. If the link is accidental - correct it.

Hope this helps,

Dirk

If the redirection was done quite recently - it's possible that Moz hasn't recrawled your site (crawls are once/week)

If this isn't the case - check the http header of the pages which Moz indicates as duplicates, to be sure that they are properly redirected (with a tool like httpstatus.io).

Adding a canonical never hurts but it's better to solve the problem at the root.

Dirk