Welcome to the Q&A Forum

If you just want to block them from the index then best practice is usually considered to do it with a robots meta tag rather than the robots.txt file. See here for a better run down of the options and why one is better than the other for various situations: http://moz.com/learn/seo/robotstxt and also a good post here on duplicate/thin content: http://moz.com/blog/fat-pandas-and-thin-content

I would think that in your case you might be able to get enough unique content on those pages (at least the company pages) to make them more acceptable to google. Thinking off the top of my head you could:

• Add information to the page meta and title tags so that the companies location is included, and your brand is also mentioned
• Force the companies to add a proper profile!
• Add some of your own unique content about the company, the areas they service, their specialties etc etc
• once you have better content then encourage users to leave comments/ratings etc

There are not THAT many company pages in your robots.txt so with a bit of brainstorming on the best way to phrase things and to increase unique content on those pages you should be able to come up with a strategy that will allow you with a bit of effort to comply with the google guidelines and also make the pages more user friendly and more informative generally.

Tom sums it up well.

In most if not all of such cases you should ask yourself if this is a 'normal' situation for a 'normal' search query and I think you will find that the answer is no. If the sites are all owned by the same person then it is certainly borderline if this situation is resulting in searchers finding enough variety in the results for them to say that google is returning the best results. Further to that, for any semi-competitive keyword it would be quite unusual for this situation to be happening so I would ask myself if it is worth the bother maintaining all these sites and tracking their rankings. Chances are that at some point the algo will mix things up and some (all?!) of the sites will move out of the top 10.

So, officially against the guidelines? (assuming no funny stuff) probably not. An un-natural situation that is unlikely to be very stable and/or worth the effort of maintaining it... probably.

Hi Wayne,

Have you seen this post (also from David)? http://www.davidmihm.com/blog/local-seo/local-search-ecosystem-canada/

I can verify that yellowpages.ca seems to be a big player since I see it mentioned in a lot of the google maps I look at as a data provider... only issue is I am in Greece looking for Greek businesses... go figure!

Hi,

If I understand right these are internal links on your own site? If that is the case then it is not a huge deal (although you should be careful to not go overboard on keyword stuffing your internal links). Sitewide footer and sidebar links on EXTERNAL sites are often done with an eye on boosting search engine rankings and as such have been de-valued by google and sites using this tactic a lot have in some cases been penalized. This is not so relevant for your own site where it is perfectly normal to expect to see footer and sidebar links highlighting main pages/categories of the site. Just be careful to not keyword stuff too much and try to link in a way that helps user experience and feels natural and you should be ok.

Hope that helps!

Hi Maja,

Any domain switch has some uncertainty involved in it in regards rankings and possible negative effects. Adding a new url structure will add more complexity to that mix and potentially make it more difficult to troubleshoot problems. That being said, if most of your branded search is coming into domain1 and you want to keep that going, it might make more sense to keep the old domain as it is and add an offline sub folder to it instead of the other way around? That might not fit your business plan or model, but I would be careful about changing domain and url structure at the same time on a site that is bringing in most of your traffic!

Check out this useful post: http://moz.com/blog/achieving-an-seo-friendly-domain-migration-the-infographic which has some links at the bottom for useful further reading.

Hope that helps!

Hi Mattew,

Honestly, that is probably not going to move the needle much.

In regards press I was not meaning a press release, these are low value links. I was meaning more links from sites like the bbc, telegraph etc, real press links in other words. Some of your competitors have a few, I thought I saw one from the ft.com in your profile. These kind of links referencing your input/expertise in regards the french property market is what I meant.

If you can replace the link in your article websites then ok, I guess worth doing just to bring the numbers down a bit, but again the crucial thing will be to bring your higher quality link ratio up and that will take time and effort. As for GWT, I would think google already had a pretty good idea you were UK based/aimed so I doubt this will change things much. When I mentioned other countries I meant seeing what kind of good qualified traffic you got from other countries (if relevant). How about Ireland for example? A good link or two from an Irish press mention (for example) might have a better chance of effecting local Irish searches in the short term while you are patiently waiting for the UK rankings to change. Same could possible go for other english speaking countries, but you will know better than I do which countries if any are relevant.

Hope that helps!

Hi Mathew,

I agree with Dennis, your link profile looks unbalanced. There is some talk about an algo update that was moving rankings around in early-mid May and my experience for a few sites has been that those that have an (even slight) overabundance of exact match links in have been pushed down a bit (after being as you say quite regular for some time). This might be the case for you, your domain seems to have a lot more exact match phrase links than the others above you (I see you at #5 for french estate agents).

I would try to reduce the number of exact match links you have, especially from low quality directories and the like, but ideally you would be supplementing these with higher quality branded links to balance out your link profile. This wont happen overnight though.

Other than working on your content/onsite stuff and trying to improve user experience where possible it might just be a matter of making the link profile look more even, getting in a few more high quality links (a couple of good press links for you/some of your competitors, maybe you can do some more work there) and then being patient to see what happens!

Also might be worth seeing if there are any country variations apparent in your dropped traffic. If the UK (say) is your main target but you also get traffic from other countries then a link or two from a regional site might push you up a bit in the country specific rankings and help offset your lost UK traffic.

Hi Bradley,

Care to give us the site and an example of some of the errors?

HI Tom

If they are two very different websites then why would it matter if they are on the same c block or not?

Personally I would not get too worried about trying to hide the connection from google, it is a losing game in most cases. That being said, most of the content you mention seems to be relevant to the ecom site, I would be tempted to host it on the same domain. Use the blog site to add relevant reviews and content and to help boost the overall domain authority of the main site. Just because the blog is hosted on the ecom site doesn't mean the reviews cannot be unbiased!

My two cents!

In regards link building it is still an important part of the mix, but it has to look natural and even if those links we are talking about are natural, they dont LOOK natural. The simple fact that you mention that you can contact the bloggers and ask for them to change the link text indicates they might not be 100% natural links if you see what I mean. If you can, I would try to reduce the number of links (less sitewides in blogroll, a lot less actually) mix up the anchor text with a very high percentage of branded terms (so domain name/company name) and if they are site wides look at making them no-follow. There is a lot of info on Q&A and blog posts here in regards natural/un-natural links, here for example: http://moz.com/blog/17-types-of-link-spam-to-avoid but search in the qa and you will get lots more info.

In regards what to do, I would say that in comparison with your competitor you have a way to go before your domain metrics will match theirs (even if they are doing similar type link building). So keep doing what you are doing in term of making the site user friendly, keep adding blog content and expanding product info (without excessive keyword stuffing). And be patient! A good video for ecommerce technical seo is here: http://moz.com/webinars/ecommerce-seo-fix-and-avoid-common-issues which might help onsite stuff.

In the meantime, the site look great, your conversion rate sounds ok for retail, so if you are not already I would be thinking about PPC campaigns which should convert well and also give some insight into real search phrases. Leverage your social media followers and try to engage them more. Make use of and try to build your email lists. The normal stuff in other words while you are waiting for your seo efforts to come into effect! Hope that helps!!

Hi,

In regards the suspicious links, if you look at the ahrefs link I mention above you will see that you have gone from about 8,000 to 68000 links in the last 3 months or so after being flat before that for many months. Worse, of those links about 80% have the term caduri ideale (ideal gifts or something like that I think?). This does not look like normal link growth from bloggers etc. This looks like keyword targeted link building on site wide links (since the number of linking domains has gone up much much less than number of links). If the rankings/visits drop is only from google, it is during (or shortly after) an algo update and it is related to keywords you see in that ahrefs link in the referring anchor phrases then I think you have a good idea where some part of the problem may be.

Hi,

Agree with Dana that the moz on page grading tool is really not that relevant, and I would stress that it is probably even less relevant to non english sites. I have sites where it tells me no instances of xxx were found in the title, body etc... but it certainly is!

Agree also with Francisco and Dennis that you should check your analytics and see if a google update coincides with your rankings drops (and also that your site looks very nice!).

Open site explorer in this case is showing slightly dated link data, if you check out your link profile in ahrefs https://ahrefs.com/site-explorer/overview/subdomains/wo-men.ro it looks like there has been a big increase in links over the last 2-3 months and a lot of those are exact match of one sort or another. Might or might not be related to the exact drops you are looking at for products, but that link profile is very suspicious looking now and it would be a shame for a site that looks good and obviously has had a lot of work put into it to be undone by something like that.

Hi Tim,

Yes, any links to those non existent pages will be causing the page not found errors, so remove them from the sitemap. You should see the errors reducing (or disappearing completely) in the next scan report. If you still see errors in the next scan you can use the same process again to find any remaining links (if there are any).

Hi Tim,

If you have deleted the pages and there are still links to them either from the same site or from external sites then these are indeed 404 (page not found) pages and they will continue to be until you either remove all links to them or 301 redirect them to new pages. Are you sure you have removed all the links to them from your site's pages? If you run the site through a moz scan then this will tell you the referring page that linked to the 404 page (the far right column) and you can go and check to see where the links are and remove them.

By the way, if it is an all html site and you have ftp access... cant you just download all the files and work on it locally?

Hope that helps!

Hi Tim,

If you go to the open site explorer there is a tab on it called 'advanced reports' which I think does the same thing if you pick the 'pages on my subdomain' report type.

Hi Phillip,

Check out this great reference by Dr Pete: http://moz.com/blog/rel-confused-answers-to-your-rel-canonical-questions

In a nutshell (if I can get my head around it)  rel canonical is 'suggesting' to google which version of a page should be indexed for search purposes while 301 redirects are basically taking you to the canonical version of the page already and therefore is more final in regards both users and bots.

The set up you have (assuming that you want to consolidate 1234.com to realsite.com permanently) has a problem in that you are telling google that realsite.com/en-us has a canonical of 1234.com (which is just a suggestion, not a permanent redirect) and therefore yes the canonical as you say is not being reognised in the way you expect it to be because it is indeed just a suggestion, not a followed link if you see what I mean. I think 301 redirecting the realsite/en-us to realsite.com is the only thing that will be a permanent solution.

Hope I got that right, maybe someone else can chime in....

Hi Dave,

I assume you are talking about webnames.ca, in which case if you drop the url for the press_releases into screaming frog then you see a whole bunch of 302's to an error page. I had a quick look and could not see an obvious link that was spawning these but something must be in there if both tools are seeing it. Most obvious thing would probably be a relative link (so like href="/default.aspx") maybe in one of the press releases themselves since your main menu items at first glance look ok.

If you export the errors list from moz and filter for the 302s, then the referring page shown in the far right column in combination with the error page shown in the first column might help track down the place where the whole thing starts off. Tracking down the link(s) causing these kind of issues can be a real pain sometimes!

Hope that helps!

That is just nasty! Which domain is showing the duplicates? I assume the realsite.com and it is showing duplicates between the / and the /en-us versions? I cannot say if moz is registering the errors in the same way exactly as a search bot would register them, but you are asking for trouble with it.

Can you not simplify the chain? It seems to me that the cross domain canonical is pointless in this case. You have all 1234.com urls 301 to their realsite.com equivalents. OK. If needed canonical the realsite.com/en-us to realsite.com or probably better yet 301 redirect it also. It might need a bit of technical fiddling to get it all set up, but surely there must be a way!

Hi,

I have not personally used it, but have read good things about search extensions using Sphinx, check out for example: http://ecommerce.aheadworks.com/magento-extensions/advanced-search.html which is from a magento platinum partner and should be reliable. There are a couple of others around also using the sphinx technology. The default magento search really does leave a bit to be desired in most cases!

I don't think changing the .com links will move the needle, there are only a couple of them and they are being 301 redirected so not going to make that much of a difference I wouldn't think.

If you are going to shift that phrase over to the internal page then in the first instance I would focus on trying to get some more  content on that page to reinforce that this is the main page about kitchens. You could add some content describing how/why each of the sub categories (worktops, doors etc) are relevant to the DIYer for example. You could use your internal blog for informational type posts that can further reinforce the main category page.

I guess what I am saying is that since the top sites for that term seem to be mainly root domains and partial/full EMDs to boot, then your content is going to need to stand out a bit for you to realistically get onto the first page. So, I would concentrate on that before I jumped into more external link building which has the potential to look slightly unnatural given the nature of the content as it is now.

Might be worth focusing on some social signals before external links also, which could sync with your internal blogging efforts, try encouraging clients to leave reviews on relevant sites, that kind of thing. Once the content is there and the social signals are supporting it then external links will seem more credible and probably be more effective.

Hope that helps!