Welcome to the Q&A Forum

Hi there

Quite a few tools out there will let you do this.  SEMRush is a popular one - simply add the competitor's URL into the field and it will show you the organic (and paid) keywords they rank for.  Searchmetrics does this too, and I've found it a bit better for EU keywords.

Keyword Spy and SpyFu do this as well.

Hope these help.

Hi Andy

Think it's very wise of you to have considered this potential duplicate content problem.

Having a rel=canonical tag on the separate categories, or even a  tag on them, would make sure that the URL is not indexed in Google, thus removing any potential duplicate content.

I can't really see a way of having both a main deals URL and a category deals URL both being indexed and ranking because, as you have said, the pages would either have zero content or duplicate content.

With that in mind, I think you're current format is the best one.  Having a big /deals page with all your offers on it will hopefully provide lots of rich content so that people will link to the page, which in turn will rank the page for a number of keywords - while you're also allowing people to filter down and get to what they want.  Just make sure that the separate sub-category pages have either enough unique content on them, or a canonical/meta noindex tag on them to avoid a duplicate content issue.

I would say though that as you're site aggregates daily deals, you are at a bit of a risk of still supplying duplicate content from other sites.  I'm not sure how the deals are fed in, but if you get a rush of deals en masse from one website and the deal's titles and descriptions are all the same, this might also be seen as duplicate.  If you can off-set this with unique content on the page and a system to put in your own titles/descriptions, then it shouldn't be a problem.

Good question this - would be interested to see some other Mozzer's POVs.

Hey there!

There's no benefit or negative effect of this either way - so there's really nothing to worry about here.

Furthermore, if you type in http://www.consumerbase.com/ you get redirected to the .html version and it's the same for internal pages.  This means you've not got any problems with duplicate URLs or content.

In short, everything is in order and from an SEO point of view there's no reason to make the change - all is well!  The only reason why you may want to make the change is from a user experience point of view - but I don't think visitors to your site will be concerned with .html extensions at all.

Hope this helps to put your mind at rest!

There's no harm at all in doing it before launch, so I would do it now.  That way, when you do go live, you're capturing 100% of the data without any delay.  If you create any new subsequent pages you can then also add as you go, rather than piling on the work.

Hope this helps!

Hi John

No, if you put the redirect in place it won't create duplicate content.  In fact, redirections are often used to avoid any potential duplication problems.

The redirection tells the Google bot that the old URL is no longer required, the URL it points to is the correct one.  This will tell the bot to stop indexing and crawling the old URL, pass on any of the links pointing to the old URL and consider the new URL to be the definitive article.

Out with the old and in with the new, so to speak!

You can read more on redirection with the Moz Guide.

Hope this helps!

Hi Ruchi

It can do, if it is overdone.  The question is of course, how much is too much?

Here is an excerpt from a great post on the topic from Danny Sullivan:

"How can you tell if you’ve got too many ads above-the-fold? When I talked with the head of Google’s web spam team, Matt Cutts, he said that Google wasn’t going to provide any type of official tools similar to how it provides tools to tell if your site is too slow (site speed is another ranking signal).

Instead, Cutts told me that Google is encouraging people to make use of its Google Browser Size tool or similar tools to understand how much of a page’s content (as opposed to ads) is visible at first glance to visitors under various screen resolutions.

But how far down the page is too far? That’s left to the publisher to decide for themselves. However, the blog post stresses the change should only hit pages with an abnormally large number of ads above-the-fold, compared to the web as a whole:

We understand that placing ads above-the-fold is quite common for many websites; these ads often perform well and help publishers monetize online content.
This algorithmic change does not affect sites who place ads above-the-fold to a normal degree, but affects sites that go much further to load the top of the page with ads to an excessive degree or that make it hard to find the actual original content on the page.

This new algorithmic improvement tends to impact sites where there is only a small amount of visible content above-the-fold or relevant content is persistently pushed down by large blocks of ads."

Hope this helps.

Hi Amelia

To follow on from Chris, can I ask if you're asking about getting the stars in your paid ads or in the organic listings?

If it's the latter, you'll need to have schema markup on the page itself, here's a Google guide to it.

Plus a helpful video.

Hi William

Just as a quick note, if you click on the "1" in the Other URL column, it will take you to a page that lists the actual URL the report is flagging as duplicate.

Sorry I can't offer any more help here, but you've done exactly all I could recommend to speed up the process - demoting the pesky sitelinks.

Only thing really to add is to ensure that the links that you would want to appear in the sitelinks are recognised by Google as the most important landing pages - so link to them internally wherever you can and of course externally (though of course the 301s will help with this)

You can't beat the SEO Log File Analyser from Screaming Frog, imo:

https://www.screamingfrog.co.uk/log-file-analyser/

Hi there

First of all, does your site use an .htaccess file? This is probably the most common solution used to implement 301 redirects.  The SEOMoz guide to redirection is really useful in explaining how to implement redirects for apache servers and others.

These three generators can create the code for you, but it's very important to learn how the .htaccess file works before trying to implement it. Therefore, you should acquaint yourself fully with the Moz guide linked above.

Alternatively, if your site is running WordPress, you can setup 301 redirects very easily with the Yoast SEO plugin.

Hope these resources help.

Hey there

It's not unreasonable at all that a company may operate from the same premises, both physically and digitally.

In terms of SEO, having 2 sites interlinking with each other that sit on the same IP won't be a problem.  In fact, a few more wouldn't be a problem either.  Matt Cutts recently answered the question on how (and how much) you should link your sites together: http://youtu.be/x0-jw_PfwtY

In short, you can do so happily in moderation - so 2 websites like yours linking together won't be a problem at all.  I'd say you'd need to look at things a bit closer if you have 5 or more.

Hope this helps.

It's certainly a safe site.

As with all article sites, its value has been diminished significantly.  As far as article sites go, SelfGrowth is probably one of the better ones out there.  I know when I've had a surplus of content on a previous site a few months back we used it.  Rankings did increase, but it's not like we used SelfGrowth exclusively, so impossible to say there's correlation.

A link from there wouldn't likely hurt and might pass on a bit of 'link-juice', for lack of a better term.  However, I'd only ever use it if you're overflowing with time-sensitive content that you don't have anywhere else to put.  Other than that circumstance, I'd nearly always recommend doing some targeted outreach and getting your article onto a related blog in your niche.

Not at all, I'm happy to help!

I can only presume that the theme query you're getting is related to the WP/Joomla theme you're using.  Wouldn't be able to help specifically without seeing it, but I would assume that the URL without the "theme=default" at the end should be the canonical URL.

For more stringent decisions - if you have a big amount of URLs and you're thinking of redirecting some, I'd start by looking at your analytics traffic - has any visitor come to your site via that URL within the last 60 days? If yes, I would definitely redirect.  If no, I'd ask this:

Do you have any inbound links to that page? If you put in the root domain into Open Site Explorer and click on top pages and export the results into a CSV - you can see which pages have inbound links.  Those without links can be ignored, those with links should be 301'd (provided you are happy that the links are of a good quality), in order for you to preserve the link equity, or SEO 'strength', of the link.

Hi Dario

If you're not intending to restrict access to certain parts of the site to certain crawlers, then you won't actually need a robots.txt file at all.  So unless you have sub-folders that you want to keep private, don't worry about it.

Links/Resources pages can be, under the right circumstance, a great place to be linked from.

Now this is dependent on a number of factors.  A website that has a resource page that contains links only from within their niche/area of expertise, on a website that you can be sure has not built dodgy links itself, then there should be no harm in having a link there.  Even if the page has a lot of outbound links, if it's a reputable website then your link will still be worth something, although might be diluted due to the other links.

Of course, a site that has one link pointing to pet supplies and another linking to viagra on the same page is going to raise suspicion.  Similarly, if the resource page sits on an authoritative domain but the PA score is nil or very low, then it may not be worth pursuing.  That's why I'd also approach the issue from an issue of branding.  I'd think to myself "Would a link to my website from this website come across as authoritative to the user?"  That is to say, if someone was on a user journey to buy car parts in michigan, and on the government resource page a link to your car parts website is there, that's going to be a ringing endorsement for your website in both the eyes of the user and a Googlebot.

In many ways, you could approach links/resource pages like web directories.  If you have the opportunity to get a link from an authoritative one on a solid website, I can't see it doing much harm.

Hi there

I definitely admire your creativity on this one, but unfortunately the Google Tag Manager loads tags asynchronously.  That means that the tags are added for users who execute javascript on page loads, but it means that the copy of the page that is crawled does not contain the tags.  All of this means that the Google crawler won't see a noindex tag on the crawled version of its page if it's loaded via the Google Tag Manager.

I think your reasoning for noindexing the pages themselves is a very good one.  You're removing pages with thin or potentially duplicate content from Google's indexing, which is healthy, while keeping the content on the page for the user, which again is a healthy thing to do.  I can definitely see the reasoning.

Unfortunately, the only way I can see the tags being implemented is manually by the webmaster.  If the site runs wordpress, you can change the robots meta data very quickly in the Yoast SEO plugin.  If you get to the page in question and scroll to the Yoast plugin section, you'll be able to select the noindex tag from a drop-down menu, meaning it can take as little as 30 seconds.

Hope this helps and good luck with the implementation.

In that case you don't need a robots.txt file at all.

Your example above is actually very close to restricting access to your entire site.  If you had a robots.txt that looked like this:

User-agent: * 
Disallow: /

You would stop any bot or visitor from seeing the site.  Definitely not recommended!

In your case, I would avoid having a robots.txt file altogether.  It's not an SEO requirement until you need to restrict access for whatever reason.

Sure does.

If you install the Mozbar to your browser as well, it comes with an option to highlight follow and no-follow links on the page.

You can find Chrome and Firefox downloads for it here.

The Mozscape index, as brilliant as it is, can in no way compete with the size of the index that Google can handle.

As a result, your WMT report should always have a bigger amount of pages, links etc crawled.  It's just bigger.