Welcome to the Q&A Forum

That post was from a week ago - it's still relevant, for sure.

The variations should be removed - canonical is an indicator to the engines, it's not going to destroy your site if you canonical pages to themselves.

I use the Yoast Wordpress SEO plugin on all of my own/client sites, and he has the self referencing canonical there as well. I think it's just good practice to prevent the types of issues you have mentioned.

To find the links to your site with these parameters, use Open site Explorer from Moz, and also check the links mentioned in Google Webmaster Tools - it's under the traffic section in the navigation. They'll both let you download lists of external links pointing to your site, and then you can see the links that are causing these parameters.

The aggregate columns you are showing are monthly averages. You can also show the column for monthly search volume - once you've turned on this setting, when you export it, the csv will include the searches per month - this is helpful for understanding seasonal keyword trends

Apparently this is reflective of code change made to the system and not algo updates here - see Dr. Pete's tweet here - https://twitter.com/dr_pete/status/288665186951380994 - stay tuned for his blog post

If the site got hit in the search engines (I see a major drop in visibility for the site in Search Metrics June 21 and 28 2012), and the client has now moved to a new domain, why would you 301 the old pages to the new ones - don't you want to start with a clean slate, which is why you started a new site?

If the 301 passes the penalty or not, why take the risk? Why not build your link profile the proper way this time?

I also see a lot of over optimized anchor text in OSE and very little branded or generic - diversify your anchor text and focus on branded, unoptimized, and naked anchor text (www.claims...) in your link building efforts.

If you're worried about visitors to the current domain, why not redirect using a 302 and then eventually let the domain die?

Instead of switching domains, you can also do a link disavow after a thorough review of your links and then do a reconsideration request, and then start the link building again on the current domain, instead of switching to a new hyphenated domain that just looks like a spammier version of the original URL. I would do a thorough back link review and then disavow and reinclusion request instead of the new domain route. You can refer to this article for how to review your links from Search Engine Journal.

That's my two cents - hope it helps

Mark

Hi Assaf,

(I'm not stalking you, I just think you've raised another interesting question)

In terms of index status/size, you don't want to create a massive index of empty/low value pages - this is food for Google's Panda algorithm, and will not be good for your site in the long run. It'll get a Panda smack if it hasn't already.

To remove these pages from the index, instead of doing hundreds of thousands of 301 redirects, which your server won't like either, I'd recommend adding the noindex meta tag to the pages.

I'd put a rule in your cms that after a certain point in time, you noindex those pages. Make sure you also have evergreen pages on your site that can serve as landing pages for the search engines and which won't need to be removed after a short period of time. These are the pages you'll want to focus your outreach and link building efforts on.

Mark

Short answer: if your data is indeed out of date, then competitive analysis based on that data isn't going to be as useful as it could be.

However, it could be that the report is just giving you different data than the data you're looking for.  Can you post what report you're using and the site you're looking at so we can take a deeper look?

In Excel, you create the formula once in the first row, and then double click on the bottom right corner of the cell and it should populate the rest of the data in that column with that formula. It's called the fill handle - double click it and it will fill the whole column.

See here - http://office.microsoft.com/en-001/excel-help/fill-data-automatically-in-worksheet-cells-HP001216366.aspx

You should remove the robots.txt block on the redirect - if the redirect is implemented properly, and it directs straight to the new page and not through some sort of infinite loop, it should be fine for you server to handle.

The bots think those pages are real pages, thus they have them indexed - you have blocked them from visiting the redirect and seeing that it points to another page and that the redirect page should be replaced with the real page. If things are implemented properly, they would follow the 301 redirect and remove the URL of the redirect from the search engines.

If you provide a link to one of these redirecting URLs, I'm happy to take a look and see if I can help you.

Exactly - I'd build  a strategy more around promoting pages that will have long lasting value.

If you use the tag noindex, follow, it will continue to spread link juice throughout the site, it's just the individual page with the tag will not be included in the search results and will not be part of the index. In order for the tag to work, they first have to crawl the page and see the tag - so it doesn't happen instantaneously - if they crawl these deeper pages once every few weeks, once a month, or even longer, it may take a while for these pages to be removed from the index.

Baron,

I believe what Michael meant (correct me if I'm wrong Michael), is that OSE runs off of Moz's crawler.  If there crawler hasn't come across your YT and G+ pages, then they wouldn't have picked up the links to your site from those pages either.

Either way, I assure you that Google knows about those links due to their more robust crawling ability and the fact that they own both of those products

It looks you have a bit of redirect loop here - that might be what's hurting the server a bit

Just an aside - you're going to have indexation issues - you have both www and non-www versions live on the site, with no canonicals pointing to one version. You also have index.php as a live page linked to from the logo. I'd definitely recommend implementing canonical tags across the site.

Mark

Hi Marisa,

It sounds like you're advocating for all of us members to get organized and create the largest link network the world has ever seen (mwahahaha). While there is some logic to it, I really think this is exactly the type of stuff that Google is going after. In the end, what it would come down to is the user. By your linking from your site to the coat hanger site, are you benefiting your user? Is the link in context and does it make sense? If someone was genuinely interested in reading your article, would following your link help them? If so, by all means do it. I don't think Google will penalize people for naturally linking to friends, business partners, associates, local businesses, etc. But what happens when this becomes organized, and your DIY furniture blog starts linking to forex sites and cheap online prescriptions? While that is clearly an exaggeration, I see it as only a matter of time before things would deteriorate. While you should by all means be building relationships and establishing connections and then leveraging those connections for all purposes, including link equity, you need to keep your user in mind in the end and be focused on them.

That being said, I'm all for contextual outbound links to relevant sites - let's all spread the link love a bit more!

These pages, which are search results pages, are often viewed by the search engines as thin pages - I would recommend adding the noindex tag to your search results pages, so that these pages won't dilute the index with thin, low quality pages that are yummy food for the algorithms.

By adding the noindex tag, you'll be fine in terms of duplicate title tags.

As an aside, you also have two versions of the meta description tag on the page - I would modify this so that for each page on your site you only provide one meta description.

Mark

Hey SaraSEO,

This might be a good solution for you - I haven't tried it, but it seems like what you need - sitemap creator that can strip out parameters and follow canonical tags - http://www.inspyder.com/products/SitemapCreator/Default.aspx

Good luck,

Mark

I agree with what William wrote, it's just not super easy. In addition to building out the great content, you have to market it. You have to get the word out - you have to do the outreach. It's not enough to build awesome content, you have to share it, via social networks, email, outreach, word of mouth, contacts you may have, the press, etc. You can build the best blog pieces out there, but if no one hears about it, it ain't gonna do you much good.

Adria wrote a great post here on SEOMoz the other day - I'd check it out - http://www.seomoz.org/blog/why-link-building-strategies-fail

I don't believe Yoast's Wordpress SEO plugin will cover Magento files. The Wordpress SEO plugin is built into the Wordpress, and pulls the page, post, category, taxonomy, etc. data from Wordpress in order to build the sitemap. The plugin itself creates a sitemap for each page, post type, etc., and is highly configurable. It then uses a sitemap index file to connect everything together.

But since the Wordpress doesn't have the Magento information, since they are separate CMS's with separate databases, the Yoast Wordpress SEO plugin won't build a sitemap file for information in the other CMS, as it has no access to it and no knowledge of it.

You can use other tools to create sitemaps for the site, either through manual crawls or installed software on your server,  but the yoast wordpress plugin won't do it.

Mark

You can use scrapebox to scrape whois info for the domains, and then use the emails you get from the scrape.

For manual guess work, you can try using rapportive and plugging in variations and seeing what you find - you can also try this nifty tool built by the team at SEO Gadget to pull contact info and other various metrics - http://tools.seogadget.co.uk/

Good luck,

Mark

You can use screaming frog to crawl the site, and it will show you the internal links and anchor text to each page across the site as well. Though this post from Branko is a bit old,it's still a great resource on using Screaming Frog to check for internal linking issues - http://www.seo-scientist.com/seo-spider-review-xenu-on-seo-steroids.html

Screaming Frog has a paid version, but the free version will you crawl up to 500 URLs, so that might be sufficient for ya.

Hope this helps and good luck.

Mark

As Saijo said above, the meta robots noindex tag is the way to go. When you block a folder via robots.txt, you prevent Google from visiting and crawling that folder and any content within it. If Google has already crawled the content, they won't remove the content from their index just if you block it with robots.txt. The old version they have of the page will be stored and saved in their index, and they just won't be able to show you an updated snippet of the page due to the robots.txt block.

To remove the pages from the index completely, you can do one of 2 things -

1. in webmaster tools, go to the url removal section, and remove that folder from the index - this will only work when it's blocked via robots.txt
2. you can add a meta robots noindex tag to the pages/page template, and remove the robots.txt block - you need to remove the robots.txt block so the search engines can recrawl the pages, see the meta robots directive, and follow the noindex guide to remove the page.

In general, I would recommend using the meta robots noindex directive over the robots.txt, because it should work for all search engines, and you won't have to go into webmaster tools for each one. You also will ensure that you don't accidentally block other urls.

From your example above, if you just blocked the folder /search/, a page that includes the word search in the url but isn't in the blocked folder shouldn't be blocked from the search engines because of that line - I would check in webmaster tools the robots.txt section, because it doesn't look to me, based on your robots.txt file, that any url with search in it should be blocked.

Good luck,

Mark