Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
Rogerbot probably missed you if he's updated since you fixed it.
Run the Crawl Test tool. You get 2 crawl credits per day so just run this, see what happens in a day or two on your campaign. You can see when it's last been crawled in the Excel file that you get with crawl test.
Also, here are a few reasons Roger can't crawl your site:
Finally, see the FAQ here: http://www.seomoz.org/help/crawl-diagnostics
That should sort you out. 
Without seeing page specifics it's hard to say if the page is affecting the result (title, content, bolds, tags, etc.)
However, "winning" Moz metrics definitely doesn't mean you'll outrank everyone for everything. I wrote a post on some factors that go beyond Moz. http://www.seomoz.org/blog/going-beyond-moz-metrics-to-answer-why-is-this-site-outranking-me
The other thing to keep in mind is - they may outrank you on one page for a huge variety of reasons: anchor text (yours OR theirs), specific weight of links to that page, specific weight and anchors of links to that page, cocitation (everytime they're mentioned is alongside that keyword, while you're mentioned alongside many.) etc.
So ... it's difficult to say without specifics but with well over 200 factors, it's worth considering that there isn't just one "easy fix" reason they're winning that term.
We have all known Penguins were sneaky since at least
http://www.youtube.com/watch?v=yyuQL1k6m2c
But yes, this is scary:
haha It was about 9 am when I replied! I hope not too many people are headed to the bar at that hour. 
(But tonight? Yes, tonight I'm going to party like a rockstar, party like a rockstar!)
Thank you! I know most people who have posted so far are the ones who aren't happy but I think it's important we know that for every site that moves down, someone moved up and is pleased with the change. We try to do everything "by the book" but since the book doesn't exist, just do what we think is best - and it seems to work, though a bit slowly sometimes.
Yes, THANKFULLY I got good news. I had a client come to me after losing 98% of their traffic in a 2 day period after Penguin 1. We have been fighting a losing battle since including disavowing old spam links, removing links, adding new and relevant links and citations, redoing some of the on-page work, etc. NOTHING has moved the needle.
Finally, yesterday, nearly all of their main keywords jumped from what Analytics said was "Average Position 400" to a few on page 1, 2, 3, lots on 5 and 6 and a lot in the top 100. Most of their keywords have recovered including ALL their most important ones. Now we have work to do but at least we know where we're at and we're not flailing in dark.
This was the client I've had the single most trouble ranking in my business' history so I'm having a really good day.
I've heard some people use it on job applications for marketing type jobs but never "used" it for anything myself. I do login once in awhile to see if there are any good perks but most are for things I wouldn't necessarily use. I think the scores are being built into Bing now so perhaps for that small portion of searches done on Bing it makes sense but otherwise, not sure why someone would use it.
I've seen people say that in "rare" cases, links blocked by Robots.txt will be shown as search results but there's no way I can imagine that would happen if it's duplicates of your content.
Robots.txt lets a search engine know not to crawl a directory - but if another resource links to it, they may know it exists, just not the content of it. They won't know if it's noindex or not because they don't crawl it - but if they know it exists, they could rarely return it. Duplicate content would have a better result, therefore that better result will be returned, and your test sites should not be...
As far as hurting your site, no way. Unless a page WAS allowed, is duplicate, is now NOT allowed, and hasn't been recrawled. In that case, I can't imagine it would hurt you that much either. I wouldn't worry about it.
(Also, noindex doesn't matter on these pages. At least to Google. Google will see the noindex first and will not crawl the page. Until they crawl the page it doesn't matter if it has one word or 300 directives, they'll never see it. So noindex really wouldn't help unless a page had already slipped through.)
My first two thoughts:
Possibly #1 and #2 are answering the query. Without knowing the actual search term, it's hard to say.
The search term may not match the meta description or the description may be (sorry) just crappy. It's hard to say without the term and link of course.
If your description isn't catching visitors, my first thought would be to try changing it and just track next month vs. this month. If you want to either PM the actual data or if you can post it, we could look further into it but without that, there are many reasons it "could" be not getting clicks.
"When I search for my blog url"
If you're searching
Then your sitemap is a relevant source of that post. In fact, probably the most reasonable result for your link other than the link itself.. Think about it like this - you're searching as if your URL is a keyword. Your "keyword" is on that page (likely the link you're searching for is on your sitemap). So Google returns a relevant site that has your "keyword" on it.
Note that this isn't the same as searching
site:website.com/blog/posttitle
It's interesting that it's returning the sitemap not the post but it may have to do with your settings on the post. For instance, if you had noindex on the post, your sitemap should come up #1. If you don't, that's curious at least. But yes, it makes sense.
There are actually a number of ways to approach what you're doing, including implementing hreflang tags for en-ca or en-us but I'm seeing the .com here in Melbourne.
I would sort out your canonicals, hreflang, and get your pages setup to show the right one per search engine (.ca on google.ca, .com on the rest maybe.)
It depends on what the user's search is.
If a user searches "cars in boston" and you have a massive amount of anchors to "cars in boston" that page isn't necessarily going to benefit from those anchors, at least nowhere near as much as they used to.
I think Cyrus' WBF and the other link both should give you some idea though. Don't control anchors. That's 2013.
Generally speaking, if you're concerned with anchor text because you control it, those are the wrong links to be building.
Exact match anchor text depends on whether your anchors match each other for your target keywords. If you have 100 links and 90 say "cars in Boston" you're going to take a beating. If 5 are "cars in boston" and 5 are "boston cars" and 3 are "Hyundai Boston" and another 10 are your brand? Not so much, really. You can also build partial match links.
and this: http://www.webmarketinggroup.co.uk/Blog/the-truth-about-exact-match-anchor-text-1760.aspx
Hope that helps!
site.com and site.com/index.html are both loading index.html So open Index.html, add the canonical tag back to your preferred version and you're done.
If you're saying that you want the link to be something different, that depends on how the site is setup, whether you can change the "home" button or whatnot. In Wordpress's new menus this is simple. In other themes, not so much. In other CMS, I don't know because "it depends."
I agree with a lot of the other answers but let me give you some backstory on what we did.
When I had a photo studio, we had 4 distinct websites. They offered something unique - each photographer's portfolio and the opportunity to book exactly who you want.
So we had:
So our main studio had our branding. It showcased everyone, was very "newsy" about the studio, etc.
Our owner site was an overlapping brand but you could only see the owners work, owners about page, owners everything. Even the contact came directly to me, not our sales@ address.
Associate 1 and Associate 2 had completely different branding. They were based on the personalities of our photographers, not our main business. If someone was more interested in a fashion photographer, they wouldn't want us, they'd want our associate1. If they wanted more of that rockstar grungyness, book associate2.
So the plan was simple - dominate the SERPs on our main keywords and then sell to those who wanted each brand. It worked beautifully. We were one of the most booked studios in my area (maybe shot the most weddings between 2009-2010.) Associate1 grew past our business and now has her own studio. It was a tremendously successful plan.
If you offer something unique per site or at least some reason to get backlinks to more than one "business" it can work. I've MADE it work. But if you're just going to double up to hedge bets, it won't work. You'd be better off ranking #1 for your main than #8 and #10 no matter which study you believe:
http://searchengineland.com/organic-click-thru-rates-tumbling-study-97338
Of course, position 1 + 3 beats 8 alone. So it just depends on the situation.
You could - although I would suggest that if you create content for the long tail keywords it will perform better than if you just slap some labels on existing content. I would think the plan would be:
Create new long tail content (blogs are #1 for this, I'd guess.)
Rank the long tail stuff and get more notice, thus gaining more backlinks and more pass through to your head keywords (short tail).
Then create a mix of long & short tail, keep growing in both areas.
I'll give you the best answer I can but at least consider the possibility that absolute URLs are actually better long term. Other than moving a site around as you're doing now, absolute URLs win on every factor.
That said, you're looking for FireLinkReport.
http://www.searchenginejournal.com/firelinkreport-research-on-page-links-firefox/17714/
It's a FFox add on that does internal vs. external, absolute vs. relative, etc. and this should create a report that helps you do what you need.
You can definitely have multiple hreflangs. This is the way to set it up:
The exact syntax to use in the header of the original would look something like this, for a site that has an original in English and alternate versions in Spanish (targeted to the audience in Mexico) and German:
You should also deal with hreflang and alt languages in your sitemap, per: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=2620865
I'm also fairly sure that Bing does not support hreflang. Only Google & Yandex do, I believe. I assume that will change as more international sites get their properties setup for Google and other engines want to stay relevant on international search.
Subdomains are considered separate sites so you.blogger.com doesn't really get any of the good juice from blogger being a Google product. (Not really anyways)
Wordpress gives you full control. G+ will be important but you can already integrate G+ comments into your site if you want. With more themes, more support, more plugins and more SEO posts already available to help you, I'd think Wordpress usually wins this debate.
(I wouldn't think you.wordpress.com is better than you.blogger.com but you.com probably kicks the crap out of both.)
Join communities in related niches and share your experience. Many niches relate to at least a few others. What's the "big" version of your niche? (If you're selling pink my little ponies only on ebay, you're still doing ecommerce. Join ecommerce groups, join toy groups, whatever suits you. It doesn't have to be a MLP group.)
Also, what is your market? What are they into? If you can't be an expert on MLP on an ecomm forum, you can share your amazon or ebay knowledge. If you're with potential clients, how can you be contribute to the discussion in a larger "view" of your niche?
