Welcome to the Q&A Forum

http://moz.com/learn/seo

Many people had big drops around the same time period, so likely an algorithm update that impacted you.

People who are disagreeing with this, explain your reasoning.

MyBlogGuest was penalized towards the end of March, that's probably not his issue.

Did you manually check your rankings? Moz is known to post massive drops like that despite little to no real movement in the SERPs.

A noindex tag specific to Googlebot would also be a good idea.

There are multiple ways to do everything in SEO. Multiple ways to rank highly, multiple ways to lose all rankings. I'd say it's pretty much 0% science, because if any part of it was then it wouldn't be nearly as hard. Even the things that one might say we have down to a science (proper title tag structure, well written meta description, etc.) aren't really because Google can display them how they want to display them - not how you wrote them.

The most important thing I can stress is don't rely solely on SEO. Don't rely solely on any one strategy. That's a guaranteed way to fail. Use SEO, PPC, email marketing, etc. Diversify your strategy and you won't freak when Google changes the algorithm and everything goes upside down.

Might be something you can get from the WMT API.

Also, to really see how many pages are indexed, do a site:xxxx.com search, go to the last page, include omitted results, go to the last page again, and add up how many you have. That's probably the most accurate number.

I don't deal with implementation or specifics of our site search, but we're on Magento and I know we've used Google site search and Solr for one of our sites. We haven't been able to find one we're really happy with.

The penalty will probably pass to site B if you do a 301 redirect. If it were that easy to get out of being penalized then you wouldn't have companies whose marketing is solely based on penalty recovery.

Go to WMT for whatever dev site you want to remove from the index. Use the URL removal tool, but in the box just enter /, nothing else. This will remove the entire site.

If you use rel=canonical on product pages, you risk not having all of them indexed and missing out on long tail traffic. Focus on making sure each page can get served in the SERPs.

All backlink analysis tools will give you a different number of links to your site. Generally, I find Moz shows the least and AHREFs shows the most. For the most complete pictures, gather link data from a variety of sources and remove duplicates.

I'm in ecommerce as well. We 301 promotion-specific pages to the homepage once the promotion is over, as each one won't come back around for a year.

Moz tells me there's an issue with the GA connection on one of my campaigns, so you're not alone.

Use this:

User-agent: Googlebot
Noindex: /

User-agent: Googlebot
Disallow: /

User-agent: *
Disallow: /

This is what I use to block our dev sites from being indexed and we've had no issues.

Digital Millennium Copyright Act being used here? No.

OP, it does sound like you have duplicate content issues. See what you can do to make those omitted pages more unique.

Why aren't 301s an option?

From a quick glance at OSE, looks like you do have some backlinks to clean up. Your anchor text is heavily weighted towards money terms - primarily [oak furniture], [solid oak furniture], and [oak office furniture]. Plus sites like http://www.lanaintl.com/basic-info-about-real-estate-agents (which you have a backlink from) are clearly spammy.