Welcome to the Q&A Forum

Hi Jared,

The comments below are correct. Moz's index isn't a complete view of the web, but we try to crawl quality sites out there. Our index is always growing. But to be frank, no one -- not even Google -- crawls the entire web. There are just too many pages out there and computing power is expensive.

We focus on quality sites because we believe (and hope) our customers and community are focusing on high quality links that are still active, which are going to give them the best SEO results. (Also a huge part of the web is just spam nonsense.) Many SEOs -- and I include myself here -- will use multiple sources of link indexing for the broadest view of the web and their sites. I use Moz's OSE and Google Webmaster Tools (free and big, but includes lots of spam, which can be great if you're fighting a Penguin penalty). Others will use OSE, GWT, and OSE competitors like Majestic and Ahrefs, especially if link building is huge for them and they want to account for every penny. Every index is a bit different.

I believe it's because of the itemscope="itemscope" call, which is saying that the area is another item. Leave out the ="itemscope" part.

Example:

[Blue Denim Castaway Flip Flops](/shoes//xsl:value-of select="translate(/page/subCategory[Name = $subCat]/Name,'ABCDEFGHIJKLMNOPQRSTUVWXYZ ,','abcdefghijklmnopqrstuvwxyz-')"/>/blue-denim-castaway-flip-flops/300) £30.00 [](/shoes//xsl:value-of select="translate(/page/subCategory[Name = $subCat]/Name,'ABCDEFGHIJKLMNOPQRSTUVWXYZ ,','abcdefghijklmnopqrstuvwxyz-')"/>/blue-denim-castaway-flip-flops/300)

Hope this helps!

Check out our SEOmoz LinkedIn Group. People add job listings all the time, and I'm pretty good at keeping it updated.

If I was in your shoes, even though your canonicals are working (great step #1), I'd be inclined to clean up the URLs are much as possible and 301 any duplicate pages that you can get approval for. You'll always want to have your canonicals coded in regardless. But having less actually pages will save you in other areas - content audits, Google bot and other bots crawl time, space on your server, confusion for customers, etc. Less can be more in this case.

We've been parring down our MozBar features to meet what most people use/want and also to make our tool unique. Since it's an SEO tool, the text to code ratio was removed as it isn't a clear SEO indicator. It doesn't tell you if your content is actually good for your SEO, just that it's there.

Antonio,

For work load and ease of change, I'd edit your robots.txt. Sometimes, search engines will still show the empty page in search results with only the URL if you don't put in a nofollow meta tag, but most of them should obey the robots.txt commands.

Of course, ideally, you'd want your website not to push those "future pages" without content live, but I understand that sometimes there are things that seem easy on a website that aren't.

Hope that helps!

Erica

I'd implement canonical tags for your duplicate content problem instead of noindex tags. This is the recommended practice for duplicate content.

As far as pages that you don't want indexed, when you use robots.txt to accomplish this, Google can/will still put the URL & title in SERPS. In order to stop this, you do need to put meta noindex tags on every page.

So there are probably two ways, they can get this information:

1. The legal way: grabbing IPs from log files and then cross-referencing them with geolocation and then providing some contact information for that company. This is hit or miss depending on if they're coming from a business or if the business's IP info actually matches where they are.

2. The illegal way: hacking into someone's browser cookies, who's visiting your site, and figuring out direct information from that.

From what I understand, it sounds like your CMS creates a bunch of 404 pages that then auto-bounce back to their parent folder. But that these are phantom pages that don't exist? My recommendation is to use robots.txt to no follow, no index. If they are blank pages, you can also add that to the blank page's meta data. Canonical tags are always a good idea.

Have no fear, MozCon will be recorded, and videos will be available in the coming months after MozCon.

Unfortunately, we cannot currently track local rankings in India. Right now, we're only covering the US, Canada, UK, and Australia. Sorry.

As the others mentioned, we can, however, track general, non-localized rankings for India.

This article lists a bunch of Grease Monkey scripts that do this.

The first thing to do is familiarize yourself with Google's rules and restrictions around adults sites. The second thing to consider is that Amazon is the biggest seller of sex toys. But before you have a panic attack at trying to take on the giant in e-commerce, I'd do a bunch of competitor research and see what they are doing. Adult sites that distinguish themselves do it in ways just like non-adult sites, and largely by being different. PornHub wrote this article on our site about what they did, which was essentially something their competitors weren't doing. Additionally, if you have a retail location, make sure your local game is on.

Unfortunately, I still don't have a date. To be transparent, most of our resources right now are focused on getting people into Moz Analytics, so we're a bit resourced-strained for getting this up. We've hoping soon, but that's really all I can tell you.

Though I can assure you that our speakers give really solid information and not fly-by-the-night techniques, so even if MozCon was a few months ago, the information isn't outdated.

Hey friends - We had this bug a while ago and fixed it, but likely our move to https caused it to happen again. We'll definitely get it taken care of, and in the meantime, sorry about the troubles!

Okay, got it. To figure out how much value you're getting from that traffic, you need to figure out what you want customers to do once they hit those pages. If the customers do the thing you want them to do (buy, lead gen, consume more content, etc), then you still want to keep those redirects. If they don't, you can probably kill them without any repercussions to the brand/site. You can also reach out to those sites that are hosting your old links and ask them to put in new, more relevant links, if those links are still valuable in that the traffic from that link drives customers to do the action you want them to do.

I also wanted to add something that, I know you may fully realize, but no one has said yet, for every page that you're 301'ing to the new structure, you can take that old page and those old assets off your server. All you need is the 301 redirect code.

Usually, the reason is because category pages tend to be light on content and thus, not as indexable. And sometimes category pages are noindex due to this. Google is likely looking at your product page and finding it more relevant, based on content, than the category page for people who are looking for "sleeping bags."

Awww. Thanks for your kind feedback. We're glad you dig the q&a forum here.

Hi friends,

Sorry for the delay in responding. I have a great news -- as of today, this bug for Q&A forum emails has been fixed. (Any emails timestamped now 1:20pm PDT and going forward.)

Unfortunately, this bug is also happening in Blog comment emails. The blog and Q&A systems are not tied together, and the development work on it rests with another dev team. The good news is that we know the issue (half the battle). We are working on prioritizing the work, and I'm hoping that in the next couple months (at most), this will be fixed.

Thanks

Update 7/28/16 - The blog comments are also fixed. This bug is fixed!

A mix of both depending on your business cases is best. I suggest reading this post from Ruth Burr Reedy about the domain changes and 301s that she did for Moz during our rebrand when she was head of SEO here. We did a mix of both killing pages and 301s depending on the page itself and its use.