Welcome to the Q&A Forum

Hi there,

I see what you're trying to do, and I think I understand it. You're attempting to conserve your link equity and flow it only to the most important pages, or what we use to call "pagerank sculpting."

The good news is you don't really need to worry about it. These days, adding nofollow to your links doesn't really increase the equity flowing through the followed links. And in fact, you could be shooting yourself in the proverbial foot by denying equity passing links to your lower product pages.

Best time to use nofollow for internal pages is typically to increase crawling efficiency, or to prevent bots from visiting pages you don't want indexed anyway. Attempting to scuplt link equity in this way could cause lots of unintended negative consequences and my advice would be in most cases to let your link equity flow freely throughout your site in a way that was natural to both humans and bots alike.

Best of luck!

Hi Jessi,

If I had to pick one of the 3 options, I'd chose the first, simply because it's the shortest.

1. In general, out of all ranking correlations we know of, longer URLs tend to rank slightly lower than shorter ones. It's not a hugely significant factor, but neither is keyword usage in the URL, as recent correlation data has also shown.

2. Many webmasters believe that URLs without numbers tend to receive a higher CTR than URLs with numbers. That's because our eyes tend to glaze over numbers, and they have an auto-generated look and feel to them. Personally, if numbers are necessary, I'd put them as close to the end as possible.

3. Along the same lines, there is some belief in the SEO world that a shorter directory structure (fewer folders and backslashes) tend to perform better in search, as this creates a virtual flat architecture for Google to crawl. That said, the benefit from this is most likely small in relation to other factors.

Just my 2 cents. Hope this helps!

Hi Maureen,

Sorry you couldn't find any link metrics for your site, but maybe we can help.

First of all, no two companies ever report the same amount of links. Most links on the Internet are "junk" and Google Webmaster has been known to over-report unimportant links, and under-report links that matter.

If OSE shows 5 links to your site, you may need to select "root domain" metrics from the various drop down options to make sure you're seeing the right metrics. Otherwise you may only see page-level metrics, which might not be what you're looking for.

For example, to see all the linking domains to your entire site, choose the "linking domains" tab and select "pages from the root domain" from the drop down. Hopefully you will see at least one domain listed. Use the "root domain" option on the other tabs to find your links. After this, the download to CSV should accurately reflect the on-screen report.

Hope this helps! Let us know if you have any problems. Best of luck with your SEO.

Sounds like you've got a good handle on your strategy, which to me seems sound.

Couple points of advice:

1. Make sure the microsites have a clean backlink profile. Use OSE or another tool to check for paid links, spammy article submissions, etc. You want to make sure not to transfer any bad links to your main site.

2. Do your best to 301 redirect individual URLs to specific URLs on your new site, keeping care to maintain the subject matter, content, structure, title tags etc. If these change too much, Google will interpret this as a change of subject, and you may lose any transferred authority.

3. Follow best practices for migrating domains.

Hope this helps. Sounds like your on the right track. Best of luck with your SEO!

It certainly won't hurt, as long as you keep the following in mind:

1. Not every directory link is right for your site. Be selective and don't go for every link out there.
2. Pace yourself. A good hint I got from Ian Lurie's Fat Free Guide is the 2:1 rule: for every two directory links you build, make sure to build one genuinely natural link. This helps to keep your link profile looking “natural”.
3. Research. Understand where your link will be placed before you go after it. Check Google’s cache of the page to make sure they are indexing it. Large directories are often plagued by bad SEO, and not every link carries the weight it should.
4. Seek relevancy. Ask yourself if this link has the potential to send qualified traffic to your site. Even if it's a small amount of traffic, it's probably worth the effort.

I originally wrote these tips in a blog post published a few months ago.

According to rule #2, you should be safe with a couple of directory links - but if you are just getting started I wouldn't push it further than this. To really add value, try to ensure they are related to your niche. The harder it is to get into a directory, the more value it adds.

Although robots.txt and meta robots appear to do similar things, they both serve different functions.

Block with Robots.txt - This tells the engines to not crawl the given URL but tells them that they may keep the page in the index and display it in in results.

Block with Meta NoIndex - This tells engines they can visit but they are not allowed to display the URL in results. (this is a suggestion only - Google may still choose to show the URL)

Source: http://www.seomoz.org/learn-seo/robotstxt

The disadvantage of robots.txt is that it blocks Google from crawling the page, meaning no link juice can flow through the page, and if Google discovers the URL through other means (external links) it may show the URL anyway in search results, usually without a meta description.

The advantage of robots.txt is it can improve crawl efficiency - useful if you find Google crawling a bunch of unnecessary pages and eating up your crawl allowance.

Most of the time, I only use robots.txt to solve problems that I can't solve at the page level. I usually prefer to keep pages out of the index using a meta NOINDEX, FOLLOW tag.

Hi Tom,

Thanks for posting about this important issue. Building links is hard work, and I'm sure this is incredibly frustrating to you.

The primary reason links don't show up - either in Linkscape or WMT - is that they are built on pages that don't get crawled. This means that the link is built on a page that doesn't have many links itself pointing to it.

For example, a link posted on the New York Times will be crawled and indexed very fast. But a link posted on an obscure article submission site may never be crawled. Additionally, links like these don't count for very much, even when they are crawled.

As you mentioned, often times you'll see links from low quality, spammy link networks in your competitors backlink profile. I know how frustrating this is! But don't be discouraged, because often these links pass no real value, and Google is really cracking down on these (at least some... finally)

When link building, pay close attention to the link metrics of the site where you want to get a link from. Does the page have good Domain and Page Authority? Is it indexed by Google? When was the last time Google crawled the page? Does it have good MozTrust? Does the domain link out to spammy sites? (hopefully no) These are all questions you should be asking when building links.

The reality is, link building takes practice and determination, and requires thinking outside the box. But if you practice enough and mix in a bit of creativity and persistence, it will pay off.

Hope this helps! Best of luck with your SEO!

It can be tricky blocking parameters with robots.txt. The first thing you want to do is make sure your are actually blocking the URLs. There are a few good robots.txt checkers out there that can help:

• http://tool.motoricerca.info/robots-checker.phtml
• http://www.frobee.com/robots-txt-check

You're file is probably going to look something like:

User-agent: *
Disallow: /*?nocrawl=1

... but this could vary depending on exactly you don't want crawled

+1 to Igal's suggestion of handling these via parameter settings in Google Webmaster Tools: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1235687

Finally, if your goal is to keep search results out of the index (it probably should be) then you should also highly consider using a meta robots NOINDEX tag on all search results pages. You can also slap a nofollow on links pointing to search results as this might also help Google steer clear of those pages.

Best of luck!

Edit: Here's what John Wu of Google Webmaster has to say...

"We show this warning when we find a high number of URLs on a site -- even before we attempt to crawl them. If you are blocking them with a robots.txt file, that's generally fine. If you really do have a high number of URLs on your site, you can generally ignore this message. If your site is otherwise small and we find a high number of URLs, then this kind of message can help you to fix any issues (or disallow access) before we start to access your server to check gazillions of URLs :-)."

I have to agree with most of what Stephen is saying. The whole concept of PageRank sculpting can quickly can quickly fall into a murky debate filled with dangerous half-truths and misguiding answers.

The simple answer is to make sure your less important pages are less important in your navigation. Google will still assign a certain amount of PageRank to these links/pages, but other link signals such as placement, context and relevance will more than compensate.

The real question is - how do you get your category pages to rank higher? In the days of Penguin, higher PageRank via sculpting through site-wide navigation links probably isn't the way you want to go.

I'd try to bolster the category pages with natural, in-body links contained within great articles/content on my site. For example, whenever we talk about link building here at SEOmoz, we often reference the link building category. This category is not linked to in the universal navigation, except in the sidebar of the blog.

By using links like the one I just inserted, I can not only vary the anchor text, but I just added relevance and context to the link, giving a possible boost to the ranking signals.

Regardless, I hope this helps. Best of luck with your SEO!

Hi Marie!

Sadly, I don't use Xenu anymore either. Most of the solutions to find orphaned pages are either hit-and-miss manual methods (search OSE, search your server files). Or you could use a method like Agents of Value describes here.

Couple of posts that may help:

1. Find Orphaned Pages From Your Sitemap.xml File with Excel and IIS Toolkit

Requires IIS toolkit, which unless your installing on an external machine, isn't mac friendly

2. 4 Tips for Technical SEO

Ian has some great tips here, including:

• Search the server log files for every unique URL loaded over a 6-month period. Compare that to all unique URLs found in a site crawl. People have a funny way of stumbling into pages you’ve accidentally blocked or orphaned. Chances are, blocked pages will show up in your log file, even if they’re blocked.
• Do a database export. If you’re using WordPress or another content management system, you can export a full list of every page/post on the site, as well as the URL generated. Then compare that to a site crawl.
• Run two crawls of your site using your favorite crawler. Do the first one with the default settings. Then do a second with the crawler set to ignore robots.txt and nofollow. If the second crawl has more URLs than the first, and you want 100% of your site indexed, then check your robots.txt and look for meta ROBOTS issues.

3. Supposedly, Webseo has an automated option to find orphaned files, but I haven't used it nor can I vouch for it:http://www.webseo.com/

Hope this helps! Let us know what works.

As others here have addressed the merits of directory links, let me address your question more narrowly. Do deep links, if they are from the same page, count?

Short answer, yes. Results from several experiments over the years have shown that Google only counts the first link on a page to a particular URL, but links to multiple URLs still count. (although there are ways to get around this as well.)

Think of them like links on your homepage, all pointing to various internal pages on your site. Each link passes value, although certain links pass more weight than others.

Almost always, the first link in any price of editorial content carries more weight than all the others. Multiple links to the same domain carry much, much less value.

Best advice: Deep linking from directories may carry value, but in light of Penguin and Google's mass de-listing of several directories, I'd be careful to only obtain links from the highest quality of human-edited niche and specialty directories. I'm no longer a fan of general directories at all, although more testing needs to be done.

I agree with Shane - this sounds like a bad idea all around. It's fine to generate the title tag dynamically, but it should generate on the backend or your CMS, before the page renders in a browser.

Google does try to parse javascript for links and content, but it's inconsistent and I'm almost certain they would either see your non-scripted title, or get really confusing signals that could cause trouble later.

The best solution may be to explain to your Devs that Google most likely wont parse the javascript, and see if they can come up with another solution.

Best of luck.

To be honest, using Web 2.0 links in this way is viewed as a low quality activity. Many of these types of links were targeted by multiple Penguin updates, and too many of them could cause you a penalty.

I'm not saying that all 2.0 links are bad - they certainly aren't. But spammers try to build 100's of 1000's of these types of links on throw-away blogger and WordPress blogs that add very little value to the web. These are the links Google tries to sniff out, so it's best to avoid them. A network of these types of links is Google's very definition of "unnatural" and could quickly lead to a loss of rankings and traffic.

Put simply, Google wants you to earn your links.

To answer your question more directly, the first link from any domain tends to count more than other links. So 100's of links from the same domain don't add as much value after the first links. And yes, this can start to look fishy.

It's best to use 2.0 links in moderation, and make sure you are targeting high authority, well trusted sites. I recommend building profiles on good sites and earning links through adding value.

Hi David,

It's odd that Roger would pick up on so many 302s. When I ran a crawl of your site with Screaming Frog and the only 302s were for static assets, which I don't think roger would be crawling anyway. What sort of URLs are you getting the 302s for? If you think these are errors, feel free to contact the help team (help@seomoz.org) and see if they can help you sort this out.

You could most likely rewrite whatever Wordpress plugin to return a 301 instead of a 302 (or have a wordpress php developer do it for you - it should be a pretty simple job) but overall, it probably won't make much a huge difference in your SEO.

But in general, you want to minimize your redirects when speeding up your site. For example, SEOmoz uses a CDN, but each static resource is linked directly to the CDN server, without the redirect.

For my own wordpress sites, I use WP Super Cache and CloudFlare. Both are free and have sped up my sites amazingly well. In particular, I'm a huge fan of CloudFlare, given the extra security measures. Well worth checking out.

Hi Daniel,

To add to what David said....

Look at the URL of this page that we are on now: http://www.seomoz.org/q/adding-keyword-to-your-url

Our CMS automatically generates the URL from the title of the question. Additionally, this also becomes part of the title tag. Many blogging platforms like Wordpress allow you to customize the URLs of your posts.

Howdy,

Most of the major Google updates in the past 12 months have targeted backlinks more than anything else. So while it's always good to look at on-site issues and improve your site in every way, a thourough backlink audit is almost always prudent when you experience a major rankings drop - especially when the drop coincides with known Google updates.

Looking at your links in Open Site Explorer, we find several types of links that seem "suspicious" or "unnatural"

• http://personalisedtshirts.blogspot.com/  (sitewide, exact match anchor text - I highly suspect this is part of a blog network)
• http://www.wokietokie.blog.seo-catalog.com/Shopping/Gifts/ (penalized, deindexed directory)
• http://ww.w.b3directory.com/Business_and_Economy/Marketing_and_Advertising/?p=8 (low quality directory)

Unfortunately, these are the types of links targeted by Google and we often see these lead to penalties and or devaluation.

Have you received any messages/warnings in Google Webmaster Tools? That would be a good place to start. Typically, the best way to recover from this is to perform a complete link audit, then go through the arduous task of trying to get the links cleaned up, submit reconsideration requests, and possibly use the disavow tool.

I've been through it myself with several client sites and it can be a lengthy process the first time you do it. Some tips to consider:

1. What links to look at? First of all, John Mueller of Google recommends starting with the links listed in Google Webmaster Tools, as these are the links most likely to effect your rankings: http://www.youtube.com/watch?v=UX...

2. Third Party Tools: A couple of companies have developed tools to help identify bad links.

• SEOGadget Link Analyser - http://tools.seogadget.co.uk/ (has a great API - highly recommend this site)
• Link Detox - http://www.linkdetox.com/ (haven't used it, but I've heard decent things)

3. Manual Review - In the end, you're own eye is the best tool you have. You want to look for:

1. links with keyword-rich, optimized anchor text
2. Comment Signature and Forum Signature links (these are different from the more legitimate forum links)
3. Sitewide links, such as in the sidebar or footer
4. Obviously paid, or suspicious looking links on low-quality sites

Our friend Paddy Moogan wrote a great guide on how to do this step by step. You should check it out. I also made a video about earning high quality links. It might be worth a look: http://www.seomoz.org/blog/high-value-tactics-futureproof-link-building-whiteboard-friday

It usually takes Google 2-4 weeks to reply to reconsideration requests. In my experience they like to see both a manual effort to clean links, and I've also had success supplementing this with the disavow tool.

Lot's of information here. Hope it helps. Best of luck getting back on track!

Couple of good posts from Rand Fishkin:

http://www.seomoz.org/blog/10-myths-that-scare-seos-but-shouldnt-whiteboard-friday

http://www.seomoz.org/blog/whiteboard-friday-sitewide-reciprocal-and-directory-links

Most of the time reciprocal links are fine, as far as we know. It's when Google detects a pattern of reciprocal linking that looks unnatural. These include link wheels and 3 way linking schemes and plans so complicated I couldn't even begin to describe them.

In your case, you should be fine.

Howdy nyc-seo,

This is a really good question with lots of implications. Although there's no single "right" answer, there are a few things you might want to consider:

1. Subfolders are good for organizational purposes and as such can help structure your content. For example, SEOmoz puts all the blog content under seomoz.org/blog
2. Subfolders can contain keywords that help with CTR and possibly with rankings. This may be good in certain situations, like in ecommerce. i.e. example.com/bird-feeders/hummingbirds
3. That said, shorter domains tend to perform better in search results, and you want to avoid keyword stuffing in your URLs.
4. Also, too many subfolders and you can run into some crawling issues. In these cases, it's best to keep your site architecture as "flat" as possible, without too many additional layers of sub-directories.

Some additional resources that may help:

• http://www.seomoz.org/blog/understanding-root-domains-subdomains-vs-subfolders-microsites
• http://www.seomoz.org/blog/site-architecture-for-seo

Hope this helps! Best of luck with your SEO.

It's unfortunate it's got to this level. Some folks charge to remove links, and unfortuneatly, there's nothing illegal about this, and not much Google can/will do.

Google suggests that if you're a victim of negative SEO, to use the disavow tool and file a reconsideration request. Matt Cutts talks about it in this video.

http://www.youtube.com/watch?v=HWJUU-g5U_I

That said, be careful with the strategy. First you want to be sure that these links are actually hurting you. Have you been hit with a penalty? Any warnings in Webmaster Tools? A lot of webmasters are rushing to disavow links that they don't need to, and this can cause more harm than good.

Regardless, hope you get the situation worked out.

Best,

Cyrus

Hi David,

Great question. Couple of points I'll go over, and realize the answer only applies specifically to Moz. Others may find different optimal solutions.

1. Crawling categories isn't a problem for us. Each blog post is linked to several juice-passing category links within the body of the post. And we have 1000s of posts all linking to individual categories.

2. Our reason for putting categories in a drop down was simply to save space. We could have made the links crawlable using pure CSS/HTML, but we suspected it really wouldn't make a difference.

3. Sidebar links have diminished value. It's doubtful much link equity was flowing through them in the first place (for other architectures it may be beneficial to have completely robot friendly sidebar category links, but in our case they were so redundant they weren't really necessary)

4. Finally, when we moved categories into the sidebar, we noticed no change to rankings/traffic to category pages, so we left it as is. Had this been different, we would have reconsidered our strategy.

Again, this strategy is outside traditional "best practices" but practically speaking, it works just fine for us, but may be different for newer sites, sites without as much link equity, etc, etc.

Hope this helps! Best of luck.