Welcome to the Q&A Forum

There shouldn't be a robots.txt file on the /blog section anyway, should always be in the root. It was just something to have a look at.

I'm having a look just now and also don't see any problems.

You've nothing in the robots.txt file and nothing in meta-robots for the header.

There's 42 pages in the site: command and a similar number in your sitemap.xml so I presume that's right. 6 pages in site:/blog which again looks right.

I've tried using SEOmoz's tools on your site though and it just tells me that your site doesn't resolve. edit Managed to get it to resolve on the 3rd try for a crawl, but using the on page report card checker it's still giving me problems.

You're definitely returning a 200 message with a site when I check using any other tool though, so I'd get in touch with SEOmoz directly and see what's wrong with their tool - help@seomoz.org

Just to confirm you're not doing anything tricky server side to prevent scraping are you?

Yes.

In your permalink settings just add .html (or whatever) to the end of your permalink settings so %postname%.html and that will work... for posts.

To add it to pages you can either use this plugin - http://wordpress.org/extend/plugins/html-on-pages/ - bearing in mind it's rarely updated, hack wp_rewrite in wp-includes/rewrite.php

add_action('init', 'change_page_permalink', -1); function change_page_permalink() { global $wp_rewrite; if ( strstr($wp_rewrite->get_page_permastruct(), '.html') != '.html' ) $wp_rewrite->page_structure = $wp_rewrite->page_structure . '.html'; }

bearing in mind you'll have to do this every time you update

or add it in your .htaccess

RewriteEngine On RewriteCond %{REQUEST_URI} !.[a-zA-Z0-9]{2,3,4} RewriteCond %{REQUEST_URI} !/$ RewriteRule ^(.*)$ $1.html

I'm not taking any responsibility for messing up that last one

A blanket answer would look something like

25% Keyword exact and almost exact match anchor text
25% Partial match anchor text and longer tail anchor text
25% Raw url and domain
25% Generic style links (click here, visit site, now, here, etc...)

For something like 50 articles, assuming you're putting them out in article directories and that was all the links that page was getting, I might not be so bothered and go for a much higher percentage of exact and partial and pick up more generic anchors elsewhere for the page.

It also depends how strong the site and page are.

I'm not quite sure exactly what it would take to trip Penguin but more than just the anchors it'll depend on where those links are coming from. If they're all good sites then you could go 50/50 on the exact match and should still be fine. If they are of, let's say, less authoritative sites then I'd still want a few other anchors in there.

If you're not taking Zach up on his offer, have a look at http://yoursite.com/robots.txt and see if it has

User-agent: *
Disallow: (your blog url in here)

If it does you'll need to edit your robots.txt file to not have anything you don't want disallowed in the disallow section. You can do this via ftp.

If it's in WP itself there may be another robots.txt file at http://yoursite.com/wp-install/robots.txt which, in theory, could also be preventing crawling if it has anything disallowed in there.

Again, editable via ftp or maybe this plugin - http://wordpress.org/extend/plugins/wp-robots-txt/

As it already says that it should be public probably not WP, but worth a look anyway.

The directives issued in a robots.txt file are just a suggestion to bots. One that Google does follow though.

Malicious bots will ignore them and occasionally even bots that follow the directives may mess up (probably what's happened here).

Google may also index pages that you've blocked as they've found them via a link as explained here - http://www.youtube.com/watch?v=KBdEwpRQRD0 - or for an overview of what Google does with robots.txt files you can read here - http://support.google.com/webmasters/bin/answer.py?hl=en&answer=156449

I'd suggest you look at other ways of fixing the problem than just blocking 1500 pages but I see you've considered what would be required to fix the issues without removing the pages from a crawl and decided the value isn't there.

If WMT is telling you the pages are blocked from being crawled I'd believe that.

Try searching for a url that should be blocked in Google and see if it's indexed or do site:http://yoursitehere.com and see if blocked pages come up.

The FTC? Oh you crazy Americans

Yeah, might all go south, but meh, get nothing if you don't try : )

I'm just being an affiliate, not going to run one (even though there's a billion scripts to do so).

That's never going to happen, too many on clearly unmoderated sites.

If I was spamming myself (not that I'd ever advocate such behaviour) I'd just have kept up the velocity until I broke through but the fact they're a year or more old makes me wonder...

I'll probably sort the site out properly, add a bit of decent content and then decide whether to hammer it with more spam and/or build some decent links

So I just bought http://penny-auctions.co/ a couple of days ago, ignore the fact it's not set up yet and generally just a mess, and something kept bugging me about why the domain authority kept showing as 25 in the SEOmoz toolbar.

Now initially I'd set it up as www. and with the trailing slash so the PA was 1, however after a bit of exploration it seems that someone has been building links to the non-www version of the domain for at least a year!

The site has never been owned before so I've now made the non-www version the default and have a 5 day old site with PA 35 and DA 25!

SEOmoz shows PA - 45 links, 20 domains and DA - 341 links, 113 domains.

Majestic Historic - 8598 links, 236 domains and Fresh - 160 links, 62 domains.

Brilliant! Except they're all spamtastic.

What do you think this will do for my future attempts at ranking?

Should I create pages that have links to them or just 301 them?

Hi Daniel, you're getting a lot of good advice in this thread and I know it's a lot to take in at first.

I've never worked in beauty so don't know off the top of my head but the beginners guide does have a section on keyword research.

And here's a timely post on simplifying keyword research which gives a brief introduction to some of the tools you have at your disposal from SEOmoz.

Could go a lot further wrong than  the SEOmoz beginners guide.

However your question is very broad. Perhaps you have a specific thing you would like to achieve first?

If I go into the adgroup I can see the cost by day no problem (just toggle graph options), but I want to do it a level up rather than collate them all individually.

Seems like something that should be possible

Easy one for someone I'm sure, but I can't figure it out.

In Adwords under campaigns "Widget" I have ad groups "Blue", "Green" and "Red" each with 3 ads in them.

How can I produce a compared report on cost per day of each ad group.

So I want something that looks like

1st     2nd   3rd    ...
Blue    £12   £14    £9    ...
Green  £8    £11     £5    ...
Red      £9    £22   £16   ...

Possible?

Well, I don't think there is any SEO going on. That site has not been made to be more accessible to search engines (or users), however still ranks number one.

The exact match title, domain age and legit info on the site appear to be the only significant factors as I don't see a much else going on I'm not convinced they're doing anything dodgy though, just lucky.

Possibly an anomaly or perhaps as other factors catch up with it the site will drop in rank.

Well, the simple answer would be that SEOmoz isn't Google.

Just because SEOmoz haven't crawled any relevant links, etc doesn't mean that Google hasn't.

It is a horrid little site though, the HTML version has "INSERT TEXT HERE" all over it. I would say activitysuper may be on to something with the domain age, plus the fact that it's for a real company (lots of contact info).

If your CMS doesn't do it then you'll need a rewrite rule, but I'd recommend that you try and sort out the CMS in the first instance.

Are you using a common CMS?

Lowercase, every time.

4 point swing?

http://blog.greenlightsearch.com/greenlights_search_blog/2011/10/how-much-do-google-rankings-fluctuate-in-a-day.html

I just checked manually using proxies, depersonalization and language settings and I see you in 11th too, but the above article might give you some sort of insight.

Unless you're checking at the exact same time it's hard to tell.

I'm not particularly up to date on how Google's datacenters work these days either, but sometimes it can be different depending on which one you hit.

Have you tried updating to the latest version of both?

Do you have in your header.php file?

You've made the changes already, without making a note of what you've changed from > to?

Well, a) Doh! b) Alan's suggestion of looking for 404 errors in webmaster tools is a decent place to start. Add redirects for anything that's throwing up an error.

I would imagine it'll be a heavily manual process if you've already changed them, though I'm not sure what your CMS is capable of or if your new URLs were sufficiently thought through to allow you to create a rewrite rule

e.g. if the changes were something like
from http://example.com/?id=123&name=product&cat=stuff
to http://example.com/stuff/name/123
then you could add just one rule for the redirects

Version of WP? Version of the plugin? Has it ever worked? Why are you asking here and not on the WP support forums?

I'm going to fire in the dark and say you need to either add to or remove from your header.