Welcome to the Q&A Forum

Check with the IT folks or hosting service for your client.  I think this is an outside chance, but if you have been running spiders from your home computer to check the site, you may have been hitting it too hard and slowed the site down and the server may be blocking your IP as you are seen as a spammer.   That is why you change ISPs you are golden as you are seen as a different "user".

I took down one of our sites once with a spidering tool.  They were pushing new code right when I hit the site.  Also, the number of requests a second I thought were ok, well, it was during peak traffic time. (DOH!)

I adjusted my crawl rate down and everything was ok.   Again, this is just a guess, but worth checking considering your symptoms.

Good luck!

Answer looks to be here

http://jensontaylor.blogspot.com/2012/03/blogger-changes-top-level-domain-names.html

"The reason my blog url is ending in .co.uk is because I access my blog from UK. If you access my blog from another country you should see the extension of that country, appended to my blog. So if you access my blog from Australia you should see my blog address as jensontaylor.blogspot.com.au"

"The reason behind the change in the top level domain names is to accommodate Google with the facility to censor blogs based on country of access."

More information here

https://support.google.com/blogger/answer/2402711?hl=en

So it looks like this is impacted by where you access the blog from.

One thing you need to consider is not just the quantity of the links you lost but the quality.  If you are losing thousands of links with a PA of 1 and DA of 0, well then good riddance!  They were not helping you to start with and if anything they were hurting you.

So, I would not think, "Well, we lost 1345 links and so we need 1345 links back".  I would say, how can we help the client to do some #RCS to earn some quality links.

http://moz.com/blog/an-interview-with-wil-reynolds

Watch the talk by Wil Reynolds about 1/4 the way down.

Good luck!

I had "via this intermediate link" when I had a link from another website to an old URL that I had a 301 redirect to the new URL.   The link was credited in GWT to the new URL.   I had a bunch of links that were being caught by a default 301 to a main directory.  I used the "via intermediate URL" to then find what old URLs still had links and then change the 301s for those specific old URLs to more appropriate URLs on my site.

Run those intermediate links through your browser etc and see if that tells you anything. What is strange is that the parameter in the URL is another URL.  You may want to check those out too separately.

If I am understanding the question - I think pulling in some body copy from each search result (and not just the whole page) would be fine.  I think Google will see that this is a search result and that you are pointing to other pages.  You are probably going to pull in text from the title too.  This is common practice in search results - heck Google does it!

If you are still concerned about the pulled in descriptions, your option is to setup the system to have an alternate description for each page.  Use the alternate description when you pull it into your main page.  It is more work, but it will eliminate this issue.

Separately, paginated pages no longer need to be canonicaled to the index page.  You can use rel next and prev.

http://googlewebmastercentral.blogspot.com/2011/09/pagination-with-relnext-and-relprev.html

https://support.google.com/webmasters/answer/1663744?hl=en

It explains to Google the relationship between P1 and P2,3,4,5,n etc.

Beyond that, you need to watch that you do not get into too many paginated pages to get to the exact same product pages.  Lets say you had 1,000 widgets that were blue, red and green and also were Free, Expensive or Cheap.  You would have several sets of paginated pages (one set for Blue, one for Red, Green, Free, Cheap, Expensive, one for Red and Expensive) etc.  It gets to be a little crazy as they all lead to the same set of widget product pages.  You need to manage how to have Google crawl all that and not have your Paginated Category pages look like duplicated. Adam Audette writes great stuff on this.  Look here for things to consider

http://www.rimmkaufman.com/blog/site-search-dynamic-content-and-seo/01032013/

Hello there!

I see that based on the date listed on the page you posted this yesterday on 7/24/2013.  Depending on how often Google visits your site, it may not spider all your pages every day.  It has just been 24 hours so you may need to give it more time.

Things that can help speed this up is to make sure that you have this page listed in your sitemap

http://www.howlatthemoon.com/sitemap.xml

I did not see the page listed there and that is a common place that Google looks for new additions etc.

Good luck

Optimal as you mention is a country specific domain, each hosted on a server in that country and you go into Google Webmaster tools and designating the country for that site.

Take a look at what Google states

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=182192

Even though you could setup one site and then setup the different country sections under different subdomains etc, it seems like they are suggesting having domain specific sites is the best option.

I think if you show them that "This is what Google suggests", that should bear some weight.

All that aside, the strongest argument comes from the user side of things.  If I am an american, will I go to a site from Spain to find what I need.  Probably not.  Unless they have customers who expect to visit a site in Spain to get a product, the separate TLD would make sense.

In my head, good UX results in good SEO.  I think you need more copy for the user to talk about all of your great Houston SEO experiences, but that will also give more fodder for the bots to read and traverse.

Other things that come to my mind.  Google search console is looking at your entire site.   On your site as a whole, the word "houston" is mentioned a lot and the word "seo" is mentioned a lot and probably "houston SEO" happens a fair amount. But if that is the case, out of all the pages on your site, which one should Google rank for "Houston SEO"?  What signals are you giving that are clear to the point of whacking a 2x4 over the head that "this" is the most important page on your site that relates to that key term.  You may need to look at how you use keywords across the site.  You may want to look at how you link internally and on what anchor text you use internally.

I know this is not the same example, but what comes to my mind is this WBF

https://moz.com/blog/why-good-unique-content-needs-to-die-whiteboard-friday

Rand states that if you really want to rank a page on something, your content has to be 10x better than everyone else's. It has to be that good to stand out to Google and users to rank in the SERPs.

Is your SEO houston page content 10x better compared to all the other pages on your site (and to your competitors' sites) as it relates to Houston SEO?  The suggestions I gave on how to add more content related to Houston SEO to a specific page underscore that idea and in the process, it also adds some additional keywords on the page without it looking too spammy.  Good for users good for Google.

The other aspect that not a lot of people talk about is how making the page "sticky" aka making it really useful, helps with SEO.  Google tracks people clicking on a result and then coming back to the SERPS with the idea that if you stay longer on a page then it must be "better" and it helps with the rankings.  (This is also why making sure your meta description is on point for conversion as you have to get people to click that result if you ever want a change to have them get to your site and stay a while).  Adding all that additional "houston SEO" content on a page will hopefully cause the user to stay longer and improve your quality score with Google from the standpoint that Google should see people spending more time on your site before or if they click back.  This is also why using bulleted lists and short paragraphs work.  People skim.  If they see a bunch of long text, they will leave quick as they do not want to process that info.  Lists, bullet points, bolded highlights, pictures, I will hang around and glance through that for sure!  

Moz blog post and video on pogo sticking.

The rel next prev is not for duplicated content - it just shows google how the parts relate to the whole.

An alternative to the rel next prev is the "Classic Pagination for SEO" that uses noindex another article by Adam

http://searchengineland.com/the-latest-greatest-on-seo-pagination-114284

If you have a duplicate issue, this would solve it as you would noindex all the duplicate pages.

What you need to do (and I can't do this for you), is to look at all the crawl paths that you are providing Google.  As I mention above, you are not doing any favors to Google or to your site when you show Google an infinite number of paths to get to the same content.  It just wastes Google's time and you don't want to do that when Google also has to crawl the rest of the internet.   If you solve this issue, you will solve your duplicate issue.

AJ Kohn just posted an article on the concept of crawl budget that talks about this.  I think the article is quite good and it explains why we need to look at all the topics of noindex, nofollow, robots, canonical and rel next prev  http://www.blindfiveyearold.com/crawl-optimization

I see the entry for the page in the XML now.  I also just searched the URL in Google and see it there as well.  Looks like this was just a timing issue.

Yes - forget the 302, I get nothing but headaches from them.  I would not use a 301 either as a permanent redirect would not be accurate in how you are relating one page to the other.   I think you need to consider if this needs to be automatic or not (regardless of how you forward people).  I would suggest using a Javascript based approach - here are the details on why.

Here is an article from Bill Hunt http://searchengineland.com/understanding-the-seo-challenges-of-language-detection-47524 he mentions how you can use an automatic IP address or a browser language based approach to send people to the proper page.  There is a problem with this for the spiders.

"Both of these methods are problematic for search engines, because spiders often crawl from a specific location and don’t signal language preference in their server request. For example, if Googlebot, crawling from Mountain View in California, requested a German language page on a site using IP detection the web server would detect an request from a IP in the U.S. and the crawler would be routed to the U.S. version and potentially never see the German language version. The same scenario on a site with browser detection would not detect any language preference and thus route the spider to the default version of the site which is typically English for US companies and the local language version for many country installations of scripts and web servers."

Matt Cutts mentions this in http://www.youtube.com/watch?v=7paVYBgH0Hw for using IP as (in 2011) Google was only crawling from US IPs.

Bill Hunt in his article mentions a solution, don't redirect the spiders "The easiest search workaround for either of these detection methods is to simply determine if the requester is a search engine and exempt them from any redirection, giving them the page they want. Note I did not say redirect them or any other action that could be misconstrued by conspiracy theorists as cloaking but simply let the spider have the page it requested. This will ensure spiders can index your local language content."

If that gives you concern about to handle the automatic redirect - see the bottom of http://moz.com/community/q/what-countries-does-google-crawl-from-is-it-only-us-or-do-they-crawl-from-europe-and-asia-etc Hannah Smith from Distilled suggests using a Javascript overlay or some other "chooser page" to direct them to it.  If you link up everything correctly, Google can crawl both versions of the site, but only the user is shown something dynamic to direct them where they want to go.   This also ensures that if there are any errors in either the IP address shown or the default language in the browser, you have a way to fail gracefully and allow the user to select where they want to go on the site.

Good luck!

One wrinkle.  If the category pages are in Google and potentially ranking well - you may want to 301 them to consolidate them into a more appropriate page (if this makes sense) or if you want to get them out of the index, use a meta noindex robots tag on the page(s) to have them removed from the index, then block them in robots.txt.

Likewise, you have to remove the links on the site that are pointing to the category pages to prevent Google from recrawling and reindexing etc.

If you have a 301, you use the 301, done, mission accomplished.  Google should drop the original page and start to use the new page in its place in the SERPs.  This is also automatic for the user as they are moved from one page to the other.  One thing, you want to make sure that the page that you are sending people to is semantically related to the page that they were sent from, otherwise you risk losing rank in the SERPs.

If you use the 301, there is no original "page" that you can put the canonical or a noindex.

If you could not 301, you would want to only use the canonical.  Google usually will treat a canonical like a 301.

If you use a canonical, you should not have to use the noindex.

I have not seen numbers in a URL hurt SEO.  I do have issues with commas in your URL as they do not play nice when you copy and paste them.  I would use dashes.

If you are looking to try and become a Google News publisher, Google requires numbers in the URL

https://support.google.com/news/publisher/answer/68323?hl=en

I have also used numbers in the URL to help with my backup 301 redirects

Say you have a url

/articles/34543-slug-goes-here.html

The # in the URL is the article ID in my database.  If I get a error on any one of these urls (and they happen)

/articles/34543-slug-goes-here.htm

/articles/34543-slug-goes-h

/articles/34543-slu

/articles/34543

/articles/34543-slug-goes-Here.html

etc

My system just has to match on the ID of the article and then will 301 to the correct URL.

I think the key is not to overkill on the numbers to make them too many digits long etc.

Also, if you are only depending on slugs to differentiate URLs then you have to have a system to make sure they are unique each time.  Using an ID number in the URL ensures they are unique.

Well, then, ignore what I said.  It did sound like you had a .com and a .co.uk.  Google likes to keep sites in a given country.

I would suggest you look at this Moz post

http://moz.com/community/q/what-is-the-best-way-to-rank-well-on-both-google-co-uk-google-com

Yes, the crawler will avoid the category pages if they are in robots.txt.  It sounded like from the question that this person was going to remove or change the category organization and so you would have to do something with the old URLs (301 or noindex) and that is why I would not use robots.txt in this case so that those directives can be seen.

If these category pages had always been blocked using robots.txt, then this whole conversation is moo as the pages never got in the index. It is when unwanted pages get in the index that you potentially want to get rid of that things get a little tricky, but workable.

I have seen issues where there are pages on sites that got into the index and ranking but they were the wrong pages and so the person just blocked with robots.txt.  Those URLs continued to rank and cause problems with the canonical pages that should be ranking.  We had to unblock, let Google see the 301, rank the new pages then put the old URLs back into robots to prevent the old URLs from getting back into the index.

Cheers!

There is logic built into Excel that allows you to both COUNT and COUNTIF or COUNTA

http://office.microsoft.com/en-us/excel-help/countif-function-HP010069840.aspx

If you are wanting to count the total number of domains you can setup a function to count non blank cells

http://office.microsoft.com/en-us/excel-help/count-nonblank-cells-HP003056101.aspx

=COUNTA(A2:A6)

This would count non blank cells from A2 though A6.

You can use this also to count if you have a value that is greater than a given number.  Say you wanted to see all the linking domains with a DA greater than 20

=COUNTIF(B2:B7,">20")

You get the idea.  Note that you may need to format your number cells as Numbers to get this to work.

You should be able to use one of the logic functions above in Excel to get you what you need.

I guess I would have to see the site and the tool to understand your point as it does not make sense if you would make a tool that is unrelated to your core business as that would take away from your core business structure.

I would do some testing with some of your users / audience to see what they think.  It may or may not be jarring to them and could help you with your answer in what direction on where to go.

Here is what I am seeing.  When I view source and look at the link for Floppy Straw Hats I see the URL

http://www.surfdome.com/baku_hats_-_baku_congo_hat_-_volcano-108584?i

and this link shows me a 200 when I run through it directly.   This is probably what Screaming Frog is doing.  I would re-run the frog and set the user agent to Google Bot just to see what happens there.

Now when I view that link in the browser and I hover over it and right click the URL and copy I get

http://go.redirectingat.com/?id=207X327&xs=2&url=http%3A%2F%2Fsurfdome.com%2F&xguid=e3d44c6f62047018ab665c4fe5392990&xword=surfdome&xcreo=300001&xpid=875260&sref=http%3A%2F%2Fwww.myfashionlife.com%2Farchives%2F2013%2F07%2F16%2Fget-anne-hathaways-paper-denim-and-cloth-look%2F&pref=

When you run this, you get the 302 redirect to the target page

If you scroll down to the bottom, you see the skimlinks JavaScript that is doing this manipulation.  FYI it is also adding redirect a link to "Surfdome" at the end of that same line.  This is not linked at all in the source code.  You have a simple JS rewirte action going on there.

So the bot sees the regular URL and the human sees a redirect via JavaScript.

Depending on if you wear a white or grey hat, this could be considered "cloaking"

https://support.google.com/webmasters/answer/66355?hl=en

"Cloaking refers to the practice of presenting different content or URLs to human users and search engines."

This is not the traditional use of the redirect. You would often see a completely different page shown to the bot vs the human using JS versus your example of just showing a 2 different links on a page.  That said, Google is reading more and more JS these days http://googlewebmastercentral.blogspot.com/2011/11/get-post-and-safely-surfacing-more-of.html

Your issue is not about the 302 passing link equity, but if you want to get penalized for cloaking or not.

The other point that comes up is that since you are paying these bloggers to have this link on the site, I would call these paid links

https://support.google.com/webmasters/answer/66356?hl=en

I know you said, "these are organic links" we are now just paying for the referrals.  Well, if Google finds that this is worth penalizing, then you have no one to argue with but yourself.

As I see it, you have 2 choices

1. No follow the links and do not use them for link juice but to pay for traffic.

2. Lose the redirects and use the links for ranking benefit (plus you can still get some traffic).

Honestly, seems like you were already getting organic links and traffic for free, so not sure why you would pay people for what you already had.  I am sure this helped to get additional links, but you just need to consider the points above to see what is more important for your site.

Hope this helps!

One other thing I forgot.  This video by Matt Cutts

It explains why Google might show a link even though the page was blocked by robots.txt

https://www.youtube.com/watch?v=KBdEwpRQRD0

Google really tries not to forget URLs and this video reminds us that Google uses links not just for ranking, but discovery so  you really have to pay attention to how you link internally.  This is especially important for large sites.