Losing indexed pages on Bing & Google - robots.txt

[Tustep]

This post is deleted!

[AlanBleiweiss]

Tustep

I am going to be as honest as possible however harsh it may seem with my answer.  If the robots.txt file you included is from your site, that means your site is billstermiteco.com

So:

There could be many reasons this is happening.  First and foremost, I can tell you that it looks like you've gotten some extremely bad guidance as far as the concept of SEO goes just by taking a very preliminary look at your site.  Whether this specific issue is a primary cause of your problem, a contributing factor, or not is too complex to know this early on however consider this:

You have a main services page called "Phoenix Pest Control".  Off of that, you have another page called "Phoenix Pests Control", and right along with that, you have a page called "Pest Control Phoenix".

Wherever you learned or were told that it was good SEO to create three separate pages for the same primary topic, that is a serious problem in 2013 as far as it being a blatant attempt to artificially rank multiple pages for one topic while simultaneously attempting to rank each page for it's perceived "unique" phrase.

Search engines in 2013 are extremely proficient in identifying those three as truly being all so similar as to be considered almost identical in their intent.

From there, I scrolled your services pages and see similar problems where you're wanting to rank for every single town in your service area by loading up on pages that really most likely don't deserve to be ranked based on SEO best practices for a host of other reasons as well.

While I totally understand you are a legitimate business, and you have a valid desire to want to be found for all of your real services and in fact be able to rank regardless of the town, the way you've implemented even just that content tells me that if I were to dig into the rest of your site, or off-site efforts I will quite likely find many other problems of a similarly severe nature.

But lets go further.  Your Twitter account is not much more than an artificial link generator for your own site, so the trust signals that could come from Twitter are non-existent.

You have severe page load speed problems - the page I tested took 5.81 seconds to process which means Google is probably seeing similar problems with their crawler speed evaluation.  That alone could be half or more of you problem, but when you combine it with what I've mentioned, plus who knows how many other major problems you have, it's no surprise at all that you're getting the results you are now that Search Engines have come as far as they have in their methods and processes.

[KeriMorgret]

I'd also take a closer look at your robots.txt. With that message about the page being blocked in the SERPs, it sounds like your robots.txt file might be excluding a few too many pages. I'd be especially worried about the wildcards that are in there.

A site: on your domain also shows this site -- did you know it's there? http://nova.billstermiteco.com/

[Tustep]

@ Alan Bleiweiss - As a man that often works outside in temps exceeding 115 degrees
your use of “harsh” puts a smile on my face.

I sincerely appreciate the time and insight you have provided.  Ensuring enough business for 24 employees so that they can feed and support their families is a responsibility not taken lightly.

Your response is a golden opportunity to right a ship that I thought was on course.  It would be easy to place blame on “bad guidance” and/or hired “experts.”  The blame and responsibility falls directly upon me.  The ole’ “trust but verify” rule was ignored.

After throwing thousands of dollars at so called “experts” who seem to disappear,
fail to deliver, and/or apparently have no clue…this termite man has attempted
to take matters into his own hands.  It appears the equivalent of handing you my equipment and sending you out to service a home for our native species termites. LOL

It would appear I should abandon work on the “Myopic SEO” site and concentrate on
building a “Sustainable SEO” site.

I've taken this opportunity to view one of your Webinars (Vertical Measures) and found it informative. Are you available for "guidance?"

[Tustep]

Thank you Keri for taking the time to look at my site.

I was not aware of the nova site on my domain...and will have it removed.  Additionally I will research the "wildcards" issue and determine the best way to correct.  I'm reading that unless I intend to block pages from the Googlebot (which I am not) there is no need to have a robots.txt file?

It appears the source of the original problem points towards a DNS issue with my host GoDaddy.  I will advise when I know for sure so that others who encounter this problem may be helped.