Welcome to the Q&A Forum

You bet.  Just to be clear, I was talking about pulling the content from the page in some automated fashion into the title.  Finding elements from each page that should be in variables and then inserting them into the title, description and H1 in a way that you can make each page unique.

You would need to make a final call on if you think the content is unique enough.  We had 5000 locations that we used data in the database to make 5000 unique pages as each location has the name of the business, city, state, zip, address phone number etc.

We are now working to have user generated content/comments/reviews on each page so that each one page becomes more unique and more useful over time.

Having a IT guy who appreciates SEO is key for this and for the URLs.  I would talk first about how he organizes the data in his system and then how this translates into the URL.  You can then then just have him rename the URL using the same logic.

Show him some data on click thru rates on more readable URLs and how Google prefers not to spider them.  I work to educate the IT guys as much as I can without making it sound like I know it all.

Wow!  Yep you gotta lock this one down.  This needs heavy automation support.  Have you looked at using the content from each build can be pulled into the title tag etc automatically?  That can help diversify how the tags look.  If the users love all the builds, then is there a way to use automation to help Google see this.

If you work it right, you can have an awesome opportunity for long tail search. I worked on a site that had a yellow pages type setup.  We had all the pages with title tags and descriptions that pulled in city state zip location address and name of location automatically.  Even changed up the order on how it was presented and had different options for filler / connective words.

Worked pretty well to show off the unique content on each page as best we could automatically.  We then paid an intern to go in and optimize page by page from there starting with the most viewed pages.

It may not be that each page is not strong enough as you mention, but you also said that users love the pages so I wanted to toss that out there.

Agreed - you already have the key term in the URL all you are doing is adding to the length of the URL and that can make it look spammy to users.

You will have other parameters added to the end of that URL and so very quickly you would be getting close to the 70 character max that SEOMoz suggests.

Hey there Kristy O,

You want to noindex follow meta tag your paginated pages beyond page 1 as you want Page 1 of a series of pages to rank for the keywords that you are optimizing for.  By preventing pages 2 though X on a given series from being indexed you "clear up" for Google what pages you want it to focus on.

You have Google follow the links on all of those pages as they all point to product pages that you do want to have indexed and ranked.  There is also the rel next and prev attributes, but I like the noindex follow meta tag approach as it gives me more control.  I

If you want to use the canonical tag in this situation.  It was recommended that you use the Canonical to link back to a master page that contains everything on page 1,2,3,4 X.   They want you to use canonical to have the parts point back to a whole.  On most larger sites this is not practical as to load a page with everything on it would take too long.

See here for when to use view all page vs not

http://googlewebmastercentral.blogspot.com/2011/09/view-all-in-search-results.html

I think that the use of the canonical in this case was more for a 3 part article vs pages after pages of a product catalog.

I really only use the canonical to help with things like having the printer friendly version point back to the main version etc.

If you want the Guru on pagination - Google Adam Audette

http://searchengineland.com/five-step-strategy-for-solving-seo-pagination-problems-95494

Good luck!

One of our sites it took about 6 months before things got back to "normal" in the SERPs.   As far as how long to leave the 301s.  I would say indefinitely.    There are links from other sites with old URLs that we still get traffic from and want to make sure we get that referral traffic.

Having the old 301s in place I do not think hurts anything as over time they will be used less and less.  You could also look at your server logs and determine that some 301s are not used anymore and then turn them off.

It just always surprises me when I see Google looking for old pages that we have 301ed for over a year and so I leave them in place.

Good luck.

I would agree with all the comments on how to technically deal with the random pages, but it is a losing battle until you get your website database/templates under control.  I once had a similar issue and had to work months to get a solution in place as the website would create all kinds of issues like this.

We had to implement a system so that the creation of these pages would be minimized.  I think the issue is that you need to make sure that any random page requests, make sure they get a 404 to start with so that the URL does not get indexed to start with.

That said, all the random URLs that are already indexed, I like the 200 options with the noindex meta tag.  My reasons: This is because otherwise with the 404s you get all these error messages that are meaningless in GWT.  The noindex also gets the page out of the index.  I have seen Google retry 404s on one of our sites, crazy.   Ever since Google started showing soft 404s for 301s that redirect many pages to a single URL, I only try to use 301s on more of a one to one basis.

Good luck.

Wouldn't you want to noindex one of the agreements vs nofollow that way only one page gets indexed.

The other thing is you could canonical link the copy page back to the original.

Yes this is a big problem.  You are showing Google, 2 pages with the same content.  This would be the case if this were any two pages with the same content on your site.

You definitely need to setup a 301 redirect (and not a 302) the slug to the main page.

You have to be careful with CMS systems. They may be generating a ton of duplicate pages and you are not even aware of it.  Check with you IT person, but then look in Google Webmaster Tools and also use other tools to check (e.g spidering tools).

You want each URL on your site to have a unique title tag, description and page content on your site.

Good luck!

Check that your canonical is indeed linking to the right page.

You can go into Google Webmaster Tools and specify that they need to ignore that parameter and that it is the same page.

Look under Configuration > URL Parameters

More info here

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1235687

I would agree with BK Search.  You want to minimize what Google has to crawl (I know this sounds backwards) so that Google focuses on the pages that you want to rank.

Long term, why would you waste GoogleBot's time on pages that don't matter as much?  What if you had an update on a more important page and GoogleBot is too busy indexing this infinite loop of pages.

At this point, I would use the noindex meta tag vs robots.txt so that google will crawl and remove all the urls from the index.  Then you can drop it in later into robots.txt so it will stop crawling.  Otherwise you may end up with a lot of junk in the index.

I think you answered your own question.  You have to compare search traffic potential for "dog training twin cities" vs "training german shepard".   If you see enough traffic on the german shepard stuff, create a new page and link to it.  Maybe focus on the top 5 types of dogs and have that as a section.

You are talking about stealing from Peter to pay Paul, but it looks like you are being smart and using analytics to try and answer your question.  There is your science.  You have to look at the long tail of search terms from analytics, and you should see some patterns there.  That will guide you on where to build content over time.

There are some cool regular expressions you can use in Google Analytics to built keyword reports around 2 and 3 word queries.  One example:

http://secretswede.net/seo/measure-longtail-traffic-google-analytics-mayday-update/

Take this and setup a segmentation to see as a whole if this type of traffic is really driving conversions.  It may be that you get traffic, but less conversions.  This adds another layer to how you look at this and if it is worth spending time building out content.

Good luck!

Me too.  It was that video that helped to clear things up for me.  Then I could see when to use robots.txt vs the noindex meta tag.  It has made a big difference in how I manage sites that have large amounts of content that can be sorted in a huge number of ways.

Take a look at

http://www.youtube.com/watch?v=KBdEwpRQRD0

to see what I am talking about.

Robots.txt does prevent crawling according to Matt Cutts.

You can use the analytics to tell where the traffic drops are coming from but I think that you need to get to why Google or some other search engine traffic is dropping.  So I totally agree with what EGOL mentions, but I think you already have a global grasp that traffic has dropped.  Here are what I would suggest are next steps to then fix the issues of why the traffic has dropped.

Odds are is that technically you may have a problem going on.

Go through your GWT reports for crawl errors and HTML optimization etc.  We relaunched and it gave us all kinds of clues on what to fix.

You need to double check all of your 301 paths - I bet that there are some holes.  This is down and dirty detail work.   I bet that they are probably not as correct as you think.  Don't assume that if you test a few the rest are ok.  Dont assume that the developer says that they are working that they are working.  Run them through a tool to make sure that you are sending a 301 response.  I had a site that was using 302 responses vs 301 (a temporary vs a permanent redirect).  Verify, verify, verify.

Did you change title tags on your pages.  Titles are a big signal and if you change them, you can see a drop.  You can look at your traffic and ranking analysis to look at sample pages.  If you did do a radical change, you may want to go back to your original title tag methods.

Run site speed tools

https://developers.google.com/speed/pagespeed/insights#url=www.brickhousesecurity.com&mobile=false

See if your speed has dropped.  The GWT will also show over time if your site is speeding up or slowing down.  You may need to look at your server setup.

Verify the HTML

http://validator.w3.org/

Did you setup

It looks like you may have changed a bunch of things at once and so it is hard to tell what you changed so see what was impacted.  Usually in cases like this, Google is trying to figure out the changes and so it may take a while to sort out.  I just gave some examples, but you need to review everything that has changed and see what the differences are.   There are some things, even with a 301 redirects that are correct, if you changed the URL structure and title tags and the new site has HTML that does not validate, Google may take a while to sort it out.  I had a site that we did a complete overhaul and it was 6 months before the traffic came back and that was with pretty good controls in place.

Good luck!

Just a couple of under the hood things to check.

1. Are you sure your robots.txt is setup correctly. Check in GWT to see that Google is reading it.

2. This may be a timing issue.  Errors take 30-60 days to drop out (as what I have seen) so did they show soft 404 and then you added them to robots.txt?

If that was the case, this may be a sequence issue.  If Google finds a soft 404 (or some other error) then it comes back to spider and is not able to crawl the page due to robots.txt - it does not know what the current status of the page is so it may just leave the last status that it  found.

3. I tend to see soft 404 for pages that you have a 301 redirect on where you have a many to one association.  In other words, you have a bunch of pages that are 301ing to a single page.  You may want to consider changing where some of the 301s redirect so that they going to a specific page vs an index page.

4. If you have a page in robots.txt - you do not want them in Google, here is what I would do.   Show a 200 on that page but then put in the meta tags a noindex nofollow.

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93710

"When we see the noindex meta tag on a page, Google will completely drop the page from our search results, even if other pages link to it"

Let Google spider it so that it can see the 200 code - you get rid of the soft 404 errors.  Then toss in the noindex nofollow meta tags to have the page removed from the Google index.  It sounds backwards that you have to let Google spider to get it to remove stuff, but it works it you walk through the logic.

Good luck!

Take the page.  Select View Source

Search for<a href<="" p=""></a>

<a href<="" p="">I get about that many links.</a>

<a href<="" p="">You have the nav on the right side (weddings by color, by season, by theme) , you have 2 links each to the different weddings (the title and the read more), social media links, etc

I think what may confuse you is that your theme does not underline links, so maybe you are not seeing them with your eyes.  View Source and look at the HTML - they are all there. :-)</a>

I think Screaming Frog has a trial version, I forget if it limits total number of pages etc. as we bought it a while ago.   At least you can try out and see.  May be others who have more tools as well.

I would say it is preferable to go ahead and setup the site on the new host first before you take down the old one.  While you should be "ok" with the downtime, I would not recommend it.  You never know when the spiders come along.  You would probably not be de-indexed, but Google would see a bunch of errors and you could potentially see a drop in the SERPS and then traffic to your site as a result.  This should all recover.

I have seen on our sites drops in traffic when we have had technical difficulties.  I usually see issues in GWT or other tools and I get them fixed and the traffic comes back.

Here is the other thing.  What if something goes wrong during that 12 hours?  What if  12 hours becomes 24, 48 etc. due to unforseen issues.  That is just bad for business/users in general when a site is down for any amount of time.  You do not want that, let alone the search engine issue. What if something goes wrong with the new host and you need to revert back to the old?  This has happened to me and trust me, you do not want this to happen to you.  Murphy likes to play games with scenarios like this, I do not mess with Mr. Murphy and his laws.

If I were you here is what I would do

1. setup the new host

2. setup your site on the new host

3. test test test on the new host

4. change the DNS from the old to the new host

5. watch the traffic move

6. test test test on the new host

7. shut down the old host

We have overlapped for up to 2 months with old and new hosts just to make sure everything is set.  You always back up your data yes?  Why would you not want to have a backup with your entire website?

Good luck!