Welcome to the Q&A Forum

I typically handle this following: URL: "Plan A" - for our overhaul is that a product URL is always going to be www.domain.com/product-name-sku.html/. Neat and clean, and avoids end-user confusion if they navigate to the product through a category that doesn't match the URL. Breadcrumbs: Option 1 - dynamically changes based on how the user navigates to the page.

Thanks Netrepid - that might be a last resort in order to just boost up the internal page, but at this stage I'm trying to get to the bottom of why the homepage is ranking so poorly and why the internal page is ranking better. Once we understand what the issues here are then we can probably put together a better plan to optimise and fix both of the pages.

James I currently use Sitelock which disallows unwanted bots to crawl the site. The MOZ is unable to crawl however we can input a white list of ip addresses comming from MOZ. How are you guys going around this issue? Raul

Makes sense - I understand now. Thanks for the clarification

http://moz.com/learn/seo/canonicalization "Another option for dealing with duplicate content is to utilize the rel=canonical tag. The rel=canonical tag passes the same amount of link juice (ranking power) as a 301 redirect, and often takes much less development time to implement."

Thank you so much everyone - that's great! I will look into all your suggestions I really appreciate it.

Thanks for the insights. Very helpful. What about the robots.txt, though? Should it stay under http://www.example.com, where crawlers can find the file?

You need to confirm that the URLs are in fact 100% of your URLs going into the site map are accessible. if it's a big issue in a big site send me the URL in a private message I will use deep crawl to create a XML sitemap for you. The screaming frog tool is excellent as well though does performance well with extremely large sites. check your robots.txt file this so great tool if in case you have more than one (it happens) http://www.internetmarketingninjas.com/seo-tools/robots-txt-generator/ or http://tools.seochat.com/tools/robots-txt-validator/ so many great free tools are found right here http://tools.seochat.com/tools/ It could be a number of things although it could be Google being finicky. Run the site through Moz crawler, use feedthebot.com using "tools SEO" or download the free version of http://www.screamingfrog.co.uk/seo-spider/  this will tell you if there is an issue. If your site is static you can even create an alternate site map with screaming frog if your site is large use deep crawl or Moz analytics be certain there are no sitemaps redirecting to each other so no redirects going from the old site map to the new site map. Make certain that the site map is in an XML format e.g.  http://example.com/sitemap.xml or if in a different format like https://example.com/sitemap_index.xml make sure the proper format That resolves when you look at the site map is what is going into Webmaster tools. Be certain the site map does not contain over 500 URLs per the site map so example.com/sitemap1.xml and so on keep  numbering them appropriately. sometimes Google is overloaded and does not seem to like to play well with certain site maps or the site map is maybe not generating very well on the server and that is fixed later on. If this is a long-term problem speak to your host or developer. My recommendation is if you've done everything I have talked about that you attempt to submit is the sitemap to to Webmaster tools or  simply build a new sitemap and submit that. so if worse comes to worse take the screaming frog and use this URL to send it to Google http://www.google.com/submityourcontent/business-owner/ I hope that helps, Thomas

From an SEO 'link building' perspective, links behind a login can not be crawled by Google & will not provide any type of traditional/old school link value (since bots cannot access that content). However, as Andy shared, those links may still be very valuable if your business is contextually relevant to the news outlet. Readers of a paid newspaper are likely serious about getting the info shared there, so visitors from the news site may be high converting if your service/product fits their interest. In general, try to think of the value to the human visitor rather than the SEO value... you'll tend have a more clear answer on what to do every time. I hope this helps!

I think thats a great Idea .net is not my thing. All the best! Tom

Thanks for your quick responses. I agree we should differentiate the product pages more

Hey!! We have a coupons and deal website. Coupons are added and removed from the website on a daily basis. But crawler isn't crawling it that often. Lately we started fetching and rendering the page, but that is a time taking task as we have more than 500 stores with coupons. So, I was looking for some API or some method using which the crawler would crawl the website as defined. Suppose "x" store page should be crawled every alternate day as we daily update the coupons there, whereas "y"  store coupons are update fortnightly so , they can be crawled weekly. Can somebody suggest me something..

https://www.youtube.com/watch?v=mQZY7EmjbMA - Here is a video on how duplicate content is handled by Google. Here are my thoughts, it would be better for you because the searcher who types in the specific make and model in your inventory will be able to find it quickly, as opposed to having to click through a site. Theoretically, that is the benefit. It could also be that pages don't rank because all of the content is viewed as duplicate. I would make the decision based on what is best for the searcher, which would be to have an organic SERP with exactly what they are looking for.

Hi Dennis, Thanks however the pages got re cached again today but still half of the links of menu bar are not visible in "Google cache text version".

Hi, I have recently had the same issue with Paypal so I placed it on the GA exclusion list. I'm still seeing 1 or 2 transactions being listed but nothing close to what was showing before. Could someone please tell me why these limited transactions are still coming through? Thank you!

Jack I will send you his email via private message.  Take a look there

I agree with Matt that there can probably be a reduction of pages, but that aside, how much of an issue this is comes down to what pages aren't being indexed. It's hard to advise without the site, are you able to share the domain? If the site has been around for a long time, that seems a low level of indexation. Is this a site where the age of the content matters? For example Craigslist? Craig

Thank you Alan for helpful insight into page . i would definitely add  these points to my priority task list. Thanks,