Welcome to the Q&A Forum

Good question, isn't it? I wish somebody knows exact answer, but I'm not friends with Matt Cutts From what I heard, read etc, my understanding is that Google knows websites, who are selling links, by simply reading the content and links on that website. Example: if website has a page with prices for articles with links or press releases, Google bots can read that content and understand what the page is about. Then they look at how many of outgoing links on website are do-follow, how often they appear, diversity of those links etc. Basically, if 99% of outgoing links are do-follow and they are all to different domains, the picture is pretty clear. If picture is not clear enough, bots would send a request to manual action team.

Without a doubt domain.com/shop. Think about it though from a customer point of view, most languages read from left to right, so with the /shop they subconsciously read your brand name before the shop, however with the shop.domain.com they may just stop when they see shop as that's where they know they need to be.

Hi guys, I thought you'd be interested in a follow up on this. So I submitted my sitemap to Bing - all good, no issues, Bing successfully retrieved it. However after submitting it, my site has effectively disappeared from the SERPs! A number of internal pages are being indexed, but the head/brand queries (where I was ranking incorrectly at #1) are now showing nothing. Very strange. Not sure what else I can do - luckily Bing accounts for a tiny fraction of my traffic! Cheers Karl

No errors with the HTTPS. Checked Ahrefs: very few direct anchors. Most anchors are on their real name from big publications and other websites citing the company... These websites always write topic-relevant content (and include the keyword close to the anchor). I'm desperate with this client

I find it impacts 22% of traffic actually.

From my experience I could see this being a risk, but, it sounds like you are aware of the important factors to consider. I'd want a good portion of unique content on any of those types of pages that is: Different from other places on the web Different form other pages on the site I'd also focus on making sure the content is utilizing correct grammar, punctuation and reads well. I'd think carefully about your plan for indexation and crawling based on how users actually search as well. URLs like this: https://www.noodle.com/colleges/williamstown-ma_location/comparative-literature_subjects Don't seem like they'd be likely targets for searchers (I didn't check volume..just a guess that there isn't much volume there). Meaning, while this page is likely useful for someone once they hit the site, it seems like it's fairly long-tail and might pick up few searches. While that's not necessarily bad, it's easy to get a large chunk of dynamically generated pages out there that: Attract few (if any) organic searches Waste crawl budget Put you at risk for algo adjustments because they can start to have very similar content, show little user activity, etc. While I haven't looked to see how some of the other big players in vertical search that you mentioned are handling this stuff, I do know that clients we've worked with in the past playing in these spaces have found benefit from carefully crafting architecture for users and bots to make sure the site aligns well with how people are actually searching and that that is the content available. I would also wager a guess that Glassdoor and other big players are getting a lot of benefit because of their off-site signals (large link profiles, brand mentions, branded searches, good user signals, etc.). That would help them be seen as legitimate if there was question about their content. Make sure you're providing good value for users that makes you the place to go, make sure you are building your brand signals, and make sure you have a strong plan that aligns your site with how users are actually searching. Hope that helps!

At first I assumed that by manually setting the crawl rate to the maximum, Google would crawl my website faster and more frequently. Our website has tens of thousands of pages so I didn't want Google missing any of it or taking a long time to index new content. We have new products added to the website daily and others that come off or change. I'll let Google decide

Thank you, Dirk. I always enjoy getting explicit answers from people who I have grown to trust. Thanks!

Keep it easy man.  If it is a one position drop, monitor your CTR and whether there are new competitors in your SERPs that have maybe just pushed you down.

My experience with subtopics in URL structure is that they are over-rated.  Use the main category to help the user know where they are in the site and potentially about the topic that the page might be about.  If you want to drop keywords, do it in the category or in the slug for the name of the page.  You can work it in there and it gives you more flexibility.  This also helps with making your page be closer to the root folder vs ending up being too far down in a folder structure. When I have used sub-categories you always end up with content that could be in two different ones and so then you have to decide which one is the better one, etc etc.  You will end up having to rework your URLs later due to issues with your sub-categories.   The only way I have seen subcats work well is when you have something like /state/city/zip or something else where your end item is only in one cat, subcat, etc. I would not do the /peanut-butter/ to redirect to /peanut-butter/subtopic-1 - that makes no sense from an organizational perspective.  If peanut butter is not a strong enough category by itself, it should not be a category to start with.   You need to rethink what your category topics are.  Ideally, /peanut-butter/ is a keyword combo you want to rank for and has great traffic potential that converts.  It should be a hub page for your site for that topic. Find good categories, work in the keyword into that category or if not work it into the slug for the name of the page.  If you want a good example, look at how the Moz site is setup.  Also, remember that keywords in the URL are good for SEO, but you really get more bang for the buck for a good title and content and links into that content.  Dont overthink the URL. Good luck!

Hello Paul , Many thanks for your assistance and comprehensive answer to solve this. You raise some very valid points and something I hadn't picked up on - the fact that currently my internal structure is referencing urls which are not the same as what the canonical tag is set for. Asyou say, this is not a good thing.. Once again, thanks for your help to solve this. My developer is on the case now. thanks Pete

Transparency!!! That is a solid point. SEO's should have nothing to hide. It isn't magic

Yea.  If you cannot do it dynamically, it gets to be a real PIA, and also, depending on how you setup the 301s, you may get an overstuffed .htaccess file that could cause problems. If these pages were so young and did not have any link equity or rank to start with, they are probably not worth 301ing. One tool you may want to consider is URLprofiler http://urlprofiler.com/  You could take all the old URLs and have URL profiler pull in GA data (from when they were live on your site) and then also pull in OSE data from Moz.  You can then filter them and see what pages got traffic and links.  Take those select "top pages" and make sure they 301 to the correct page on the new URL structure and then go from there.  URL profiler has a free 15 day trial that you could use for this project and get done at no charge.  But after using the product, you will see it is pretty handy and may buy anyway. Ideally, if you could have dynamically 301ed the old pages to the new, that would have been the simplest method, but with your situation, I think you are ok.    Google is just trying to help to make sure you did not "mess up" and 404 those old pages on accident.  It wants to give you the benefit of the doubt.  It is crazy sometimes how they keep things in the index. I am monitoring a site that scraped one of my sites.  They shut the entire site down after we threatened legal action.  The site has been down for weeks and showing 404s, but I can still do a site: search and see them in the index.  Meh.

Before you start chasing new keywords, make sure that they are relevant and useful keywords for your customers. Remember that the purpose is always to bring lots of the right traffic to your site, not just lots of traffic. So before you do anything, check that all that great traffic you're getting now is actually useful to you. Is it converting to sales, if that's what you ultimately want? If you haven't got goals set up already in google analytics, go do it now. Assuming that the traffic really is beneficial to you - ie visitors are completing your goals - then analyse where the traffic is coming from, and on what keywords. You might find that traffic from a certain website brings in higher value visitors than organic search, for example - or vice versa. Ok so now you know you're getting loads of traffic, it's converting well, and you know exactly what types of traffic converts best. So you want to replicate that and get even more traffic. Maybe all the traffic and goals are around one product or section of your website? If there is content that no one is finding, that's what you should concentrate on. There's no point in ranking well for keywords that have nothing to do with your website (plus it's really hard to do). So find that area that is getting ignored, and create some great additional content on it. Blog posts, infographics, beautiful photography.... whatever is right for your content. Now go back to your analytics, and see what you can learn from the bits of your website that already do well. Go to websites that are linking to that content, and see if they might want to link to your new content. Getting loads of interest from social media? Seed your new content there too. What you're trying to do is replicate the success you already have. Ultimately what you need is good quality links from good quality websites, pointing to your new content. This will do wonders for your SEO and you'll start getting great organic keyword traffic.

Awesome responses everyone - really appreciate this. It seems I have my work cut out across the various tools to try to identify root cause. At first, quick glance I don't see anything obvious popping up (for example to organic vs. direct) but I have more to check, based on the excellent input from the three of you. Thank you again! Mark

Thanks - I'll give it a try!

That is really helpful! Thank you! Looks like I'm going to have to get cozy with Screaming Frog, but that's cool. The good news is that his social signals and inbound links are really strong, so I would expect any damage to be short-lived.

Google will do this as they see fit in an effort to connect users with the content they are looking for. They will even serve multiple page titles for the same page based on the user search. The factors that determine that are anchor text to the URL and click through rates for specific kws. Essentially Google will treat your SERPs like their own personal Adwords ad, where they will tinker with the ad copy to achieve their desired result.

Interesting.  Thanks for the insight! The reason the sitemap you viewed does not contain links to the new shop products is because that sitemap is generated automatically via the yoast seo plugin.  It can't detect the new product pages since they're on the new WP installation in the /shop/ directory.  However, I do have a new sitemap here, http://startupfashion.com/shop/sitemap_index.xml with the current product URLs and Categories and it has been submitted to GWT.  I initially had those old products and categories removed from the old sitemap, but when I saw that the value wasn't being passed, I decided to put them back as a test because i wondered if perhaps the 301s weren't passing value because Google wasn't seeing those pages anymore.  But I'll definitely take them out again, thanks! As for the double redirect with the forward slash, that's strange.  I'll definitely look into it!   Also... i have no clue whats' up with that category typo.  I don't think I ever touched that, but it's another thing to look into. THANKS!

Hi there, I'd agree with the other comments here in that those particular tactics are not as strong as they used to be and I'd be very surprised if I saw that they were actually improving your search results. They may work for a short period of time but sooner or later, they are going to stop working or worse, they could start hurting you if they cause a penalty. I'd recommend focusing on tactics that are going to work in the long-term and not cause you long-term problems. It's just not worth taking shortcuts. I'd recommend taking a look at the Moz beginner's guide to link building: https://moz.com/beginners-guide-to-link-building I hope that helps! Paddy