Welcome to the Q&A Forum

Thanks that is what i suspected. Very helpful, many thanks.

Yes, example.com/blog/ will typically perform better than blog.example.com and exampleblog.com, and the more decent links that are pointed to the blog and the main site, the bigger the expected impact from combining them. One word of warning - reverse proxies are complex and many teams encounter major developer headaches. Oftentimes it is easier to find a different way to set up content to be published on a subfolder like /blog/. If you want to leave notes on what type of website you are working on (CMS or other software type for both regular site and blog) I am happy to leave notes if it's one I recognize.

You're so welcome, Kathy. We're glad to have you here and please keep your good questions coming as you move along in the learning process.

Hello There, The display you are talking about is referred to as "site links" and you can read more about them here: https://support.google.com/webmasters/answer/47334?hl=en https://moz.com/learn/seo/serp-features Hope this helps!

Thank you! I think the problem is what you mentioned in the previous post. So MOST of them are indexing but I guess really it is a ranking problem. I have been banging my head against a wall and I cannot figure out why this site isn't ranking, driving me nuts!

Yes, I have the same problem.  But certain kind of people do collabs based on DA number and I'm missing 4 very important backlinks that should be seen by Google and that would maybe make my number higher. But why certain backlinks are not seen by the moz crawler?

Hi there, Sam from Moz's Help Team here! So, I took a look at your site, and unfortunately I believe this is a result of our crawler's inability to work with Javascript.  Any client side loaded content will not be rendered to our crawler as we are only seeing what is rendered by your server. Our crawler works by parsing the source code of your site, looking at HTML elements.  If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but there are some good blog posts and discussions in the Q&A about this if you head over to our Help Hub. While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine. You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog. I'm really sorry I can't be of more help here; Please let me know if there's anything else I can help with, or if there are any follow-up questions you might have!

thanks Alex

No problem Matthew, we all have lives away from the internet. I am glad your issue is now resolved. Steve

thanks!  Such a great answer.

Thanks Donna a ton

Thank you Miriam for your response. A couple of follow up questions. What would you recommend I do about the high spam score? Disavow links with a score over a certain %?

Hi Jarod, What you should be asking yourself is whether you need those pages to be indexable and having information available to Google. If you don't want them to rank well, they probably won't need any meta description. Remember that if a page has no meta description, Google will take some of its content and place it as meta description. On a side note, Moz reports will show errors and analyze every page that you have enabled to be indexable.  Also, you can choose to ignore what Moz's report tells you. You might probably know more about your site than Moz. Best luck. Hope I helped. Gaston

Thank you Miriam- thanks for pointing me to those guidelines!

Thanks for confirming Meghan Have a great weekend ! All Best Dan

Sounds like you had the best of intentions by giving a non-JS fallback but that it came back to bite you By the way, this gives evidence to something else that I'm always, always banging on about - Google 'can' render JS and do headless browser renders of a web-page when crawling, but they don't do this for everyone and they don't do it all the time (even for sites large enough to warrant such increased crawl resources). Rendered crawling is like 10x slower than basic source code scraping, and Google's mission is to index the web. Obviously they're not going to take a 10x efficiency hit on their MO for just anyone Sorry about that, needed to get it off my chest as people are always linking articles saying "LOOK! Google can do JS crawling now we don't have to make sure our non-modified source code is solid any more". YES YOU DO - INTERNET Ok done now. Let's focus on the query at hand So you have this lovely page here which you have quoted: https://www.yoursclothing.de/kleider-grosse-groessen It looks like this: https://d.pr/i/QVNfKR.png (screenshot) And you can scroll it down, and it infinitely loads - and you only see the bottom of the results (with no page changing button) when results run out, like this: https://d.pr/i/XECK5Q.png (screenshot) But when JS is disabled (or if you're fast like some kind of ninja cat, and you scroll down to the bottom of the page and find the button before the infinite load modifies the page-contents... but no mainly, just when JS is disabled) - then you get this button here: https://d.pr/i/4Y9T9Y.png (screenshot) ... and when you click the button you end up on another page like this one: https://www.yoursclothing.de/kleider-grosse-groessen?filter=true&view=32&categoryid=3440&page=2 ... where you see "&page=2" at the end there, which is the parameter modifier which changes the active page of contents Google are sometimes choosing the sub-pages of results as canonical when you guys don't want them to do that. You want to know why, what you have done isn't really working and what you could do instead. Got it IMPORTANT Disclaimer: Google decides to rank pages for a number of reasons. If Google really does feel that sometimes, sub-pages of your results are 'better' (maybe they have better products on some of the paginated URLs, a better mix of products or products which fit Google's idea of fair pricing better than the default feed...)  - there is no guarantee that 'correcting' this 'error' will result in the same rankings you have now. I just want to be 100% clear on that point, you might even lose some rankings if Google is really decided. They have told you, they are overriding your choice and usually there's some kind of reason on that. Sometimes it's a 'just past the post' decision where you can correct them and get basically the same rankings on other pages, other times you can lose rankings or they just won't shift it Still with me? Ok let's look at what you did here: On the page 2  (and page 3, and however many paginated URLs there are) you have a canonical tag pointing to the parent And you have blocked the paginated URLs in robots.txt I need to start by querying the fact that you say the page 2s (and assumedly other sub pages, like page 3s - e.g: https://www.yoursclothing.de/kleider-grosse-groessen?filter=true&view=32&categoryid=3440&page=3) - are blocked in robots.txt DeepCrawl's indexation plugin doesn't see them as blocked: https://d.pr/i/1cRShK.png (screenshot) It says about the canonical tag, but it says nothing about the robots.txt at all! So lets look at your robots.txt file: https://www.yoursclothing.de/robots.txt https://d.pr/i/YbyEGl.png (screenshot) Nothing under # BlockSecureAreas handles pagination But then under # NoIndex we have this entry: Disallow: /filter=true That _should _handle it, as pagination never occurs without a filter being applied (at least as far as I can see) Indeed using this tool that I like, if I just paste in only the relevant parts: https://d.pr/i/TVafTL.png (screenshot) **We can see that the block is effective **(so DeepCrawl, your Chrome tool is probably wrong somehow - maybe they will see this new link, read and fix it!) I did notice that there's some weird, unrequired indentation in your robots.txt file. Could that cause problems for Google? Could it, at the least - make Google think "well if there's syntax errors in here, maybe it's not worth obeying as it's probably wrong" - quite possibly In my opinion that's not likely to be part of it So if it's not that, then what!? Well it could be that you're using robots.txt in the wrong capacity. Robots.txt _doesn't _stop Google from indexing web pages or tell them not to index web-pages (which is why it's funny that you have commented with "# NoIndex" - that's not what robots.txt does!) Robots.txt dissuades Google from 'crawling' (but not indexing) a URL. If they can find signals from around the web (maybe backlinks) or if they believe the content on the URL is better via other means, they can (and will) still index a URL without necessarily crawling it. Robots.txt does not do, what Meta no-index does (which can be fired through the HTTP header, or via HTML) Also, riddle me this if you will. If Google isn't allowed to crawl your URLs any more, how will it continue to find your canonical tags and find any new no-index tags? Why give Google a directive (canonical tags) on a URL which Google isn't allowed to crawl, and thus they will never see the directive? Sounds backwards to me My proposed steps: Read, understand and make  your own decision on the "disclaimer" I wrote up earlier in this very post If you still want to go ahead, enact the following (otherwise don't!) Remove the robots.txt block so Google can crawl those URLs, or if that rule covers more than just the paginated URLs - leave it in place but add an exclusion for the paginated URLs so they may be crawled Leave all the canonical tags on, good work. Maybe supplement these with a 'no-index' directive which would tell Google not to index those pages (there is no guarantee the canonical URL will replace the no-indexed URL, but you can try your luck - read the disclaimer) Maybe serve status code 410, only to Googlebot (user-agent) when it visits the paginated URLs specifically - to try and encourage Google to think of those URLs as gone. Leave the contents alone, otherwise it's cloaking. Serve the same content to Google and users, but serve googlebot a 410 (gone) status Before enacting the super-aggressive 410 stance, give Google plenty of time to swallow the new "no-index" tags on paginated URLs which weren't there before. A 410 whilst powerful, may cause these not to be read - so do give Google time (a few weeks IMO) If you do adopt the 410 stance, one down-side will be that Google will think your JS fallback is a broken link and this will appear in Google Search Console. To make this less severe (though it probably still will happen), add no-follow directives to the pagination JS-fallback link / button where it appears Once Google seems to have swallowed your wishes and seems to have removed most of these URLs from their index, THEN put the robots.txt block for paginated URLs back on (so it won't all happen again in the future) Try removing the weird indentation formatting from your robots.txt file Smile Well, that's it from me. Thanks for this one, it was pretty interesting