Welcome to the Q&A Forum

User-agent: * Disallow: /cgi-bin/ Disallow: /wp-admin/ Disallow: /archives/ Disallow: /? Allow: /comments/feed/ Disallow: /refer/ Disallow: /index.php Disallow: /wp-content/plugins/ Allow: /wp-admin/admin-ajax.php User-agent: Mediapartners-Google* Allow: / User-agent: Googlebot-Image Allow: /wp-content/uploads/ User-agent: Adsbot-Google Allow: / User-agent: Googlebot-Mobile Allow: / Sitemap: https://site.com/sitemap_index.xml use this it will help you and your problem will solve Regards Chotapao

Hi Rogier, Yes, this is usually counting towards crawl budgets as Googlebot is doing this per request. It depends on how your request is being set up obviously, otherwise, I would advise going with the exclusion for the robots.txt that you're already heading towards. Hope this helps!

It works on my three sites Blogger SEO Harbor 347 Bukase Mangatbaru

Sorry Lisa, this got lost in my email box. You are looking for tips for how to set things up for an international site? Can you tell me more about your site and what you are looking to get out of your Moz account when it comes to international search?

Hi Choice This will clear it up for you: 1. You don't need index follow on any of these pages as that is the default setting anyway. The only reason I would use a robots tag is if I wanted to noindex a page. 2. Sorted Page and Pagination of sorted pages - remove the index/follow and replace with a self-referencing canonical tag to the main category rel="canonical" href= https://www.site.com/category/ You do not want sorted pages and pagination of sorted pages appearing in Google. You just want them pointing back to the main category. That will tell Google to ignore the sorted URL and index the core URL. 3. Paginated page For pagination, you need to add rel=prev and rel=next (You don't need a canonical) - this is just for category pagination. Still relevant for pagination: https://moz.com/blog/pagination-best-practices-for-seo-user-experience Just don't get confused between sort pages (low grade) and pagination (needed for Google to crawl all the content & links) and don't let Google index any of the sorted pages. Regards Nigel

Lil help here..

Hi Curtis, By the sounds of things I think you might well have over-engineered your responsive website! You clearly understand the principles of responsive design and are a better developer than me, but unfortunately it seems the use of media queries here isn't optimal compared to a fluid grid system: https://www.w3schools.com/w3css/w3css_grid.asp  Why don't you make your question redundant by trying to incorporate a single div? Plus then you might be able to remove some jQuery which could speed up your site. To specifically answer your question about duplicate content...the only way of knowing would be to test rankings before and after. Generally it's believed that Googlebot would be smart enough to realise why there's a small amount of "hidden" content, but personally as an SEO (someone trying to optimise as efficiently as possible), I wouldn't want to risk it! How Google Defines Cloaking - https://support.google.com/webmasters/answer/66355?hl=en  A Google product forums discussion around display:none and mobile design; - https://productforums.google.com/forum/#!msg/webmasters/zB0Gqjajfis/E6tgJs7_AwAJ  Best of luck, Nick

My pleasure, Joe!

There were several that had good traffic and sales pre-migration.

Hi Davit, Thanks for the links! I agree with you that it seems a bit weird seeing this business, with no GMB listing, doing so well organically. I looked at a few things I normally look at, and couldn't see a "easy" explanation. So, I asked one of our organic SEO experts, Britney Muller, if she could take a glance at this, and she suggested a couple of things: Your competitor has been around since 2006, so pretty established Your competitor has invested in links, which appear to be paying off "LAX car service" is one of the phrases they are doing best on, though they are not doing so well on other terms They are investing heavily in PPC, which could be producing what she described as some sort of "halo" effect Taking all of that into consideration, I do consider this competitor of yours as having earned rankings for this term that don't make immediate sense. Give this, I believe if you could hire a good company to do a full competitive audit for you, you should be able to map out a strategy to surpass them, but it may take some doing. Hope you'll receive other feedback.

Thanks Dr. Pete

Hi, thanks for your question. You would do this in Pardot. Accordingly, I recommend reaching out to their support team for details on how to do this. You may want to ask them how you can edit the landing page template(s)  you are using so that title tags are dynamically inserted (into your landing pages). Hope that helps! Christy

Yep when/if Google shut down search operators there are a lot of processes that will be FUBAR! 100% agree a full ecommerce related Technical Audit is a must as a starting point. WQe recently saw a spider trap issue that was preventing a high percentage of  a clients product pages being crawled and indexed. Resolving was a major 'quick win'. Thanks for weighing in Everett. Appreciate it!

Hi Kingsplan The problem here is that you would still have them pointing at your domain. Even if they 404'd, the links would still be hitting your site. I would: 1. Ascertain if they are actually toxic - look at SEMrush or similar (anything over 65 toxicity is worth removing) - we have had a lot from 'the globe' recently. I have disavowed them for some sites but honestly, they don't appear to have done any harm to others. 2. Email the 4 or 5 domains and ask them to remove the links. 3. If you don't get a response after 2 weeks or so simply add those domains to the disavow file. Regards Nigel

Hi Lauren: Do on page factors influence Domain Authority as much as building back links? This is a really key question! I need to decide whether to invest resources in building back links or improving on site SEO. How about internal links? How much do they contribute to domain authority? At the moment my site lacks internal links, would adding them in a relevant manner improve domain authority? While I greatly appreciate your last response it is somewhat ambiguous. Thanks, Alan

That's what I would do Joe!

This all depends on what your business wants to sell, to whom & where. As for keyword research, I'd recommend using: Keywords Everywhere SEM Rush Google KW planner/ Moz Ahrefs AnswerThePublic

I have never been worried that Google prefers to choose a copy from the page instead of the meta description. On the contrary, I'm happy with it. It usually makes it longer (up to 300 characters in some cases). And this makes the visible result seem a lot bigger in the rankings. As mentioned above that could really help with CTR.