Welcome to the Q&A Forum

Perfect! Happy to help!

Hey! If you have access to the code, you can add this to the meta data: You can also request to have it removed via the Google Search Console: https://www.sistrix.com/ask-sistrix/google-index-google-bot-crawler/how-can-i-remove-a-url-on-my-website-from-the-google-index/ If you have a CMS like Wordpress, there are free plugins that you can find that will no-index your pages fairly easily. I hope that helps!

Hey Tommy! Are you able to post a screen shot of what you are seeing in the search console. Is your web host reliable to where your site is going down periodically? Just curious. I look forward to seeing the screen shot.

Sure! Happy to help!

I wanted to reinforce with Martijn and Gaston had said. I would just have your web person 301 redirect the old URLs to the new URLs. I would add to keep an eye out for any stray URL's that you may have missed in the Search console and redirect those too. They tend to pop up 1-2 weeks after a site move. Thanks!

Hi Christian, I'm sorry to hear about the issue. Here's what I would do: 1. Try to use a free xml sitemap generator (Google free xml sitemap generator) and manually upload it to your server and then submit that. That may involve turning off the Yoast xml sitemap generator. You can see if you still get the error 2. Try a different plugin to generate the sitemap - I'd recommend this one: https://wordpress.org/plugins/google-sitemap-generator/ If you are still getting the same error, I would take Everett's advice and call your web host.

All of the plugins I can find allow the tag to be deployed on pages, posts etc. You pick from a pre-defined list of existing content, instead of just whacking in a URL and having it inserted (annoying!) If you put an index.php at that location (the location of the 404), you could put whatever you wanted in it. Might work (maybe test with one). Would resolve a 200 so you'd then need to force a 410 over the top. Not very scalable though...

Hi Rajesh, Based on what you've described, it sounds like a desktop PWA could be a good way to go! But I would avoid hiding any content from Google crawlers. Instead, check out these articles for some guidelines on how to optimize a PWA site for SEO: https://moz.com/blog/introducing-progressive-web-apps https://searchengineland.com/google-progressive-web-apps-mobile-experience-seo-259866 https://www.keylimetoolbox.com/news/technical-seo-progressive-web-apps-pwa-javascript-ajax/ Also, be aware that PWAs are not supported by all browsers, so before you go forward with that approach and ditch a regular website, make sure you look into what browsers your customers are using and ensure that all browsers which are used by a significant percentage of your audience have a supported experience. Hope this helps!

Great, thanks for your note Paul, I will filter through as you suggest!

Knowing that with a large body of documentation like this, the chances of being able to rewrite it all to combine into a single page are pretty slim (and knowing that might be a very negative user experience) you're really only left with the canonical tag option - assuming the older docs need to be maintained. You're right to be concerned, as Google has been clear that canonical only applies to pages that have substantially identical content. Unfortunately, they do a really poor job of explaining just how much variation would be allowed. Is it okay if the canonical is not an exact duplicate of the content? We allow slight differences, e.g., in the sort order of a table of products. We also recognize that we may crawl the canonical and the duplicate pages at different points in time, so we may occasionally see different versions of your content. All of that is okay with us. ~ https://webmasters.googleblog.com/2009/02/specify-your-canonical.html My impression is that they would honour canonical in your use case. Really, the only way to know is to select a couple of products' documentation pages and conduct a test. Canonicalise all old version to the current version and request re-indexing for each page. Then monitor the results (The new index monitoring tools in the new GSC are useful for this). You'll want to choose at least one test case that involves featured snippets - it would be incredibly useful to know if the FS transfers across to the new canonical page! Do note that you'll need an ongoing process for managing the canonicals s each new iteration of documentation is added - all related pages will need to have their canonicals updated to point to the newest each time new docs are published. Interesting conundrum. Please let us know the results if you decide to try a test! Is that useful? Paul

Awesome!

no at all

The correct way to use UTM tracking without hurting your SEO efforts is to make sure you’ve implemented your canonical tags correctly. You should add self-referring canonical tags, which will prevent multiple versions of the same page from being indexed. For example: http://www.yoursite.com/some-page?utm_source=facebook&utm_medium=social should have a canonical tag that looks like this: If you have pages with these parameters on your site then you should use the rel=”canonical” tag to specify the canonical URL that you’d like Google to rank. I hope this information answers your question. if you consider that my answer is good enough don't  forget to mark it as a Good Answer Regards and have a great day

WPEis run on a base server using Apache with a Nginx proxy so you can use the WP engine  301 redirect system built-in or you can simply add a redirect to the HTAccess file. If you would like to use a tool to do this I recommend this one another alternative is to ask WP engine to make a change for you.https://www.aleydasolis.com/htaccess-redirects-generator/non-slash-vs-slash-urls/ApacheJust copy to your htaccess:``` https://example.com/page/ **https://example.com/page** ```  <label for="nonslash">**Slash to Non-Slash URLs**</label> > <ifmodule mod_rewrite.c="">RewriteEngine on > RewriteCond %{REQUEST_FILENAME} !-d > RewriteRule ^(.*)/$ /$1 [L,R=301]</ifmodule> **Non-Slash to Slash URLs** ``` ****Apache** https://example.com/page** https://example.com/page/ > <ifmodule mod_rewrite.c="">RewriteEngine on > RewriteCond %{REQUEST_FILENAME} !-f > RewriteRule ^(.*[^/])$ /$1/ [L,R=301]</ifmodule> USEING Nginx to do **https://example.com/page/** https://example.com/page As you see, there is one tiny difference between those two URLs, and it’s the trailing slash at the end. In order to avoid duplicate content, if you are using Nginx you can **remove the trailing slash from Nginx** URLs. Place this inside your virtual host file in the server {} block configuration: > ``` > rewrite ^/(.*)/$ /$1 permanent; > ``` Full example: > ``` > server { > listen 80; > server_name www.mysite.com; > rewrite ^/(.*)/$ /$1 permanent; > } > ``` All done, now Nginx will remove all those trailing slashes. USEING Nginx to do https://example.com/page https://example.com/page/ Add a trailing slash by placing this inside your virtual host file in the server {} block configuration: > ``` > rewrite ^(.*[^/])$ $1/ permanent; > ``` Full example: > ``` > server { > listen 80; > server_name www.mysite.com; > rewrite ^(.*[^/])$ $1/ permanent; > } > ``` From now on, Nginx should add your trailing slash automatically to every url * https://www.scalescale.com/tips/nginx/add-trailing-slash-nginx/ * https://www.scalescale.com/tips/nginx/nginx-remove-trailing-slash/  I hope this helps, Tom

No index & test Indexing Before You Launch The domains are intended for development use and cannot be used for production. A custom or CMS-standard will only work robots.txt on Live environments with a custom domain. Adding sub-domains (i.e., dev.example.com , ``test.example.com) for DEV or TEST will remove the header only, X-Robots-Tag: noindex but still, serve the domain. robots.txt  To support pre-launch SEO testing, we allow the following bots access to platform domains: Site Auditor by Raven SEMrush RogerBot by Moz Dotbot by Moz If you’re testing links or SEO with other tools, you may request the addition of the tool to our robots.txt Pantheon's documentation on robots.txt: http://pantheon.io/docs/articles/sites/code/bots-and-indexing/User-agent: * Disallow: / User-agent: RavenCrawler User-agent: rogerbot User-agent: dotbot User-agent: SemrushBot User-agent: SemrushBot-SA Allow: /

Awesome! I know how clients can be. Good thing they had someone like you! Have a good one

Thank you for the clarification Rajesh, much appreciated