Welcome to the Q&A Forum

Canonical reference links are the preferred technique for this.

If you do nothing, very likely the search engines will decide for you which variations of your pages to index, and the selection may not be ideal. If an index page can be filtered many different ways, the unfiltered version should be referenced as the canonical on each, and a self-referencing canonical link should also be specified on the unfiltered version.

You don't really yet want to disallow the crawling of the refinement paths, because without canonicals implemented, you might very well do more harm than good, finding important pages getting de-indexed. If at some point in the future you find that all the URLs from the refinement paths have been disappeared from the index, and your desired pages are all indexed properly, then at that future date you might want to disallow crawling of the refinement paths (in your robots.txt file). But, not yet, IMO.

From my experience, there is not an issue with too many redirects in parallel. However, too many redirects in sequence, or a "redirect chain" definitely is problematic. If a page redirects too many times, the search engine might not follow until the very end.

With regard to redirects in general. On occasion, Google has stated that some small amount rank equity is lost with each redirect. So, we do tend to assume that a direct link is preferred over a redirected link. When we restructure our URLs, we generally clean up all the internal linking to point directly at the new links, rather than rely on redirects. With internal links, that's feasible. With external links, while you can reach out and request link updates, you do also need to rely on redirects.

So, regarding NOINDEX vs. DISALLOW, there is a significant difference there.

If you disallow in robots, you are asking the search engine to not even crawl that page. Whereas if you NOINDEX in the page head, then the search engine may still crawl the page but should not index it.

There are a few impacts of this difference. For one, if you use NOINDEX but still allow the search engine to FOLLOW, then it may discover pages which otherwise might not have been discovered (if that page has unique links, for example). So in this case, you might prefer to use (NOINDEX, FOLLOW) if you want that discovery to happen. On the other hand, if you have many pages and you are trying to wisely use the search engine's crawl "budget", then you might in some cases prefer to disallow some paths in the robots.txt file.

It's also common to use robots.txt to disallow some files where you do not have control over the response. Non-html files, where you might not be able to easily administer noindex directives. Or dynamic pages your web application may serve but not allow you to administer head tags for.

All of that said, robots.txt files have been shrinking ever since the search engines began to render javascript, since now they need access to a lot of resource files which they previously did not. Much of the old advice of disallowing scripts and admin folder paths may be obsolete now, if those files are needed to properly render pages.

An interesting dilemma. Often, UGC like forums or reviews is considered as a boost for SEO. But it seems you have concluded that in your case, it is a detriment.

I'm not completely certain of what would be the best strategy in your case, but I suggest you might start with exploring the main reasons why the forum is hurting your rankings (assuming it is). One possibility is that the forum content might contain links to spammy sites. To combat that, you might nofollow those links which are inserted into your user generated content. Perhaps even use the new-ish rel=ugc tag whcih Google introduced (https://webmasters.googleblog.com/2019/09/evolving-nofollow-new-ways-to-identify.html)

Another strategy might be to leave the UGC in place, but to make it not crawlable. For example, reading it into the page using a script from another URL which is disallowed in your robots.txt file. If I were to explore this method, I woudl probably start with some experiements on just a few pages, to measure the impact (out of caution in case the assumption that UGC is the culprit might be incorrect).

Lastly, the forum does not need to be on the same root domain as the website. Again, I'm not sure this would be a positive move or not, but you could register an entirely different domain, and even disallow crawling. Or, it could be another subdomain on your root domain, but again with disallowed crawling. If you want people to be able to find the forum, but are worried about the content being indexed, you could only allow crawling of a few key pages on it whcih are non-UGC.

But mainly, I think your challenge seems to be a very good candidate for running some experiments, rather than relying on opinions from those of us (myself included) who haven't been in your exact scenario before.

On the Redirect Chain Report in Moz (Site Crawl > Redirect Issues > Redirect Chain), click the preview arrow. You should see a set of URLs in the chain. I don't think it specifically tells you the sequence of the chain, but that's easy to find out.

You can do that with Chrome Dev Tools, using the "Preserve Log" function. But for ease of use, I recommend to download the Chrome extension  "Redirect Path" by Ayima. Then request one of the URLs in the chain (pick one). And then use that extension to see what redirects happen. Do that for the others in the chain, and you'll be able to figure out the order which produces the longest chain.

So, then let's say A redirects to B and then to C. You probably want to redirect A to C, and B to C. That way, each URL only has one redirect, not a chain.

That said, might get a whole bunch just because of http to https, and from there onward. If so, you need to figure out where on your site (or off) you have links to your http pages and if possible update them.

I also suggest to double check that the actual title tag (the first one you included in your post) is in the section of the page, and not the . And preferably high up in the section before any scripts or other elements which could take significant time to load. Check this in "view page source" and not just using the "inspect" function of the browser's dev tools, to ensure that some scripts on the page aren't altering the title tag after the document is loaded, since the search engines may instead use whatever is initially set in the actual of the document before scripts execute.

One alternative to the "noindex" directives described below is to instead implement a canonical link tag to an alternative page you would prefer to be indexed for the same content. This would only be relevant if there was another very similar page. And, it would not be as strong of a directive as the "noindex", so I would not use canonical if it's extremely important that it not be indexed, like perhaps some sensitive info.

If you do go the NOINDEX route, then I would suggest, after you verify it has been properly implemented, to use the removal tool in Google Search Console. It technically isn't needed, if you have the NOINDEX directive, but it is usually recognized more quickly, and won't conflict. And then, once the page is out of Google's index, the next step (and only after it is removed and has the noindex tag), is to disallow crawling of that page in your robots.txt file.

This is because Moz hasn't updated their crawling tool to consider hreflang in the equation of reporting "duplicates". They've acknowledged that. They might update it in the future. But for now, you just have to ignore pages being reported  as duplicate if you know that they are properly linked by hreflang to distinguish countries or languages.

Self-referencing canonical tags are a best practice, and will give an important correct signal to the search engines, which is more important than cleaning up reported warnings in the Moz crawl.

So, if both versions of each English pages are identical, and both for the US market, then I think you may need to utilize two different approaches together. First, you can use cross-domain canonicalization on the English pages on the secondary domain to point them to the primary domain (along with a self-referencing canonical on the primary domain). And second, you can use hreflang tags on the English pages on the primary domain and the other languages on the secondary domain, to specify which versions of each page are for which locales (along with x-default and self-referencing hreflang tags).

On the other hand, if the English pages on the secondary domain are for another English language market (non-US), then you could use just a full-blown hreflang solution.

To my understanding, no. Duplicate content generally occurs when two or more pages have very similar content, and the search engine decides that only one of those should be indexed (after crawling each). The concept of "duplicate content" as a "penalty" I think is much misunderstood. There's not actually a penalty, like one would see for a manual action due to spam or hacked content. But rather, if those multiple pages really needed to be separately indexed, then the "penalty" is that only one will be indexed. And, if you don't make smart use of techniques like canonicalization, then the one the search engine chooses to index may not be the one you want indexed. But anyway, none of that really applies in the case you're put forward. As best I can tell from very quickly looking at your site, the pages are unique, and yes, multiple pages might include the same thumbnail(s). But as long as they also include enough unique content besides the thumbnails (and/or don't include exactly the same set of thumbnails as another page), they are likely to be treated as unique pages by the search engine and each indexed. But, you don't need to guess. Just look to see whcih of your site's pages are being indexed. If all the pages you want indexed are being indexed, and none that you don't want indexed, then you don't have a duplicate content issue to worry about.

Typically, a "vanity URL" is an address you use in advertisements which is redirected to your actual page, sometimes along with some tracking parameters in the query strings portion of the URL. This is usually for the purpose of shortening the URL, or making it easy to remember, for use on radio, television, print, billboards, etc. So, in this context, yes a redirect (usually 301) takes care of sending the traffic (and the search engines if any happen to discover one of the vanity URLs) to the real page.

But the phrasing of your question makes me think you are not using "vanity URLs" in this context. If they are being indexed, I assume this mean you are creating actual copies of the pages, or possibly forwarding the page request without changing the URL. If this is the case, then using canonicals (one from the secondary to the primary page, and a self-referencing one on the primary page itself) can be of help to you here. But, I would first question why you are ending up with multiple URLs for one page in the first place.

Yes, you generally should handle your version 1 URLs and your version 2 URLs to redirect to your version 3. That's assuming that there are still some backlinks out there pointing to your version 1 URLs.

But, a best way to do this is to eliminate the hops. So, instead of having a redirect from version 1 to version 2 to version 3, you would update all the oldest ones so they go from version 1 directly to version 3. And also version 2 would go directly to version 3. That way you reduce your "redirect chains".

I think that might be a successful approach under some circumstances. For example, if the company is a brand, and their storefront is only one aspect of that brand but you think that they might otherwise rank for searches of non-transactional intent. An example might be a museum which also runs a gift shop. Or a manufacturer who also manages a direct-to-consumer storefront but where that is not the focus of their business. In these and similar cases, having a separate set of pages (whether on a subdomain or preferably just in a subfolder if feasible) for the commerce isn't necessarily a bad idea. I'm assuming when you wrote "proving to be highly restrictive", you meant more than just for example not being able to set the exact H1 tags you might want on a page or not being able to insert schema markup for certain types of objects. There are going to be those kinds of tactical challenges for on-page SEO in every platform, just varying degrees between the platforms, and I wouldn't take a drastic approach like separating the storefront just because of those kinds of issues. But, if the SEO challenges with the current platform are really of the highest severity and can't be addressed within that platform, then the approach of a separate storefront might make sense in the kinds of scenarios like the museum or the manufacturer mentioned above.

Yes. As I understand it, hreflang is a page-specific directive. Each page can have different hreflang tags, and this is a very common approach. You could also just tag those single language pages with a self-referencing hreflang tag (EN) and x-default. Those two tags could be on all your US pages, and then add additional tags to the pages whcih have siblings in other languages/countries.

I would recommend to canonicalize these to a version of the page without query strings, IF you are not trying to optimize different version of the page for different keyword searches, and/or if the content doesn't change in a way which is significant for purpose of SERP targeting. From what you described, I think those are the case, and so I would canonicalize to a version without the query strings.

An example where you would NOT want to do that would be on an ecommerce site where you have a URL like www.example.com/product-detail.jsp?pid=1234. Here, the query string is highly relevant and each variation should be indexed uniquely for different keywords, assuming the values of "pid" each represent unique products. Another example would be a site of state-by-state info pages like www.example.com/locations?state=WA. Once again, this is an example where the query strings are relevant, and should be part of the canonical.

But, in any case a canonical should still be used, to remove extraneous query strings, even in the cases above. For example, in addition to the "pid" or "state" query strings, you might also find links which add tracking data like "utm_source", etc. And you want to make sure to canonicalize just to the level of the page which you want in the search engine's index.

You wrote that the query strings and page content vary based on years and quarters. If we assume that you aren't trying to target search terms with the year and quarter in them, then I would canonicalize to the URL without those strings (or to a default set). But if you are trying to target searches for different years and quarters in the user's search phrase, then not only would you include those in the canonical URL, but you would also need to vary enough page content (meta data, title, and on-page content) to avoid being flagged as duplicates.

If you are keeping them, rather than redirecting them, I assume that means you have a reason for people to be able to find those pages so that they can get some information abot the discontinued product, or at least understand that it was discontinued. If that's the case, then I don't think you would want to noindex or 404 them. On the other hand, if there is no reason for those pages to still exist, from a visitor standpoint, then usually I would redirect them to a category page (generally the parent category the product belonged to), to preserve any link equity, even if the number of links are low. Especially if you have a lot of discontinued products from a category, even if each product had let's say on average 0.1 links, then if you have 1,000 of those pages you would end up with 10 backlinks to your category page, which could be valuable. Again, this is assuming that you don't want/need to preserve the pages for your users to be able to find the info.

Yes, I meant 301, server-side redirects.

Regarding performance, I currently have a little over 50,000 entries in my redirects file with no discernable performance impact. But, different platforms handle differently, and also we have a CDN which caches redirects too, so that could make a difference. I guess the safest approach would be to insert a hundred thousand or more dummy redirect entries into your redirect file, temporarily, and stress test it.