Welcome to the Q&A Forum

thanks a lot, My team will be working on this and lets hope we do well.

As Thomas said, 50k is the max but you can also have an index file with multiple sitemaps within it so you can have lots more if needed. A couple of things I would add. Manual crawling is fine but remember, that also means that pages that can't be crawled will be excluded from the sitemap which defeats the purpose of the sitemap As for why the indexation levels are so low, the first thing I would check is if everything in the sitemap is a 200 response. Make sure there aren't any redirects or 404s otherwise Google may decide not to trust the sitemap. I hope this helps

Thanks Bernadette! What would you say is the best way to identify toxic links?

Tnks a lot!!

If they do really want to know, they could just copy the text into a standard content box. It'll tell you for that content if it's good/bad etc.

Google search console Index Status report Google Webmaster tools tell you in the https://support.google.com/webmasters/answer/2642366?hl=en if you want to know without a doubt you have to connect Google Analytics to Deepcrawl it will say exactly how many URLs are in the index. https://www.deepcrawl.com https://www.deepcrawl.com/knowledge/best-practice/how-to-measure-indexed-pages-more-accurately/ https://www.deepcrawl.com/knowledge/best-practice/how-can-non-indexable-pages-receive-google-organic-traffic/ http://www.stateofdigital.com/40-deepcrawl-tweaks/ Now that Google Analytics can be added to screaming frog you can do something very similar. https://www.screamingfrog.co.uk/seo-spider/ https://www.screamingfrog.co.uk/seo-spider/user-guide/general/#crawling https://www.screamingfrog.co.uk/seo-spider/faq/ https://moz.com/community/q/how-to-determine-which-pages-are-not-indexed I Hope this helps, Tom

Chris - thanks for the response, the kind words, and for pointing out the footer link issues. We're still cleaning a few things up. Do you know if there are any concerns associated with using a 302 long term, say 3+ years? I'm contemplating using a 302 or just adding a generic paragraph about our fundraising platform to the empty page and leaving it out of our menu drop downs. This way I won't get dinged by Google crawlers for a blank page. Any suggestions are appreciated. Thanks, Kurt

I just took the first domain you gave me I tested them on two tools you lack canonical's on all but the homepage for all three and all three failed the https://varvy.com test imranlaw.ca dietrichlaw.ca canadaenergy.ca burntucker.com  past the Varvy test but has only one canonical https://cl.ly/hPdN https://cl.ly/hPoe bolandhowe.com  is the probably the most affected it has way too many 200 code URLs canonical's pointing to the HTTPS however they should be using a 301 redirect See  search engine land post below & these photos https://cl.ly/hPyM & https://cl.ly/hPUj Preform a search and replace see: https://cl.ly/hPe6 https://searchenginewatch.com/sew/how-to/2291162/seo-audit-findings-4-hidden-technical-problems-that-can-send-dangerous-signals-to-search-engines https://searchenginewatch.com/sew/how-to/2300520/technical-seo-for-nontechnical-people I took the domainIn number three above and ran it through screaming frog I found no canonical's for all but one URL. Take a look at what most of the URLs appear like. https://cl.ly/hPNg In addition found that you have a redirect chain photos below they should go straight to HTTPS://www.canadaenergy.ca I would utilize HSTS as well this will help considerably. And adding canonical's https://cl.ly/hPJd to https://cl.ly/hPr1 to https://cl.ly/hPyj Domain number two dietrichlaw.ca the same situation you have one canonical URL homepage nothing else has a canonical https://cl.ly/hPzk https://cl.ly/hPlk domain number one imranlaw.ca  same situation see below no canonical except for the homepage | Address | http://www.imranlaw.ca/ | | URL Encoded Address | http://www.imranlaw.ca/ | | Status Code | 200 | | Status | OK | | Content | text/html; charset=ISO-8859-1 | | Size | 13160 | | Title 1 | Mississauga Immigration Lawyer & Canadian Citizenship Attorney | | Meta Description 1 | Imran Khan - Canada Immigration lawyer and Canadian Citizenship attorney Mississauga Immigration Lawyer | | H1-1 | Canadian Immigration & Naturalization Lawyer | | H2-1 | Imran Khan Law Office offers Legal Services in Immigration Law and Real Estate Law Matters. | | Meta Robots 1 | index,follow | | Canonical Link Element 1 | http://www.imranlaw.ca/ | | Word Count | 275 | | Level | 1 | | Inlinks | 28 | | Outlinks | 19 | | Address | http://www.imranlaw.ca/contact | | URL Encoded Address | http://www.imranlaw.ca/contact | | Status Code | 200 | | Status | OK | | Content | text/html; charset=ISO-8859-1 | | Size | 14503 | | Title 1 | Mississauga Immigration Lawyer - Contact | | Meta Description 1 | Imran Khan - Canada Immigration lawyer and Canadian Citizenship attorney Contact Imran Khan | | H1-1 | Contact Imran Khan Law Office | | Meta Robots 1 | index,follow | | Word Count | 276 | | Level | 2 | | Inlinks | 28 | | Outlinks | 17 | https://cl.ly/hPsc https://cl.ly/hPdC https://cl.ly/hQ1I A few domains the ones above which are listed below as well fail to be able to be seen by a synthetic Googlebot. Are you running them all on the same server? You have some domains and in .com and others that end in .ca if you are looking in Google.ca and have geo-targeted the .com domains to Canada you should see them there. However if you're looking in Google.com obviously you cannot geo-target .CA domains to the United States therefore they would not show up in .com unless very rarely. Deep crawl and screaming frog are going to be a best friends on this one. Please let me know if I can be of more help here are my findings using a basic tool dietrichlaw.ca and put it into https://varvy.com The results were Findable links Well formed static links not found. [image: AAAAAElFTkSuQmCC]Page has no findable links. Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.' Learn about links and site hierarchy[image: HzV01FDqGCgANXA9tQw7D8UCo4YNZcOgBlLPMKiBAVQzbFABAJmGdLl5mYesAAAAAElFTkSuQmCC] [image: png;base64,iVBORw0KGgoAAAANSUhEUgAAAB4AAAAeCAQAAACROWYpAAAATElEQVR42u3RsQ3AMAzEwIzgEb2RR9BIHolJ95UBg0UqsdbhCz3dfQxPi+0pzKZNf6BMll5lAWVojkrRcEXDFQ1XNDxUcEHzuI92x14P+4pBnwcC8gAAAABJRU5ErkJggg==]HTTP headers Page headers when accessed as Googlebot. Headers: Intro to HTTP[image: HzV01FDqGCgANXA9tQw7D8UCo4YNZcOgBlLPMKiBAVQzbFABAJmGdLl5mYesAAAAAElFTkSuQmCC] pages could not be found https://varvy.com/hierarchyandlinks.html Same thing for imranlaw.ca Findable links Well formed static links not found. [image: AAAAAElFTkSuQmCC]Page has no findable links. Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.' Learn about links and site hierarchy[image: HzV01FDqGCgANXA9tQw7D8UCo4YNZcOgBlLPMKiBAVQzbFABAJmGdLl5mYesAAAAAElFTkSuQmCC] For canadaenergy.ca Findable links Well formed static links not found. [image: AAAAAElFTkSuQmCC]Page has no findable links. Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.' Learn about links and site hierarchy[image: HzV01FDqGCgANXA9tQw7D8UCo4YNZcOgBlLPMKiBAVQzbFABAJmGdLl5mYesAAAAAElFTkSuQmCC] Amount of links Amount of links not excessive. [image: 9hAAAAP0lEQVR42mNgGLLAc0OAAQhTovk9EAdQojlhVDNCgQMQz6fIZpAB6IaQ7GxkQ8j2M9SQ9WQHGNSQfrI1DwgAAK8CSWBkTjaiAAAAAElFTkSuQmCC]0 links found on page. Guideline states: 'Limit the number of links on a page to a reasonable number (a few thousand at most).' Considering the amount of links on a page[image: HzV01FDqGCgANXA9tQw7D8UCo4YNZcOgBlLPMKiBAVQzbFABAJmGdLl5mYesAAAAAElFTkSuQmCC] **I wouldUse a tool like deepcrawl.com  or screamingfrog.co.uk/seospider ** two determined exactly what is wrong with all three Domains which failed a very basic test of being able to be detected by Googlebot. Hope this helps, Tom

I understand the frustration of working with a big company where "because it is new" is a reason not to change the site. That is not a good way to run a business but inertia and people trying to protect their own positions often result in this type of situation. And there generally isn't much you can do to change this, sadly. How terrible is the new site? Does your client just not like it? Or did rankings/search volume/leads/sales plummet when it was introduced? If it is the former, your client should just suck it up and stick with the current site design. It is the customers' preferences that matter and if the site is doing well, that is what counts. If the latter then your client is right to want something different but I wouldn't count much on organic search traffic in that scenario. And I don't know that one working piece will drive company success, though perhaps your client hopes to use that smaller success to drive change for the rest of the site. Good luck with that. Either way it solves your subdomain dilemma--in the first case you won't need one and in the second case it probably won't make a difference.

Hi Andy This is why I have asked the question as the wiki is on its own domain so these aren't internal links. Andrew

Hi, It isn't something I would recommend. The title would keep changing as the stock changed, but as Ikkie said, Google will just ignore this element. Try and think about you you can make the title engaging at the same time as telling Google what the page is about - and remember, you have 70 characters (approx) to play with now. -Andy

Hi Paul, What Bernadette says has a lot of truth. Even, there's been some recently changes in 3xx redirection rules. And a great professional (Cyrus Shepard) wrote a nice piece of text about that in the Moz Blog. Check it out! 301 Redirects Rules Change: What You Need to Know for SEO In my opinion, historically 301 are better than 302, if you can set 301, do it. Best Luck. GR.

The pages are loading now not sure what was causing that but Id keep an eye on it. Feel free to shoot me a message if you have any other questions. I don't mind helping out if I can.

Hi there. There is the rule: do not show google duplicate content. So, having both, with and without the trailing slashes is an issue.  If you dont want or think is not necessary a redirect... you can set a canonical.  As you said, moz doesnt have a redirect, instead has a canonical. Check it out: view-source:https://moz.com/community/q/ Hope it helps. GR.

Thank you. I want to disallow specific URLs on the subdomain and add the shop sitemap in the robots.txt file. So I'll go ahead and create another!

Thanks, both. We'll explore a better solution with Demandware.

As Logan pointed out, the most likely cause of this is because of the canonical tags pointing to the HTTP variation. I'd probably suggest carrying out the following: Download a tool like Screaming Frog SEO Spider and crawl your full website - https://www.screamingfrog.co.uk/seo-spider - Once the crawl has run, export the data and check both the URLs and the Canonicals across all pages to ensure only HTTPS variants appear. If you find http variants, then it's likely that the redirect rules you have in place aren't working. This is a global rewrite rule that can be applied to force the https:// version to show at all times, so you shouldn't need to make the changes once by one. I'd also make sure that if you're linking to any pages internally, that you're linking to them using a relative path, or straight to the https:// version. If your canonicals are causing issues, just ensure they're always choosing the HTTPS version and you shouldn't have any issues! Finally, in GSC, just make sure you've got the HTTPS version of the site setup and that your sitemap includes only HTTPS URLs. Hope this helps!

You would not set the canonical URL on the product page to your preferred category, that would send the signal that instead of having the product URL rank in search, you would prefer one of the product's categories, which would be incorrect. It sounds like since you've already set a canonical URL for certain categories, so you're on the right track. For instance if you have 5 categories with very similar lists of products on those pages, it sounds like you are canonicalizing 4 of those pages to the 1 most authoritative of the 5. In summary, you would only set a canonical for a product page if you had the same exact product spread across 5 different product pages. For example, if you had /mens-glasses/versace-primo/ and then for some reason every color had its own page because of your system, such as /mens-glasses/versace-primo/black, /mens-glasses/versace-primo/blue, /mens-glasses/versace-primo/red - then you would need the canonical on the product page back to the main page without the color subfolder. To address a part of your question, it's generally ok to assign the same product to multiple categories if that's useful to the user. Unless you have a massive site and you're maxing out your crawl budget, I don't think you need to sweat too much on duplicate content issues, unless you're receiving really strong negative signals. For instance, Moz might show that some of your pages are duplicate, but it's more informational, and you don't always need to take sweeping action. One strategy to fix duplicate content issues on category pages, is to write unique content for each category page and make the category page an actual destination page rather than just a navigational page.

Hi Chris, Thanks for your response and suggestions. I think I've found a solution to all of the issues I'm currently experiencing! https://www.perfectseourl.com/ This should fix the -2 problem, as I can now dispense with the "product-category" slug completely and also follow your suggestion of going with the "womens" category name etc. Many thanks!

Thanks ! I turned of Geolocate (with page caching support), and as you said, it corrected the problem. Thanks again. Bob