Welcome to the Q&A Forum

Not 100% sure if Google even reads  AutoRental schema on web-pages, though there is some evidence to suggest that Google sees valid usage of AutoRental in emails

If you go here:

https://developers.google.com/search/docs/data-types/product

On the left-hand sidebar, you can see a list of all the different schemas which Google documents that they support. AutoRental isn't present there. A Google search helps to confirm this. But they do list "LocalBusiness" schema, of which "AutomotiveBusiness" and "AutoRental" are valid sub types, so I assume that using AutoRental would be ok and acceptable by Google

It does seem that this site: https://search.google.com/structured-data/testing-tool#url=https%3A%2F%2Fwww.kayak.co.uk%2FCheap-Leicester-Car-Hire.6700.cars.ksp (Structured Data results for a car rental site) is indeed using product schema to list all the vehicles on offer, so I think it could be a good supplementary schema to go alongside AutoRental

These guys: https://search.google.com/structured-data/testing-tool#url=https%3A%2F%2Fwww.enterprise.com%2Fen%2Fcar-rental%2Flocations%2Fus%2Fny%2Fnew-york.html - are using AutoRental, and Google's structured data tool does indeed pick it up

Check more of your competitors using Google's Structured Data testing tool, if enough of them are using product schema on the vehicular product listings then I'd see no good reason to omit it

Also: "50%"

I personally don't think that Google handles this data exceptionally well:

https://d.pr/i/2Y562I.png (Keyword Revealer screenshot)

https://d.pr/i/El2skX.png (Ahrefs screenshot)

https://d.pr/i/Y3bQ3p.png (Google keyword planner screenshot)

... however, I do sometimes see such keywords returned from Google Search Console and / or Google Analytics under GSC's "Search Queries" (search terms) report. So it makes me wonder, if Google really has such trouble, why does it highlight and record such keywords, passing them to me for further analysis?

Maybe it's actually not a big deal, it's just that Google's keyword planner (in terms of full unicode support) is way, WAY out of date (something they should have patched and fixed 5-6 years ago IMO)

Regardless of this though, more people do seem to search by 'half' or '50%', people 'almost' never type "½" as it's so hard to type in a web browser, you almost always have to copy and paste the symbol unless you have some kind of rich-text field entry add-in / extension

Google can process the symbol as search entry text:

https://www.google.com/search?q=%C2%BD

Google often states that actually, using unicode characters (even in URLs, in UTF-8) is ok in modern times. This is a compromise they have had to make, as many foreign characters are packaged  in various unicode character sets

This is the full list of UTF-8 symbols:

http://www.fileformat.info/info/charset/UTF-8/list.htm

If you Ctrl+F for '½', it is technically in that list. As early as 2008 Google was recorded indexing UTF-8 URLs:

https://www.seroundtable.com/archives/018137.html

Much more recently, the debate has been raised again:

https://searchengineland.com/google-using-non-english-urls-non-english-websites-fine-294758

"For domain names and top-level domains non-Latin characters are represented with Unicode encoding. This can look a little bit weird at first. For example, if you take Mueller, my last name, with the dots on the U, that would be represented slightly differently as a domain name. For browsers and for Google search, both versions of the domain name are equivalent; we treat them as one and the same. The rest of the URL can use unicode utf-8 encoding for non-Latin characters. You can use either the escape version or the unicode version within your website; they’re also equivalent to Google."

Obviously Google is talking about URLs here, but usually Google becomes capable of reading characters in markup (content, Page Titles etc) first and then accepts them for valid URL usage later. I would surmise that it probably is 'ok' to use them, but it probably would not be 'optimal' or 'the best idea'

If you have a general site which happens to have a search facility, blocking search results is quite usual. If your site is all 'about' searching (e.g: Compare The Market, stuff like that) then the value-add of your site is how it helps people to find things. In THAT type of situation, you absolutely do NOT want to block all your search URLs

Also, don't rule out seasonality. Traffic naturally goes up and down, especially at this time of year when everyone is on holiday. How many people spend their holidays buying stuff or doing business stuff online? They're all at the beach - mate!

It will be strongly de-valued and the links may or may not even be noticed / seen at all. Googlebot can leverage headless browsers (something similar to Selenium or Windmill in Python, with targeting handled via XPath maybe). The only thing is, this takes ages longer than basic source-code scraping. To scrape the modified source with a headless browser can take, 5-10 seconds instead of less than 1 second

Since Google's mission is the 'index the web', you have to fathom that they wouldn't take this colossal efficiency hit all the time, or for everyone. Certainly looking at the results of many sites and their different builds, that's exactly what I see. Just because 'Google can' that doesn't mean that 'Google will' on all crawls and websites

Some very large websites rely on such technologies, but usually they're household name sites which offer a unique value-proposition of cultural trust signals for the specified audience. If you're not a titan of industry, then you're likely not one of the favoured few who gets such special treatment from Googlebot so regularly

This is an interesting post to read:

https://medium.com/@baphemot/whats-server-side-rendering-and-do-i-need-it-cb42dc059b38

... you may also have the option of building the HTML on the server side and then serving it in different URLs to the user. To me it sounds like a case where SSR might be the best option. That way you can still use your existing technologies (which are FAST) to render the modified HTML, but render it on the server side and then serve the static HTML (after the render) to users using SSR. That's personally what I would start looking at as it will keep the best of both worlds

Implementation could be costly though!

I don't think you'd get accused of cloaking but that doesn't change the fact, part of your site's architecture will 90% become invisible to Google 90% of the time which is not really very good for SEO (at all)

Another option, instead of building all the post listings on page-load (which will cause stutter between pages), just load all of them at once in the source code and use the JavaScript to handle the visual navigation (from page to page) only. Let JS handle the visual effect, but keep all listings in the HTML right from the get-go. That can work fine too, but maybe SSR would be better for you (I don't know)

...

after looking at your source code, it seems you have already done this. The only real problem would be if the links themselves were 'created' through the JS, which they are not (they all start visible in your non-modified source code). Yes, things which begin hidden, are slightly de-valued (but not completely). This might impact you slightly, but to be honest I don't think separating them out and making the pages load entirely separately would be much better. It would help architectural internal-indexation slightly, but likely would hamper content-loading speeds significantly

Maybe think about the SSR option. You might get the best of both worlds and you might be able to keep the JS intact whilst also allowing deep-linking of paginated content (which currently is impossible, can't link to page 2 of results)

Let me know if you have previously thought about SSR

I'm with Joe on the over 80% rule myself

You could always force trailing slashes instead of removing all trailing slashes.

What you really want to establish, is which structure has been linked to more often (internally and externally). A 301 redirect, even a deeper more complex rule - is seldom the answer in isolation. What are you going to do (for example) when you implement this, then you realise most of the internal links use the opposite structure to the one which you picked, and then all your internal redirects get pushed through 301s and your page-speed scores go down?

What you have to do is crawl the site now, in advance - and work out the internal structure. Spend a lot of time on it, days if you have to, get to grips with the nuts and bolts of it. Figure out which structure most internal/external links utilise and then support it

Likely you will need a more complex rule than 'force all' or 'strip all' trailing slashes. It may be the case that most pages contain child URLs or sub-pages, so you decide to force the railing slash (as traditionally that denotes further layers underneath). But then you'll realise you have embedded images in some pages with URLs ending in ".jpg" or ".png". With those, they're files (hence the file extension at the end of the URL) so with those you'd usually want to strip the slash instead of forcing it

At that point you'd have to write something that said, force trailing slash unless the URL ends with a file extension, in which case always remove the slash (or similar)

Picking the right structural format for any site usually takes a while and involves quite a bit of research. It's a variable answer, depending upon the build of the site in question - and how it has been linked to externally, from across the web

I certainly think, that too many people use the canonical tag as a 'cop out' for not creating a unified, strong, powerful on-site architecture. I would say do stick with the 301s and consolidate your site architecture, but do some crawling and backlink audits - really do it properly, instead of just taking someone's 'one-liner' answer online. Here at Moz Q&A, there are a lot of people who really know their stuff! But there's no substitute for your own research and data

If you're aiming for a specific architecture and have been told it could break the site, ask why. Try and get exceptions worked into your recommendations which flip the opposite way -  i.e: "always strip the trailing slash, except in X situation where it would break the site. In X situation always force the trailing slash instead"

Your ultimate aim is to make each page accessible from just one URL (except where parameters come into play, that's another kettle of fish to be handled separately). You don't have to have EVERYTHING on the site one way or the other in 'absolute' terms. If some URLs have to force trailing slash whilst others remove it, fine. The point is to get them all locked down to one accessible format, but you can have varied controlled architectures inside of one website

Firstly see what Google thinks of the page. On-page SEO checkers require wider experience to use accurately and efficiently. You can't just take insights from online tools and run with them

If the page has no associated keywords / search queries (in Ahrefs, GSC, Google Analytics, SEMRush etc) then that would show Google isn't really that interested. Because keyword data (even from Google) is always heavily sampled, you'd also want to check if, as a 'landing page' (in Google Analytics), the page had been receiving any traffic from Google (Organic Search segment)

If the page is doing well despite what online tools say, rules be damned. If the page isn't performing well (or at all) then just re-build it from scratch in line with best practices, and 301 redirect the old URL to the new one. If the URL stays the same then just rebuild on the active URL, that's fine too (but don't publish until the complete re-build is 100% finished and you are 110% happy with it)

Just so you know, EMA (exact-match anchor text), which is also referred to as 'over' link optimisation, is more a concern for your off-site links. In terms of your internal site structure, that's much more lenient. Obviously if it impacted UX (e.g: site nav buttons with ridiculous amounts of text that become over-chunky, annoying users) then that's bad. If you can satisfy UX and also do some light keyword optimisation of your internal site links, I honestly don't see that as a massive problem. If anything it just gives Google more context and direction

I don't think internal link over-optimisation is a myth, because there's always someone stupid enough to pick up a spoon and run with it (taking it to ridiculous extremes that would also impact UX and the readability of the site). But as long as you don't go completely mental and the links make sense for users (they end up where they would expect to end up, with concise link / button text that doesn't bloat the UI) then you're fine. Don't worry about this overly much, but don't take it to an unreasonable extreme

If the errors are detected by Moz's crawler and Google Search Console (both at the same time) then I'd be much more concerned. It does also depend on the volume of them, if there are like three then it's probably not worth your time to sort it out. If there are hundreds or thousands, you might want to think about that

If you have hidden links in the coding which Moz is picking up on (that's how Moz's crawler works, by following links) then you can't really say: "We've checked each page and know that we are not linking to them anywhere on our site" - the fact that the crawler found the links means they exist and are there (even if you can't see them or find them). That is of course, unless your site is on one of the unusual architecture that Rogerbot (Moz's crawler) has difficulties with. That shouldn't be your first assumption, though - he usually knows where he's going

Where you say this:

"since we migrated our blog to Hubspot so we think it has something to do with the test pages their developers had set up" - pull them up on it! If their developers coded a load of errors into your site, that's their fault not yours and it should be their expense (not yours) to fix it

This is the page regarding their CMS:

https://www.hubspot.com/products/marketing/content-management-system

It does say "A Content Management System Built for Professional Marketers" - so migrating to it, shouldn't cause loads of SEO problems, as SEO is still the largest chunk of most site's online marketing and traffic. That should be nailed down, no problems, fewer problem than your prior system

In-fact, HubSpot know that SEO is important for a CMS: https://www.hubspot.com/cms-and-seo - "Every marketer has been told that they need to consider SEO when creating content. But what makes SEO a unique marketing strategy that marketers should prioritize? And why should your CMS have tools that help you execute your SEO strategy?" - I would argue that a load of 404 errors, could not be considered "tools that help you execute your SEO strategy"

Whether their developers messed up or their CMS is at fault is not really relevant. The main point is, the responsibility to sort it out should be on their side (not yours, IMO)

I would adjust it personally. If for no other reason than, if someone analyses the site later on they should be able to get a good idea of your strategy. Even if you come back to the site later and re-crawl it. Leaving it as it is will inevitably cause strategic confusion down the line. Just set it to your original vision, keep all the signals under your control pointing in a single direction

There is a risk of the new domain being infected with the prior domain's negative equity. Basically if you put exactly the same site up as before with the same content and the domain-name is very similar, Google will assume it's the same site. For a lot of people that can be a good thing (preserving their SEO authority) but if you have negative equity it could be a very bad thing

If Google has mislabeled your site, that's what they believe your site is. Since they believe your site IS a porn site, their algorithms will probably see the new domain (with the same site on it) and interpret that as, hey this porn-site owner is trying to evade our listing bans. Ban evaders are bad, slap it back on!

Since you're already at a point where the algorithm(s) have failed you, I wouldn't anticipate them to randomly be more generous in the future. You might get lucky, but you might find the issue does creep across

The good news I guess is that usually, web-hosting and domain purchases aren't bank-breakingly expensive. As such it's worth a try I suppose. Just don't be shocked if it does well for a few days, and then suddenly declines again

I know that content similarity matters a lot for positive redirects (e.g: if you want to move SEO authority from one page to another, you'd better be sure that the content is very similar - otherwise it has little to no effect even with 301s). Does this also hold true for negative redirects? I'm not certain but I would suspect that the same logic would be at play. In this situation though, since it's exactly the same site and content it could work against you and let the negativity flow through

Maybe you could create 302 redirects instead of 301 redirects (keeps the authority, in this case negative, on the redirecting URL - not the redirect destination). Maybe you could also exempt the Googlebot user-agent from the redirects so that it doesn't see them at all

That would be some stuff I would try, but I wouldn't be amazed if Google got really cross with you for SERP ban-evasion behavior. If you could possibly re-brand slightly and re-write a load of the content that might help significantly. It's a lot of money though when in reality, it's a 50/50 gambit

To be honest that's exactly right. In actuality you aren't ban evading, because you aren't that site and it no longer lives on that domain. But if Google's algorithms are unaware of that, more issues could arise

It's really unfortunate this has happened to you! One thing you might want to check for is compromised / hacked content as well, since here in the Moz community (and in-fact the wider SEO community) we are seeing increased instances of negative SEO combined with hack attacks

Another thing, you have to be really careful when updating your disavow file. If you don't download the existing disavow file first and add your entries on to it, then all of your prior disavow work will be undone. When you upload the disavow txt file, that is your COMPLETE disavow, not simply some additions to it. So if you handled that wrong, you may have disavowed the new stuff and un-disavowed loads of legacy work (which could land you right back at square 1)

In general, disavow work does not result in increased SEO performance. If a load of sites were giving you boosted SEO metrics, and then Google decides they are bad and nullifies them, doing a disavow doesn't magically make Google give back SEO authority that you should never have had in the first place. Those pipelines are severed

Have you considered that in its early stages, a negative SEO attack often actually boosts rankings? Then, when the network is uncovered, that juice is all cut off. Maybe the past positive performance, wasn't all down to you. The negative SEO attack (before it reached critical velocity) may have been boosting your rankings. Now that it's flipped negative, most of that SEO authority will be cancelled out. That could be a bitter pill to swallow, but I have seen it a lot of times

Maybe this drop in performance, is actually where the site should be ranking under your own efforts. If the negative SEO attack initially went undetected and boosted certain rankings, then was discovered and spun around - then actually you are where you should be. Could be complicated to explain to an enraged client though

The reason you process a disavow file, is to list all the spammy links and disavow them, so that Google will not (in the future) give you a manual penalty which nullifies all (or most of) your rankings. Since you don't know which links Google currently does or does not consider to be spam, this inevitably results in a few non-spammy links (which were contributing SEO authority) getting nullified, which makes results go down NOT up. What a disavow does, is trade a very small amount of your current performance away, in return for mid-term insulation against manual actions (which believe me, are truly horrible)

It never ceases to amaze me how people don't understand what disavows are truly for and how many people say "hey I did a disavow and I saw slight drops and no gains". It's only under very exceptional circumstances (like manual actions) where disavows can potentially (if they are accompanied with linked Google Word Docs, explaining in detail what has happened to Google, through a reconsideration request) result in raised rankings. 99% of the time they just insulate you form things getting worse and give you a very slight results dip

One thing you may want to look at is if your disavow was too aggressive. Spam can mean different things to different people. Did you download all links from all tools (Majestic, Ahrefs, Moz, Google Search Console), re-crawl them in SF to see if they are all still live? Did you then fetch metrics (Ahrefs URL Rating, number of linking domains to each URL, PA and DA from Moz, CF and TF from Majestic, Sessions from domain from GA) for all URLs using service APIs and URL Profiler / Netpeak Checker?

Did you put all the metrics against each URL in a massive spreadsheet? Did you normalise and boil the metrics down to a final score to see which links have good SEO authority, and which ones don't?

If you just looked at them with your human eyes, if you only looked at links from one backlink source without doing a master de-dupe from all backlink data suppliers, then you probably didn't do something anywhere near extensive enough to clean up properly. Cleaning up a negative SEO attack properly with minimal risk, is a huge undertaking that could take an expert around 5 days to compile and update the disavow file

Remember you're dealing with an algorithmic devaluation. Algorithms see in numbers, not with your actual eyes. If you didn't have a data-led approach to your work you are extremely likely to continue suffering in the mid to long term

I would go over the work again much more forensically

Finally: Remember that Google and Moz aren't connected data sources. DA (Domain Authority) is NOT used in Google's ranking algorithm (at all). It's  an estimate only (to replace Toolbar PageRank which Google took away, and stopped SEOs seeing). Even TBPR wasn't great, but it was at least aware of disavows, Google penalties and algorithmic devaluations. Moz's DA is 100% unaware of these things and doesn't factor them, so if you are looking to DA to save you - or for things to 'just get better' over time - think again!

First of all those more detailed URLs should have been handled via canonical tags and not via robots.txt

You are probably safe to allow the detailed URLs to rank, try allowing a sample of them to rank whilst keeping others disallowed. First, fix the architecture. Stop using robots.txt and on the detailed URLs, make them canonical to their parents

Once that is done, select a volume of the detailed URLs as a test. Remove the canonical tags from those URLs, allowing them to index. Do they start ranking, performing? Do you get duplicate content warnings?

Depending on the outcome, you may want to lift the canonical tags from all detailed URLs, or even reverse the canonicals so that the detailed pages have ranking preference

I would gravitate to marking everything up and letting Google decide what they want to show. Most of the time when you try to 'sculpt' what Google can see in terms of structured data, it usually results in a structured data spam action. Sometimes it can take weeks, months or years for that to happen - but Google always want to be given the full picture.

Google don't take too kindly to being funneled in a certain direction. Schema and rich snippet spam have been a big headache for Google since they started utilising structured data more, some stuff (like author avatars for posts in SERPs) has been entirely taken away in the past (though someone has told me recently, they have been seeing these again for Google mobile layout only).

Google do have some official guidance here:

https://developers.google.com/search/docs/data-types/faqpage

They give a microdata example of implementation: https://search.google.com/test/rich-results?utm_campaign=devsite&utm_medium=microdata&utm_source=faq-page

In their example, nothing is missing or has been left out. Since that's how Google have illustrated their example, that's what I'd aim for myself

I think you tried to attach two charts there but for me at least, they look like exactly the same chart. Entrances isn't the metric you usually use to evaluate what people call 'traffic', that's actually more commonly associated with the sessions metric

Let's look at what you said you did:

"1. Tried contacting website owner which we  think spam and add all such domains to our disavow list" - this can and usually will only make your results go down further. Your view of what spam is, will not be perfectly aligned with Google's view. As such, some of the links which you think are spam, may have been giving you ranking power - which you have killed off using the disavow tool. If Google think a link is spammy, they nullify it themselves without disavow. If you mark a link as disavow, Google do not give you back the ranking power. Why would they? You are agreeing with them that the link is bad, so why would they give you ranking power for it? Disavows can (under normal circumstances) only make results dip or drop further. The use-case for a disavow, is if you believe your backlink profile is SO bad, that Google are about to give you a manual link penalty (which will kill ALL of your results). If you think that is about to happen, you can stop it from happening using a disavow submission. Disavow trades away (loses) some of your current performance, in exchange for future insulation against manual actions (which are WAY worse than algorithmic devaluation)

"2. We found little duplicate content on sites like Quora, we made those answers down by reporting to Quora"

This probably won't do anything for your SEO. Quora probably still won't delete the posts. If they're still there, it's still duplicate content (down vote or not)

"3. Reported to DMCA on 3 articles articles(partial) from our website."

This is a good move keep doing this. Actually focus more on this if you can find more stolen content! Use CopyScape to track it down

"4. We are trying improving user experience"

This is always good and you should always be doing this

"5. Removed one of our page that shared by many people but our page was not indexed by Google."

I mean, it won't really help to do this as a one off. If you have loads of pages that Google refuses to index, then action might be needed. But if it's just one page and people liked to share it, it seems kind of crazy to erase it to me

"6. Checked and modified content if any our articles are having more keywords than what SEO experts recommend."

Just so you know keyword density is not an accepted measurement in modern SEO. If your articles previously read like they were really spammy, you did the right thing. If they were reading fine anyway AND getting the keywords in, you may have hurt yourself more by doing this

"7. We are working on researching more and figuring our what else can might have gone wrong with our traffic."

Focus on R&I is healthy

"8. Working on improving EAT "

A lot of sites got stung, are still getting stung and will continue to be hurt by this. If you're writing for finance this massively concerns you and should probably be your #1 thing, not your #8 thing. Really focus on this a lot

"I attached our traffic drop graph. I believe this drop is not natural it happened because of some issue at our end and we are not able to figure out the exact reasons."

It could be many things but poor EAT on a finance site will kill your site in 2019

"S_urprisingly another site with not so high quality content started ranking now in the top._"

That's your opinion and you are welcome to it, but the quality of your sites and other sites is being determined by mathematical algorithms and not by human minds. What you think is quality content, may be very far removed from what Google's mechanical mind perceives as high quality. Another thing, older sites which are more established can rank above your with lower quality content than you have (as their SEO authority and trust is higher). You need to think about winning trust and links. Maybe some crappy sites do rank well, but when they were first made they filled a hole (in the query-verse) or they were good for their time. You have to be good in YOUR time. What they did to earn their success (which they ride along on) may have been drastically less than what you have to do in 2019. Never forget that. Comparative analysis-paralysis doesn't get you ahead, it holds you back. Vision is what's needed now

"I am here to get community members/experts help on this. I could provide you if you need any further details.    Thanks a lot for your time. We really appreciate any tips that you can share with us. "

Not a problem hopefully some of my comments have proven useful to you guys

Self-referencing hyperlinks ( <a>tags) are pretty much pointless for SEO and in general</a>

<a>For specific sets of Meta links (like hreflangs or canonical tags) - self-referencing URLs are often a requirement and are part of SEO best practice</a>

I can't seem to replicate your problem

https://d.pr/i/l9sPYn.png (SERPs screenshot)

For me, your site is loading ok in Google's SERPs and showing all the Meta data you have set

The URL which you have shared says that it is indexable with DeepCrawl's indexability checker. Can you give more details and screenshots, or let us know if things have changed?

For SEO it's always better to link directly, rather than linking to a page which then redirects. Redirects only pass large amounts of SEO authority to 'similar' pages (there are checks and balances on this). You could experiment with something like this, but I doubt it will result in much uplift really