Welcome to the Q&A Forum

Following the advice here: https://support.google.com/webmasters/answer/80553?hl=en I can confirm that they are indeed Googlebot.

If you use Fetch as Google in Search Console, are you seeing errors?

Cloudflare has an article here on the subject:

https://support.cloudflare.com/hc/en-us/articles/200169806-I-m-getting-Google-Crawler-Errors-What-should-I-do-

My first question is whether you have a copy of the old site or a site map? Secondly, was all the content optimised, e.g. did it all have sensible urls and live in a sensible directory structure?

Ideally, however you do your redirects, you'd want to create a map of your required redirects. It could be that all the new content is going to be in the same location, but on the new URL (e.g. http://olddomain.com/directory/content -> https://newdomain.com/directory/content), in which case you could use .htaccess to simply forward to the new domain with the same path, or it could be that you're going to move everything about (e.g. http://olddomain.com/directory/content -> https://newdomain.com/new-directory/new-content-location)

Should I re-build the old site first, later build the new site, and then make 301 redirections from the old site to the new site? Or I just build the new site and redirect all the links from old site to my new site?

No need to rebuild the old site first, simply start your new site.

1. How to do 301 correctly to transfer the seo authory in wordpress? Should I set the 301 redirections from my host or should I have to set it in .htaccess? Does anyone know the code?

You could install a WordPress instance on the old domain and then use a plugin to manage your redirects if that's what you're more comfortable with, or you could do them using .htaccess.

2. Should I set every link, every page, every .jpg link from the old site to new site's related address one by one?

This is up to you. Depending upon the scale of your site, doing everything might be safest. However, if the old site was huge and you don't have a sitemap, that is going to take some serious time. I would probably use a tool to discover any existing links and then make sure I was redirecting them, while at the same time having a tool to catch any 404s for redirects I hadn't set up. (You could either give a 404 page or alternatively forward everything to the new homepage while still tracking). You could then add proper redirects for any relevant old pages that were still receiving traffic.

3. There are some good backlinks for the old domain, but now all the old site's links are broken, how to deal with the old urls that the backlinks pointed to?

I think that question is already covered above. 301 them to the new content on the new site.

Hope that helps!

I disagree, but I know there are lots of people on both sides of the fence on this one!

"The globe is a known spam network", so one would assume Google is aware and just ignores it.

Google say (highlighting is me):
"Google works very hard to make sure that actions on third-party sites do not negatively affect a website. In some circumstances, incoming links can affect Google’s opinion of a page or site. For example, you or a search engine optimizer (SEO) you’ve hired may have built bad links to your site via paid links or other link schemes that violate our quality guidelines." - https://support.google.com/webmasters/answer/2648487?hl=en

And John Mueller said (highlighting is me):
"Random links collected over the years aren't necessarily harmful, we've seen them for a long time too and can ignore all of those weird pieces of web-graffiti from long ago. Disavow links that were really paid for (or otherwise actively unnaturally placed), don't fret the cruft." - https://twitter.com/JohnMu/status/1088929651593039872

If you didn't go out and generate those links, I wouldn't worry about them.

Disavowing a known spam link site does not help Google clear up

Sites like that will not do any harm (they won't do anything for your rankings to be fair). I wouldn't worry about disavowing, Google will be well aware of them and discount them, they won't penalise you.

A brilliant response EGOL, however, I would like to put to you, that each of your pages/articles would still be focussing on a single keyword (or keyword cluster) while bearing in mind the overall goals.

You wouldn't have two pages competing for "Stihl Chainsaw MS170 Maintenance" for example, but you might have multiple pages that are talking about "Stihl Chainsaw MS170" in various ways, all probably linking to the page where the customer could buy the actual product or parts, etc.

Of course, if a customer were to search "Stihl Chainsaw MS170" and not be showing any clear intent, then Google may well show multiple results from your site, which, I agree, would be great. (And you'd also likely show up in the SERP generated by clicking on the various, "People also search for", etc)

Firstly, I would definitely take the opportunity to switch to SSL. A migration to SSL shouldn't be something to worry about if you set up your redirects properly, but given that most of your pages aren't indexed at all, it is even less risky.

You will eventually get the traffic back, as far as how long, it's very difficult to say.

I would concentrate on crawlability, and make sure your structure makes sense, and that you aren't linking any 404's or worse. Given the size of your site, that wouldn't be a bad thing anyway.

From your description of your pages, I'm not sure there is any "importance hierarchy", so my suggestion may not help, but you could make use of Google's API to submit pages for crawling. Unfortunately, you can only submit in batches of 100 and you are limited to 200 a day. You could, of course, prioritise or cherry pick some important pages and "hub" pages, if such things exist within your site, and then start working through those.

Following the recent Google blunder where they deindexes huge swathes of the web and, in the short term, the only way to get them back in the index was to resubmit them, someone has provided a tool to interact with the API, which you can find here: https://github.com/steve-journey-further/google-indexing-api-bulk

Hi,

For your more generic keywords, perhaps general industry or overall business, I would keep those throughout the site. For more focussed keywords, maybe individual products, services, content pieces; I would keep those to individual pages.

A very brief example, if you were a car servicing centre, I would be using terms like "car maintenance" and "car repair" across the site, but focussing specific pages on "Tyres", "Brakes", "MoTs", "Car Air Conditioning", etc.

Well, I guess that's the million dollar question.

It's not as simple that Google will simply replace the SERP with the new page. That will be the apparent behaviour until Google updates the listing, as anyone who clicks the link will be redirected, but Google will quickly "notice" and then reapply the algorithm and decide whether the new page should be in the same place. I wouldn't expect that fact that the old page ranked to directly affect the ranking of the newly redirected page, however, the fact that any links to the old page will be being redirected will have an impact.

As far as new rank, I would expect a similar effect to that of simply updating all the content and not changing the URL, and of course, we don't know what exactly would happen then.

If I had to guess, given what you've said, I would say that very specific searches may rank worse (E.g. "double-sided bookmarks") but that more generic terms might rank better (E.g. "customizable bookmarks")

You're right that you can't use Search Console for this kind of redirect, unfortunately, you'll just have to do it the old fashioned way, as far as I'm aware, and just set up your redirects and wait for Google to catch up.

I certainly wouldn't set up a redirect to a subdomain and then redirect that. Google will not appreciate being "lied" to and I suspect it will take just as long for Google to update the index. I would simply set up the redirects and wait patiently. In the meantime, the customer experience will be fine as they'll click an outdated link and be redirected to the page they're expecting anyway.

Does it give a warning/error, or is it simply returning no pages?

When you say migration, are you talking about moving the site to a new domain, or simply to a new platform while maintaining the current domain? To be fair, I don't think it makes too much difference either way, I was just trying to get it clear in my head.

I think you may be over thinking it.

• _What is the best way to make this transition without losing too much of our SEO value?  I understand there is nearly always traffic loss with URL changes but I'd like to at least minimize the damage as best I can.  We have backlinks and ranks for many product pages so I want to make sure we pass as much of this as we can. _

  I would simply redirect both the old URLs to the new URL with a 301, I don't see any issue with doing this as the new page will have all the relevant content.

• (And is this at all further complicated by the fact that URL A & B won't exist on the new site, and URL C doesn't exist on the current site?  Does this impact the use of the 301 redirects and if so, how?)
  No, not really. So long as the new page exists before you create the 301 (or at the same time) there is no issue there.

• Are we better off to approach this page consolidation after the site migration and treat it as a separate project?  This is something that is important to our user experience, and is definitely a change we want to make.
  I don't think so, I would definitely do it as a single project. Except from a "it's slightly less complicated if we break it into parts" point of view, the only benefit SEO-wise in breaking it into two projects would be from a monitoring angle, i.e. if something were to go wildly wrong with your rankings you would know which part of the transition had the impact and maybe be able to diagnose quicker.

Hope that helps!

Alex

This sounds like it might be an extension of your other post I've just replied to...

To expand on what I said in my other response, I would certainly be considering your site structure, if Google isn't finding and ranking the pages, how will your direct customers. Maybe some sub category pages might be a good idea? By location, by type, etc.

Also, have you checked your sitemaps are working correctly? Are the "obscure properties" actually not indexed, or is it just a ranking problem?

Hi,

Your developer hasn't added it as a class, he has added the attribute to the paragraph tag, however, it still isn't right.

The "rel" attribute can only be used with a or link tags. It does not cascade down, i.e. you can't put it higher up to cover all child tags, which is what I think the developer is trying to do.

As you've suggested, he needs to go back and change it.

Hi,

Firstly, yes, that robots.txt is valid and would work for your purpose.

There's a great tool (https://technicalseo.com/tools/robots-txt/) that allows you to put in your proposed robots.txt file contents, the URL you want to test and even the robot you want to test against and it lets you know the result.

My initial reaction really revolves around how well you are ranking for "Jaipur Dentists" and how much traffic that is driving. Domain names are not a massive ranking factor any more, but Google may be giving you a bit of a boost for that term.

If you are concerned that your competitor is "stealing" branded traffic, I would consider PPC, as I suspect that even if you change your domain name you will still have a fight on your hands competing.

Also, given that you currently outrank them for your brand (certainly from where I am searching), it may not be worth the risk. I would probably work on why their knowledge graph is showing instead of yours...

Google's John Mueller has stated that exit intent pop-ups do not attract a Google penalty:

https://www.youtube.com/watch?time_continue=746&v=gS4_JH-QqSg
"What we’re looking for is really interstitials that show up on the interaction between the search click and going through the page and seeing the content. So that’s kind of the place we’re looking for those interstitials. What you do afterwards like if someone clicks on stuff within your website or closes the tab or something like that then that’s kind of between you and the user."

Google won't be monitoring the user's full behaviour on your site beyond the initial bounce/non-bounce (at least as far as the search arm is considered, Analytics is, of course, different!), e.g. they won't see that x happened that caused (or appeared to cause) the user to leave.

Hi Jim,

The key thing, as I'm sure you're aware, will be to manage all of your redirects! It looks to me that there will be quite a lot of duplicated/similar content from across the three existing domains that you'll want to combine, and then ensure that old pages redirect to the new content; this makes it quite complicated and will require lots of custom redirects rather than a simpler bulk https://blog.lucybee.com/* -> https://lucybee.com/blog/*

Depending upon the new structure for the shop element of your site, you might be able to use wildcard redirects (E.g. https://shop.lucybee.com/collections/* -> https://lucybee.com/products/*) but you'd obviously need to be very careful to make sure they all match up.

I'm not familiar with Shopify, but I would suggest getting in touch with them and explaining what you plan to do. If I were carrying out your project I would want to set up a dev site to make all the content and structure changes, and then prepare my redirects, before then putting the whole thing live.

A crawl of the existing three sites before you start and then a map of your redirects is probably best. You can then bulk check the old URLs once you go live to make sure you haven't missed anything. (https://httpstatus.io/)

Have you considered the ongoing video series by Rand on Moz's Whiteboard Friday's? Up to date and a great format.

The first one is here: https://moz.com/blog/one-hour-seo-guide-part-1-seo-strategy

There are links to the others in the series so far at the bottom. (Apart from part 5, which hasn't been linked yet https://moz.com/blog/one-hour-guide-to-seo-technical-seo)

Are the pages linked elsewhere with follow links? And therefore, are those pages already indexed? (I would be surprised if they weren't, especially "About Us"). If so, changing the follow status won't have any impact on crawl budget but will demonstrate to Google that those pages are more important to you.

It seems to me that the pages you've listed would certainly be pages you would want to be indexed. And give the small number of pages you're talking about, it's not going to have any real impact on your crawl budget.

The fact that you're talking about less than 10 additional pages aside, if you're worried about crawl budget (as you might need to be if your e-commerce site is vast), then there is a good article here from Yoast: https://yoast.com/crawl-budget-optimization/
The pertinent section, and to work out if you really need to worry, is:

1. Determine how many pages you have on your site, the number of your URLs in your XML sitemaps might be a good start.
2. Go into Google Search Console.
3. Go to Crawl -> Crawl stats and take note of the average pages crawled per day.
4. Divide the number of pages by the “Average crawled per day” number.
5. If you end up with a number higher than ~10 (so you have 10x more pages than what Google crawls each day), you should optimize your crawl budget. If you end up with a number lower than 3, go read something else.

I would definitely not allow search engines to index those type of results pages. To be fair, they're unlikely to come across them as a bot wouldn't typically fill in a search box to search, but ey might follow a link from somewhere else. For products, I would definitely want to be using category (or similar) pages to define what the search engines saw.