Welcome to the Q&A Forum

Hey Jacob:

• Do you use Screaming Frog? would be great to double check if there's any directive with noindex that it's hurting your .be visibility (about a few of your pages are being indexed). The "site:" command it's pretty useful to use it on-the-fly, but I would recommend always to check if the URLs in the sitemap.xml are being indexed. Wait 1-2 days to see if after submiting your sitemap there's any change
• I assume you are using Wordpres in a Apache server running php, so, in your File Manager (cPanel) or your FTP software, go to the root directory (one level up to public_html), you should have a "logs" folder with a couple of compressed files. Un-zip them and open it with Notepad or any text editor. Search for Googlebot in the logs and see the most recent request from Googlebot
• Yoast it's a good plugin, I use it, but for this case, maybe should be good to deactivate this feature of the plugin and search for another than can handle hreflang, or do it manually
• Yes, maybe your .be ecosystem is pointing to the .nl site, check it with Open Site Explorer and if this is the case, request a change of domain of each site owner. If not, you should begin to build those links in a proper way

In my experience this should take no more than 2 weeks after checking href lang are set up properly (but will depend if Googlebot crawl both sites frecuently), the questions I will ask myself in this case are:

• It's pretty dumb, but sometimes we forget the basics, like: are you blocking the site with the robots.txt? noindex tags? something?
• Double check if the href lang is properly implemented
• In your log files there's any presence of Google bot on both sites?
• Assuming you are using tags in the header for href lang: Have you tried to force the href lang implementation with sitemap.xml? https://support.google.com/webmasters/answer/189077?hl=en
• Have you tried to backlink the .be domain from business partners in Belgium?

Hey:

Migrating a site to a brand new domain after doing all the things to do doesn't guarantee you will keep all your rankings, but if the change is mandatory, and assuming you site will be the same (just a domain change) I suggest to:

• As you said, make sure you are doing a cross domain 301 redirect, so if someone is trying to request http://olddomain.com/fake-page-123 the user should be redirected to http://newdomain.com/fake-page-123\. Usually you can do this in Apache servers modifying the .htaccess with this:

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} olddomain.com$
RewriteRule ^(.*)$ http://newdomain.com/$1 [R=301,L]

• Make sure validate both domains in Google Search Console (and if in your market it's relevant, Bing Webmaster Tools) before migrate. After migrate, make a Change of Address Request from olddomain.com to newdomain.com

• As usual, make sure your sitemap.xml it's up to date in the new domain and try to block all the innecesary pages on robots.txt to avoid spend your crawl budget (just if neccessary, in large site it's more common). Send them to Google through Google Search Console as soon as your site is migrated.

• Optional: Before migration, would be great to check any 404 errors on the new site to address them when you migrate your site, I suggest to use:

• Google Search Console and Bing Webmaster Tools (Crawl Errors)

• Screaming Frog search from broken links

• Open Site Explorer from Moz (Top Pages) and if necesary in large sites, Majestic.com

• Logs files

• Google Analytics history (page views)

• After gathering all this data, make sure the URLs are returning 404 with Screaming Frog, List Mode

Let me know if this is helpful for you.

If I understood, ashortcut.com always redirect to some a.com page. If this is correct, unfortunatelly Google will rank a.com page instead ashortcut.com, as the ashortcut.com page is a redirection pointing to a.com/newslettersignup.

If ashortcut.com isn't a redirection and you just duplicated the pages, yes, the option would be a rel canonical tag on both pages pointing to ashortcut.com. You should determine which domain/page has more relevancy from an SEO POV, so using Open Site Explorer from Moz will be helpful (maybe ashortcut.com doesn't have more Page Authority than a.com/newslettersignup

Let me know if I miss something.

Remember: if any page inside /mobile/ match with another page than the home, you should redirect it to the right page, as Aaron said, not redirect all the folder to the home. E.g.

/mobile/page-about-vegetarian-food > /vegerarian-food

Today doing what we know as "PageRalk sculping" (adding nofollow on links with no relevancy) is not a good idea since 6 years ago.

If you want to give more relevancy to certain pages, you can just link to the important ones and try to link to the pages with little relevancy in more internal section (try also to merge legal pages into just one). Also remember that this isn't the only factor to give more link juice to your most important pages.

Finally, just think about your visitors and show them the important info and links when needed.

Thanks Dirk. At the moment I couldn't find any alternative, so maybe will be a good idea put some hands on this.
If any other person solved this, would be great if can share it with us the solution

Hey guys:

I tried to download all the crawl data from Google Search Console using the API and solutions like this one: https://github.com/eyecatchup/php-webmaster-tools-downloads but seems that is not longer working (or I made something wrong, I just receive a blank page when running the PHP file after some load time)... I needed to download more than 1.000 URLs long time ago, so I didn't tried to use this method since then.

Is there any other solution using the API to grab all the crawl errors, or today this is not possible anymore?

Thanks!