MA root domain settings correct for new set up ?

[Dan-Lawrence]

Hi

I've a client project set up in MA set to the root domain so 'catches all'

Dev is currently setting site up for new international config so goes from domain.com to domain.com/uk & domain.com/us

As a result i presumed that i don't need to change MA campaign settings since should catch these since all 'off the root' ?

However i note the latest crawl report shows a massive drop in reported errors (there were thousands) and i know dev havn't fixed these yet, so seems very strange!  The only crawl error that does remain is a 302 redirect

I also note that, despite instructing dev to 301 redirect all old pages to new pages (which they say they will do soon) they have set up a temporary 302 redirect whilst they complete the international config (from domain.com to domain.com/c-uk according to MA crawl error report page). This also seems very strange since the new int HP is domain.com/uk but when i enter domain.com/c-uk it does resolve with domain.com/uk. Also weirdly the domain.com is still resolving.

My questions are:

• What do i need change in MA to report correctly, if anything since surely current setting should 'catch all' inc these sub-folders ? And if this problem is due to a temporary redirect to HP which will soon revert back to 301 redirects to new UK & US sub folder HP's then i should leave 'as is' and previous crawl errors etc will come back?

• Rankings have plummeted since this change which i presume is down to this 302 redirect to the new HP since all the old main optimised pages are now being redirected to the HP. So should i go nuts and demand immediate 301'ing as I originally instructed them or, so long as they do in the next few days problems should right themselves ?

All Best

Dan

[JoelDay]

Hey Dan,

Thanks for the question. Without going into specifics about your page (for privacy) I can say that what's happening is that when we hit your root domain we get a 301 to the www. From there we get a 302 to a regional page which in turn delivers our crawler a second 302 back onto itself. You can see this as well in your crawl diagnostic CSV.

That stops the crawl immediately since we've already been there we cannot find any additional pages.

Geo-based redirects are tough sometimes because it's really up to your server to deliver the right responses. If we get dumped to a specific region it will likely preclude us from hitting other subfolders. Which may or may not meet your needs.

I would advise to see if you can have your sever admin add a directive to allow RogerBot to circumvent that redirect. That or have Roger directed to whatever subfolder you feel would be best representative of your site as a whole.

I hope that helps. Maybe someone else in the community who has dealt with this can chime in with what they did.
Cheers,
Joel.

[Dan-Lawrence]

Thanks Joel

so do you think best/simplest to set up new campaign/s in MA for both domain.com/US & domain.com/UK ?

And if so can i edit the settings on the exsting domain to monitor the new domain.com/uk so maintains the historical data (rankings crawl errors etc) ? since if i did set up a new campaign for domain.com/uk would lose it surely ?

Also i know your suffering a DDOS attack preventing some GA data and i see that at this campaign doesnt have any Search data in reporting at the moment.  I need to provide clients reports early next week will it repopulate that data by then do you know ?

Thanks

Dan