Welcome to the Q&A Forum

I'd also add https://www.responsetap.com/ to that list.

Have you implemented hreflang to all domains and URLs to indicate to Google your intentions with the domains/URLs?

Also have you checked Analyics >> Audience >> Geo >> Location of www.example.com.au for a period prior to your implementation to see if you have excluded a section of traffic?  I know you said ranking but maybe you refer to traffic.

Either switch javascript off in the browser or search cache:www.yoursite and see if you spot any content missing.

I don't think you can filter out on response codes.

However, first I would ensure you are running the right version of Java if you are on a 64bit machine.  The 32bit version functions but you cannot increase the memory allocation which is why you could be running into problems.  Take a look at http://www.screamingfrog.co.uk/seo-spider/user-guide/general/ under Memory.

ScreamingFrog gives all the data you want.  Tools for the purpose of a creating a sleek report usually don't give the full picture.  It's those issues you draw out yourself that makes a difference.

IMHO directories in the main are dead.  Where they do succeed is when they are very niche and actually human managed.  If you have the skills and passion in your niche then there is a good possibility you will make it work and become a valuable resource.  Using NAP in the detail will provide valuable citation for the business listing.  SEO impact will be minimal, but for the business it is still a movement in the right direction.

You'll need to heavily market the directory to the business customer base as an invaluable one stop shop for their needs.  That will then drive referral traffic to the listings and allow you to charge a reasonable fee for inclusion.

You want Settings >> Show Advanced Settings >> (Privacy) Content Settings >> (Javascript) Do not allow any site to run javascript >> Finished.

Reload the site and check what you can see, or open up.

Yes a toxic domain will infect a new domain if 301 is implemented.

I would lean towards cleaning up the existing domain.  Even if you end up disavowing every linking domain the existing domain is likely to have created more trust than starting from scratch. If it is a manual penalty ensure you document all steps to try and clean up so you can detail in the reconsideration request.

I can answer that question from a slightly different perspective.  I worked with software that parsed varying file types and csv files were so slow to process because of their size.  It was the same scenario writing to them.  So i'm not at all surprised at the slowness of exporting your data to them - not really Moz's fault.

I have understood the length of the URL to be calculated post http:// unless it's https://.

Just the way it is displayed in the results.

yep, sound good.

I was working on a site last year and they switched a DNN module based on your scenario without letting me know, having already tested the existing module.  First I saw was when rankings and traffic wobbled.  In this case the text was lost in the javascript and accounted for about 25-30% of content on all their main pages. Nightmare!

Took another look, also looked at documentation/online and don't see any way to exclude URLs from crawl based on response codes.  As I see it you would only want to exclude on name or directory as response code is likely to be random throughout a site and impede a thorough crawl.

Have you only recently updated your website, maybe the tool has not re-crawled yet?

What results do you get in Google for site:www.yourdomain.com?

What results do you get with https://moz.com/researchtools/crawl-test?

If I understand you correctly and you want to stop a particular URL from being indexed you would be best using the no-index meta tag within the head section of that page:

You probably want the search engines to follow the links on the page even if the page should not be indexed, so add the 'follow'.

I've just had fresh content crawled and indexed that is in this scenario.  Basically we are saying to the visitor "if you really want to know some more boring technical information then expand this, but we don't want to spoil your experience by vomiting all the data at you at once".  Crazy if that is changed.

For the directories I think you need to go down the rewrite route.  This isn't probably functioning correctly but you get the principle.

RewriteCond %{THE_REQUEST} ^GET\ /category/
RewriteCond %{REQUEST_URI} !^/category/sub1/
RewriteCond %{REQUEST_URI} !^/category/sub2/
RewriteRule ^category/(.*) http://www.newdomain.com/category/$1 [L,R=301]

RewriteCond %{THE_REQUEST} ^GET\ /category/sub1
RewriteRule ^category/sub1(.*) http://www.newdomain.com/category/sub1/$1 [L,R=301]

RewriteCond %{THE_REQUEST} ^GET\ /category/sub2
RewriteRule ^category/sub2(.*) http://www.newdomain.com/category/sub2/$1 [L,R=301]

There are site-wide footer links too.

If the aged domain has links going deeper than the homepage you should look at redirecting those pages/directories directly to their associated pages/directories on the new site.  Otherwise all passed link equity will go to the homepage.

You could legitimately have hidden content in terms of category/URLs - maybe only made available if you sign up as an example.  The guidelines are referring to manipulating on page content so there are elements within that page that are hidden and by default manipulating.

Maybe there are multiple URL variations created.  For example, URL parameters, which will create multiple URLs to be indexed in Google.