Welcome to the Q&A Forum

Hey Shawn, 1. If GWT is showing a 404 without a "Linked from," I wouldn't worry about it too much. Sometimes Google tries to recrawl URLs it crawled once in the past, but that doesn't mean it's causing a problem. I would double check that you don't have any internal or external links to "blog/blog," then just "mark as fixed." There's no point to disallow Google from crawling a page that isn't there in the first place. 2. Nope. The only danger of having a long robots.txt is that you may get lost while checking your rules and accidentally disallow a page you want in the index. Whenever you add a rule to robots.txt, make sure you go to Google Search Console and use the robots.txt Tester to check if you've disallowed any important pages. Good luck! Kristina

Very Thanks to you Sir.

Thanks Matt and Erica. I think we'll have to take the band aid approach at first, but it's definitely something I'll now think about with future updates and developments

Thanks friends, for your reply. As you said, I will also make a deep analysis in to those spammy links manually. If I feel really those links are spam then will get rid off them.

If anybody ends up searching for this topic please view this Whiteboard Friday with Rand. I think he actually ended up reading my question and answering it: https://www.youtube.com/watch?v=oG3mxJnA8Zk

There are two or more different URLs on the same website competing against each other for that search query. If you hover over them or scroll down you can see which of the URLs they are. Trouble is, since they are competing against each other, it may be harder to climb the ranks as Google is getting confused and doesn't know which one to rank. Better to have one single URL with content optimised to target the search topic and keywords, as it relieves Google of having to make the choice and ultimately... screwing it up.

Hi there CSobus, According to Matt Cutts, "custom URL shorteners are essentially like any other redirect... and will pass page rank to the final destination". The negative impact would be the same as with any other redirect, like when you set up a chain of redirects that dilutes your link equity.

Hi Andrew! I checked out the site and got the same result. Unfortunately, the MozBar doesn't work on some sites because of the way those sites are structured. But! There is a workaround for this issue. Open the MozBar on a different site and click on the "dock on bottom" button next to the gear icon. Then return to the website you are having issues with and refresh the page. The MozBar will appear on the bottom. Thanks and let me know if you have any questions! Roxana

Roman More than crawl issues, but in short yes, broken links, site loading speed, mobile optimization etc.  All the basics.  The technical seo audit by Moz is an great checklist to work through. Yes they will be if they are from bought links, or spam directories, but if they are earn't links then they would be natural and should not incur a penalty.  It is hard work getting one or two.  We use Moz and majestic and do a link analysis.  Have you done that? How many links do you actually have with the keyword anchor text you are targeting on the specific page? Your own blog, and own socials etc and the posts get re-published because you created such awesome content. Hope that assists.  Tragically there is nothing short of back breaking work ahead...

Hi Guys, OK so we resolved this and it's a Microsoft issue!!  and a schoolboy mistake from me! We use "Web User Controls" on the website that allow us to write re-usable modules for things like headers, burger menus and footers in an ASP.NET web solution.  We then drag and drop the onto any page and if we need to change them then we just change the core web user control HTML. The rogue links were coming from these web user controls, when they were on pages in the rogue directories. Now the reason we had this fault on this website on not on any other was because we made the mistake of using standard HTML Well if the page is in the root directory then no problem.  If you want to go to the home.aspx page then the link is fine (HOME However if you put this footer on  page in a sub folder then you need to change the link to ../home.aspx, but you can;t do that or all the pages in the root folder go squeewy.  So this is why you use an ASP:Hyperlink control instead, this allows you to define a page from root with the ~/home.aspx link. This is where all of these rogue pages were coming form. TY for the help guys.

You bet Russ. I appreciate what you guys do and am happy to participate and share as I can!

Pleasure - now go smash them..

We had this issue when we moved from staging to production.  Worked fine on staging, but not on production. It ended up the clock on one of our production servers was 6 minutes behind.  This caused the expiration to error out.  Immediately we increased the expiration, and ultimately set the proper time on the server. This resolved all issues relating to the following error: 401 Error - Your authentication has expired. Re-send your authentication and try again. For more information on signed authentication, see: http://apiwiki.moz.com/signed-authentication

Sorry to revive this old thread, but I don't see a better place to put this. I am looking at the same report - I have a ton of 404's on the list, however I have implemented 301 redirects already.  Will this list update as the 301's get indexed?  Is it possible to have this list reflect the new pages that the 301's point to?  As it sits right now, the "Top pages by Authority" report is useless as it's showing me old pages that no longer exist. Thanks in advance for your answer. Chad

Hi, It is worth remembering that a canonical just suggests to Google, which is the preferred page to index and serve. This helps stop duplication issues. What is doesn't do is place a block on a page and prevent anything else being crawled from there on in. You would need to add a nofollow and noindex for that to happen. If someone lands on a canonical page within the site, they can still browse around from there. Think of a canonical as a less drastic action than a 301. PR will flow via a canonical in the same way as a 301 as well, but doesn't just stop there. I hope this helps. -Andy

You guys are awesome. These are very very helpful answers I'll get to work!

Hi again Tymen, The mozscape crawl didn't work as intended and the data is incomplete from the last crawl, (see: https://moz.com/community/q/september-s-mozscape-update-broke-we-re-building-a-new-index) so you might not see these updated links until the next scheduled crawl (early October) Are you see the new links in Google Webmaster Tools (AKA search console)? I wouldn't focus on seeing the backlinks in Moz and focus more on continuing to build more links. As the saying goes "a watched kettle never boils" set a reminder to check back in early October to see if your links are showing up, but in the meantime continue to focus on building more links. I hope that helps. https://moz.com/community/q/september-s-mozscape-update-broke-we-re-building-a-new-index

Thanks!  Yes, I'm talking about "referring domain counts". That's what I was thinking. After looking at link building strategy and individual links, nothing else makes much sense. I've included the graph, just to confirm. I was referring to the top two lines. zhe37im.png

Joseph, Yes, GWT = Google Webmaster Tools In GA, you will want to take a look at traffic levels, Bounce Rate, Page views, Time On Page, etc. If any of these metrics show a significant decrease instantaneously (over a couple of days) you can be fairly sure there is something going on from a penalization standpoint. That, or the GA tracking code may have been tampered with during a site redesign. The way to know this is occurring is if your rankings are staying relatively high, but your measured traffic is decreasing drastically. We had a client who went through this just this week - it's a small problem to fix, but it can make your heart race when you think it's a penalty. GWT will inform you of any penalization taken against your website (if it is manual). If it is algorithmic, your only real warning will be your ranking and traffic drops. Besides that, it is also fairly good for security issues, but these may not be alluded to directly. A great way to determine whether a site is at risk/hacked is to check the link profile - if there is an unnaturally large number of incoming spammy links, there are good odds you are the target of a negative SEO attack, or the site is hacked and being used for spam. Use a site: search to determine if new pages are being created on your sitemap and what they are targeting. These are worst-case scenarios, so I don't know if you will be tested on them. More likely you will have to make adjustments to some basic on-site ranking factors like H1's or Title Tags. Feel free to touch base any time if you need additional tips - you can PM me anytime. Best of luck!