Welcome to the Q&A Forum

Hi there! I just ran the site through our crawler using a few different cURLs and it does look like the homepage URL is returning a 404 to our crawler: http://www.screencast.com/t/fgoeCGHu and http://www.screencast.com/t/SRzcBXJIZmw. It may be a user agent specific issue, where the server may be trying to specifically block our crawler, but it is set to use the wrong http status so it returns a 404 rather than a 403. If you look at your server logs for the time of the crawl, you should be able to see the exact response of your server to our crawler, rogerbot. I would recommend working with your webmaster to look into how the site is responding to our crawler further. If you have more questions or need specific information, I would recommend emailing help@moz.com so that we can look into the issue further for you.

No problem! Let me know if you need anything else!

I don't believe there's a way from the user side to cancel a crawl. However, your competitor won't get notified of anything. Moz doesn't have their email address or phone number to contact them. The tool crawls their site like Google, Yahoo, Ahrefsbot, Majesticbot, and all the other web search engines do. They will not even notice that you ran this accidentally. It's similar to doing a crawl of a site with any other search engine or tool. It will literally never come to this but if they question it, blame your SEO guy. Everyone else does. lol Say "my SEO guy must be doing some competitor research."  Us SEO guys do that all the time for pretty much every client. We'd not really be doing our job if we didn't.

There can be multiple reasons why this could have happened but before I start, I personally do not trust the data by search console and the main reason is because its delayed but I don’t think it can’t be that wrong that it will challenge the data provided in Google Analytics. Here are some of the possibilities: Check your analytics IP data (ignore traffic from…) There is a possibility that Analytics is ignoring some of the data as instructed whereas webmaster console is looking into every detail. Very nice point by Matt. Maybe analytics data is not available on every page on the website and Search Console is tracking that traffic as well. Also, make sure that you are looking in to the full traffic data in analytics and not just the search traffic only as search console will be tracking the data from search, referral, paid and more. I believe a quick audit should point where the problem is! Hope this helps!

My questions were just the tip of what could be wrong. As I said and Moosa repeated, you are likely going to have to hire someone as a consultant to solve this. There are way too many factors at play on why you may have lost some organic lately. Competitor improvements, your own site not crawling properly, parameters & duplicate content confusing Google, new site structure loads slower, etc. There aren't another 3-5 questions we can ask to sort this unfortunately. Someone needs to dig in with your site, analytics, etc. to see what's going on. Exact dates matter (algorithm updates) as do how everything is setup for your redirects and all that. As I said, Archive.org saw your redirect as a 302. That could hurt you. There could just be so many reasons.

Maybe your hosting also include certificate for your site and support secure connection too. But since you didn't enable this visitors may get other site (yeah, that's right!) and also invalid certificate (because certificate mismatch with domain). Isn't great situation, but you can check your site with https:// and disable it on your hosting control panel.

I suggest you to dig into the possible issues Dirk is talking about, especially if there are some JavaScript and cookies on the works. In fact, I have seen several cases where a not perfect JavaScript use may end up screwing things. i.e.: I go to domain.com - english - and then go to its russian version - domain.ru - from the selector and then go back to domain.com, but now the version I see is the russian in domain.ru because of bad JScript and cookie management.

Just updating the URL of the post Miriam mentioned. It's been moved and is returning a 404 error. Try https://www.imprezziomarketing.com/is-permanently-closed-killing-your-ranking-4-case-studies/ instead.

Private network - essentially if you own all those sites that are linking to you, all using the same template, etc. that's potentially very bad. Especially when done the way you've done it. If you're using an SEO agency who is building links from sites like hottubgalaxy (dotcom) for you, they're doing a very poor job of hiding their intent. If you are doing this yourself, stop it. You obviously know a little about SEO given your exact anchor text link to start your question ... but even that, here on Moz, is kind of suspect in most cases. You could just give us the URL.

Thank you for the info. For this: "update your home page meta data to focus on your brand and overall category (baby accessories maybe? sorry, not too familiar with your niche)" Ok so to Clarify, I should focus the meta data on "baby Headbands, Baby Girl Headbands, Headbands for Babies" and then mention that we sell other things. So maybe this meta Title: Baby Headbands - Baby Girl Headbands - Infant Headbands - Baby Bow Headbands - Hair Bows for Little Girls at Princess Bowtique! Meta Description: Baby Headbands or Flower Baby Headbands  Beautiful Infant Headbands, Tutus, and Baby Hair Bows, Pettiskirts, and Baby Clothes at Princess Bowtique! 100% Satisfaction Guarantee. We sell the cutest clothes for babies! Thank you, Tiffiny

Should I still try to rank my page even though the image is ranking as the first image displayed in the google results. Absolutely YES.  I have lots of first image in the SERPs and first listing in the organics.  Both of these bring traffic.  That traffic is generally not as valuable as traffic from organic web search but you can still make money from it.  (Visitors to Google image search are given the option to view your image or visit your page.  Sometimes when they view your image that viewing is done on Google's website, sometimes the visitor is moved to your website.  Google is known sometimes to frame your website within Google image search.  Google changes how image search works from time to time and some of their configurations produce almost no traffic for the owner of the image.) Will this count against me by Google if I do? Absolutely NOT.   It may actually increase your opportunities.  You then have great content from web search and great content from image search.  They should want to show that page to everyone! Or should I try to make the image being ranked more enticing for google searchers to try to increase traffic? Absolutely NOT.   Get two images on that page and attack with both.  Three would be better.  Four or five, better still. As in other things, much of this depends upon how much time you have to spend on it, how valuable the traffic might be and whether you are reaching for the competitor's throat or farting around. 

Thanks!!

Hi, This is a good question; for now my answer would be no, I wouldn't bother with rolling it out to non-news pages at the moment. Right now it's only for news/article pages, and if you do have any of that content on your site, it would certainly be worth rolling it out to those, but we don't know how it will play out. Although it is likely that we could see it roll out further in the near future, for now I think you're best off simply improving other areas (including mobile-related things like mobile-friendliness of the UX and site speed) until we see what direction Google ends up taking this in.

You have to use the home address as the main point to send the postcard & list on the Google business page, then hide the address. Do not use virtual offices, po boxes, or anything else to try and game the system. The reason Google is tougher on this niche is because so many locksmiths have done shady things to manipulate the system and gain an advantage. Follow the rules and set up the page legitimately to have longer term success. If there is no physical office location, you must use the home address. Optimize the website to cover the different service areas that you want to target (use local landing pages), which will help improve organic ranking in those areas. Do not create multiple pages for each location (that's another violation), just use the home address & hide it on maps when you select a service area.

It will probably help if you delete the following line from your robots.txt file: User-Agent: rogerbot Disallow: / Dirk PS You also block /js/ for all bots - Google doesn't really recommend that

I understand the difference between what you're doing and what Google shows, I guess I'm just not sure when I'd want to know that something could technically be indexed, but isn't? I guess I'm not your target market! Good luck with your tool.