I highly doubt it would effect rankings due to low quality issues but it will show that you have site map error warnings in your GWT console. That issue is technically classified as 'Warnings' and not 'Errors'. The right thing to do in that scenario is take the robots.txt block off and just use a 'noindex' tag on the pages. That way they can stay in the site map but they won't show up in the index. Otherwise you should remove them from the sitemap if you don't want the warnings in GWT.
Best posts made by Dezzign
-
RE: What are the negative implications of listing URLs in a sitemap that are then blocked in the robots.txt?
-
RE: Seeing steady drop in Domain Authority?
Hello Becky, Yes it could be interpreted as their gaining links and your not. But it could also be interpreted as the sites their backlinks are on are gaining links, and the sites your backlinks are on are losing links.
If your backlink is on my site and my site's DA goes up by 30 points, then your site's DA will also go up by a certain number of points. Your DA increase would be dependent on where your backlink is on my site. If your backlink was on my homepage, then I'd say your DA would probably go up by about 4 points.
And yes Becky, the only way to increase DA is via inbound links. But I would also try to move my clients focus away from this metric as it's not that relevant to traffic or rankings. In many niches, I see sites with lower DA dominating the rankings and getting more traffic.
-
RE: What are the pros & cons of recycling an old domain name?
Re-purposing domains is no where near as safe or effective as it used to be. Your asking for trouble if your going to completely change the topic of the website and expect easy ranking benefits from the old off-topic inbound links. (this is one of the footprints they used to recently de-index low quality PBN's because the sites topic had completely changed after the domain was re-birthed)
In saying that, I wouldn't start with a fresh domain either.
I would do my homework on domcop or register-compass and find an expiring/expired/deleted domain with great on-topic link metrics and start the new site that way. If your niche isn't that big then you may be able to find an on-topic domain with metrics like DA40 - PA45 - TF35 - CF40 for under $500. That would give your new site a huge kick start. Hunt for decent Trust Flow with good topical relevance. If you build a great site under a domain like that your rankings will come much easier than using a fresh domain.
-
RE: URL path randomly changing
Those paths all seem fine as they are all legitimate ways of getting to that bunch of products. I'm also assuming that the final page on each of the below urls is a page that contains a selection of products and you can still click on an individual product from the list and go to its url.
1)www.domain.com/productx
2)www.domain.com/category/productx
3)www.domain.com/category/subcategory/productx
4)www.domain.com/bestsellers/productxIt's even debatable whether you need canonical tags on any of those above urls, it all depends on how different those pages are from each other with regards to the content on the page before the products. If all of those above urls had different H1 tags and different content before the product feed then they are all stand alone legitimate pages that don't need canonical tags. But if they're all virtually the same and not much customization has been done to the auto generated pages then yes, they should all have a canonical tag back to www.domain.com/productx perhaps, or the most suitable page.
A bigger issue I think you may have is the url of the final single product page. It shouldn't be like this:
www.domain.com/category/subcategory/productx/final-product/
or like this either:
www.domain.com/bestsellers/productx/final-product/Optimally, it should read like this no matter how the visitor got there:
www.domain.com/final-product/In most cases, for optimal Seo, an extension, plugin, or htaccess command should rewrite the final product url to strip out all of the category url paths or best seller url paths from the urls so your final product page url is short and clean like this: www.domain.com//final-product/ even though the path is really ths: www.domain.com/category/subcategory/productx/final-product/
It's pretty hard to tell what the optimal solution for your site is without having a look at it and understanding your product range and categorization a bit better, but I hope that helps a bit.
-
RE: "Https://" version of my home page has PA of 39, while the "https://www" version has a PA of 43?
If there is a 301 already in place then you have nothing to worry about. Google won't see it as two different pages. As for the PA variation, this is a very minor variation and nothing to worry about either. I would only be concerned if your non www site had a much higher PA than your www site, then you may be redirecting the wrong site. For instance, you wouldn't redirect your PA55 version of your domain to the PA12 version. Hope that helps.
-
RE: URL Injection Hack - What to do with spammy URLs that keep appearing in Google's index?
If the urls follow any particular pattern then you can use a htaccess redirect and return the header code 410 / 403 / 404 to Google. (I suggest 410) They will soon drop out of the index.
I don't know the exact .htaccess syntax off the top of my head but it will be something like this:
If they all come from the same folder then it would look something like this:
RedirectMatch 410 ^/folder/.*$If they have a common character string after the forward slash (such as xyz) then it would look something like this:
RedirectMatch 410 ^/xyz.*$If they have any common character string footprints at all (such as xyz) then it would look something like this (now I'm guessing):
RedirectMatch 410 ^/()xyz.$This would be a pretty easy fix if all of those spammy urls have any common characters after the forward slash or they all originate from a certain folder.
-
RE: Shortened URLs ??
Hello Adam, They most certainly can affect your site's SEO. Every time you create or distribute one of your Bitly shortened URL's you're creating a 301 redirect (From Libya) back to your site. Using Url shorteners is also a spam tactic too. Spammers get their URL shortened by every Url shortening service on the net as a means of sending links back to their sites. It's a very easy way to get 100's of new links pretty quickly, but don't expect the money site to last for long.
On domains I care about, I always try to keep the threshold of 301's against my domain name very low. If my domain is two years old and only has one inbound 301 pointing to it, then I have done my job. You must understand that 301's are one of the most sacred black hat tactics. The minute your domain begins to accumulate an above average amount of inbound 301's expect trouble.
Some sites have 1000's of 301's from legitimate domain migrations because they moved their huge eCommerce site from one domain to another. Google understands this, however, lately even these legitimate authority sites are having trouble maintaining their rankings after domain migrations. Having scores of inbound 301's from URL shorteners isn't optimal in my opinion.
As always, there are exceptions to the rule and it applies to social media in this scenario. If you Tweet a shortened version of your Url and the Tweet goes viral, then it's a little different. If the Tweet gets embedded in 1000's of sites Twitter feeds because they retweeted your tweet, then it's good to have a heap of inbound 301's. It now sounds contradictory I know. But Google can tell the difference, and more often than not, a viral Tweet often spreads to high-quality sites.
The short version of this answer is that 301's are used heavily by dark grey hat Seo's. The less inbound 301's to your domain, the better.
-
RE: URL path randomly changing
If that was the final product page, then yes, you should be using a standard htacess rewrite command to ensure that the final product urls are always www.domain.com/productx But in saying that, the way you had it is totally fine if all the other url possibilities have a canonical tag that points back to optimised version (original product url - www.domain.com/productx)
The htacess rewrite it's not something you should be handling manually. Magento has that option inbuilt into it. It would be a fair amount of work if you had to do that manually. and I would just run with the canonical option if that were the case. Any good eCommerce platform should have the inbuilt ability to automatically remove the category folders and other search queries from the final product url.
Sometimes it's ok to leave the category folders in the url, it just depends on the products being sold. Below would be an example where I would leave the category folders in the url if I was selling different colored soccer balls.
www.sports.com/soccer-balls/black-white/
www.sports.com/soccer-balls/blue-white/
www.sports.com/soccer-balls/red-yellow/ -
RE: Disallow: /jobs/? is this stopping the SERPs from indexing job posts
I don't think it should be blocked by robots.txt at all. It's stopping Google from crawling the site fully. And they may even treat it negatively as they've been really clamping down on blocking folders with robots.txt lately. I've seen sites with warning in search console for: Disallow: /wp-admin
You may want to consider just using a noindex tag on those pages instead. And then also use a canonical tag that points back to the main job category page. That way Google can crawl the pages and perhaps pass all the juice back to the main job category page via the canonical. Then just make sure those junk job pages aren't in the sitemap either.
-
RE: Disallow: /jobs/? is this stopping the SERPs from indexing job posts
Ah yes when it's pointed out like that, it's a conflicting signal isn't It. Makes sense in theory, but if you're setting it to noindex and then passing that on via a canonical it's probably not the best is it.
They're was link out in that thread to a discussion of people who still do that with success, but after reading that I would just use noindex only as you said. (Still prefer the no index on the robots block though)