Pages not indexable?

[KenSchaefer]

Hello,

I've been trying to find out why Google Search Console finds these pages non-indexable:

https://www.visitflorida.com/en-us/eat-drink.html

https://www.visitflorida.com/en-us/florida-beaches/beach-finder.html

Moz and SEMrush both crawl the pages and show no errors but GSC comes back with, "blocked by robots.txt" but I've confirmed it is not.

Anyone have any thoughts?

6AYn1TL

[Specscartuk]

Hi 

I have found issue with robots.txt.

User-agent: *

Allow: /

Disallow: /content/lookup

Disallow: /en-us/deals/*
Disallow: /content/visitflorida/en-us/deals/*

Sitemap: https://www.visitflorida.com/sitemap.xml

Do some change in robots.txt file.This issue will be corrected.

Disallow: /en-us/deals/* (you wanted to disallow deals page. but google will cosider disallow page started with /en-us )

Mention only Disallow: /deals/*

If you will correct this thing. this issue will be resolve.

[WebQuest]

Hi Ken,

Running a  quick "site:" query on both URLs, I can see that they are indexed.

You can use the robots.txt tester from the Old Search Console interface and comment out the lines that could be contributing to the block.

In all cases, the pages are sitting in the index as of this writing.

[KenSchaefer]

Hi Specscart,

I think if your assessment was correct the entire site would not be indexed as it all falls under /en-us/.

Ken

[KenSchaefer]

Thanks, Mazen.

robots.txt tester doesn't show that these URL's are blocked but I still get the error upon submitting the URL.  I also see that Google has indexed them at some point, yes.  Thanks for that.  The dev. team is reviewing further for any issues.

Ken

[AlexSmithh]

Hi, I am also facing a similar issue. Do let me know as well if you find out any helpful guide.

[randomghrytu]

Hello please also guide me about my website pages are not indexing too.