The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. My wepgages aren't crawled by google

    My wepgages aren't crawled by google

    Technical SEO Issues
    9 4 131
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Poutokas
      Poutokas last edited by

      Most of my webpages aren't crawled by google.
      Why is that and what can i do to make google index at least most of my webpages?

      1 Reply Last reply Reply Quote 0
      • ZoeRigley
        ZoeRigley last edited by

        Hi,

        Firstly I'd check that Google can index your website. Have you added your site to Google Webmaster Tools? I'd start there and check for any crawl issues, especially your robots.txt file and any no-indexing of pages.

        Secondly, if your website is brand new, I'd add your website to Dmoz & some relevant good quality sites like Yelp, Yell, Yellowpages, Google Plus (where relevant). Make sure the details you add to each match exactly with the details on your website. It will take some time for your site to appear in Google's index- sometimes a few days, sometimes a week or so- you can check by typing site:yourdomainname.com into a Google search to find the pages.

        If your website is not new & has been indexed by Google before, I'd investigate whether you have a penalty. This post on penalties from white.net is really useful!

        Hope this helps, 
        Zoe

        Poutokas 1 Reply Last reply Reply Quote 2
        • Poutokas
          Poutokas @ZoeRigley last edited by

          Hi

          I added my website to Google Webmaster Tools and i checked my website and i don't have any crawling issues.
          I added my website to Dmoz but the backling didn't appear yet.My site is live for about a year and google doesn't crawl most of my webpages yet.
          Is it because i don't have quality backlinks?

          Thank you

          ZoeRigley 1 Reply Last reply Reply Quote 0
          • ZoeRigley
            ZoeRigley @Poutokas last edited by

            Hi,

            Backlinks certainly help, if there's no links at all to your site that could be a reason, but it's hard to say without looking deeper.

            Are your internal pages all linked to each other? Does your website have a structured navigation system? This is also really necessary to ensure Google will index your whole site, not just a couple of pages.

            Zoe

            1 Reply Last reply Reply Quote 1
            • DirkC
              DirkC last edited by

              Hi

              You didn't answer the first part of Zoe's question - are you sure that your site is crawlable and that there are no issues with the robots.txt / noindex tags, ip detection  systems, canonicals on all pages pointing to the home and so on. It's not because you can see all pages of your site in a browser that they are accessible/crawlable/indexable by google.

              Try a crawl with Screaming Frog and user agent Googlebot to see if your pages can be crawled and indexed.

              Backlinks are needed to have your site ranked for keywords - but it's not a prerequisite to have your site crawled. (noticed that a few times when a dev site was indexed by accident)

              Without the actual url it's impossible to give a more detailed answer.

              Dirk

              Poutokas 1 Reply Last reply Reply Quote 2
              • Poutokas
                Poutokas @DirkC last edited by

                Hi
                Yes my site is crawlable.
                I checked if robots.txt or noindex tags and canonical urls and everything is fine.
                Maybe is it because my website has over 400.000 pages

                1 Reply Last reply Reply Quote 0
                • MichaelGregory
                  MichaelGregory last edited by

                  The only site I can think of that would legitimately have 400,000 pages is Amazon.com. Google probably thinks your site is full of a ton of low quality content. Why in the world do you have that many pages? Are they low quality garbage? Do any visitors actually engage this even a fraction of them?

                  Poutokas 1 Reply Last reply Reply Quote 1
                  • Poutokas
                    Poutokas @MichaelGregory last edited by

                    My website is a yellow pages site from Greece www.vreite.gr
                    It has registered more than 175.000 businesses and every business has 6 profile pages(main page,product page,feed page etc.)
                    Many visitors engage with these pages and are absolutely dunamic pages.
                    Is that a problem?

                    MichaelGregory 1 Reply Last reply Reply Quote 0
                    • MichaelGregory
                      MichaelGregory @Poutokas last edited by

                      Well, Google does have a crawl budget, they might be using that for your most popular pages. As long as your indexed pages number is going up, that means google is working its way through the backlog.

                      1 Reply Last reply Reply Quote 0
                      • 1 / 1
                      • First post
                        Last post
                      • Google has deindexed a page it thinks is set to 'noindex', but is in fact still set to 'index'
                        d.bird
                        d.bird
                        0
                        5
                        93

                      • Google Crawling Issues! How Can I Get Google to Crawl My Website Regularly?
                        libero_net
                        libero_net
                        1
                        8
                        142

                      • Google Cache can't keep up with my 403s
                        HireSpace
                        HireSpace
                        1
                        4
                        525

                      • Why isn't my site not searchable from google?
                        Czubmeister
                        Czubmeister
                        0
                        15
                        1.8k

                      • When do you use 'Fetch as a Google'' on Google Webmaster?
                        MagicDude4Eva
                        MagicDude4Eva
                        1
                        13
                        7.5k

                      • Why aren't certain links showing in SEOMOZ?
                        trophycentraltrophiesandawards
                        trophycentraltrophiesandawards
                        0
                        9
                        392

                      • Google is somehow linking my two sites that aren't linked! HELP
                        jenmcardle
                        jenmcardle
                        0
                        4
                        601

                      • Google Has Indexed Most of My Site, why won't Bing?
                        SlavaRybalka
                        SlavaRybalka
                        0
                        3
                        2.3k

                      Get started with Moz Pro!

                      Unlock the power of advanced SEO tools and data-driven insights.

                      Start my free trial
                      Products
                      • Moz Pro
                      • Moz Local
                      • Moz API
                      • Moz Data
                      • STAT
                      • Product Updates
                      Moz Solutions
                      • SMB Solutions
                      • Agency Solutions
                      • Enterprise Solutions
                      • Digital Marketers
                      Free SEO Tools
                      • Domain Authority Checker
                      • Link Explorer
                      • Keyword Explorer
                      • Competitive Research
                      • Brand Authority Checker
                      • Local Citation Checker
                      • MozBar Extension
                      • MozCast
                      Resources
                      • Blog
                      • SEO Learning Center
                      • Help Hub
                      • Beginner's Guide to SEO
                      • How-to Guides
                      • Moz Academy
                      • API Docs
                      About Moz
                      • About
                      • Team
                      • Careers
                      • Contact
                      Why Moz
                      • Case Studies
                      • Testimonials
                      Get Involved
                      • Become an Affiliate
                      • MozCon
                      • Webinars
                      • Practical Marketer Series
                      • MozPod
                      Connect with us

                      Contact the Help team

                      Join our newsletter
                      Moz logo
                      © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                      • Accessibility
                      • Terms of Use
                      • Privacy