The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Local
    4. My crawl is only crawling only the 1st page. What’s going wrong?

    My crawl is only crawling only the 1st page. What’s going wrong?

    Moz Local
    2 2 119
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Unidev
      Unidev last edited by

      When I crawl the website kmtstore.com usiing the moz tool , it crawls the product urls only the fist page and skips the rest of the products on the remainaing pages .

      Here are the produclist page urls which has the products on many pages but on crawling only the 1st page products show up . I have crawled this website kmtstore.com yesterday and all the details being given is as per the latest crawl report generated yesterday./

      http://kmtstore.com/pl/11099/H2O-Jet-Parts-Flow-Style.aspx
      http://kmtstore.com/pl/11103/Pneumatic-Control-Valves.aspx
      http://kmtstore.com/pl/11104/Swivel-Joint-Assemblies.aspx
      http://kmtstore.com/pl/11105/Cutting-Head-Components/Cutting-Head-Replacement-Parts.aspx
      http://kmtstore.com/pl/11106/Cutting-Head-Components/Focusing-Tubes.aspx
      http://kmtstore.com/pl/11107/Cutting-Head-Components/Cutting-Heads.aspx
      http://kmtstore.com/pl/11108/Cutting-Head-Components/Orifice.aspx
      http://kmtstore.com/pl/11109/HP-Valves-Fittings-and-Tubing/Tubing.aspx
      http://kmtstore.com/pl/11110/HP-Valves-Fittings-and-Tubing/Inline-Water-Filter.aspx
      http://kmtstore.com/pl/11111/HP-Valves-Fittings-and-Tubing/High-Pressure-Fittings.aspx
      http://kmtstore.com/pl/11112/HP-Valves-Fittings-and-Tubing/Gauges.aspx
      http://kmtstore.com/pl/11113/Intensifier-Parts/Electrical.aspx
      http://kmtstore.com/pl/11114/Intensifier-Parts/Low-Pressure-Water-Systems.aspx
      http://kmtstore.com/pl/11115/Intensifier-Parts/Topworks-Replacement-Parts.aspx
      http://kmtstore.com/pl/11116/Intensifier-Parts/Hydraulic-System.aspx
      http://kmtstore.com/pl/11117/Intensifier-Parts/Sealing-Head-Replacement-Parts.aspx
      http://kmtstore.com/pl/11118/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Coning-and-Threads.aspx
      http://kmtstore.com/pl/11119/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Topworks.aspx
      http://kmtstore.com/pl/11120/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Lubricants.aspx
      http://kmtstore.com/pl/11121/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Pneumatic-Valves.aspx
      http://kmtstore.com/pl/11122/Maintenance-Tools-and-Lubricants/Manuals.aspx
      http://kmtstore.com/pl/11123/Pneumatic-Control-Valves/Pneumatic-Control-Valve-Replacement-Parts.aspx
      http://kmtstore.com/pl/11124/Pneumatic-Control-Valves/Cutting-Head-Nozzle-Tubes.aspx

      1 Reply Last reply Reply Quote 0
      • ErinMcCaul
        ErinMcCaul last edited by

        Hi there!

        Thanks for reaching out! My name is Erin, and I'm on the Moz Help Team. I appologize for any confusion here. I downloaded your crawl CSV for kmtstore.com, and everything looked fine. From what I can tell Google has 296 pages indexed for your site, and we crawled 241.

        Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. Thus, your homepage is the only seed we use to crawl your site. We know that Google uses multiple seeds, so it could be possible Google is indexing more pages that way. Since we use recursive crawling, if these product pages aren't linked to your homepage, our crawler won't be able to pick them up.

        I hope this helps, and if you have any other questions, please reach out to us at help@moz.com! We'll be able to take this conversation offline and answer any other questions for you from there!

        Cheers!

        Erin

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Correct GMB page is not being pulled to validate in Moz Dashboard
          samantha.chapman
          samantha.chapman
          0
          2
          66

        • What do I do if the business listed in Moz Local Business Listing for Yellow Pages is wrong?
          moz_support
          moz_support
          0
          2
          172

        • What's Going On with Moz Social
          moz_support
          moz_support
          0
          2
          195

        • Facebook page not appearing in the Moz Local listing report
          GleamingGlass
          GleamingGlass
          1
          4
          1.1k

        • Facebook page has disappeared from moz local
          JordanRailsback
          JordanRailsback
          0
          2
          186

        • Insider Pages and Citysearch listing issues
          Ree1701
          Ree1701
          0
          3
          723

        • What do I do if Moz Local is pulling the wrong business citation?
          wparlaman
          wparlaman
          0
          3
          95

        • Error's while trying to upload listings
          Abe_Schmidt
          Abe_Schmidt
          0
          2
          98

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy