The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. E-Commerce Site Crawling Problem

    E-Commerce Site Crawling Problem

    Technical SEO Issues
    3 3 720
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • 13375auc3
      13375auc3 last edited by

      Our website displays all of the products in our website If you attempt to visit a category or page that doesn't exist but conforms to our site url structure. Somehow google crawled these pages and indexed them, and they have TONS of duplicate content that hurt us. How do I deal with this problem?

      1 Reply Last reply Reply Quote 0
      • EGOL
        EGOL last edited by

        Can you block them from indexing with robots.txt.....

        .... or redirect them with .htaccess.?

        1 Reply Last reply Reply Quote 0
        • AdoptionHelp
          AdoptionHelp last edited by

          Two ways:

          1. Find out where Google followed a link to the non-existent category pages, and get those links removed and the category pages redirected or blocked as EGOL mentioned.

          2. Change your code so that non-existent categories show a 404 page, preferably a 404 crafted to gently push your user to something they may be interested in.

          1 Reply Last reply Reply Quote 1
          • 1 / 1
          • First post
            Last post
          • Blog on subdomain of e-commerce site
            KaneJamison
            KaneJamison
            0
            2
            247

          • I have 2 E-commerce sites - Can i cross link?
            AlanMosley
            AlanMosley
            0
            9
            410

          • Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?
            Nobody1560986989723
            Nobody1560986989723
            0
            5
            422

          • Site Crawl
            Sean_Dawes
            Sean_Dawes
            0
            2
            414

          • E-Commerce site and blogs
            ResslerMotors
            ResslerMotors
            0
            2
            400

          • WordPress E-Commerce Plugin Duplicate Content Problem
            stefanok
            stefanok
            0
            3
            879

          • Site problem
            Entrusteddev
            Entrusteddev
            0
            9
            920

          • Problem with my site
            KeriMorgret
            KeriMorgret
            0
            10
            946

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy