Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
Hey Becky, I definitely question if they're being crawled at all. Do you have access to your server logs at all? If so, you could then use Screaming Frog's Log Analyser (https://www.screamingfrog.co.uk/log-file-analyser/) to parse through them and find if Googlebot is indeed hitting those pages. It would be worth the investigation!
Totally agreed, Daniel! I'd also say it's our job to set expectations and be clear about when something is a test vs when something will more than likely work. Consulting is all about setting expectations!
This is actually a really interesting question. I looked at their category pages (eg http://venturebeat.com/tag/ar-vr-weekly/) and those seem to be set up correctly to handle infinite scroll as it sends the search engines to the next page.
I've not come across this with infinite scroll on articles, though. I'm sure they've tested it extensively to figure out the best way to send search engines to future articles, but who really knows if it's being effective. If it's still there, I'd assume that they've seen positive signs but it is definitely a non-standard implementation of rel-next/prev!
This does bring up a good point about copying/not copying a competitor's strategy. They have this implemented, but would it work for your own site/business? Maybe, but maybe not. We can't be sure until we test it ourselves (or speak with someone at VentureBeat who wants to share their learnings :-)). If you know when it was rolled out you could benchmark there and look at SEMrush or another tool to see their organic visibility and from there draw at least some correlation, if not causation.
Thanks for flagging this up! It's cool to see.
Hey Graeme -
Good questions here. I think there is something else going on with your page than over-optimization, though you could definitely remove some mentions of your keyword in that top chunk of text. Any idea if some of your competitors have done anything to move up?
I checked your page's cache and it hasn't been re-cached since November 20th, which surprises me. I checked your robots.txt and there's nothing there that causes alarm. Have you made any other changes to your page?
A few things I'd also think about on this page and your site:
Good luck!
Hey Leigh! Can you clarify if the search engines are crawling the /category/article-title/ URLs or if your breadcrumbs just hint that it is the structure? Feel free to PM me the site if you want.
Whether it's worth restructuring or not to add /category/ to the URLs depends on a lot of things. It's definitely helped on ecomm sites and I've recently done this with a marketplace I've worked with, which has really helped with rankings and longterm will really help with traffic.
But you need to weigh it and the different considerations - all the redirects that need to take place, updating sitemaps, internal links, etc - against other things you could do to improve your rankings like building links and developing new content, as well as generally speeding up your site if needed. URL changes are never something to take lightly!
It's a really interesting question and I wonder if they are being crawled. The link destination on them in the right sidebar goes to /#, which shouldn't let the search engines crawl these links.
Are you seeing these parameters in Search Console or your log files? That is where I would look to see if they are actually being hit by Googlebot.
If they are, then you should remove that anchor link and let the checkboxes activate the facets. Not sure how easy this is to do technically, but it's the right way to do it.
Hi Emjmoz! Lots of stuff going on here. On to your questions.
First about redirects. 301 redirects are permanent redirects, which tells the search engines that a page has permanently moved to the location where the redirect ends (eg you redirect /page/ to /page1/, /page1/ is the final destination). Historically, only 301 redirects have passed link equity through them and been guaranteed to drop the original page (so /page1/ should now rank and /page/ should drop from the search index). Google has, in recent years, been slower to honor 301 redirects but all of this still holds true, and if you're having a big issue with Google not dropping a specific URL you can always Fetch as Googlebot within Search Console. To drop them out at scale, you can short term create a specific XML sitemap with the old URLs and submit in Search Console. Once you see the majority drop out, unsubmit that sitemap.
302 redirects are temporary, meaning the search engines will think that the original page may come back. 302 redirects historically do not pass link equity and do not drop the original URL out of the index. Some search engine representatives have said that if a 302 redirect is left in place for a long time they will start to treat it as a 301, but this is really in answer to some major CMSs using 302s by default and thus hurting their customers.
If you are doing a proper site migration to new URLs, use a 301.
To your question about duplicating the site on a different domain (eg .co.uk) for about a month and then redirecting it back to the original, I would question this. It would be better to put the new site on a subdomain or with a specific parameter on the end of the URL with those URLs canonical'd back to your current existing URLs. Otherwise, you risk duplicate content and hurting your search performance. As you roll the new site out, you should also think about a) sending small amounts of traffic to your new pages to track conversions and interaction and b) you can expose some specific pages to the search engines to see how the search engines treat them. This would be very hard to do with a separate site.
Good luck!
I also see that you mention that you'd like to find someone you could call on for future questions. My company GetCredo.com helps with this. Just search "technical seo getcredo" in Google and you'll find a list that you can contact. Or, feel free to reach out to me on Clarity.fm.
Hey Becky, I see you posting a bunch about your technical SEO and internal linking/indexation discoveries. Great to see that you're digging in deep!
When you say a "facet", do you mean a link like this - http://www.key.co.uk/en/key/multipurpose-storage-lockers#facet:-70000000000000105744949554832109109&productBeginIndex:0&orderBy:5&pageView:grid& ?
If that's the case, that page has a canonical on it back to the base of http://www.key.co.uk/en/key/multipurpose-storage-lockers, but you should take a look in your server logs (this is a good place to start - https://builtvisible.com/log-file-analysis/) to see if these are being hit by Googlebot.
Just trying to figure out what you're asking so I can try to help!
Hey Mirabile, it's a good question and definitely fun to think about. Honestly, I think it's going to be a bit like "Mobilegeddon" last year which ended up being a whimper at the time, but has set Google up to do this. They've been moving in the mobile direction for quite a long time, and this is a further step.
Unfortunately, we don't yet know how all this is going to work. I think we can be certain that Google doesn't want to make their search results worse by hurting e.g. large companies that deserve to rank simply because they move at glacial speeds (super slow) and don't have a mobile friendly site yet. I think we'll also see that in verticals that have way less mobile traffic (eg very B2B niches) there will be much less of an effect.
If it's anything like Mobilegeddon, we'll only really see the effect a year-ish on as they slowly crank up the dial. Specific questions like which content will be used for ranking, how important internal links become, and all of that can only be answered after the fact.
That said, I'll be watching all of this closely 
David -
Thanks for your question, and it's one I see often. I would say this is a much bigger question than "subdomain v subfolder", but really the ability to affect your own SEO.
In direct answer to your questions:
Point 2 brings up the bigger question of if you should host your blog on Medium. While it is indeed a beautiful platform and writing on it is a joy (I actually do a lot of blog drafting in their editor), you don't have control over a lot of things such as:
If you are concerned about the SEO implications (as you seem to be and should be), I'd definitely recommend investigating a self-hosted blog platform like WordPress instead of Medium.
Good luck!
I definitely agree with Logan. The max for an XML sitemap for Search Console is 50,000 URLs, so you won't be able to fit all of yours into one.
That being the case, divide them into different sitemaps by category or type, then list all of those in one directory sitemap and submit that. Now you can see indexation by page type on your website.
Finally, I have to ask why you are doing this with a third party tool and creating a static sitemap as opposed to creating a dynamic one that can update automatically when you publish new content? If your site is static and you're not creating new pages, then your approach might be ok, but otherwise I'd recommend investigating how you build a dynamic XML sitemap that updates with new content.
Cheers!
Hi Becky! Thanks for your question!
I wouldn't be so concerned about what that anchor text says as the fact that your filters are being crawled and indexed. This can be a huge duplicate content issue. I wrote about it here a few years ago - http://www.johnfdoherty.com/facets-filters-considerations-ecommerce/
You should investigate if this is causing more pages than you have on your site to be indexed, which can be quite bad for your site's organic traffic. Sounds like you're a pretty good SEO already, so let me know if you need some pointers on doing this!
Hi Leggo, thanks for the question!
Your current setup is definitely not optimal for SEO, as you are discovering. There are a few things that you can do to help yourself out here. First, there are three good resources here on Moz that you should check out (the last 2 done by me in 2012, but still very relevant):
On to your specific questions.
You ask how to set up your internal linking. Your current setup of linking to just 20 pages internally is definitely not helping you out because you are effectively just telling Google about those 20 articles and not the longtail of articles that you have. XML sitemaps are for discovery purposes for Google, not ranking. An HTML sitemap is a must for you if you have a lot of content and can also help with discoverability and a bit with rankings as well, but it's definitely not all you need to do.
If your content is structured well (in categories that make sense, tagged, etc), then you can set up a system (all depends on your CMS how this is configured) that links similar articles with other similar articles. Some sites have done smart things with linking to specific articles that have more potential search volume, and deprioritizing the number of internal links to articles that don't have much potential.
In the latter case, eventually you should audit if you should remove those lower volume or quality pages to help your site overall. You say that it is indeed the case that you have these pages. I would at first not put them in the sitemap. Then, decide if you want to a) update them to make them quality or b) remove them. That's an internal decision based off of current available resources (time, people) and opportunity. I've done both in the past (updated or removed) and both can definitely help. It just depends on your overall strategy for your site!
Good luck!
Hey Joseph -
Good question here. It's pretty impossible to say what percentage better a subfolder (/blog/) is than a subdomain (blog.mycompany.com), but there have been many case studies done over the years that it is better for SEO overall. At my old company, we moved from a subdomain to a subfolder and sorted out some technology (went to WordPress from TypePad), usability, and internal linking issues. Over the course of about 3 months we saw a great (200%+) increase in traffic. BUT, we also had a lot of content that should have been ranking, were cranking up the quality and quantity of content we put on there, and actively improving our site's overall SEO. These results not typical, your mileage may vary, etc.
I think in your case, I would say that you need to weigh the pros and cons of the effort required to make this work technically with the focus on creating amazing content that gets shared and linked to. I think often we focus too much on smaller technical things like subfolder v subdomain, while neglecting the real reason we are blogging/creating content (to build an audience).
I don't necessarily think this is a make it or break it thing for you, but if you have the chance to put it on a subfolder and it's not too challenging technically (I'm not familiar with the ins-and-outs of this with HubSpot), I'd recommend a subfolder like mycompany.com/blog/
Good luck!
Hi Steven -
Great question here and one I have heard a lot over the last few years. A few questions for you:
In the good old days of SEO, it was quite common to see a page jump in rankings (like you seem to have seen) from building some links. I've seen it happen within a matter of hours, actually. But things have changed (probably to keep people from gaming the SERPs too hard) and sometimes it can take weeks or even months for links to have their full effect on your rankings.
So I'd check that the links are indexed and that you didn't change anything on your page to affect your rankings. Then, I'd make sure that your page is structured to meet your user's needs (eg if it's an informational query as evidenced by what else is ranking, don't have a super transactional page). Then, build more links and keep calm and carry on.
I hope that helps!
John
Thanks for the plug, Anthony! I'm the founder of Credo (I feel like I'm on ProductHunt right now!) and always happy to answer questions and point people in the right direction for which agencies to contact!
This is definitely a good answer and the way to be thinking about it. You're never going to outrank Z/T/HP/RDC based off domain authority if you're a small site. First you need to have your onpage SEO absolutely dialied (H1s/H2s/keyword mentions (in this case address)). Then as Matt suggested pull in other content that is related to that listing or area, such as pricing and information about other homes selling or being listed in that area. The more of this rich content you have, the more you will be able to compete. But as others have said, you also need to get more links. If I were you, I'd find aggregators that you can form partnerships with as the originator of the content.
Qualifications: I ran SEO at HotPads and Trulia Rentals for a while
The solution completely depends on what your goals are. Are you investing in those tag pages? Is there a reason to keep them around?
If you noindex as the others here suggested, you're not only not going to rank with that page, you probably won't with the other. I'd decide what you want to support (it seems categories) and redirect the tag pages. Second best option is a canonical, but then you still have the support the tag pages, the search engines might not respect it, and you're putting a bandaid on a bigger wound.
Hope this helps you think through it!
I am also not familiar with the plugin. I use the Redirection plugin (that's its actual name!) for WordPress redirects.
Can you not specify the full URL? Otherwise, can you use /news/www.website.com because that is the path after your website.com actual URL? From the screenshots of the plugin it looks like that might work...
Hi Terrysun -
It's a good question to be sure. Why not have separate pages targeted at the different keywords? Since you have both iOS and Android apps, your homepage can be targeted at [Property Inspection Software] and then you can use site.com/ios-property-inspection-app and site.com/android-property-inspection-app to rank for those specific keywords. If you want, then have a page a level higher at site.com/property-inspection-app to target that specific keyword.
Hope this helps!