Welcome to the Q&A Forum

Hi there,

I have a robots.txt query which I haven't tried before and as we're nearing a big time for sales, I'm hesitant to just roll out to live!

Say for example, in my robots.txt I disallow the URL 'example1.html'.

In reality, 'example1.html' 301s/302s to 'example2.html'. Would the robots.txt directive also apply to 'example2.html' (disallow) or as it's a separate URL, would the directive be ignored as it's not valid?

I have a feeling that as it's a separate URL, the robots disallow directive won't apply. However, just thought I'd sense-check with the community.

Google's announcement last month about depreciating their AJAX crawl directive and Distilled's recent article have got me thinking a lot about how this change impacts frameworks like AngularJS.

For those of you that use or are considering using frameworks like AngularJS, does this change impact you? Has it changed your mind about services like Prerender etc?

All discussions relating to AJAX crawling welcome. Some resources to get started:

https://prerender.io/js-seo/angularjs-seo-get-your-site-indexed-and-to-the-top-of-the-search-results/

https://www.distilled.net/resources/prerender-and-you-a-case-study-in-ajax-crawlability/

To add to this, I'd also recommend having a look around in /lib/ just to make sure you aren't blocking important javascript and css files (I've been bitten by this!).

More guidance here: https://developers.google.com/webmasters/mobile-sites/mobile-seo/common-mistakes/blocked-resources?hl=en

Hi Ryan,

Thanks for the great resources. I was just wondering if you had any tips on how best to set up Fiddler to monitor for code changes?

Thanks,

Holly

Hi all,

I was wondering if you have any recommendations for a code monitor? We'd like to keep track of any code and content changes on a couple of websites. We've taken a look at Page Monitor: https://chrome.google.com/webstore/detail/page-monitor/pemhgklkefakciniebenbfclihhmmfcd?hl=en but I'm not sure if it tracks code changes? Any suggestions for free or paid tools would be appreciated.

Edit: We'd also like to avoid a tool that requires any tracking code changes or anything that involves a database/FTP connection.

That's really interesting - I hadn't thought of the impact to the Ukraine, but that definitely makes sense.

I think as you've highlighted, Google and Europe more generally is a tricky issue at the moment, what with the recent issues in Spain and the not-very-helpful suggestion that Google should 'break up'. In the UK, the chancellor has just announced a 'Google Tax' which will no doubt go down as well as the legislative changes in Spain.

This article has a kind of doomsday scenario where Google pulls out of Europe, but I can't really foresee that happening. What do you reckon? It does worry me the way Google and Europe are colliding so much at the moment.

We've all seen the news recently that Google will be closing its engineering offices in Russia due to new data protection laws coming into place in January 2015. The same law has also led to Adobe pulling out of Russia earlier in the year.

I was wondering how you think this will impact SEOers and small/medium businesses that market _to _Russia, but are based outside of the country? Personal data has been defined in the new legislation as:

• Personal data means any information directly or indirectly related to any identified or potentially identifiable person. It includes, among other things, first name and family name, date and place of birth, address, information about family status, education, profession, income

Source

For those businesses which don't process personal data (affiliates etc), will there be any foreseeable impact?

On the flipside, are there any benefits here for affiliate businesses inside of Russia? I'm using affiliates as an example to get the ball rolling, but I'm sure there's numerous more. Personally, I'd be interested to hear if you think this may impact corporate websites which don't process personal data, but operate outside of Russia.

Hi there,

I was wondering if anyone had any guidance on integrating Dynamic Remarketing with Universal Analytics? Integrating standard remarketing into the tag is pretty straight forward:

But there seems to be a bit of mixed advice on integrating dynamic remarketing with Universal Analytics. As far as I can see, this is the guide specific for UA users - https://support.google.com/analytics/answer/3455600?hl=en-GB

In terms of specific examples, we're looking for an ecommerce/retail integration, so the code example most relevant for us is:

I have set up the e-commerce custom dimensions within Google Analytics, but I was wondering if the ga('set', 'dimension1')... examples are the best method for integrating dynamic remarketing with Universal Analytics?

This is really helpful - thank you.

Something I am struggling with is finding a solid tool that detects if text is in simplified or traditional Chinese. I must admit, I'm not too familiar with either to know the nuances of each one, aside from a few obvious markers like strokes. Google has the best translate tool I know of, but it just translates into 'Chinese'. Are there any language detection tools you'd recommend for detecting if text is simplified or traditional?

Hi guys,

I was wondering if you could help me with an SEO query for language usage in Hong Kong? Specifically, I'm aware that in mainland China it's preferred to use simplified Chinese.

However, in Hong Kong, if you want to rank well in Google and Yahoo! HK, should you be use traditional or simplified Chinese in your web content?

Any guidance would be much appreciated.

Glad to hear those rankings are improving! I'd be really keen to hear how you get on, as I really can't see anything obvious as to why those wouldn't be ranking, aside from looking into the site structure more and use of 'cortinas' anchors.

Looking forward to seeing the site ranking well in the future!

Hi there,

When did you join Moz? On the top left of your charts you should be able to switch to monthly view which incorporates your data since joining (screenshot: http://postimg.org/image/zayivcumt/). However, if you're new to Moz, it doesn't pull in a whole lot retroactively unfortunately.

Unfortunately, this issue has been up for debate over the past few years and there's no clear-cut answer. You might like to take a look at this staff-endorsed answer from 2012, which indicated that Google will look at how much unique content is above the fold. If your content is buried deep in the source code and duplicate copy comes first, then this _could _be an issue.

If you can place your unique content higher up within the source code, then this wouldn't be a bad thing, but I have suspicion that chances are there's always more pressing things to optimise.

Sorry there's no '100% yes!' on this, but IMO this has always been one of the vaguest areas of SEO; especially as we know G has become more adept at crawling in the past few years (javascript support, etc).

Hey,

I don't think I can answer this one for you completely, but having a look around the site, I can hopefully get some ideas flowing for you?

There's quite a few different pages targeting cortinas; using site:kaaten.com, I can see that there's this page which includes a KW-rich link to the homepage using the phrase 'cortinas'. Then there's also this page, which seems to serve a similar purpose?

I must admit, I don't know the cortinas/curtains market too well, but I have a suspicion that the way the site is structured might be worth looking at. There's a lot of great content, but perhaps some of it could be consolidated.

How are your rankings for other product related terms? E.g. specific types of cortinas

Hi there,

I'm not hugely familiar with Adobe BC, but on slide 30 of this Adobe BC presentation it suggests there's a bit of a hack to implement rel="canonicals". There's also a bit of discussion here, but must admit I know Adobe Analytics better than I do Business Catalyst.

Unfortunately, as you suggest it doesn't look like those 301s are coming through; they're both returning a 200 status code. I don't really want to recommend using the URL removal tool in Google Webmaster Tools, as this is quite a severe solution and doesn't really address the underlying issue of these redirects not working.

When you implement 301s via htaccess, there's no rule of too many 301s; you can redirect entire sites with 1000s of pages with no problem. In fact, based on what you've suggested this sounds like this is an Adobe Business Catalyst issue. There's a discussion going on over in the Adobe forums about redirecting uppercase URLs to lowercase, and while I can't say I massively agree with their view of this not being an issue (it can be), they seem to be suggesting your best bet would be to implement rel="canonical". So, based on this, I'd have a look at using rel="canonical" within BC. It doesn't look like it's supported out of the box, but the Adobe Discussion team may be able to help you.

Sorry there's no hard and fast answer for this; it seems like the issue you're facing is a mix of Adobe BC, 301s behaving differently on the platform (compared to normal .htaccess) and the fact that canonical functionality isn't baked into the platform. I don't envy you with this one!

Yep, that's perfect! Let us know how you get on.

Hey,

I'm not sure if this is resolved for you, but as you suggest you can do something with robots.txt. Specifically, you could use wildcards to capture these URLs and tell Rogerbot (Moz's crawler) to ignore them. Here's a great Stackoverflow query to get you started and details on how block Rogerbot, you can take a look here.

Hi there,

Even though you don't have that page, Google is still indexing it, because it returns a 200 status code - it doesn't 301 to the lowercase version. This means that even though you haven't created it, the website is still returning /SERVICES/ as individual web pages, separate from the lowercase ones. Even though the URL seems wrong to you, to Google, they're being told by the website 'hey, you should index these /SERVICES/ pages', due to the 200 status code.

Using the canonical you suggest would tell Google that the subfolder they should index is SERVICES. This would mean Google would drop (if it follows the canonical rule correctly) all lowercase iterations of /services/ and instead index /SERVICES/. So therefore, you do not want to add a canonical with /SERVICES/, as this would not get rid of /SERVICES/, it would tell Google that **this (/SERVICES/) is the version you want indexed. **

You should add a canonical tag to all of your /services/ pages indicating that the correct canonical version is /services/ - with the lowercase. This should then propagate to your /SERVICES/ pages, indicating the version you want Google to index is the lowercase version; this will get rid of /SERVICES/.

Alternatively, if this isn't clear for you, I would recommend you set up a 301 rule to redirect all versions of /SERVICES/ to /services/.

Hope this helps.

Hi there,

It's no problem at all!

If you want to get rid of the capitalisation, you would switch the order around, so on the /SERVICES/ pages, e.g. /SERVICES/wordpress-development-india.asp use the rel=canonical:

This tells Google that the preferred URL to index is the non-capitalised one. Here's some more details from Google on this:

Indicate the preferred URL with the rel="canonical" link element

Suppose you want http://blog.example.com/dresses/green-dresses-are-awesome/ to be the preferred URL, even though a variety of URLs can access this content. You can indicate this to search engines as follows:

• Mark up the canonical page and any other variants with a rel="canonical" link element.
  Add a  element with the attribute rel="canonical" to the  section of these pages:

  This indicates the preferred URL to use to access the green dress post, so that the search results will be more likely to show users that URL structure. (Note: We attempt to respect this, but cannot guarantee this in all cases.)

Source

So, to recap, if you want to get rid of the capitalised subfolders, on the /SERVICES/ pages use a rel="canonical" referring to the non-capitalised (/services/) versions. If it's the other way around and you want to **keep **the capitalised pages, then use a rel="canonical" with /SERVICES/ in on the non-capitalised pages.

Sounds good! Glad to hear you got a solution sorted. Will be interested to hear how it goes.