Welcome to the Q&A Forum

Campaign Monitor is good, but have always preferred Mailchimp. The industry standard I would say, free up to 2000 subscribers, loads of integration tools available for social and ecommerce setups and great reports and analytics integration.

Hi Eduardo,

For the titles this is probably due to google rewriting page titles based on brand searches. They have been experimenting with various ways of displaying titles in the serps for branded searches and if you are searching for 'jobsandtalent' with no spaces then this is a pretty specific search and google is rewriting you title based on it. If you search for your whole page title + brand you will see the normal title as expected. It does not have anything to do with Ruby on Rails.

As for the page rank, this is not a number I place much importance in. I cant remember off hand how often it is updated but it is not all the time. More to the point to be looking a moz domain and page metrics if you ask me. That being said I see your pr as 5 for the root domain www.jobandtalent.oom.

I noticed you seem to be using cookie based redirects from the main domain to the language folder so that if you have entered /es once then going to the .com main page automatically pushes you to .com/es. This can potentially be problematic in terms of google properly indexing you site. I cannot say if this is responsible for your difficulties in rankings but in a competitive sector like job postings I would certainly look changing that so that google (and users) can view all pages of the site in whichever language they choose without being pushed into a language based on cookies.

Hope that helps!

Hi Tim,

Glad it helped. It might be worth asking your host what kind of features they have for preventing flooding attacks, there are various ways of addressing them on the server side that most hosts will have enabled in one way or another. Unless you have a specific issue with these kind of attacks, it seems to me that this part of the module is causing more harm than good as it is now.

Hi,

I believe the sitelinks shown are dependent on what main page is being shown in the serp results so maybe you have demoted the link for the homepage but not for some other page that is being brought up for specific searches? If so you can add the specific page that is bringing up the result in the GWT sitelinks admin page and then demote the link you do not want for this page also.

If this is not the case worth making sure you have not made a typo or anything. You should see a list of demoted links and the dates they are valid to. When I have demoted links it took a few days to filter through but certainly not a month. Might also be worth looking at where this page is being linked to in your site, if it is a non important page but has a 'prominent' sitewide link on all pages then this can lead google to conclude it is an important page and if it is being linked to from outside domains this can be reinforced even more.  Linking to it from less of your internal pages can sometimes help reduce its importance, but really the GWT demotion tool usually works pretty well so I would check the homepage vs internal pages and any typo issues first.

Hope that helps!

Hi Tim,

Did you ever get to the bottom of the issue mentioned in this question? It is almost certainly the same problem.

Have a look at this page and try either turning of the sh404SEF anti flooding feature or else boosting the max number of requests allowed. http://forum.joomla.org/viewtopic.php?p=1368937

The anti flooding part of this component is basically blocking requests for pages if it thinks someone is trying to do a dos attack on your site. The current setup seems to be too sensitive and is bloocking screaming frog after the first few requests, quite possibly blocking the google bots, maybe blocking the moz crawler also, so certainly something you should address.

Hi,

Technically if your cms and server can handle incoming urls with utf8 characters in them then you should be ok. I have seen some instances where the setup does not like them and produces 404 errors when you try to include these characters in the url, but most times it is fine. Google will display utf8 urls in the search results without problem (check out the wikipedia result here)

There is a further consideration though which is one I face a lot with Greek (similar to russian in some ways) which is how these urls are shared in mails, social media etc. A lot of the times these urls end up getting automatically url encoded and the url shown in the mail or facebook etc is a long, long string of url encoded characters which is impossible to read, gives no indication of what the page is about and generally looks bad (try putting that link above into facebook....nasty). For this reason I usually choose to do 'greeklish' urls which is a latin character representation of the greek characters. There are usually some common practices in regards how the local language is 'recreated' in latin characters, there are for Greek, I would assume there are for Russian also. So with that in mind, if you have a russian speaker who is familiar with that kind of thing I would be inclined to make the urls themselves 'russianlish'.

My two cents!

Hi Barb,

I am sure Joel will chime in also but just to clarify that it is probably not the utf8 encoding or lack of it that is causing the issue. At least with the sitemap urls it is simply the formatting of the xml that is being produced. As to if the other errors you are seeing are caused by the same kind of thing, if you are seeing references to the same encoded characters (%0A%09%) then the answer is most likely yes.

So the issue is not utf8 encoding related (there are plenty of non utf8 encoded sites on the web still!) but how the moz crawler is reading your links and if other tools/systems will be having the same troubles. Have you looked in google webmaster tools to see if it reports similar 404 errors from the sitemap or elsewhere? If you see similar errors in GWT then the issue is likely not restricted to the moz crawler only.

Beyond that, since for the sitemap at least the fix should be relatively simple and quite possibly the other moz errors you see will also be able to be fixed easily by making small adjustments to the templates and removing the extra line breaks/tabs which are creating the issue then it is worth doing so that these errors are removed and you can concentrate on the 'real' errors without all the noise.

Thats a pretty good video, hadn't seen it before. Check out this article for another good rundown on using rel canonical and prev/next and implementation: http://www.ayima.com/seo-knowledge/conquering-pagination-guide.html using them both alone or in combination depending on the situation is certainly the best way to approach this issue.

Ah ok now I understand, misread it a bit.

Well, 2 ways to do it then:

1. Rel canonical to a 'view all products page', in this case the rel canonical is a valid option.

2. Implement pagination with rel next/prev - this will work also.

The preferred option would usually be the first, but this does mean that search visitors would normally be landing on the all products view. Depends on how many products each brand has to a degree, how user friendly seeing all the products together is, page load times etc.

Check out this page for a good rundown on the options and implementation.

Hi,

I'm not sure I understand your page/structure setup. It is only one brand's products per page right? You say the cart sorts the brands, but what does this mean in practice? Does http://www.xyz.com/brand.html?page=2 always show the products of one specific brand or can it be different brands depending on the sorting? If the first then you shouldn't have a problem, if the latter then yes this could be a problem.

Regardless adding the rel canonical as you describe it is not the way you want to go. This is in effect saying you have only one page/brand you want to index (whichever brand is on http://www.xyz.com/brand.html).

Hi,

It can be frustrating I know, but if you are methodical you will get to the bottom of all errors and then feel much better

Not sure why the number of 404s would have gone down, but in regards the sitemap itself the moz team might be right that utf-8 encoding could be part of the problem. I think it might be more to do with some non visible formatting/characters being added to your site map during creation. %09 is a url encoded tab and %0A is a url encoded line feed, it looks to me that these are getting into your sitemap even though they are not actually visible.

If you download your site map you will see that many (but not all) the urls look like this:
<loc>http://www.cmswire.com/cms/web-cms/david-hillis-10-predictions-for-web-content-management-in-2011-009588.php</loc>

Note the new lines and the indent. Some other urls do not have this format for example:

<loc>http://www.cmswire.com/news/topic/impresspages</loc>

It would be wise to ensure both the file creating the sitemap and the sitemap itself are in utf-8, but also it could be as simple as going into the file creating the sitemap and removing those line breaks. Once that is done wait for the next crawl and see if it brings the error numbers down (it should). As for the rest of the warnings, just be methodical, identify where they are occurring and why and work through them. You will get to few or zero warnings, and you will feel good about it!

Has this changed recently with the whole moz update? I was under the impression entering a root domain without the www would allow tracking of all sub domains including www., so you could actually track multiple sub domains in a single campaign. Now the help page is a bit unclear on that, above says you can track the root domain if you like, below says you cannot track sub domains...

Indeed it might be better to track the blog. sub domain in a separate campaign, but now I am confused if you can actually track them all in one or not...

Hi Ash,

Take a deep breath

You have a couple of things going on including the change in site structure as well as moz and GWT data telling you various things. so the first thing to do is figure out what exactly each tool is telling you and why. Then you know what you need to fix.

For the moz report, you should check out where the 404s are coming from by downloading the report as csv and opening in excel. Filter by 404 errors and on the left column you have the error page and the far right column the page that linked to it. It is likely you have links to old pages on a page (or pages) of your new site in which case you should adjust these links to reflect the new architecture. That should take care of most of the 404's but of course you need to see the exact pages in the report to know for sure where and why the errors are happening.

For GWT I would think that maybe your old site had a sitemap and the new one does not? I do not see a sitemap at /sitemp.xml. This would potentially account for the 163 submitted but only 2 indexed. Google does not easily forget old submitted sitemaps so if it still has the old sitemap in its 'memory' and the new site only has 2 of the same urls, you would get some numbers like what you see. I would create and submit a new sitemap reflecting the new architecture and see what that does for the numbers. Be patient, it can take a few days or even a week or two for google to completely spider the new architecture. Of course you should also 301 redirect all the old page urls to their relevant new pages if you have not done so already.

Hope that helps! Main thing is to cut the data into more understandable chunks so you can get a better handle on what is going on.

Hi Brice,

What makes you think the issue is that moz cannot read the urls? In the first instance I would want to make sure that something else is not going wrong by checking the urls moz is flagging as 404s, ensuring they actually do or do not exist and if the latter finding out where the link is coming (be it the sitemap or another page on the site). You may have already done this, but if not you can get all this information by downloading the error report in csv and then filtering in excel to get data for 404 pages only.

If you have done this already then if you give us a sample or two of the urls moz is flagging along with the referring url and your sitemap url we might be able to diagnose the issue better. It would be unusual for the moz crawler to start throwing errors all of a sudden if nothing else has changed. Not saying it is impossible for it to be an error with moz, just saying that the chances are on the side of something else going on.

Hope that helps!

Hi Neville,

Article links are pretty low value most of the time, so instead of answering that question can I suggest you look at guest blogging instead which while also sometimes lending itself to low value links and content is at least usually a step up from article spinning sites.

Check out this recent post on guest blogging, lots of good info in there and then rework your articles a bit to be suitable for your selected blogs. You will need to be creative in your blog selection and your communications with the owner to ensure best results but needless to say if the process of publishing the article/post and getting the link does not involve real communication with a real person then 9 times out of 10 that link is going to be pretty much worthless in the best case and potentially penalty inducing in the worst. Also, take it easy on the keyword links! Any decent blog will usually insist on only a branded author bio link, something to keep in mind.

My two cents!

Hi Tom,

I think this is getting into specifics of your particular cms/site setup that we are not going to be able to easily answer.

it seems that your web guys are saying that you can set up a url that will contain /en-us/ by adding that info to the folder field. If this is the case then it should work in terms of giving you the language sub folder in the sense that you do not actually have to have a real en-us folder on your server to have urls like www.example.com/en-us/page.html. It is creating a virtual sub folder so to speak that exists only in the url but not as a real folder. In technical terms, all other things being ok this setup should work as expected as long as you recreate all the relevant pages of the site in the language you want and add the same folder name in that field. I assume you will probably have to edit all the onpage links also to reflect the /en-us/ url.

In term of the TLD/redirect setup again this seems specific to your setup, To be honest it seem like they are suggesting exactly the same thing in this case, I still don't see a reference to a 'real' en-us subfolder. It looks like  your site/cms is not ideally setup to handle languages and also that it is not easy to copy the entire site into a subfolder to have a second language version of the site available in this way. It is impossible to advise on the best way around this without having a much more thorough look at your setup, there are many variables involved.

Glad it helped! I think the issue might be with excel more than Moz, its handling of utf8 csv's has been terrible since day 1! I think there is a way you can use the excel import data function to get the same result but I never had much luck with it and the open office trick seemed less painful.

Hi Ash,

I had this problem too and here is how I solved it (there might be better ways).

If the characters are in the page titles, meta tags etc you can open the csv file in open office and then choose save as xls and it will save an excel file which you can then open in excel and the utf8 characters will read ok. This method works great for titles etc but does not decode foreign characters in the urls themselves.

If the characters are in the url then a way I have found is to download this pretty awesome excel addon (site is in german, I used google translate to figure out what was going on). Then you have some new functions in excel where you can create a 2nd column next to the url column, apply the url decode function to the first column and get readable urls in the second. This addon saved me sooo much time and trouble! It works for greek which I need it for, I assume it will work for chinese also. Let me know if you need more detailed instructions, it took a bit of trial and error to figure out the exact moves needed to get the results you want.

Hope that helps!

Hi Jesse,

If you change the url even a bit and let the old one 404 then you will accomplish what you want in terms of cutting the bad incoming links, so if all of them are poison as you say, then this is probably a logical option. You could also consider a 410 response which might remove the page from the index faster and is considered more permanent (gone forever).

In terms of the incoming traffic I would keep two things in mind.

1. Firstly it would be nice to identify where that direct traffic is coming from as much as possible. Check out this article for a couple of ideas on what traffic might be hiding behind those numbers and apply to your situation as relevant. If you have a couple of days/weeks to be patient you can manually tag some of the likely sources to see what data that gives you.

2. If possible consider making a custom 404/410 page for this instance giving real users a link to the new page. Not 100% sure on the technicalities of how google will assess a link to the new page from the now old page which is returning a 4xx status. You could meta tag the 404 noindex, follow or even noindex, nofollow I suppose to further enforce the disconnect between the old and the new while still keeping the link available for human visitors.

Hope that helps.

Hi Angela,

Officially google recommend that for search results without filters you should use either a rel canonical to a 'view all' results pages OR rel prev/next (not both in other words). Now with filters that changes slightly and then you should implement rel prev/next as normal but also rel canonical each filtered page to its relevant NON filtered page.

So for example you have:

Page is /all/best/2** and **meta data is:

So you rel canonical not to the first non filtered page as you mention but to the relevant paginated non filtered page if you see what I mean. It can get a bit confusing, be sure to check the final results well to avoid unpleasant surprises.

Hope that helps!