Not getting foreign characters in crawl diagnostics .csv

[trainSEM]

The crawl diagnostics .csv file is showing high-ascii characters instead of the correct language (foreign language website) e.g. Vietnamese, Chinese (both kinds), etc. Is there a way to get this right?

[LynnPatchett]

Hi Ash,

I had this problem too and here is how I solved it (there might be better ways).

If the characters are in the page titles, meta tags etc you can open the csv file in open office and then choose save as xls and it will save an excel file which you can then open in excel and the utf8 characters will read ok. This method works great for titles etc but does not decode foreign characters in the urls themselves.

If the characters are in the url then a way I have found is to download this pretty awesome excel addon (site is in german, I used google translate to figure out what was going on). Then you have some new functions in excel where you can create a 2nd column next to the url column, apply the url decode function to the first column and get readable urls in the second. This addon saved me sooo much time and trouble! It works for greek which I need it for, I assume it will work for chinese also. Let me know if you need more detailed instructions, it took a bit of trial and error to figure out the exact moves needed to get the results you want.

Hope that helps!

[trainSEM]

Open Office did the trick! Thank you. Would be nice if the Moz app could do UTF-8 natively.

[LynnPatchett]

Glad it helped! I think the issue might be with excel more than Moz, its handling of utf8 csv's has been terrible since day 1! I think there is a way you can use the excel import data function to get the same result but I never had much luck with it and the open office trick seemed less painful.