Long URLs due to foreign characters

[RyanKent]

I have a site which provides forum sections for various languages. When foreign characters are used in the post title, each letter is replace by a three character replacement such as %93. This conversion makes the URLs long.

The site's software automatically uses the thread's title in the URL. It is never a problem except in these instances.

Any suggestions on how to handle this issue?

[blu42media]

Hey Ryan-

What software are you using?

Depending on your coding experience, you may be able to set up replacements for the foreign characters and override the URL generating function.

Just let me know, I may be able to help you out.

-John

[RyanKent]

XenForo is the forum software in use.

I was really wondering what type of replacement process would be used?

When Google crawls a russian or korean site, do they convert the characters? If not, is there a way of telling Google "hey, this title is from the Russian forums so please use the Russian alphabet?"

If they do still convert the characters, how do other countries handle this change? The title length would be reduced by two-thirds.

[blu42media]

You should have a meta tag for the page language (adjust language code as needed):

As far as the URLs go... many sites are converting these to non-escaped variants on save.  Magento, for example, treats e, é, and ê as e in the url.  Check out Lemonde.fr, french news source.  They are just stripping the accents as well.

To adjust for the accents, you would need to transliterate them.  First, find the function that is generating the URL.  Next, if your system allows has the iconv() function:

$new_url = iconv('utf-8', 'us-ascii//IGNORE//TRANSLIT', $old_url);

If not... then you could go this sort of route:

$table = array(

'Š'=>'S', 'š'=>'s', 'Đ'=>'Dj', 'đ'=>'dj', 'Ž'=>'Z',

'ž'=>'z', 'Č'=>'C', 'č'=>'c', 'Ć'=>'C', 'ć'=>'c',

'À'=>'A', 'Á'=>'A', 'Â'=>'A', 'Ã'=>'A', 'Ä'=>'Ae',

'Å'=>'A', 'Æ'=>'A', 'Ç'=>'C', 'È'=>'E', 'É'=>'E',

'Ê'=>'E', 'Ë'=>'E', 'Ì'=>'I', 'Í'=>'I', 'Î'=>'I',

'Ï'=>'I', 'Ñ'=>'N', 'Ò'=>'O', 'Ó'=>'O', 'Ô'=>'O',

'Õ'=>'O', 'Ö'=>'Oe', 'Ø'=>'O', 'Ù'=>'U', 'Ú'=>'U',

'Û'=>'U', 'Ü'=>'Ue', 'Ý'=>'Y', 'Þ'=>'B', 'ß'=>'ss',

'à'=>'a', 'á'=>'a', 'â'=>'a', 'ã'=>'a', 'ä'=>'ae',

'å'=>'a', 'æ'=>'ae', 'ç'=>'c', 'è'=>'e', 'é'=>'e',

'ê'=>'e', 'ë'=>'e', 'ì'=>'i', 'í'=>'i', 'î'=>'i',

'ï'=>'i', 'ð'=>'o', 'ñ'=>'n', 'ò'=>'o', 'ó'=>'o',

'ô'=>'o', 'õ'=>'o', 'ö'=>'oe', 'ø'=>'o', 'ù'=>'u',

'ú'=>'u', 'û'=>'u', 'ü'=>'ue', 'ý'=>'y', 'ý'=>'y',

'þ'=>'b', 'ÿ'=>'y', 'Ŕ'=>'R', 'ŕ'=>'r', 'Ā'=>'A',

'ā'=>'a', 'Ē'=>'E', 'ē'=>'e', 'Ī'=>'I', 'ī'=>'i',

'Ō'=>'O', 'ō'=>'o', 'Ū'=>'U', 'ū'=>'u', 'œ'=>'oe',

'ß'=>'ss', 'ij'=>'ij'

); $new_url = strtr($old_url, $table);

I'm not sure about Korean handling - perhaps someone else knows how these are being handled?

-John

[RyanKent]

Thank you John.

The solution you offered works if a site is geared for one particular language. The site I am working with has language dedicated forums covering more then a dozen languages. The end solution will need to adjust for all of them.

I will speak to the forum software about your idea and hopefully we can build something off your suggestion. Thanks for taking the time to share your experience.