I am currently working with a codeigniter PHP based application and have come to the point where it's about to go off with multiple languages.
Is codeigniters own language class the most effective way to handle languages?
Is there any specific language-tools/libraries that are commonly used in PHP apps?
Thanks!
I've never used CI_Language but it appears to use language arrays to do the translation.
Overly simplified example of this type method:
$trans = array(
'MAIN_TITLE' => 'Title Here'
);
echo $trans['MAIN_TITLE'];
Personally I find this really annoying because you're then editing views that are cluttered with array key names instead of useful text. Which can be quite annoying at times. Not to mention you have to remember which keys correlate to which strings if you are using them in multiple places.
I use Gettext which I find much easier. You just have to wrap your strings with the translate method: _(). Then once you're done with your app, you open up PoEdit and create the new language file. PoEdit will parse all of my source files looking for strings wrapped like this <?php echo _('Title here') ?> and insert them into the .po language file. You can then go string by string and translate the text easily within PoEdit. The benefit of this is you have the source translation right there within PoEdit, instead of a meaningless array key name in some include file
This all makes my life much easier in that I can update my language files every Friday with one click. Any new or modified translations will automatically be added to my language file, and any unused translations will automatically be removed. I send the files off to my 3 international branchs for translation, and my changes and updated language files are ready to be deployed Monday morning
You may want to have a look into php intl library. http://php.net/intl
Related
I'm writing a webapp in german, so all buttons,text,tooltips etc. are in german for now. But I want to use some kind of template file for the webapp so I can quick change to another language if needed. I thought about textfile that I explode with "\n" and load into a sessionvariable to have always all text the user will need in his session. An other approach would be to parse such a file i.e. a XML document like this:
<?xml version="1.0"?>
<phrase>
<placeholder></placeholder>
<value></value>
</phrase>
where every field has it's own name/value that represents a textsnippet or button or whatever on the website, and then cast it into an object an cache it for everyone. I think the second approach is the best for working with multiple languages for a webapp. Anybody perhaps some pointers what I could do even better, or just post how you did this kind of languagetemplating for mutlinational webpages/webapps in the past.
Since you are looking for a translation solution, I understand you don't use a framework to develop your site, since most of them provide you with solutions to handle translations.
Most frameworks and apps I've seen in php use arrays, where the original sentence is the key and the translation is the value. So, to make easier to translate it to several languages, the key is in english.
In case you use gettext as suggested, or another aproach, it'll be useful also to parse your code to catch all strings to be translated automatically, since it can be a mess doing it manually when the base code grows and you want to keep up to date your translations.
Take a look at GNU Gettext, its very handy for multilanguage support.
The main idea is that you just wrap your words or phrases into a function, like
echo _g('Hello');
so you do not have any engine changes. You will have to add translation files for each language you are using.
You've come up with 2 solutions for storing the data, but I suggest you need to think further about the architecture and take a more complete view of the lifecycle of each request.
Regarding architecture: neither solution scales up to describe an extensive vocabulary very well - although for one or 2 pages it will suffice. The alternative approach, to manage a translation database (such as gettext) which might be overkill - and performs less optimally with small numbers of pages but importantly performance does not deteriorate significantly with large/multiple dictionaries. A compromise solution might be to have a dataset for each URL/language (which might be extracted from a consolidated database).
If it were me, I would not use either method you proposed for storing the data: parsing XML creates a sginficant overhead for each page request : using \n as a delimiter precludes the use of \n within a translation. Using a serialized PHP array seems to be the least expensive solution.
I need to translate several PHP files (HTML Code + PHP Tags) into another language.
Google Translator's Kit allows this, but clears the PHP Tags, erases class="" attributes (?!) and adds html, head tags & what not. Completely useless.
How can I (ideally in batch) translate these files using any kind of automated translation service?
Thanks.
bmargulies is the most clear-cut way of doing it. However, it takes times.
If you're in a pinch, or want to cut corners, a relatively simple way to do it is to use regular expressions to filter your code out yourself. Match over multiple lines (/s flag in preg), store the match, and replace with a hash. Any hash. Just make sure it doesn't map to anything in any language.
Do the same for HTML tags if they are proving to be annoying to Google.
Translate with Google.
Replace back the hashes. Voila! Job done! If you're feeling even more daring, instead of replacing the hashes back, replace them with an l18n-suitable structure might prove to be even more worthwhile.
You need to internationalize the code. You need to move all the translatable strings out into a separate file, so that you can shove that through Google and then easily drop in the results.
Researching the topic of PHP I18N will prove rewarding.
Google Translator toolkit is for documents - not so much for source code. You can organize your program's strings as documents and translate them in Google Translator toolkit, and there are, in fact, software projects that do it, but it's contrived. It would be much better to use a different method, as the other people here say.
Put the translatable strings in separate files - you can use something like YAML or JSON, for example, or to just organize your strings as PHP arrays (that's how it's done in MediaWiki, for example). Each message should have a key. Use one file per language or one file with all the languages, and the strings grouped by languages. (By the way, use ISO 639-3 language codes - don't make up your own. Then you'll be able to reuse them in HTML lang attributes.)
After you organized your strings like that, write functions that load the strings from these files by message key and language code, and use these functions to display the messages - never use hardcoded strings.
Finally, put your files up for translation using software such as Pootle, Transifex, Zanata, or the MediaWiki Translate extension.
(Disclaimer: I am a developer of the MediaWiki Translate extension.)
Using something like Gettext (namely php-gettext) is IMHO best approach to do that. Another widely used option is to simply extract strings to separate files (be it PHP or JSON) and translate these. However I'd recommend to use Gettext as you will be using standard format with wide range of available tools.
I did some search about localization files, see that we can use .po file or gettext but, is there any tutorial or sample of a unique solution that will work both in javascript and in php.
I want to only maintain one localization file per language that will work with both JS and PHP languages.
Would appreciates if someone can point me to some links or samples...
I found that it is typically a sign of a questionable design when translatable text is coded inside JS functions. JS is meant to implement logic, not content. The content should be provided by PHP (typically by using a templating engine) and should be used by JS. That way you only need a localization solution for PHP.
If (exceptions always occur) you really need to translate a phrase inside a JS routine you use an ajax call to fetch the translation. This also simplifies the access to the dictionary holding the translation tokens since it is again done by PHP. The dictionary can be kept in a single place.
Yep, there is. I've successfully used gettext.js a while ago, which is operating on .json or .po files. This way, you only have to maintain one translation source. The webpage I've used this for is located here so you can check out how I've did it. Good luck!
First, try to avoid gettext if you can. It's popular and stable, but it puts some burden on the translations maintenance: you will need to change the source strings, and when this happens, gettext needs to update all the message keys, and this may mess up the existing translations. An approach with constant message keys is much easier to maintain in the long run - you will need to remember to delete the keys you don't use any more, but it's a very small burden.
You can use the same translations storage format for PHP and JavaScript. If you use a key-based approach, as I suggest, it will probably be some JSON-based format. JSON is easily accessible in both PHP and JavaScript.
There are several ready-made JavaScript libraries for JSON-based internationalization. I happen to be a developer of one such library: https://github.com/wikimedia/jquery.i18n . It should be reasonably easy to adapt it to PHP.
We are in the process of making our website international, allowing multiple languages.
I've looked into php's "gettext" however, if I understand it right, I see a big flaw:
If my webpage has let's say "Hello World" as a static text. I can put the string as <?php echo gettext("Hello World"); ?>, generate the po/mo files using a tool. Then I would give the file to a translator to work on.
A few days later we want to change the text in English to say "Hello Small World"?
Do I change the value in gettext? Do I create an english PO file and change it there?
If you change the gettext it will consider it as a new string and you'll instantly loose the current translation ...
It seems to me that gradually, the content of the php file will have old text everywhere.
Or people translating might have to be told "when you see Hello World, instead, translate Hello Small World".
I don't know I'm getting confused.
In other programming languages, I've seen that they use keywords such as web.home.featured.HelloWorld.
What is the best way to handle translations in PHP?
Thanks
You basically asked and answered your own question, the answer might just be having a slightly better understanding of how PO files work.
Within the PO file you have a msgid and a msgstr. The msgid is the value which is replaced with the msgstr within the PHP file depending on the localization.
Now you can make those msgid's anything you would like, you could very well make it:
<?php echo _("web.home.featured.HelloWorld"); ?>
And then you would never touch this string again within the source, you only edit the string through the PO files.
So basically the answer to your question is you make the gettext values identifiers for what the string should say, however the translators typically use the default language files text as the basis for conversion, not the identifier itself.
I hope this is clear.
I know an answer has been accepted, and the above answer is good. But there is another issue with using permanent machine-style keys like thing.stuff.widget when working with Gettext.
While using permanent keys is a better approach to development, Gettext is not set up for that style of working and this can complicate your workflow.
If you present a translator with a PO file populated with keys in place of source text, they may not know what the English should be. So you'd have to provide them with a second file containing source language translations for them to compare to. Not the end of the world, but more fiddly for them and not how Gettext was designed. (square peg, round hole etc..)
I think PO is perfectly fine as a file format for translations in PHP, and especially recommended if you're not working with a framework that has a good l10n module, but that doesn't mean it's good for workflow and your translation process.
I suggest you arrive at a workflow that allows your programmers to work with permanent keys, your translators work with words, and gives you a MO file out the other end. Take a look at Loco for one solution to this.
Alternatively use a different interim file format that allows the use of keys and words. TMX is one example. If you still want to use Gettext at runtime you can convert the files.
Currently, I am dealing with the same issue. The common practice with gettext is to use the English text as the key. Recently, our copy editor changed whole bunch of English text (other languages are hardly touched) so we have to change all the source code all the PO files.
We are switching to a neutral key. Since we already have some sites on Java. We will use the same property name format.
I am working on a large multilingual website and I am considering different approaches for making it multilingual. The possible alternatives I can think of are:
The Gettext functions with generation of .po files
One MySQL table with the translations and a unique string ID for each text
PHP-files with arrays containing the different translations with unique string IDs
As far as I have understood the Gettext functions should be most efficient, but my requirement is that it should be possible to change a text string in the original reference language (English) without the other translations of that string automatically reverting back to English just because a couple of words changed. Is this possible with Gettext?
What is the least resource demanding solution?
Is using the Gettext functions or PHP files with arrays more or less equally resource demanding?
Any other suggestions for more efficient solutions?
A few considerations:
1. Translations
Who will be doing the translations? People that are also connected to the site? A translation agency? When using Gettext you'll be working with 'pot' (.po) files. These files contain the message ID and the message string (the translation). Example:
msgid "A string to be translated would go here"
msgstr ""
Now, this looks just fine and understandable for anyone who needs to translate this. But what happens when you use keywords, like Mike suggests, instead of full sentences? If someone needs to translate a msgid called "address_home", he or she has no clue if this is should be a header "Home address" or that it's a full sentence. In this case, make sure to add comments to the file right before you call on the gettext function, like so:
/// This is a comment that will be included in the pot file for the translators
gettext("ready_for_lost_episode");
Using xgettext --add-comments=/// when creating the .po files will add these comments. However, I don't think Gettext is ment to be used this way. Also, if you need to add comments with every text you want to display you'll a) probably make an error at some point, b) you're whole script will be filled with the texts anyway, only in comment form, c) the comments needs to be placed directly above the Gettext function, which isn't always convient, depending on the position of the function in your code.
2. Maintenance
Once your site grows (even further) and your language files along with it, it might get pretty hard to maintain all the different translations this way. Every time you add a text, you need to create new files, send out the files to translators, receive the files back, make sure the structure is still intact (eager translators are always happy to translate the syntax as well, making the whole file unusable :)), and finish with importing the new translations. It's doable, sure, but be aware with possible problems on this end with large sites and many different languages.
Another option: combine your 2nd and 3rd alternative:
Personally, I find it more useful to manage the translation using a (simple) CMS, keeping the variables and translations in a database and export the relevent texts to language files yourself:
add variables to the database (e.g.: id, page, variable);
add translations to these variables (e.g.: id, varId, language, translation);
select relevant variables and translations, write them to a file;
include the relevant language file in your site;
create your own function to display a variables text:
text('var'); or maybe something like __('faq','register','lost_password_text');
Point 3 can be as simple as selecting all the relevant variables and translations from the database, putting them in an array and writing the serlialized array to a file.
Advantages:
Maintenance. Maintaining the texts can be a lot easier for big projects. You can group variables by page, sections or other parts within your site, by simply adding a column to your database that defines to which part of the site this variable belongs. That way you can quickly pull up a list of all the variables used in e.g. the FAQ page.
Translating. You can display the variable with all the translations of all the different languages on a single page. This might be useful for people who can translate texts into multiple languages at the same time. And it might be useful to see other translations to get a feel for the context so that the translation is as good as possible. You can also query the database to find out what has been translated and what hasn't. Maybe add timestamps to keep track of possible outdated translations.
Access. This depends on who will be translating. You can wrap the CMS with a simple login to grant access to people from a translation agency if need be, and only allow them to change certain languages or even certain parts of the site. If this isn't an option you can still output the data to a file that can be manually translated and import it later (although this might come with the same problems as mentioned before.). You can add one of the translations that's already there (English or another main language) as context for the translator.
All in all I think you'll find that you'll have a lot more control over the translations this way, especially in the long run. I can't tell you anything about speed or efficiency of this approach compared to the native gettext function. But, depending on the size of the language files, I don't think it'll be a big difference. If you group the variables by page or section, you can alway include only the required parts.
After some testing I finally decided to go more or less with the lines of Alecs' combination of the second and third alternative.
Gettext problem
I tried to set up the whole gettext-system first to try it out, but it turned out to be much more complicated then I thought. The problem is that Windows and Unix systems use different language shortnames for setlocale(). For the moment I'm running my dev-server on Windows with Wamp, while the final site will run on Linux. After I went through a couple of dozen guides, forums, questions etc. and restarting the server after each modification. I couldn't get it setup properly in any easy way it seemed. Additionally gettext is not threadsafe, to update the language file the server needs to be restarted or a hack needs to be used, there is no easy way of handling different versions of language files or handling the original English text without modifying the source or using Mikes suggestion, which as Alec pointed out isn't optimal.
Solution
So I ended up with what I think is the best solution based on Alecs response:
Save all the translations in a DB with the fields; language, page, var_key, version, revision and last_modified_time - where the version is corresponds to versions of the original translation (English), while revision allows the translator to modify/correct the finalized translations within a version.
Use a kind of CMS for translation, which is connected to the DB and handles different versions and allows for an easy overview of which languages are translated, in which version and how complete the translations are.
When a revision of a version is finalized a cache files are generated - each file contains an array with only the var_key and text-translation for one language and one page and are named with the ISO 639-1 names of the languages and the page name like: lang/en_index.php These language files are then simply included and wrapped in a function t($var_key) which allows for using the DB during the development, while then changed to only use the cache files.
Performance
I never got around to test gettext, but according to the link Mike posted the difference in performance between using an array and gettext is totally acceptable for me for the benefits which a custom system gives as described above. However, I compared using an array with 20 translated text-strings in an array compared to retrieving the same 20 text-strings from a MySQL DB. It turned out that using an array included from a file was aeound 6 times faster than retrieving all the 20 strings at the same time from the MySQL DB. It was no really scientific benchmark and the results may surely vary on different systems and setups, but it clearly shows exactly what I expected - that it would be much slower using a DB than using an array directly, which is why I choose to generate cache-files for the array instead of using the DB.
As a comparison I also tested how fast it was to only output simple echos with the same text. It turned out to be around 20 times faster than using arrays from an included file, but well - then it is not possible to translate without having different versions of the page for different languages, which defies the purpose of dynamic pages. Then it is better to also use a good cachesystem.
Performance test source files:
PHP: http://pastie.org/964082
MySQL table: http://pastie.org/964115
It is surely not perfect, but at least creates an idea about the performance differences.
Rather than having to use the English text as the keys you could arbitrarily do this but also provide english translations i.e.
gettext key is 'hello'
You then have your various language translations of this and an english translation of this that is also 'hello', then if you want to update the english version of the string you can leave the key alone and just update the english translation.