GSoC: Implementation of additional i18n features on Django

Here you have my proposal for Google Summer of Code 2009. It was approved previous week, and I'll be working on it during this summer.

The problem

While Django provides an amazing system to translate texts, and displays localized dates in some parts of the admin; it has many data that could be internationalized, not it's not yet.

The information that developers should be able to localize/translate is mainly:

More information on these issues can be found in the following blog post and this ticket:

[http://vaig.be/2008/07/django-i18n-status.html](http://vaig.be/2008/07/django-i18n-status.html)

[http://code.djangoproject.com/ticket/7980](http://code.djangoproject.com/ticket/7980)

Proposal

The proposed solution for improving Django i18n includes several different tasks. Those tasks are:

• Import locale data from CLDR

• Apply i18n to Django dates and times

• Apply i18n to Django numbers

• Allow translating content on the database

• Fix already reported bug about i18n

Next are the details for every task. Note that all those specifications are subject to change, according to discussions with the mentor of the project, Django core developers team, and the main Django community.

Importing locale data

The main repository of locale data is the Common Locale Data Repository (CLDR) by the Unicode Consortium http://cldr.unicode.org/. It provides a set of XML files with information such as date, time and number formatting for most languages.

The idea of this task would be to create a python script (probably as a django-admin command), that will extract all necessary data from those XML files and put it into configuration files on the Django structure. This information will be used by Django to internationalize data on applications.

The idea of this script is to be used just by Django developers. It would mainly be a one-time execution script, and then it would be executed just when new locales are added (are some are changed).

All information gathered from CLDR files could be saved on django/conf/locale/{ language code }/formats/django.po

Specific settings imported from CLDR could be (with English localized example):

• DECIMAL_NUMBER FORMAT (1,000,000.123)

There are some locale based parameters that already exist on Django, on translation files (LC_MESSAGES) and could be deprecated on future releases of Django (when breaking backward compatibility). Those are:

For keeping the system flexible, existing default values on settings will be kept. Probably it would be worth to add new ones for the new customizable formats.

Dates, times and calendar i18n

All dates and times displayed using Django should use the format defined for the current session locale. This is already implemented for some dates, like the ones displayed in admin's lists. Also a filter for formatting dates already exists in templates, which, together with the formats in the translation files, can do the job. But the good way to do that would be displaying the date by default on the session locale.

All Django forms (including admin forms) should accept the short date/datetime format of the current locale. Now it's possible to define the accepted formats using parameters of the widget, and this can be kept, but at least support for entering data formatted in current locale should be added. ISO and/or English locale can be kept as well. Existing data on input fields should be displayed in current locale too.

As Django 1.0 series is maintaining backward compatibility, those changes have to be implemented being compatible with existing behavior by default.

The calendar on admin's date/datetime field should also be displayed according to user session locale.

So basically those are the main tasks required for internationalizing Django dates:

With those changes next tickets would be fixed:

Number i18n

Right now, Django doesn't provide anything for localizing numbers on applications. All numeric values within Django applications are formatted using American formats. Users from many countries are not used to dealing with the American format, and a simple shop using Django can create confusion among users who, for example, expect the comma to be the decimal separator, and they find the point on prices.

As for the previous section, changes must be applied keeping backward compatibility.

So Django should display, and use by default the language of the current locale to format numbers. Basically that means:

With those changes next ticket should be fixed:

Translating dynamic content

Django has an amazing system for translating texts to any language. The only problem of this system is that it can just be used for static content (defined on source files, including templates), and not for dynamic content, created by users after deploying the web site. This can be useful for many different situations like an application that has a product catalog where product names and descriptions have to be translated, or a news website, where news can be translated to any language.

There are some external applications, widely used, that allow to do that on Django, but all of them have many different problems, like complex and tricky syntax for developers, ugly interface for users, bad design, bad scalability... Main applications are:

A comparison of the two first applications, and some ideas for a better solution, can be found on a presentation at

[http://docs.google.com/Presentation?docid=dfbzs3ks_7f2z85hvr&hl=en](http://docs.google.com/Presentation?docid=dfbzs3ks_7f2z85hvr&hl=en)

Basically, a good solution to allow Django developers to translate their models should include:

To achieve those targets a lot of analysis is required, so, just some ideas are detailed here.

For the model syntax there are many different options, some of them can be checked on this blog post, and this poll:

[http://vaig.be/2009/03/django-multilingual-syntax-poll.html](http://vaig.be/2009/03/django-multilingual-syntax-poll.html)

[http://doodle.com/aicvayf8ss2mxm2h](http://doodle.com/aicvayf8ss2mxm2h)

The most popular one is (using an example):

:::: class MyModel(model.Model): my_field = CharField() my_i18n_field = CharField()

 <span>class Meta:</span>
     <span>translate = ('my_i18n_field',)</span>

A way to translate models (and whole applications) without modifying its code would be great, in order to translate applications that already exist.

For the database backend there are also different options, including:

• To create a field on the model for every translation

There is just one generic ticket on Django that would be fixed:

May be it's not possible having a generic solution that fits most of the user-cases, and in that case could be worth making some modifications on Django to make it easier creating external applications that can do this job.

Fix i18n bugs

There are many bugs already accepted on Django trac, that would be fixed on this Summer of Code. A better review will be done, but some of them could be:

• #3907: LocaleMiddleware allows languages not supported by Django

Timeline

The estimated time line for this project, detailed in a weekly basis is:

• Week 01: Analysis and working environment setup

• Week 02: Import CLDR

• Week 03: Import CLDR

• Week 04: I18n of dates and numbers

• Week 05: I18n of dates and numbers

• Week 06: I18n of dates and numbers

• Week 07: Translation of dynamic content

• Week 08: Translation of dynamic content

• Week 09: Translation of dynamic content

• Week 10: Translation of dynamic content

• Week 11: Fix i18n bugs

• Week 12: Fix i18n bugs

My dedication to the project will be full time, around 40 hours per week. A total of 480 hours are estimated for the whole project.

About me

My name is Marc Garcia, I'm from Barcelona, Europe, and I'm 29 years old.

I am studying computer science at Universitat Oberta de Catalunya, an Internet-based university from Barcelona. Currently I'm not working but I have almost 8 years of programming experience (with different technologies, mainly Python, PHP and VB).

I started using Django in 2006, and at this time I developed and participated on the development of many websites, as well as many reusable applications for Django.

As examples of reusable Django applications note:

As examples of websites, note next ones:

I'm also one of the two official translators of Django to Castilian Spanish and Catalan. In addition, I was interviewed about localization on Django on This Week in Django 20 (on 2008-04-27). I maintain a blog with many Django related posts at http://vaig.be.