Internationalization for developers

Zulip, like many popular applications, is designed with internationalization (i18n) in mind, which means users can fully use the Zulip UI in their preferred language.

This article aims to teach Zulip contributors enough about internationalization and Zulip's tools for it so that they can make correct decisions about how to tag strings for translation. A few principles are important in how we think about internationalization:

This article aims to provide a brief introduction. We recommend the EdX i18n guide as a great resource for learning more about internationalization in general; we agree with essentially all of their style guidelines.

Key details about human language

There are a few critical details about human language that are important to understand when implementing an internationalized application:

There's a lot of other interesting differences that are important for i18n (e.g. Zulip has a "full name" field rather than "first name" and "last name" because different cultures order the surnames and given names differently), but the above issues are likely to be relevant to most people working on Zulip.

Translation process

The end-to-end tooling process for translations in Zulip is as follows.

  1. The strings are marked for translation (see sections for backend and frontend translations for details on this).

  2. Translation resource files are created using the ./manage.py makemessages command. This command will create, for each language, a resource file called translations.json for the frontend strings and django.po for the backend strings.

The makemessages command is idempotent in that:

If you're interested, you may also want to check out the translators' workflow, just so you have a sense of how everything fits together.

Translation resource files

All the translation magic happens through resource files, which hold the translated text. Backend resource files are located at locale/<lang_code>/LC_MESSAGES/django.po, while frontend resource files are located at locale/<lang_code>/translations.json (and mobile at mobile.json).

These files are uploaded to Transifex, where they can be translated.

HTML Templates

Zulip makes use of the Jinja2 templating system for the backend and Handlebars for the frontend. Our HTML templates documentation includes useful information on the syntax and behavior of these systems.

Backend translations

Jinja2 templates

All user-facing text in the Zulip UI should be generated by an Jinja2 HTML template so that it can be translated.

To mark a string for translation in a Jinja2 template, you can use the _() function in the templates like this:

{{ _("English text") }}

If a piece of text contains both a literal string component and variables, you can use a block translation, which makes use of placeholders to help translators to translate an entire sentence. To translate a block, Jinja2 uses the trans tag. So rather than writing something ugly and confusing for translators like this:

# Don't do this!
{{ _("This string will have") }} {{ value }} {{ _("inside") }}

You can instead use:

{% trans %}This string will have {{ value }} inside.{% endtrans %}

Python

A string in Python can be marked for translation using the _() function, which can be imported as follows:

from django.utils.translation import gettext as _

Zulip expects all the error messages to be translatable as well. To ensure this, the error message passed to JsonableError should always be a literal string enclosed by _() function, e.g.:

JsonableError(_('English text'))

If you're declaring a user-facing string at top level or in a class, you need to use gettext_lazy instead, to ensure that the translation happens at request-processing time when Django knows what language to use, e.g.:

from zproject.backends import check_password_strength, email_belongs_to_ldap

AVATAR_CHANGES_DISABLED_ERROR = gettext_lazy("Avatar changes are disabled in this organization.")

def confirm_email_change(request: HttpRequest, confirmation_key: str) -> HttpResponse:
  ...

class Realm(models.Model):
    MAX_REALM_NAME_LENGTH = 40
    MAX_REALM_SUBDOMAIN_LENGTH = 40

    ...
    ...

    STREAM_EVENTS_NOTIFICATION_TOPIC = gettext_lazy('stream events')

To ensure we always internationalize our JSON errors messages, the Zulip linter (tools/lint) attempts to verify correct usage.

Frontend translations

We use the FormatJS library for frontend translations when dealing with Handlebars templates or JavaScript.

To mark a string translatable in JavaScript files, pass it to the intl.formatMessage function, which we alias to $t in intl.js:

$t({defaultMessage: "English text"})

The string to be translated must be a constant literal string, but variables can be interpolated by enclosing them in braces (like {variable}) and passing a context object:

$t({defaultMessage: "English text with a {variable}"}, {variable: "Variable value"})

FormatJS uses the standard ICU MessageFormat, which includes useful features such as plural translations.

$t does not escape any variables, so if your translated string is eventually going to be used as HTML, use $t_html instead.

$("#foo").html(
    $t_html({defaultMessage: "HTML with a {variable}"}, {variable: "Variable value"})
);

The only HTML tags allowed directly in translated strings are the simple HTML tags enumerated in default_html_elements (static/js/i18n.js) with no attributes. This helps to avoid exposing HTML details to translators. If you need to include more complex markup such as a link, you can define a custom HTML tag locally to the translation:

$t_html(
    {defaultMessage: "<b>HTML</b> linking to the <z-link>login page</z-link>"},
    {"z-link": (content_html) => `<a href="/login/">${content_html}</a>`},
)

Handlebars templates

For translations in Handlebars templates we also use FormatJS, through two Handlebars helpers that Zulip registers. The syntax for simple strings is:

{{t 'English text' }}

If you are passing a translated string to a Handlebars partial, you can use:

{{> template_name
    variable_name=(t 'English text')
    }}

The syntax for block strings or strings containing variables is:

{{#tr}}
    Block of English text.
{{/tr}}

{{#tr}}
    Block of English text with a {variable}.
{{/tr}}

Just like in JavaScript code, variables are enclosed in single braces (rather than the usual Handlebars double braces). Unlike in JavaScript code, variables are automatically escaped by our Handlebars helper.

Handlebars expressions like {{variable}} or blocks like {{#if}}...{{/if}} aren't permitted inside a {{#tr}}...{{/tr}} translated block, because they don't work properly with translation. The Handlebars expression would be evaluated before the string is processed by FormatJS, so that the string to be translated wouldn't be constant. We have a linter to enforce that translated blocks don't contain handlebars.

Restrictions on including HTML tags in translated strings are the same as in JavaScript. You can insert more complex markup using a local custom HTML tag like this:

{{#tr}}
    <b>HTML</b> linking to the <z-link>login page</z-link>
    {{#*inline "z-link"}}<a href="/login/">{{> @partial-block}}</a>{{/inline}}
{{/tr}}

Transifex config

The config file that maps the resources from Zulip to Transifex is located at .tx/config.

Transifex CLI setup

In order to be able to run tx pull (and tx push as well, if you're a maintainer), you have to specify your Transifex credentials in a config file, located at ~/.transifexrc.

You can find details on how to set it up here, but it should look similar to this (with your credentials):

[https://www.transifex.com]
username = user
token =
password = p@ssw0rd
hostname = https://www.transifex.com

This basically identifies you as a Transifex user, so you can access your organizations from the command line.