Configuration
One of the dependencies in the SIMS Portal project is a package called Flask-Babel, which supports this process. A prerequisite for the steps below is to have a special configuration file saved at the top level of the flask_app directory, called babel.cfg. This is a very simple file that (as of the Portal launch) only requires two lines:
[python: SIMS_Portal/**.py]
[jinja2: SIMS_Portal/templates/**.html]The first line helps the engine find strings in the Python files, and the second inside the HTML files. If you need to edit this configuration file, use your text editor or terminal to modify the .cfg file.
Next, we need to update the Flask app’s Config class with an additional variable that establishes which languages our text is in and will be translated to. As of the initial launch of the Portal, I only supported Spanish, so the config file looks like this:
class Config(object):
# ...
LANGUAGES = ['en', 'es']If you want to add an additional language in the future, append the shortcode inside the list. For example, adding French would mean the line would read LANGUAGES = ['en', 'es', 'fr']
The last configuration-related item shouldn’t be an issue in the future if you’re referencing this guide, but I’m listing here for posterity. When a user loads the site and the server launches the app, Flask generates a new instance with the create_app() function in the __init__.py file. The get_locale() function nested inside of that uses an attribute of Flask’s request object called accept_languages along with the best_match() method.
@babel.localeselector
def get_locale():
return request.accept_languages.best_match(app.config['LANGUAGES'])Essentially what this is doing is asking the user’s browser for their defined language. The best_match() does what its name implies and looks for the best it can with finding the right language. We have to do fuzzy matching here as some users have a country specified here as well (e.g. en-US for US English versus en-GB for British English), and best_match() avoids exact-match issues. If you want to test out your translations, you can either change your language setting on your browser, or just temporarily hardcode your language in that function:
@babel.localeselector
def get_locale():
return 'es' # manually forces Flask to serve Spanish translationsText Tagging
The Portal serves up text to end users from two sources. First and more common is through HTML, where the bulk of text is stored. Second is within the Python code that handles the backend, where specialized messages are sometimes generated. The tagging process is slightly different for each type.
Tagging HTML
The Portal takes advantage of Jinja, a powerful templating engine, to translate data from the database and other sources into legible text. You can find where Jinja is being used when you look at an HTML template and see double curly brackets (e.g. {{ }}) or curly brackets with percentage signs (e.g. {% %}). Without translations, you’d only see these when the template is displaying variable data, rather than static. For example, if a string on a certain page said “Hello, Jonathan”, that first name would actually be a variable wrapped inside Jinja tags, like <h1>Hello, {{user.first_name}}</h1>.
When we tag text for translation, we have to essentially wrap all our text in similar Jinja tags with a slightly modified syntax: an underscore, parentheses, and single quotes inside double curly brackets. For example:
<h3 class='text-red'>View My Profile</h3>becomes:
<h3 class='text-red'>{{ _('View My Profile') }}</h3>If we have existing Jinja variables as in the Hello, Jonathan example, we have to change our syntax slightly and make use of arguments:
<h1>{{ _('Hello, %(username)s', username=user.first_name) }}</h1>As you can see, we used string formatting with the username variable (with the %(VARIABLE_NAME)s syntax), then passed the value as an argument. This is not, unfortunately, the more modern way of handling text replacement in Python—that would be to use curly brackets with a .format()—so keep that in mind if you’re used to Python3.
Tagging Python Code
The process for tagging text in Python code is similar, except that we aren’t dealing with Jinja like we did with HTML. However, there are some different methods you have to keep in mind when tagging the code.
In most cases, you simply have to wrap string literals inside _( ). If there are no variables to worry about, that’s all you do. For example:
from flask_babel import _
def flash_message_on_logout():
message = flash( _('You have logged out.') )
return messageIf you are including variable data, then you need to use the same strategy shown in the HTML section above wherein we wrap the variable and then pass it as an argument. So in the example above, if we wanted to add the user’s name to the message:
from flask_babel import _
def flash_message_on_logout(user_id):
user = db.session.query(User).filter(User.id == user_id).first()
message = flash( _('You have logged out, %(username)s.', username=user.first_name ) )
return messageAdapting Tagging for Text Outside of Requests
The text that we’ve dealt with to this point in this guide has been that which gets loaded when the user loads a specific page or fires off a specific Flask route. However, some string literals are loaded at the time of the application’s startup, before a language is assigned to the request object. One example of a place where this would be a problem is on forms, where the Flask-WTForms package is handling the labels and fields. Those labels would be loaded outside of a request, and so need to utilize a lazy evaluation technique called lazy_gettext().
To use this lazy evaluation, you need to import it and alias it (the common syntax is _l for the alias):
from flask_babel import lazy_gettext as _l
class UserLogin(FlaskForm):
username = StringField(_l('Username'), validators=[DataRequired()])
# ...Extracting Text for Translation
Now that we’ve tagged all of our text, we’re ready to extract it. This extraction means we’ll be able to more easily manage the process of translating each snippet of text in a clean view rather than cluttered HTML or Python code. This step requires a basic understanding of the command line, but I’ll provide the code necessary to follow along.
Before getting into the text extraction process, let’s quickly review what the Portal’s folder structure looks like (this is a simplified view of just the relevant files).
- flask_app
- babel.cfg
- run.py
- SIMS_Portal
- database.sqlite
- folders for each database table with relevant python files
- static
- js
- css
- img
- templates- Verify that the
babel.cfgconfiguration file described above is in place and that the routes to the HTML and Python files are correct. - Open up your terminal. If you’re using an IDE with an integrated terminal, use that as it will make navigating to your directory more straightforward. If not, navigate to the correct folder, which would be the same place as the
babel.cfg. That means your terminal should look something like this:(venv) jonathan.garro@jonathans-mbp flask_app %. If the last bit of that doesn’t sayflask_app, you’re not at the right level of the folder structure—usepwdto find where you currently are,lsto list out the files and folders available to drill down into, andcd FOLDER_NAMEto navigate around. - Run the following command:
pybabel extract -F babel.cfg -k _l -o messages.pot .– This will loop over all of the files that thebabel.cfgfile has pointed it towards, and generate a file which I’ve specified as being calledmessages.pot. This file should appear at the same level as the configuration file. We’ll deal with this file when we actually write out our translations. - Generate a catalogue for each language. As I prepared the Portal for Spanish, you should already see a folder for it. But you’ll need to do this step if you are adding any new languages. The command in the terminal is:
pybabel init -i messages.pot -d app/translations -l es– if you were running this for, say, French, then you’d replace theeswithfr. This will generate a folder with the following structure:app/translations/es/LC_MESSAGES/messages.po.
See the updated folder structure below to understand where this folder should be. I’ve flagged the two changes with (NEW).
- flask_app
- babel.cfg
- (NEW) messages.pot
- run.py
- SIMS_Portal
- database.sqlite
- folders for each database table with relevant python files
- static
- js
- css
- img
- templates
- (NEW) translations
- es
- LC_MESSAGES
- messages.poThe messages.pot is a text file you should be able to open with a text editor. It looks something like this:
# Translations template for PROJECT.
# Copyright (C) 2023 ORGANIZATION
# This file is distributed under the same license as the PROJECT project.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2023.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PROJECT VERSION\n"
"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
"POT-Creation-Date: 2023-01-03 15:25-0500\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.11.0\n"
#: SIMS_Portal/templates/index.html:9
msgid "View My Profile"
msgstr ""After the header info and some metadata, you see what the structure of the translations looks like with an example at the bottom. The “View My Profile” string was picked up by the tagging process. The mgstr is blank—that’s where the translation for that string will go.
Translate the Text
Now that we’ve generated our messages.pot we’re ready translate! As we saw above, the messages.pot isn’t great to work with if we’re dealing with loads of text to be translated. Fortunately, there are specialized programs for this part of our process. I recommend downloading the free, open-source translation editor called POedit. Open the messages.pot file with this program, and you’ll see all of your source text in a single column. Click Create new translation and specify the language, and you’ll get a second column for the translation. Clicking on each shows the source and translation as two boxes.
You can either manually translate, or take advantage of the deepl-powered translator. If you decide on the latter, just make sure a native speaker reviews everything once you’re done. Save the file (make sure it’s being saved in the same location as where you opened it). Now the raw messages.pot file will have the msgstr filled in with your translations:
#: SIMS_Portal/templates/index.html:9
msgid "View My Profile"
msgstr "Ver Mi Perfil"Compile the Translations
The last step will be to compile the translations that we have saved to the messages.pot file into the relevant language catalogue we created earlier. Run the following command in your terminal: pybabel compile -d SIMS_Portal/translations. If you get a path error, it’s because the app/translation structure isn’t accurate. You might need to adjust it to find the translations folder.
That should be it! If you want to check that it worked, restart the Flask server in your local virtual environment, and change the browser settings to the language you want to test. That process will be different for each browser, but in Chrome, go to Settings > Languages, then add the language you want to test, and move it to the top of the list.
Making Updates
If you’re reading this guide, at least some of the Portal has already been translated, but keeping things up to date will have to be a periodic process. Whenever you need to make new translations, first extract a new .pot file with pybabel extract -F babel.cfg -k _l -o messages.pot . (it will override the existing file with any newly-tagged text), do your translations in POedit and save the file, then run pybabel update -i messages.pot -d SIMS_Portal/translations.