Pootle is an online tool that makes the process of translating so much simpler. It allows crowd-sourced translations, easy volunteer contribution and gives statistics about the ongoing work.
Pootle is built using the powerful API of the Translate Toolkit and the Django framework. If you want to know more about these, you can dive into their own documentation.
The following pages cover the documentation of Pootle from a wide variety of perspectives, server administrator’s, and developer’s view.
Pootle FS is Pootle’s integration with version control plugin systems. It allows Pootle to synchronize with an external repository containing your translations, keep them synchronised and manage and resolve any conflicts either automatically or via user input.
At any time we are able to query the state of Pootle FS using fs
state
command. The results of this operation will indicate if there
are any actions you need to specify to resolve any conflicts or if there are
untracked files.
You specify Actions that need to be taken to resolve conflicts or to ensure that files are tracked. This could be adding a file, removing a file or merging conflicting translations. This is the process of staging actions.
The final step is to synchronise Pootle and your filesystem. This operation takes your staged actions and executes them.
A filesystem is actually itself a Pootle FS plugin. Currently two exist:
You can write a plugin for any version control system, Pootle FS will ensure that the same commands and operations are used to ensure Pootle and your filesystem stay synchronized.
Note
Read this if you have used previous versions of Pootle.
Previous versions of Pootle made use of two commands, update_stores
and sync_stores
, to allow translations to be pushed into Pootle or
pulled from Pootle.
These two commands still exist but are now deprecated. We will be phasing these out in the long term to make everything use Pootle FS, but you can replace these commands with Pootle FS equivalents.
You can find an outline of how to use Pootle FS in the using Pootle FS instructions. If you have a existing project you want to migrate to Pootle FS please read the migrating your projects to Pootle FS instructions. You might also have an already correctly setup project that you want to integrate directly with a version control system.
When upgrading Pootle your projects will be automatically migrated to use the
Pootle FS localfs
backend.
Note
Before continuing ensure all projects were properly migrated to Pootle FS when upgrading Pootle.
While Pootle will continue to support update_stores
and
sync_stores
these are now deprecated, so it is advisable you start
to adjust your workflow to use Pootle FS.
If you have scripts using sync_stores
and update_stores
then you might want to continue using those until you can migrate them to
Pootle FS commands.
sync_stores
and update_stores
make use of Pootle FS
infrastructure so they are in fact still using Pootle FS. The difference is
that they mimic the monodirectional behaviour of the old commands. Pootle FS
will synchronise in both directions at a unit level, while
update_stores
will only load new and changed units and
sync_stores
will only synchronise Pootle changes to disk.
The advantage of this monodirectional mode is that you can add scripts to adapt files after synchronising or before loading into Pootle. Your scripts changing files on disk will likely mess with direct Pootle FS change detection.
You may want to look at the format adaptors for future massaging or formats.
update_stores
can be replaced with the following set of Pootle FS
commands:
(env) $ pootle fs fetch my-project
(env) $ pootle fs resolve my-project --overwrite
(env) $ pootle fs sync my-project --update=pootle
Note
To narrow down the execution to a particular language in a project
you must append the --fs-path
argument for each of command in the
previous snippet. For example --fs-path=my-project/fr/*
constrains to
the project’s French filesystem files.
sync_stores
can be replaced with the following set of Pootle FS
commands:
(env) $ pootle fs fetch my-project
(env) $ pootle fs resolve my-project --overwrite --pootle-wins
(env) $ pootle fs sync my-project --update=fs
Note
To narrow down the execution to a particular language in a project
you must append the --pootle-path
argument for each of command in the
previous snippet. For example --pootle-path=/de/my-project/*
constrains
to the project’s German database stores.
Note
Pootle FS will work out of the box when synchronizing with the local file system. If this is the case you can safely skip the integration with version control.
If the translations for your project are stored in a version control system
(VCS in short), then might be a good idea to directly integrate with the VCS.
The following instructions work either if you project is already setup to use
the localfs
Pootle FS backend, or if you are creating and setting a new
project to directly work with the VCS.
Pootle FS provides support for different VCS systems through plugins, so in order for Pootle to work with a specific VCS it is necessary to install its plugin. For example for Git:
Install the plugin:
(env) $ pip install --pre --process-dependency-links Pootle[git]
Add the plugin to INSTALLED_APPS
in your custom Pootle settings:
INSTALLED_APPS += ['pootle_fs_git']
This is done once for the whole Pootle server.
The version control system also must provide access for Pootle FS to synchronize:
Create a SSH key:
$ sudo -u USER-RUNNING-POOTLE ssh-keygen -b 4096
Tell your upstream repository about the public key, allowing Pootle to be able to push to the repository. For example for GitHub:
After installing the necessary Pootle FS plugin and connecting Pootle FS with the VCS repository, it is now necessary to alter the project configuration:
Deactivate any existing automatic synchronization (like cron entries).
Disable the project to prevent changes from translators.
Ensure you have synchronized all the translation files to disk.
Ensure you have committed all the translation files to your version control system.
Set the project’s Filesystem backend to the appropriate VCS backend.
Set the URL to your version control repository in the project’s Path or
URL, e.g. git@github.com:user/repo.git
.
Synchronize as follows:
(env) $ pootle fs fetch $MYPROJECT
(env) $ pootle fs sync $MYPROJECT
Enable the project again.
Enable automatic synchronization again.
Your project is now ready to synchronize with the configured repository using Pootle FS. You might want to learn more about how to use Pootle FS.
The task of Pootle FS is to keep the filesystem and Pootle in sync. There are scenarios where items are not in sync and Pootle FS requires your intervention, these are the commands you will use to bring things back into sync and to resolve conflicts.
To clarify the terminology that we use in Pootle FS:
file
- a translation file on diskstore
- a translation file in the Pootle databaseFiles and stores are usually associated and thus we are able to keep them synchronised. But there might be files with no store (the store for a new file has not yet been created in the Pootle database), and stores with no file (the file has been removed from the filesystem).
Pootle FS works in these stages:
Files that have never been synced are untracked, need to be explicitly staged. Files previously synced are tracked, will be automatically staged if there are any changes. In the case of conflicts (changes both on disk and in Pootle) it is also necessary to manually stage these to resolve which version should prevail.
When staging it is possible to specify specific stores or files, or groups of
them using the -P
and -p
options. It is also possible to limit which
staged actions are executed by using these same options on the sync
command.
When a store and its corresponding file are tracked and previously synced, then they are automatically staged for syncing if either changes.
If both have changed then we will need to specify how to resolve the conflict.
To re-sync stores and files run:
(env) $ pootle fs sync MYPROJECT
When new files appear on the filesystem that we want to bring into Pootle we
use add
. And when new stores have appeared on Pootle that we want to
push to the filesystem we also use add
:
(env) $ pootle fs add MYPROJECT
(env) $ pootle fs sync MYPROJECT
Where add
will stage the previously untracked files or stores.
While sync
will synchronize, pulling the translations in the file
into the Pootle database or pushing translations from the stores in the
database to files on the filesystem.
Following this the file and store are now tracked.
A store or file can be missing from Pootle or the filesystem because it has
been removed, we use rm
to remove such files and stores:
(env) $ pootle fs rm MYPROJECT
(env) $ pootle fs sync MYPROJECT
This will remove the store or file, depending on whether it is the file or store that remains.
Following this there is no such file or store on the filesystem or on Pootle.
Conflicts can occur if a tracked Pootle store and its corresponding file have both changed. They can also arise if a new Pootle store is added and a matching file has been added in the filesystem simultaneously.
Using the resolve
command we have four possible ways to resolve such
conflicts:
The merge options are most useful where you need translators to resolve the conflict.
The default options for resolve
ensure that the filesystem always
wins and that translators will be given an opportunity to resolve the conflict.
This ensures that changes on version control are authoritative, a fix can land
there without anyone knowing Pootle is involved. It also means that we have no
data loss in that any conflicts will be presented to translators as
suggestions, allowing them to resolve the conflicts by reviewing the
suggestions.
You want to keep the version that is currently on the filesystem, discarding all changes in Pootle:
(env) $ pootle fs resolve --overwrite MYPROJECT
(env) $ pootle fs sync MYPROJECT
You wish to keep the version that is currently in Pootle, discarding all changes in the filesystem:
(env) $ pootle fs resolve --overwrite --pootle-wins MYPROJECT
(env) $ pootle fs sync MYPROJECT
You want to retain all translations and allow translators to resolve conflicts. This will merge any non-conflicting units. For conflicting units, keep the filesystem translation and convert the Pootle translation into a suggestion:
(env) $ pootle fs resolve MYPROJECT
(env) $ pootle fs sync MYPROJECT
The result is that all non-conflicting units have been synchronised. For any unit where both the store unit and file unit changed the translation is set to the file unit translation with the store unit translation converted into a suggestion. You can now review these suggestions to resolve the conflicts.
You want to retain all translations and allow translators to resolve conflicts.
This will merge any non-conflicting units. For conflicting units, keep the
Pootle translation (due to the --pootle-wins
option) and convert the filesystem translations into a suggestion:
(env) $ pootle fs resolve --pootle-wins MYPROJECT
(env) $ pootle fs sync MYPROJECT
Pootle FS uses a set of statuses for the files or stores it manages, using these it is able to determine what to do to resolve the scenario.
There are two groups of statuses:
resolve
, rm
and add
commands.sync
’ed.Statuses that need an action to be specified in order for Pootle FS to be able to resolve them:
conflict
resolve
to merge the two and manage conflict
resolution. Or if you wish to discard one or the other use resolve
--overwrite
.conflict_untracked
resolve
to combine the translation and manage
conflict resolution for each unit. Or to force taking the whole file or store
use either resolve --overwrite
and optionally
--pootle-wins
depending on whether you want
to keep the filesystem file or the Pootle store.pootle_untracked
add
to create the file on the
filesystem and push the translations on the store to it, or alternatively use
rm --force
to stage the store for removal.fs_untracked
add
to pull the file into
Pootle or alternatively use rm --force
to stage the file for
removal.pootle_removed
add --force
to restore
the filesystem version, or use rm
to stage for removal from
filesystem.fs_removed
add --force
to restore the Pootle version, or use rm
to
stage for removal from Pootle.These statuses reflect changes that can be either unstaged by using
unstage
or executed with sync
:
pootle_ahead
sync
will push the changes to the filesystem.fs_ahead
sync
will pull the changes to Pootle.pootle_staged
sync
will create the file on the filesystem.fs_staged
sync
will create the store on Pootle.merge_pootle_wins
merge_fs_wins
remove
sync
will remove the file or store.both_removed
sync
will remove both the file and
store.Pootle supports many file formats through the powerful Translate Toolkit API. The Toolkit also provides several format converters for other formats, this will allow you to host a lot of translatable content on Pootle.
All these formats can be downloaded for offline use/or translation (for example in Virtaal). We recommend Virtaal for offline translation. They can also be downloaded in XLIFF format.
Pootle gives translators and project developers an easy way to see progress on the translation work. Progress is indicated by a coloured graph to easily see how much work is complete, and how much work remains. Pootle can also give detailed statistics about the progress in translation work.
Statistics report on both the progress in the number of messages, and in the number of words. The number of words gives a much better impression of how much work is involved, and allows for more accurate time estimation.
Pootle also assists in translation quality assurance, by performing several Quality checks on the translations which can help in review. These quality checks correspond to the quality checks performed by pofilter from the Translate Toolkit.
Translation templates are translation files that contain only the source text (original text). These files are used as a template to create target files for each language.
Users familiar with Gettext know translation templates as POT files. For other bilingual formats (like XLIFF) untranslated files with the same extension are used as templates.
Pootle can manage a special language called Templates. This is not strictly speaking a language but rather a place to store translation templates for a project.
If the Templates language is present then Pootle will initialise brand new languages from the Templates files present in Pootle.
If the Templates language is absent from a project, Pootle will assume all initialisation of files for new languages happens outside of Pootle.
It is helpful to understand in more detail how a new language is created or added to Pootle.
When adding a new language to a project from the Pootle interface, and the Templates language exists for the project in Pootle then a fresh copy will be generated based on those template files.
If there is no Templates language it is necessary to manage all
initialisation of languages from the Pootle command line. When using
update_stores
new languages will be initialised if they are present
on the filesystem. You are responsible for initialisation of these new
languages from template files as required.
Pootle will not update existing translations if new template files are added to Pootle. Updating of translations is managed outside of Pootle. You can update your translations as follows:
sync_stores
to sync all translations to the filesystem.
These files will now contain the latest translations from Pootle users.update_stores
to push the updated translations to Pootle.A detailed example can be found in Updating strings for existing project.
Pootle has the ability to display alternative source languages while translating. Thus, translators who know another language better than English can take part in the translation project. Also, it provides a way to disambiguate terminology by seeing how other languages have translated the same string.
Users who want to use the functionality need to specify the desired alternative
source languages in their account configuration. Alternatively, Pootle will try
to guess the user’s alternative source language by looking at the browser’s
Accept-Lang
header.
Note
If the selected project doesn’t have translations in the alternative source language then no alternative will be displayed.
This feature is enabled by default.
In order to show suggestions from another language, the following is needed:
Pootle can display clickable characters which might be difficult to type or unavailable to the translator on their keyboard. These appear below the translation widget as we see below for Afrikaans.
Clicking on any of the ëïêôûáéíóúý
characters will insert that character
into the translation.
Many languages, e.g. those using the Latin script with diacritics, will find this very helpful, especially where keyboard layouts are not readily available.
Anyone with admin rights for Pootle or for a specific language can adjust the special characters.
To adjust the characters open the Special characters page accesssed via the admin dropdown in the navigation bar.
Adjust the needed characters by adding and deleting characters.
Special characters do not solve the input needs for all languages, but has been a very useful help for many languages, especially in translate@thons.
For people using non-Latin scripts, consider if it will be useful to perhaps include things that can’t be easily typed by translators in your language. You will probably need to limit the number of characters, but hopefully you can find a reasonable compromise that will help many people.
Pootle provides a powerful way of reviewing translations for quality. It exposes most of the pofilter checks that can check for several issues that can affect the quality of your translations.
If Pootle indicates a possible problem with a translation, it doesn’t mean that the translation is necessarily wrong, just that you might want to review it. Pootle administrators should indicate the correct project type (GNOME, KDE, Mozilla, etc.) in the administration pages. This will improve the accuracy of the quality checks.
Critical checks are prominently displayed through the browsing UI.
Any extra failing checks can be accessed by clicking the
button located below the navigation breadcrumbs.
Clicking on the name of a check will step you through the translations that
fail that check.
While in the translation editor, submissions resulting in critical failing
checks will be immediately reported, preventing you from automatically
continuing until the issues have been resolved or muted as false positives by
using the mute button. Non-critical checks flag potential
problems but can be ignored or muted as needed.
To understand the meaning of each check, Pootle displays the failing checks right on top of the submission button, with a link to the online documentation. You can also read the detailed descriptions of the pofilter checks.
It is possible to mute the quality check if the translation is correct.
Reviewers are able to remove the check for a certain string, by clicking on the
button, to indicate that the string is correctly translated.
This avoids having to review the same check multiple times. You can unmute any
muted check using the
button.
If the source text of the translation is changed then the mute is discarded to ensure that the new translation is checked again for any possible issues.
Pootle provides matching translations to the current string. Translator can use these matches as their translation or to aid their translation.
Matches are based on previous translations of similar strings. These Translation Memory (TM) matches mean that you can speed up your translation and ensure consistency across your work.
Translation Memory matches are automatically retrieved when you enter a new translation unit. These are displayed below the editing widget. You can insert a TM match by clicking on a match row.
The differences between the current string and the match are highlighted, this allows you to see how the two differ and helps you make changes to the match to adapt it for use with the current string.
Translation Memory will work out of the box with a default Pootle installation. There are three methods of getting Translation Memory.
By default Pootle will query Translate’s Amagama Translation Memory server, which hosts translations of an extensive collection of Opensource software.
If you want to setup and connect to your own TM server then the
AMAGAMA_URL
will allow you to point to a private TM server.
To disable Amagama set AMAGAMA_URL
to ''
.
New in version 2.7.
Pootle can also retrieve TM matches stored on Elasticsearch-based TM servers. These TM servers require Elasticsearch to be installed and running.
Note
Elasticsearch depends on Java. Note that some systems may ship with OpenJDK, however elasticsearch recommends using Oracle JDK.
Install the required Python libraries:
(env) $ pip install --pre --process-dependency-links Pootle[es5]
Note
Elasticsearch TM should work with any version of Elasticsearch, our tests run against Elasticsearch 5.x. For support for Elasticsearch 1.x and Elasticsearch 2.x, simply replace es5 with es1 and es2 respectively in the above command.
Pootle supports two types of Elasticsearch-based TMs:
local
) is populated using translations
stored in Pootle database and every new translation gets automatically
imported to it.Both local and external TM settings can be adjusted in
POOTLE_TM_SERVER
. A configuration example for local and external TM
can be found in the default ~/.pootle/pootle.conf
, and can be enabled by
uncommenting the example.
Please see the POOTLE_TM_SERVER-WEIGHT
for a full example of the
configuration necessary to set up local/external TM.
Both Amagama and Elasticsearch based TMs can operate together. Though you may want to disable Amagama.
New in version 2.7.
To use it, the local
TM must be enabled in POOTLE_TM_SERVER
and
will need to be populated using the update_tmserver
command:
(env) $ pootle update_tmserver
Once populated Pootle will keep Local TM up-to-date.
New in version 2.7.3.
In order to use them they must be enabled in POOTLE_TM_SERVER
and
you will need to populate them using the update_tmserver
command
specifying the TM to use with --tm
and the
display name with --display-name
:
(env) $ pootle update_tmserver --tm=external --display-name=Pidgin af.po gl.tmx
A display name is a label used to group translations within a TM. A given TM
can host translations for several labels. Just specify them with
--display-name
:
(env) $ pootle update_tmserver --tm=external --display-name=GNOME pt.tmx eu.po xh.po
It is possible to have several Elasticsearch-based external TM servers working
at once, along with the Elasticsearch-based local TM server. In order to do so
just add new entries to POOTLE_TM_SERVER
:
POOTLE_TM_SERVER = {
...
'libreoffice': {
'ENGINE': 'pootle.core.search.backends.ElasticSearchBackend',
'HOST': 'localhost',
'PORT': 9200,
'INDEX_NAME': 'whatever',
'WEIGHT': 0.9,
'MIN_SCORE': 'AUTO',
},
}
Make sure INDEX_NAME
is unique. You
might also want to tweak WEIGHT
to change
the score of the TM results in relation to other TM servers (valid values are
between 0.0
and 1.0
).
To use these additional external TMs you will need to populate them using the
update_tmserver
command specifying the TM server with --tm
:
(env) $ pootle update_tmserver --tm=libreoffice --display-name=LibreOffice af.po gl.tmx
Check update_tmserver
for more options.
Note that Pootle will not push new translations to these TM servers unless you
explicitly use the update_tmserver
command, giving you full control
of which translations make into them.
Pootle has the ability to use online Machine Translation (MT) Services to give suggestions to translators. This feature has to be enabled by the server administrators.
Note
Machine Translations are not meant to replace human translations but to give a general idea or understanding of the source text. It can be used as suggestion of a translation, but don’t forget to review the suggestion given.
If the server administrator has enabled machine translation then an icon
will be displayed for each source text (English or
alternative source language) next to the Copy button. Clicking the relevant
buttons will retrieve translation suggestions from the online services and
will mark the current string as fuzzy to indicate that review is required.
To enable a certain Machine Translation Service, edit your configuration
file and add the desired service within the
POOTLE_MT_BACKENDS
setting.
Each line is a tuple which has the name of the service and an optional API key. Some services may not require API keys but others do, so please take care of getting an API key when necessary.
Supported Services:
Google Translate
Yandex.Translate
New in version 2.7: Yandex.Translate
Google Translate is widely used and supports a number of languages. It is a paid service requiring an account and API key.
Yandex.Translate is the free alternative to Google.
Pootle provides search functionality that allows translators and reviewers to search through translations for some text. The search box is shown close to the top of the page. Searching can be used to find specific things you want to work on, see how issues were solved before, or to verify consistency in your translations.
Search results are up to date, and will reflect the current translations in Pootle.
It is important to realize that when a new search term is entered, searching will take place inside the currently viewed domain. If you are currently at the top level of your project, the whole project will be searched. If you are viewing a directory, only files under that directory will be searched. If you are already viewing/translating a file, only that file will be searched.
The first result will be shown in context in the file where it is found. When you click “Skip”, “Suggest” or “Translate” it will provide the next match to the search (in the original domain) until all matches were presented. Remember that if you edit the search query while viewing search results in a specific file, your new query will only search in that specific file.
When you enter a search box a dropdown will open allowing you to limit or expand your search to specific fields. Any combination of these fields and options is accepted.
Fields that you can search in include:
Options:
Action | Current shortcut | Proposed shortcut |
---|---|---|
Submit and move to next translation | Ctrl+Enter | |
Toggle the ‘Needs work’ flag | Ctrl+Space | |
Toggle the suggest/submit mode | Ctrl+Shift+Space | |
Copy the contents from the original language | Alt+Down | |
Focus on comments field | Ctrl+Shift+C |
Pootle has the ability to optionally allow users to provide suggestions that need to be reviewed before they are accepted into the real translation files. Who is allowed to do what, is determined by the configuration of User permissions for the project or the server.
This allows for a team to form with different roles for different team members, and makes it possible to have a more explicit review step that requires suggestions to be checked before they become the real translations. This also allows the collection of different ideas for translating a single string.
When translating, suggestions are shown inline so they’re always visible. If a user wants to view all the suggestions within a project scope, it just needs to go to the “Review” tab and click on the “View Suggestions” link. Users with rights to translate will be shown a “Review Suggestions” link and will be able to accept and reject suggestions.
Users with rights for making suggestions will see a “Suggest” button next to “Submit”. Making a suggestions is as easy as clicking the button – hey, did you expect more steps involved?
In order to review suggestions, users must have privileges to translate. There are two ways for reviewing suggestions: going through all of them, or reviewing while translating.
To go through all of them, the reviewer must click on “Review Suggestions” within the “Review” tab of the project. This would guide her/him through all the suggestions available for the current view.
The second method is straightforward, since suggestions are shown throughout the translation process. Additionally, buttons for accepting and rejecting the suggestions are displayed.
While reviewing a suggestion, a coloured difference between the current translation and the suggestion is displayed. If available, the username is provided of the user that gave the suggestion.
A click on the green tick icon approves the selected suggestion while the red cross rejects the selected suggestion. A suggestion approval doesn’t imply the rejection of the remaining suggestions.
Pootle can help translators with terminology. Terminology can be specified to be global per language, and can be overridden per project for each language. A project called terminology (with any full name) can contain any files that will be used for terminology matching. Alternatively a file with the name pootle-terminology.po (in a PO project) can be put in the directory of the project, in which case the global one (in the terminology project) will not be used. Matching is done in real time.
Ideally, the source term should be the shortest, simplest form of a word. Therefore cat, dog, house are good, but cats, dogged and housing are bad.
Context indicators are allowed in the source text, in brackets after the term, but keep them short, eg file (noun), view (verb), etc.
The ideal is therefore that the target term be something that you’d like the translator to be able to insert… but strictly speaking the target text can be anything, including a definition.
If the terminology PO file has translator comments, they will be displayed as a tooltip in Pootle.
If our glossary has an entry: file->lêer, and we translate a sentence like The file was not found, we can suggest the glossary entry file->lêer as relevant to the translation, even if we don’t have any TM entry that is related to the complete sentence that is available for translation.
Say our glossary has an entry category->kategorie and we translate a sentence like Please enter the categories for this photo, we can suggest the glossary entry category->kategorie, even though the letters category doesn’t occur anywhere in the original string.
Currently a single term entry can be up to 30 characters long (including context information), and the first 500 characters of each translation are scanned. Terms can consist of many words, but consider making them as general or simple as possible for maximum impact.
If these limits prove too restrictive, feel free to point out use cases where this is not sufficient.
Since the terminology matching is performed in real-time, you might want to keep an eye on the size of your terminology project to ensure that performance is not affected too much by having too many terms. This is highly dependent on your server abilities and the nature of what you are translating.
New in version 2.7.
Virtual folders provide a way to group translations based on any criteria, including a file across all the languages in a project, or files on specific locations. Virtual folders have priority, so they can be used to allow translators to focus on the most important work.
Virtual folders have several attributes:
The location indicates the root place where the virtual folder applies. It can
use placeholders for language ({LANG}
) and project ({PROJ}
).
Note
The /
location is not valid and must be replaced by
/{LANG}/{PROJ}/
. The locations starting with /projects/
are also
not valid and must be changed so they instead start with /{LANG}/
.
Each virtual folder must have a unique combination of name and location. This means that there can exist two different virtual folders with the same name if they have different locations.
The priority defaults to 1
and accepts any value greater than 0
,
including numbers with decimals, like 0.75
. Higher numbers means higher
priority.
By default virtual folders are public. If they are not public then they won’t be displayed, but they are still used for sorting.
Also the virtual folders can have a description which might be useful to explain the contents of the folder or provide additional instructions. This might be handy when using the virtual folders as goals.
The filtering rules specify which translation units are included within a virtual folder. Currently the only supported filtering rule consists of a list of file or directory paths relative to the virtual folder location. Note that it is required to set some filtering rule.
To add or modify the properties of virtual folders use the
add_vfolders
management command.
This command imports a JSON file holding a list of virtual folders, and the files included on each virtual folder along with all their attributes. Check the specs for the JSON format in order to know how to craft a JSON file that fits your needs.
Translation stats of virtual folders are automatically calculated.
If a virtual folder applies in the current location, then clicking on the links on the overview page will provide the units in priority order when translating in the editor. The priority sorting on the translation editor is calculated taking into account all the applicable virtual folders in the current location, including the not public ones.
The JSON file used to import virtual folders consists of a list of virtual folder definitions with the same fields as the virtual folders, except for two differences:
The following example depicts a basic JSON file:
[
{
"name": "user1",
"location": "/{LANG}/firefox/browser/",
"priority": 999.99,
"is_public": true,
"description": "Most visible strings for the user.",
"filters": {
"files": [
"branding/official/brand.dtd.po",
"chrome/browser/aboutDialog.dtd.po"
]
}
},
{
"name": "user2",
"location": "/gl/firefox/",
"priority": 7.5,
"is_public": false,
"filters": {
"files": [
"browser/chrome/browser/aboutSessionRestore.dtd.po",
"browser/chrome/browser/downloads/downloads.dtd.po"
]
}
},
{
"name": "user3",
"location": "/ru/{PROJ}/",
"priority": 0.3,
"filters": {
"files": [
"browser/chrome/browser/engineManager.dtd.po"
]
}
},
{
"name": "directories-for-lang",
"location": "/{LANG}/",
"filters": {
"files": [
"firefox/browser/profile/",
"firefox/browser/chrome/browser/"
]
}
},
{
"name": "directories-and-files-for-tp",
"location": "/{LANG}/firefox/",
"filters": {
"files": [
"browser/updater/",
"browser/chrome/browser/devtools/appcacheutils.properties.po",
"browser/chrome/browser/migration/"
]
}
},
{
"name": "default",
"location": "/{LANG}/{PROJ}",
"description": "All files in all projects for all languages.",
"filters": {
"files": [
"/"
]
}
},
{
"name": "other",
"location": "/af/firefox/",
"is_public": true,
"filters": {
"files": [
"browser/chrome/browser/aboutCertError.dtd.po"
]
}
},
{
"name": "developer",
"location": "/af/firefox/",
"priority": 0.9,
"description": "As you can see this\\n description spans\\n several lines.",
"filters": {
"files": [
"browser/chrome/browser/devtools/appcacheutils.properties.po",
"browser/chrome/browser/devtools/debugger.dtd.po"
]
}
},
{
"name": "install",
"location": "/ru/{PROJ}/",
"priority": 5,
"is_public": true,
"description": "Installation related strings.",
"filters": {
"files": [
"browser/chrome/browser/migration/migration.dtd.po",
"browser/chrome/browser/migration/migration.properties.po"
]
}
}
]
You can export files for offline translation. Once translated you can import them again and Pootle will manage updating the translation in Pootle based on your changes.
This feature is ideal for teams who have poor connectivity or if you prefer to use an offline translation tool.
To export, simply click on the “Download for offline translation” link on the sidebar in Pootle’s overview page. To import simply click the “Upload translations” link and select the file you wish to upload.
Changed in version 2.7.1.
If a string has been translated on Pootle and changed in your uploaded file then your change will still be uploaded but it will be converted into a suggestion which you can resolve in Pootle.
Note
If there are any errors in the upload then Pootle will warn you and the file will be rejected.
Pootle makes it easy to setup additional custom content without too much effort.
There are three types of static pages:
Use Admin – Static Pages to create and manage static pages.
The static pages are by default formatted using HTML. But you can use Markdown
or RestructuredText by setting POOTLE_MARKUP_FILTER
correctly.
When linking to a static page externally or in any customisations, your links
would be pointing to /pages/$slug
, such as /pages/gettting-started
.
For linking to another static page from within a static page use the
#/$slug
syntax. Thus, if you created a Getting Started page as a static
page which pointed to your Licence Statement legal page we’d use this
#/licence_statement
in the URL.
When creating an announcement page use a slug projects/$project
so that the
page will be used on the $project
project.
Other slug names may be used:
$lang
- for an announcement page that will appear on every single project
enabled for the $lang
languages.$lang/$project
- for an announcement page that is specific to the
$project
project in the $lang
language.The prefered model though is to use the projects/$project
convention for a
single easy-to-maintian page.
In many cases you have URLs in announcement pages that would be identical except for variations in the language code. Examples would include links to team wiki pages, signoff pages, progress dashboards, live test versions, etc.
Any link within your announcement page that uses a fake language code of
/xx/
will be rewritten with the language code for this translation. Thus
if you insert a link such as http://example.com/signoff/xx/
then that will
be rewritten to http://example.com/signoff/af/
for a user viewing this
announcement page for the Afrikaans language translation.
With Pootle’s flexible permissions several ways of interacting with your translation community are possible. If you have a very open Pootle server, you might want to ensure that spammers don’t abuse it by enabling captchas.
If you have no need for captchas, e.g. at a translation sprint, you might want
to remove captcha support. To disable it, set POOTLE_CAPTCHA_ENABLED
in your configuration file to False
. Restart your server for the setting
to take effect.
The captchas can be customized. Look at the captcha template and code:
and make the changes you need.
Pootle’s team page is designed to assist in managing teams. These includes team members and their roles within the team, and other team related aspects and features. The team page also provides overall statistics and access to the bulk handling for pending suggestions.
Language administrators will find the team interface on the first dropdown in their language’s browse page.
The team page is divided in two parts, on the left the management of the team members and on the right a list of the projects enabled for the language, and the overall translation statistics with links to the editor and the bulk suggestions manager.
The UI for team members management is divided in two parts:
There are four different roles, which provide incremental permissions within the team:
Add new members using the form at the top left. Simply, select the user, choose a role and click Add team member button.
To remove members check the box next to their username and then click on the Remove selected button.
To change a member’s role first remove the member from the team, then add the member back with the desired role.
There are several rights which can be assigned to users or to a group of users, such as to all logged in users. The default site-wide permissions are configured by the server administrator. These are the permissions that will be used in each project unless other permissions are configured.
Permissions can be customized server-wide, per-language, per-project or language/project combination (translation project).
Permissions apply recursively, so server-wide permissions will apply to all languages and projects unless there is a more specific permission. Language permission applies to all translation projects under that language, etc.
Pootle has two special users, nobody and default, which are used to assign permissions to more than one user at once. The user nobody represents any non-logged in user, and default represents any logged in user.
If a user has permissions assigned to her user account they override any default permissions even those applied to more specific objects (i.e. a user who has specific rights on a language will override default rights on translation projects).
Server administrators can be specified in the users page of the admin section. Server administrators have full rights on all languages and projects and override all permissions.
Access rights can be set server-wide or for projects. Bear in mind that when limiting access to projects the permissions affect to all the languages available in the project.
Permissions restricting actions can be set server-wide, per language, or language-project combination:
Users with administrative rights for projects, languages or translation projects can access the permissions interface by selecting “Permissions” in the navigation dropdown on the project, language or translation project browsing pages.
Pootle administrators will find the default permissions interface on the administration page, at the “Permissions” tab, where they can set server-wide permissions.
The current rights are listed as they are assigned. The user “nobody” refers to any user that is not logged in (an anonymous, unidentified user). The user “default” refers to the rights that all logged in users will have by default, unless other specific rights were assigned to them. The rest of the users are users of the Pootle server for which non-default rights were assigned.
In the list of permissions, you can simply select which rights must be assigned to each user by picking new permissions or unassigning them. Changes will be updated when you submit the form.
To set permissions for a specific user, select the user in the dropdown list and set the specific rights for that user. This is only necessary if the user does not yet have their own set of rights defined.
To reset some user’s rights to the default rights, select the “Delete” tick box next to their name and permissions list. When you submit, their rights will be reset to the default rights.
Warning
A user with administrative rights can remove his own administrative rights.
These instructions will guide you through installing Pootle and its requirements in a virtual environment.
If you only want to have a sneak peek of Pootle then the default configuration and the built-in server will suffice.
Follow the Pootle installation video tutorial, which starts after some basic setup, to see the installation steps in action and expected results.
For a production deployment we strongly recommend that you set up the following:
Note
Before installing please ensure that you have all the necessary requirements.
Warning
It is important to install Pootle into a virtual environment to ensure the correct packages and permissions. It’s even more important not to install Pootle as the root user on your system. Installing or running Pootle as the root user will expose your system to many potential security vulnerabilities
We’ve made some assumptions in these instructions, adjust as needed:
~/dev/pootle
.In order to install Pootle first create a virtual environment. The virtual environment allows you to install dependencies independent of your system packages.
Please install virtualenv from your system packages, e.g. on Debian:
$ sudo apt-get install python-virtualenv
Otherwise you can install virtualenv using pip:
$ sudo pip install virtualenv
Now create a virtual environment on your location of choice by issuing the virtualenv command:
$ cd ~/dev/pootle
$ virtualenv env
Note
for versions of virtualenv prior to 1.10, you may need to
call virtualenv with the --setuptools
option, to ensure the
correct environment.
To activate the virtual environment run the activate script:
$ source env/bin/activate
Once activated the virtual environment name will be prepended to the shell prompt.
Lastly, we want to make sure that we are using the latest version of pip and setuptools:
(env) $ pip install --upgrade pip setuptools
Use pip to install Pootle into the virtual environment:
(env) $ pip install --pre --process-dependency-links Pootle
This will also fetch and install Pootle’s dependencies.
Note
pip requires –pre to install pre-release versions of Pootle, i.e. alpha, beta and rc versions. You may require –process-dependency-links if Pootle depends on unreleased versions of third-party software.
To verify that everything installed correctly, you should be able to access the pootle command line tool within your environment.
(env) $ pootle --version Pootle 2.9.0rc1 (Django 1.11.12, Translate Toolkit 2.3.0)
Once Pootle has been installed, you will need to initialize a configuration file:
(env) $ pootle init
By default the configuration file is saved as ~/.pootle/pootle.conf
. You can pass
an alternative path as an argument if required - see the init
command for all
of the options.
Warning
This default configuration is enough to experiment with Pootle. Don’t use this configuration in a production environment.
The initial configuration includes the settings that you’re most likely to change. For further customization, see the full list of available settings.
Statistics tracking and various other background processes are managed by RQ. The rqworker
command needs to be run
continuously in order to process the jobs.
If you have not already done so you should install and start a Redis server.
You can start the worker in the background with the following command:
(env) $ pootle rqworker &
In a production environment you may want to run RQ workers as services.
See here for further information about RQ jobs in Pootle.
Before you run Pootle for the first time, you need to create the schema for
the database and populate it with initial data. This is done by executing the
migrate
and initdb
management commands:
Note
You will need to have an RQ worker running to complete this. Alternately, you can
use the --no-rq
.
(env) $ pootle migrate
(env) $ pootle initdb
Running initdb
will take some time as it will create the default
projects and stores.
Pootle needs at least one user with superuser rights which we create with the
createsuperuser
command.
(env) $ pootle createsuperuser
All users are required to verify their email before logging in. If you wish to
bypass this step you can use the verify_user
command.
For example to allow a user named admin
to log in without having to verify
their email address:
(env) $ pootle verify_user admin
The Django default server will be enough for quickly testing the software. To run it, just issue:
(env) $ pootle runserver --insecure
Warning
There are serious drawbacks to using runserver. Never use it in production.
And the server will start listening on port 8000. Pootle can then be accessed from your web browser at localhost:8000.
Now that you have Pootle up and running you may want to consider some of the following in order to build a production environment.
These are the instructions for upgrading Pootle from an older version to the current release.
Stop your running Pootle while you upgrade to prevent updates to your data during the migration process. If you have RQ workers running stop those also.
Warning
Before upgrading we strongly recommend that you backup your current system.
You must synchronize all your translations to disk:
(env) $ pootle sync_stores
If you are currently using SQLite for your database you will need to migrate to either MySQL (InnoDB) or PostgreSQL before you upgrade.
Before upgrading Pootle familiarize yourself with important changes since the version that you are upgrading from.
You should check that you have all of the necessary Pootle requirements and have installed all required system packages.
Warning
Pootle 2.7.0 or newer requires Python 2.7
If you are upgrading from a virtual environment using an earlier Python version, you must upgrade or rebuild your virtual environment first.
These instructions assume that you are using virtualenv and you have
activated a virtual environment named env
as follows:
$ source env/bin/activate
(env) $
You should now upgrade pip and setuptools to the latest version:
(env) $ pip install --upgrade pip setuptools
If you are upgrading from a version older than 2.6 you will need to first upgrade to the latest 2.6.x version and then you will be able to upgrade to the latest version.
(env) $ pip install --upgrade "Pootle>=2.6,<2.7"
(env) $ pootle setup
Warning
The 2.6.x releases are meant only as a migration step.
You must upgrade immediately to the latest version once setup has completed.
You should remove any stale Python bytecode files before upgrading.
Assuming you are in the root of your virtualenv folder you can run:
(env) $ pyclean .
Upgrade to the latest Pootle version:
(env) $ pip install --pre --process-dependency-links --upgrade Pootle
You should now update your custom Pootle settings to add, remove or adjust any settings that have changed. You may want to view the latest available settings.
You can check to see if there are any issues with your configuration settings that need to be resolved:
(env) $ pootle check
Note
If you are upgrading from a version of Pootle that uses
localsettings.py
then you may want to merge your old custom settings
with your settings conf file (default location
~/.pootle/pootle.conf
).
Statistics tracking and various other background processes are managed by RQ. The rqworker
command needs to be run
continuously in order to process the jobs.
If you have not already done so you should install and start a Redis server.
You can start the worker in the background with the following command:
(env) $ pootle rqworker &
In a production environment you may want to run RQ workers as services.
See here for further information about RQ jobs in Pootle.
Review the MySQL or PostgreSQL installation instructions for any changes that you need to make to your database.
If you run MySQL you will almost certainly need to make sure you have Time zone definition files loaded into the database.
Once you have updated your settings you can perform the database schema and data upgrade by running. This is done as follows:
(env) $ pootle migrate --fake-initial
You will also need to update the stats data held in Pootle
(env) $ pootle update_data
You must now update the translation checks. You will need to have an RQ worker running to complete this.
(env) $ pootle calculate_checks
This command will dispatch jobs to the RQ worker and may take some time.
If you wish to run calculate_checks
in the foreground without using
the RQ worker you can use the --no-rq
option.
If you are upgrading from a version prior to 2.8rc6, you will need to update
user scores using refresh_scores
.
(env) $ pootle refresh_scores --reset
(env) $ pootle refresh_scores
Redis might have cached HTML snippets referring to outdated static assets. In order for Pootle to return references to the newest assets these cached snippets must go away:
(env) $ pootle flush_cache --django-cache
Any accounts that do not have an email address registered will not be able to
log in. You can set the email for a user using the update_user_email
command.
For example to set the email for user admin
to admin@example.com
:
(env) $ pootle update_user_email admin admin@example.com
As of Pootle 2.7 users must now verify their email before they can log in.
You can use the verify_user
command to bypass email verification for
a specific user.
For example to automatically verify the admin user:
(env) $ pootle verify_user admin
If you wish to verify all of your existing users please see the
verify_user
command for further options.
Your projects should have been automatically migrated to use the Pootle FS
localfs
backend, so you need to check that everything was migrated
correctly:
(env) $ pootle fs
This command lists all the available Pootle FS projects. Make sure that all the existing projects in Pootle are listed here.
(env) $ pootle fs state MYPROJECT
This command will show the state of tracked files for a specific project. Run it for each of the projects listed by the previous command.
Ideally we want the state to show no results, i.e. that all files on disk and in Pootle are in sync and are being tracked. If the migration to Pootle FS was not able to fully understand your layout then there may be untracked files.
If there are untracked files you will want do some of these steps:
fs add
or fs rm
any files that should be
tracked but are currently untracked.Now that you have Pootle up and running you may want to consider some of the following in order to build a production environment.
Your Pootle installation will need to be flexible enough to handle the translation load. The recommended hardware depends highly on the performance you expect, the number of users you want to support, and the number and size of the files you want to host.
Whatever hardware you have, you will still benefit from performance improvements if you can optimize your system.
Your disk space should always be enough to store your files and your Pootle database, with some extra space available.
To run Pootle you need a computer running:
Or, any other Unix-like system.
Note
Pootle will not run on Windows since it uses RQ, whose workers cannot run on Windows.
Some developers do develop on Windows so these problems can be worked around for some of the development tasks.
Pootle should be able to run on any system that implements fork()
.
Python 2.7 is required, 3.x is not yet supported.
You will need a C compiler and the development libraries for Python and XML to be available on your system before you will be able to install your virtual environment. You will also need pip.
Eg. on a Debian-based system:
$ sudo apt-get install build-essential libxml2-dev libxslt-dev python-dev python-pip zlib1g-dev
You will also need to access to a working Redis server to provide caching to Pootle and for managing asynchronous workers.
To install and run Redis on a Debian-based system:
$ sudo apt-get install redis-server
$ sudo service redis-server start
Note
Pootle requires a minimum Redis server version of 2.8.4. If you are using Debian Wheezy you will need to install redis-server from backports.
Make sure to install the requirements for your chosen database, either MySQL or PostgreSQL.
In order to customise static resources such as CSS or JavaScript, you must install Node.js and npm.
On a Debian-based system you can install these with:
$ sudo apt-get install nodejs npm
On Debian Jessie and perhaps other distributions you also need to link the
nodejs
command to node
:
$ sudo update-alternatives --install /usr/bin/node node /usr/bin/nodejs 99
Now that you have the server running, you can setup a project to translate on the server.
To simplify the example we assume that:
You need to place the translation files for your new project in a location
where Pootle can find, read and write them. Pootle uses the
POOTLE_TRANSLATION_DIRECTORY
setting to find out where translation
files are stored on the server.
Our example project uses a GNU layout.
A GNU layout means that our project contains translation files named using language codes. Within the project there are no directories, just files. There can only be a single translation file per language in a project using this layout.
This is the simplest layout possible and the reason we are using it in our example.
Below you can see an example with two projects using the GNU layout:
`-- translations
`-- project1
| `-- po
| |-- de.po
| |-- fr.po
| |-- gl.po
| |-- pt_BR.po
| `-- templates.pot
`-- project2
`-- po
|-- af.po
|-- eu.po
|-- pt_BR.po
|-- templates.pot
`-- zu.po
Among the regular translation files there are two files named
templates.pot
. These are the template (master or reference) files that
contain the original strings. Usually these files contain only English strings,
however it is much less confusing to use the term templates
than e.g.
en
or English
.
To get started, create a my-project
directory in the location pointed
to by POOTLE_TRANSLATION_DIRECTORY
and place within it the
translation files for your new project. Make sure you have a
templates.pot
among those project translation files.
Note
If you want to integrate your project with a version control system you need to install the Pootle FS plugins for the VCS and connect Pootle FS with the VCS repository before creating the project in Pootle.
At the top of the user interface, you will see your newly created administrator username. Click on it and the main top menu will be displayed, then click on Admin (highlighted in red):
Now you are in the administration interface. Go to the Projects tab and you will see a New Project button:
Click on that button and the Add Project form will be displayed. Enter the new project’s details.
Code is a unique string that identifies the project. Also you must specify the File Types used in this new project.
The Filesystem backend tells Pootle FS how to synchronize the translations
in the project. The default option is localfs
which tells to synchronize
with the disk. Other backends allow to synchronize with version control systems
and they might require the installation of additional Pootle FS plugins.
Path or URL either is a path pointing to the translation files on the disk
if you are using the localfs
Filesystem backend, or is a URL pointing
to the VCS repository. The path can be an absolute path like
/path/to/translations/my-project/
or it can use the
{POOTLE_TRANSLATION_DIRECTORY}
placeholder if you are using the localfs
Filesystem backend, like {POOTLE_TRANSLATION_DIRECTORY}my-project
(the placeholder will be transparently replaced by the value of the
POOTLE_TRANSLATION_DIRECTORY
setting). If you are using a
Filesystem backend that uses a URL instead of a Path, for example for the
Git backend, this would be like git@github.com:user/repo.git
for a
GitHub repository.
The Path mapping field specifies the project layout using a glob like
/path/to/translation/files/<language_code>/<dir_path>/<filename>.<ext>
that
must start with /
, end with .<ext>
, and contain <language_code>
(the rest of the placeholders are optional). If you are using the localfs
Filesystem backend the Path mapping will be combined with the path
specified in the Path or URL field. For other backends it will be relative
to the root of the repository. Note you can easily fill this field by selecting
one of the available Path mapping presets.
You can also provide a Full Name easily readable for humans, if not the Code will be used. You don’t need to change the rest of the fields unless you need to further customize your project.
Note
If you are creating a project that is integrated with a version control repository then configure as follows:
git@github.com:user/repo.git
for a GitHub repository.In our example set the following:
my-project
.My project
Gettext PO (po/pot)
.localfs
.{POOTLE_TRANSLATION_DIRECTORY}my-project
.GNU style
.Once you are done click on the Save button below the form to create the project and save its Pootle FS configuration.
If not all of your project’s language codes do not match those available in Pootle, then you must add language mapping configurations for those languages. You can do that by clicking on the Languages link that is displayed below your project form:
The existing languages enabled for the project are listed, including an
optional mapping to the language code used in the filesystem. In the
screenshot below you can see that fr_FR
on filesystem maps to fr
on
Pootle:
In our example this form won’t list any language, so you must add new entries for the languages that need to be mapped specifying their Filesystem language code field.
Creating the project doesn’t actually import all the translations to Pootle. To do that you need to run the following on the command line of the Pootle server:
(env) $ pootle fs fetch my-project
(env) $ pootle fs state my-project
(env) $ pootle fs add my-project
(env) $ pootle fs sync my-project
Note
Read the using Pootle FS instructions in order to learn more about Pootle FS usage.
This will import all the translations from disk into Pootle, calculate the translation statistics and calculate the quality check failures. This might take a while for a large project.
Looking at your new project you will see that Pootle has imported all the
existing translations for the existing languages that you copied to the
my-project
directory within POOTLE_TRANSLATION_DIRECTORY
.
When you want to add a new language to the project, follow these steps.
Go to your project overview and select Languages in the navigation dropdown:
Note
Alternatively you can get the same result by clicking on the Languages link that is displayed below your project form in the administration interface:
The existing languages enabled for the project are listed, including an
optional mapping to the language code used in the filesystem (in the
screenshot below you can see that fr_FR
on filesystem maps to fr
on
Pootle). In our example we are adding Arabic to the project:
When you click the Save button the new language will be added for
translation. In large projects it may take some time to create the new
translation files from the templates
.
Note
If you want to enable translation to a language that doesn’t yet exist in your Pootle instance, then you will first have to add the language in the Languages tab in the administration interface, in a similar way to creating a new project.
Once the language is created you can enable translation to that new language in any project by following the instructions above.
Whenever developers introduce new strings, deprecate older ones, or change some of them this impacts Pootle and the languages being translated.
When any of these changes occur, you will need to generate a new
templates.pot
and use it to bring the translations in Pootle up-to-date
with the new templates.
Once you have created the new templates.pot
place it within your
project’s directory in POOTLE_TRANSLATION_DIRECTORY
, replacing the
file with the same name. After that, invoke the following command which will
update the template translations in the Pootle database.
(env) $ pootle update_stores --project=my-project --language=templates
This command will ensure that new strings are added to the project and any strings which have been removed are marked as deprecated, and thus will not be available for translation.
Now each of the languages will need to be brought into sync with the template language. The first step is to save all the Pootle translations to disk:
(env) $ pootle sync_stores --project=my-project
Then update all those translations on disk against the newer templates. We recommend you to update them on disk using the pot2po command line tool because it can handle other formats besides Gettext PO.
(env) $ cd $POOTLE_TRANSLATION_DIRECTORY # Use the actual path!
(env) $ cd my-project
(env) $ pot2po -t af.po templates.pot af.po # Repeat for each language by changing the language code.
Note
To preserve the existing translations we pass the previous
translation file to the -t
option.
When all the languages in the project have been updated you can push them back to Pootle:
(env) $ pootle update_stores --project=my-project
Note
If your project languages contain many translations you might want to perform the update against newer templates on a language by language basis.
These instructions provide additional steps for setting up Pootle with MySQL.
You should read the full installation instructions in order to install Pootle.
Pootle supports the versions of MySQL supported by Django, make sure that your installed version is supported.
Use the mysql command to create the user and database:
$ mysql -u root -p # You will be asked for the MySQL root password to log in
> CREATE DATABASE pootledb CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
> GRANT ALL PRIVILEGES ON pootledb.* TO pootle@localhost IDENTIFIED BY 'secret';
> FLUSH PRIVILEGES;
In addition to the system packages set out in the general installation requirements you will also require the MySQL client development headers in order to build the Python bindings, e.g. on a Debian-based system:
$ sudo apt-get install libmysqlclient-dev
Once you have set up and activated your virtual environment, you will need to install the MySQL driver.
You can do so as follows:
(env) $ pip install --pre --process-dependency-links Pootle[mysql]
When initializing your configuration you can specify params to set up your database, e.g.:
(env) $ pootle init --db mysql --db-name pootledb --db-user pootle
This will create a configuration file to connect to a MySQL database named
pootledb
hosted on localhost as the user pootle
. Please see the
init
command for all of the available options.
You will most likely want to edit your Pootle configuration (default location:
~/.pootle/pootle.conf
) to set your password.
Pootle makes use of time zones, follow Django’s instructions to load time zone tables into the MySQL database. This needs to be done just once for your MySQL server, not per database.
MySQL terminates idle connections after wait_timeout
seconds. Thus setting CONN_MAX_AGE
to a lower
value will be fine (it defaults to 0
). Persistent connections where
CONN_MAX_AGE
is None
can’t be used with
MySQL.
To learn more please check Django’s docs on persistent connections and connection management.
DATABASES = {
'default': {
...
'CONN_MAX_AGE': 0,
...
}
}
These instructions provide additional steps for setting up Pootle with PostgreSQL.
You should read the full installation instructions in order to install Pootle.
Pootle supports the versions of PostgreSQL supported by Django, make sure that your installed version is supported.
As the postgres
user you must create a database and database user:
$ sudo su postgres # On Ubuntu, may be different on your system
postgres@ $ createuser -P pootle # This will ask you to define the users password.
postgres@ $ createdb --encoding='utf8' --locale=en_US.utf8 --template=template0 --owner=pootle pootledb
In addition to the system packages set out in the general installation requirements you will also require the PostgreSQL client development headers in order to build the Python bindings, e.g. on Debian Jessie:
$ sudo apt-get install postgresql-server-dev-9.4
Once you have set up and activated your virtual environment, you will need to install the PostgreSQL bindings.
You can do so as follows:
(env) $ pip install --pre --process-dependency-links Pootle[postgresql]
When initializing your configuration you can specify params to set up your database, e.g.:
(env) $ pootle init --db postgresql --db-name pootledb --db-user pootle
This will create a configuration file to connect to a PostgreSQL database named
pootledb
hosted on localhost as the user pootle
. Please see the
init
command for all of the available options.
You will most likely want to edit your Pootle configuration (default location:
~/.pootle/pootle.conf
) to set your password.
The default value for CONN_MAX_AGE
is
0
. It means that Django creates a connection before every request and closes
it at the end. PostgreSQL supports persistent connections, and it will be fine
to set CONN_MAX_AGE
to None
.
To learn more please check Django’s docs on persistent connections and connection management.
DATABASES = {
'default': {
...
'CONN_MAX_AGE': None,
...
}
}
Running Pootle with a front end web server will improve performance, give you more flexibility, and might be better for security. It is strongly recommended to run Pootle under Apache, Nginx, or a similar web server.
If you plan to run Pootle and/or RQ workers as system services, you can use whatever software you are familiar with for that purpose. For example Supervisor, Circus or daemontools might fit your needs.
You can use Apache either as a reverse proxy or straight with mod_wsgi.
If you want to reverse proxy through Apache, you will need to have mod_proxy installed for forwarding requests and configure it accordingly.
ProxyPass / http://localhost:8000/
ProxyPassReverse / http://localhost:8000/
ProxyPreserveHost On
Make sure to review your global Apache settings (something like
/etc/apache2/httpd.conf or /etc/httpd/conf/httpd.conf) for the server-pool
settings. The default settings provided by Apache are too high for running a web
application like Pootle. The ideal settings depend heavily on your hardware and
the number of users you expect to have. A moderate server with 1GB memory might
set MaxClients
to something like 20
, for example.
Make sure Apache has read access to all of Pootle’s files and write access to
the POOTLE_TRANSLATION_DIRECTORY
directory.
Note
Most of the paths present in the examples in this section are the result of deploying Pootle using a Python virtualenv as told in the Setting up the Environment section from the Quickstart installation instructions.
If for any reason you have different paths, you will have to adjust the examples before using them.
For example the path /var/www/pootle/env/lib/python2.7/site-packages/
will be different if you have another Python version, or if the Python
virtualenv is located in any other place.
First it is necessary to create a WSGI loader script:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import site
import sys
# You probably will need to change these paths to match your deployment,
# most likely because of the Python version you are using.
ALLDIRS = [
'/var/www/pootle/env/lib/python2.7/site-packages',
'/var/www/pootle/env/lib/python2.7/site-packages/pootle/apps',
]
# Remember original sys.path.
prev_sys_path = list(sys.path)
# Add each new site-packages directory.
for directory in ALLDIRS:
site.addsitedir(directory)
# Reorder sys.path so new directories at the front.
new_sys_path = []
for item in list(sys.path):
if item not in prev_sys_path:
new_sys_path.append(item)
sys.path.remove(item)
sys.path[:0] = new_sys_path
# Set the Pootle settings module as DJANGO_SETTINGS_MODULE.
os.environ['DJANGO_SETTINGS_MODULE'] = 'pootle.settings'
# Set the WSGI application.
def application(environ, start_response):
"""Wrapper for Django's WSGIHandler().
This allows to get values specified by SetEnv in the Apache
configuration or interpose other changes to that environment, like
installing middleware.
"""
try:
os.environ['POOTLE_SETTINGS'] = environ['POOTLE_SETTINGS']
except KeyError:
pass
from django.core.wsgi import get_wsgi_application
_wsgi_application = get_wsgi_application()
return _wsgi_application(environ, start_response)
Place it in /var/www/pootle/wsgi.py
. If you use a different location
remember to update the Apache configuration accordingly.
A sample Apache configuration with mod_wsgi might look like this:
WSGIRestrictEmbedded On
WSGIPythonOptimize 1
<VirtualHost *:80>
# Domain for the Pootle server. Use 'localhost' for local deployments.
#
# If you want to deploy on example.com/your-pootle/ rather than in
# my-pootle.example.com/ you will have to do the following changes to
# this sample Apache configuration:
#
# - Change the ServerName directive to:
# ServerName example.com
# - Change the WSGIScriptAlias directive to (note that /your-pootle must
# not end with a slash):
# WSGIScriptAlias /your-pootle /var/www/pootle/wsgi.py
# - Change the Alias directive for 'assets' to include the '/your-pootle'.
# - Include the following settings in your custom Pootle settings:
# STATIC_URL = '/your-pootle/assets/'
# FORCE_SCRIPT_NAME = '/your-pootle'
# - If you have previously calculated the stats:
# - Restart the RQ workers.
# - Run refresh_stats to recalculate the stats data.
ServerName my-pootle.example.com
# Set the 'POOTLE_SETTINGS' environment variable pointing at your custom
# Pootle settings file. An initial settings file can be created using
# 'pootle init'
#
# This might require enabling the 'env' module.
SetEnv POOTLE_SETTINGS /var/www/pootle/your_custom_settings.conf
# The following two optional lines enable the "daemon mode" which
# limits the number of processes and therefore also keeps memory use
# more predictable.
WSGIDaemonProcess pootle processes=2 threads=3 stack-size=1048576 maximum-requests=500 inactivity-timeout=300 display-name=%{GROUP} python-path=/var/www/pootle/env/lib/python2.7/site-packages
WSGIProcessGroup pootle
# Point to the WSGI loader script.
WSGIScriptAlias / /var/www/pootle/wsgi.py
# Turn off directory listing by default.
Options -Indexes
# Compress before being sent to the client over the network.
# This might require enabling the 'deflate' module.
SetOutputFilter DEFLATE
AddOutputFilterByType DEFLATE text/html text/css text/plain text/xml application/x-javascript
# Set expiration for some types of files.
# This might require enabling the 'expires' module.
ExpiresActive On
ExpiresByType image/jpg "access plus 10 years"
ExpiresByType image/png "access plus 10 years"
ExpiresByType text/css "access plus 10 years"
ExpiresByType application/x-javascript "access plus 10 years"
# Optimal caching by proxies.
# This might require enabling the 'headers' module.
Header set Cache-Control "public"
# Directly serve static files like css and images, no need to go
# through mod_wsgi and Django. For high performance consider having a
# separate server.
Alias /assets /var/www/pootle/env/lib/python2.7/site-packages/pootle/assets
<Directory /var/www/pootle/env/lib/python2.7/site-packages/pootle/assets>
Require all granted
# For apache 2.2, comment the line directly above, and uncommend the two lines directly below
#Order deny,allow
#Allow from all
</Directory>
</VirtualHost>
You can find more information in the Django docs about Apache and mod_wsgi.
If you do not have access to the main Apache configuration, you should still be able to configure things correctly using the .htaccess file.
More information on configuring mod_wsgi (including .htaccess)
Running Pootle under a web server such as Nginx will improve performance. For more information about Nginx and WSGI, visit Nginx’s WSGI module page
A Pootle server is made up of static and dynamic content. By default Pootle serves all content, and for low-latency purposes it is better to get other webserver to serve the content that does not change, the static content. It is just the issue of low latency and making the translation experience more interactive that calls you to proxy through Nginx. The following steps show you how to setup Pootle to proxy through Nginx.
The default Pootle server runs at port 8000 and for convenience and simplicity does ugly things such as serving static files — you should definitely avoid that in production environments.
By proxying Pootle through nginx, the web server will serve all the static media and the dynamic content will be produced by the app server.
server {
listen 80;
server_name pootle.example.com;
access_log /path/to/pootle/logs/nginx-access.log;
gzip on; # Enable gzip compression
charset utf-8;
location /assets {
alias /path/to/pootle/env/lib/python2.6/site-packages/pootle/assets/;
expires 14d;
access_log off;
}
location / {
proxy_pass http://localhost:8000;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
Note
Please note that the database migration must be performed before upgrading Pootle.
Using dumpdata
and loaddata
commands to migrate between
databases is no longer supported.
The MySQL MyISAM backend is no longer supported. Use InnoDB instead.
There are several tools available to migrate between databases. We recommend having a look through this list for the following supported backends:
You will find all the Pootle-specific settings in this document.
If you have upgraded, you might want to compare your previous copy to the one distributed with the Pootle version for any new settings you might be interested in.
When starting Pootle with the pootle runner script, by default it
will try to load custom settings from the ~/.pootle/pootle.conf
file.
These settings will override the defaults set by Pootle.
An alternative location for the settings file can be specified by setting the
-c </path/to/settings.conf/>
flag when executing the runner. You can also
set the POOTLE_SETTINGS
environment variable to specify the path to
the custom configuration file. The environment variable will take precedence
over the command-line flag.
This is a list of Pootle-specific settings grouped by the file they’re contained and ordered alphabetically.
This file contains base configuration settings.
POOTLE_INSTANCE_ID
POOTLE_TITLE
Default: 'Pootle Translation Server'
The name of the Pootle server.
POOTLE_SQL_MIGRATIONS
Default: True
New in version 2.8.
Some migrations have been optimized by using SQL directly and bypassing the Django ORM (Currently only for mysql migrations).
This could cause problems if, for example, you have developed custom models with foreign keys to Pootle models.
Backend and caching settings.
POOTLE_CACHE_TIMEOUT
Default: 604800
(a week)
New in version 2.7.
Time in seconds to keep certain objects cached in memory (template fragments, language and project lists, permissions, etc.).
Note that for anonymous users Pootle also uses Django’s caching middleware, and its settings can be configured separately.
POOTLE_LOG_DIRECTORY
Default: working_path('log')
New in version 2.7.
The directory where Pootle writes event logs to. These are high-level logs of events on store/unit changes and pootle commands executed.
Site-specific settings.
POOTLE_CONTACT_ENABLED
Default: True
Controls whether users will be able to use the contact form. The address to
receive messages is controlled by POOTLE_CONTACT_EMAIL
.
POOTLE_CONTACT_EMAIL
Default: info@YOUR_DOMAIN.com
Address to receive messages sent through the contact form. This will only
have effect if POOTLE_CONTACT_ENABLED
is set to True
.
POOTLE_CONTACT_REPORT_EMAIL
Default: POOTLE_CONTACT_EMAIL
New in version 2.7.
Email address to report errors on strings.
POOTLE_EMAIL_FEEDBACK_ENABLED
Default: False
New in version 2.8.
Controls whether emails are sent to suggesters when a reviewer accepts or rejects their suggestions providing some comment for the suggester.
POOTLE_CANONICAL_URL
Default: http://localhost
New in version 2.8.
Canonical URL, without trailing slash, used when deriving the URLs to
send out emails. If you use the django.contrib.sites
framework set this
to blank string.
POOTLE_CUSTOM_LOGO
Default: ""
New in version 2.8.
Custom logo URL - this can be an absolute or relative URL.
POOTLE_FAVICONS_PATH
Default: "/assets/favicon"
New in version 2.8.
Customisable favicon path. Should not end with trailing slash.
Configuration settings for applications used by Pootle.
POOTLE_SIGNUP_ENABLED
Default: True
Changed in version 2.7.
Controls whether user sign ups are allowed or not. If set to False
,
administrators will still be able to create new user accounts.
POOTLE_CUSTOM_TEMPLATE_CONTEXT
Default: {}
Changed in version 2.7.
Custom template context dictionary. The values will be available in the
templates as {{ custom.<key> }}
.
POOTLE_LEGALPAGE_NOCHECK_PREFIXES
Default: ('/about', '/accounts', '/admin', '/contact', '/jsi18n', '/pages', )
Changed in version 2.7.
List of path prefixes where the LegalAgreementMiddleware
will check
if the current logged-in user has agreed all the legal documents defined
for the Pootle instance. Don’t change this unless you know what you’re
doing.
POOTLE_META_USERS
Default: ()
New in version 2.7.
Additional meta, or non-human, accounts. Pootle already manages the ‘system’ and ‘nobody’ users who own system updates to translations and submissions by anonymous users. These meta accounts have their own simple public profiles and won’t track scores.
POOTLE_MARKUP_FILTER
Default: (None, {})
Two-tuple defining the markup filter to apply in certain textareas.
markdown
and html
(deprecated).Changed in version 2.8: Support for textile
, restructuredtext
and html
formats has
been deprecated.
Examples:
POOTLE_MARKUP_FILTER = ('markdown', {})
POOTLE_MARKUP_FILTER = ('markdown', {
'clean': {
'extra_tags': ['div'],
'extra_attrs': {
'*': ['style']
'img': ['alt'],
},
'extra_styles': [
'color',
'font-weight'
],
},
})
POOTLE_CAPTCHA_ENABLED
Default: True
Enable spam prevention through a captcha.
POOTLE_REPORTS_MARK_FUNC
Default: ''
(empty string)
New in version 2.7.
The graph of a user’s activity, within reports, can be marked to indicate events by using this function. The setting must contain an import path to such a marking function (string).
The function receives the user and graph ranges and returns an array of applicable marks.
Parameters:
username
- user for whom we’re producing this graphstart
(datetime) - start date of the graphend
(datetime) - end date of the graphThe function must return an array of dictionaries (marks), where every mark has the following properties:
position
, specifying the point in the x-axis where the mark should
be set (UNIX timestamp multiplied by 1000), andlabel
specifying the text that will be displayed next to the mark.POOTLE_SCORES
Default:
{
'suggestion_add': 0,
'suggestion_accept': .1,
'suggestion_reject': .1,
'comment_updated': .1,
'target_updated': .3,
'state_translated': .6,
'state_fuzzy': .1,
'state_unfuzzy': .1,
}
New in version 2.8.
The default score is based on the wordcount of the source text. The values of the various parameters are used as a multiplier to arrive at the score atributed to the translators and reviewers.
Thus:
score = source wordcount * multiplier
When a translator translates a 10 word string they would get a score using
the state_translated
multiplier of 0.6.
6 = 10 words * 0.6
Parameters:
suggestion_*
- scoring for the events of adding a new suggestions,
accepting or rejecting existing suggestions. By default adding a
suggestion gives no score, to prevent users from gaming the system.comment_*
- making changes to the string comment.target_*
- adjusting the existing translation.state_*
- changing the state of the string, this includes translation
and fuzzy setting.POOTLE_FS_WORKING_PATH
Default: working_path('.pootle_fs/tmp/')
New in version 2.8.
The directory that Pootle FS uses to store temporary data for handling the projects.
Warning
This directory can potentially get very large, so you need to place it somewhere with plenty of room.
Translation environment configuration settings.
AMAGAMA_URL
Default: https://amagama-live.translatehouse.org/api/v1/
URL to an amaGama Translation Memory server. The default service should work fine, but if you have a custom server set it here.
This URL must point to the public API URL which returns JSON. Don’t forget the trailing slash.
AMAGAMA_SOURCE_LANGUAGES
Default: ('en', 'en_US', 'en-US')
List of available source languages in amaGama server pointed to by
AMAGAMA_URL
.
POOTLE_SYNC_FILE_MODE
Default: 0o644
Changed in version 2.7.
On POSIX systems, files synchronized to disk will be assigned this
permission. Use 0o644
for publically-readable files or 0o600
if you
want only the Pootle user to be able to read them.
POOTLE_TM_SERVER
New in version 2.7.
Changed in version 2.7.3: Added the WEIGHT
and
MIN_SIMILARITY
options. Also
added another default TM used to import external translations from files.
Default: {}
(empty dict)
Example configuration for local/external TM server:
{
'local': {
'ENGINE': 'pootle.core.search.backends.ElasticSearchBackend',
'HOST': 'localhost',
'PORT': 9200,
'INDEX_NAME': 'translations',
'WEIGHT': 1,
},
'external': {
'ENGINE': 'pootle.core.search.backends.ElasticSearchBackend',
'HOST': 'localhost',
'PORT': 9200,
'INDEX_NAME': 'external-translations',
'WEIGHT': 0.9,
},
}
This is configured to access a standard Elasticsearch setup. Change the
settings for any non-standard setup. Change HOST
and PORT
settings
as required.
The default local
TM is automatically updated every time a new
translation is submitted. The other TMs are not automatically updated so they
can be trusted to provide selected high quality translations.
Every TM server must have its own unique INDEX_NAME
.
WEIGHT
provides a weighting factor to alter the final score for TM
results from this TM server. Valid values are between 0.0
and 1.0
,
both included. Defaults to 1.0
if not provided.
MIN_SIMILARITY
serves as a threshold value to filter out results that are
potentially too far from the source text. The Levenshtein distance is
considered when measuring how similar the text is from the source text, and
this represents a real value in the (0..1) range, 1 being 100% similarity.
The default value (0.7) should work fine in most cases, although your mileage
might vary.
POOTLE_MT_BACKENDS
Default: []
(empty list)
This setting enables translation suggestions through several online services.
The elements for the list are two-element tuples containing the name of the service and an optional API key.
Available options are:
GOOGLE_TRANSLATE
: Google Translate service.YANDEX_TRANSLATE
: Yandex.Translate service.PARSE_POOL_CULL_FREQUENCY
Default: 4
When the pool fills up, 1/PARSE_POOL_CULL_FREQUENCY number of files will be removed from the pool.
PARSE_POOL_SIZE
Default: 40
To avoid rereading and reparsing translation files from disk on every request, Pootle keeps a pool of already parsed files in memory.
Larger pools will offer better performance, but higher memory usage (per server process).
POOTLE_TRANSLATION_DIRECTORY
Default: working_path('translations')
The directory where projects hosted on Pootle store their translation files.
sync_stores
will write to this directory and
update_stores
will read from this directory.
POOTLE_WORDCOUNT_FUNC
Default: translate.storage.statsdb.wordcount
New in version 2.7.
The import path to a function that provides wordcounts for Pootle.
Current options:
Adding a custom function allows you to alter how words are counted.
Warning
Changing this function requires that you recalculate the associated wordcounts.
ENABLE_ALT_SRC
Deprecated since version 2.5: Alternate source languages are now on by default. This ensures that translators have access to as much useful information as possible when translating.
POOTLE_TOP_STATS_CACHE_TIMEOUT
Deprecated since version 2.7: The overview page statistics rewrite has removed these statistics and the RQ based statistics has also removed the load of this type of data so this setting has been removed.
VCS_DIRECTORY
Deprecated since version 2.7: Version Control Support has been removed from Pootle. We feel we can
support version control better in future. You can currently make use of
sync_stores
and update_stores
to automate your own
integration.
CONTRIBUTORS_EXCLUDED_NAMES
Deprecated since version 2.7: The contributors page has been removed and is being replaced with better user statistics.
CONTRIBUTORS_EXCLUDED_PROJECT_NAMES
Deprecated since version 2.7: The contributors page has been removed and is being replaced with better user statistics.
MIN_AUTOTERMS
Deprecated since version 2.7: Terminology auto-extraction feature has been removed.
MAX_AUTOTERMS
Deprecated since version 2.7: Terminology auto-extraction feature has been removed.
DESCRIPTION
Deprecated since version 2.7: Pootle no longer displays site description on the landing page, but rather makes use of static pages to convey information to users in the sidebar. Use static pages and customization if you want to give users information about the Pootle site.
FUZZY_MATCH_MAX_LENGTH
Deprecated since version 2.7: Update against templates feature has been removed.
FUZZY_MATCH_MIN_SIMILARITY
Deprecated since version 2.7: Update against templates feature has been removed.
EXPORTED_DIRECTORY_MODE
Deprecated since version 2.7: Offline translation support was rewritten and the setting was unused.
POOTLE_QUALITY_CHECKER
Deprecated since version 2.8: To simplify checker code, the ability to have a custom quality checker was removed. To create custom checks, write them within the Translate Toolkit.
POOTLE_SCORE_COEFFICIENTS
Removed in version 2.8: A rewrite of the scoring in Pootle now uses POOTLE_SCORE
to
store score adjustment coefficients.
Pootle’s default logging has configurations for all important aspects of the server that we want to log. Pootle also logs to the ‘action’ logger that will log every user, system and command action executed on the server.
You can override the default logging directory by specifying the
POOTLE_LOG_DIRECTORY
setting.
The action logger logs each activity related to translation, units changes, store changes, command execution and other activities.
The generic log message is as follows (though some actions do produce slightly different log entries):
[2015-05-04T15:06:39] system X pootle update_tmserver
That is:
[date] user type message
Current action types are as follows:
Action | Group | Description |
---|---|---|
A | Translation | Translation submission added a translation |
C | Translation | An existing translation was changed |
D | Translation | An existing translation was deleted |
UA | Unit | A new unit was added |
UO | Unit | An existing unit was made obsolete |
UR | Unit | An obsolete unit was resurected i.e. reinstated |
UD | Unit | An existing unit was deleted |
SA | Store | A new store was added |
SO | Store | An existing store was made obsolete |
SR | Store | An obsolete store was reinstated |
SD | Store | An existing store was deleted |
X | Command | A pootle command was executed |
QM | Quality check | A quality check was muted (marked as a false positive) |
QU | Quality check | A quality check was unmuted (reenabled after having been muted) |
SC | Score | A users score has changed because of an action |
In addition the SC action type also has its own actions which track the actual type of activity that leads to changes in translation. These are used to track scores for the translators.
Action | Description |
---|---|
TA | unit translated |
TE | unit edited after someone else |
TX | unit edited after themselves |
TD | translation deleted by admin |
R | translation reviewed |
TF | translation’s fuzzy flag is set by admin |
XE | translation penalty [when translation deleted] |
XR | translation penalty [when review canceled] |
S | suggestion added |
SA | suggestion accepted (counted towards the suggestion author) |
SR | suggestion rejected (counted towards the suggestion author) |
RA | suggestion accepted (counted towards the reviewer) |
RR | suggestion rejected (counted towards the reviewer) |
Various of the action groups have different message structures as outlined here:
Translation:
date user action lang unit path translation
[2015-05-19T14:11:18] admin C af 2 /af/tutorial/stats-test.po Twee
[2015-05-19T14:12:17] admin A af 3 /af/tutorial/stats-test.po Drie
[2015-05-19T14:13:05] admin D af 1 /af/tutorial/stats-test.po
Unit:
date user action language unit file translation
[2015-05-06T16:25:20] system UA am 4109 /am/terminology/gnome/am.po MSDOS
[2015-05-06T16:37:05] system UA cs 12043 /cs/terminology/gnome/cs.po přepínač
Store:
date user action path store
[2015-05-05T20:23:37] system SA /templates/tutorial/tutorial.pot 1
Command:
date user action command
[2015-05-06T11:24:28] system X pootle update_stores --project=vfolders
[2015-05-05T20:22:46] system X pootle migrate
Quality check:
date user action lang unit path translation
[2015-05-19T14:16:36] admin QM af 855 /af/terminology/gnome-terminologie.po lug
[2015-05-19T14:17:44] admin QU af 855 /af/terminology/gnome-terminologie.po lug
Score:
date user SC score_delta score_action #unit NS=wordcount S=similarity total
[2015-05-19T14:19:11] admin SC 1.0 TA #1 NS=1 S=0.0 (total: 2.28571428571)
The sync_stores
and update_stores
commands will produce a
number of logs to report any activity that results from those commands.
update_stores:
[$date] [update] updated $number units in $store_path [revision: $revision]
[2015-05-19T21:06:24] [update] updated 1 units in /an/libo_ui/dictionaries/pt_PT.po [revision: 58]
sync_stores:
[$date] [sync] File saved; updated $number units in $store_path [revision: $revision]
[2015-05-19T23:11:50] [sync] File saved; updated 1 units in /an/libo_ui/avmedia/source/viewer.po [revision: 0]
This page lists extra optional software you can install to improve Pootle’s performance. Some configuration tips are given too.
By installing optional software you can gain performance and extra features.
You should really switch to a real database backend in production environments.
Adjust the DATABASES
setting accordingly.
You should really run Pootle behind a real web server, at least to serve static content. For generating the dynamic content, you can also use alternative WSGI servers that might better suit your environment.
With a few extra steps, you can support more users and more data. Here are some tips for performance tuning on your Pootle installation.
DEBUG
mode is disabled.PARSE_POOL_SIZE
if you have enough memory available.'django.contrib.sessions.backends.cached_db'
.For Apache, review your server settings so that you don’t support too many or too few clients. Supporting too many clients increases memory usage, and can actually reduce performance.
No specific settings can be recommended, since this depends heavily on your
users, your files, and your hardware. However the default value for the
MaxClient
directive (usually 256) is almost always too high. Experiment
with values between 10 and 80.
Using MySQL with InnoDB backend is well tested. MyISAM is no longer supported. You can migrate your current database if you already have data you don’t want to lose.
Pootle uses a caching system to improve performance. It is an essential part of your Pootle installation. It is based on Django’s caching system, and is used for various things:
Without a well functioning cache system, Pootle could be slow.
Pootle is configured with a these named caches:
'default'
– all non specified cache data and all Django cache data.'redis'
– all RQ data related and revision counter.'lru'
– all lru cache data.In large installations you may want to setup separate caches to improve cache performance. You can then setup caching parameters for each cache separately.
Django supports multiple cache backends
(methods of storing cache data). However, Redis is the only cache backend
supported by Pootle. We use some custom features of Redis so cannot support
other backends. You can customise the Redis cache settings by overriding the
value of CACHES
in your configuration file.
Pootle’s backend for authenticating and authorizing users is provided by django-allauth, and it comes with a heavily-customized client-side user interface.
Note that while Allauth supports local and social sign-in flows, not all of them have been equally-tested on Pootle, so your mileage might vary.
At the same time, Allauth also provides tons of settings
which deployments can configure to their needs, although some of them
clash directly with how our workflow has been designed. For instance,
leaving UNIQUE_EMAIL = True
becomes a hard requirement.
Each third party social authentication provider has its own requirements, although most of them implement similar protocols (OAuth, OAuth2, OpenID etc.).
Usually providers require consumers to register their apps on the provider website. On the Pootle side of things, your provider might need to be registered as a social app against your host. In order to do this you will need to insert a few records into your SQL database.
An example with GitHub follows.
Register your app against your host.
Some descriptive name
URL to your Pootle server, e.g. http://foo.bar.tld
Some descriptive text
URL to the callback endpoint of your provider in the Pootle server, e.g.
http://foo.bar.tld/accounts/github/login/callback/
Let Allauth know about your social provider.
UPDATE django_site SET DOMAIN = 'foo.bar.tld', name = 'Site name' WHERE id=1;
INSERT INTO socialaccount_socialapp (provider, name, secret, client_id, 'key')
VALUES ("github", "GitHub", "---Client-Secret-from-above---",
"---Client-ID-from-above---", "");
INSERT INTO socialaccount_socialapp_sites (socialapp_id, site_id)
VALUES (1,1);
Note the first line simply sets the domain name for the default site; you can omit it if it’s already up-to-date.
The management commands are administration commands provided by Django, Pootle
or any external Django app being used with Pootle. You will usually run these
commands by issuing pootle <command> [options]
.
For example, to get information about all available management commands, you will run:
(env) $ pootle help
These commands will go through all existing projects performing maintenance tasks. The tasks are all available through the web interface but on a project by project or file by file basis.
--project
,
--language
¶The commands target can be limited in a more flexible way using the
--project
--language
command line options. They can be
repeated to indicate multiple languages or projects. If you use both options
together it will only match the files that match both languages and projects
selected.
For example, to calculate_checks for the tutorial project only, run:
(env) $ pootle calculate_checks --project=tutorial
To only calculate the Zulu and Basque language files within the tutorial project, run:
(env) $ pootle calculate_checks --project=tutorial --language=zu --language=eu
--no-rq
¶New in version 2.7.1.
Some of the commands work asynchronously and will schedule jobs to RQ workers,
rather than running them in the command process. You can change this behaviour
using the --no-rq
command line option.
This can be useful for running pootle commands in bash scripts or automating installation/upgrade/migration. It can also be useful for debugging otherwise asynchronous jobs.
For example, to run calculate_checks
in the command process and wait
for the process to terminate:
(env) $ pootle calculate_checks --no-rq
It is not generally safe to run commands in this mode if you have RQ workers active at the same time, as there is a risk that they conflict with other jobs dispatched to the workers.
--atomic
¶New in version 2.8: Default: tp
.
Available choices: tp
, all
, none
.
This option allows you to run CLI commands with atomic transactions.
The default is to commit changes on per-translation-project basis.
For example to run update_stores against all translation projects in a single transaction.
(env) $ pootle update_stores --atomic=all
--noinput
¶If there are RQ workers running, the command will ask for confirmation before
proceeding. This can be overridden using the --noinput
flag, in
which case the command will run even if there are.
retry_failed_jobs
¶New in version 2.7.
Requeue failed RQ jobs.
Background RQ jobs can fail for various reasons. To push them back into the queue you can run this command.
Examine the RQ worker logs for tracebacks before trying to requeue your jobs.
update_data
¶New in version 2.8.
This command updates the stats data. The stats data update can be triggered for specific languages or projects.
--store
¶Use the --store
option to narrow the stats data calculation to a
specific store:
(env) $ pootle update_data --store=/fr/proj/mydir/mystore.po
Note this will also trigger the update of the stats data for items above the store, like for example directories above it, its language and its project.
calculate_checks
¶New in version 2.7.
This command will create a background job to go through all units and recalculate quality checks.
Note
Disabled projects are processed.
calculate_checks
will flush existing caches and update the quality
checks cache.
It’s necessary to run this command after upgrading Pootle if new quality checks are added.
The time it takes to complete the whole process will vary depending on the number of units you have in the database. If a user hits a page that needs to display stats but they haven’t been calculated yet, then a message will be displayed indicating that the stats being calculated.
--check
¶Use the --check
option to force calculation of a specified check. To
recalculate only the date_format
quality checks, run:
(env) $ pootle calculate_checks --check=date_format
Multiple checks can be specifed in one run as well:
(env) $ pootle calculate_checks --check=date_format --check=accelerators
flush_cache
¶New in version 2.8.0.
Flush cache.
Warning
You must first stop the workers if you flush redis cache.
--django-cache
¶Use the --django-cache
to flush the default
cache which keeps
Django templates, project permissions etc.
--rqdata
¶Use the --rqdata
to flush all data contained in redis
cache:
pending jobs, revision (which will be automatically restored), all data from
queues.
--lru
¶Use the --lru
to flush all lru cache data contained
in lru
cache.
--all
¶Use the --all
to flush all caches (default
, redis
, lru
) data.
refresh_scores
¶New in version 2.7.
Recalculates the scores for all users. It is possible to narrow down the calculation to specific projects and/or languages.
Warning
It is advisable to run this command while Pootle server is offline since the command can fail due to data being changed by users.
--reset
¶When the --reset
option is used , all score log data is removed and
zero score is set for all users.
sync_stores
¶Deprecated since version 2.9: Deprecated in favor of Pootle FS equivalent.
Note
Since version 2.9 all projects are managed by Pootle FS and therefore this command is now able to work with those projects.
Save all translations currently in the database to the file system, thereby
bringing the files under the POOTLE_TRANSLATION_DIRECTORY
directory
in sync with the Pootle database.
Note
Disabled projects are skipped.
You must run this command before taking backups or running scripts that modify the translation files directly on the file system, otherwise you might miss out on translations that are in the database but not yet saved to disk. In other words, translations are saved to disk only when you explicitly do so using this command.
For every file being synced, the in-DB Store
will be updated to
reflect the latest revision across the units in the file at the time of
syncing. This allows Pootle to make optimizations when syncing and
updating files, ignoring files that haven’t change.
The default behavior of sync_stores
can be altered by specifying
these parameters:
--force
¶Changed in version 2.9: This option has no effect anymore.
--overwrite
¶Changed in version 2.9: This option has no effect anymore.
--skip-missing
¶Ignores files missing on disk, and no new files will be created.
update_stores
¶Deprecated since version 2.9: Deprecated in favor of Pootle FS equivalent.
Note
Since version 2.9 all projects are managed by Pootle FS and therefore this command is now able to work with those projects.
Load translation files currently on the file system into the database, thereby
bringing the Pootle database in sync with the files under the
POOTLE_TRANSLATION_DIRECTORY
directory. Pootle will not detect
changes in the file system on its own. This is the opposite of
sync_stores
.
Note
Disabled projects are skipped.
Note
update_stores
does not manage the updating of translations
against templates, it simply loads translation files and translation
templates into Pootle. For a full understanding of the role of templates
and updating translations against templates read the templates section.
It also discovers new units, files and translation projects that were added on disk:
You must run this command after running scripts that modify translation files directly on the file system.
update_stores
accepts several options:
--force
¶Changed in version 2.9: This option has no effect anymore.
--overwrite
¶Mirrors the on-disk contents of the file. If there have been changes in the database since the last sync operation, these will be overwritten.
Warning
If files on the file system are corrupt, translations might be deleted from the database. Handle with care!
list_serializers
¶New in version 2.8.0.
List the installed serializers and deserializers on your system.
Available options:
-m
,
--model
¶List serializers for specified model. The model should be expressed as a
contenttype label - eg app_name
.``model_name``
-d
,
--deserializers
¶List available deserializers set up for our system.
list_languages
¶Lists all the language codes for languages hosted on the server. This can be useful for automation.
--modified-since
¶Accepts the --modified-since
parameter to list only those languages
modified since the revision given by revision
.
list_projects
¶Lists all the project codes on the server. This might can be useful for automation.
--modified-since
¶Accepts the --modified-since
parameter to list only those projects
modified since the revision given by revision
.
contributors
¶New in version 2.7.1.
Lists the contributors to a language, project or overall and the amount of contributions they have.
Available options:
--sort-by
¶Changed in version 2.8.0.
Specifies the sorting to be used. Valid options are contributions
(sort
by decreasing number of contributions) and username
(sort by user name,
alphabetically).
Default: username
.
--mailmerge
¶New in version 2.8.0.
Specifies to only output user names and emails. Users with no email are skipped.
--mailmerge
and
--include-anonymous
are mutually
exclusive.
--include-anonymous
¶New in version 2.8.0.
Specifies to include anonymous contributions.
--include-anonymous
and
--mailmerge
are mutually exclusive.
--since
¶New in version 2.8.0.
Only consider contributions since the specified date or datetime.
Date or datetime can be in any format accepted by python-dateutil
library, for example ISO 8601 format (2016-01-24T23:15:22+0000
or
2016-01-24
) or a string formatted like "2016-01-24 23:15:22 +0000"
(quotes included).
--until
¶New in version 2.8.0.
Only consider contributions until the specified date or datetime.
Date or datetime can be in any format accepted by python-dateutil
library, for example ISO 8601 format (2016-01-24T23:15:22+0000
or
2016-01-24
) or a string formatted like "2016-01-24 23:15:22 +0000"
(quotes included).
set_filetype
¶New in version 2.8.
This command sets file formats for projects, and also allows to convert files to another format.
--from-filetype
¶Convert stores of this file type.
--matching
¶Convert stores matching this path glob within the project.
For example, to add the properties format to a project, run:
(env) $ pootle set_filetype --project=myproj properties
To convert stores of po format to properties, run:
(env) $ pootle set_filetype --project=myproj --from-filetype=po properties
To convert stores matching a given path glob to properties format, run:
(env) $ pootle set_filetype --project=myproj --matching=mydir/myfile-* properties
revision
¶New in version 2.7.
Print the latest revision number.
The revision is a common system-wide counter for units. It is incremented with every translation action made from the browser. Zero length units that have been auto-translated also increment the unit revision.
--restore
¶The revision counter is stored in the database but also in cache for faster
retrieval. If for some reason the revision counter was removed or got
corrupted, passing the --restore
flag to the command will restore the
counter’s value based on the revision data available on the relational DB
backend. You shouldn’t need to ever run this, but if for instance you deleted
your cache you will need to restore the counter to ensure correct operation.
test_checks
¶New in version 2.7.
Tests any given string pair or unit against all or certain checks from the command line. This is useful for debugging and developing new checks.
--source
,
--target
¶String pairs can be specified by setting the values to be checked in the
--source=<"source_text">
and --target="<target_text>"
command-line arguments.
--unit
¶Alternatively, --unit=<unit_id>
can be used to reference an existing
unit from the database.
--check
¶By default, test_checks
tests all existing checks. When
--check=<checkname>
is set, only specific checks will be tested against.
dump
¶New in version 2.7.
Prints data or stats data (depending on --data
or --stats
option)
in specific format.
--data
¶object_id:class_name
8276:Directory name=android parent=/uk/ pootle_path=/uk/android/
24394:Store file=android/uk/strings.xml.po translation_project=/uk/android/ pootle_path=/uk/android/strings.xml.po name=strings.xml.pstate=2
806705:Unit source=Create Account target=Створити аккаунт source_wordcount=2 target_wordcount=2 developer_comment=create_account translator_commentlocations=File:\nstrings.xml\nID:\ne82a8ea14a0b9f92b1b67ebfde2c16e9 isobsolete=False isfuzzy=False istranslated=True
115654:Suggestion target_f=Необхідна електронна адреса user_id=104481
--stats
¶pootle_path total,translated,fuzzy,suggestions,criticals,is_dirty,last_action_unit_id,last_updated_unit_id
/uk/android/strings.xml.po 11126,10597,383,231,0,False,4710214,4735242
/uk/android/widget/strings.xml.po 339,339,0,26,0,False,2277376,3738609
/uk/android/widget/ 339,339,0,26,0,False,2277376,3738609
/uk/android/ 11465,10936,383,257,0,False,4710214,4735242
This command can be used by developers to check if all data kept after migrations or stats calculating algorithm was changed.
config
¶New in version 2.8.
Gets, sets, lists, appends and clears pootle configuration settings.
content_type
¶Optional positional argument to specify a model to manage configuration for.
object
¶Optional positional argument to specify the primary key of an object to
manage configuration for. You can use a field other than the primary key by
specifying -o
, but the field must be unique for the
request object when doing so.
-o
<field>
,
--object-field
<field>
¶Specify a field other than the primary key when specifying an object. It must be unique to the object specified.
-g
<key>
,
--get
<key>
¶Get value for specified key.
-l
<key>
,
--list
<key>
¶List values for specified key(s). This option can be specified multiple times.
-s
<key> <value>
,
--set
<key> <value>
¶Set value for specified key. The key must be unique or not exist already.
-a
<key> <value>
,
--append
<key> <value>
¶Append value for specified key.
-c
<key>
,
--clear
<key>
¶Clear value(s) for specified key.
-j
,
--json
¶Treat data as JSON when getting, setting, or appending values.
schema
¶New in version 2.8.
Dumps a JSON representation for the Pootle database schema, currently only MySQL, for debugging and comparison to a reference database schema.
These commands allow you to setup and manage Translation Memory.
update_tmserver
¶New in version 2.7.
Changed in version 2.7.3: Renamed --overwrite
to --refresh
.
Disabled projects’ translations are no longer added by default. It is also
possible to import translations from files.
Updates the local
server in POOTLE_TM_SERVER
. The command
reads translations from the current Pootle install and builds the TM resources
in the TM server.
If no options are provided, the command will only add new translations to the server.
--refresh
¶Use --refresh
to also update existing translations that have
been changed, besides adding any new translation.
--rebuild
¶To completely remove the TM and rebuild it adding all existing translations use
--rebuild
.
--tm
¶If no specific TM server is specified using --tm
, then the default
local
TM will be used. If the specified TM server doesn’t exist it will
be automatically created for you.
--include-disabled-projects
¶By default translations from disabled projects are not added to the TM, but
this can be changed by specifying --include-disabled-projects
.
--dry-run
¶To see how many units will be loaded into the server use --dry-run
,
no actual data will be loaded or deleted (the TM will be left unchanged):
(env) $ pootle update_tmserver --dry-run
(env) $ pootle update_tmserver --refresh --dry-run
(env) $ pootle update_tmserver --rebuild --dry-run
This command also allows to read translations from files and build the TM
resources in the external TM server. In order to do so it is mandatory to
provide the --tm
and --display-name
options, along with
some files to import.
--display-name
¶The display name is a label used to group translations within a TM. A given TM
can host translations for several display names. The display name can be used
to specify the name of the project from which the translations originate. The
display name will be shown on TM matches in the translation editor. To specify
a name use --display-name
:
(env) $ pootle update_tmserver --tm=libreoffice --display-name="LibreOffice 4.3 UI" TM_LibreOffice_4.3.gl.tmx
By default the command will only add new translations to the server. To rebuild
the server from scratch use --rebuild
to completely remove the TM and
rebuild it before importing the translations:
(env) $ pootle update_tmserver --rebuild --tm=mozilla --display-name="Foo 1.7" foo.po
Option --refresh
doesn’t apply when adding translations from files
on disk.
To see how many units will be loaded into the server use --dry-run
,
no actual data will be loaded:
(env) $ pootle update_tmserver --dry-run --tm=mozilla --display-name="Foo 1.7" foo.po
175045 translations to index
This command is capable of importing translations in multiple formats from several files and directories at once:
(env) $ pootle update_tmserver --tm=mozilla --display-name="Foo 1.7" bar.tmx foo.xliff fr/
--target-language
¶Use --target-language
to specify the target language ISO code for the
imported translations in case it is not possible to guess it from the
translation files or if the code is incorrect:
(env) $ pootle update_tmserver --target-language=af --tm=mozilla --display-name="Foo 1.7" foo.po bar.tmx
These commands allow you to perform tasks with virtual folders from the command line.
add_vfolders
¶New in version 2.7.
Creates virtual folders from a JSON file. If the specified virtual folders already exist then they are updated.
The vfolder format defines how to specify a virtual folder that fits your needs.
This command requires a mandatory filename argument.
(env) $ pootle add_vfolders virtual_folders.json
Export and Import translation files in Pootle. The operation can be thought of
best as offline operations to assist with offline translation, unlike
sync_stores
and update_stores
the operations here are
designed to cater for translators working outside of Pootle.
The import
and export
commands are designed to mimic the
operations of Download and Upload from the Pootle UI.
export
¶New in version 2.7.
Download a file for offline translation.
Note
This mimics the editor’s download functionality and its primary purpose is to test the operation of downloads from the command line.
A file or a .zip of files is provided as output. The file headers include a revision counter to assist Pootle to detetmine how to handle subsequent uploads of the file.
Available options:
--tmx
¶New in version 2.8.0.
Export every translation project as one zipped TMX file
into MEDIA_ROOT
directory.
--rotate
¶New in version 2.8.0.
Remove old exported zipped TMX files (except previous one)
from MEDIA_ROOT
directory after current exported file is saved.
import
¶New in version 2.7.
Upload a file that was altered offline.
Note
This mimics the editor’s upload functionality and its primary purpose is to test the operation of uploads from the command line.
A file or a .zip is submitted to Pootle and based on the revision counter of
the Store
on Pootle it will be uploaded or rejected. If the revision
counter is older than on Pootle, that is someone has translated while the file
was offline, then it will be rejected. Otherwise the translations in the file
are accepted.
Available options:
--user
¶New in version 2.7.3.
Import file(s) as given user. The user with the provided username must exist.
Default: system
.
These commands expose the database installation and upgrade process from the command line.
init
¶Create the initial configuration for Pootle.
Available options:
--db
¶New in version 2.7.1.
The database backend that you are using
Default: sqlite
.
Available options: sqlite
, mysql
, postgresql
.
--db-name
¶New in version 2.7.1.
The database name or path to database file if you are using sqlite.
Default for sqlite: dbs/pootle.db
.
Default for mysql/postgresql: pootledb
.
--db-user
¶New in version 2.7.1.
Name of the database user. Not used with sqlite.
Default: pootle
.
--db-host
¶New in version 2.7.1.
Database host to connect to. Not used with sqlite.
Default: localhost
.
--db-port
¶New in version 2.7.1.
Port to connect to database on. Defaults to database backend’s default port. Not used with sqlite.
--dev
¶New in version 2.8.
Creates a development configuration instead.
--yes
¶New in version 2.9.
Answer ‘yes’ to any questions blocking overwrite of existing config files.
initdb
¶Initializes a new Pootle install.
This is an optional part of Pootle’s install process, it creates the default admin user, populates the language table with several languages, initializes the terminology project, and creates the tutorial project among other tasks.
initdb
can only be run after migrate
.
initdb
accepts the following option:
New in version 2.7.3.
--no-projects
¶Don’t create the default terminology
and tutorial
projects.
Note
initdb
will import translations into the database, so
can be slow to run. You should have an rqworker
running or run with
the –no-rq.
Running the Django admin collectstatic
command finds and
extracts static content such as images, CSS and JavaScript files used by the
Pootle server, so that they can be served separately from a static webserver.
Typically, this is run with the --clear
--noinput
options, to flush any
existing static data and use default answers for the content finders.
Pootle uses the Django app django-assets interface of webassets to minify
and bundle CSS and JavaScript; this app has a management command that is used
to make these preparations using the command assets build
. This command is
usually executed after the collectstatic one.
webpack
¶New in version 2.7.
The webpack tool is used under the hood to bundle JavaScript scripts, and this management command is a convenient wrapper that sets everything up ready for production and makes sure to include any 3rd party customizations.
--dev
¶When the --dev
flag is enabled, development builds will be created
and the process will start a watchdog to track any client-side scripts for
changes. Use this only when developing Pootle.
fs
¶To interact with Pootle FS we use multiple subcommands:
sync
- Execute staged actionsNote
The action staging commands require that you run
sync
in order to actually perform the staged actions.
Pootle FS action and execution subcommands take the -p
and
-P
options which allow you to specify a glob to limit which files or
stores are affected by the command.
-p
--fs-path
¶Only affect files whose filesystem path matches a given glob.
(env) $ pootle fs add --fs-path=MYPROJECT/af/directory/file.po MYPROJECT
(env) $ pootle fs add --fs-path=MYPROJECT/af/* MYPROJECT
(env) $ pootle fs add --fs-path=MYPROJECT/af/*/file.po MYPROJECT
(env) $ pootle fs add --fs-path=MYPROJECT/af/directory/*.po MYPROJECT
Note
The path should be relative to the Pootle FS URL setting for the project.
-P
--pootle-path
¶Only affect files whose Pootle path matches a given glob.
(env) $ pootle fs add --pootle-path=/af/MYPROJECT/directory/file.po MYPROJECT
(env) $ pootle fs add --pootle-path=/af/MYPROJECT/* MYPROJECT
(env) $ pootle fs add --pootle-path=/af/MYPROJECT/*/file.po MYPROJECT
(env) $ pootle fs add --pootle-path=/af/MYPROJECT/directory/*.po MYPROJECT
Note
Keep in mind that Pootle paths always start with /.
add
¶New in version 2.8.0.
Stage for adding any new or changed stores from Pootle to the filesystem:
(env) $ pootle fs add MYPROJECT
This command is the functional opposite of the fetch
command.
--force
¶Conflicting files on the filesystem will be staged to be overwritten by the Pootle store.
(env) $ pootle fs add --force MYPROJECT
fetch
¶New in version 2.8.0.
Stage for fetching any new or changed files from the filesystem to Pootle:
(env) $ pootle fs fetch MYPROJECT
This command is the functional opposite of the add
command.
--force
¶Conflicting stores in Pootle to be overwritten with the filesystem file.
(env) $ pootle fs fetch --force MYPROJECT
info
¶New in version 2.8.0.
Retrieve the filesystem info for a project.
(env) $ pootle fs info MYPROJECT
resolve
¶New in version 2.8.0.
Stage for merging any stores/files that have either been updated both in Pootle and filesystem.
When merging, if there are conflicts in any specific translation unit the default behavior is to keep the filesystem version and convert the Pootle version into a suggestion. Suggestions can then we reviewed by translators to ensure any corrections are correctly incorporated.
When there are no conflicts in unit resolve
will handle the merge
without user input:
(env) $ pootle fs resolve MYPROJECT
--pootle-wins
¶Alter the default conflict resolution of filesystem winning to instead use the Pootle version as the correct translation and converting the filesystem version into a suggestion.
(env) $ pootle fs resolve --pootle-wins MYPROJECT
--overwrite
¶Discard all translations. Use only those translations from the filesytem,
by default, or from Pootle if used together with --pootle-wins
(env) $ pootle fs resolve --overwrite MYPROJECT
rm
¶New in version 2.8.0.
Remove any matched:
(env) $ pootle fs rm MYPROJECT
--force
¶Stage for removal conflicting/untracked files and/or stores.
(env) $ pootle fs rm --force MYPROJECT
state
¶New in version 2.8.0.
List the status of stores in Pootle and files on the filesystem.
(env) $ pootle fs state MYPROJECT
-t
--type
¶Restrict to specified Pootle FS status.
(env) $ pootle fs state -t pootle_staged MYPROJECT
sync
¶New in version 2.8.0.
Commit any staged changes, effectively synchronizing the filesystem and Pootle. This command is run after other Pootle FS commands have been used to stage changes.
(env) $ pootle fs sync MYPROJECT
unstage
¶New in version 2.8.0.
Unstage any staged Pootle FS actions. This allows you to remove any staged actions which you might have added erroneously.
(env) $ pootle fs unstage MYPROJECT
find_duplicate_emails
¶New in version 2.7.1.
As of Pootle version 2.8, it will no longer be possible to have users with duplicate emails. This command will find any user accounts that have duplicate emails. It also shows the last login time for each affected user and indicates if they are superusers of the site.
(env) $ pootle find_duplicate_emails
merge_user
¶New in version 2.7.1.
This can be used if you have a user with two accounts and need to merge one account into another. This will re-assign all submissions, units and suggestions, but not any of the user’s profile data.
This command requires 2 mandatory arguments, src_username
and
target_username
, both should be valid usernames for users of your site.
Submissions from the first are re-assigned to the second. The users’ profile
data is not merged.
--no-delete
¶By default src_username
will be deleted after the contributions have been
merged. You can prevent this by using the --no-delete
option.
(env) $ pootle merge_user src_username target_username
purge_user
¶New in version 2.7.1.
This command can be used if you wish to permanently remove a user and revert the edits, comments and reviews that the user has made. This is useful for removing a spam account or other malicious user.
This command requires a mandatory username
argument, which should be a valid
username for a user of your site.
Changed in version 2.7.3: purge_user
can accept multiple user
accounts to purge.
(env) $ pootle purge_user username [username ...]
update_user_email
¶New in version 2.7.1.
(env) $ pootle update_user_email username email
This command can be used if you wish to update a user’s email address. This might be useful if you have users with duplicate email addresses.
This command requires a mandatory username
, which should be a valid
username for a user of your site, and a mandatory valid email
address.
(env) $ pootle update_user_email username email
verify_user
¶New in version 2.7.1.
Verify a user without the user having to go through email verification process.
This is useful if you are migrating users that have already been verified, or if you want to create a superuser that can log in immediately.
This command requires either mandatory username
arguments, which should be
valid username(s) for user(s) on your site, or the --all
flag if you
wish to verify all users of your site.
Changed in version 2.7.3: verify_user
can accept multiple user
accounts to verify.
(env) $ pootle verify_user username [username ...]
Available options:
--all
¶Verify all users of the site
There are multiple ways to run Pootle, and some of them rely on running WSGI servers that can be reverse proxied to a proper HTTP web server such as nginx or lighttpd.
The following are commands that have been removed or deprecated:
refresh_stats
¶Removed in version 2.8.
With the new stats infrastructure this is not needed anymore.
clear_stats
¶Removed in version 2.8.
With the new stats infrastructure this is not needed anymore.
last_change_id
¶Deprecated since version 2.7.
With the change to revisions the command you will want to use is
revision
, though you are unlikely to know a specific revision
number as you needed to in older versions of update_stores
.
commit_to_vcs
¶Deprecated since version 2.7.
Version Control support has been removed from Pootle and will reappear in a later release.
update_from_vcs
¶Deprecated since version 2.7.
Version Control support has been removed from Pootle and will reappear in a later release.
run_cherrypy
¶If you want to schedule certain actions on your Pootle server, using management commands with cron might be a solution.
The management commands can perform certain batch commands which you might want to have executed periodically without user intervention.
For the full details on how to configure cron, read your platform documentation
(for example man crontab
). Here is an example that runs the
calculate_checks
command daily at 02:00 AM:
00 02 * * * www-data source /var/www/sites/pootle/env/bin/activate; pootle calculate_checks
Test your command with the parameters you want from the command line. Insert it in the cron table, and ensure that it is executed as the correct user (the same as your web server) like www-data, for example. The user executing the command is specified in the sixth column. Cron might report errors through local mail, but it might also be useful to look at the logs in /var/log/cron/, for example.
If you are running Pootle from a virtualenv, or if you set any custom
PYTHONPATH
or similar, you might need to run your management command
from a bash script that creates the correct environment for your command to run
from. Call this script then from cron. It shouldn’t be necessary to specify
the settings file for Pootle — it should automatically be detected.
Pootle makes use of RQ to manage background jobs.
Some tasks are performed using background jobs and we expect more components to use it in future.
The RQ queue is managed by Redis and it is setup in the RQ_QUEUES and CACHES
settings.
The queue is processed by Workers. Any number of workers may be started and
will process jobs in the default queue. The rqworker
command is
used to start a Worker.
At the simplest level the Admin dashboard will tell you if the queue is active and how many workers are available to service the queue. It also lists the number of pending jobs and the number of failed jobs. This gives you a quick way to see if anything is wrong.
If a job fails it needs to be investigated. In most cases a traceback will indicate why the job failed.
The simplest way to work with queues and jobs is to use rq-dashboard, though you likely don’t want to deploy that on a production server. With this you can see the jobs in the queue, you can check the tracebacks and you can retry failed jobs.
In the case of a production server you can make use of the following commands to manage jobs:
$ redis-cli -n 2 lrange rq:queue:default 0 -1
03135097-00f8-46eb-b084-6f34a16d9940
a07309b3-f056-47e7-856c-c608bda2f171
3df6a559-2e3c-4c0c-b09c-1948b4bacda2
This will display all pending job IDs in the default queue. We’re using
the Redis DB number 2
, the default RQ queue on a standard Pootle install.
$ redis-cli -n 2 lrange rq:queue:failed 0 -1
60ed13df-0ce5-4b98-96f0-f8e0294ba421
3240527f-58b9-40fe-b0c5-b8d3fcaa06b6
This will display the failed job IDs.
To investigate a failed job simply add rq:job:
prefix to a job ID and
use a command such as this:
$ redis-cli -n 2 hgetall rq:job:60ed13df-0ce5-4b98-96f0-f8e0294ba421
This will allow you to see any traceback and investigate and solve them.
To push failed jobs back into the queue we simply run the
retry_failed_jobs
management command.
Sometimes failed jobs no longer apply since they refer to removed items, so no
matter how many times you run them they will keep failing. Note that sometimes
those unrecoverable failed jobs are in company of other failed jobs that can be
re-run by using the retry_failed_jobs
management command:
(env) $ pootle retry_failed_jobs
In order to delete all the failed jobs you must first stop the workers.
Once the workers are stopped make sure that there are no failed jobs that you
don’t want to remove. In case there is any restart the workers to re-run them
with retry_failed_jobs
. Stop the workers again once those jobs are
completed. Check again that all the failed jobs are the ones you want to
remove.
In order to perform a bulk delete of all failed jobs run the following commands:
$ redis-cli -n 2 LRANGE "rq:queue:failed" 0 -1 | perl -nE 'chomp; `redis-cli DEL rq:job:$_`;'
Now remove the list of failed jobs:
$ redis-cli -n 2 DEL "rq:queue:failed"
Do not forget to restart the workers.
In particular you should backup:
POOTLE_TRANSLATION_DIRECTORY
). Use the sync_stores
command to synchronize all your translation files to disk before making any
backup.If you are a developer and are willing to hack on Pootle or contribute in some other way, make sure to read through this part.
There are several ways you can contribute to improve Pootle, even if you don’t know programming! Want to know how? Please keep reading.
Sometimes Pootle doesn’t quite meet your expectations or you have an idea for a great new feature.
It might help to understand how Pootle developers evaluate new features:
If you really do want your feature to succeed here are some options to help you when reporting or requesting the feature.
In order to best solve the problem we need good bug reports. Reports that do not give a full picture or which coders are unable to reproduce, end up wasting a lot of time. If you, the expert in your bug, spend a bit of time you can make sure your bug gets fixed.
First see if the bug is not already reported. Perhaps someone already reported it and you can provide some extra information in that bug report. You can also add yourself in the CC field so that you get notified of any changes to the bug report.
If you could not find the bug, you should report it. Look through each of the following sections and make sure you have given the information required.
Tell us exactly how came to see this bug. Don’t say:
Suggesting doesn't work
Rather say:
In a translation project with proper permissions when I try to suggest I
get a 404 error.
So we need to know:
Tell us exactly how to reproduce the error. Mention the steps if needed, or give an example. Without being able to reproduce the error, it will not easily get fixed.
If you are a server administrator you can get this information from the web server’s error log. In case you’re hacking on Pootle, the traceback will be displayed both in the console and the browser.
A traceback will give a much better clue as to what the error might be and send the coder on the right path. It may be a very simple fix, may relate to your setup or might indicate a much more complex problem. Tracebacks help coders get you information quicker.
If you can be on Pootle channel or the mailing list to answer questions and test possible fixes then this will help to get your problem fixed quickly.
Pootle’s User Interface translations are kept in the official Pootle server. If you have a user in that server, you can start translating right away. Otherwise, just create a new user and start translating.
If your language already has a translation and you want to further improve or complete it, you can contribute suggestions that will later be reviewed by the language administrators.
If you can’t find your language and want to have that added or have concerns of any other means, contact us on the Pootle channel.
Although desirable, it’s not mandatory to use the official Pootle server to translate Pootle itself. In case you feel more comfortable working with files and offline tools, just head to the code repository at GitHub, create your localization based on the latest template and submit it to us by opening a bug or by sending us a pull request.
There are some addtional localization requirements beyond translation, so please review those to ensure that your language is 100% translated.
You can help us documenting Pootle by just mentioning typos, providing reworded alternatives or by writing full sections.
Pootle’s documentation is written using reStructuredText and Sphinx.
If you intend to build the documentation yourself (it’s converted from reST to HTML using Sphinx), you may want to setup a development environment for that.
We track the development roadmap using Github Milestones. These match the version numbers of future releases.
Project specific milestones are tracked using mstone-$name
milestones.
Note
Development partners may use their own systems to track their roadmaps.
Want to fix a bug in Pootle? Want to change the behaviour of an existing feature or add new ones? This section is all about hacking on Pootle, so if you are interested on the topic, keep reading.
Before starting any actual work on the source code, make sure that:
Although Pootle should only be deployed to production on a Linux server, it is possible to get a viable development environment up and running on Windows with some slightly different steps.
Any time you want to fix a bug or work on a new feature, create a new local branch:
$ git checkout -b <my_new_branch>
Then safely work there, create the needed commits and once the work is ready for being incorporated upstream, either:
HEAD
of the master
branch using
git diff or git format-patch and attach it to the
affected issue.When creating commits take into account the following:
As far as possible, try to commit individual changes in individual commits. Where different changes depend on each other, but are related to different parts of a problem / solution, try to commit them in quick succession.
If a change in the code requires some change in the documentation then all those changes must be in the same commit.
If code and documentation changes are unrelated then it is recommended to put them in separate commits, despite that sometimes it is acceptable to mix those changes in the same commit, for example cleanups changes both in code and documentation.
Begin the commit message with a single short (less than 50 character) line summarizing the change, followed by a blank line and then a more thorough (and sometimes optional) description.
Cleanups
Another example:
Factor out common behavior for whatever
These reduces lines of code to maintain, and eases a lot the maintenance
work.
Also was partially reworked to ease extending it in the future.
If your change fixes a bug in the tracker, mention the bug number. This way the bug is automatically closed after merging the commit.
Docs: Update code for this thing
Now the docs are exact and represent the actual behavior introduced in
commits ef4517ab and abc361fd.
Fixes #2399
If you are reverting a previous commit, mention the sha1 revision that is being reverted.
Revert "Fabric: Cleanup to use the new setup command"
This reverts commit 5c54bd4.
The minimum software packages you need for setting up a development environment include git and a Python interpreter along with the pip installer. Consult the specifics for your operating system in order to get each package installed successfully.
Once you have the basic requirements in place, you will need to install Pootle’s dependencies, which come in shape of Python packages. Instead of installing them system-wide, we recommend using virtualenv (and virtualenvwrapper for easing the management of multiple virtualenvs). This way you can install all the dependencies at specific versions without interfering with system-wide packages. You can test on different Python/Django versions in parallel as well.
For installing the dependencies in an isolated environment, we will use virtualenv – more specifically virtualenvwrapper, which eases the process of managing and switching between multiple virtual environments. Install virtualenwrapper as follows for bash (examine platform specific installation instructions for other environments):
$ sudo pip install virtualenvwrapper
virtualenvwrapper will need to be configured in order to specify where to store the created environments:
$ export WORKON_HOME=~/envs
$ mkdir -p $WORKON_HOME
$ source /usr/local/bin/virtualenvwrapper.sh # Or /usr/bin/virtualenvwrapper.sh
Note
You may want to add the above-mentioned commands and environment
variables to your .bashrc
file (or whatever file your shell uses for
initializing user customizations).
Now that the commands provided by virtualenv and virtualenvwrapper are available, we can create our virtual environment.
$ mkvirtualenv pootle_env
Replace pootle_env
with a meaningful name that describes the environment
you are creating. mkvirtualenv
accepts any options that virtualenv accepts. We could for example
specify to use the Python 3.3 interpreter by passing the -p python3.3 option.
Note
After running mkvirtualenv, the newly created environment is activated. To deactivate it just run:
(pootle_env) $ deactivate
To activate a virtual environment again use workon as follows:
$ workon pootle_env
First, upgrade the version of pip and setuptools as follows:
(pootle_env) $ pip install --upgrade pip setuptools
Time to clone Pootle’s source code repository. The main repository lives under translate/pootle in GitHub.
Note
If you have a GitHub account, fork the main translate/pootle
repository and replace the repository URL with your own fork.
(pootle_env) $ git clone https://github.com/translate/pootle.git
Install Pootle and its development dependencies into your virtualenv. This
makes it easy to run Pootle locally and is needed for various development
actitivies. The [dev]
target will install some extra packages to aid
development (you can examine these in requirements/dev.txt
).
(pootle_env) $ pip install -e .[dev]
Note
Some requirements may depend on external packages. For these you may need to install extra packages on your system in order to complete their installation.
With all the dependencies installed within the virtual environment, Pootle is almost ready to run. In development environments you will want to use settings that differ from those used in production environments.
(pootle_env) $ pootle init --dev
Note
To learn more about how settings work in Pootle read the settings documentation.
Once the configuration is in place, you’ll need to setup the database schema and add initial data.
(pootle_env) $ pootle migrate
(pootle_env) $ pootle initdb
Now ensure that you have built the assets by following the instructions for frontend development.
Finally, run the development server.
(pootle_env) $ pootle runserver
Once all is done, you can start the development server anytime by enabling the
virtual environment (using the workon
command) and running the pootle runserver
command.
Happy hacking!!
Note
Ensure that you are executing all of the following steps with Administrator privileges!
During the installation you will be asked to set what port Redis should listen on; leave it at the default (6379).
Note
For convenience these instructions consistently specify paths
C:\pootle_venv
, C:\git\pootle
and C:\temp
, but you can change
these to suit your environment and needs.
Note
Depending on how correctly your environment is set up (depending on
factors beyond your control such as virus scanners, Windows system health,
and so on), you may need to use the command python -m pip
for the
following steps if the basic pip
commands fail. Similarly, any other
Python command that should ‘just work’ might need to be invoked with
python -m
to avoid issues.
For installing the dependencies in an isolated environment, set up a fresh virtualenv.
> pip install virtualenv
> virtualenv C:\pootle_venv
Activate the new virtualenv and upgrade pip:
> C:\pootle_venv\Scripts\activate
(pootle_venv)> pip install --upgrade pip setuptools
Clone your fork of the Pootle master using your favourite Windows
implementation of Git so that you have a working copy somewhere accessible on
your computer (e.g. C:\git\pootle
).
Then go to the Pootle requirements\base.txt
and comment out the following
packages:
# lxml
# python-levenshtein
# scandir
These three packages are difficult to build on Windows, so we will download pre-built versions to install manually, saving them into your temporary folder:
Now install them explicitly:
(pootle_venv)> pip install C:\temp\lxml-3.6.4-cp27-cp27m-win32.whl
(pootle_venv)> pip install C:\temp\python_Levenshtein-0.12.0-cp27-none-win32.whl
(pootle_venv)> pip install C:\temp\scandir-1.2-cp27-none-win32.whl
At this point, you may be able to install Pootle and its requirements using the following command. However, pip installation of requirements may fail with “directory was not empty” or “file not found” issues, in which case you need to use the commands in the next note block.
(pootle_venv)> cd C:\git\pootle
(pootle_venv)> pip install -e .[dev]
Note
“Directory was not empty” and “file not found” issues come from
modern versions of Windows’ tighter control over permissions for special
folders. By default, pip stores temporary files in your user\AppData
folder which may not allow access in all circumstances. By downloading the
packages to a folder with no special permissions and building and installing
them from there we can circumvent these problems:
(pootle_venv)> pip download -d C:\temp -r requirements\dev.txt -b C:\temp
(pootle_venv)> pip install -r requirements\dev.txt -b C:\temp -t C:\pootle_venv\Lib\site-packages\ --no-index --find-links="C:\temp"
(pootle_venv)> cd C:\git\pootle
(pootle_venv)> pip install -e .
Now that all the requirements are lined up, we are ready to initialise Pootle. You should be able to initialise the Pootle demo database the same way as on a Linux system.
Note
Depending on how successfully your system has engaged the virtual
environment, you may have to execute pootle
commands with python
pootle/runner.py
from the pootle root folder instead (e.g. python
pootle/runner.py migrate
instead of pootle migrate
).
(pootle_venv)> pootle init --dev
(pootle_venv)> pootle migrate
(pootle_venv)> pootle initdb
Next, you will need to set up the client-side bundles with NPM. It might be necessary to deactivate the virtual environment or use a separate command window to perform this step, but it might also ‘just work’ from within the venv.
C:\git\pootle> cd pootle\static\js
C:\git\pootle\pootle\static\js> npm install
Once NPM install has completed, the actual javascript bundles can be compiled:
(pootle_venv)> cd C:\git\pootle
(pootle_venv)> pootle webpack --dev
The webpack
command will keep running after it’s completed, to
monitor your javascript files for changes so that it can auto-recompile as you
work. You’ll need to either exit it with Ctrl+C
once it has settled down,
or else open up a new command prompt and activate your virtual environment
there too.
One last javascript pack needs to be compiled to complete the client-side preparations:
(pootle_venv)> pootle compilejsi18n
Now create and verify a super-user as normal:
(pootle_venv)> pootle createsuperuser
[Follow on-screen prompts.]
(pootle_venv)> pootle verify_user [username]
Pootle is now ready to be fired up!
You will need to run one RQWorker and one Pootle server, so you’ll need two command prompt windows (as both will remain active until you disable the server):
(pootle_venv)> pootle rqworker
(pootle_venv)> pootle runserver
Congratulations, Pootle should now be running comfortably! Happy hacking on Windows!!
Parts of Pootle front-end development require a Node.js run-time and packages installed via npm. This is only the case for developing or building Pootle.
In order to setup the front-end development enviroment, it’s necessary to have Node.js installed. Please check the installation instructions for your OS.
Warning
If you are using versions provided by you system then you need at least npm
>= v1.4.3 for installation to work correctly. To upgrade, use [sudo]
npm install npm@latest -g
.
Once Node.js is available, Pootle dependencies need to be installed.
$ cd pootle/static/js
$ npm install
This will read the package.json
file and install the development
dependencies.
Simply run:
(env) $ pootle compilejsi18n
(env) $ pootle webpack --dev
This will make sure to build all the necessary scripts and create the relevant bundles with source maps support. It will also watch for changes in scripts so you don’t need to constantly be running this.
For creating a production-ready build, use:
(env) $ pootle webpack
This will also run the output through UglifyJS, making the output build considerably lighter in size.
Note that this step is also done as part of the make assets command, so you may only want to run the latter.
In some cases it might be desirable to customize the styling of Pootle to fit in with your other websites or other aspects of your identity. It might also be required to add a common header or footer for proper visual integration and even adjust and enhance existing functionality.
It’s highly recommended to put any custom changes separate from the distributed files, so that upgrades are unlikely to affect your customizations.
In case you need to change a template, copy it into your custom
TEMPLATE:DIRS
directory with the same path name as
it had before.
Warning
If you edit any templates, keep in mind that changes to the text could result in untranslated text for users of the non-English user interface.
You can customize specific blocks of templates by indicating which template the
current template is customizing. Use the {% overextends %}
template tag for
that (requires to install the django-overextends package). This must be the first
tag in the template.
{% overextends 'browser/overview.html' %}
{% block pre_content %}
{{ block.super }}
<h1>My custom content</h1>
{% endblock %}
Check the original templates in order to know which blocks can be customized.
On upgrades, you will want to check if the templates and the contained blocks differ.
You can place any custom scripts in your custom STATICFILES_DIRS
directory and make them part of the default Pootle bundles by adding a very
simple manifest.json file under the js/ directory of your custom
STATICFILES_DIRS
.
This file must contain an object of key-values where the keys correspond to the entry points defined by Pootle and the values are arrays of module names to include in the output bundle. Check out the pootle/static/js/webpack.config.js file to see the existing entry points.
Example:
{
"common": ["login.js", "extra_module.js"]
}
In the example above, the login.js and the extra_module.js JavaScript modules will be added as part of the common bundle. If common didn’t exist as an entry point before, a new bundle will be output.
Note that the manifest.json file has to be valid JSON, otherwise it will be omitted.
Custom scripts can require()
Pootle modules that are part of the core
bundles by prefixing paths with pootle/
. For instance the
require('pootle/models')
call will make Pootle’s own models
module
available in the scope of a 3rd party script.
Needless to say, you can refer to your custom scripts the same way as you
would refer to any other static asset, i.e. by using the {% static %}
template tag.
Create any needed files under your custom STATICFILES_DIRS
and
reference them from your custom templates using the {% static %}
template
tag. You can also inline styles in your templates as usual.
You should put your custom images in your custom STATICFILES_DIRS
.
From CSS you would just reference them using a relative path.
On the contrary, if you want to reference images from HTML code or inline CSS,
you should use the {% static %}
template tag.
The site robots.txt file uses a static template. There is no dynamic content. Place your custom robots.txt file in your custom templates folder and this will be served instead of Pootle’s default robots.txt file.
Larger installs will want to serve this file statically, bypassing Django completely. You can safely serve the robots.txt template file as is, statically, via your configured webserver.
Before you can rebuild your static assets with any CSS or JavaScript customisations, you will need to install some Node.js libraries.
Before proceeding please make sure you have Node.js and npm installed in your system.
(env) $ cd $pootle_dir
(env) $ cd pootle/static/js/
(env) $ npm install
$pootle_dir
is the directory where Pootle is installed.
Before rebuilding your assets for the first time you must install the JavaScript build libraries.
After doing any customizations, you will need to regenerate any modified bundles and gather all the static assets in a single place for public consumption.
You will need to activate your virtual environment before running these commands.
(env) $ pootle webpack
(env) $ pootle collectstatic --noinput --clear -i node_modules
(env) $ pootle assets build
Pootle targets the latest stable versions of major modern web browsers.
Pootle should not only work correctly, but it should also look great in Firefox, Chrome and Safari.
Internet Explorer is an exception, where we support the latest two stable versions (as of today, IE11+). Here Pootle should work well, but might look imperfect.
Older browser versions might work properly too, but we are not committed to ensure such support.
A nice to have goal is making Pootle usable in smaller screens such as iPads. But this is not a hard requirement.
If you are about to use a feature which might not be available in the set of supported browsers, check the Can I Use… website first.
Warning
Work in progress. For now only Python testing is being added. Future coverage will include JavaScript code too.
Pootle tests use the full-featured pytest testing tool and its integration with Django via pytest-django.
To test simply run:
.. code-block:: console
(env) $ py.test
from the root of the repository. Note that you need to install the testing requirements into your virtualenv first (requirements/tests.txt).
Note
Since the test runner automatically sets the DEBUG
setting to
False
, the static assets need to be collected before running the view
tests. You can run make assets
for building them.
The py.test
runner command offers several options which are extended by
plugins as well. Check its documentation
for further details.
Some testing-specific settings are loaded from the tests/settings.py file and override any previous setting you might have set in the settings/*.conf files.
Writing new tests is easy. Just write a function whose name starts with
test_
and place it in an appropriate module under the tests/
subdirectory.
You’ll need to use plain Python assertions in test functions. Check pytest’s documentation for more information on assertions.
In order to use a fixture, you simply need to reference its name as a function argument. Pytest does the rest of the magic. There are other ways to reference and use fixtures as well, but most of the time you’ll find yourself passing them as function arguments.
You’ll usually want to test model behavior. These tests should test one function or method in isolation. If you end up needing to test for multiple things, then you might need to split the function/method into more specific units. This allows to structure the code better.
When testing models, it’s a suggested practice to avoid DB access because it makes the tests run slower, so think twice if your test actually needs DB access. At the same time, pytest-django encourages you to follow these best practices and disables DB access by default. If your test needs DB access, you need to explicitly request it by using the @`pytest.mark.django_db marker.
While testing views/integration tests can also help catch regressions, they’re slower to run and end up in less useful failures, so better to write fewer of these.
Pootle tests include some pytest fixtures you can reuse. They’re located in tests/fixtures/ and are loaded when the test runner is being set up.
If you have a fixture which is very specific you can place it in a usual
conftest.py
file in its proper context, whereas the aforementioned
directory is meant to be for storing shared or general-purpose fixtures.
Model fixtures are stored under tests/fixtures/models/, and they are basically factory functions returning an instance of the desired model. Note that these might depend on other fixtures too.
For now these model fixtures require DB access, but since that’s not what every single test might need, we might want to combine this with other more complete solutions like factory_boy in the future.
This document describes the release process Pootle follows starting from version 2.5.
The principles above extended into these rules.
A Pootle version number consists of Major-Minor-Point-Bugfix
as in
2.5.0
or 2.6.1.2
Pootle’s minor number is changed to indicate the latest version of Django that is supported. Thus when the latest version of Django is released, and Pootle gains support for this version, then the Pootle minor number will change.
Note
Pootle 2.5.0 is an exception to this rule. It did not support Django 1.5 at the time of release.
Every six months, when a new release train is ready to be shipped, Pootle’s point version will be incremented.
Any critical security fixes will automatically result in a new bugfix release.
Understanding the number and release train with some examples:
Django 1.5 is the latest version of Django:
2.5
and should support Django 1.5.2.5.0
is released as the first time-based release.2.5.1
.A security issue is detected in Pootle 2.5.0
2.5.0.1
is made2.5.1
Django 1.6 is released:
2.5.4
, next Pootle release will be 2.6.0
2.6.0
is out we will support Pootle 2.6.0
and 2.5.4
, all
previous versions will be unsupported.A security issue is discovered which impacts all our supported time-based releases:
2.6.0.1
and 2.5.4.1
Time-based release 2.6.1
is released six months after 2.6.0
2.6.1
and 2.6.0
2.5.4
which is now a year old.Within the priciple that master is always deployable we aim to ensure a period of stability to allow easier release in the month prior to a time-based release.
If for some reason there’s feature work that changes the schema during month six of the release train, the feature will go in its own branch and won’t be merged until the next release train starts.
Security fixes are applied anytime in the release train.
The next Pootle version is always baked in the master branch. Exceptions are security fixes which are committed in master and cherry-picked to the currently supported time-based release branches.
A new time-based release is made off of master, incrementing the point
version. Every time a new release happens, a new branch is created. These
branches are named after their version numbers: if master is to become
version 2.6.2
, then the new branch will be named stable/2.6.2. The actual
release is also tagged, in this case as 2.6.2.
Security fixes are made on the relevant release branches. So the first security release on stable/2.6.2 would be tagged as 2.6.2.1.
Features that produce schema changes or are quite invasive go into feature branches named feature/<feature-name>. Once the feature is ready to be integrated within the first phase of the release train, they’re merged into master.
A file that stores translations (e.g. a PO file) — although it could also be used to refer to other ways of storing translations.
Contains a number of Translation Units, which contain messages.
At the simplest level contains a single source string (the original message) and a single target string (the translated message).
XLIFF refers to this as a unit, Gettext calls it a message or string. Some industry tools talk of segments. To maintain consistency we refer to string in the GUI and unit in the code.
Monolingual formats (like .properties, OpenOffice SDF, DTD, HTML, etc.) only contain a source strings.
However when handling plurals the source may actually contain different variants of a message for different plural forms (e.g. in English, the singular and plural), and the target as well (the number of variants in source and target strings are often different because different languages handle plurals differently).
ctx_obj
)An object representing the context that encloses the current view.
If we are navigating through the files for an existing translation project, the context object will refer to the current translation project.
Similarly, if we are in the overview page for a language, the context will point to the current language object. In the overview page for a project, the context object points to the current project.
At a higher level, the root directory is considered the context object.
resource_obj
)An object representing the resource that the current view is referring to.
For example, if we are navigating through the files and directories for an existing translation project, the resource object will refer to the current file or directory object.
If the current view refers to multiple resources, the resource object is the same as the context object.
Pootle developers try to stick to some development standards that are gathered in this document.
For Python code and documentation Pootle follows the Translate Styleguide adding extra clarifications listed below.
Pootle has specific conventions for Python coding style.
Like in Python import conventions in Translate styleguide, but imports should be grouped in the following order:
Check Python import conventions in Translate styleguide for other conventions that the imports must follow.
import re
import sys.path as sys_path
import time
from datetime import timedelta
from os import path
from lxml.html import fromstring
from translate.storage import versioncontrol
from django.contrib.sites.models import Site
from django.db import models
from django.db.models import Q
from django.db.models.signals import post_save
from tastypie.models import ApiKey
from pootle_language.models import Language
from pootle_translationproject.models import TranslationProject
from .forms import GoalForm
from .models import Tag
Model’s inner classes and methods should keep the following order:
objects
managerclass Meta
def natural_key()
(Because it is tightly related to model fields)@classmethod
def __unicode__()
def __str__()
__
(for example __init__()
)def save()
def delete()
def get_absolute_url()
def get_translate_url()
class SampleForm(forms.Form):
# Field declaration that spans to several lines.
language = forms.ChoiceField(
label=_('Interface Language'),
initial="",
required=False,
widget=forms.Select(attrs={
'class': 'js-select2 select2-language',
}),
help_text=_('Default language for using on the user interface.'),
)
# One line field declaration.
project = forms.ModelChoiceField(Project, required=True)
When writing the URL patterns:
url()
function, not a tuple.pootle-{app}-{view}
(except in some
specific cases):{app}
is the app name, which sometimes can be shortened, e.g. using
tp to avoid the longish translationproject. The chosen app name
must be used consistently across all the URL patterns for the app.{view}
is a unique string which might consist on several words,
separated with hyphens, that might not match the name of the view that is
handled by the URL pattern.pootle-xhr-{view}
.pootle-admin-{view}
pootle-{view}
.urlpatterns = patterns('pootle_project.views',
# Listing of all projects.
url(r'^$',
'projects_index'),
# Whatever URLs.
url(r'^incredibly-stupid/randomly-long-url-with-hyphens-that-is-split-'
r'and-continued-on-next-line.html$',
'whatever',
name='pootle-project-whatever'),
# Admin URLs.
url(r'^(?P<project_code>[^/]*)/admin.html$',
'project_admin'),
url(r'^(?P<project_code>[^/]*)/permissions.html$',
'project_admin_permissions',
name='pootle-project-admin-permissions'),
)
In order to have a more consistent code the use of specific names for some heavily used variables is encouraged:
ctx
: Name for the dictionary with the context passed to a template for
rendering. Also known as context, template variables or template vars.
# Good.
ctx = {
'language': language,
}
# Bad.
context = {
...
templatevars = {
...
template_vars = {
...
Pootle specific settings must be named like POOTLE_*
, for example:
POOTLE_ENABLE_API
, POOTLE_VCS_DIRECTORY
or POOTLE_MARKUP_FILTER
For documenting several things, Pootle defines custom Sphinx roles.
Settings:
.. setting:: POOTLE_TITLE
To link to a setting, use :setting:`POOTLE_TITLE`
.
Icons:
Some reference to |icon:some-icon| in the text.
This allows you to easily add inline images of icons used in Pootle.
The icons are all files from pootle/static/images/sprite
. If you
were referring to an icon icon-edit.png
then you would use the syntax
|icon:icon-edit|
. The icon reference is always prefixed by icon:
and the name of the icon is used without the extension.
E.g. |icon:icon-google-translate|
will insert this
icon.
Pootle and Django commands:
.. django-admin:: sync_stores
To link to a command, use :djadmin:`sync_stores
Follow the great Airbnb JavaScript Style Guide. Go check it out for all the details.
As a summary, that includes:
pascalCase
variable naming.In addition to that:
When dealing with existing or legacy code, also keep in mind to:
$
Variables holding jQuery objects.js-
to prefix selectors for elements queried via JavaScript.For React + JSX code also follow the Airbnb React/JSX Style Guide, with the following exceptions:
.js
extension for React components (not .jsx
).React.createClass({})
over extending React.Component
.Also bear in mind the following:
handle*()
for methods, on*()
for props.propTypes
: sort them alphabetically, but also group them to place
isRequired
types first.Always use double quotes for HTML attribute values.
Always use single quotes for Django template tags and template filters located inside HTML attribute values.
<!-- Good -->
<a href="{% url 'whatever' %}" class="highlight">
Good:
.foo-bar,
.foo-bar:hover
{
background-color: #eee;
}
Bad:
.foo-bar, .foo-bar:hover {
background-color: #eee;
}
.tm-results
and not
.TM_results
.From time to time features, commands, configurations will be deprecated. We deprecate and manage backward compatibility within the following guidelines:
So some rough “rules”. These apply to features, management commands and settings.
This page is divided in four sections. The first one lists the tasks that must be performed before creating a package. The second section includes a list of tasks to get a valid package. The third one to get the package published and the release announced. The last one lists and suggests some possible cleanup tasks to be done after releasing.
Note
Please note that this is not a complete list of tasks. Please feel free to improve it.
Before starting the release process it is necessary to perform some previous tasks.
We need to give localizers enough time to localize Pootle. They need time to do the actual translation and to feedback on any errors that they might encounter.
First upload the new translations:
Create the new templates:
$ git clone git@github.com:translate/pootle.git pootle-translations
$ cd pootle-translations
$ make pot
Upload the templates to Pootle for translation.
Update current translations against templates either on Pootle or in code and commits these updated files to Git.
Announce the new translations on the following channels:
A string freeze would normally run between an RC and a final version. We want to give a string freeze at least 2-4 weeks before a release. They must be announced, explicitly stating the duration, on the translate-announce@lists.sourceforge.net and the translate-pootle@lists.sourceforge.net mailing lists.
Note
If we do have a string freeze break then announce it to people. The string freeze breaks usually are only allowed to fix mistakes on the translatable strings.
The first steps are to create and validate a package for the next release.
We work from a clean checkout to ensure that everything you are adding to the build is what is in the repository and doesn’t contain any of your uncommitted changes. It also ensures that someone else could replicate your process.
$ git clone git@github.com:translate/pootle.git pootle-release
$ cd pootle-release
$ git submodule update --init
It is extremely advisable to create a Pull Request with the changes necessary for the release. This means that it is best to create a working branch to clearly separate those changes from master branch, which can be altered while doing the release, and ease rebasing if needed.
$ git remote add my-fork git@github.com:my-fork/pootle.git
$ git fetch my-fork
$ git checkout -b releasing
Update the minimum version number for the requirements in:
requirements/
pootle/checks.py
docs/server/requirements.rst
Make sure version numbers displayed on documentation examples match the latest requirements on the above files.
Update any copyright dates in docs/conf.py:copyright
and anywhere else
that needs fixing.
$ git grep 2013 # Should pick up anything that should be examined
Create ~/.pootle/pootle_build.conf
with the following content:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Configuration file to build Pootle.
Must be placed in ~/.pootle/pootle_build.conf
"""
# Django now requires to set some secret key to be set.
SECRET_KEY = '__BuildingPootle_1234567890__'
# Silence some checks so the build output is cleaner.
SILENCED_SYSTEM_CHECKS = [
'pootle.W004', # Pootle requires a working mail server
'pootle.W006', # sqlite database backend is unsupported
'pootle.W010', # DEFAULT_FROM_EMAIL has default setting
'pootle.W011', # POOTLE_CONTACT_EMAIL has default setting
]
The quality checks descriptions are kept as a static HTML page that has to be regenerated in order to ensure the descriptions match the currently available quality checks.
$ mkvirtualenv build-checks-templates
(build-checks-templates)$ pip install --upgrade setuptools pip
(build-checks-templates)$ pip install -r requirements/build.txt
(build-checks-templates)$ export POOTLE_SETTINGS=~/.pootle/pootle_build.conf
(build-checks-templates)$ DJANGO_SETTINGS_MODULE=pootle.settings ./setup.py build_checks_templates
(build-checks-templates)$ deactivate
$ unset POOTLE_SETTINGS
$ rmvirtualenv build-checks-templates
Update the translations from the Pootle server
Download all translations
$ make get-translations
Update pootle/locale/LINGUAS
to list the languages we would like to
ship. While we package all PO files, this is an indication of which ones we
want packagers to use. The requirement is roughly 80% translated with no
obvious variable errors. Languages with a small userbase can be included.
$ make linguas
Check the output and make any adjustments such as adding back languages that don’t quite make the target but you wish to ship.
Build translations to check for errors:
$ ./setup.py build_mo # Build all LINGUAS enabled languages
$ ./setup.py build_mo --check # Not all of these are errors
We create our release notes in reStructured Text, since we use that elsewhere and since it can be rendered well in some of our key sites.
First we need to create a log of changes in Pootle, which is done generically like this:
$ git log $previous_version..HEAD > docs/releases/$version.rst
Or a more specific example:
$ git log 2.5.0..HEAD > docs/releases/2.5.1.rst
Edit this file. You can use the commits as a guide to build up the release notes. You should remove all log messages before the release.
Note
Since the release notes will be used in places that allow linking we use links within the notes. These should link back to products websites (Virtaal, Pootle, etc), references to Translate and possibly bug numbers, etc.
Read for grammar and spelling errors.
Note
When writing the notes please remember:
We create a list of contributors using this command:
$ git log 2.5.0..HEAD --format='%aN, ' | awk '{arr[$0]++} END{for (i in arr){print arr[i], i;}}' | sort -rn | cut -d\ -f2-
Bump the version for each of the apps so the caches are caned after upgrade:
apps.py
for each of the apps that have a versionUpdate the version number in:
pootle/__init__.py:VERSION
docs/server/installation.rst
and
docs/server/upgrading.rst
The version tuple should follow the pattern:
(major, minor, micro, candidate, extra)
E.g.
(1, 10, 0, 'final', 0)
(2, 7, 0 'alpha', 1)
When in development we use ‘alpha’ with extra
of 0. The first release of a
minor
version will always have a micro
of .0
. So 2.6.0
and
never just 2.6
.
Most likely your system will provide a nodejs version older than the one that is required. nvm is a tool that allows to quickly install and switch nodejs versions.
Follow the nvm installation instructions.
Building is the first step to testing that things work. From your clean checkout run:
$ mkvirtualenv build-pootle-release
(build-pootle-release)$ nvm install stable
(build-pootle-release)$ pip install --upgrade setuptools pip
(build-pootle-release)$ pip install -r requirements/build.txt
(build-pootle-release)$ pip install -e .[dev]
(build-pootle-release)$ export PYTHONPATH="${PYTHONPATH}:`pwd`"
(build-pootle-release)$ export POOTLE_SETTINGS=~/.pootle/pootle_build.conf
(build-pootle-release)$ ./setup.py build_mo # Build all LINGUAS enabled languages
(build-pootle-release)$ ./setup.py build_mo --all # If we are shipping an RC
(build-pootle-release)$ make build
(build-pootle-release)$ deactivate
$ unset POOTLE_SETTINGS
This will create a tarball in dist/
which you can use for further
testing.
Note
We use a clean checkout just to make sure that no inadvertant changes make it into the release.
The easiest way to test is in a virtualenv. You can test the installation of the new release using:
$ mkvirtualenv test-pootle-release
(test-pootle-release)$ pip install --upgrade setuptools pip
(test-pootle-release)$ pip install dist/Pootle-$version.tar.bz2
(test-pootle-release)$ pip install mysqlclient
(test-pootle-release)$ pootle init
You can then proceed with other tests such as checking:
Documentation is available in the package
Assets are available in the package
Quick SQLite installation check:
(test-pootle-release)$ pootle migrate
(test-pootle-release)$ pootle initdb
(test-pootle-release)$ pootle runserver --insecure
(test-pootle-release)$ # Browse to localhost:8000
MySQL installation check:
Create a blank database on MySQL:
mysql -u $db_user -p$db_password -e 'CREATE DATABASE `test-mysql-pootle` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;'
Change the database settings in the settings file created by
pootle init
(by default ~/.pootle/pootle.conf
)
to use this new MySQL database
Run the following:
(test-pootle-release)$ pootle migrate
(test-pootle-release)$ pootle initdb
(test-pootle-release)$ pootle runserver --insecure
(test-pootle-release)$ # Browse to localhost:8000
Drop the MySQL database you have created:
mysql -u $db_user -p$db_password -e 'DROP DATABASE `test-mysql-pootle`;'
MySQL upgrade check:
Download a database dump from Pootle Test Data repository
Create a blank database on MySQL:
mysql -u $db_user -p$db_password -e 'CREATE DATABASE `test-mysql-pootle` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;'
Import the database dump into the MySQL database:
mysql -u $db_user -p$db_password test-mysql-pootle < $db_dump_file
Run the following:
(test-pootle-release)$ pootle migrate
(test-pootle-release)$ pootle runserver --insecure
(test-pootle-release)$ # Browse to localhost:8000
Drop the MySQL database you have created:
mysql -u $db_user -p$db_password -e 'DROP DATABASE `test-mysql-pootle`;'
Check that the instructions in the Installation guide are correct
Check that the instructions in the Upgrade guide are correct
Check that the instructions in the Hacking guide are correct
Meta information about the package is correct. This is stored in
setup.py
, to see some options to display meta-data use:
$ ./setup.py --help
Now you can try some options like:
$ ./setup.py --name
$ ./setup.py --version
$ ./setup.py --author
$ ./setup.py --author-email
$ ./setup.py --url
$ ./setup.py --license
$ ./setup.py --description
$ ./setup.py --long-description
$ ./setup.py --classifiers
The actual long description is taken from /README.rst
with some
tweaking for releasing.
Finally clean your test environment:
(test-pootle-release)$ deactivate
$ rmvirtualenv test-pootle-release
Once we have a valid package it is necessary to publish it and announce the release.
Before merging the changes you must create a Pull Request to ensure that all tests and checks pass:
$ git push my-fork releasing
Note
Of course you must wait until all automatic checks pass.
You should branch only when you are releasing the first stable version of a new version series, since betas and release candidates can be developed in the master branch. To branch do:
$ git checkout -b stable/2.8.x
$ git push origin stable/2.8.x
If you branch you will want to update the README.rst
file so that it
points correctly to branched versions of badges and documentation. Review and
test the actual links created, you don’t need to commit everything.
$ workon build-pootle-release
(build-pootle-release)$ ./setup.py update_readme -w
(build-pootle-release)$ git diff README.rst
(build-pootle-release)$ git commit README.rst -m "Adjust README to branch"
(build-pootle-release)$ deactivate
Also if you branch you will want to limit requires.io to requirements in the branch. To do so check how it was done in this commit:
$ nano .requires.yml
$ git add .requires.yml
$ git commit -m "Requirements: Limit requires.io to branch requirements"
You should only tag once you are happy with your release as there are some things that are difficult to undo:
$ git tag -a 2.8.0 -m "Tag version 2.8.0"
$ git push --tags
We need a tagged release or branch before we can do this. The docs are published on Read The Docs.
Use the admin pages to flag a version that should be published. When we have
branched the stable release we use the branch rather then the tag i.e.
stable/2.5.x
rather than 2.5.0
as that allows any fixes of
documentation for the 2.5
releases to be immediately available.
Change all references to docs in the Pootle code to point to the branched version as apposed to the latest version.
Deactivate documentation that is no longer applicable.
Note
You need a username and password on Python Package Index (PyPI) and have rights to the project before you can proceed with this step.
These can be stored in $HOME/.pypirc
and will contain your username
and password. Check Create a PyPI account
for more details.
Run the following to publish the package on PyPI:
$ workon build-pootle-release
(build-pootle-release)$ pip install --upgrade pyopenssl ndg-httpsclient pyasn1 twine
(build-pootle-release)$ twine upload dist/Pootle-*
(build-pootle-release)$ deactivate
$ rmvirtualenv build-pootle-release
Do the following to create the release:
We use github pages for the website. First we need to checkout the pages:
$ git checkout gh-pages
_posts/
add a new release posting. Use the same text used for the
Github release description,
including the link to the full release notes.$version
as needed. See _config.yml
and
git grep $old_releaseLet people know that there is a new version:
Announce on mailing lists using plain text emails using the same text (adjusting what needs to be adjusted) used for the Github release description:
Warning
This has to be explicitly reviewed and approved by Dwayne so we don’t repeat the same email over and over.
Adjust the Pootle channel notice.
Use /topic [new topic]
to change the topic. It is easier if you copy the
previous topic and adjust it.
Email important users
Tweet about it
Update Pootle’s Wikipedia page
These are tasks not directly related to the releasing, but that are nevertheless completely necessary.
If this new release is a stable one, bump the version in master
to
{N+1}-alpha1
. The places to be changed are the same ones listed in
Up version numbers. This prevents anyone
using master
being confused with a stable release and we can easily check
if they are using master
or stable
.
After updating the release notes for the about to be released version, it is
necessary to add new release notes for the next release, tagged as dev
.
Some possible cleanup tasks:
pootle-release
checkout.master
.$version
docs rather then latest
?Warning
Pootle’s plugin system is currently in an early stage of development, and may be subject to change in the future. If you have any questions or are intending to use it in your own applications you can chat with us on the Translate development channel.
You can customize or extend Pootle using plugins.
A Pootle plugin is a Django application that hooks into the core functionality in Pootle.
Pootle emits Signals
when key events happen. You can listen to these
signals using a receiver
to trigger custom behaviour. Pootle uses
Django’s Signals framework for handling these types of events.
Pootle allows plugins to override the default behaviour using a Getter
function, which are decorated with the pootle.core.plugin.getter
decorator.
Once Pootle has received a response from a plugin for a Getter
function
it stops processing any further configured functions.
Pootle allows developers to change or extend the data used by the system,
by adding Provider
functions, which are decorated with the
pootle.core.plugin.provider
decorator. With Provider
functions Pootle
will gather data from all plugins configured to provide for a given
Provider
function.
__init__.py
apps.py
- Django application configurationreceivers.py
- receivers for signalsgetters.py
- getter functionsproviders.py
- provider functionsYour application requires a Django application configuration
For an application named pootle_custom
you need to add lines similar to the
following in the __init__.py
:
default_app_config = 'pootle_custom.apps.PootleCustomConfig'
With the above configuration you should add an apps.py
.
At a minimum this should define the PootleCustomConfig
class with its
name
and verbose_name
.
It can also be used to activate receivers, providers and getters. The following application configuration activates all of them for the “custom” application.
import importlib
from django.apps import AppConfig
class PootleCustomConfig(AppConfig):
name = "pootle_custom"
verbose_name = "Pootle Custom"
def ready(self):
importlib.import_module("pootle_custom.receivers")
importlib.import_module("pootle_custom.providers")
importlib.import_module("pootle_custom.getters")
The following is an example of providing custom context_data
to the Pootle
LanguageView
.
Add a file called providers.py
with the following:
from pootle.core.delegate import context_data
from pootle.core.plugin import provider
from pootle_language.views import LanguageView
@provider(context_data, sender=LanguageView)
def provide_context_data(**kwargs):
return dict(
custom_var1="foo",
custom_var2="bar")
The following is an example of customizing the Unit
search_backend
for an
application.
Add a file called getters.py
with the following:
from pootle.core.delegate import search_backend
from pootle.core.plugin import getter
from pootle_store.models import Unit
from pootle_store.unit.search import DBSearchBackend
class CustomSearchBackend(DBSearchBackend):
pass
@getter(search_backend, sender=Unit)
def get_search_backend(**kwargs):
return CustomSearchBackend
Pootle uses the django.core.signals module to handle events.
The following is an example of a receiver
that emits a log warning whenever
a Store
cache is expired.
Add a file called receivers.py
with the following code:
import logging
from django.core.signals import receiver
from pootle.core.signals import cache_cleared
from pootle_store.models import Store
@receiver(cache_cleared, sender=Store)
def handle_cache_cleared(**kwargs):
logging.warning(
"Store cache cleared: %s"
% kwargs["instance"].pootle_path)
You can install a development and test environment using docker.
Installing with virtualenv keeps your host environment clean, and separates your development environment from the host.
For example:
$ mkvirtualenv pootle
$ cd ~/virtualenvs/pootle
You most likely want to clone your fork of the pootle repository, so you can easily create Pull Requests for your changes.
(pootle): git clone git@github.com:$USER/pootle
(pootle): cd pootle/
(pootle): pip install -r requirements/host.txt
The default installer will create a postgresql database with
(pootle): makey install-dev
This will take some time as it loads the default projects
Once Pootle is installed you can run the development server with
(pootle): makey runserver
Caught out by a problem installing or running Pootle? We hope you’ll find some answers here. Ideal candidates are specific installation issues that we can’t integrate into the main docs. Feel free to provide updates with your own findings.
Pootle does not, yet, support Python 3 but it definitely is a goal.
Our first priority has been cleaning up the code and getting onto the latest version of Django. We’ve achieved that with Pootle 2.8.0.
We also want to be Django warning free, we’ve also achieved that in Pootle 2.8.0.
All of these where needed to ease to migration to Python 3.
Currently, we’re trying to eliminate Python 2 specific changes and we’re coding pylint checks to prevent any regression.
If you want to help make this happen sooner, patches are welcome.
File "/home/pootle/env/lib/python3.6/site-packages/pootle/runner.py", line 19, in <module>
import syspath_override # noqa
ModuleNotFoundError: No module named 'syspath_override'
You are running Pootle using Python 3, change your virtual environment to Python 2 and try again.
Something like this will be needed to setup your virtual environment.
$ mkvirtualenv --python=/path/to/python2 pootle
Pootle assumes that you have the en_US.utf8
locale installed on your
server. If for some reason your server does not include this (you’re not
American or you are using a very minimal server) then you need to install that
locale.
On a Debian based server simply run:
$ sudo dpkg-reconfigure locales
Pootle may require you to install additional system dependencies. The majority of these relate to the installation of lxml, required by Pootle for XLIFF and other XML based support.
lxml requires compilation so we depend on build components as well as libraries
for libxml
, libxslt
and Python.
On Debian based system the following will install all additional system requirements:
$ sudo apt-get install build-essential libxml2-dev libxslt-dev python-dev python-pip zlib1g-dev
There are too many variables to give a definitive numbers.
In terms of a servers ability to handle large files, this will depend on the size or shape of the database, available system resources, the database configuration and the activity on the site.
What is helpful to be aware of is that Pootle does work on a file level. So really large translation files might become unwealdy to process, and queries to find untranslated units in the file may take longer then expected.
Our general advice is to keep related translations in the same file and this should work fine. If performance does appear to be a problem then break the large files into logical divisions to create smaller files.
The following are release notes used on PyPI and mailing lists for Pootle releases.
These are the changes that have happened in Pootle and may affect your server. Also be aware of the important changes in the Translate Toolkit as many of these also affect Pootle.
If you are upgrading Pootle, you might want to see some tips to ensure your upgrade goes smoothly.
Not yet released
Welcome to Pootle 2.9!
If you want to try it, check one of the following:
sync_stores
:--force
and
--overwrite
arguments no longer have
any effect in the command execution.update_stores
:--force
argument no longer has any
effect in the command execution.changed_languages
command. Use list_languages
instead.--yes
argument to init
command.This release was made possible by the following people:
Ryan Northey, Leandro Regueiro, Dwayne Bailey, Taras Semenenko, boite.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 15 September 2017
Welcome to Pootle 2.8.2!
If you want to try it, check one of the following:
This release was made possible by the following people:
Leandro Regueiro, Dwayne Bailey, Ryan Northey.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 31 August 2017
Welcome to Pootle 2.8.1!
If you want to try it, check one of the following:
VIRTUAL_ENV
environment
variable set.This release was made possible by the following people:
Leandro Regueiro, boite.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 16 August 2017
Welcome to Pootle 2.8!
If you want to try it, check one of the following:
Below we provide much more detail. These are by no means exhaustive, view the git log for complete information.
mysqlclient
database driver instead of MySQLdb
,
this is Django’s preferred database driver.robots.txt
. It is now a static file which you can
adjust for your site. The site is allow
by default.POOTLE_EMAIL_FEEDBACK_ENABLED
settingcan review
permissions if submitted
translation is 100% match.TypeError
error when filter in editor gets no unitsPOOTLE_CUSTOM_LOGO
Pootle FS enables synchronization of Pootle against a filesystem, or version control system, handling conflict resolution and other situations of two files being out of sync.
Pootle FS follows a git like command execution. We’ve designed it such that we expect there to be no data loss when conflicts are discovered. Any conflicts are turned into suggestions which can be resolved in Pootle.
Pootle FS is still in beta as we’d like to make sure that all the bugs are washed out before making it an official and default part of Pootle.
sync_stores and update_stores are still the default method of interacting with Pootle. We expect these to remain for some time, but expect the next version of Pootle to use to Pootle FS infrastructure to manage and handle these commands.
info
, fetch
, resolve
,
sync
, add
and rm
commandsverify_user
and purge_user
now accept multiple
usernames.refresh_scores
now recalculates user scores and accepts
multiple usernames. It can be run across projects and/or languages.contributors
command has been refactored in order to return more
accurate results and has new options
--since
,
--until
and
--mailmerge
. The --from-revision
option has been removed.flush_cache
flushes default
, redis
caches,
accepts --rqdata
,
--django-cache
options.export
is now able to export zipped TMX files per translation
project with the --tmx
option.
--rotate
option allows old files to be removed.init
now creates a development configuration with
--dev
option.list_serializers
allows to view serializers and deserializers
installed on your system.config
allows to get, set, list, append and clear configuration
settings.init_fs_project
.set_filetype
.schema
allows to dump the database schema on MySQL which is
useful for diagnosing differences in database schema.update_data
allows to update the stats data.runserver
instead.STRICT_TRANS_TABLES
.POOTLE_TM_SERVER
no longer receives the MIN_SCORE
parameter,
as it was misleading and had questionable effects.POOTLE_TM_SERVER
now accepts a MIN_SIMILARITY
parameter, to
filter out results which might be irrelevant. To learn more, check the
documentation on
MIN_SIMILARITY
.POOTLE_MARKUP_FILTER
defaults to 'markdown'
, and None
,
'html'
, 'textile'
and 'restructuredtext'
values have been
deprecated. Deployments using any deprecated markup must migrate manually
to Markdown. This setting will be removed in the future since Markdown will
be the only available markup.POOTLE_SCORES
accepts custom settings for user scores
calculation.POOTLE_SEARCH_BACKEND
to allow configuring the search backend to
be used.POOTLE_EMAIL_FEEDBACK_ENABLED
to allow disabling sending emails
to suggesters when suggestions are accepted or rejected.POOTLE_CUSTOM_LOGO
, POOTLE_FAVICONS_PATH
,
POOTLE_FS_WORKING_PATH
and POOTLE_CANONICAL_URL
settings to allow easy customisations.POOTLE_SQL_MIGRATIONS
.AMAGAMA_SOURCE_LANGUAGES
.POOTLE_QUALITY_CHECKER
since the custom quality checkers feature is
gone.POOTLE_SCORE_COEFFICENTS
has been removed and replaced with
POOTLE_SCORES
.This release was made possible by the following people:
Ryan Northey, Dwayne Bailey, Julen Ruiz Aizpuru, Taras Semenenko, Leandro Regueiro, Igor Afanasyev, Claude Paroz, Safa Alfulaij, Rene Ladan, Kevin Scannell, Jason P. Pickering, Eamonn Lawlor, Alexander Lakhin, Robbie Cole, Rhoslyn Prys, Prasasto Adi, Nootan Ghimire, Mikhail Paulyshka, Mike Robinson, leonardcj, Henrik Feldt, Francesc Ortiz, Allan Nordhøy, Christian Lohmaier, Burhan Khalid, benbankes, Arash Mousavi, Andy Kittner, Adam Chainz.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 20 June 2016
Bugfix release for 2.7.5.
This release was made possible by the following people:
Leandro Regueiro.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 21 May 2016
Bugfix release for 2.7.4.
Changes since 2.7.4:
View the git log for complete information.
This release was made possible by the following people:
Leandro Regueiro, Dwayne Bailey, Ryan Northey, Jason P. Pickering.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 19 May 2016
Bugfix release for 2.7.3.
Changes since 2.7.3:
View the git log for complete information.
This release was made possible by the following people:
Leandro Regueiro, Taras Semenenko, Mikhail Paulyshka, Dwayne Bailey.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 27 April 2016
Bugfix release for 2.7.2.
Below we provide much more detail. These are by no means exhaustive, view the git log for complete information.
There are some changes that haven’t being reported on their corresponding release notes at the time:
update_tmserver
:--overwrite
to --refresh
.--include-disabled-projects
.POOTLE_TM_SERVER
:default
TM server has been renamed to local
. Make sure to
adjust your settings.WEIGHT
option to raise
or lower the TM results score for each specific TM server.export
command without options.--user
to import
to
attribute changes to specified user on file import.initdb
:--no-projects
option to prevent
creating the default projects at set up.test_checks
errors when being run with no options and
without the --check
option.…and lots of refactoring, new tests, cleanups, improved documentation and of course, loads of bugs were fixed.
This release was made possible by the following people:
Julen Ruiz Aizpuru, Leandro Regueiro, Ryan Northey, Dwayne Bailey, Taras Semenenko.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 22 September 2015
Bugfix release for 2.7.1.
Below we provide much more detail. These are by no means exhaustive, view the git log for complete information.
elasticsearch
is unavailable.
elasticsearch
version must now be 1.6.0 at most.…and cleanups, improved documentation.
This release was made possible by the following people:
Dwayne Bailey, Leandro Regueiro, Julen Ruiz Aizpuru, Ryan Northey, Taras Semenenko.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 18 September 2015
Bugfix release for 2.7.0.
Below we provide much more detail. These are by no means exhaustive, view the git log for complete information.
refresh_stats
command (issue 3835).refresh_stats_rq
command to refresh_stats
. The
old refresh_stats
command is now gone.On deleting a user account their submissions, suggestions and reviews are now re-assigned to the “nobody” user.
If you wish to remove the user’s contributions also, you can use the
purge_user
command, or call user.delete(purge=True)
to delete the
user programatically.
contributors
command to get the list of contributors
(issue 3867).find_duplicate_emails
command to find duplicate emails.merge_user
command to get merge submissions, comments and
reviews from one user account to another. This is useful for fixing users
that have multiple accounts and want them to be combined. No profile data
is merged. By default it removes the original user account after successful
merge.purge_user
command to purge a user from the site and revert
any submissions, comments and reviews that they have made. This is useful to
revert spam or a malicious user.verify_user
command to automatically verify a user accountrefresh_stats_rq
command to refresh_stats
, replacing
the old command of the same name. refresh_stats
is able to
calculate the stats for disabled projects (old refresh_stats_rq
was
unable to do it).refresh_stats
has removed the
following options:--calculate-checks
and --check
– Use calculate_checks
instead.--calculate-wordcount
update_user_email
command to update a user’s email
address.--no-rq
option to run commands in a single process without
using RQ workers.init command
.Disabled projects are visually differentiated in the projects drop-down
(issue 3996).
Since the in-cache data structure supporting this changed, it’s necessary to
clear the cache. Assuming your default
cache lives in the DB number 1
,
you can clear it as follows:
$ redis-cli -n 1 KEYS "*method-cache:Project:cached_dict:*" | xargs redis-cli -n 1 DEL
Admins can now always see and navigate disabled projects.
Pulled latest translations.
Scores now include suggestions.
A link is now displayed on the sidebar so admin users can quickly edit the announcements.
Now previously hidden errors during login and sign up are displayed to the user.
Improved usage of system checks so sysadmins get better feedback on whether something is wrong with Pootle.
…and lots of refactoring, new tests, cleanups, improved documentation and of course, loads of bugs were fixed.
This release was made possible by the following people:
Julen Ruiz Aizpuru, Ryan Northey, Taras Semenenko, Leandro Regueiro, Dwayne Bailey, Jerome Leclanche, Kevin Scannell, Daniel Widerin.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 4 August 2015
This is the first release of Pootle that combines the work of Evernote and Translate.
Below we provide much more detail. These are by no means exhaustive, view the git log for complete information.
POOTLE_WORDCOUNT_FUNC
allows a custom word counting method to
be used.pootle.core.utils.wordcount.wordcount
has been incorporated
(it omits placeholders and words that shouldn’t be translated). Non-empty
units with 0 words are immediately translated and marked as fuzzy.view
permission is now project-centric, it can be set server-wide or
for projects, but not for individual language-project combinations.POOTLE_TM_SERVER
and to load the TM use the update_tmserver
management command.POOTLE_CONTACT_REPORT_EMAIL
.dumpdata
and loaddata
commands to move
between databases is no longer supported. If you need to move, please use
proper SQL scripts instead.POOTLE_QUALITY_CHECKER
can be used to point to a custom
quality check handler.POOTLE_REPORTS_MARK_FUNC
allows a site wide function to provide
marks to user graphs.fork()
therefore Pootle will only run on systems that implement
fork()
. Importantly that means that Pootle is no longer supported on
Windows. It would be possible to run Pootle on Windows if the rqworkers are
run on a system that supports fork()
.update_stores
inserts and deletes units in the
store (issue 3802).update_stores
if a directory doesn’t exist while running the
command, the project will be disabled. Thus the
update_translation_projects
command has been removed, it’s functionality
has been merged into update_stores
with this change.changed_languages
management command.--check
flag
passed to the refresh_stats
management command.--calculate-checks
parameter to the refresh_stats
command.refresh_stats_rq
was added to allow statistics to be refresh when
running with multiple RQ workers.system
user to attribute changes done by the management
commands.test_checks
management command.--directory
and --path-prefix
parameters from management
commands. --project
and --language
should be used instead
to reduce the scope of commands.--modified-since
flag from sync_stores
and
update_stores
. Optimizations will automatically be done based on
the latest sync revision.revision
, refresh_scores
,
retry_failed_jobs
, import
, export
,
dump
and calculate_checks
.POOTLE_
.
The following settings are impacted and should be renamed accordingly in your
settings file:TITLE
-> POOTLE_TITLE
CAN_CONTACT
-> POOTLE_CONTACT_ENABLED
CAN_REGISTER
-> POOTLE_SIGNUP_ENABLED
CONTACT_EMAIL
-> POOTLE_CONTACT_EMAIL
PODIRECTORY
-> POOTLE_TRANSLATION_DIRECTORY
MARKUP_FILTER
-> POOTLE_MARKUP_FILTER
USE_CAPTCHA
-> POOTLE_CAPTCHA_ENABLED
MT_BACKENDS
-> POOTLE_MT_BACKENDS
POOTLE_CONTACT_REPORT_EMAIL
-> POOTLE_REPORT_STRING_ERRORS_EMAIL
EXPORTED_FILE_MODE
-> POOTLE_SYNC_FILE_MODE
OBJECT_CACHE_TIMEOUT
-> POOTLE_CACHE_TIMEOUT
LEGALPAGE_NOCHECK_PREFIXES
-> POOTLE_LEGALPAGE_NOCHECK_PREFIXES
CUSTOM_TEMPLATE_CONTEXT
-> POOTLE_CUSTOM_TEMPLATE_CONTEXT
POOTLE_TOP_STATS_CACHE_TIMEOUT
has been removed with the old top
stats rendering and is replaced by the new browsing UI.VCS_DIRECTORY
is now deprecated as the integrated Version Control
feature has been removed to come back at a later date.CONTRIBUTORS_EXCLUDED_PROJECT_NAMES
and
CONTRIBUTORS_EXCLUDED_NAMES
have been removed along with the
contributors’ page.DESCRIPTION
has been removed, use static pages instead.ENABLE_ALT_SRC
has been removedMIN_AUTOTERMS
has been removedMAX_AUTOTERMS
has been removedFUZZY_MATCH_MAX_LENGTH
has been removedFUZZY_MATCH_MIN_SIMILARITY
has been removedEXPORTED_DIRECTORY_MODE
has been removedauth.User
and PootleProfile
before, and has allowed to
remove the dependency on deprecated third party apps that were bundled in the
code.Suggestion
models have been merged into a single model.last_sync_revision
field of the store.creation_time
field.reverse()
and {% url %}
are used almost
everywhere.There are two groups of features that have been dropped:
The following features are removed from Pootle since 2.5.1.3 and will be recovered at some time. Where possible we provide alternate approaches that can be used.
Note
sysadmins should take note of these changes and determine if this prevents use of Pootle within their environment. Essentially you will need to evaluate the use and need for each missing feature.
update_stores
to load the changed files.We have dropped these features, some of which have been kept around to allow easy upgrades in the past:
Note
The removal of some of these feature required extensive changes to the upgrading code, which means that upgrading directly from very old Pootle versions is no longer possible. In case you are trying to upgrade you must first upgrade to 2.6 before continuing the upgrade process.
updatetm
tool was
removed in Pootle 2.5.0.sync_stores
instead.update_translation_projects
, updatedb
,
upgrade
, setup
, assign_permissions
.…and lots of refactoring, upgrades of upstream code, cleanups to remove old Django versions specifics, improved documentation and of course, loads of bugs were fixed.
This release was made possible by the following people:
Julen Ruiz Aizpuru, Taras Semenenko, Dwayne Bailey, Leandro Regueiro, Igor Afanasyev, Jerome Leclanche, Khaled Hosny, pfennig59, Zahim Anass, Trejkaz (pen name), safaalfulaij, Peter Bengtsson, msaad, Mikhail Paulyshka, Miha Vrhovnik, Kevin Scannell, Edmund Huber, Dídac Rios, Andras Timar.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 04 April 2017
The 2.6.4 release is an interim release. It is used to migrate from Pootle 2.5.0 or newer to Pootle 2.7.x or newer releases.
Warning
Do not run a Pootle instance using this version.
This release fixes issues that some users experienced upgrading via 2.6.3
Warning
If you are upgrading from Pootle 2.1.0 or older you must first upgrade to 2.1.6 before upgrading to this version.
Warning
If you are upgrading from Pootle older than 2.5.0 you must first upgrade to 2.5.1.3 before upgrading to this version.
This release was made possible by the following people:
Ryan Northey, Leandro Regueiro
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 30 March 2017
The 2.6.3 release is an interim release. It is used to migrate from Pootle 2.5.0 or newer to Pootle 2.7.x or newer releases.
Warning
Do not run a Pootle instance using this version.
This release fixes issues that some users experienced upgrading via 2.6.2
Warning
If you are upgrading from Pootle 2.1.0 or older you must first upgrade to 2.1.6 before upgrading to this version.
Warning
If you are upgrading from Pootle older than 2.5.0 you must first upgrade to 2.5.1.3 before upgrading to this version.
This release was made possible by the following people:
Ryan Northey, Leandro Regueiro
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 28 September 2015
The 2.6.2 release is an interim release. It is used to migrate from Pootle 2.5.0 or newer to Pootle 2.7.x or newer releases.
Warning
Do not run a Pootle instance using this version.
This release fixes issue issue 4101 that some users experienced upgrading via 2.6.1.
Warning
If you are upgrading from Pootle 2.1.0 or older you must first upgrade to 2.1.6 before upgrading to this version.
Warning
If you are upgrading from Pootle older than 2.5.0 you must first upgrade to 2.5.1.3 before upgrading to this version.
This release was made possible by the following people:
Ryan Northey, Leandro Regueiro
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 15 September 2015
The 2.6.1 release is an interim release. It is used to migrate from Pootle 2.5.0 or newer to Pootle 2.7.x or newer releases.
Warning
Do not run a Pootle instance using this version.
This release fixes issues that some users experienced upgrading via 2.6.0
Warning
If you are upgrading from Pootle 2.1.0 or older you must first upgrade to 2.1.6 before upgrading to this version.
Warning
If you are upgrading from Pootle older than 2.5.0 you must first upgrade to 2.5.1.3 before upgrading to this version.
This release was made possible by the following people:
Ryan Northey, Leandro Regueiro
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 29 June 2015
The 2.6.0 release is an interim release. It is used to migrate from Pootle 2.5.0 or newer to Pootle 2.7.0.
Warning
Do not run a Pootle instance using this version.
Direct upgrade is now only possible from 2.5.0 and later.
We have dropped some legacy upgrade features. The removal of some of these feature means that upgrading directly from ancient Pootle versions is no longer possible.
Warning
If you are upgrading from Pootle 2.1.0 or older you must first upgrade to 2.1.6 before upgrading to this version.
Warning
If you are upgrading from Pootle older than 2.5.0 you must first upgrade to 2.5.1.3 before upgrading to this version.
This release was made possible by the following people:
Leandro Regueiro, Julen Ruiz Aizpuru, Jerome Leclanche, Igor Afanasyev, Taras Semenenko, Dwayne Bailey, Khaled Hosny, Arky, Peter Bengtsson, அருண் குமார், Sebastian Silva, ricordisamoa, Miha Vrhovnik, Kevin KIN-FOO, Henrik Saari, Greg Slepak, Folkert van Heusden, Clement Wong, Alexandre Segura, afan.
And to all our bug finders, testers and translators, a Very BIG Thank You.
Released on 2015-06-03
This is a bugfix release for the 2.5.1 branch. It is meant to provide a newer stable version until Pootle 2.7.0 is released.
For a full list of changes, please check the git log.
xliff
extension for XLIFF filesThe following people have made this release possible:
Dwayne Bailey, Leandro Regueiro, Miha Vrhovnik, Kevin KIN-FOO, Julen Ruiz Aizpuru.
Released on 2015-06-01
The 2.5.1.2 release is a bugfix release for the 2.5.1 branch. It is meant to provide a newer stable version until Pootle 2.7.0 is released.
For a full list of changes, please check the git log.
xliff
extension for XLIFF filesThe following people have made Pootle 2.5.1.2 possible:
Dwayne Bailey, Leandro Regueiro, Miha Vrhovnik, Kevin KIN-FOO, Julen Ruiz Aizpuru.
Released on 2014-04-29
The 2.5.1.1 release is a bugfix release for the 2.5.1 branch.
For a full list of changes, please check the git log.
POOTLE_TOP_STATS_CACHE_TIMEOUT
.The following people have made Pootle 2.5.1.1 possible:
Julen Ruiz Aizpuru, Leandro Regueiro, Dwayne Bailey, Khaled Hosny, Jerome Leclanche, Igor Afanasyev and @qdinar.
Released on 24 January 2014
Yes, we did miss our 6 month release cycle! Many changes have gone into Pootle 2.5.1 which follows on from 2.5.0 released in May.
Pootle 2.5.1 has been in production for a number of users, so although it is a new official release, we’ve had many people running their production Pootle server off this code. This includes Mozilla and Evernote. So you are in good company.
For those who can’t wait you might be interested to know what we’ve got planned on our roadmap for Pootle 2.5.2.
These are by no means exhaustive, check the git log for more details.
pootle.core.auth.ldap_backend.LdapBackend
and received various fixes.…and lots of refactoring, upgrades of upstream code, cleanups to remove Django 1.3 specifics, missing documentation and of course, loads of bugs were fixed
The following people have made Pootle 2.5.1 possible:
Julen Ruiz Aizpuru, Leandro Regueiro, Dwayne Bailey, Alexander Dupuy, Khaled Hosny, Arky, Fabio Pirola, Christian Hitz, Taras Semenenko, Chris Oelmueller, Peter Bengtsson, Yasunori Mahata, Denis Parchenko, Henrik Saari, Hakan Bayindir, Edmund Huber, Dmitry Rozhkov & Darío Hereñú
Released on 18 May 2013
Finally! Translate has a new baby and we’re pretty proud of her. Many changes have gone into 2.5.0 which follows on from 2.1.6 released more then two years ago. So many changes that it’s quite hard to list them all.
Why so long? Well we had the Egyptian revolution, a complete change in UI, and a load of features we wanted you to have. It took much longer to stabilise it for you to enjoy.
Pootle 2.5.0 has been in production with many users, so although it is a new official release, we’ve had many people running their production server off this code. This includes LibreOffice, Mozilla and Evernote. So you are in good company.
These are by no means exhaustive, check the git log for more details
We undertook a major UI rework – we now have a clean new translation interface, and overview page.
In the editor:
In the overview page:
VCS_DIRECTORY
for VCS checkout is where Pootle now does all VC related work – this ensures
that we can work well with DVCS like Git.New and changed commands:
django.contrib.staticfiles
module.
This means you will need to run the pootle collectstatic
command on production and serve the pootle/assets/
directory from your webserver at /assets/. If you are upgrading from a
previous version, you will need to replace the occurrences of static with
assets within your web server configuration.PootleServer
script has been phased out in favor of a pootle
runner script.UTC
, unless you are using PostgreSQL. Users of PostgreSQL or Django
1.4 or later are free to set the time zone as they prefer. Also make sure to
use the minimum required South version when performing database upgrades.…and of course, loads of bugs were fixed
The following people have made Pootle 2.5.0 possible:
Julen Ruiz Aizpuru, Friedel Wolff, Alaa Abd el Fattah, Igor Afanasyev, Dwayne Bailey, Leandro Regueiro, Claude Paroz, Chris Oelmueller, Taras Semenenko, Kevin Scannell, Christian Hitz, Thomas Kinnen, Alexander Dupuy, khagaroth, dvinella, Stuart Prescott, Roman Imankulov, Peter Bengtsson, Nagy Akos, Michael Tänzer, Gregory Oschwaldi & Andy Nicholson.
Released on 13 April 2011
It’s been 3 months since our last bug fix releases, it’s about time we give you Pootle 2.1.6.
Pootle is a web based system for translation and translation management.
Main focus of the release is incompatibility issues with the latest versions of Django (1.2.5 and 1.3.0).
Apart from that, version 2.1.6 has a handful of fixes. Here are the highlights:
On the first visit after upgrading upgrade screen will flash for a short period while translation statistics are recalculated, if running under Translate Toolkit version 1.9.0 it might last longer as Qt TS files will be reparsed to benefit from improvements to the format support.
Django 1.2.5 and 1.3.0 compatibility depends on Translate Toolkit version 1.9.0 or above but all users are encouraged to upgrade their versions of Translate Toolkit. As always Pootle will benefit from fixes and performance improvements in the latest versions.
Released on 18 Jan 2011
A quick bug fix release to celebrate the new Year. Please welcome Pootle 2.1.5!
Pootle is a web based system for translation and translation management.
This release fixes a couple of regressions introduced in the previous 2.1.4 release. Including a build mistake where the files in the 2.1.4 tarball had very restrictive permissions.
Apart from that, version 2.1.5 has a handful of fixes. Here are the highlights:
As always Pootle will benefit from fixes and performance improvements in the latest versions of Translate Toolkit.
Enjoy it, The Translate Team
Released on 17 Dec 2010
We thought we’d wrap up the year with one more bug fix release, Please welcome Pootle 2.1.4
Pootle is a web based system for translation and translation management.
This release fixes a nasty bug where quality checks failed to update on file uploads. the upgrade screen will flash on first visit after upgrade for a minute or two to correct this problem (might take longer if you used the quality checks feature extensively).
Apart from that, version 2.1.4 has a handful of fixes. Here are the highlights:
As always Pootle will benefit from fixes and performance improvements in the latest versions of Translate Toolkit.
Released on 26 Nov 2010
It’s been less than three weeks since the we released Pootle 2.1.2 but we’ve fixed a couple of critical bugs affecting many users so it’s time for another bug fix release. Please welcome Pootle 2.1.3
Pootle is a web based system for translation and translation management.
This release includes a fix to a data loss bug, where recent translations are lost when updating from version control. Users who depend on version control support are encouraged to upgrade immediately.
We’ve added support for CSV format. This will hopefully make it easier for less technical users to get their strings inside Pootle by exporting from spreadsheet or similar office software. But it should not be treated as a replacement for more solid formats like PO, Qt ts or XLIFF.
By popular demand we’ve improved Java properties support to accept properties files in any encoding. including UTF-8.
Improved format support depends on the recently release Translate Toolkit 1.8.1
We also bring you translations for Chiga and Latvian.
Apart from that, version 2.1.2 has many bug fixes. Here are the highlights:
As always Pootle will benefit from fixes in any the latest versions of Translate Toolkit, the recently released 1.8.1 includes many fixes specifically for Pootle 2.1.3 so upgrading translate toolkit is highly recommended.
Released on 15 Nov 2010
https://sourceforge.net/projects/translate/files/Pootle/2.1.2/Pootle-2.1.2.tar.bz2
This release includes an important security fix to a cross site scripting vulnerability in the translate page. All users are encouraged to upgrade immediately.
The release also includes many improvements to the support of monolingual translation formats (like subtitles files and Java properties) and to “GNU style” projects.
We also bring you translations for five new language (Zulu, Greek, Danish, Acoli and Fulah) and six more translations are now 100% complete (Uighur, Chinese (China), Catalan, Asturian, Akan and Ganda).
Highlighted fixed and improvements:
Pootle 2.1.1 depends on at least version 1.8.0 of Translate Toolkit, and as always will benefit from fixes in any later versions. so always use the latest.
This work was made possible by many volunteers and our funders:
Released on 03 September 2010
With the coming of spring we thought it’s a good time to make the first bug fix release of the exciting new Pootle. We bring you Pootle 2.1.1 get it while it is blooming from https://sourceforge.net/projects/translate/files/Pootle/2.1.1/
Pootle is a web based system for translation and translation management.
This release finally brings the ability to migrate data between different database engines. This means all of you stuck with the default sqlite3 can now move to a database engine that scales better like MySQL or PostgreSQL.
Note that database migration depends on Django 1.2 or later.
As an added bonus we added database migration to the 2.0 branch and quietly slipped in the last bug fix release for that series https://sourceforge.net/projects/translate/files/Pootle/2.0.6/ we made this bonus release so users still on the 2.0 branch using sqlite can migrate databases before they upgrade to 2.1 since the upgrade process is slow and the database size under 2.1 is considerably larger.
For instructions and more details check Database migration docs.
We noticed some users running Pootle under apache fail to use memcached for caching and stick to the default local memory cache backend. This causes buggy behavior as the default is not compatible with multiprocess servers. So for 2.1.1 we changed the default to a database cache backend. We still recommend using memcached but if for any reason you can’t please update your localsettings.py.
Users upgrading from 2.1.0 will see the upgrade screen appear for a few seconds while Pootle prepares the database for the new cache backend.
For more information check Caching System docs.
Apart from these two major changes 2.1.1 includes four new translations (Slovenian, Songhai, Tamil and Faroese) and many fixes and performance improvements. Here are the highlights:
Pootle 2.1.1 depends on at least version 1.8.0 of Translate Toolkit, and as always will benefit from fixes in any later versions. So always use the latest.
This work was made possible by many volunteers and our funders:
Older release more for your entertainment and to track Pootle’s history.
Released on August 17th 2010.
manage.py
command can sync to files on the
command line.easy_install South
(the upgrade could take quite a while, depending on your installation size).Released on December 7th 2009.
Released on October 8th 2008.
The name of the directory for indexing databases changed from .poindex-PROJECT-LANGUAGE to .translation_index. Administrators may want to remove the old indexing directories manually.
The enhanced search function needs all indexing databases to be regenerated, otherwise it won’t find anything. To achieve this, just remove all .translation_index directories under your projects:
find /path/to/projects/ -type d -name ".translation_index" -exec rm -rf {} \;
If you used testing versions of Pootle 1.2, you almost definitely need to regenerate your statistics database. Pootle might be able to do it automatically, but if not, delete ~/.translate_toolkit/stats.db.
Released on May 25th 2007.
localfiletype
for each project.treestyle
attribute for the project in pootle.prefs.
Currently this can not be specified through the admin interface.Released on March 8th 2007.
Released on August 29th 2006.
The statistics pages are greatly reworked. We now have a page that shows a nice table, that you can sort, with graphs of the completeness of the files. This is the default view. What is confusing is that the stats page does not work directly with editing. To get the editing features, click on the editing link in the top bar.
The quick statistics files (pootle-projectname-zu.stats) now also store the fuzzy stats that are needed to render the statistics tables. Your previous files from 0.9 can not supply this information. Pootle 0.10 will automatically update these files, but if you (for some reason) want/need to go back to Pootle 0.9, you will have to delete these files. Not all .stats files need to be deleted, only the ones starting with pootle-projectname.
The Pootle code and documentation is released under the GNU General Public License (GPL), version 3 or later.