Pootle Documentation¶

The sync_stores in our recipe above ensures that everything is in sync. However if you have scripts that commit and update files you might prefer to let the filesystem win in which case rather use:

(env) $ pootle fs fetch --force MYPROJECT

1. Make sure you have installed the needed Pootle FS plugin for the version control backend you are using.

2. (optional but recommended) Disable the project.

3. Ensure you have synchronized all your files and committed them to your version control system.

4. Instead of localfs, set the backend appropriately.

5. Set the URL to your version control repository.

6. Synchronize as follows:

   (env) $ pootle fs fetch --force MYPROJECT
   (env) $ pootle fs sync MYPROJECT
   

You want to keep the version that is currently on the filesystem, discarding all changes in Pootle:

(env) $ pootle fs resolve --overwrite --pootle-wins MYPROJECT
(env) $ pootle fs sync MYPROJECT

You wish to keep the version that is currently in Pootle, discarding all changes in the filesystem:

(env) $ pootle fs resolve --overwrite MYPROJECT
(env) $ pootle fs sync MYPROJECT

To retain all translation and allow translators to resolve conflicts use resolve. This will merge any non-conflicting units and convert conflicts into suggestions, by default we use filesystem translations:

(env) $ pootle fs resolve MYPROJECT
(env) $ pootle fs sync MYPROJECT

The result is that all non-conflicting units have been synchronised. For any unit where both the store unit and file unit changed the translation is set to the file unit translation with the store unit translation converted into a suggestion. You can now review these suggestions to resolve the conflicts.

To retain all translation and allow translators to resolve conflicts use resolve. This will merge any non-conflicting units and convert conflicts into suggestions, the --pootle-wins option ensures that we use Pootle translations and convert filesystem translations into suggestions:

(env) $ pootle fs resolve --pootle-wins MYPROJECT
(env) $ pootle fs sync MYPROJECT

Special characters do not solve the input needs for all languages, but has been a very useful help for many languages, especially in translate@thons.

For people using non-Latin scripts, consider if it will be useful to perhaps include things that can’t be easily typed by translators in your language. You will probably need to limit the number of characters, but hopefully you can find a reasonable compromise that will help many people.

By default Pootle will query Translate’s Amagama Translation Memory server, which hosts translations of an extensive collection of Opensource software.

If you want to setup and connect to your own TM server then the AMAGAMA_URL will allow you to point to a private TM server.

To disable Amagama set AMAGAMA_URL to ''.

Pootle can also retrieve TM matches stored on Elasticsearch-based TM servers. These TM servers require Elasticsearch to be installed and running.

Install the required Python libraries:

(env) $ pip install --process-dependency-links Pootle[es5]

Pootle supports two types of Elasticsearch-based TMs:

• Local TM: (just one, named local) is populated using translations stored in Pootle database and every new translation gets automatically imported to it.
• External TMs: (several) are populated from translation files specifically provided by the server admins, and are not automatically updated.

Both local and external TM settings can be adjusted in POOTLE_TM_SERVER. A configuration example for local and external TM can be found in the default ~/.pootle/pootle.conf, and can be enabled by uncommenting the example.

Please see the POOTLE_TM_SERVER-WEIGHT for a full example of the configuration necessary to set up local/external TM.

Both Amagama and Elasticsearch based TMs can operate together. Though you may want to disable Amagama.

(env) $ pootle update_tmserver

(env) $ pootle update_tmserver --tm=external --display-name=Pidgin af.po gl.tmx

(env) $ pootle update_tmserver --tm=external --display-name=GNOME pt.tmx eu.po xh.po

POOTLE_TM_SERVER = {

    ...

    'libreoffice': {
        'ENGINE': 'pootle.core.search.backends.ElasticSearchBackend',
        'HOST': 'localhost',
        'PORT': 9200,
        'INDEX_NAME': 'whatever',
        'WEIGHT': 0.9,
        'MIN_SCORE': 'AUTO',
    },
}

(env) $ pootle update_tmserver --tm=libreoffice --display-name=LibreOffice af.po gl.tmx

When creating an announcement page use a slug projects/$project so that the page will be used on the $project project.

Other slug names may be used:

• $lang - for an announcement page that will appear on every single project enabled for the $lang languages.
• $lang/$project - for an announcement page that is specific to the $project project in the $lang language.

The prefered model though is to use the projects/$project convention for a single easy-to-maintian page.

In many cases you have URLs in announcement pages that would be identical except for variations in the language code. Examples would include links to team wiki pages, signoff pages, progress dashboards, live test versions, etc.

Any link within your announcement page that uses a fake language code of /xx/ will be rewritten with the language code for this translation. Thus if you insert a link such as http://example.com/signoff/xx/ then that will be rewritten to http://example.com/signoff/af/ for a user viewing this announcement page for the Afrikaans language translation.

There are four different roles, which provide incremental permissions within the team:

Member

May submit suggestions.

Submitter

Can translate and make suggestions.

Reviewer

In addition to translate and submit suggestions, can also review suggestions.

Administrator

Can administer the team, adding team members and adjusting roles. May edit the announcement of the team. Plus all rights of the Reviewer.

Add new members using the form at the top left. Simply, select the user, choose a role and click Add team member button.

To remove members check the box next to their username and then click on the Remove selected button.

To change a member’s role first remove the member from the team, then add the member back with the desired role.

Access rights can be set server-wide or for projects. Bear in mind that when limiting access to projects the permissions affect to all the languages available in the project.

view

Gives access to a project.

hide

Forbids access to a project.

Permissions restricting actions can be set server-wide, per language, or language-project combination:

suggest

The right to suggest a translation for a specific string, also implies the right to upload file using suggest only method.

review

The right to review the suggested translations and accept or reject them, as well as the right to reject false positive quality checks

translate

The right to supply a translation for a specific string or to replace the existing one. This implies the right to upload files using the merge method.

administrate

The right to administrate the project or language including administer permissions and delegating rights to users (this is not the same as the site administrator)

In the list of permissions, you can simply select which rights must be assigned to each user by picking new permissions or unassigning them. Changes will be updated when you submit the form.

To set permissions for a specific user, select the user in the dropdown list and set the specific rights for that user. This is only necessary if the user does not yet have their own set of rights defined.

To reset some user’s rights to the default rights, select the “Delete” tick box next to their name and permissions list. When you submit, their rights will be reset to the default rights.

You need to place the translation files for your new project in a location where Pootle can find, read and write them. Pootle uses the POOTLE_TRANSLATION_DIRECTORY setting to find out where translation files are stored on the server.

Our example project uses a GNU layout.

A GNU layout means that our project contains translation files named using language codes. Within the project there are no directories, just files. There can only be a single translation file per language in a project using this layout.

This is the simplest layout possible and the reason we are using it in our example.

Below you can see an example with two projects using the GNU layout:

`-- translations
    `-- project1
    |   |-- de.po
    |   |-- fr.po
    |   |-- gl.po
    |   |-- pt_BR.po
    |   `-- templates.pot
    `-- project2
        |-- af.po
        |-- eu.po
        |-- pt_BR.po
        |-- templates.pot
        `-- zu.po

Among the regular translation files there are two files named templates.pot. These are the template (master or reference) files that contain the original strings. Usually these files contain only English strings, however it is much less confusing to use the term templates than e.g. en or English.

To get started, create a my-project directory in the location pointed to by POOTLE_TRANSLATION_DIRECTORY and place within it the translation files for your new project. Make sure you have a templates.pot among those project translation files.

At the top of the user interface, you will see your newly created administrator username. Click on it and the main top menu will be displayed, then click on Admin (highlighted in red):

Now you are in the administration interface. Go to the Projects tab and you will see a New Project button:

Click on that button and the Add Project form will be displayed. Enter the new project’s details. Code must match the name of the directory within POOTLE_TRANSLATION_DIRECTORY that contains the project translation files, in our example my-project. Also you must specify the File Types used in this new project, in our example Gettext PO (po/pot).

You can also provide a Full Name easily readable for humans. You don’t need to change the rest of the fields unless you need to further customize your project.

Once you are done click on the Save button below the form to create the project. Creating the project doesn’t actually import all the translations to Pootle. To do that you need to run update_stores on the command line of the Pootle server:

(env) $ pootle update_stores --project=my-project

This will import all the translations from disk into Pootle, calculate the translation statistics and calculate the quality check failures. This might take a while for a large project.

Looking at your new project you will see that Pootle has imported all the existing translations for the existing languages that you copied to the my-project directory within POOTLE_TRANSLATION_DIRECTORY.

MySQL terminates idle connections after wait_timeout seconds. Thus setting CONN_MAX_AGE to a lower value will be fine (it defaults to 0). Persistent connections where CONN_MAX_AGE is None can’t be used with MySQL.

To learn more please check Django’s docs on persistent connections and connection management.

DATABASES = {
    'default': {
        ...
        'CONN_MAX_AGE': 0,
        ...
    }
}

The default value for CONN_MAX_AGE is 0. It means that Django creates a connection before every request and closes it at the end. PostgreSQL supports persistent connections, and it will be fine to set CONN_MAX_AGE to None.

To learn more please check Django’s docs on persistent connections and connection management.

DATABASES = {
    'default': {
        ...
        'CONN_MAX_AGE': None,
        ...
    }
}

If you want to reverse proxy through Apache, you will need to have mod_proxy installed for forwarding requests and configure it accordingly.

ProxyPass / http://localhost:8000/
ProxyPassReverse / http://localhost:8000/
ProxyPreserveHost On

Make sure to review your global Apache settings (something like /etc/apache2/httpd.conf or /etc/httpd/conf/httpd.conf) for the server-pool settings. The default settings provided by Apache are too high for running a web application like Pootle. The ideal settings depend heavily on your hardware and the number of users you expect to have. A moderate server with 1GB memory might set MaxClients to something like 20, for example.

Make sure Apache has read access to all of Pootle’s files and write access to the POOTLE_TRANSLATION_DIRECTORY directory.

First it is necessary to create a WSGI loader script:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import site
import sys


# You probably will need to change these paths to match your deployment,
# most likely because of the Python version you are using.
ALLDIRS = [
    '/var/www/pootle/env/lib/python2.7/site-packages',
    '/var/www/pootle/env/lib/python2.7/site-packages/pootle/apps',
]

# Remember original sys.path.
prev_sys_path = list(sys.path)

# Add each new site-packages directory.
for directory in ALLDIRS:
    site.addsitedir(directory)

# Reorder sys.path so new directories at the front.
new_sys_path = []

for item in list(sys.path):
    if item not in prev_sys_path:
        new_sys_path.append(item)
        sys.path.remove(item)

sys.path[:0] = new_sys_path

# Set the Pootle settings module as DJANGO_SETTINGS_MODULE.
os.environ['DJANGO_SETTINGS_MODULE'] = 'pootle.settings'


# Set the WSGI application.
def application(environ, start_response):
    """Wrapper for Django's WSGIHandler().

    This allows to get values specified by SetEnv in the Apache
    configuration or interpose other changes to that environment, like
    installing middleware.
    """
    try:
        os.environ['POOTLE_SETTINGS'] = environ['POOTLE_SETTINGS']
    except KeyError:
        pass

    from django.core.wsgi import get_wsgi_application
    _wsgi_application = get_wsgi_application()
    return _wsgi_application(environ, start_response)

Place it in /var/www/pootle/wsgi.py. If you use a different location remember to update the Apache configuration accordingly.

A sample Apache configuration with mod_wsgi might look like this:

WSGIRestrictEmbedded On
WSGIPythonOptimize 1

<VirtualHost *:80>
    # Domain for the Pootle server. Use 'localhost' for local deployments.
    #
    # If you want to deploy on example.com/your-pootle/ rather than in
    # my-pootle.example.com/ you will have to do the following changes to
    # this sample Apache configuration:
    #
    # - Change the ServerName directive to:
    #   ServerName example.com
    # - Change the WSGIScriptAlias directive to (note that /your-pootle must
    #   not end with a slash):
    #   WSGIScriptAlias /your-pootle /var/www/pootle/wsgi.py
    # - Change the Alias directive for 'assets' to include the '/your-pootle'.
    # - Include the following settings in your custom Pootle settings:
    #   STATIC_URL = '/your-pootle/assets/'
    #   FORCE_SCRIPT_NAME = '/your-pootle'
    # - If you have previously calculated the stats:
    #   - Restart the RQ workers.
    #   - Run refresh_stats to recalculate the stats data.
    ServerName my-pootle.example.com

    # Set the 'POOTLE_SETTINGS' environment variable pointing at your custom
    # Pootle settings file.  An initial settings file can be created using
    # 'pootle init'
    #
    # This might require enabling the 'env' module.
    SetEnv POOTLE_SETTINGS /var/www/pootle/your_custom_settings.conf


    # The following two optional lines enable the "daemon mode" which
    # limits the number of processes and therefore also keeps memory use
    # more predictable.
    WSGIDaemonProcess pootle processes=2 threads=3 stack-size=1048576 maximum-requests=500 inactivity-timeout=300 display-name=%{GROUP} python-path=/var/www/pootle/env/lib/python2.7/site-packages
    WSGIProcessGroup pootle

    # Point to the WSGI loader script.
    WSGIScriptAlias / /var/www/pootle/wsgi.py

    # Turn off directory listing by default.
    Options -Indexes

    # Compress before being sent to the client over the network.
    # This might require enabling the 'deflate' module.
    SetOutputFilter DEFLATE
    AddOutputFilterByType DEFLATE text/html text/css text/plain text/xml application/x-javascript

    # Set expiration for some types of files.
    # This might require enabling the 'expires' module.
    ExpiresActive On

    ExpiresByType image/jpg "access plus 10 years"
    ExpiresByType image/png "access plus 10 years"
    ExpiresByType text/css "access plus 10 years"
    ExpiresByType application/x-javascript "access plus 10 years"

    # Optimal caching by proxies.
    # This might require enabling the 'headers' module.
    Header set Cache-Control "public"

    # Directly serve static files like css and images, no need to go
    # through mod_wsgi and Django. For high performance consider having a
    # separate server.
    Alias /assets /var/www/pootle/env/lib/python2.7/site-packages/pootle/assets
    <Directory /var/www/pootle/env/lib/python2.7/site-packages/pootle/assets>
        Require all granted
        # For apache 2.2, comment the line directly above, and uncommend the two lines directly below
        #Order deny,allow
        #Allow from all
    </Directory>

</VirtualHost>

You can find more information in the Django docs about Apache and mod_wsgi.

The default Pootle server runs at port 8000 and for convenience and simplicity does ugly things such as serving static files — you should definitely avoid that in production environments.

By proxying Pootle through nginx, the web server will serve all the static media and the dynamic content will be produced by the app server.

server {
   listen  80;
   server_name  pootle.example.com;

   access_log /path/to/pootle/logs/nginx-access.log;
   gzip on; # Enable gzip compression

   charset utf-8;

   location /assets {
       alias /path/to/pootle/env/lib/python2.6/site-packages/pootle/assets/;
       expires 14d;
       access_log off;
   }

   location / {
     proxy_pass         http://localhost:8000;
     proxy_redirect     off;

     proxy_set_header   Host             $host;
     proxy_set_header   X-Real-IP        $remote_addr;
     proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
   }
 }

This file contains base configuration settings.

POOTLE_INSTANCE_ID

Instance ID. This is to differentiate multiple instances of the same app (e.g. development, staging and production). By default this value is exposed as a global <html> class name to allow overriding CSS rules based on the instance type.

POOTLE_TITLE

Default: 'Pootle Translation Server'

The name of the Pootle server.

POOTLE_SQL_MIGRATIONS

Default: True

New in version 2.8.

Some migrations have been optimized by using SQL directly and bypassing the Django ORM (Currently only for mysql migrations).

This could cause problems if, for example, you have developed custom models with foreign keys to Pootle models.

Backend and caching settings.

POOTLE_CACHE_TIMEOUT

Default: 604800 (a week)

New in version 2.7.

Time in seconds to keep certain objects cached in memory (template fragments, language and project lists, permissions, etc.).

Note that for anonymous users Pootle also uses Django’s caching middleware, and its settings can be configured separately.

POOTLE_LOG_DIRECTORY

Default: working_path('log')

New in version 2.7.

The directory where Pootle writes event logs to. These are high-level logs of events on store/unit changes and pootle commands executed.

Site-specific settings.

POOTLE_CONTACT_ENABLED

Default: True

Controls whether users will be able to use the contact form. The address to receive messages is controlled by POOTLE_CONTACT_EMAIL.

POOTLE_CONTACT_EMAIL

Default: info@YOUR_DOMAIN.com

Address to receive messages sent through the contact form. This will only have effect if POOTLE_CONTACT_ENABLED is set to True.

POOTLE_CONTACT_REPORT_EMAIL

Default: POOTLE_CONTACT_EMAIL

New in version 2.7.

Email address to report errors on strings.

POOTLE_EMAIL_FEEDBACK_ENABLED

Default: False

New in version 2.8.

Controls whether emails are sent to suggesters when a reviewer accepts or rejects their suggestions providing some comment for the suggester.

POOTLE_CANONICAL_URL

Default: http://localhost

New in version 2.8.

Canonical URL, without trailing slash, used when deriving the URLs to send out emails. If you use the django.contrib.sites framework set this to blank string.

POOTLE_CUSTOM_LOGO

Default: ""

New in version 2.8.

Custom logo URL - this can be an absolute or relative URL.

POOTLE_FAVICONS_PATH

Default: "/assets/favicon"

New in version 2.8.

Customisable favicon path. Should not end with trailing slash.

Configuration settings for applications used by Pootle.

POOTLE_SIGNUP_ENABLED

Default: True

Changed in version 2.7.

Controls whether user sign ups are allowed or not. If set to False, administrators will still be able to create new user accounts.

POOTLE_CUSTOM_TEMPLATE_CONTEXT

Default: {}

Changed in version 2.7.

Custom template context dictionary. The values will be available in the templates as {{ custom.<key> }}.

POOTLE_LEGALPAGE_NOCHECK_PREFIXES

Default: ('/about', '/accounts', '/admin', '/contact', '/jsi18n', '/pages', )

Changed in version 2.7.

List of path prefixes where the LegalAgreementMiddleware will check if the current logged-in user has agreed all the legal documents defined for the Pootle instance. Don’t change this unless you know what you’re doing.

POOTLE_META_USERS

Default: ()

New in version 2.7.

Additional meta, or non-human, accounts. Pootle already manages the ‘system’ and ‘nobody’ users who own system updates to translations and submissions by anonymous users. These meta accounts have their own simple public profiles and won’t track scores.

POOTLE_MARKUP_FILTER

Default: (None, {})

Two-tuple defining the markup filter to apply in certain textareas.

• Acceptable values for the first element are markdown and html (deprecated).
• The second element should be a dictionary of keyword arguments that will be passed to the markup function

Changed in version 2.8: Support for textile, restructuredtext and html formats has been deprecated.

Examples:

POOTLE_MARKUP_FILTER = ('markdown', {})

POOTLE_MARKUP_FILTER = ('markdown', {
                            'clean': {
                                'extra_tags': ['div'],
                                'extra_attrs': {
                                    '*': ['style']
                                    'img': ['alt'],
                                },
                                'extra_styles': [
                                    'color',
                                    'font-weight'
                                ],
                            },
                        })

POOTLE_CAPTCHA_ENABLED

Default: True

Enable spam prevention through a captcha.

POOTLE_REPORTS_MARK_FUNC

Default: '' (empty string)

New in version 2.7.

The graph of a user’s activity, within reports, can be marked to indicate events by using this function. The setting must contain an import path to such a marking function (string).

The function receives the user and graph ranges and returns an array of applicable marks.

Parameters:

• username - user for whom we’re producing this graph
• start (datetime) - start date of the graph
• end (datetime) - end date of the graph

The function must return an array of dictionaries (marks), where every mark has the following properties:

• position, specifying the point in the x-axis where the mark should be set (UNIX timestamp multiplied by 1000), and
• label specifying the text that will be displayed next to the mark.

POOTLE_SCORES

Default:

{
    'suggestion_add': 0,
    'suggestion_accept': .1,
    'suggestion_reject': .1,
    'comment_updated': .1,
    'target_updated': .3,
    'state_translated': .6,
    'state_fuzzy': .1,
    'state_unfuzzy': .1,
}

New in version 2.8.

The default score is based on the wordcount of the source text. The values of the various parameters are used as a multiplier to arrive at the score atributed to the translators and reviewers.

Thus:

score = source wordcount * multiplier

When a translator translates a 10 word string they would get a score using the state_translated multiplier of 0.6.

6 = 10 words * 0.6

Parameters:

• suggestion_* - scoring for the events of adding a new suggestions, accepting or rejecting existing suggestions. By default adding a suggestion gives no score, to prevent users from gaming the system.
• comment_* - making changes to the string comment.
• target_* - adjusting the existing translation.
• state_* - changing the state of the string, this includes translation and fuzzy setting.

POOTLE_FS_WORKING_PATH

Default: working_path('.pootle_fs/tmp/')

New in version 2.8.

The directory that Pootle FS uses to store temporary data for handling the projects.

Warning

This directory can potentially get very large, so you need to place it somewhere with plenty of room.

Translation environment configuration settings.

AMAGAMA_URL

Default: https://amagama-live.translatehouse.org/api/v1/

URL to an amaGama Translation Memory server. The default service should work fine, but if you have a custom server set it here.

This URL must point to the public API URL which returns JSON. Don’t forget the trailing slash.

AMAGAMA_SOURCE_LANGUAGES

Default: ('en', 'en_US', 'en-US')

List of available source languages in amaGama server pointed to by AMAGAMA_URL.

POOTLE_SYNC_FILE_MODE

Default: 0o644

Changed in version 2.7.

On POSIX systems, files synchronized to disk will be assigned this permission. Use 0o644 for publically-readable files or 0o600 if you want only the Pootle user to be able to read them.

POOTLE_TM_SERVER

New in version 2.7.

Changed in version 2.7.3: Added the WEIGHT and MIN_SIMILARITY options. Also added another default TM used to import external translations from files.

Default: {} (empty dict)

Example configuration for local/external TM server:

{
    'local': {
        'ENGINE': 'pootle.core.search.backends.ElasticSearchBackend',
        'HOST': 'localhost',
        'PORT': 9200,
        'INDEX_NAME': 'translations',
        'WEIGHT': 1,
    },
    'external': {
        'ENGINE': 'pootle.core.search.backends.ElasticSearchBackend',
        'HOST': 'localhost',
        'PORT': 9200,
        'INDEX_NAME': 'external-translations',
        'WEIGHT': 0.9,
    },
}

This is configured to access a standard Elasticsearch setup. Change the settings for any non-standard setup. Change HOST and PORT settings as required.

The default local TM is automatically updated every time a new translation is submitted. The other TMs are not automatically updated so they can be trusted to provide selected high quality translations.

Every TM server must have its own unique INDEX_NAME.

WEIGHT provides a weighting factor to alter the final score for TM results from this TM server. Valid values are between 0.0 and 1.0, both included. Defaults to 1.0 if not provided.

MIN_SIMILARITY serves as a threshold value to filter out results that are potentially too far from the source text. The Levenshtein distance is considered when measuring how similar the text is from the source text, and this represents a real value in the (0..1) range, 1 being 100% similarity. The default value (0.7) should work fine in most cases, although your mileage might vary.

POOTLE_MT_BACKENDS

Default: [] (empty list)

This setting enables translation suggestions through several online services.

The elements for the list are two-element tuples containing the name of the service and an optional API key.

Available options are:

GOOGLE_TRANSLATE: Google Translate service.

For this service you need to obtain an API key. Note that Google Translate API is a paid service.

YANDEX_TRANSLATE: Yandex.Translate service.

For this service you need to obtain a Yandex API key.

PARSE_POOL_CULL_FREQUENCY

Default: 4

When the pool fills up, 1/PARSE_POOL_CULL_FREQUENCY number of files will be removed from the pool.

PARSE_POOL_SIZE

Default: 40

To avoid rereading and reparsing translation files from disk on every request, Pootle keeps a pool of already parsed files in memory.

Larger pools will offer better performance, but higher memory usage (per server process).

POOTLE_TRANSLATION_DIRECTORY

Default: working_path('translations')

The directory where projects hosted on Pootle store their translation files. sync_stores will write to this directory and update_stores will read from this directory.

POOTLE_WORDCOUNT_FUNC

Default: translate.storage.statsdb.wordcount

New in version 2.7.

The import path to a function that provides wordcounts for Pootle.

Current options:

• Translate Toolkit (default) - translate.storage.statsdb.wordcount

Adding a custom function allows you to alter how words are counted.

Warning

Changing this function requires that you recalculate the associated wordcounts.

Current action types are as follows:

In addition the SC action type also has its own actions which track the actual type of activity that leads to changes in translation. These are used to track scores for the translators.

Various of the action groups have different message structures as outlined here:

Translation:

date  user  action  lang    unit    path    translation
[2015-05-19T14:11:18]   admin   C       af      2       /af/tutorial/stats-test.po      Twee
[2015-05-19T14:12:17]   admin   A       af      3       /af/tutorial/stats-test.po      Drie
[2015-05-19T14:13:05]   admin   D       af      1       /af/tutorial/stats-test.po

Unit:

date  user    action  language    unit    file    translation
[2015-05-06T16:25:20] system  UA      am      4109    /am/terminology/gnome/am.po     MSDOS
[2015-05-06T16:37:05] system  UA      cs      12043   /cs/terminology/gnome/cs.po     přepínač

Store:

date  user    action  path    store
[2015-05-05T20:23:37] system  SA      /templates/tutorial/tutorial.pot        1

Command:

date  user  action command
[2015-05-06T11:24:28] system  X       pootle update_stores --project=vfolders
[2015-05-05T20:22:46] system  X       pootle migrate

Quality check:

date  user    action  lang    unit    path    translation
[2015-05-19T14:16:36]   admin   QM      af      855     /af/terminology/gnome-terminologie.po   lug
[2015-05-19T14:17:44]   admin   QU      af      855     /af/terminology/gnome-terminologie.po   lug

Score:

date  user    SC  score_delta  score_action    #unit  NS=wordcount    S=similarity   total
[2015-05-19T14:19:11]   admin   SC      1.0     TA      #1      NS=1    S=0.0   (total: 2.28571428571)

The sync_stores and update_stores commands will produce a number of logs to report any activity that results from those commands.

update_stores:

[$date] [update] updated $number units in $store_path [revision: $revision]
[2015-05-19T21:06:24]   [update] updated 1 units in /an/libo_ui/dictionaries/pt_PT.po [revision: 58]

sync_stores:

[$date]   [sync] File saved; updated $number units in $store_path [revision: $revision]
[2015-05-19T23:11:50]   [sync] File saved; updated 1 units in /an/libo_ui/avmedia/source/viewer.po [revision: 0]

You should really switch to a real database backend in production environments. Adjust the DATABASES setting accordingly.

• MySQL
• PostgreSQL

You should really run Pootle behind a real web server, at least to serve static content. For generating the dynamic content, you can also use alternative WSGI servers that might better suit your environment.

Apache

Apache web server.

Nginx

Ngninx web server.

gunicorn

Python WSGI HTTP server.

iso-codes

Enables translated language and country names.

raven

Enables logging server exceptions to a Sentry server. If installed and configured, Pootle will automatically use the raven client.

For Apache, review your server settings so that you don’t support too many or too few clients. Supporting too many clients increases memory usage, and can actually reduce performance.

No specific settings can be recommended, since this depends heavily on your users, your files, and your hardware. However the default value for the MaxClient directive (usually 256) is almost always too high. Experiment with values between 10 and 80.

Using MySQL with InnoDB backend is well tested. MyISAM is no longer supported. You can migrate your current database if you already have data you don’t want to lose.

1. Register your app against your host.

   Application name

   Some descriptive name

   Homepage URL

   URL to your Pootle server, e.g. http://foo.bar.tld

   Application description

   Some descriptive text

   Authorization callback URL

   URL to the callback endpoint of your provider in the Pootle server, e.g. http://foo.bar.tld/accounts/github/login/callback/

2. Let Allauth know about your social provider.

   UPDATE django_site SET DOMAIN = 'foo.bar.tld', name = 'Site name' WHERE id=1;
   
   INSERT INTO socialaccount_socialapp (provider, name, secret, client_id, 'key')
          VALUES ("github", "GitHub", "---Client-Secret-from-above---",
                  "---Client-ID-from-above---", "");
   
   INSERT INTO socialaccount_socialapp_sites (socialapp_id, site_id)
          VALUES (1,1);
   

Note the first line simply sets the domain name for the default site; you can omit it if it’s already up-to-date.

--no-rq¶

Some of the commands work asynchronously and will schedule jobs to RQ workers, rather than running them in the command process. You can change this behaviour using the --no-rq command line option.

This can be useful for running pootle commands in bash scripts or automating installation/upgrade/migration. It can also be useful for debugging otherwise asynchronous jobs.

For example, to run calculate_checks in the command process and wait for the process to terminate:

(env) $ pootle calculate_checks --no-rq

It is not generally safe to run commands in this mode if you have RQ workers active at the same time, as there is a risk that they conflict with other jobs dispatched to the workers.

--atomic¶

(env) $ pootle update_stores --atomic=all

--noinput¶

If there are RQ workers running, the command will ask for confirmation before proceeding. This can be overridden using the --noinput flag, in which case the command will run even if there are.

retry_failed_jobs¶

Requeue failed RQ jobs.

Background RQ jobs can fail for various reasons. To push them back into the queue you can run this command.

Examine the RQ worker logs for tracebacks before trying to requeue your jobs.

update_data¶

This command updates the stats data. The stats data update can be triggered for specific languages or projects.

--store¶

Use the --store option to narrow the stats data calculation to a specific store:

(env) $ pootle update_data --store=/fr/proj/mydir/mystore.po

Note this will also trigger the update of the stats data for items above the store, like for example directories above it, its language and its project.

calculate_checks¶

This command will create a background job to go through all units and recalculate quality checks.

calculate_checks will flush existing caches and update the quality checks cache.

It’s necessary to run this command after upgrading Pootle if new quality checks are added.

The time it takes to complete the whole process will vary depending on the number of units you have in the database. If a user hits a page that needs to display stats but they haven’t been calculated yet, then a message will be displayed indicating that the stats being calculated.

--check¶

Use the --check option to force calculation of a specified check. To recalculate only the date_format quality checks, run:

(env) $ pootle calculate_checks --check=date_format

Multiple checks can be specifed in one run as well:

(env) $ pootle calculate_checks --check=date_format --check=accelerators

flush_cache¶

Flush cache.

--django-cache¶

Use the --django-cache to flush the default cache which keeps Django templates, project permissions etc.

--rqdata¶

Use the --rqdata to flush all data contained in redis cache: pending jobs, revision (which will be automatically restored), all data from queues.

--lru¶

Use the --lru to flush all lru cache data contained in lru cache.

--all¶

Use the --all to flush all caches (default, redis, lru) data.

refresh_scores¶

Recalculates the scores for all users. It is possible to narrow down the calculation to specific projects and/or languages.

--reset¶

When the --reset option is used , all score log data is removed and zero score is set for all users.

sync_stores¶

Save all translations currently in the database to the file system, thereby bringing the files under the POOTLE_TRANSLATION_DIRECTORY directory in sync with the Pootle database.

You must run this command before taking backups or running scripts that modify the translation files directly on the file system, otherwise you might miss out on translations that are in the database but not yet saved to disk. In other words, translations are saved to disk only when you explicitly do so using this command.

For every file being synced, the in-DB Store will be updated to reflect the latest revision across the units in the file at the time of syncing. This allows Pootle to make optimizations when syncing and updating files, ignoring files that haven’t change.

The default behavior of sync_stores can be altered by specifying these parameters:

--force¶

Synchronizes files even if nothing changed in the database.

--overwrite¶

Copies the current state of the DB stores (not only translations, but also metadata) regardless if they have been modified since the last sync or not. This operation will (over)write existing on-disk files.

--skip-missing¶

Ignores files missing on disk, and no new files will be created.

update_stores¶

Load translation files currently on the file system into the database, thereby bringing the Pootle database in sync with the files under the POOTLE_TRANSLATION_DIRECTORY directory. Pootle will not detect changes in the file system on its own. This is the opposite of sync_stores.

It also discovers new units, files and translation projects that were added on disk:

• Projects that exist in the DB but ceased to exist on disk will be disabled (not deleted). If a project is recovered on disk it can be enabled via the admin UI only.
• Translation projects will be scanned for new files and directories. In-DB files and directories that no longer exist on disk will be marked as obsolete. Also any in-DB directory will be marked as obsolete if this directory is empty or contains empty directories only.
• In-DB stores will be updated with the contents of the on-disk files. New units will be added to the store, units that ceased to exist will be marked as obsolete. Translations that were updated on-disk will be reflected in the DB.

You must run this command after running scripts that modify translation files directly on the file system.

update_stores accepts several options:

--force¶

Updates in-DB translations even if the on-disk file hasn’t been changed since the last sync operation.

--overwrite¶

Mirrors the on-disk contents of the file. If there have been changes in the database since the last sync operation, these will be overwritten.

list_serializers¶

List the installed serializers and deserializers on your system.

Available options:

-m, --model¶

List serializers for specified model. The model should be expressed as a contenttype label - eg app_name.``model_name``

-d, --deserializers¶

List available deserializers set up for our system.

list_languages¶

Lists all the language codes for languages hosted on the server. This can be useful for automation.

--modified-since¶

Accepts the --modified-since parameter to list only those languages modified since the revision given by revision.

list_projects¶

Lists all the project codes on the server. This might can be useful for automation.

--modified-since¶

Accepts the --modified-since parameter to list only those projects modified since the revision given by revision.

contributors¶

Lists the contributors to a language, project or overall and the amount of contributions they have.

Available options:

--sort-by¶

Changed in version 2.8.0.

Specifies the sorting to be used. Valid options are contributions (sort by decreasing number of contributions) and username (sort by user name, alphabetically).

Default: username.

--mailmerge¶

New in version 2.8.0.

Specifies to only output user names and emails. Users with no email are skipped.

--mailmerge and --include-anonymous are mutually exclusive.

--include-anonymous¶

New in version 2.8.0.

Specifies to include anonymous contributions.

--include-anonymous and --mailmerge are mutually exclusive.

--since¶

New in version 2.8.0.

Only consider contributions since the specified date or datetime.

Date or datetime can be in any format accepted by python-dateutil library, for example ISO 8601 format (2016-01-24T23:15:22+0000 or 2016-01-24) or a string formatted like "2016-01-24 23:15:22 +0000" (quotes included).

--until¶

New in version 2.8.0.

Only consider contributions until the specified date or datetime.

Date or datetime can be in any format accepted by python-dateutil library, for example ISO 8601 format (2016-01-24T23:15:22+0000 or 2016-01-24) or a string formatted like "2016-01-24 23:15:22 +0000" (quotes included).

set_filetype¶

This command sets file formats for projects, and also allows to convert files to another format.

--from-filetype¶

Convert stores of this file type.

--matching¶

Convert stores matching this path glob within the project.

For example, to add the properties format to a project, run:

(env) $ pootle set_filetype --project=myproj properties

To convert stores of po format to properties, run:

(env) $ pootle set_filetype --project=myproj --from-filetype=po properties

To convert stores matching a given path glob to properties format, run:

(env) $ pootle set_filetype --project=myproj --matching=mydir/myfile-* properties

revision¶

Print the latest revision number.

The revision is a common system-wide counter for units. It is incremented with every translation action made from the browser. Zero length units that have been auto-translated also increment the unit revision.

--restore¶

The revision counter is stored in the database but also in cache for faster retrieval. If for some reason the revision counter was removed or got corrupted, passing the --restore flag to the command will restore the counter’s value based on the revision data available on the relational DB backend. You shouldn’t need to ever run this, but if for instance you deleted your cache you will need to restore the counter to ensure correct operation.

changed_languages¶

Produces a comma-separated list of language codes that changed since the last sync operation.

--after-revision¶

When --after-revision is specified with a revision number as an argument, it will print the language codes for languages that have changed since the specified revision.

test_checks¶

Tests any given string pair or unit against all or certain checks from the command line. This is useful for debugging and developing new checks.

--source, --target¶

String pairs can be specified by setting the values to be checked in the --source=<"source_text"> and --target="<target_text>" command-line arguments.

--unit¶

Alternatively, --unit=<unit_id> can be used to reference an existing unit from the database.

--check¶

By default, test_checks tests all existing checks. When --check=<checkname> is set, only specific checks will be tested against.

dump¶

Prints data or stats data (depending on --data or --stats option) in specific format.

--data¶

object_id:class_name
8276:Directory        name=android    parent=/uk/     pootle_path=/uk/android/
24394:Store   file=android/uk/strings.xml.po  translation_project=/uk/android/        pootle_path=/uk/android/strings.xml.po  name=strings.xml.pstate=2
806705:Unit   source=Create Account   target=Створити аккаунт source_wordcount=2      target_wordcount=2      developer_comment=create_account        translator_commentlocations=File:\nstrings.xml\nID:\ne82a8ea14a0b9f92b1b67ebfde2c16e9   isobsolete=False        isfuzzy=False   istranslated=True
115654:Suggestion     target_f=Необхідна електронна адреса    user_id=104481

--stats¶

pootle_path total,translated,fuzzy,suggestions,criticals,is_dirty,last_action_unit_id,last_updated_unit_id
/uk/android/strings.xml.po  11126,10597,383,231,0,False,4710214,4735242
/uk/android/widget/strings.xml.po  339,339,0,26,0,False,2277376,3738609
/uk/android/widget/  339,339,0,26,0,False,2277376,3738609
/uk/android/  11465,10936,383,257,0,False,4710214,4735242

This command can be used by developers to check if all data kept after migrations or stats calculating algorithm was changed.

config¶

Gets, sets, lists, appends and clears pootle configuration settings.

content_type¶

Optional positional argument to specify a model to manage configuration for.

object¶

Optional positional argument to specify the primary key of an object to manage configuration for. You can use a field other than the primary key by specifying -o, but the field must be unique for the request object when doing so.

-o <field>, --object-field <field>¶

Specify a field other than the primary key when specifying an object. It must be unique to the object specified.

-g <key>, --get <key>¶

Get value for specified key.

-l <key>, --list <key>¶

List values for specified key(s). This option can be specified multiple times.

-s <key> <value>, --set <key> <value>¶

Set value for specified key. The key must be unique or not exist already.

-a <key> <value>, --append <key> <value>¶

Append value for specified key.

-c <key>, --clear <key>¶

Clear value(s) for specified key.

-j, --json¶

Treat data as JSON when getting, setting, or appending values.

schema¶

Dumps a JSON representation for the Pootle database schema, currently only MySQL, for debugging and comparison to a reference database schema.

Updates the local server in POOTLE_TM_SERVER. The command reads translations from the current Pootle install and builds the TM resources in the TM server.

If no options are provided, the command will only add new translations to the server.

--refresh¶

Use --refresh to also update existing translations that have been changed, besides adding any new translation.

--rebuild¶

To completely remove the TM and rebuild it adding all existing translations use --rebuild.

--tm¶

If no specific TM server is specified using --tm, then the default local TM will be used. If the specified TM server doesn’t exist it will be automatically created for you.

--include-disabled-projects¶

By default translations from disabled projects are not added to the TM, but this can be changed by specifying --include-disabled-projects.

--dry-run¶

To see how many units will be loaded into the server use --dry-run, no actual data will be loaded or deleted (the TM will be left unchanged):

(env) $ pootle update_tmserver --dry-run
(env) $ pootle update_tmserver --refresh --dry-run
(env) $ pootle update_tmserver --rebuild --dry-run

This command also allows to read translations from files and build the TM resources in the external TM server. In order to do so it is mandatory to provide the --tm and --display-name options, along with some files to import.

--display-name¶

The display name is a label used to group translations within a TM. A given TM can host translations for several display names. The display name can be used to specify the name of the project from which the translations originate. The display name will be shown on TM matches in the translation editor. To specify a name use --display-name:

(env) $ pootle update_tmserver --tm=libreoffice --display-name="LibreOffice 4.3 UI" TM_LibreOffice_4.3.gl.tmx

By default the command will only add new translations to the server. To rebuild the server from scratch use --rebuild to completely remove the TM and rebuild it before importing the translations:

(env) $ pootle update_tmserver --rebuild --tm=mozilla --display-name="Foo 1.7" foo.po

Option --refresh doesn’t apply when adding translations from files on disk.

To see how many units will be loaded into the server use --dry-run, no actual data will be loaded:

(env) $ pootle update_tmserver --dry-run --tm=mozilla --display-name="Foo 1.7" foo.po
175045 translations to index

This command is capable of importing translations in multiple formats from several files and directories at once:

(env) $ pootle update_tmserver --tm=mozilla --display-name="Foo 1.7" bar.tmx foo.xliff fr/

--target-language¶

Use --target-language to specify the target language ISO code for the imported translations in case it is not possible to guess it from the translation files or if the code is incorrect:

(env) $ pootle update_tmserver --target-language=af --tm=mozilla --display-name="Foo 1.7" foo.po bar.tmx

Creates virtual folders from a JSON file. If the specified virtual folders already exist then they are updated.

The vfolder format defines how to specify a virtual folder that fits your needs.

This command requires a mandatory filename argument.

(env) $ pootle add_vfolders virtual_folders.json

Download a file for offline translation.

A file or a .zip of files is provided as output. The file headers include a revision counter to assist Pootle to detetmine how to handle subsequent uploads of the file.

Available options:

--tmx¶

New in version 2.8.0.

Export every translation project as one zipped TMX file into MEDIA_ROOT directory.

--rotate¶

New in version 2.8.0.

Remove old exported zipped TMX files (except previous one) from MEDIA_ROOT directory after current exported file is saved.

import¶

Upload a file that was altered offline.

A file or a .zip is submitted to Pootle and based on the revision counter of the Store on Pootle it will be uploaded or rejected. If the revision counter is older than on Pootle, that is someone has translated while the file was offline, then it will be rejected. Otherwise the translations in the file are accepted.

Available options:

--user¶

New in version 2.7.3.

Import file(s) as given user. The user with the provided username must exist.

Default: system.

Create the initial configuration for Pootle.

Available options:

--config¶

The configuration file to write to.¶

Default: ~/.pootle/pootle.conf.

--db¶

New in version 2.7.1.

The database backend that you are using

Default: sqlite. Available options: sqlite, mysql, postgresql.

--db-name¶

New in version 2.7.1.

The database name or path to database file if you are using sqlite.

Default for sqlite: dbs/pootle.db. Default for mysql/postgresql: pootledb.

--db-user¶

New in version 2.7.1.

Name of the database user. Not used with sqlite.

Default: pootle.

--db-host¶

New in version 2.7.1.

Database host to connect to. Not used with sqlite.

Default: localhost.

--db-port¶

New in version 2.7.1.

Port to connect to database on. Defaults to database backend’s default port. Not used with sqlite.

--dev¶

New in version 2.8.

Creates a development configuration instead.

initdb¶

Initializes a new Pootle install.

This is an optional part of Pootle’s install process, it creates the default admin user, populates the language table with several languages, initializes the terminology project, and creates the tutorial project among other tasks.

initdb can only be run after migrate.

initdb accepts the following option:

--no-projects¶

Don’t create the default terminology and tutorial projects.

Running the Django admin collectstatic command finds and extracts static content such as images, CSS and JavaScript files used by the Pootle server, so that they can be served separately from a static webserver. Typically, this is run with the --clear --noinput options, to flush any existing static data and use default answers for the content finders.

Pootle uses the Django app django-assets interface of webassets to minify and bundle CSS and JavaScript; this app has a management command that is used to make these preparations using the command assets build. This command is usually executed after the collectstatic one.

webpack¶

The webpack tool is used under the hood to bundle JavaScript scripts, and this management command is a convenient wrapper that sets everything up ready for production and makes sure to include any 3rd party customizations.

--dev¶

When the --dev flag is enabled, development builds will be created and the process will start a watchdog to track any client-side scripts for changes. Use this only when developing Pootle.

To interact with Pootle FS we use multiple subcommands:

• Admin:
  • info - Display filesystem info
  • state - Show current state
• Action:
  • fetch - Add a file from the filesystem to Pootle
  • add - Add a store from Pootle to the filesystem
  • merge - Handle conflicts in stores and files
  • rm - Remove a store and file from both Pootle and the filesystem
  • resolve - Revert a staged action
  • unstage - Revert a staged action
• Execute:
  • sync - Execute staged actions

Pootle FS action and execution subcommands take the -p and -P options which allow you to specify a glob to limit which files or stores are affected by the command.

-p --fs_path¶

Only affect files whose filesystem path matches a given glob.

(env) $ pootle fs add --fs_path MYPROJECT/af/directory/file.po MYPROJECT

Note

The path should be relative to the Pootle FS URL setting for the project.

-P --pootle_path¶

Only affect files whose Pootle path matches a given glob.

(env) $ pootle fs add --pootle_path /af/MYPROJECT/directory/file.po MYPROJECT

Note

Keep in mind that Pootle paths always start with /.

add¶

(env) $ pootle fs add MYPROJECT

(env) $ pootle fs fetch MYPROJECT

(env) $ pootle fs info MYPROJECT

(env) $ pootle fs merge MYPROJECT

(env) $ pootle fs rm MYPROJECT

(env) $ pootle fs state MYPROJECT

(env) $ pootle fs sync MYPROJECT

(env) $ pootle fs unstage MYPROJECT

As of Pootle version 2.8, it will no longer be possible to have users with duplicate emails. This command will find any user accounts that have duplicate emails. It also shows the last login time for each affected user and indicates if they are superusers of the site.

(env) $ pootle find_duplicate_emails

merge_user¶

This can be used if you have a user with two accounts and need to merge one account into another. This will re-assign all submissions, units and suggestions, but not any of the user’s profile data.

This command requires 2 mandatory arguments, src_username and target_username, both should be valid usernames for users of your site. Submissions from the first are re-assigned to the second. The users’ profile data is not merged.

--no-delete¶

By default src_username will be deleted after the contributions have been merged. You can prevent this by using the --no-delete option.

(env) $ pootle merge_user src_username target_username

purge_user¶

This command can be used if you wish to permanently remove a user and revert the edits, comments and reviews that the user has made. This is useful for removing a spam account or other malicious user.

This command requires a mandatory username argument, which should be a valid username for a user of your site.

(env) $ pootle purge_user username [username ...]

update_user_email¶

(env) $ pootle update_user_email username email

This command can be used if you wish to update a user’s email address. This might be useful if you have users with duplicate email addresses.

This command requires a mandatory username, which should be a valid username for a user of your site, and a mandatory valid email address.

(env) $ pootle update_user_email username email

verify_user¶

Verify a user without the user having to go through email verification process.

This is useful if you are migrating users that have already been verified, or if you want to create a superuser that can log in immediately.

This command requires either mandatory username arguments, which should be valid username(s) for user(s) on your site, or the --all flag if you wish to verify all users of your site.

(env) $ pootle verify_user username [username ...]

Available options:

--all¶

Verify all users of the site

With the new stats infrastructure this is not needed anymore.

clear_stats¶

With the new stats infrastructure this is not needed anymore.

last_change_id¶

With the change to revisions the command you will want to use is revision, though you are unlikely to know a specific revision number as you needed to in older versions of update_stores.

commit_to_vcs¶

Version Control support has been removed from Pootle and will reappear in a later release.

update_from_vcs¶

Version Control support has been removed from Pootle and will reappear in a later release.

run_cherrypy¶

Run the CherryPy server bundled with the Translate Toolkit.

start¶

Use runserver instead.

Run Pootle using the default Django server.

Sometimes failed jobs no longer apply since they refer to removed items, so no matter how many times you run them they will keep failing. Note that sometimes those unrecoverable failed jobs are in company of other failed jobs that can be re-run by using the retry_failed_jobs management command:

(env) $ pootle retry_failed_jobs

In order to delete all the failed jobs you must first stop the workers.

Once the workers are stopped make sure that there are no failed jobs that you don’t want to remove. In case there is any restart the workers to re-run them with retry_failed_jobs. Stop the workers again once those jobs are completed. Check again that all the failed jobs are the ones you want to remove.

In order to perform a bulk delete of all failed jobs run the following commands:

$ redis-cli -n 2 LRANGE "rq:queue:failed" 0 -1 | perl -nE 'chomp; `redis-cli DEL rq:job:$_`;'

Now remove the list of failed jobs:

$ redis-cli -n 2 DEL "rq:queue:failed"

Do not forget to restart the workers.