Management commands

The management commands are administration commands provided by Django, Pootle or any external Django app being used with Pootle. You will usually run these commands by issuing pootle <command> [options].

For example, to get information about all available management commands, you will run:

$ pootle help

Note

If you run Pootle from a repository checkout you can use the manage.py file found in the root of the repository. Note that this approach is deprecated and may not be supported in future.

Managing Pootle projects

These commands will go through all existing projects performing maintenance tasks. The tasks are all available through the web interface but on a project by project or file by file basis.

--project, --language

The commands target can be limited in a more flexible way using the --project --language command line options. They can be repeated to indicate multiple languages or projects. If you use both options together it will only match the files that match both languages and projects selected.

For example, to calculate_checks for the tutorial project only, run:

$ pootle calculate_checks --project=tutorial

To only calculate the Zulu and Basque language files within the tutorial project, run:

$ pootle calculate_checks --project=tutorial --language=zu --language=eu

Running commands with –no-rq option

--no-rq

New in version 2.7.1.

Some of the commands work asynchronously and will schedule jobs to RQ workers, rather than running them in the command process. You can change this behaviour using the --no-rq command line option.

This can be useful for running pootle commands in bash scripts or automating installation/upgrade/migration. It can also be useful for debugging otherwise asynchronous jobs.

For example, to run calculate_checks in the command process and wait for the process to terminate:

$ pootle calculate_checks --no-rq

It is not generally safe to run commands in this mode if you have RQ workers active at the same time, as there is a risk that they conflict with other jobs dispatched to the workers.

--noinput

If there are RQ workers running, the command will ask for confirmation before proceeding. This can be overridden using the --noinput flag, in which case the command will run even if there are.

retry_failed_jobs

retry_failed_jobs

New in version 2.7.

Requeue failed RQ jobs.

Background RQ jobs can fail for various reasons. To push them back into the queue you can run this command.

Examine the RQ worker logs for tracebacks before trying to requeue your jobs.

calculate_checks

calculate_checks

New in version 2.7.

This command will create a background job to go through all units and recalculate quality checks.

Note

Disabled projects are processed.

calculate_checks will flush existing caches and update the quality checks cache.

It’s necessary to run this command after upgrading Pootle if new quality checks are added.

The time it takes to complete the whole process will vary depending on the number of units you have in the database. If a user hits a page that needs to display stats but they haven’t been calculated yet, then a message will be displayed indicating that the stats being calculated.

--check

Use the --check option to force calculation of a specified check. To recalculate only the date_format quality checks, run:

$ pootle calculate_checks --check=date_format

Multiple checks can be specifed in one run as well:

$ pootle calculate_checks --check=date_format --check=accelerators
flush_cache

flush_cache

New in version 2.8.0.

Flush cache.

Warning

You must first stop the workers if you flush redis cache.

--django-cache

Use the --django-cache to flush the default cache which keeps Django templates, project permissions etc.

--rqdata

Use the --rqdata to flush all data contained in redis cache: pending jobs, revision (which will be automatically restored), all data from queues.

--lru

Use the --lru to flush all lru cache data contained in lru cache.

--all

Use the --all to flush all caches (default, redis, lru) data.

refresh_scores

refresh_scores

New in version 2.7.

Recalculates the scores for all users.

--reset

When the --reset option is used , all score log data is removed and zero score is set for all users.

sync_stores

sync_stores

Changed in version 2.7.

Save all translations currently in the database to the file system, thereby bringing the files under the POOTLE_TRANSLATION_DIRECTORY directory in sync with the Pootle database.

Note

Disabled projects are skipped.

You must run this command before taking backups or running scripts that modify the translation files directly on the file system, otherwise you might miss out on translations that are in the database but not yet saved to disk. In other words, translations are saved to disk only when you explicitly do so using this command.

For every file being synced, the in-DB Store will be updated to reflect the latest revision across the units in the file at the time of syncing. This allows Pootle to make optimizations when syncing and updating files, ignoring files that haven’t change.

The default behavior of sync_stores can be altered by specifying these parameters:

--force

Synchronizes files even if nothing changed in the database.

--overwrite

Copies the current state of the DB stores (not only translations, but also metadata) regardless if they have been modified since the last sync or not. This operation will (over)write existing on-disk files.

--skip-missing

Ignores files missing on disk, and no new files will be created.

update_stores

update_stores

Changed in version 2.7.

Load translation files currently on the file system into the database, thereby bringing the Pootle database in sync with the files under the POOTLE_TRANSLATION_DIRECTORY directory. Pootle will not detect changes in the file system on its own. This is the opposite of sync_stores.

Note

Disabled projects are skipped.

Note

update_stores does not manage the updating of translations against templates, it simply loads translation files and translation templates into Pootle. For a full understanding of the role of templates and updating translations against templates read the templates section.

It also discovers new units, files and translation projects that were added on disk:

  • Projects that exist in the DB but ceased to exist on disk will be disabled (not deleted). If a project is recovered on disk it can be enabled via the admin UI only.
  • Translation projects will be scanned for new files and directories. In-DB files and directories that no longer exist on disk will be marked as obsolete. Also any in-DB directory will be marked as obsolete if this directory is empty or contains empty directories only.
  • In-DB stores will be updated with the contents of the on-disk files. New units will be added to the store, units that ceased to exist will be marked as obsolete. Translations that were updated on-disk will be reflected in the DB.

You must run this command after running scripts that modify translation files directly on the file system.

update_stores accepts several options:

--force

Updates in-DB translations even if the on-disk file hasn’t been changed since the last sync operation.

--overwrite

Mirrors the on-disk contents of the file. If there have been changes in the database since the last sync operation, these will be overwritten.

Warning

If files on the file system are corrupt, translations might be deleted from the database. Handle with care!

list_serializers

list_serializers

New in version 2.8.0.

List the installed serializers and deserializers on your system.

Available options:

-m, --model

List serializers for specified model. The model should be expressed as a contenttype label - eg app_name.``model_name``

-d, --deserializers

List available deserializers set up for our system.

list_languages

list_languages

Lists all the language codes for languages hosted on the server. This can be useful for automation.

--modified-since

Accepts the --modified-since parameter to list only those languages modified since the revision given by revision.

list_projects

list_projects

Lists all the project codes on the server. This might can be useful for automation.

--modified-since

Accepts the --modified-since parameter to list only those projects modified since the revision given by revision.

contributors

contributors

New in version 2.7.1.

Lists the contributors to a language, project or overall and the amount of contributions they have.

Available options:

--sort-by

Changed in version 2.8.0.

Specifies the sorting to be used. Valid options are contributions (sort by decreasing number of contributions) and username (sort by user name, alphabetically).

Default: username.

--mailmerge

New in version 2.8.0.

Specifies to only output user names and emails. Users with no email are skipped.

--mailmerge and --include-anonymous are mutually exclusive.

--include-anonymous

New in version 2.8.0.

Specifies to include anonymous contributions.

--include-anonymous and --mailmerge are mutually exclusive.

--since

New in version 2.8.0.

Only consider contributions since the specified date or datetime.

Date or datetime can be in any format accepted by python-dateutil library, for example ISO 8601 format (2016-01-24T23:15:22+0000 or 2016-01-24) or a string formatted like "2016-01-24 23:15:22 +0000" (quotes included).

--until

New in version 2.8.0.

Only consider contributions until the specified date or datetime.

Date or datetime can be in any format accepted by python-dateutil library, for example ISO 8601 format (2016-01-24T23:15:22+0000 or 2016-01-24) or a string formatted like "2016-01-24 23:15:22 +0000" (quotes included).

revision

revision

New in version 2.7.

Print the latest revision number.

The revision is a common system-wide counter for units. It is incremented with every translation action made from the browser. Zero length units that have been auto-translated also increment the unit revision.

--restore

The revision counter is stored in the database but also in cache for faster retrieval. If for some reason the revision counter was removed or got corrupted, passing the --restore flag to the command will restore the counter’s value based on the revision data available on the relational DB backend. You shouldn’t need to ever run this, but if for instance you deleted your cache you will need to restore the counter to ensure correct operation.

changed_languages

changed_languages

New in version 2.7.

Produces a comma-separated list of language codes that changed since the last sync operation.

--after-revision

When --after-revision is specified with a revision number as an argument, it will print the language codes for languages that have changed since the specified revision.

test_checks

test_checks

New in version 2.7.

Tests any given string pair or unit against all or certain checks from the command line. This is useful for debugging and developing new checks.

--source, --target

String pairs can be specified by setting the values to be checked in the --source=<"source_text"> and --target="<target_text>" command-line arguments.

--unit

Alternatively, --unit=<unit_id> can be used to reference an existing unit from the database.

--check

By default, test_checks tests all existing checks. When --check=<checkname> is set, only specific checks will be tested against.

dump

dump

New in version 2.7.

Prints data or stats data (depending on --data or --stats option) in specific format.

--data
object_id:class_name
8276:Directory        name=android    parent=/uk/     pootle_path=/uk/android/
24394:Store   file=android/uk/strings.xml.po  translation_project=/uk/android/        pootle_path=/uk/android/strings.xml.po  name=strings.xml.pstate=2
806705:Unit   source=Create Account   target=Створити аккаунт source_wordcount=2      target_wordcount=2      developer_comment=create_account        translator_commentlocations=File:\nstrings.xml\nID:\ne82a8ea14a0b9f92b1b67ebfde2c16e9   isobsolete=False        isfuzzy=False   istranslated=True
115654:Suggestion     target_f=Необхідна електронна адреса    user_id=104481
--stats
pootle_path total,translated,fuzzy,suggestions,criticals,is_dirty,last_action_unit_id,last_updated_unit_id
/uk/android/strings.xml.po  11126,10597,383,231,0,False,4710214,4735242
/uk/android/widget/strings.xml.po  339,339,0,26,0,False,2277376,3738609
/uk/android/widget/  339,339,0,26,0,False,2277376,3738609
/uk/android/  11465,10936,383,257,0,False,4710214,4735242

This command can be used by developers to check if all data kept after migrations or stats calculating algorithm was changed.

config

config

New in version 2.8.

Gets, sets, lists, appends and clears pootle configuration settings.

content_type

Optional positional argument to specify a model to manage configuration for.

object

Optional positional argument to specify the primary key of an object to manage configuration for. You can use a field other than the primary key by specifying -o, but the field must be unique for the request object when doing so.

-o <field>, --object-field <field>

Specify a field other than the primary key when specifying an object. It must be unique to the object specified.

-g <key>, --get <key>

Get value for specified key.

-l <key>, --list <key>

List values for specified key(s). This option can be specified multiple times.

-s <key> <value>, --set <key> <value>

Set value for specified key. The key must be unique or not exist already.

-a <key> <value>, --append <key> <value>

Append value for specified key.

-c <key>, --clear <key>

Clear value(s) for specified key.

-j, --json

Treat data as JSON when getting, setting, or appending values.

Translation Memory

These commands allow you to setup and manage Translation Memory.

update_tmserver

update_tmserver

New in version 2.7.

Changed in version 2.7.3: Renamed --overwrite to --refresh. Disabled projects’ translations are no longer added by default. It is also possible to import translations from files.

Updates the local server in POOTLE_TM_SERVER. The command reads translations from the current Pootle install and builds the TM resources in the TM server.

If no options are provided, the command will only add new translations to the server.

--refresh

Use --refresh to also update existing translations that have been changed, besides adding any new translation.

--rebuild

To completely remove the TM and rebuild it adding all existing translations use --rebuild.

--tm

If no specific TM server is specified using --tm, then the default local TM will be used. If the specified TM server doesn’t exist it will be automatically created for you.

--include-disabled-projects

By default translations from disabled projects are not added to the TM, but this can be changed by specifying --include-disabled-projects.

--dry-run

To see how many units will be loaded into the server use --dry-run, no actual data will be loaded or deleted (the TM will be left unchanged):

$ pootle update_tmserver --dry-run
$ pootle update_tmserver --refresh --dry-run
$ pootle update_tmserver --rebuild --dry-run

This command also allows to read translations from files and build the TM resources in the external TM server. In order to do so it is mandatory to provide the --tm and --display-name options, along with some files to import.

--display-name

The display name is a label used to group translations within a TM. A given TM can host translations for several display names. The display name can be used to specify the name of the project from which the translations originate. The display name will be shown on TM matches in the translation editor. To specify a name use --display-name:

(env) $ pootle update_tmserver --tm=libreoffice --display-name="LibreOffice 4.3 UI" TM_LibreOffice_4.3.gl.tmx

By default the command will only add new translations to the server. To rebuild the server from scratch use --rebuild to completely remove the TM and rebuild it before importing the translations:

(env) $ pootle update_tmserver --rebuild --tm=mozilla --display-name="Foo 1.7" foo.po

Option --refresh doesn’t apply when adding translations from files on disk.

To see how many units will be loaded into the server use --dry-run, no actual data will be loaded:

(env) $ pootle update_tmserver --dry-run --tm=mozilla --display-name="Foo 1.7" foo.po
175045 translations to index

This command is capable of importing translations in multiple formats from several files and directories at once:

(env) $ pootle update_tmserver --tm=mozilla --display-name="Foo 1.7" bar.tmx foo.xliff fr/
--target-language

Use --target-language to specify the target language ISO code for the imported translations in case it is not possible to guess it from the translation files or if the code is incorrect:

(env) $ pootle update_tmserver --target-language=af --tm=mozilla --display-name="Foo 1.7" foo.po bar.tmx

Virtual Folders

These commands allow you to perform tasks with virtual folders from the command line.

add_vfolders

add_vfolders

New in version 2.7.

Creates virtual folders from a JSON file. If the specified virtual folders already exist then they are updated.

The vfolder format defines how to specify a virtual folder that fits your needs.

This command requires a mandatory filename argument.

$ pootle add_vfolders virtual_folders.json

Import and Export

Export and Import translation files in Pootle. The operation can be thought of best as offline operations to assist with offline translation, unlike sync_stores and update_stores the operations here are designed to cater for translators working outside of Pootle.

The import and export commands are designed to mimic the operations of Download and Upload from the Pootle UI.

export

export

New in version 2.7.

Download a file for offline translation.

Note

This mimics the editor’s download functionality and its primary purpose is to test the operation of downloads from the command line.

A file or a .zip of files is provided as output. The file headers include a revision counter to assist Pootle to detetmine how to handle subsequent uploads of the file.

import

import

New in version 2.7.

Upload a file that was altered offline.

Note

This mimics the editor’s upload functionality and its primary purpose is to test the operation of uploads from the command line.

A file or a .zip is submitted to Pootle and based on the revision counter of the Store on Pootle it will be uploaded or rejected. If the revision counter is older than on Pootle, that is someone has translated while the file was offline, then it will be rejected. Otherwise the translations in the file are accepted.

Available options:

--user

New in version 2.7.3.

Import file(s) as given user. The user with the provided username must exist.

Default: system.

Manually Installing Pootle

These commands expose the database installation and upgrade process from the command line.

init

init

Create the initial configuration for Pootle.

Available options:

--config
The configuration file to write to.

Default: ~/.pootle/pootle.conf.

--db

New in version 2.7.1.

The database backend that you are using

Default: sqlite. Available options: sqlite, mysql, postgresql.

--db-name

New in version 2.7.1.

The database name or path to database file if you are using sqlite.

Default for sqlite: dbs/pootle.db. Default for mysql/postgresql: pootledb.

--db-user

New in version 2.7.1.

Name of the database user. Not used with sqlite.

Default: pootle.

--db-host

New in version 2.7.1.

Database host to connect to. Not used with sqlite.

Default: localhost.

--db-port

New in version 2.7.1.

Port to connect to database on. Defaults to database backend’s default port. Not used with sqlite.

initdb

initdb

Initializes a new Pootle install.

This is an optional part of Pootle’s install process, it creates the default admin user, populates the language table with several languages, initializes the terminology project, and creates the tutorial project among other tasks.

initdb can only be run after migrate.

initdb accepts the following option:

New in version 2.7.3.

--no-projects

Don’t create the default terminology and tutorial projects.

Note

initdb will import translations into the database, so can be slow to run. You should have an rqworker running or run with the –no-rq.

collectstatic

Running the Django admin collectstatic command finds and extracts static content such as images, CSS and JavaScript files used by the Pootle server, so that they can be served separately from a static webserver. Typically, this is run with the --clear --noinput options, to flush any existing static data and use default answers for the content finders.

assets

Pootle uses the Django app django-assets interface of webassets to minify and bundle CSS and JavaScript; this app has a management command that is used to make these preparations using the command assets build. This command is usually executed after the collectstatic one.

webpack

webpack

New in version 2.7.

The webpack tool is used under the hood to bundle JavaScript scripts, and this management command is a convenient wrapper that sets everything up ready for production and makes sure to include any 3rd party customizations.

--dev

When the --dev flag is enabled, development builds will be created and the process will start a watchdog to track any client-side scripts for changes. Use this only when developing Pootle.

Pootle FS

fs

fs

To interact with Pootle FS we use multiple subcommands:

  • Admin:
    • info - Display filesystem info
    • state - Show current state
  • Action:
    • fetch - Add a file from the filesystem to Pootle
    • add - Add a store from Pootle to the filesystem
    • merge - Handle conflicts in stores and files
    • rm - Remove a store and file from both Pootle and the filesystem
    • unstage - Revert a staged action
  • Execute:
    • sync - Execute staged actions

Note

The action staging commands require that you run :djadmin:sync in order to actually perform the staged actions.

Common options

Pootle FS action and execution subcommands take the -p and -P options which allow you to specify a glob to limit which files or stores are affected by the command.

-p --fs_path

Only affect files whose filesystem path matches a given glob.

(env) $ pootle fs add --fs_path MYPROJECT/af/directory/file.po MYPROJECT

Note

The path should be relative to the Pootle FS URL setting for the project.

-P --pootle_path

Only affect files whose Pootle path matches a given glob.

(env) $ pootle fs add --pootle_path /af/MYPROJECT/directory/file.po MYPROJECT

Note

Keep in mind that Pootle paths always start with /.

Pootle FS subcommands

add

fs add

New in version 2.8.0.

Stage for adding any new or changed stores from Pootle to the filesystem:

(env) $ pootle fs add MYPROJECT

This command is the functional opposite of the fetch command.

--force

Conflicting files on the filesystem will be staged to be overwritten by the Pootle store.

(env) $ pootle fs add --force MYPROJECT
fetch

fs fetch

New in version 2.8.0.

Stage for fetching any new or changed files from the filesystem to Pootle:

(env) $ pootle fs fetch MYPROJECT

This command is the functional opposite of the add command.

--force

Conflicting stores in Pootle to be overwritten with the filesystem file.

(env) $ pootle fs fetch --force MYPROJECT
info

fs info

New in version 2.8.0.

Retrieve the filesystem info for a project.

(env) $ pootle fs info MYPROJECT
merge

fs merge

New in version 2.8.0.

Stage for merging any stores/files that have either been:

  1. Independently added both on the filesystem and on Pootle, or
  2. That have been updated both in Pootle and filesystem.

When merging, if there are conflicts in any specific translation unit the default behavior is to keep the filesystem version and convert the Pootle version into a suggestion.

Suggestions can then we reviewed by translators to ensure any corrections are correctly incorporated.

When there are no conflicts in unit merge will handle the merge without user input:

(env) $ pootle fs merge MYPROJECT
--pootle-wins

Alter the default conflict resolution of filesystem winning to instead use the Pootle version as the correct translation and converting the filesystem version into a suggestion.

(env) $ pootle fs merge --pootle-wins MYPROJECT
rm

fs rm

New in version 2.8.0.

Remove any matched:

  • Store that do not have a corresponding file in filesystem.
  • File that do not have a corresponding store in Pootle.
(env) $ pootle fs rm MYPROJECT
--force

Stage for removal conflicting/untracked files and/or stores.

(env) $ pootle fs rm --force MYPROJECT
state

fs state

New in version 2.8.0.

List the status of stores in Pootle and files on the filesystem.

(env) $ pootle fs state MYPROJECT
-t --type

Restrict to specified Pootle FS status.

(env) $ pootle fs state -t pootle_staged MYPROJECT
sync

fs sync

New in version 2.8.0.

Commit any staged changes, effectively synchronizing the filesystem and Pootle. This command is run after other Pootle FS commands have been used to stage changes.

(env) $ pootle fs sync MYPROJECT
unstage

fs unstage

New in version 2.8.0.

Unstage any staged Pootle FS actions. This allows you to remove any staged actions which you might have added erroneously.

(env) $ pootle fs unstage MYPROJECT

Managing users

find_duplicate_emails

find_duplicate_emails

New in version 2.7.1.

As of Pootle version 2.8, it will no longer be possible to have users with duplicate emails. This command will find any user accounts that have duplicate emails. It also shows the last login time for each affected user and indicates if they are superusers of the site.

$ pootle find_duplicate_emails
merge_user

merge_user

New in version 2.7.1.

This can be used if you have a user with two accounts and need to merge one account into another. This will re-assign all submissions, units and suggestions, but not any of the user’s profile data.

This command requires 2 mandatory arguments, src_username and target_username, both should be valid usernames for users of your site. Submissions from the first are re-assigned to the second. The users’ profile data is not merged.

--no-delete

By default src_username will be deleted after the contributions have been merged. You can prevent this by using the --no-delete option.

$ pootle merge_user src_username target_username
purge_user

purge_user

New in version 2.7.1.

This command can be used if you wish to permanently remove a user and revert the edits, comments and reviews that the user has made. This is useful for removing a spam account or other malicious user.

This command requires a mandatory username argument, which should be a valid username for a user of your site.

Changed in version 2.7.3: purge_user can accept multiple user accounts to purge.

$ pootle purge_user username [username ...]
update_user_email

update_user_email

New in version 2.7.1.

$ pootle update_user_email username email

This command can be used if you wish to update a user’s email address. This might be useful if you have users with duplicate email addresses.

This command requires a mandatory username, which should be a valid username for a user of your site, and a mandatory valid email address.

$ pootle update_user_email username email
verify_user

verify_user

New in version 2.7.1.

Verify a user without the user having to go through email verification process.

This is useful if you are migrating users that have already been verified, or if you want to create a superuser that can log in immediately.

This command requires either mandatory username arguments, which should be valid username(s) for user(s) on your site, or the --all flag if you wish to verify all users of your site.

Changed in version 2.7.3: verify_user can accept multiple user accounts to verify.

$ pootle verify_user username [username ...]

Available options:

--all

Verify all users of the site

Running WSGI servers

There are multiple ways to run Pootle, and some of them rely on running WSGI servers that can be reverse proxied to a proper HTTP web server such as nginx or lighttpd.

There are many more options such as uWSGI, Gunicorn, etc.

Deprecated commands

The following are commands that have been removed or deprecated:

refresh_stats

refresh_stats

Removed in version 2.8.

With the new stats infrastructure this is not needed anymore.

clear_stats

clear_stats

Removed in version 2.8.

With the new stats infrastructure this is not needed anymore.

last_change_id

last_change_id

Deprecated since version 2.7.

With the change to revisions the command you will want to use is revision, though you are unlikely to know a specific revision number as you needed to in older versions of update_stores.

commit_to_vcs

commit_to_vcs

Deprecated since version 2.7.

Version Control support has been removed from Pootle and will reappear in a later release.

update_from_vcs

update_from_vcs

Deprecated since version 2.7.

Version Control support has been removed from Pootle and will reappear in a later release.

run_cherrypy

run_cherrypy

Deprecated since version 2.7.3.

Run the CherryPy server bundled with the Translate Toolkit.

start

start

Removed in version 2.7.3.

Use runserver instead.

Run Pootle using the default Django server.

Running Commands in cron

If you want to schedule certain actions on your Pootle server, using management commands with cron might be a solution.

The management commands can perform certain batch commands which you might want to have executed periodically without user intervention.

For the full details on how to configure cron, read your platform documentation (for example man crontab). Here is an example that runs the calculate_checks command daily at 02:00 AM:

00 02 * * * www-data /var/www/sites/pootle/manage.py calculate_checks

Test your command with the parameters you want from the command line. Insert it in the cron table, and ensure that it is executed as the correct user (the same as your web server) like www-data, for example. The user executing the command is specified in the sixth column. Cron might report errors through local mail, but it might also be useful to look at the logs in /var/log/cron/, for example.

If you are running Pootle from a virtualenv, or if you set any custom PYTHONPATH or similar, you might need to run your management command from a bash script that creates the correct environment for your command to run from. Call this script then from cron. It shouldn’t be necessary to specify the settings file for Pootle — it should automatically be detected.