storage

Classes that represent various storage formats for localization.

base

Base classes for storage interfaces.

class translate.storage.base.DictStore(unitclass=None, encoding=None)
Extensions = None

A list of file extentions associated with this store type

Mimetypes = None

A list of MIME types associated with this store type

Name = 'Base translation store'

The human usable name of this store type

UnitClass

alias of TranslationUnit

add_unit_to_index(unit)

Add a unit to source and location idexes

addsourceunit(source)

Add and returns a new unit with the given source string.

Return type

TranslationUnit

addunit(unit)

Append the given unit to the object’s list of units.

This method should always be used rather than trying to modify the list manually.

Parameters

unit (TranslationUnit) – The unit that will be added.

detect_encoding(text: bytes, default_encodings: Optional[List[str]] = None) Tuple[str, str]

Try to detect a file encoding from text, using either the chardet lib or by trying to decode the file.

static fallback_detection(text)

Simple detection based on BOM in case chardet is not available.

findid(id)

find unit with matching id by checking id_index

findunit(source)

Find the unit with the given source string.

Return type

TranslationUnit or None

findunits(source)

Find the units with the given source string.

Return type

TranslationUnit or None

getids()

return a list of unit ids

getprojectstyle()

Get the project type for this store.

getsourcelanguage()

Get the source language for this store.

gettargetlanguage()

Get the target language for this store.

getunits()

Return a list of all units in this store.

isempty()

Return True if the object doesn’t contain any translation units.

makeindex()

Indexes the items in this store. At least .sourceindex should be useful.

property merge_on

The matching criterion to use when merging on.

Returns

The default matching criterion for all the subclasses.

Return type

string

parse(data)

parser to process the given source string

classmethod parsefile(storefile)

Reads the given file (or opens the given filename) and parses back to an object.

classmethod parsestring(storestring)

Convert the string representation back to an object.

remove_unit_from_index(unit)

Remove a unit from source and locaton indexes

removeunit(unit)

Remove the given unit to the object’s list of units.

This method should always be used rather than trying to modify the list manually.

Parameters

unit (TranslationUnit) – The unit that will be added.

require_index()

make sure source index exists

save()

Save to the file that data was originally read from, if available.

savefile(storefile)

Write the string representation to the given file (or filename).

serialize(out)

Converts to a bytes representation that can be parsed back using parsestring(). out should be an open file-like objects to write to.

setprojectstyle(project_style)

Set the project type for this store.

setsourcelanguage(sourcelanguage)

Set the source language for this store.

settargetlanguage(targetlanguage)

Set the target language for this store.

suggestions_in_format = False

Indicates if format can store suggestions and alternative translation for a unit

translate(source)

Return the translated string for a given source string.

Return type

String or None

unit_iter()

Iterator over all the units in this store.

class translate.storage.base.DictUnit(source=None)
adderror(errorname, errortext)

Adds an error message to this unit.

Parameters
  • errorname (string) – A single word to id the error.

  • errortext (string) – The text describing the error.

addlocation(location)

Add one location to the list of locations.

Note

Shouldn’t be implemented if the format doesn’t support it.

addlocations(location)

Add a location or a list of locations.

Note

Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation().

Warning

This method might be removed in future.

addnote(text, origin=None, position='append')

Adds a note (comment).

Parameters
  • text (string) – Usually just a sentence or two.

  • origin (string) – Specifies who/where the comment comes from. Origin can be one of the following text strings: - ‘translator’ - ‘developer’, ‘programmer’, ‘source code’ (synonyms)

classmethod buildfromunit(unit)

Build a native unit from a foreign unit, preserving as much information as possible.

static getcontext()

Get the message context.

static geterrors()

Get all error messages.

Return type

Dictionary

getid()

A unique identifier for this unit.

Return type

string

Returns

an identifier for this unit that is unique in the store

Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.

static getlocations()

A list of source code locations.

Return type

List

Note

Shouldn’t be implemented if the format doesn’t support it.

getnotes(origin=None)

Returns all notes about this unit.

It will probably be freeform text or something reasonable that can be synthesised by the format. It should not include location comments (see getlocations()).

gettargetlen()

Returns the length of the target string.

Return type

Integer

Note

Plural forms might be combined.

getunits()

This unit in a list.

getvalue()

Returns dictionary for serialization.

static hasplural()

Tells whether or not this specific unit has plural strings.

infer_state()

Empty method that should be overridden in sub-classes to infer the current state(_n) of the unit from its current state.

isblank()

Used to see if this unit has no source or target string.

Note

This is probably used more to find translatable units, and we might want to move in that direction rather and get rid of this.

static isfuzzy()

Indicates whether this unit is fuzzy.

static isheader()

Indicates whether this unit is a header.

static isobsolete()

indicate whether a unit is obsolete

static isreview()

Indicates whether this unit needs review.

istranslatable()

Indicates whether this unit can be translated.

This should be used to distinguish real units for translation from header, obsolete, binary or other blank units.

istranslated()

Indicates whether this unit is translated.

This should be used rather than deducing it from .target, to ensure that other classes can implement more functionality (as XLIFF does).

makeobsolete()

Make a unit obsolete

markfuzzy(value=True)

Marks the unit as fuzzy or not.

markreviewneeded(needsreview=True, explanation=None)

Marks the unit to indicate whether it needs review.

Parameters
  • needsreview – Defaults to True.

  • explanation – Adds an optional explanation as a note.

merge(otherunit, overwrite=False, comments=True, authoritative=False)

Do basic format agnostic merging.

multistring_to_rich(mulstring)

Convert a multistring to a list of “rich” string trees:

>>> target = multistring(['foo', 'bar', 'baz'])
>>> TranslationUnit.multistring_to_rich(target)
[<StringElem([<StringElem(['foo'])>])>,
 <StringElem([<StringElem(['bar'])>])>,
 <StringElem([<StringElem(['baz'])>])>]
removenotes(origin=None)

Remove all the translator’s notes.

rich_parsers = []

A list of functions to use for parsing a string into a rich string tree.

property rich_source
property rich_target
classmethod rich_to_multistring(elem_list)

Convert a “rich” string tree to a multistring:

>>> from translate.storage.placeables.interfaces import X
>>> rich = [StringElem(['foo', X(id='xxx', sub=[' ']), 'bar'])]
>>> TranslationUnit.rich_to_multistring(rich)
multistring('foo bar')
setcontext(context)

Set the message context

setid(value)

Sets the unique identified for this unit.

only implemented if format allows ids independant from other unit properties like source or context

unit_iter()

Iterator that only returns this unit.

exception translate.storage.base.ParseError(inner_exc)
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class translate.storage.base.TranslationStore(unitclass=None, encoding=None)

Base class for stores for multiple translation units of type UnitClass.

Extensions = None

A list of file extentions associated with this store type

Mimetypes = None

A list of MIME types associated with this store type

Name = 'Base translation store'

The human usable name of this store type

UnitClass

The class of units that will be instantiated and used by this class

alias of TranslationUnit

add_unit_to_index(unit)

Add a unit to source and location idexes

addsourceunit(source)

Add and returns a new unit with the given source string.

Return type

TranslationUnit

addunit(unit)

Append the given unit to the object’s list of units.

This method should always be used rather than trying to modify the list manually.

Parameters

unit (TranslationUnit) – The unit that will be added.

detect_encoding(text: bytes, default_encodings: Optional[List[str]] = None) Tuple[str, str]

Try to detect a file encoding from text, using either the chardet lib or by trying to decode the file.

static fallback_detection(text)

Simple detection based on BOM in case chardet is not available.

findid(id)

find unit with matching id by checking id_index

findunit(source)

Find the unit with the given source string.

Return type

TranslationUnit or None

findunits(source)

Find the units with the given source string.

Return type

TranslationUnit or None

getids()

return a list of unit ids

getprojectstyle()

Get the project type for this store.

getsourcelanguage()

Get the source language for this store.

gettargetlanguage()

Get the target language for this store.

getunits()

Return a list of all units in this store.

isempty()

Return True if the object doesn’t contain any translation units.

makeindex()

Indexes the items in this store. At least .sourceindex should be useful.

property merge_on

The matching criterion to use when merging on.

Returns

The default matching criterion for all the subclasses.

Return type

string

parse(data)

parser to process the given source string

classmethod parsefile(storefile)

Reads the given file (or opens the given filename) and parses back to an object.

classmethod parsestring(storestring)

Convert the string representation back to an object.

remove_unit_from_index(unit)

Remove a unit from source and locaton indexes

removeunit(unit)

Remove the given unit to the object’s list of units.

This method should always be used rather than trying to modify the list manually.

Parameters

unit (TranslationUnit) – The unit that will be added.

require_index()

make sure source index exists

save()

Save to the file that data was originally read from, if available.

savefile(storefile)

Write the string representation to the given file (or filename).

serialize(out)

Converts to a bytes representation that can be parsed back using parsestring(). out should be an open file-like objects to write to.

setprojectstyle(project_style)

Set the project type for this store.

setsourcelanguage(sourcelanguage)

Set the source language for this store.

settargetlanguage(targetlanguage)

Set the target language for this store.

suggestions_in_format = False

Indicates if format can store suggestions and alternative translation for a unit

translate(source)

Return the translated string for a given source string.

Return type

String or None

unit_iter()

Iterator over all the units in this store.

class translate.storage.base.TranslationUnit(source=None)

Base class for translation units.

Our concept of a translation unit is influenced heavily by XLIFF.

As such most of the method- and variable names borrows from XLIFF terminology.

A translation unit consists of the following:

  • A source string. This is the original translatable text.

  • A target string. This is the translation of the source.

  • Zero or more notes on the unit. Notes would typically be some comments from a translator on the unit, or some comments originating from the source code.

  • Zero or more locations. Locations indicate where in the original source code this unit came from.

  • Zero or more errors. Some tools (eg. pofilter) can run checks on translations and produce error messages.

adderror(errorname, errortext)

Adds an error message to this unit.

Parameters
  • errorname (string) – A single word to id the error.

  • errortext (string) – The text describing the error.

addlocation(location)

Add one location to the list of locations.

Note

Shouldn’t be implemented if the format doesn’t support it.

addlocations(location)

Add a location or a list of locations.

Note

Most classes shouldn’t need to implement this, but should rather implement TranslationUnit.addlocation().

Warning

This method might be removed in future.

addnote(text, origin=None, position='append')

Adds a note (comment).

Parameters
  • text (string) – Usually just a sentence or two.

  • origin (string) – Specifies who/where the comment comes from. Origin can be one of the following text strings: - ‘translator’ - ‘developer’, ‘programmer’, ‘source code’ (synonyms)

classmethod buildfromunit(unit)

Build a native unit from a foreign unit, preserving as much information as possible.

static getcontext()

Get the message context.

static geterrors()

Get all error messages.

Return type

Dictionary

getid()

A unique identifier for this unit.

Return type

string

Returns

an identifier for this unit that is unique in the store

Derived classes should override this in a way that guarantees a unique identifier for each unit in the store.