misc

Miscellaneous modules for translate - including modules for backward compatibility with pre-2.3 versions of Python.

dictutils

Implements a case-insensitive (on keys) dictionary and order-sensitive dictionary.

class translate.misc.dictutils.cidict(fromdict=None)
clear() None.  Remove all items from D.
copy() a shallow copy of D
fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

get(key, default=None)

Return the value for key if key is in the dictionary, else default.

items() a set-like object providing a view on D's items
keys() a set-like object providing a view on D's keys
pop(k[, d]) v, remove specified key and return the corresponding value.

If the key is not found, return the default if given; otherwise, raise a KeyError.

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

setdefault(key, default=None, /)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update(updatedict) None

Update from a dictionary.

D.update(E) -> None. Update D from E: for k in E.keys(): D[k] = E[k].

values() an object providing a view on D's values

file_discovery

translate.misc.file_discovery.get_abs_data_filename(path_parts, basedirs=None)

Get the absolute path to the given file- or directory name in the current running application’s data directory.

Parameters:

path_parts (list) – The path parts that can be joined by os.path.join().

multistring

Supports a hybrid Unicode string that can also have a list of alternate strings in the strings attribute.

class translate.misc.multistring.multistring(string='')
capitalize()

Return a capitalized version of the string.

More specifically, make the first character have upper case and the rest lower case.

casefold()

Return a version of the string suitable for caseless comparisons.

center(width, fillchar=' ', /)

Return a centered string of length width.

Padding is done using the specified fill character (default is a space).

count(sub[, start[, end]]) int

Return the number of non-overlapping occurrences of substring sub in string S[start:end]. Optional arguments start and end are interpreted as in slice notation.

encode(encoding='utf-8', errors='strict')

Encode the string using the codec registered for encoding.

encoding

The encoding in which to encode the string.

errors

The error handling scheme to use for encoding errors. The default is ‘strict’ meaning that encoding errors raise a UnicodeEncodeError. Other possible values are ‘ignore’, ‘replace’ and ‘xmlcharrefreplace’ as well as any other name registered with codecs.register_error that can handle UnicodeEncodeErrors.

endswith(suffix[, start[, end]]) bool

Return True if S ends with the specified suffix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. suffix can also be a tuple of strings to try.

expandtabs(tabsize=8)

Return a copy where all tab characters are expanded using spaces.

If tabsize is not given, a tab size of 8 characters is assumed.

find(sub[, start[, end]]) int

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

format(*args, **kwargs) str

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{’ and ‘}’).

format_map(mapping) str

Return a formatted version of S, using substitutions from mapping. The substitutions are identified by braces (‘{’ and ‘}’).

index(sub[, start[, end]]) int

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

isalnum()

Return True if the string is an alpha-numeric string, False otherwise.

A string is alpha-numeric if all characters in the string are alpha-numeric and there is at least one character in the string.

isalpha()

Return True if the string is an alphabetic string, False otherwise.

A string is alphabetic if all characters in the string are alphabetic and there is at least one character in the string.

isascii()

Return True if all characters in the string are ASCII, False otherwise.

ASCII characters have code points in the range U+0000-U+007F. Empty string is ASCII too.

isdecimal()

Return True if the string is a decimal string, False otherwise.

A string is a decimal string if all characters in the string are decimal and there is at least one character in the string.

isdigit()

Return True if the string is a digit string, False otherwise.

A string is a digit string if all characters in the string are digits and there is at least one character in the string.

isidentifier()

Return True if the string is a valid Python identifier, False otherwise.

Call keyword.iskeyword(s) to test whether string s is a reserved identifier, such as “def” or “class”.

islower()

Return True if the string is a lowercase string, False otherwise.

A string is lowercase if all cased characters in the string are lowercase and there is at least one cased character in the string.

isnumeric()

Return True if the string is a numeric string, False otherwise.

A string is numeric if all characters in the string are numeric and there is at least one character in the string.

isprintable()

Return True if the string is printable, False otherwise.

A string is printable if all of its characters are considered printable in repr() or if it is empty.

isspace()

Return True if the string is a whitespace string, False otherwise.

A string is whitespace if all characters in the string are whitespace and there is at least one character in the string.

istitle()

Return True if the string is a title-cased string, False otherwise.

In a title-cased string, upper- and title-case characters may only follow uncased characters and lowercase characters only cased ones.

isupper()

Return True if the string is an uppercase string, False otherwise.

A string is uppercase if all cased characters in the string are uppercase and there is at least one cased character in the string.

join(iterable, /)

Concatenate any number of strings.

The string whose method is called is inserted in between each given string. The result is returned as a new string.

Example: ‘.’.join([‘ab’, ‘pq’, ‘rs’]) -> ‘ab.pq.rs’

ljust(width, fillchar=' ', /)

Return a left-justified string of length width.

Padding is done using the specified fill character (default is a space).

lower()

Return a copy of the string converted to lowercase.

lstrip(chars=None, /)

Return a copy of the string with leading whitespace removed.

If chars is given and not None, remove characters in chars instead.

static maketrans()

Return a translation table usable for str.translate().

If there is only one argument, it must be a dictionary mapping Unicode ordinals (integers) or characters to Unicode ordinals, strings or None. Character keys will be then converted to ordinals. If there are two arguments, they must be strings of equal length, and in the resulting dictionary, each character in x will be mapped to the character at the same position in y. If there is a third argument, it must be a string, whose characters will be mapped to None in the result.

partition(sep, /)

Partition the string into three parts using the given separator.

This will search for the separator in the string. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing the original string and two empty strings.

removeprefix(prefix, /)

Return a str with the given prefix string removed if present.

If the string starts with the prefix string, return string[len(prefix):]. Otherwise, return a copy of the original string.

removesuffix(suffix, /)

Return a str with the given suffix string removed if present.

If the string ends with the suffix string and that suffix is not empty, return string[:-len(suffix)]. Otherwise, return a copy of the original string.

replace(old, new, count=None)

Return a copy with all occurrences of substring old replaced by new.

count

Maximum number of occurrences to replace. -1 (the default value) means replace all occurrences.

If the optional argument count is given, only the first count occurrences are replaced.

rfind(sub[, start[, end]]) int

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

rindex(sub[, start[, end]]) int

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

rjust(width, fillchar=' ', /)

Return a right-justified string of length width.

Padding is done using the specified fill character (default is a space).

rpartition(sep, /)

Partition the string into three parts using the given separator.

This will search for the separator in the string, starting at the end. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing two empty strings and the original string.

rsplit(sep=None, maxsplit=-1)

Return a list of the substrings in the string, using sep as the separator string.

sep

The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit

Maximum number of splits (starting from the left). -1 (the default value) means no limit.

Splitting starts at the end of the string and works to the front.

rstrip(chars=None, /)

Return a copy of the string with trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

split(sep=None, maxsplit=-1)

Return a list of the substrings in the string, using sep as the separator string.

sep

The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit

Maximum number of splits (starting from the left). -1 (the default value) means no limit.

Note, str.split() is mainly useful for data that has been intentionally delimited. With natural text that includes punctuation, consider using the regular expression module.

splitlines(keepends=False)

Return a list of the lines in the string, breaking at line boundaries.

Line breaks are not included in the resulting list unless keepends is given and true.

startswith(prefix[, start[, end]]) bool

Return True if S starts with the specified prefix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. prefix can also be a tuple of strings to try.

strip(chars=None, /)

Return a copy of the string with leading and trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

swapcase()

Convert uppercase characters to lowercase and lowercase characters to uppercase.

title()

Return a version of the string where each word is titlecased.

More specifically, words start with uppercased characters and all remaining cased characters have lower case.

translate(table, /)

Replace each character in the string using the given translation table.

table

Translation table, which must be a mapping of Unicode ordinals to Unicode ordinals, strings, or None.

The table must implement lookup/indexing via __getitem__, for instance a dictionary or list. If this operation raises LookupError, the character is left untouched. Characters mapped to None are deleted.

upper()

Return a copy of the string converted to uppercase.

zfill(width, /)

Pad a numeric string with zeros on the left, to fill a field of the given width.

The string is never truncated.

optrecurse

class translate.misc.optrecurse.ManHelpFormatter(indent_increment=0, max_help_position=0, width=80, short_first=1)
format_option_strings(option)

Return a comma-separated list of option strings & metavariables.

class translate.misc.optrecurse.ManPageOption(*opts, **attrs)
take_action(action, dest, opt, value, values, parser)

take_action that can handle manpage as well as standard actions.

class translate.misc.optrecurse.RecursiveOptionParser(formats, usetemplates=False, allowmissingtemplate=False, description=None)

A specialized Option Parser for recursing through directories.

add_option(Option)
add_option(opt_str, ..., kwarg=val, ...) None
check_values(values: Values, args: [string])

-> (values : Values, args : [string])

Check that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new – whatever you like). Default implementation just returns the passed-in values; subclasses may override as desired.

checkoutputsubdir(options, subdir)

Checks to see if subdir under options.output needs to be created, creates if neccessary.

define_option(option)

Defines the given option, replacing an existing one of the same short name if neccessary…

destroy()

Declare that you are done with this OptionParser. This cleans up reference cycles so the OptionParser (and all objects referenced by it) can be garbage-collected promptly. After calling destroy(), the OptionParser is unusable.

disable_interspersed_args()

Set parsing to stop on the first non-option. Use this if you have a command processor which runs another command that has options of its own and you want to make sure these options don’t get confused.

enable_interspersed_args()

Set parsing to not stop on the first non-option, allowing interspersing switches with command arguments. This is the default behavior. See also disable_interspersed_args() and the class documentation description of the attribute allow_interspersed_args.

error(msg: string)

Print a usage message incorporating ‘msg’ to stderr and exit. If you override this in a subclass, it should not return – it should either exit or raise an exception.

finalizetempoutputfile(options, outputfile, fulloutputpath)

Write the temp outputfile to its final destination.

format_manpage()

Returns a formatted manpage.

static getformathelp(formats)

Make a nice help string for describing formats…

static getfullinputpath(options, inputpath)

Gets the full path to an input file.

static getfulloutputpath(options, outputpath)

Gets the full path to an output file.

getfulltemplatepath(options, templatepath)

Gets the full path to a template file.

getoutputname(options, inputname, outputformat)

Gets an output filename based on the input filename.

getoutputoptions(options, inputpath, templatepath)

Works out which output format and processor method to use…

getpassthroughoptions(options)

Get the options required to pass to the filtermethod…

gettemplatename(options, inputname)

Gets an output filename based on the input filename.

static getusageman(option)

Returns the usage string for the given option.

static getusagestring(option)

Returns the usage string for the given option.

static isexcluded(options, inputpath)

Checks if this path has been excluded.

static isrecursive(fileoption, filepurpose='input')

Checks if fileoption is a recursive file.

isvalidinputname(inputname)

Checks if this is a valid input filename.

static mkdir(parent, subdir)

Makes a subdirectory (recursively if neccessary).

static openinputfile(options, fullinputpath)

Opens the input file.

static openoutputfile(options, fulloutputpath)

Opens the output file.

opentemplatefile(options, fulltemplatepath)

Opens the template file (if required).

static opentempoutputfile(options, fulloutputpath)

Opens a temporary output file.

parse_args(args=None, values=None)

Parses the command line options, handling implicit input/output args.

print_help(file: file = stdout)

Print an extended help message, listing all options and any help text provided with them, to ‘file’ (default stdout).

print_manpage(file=None)

Outputs a manpage for the program using the help information.

print_usage(file: file = stdout)

Print the usage message for the current program (self.usage) to ‘file’ (default stdout). Any occurrence of the string “%prog” in self.usage is replaced with the name of the current program (basename of sys.argv[0]). Does nothing if self.usage is empty or not defined.

print_version(file: file = stdout)

Print the version message for this program (self.version) to ‘file’ (default stdout). As with print_usage(), any occurrence of “%prog” in self.version is replaced by the current program’s name. Does nothing if self.version is empty or undefined.

processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)

Process an individual file.

recurseinputfilelist(options)

Use a list of files, and find a common base directory for them.

recurseinputfiles(options)

Recurse through directories and return files to be processed.

recursiveprocess(options)

Recurse through directories and process files.

run()

Parses the arguments, and runs recursiveprocess with the resulting options…

set_usage(usage=None)

sets the usage string - if usage not given, uses getusagestring for each option.

seterrorleveloptions()

Sets the errorlevel options.

setformats(formats, usetemplates)

Sets the format options using the given format dictionary.

Parameters:

formats (Dictionary or iterable) –

The dictionary keys should be:

  • Single strings (or 1-tuples) containing an input format (if not usetemplates)

  • Tuples containing an input format and template format (if usetemplates)

  • Formats can be None to indicate what to do with standard input

The dictionary values should be tuples of outputformat (string) and processor method.

setmanpageoption()

creates a manpage option that allows the optionparser to generate a manpage.

setprogressoptions()

Sets the progress options.

static splitext(pathname)

Splits pathname into name and ext, and removes the extsep.

Parameters:

pathname (string) – A file path

Returns:

root, ext

Return type:

tuple

splitinputext(inputpath)

Splits an inputpath into name and extension.

splittemplateext(templatepath)

Splits a templatepath into name and extension.

templateexists(options, templatepath)

Returns whether the given template exists…

warning(msg, options=None, exc_info=None)

Print a warning message incorporating ‘msg’ to stderr.

ourdom

module that provides modified DOM functionality for our needs.

Note that users of ourdom should ensure that no code might still use classes directly from minidom, like minidom.Element, minidom.Document or methods such as minidom.parseString, since the functionality provided here will not be in those objects.

class translate.misc.ourdom.Document
property documentElement

Top-level element of this document.

property firstChild

First child node, or None.

property lastChild

Last child node, or None.

property localName

Namespace-local name of this node.

class translate.misc.ourdom.Element(tagName, namespaceURI=None, prefix=None, localName=None)
property attributes

NamedNodeMap of attributes on the element.

property firstChild

First child node, or None.

getAttribute(attname)

Returns the value of the specified attribute.

Returns the value of the element’s attribute named attname as a string. An empty string is returned if the element does not have such an attribute. Note that an empty string may also be returned as an explicitly given attribute value, use the hasAttribute method to distinguish these two cases.

getElementsByTagName(name)

Returns all descendant elements with the given tag name.

Returns the list of all descendant elements (not direct children only) with the specified tag name.

hasAttribute(name)

Checks whether the element has an attribute with the specified name.

Returns True if the element has an attribute with the specified name. Otherwise, returns False.

property lastChild

Last child node, or None.

property localName

Namespace-local name of this element.

writexml(writer, indent, addindent, newl)

Write an XML element to a file-like object

Write the element to the writer object that must provide a write method (e.g. a file or StringIO object).

class translate.misc.ourdom.ExpatBuilderNS(options=None)
createParser()

Create a new namespace-handling parser.

getParser()

Return the parser object, creating a new one if needed.

install(parser)

Insert the namespace-handlers onto the parser.

parseFile(file)

Parse a document from a file object, returning the document node.

parseString(string)

Parse a document from a string, returning the document node.

reset()

Free all data structures used during DOM construction.

start_namespace_decl_handler(prefix, uri)

Push this namespace declaration on our storage.

translate.misc.ourdom.getElementsByTagName_helper(parent, name, dummy=None)

A reimplementation of getElementsByTagName as an iterator.

Note that this is not compatible with getElementsByTagName that returns a list, therefore, the class below exposes this through yieldElementsByTagName

translate.misc.ourdom.getnodetext(node)

Returns the node’s text by iterating through the child nodes.

translate.misc.ourdom.parse(file, parser=None, bufsize=None)

Parse a file into a DOM by filename or file object.

translate.misc.ourdom.parseString(string, parser=None)

Parse a file into a DOM from a string.

translate.misc.ourdom.searchElementsByTagName_helper(parent, name, onlysearch)

Limits the search to within tags occuring in onlysearch.

translate.misc.ourdom.writexml_helper(self, writer, indent='', addindent='', newl='')

A replacement for writexml that formats it like typical XML files. Nodes are intendented but text nodes, where whitespace can be significant, are not indented.

progressbar

Progress bar utilities for reporting feedback on the progress of an application.

class translate.misc.progressbar.DotsProgressBar

An ultra-simple progress indicator that just writes a dot for each action.

show(verbosemessage)

Show a dot for progress :-).

class translate.misc.progressbar.HashProgressBar(*args, **kwargs)

A ProgressBar which knows how to go back to the beginning of the line.

show(verbosemessage)

Displays the progress bar.

class translate.misc.progressbar.MessageProgressBar(*args, **kwargs)

A ProgressBar that just writes out the messages without any progress display.

show(verbosemessage)

Displays the progress bar.

class translate.misc.progressbar.NoProgressBar

An invisible indicator that does nothing.

show(verbosemessage)

Show nothing for progress :-).

class translate.misc.progressbar.ProgressBar(minValue=0, maxValue=100, totalWidth=50)

A plain progress bar that doesn’t know very much about output.

show(verbosemessage)

Displays the progress bar.

class translate.misc.progressbar.VerboseProgressBar(*args, **kwargs)
show(verbosemessage)

Displays the progress bar.

quote

String processing utilities for extracting strings with various kinds of delimiters.

translate.misc.quote.entitydecode(source: str, name2codepoint: dict[str, str]) str

Decode source using entities from name2codepoint.

Parameters:
  • source (unicode) – Source string to decode

  • name2codepoint (dict()) – Dictionary mapping entity names (without the the leading & or the trailing ;) to code points

translate.misc.quote.entityencode(source: str, codepoint2name: dict[str, str]) str

Encode source using entities from codepoint2name.

Parameters:
  • source (unicode) – Source string to encode

  • codepoint2name (dict()) – Dictionary mapping code points to entity names (without the the leading & or the trailing ;)

translate.misc.quote.escapecontrols(source: str) str

Escape control characters in the given string.

translate.misc.quote.extract(source, startdelim, enddelim, escape=None, startinstring=False, allowreentry=True)

Extracts a doublequote-delimited string from a string, allowing for backslash-escaping returns tuple of (quoted string with quotes, still in string at end).

translate.misc.quote.extractwithoutquotes(source, startdelim, enddelim, escape=None, startinstring=False, includeescapes=True, allowreentry=True)

Extracts a doublequote-delimited string from a string, allowing for backslash-escaping includeescapes can also be a function that takes the whole escaped string and returns the replaced version.

translate.misc.quote.find_all(searchin: str, substr: str) list[int]

Returns a list of locations where substr occurs in searchin locations are not allowed to overlap.

translate.misc.quote.htmlentitydecode(source: str) str

Decode source using HTML entities e.g. © -> ©.

Parameters:

source (unicode) – Source string to decode

translate.misc.quote.htmlentityencode(source)

Encode source using HTML entities e.g. © -> ©.

Parameters:

source (unicode) – Source string to encode

translate.misc.quote.java_utf8_properties_encode(source: str) str

Encodes source in the escaped-unicode encoding used by java utf-8 .properties files.

translate.misc.quote.javapropertiesencode(source: str) str

Encodes source in the escaped-unicode encoding used by Java .properties files.

translate.misc.quote.mozillaescapemarginspaces(source: str) str

Escape leading and trailing spaces for Mozilla .properties files.

translate.misc.quote.propertiesdecode(source: str) str

Decodes source from the escaped-unicode encoding used by .properties files.

Java uses Latin1 by default, and Mozilla uses UTF-8 by default.

Since the .decode(“unicode-escape”) routine decodes everything, and we don’t want to we reimplemented the algorithm from Python Objects/unicode.c in Python and modify it to retain escaped control characters.

wsgi

Wrapper to launch the bundled CherryPy server.

translate.misc.wsgi.launch_server(host, port, app, **kwargs)

Use cheroot WSGI server, a multithreaded scallable server.

xml_helpers

Helper functions for working with XML.

translate.misc.xml_helpers.clear_content(node)

Removes XML node content.

Unlike clear() this is not removing attributes.

translate.misc.xml_helpers.expand_closing_tags(elem)

Changes value of empty XML tags to empty string.

This changes lxml behavior to render these tags as <tag></tag> instead of <tag />

translate.misc.xml_helpers.getText(node, xml_space='preserve')

Extracts the plain text content out of the given node.

This method checks the xml:space attribute of the given node, and takes an optional default to use in case nothing is specified in this node.

translate.misc.xml_helpers.getXMLlang(node)

Gets the xml:lang attribute on node.

translate.misc.xml_helpers.getXMLspace(node, default=None)

Gets the xml:space attribute on node.

translate.misc.xml_helpers.namespaced(namespace, name)

Returns name in Clark notation within the given namespace.

For example namespaced(“source”) in an XLIFF document might return:

{urn:oasis:names:tc:xliff:document:1.1}source

This is needed throughout lxml.

translate.misc.xml_helpers.normalize_space(text: str)

Normalize the given text for implementation of xml:space="default".

translate.misc.xml_helpers.normalize_xml_space(node, xml_space: str, remove_start: bool = False)

normalize spaces following the nodes xml:space, or alternatively the given xml_space parameter.

translate.misc.xml_helpers.reindent(elem, level: int = 0, indent: str = '  ', max_level: int = 4, skip: set[str] | None = None, toplevel=True, leaves: set[str] | None = None, *, ignore_preserve: set[str] | None = None)

Adjust indentation to match specification.

Each nested tag is identified by indent string, up to max_level depth, possibly skipping tags listed in skip.

translate.misc.xml_helpers.safely_set_text(node, text: str) None

Safe updating of ElementTree text of a node.

In case of ValueError it strips any characters refused by lxml.

translate.misc.xml_helpers.setXMLlang(node, lang)

Sets the xml:lang attribute on node.

translate.misc.xml_helpers.setXMLspace(node, value)

Sets the xml:space attribute on node.

translate.misc.xml_helpers.string_xpath = string()

Return a non-normalized string in the node subtree

translate.misc.xml_helpers.string_xpath_normalized = normalize-space()

Return a (space) normalized string in the node subtree

translate.misc.xml_helpers.valid_chars_only(text: str) str

Prevent to crash libxml with unexpected chars.

translate.misc.xml_helpers.xml_preserve_ancestors = ancestor-or-self::*[attribute::xml:space='preserve']

All ancestors with xml:space=’preserve’

translate.misc.xml_helpers.xml_space_ancestors = ancestor-or-self::*/attribute::xml:space

All xml:space attributes in the ancestors