Testing¶
The pytest Suite¶
Real, automated, runs in CI on every push against the actively
supported Python versions - see .github/workflows/ci.yml’s
test job for the current matrix. Lives under
virtaal/ as test_*.py files next to the code they cover (e.g.
virtaal/test/test_storemodel.py,
virtaal/plugins/test_spellchecker.py), plus a couple of
demo_*.py files (virtaal/views/widgets/demo_textbox.py and
siblings) that are deliberately not picked up by pytest - those are
manual, interactive GTK demo runners predating the automated suite,
kept for hands-on widget debugging, not real coverage.
Besides the Building requirements, the suite uses these system
tools - see the test job in .github/workflows/ci.yml for the
exact Linux packages:
ps(procps or procps-ng)GNU gettext’s
msgfmtandmsgmerge, and libgettextpo - tests that need them are skipped without themEnchant, GtkSpell3 and an
en_UShunspell dictionary - the spell checking tests are skipped without themXvfb, on Linux without a real display (see below)
Run it the same way CI does:
pip install .[test,spellcheck]
pytest -rvxs virtaal
On Linux without a real display, wrap it in xvfb-run (CI does):
xvfb-run -a --server-args="-screen 0 1024x768x24" pytest -rvxs virtaal
Real translation files for manual testing live under
devsupport/testfiles/, one per format or scenario worth testing
directly:
checks.po- exercises most quality checks.msgfmt.po- one unit for each message of the msgfmt check, the message noted on it.plurals.po/plurals-zero.po- nplurals=3 and nplurals=1 respectively.workflow.po/workflow.xliff- between the two, covers five ofWorkflowMode’s six states.workflow.mo- a compiled Gettext catalog (pocompile), one translated unit; compilation itself drops untranslated/fuzzy entries.workflow.tmx,workflow.tbx,workflow.ts(Qt Linguist),workflow.qph(Qt Phrase Book),workflow.ftl(Fluent) - cover translate-toolkit’s other supported formats beyond Gettext PO.workflow.qm- compiled fromworkflow.tsvia Qt’s ownlrelease(not translate-toolkit - it can’t write.qm), so it’s real Qt output, not something translate-toolkit round-tripped itself. Includes anumerus="yes"plural message; regenerate withlrelease workflow.ts -qm workflow.qmafter editing the.ts.
Add more here as a format or scenario needs its own dedicated coverage.
Pseudo-Translation¶
Run against synthetic locales, covering every translatable string
without needing a real translation - each regenerates its locale
fresh from the current po/virtaal.pot on every run, so there’s no
separate generation step:
virtaal --pseudo-translation devsupport/testfiles/checks.po
virtaal --pseudo-translation-bidi devsupport/testfiles/checks.po
virtaal --pseudo-translation wraps every string in brackets
([Save]) - useful for spotting hardcoded strings and layout
truncation. virtaal --pseudo-translation-bidi wraps every
string in Unicode RTL isolate marks too, simulating a right-to-left
translation’s text runs while keeping the text itself readable Latin
script.
virtaal --pseudo-translation-source prefixes every string with
the catalog it comes from (vt:Save, gtk:_Open, glib:%.1f MB,
spell:Ignore All, mac:Quit %s, iso:German), showing which
visible strings come from GTK, GLib, gtkspell, the macOS menu
integration or the language and country name lists rather than
Virtaal, and which aren’t translatable at all. Library strings are
taken from their installed catalogs (macOS and Linux only):
virtaal --pseudo-translation-source devsupport/testfiles/checks.po
devsupport/pseudo-translation/generate_pseudo_translation.py also
writes a third, fa-tagged locale with every string’s glyphs
visually flipped (not a real Farsi translation - a real
RTL-recognised language code, for exercising actual whole-window RTL
mirroring under a genuine locale rather than just isolate-wrapped
text). Unlike the two above, this one has no dedicated
virtaal --lang shortcut that regenerates it automatically,
so run the script first:
python devsupport/pseudo-translation/generate_pseudo_translation.py
LANG=fa_IR.UTF-8 LANGUAGE=fa virtaal devsupport/testfiles/checks.po
See devsupport/testing/windows/Enable-VirtaalRtlDebug.ps1 for
installing this into a frozen build instead of a dev checkout.
Windows CI Checks¶
test-windows runs the pytest suite above under gvsbuild’s GTK3 build.
build-windows-installer goes further: it builds the real frozen
.exe (PyInstaller + Inno Setup, see Building) and drives
it directly, since a packaged build can hit bugs a dev checkout
never does (missing bundled data files, frozen-mode-only code paths).
devsupport/testing/windows/virtaal_ui_test_helpers.ps1 provides
the Win32 plumbing this needs - launch the real .exe, get a
window handle, read its geometry/title, send keystrokes, detect and
drive popup dialogs, clean up - so each check doesn’t reinvent P/Invoke
boilerplate. Two checks currently use it: a basic launch-and-verify
(open a real file, confirm the process stays alive, check the frozen
build’s log files for anything unexpected), and a regression check for
a real bug found this way (the main window growing wider on every
Enter-to-advance during translation - see
virtaal/views/widgets/storetreeview.py’s select_index()).
This is CI-only tooling - see below for a local, hands-on Windows test harness.
Local Windows Testing¶
CI’s checks above run headless against a raw PyInstaller bundle, not the actual installer/uninstaller a real user goes through - at least one real report (“the unsaved marker seems to persist between reinstalls”) was specifically about behaviour across that cycle, something CI’s bundle-only checks can’t reach at all. It’s also worth actually watching Virtaal run on a real Windows desktop rather than only trusting a headless pass.
devsupport/testing/windows/ has the pieces for this:
virtaal_install_helpers.ps1- installs/uninstalls Virtaal’s real Inno Setup package silently and idempotently.Invoke-VirtaalLocalTestPass.ps1- orchestrates a full pass: uninstall, install, a battery of UI regression checks against the real installed app, uninstall again, with a pass/fail summary and a process exit code.Enable-VirtaalRtlDebug.ps1- installs a synthetic right-to-left UI catalog for manually checking that the app’s whole chrome (menus, toolbar, status bar), not just editor text, mirrors correctly under an RTL language.
Run from the repo root, in PowerShell, on Windows:
.\devsupport\testing\windows\Invoke-VirtaalLocalTestPass.ps1
See devsupport/testing/windows/README.md for the full detail -
useful switches, what the battery covers, and using the helper
functions directly for a one-off manual check.
Getting a real Windows 11 desktop¶
On an Apple Silicon Mac, UTM (free) runs Windows 11 ARM64 natively via Apple’s own virtualization framework - fast, no CPU emulation involved.
Install UTM from mac.getutm.app (or the Mac App Store).
Get a Windows 11 ARM64 installer ISO. Easiest: install CrystalFetch (free, App Store) - it downloads the current official build directly from Microsoft. Alternatively, UTM’s own guide links a direct Windows 11 for Apple Silicon Macs ISO download if you’d rather not install another app. Either way, you need a real Windows license to activate it - a VM doesn’t get you around that.
In UTM: + -> Virtualize -> Windows -> pick RAM/CPU (4GB/2 cores minimum; more if your Mac has the headroom) -> Continue -> make sure “Install Windows 10 or higher” and “Install drivers and SPICE tools” are both checked -> Browse to the ISO -> Continue -> give it at least 64GB of disk -> Continue -> Save.
Boot it, press any key when prompted to boot from the ISO. Newer UTM versions handle Secure Boot/TPM automatically; if Windows Setup refuses with “This PC can’t run Windows 11,” see UTM’s troubleshooting section for the
LabConfigregistry bypass.To skip the Microsoft-account requirement and set up a local account instead: at the network-connection screen during Setup, press Shift+F10 for a command prompt and run
start ms-cxh:localonly(the olderOOBE\BYPASSNROtrick is now blocked by Microsoft on current builds).Known UTM gotcha on Windows 11 24H2: the installer can go to a black screen because of the bundled guest-tools graphics driver. If that happens, eject the guest-tools ISO from UTM’s CD menu, finish Windows Setup without it, then remount the tools ISO afterward and run “Install Windows Guest Tools” - you may need to reset the VM once more for the drivers to load cleanly.
Install the SPICE guest tools if they didn’t run automatically (needed for working networking - without them Windows may insist there’s no internet connection even once you’re on the desktop).
Match what test-windows in .github/workflows/ci.yml actually
installs inside the VM, rather than improvising a different setup:
Install Git for Windows and a current Python (matching what
windows-latestruns) from python.org - ARM64 builds are available directly.Install Visual Studio Build Tools with the “Desktop development with C++” workload - this gives you
cl.exe(needed to build PyGObject) and, usefully for debugging,cdb.exe/WinDbg underWindows Kits\10\Debuggers\.Download the same
gvsbuildGTK3 build CI uses (checkgvsbuild_versioninci.ymlfor the current pinned version) from wingtk/gvsbuild releases and extract toC:\gtk.Clone the repo, then in a Developer PowerShell for VS (so
cl.exeis onPATH), set the same environment variablesci.yml’s Windows job sets -PKG_CONFIG_PATH,GI_TYPELIB_PATH,INCLUDE/LIBpointing atC:\gtk, and addC:\gtk\bintoPATH- beforepip install .[test]. Read through thetest-windowsjob’s steps directly for the exact current values and ordering (some of them, likePKG_CONFIG_PATH, get silently overwritten by other tools if set in the wrong order - see that job’s own comments for why).
This is also the most useful environment for chasing a Windows-only
native crash - no 30-minute tmate session time limit, real WinDbg
instead of just cdb.exe’s command-line interface, and a real GUI
to actually watch what happens.