Welcome | Get started | Dive into Lino | Contribute | Topics | Reference | More

Run the Lino test suite

How to

Once your environment is correctly set up you can run the test suite for the Lino framework as follows:

$ pp inv prep test

The pp command loops over all your projects, initializes their demo databases (inv prep) and then runs the test suite (inv test. The whole process takes 20 minutes on my machine when there's no error. It produces a lot of output of this style:

==== lino ====
no previously-included directories found matching 'docs'
.
----------------------------------------------------------------------
Ran 1 test in 0.013s
OK

(...)

==== noi ====
--------------------------------------------------------------------------------
In demo project lino_noi.projects.team.settings.demo:
`initdb std demo demo2` started on database .../default.db.
Operations to perform:
...
Installed 148 object(s) from 11 fixture(s)
...................
----------------------------------------------------------------------
Ran 19 tests in 34.440s

OK

(...)

If any error occurs, then you need to find out the reason. And we recommend that you call for our help by reporting your error.

Possible reasons for failures are:

  • PodError: An error occurred during the conversion. Could not connect to LibreOffice on port 8100. UNO (LibreOffice API) says: Connector : couldn't connect to socket (Success).

    This means that you don't have the LibreOffice server running. See Running a LibreOffice server.

  • Some dependency is not installed on your machine.

  • Some test suite is currently broken (also on GitLab).

In order to check whether everything worked well, we are now going to run the test suite.

Make sure that your demo databases are initialized and that you did not do any manual changes therein. Because the test suite has many test cases which would fail if these demo databases were missing or not in their virgin state. In case you did write into some database during the previous section, just run inv prep once more.

Tips & tricks

  1. You can split the process into two:

    $ pp inv prep
$ pp inv test

    This can save you time if there is some problem. Once inv initdb has run successfully for all projects, you can focus on inv test.

  2. You can run the test suite for one project at a time by doing:

    $ go <prjname>
$ inv prep test

  3. You can prepend the standard Unix time command if you want to know how much time it took on your machine:

    $ time pp inv prep test

  4. You can use the -v option of pp so that you can peacefully go to the kitchen and make yourself a cup of tea, knowing that your computer will announce the result through its speakers when the process has finished:

    $ time pp -v inv prep test

    You can try this by issuing:

    $ pp -v ls

    Your computer should then say the words "Successfully terminated 'ls' for all projects" with a more or less clear male voice.

    Note that this requires the espeak package to be installed on your machine:

    $ sudo apt-get install espeak

Choosing the test runner

A repository can specify a test_command setting in its tasks.py file. The default value runs python -m unittest discover -s tests (unless there is no directory named tests, in which case it does nothing).

In the Developer Guide repository we use pytest instead of the default.

We do not use any pytest-specific features in order to remain testable with other test runners. The only reason for using pytest is that its reporting is more convenient: when a testcase fails, it shows the filename of the failure immediately, while unittest shows only an F instead of a dot (.).

We need pytest-forked (which adds the --forked option) because otherwise pytest would run all doctests in a single process. They must run in separate subprocesses because you cannot unload Django settings once they have been loaded.

We do not use pytest-xdist because each test must run in a subprocess, but we set --numprocesses to 1 because running multiple tests at the same time can cause side effects.