`linod`: The Lino daemon¶

This page documents the linod plugin for developers. We assume that you have read the end-user docs.

This plugin defines the linod admin command, which is responsible for running the background tasks and the socket-based log server.

The “d” stands for “daemon”, like in sshd, cupsd, systemd and other background processes on a Linux system.

When linod.use_channels is True, this plugin uses channels to provide an ASGI application, and the linod command then includes Channels’ runworker command.

This document contains code snippets (lines starting with >>>) that get tested as part of our development workflow.

>>> import lino
>>> lino.startup('lino_book.projects.noi1e.settings.demo')
>>> from lino.api.doctest import *

Usage ¶

Other plugins can register a background task using the dd.api.schedule_often() or dd.api.schedule_daily() decorators. For example (taken from lino.modlib.checkdata):

@dd.schedule_daily()
def checkdata(ar):
    """Run all data checkers."""
    check_data(fix=False)

The code for the example above should be in your plugin’s models.py modules.

Plugin configuration ¶

linod.use_channels¶: Whether to use channels and daphne to run in asynchronous mode.

linod.background_sleep_time¶: How many seconds the background task runner should sleep when there is nothing to do.

Background procedures ¶

>>> rt.show(linod.Procedures)  
============================== ============================== ============================== ================== ================================
 value                          name                           text                           Task class         Suggested recurrency
------------------------------ ------------------------------ ------------------------------ ------------------ --------------------------------
 event_notification_scheduler   event_notification_scheduler   event_notification_scheduler   linod.SystemTask   every=300, every_unit=secondly
 generate_calendar_entries      generate_calendar_entries      generate_calendar_entries      linod.SystemTask   every=1, every_unit=daily
 delete_older_changes           delete_older_changes           delete_older_changes           linod.SystemTask   every=1, every_unit=daily
 checksummaries                 checksummaries                 checksummaries                 linod.SystemTask   every=1, every_unit=daily
 checkdata                      checkdata                      checkdata                      linod.SystemTask   every=1, every_unit=daily
 send_pending_emails_often      send_pending_emails_often      send_pending_emails_often      linod.SystemTask   every=10, every_unit=secondly
 send_pending_emails_daily      send_pending_emails_daily      send_pending_emails_daily      linod.SystemTask   every=1, every_unit=daily
 clear_seen_messages            clear_seen_messages            clear_seen_messages            linod.SystemTask   every=1, every_unit=daily
 run_invoicing_tasks            run_invoicing_tasks            run_invoicing_tasks            invoicing.Task     every=1, every_unit=daily
============================== ============================== ============================== ================== ================================

While the procedures are in a choicelist (i.e. end users cannot edit them), the list of system tasks is configurable. The default situation is that every procedure has created one system task:

Logging levels ¶

>>> rt.show(linod.LogLevels)
========== ========== ===============
 value      text       Numeric value
---------- ---------- ---------------
 DEBUG      DEBUG      10
 INFO       INFO       20
 WARNING    WARNING    30
 ERROR      ERROR      40
 CRITICAL   CRITICAL   50
========== ========== ===============

DEBUG means to include detailed debug messages. You should not set this for a longer period on a production site because it bloats the log files.

INFO means to show informative messages.

WARNING is the recommended value for most tasks. Only warnings and error messages are logged.

The levels ERROR and CRITICAL (log only errors and critical messages) exist only for exceptional situations. You should probably not use them.

System tasks ¶

>>> rt.show(linod.SystemTasks) 
===== ============================== =============== ========== ======================= ==============================
 No.   Name                           Logging level   Disabled   Status                  Background procedure
----- ------------------------------ --------------- ---------- ----------------------- ------------------------------
 1     event_notification_scheduler   WARNING         No         Scheduled to run asap   event_notification_scheduler
 2     generate_calendar_entries      INFO            No         Scheduled to run asap   generate_calendar_entries
 3     delete_older_changes           INFO            No         Scheduled to run asap   delete_older_changes
 4     checksummaries                 INFO            No         Scheduled to run asap   checksummaries
 5     checkdata                      INFO            No         Scheduled to run asap   checkdata
 6     send_pending_emails_often      WARNING         No         Scheduled to run asap   send_pending_emails_often
 7     send_pending_emails_daily      INFO            No         Scheduled to run asap   send_pending_emails_daily
 8     clear_seen_messages            INFO            No         Scheduled to run asap   clear_seen_messages
===== ============================== =============== ========== ======================= ==============================

>>> rt.show(invoicing.Task) 
===== ============ ============================= ===================================== =============== ========== =========== =======================
 No.   Author       Target journal                Invoice generators                    Logging level   Disabled   When        Status
----- ------------ ----------------------------- ------------------------------------- --------------- ---------- ----------- -----------------------
 1     Robin Rood   Service reports (SRV)         working.Session                       INFO            No         Every day   Scheduled to run asap
 2     Robin Rood   Sales invoices (SLS)          storage.Filler, trading.InvoiceItem   INFO            No         Every day   Scheduled to run asap
 3     Robin Rood   Subscription invoices (SUB)   subscriptions.SubscriptionPeriod      INFO            No         Every day   Scheduled to run asap
===== ============ ============================= ===================================== =============== ========== =========== =======================

Dependencies ¶

When linod.use_channels is True, the lino.modlib.linod plugin requires the django-channels and channels-redis Python packages to be installed, as well as a running redis-server.

To install redis on a Debian-based Linux distribution, run the following command as root:

$ apt update
$ apt install redis

To install the required Python packages, run the following command after activating your Python environment:

$ pm install

>>> list(dd.plugins.linod.get_requirements(settings.SITE))
['channels', 'channels_redis', 'daphne']

Usage for developers ¶

To run the Lino daemon in a development environment, run pm linod in your project directory:

$ cd ~/lino/lino_local/mysite
$ pm linod

This process will run as long as you don’t kill it, e.g. until you hit Ctrl-C.

Another way to kill the linod process is using the kill command:

$ kill -s SIGTERM 123456

If you kill linod with another signal than SIGTERM, Lino will not run it shutdown method, which is responsible e.g. for removing the socket file of the log server.

You may change the logging level by setting LINO_LOGLEVEL:

$ LINO_LOGLEVEL=debug pm linod

Testing instructions for developers ¶

General remarks:

The following demo projects are useful when testing linod:
- cms1 has some background tasks, but linod.use_channels set to False
- noi1r has linod.use_channels set to True.
- chatter : an instant messaging system has no background tasks, but it has linod.use_channels set to True.
linod.use_channels changes the way pm linod works.
For testing the log server you need to create a log directory, and don’t forget to remove it after your tests because a log directory causes different output for certain commands and would cause the unit test suite to fail if you forget to delete it.
When you start pm runserver before pm linod, runserver will write directly to the lino.log file because there is no socket file. Two processes writing to the same file is likely to cause unpredictable results.
You can set Site.log_each_action_request to True

Example testing session 1

In terminal 1:

go noi1e
mkdir settings/log
LINO_LOGLEVEL=debug pm linod

In terminal 2:

go noi1r
pm runserver

In your browser: sign in as robin, go to Configure ‣ System ‣ System tasks, click “Run now” on one of them. The linod process in terminal 1 should run the task.

In terminal 1, hit Ctrl-C to stop the linod. Then do something in the browser and verify that runserver no longer writes to the lino.log. That’s normal because the runserver process believes that a socket server is running. Now restart the linod process and verify that runserver is again being logged. The socket file did not exist for some time and now it’s a new socket file, but this doesn’t disturb logging.

In terminal 1:

go noi1e
rm -rf settings/log

If you remove the log directory before stopping the linod, you will get the following exception when linod stops:

FileNotFoundError: [Errno 2] No such file or directory: '.../noi1e/settings/log/lino.log'

Example testing session 2

In terminal 1:

go cms1
LINO_LOGLEVEL=debug pm linod

Expected output:

actors.discover() : registering 135 actors
actors.initialize()
Analyzing Tables...
Analyze 22 slave tables...
Discovering choosers for database fields...
No log server because there is no directory .../lino_book/projects/cms1/log.
Start task runner using <Logger lino (DEBUG)>...
Start next task runner loop.
Too early to start System task #1 (update_publisher_pages)
Too early to start System task #2 (checkdata)
Let task runner sleep for 4.996284 seconds.
...
(etc until you hit Ctrl-C)

Class reference ¶

class lino.modlib.linod.Procedure¶

A callable function designed to run in background at default interval given by every_unit and every_value.

The default interval can be overridden by SystemTask.

func¶

The function to run as a system task.

Type:: Callable[[BaseRequest], None]

every_unit¶

The default unit of the interval at which the task func will run.

Type:: str

every_value¶

The default value of the interval at which the task func will run.

Type:: int

start_datetime¶

The time at which this task should run first.

Type:: datetime.datetime

run(self, ar)¶

Calls the function stored in func passing ar as a positional argument.

Parameters:: ar – an instance of BaseRequest

class lino.modlib.linod.Procedures¶: The choicelist of background procedures available in this application.

class lino.modlib.linod.LogLevels¶

A choicelist of logging levels available in this application.

See Logging levels

class lino.modlib.linod.SystemTask¶

Django model used to represent a background task.

Overrides the recurrent rule of a Procedure.

A subclass of Sequenced and RecurrenceSet.

procedure¶: Pointer to an instance of Procedure.

start_datetime¶

Tells at what time exactly this job started.

Type:: datetime.datetime

message¶: Stores information about the job, mostly logs.

disabled¶

Tells whether the task should be ignored.

Lino sets this to True when the tasks fails and raises an exception. But it can also be checked by an end user in the web interface.

log_level¶

The logging level to apply when running this task.

See LogLevels.

run(self, ar, lgr=None) → Job¶

Performs a routine job.

Calls self.procedure.run.
Cancels the rule in case of a failure.
Creates an instance of Job

Parameters:

ar – An instance of BaseRequest
lgr – Logger obtained by calling logging.getLogger.

Returns:

An instance of Job.

class lino.modlib.linod.SystemTasks¶: The default actor for the SystemTask model.

linod: The Lino daemon¶

`linod`: The Lino daemon¶