`linod`: The Lino daemon¶

The lino.modlib.linod plugin adds a framework for defining and handling scheduled background tasks such as sending notifications or verifying data.

The "d" stands for "daemon", like in sshd, cupsd, systemd and other background processes on a Linux system.

It defines the linod command, which is responsible for running the background procedures defined by your application. Background procedures are used for example for sending out notification emails or running maintenance tasks such as checkdata or checksummaries.

The linod command also contains the log server, which is used in production sites where multiple processes need to log to the lino.log.

When linod.use_channels is True, this plugin uses channels to provide an ASGI interface.

background task¶: A database row where the site manager specifies when to run a given background procedure on this site.
background procedure¶: A concrete job defined by an application for running in background.

Usage¶

Other plugins can register a background task using the dd.api.schedule_often() or dd.api.schedule_daily() decorators. For example (taken from lino.modlib.checkdata):

@dd.schedule_daily()
def checkdata(ar):
    """Run all data checkers."""
    check_data(fix=False)

The code for the example above should be in one of your application's models.py modules.

>>> import lino
>>> lino.startup('lino_book.projects.noi1e.settings.demo')
>>> from lino.api.doctest import *

Background tasks¶

class lino.modlib.linod.Procedure¶

Is basically a Job which contains a function func to run at default interval given by every_unit and every_value.

The default interval can be overridden by BackgroundTask.

func¶

The function to run as a system task.

Type:: Callable[[BaseRequest], None]

every_unit¶

The default unit of the interval at which the task func will run.

Type:: str

every_value¶

The default value of the interval at which the task func will run.

Type:: int

start_datetime¶

The time at which this task should run first.

Type:: datetime.datetime

run(self, ar)¶

Calls the function stored in func passing ar as a positional argument.

Parameters:: ar -- an instance of BaseRequest

class lino.modlib.linod.Procedures¶: The choicelist of background procedures available in this application.

class lino.modlib.linod.LogLevels¶

A choicelist of logging levels available in this application.

>>> rt.show(linod.LogLevels)
========== ========== ===============
 value      text       Numeric value
---------- ---------- ---------------
 DEBUG      DEBUG      10
 INFO       INFO       20
 WARNING    WARNING    30
 ERROR      ERROR      40
 CRITICAL   CRITICAL   50
========== ========== ===============

DEBUG means to include detailed debug messages. You should not set this for a longer period on a production site because it bloats the log files.

INFO means to show informative messages.

WARNING is the recommended value for most tasks. Only warnings and error messages are logged.

The levels ERROR and CRITICAL (log only errors and critical messages) exist only for exceptional situations. You should probably not use them.

class lino.modlib.linod.BackgroundTask¶

Django model used to represent a background task.

Overrides the recurrent rule of a Procedure.

A subclass of Sequenced and RecurrenceSet.

procedure¶: Pointer to an instance of Procedure.

start_datetime¶

Tells at what time exactly this job started.

Type:: datetime.datetime

message¶: Stores information about the job, mostly logs.

disabled¶

Tells whether the task should be ignored.

Lino sets this to True when the tasks fails and raises an exception. But it can also be checked by an end user in the web interface.

log_level¶

The logging level to apply when running this task.

See LogLevels.

run(self, ar, lgr=None) → Job¶

Performs a routine job.

Calls self.procedure.run.
Cancels the rule in case of a failure.
Creates an instance of Job

Parameters:

ar -- An instance of BaseRequest
lgr -- Logger obtained by calling logging.getLogger.

Returns:

An instance of Job.

class lino.modlib.linod.BackgroundTasks¶: The default actor for the BackgroundTask model.

class lino.modlib.linod.RunJob¶: Manually run this task immediately.

Background procedures¶

>>> rt.show(linod.Procedures)
============================== ============================== ============================== ================================
 value                          name                           text                           Suggested recurrency
------------------------------ ------------------------------ ------------------------------ --------------------------------
 event_notification_scheduler   event_notification_scheduler   event_notification_scheduler   every=300, every_unit=secondly
 generate_calendar_entries      generate_calendar_entries      generate_calendar_entries      every=1, every_unit=daily
 checksummaries                 checksummaries                 checksummaries                 every=1, every_unit=daily
 checkdata                      checkdata                      checkdata                      every=1, every_unit=daily
 send_pending_emails_often      send_pending_emails_often      send_pending_emails_often      every=10, every_unit=secondly
 send_pending_emails_daily      send_pending_emails_daily      send_pending_emails_daily      every=1, every_unit=daily
 clear_seen_messages            clear_seen_messages            clear_seen_messages            every=1, every_unit=daily
 update_publisher_pages         update_publisher_pages         update_publisher_pages         every=1, every_unit=daily
============================== ============================== ============================== ================================

While the procedures are in a choicelist (i.e. end users cannot edit them), the list of background tasks is configurable. The default situation is that every procedure has created one background task:

>>> rt.show(linod.BackgroundTasks) 
===== ============================== =============== ========== =============
 No.   Background procedure           Logging level   Disabled   Status
----- ------------------------------ --------------- ---------- -------------
 1     event_notification_scheduler   WARNING         No         Not started
 2     generate_calendar_entries      INFO            No         Not started
 3     checksummaries                 INFO            No         Not started
 4     checkdata                      INFO            No         Not started
 5     send_pending_emails_often      WARNING         No         Not started
 6     send_pending_emails_daily      INFO            No         Not started
 7     clear_seen_messages            INFO            No         Not started
 8     update_publisher_pages         INFO            No         Not started
===== ============================== =============== ========== =============

Simulating the daemon in a developer environment¶

To run the Lino daemon in a development environment run pm linod in a separate terminal.

Usage for developers¶

The lino.modlib.linod plugin requires a running redis-server in the background and also the django-channels, channels-redis Python packages to be installed.

To install redis on a Debian-based Linux distribution, run the following command as root:

$ apt update
$ apt install redis

To install the required Python packages, run the following command after activating your Python environment:

$ pm install

>>> list(dd.plugins.linod.get_requirements(settings.SITE))
['channels', 'channels_redis', 'daphne']

Now you simply go to your project directory and invoke the admin command:

$ cd ~/projects/mysite
$ pm linod

This process will run as long as you don't kill it, e.g. until you hit Ctrl-C.

Don't read this¶

>>> bt = linod.BackgroundTask.objects.get(procedure=linod.Procedures.update_publisher_pages)
>>> bt.status
'Not started'
>>> ar = rt.login("robin")
>>> from asgiref.sync import async_to_sync
>>> async_to_sync(bt.start_task)(ar)
Start Background task #8 update_publisher_pages...
Update published pages...
72 pages have been updated.
>>> bt.status  
'Scheduled to run at ... (... from now)'

Restore database state:

>>> bt.last_start_time = None
>>> bt.save()

linod: The Lino daemon¶