search : Add search functionality

The lino.modlib.search plugin adds functionality for doing site-wide searches across all tables of the application.

This plugin can optionally use an ElasticSearch server to have faster and intelligent search functionalities.

Table of contents:

This is a tested document. The following instructions are used for initialization:

>>> import lino
>>> lino.startup('lino_book.projects.noi1r.settings')
>>> from lino.api.doctest import *

Configuration settings

use_elasticsearch

Whether to use ElasticSearch.

This is a site attribute.

search.elasticsearch_url

The URL of the ElasticSearch server.

This is a plugin attribute.

The SiteSearch table

class lino.modlib.search.SiteSearch

The virtual table used to implement site-wide searches using default Django ORM filtering.

>>> rt.show('search.SiteSearch', quick_search="foo", limit=5)
+--------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Description              | Matches                                                                                                                                                                                                                                                                                                                                                                                                                |
+==========================+========================================================================================================================================================================================================================================================================================================================================================================================================================+
| *Comment #114* (Comment) | body:<table><thead>                                                                                                                                                                                                                                                                                                                                                                                                    |
|                          | <tr><td>Who</td><td>What</td><td>Done?</td></tr></thead><tbody><tr><td>Him</td><td>Bar</td><td>&nbsp;</td></tr><tr><td>Her</td><td>**Foo** the Bar</td><td><strong>x</strong></td></tr><tr><td>Them</td><td><span>Floop the pig<br></span></td><td>x</td></tr></tbody></table>, body_short_preview:<table><thead>                                                                                                      |
|                          | <tr><td>Who</td><td>What</td><td>Done?</td></tr></thead><tbody><tr><td>Him</td><td>Bar</td><td> </td></tr><tr><td>Her</td><td>**Foo** the Bar</td><td><strong>x</strong></td></tr><tr><td>Them</td><td><span>Floop the pig<br/></span></td><td>x</td></tr></tbody></table>, body_full_preview:<table><thead>                                                                                                           |
|                          | <tr><td>Who</td><td>What</td><td>Done?</td></tr></thead><tbody><tr><td>Him</td><td>Bar</td><td> </td></tr><tr><td>Her</td><td>**Foo** the Bar</td><td><strong>x</strong></td></tr><tr><td>Them</td><td><span>Floop the pig<br/></span></td><td>x</td></tr></tbody></table>                                                                                                                                             |
+--------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| *Comment #117* (Comment) | body:<p>breaking </p>                                                                                                                                                                                                                                                                                                                                                                                                  |
|                          | <p></p>                                                                                                                                                                                                                                                                                                                                                                                                                |
|                          | <p><strong><span>De&nbsp;:</span></strong><span> <a href="mailto:lino@**foo**.net">lino@**foo**.net</a> [<a href="mailto:**foo**@bar.com">mailto:**foo**@bar.com</a>] <br> <strong>Envoy&eacute;&nbsp;:</strong> mardi 18 octobre 2016 08:52<br> <strong>&Agrave;&nbsp;:</strong> <a href="mailto:Far@baz.net">eexample@**foo**.com</a><br> <strong>Objet&nbsp;:</strong> [welcht] YOU modified **FOO** BAR</span></p> |
|                          | <p>&nbsp;</p>                                                                                                                                                                                                                                                                                                                                                                                                          |
|                          | <p>Dear Aur&eacute;lie ,</p>                                                                                                                                                                                                                                                                                                                                                                                           |
|                          | <p>this is to notify  / BAR</p>                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | <p>BAR modified </p>                                                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>TODO: include a summary of the modifications.</p>                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>Any subsequent notifications about **foo**/  until you view this notification in the Lino web interface. Please visit</p>                                                                                                                                                                                                                                                                                           |
|                          | <p><a href="None">None</a></p>                                                                                                                                                                                                                                                                                                                                                                                         |
|                          | <p>and follow your welcome messages.</p>, body_short_preview:breaking                                                                                                                                                                                                                                                                                                                                                  |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | De : lino@**foo**.net [mailto:**foo**@bar.com]  Envoyé : mardi 18 octobre 2016 08:52 À : eexample@**foo**.com Objet : [welcht] YOU modified **FOO** BAR                                                                                                                                                                                                                                                                |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | Dear Aurélie ,                                                                                                                                                                                                                                                                                                                                                                                                         |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | this is to notify  / BAR                                                                                                                                                                                                                                                                                                                                                                                               |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | BAR modified                                                                                                                                                                                                                                                                                                                                                                                                           |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | TODO: include a summary of the modifications.                                                                                                                                                                                                                                                                                                                                                                          |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | Any subsequent notifications about **foo**/  un..., body_full_preview:<p>breaking </p>                                                                                                                                                                                                                                                                                                                                 |
|                          | <p></p>                                                                                                                                                                                                                                                                                                                                                                                                                |
|                          | <p><strong><span>De :</span></strong><span> <a href="mailto:lino@**foo**.net">lino@**foo**.net</a> [<a href="mailto:**foo**@bar.com">mailto:**foo**@bar.com</a>] <br/> <strong>Envoyé :</strong> mardi 18 octobre 2016 08:52<br/> <strong>À :</strong> <a href="mailto:Far@baz.net">eexample@**foo**.com</a><br/> <strong>Objet :</strong> [welcht] YOU modified **FOO** BAR</span></p>                                |
|                          | <p> </p>                                                                                                                                                                                                                                                                                                                                                                                                               |
|                          | <p>Dear Aurélie ,</p>                                                                                                                                                                                                                                                                                                                                                                                                  |
|                          | <p>this is to notify  / BAR</p>                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | <p>BAR modified </p>                                                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>TODO: include a summary of the modifications.</p>                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>Any subsequent notifications about **foo**/  until you view this notification in the Lino web interface. Please visit</p>                                                                                                                                                                                                                                                                                           |
|                          | <p><a href="None">None</a></p>                                                                                                                                                                                                                                                                                                                                                                                         |
|                          | <p>and follow your welcome messages.</p>                                                                                                                                                                                                                                                                                                                                                                               |
+--------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| *Comment #130* (Comment) | body:<table><thead>                                                                                                                                                                                                                                                                                                                                                                                                    |
|                          | <tr><td>Who</td><td>What</td><td>Done?</td></tr></thead><tbody><tr><td>Him</td><td>Bar</td><td>&nbsp;</td></tr><tr><td>Her</td><td>**Foo** the Bar</td><td><strong>x</strong></td></tr><tr><td>Them</td><td><span>Floop the pig<br></span></td><td>x</td></tr></tbody></table>, body_short_preview:<table><thead>                                                                                                      |
|                          | <tr><td>Who</td><td>What</td><td>Done?</td></tr></thead><tbody><tr><td>Him</td><td>Bar</td><td> </td></tr><tr><td>Her</td><td>**Foo** the Bar</td><td><strong>x</strong></td></tr><tr><td>Them</td><td><span>Floop the pig<br/></span></td><td>x</td></tr></tbody></table>, body_full_preview:<table><thead>                                                                                                           |
|                          | <tr><td>Who</td><td>What</td><td>Done?</td></tr></thead><tbody><tr><td>Him</td><td>Bar</td><td> </td></tr><tr><td>Her</td><td>**Foo** the Bar</td><td><strong>x</strong></td></tr><tr><td>Them</td><td><span>Floop the pig<br/></span></td><td>x</td></tr></tbody></table>                                                                                                                                             |
+--------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| *Comment #133* (Comment) | body:<p>breaking </p>                                                                                                                                                                                                                                                                                                                                                                                                  |
|                          | <p></p>                                                                                                                                                                                                                                                                                                                                                                                                                |
|                          | <p><strong><span>De&nbsp;:</span></strong><span> <a href="mailto:lino@**foo**.net">lino@**foo**.net</a> [<a href="mailto:**foo**@bar.com">mailto:**foo**@bar.com</a>] <br> <strong>Envoy&eacute;&nbsp;:</strong> mardi 18 octobre 2016 08:52<br> <strong>&Agrave;&nbsp;:</strong> <a href="mailto:Far@baz.net">eexample@**foo**.com</a><br> <strong>Objet&nbsp;:</strong> [welcht] YOU modified **FOO** BAR</span></p> |
|                          | <p>&nbsp;</p>                                                                                                                                                                                                                                                                                                                                                                                                          |
|                          | <p>Dear Aur&eacute;lie ,</p>                                                                                                                                                                                                                                                                                                                                                                                           |
|                          | <p>this is to notify  / BAR</p>                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | <p>BAR modified </p>                                                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>TODO: include a summary of the modifications.</p>                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>Any subsequent notifications about **foo**/  until you view this notification in the Lino web interface. Please visit</p>                                                                                                                                                                                                                                                                                           |
|                          | <p><a href="None">None</a></p>                                                                                                                                                                                                                                                                                                                                                                                         |
|                          | <p>and follow your welcome messages.</p>, body_short_preview:breaking                                                                                                                                                                                                                                                                                                                                                  |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | De : lino@**foo**.net [mailto:**foo**@bar.com]  Envoyé : mardi 18 octobre 2016 08:52 À : eexample@**foo**.com Objet : [welcht] YOU modified **FOO** BAR                                                                                                                                                                                                                                                                |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | Dear Aurélie ,                                                                                                                                                                                                                                                                                                                                                                                                         |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | this is to notify  / BAR                                                                                                                                                                                                                                                                                                                                                                                               |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | BAR modified                                                                                                                                                                                                                                                                                                                                                                                                           |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | TODO: include a summary of the modifications.                                                                                                                                                                                                                                                                                                                                                                          |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | Any subsequent notifications about **foo**/  un..., body_full_preview:<p>breaking </p>                                                                                                                                                                                                                                                                                                                                 |
|                          | <p></p>                                                                                                                                                                                                                                                                                                                                                                                                                |
|                          | <p><strong><span>De :</span></strong><span> <a href="mailto:lino@**foo**.net">lino@**foo**.net</a> [<a href="mailto:**foo**@bar.com">mailto:**foo**@bar.com</a>] <br/> <strong>Envoyé :</strong> mardi 18 octobre 2016 08:52<br/> <strong>À :</strong> <a href="mailto:Far@baz.net">eexample@**foo**.com</a><br/> <strong>Objet :</strong> [welcht] YOU modified **FOO** BAR</span></p>                                |
|                          | <p> </p>                                                                                                                                                                                                                                                                                                                                                                                                               |
|                          | <p>Dear Aurélie ,</p>                                                                                                                                                                                                                                                                                                                                                                                                  |
|                          | <p>this is to notify  / BAR</p>                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | <p>BAR modified </p>                                                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>TODO: include a summary of the modifications.</p>                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>Any subsequent notifications about **foo**/  until you view this notification in the Lino web interface. Please visit</p>                                                                                                                                                                                                                                                                                           |
|                          | <p><a href="None">None</a></p>                                                                                                                                                                                                                                                                                                                                                                                         |
|                          | <p>and follow your welcome messages.</p>                                                                                                                                                                                                                                                                                                                                                                               |
+--------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| *Comment #141* (Comment) | body:<p>breaking </p>                                                                                                                                                                                                                                                                                                                                                                                                  |
|                          | <p></p>                                                                                                                                                                                                                                                                                                                                                                                                                |
|                          | <p><strong><span>De&nbsp;:</span></strong><span> <a href="mailto:lino@**foo**.net">lino@**foo**.net</a> [<a href="mailto:**foo**@bar.com">mailto:**foo**@bar.com</a>] <br> <strong>Envoy&eacute;&nbsp;:</strong> mardi 18 octobre 2016 08:52<br> <strong>&Agrave;&nbsp;:</strong> <a href="mailto:Far@baz.net">eexample@**foo**.com</a><br> <strong>Objet&nbsp;:</strong> [welcht] YOU modified **FOO** BAR</span></p> |
|                          | <p>&nbsp;</p>                                                                                                                                                                                                                                                                                                                                                                                                          |
|                          | <p>Dear Aur&eacute;lie ,</p>                                                                                                                                                                                                                                                                                                                                                                                           |
|                          | <p>this is to notify  / BAR</p>                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | <p>BAR modified </p>                                                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>TODO: include a summary of the modifications.</p>                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>Any subsequent notifications about **foo**/  until you view this notification in the Lino web interface. Please visit</p>                                                                                                                                                                                                                                                                                           |
|                          | <p><a href="None">None</a></p>                                                                                                                                                                                                                                                                                                                                                                                         |
|                          | <p>and follow your welcome messages.</p>, body_short_preview:breaking                                                                                                                                                                                                                                                                                                                                                  |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | De : lino@**foo**.net [mailto:**foo**@bar.com]  Envoyé : mardi 18 octobre 2016 08:52 À : eexample@**foo**.com Objet : [welcht] YOU modified **FOO** BAR                                                                                                                                                                                                                                                                |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | Dear Aurélie ,                                                                                                                                                                                                                                                                                                                                                                                                         |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | this is to notify  / BAR                                                                                                                                                                                                                                                                                                                                                                                               |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | BAR modified                                                                                                                                                                                                                                                                                                                                                                                                           |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | TODO: include a summary of the modifications.                                                                                                                                                                                                                                                                                                                                                                          |
|                          |                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | Any subsequent notifications about **foo**/  un..., body_full_preview:<p>breaking </p>                                                                                                                                                                                                                                                                                                                                 |
|                          | <p></p>                                                                                                                                                                                                                                                                                                                                                                                                                |
|                          | <p><strong><span>De :</span></strong><span> <a href="mailto:lino@**foo**.net">lino@**foo**.net</a> [<a href="mailto:**foo**@bar.com">mailto:**foo**@bar.com</a>] <br/> <strong>Envoyé :</strong> mardi 18 octobre 2016 08:52<br/> <strong>À :</strong> <a href="mailto:Far@baz.net">eexample@**foo**.com</a><br/> <strong>Objet :</strong> [welcht] YOU modified **FOO** BAR</span></p>                                |
|                          | <p> </p>                                                                                                                                                                                                                                                                                                                                                                                                               |
|                          | <p>Dear Aurélie ,</p>                                                                                                                                                                                                                                                                                                                                                                                                  |
|                          | <p>this is to notify  / BAR</p>                                                                                                                                                                                                                                                                                                                                                                                        |
|                          | <p>BAR modified </p>                                                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>TODO: include a summary of the modifications.</p>                                                                                                                                                                                                                                                                                                                                                                   |
|                          | <p>Any subsequent notifications about **foo**/  until you view this notification in the Lino web interface. Please visit</p>                                                                                                                                                                                                                                                                                           |
|                          | <p><a href="None">None</a></p>                                                                                                                                                                                                                                                                                                                                                                                         |
|                          | <p>and follow your welcome messages.</p>                                                                                                                                                                                                                                                                                                                                                                               |
+--------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

If your search contains more than one word, it shows all rows containing both words.

>>> rt.show('search.SiteSearch', quick_search="est land", limit=5)
+------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Description                  | Matches                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+==============================+=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
| *Estonia* (Country)          | name:**Est**onia, name_de:**Est****land**, name_fr:**Est**onie                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
+------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| *Flandre de l'Est* (Place)   | name:F**land**re de l'**Est**, name_de:Ostf**land**ern, name_fr:F**land**re de l'**Est**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| *Flandre de l'Ouest* (Place) | name:F**land**re de l'Ou**est**, name_de:W**est**f**land**ern, name_fr:F**land**re de l'Ou**est**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
+------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| *Comment #115* (Comment)     | body:<p>Lorem ipsum<strong> dolor sit amet</strong>, consectetur adipiscing elit. Nunc cursus felis nisi, eu pellentesque lorem lobortis non. Aenean non sodales neque, vitae venenatis lectus. In eros dui, gravida et dolor at, pellentesque hendrerit magna. Quisque vel lectus dictum, rhoncus massa feugiat, condimentum sem. Donec elit nisl, placerat vitae imperdiet eget, hendrerit nec quam. Ut elementum ligula vitae odio efficitur rhoncus. Duis in b**land**it neque. Sed dictum mollis volutpat. Morbi at **est** et nisi euismod viverra. Nulla quis lacus vitae ante sollicitudin tincidunt. Donec nec enim in leo vulputate ultrices. Suspendisse potenti. Ut elit nibh, porta ut enim ac, convallis mol**est**ie risus. Praesent consectetur lacus lacus, in faucibus justo fringilla vel.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
|                              | <p>Donec fermentum enim et maximus v**est**ibulum. Sed mollis lacus quis dictum fermentum. Maecenas libero tellus, hendrerit cursus pretium et, hendrerit quis lectus. Nunc bibendum nunc nunc, ac commodo sem interdum ut. Quisque vitae turpis lectus. Nullam efficitur scelerisque hendrerit. Fusce feugiat ullamcorper nulla. Suspendisse quis placerat ligula. Etiam ullamcorper elementum consectetur. Aenean et diam ullamcorper, posuere turpis eget, eg**est**as nibh. Quisque condimentum arcu ac metus sodales placerat. Quisque placerat, quam nec tincidunt pharetra, urna justo scelerisque urna, et vulputate ipsum lacus at ligula.</p>, body_full_preview:<p>Lorem ipsum<strong> dolor sit amet</strong>, consectetur adipiscing elit. Nunc cursus felis nisi, eu pellentesque lorem lobortis non. Aenean non sodales neque, vitae venenatis lectus. In eros dui, gravida et dolor at, pellentesque hendrerit magna. Quisque vel lectus dictum, rhoncus massa feugiat, condimentum sem. Donec elit nisl, placerat vitae imperdiet eget, hendrerit nec quam. Ut elementum ligula vitae odio efficitur rhoncus. Duis in b**land**it neque. Sed dictum mollis volutpat. Morbi at **est** et nisi euismod viverra. Nulla quis lacus vitae ante sollicitudin tincidunt. Donec nec enim in leo vulputate ultrices. Suspendisse potenti. Ut elit nibh, porta ut enim ac, convallis mol**est**ie risus. Praesent consectetur lacus lacus, in faucibus justo fringilla vel.</p> |
|                              | <p>Donec fermentum enim et maximus v**est**ibulum. Sed mollis lacus quis dictum fermentum. Maecenas libero tellus, hendrerit cursus pretium et, hendrerit quis lectus. Nunc bibendum nunc nunc, ac commodo sem interdum ut. Quisque vitae turpis lectus. Nullam efficitur scelerisque hendrerit. Fusce feugiat ullamcorper nulla. Suspendisse quis placerat ligula. Etiam ullamcorper elementum consectetur. Aenean et diam ullamcorper, posuere turpis eget, eg**est**as nibh. Quisque condimentum arcu ac metus sodales placerat. Quisque placerat, quam nec tincidunt pharetra, urna justo scelerisque urna, et vulputate ipsum lacus at ligula.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| *Comment #131* (Comment)     | body:<p>Lorem ipsum<strong> dolor sit amet</strong>, consectetur adipiscing elit. Nunc cursus felis nisi, eu pellentesque lorem lobortis non. Aenean non sodales neque, vitae venenatis lectus. In eros dui, gravida et dolor at, pellentesque hendrerit magna. Quisque vel lectus dictum, rhoncus massa feugiat, condimentum sem. Donec elit nisl, placerat vitae imperdiet eget, hendrerit nec quam. Ut elementum ligula vitae odio efficitur rhoncus. Duis in b**land**it neque. Sed dictum mollis volutpat. Morbi at **est** et nisi euismod viverra. Nulla quis lacus vitae ante sollicitudin tincidunt. Donec nec enim in leo vulputate ultrices. Suspendisse potenti. Ut elit nibh, porta ut enim ac, convallis mol**est**ie risus. Praesent consectetur lacus lacus, in faucibus justo fringilla vel.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
|                              | <p>Donec fermentum enim et maximus v**est**ibulum. Sed mollis lacus quis dictum fermentum. Maecenas libero tellus, hendrerit cursus pretium et, hendrerit quis lectus. Nunc bibendum nunc nunc, ac commodo sem interdum ut. Quisque vitae turpis lectus. Nullam efficitur scelerisque hendrerit. Fusce feugiat ullamcorper nulla. Suspendisse quis placerat ligula. Etiam ullamcorper elementum consectetur. Aenean et diam ullamcorper, posuere turpis eget, eg**est**as nibh. Quisque condimentum arcu ac metus sodales placerat. Quisque placerat, quam nec tincidunt pharetra, urna justo scelerisque urna, et vulputate ipsum lacus at ligula.</p>, body_full_preview:<p>Lorem ipsum<strong> dolor sit amet</strong>, consectetur adipiscing elit. Nunc cursus felis nisi, eu pellentesque lorem lobortis non. Aenean non sodales neque, vitae venenatis lectus. In eros dui, gravida et dolor at, pellentesque hendrerit magna. Quisque vel lectus dictum, rhoncus massa feugiat, condimentum sem. Donec elit nisl, placerat vitae imperdiet eget, hendrerit nec quam. Ut elementum ligula vitae odio efficitur rhoncus. Duis in b**land**it neque. Sed dictum mollis volutpat. Morbi at **est** et nisi euismod viverra. Nulla quis lacus vitae ante sollicitudin tincidunt. Donec nec enim in leo vulputate ultrices. Suspendisse potenti. Ut elit nibh, porta ut enim ac, convallis mol**est**ie risus. Praesent consectetur lacus lacus, in faucibus justo fringilla vel.</p> |
|                              | <p>Donec fermentum enim et maximus v**est**ibulum. Sed mollis lacus quis dictum fermentum. Maecenas libero tellus, hendrerit cursus pretium et, hendrerit quis lectus. Nunc bibendum nunc nunc, ac commodo sem interdum ut. Quisque vitae turpis lectus. Nullam efficitur scelerisque hendrerit. Fusce feugiat ullamcorper nulla. Suspendisse quis placerat ligula. Etiam ullamcorper elementum consectetur. Aenean et diam ullamcorper, posuere turpis eget, eg**est**as nibh. Quisque condimentum arcu ac metus sodales placerat. Quisque placerat, quam nec tincidunt pharetra, urna justo scelerisque urna, et vulputate ipsum lacus at ligula.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
>>> rt.show('search.SiteSearch', quick_search="123", limit=5)  
===================================================== ========================
 Description                                           Matches
----------------------------------------------------- ------------------------
 *Arens Andreas* (Partner)                             phone:+32 87**123**456
 *Arens Annette* (Partner)                             phone:+32 87**123**457
 *Dobbelstein-Demeulenaere Dorothée* (Partner)         id:123
 *Mr Andreas Arens* (Person)                           phone:+32 87**123**456
 *Mrs Annette Arens* (Person)                          phone:+32 87**123**457
 *Mrs Dorothée Dobbelstein-Demeulenaere* (Person)      id:123
 *+32 87123456* (Contact detail)                       value:+32 87**123**456
 *+32 87123457* (Contact detail)                       value:+32 87**123**457
 *Diner (09.05.2015 13:30)* (Calendar entry)           id:123
 *SLS 9.2* (Movement)                                  id:123
 *DOBBELSTEIN-DEMEULENAERE Dorothée (123)* (Patient)   id:123
===================================================== ========================

Dobbelstein-Demeulenaere Dorothée (id 123) is Partner, Person and Patient. All three instances are listed in the SiteSearch.

The ElasticSiteSearch table

class lino.modlib.search.ElasticSiteSearch

A virtual table used to search on this Lino site using ElasticSearch.

It exists only when settings.SITE.use_elasticsearch is True.

Install an ElasticSearch server

This section explains how an application developer can install ElasticSearch on their machine.

General instructions are here. For our demonstration purposes we will install ElasticSearch in a docker container using the instructions provided here.

Pull the docker image of ElasticSearch:

$ sudo docker pull docker.elastic.co/elasticsearch/elasticsearch:7.15.0

Run an ElasticSearch instance on a docker container:

$ sudo docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.15.0

PS: Always try to use the latest ElasticSearch version and replace 7.15.0 with current x.xx.x version.

To run a docker container instance with minimal security enabled, use the following command:

$ sudo docker run -p 127.0.0.1:9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e "xpack.security.enabled=true" -e ELASTIC_PASSWORD="elastic" docker.elastic.co/elasticsearch/elasticsearch:7.15.0

Here we replaced -p 9200:9200 with -p 127.0.0.1:9200:9200 so that port 9200 is only accessible from the localhost and not open to the external network. The option -e "xpack.security.enabled=true" enables the security feature of the docker container and -e ELASTIC_PASSWORD="elastic" set the password to elastic for the default user: elastic.

Source: https://www.elastic.co/guide/en/elasticsearch/reference/7.15/security-minimal-setup.html

Preparing Indexes

lino.modlib.search provides a management command python manage.py createindexes (recommended to run before publishing a site) that has the following grounds on a lino site:

The following method search.utils.ESResolver.resolve_es_indexes() gathers all the indexes, defined in the model definitions as lino.core.model.Model.ES_indexes, into a dictionary for further uses in the app elasticsearch_django's settings:

>>> from lino.modlib.search.utils import ESResolver
>>> indexes, _ = ESResolver.resolve_es_indexes()
>>> indexes
{'comment': {'models': [<class 'lino.modlib.comments.models.Comment'>]}, 'ticket': {'models': [<class 'lino_noi.lib.tickets.models.Ticket'>]}}
>>> ESResolver.read_indexes()
{'comment': {'models': ['comments.Comment']}, 'ticket': {'models': ['tickets.Ticket']}}

URL to ElasticSearch instance:

>>> es_url = dd.plugins.search.ES_url
>>> print(es_url)
https://elastic:mMh6KlFP0UAooywwsWPLJ3ae@lino.es.us-central1.gcp.cloud.es.io:9243

Create ElasticSearch index mapping files:

>>> # ESResolver.create_index_mapping_files()
>>> mappings_dir = dd.plugins.search.mappings_dir
>>> print(mappings_dir) 
/.../lino/modlib/search/search/mappings

Content of the mapping files:

>>> import json
>>> def get_mappings_from_files(mappings_dir):
...     maps = dict()
...     for index in ESResolver.get_indexes():
...         filename = mappings_dir / (index + '.json')
...         with open(filename, 'r') as f:
...             obj = json.load(f)
...         maps[index] = obj
...     print(maps)
>>> get_mappings_from_files(mappings_dir)
{'comment': {'mappings': {'properties': {'body': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}, 'analyzer': 'autocomplete', 'search_analyzer': 'autocomplete_search'}, 'body_full_preview': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}, 'analyzer': 'autocomplete', 'search_analyzer': 'autocomplete_search'}, 'body_short_preview': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}, 'analyzer': 'autocomplete', 'search_analyzer': 'autocomplete_search'}, 'model': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}, 'analyzer': 'autocomplete', 'search_analyzer': 'autocomplete_search'}, 'modified': {'type': 'date'}, 'owner_id': {'type': 'long'}, 'owner_type': {'type': 'long'}, 'private': {'type': 'boolean'}, 'user': {'type': 'long'}}}}, 'ticket': {'mappings': {'properties': {'closed': {'type': 'boolean'}, 'description': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}, 'analyzer': 'autocomplete', 'search_analyzer': 'autocomplete_search'}, 'end_user': {'type': 'long'}, 'feedback': {'type': 'boolean'}, 'model': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}, 'analyzer': 'autocomplete', 'search_analyzer': 'autocomplete_search'}, 'priority': {'type': 'long'}, 'private': {'type': 'boolean'}, 'site': {'type': 'long'}, 'standby': {'type': 'boolean'}, 'state': {'properties': {'text': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}}, 'value': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}}}}, 'summary': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}, 'analyzer': 'autocomplete', 'search_analyzer': 'autocomplete_search'}, 'ticket_type': {'type': 'long'}, 'upgrade_notes': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}, 'analyzer': 'autocomplete', 'search_analyzer': 'autocomplete_search'}, 'user': {'type': 'long'}, 'waiting_for': {'type': 'text', 'fields': {'keyword': {'type': 'keyword', 'ignore_above': 256}}, 'analyzer': 'autocomplete', 'search_analyzer': 'autocomplete_search'}}}}}

Don't read on

The remaining part of this page is just technical stuff.

>>> from lino.core.utils import get_models
>>> ar = rt.login()
>>> user_type = ar.get_user().user_type
>>> count = 0
>>> for model in get_models():
...     t = model.get_default_table()
...     if not t.get_view_permission(user_type):
...         print("Not visible: {}".format(t))
...     count += 1
>>> print("Verified {} models".format(count))
Verified 92 models
>>> rt.models.contacts.Person.quick_search_fields_digit
(<django.db.models.fields.BigAutoField: id>,)

The following request caused problems before 20180920 (#2544 SiteSearch fails when it finds a name containing "&")

>>> rt.show('search.SiteSearch', quick_search="rumma")
================================ =================================
 Description                      Matches
-------------------------------- ---------------------------------
 *Rumma & Ko OÜ* (Partner)        name:**Rumma** & Ko OÜ
 *Saffre-Rumma* (Partner)         name:Saffre-**Rumma**
 *Rumma & Ko OÜ* (Organization)   name:**Rumma** & Ko OÜ
 *Saffre-Rumma* (Organization)    name:Saffre-**Rumma**
 *SLS 5/2015* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 3/2015* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 1/2015* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 23/2014* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 21/2014* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 19/2014* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 17/2014* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 15/2014* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 13/2014* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 11/2014* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 9/2014* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 7/2014* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 5/2014* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 3/2014* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 1/2014* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 23/2013* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 21/2013* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 19/2013* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 17/2013* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 15/2013* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 13/2013* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 11/2013* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 9/2013* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 7/2013* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 5/2013* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 3/2013* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 1/2013* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 23/2012* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 21/2012* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 19/2012* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 17/2012* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 15/2012* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 13/2012* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 11/2012* (Sales invoice)    partner__name:**Rumma** & Ko OÜ
 *SLS 9/2012* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 7/2012* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 5/2012* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 3/2012* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLS 1/2012* (Sales invoice)     partner__name:**Rumma** & Ko OÜ
 *SLA 1/2014* (Order)             project__name:**Rumma** & Ko OÜ
================================ =================================

Using Solr as search backend

Lino utilizes django-haystack app along with apache Solr backend server to provide intelligent search functionalities.

Installing Solr and dependencies

The following command will set the application developer for a compatible Solr - django-haystack environment.

Install pysolr:

$ pip install pysolr

Install django-haystack:

$ pip install django-haystack

Install java-1.8 (Linux):

jre1.8 is available for download here however, to make it more command line friendly, lino team made downloading jre1.8 available from another source for Linux (x64) based systems. Follow the commands below:

$ cd ~
$ curl -o jre-8u311-linux-x64.tar.gz "link"
$ tar -xvf jre-8u311-linux-x64.tar.gz

Change JAVA_HOME to our downloaded jre1.8:

$ export JAVA_HOME="$HOME/jre1.8.0_311"

Install solr-8.11.0:

$ cd ~
$ curl -o solr-8.11.0.zip "https://dlcdn.apache.org/lucene/solr/8.11.0/solr-8.11.0.zip"
$ unzip solr-8.11.0.zip

Add solr-6.6.6 to unix PATH:

$ export PATH="$HOME/solr-8.11.0/bin:$PATH"

Run solr and create a collection named lino:

$ solr start
$ solr create -c lino -n basic_config

Now we are ready to use solr with a lino project. Go to the root directory of you lino project (where manage.py is located) and run the following command:

$ python manage.py build_solr_schema_v2 --configure-directory="$HOME/solr-8.11.0/server/lino/conf" --reload-core=lino

The last command will build solr schema and reload the solr core name lino. And now your solr core/collection lino is ready for indexing and searching search documents.