Tip

Found anything unclear or needy of further explanation? Do send us the feedback at docs@tattler.dev !

Quick start#

This guides you to:

  1. install tattler into a virtual environment

  2. run tattler_server

  3. trigger a notification through it, using any of your 3 available options:

  4. Write your first notification template.

  5. Organize your configuration.

  6. Write an addressbook plug-in to load contact data of your users.

  7. Write a context plug-in to load common variables for your templates.

Install#

# create and load a virtualenv to install into
mkdir ~/tattler_quickstart
python3 -m venv ~/tattler_quickstart/venv
. ~/tattler_quickstart/venv/bin/activate

# install tattler into it
pip install tattler

Run tattler server#

export TATTLER_MASTER_MODE=production

# if you need to customize your SMTP settings
export TATTLER_SMTP_ADDRESS="127.0.0.1:25"
export TATTLER_SMTP_AUTH="username:password" # you will learn secure configuration later
export TATTLER_SMTP_TLS=yes

# run tattler server on default 127.0.0.1:11503
tattler_server

This causes the following basic scenario:

  • tattler_server listens for requests on http://127.0.0.1:11503.

  • tattler_server delivers notification to the actual recipient (no dry-run!) due to the TATTLER_MASTER_MODE environment variable.

  • tattler_server uses SMTP server 127.0.0.1:25 to deliver email, using STARTTLS and the given authentication credentials.

Don’t worry, you’ll soon learn a simpler, secure way to manage this configuration.

Send a notification via HTTP#

You can do this via plain HTTP request, e.g. using curl:

# in a new terminal:

# replace ``your@email.com`` with your actual email address
curl -X POST 'http://127.0.0.1:11503/notification/demoscope/demoevent/?mode=production&user=your@email.com'

Here’s what this does:

  • It contacts tattler_server at its REST endpoint http://127.0.0.1:11503/notification/.

  • It asks to send the notification for demoevent, which is built into tattler’s distribution.

  • It asks to send it to your@email.com.

  • It asks to actually send it to the indicated recipient, with mode=production.

In the terminal running tattler_server you’ll see the notification go out successfully:

INFO:tattler.server.tattler_utils:Sending demoevent:None (evname:language) to #your@email.com@email => [your@email.com], context={'user_id': 'your@email.com', 'user_email': 'your@email.com', 'user_sms': None, 'user_firstname': 'Your', 'user_account_type': None, 'user_language': None, 'correlation_id': 'tattler:b8357483-1115-4e8a-9b5e-4840cbd4a285', 'notification_id': '4840cbd4a285', 'notification_mode': 'production', 'notification_vector': 'email', 'notification_scope': 'demoscope', 'event_name': 'demoevent'} (cid=tattler:b8357483-1115-4e8a-9b5e-4840cbd4a285)
INFO:tattler.server.sendable.vector_sendable:neb59232b-8819-4d2c-93f4-2791cda09f50: Sending 'demoevent'@'email' to: {'your@email.com'}
INFO:tattler.server.tattlersrv_http:Notification sent. [{'id': 'email:fda614e8-61ef-4a00-98b6-6c993e90f1f0', 'vector': 'email', 'resultCode': 0, 'result': 'success', 'detail': 'OK'}]
127.0.0.1 - - [11/Feb/2024 21:08:33] "POST /notification/demoscope/demoevent/?mode=production&user=your@email.com HTTP/1.1" 200 -

… and your mailbox will show the demo notification:

_images/tattler-notification-demo-email-html-light.png _images/tattler-notification-demo-email-plaintext-light.png

Why demo? Why email?#

Now here’s a couple of things which might turn your nose:

“Why demoevent? I thought tattler allowed me to send my own notifications!”

That’s right. demoevent is a template built into tattler to allow demos. We’ll look into writing your own notifications soon.

“Why your@email.com? I thought tattler would look up user information for me!”

That’s right. Tattler really shines when it loads your data for you. We’ll look into that in the plug-ins section.

Send a notification via command-line#

An alternative is for you to trigger the notification with a command line tool.

Tattler includes a little utility to easily trigger notifications from the command line:

# load the same virtual environment where you installed tattler server
. ~/tattler_quickstart/venv/bin/activate

# replace ``your@email.com`` with your actual email address
tattler_notify -s '127.0.0.1:11503' -m production your@email.com demoscope demoevent

Done!

This does exactly the same as Send a notification via HTTP, using the same REST API, and actually relying on tattler’s python client SDK which we’ll look into next.

  • Argument -s provides the SMTP address to use to deliver email.

  • Argument -m controls what notification mode the client should request, as TATTLER_MASTER_MODE did for the server.

Send a notification via python#

A third option is for you to trigger the notification from python code.

Tattler includes a little python client library:

from tattler.client.tattler_py import send_notification

# replace ``your@email.com`` with your actual email address
send_notification('demoscope', 'demoevent', 'your@email.com', mode='production', srv_addr='127.0.0.1', srv_port=11503)

Again, this code does the same as shown in Send a notification via HTTP: it contacts tattler_server on the same REST API endpoint.

Write your own notification templates#

What actual content should tattler send, for the event we requested? Event templates tell tattler that.

# create a directory to host notification templates and change into it
mkdir -p ~/tattler_quickstart/templates/mywebapp
cd ~/tattler_quickstart/templates/mywebapp

# create a template for an event titled "password changed"
mkdir password_changed

# we want event 'password_changed' to send an email notification
mkdir password_changed/email
cd password_changed/email
# so we need a subject and body
echo 'You successfully changed your password!' > subject
echo 'Hey!\n\nAccount password changed!' > body_plain

Done. Our notification templates directory now looks like this:

tattler_quickstart/
└── templates/
    └── mywebapp/                 # scope  = mywebapp
        └── password_changed/     # event  = password_changed
            └── email/            # vector = email
                ├── body_plain
                └── subject

Find more information about designing templates in the documentation for template designers.

Tell tattler server where to find event templates#

tattler_server takes the path holding notification event templates from the TATTLER_TEMPLATE_BASE environment variable. So let’s restart it with it:

# in the terminal which was running script 'tattler_server':

# Stop the running instance with Ctrl-c

# Re-start the instance with the new path
TATTLER_TEMPLATE_BASE=~/tattler_quickstart/templates TATTLER_MASTER_MODE=production tattler_server

… and tell tattler to send a notification for your new event (replace your email address, as usual):

curl -X POST 'http://127.0.0.1:11503/notification/mywebapp/password_changed/?user=your@email.com&mode=production'

Upon which tattler_server will confirm:

INFO:tattler.server.tattler_utils:Sending password_changed:None (evname:language) to #your@email.com@email => [your@email.com], context={'user_id': 'your@email.com', 'user_email': 'your@email.com', 'user_sms': None, 'user_firstname': 'user', 'user_account_type': None, 'user_language': None, 'correlation_id': 'tattler:a6c81356-662a-4b1a-a6ca-a0b9dbcbe34e', 'notification_id': 'a0b9dbcbe34e', 'notification_mode': 'production', 'notification_vector': 'email', 'notification_scope': 'mywebapp', 'event_name': 'password_changed'} (cid=tattler:a6c81356-662a-4b1a-a6ca-a0b9dbcbe34e)
INFO:tattler.server.sendable.vector_sendable:n83311a12-f922-400b-8bef-362a84289f9e: Sending 'password_changed'@'email' to: {'your@email.com'}
INFO:tattler.server.tattlersrv_http:Notification sent. [{'id': 'email:56985c3c-b5c9-4c87-8329-75c2335d28de', 'vector': 'email', 'resultCode': 0, 'result': 'success', 'detail': 'OK'}]
127.0.0.1 - - [11/Feb/2024 21:33:55] "POST /notification/mywebapp/password_changed/?user=your@email.com&mode=production HTTP/1.1" 200 -

Then go check your mailbox to confirm that you received the notification with the content you put into your template.

Hint

Exploit the demo template!

The demo template you used earlier is your friend. It includes tips on writing effective, portable HTML emails – and most of all it includes some code snippets which you can simply copy and paste to write your own templates.

Find the source code of the demo templates in tattler’s repository.

Organize your configuration#

Tattler uses environment variables for configuration as they are highly flexible and avoid overbloating code with parsers.

envdir comes to your rescue to organize configuration cleanly, maintainably and securely. Envdir loads environment variables from a directory that holds one file per configuration key, whose content is the configuration value.

Here’s how you setup your configuration using envdir:

# create the directory holding your configuration, and change into it
mkdir -p ~/tattler_quickstart/etc
cd ~/tattler_quickstart/etc

# create file "TATTLER_MASTER_MODE" to hold content "staging",
# to have tattler copy notifications to the actual recipient and to NOTIF_DEBUG_RECIPIENT_EMAIL
echo "staging" > TATTLER_MASTER_MODE

# add more non-sensitive configuration values
echo support@myorganization.org > TATTLER_EMAIL_SENDER
echo "127.0.0.1:11503" > TATTLER_LISTEN_ADDRESS
echo ~/tattler_quickstart/templates > TATTLER_TEMPLATE_BASE
echo "your_own_email@company.com" > NOTIF_DEBUG_RECIPIENT_EMAIL
echo "127.0.0.1:25" > TATTLER_SMTP_ADDRESS
echo yes > TATTLER_SMTP_TLS

# And here is how to add sensitive configuration values:

# 1. create a file and restrict access to it
touch TATTLER_SMTP_AUTH
chown tattler TATTLER_SMTP_AUTH
chmod 0400 TATTLER_SMTP_AUTH
# 2. add content using an editor, so it does not get stored into your shell history
vim TATTLER_SMTP_AUTH

And finally start tattler having envdir loads the configuration above:

# load the virtualenv where you installed tattler
. ~/tattler_quickstart/venv/bin/activate

# have envdir load the configuration into an environment where tattler is started
envdir ~/tattler_quickstart/etc tattler_server &

Refer to available configuration options and documentation for sysadmins for further information.

Write an addressbook plug-in#

An addressbook plug-in enables tattler to automatically retrieve contact data about your users.

Once you have this, your applications no longer need to provide contact data to notify users.

They simply tell tattler “notify user #123 about event X”, and tattler figures out who is 123, what vectors it provided (email address, phone number), what are the concrete addresses, etc.

Start by creating a directory to hold your plugin(s):

# create directory to hold tattler plug-ins
mkdir -p ~/tattler_quickstart/plugins
cd ~/tattler_quickstart/plugins

# start editing your addressbook plug-in
vim myaddressbook_tattler_plugin.py

You may fill your plugin file myaddressbook_tattler_plugin.py starting from the sample addressbook plugin in tattler’s repository. Just replace the names of your tables and fields to your own schema:

"""Example Addressbook plug-in for tattler accessing SQL database"""

import os
import logging
from typing import Mapping, Optional, Tuple

from sqlalchemy import create_engine, MetaData, Table, select
from sqlalchemy.engine import Engine

from tattler.server.pluginloader import AddressbookPlugin


logging.basicConfig(level=os.getenv('LOG_LEVEL', 'debug').lower())
log = logging.getLogger(__name__)


class SQLAddressbookPlugin(AddressbookPlugin):
    """Sample plug-in to extract recipient information from a SQL database.
    
    This plug-in does the following:
    
    1. it connects to a SQL database using a connection URI passed via environment variable ``DATABASE``.
    2. it loads 2 tables from it: ``auth_user`` and ``userprofile``.
    3. during lookups, it extracts columns "email", "first_name", "mobile_number" and "telegram_id" after a natural join of the tables.

    Set an environment variable 'DATABASE' to point this code to connect to your database,
    for example:
    
    - sqlite:       ``DATABASE=sqlite://///Users/foo/tattler/sqlplugins.db``
    - postgresql:   ``DATABASE=postgresql://username:password@192.168.12.15:6432/dbname``
    - mysql:        ``DATABASE=mysql+pymysql://user:pass@some_mariadb/dbname?charset=utf8mb4``
    - mariadb:      ``DATABASE=mariadb+pymysql://user:pass@some_mariadb/dbname?charset=utf8mb4``

    also make sure you install a driver for SQLAlchemy to access your database:

    - sqlite:       driver is embedded in python
    - postgresql:   ``pip install psycopg``
    - mysql:        ``pip install PyMySQL``
    - mariadb:      ``pip install PyMySQL``

    See `SQLAlchemy's documentation <https://docs.sqlalchemy.org/en/20/dialects/mysql.html#dialect-mysql>`_
    for more information on database drivers and connection URIs:
    """

    def _connect_db(self) -> Tuple[Engine, Mapping[str, Table]]:
        """Connect to the database and load the required tables.
        
        Relies on environment variable ``DATABASE`` as a database connection URI, e.g. 'postgresql://user:pass@host:port/dbname'.

        :return:    tuple of SQLAlchemy engine and tables map.
        """
        db_path = os.getenv('DATABASE')
        if not db_path:
            log.error("Unable to look up contacts: DATABASE envvar is not set or empty. Set to a database connection URI, e.g. 'postgresql://user:pass@host:port/dbname'.")
            raise RuntimeError("Unable to look up contacts: DATABASE envvar is not set or empty. Set to a database connection URI, e.g. 'postgresql://user:pass@host:port/dbname'.")
        dbengine: Engine = create_engine(db_path, echo=False, future=True)
        metadata_obj = MetaData()
        tables = {}
        # load SQL tables 'auth_user' and 'userprofile'
        tables['user'] = Table("auth_user", metadata_obj, autoload_with=dbengine)
        tables['profile'] = Table("userprofile", metadata_obj, autoload_with=dbengine)
        return dbengine, tables

    def _get_recipient_data(self, recipient_id: str, role: Optional[str]=None) -> Mapping[str, str]:
        """Query database to return all known data about the recipient.
        
        Put the actual database lookup here. Do not stress about performance:
        premature optimization is the root of all evil!
        """
        res = super().attributes(recipient_id, role)
        recipient_id = '1'
        dbconn, tabs = self._connect_db()
        # select data by joining 'user' and 'profile' tables on id = user_id, then extracting columns "email", "first_name", "mobile_number", "telegram_id"
        qr = select(tabs['user'].c.email, tabs['profile'].c["first_name", "mobile_number", "telegram_id"]).join_from(tabs['user'], tabs['profile'], tabs['user'].c.id == tabs['profile'].c.user_id).where(tabs['user'].c.id==int(recipient_id))
        with dbconn.connect() as conn:
            res = conn.execute(qr).one_or_none()
            if res is None:
                return {}
            return {
                'email': res.email,
                'first_name': res.first_name or None,
                'mobile': res.mobile_number or None,
                'telegram': res.telegram_id or None,
            }

    def attributes(self, recipient_id: str, role: str | None = None) -> Mapping[str, str | None]:
        """Return all known properties for user in one go"""
        data = self._get_recipient_data(recipient_id, role)
        if 'mobile' in data:
            data['sms'] = data['mobile']
            data['whatsapp'] = data['mobile']
        return data

Now enable tattler to load the plugin:

# load the virtualenv where you installed tattler
cd ~/tattler_quickstart
. venv/bin/activate

# install the dependencies required by your plug-in. In this case, let's assume
# sqlalchemy
pip install sqlalchemy

# use the sample database from our codebase to test this
curl -O https://raw.githubusercontent.com/tattler-community/tattler-community/main/plugins/sqlplugins.sql
sqlite3 sqlplugins.db < sqlplugins.sql
# check out the content of the sample DB. Feel free to replace e.g. your email address
sqlite3 sqlplugins.db 'select * from auth_user join userprofile on auth_user.id = user_id'

# configure tattler to load your new plug-in
echo ~/tattler_quickstart/plugins/ > ~/tattler_quickstart/etc/TATTLER_PLUGIN_PATH
echo sqlite:////$HOME/tattler_quickstart/sqlplugins.db > ~/tattler_quickstart/etc/DATABASE

# you may want to enable debug-level logging when testing new logic
echo debug > ~/tattler_quickstart/etc/LOG_LEVEL

# set email address to have tattler use as sender
echo support@myorganization.org > TATTLER_EMAIL_SENDER

# set tattler in 'staging' mode, i.e. copy every notification to a 'supervisor' address
echo staging > TATTLER_MASTER_MODE

# set email address to have tattler copy every notification to ()
echo notification@myorganization.org > TATTLER_SUPERVISOR_RECIPIENT_EMAIL

# restart tattler_server with the new configuration
envdir ~/tattler_quickstart/etc tattler_server

Have a look at the log output here. You’ll see some message like the following:

...
INFO:tattler.server.pluginloader:Loading plugin SQLAddressbookPlugin (<class 'myaddressbook_tattler_plugin.SQLAddressbookPlugin'>) from module myaddressbook_tattler_plugin
...

and now trigger a notification using a user ID:

curl -X POST 'http://127.0.0.1:11503/notification/demoscope/demoevent/?user=2&mode=production'

… and there you go: tattler delivers the notification to email address your@email.net as stored in the database:

INFO:tattler.server.pluginloader:Looking up recipient 2 with addressbook plugin #1 'SQLAddressbookPlugin'
# ..
DEBUG:tattler.server.tattler_utils:Contacts for recipient 2 are: {'email': 'your@email.net', 'first_name': 'Michelle', 'mobile': '+1789456321', 'telegram': '5689234578', 'sms': '+1789456321', 'whatsapp': '+1789456321'}
INFO:tattler.server.tattler_utils:Recipient 2 is reachable over 1 vectors of the 1 requested: {'email'}

More details on this in addressbook plug-in documentation.

Write a context plug-in#

A context plug-in enables tattler to automatically retrieve data to make available to all your templates.

Once you have this, your applications no longer need to collect and supply data to expand templates. Instead, tattler autonomously pre-loads the necessary data through your context plug-in.

They are called “context” because they provide additional context to expand templates with, i.e. a collection of additional variables that templates may access.

Create your context plug-in in the same plug-in folder you created previously:

# as created in the example before
cd plugins

# start editing your context plug-in
vim mycontext_tattler_plugin.py

Feel free to fill your plugin file mycontext_tattler_plugin.py starting from the sample context plugin in tattler’s repository. Just replace the names of your tables and fields to your own schema:

"""Example context plug-in for tattler loading data from a SQL database"""

import os
import logging
from typing import Mapping, Tuple, Any

from sqlalchemy import create_engine, MetaData, Table, select
from sqlalchemy.engine import Engine

from tattler.server.pluginloader import ContextPlugin, ContextType


logging.basicConfig(level=os.getenv('LOG_LEVEL', 'debug').lower())
log = logging.getLogger(__name__)


class SQLContextTattlerPlugin(ContextPlugin):
    """Sample plug-in to extract context information from a SQL database to expand templates.

    This plug-in does the following:

    1. it omits overriding :meth:`processing_required()`, thereby loading at every notification.
    2. it loads some columns from 2 tables.

    Set an environment variable 'DATABASE' to point this code to connect to your database,
    for example:
    
    - sqlite:       ``DATABASE=sqlite://///Users/foo/tattler/sqlplugins.db``
    - postgresql:   ``DATABASE=postgresql://username:password@192.168.12.15:6432/dbname``
    - mysql:        ``DATABASE=mysql+pymysql://user:pass@some_mariadb/dbname?charset=utf8mb4``
    - mariadb:      ``DATABASE=mariadb+pymysql://user:pass@some_mariadb/dbname?charset=utf8mb4``

    also make sure you install a driver for SQLAlchemy to access your database:

    - sqlite:       driver is embedded in python
    - postgresql:   ``pip install psycopg``
    - mysql:        ``pip install PyMySQL``
    - mariadb:      ``pip install PyMySQL``

    See `SQLAlchemy's documentation <https://docs.sqlalchemy.org/en/20/dialects/mysql.html#dialect-mysql>`_
    for more information on database drivers and connection URIs:
    """

    def _connect_db(self) -> Tuple[Engine, Mapping[str, Table]]:
        """Connect to the database and load the required tables.
        
        Relies on environment variable ``DATABASE`` as a database connection URI, e.g. 'postgresql://user:pass@host:port/dbname'.

        :return:    tuple of SQLAlchemy engine and tables map.
        """
        db_path = os.getenv('DATABASE')
        if not db_path:
            log.error("Unable to look up contacts: DATABASE envvar is not set or empty. Set to a database connection URI, e.g. 'postgresql://user:pass@host:port/dbname'.")
            raise RuntimeError("Unable to look up contacts: DATABASE envvar is not set or empty. Set to a database connection URI, e.g. 'postgresql://user:pass@host:port/dbname'.")
        dbengine: Engine = create_engine(db_path, echo=False, future=True)
        metadata_obj = MetaData()
        tables = {}
        # load SQL tables 'resource' and 'billing'
        tables['resource'] = Table("resource", metadata_obj, autoload_with=dbengine)
        tables['billing'] = Table("billing", metadata_obj, autoload_with=dbengine)
        return dbengine, tables

    def _get_resource_consumption(self, user_id: str, dbconn: Engine, tabs: Mapping[str, Table]) -> Mapping[str, Any]:
        qr = select(tabs['resource'].c['traffic']).where(tabs['resource'].c.user_id==int(user_id))
        with dbconn.connect() as conn:
            res = conn.execute(qr).one_or_none()
            if res is None:
                return {}
            return {
                'traffic': res.traffic,
            }

    def _get_invoices(self, user_id: str, dbconn: Engine, tabs: Mapping[str, Table]) -> Mapping[str, Any]:
        qr = select(tabs['billing'].c['number', 'paid']).where(tabs['billing'].c.user_id==int(user_id))
        with dbconn.connect() as conn:
            res = conn.execute(qr)
            if res is None:
                return {}
            return {
                'invoice': [[inv.number, bool(inv.paid)] for inv in res],
            }

    def process(self, context: ContextType) -> ContextType:
        """Run the plug-in to generate new context data.

        The latest (previous) context is passed as input to this method, allowing
        the method to add, change or remove variables from it.
        
        :param context:     The latest context resulting from the previous plug-in, or tattler's native context for the first-running context plug-in.
        :return:            The generated context to either feed to the template, or to the next context plug-in in the chain.
        """
        dbconn, tables = self._connect_db()
        user_id = context['user_id']
        new_context = {}
        # add resource variables to context
        new_context |= self._get_resource_consumption(user_id, dbconn, tables)
        # add invoice variables to context
        new_context |= self._get_invoices(user_id, dbconn, tables)
        return context | new_context

How to enable the new plug-in? If you already followed Write an addressbook plug-in , you have already setup what’s needed. If not, go do so 🙂

Then simply reload tattler and you’ll see the new plug-in loaded:

INFO:tattler.server.pluginloader:Loading plugin SQLContextTattlerPlugin (<class 'sqlcontext_tattler_plugin.SQLContextTattlerPlugin'>) from module sqlcontext_tattler_plugin

and now trigger a notification again:

curl -X POST 'http://127.0.0.1:11503/notification/demoscope/demoevent/?user=2&mode=production'

… and observe that tattler loads new context variables:

INFO:tattler.server.pluginloader:Context after plugin SQLContextTattlerPlugin (in 0:00:00.002836): {'user_id': '2', 'user_email': 'your@email.net', 'user_sms': '+1789456321', 'user_firstname': 'Your', 'user_account_type': 'unknown', 'user_language': None, 'correlation_id': 'tattler:af4054ab-fac7-4b3a-ad80-0a9fa4326d36', 'notification_id': '0a9fa4326d36', 'notification_mode': 'debug', 'notification_vector': 'email', 'notification_scope': 'mywebapp', 'event_name': 'password_changed', 'traffic': 1224994602, 'invoice': [['2023123001', True]]}

More details on this in context plug-in documentation.

Done!#

You have gotten to know most of Tattler’s capabilities already 👏🏻

If this journey scratched your itch, consider giving a star to our repo. And if you run a tech blog – would Tattler possibly be a suitable next topic?

If you found friction at any step of the way, do let us know. We take documentation seriously and look forward for feedback!