`application` — Applications¶

This module helps with developing event-sourced applications.

An event-sourced application object has command and query methods used by clients to interact with its domain model. An application object has an event-sourced repository used to obtain already existing event-sourced aggregates. It also has a notification log that is used to propagate the state of the application as a sequence of domain event notifications.

Domain-driven design¶

The book Domain-Driven Design describes a “layered architecture” with four layers: interface, application, domain, and infrastructure. The application layer depends on the domain and infrastructure layers. The interface layer depends on the application layer.

Generally speaking, the application layer implements commands which change the state of the application, and queries which present the state of the application. The commands and queries (“application services”) are called from the interface layer. By keeping the application and domain logic in the application and domain layers, different interfaces can be developed for different technologies without duplicating application and domain logic.

The discussion below continues these ideas, by combining event-sourced aggregates and persistence objects in an application object that implements “application services” as object methods.

Application objects¶

In general, an application combines a stand-alone domain model and infrastructure that persists the state of that model. In this library, an event-sourced application object combines an event-sourced domain model with a cohesive mechanism for storing and retrieving domain events.

The library’s Application class brings together objects from the domain and persistence modules. It can be used as it is, or subclassed to develop domain-specific event-sourced applications.

The general idea is to name your application object class after the domain supported by its domain model, or after the “bounded context” supporting a “subdomain” if you are doing DDD. And then define command and query methods that allow clients to (let’s say) create, read, update and delete your event-sourced domain model aggregates. Domain model aggregates are discussed in the domain module documentation. The “ubiquitous language” of your project should guide the names of the application’s command and query methods, along with those of its domain model aggregates.

The main features of an application are:

the save() method, used for collecting and recording new aggregate events;
the repository attribute, with which aggregates are reconstructed;
the notification_log attribute, from which the state of the application can be propagated;
the take_snapshot() method;
the application environment, used to configure an application;
the command and query methods you define, which implement your “application services”.

The Application class defines an object method save() which can be used to update the recorded state of one or many domain model aggregates. The save() method works by using the aggregate’s collect_events() method to collect pending domain events; the pending domain events are stored by calling the put() method of application’s event store.

The Application class defines an object attribute repository which holds an event-sourced repository. The repository’s get() method can be used by your application’s command and query methods to obtain already existing aggregates.

The Application class defines an object attribute notification_log which holds a local notification log. The notification log can be used to propagate the state of an application as a sequence of domain event notifications.

The Application class defines an object method take_snapshot() which can be used for snapshotting existing aggregates. Snapshotting isn’t necessary, but can help to reduce the time it takes to access aggregates that would otherwise be reconstructed from a large number of recorded domain events.

The Application class has an env attribute, which can be redefined on your application classes. Application objects also have an env attribute which is determined by a combination of the application class attribute, the operating system environment, and by an optional constructor argument.

The Application class can be extended with command and query methods that use event-sourced aggregates.

Simple example¶

In the example below, the DogSchool application extends the library’s application object base class. The Dog aggregate is defined and discussed as the simple example in the domain module documentation.

from typing import List
from uuid import UUID

from eventsourcing.application import Application


class DogSchool(Application):
    def register_dog(self) -> UUID:
        dog = Dog.create()
        self.save(dog)
        return dog.id

    def add_trick(self, dog_id: UUID, trick: str):
        dog = self.repository.get(dog_id)
        dog.add_trick(trick)
        self.save(dog)

    def get_tricks(self, dog_id: UUID) -> List[str]:
        dog = self.repository.get(dog_id)
        return list(dog.tricks)

The register_dog() method is defined as a command method that creates and saves a new Dog aggregate. It returns a new dog_id that can be used to identify the aggregate on subsequence method calls. It saves the new aggregate by calling the base class save() method.

The add_trick() method is also defined as a command method. It works by obtaining an existing Dog aggregate from the repository. Then it calls the aggregate’s command method add_trick(), and then saves the aggregate by calling the application’s save() method.

The application’s get_tricks() method is defined as a query method that presents the current tricks of an existing Dog aggregate. It works by calling the repository get() method to reconstruct a Dog aggregate object from previously saved aggregate events, and then returns the value of its tricks attribute.

Having define an application object, we can use it. Below, an instance of the DogSchool application is constructed. A new Dog aggregate is created by calling register_dog(). Three items are added to its tricks by calling add_trick() three times. The recorded tricks of the aggregate is then obtained by calling get_tricks() method.

application = DogSchool()

dog_id = application.register_dog()

application.add_trick(dog_id, "roll over")
application.add_trick(dog_id, "fetch ball")
application.add_trick(dog_id, "play dead")

tricks = application.get_tricks(dog_id)
assert tricks[0] == "roll over"
assert tricks[1] == "fetch ball"
assert tricks[2] == "play dead"

In the example above, the application object uses the library’s “plain old Python objects” infrastructure, which keeps stored event objects in memory. This is the default for all application objects. To store the domain events in a real database, simply configure persistence by setting environment variables for the application.

Repository¶

The application repository is used to get the already existing aggregates of the application’s domain model. It is an instance of the library’s Repository class. The repository’s get() method is used to obtain already existing aggregates. It works by calling the get() method of an event store to retrieve already existing domain event objects for the requested aggregate, and then using these to reconstruct an aggregate object.

dog_latest = application.repository.get(dog_id)

assert len(dog_latest.tricks) == 3
assert dog_latest.version == 4

The Repository class implements a __contains__() method, so that you can use the Python in keyword to see whether or not an aggregate exists in the repository.

assert dog_id in application.repository

assert uuid4() not in application.repository

The repository get() method accepts three arguments: aggregate_id, version, and projector_func. The aggregate_id argument is required, and should be the ID of an already existing aggregate. If the aggregate is not found, the exception AggregateNotFound will be raised.

The version argument is optional, and represents the required version of the aggregate. If the requested version is greater than the highest available version of the aggregate, the highest available version of the aggregate will be returned.

dog_v1 = application.repository.get(dog_id, version=1)

assert dog_v1.version == 1
assert len(dog_v1.tricks) == 0

dog_v2 = application.repository.get(dog_id, version=2)

assert dog_v2.version == 2
assert len(dog_v2.tricks) == 1
assert dog_v2.tricks[-1] == "roll over"

dog_v3 = application.repository.get(dog_id, version=3)

assert dog_v3.version == 3
assert len(dog_v3.tricks) == 2
assert dog_v3.tricks[-1] == "fetch ball"

dog_v4 = application.repository.get(dog_id, version=4)

assert dog_v4.version == 4
assert len(dog_v4.tricks) == 3
assert dog_v4.tricks[-1] == "play dead"

dog_v5 = application.repository.get(dog_id, version=5)

assert dog_v5.version == 4  # There is no version 5.
assert len(dog_v5.tricks) == 3
assert dog_v5.tricks[-1] == "play dead"

The projector_func argument is also optional, and can be used to pass in an alternative “mutator function” that will be used as the “aggregate projector” to reconstruct the current state of the aggregate from stored snapshots and domain events. By default, the repository will use the mutate() methods of domain event objects to reconstruct the state of the requested aggregate.

It is possible to enable caching of aggregates in the application repository. See Configuring aggregate caching for more information.

Notification log¶

A notification log can be used to propagate the state of an application as a sequence of domain event notifications. The library’s LocalNotificationLog class presents the event notifications of an application. The application object attribute notification_log is an instance of LocalNotificationLog.

from eventsourcing.application import LocalNotificationLog

assert isinstance(application.notification_log, LocalNotificationLog)

This class implements an abstract base class NotificationLog which defines method signatures needed by the NotificationLogReader class, so that a reader can get event notifications from both “local” and “remote” notification logs. The event notifications themselves are instances of the library’s Notification class, which is defined in the persistence module and discussed in the Notification objects section.

The general idea is that, whilst the aggregate events are recorded in a sequence for the aggregate, all of the aggregate events are also given a “total order” by being placed in a sequence of event notifications. When recording aggregate events using an application recorder, event notifications are automatically and atomically recorded along with the stored events.

Outbox pattern¶

The “notification log pattern” is also referred to as the “outbox pattern”. The general idea is to avoid “dual writing” when recording updates to application state and then messaging those updates to others. The important thing is that either both the stored events and the notifications are recorded together, or nothing is recorded. Unless these two things are recorded atomically, in the same database transaction, it is impossible to exclude the possibility that one will happen but not the other, potentially causing a permanent and perhaps catastrophic inconsistency in the state of downstream applications. But it is equally important to avoid “dual writing” in the consumption of event notifications, which is why “tracking” records need to be written atomically when recording the result of processing event notifications. This is a topic we shall return to when discussing the system module.

Selecting notifications from the log¶

The select() method of a notification log can be used to directly select a sub-sequence of event notification objects from a notification log. In the example below, the first two event notifications are selected from the notification log of the Application object.

notifications = application.notification_log.select(start=1, limit=2)

The start and limit arguments are used to specify the selection. The selection will contain no more than the specified limit.

assert len(notifications) == 2

We can see they are all instances of the Notification class.

from eventsourcing.persistence import Notification

assert isinstance(notifications[0], Notification)
assert isinstance(notifications[1], Notification)

Each event notification has an id that is the unique integer ID of the event notification. The event notifications are ordered by their IDs, with later event notifications having higher values than earlier ones.

The selection of event notifications will have notification IDs which are greater or equal to the given value of start. Please note, by default, the first recorded event notification will have ID equal to 1.

assert notifications[0].id == 1
assert notifications[1].id == 2

We can see these notifications represent the facts that a Dog aggregate was created, and then two TrickAdded events occurred (“roll over”, “fetch ball”).

notification = notifications[0]
assert "Dog.Created" in notification.topic
assert notification.originator_id == dog_id

notification = notifications[1]
assert "Dog.TrickAdded" in notification.topic
assert b"roll over" in notification.state
assert notification.originator_id == dog_id

We can continue to select event notifications, by using the last event notification ID to calculate the next start value.

notifications = application.notification_log.select(
    start=notifications[-1].id + 1, limit=2
)

assert notifications[0].id == 3
assert notifications[1].id == 4

notification = notifications[0]
assert "Dog.TrickAdded" in notification.topic
assert b"fetch ball" in notification.state

notification = notifications[1]
assert "Dog.TrickAdded" in notification.topic
assert b"play dead" in notification.state

This method is used in a NotificationLogReader in a ProcessApplication to pull event notifications from upstream applications in an event-driven system, using the max_tracking_id() method of a ProcessRecorder to calculate the next start value from which to pull unprocessed event notifications. See the system and persistence module documentation for more information.

Linked list of sections¶

The notification log also presents linked sections of notification objects. The sections are instances of the library’s Section class.

A notification log section is identified by a section ID string that comprises two integers separated by a comma, for example "1,10". The first integer specifies the notification ID of the first event notification included in the section. The second integer specifies the notification ID of the second event notification included in the section. Sections are requested from the notification using the Python square bracket syntax, for example application.notification_log["1,10"].

The notification log will return a section that has no more than the requested number of event notifications. Sometimes there will be less event notifications in the recorded sequence of event notifications than are needed to fill the section, in which case less than the number of event notifications will be included in the returned section. On the other hand, there may be gaps in the recorded sequence of event notifications, in which case the last event notification included in the section may have a notification ID that is greater than that which was specified in the requested section ID.

A notification log section has an attribute section_id that has the section ID. The section ID value will represent the event notification ID of the first and the last event notification included in the section. If there are no event notifications, the section ID will be None.

A notification log section has an attribute items that has the list of notification objects included in the section.

A notification log section has an attribute next_id that has the section ID of the next section in the notification log. If the notification log section has less event notifications that were requested, the next_id value will be None.

In the example above, there are four domain events in the domain model, and so there are four notifications in the notification log.

section = application.notification_log["1,10"]

assert len(section.items) == 4
assert section.id == "1,4"
assert section.next_id is None

assert isinstance(section.items[0], Notification)
assert section.items[0].id == 1
assert section.items[1].id == 2
assert section.items[2].id == 3
assert section.items[3].id == 4

assert section.items[0].originator_id == dog_id
assert section.items[1].originator_id == dog_id
assert section.items[2].originator_id == dog_id
assert section.items[3].originator_id == dog_id

assert section.items[0].originator_version == 1
assert section.items[1].originator_version == 2
assert section.items[2].originator_version == 3
assert section.items[3].originator_version == 4

assert "Dog.Created" in section.items[0].topic
assert "Dog.TrickAdded" in section.items[1].topic
assert "Dog.TrickAdded" in section.items[2].topic
assert "Dog.TrickAdded" in section.items[3].topic

assert b"roll over" in section.items[1].state
assert b"fetch ball" in section.items[2].state
assert b"play dead" in section.items[3].state

A domain event can be reconstructed from an event notification by calling the application’s mapper method to_domain_event(). If the application is configured to encrypt stored events, the event notification will also be encrypted, but the mapper will decrypt the event notification.

domain_event = application.mapper.to_domain_event(section.items[0])
assert isinstance(domain_event, Dog.Created)
assert domain_event.originator_id == dog_id

domain_event = application.mapper.to_domain_event(section.items[3])
assert isinstance(domain_event, Dog.TrickAdded)
assert domain_event.originator_id == dog_id
assert domain_event.trick == "play dead"

Registering custom transcodings¶

The application serialises and deserialises domain events using a transcoder object. By default, the application will use the library’s default JSON transcoder. The library’s application base class registers transcodings for UUID, Decimal, and datetime objects.

If your domain model uses types of object that are not already supported by the transcoder, then custom transcodings for these objects will need to be implemented and registered with the application’s transcoder.

The application method register_transcodings() can be overridden or extended to register the transcodings required by your application.

For example, to define and register a Transcoding for the Python date class, define a class such as the DateAsISO class below, and extend the application register_transcodings() method by calling the super() method with the given transcoder argument, and then the transcoder’s register() method once for each of your custom transcodings.

from datetime import date
from typing import Union

from eventsourcing.persistence import Transcoder, Transcoding


class MyApplication(Application):
    def register_transcodings(self, transcoder: Transcoder):
        super().register_transcodings(transcoder)
        transcoder.register(DateAsISO())


class DateAsISO(Transcoding):
    type = date
    name = "date_iso"

    def encode(self, o: date) -> str:
        return o.isoformat()

    def decode(self, d: Union[str, dict]) -> date:
        assert isinstance(d, str)
        return date.fromisoformat(d)

Saving multiple aggregates¶

In some cases, it is necessary to save more than one aggregate in the same atomic transaction. The persistence modules included in this library all support atomic recording of aggregate events into more than one aggregate sequence. However, not all databases can support this, and so this isn’t allowed on the library extensions that adapt these databases.

The example below continues the example from the discussion of namespaced IDs in the domain module documentation. The aggregate classes Page and Index are defined in that section.

We can define a simple wiki application, which creates named pages. Pages can be retrieved by name. Names can be changed and the pages can be retrieved by the new name.

class Wiki(Application):
    def create_page(self, name: str, body: str) -> None:
        page = Page.create(name, body)
        index = Index.create(page)
        self.save(page, index)

    def rename_page(self, name: str, new_name: str) -> None:
        page = self.get_page(name)
        page.update_name(new_name)
        index = Index.create(page)
        self.save(page, index)
        return page.body

    def get_page(self, name: str) -> Page:
        index_id = Index.create_id(name)
        index = self.repository.get(index_id)
        page_id = index.ref
        return self.repository.get(page_id)

Now let’s construct the application object and create a new page (with a deliberate spelling mistake).

wiki = Wiki()

wiki.create_page(name="Erth", body="Lorem ipsum...")

We can use the page name to retrieve the body of the page.

assert wiki.get_page(name="Erth").body == "Lorem ipsum..."

We can also update the name of the page, and then retrieve the page using the new name.

wiki.rename_page(name="Erth", new_name="Earth")

assert wiki.get_page(name="Earth").body == "Lorem ipsum..."

The uniqueness constraint on the recording of stored domain event objects combined with the atomicity of recording domain events means that name collisions in the index will result in the wiki not being updated.

from eventsourcing.persistence import RecordConflictError

# Can't create another page using an existing name.
try:
    wiki.create_page(name="Earth", body="Neque porro quisquam...")
except RecordConflictError:
    pass
else:
    raise AssertionError("RecordConflictError not raised")

assert wiki.get_page(name="Earth").body == "Lorem ipsum..."

# Can't rename another page to an existing name.
wiki.create_page(name="Mars", body="Neque porro quisquam...")
try:
    wiki.rename_page(name="Mars", new_name="Earth")
except RecordConflictError:
    pass
else:
    raise AssertionError("RecordConflictError not raised")

assert wiki.get_page(name="Earth").body == "Lorem ipsum..."
assert wiki.get_page(name="Mars").body == "Neque porro quisquam..."

A more refined implementation might release old index objects when page names are changed so that they can be reused by other pages, or update the old index to point to the new index, so that redirects can be implemented. See the Wiki application example to see how this can be done.

Using index aggregates, or aggregates with version-5 UUIDs is one way to discover version-4 aggregate IDs. By knowing the name, the IDs can be retrieved. A more powerful alternative is to implement a custom table of current values that can be used to index a collection of aggregates, but this requires a little bit more work. Another alternative is to have an aggregate that is a collection of many version-4 aggregate IDs, or even other attribute values, that can be used to the same effect. But this approach has limits because there may be too many aggregate IDs to contain in a single collection aggregate without suffering from performance issues. Another way that avoids the limits of collecting version-4 aggregate IDs in an event-sourced aggregate identified with a version-5 UUID is to use an event-sourced log.

Event-sourced log¶

As we have seen, event-sourced aggregates generate a sequence of events. The stored sequence of events can be retrieved and projected to reconstruct the current state of the aggregate. However, the mechanism for storing and retrieving events in a sequence is useful in itself. We can usefully log a sequence of events, and use this sequence without projecting the entire sequence into the current state of an aggregate. The term ‘event-sourced log’ refers to using the persistence mechanism for event sourcing without aggregate projections for the purpose of logging information that is useful in an application.

An event-sourced log object can be constructed as a part of an application. The library class EventSourcedLog can be used as an event-sourced log. An event-sourced log object is typically constructed with a version-5 UUID that identifies the sequence of logged events, and a subclass of DomainEvent defined with attributes for the information to be logged. A “trigger” method can be defined that expects to be given the kind of information to be logged. The “trigger” method constructs and returns a new event object of the given type with the given information. The event can then be saved atomically with other things, by passing it to the application’s save() method. A subsequence of logged events, of a given size and from a given position, can be selected from log. The logged information can be used directly.

This technique can be used, for example, to record and later discover the version-4 UUIDs of a particular type of aggregate. In the example below, aggregate IDs of newly created aggregates are logged, so that the aggregate IDs of stored aggregates can be discovered. The aggregate IDs are then accessed in pages of a fixed size.

from eventsourcing.application import EventSourcedLog
from eventsourcing.domain import DomainEvent


class LoggedID(DomainEvent):
    aggregate_id: UUID


class MyApplication(Application):
    def __init__(self, env=None) -> None:
        super().__init__(env=env)
        self.aggregate_log = EventSourcedLog(
            events=self.events,
            originator_id=uuid5(NAMESPACE_URL, "/aggregates"),
            logged_cls=LoggedID,
        )

    def create_aggregate(self):
        aggregate = Aggregate()
        log_event = self.aggregate_log.trigger_event(
            aggregate_id=aggregate.id
        )
        self.save(aggregate, log_event)
        return aggregate.id


app = MyApplication()

# Get last created aggregate ID.
assert app.aggregate_log.get_last() == None

aggregate1_id = app.create_aggregate()
last = app.aggregate_log.get_last()
assert last.aggregate_id == aggregate1_id

aggregate2_id = app.create_aggregate()
last = app.aggregate_log.get_last()
assert last.aggregate_id == aggregate2_id

# Get all created aggregate ID.
aggregate_ids = [i.aggregate_id for i in app.aggregate_log.get()]
assert aggregate_ids == [aggregate1_id, aggregate2_id]

aggregate_ids = [i.aggregate_id for i in app.aggregate_log.get(desc=True)]
assert aggregate_ids == [aggregate2_id, aggregate1_id]

# Get ascending pages of aggregate IDs.
log_events = list(app.aggregate_log.get(limit=1))
aggregate_ids = [i.aggregate_id for i in log_events]
assert aggregate_ids == [aggregate1_id]

next = log_events[-1].originator_version
log_events = app.aggregate_log.get(limit=1, gt=next)
aggregate_ids = [i.aggregate_id for i in log_events]
assert aggregate_ids == [aggregate2_id]

# Get descending pages of aggregate IDs.
log_events = list(app.aggregate_log.get(desc=True, limit=1))
aggregate_ids = [i.aggregate_id for i in log_events]
assert aggregate_ids == [aggregate2_id]

next = log_events[-1].originator_version - 1
log_events = app.aggregate_log.get(desc=True, limit=1, lte=next)
aggregate_ids = [i.aggregate_id for i in log_events]
assert aggregate_ids == [aggregate1_id]

This technique can be used to implement “list-detail” views common in non-event-sourced CRUD or ORM-based domain models in which the list is ordered by the aggregates’ created_on attribute (ascending or descending).

To order aggregates by another attribute, such as their modified_on value, or a custom attribute, it will be necessary somehow to define and maintain a custom index of the current values. Similarly, a custom index will need to be maintained if aggregates are to be discarded, since it isn’t possible to delete events from the log.

Application configuration¶

An application can be configured using environment variables. You can set the application’s environment either on the env attribute of the application class, in the operating system environment, or by passing them into the application using the constructor argument env. You can use all three ways for configuring an application in combination.

import os

# Configure by setting class attribute.
class MyApplication(Application):
    env = {"SETTING_A": "1", "SETTING_B": "1", "SETTING_C": "1"}

# Configure by setting operating system environment.
os.environ["SETTING_B"] = "2"
os.environ["SETTING_C"] = "2"

# Configure by setting constructor argument.
app = MyApplication(env={"SETTING_C": "3"})

The order of precedence is: constructor argument, operating system, class attribute. This means a constructor argument setting will override both an operating system and a class attribute setting. And an operating system setting will override a class attribute setting.

assert app.env["SETTING_A"] == "1"
assert app.env["SETTING_B"] == "2"
assert app.env["SETTING_C"] == "3"

The resulting settings can be seen on the env attribute of the application object. In the example above, we can see that the settings from the construct argument have overridden the settings from the operating system environment, and the settings from the operating system environment have overridden the settings from the class attribute.

Please note, all values are expected to be strings, as would be the case if the values are set in the actual operating system process environment.

Configuring aggregate caching¶

To enable caching of aggregates in an application repository, set AGGREGATE_CACHE_MAXSIZE in the application environment to a string representing an integer value. A positive integer value such as '1000' will enable a “least recently used” cache, in which the least recently used aggregate will be evicted from the cache when it is full. A value of '0' will enable an unlimited cache. The default is for aggregate caching not to be enabled.

When getting an aggregate that is not found in the cache, the aggregate will be reconstructed from stored events, and then placed in the cache. When getting an aggregate that is found in the cache, it will be “deep copied” to prevent the cache being corrupted with partial or aborted changes made by an application command method. After a mutated aggregate is successfully saved, the cache is updated by replacing the old state of the aggregate with the new.

To avoid an application getting stale aggregates from a repository when aggregate caching is enabled (that is, aggregates for which subsequent events have been stored outside of the application instance) cached aggregates are “fast-forwarded” with any subsequent events. This involves querying for subsequent events, and updating the state of the aggregate.

To disable fast-forwarding of cached aggregates, set AGGREGATE_CACHE_FASTFORWARD in the application environment to a false value as interpreted by strtobool(), that is one of 'n', 'no', 'f', 'false', 'off', or '0'. Only do this when only one instance of the application is being used to evolve the state of the aggregates, because integrity errors will certainly occur when attempting to evolve the state of stale aggregates. The default is for fast-forwarding to be enabled when aggregate caching is enabled. But in cases where there is only one application instance, querying for new events is unnecessary, and a greater performance improvement will be obtained by disabling fast-forwarding because querying for new events will be avoided.

Configuring persistence¶

By default, application objects will be configured to use the library’s “plain old Python objects” persistence module, and will store events in memory. The example above uses the application’s default persistence module. This is good for development, because it is the fastest option. But the state of the application will be lost when the application object is deleted. So it is not very good for production.

If you want the state of the application object to endure, you will need to use an alternative persistence module. To use alternative persistence infrastructure, set the variable PERSISTENCE_MODULE in the application environment. Using an alternative persistence module may involve setting further environment variables, perhaps to configure access to a real database, such as a database name, a user name, and a password.

For example, to use the library’s SQLite persistence module, set PERSISTENCE_MODULE to the value "eventsourcing.sqlite". The environment variable SQLITE_DBNAME must also be set. This value will be passed to Python’s sqlite3.connect() function.

from tempfile import NamedTemporaryFile

tmpfile = NamedTemporaryFile(suffix="_eventsourcing_test.db")

os.environ["PERSISTENCE_MODULE"] = "eventsourcing.sqlite"
os.environ["SQLITE_DBNAME"] = tmpfile.name
application = DogSchool()

dog_id = application.register_dog()

application.add_trick(dog_id, "roll over")
application.add_trick(dog_id, "fetch ball")
application.add_trick(dog_id, "play dead")

By using a file on disk, as we did in the example above, the state of the application will endure after the application object has been deleted and reconstructed.

del application

application = DogSchool()

tricks = application.get_tricks(dog_id)
assert tricks[0] == "roll over"
assert tricks[1] == "fetch ball"
assert tricks[2] == "play dead"

See SQLite module documentation for more information about using SQLite.

To use the library’s PostgreSQL persistence module, set PERSISTENCE_MODULE to the value "eventsourcing.postgres". See PostgreSQL module documentation for more information about using PostgreSQL.

Configuring compression¶

Compression is useful for reducing the total size of recorded application state, and for reducing transport time of domain events and domain event notifications across a network.

To enable compression, set the environment variable COMPRESSOR_TOPIC as the topic of a compressor class or module. The library’s ZlibCompressor can be used to compress stored domain events.

# Configure compressor topic.
os.environ["COMPRESSOR_TOPIC"] = "eventsourcing.compressor:ZlibCompressor"

Configuring encryption¶

Application-level encryption is useful for encrypting the state of the application “on the wire” and “at rest”.

To enable application-level encryption, set the environment variable CIPHER_TOPIC to be the topic of a cipher class.

The library’s AESCipher class can be used to encrypt stored domain events. When using the library’s AESCipher class, set environment variable CIPHER_KEY to be a valid encryption key. You can use the static method create_key() on the AESCipher class to generate a valid encryption key.

from eventsourcing.cipher import AESCipher

# Generate a cipher key (keep this safe).
cipher_key = AESCipher.create_key(num_bytes=32)

# Configure cipher topic.
os.environ["CIPHER_TOPIC"] = "eventsourcing.cipher:AESCipher"

# Configure cipher key.
os.environ["CIPHER_KEY"] = cipher_key

Snapshotting¶

If the reconstruction of an aggregate depends on obtaining and replaying a relatively large number of domain event objects, it can take a relatively long time to reconstruct the aggregate. Snapshotting aggregates can help to reduce the time it takes to reconstruct aggregates, and hence reduce the access time of aggregates with lots of domain events.

Snapshots are stored separately from the aggregate events. When snapshotting is enabled, the application object will have a snapshot store assigned to the attribute snapshots. By default, snapshotting is not enabled, and the application object’s snapshots attribute will have the value None.

assert application.snapshots is None

Enabling snapshotting¶

To enable snapshotting in application objects, the environment variable IS_SNAPSHOTTING_ENABLED may be set to a valid “true” value. The function strtobool() is used to interpret the value of this environment variable, so that strings "y", "yes", "t", "true", "on" and "1" are considered to be “true” values, and "n", "no", "f", "false", "off" and "0" are considered to be “false” values, and other values are considered to be invalid. The default is for an application’s snapshotting functionality not to be enabled.

application = DogSchool(env={"IS_SNAPSHOTTING_ENABLED": "y"})

assert application.snapshots is not None

Snapshotting can also be enabled for all instances of an application class by setting the boolean attribute ‘is_snapshotting_enabled’ on the application class.

class SnapshottingApplication(Application):
    is_snapshotting_enabled = True

application = SnapshottingApplication()
assert application.snapshots is not None

Taking snapshots¶

The application method take_snapshot() can be used to create a snapshot of the state of an aggregate. The ID of an aggregate to be snapshotted must be passed when calling this method. By passing in the ID (and optional version number), rather than an actual aggregate object, the risk of snapshotting a somehow “corrupted” aggregate object that does not represent the actually recorded state of the aggregate is avoided.

application = DogSchool(env={"IS_SNAPSHOTTING_ENABLED": "y"})
dog_id = application.register_dog()

application.add_trick(dog_id, "roll over")
application.add_trick(dog_id, "fetch ball")
application.add_trick(dog_id, "play dead")

application.take_snapshot(dog_id)

Snapshots are stored separately from the aggregate events, but snapshot objects are implemented as a kind of domain event, and snapshotting uses the same mechanism for storing snapshots as for storing aggregate events. When snapshotting is enabled, the application object attribute snapshots is an event store dedicated to storing snapshots. The snapshots can be retrieved from the snapshot store using the get() method. We can get the latest snapshot by selecting in descending order with a limit of 1.

snapshots = application.snapshots.get(dog_id, desc=True, limit=1)

snapshots = list(snapshots)
assert len(snapshots) == 1, len(snapshots)
snapshot = snapshots[0]

assert snapshot.originator_id == dog_id
assert snapshot.originator_version == 4

When snapshotting is enabled, the application repository looks for snapshots in this way. If a snapshot is found by the aggregate repository when retrieving an aggregate, then only the snapshot and subsequent aggregate events will be retrieved and used to reconstruct the state of the aggregate.

Automatic snapshotting¶

Automatic snapshotting of aggregates at regular intervals can be enabled for an application class be setting the class attribute ‘snapshotting_intervals’. The ‘snapshotting_intervals’ should be a mapping of aggregate classes to integers which represent the snapshotting interval. Setting this attribute implicitly enables snapshotting as described above. When aggregates are saved, snapshots will be taken if the version of aggregate coincides with the specified interval.

The example extends the DogSchool application class and specifies that Dog aggregates are to be automatically snapshotted every 2 events. In practice, a suitable interval would most likely be larger than 2, perhaps more like 100.

class DogSchoolWithAutomaticSnapshotting(DogSchool):
    snapshotting_intervals = {Dog: 2}


application = DogSchoolWithAutomaticSnapshotting()

dog_id = application.register_dog()

application.add_trick(dog_id, "roll over")
application.add_trick(dog_id, "fetch ball")
application.add_trick(dog_id, "play dead")

snapshots = application.snapshots.get(dog_id)
snapshots = list(snapshots)

assert len(snapshots) == 2

assert snapshots[0].originator_id == dog_id
assert snapshots[0].originator_version == 2

assert snapshots[1].originator_id == dog_id
assert snapshots[1].originator_version == 4

Classes¶

eventsourcing.application.project_aggregate(aggregate: TMutableOrImmutableAggregate | None, domain_events: Iterable[DomainEventProtocol]) → TMutableOrImmutableAggregate | None[source]¶: Projector function for aggregate projections, which works by successively calling aggregate mutator function mutate() on each of the given list of domain events in turn.

class eventsourcing.application.Cache[source]¶

Bases: Generic[S, T]

__init__() → None[source]¶

class eventsourcing.application.LRUCache(maxsize: int)[source]¶

Bases: Cache[S, T]

Size limited caching that tracks accesses by recency.

This is basically copied from functools.lru_cache. But we need to know when there was a cache hit, so we can fast-forward the aggregate with new stored events.

__init__(maxsize: int)[source]¶

class eventsourcing.application.Repository(event_store: EventStore, *, snapshot_store: EventStore | None = None, cache_maxsize: int | None = None, fastforward: bool = True, fastforward_skipping: bool = False, deepcopy_from_cache: bool = True)[source]¶

Bases: object

Reconstructs aggregates from events in an EventStore, possibly using snapshot store to avoid replaying all events.

__init__(event_store: EventStore, *, snapshot_store: EventStore | None = None, cache_maxsize: int | None = None, fastforward: bool = True, fastforward_skipping: bool = False, deepcopy_from_cache: bool = True)[source]¶: Initialises repository with given event store (an EventStore for aggregate AggregateEvent objects) and optionally a snapshot store (an EventStore for aggregate Snapshot objects).

get(aggregate_id: UUID, *, version: int | None = None, projector_func: ProjectorFunction[TMutableOrImmutableAggregate, TDomainEvent] = <function project_aggregate>, fastforward_skipping: bool = False, deepcopy_from_cache: bool = True) → TMutableOrImmutableAggregate[source]¶: Reconstructs an Aggregate for a given ID from stored events, optionally at a particular version.

__contains__(item: UUID) → bool[source]¶: Tests to see if an aggregate exists in the repository.

class eventsourcing.application.Section(id: str | None, items: List[Notification], next_id: str | None)[source]¶

Bases: object

Frozen dataclass that represents a section from a NotificationLog. The items attribute contains a list of Notification objects. The id attribute is the section ID, two integers separated by a comma that described the first and last notification ID that are included in the section. The next_id attribute describes the section ID of the next section, and will be set if the section contains as many notifications as were requested.

Constructor arguments:

Parameters:

id (Optional[str]) – section ID of this section e.g. “1,10”
items (List[Notification]) – a list of event notifications
next_id (Optional[str]) – section ID of the following section

__delattr__(name)¶: Implement delattr(self, name).

__eq__(other)¶: Return self==value.

__hash__()¶: Return hash(self).

__init__(id: str | None, items: List[Notification], next_id: str | None) → None¶

__repr__()¶: Return repr(self).

__setattr__(name, value)¶: Implement setattr(self, name, value).

class eventsourcing.application.NotificationLog[source]¶

Bases: ABC

Abstract base class for notification logs.

abstract __getitem__(section_id: str) → Section[source]¶: Returns a Section of Notification objects from the notification log.

abstract select(start: int, limit: int, stop: int | None = None, topics: Sequence[str] = ()) → List[Notification][source]¶: Returns a selection of Notification objects from the notification log.

class eventsourcing.application.LocalNotificationLog(recorder: ApplicationRecorder, section_size: int = 10)[source]¶

Bases: NotificationLog

Notification log that presents sections of event notifications retrieved from an ApplicationRecorder.

__init__(recorder: ApplicationRecorder, section_size: int = 10)[source]¶

Initialises a local notification object with given ApplicationRecorder and an optional section size.

Constructor arguments:

Parameters:

recorder (ApplicationRecorder) – application recorder from which event notifications will be selected
section_size (int) – number of notifications to include in a section

__getitem__(requested_section_id: str) → Section[source]¶: Returns a Section of event notifications based on the requested section ID. The section ID of the returned section will describe the event notifications that are actually contained in the returned section, and may vary from the requested section ID if there are fewer notifications in the recorder than were requested, or if there are gaps in the sequence of recorded event notification.

select(start: int, limit: int, stop: int | None = None, topics: Sequence[str] = ()) → List[Notification][source]¶: Returns a selection of Notification objects from the notification log.

class eventsourcing.application.ProcessingEvent(tracking: Tracking | None = None)[source]¶

Bases: object

Keeps together a Tracking object, which represents the position of a domain event notification in the notification log of a particular application, and the new domain events that result from processing that notification.

__init__(tracking: Tracking | None = None)[source]¶: Initialises the process event with the given tracking object.

collect_events(*objs: ImmutableAggregateProtocol | MutableAggregateProtocol | DomainEventProtocol | None, **kwargs: Any) → None[source]¶: Collects pending domain events from the given aggregate.

class eventsourcing.application.Application(env: Mapping[str, str] | None = None)[source]¶

Bases: object

Base class for event-sourced applications.

snapshot_class¶: alias of Snapshot

classmethod __init_subclass__(**kwargs: Any) → None[source]¶

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__init__(env: Mapping[str, str] | None = None) → None[source]¶: Initialises an application with an InfrastructureFactory, a Mapper, an ApplicationRecorder, an EventStore, a Repository, and a LocalNotificationLog.

property repository: Repository¶: An application’s repository reconstructs aggregates from stored events.

property notification_log: LocalNotificationLog¶: An application’s notification log presents all the aggregate events of an application in the order they were recorded as a sequence of event notifications.

construct_env(name: str, env: Mapping[str, str] | None = None) → Environment[source]¶: Constructs environment from which application will be configured.

construct_factory(env: Environment) → InfrastructureFactory[source]¶: Constructs an InfrastructureFactory for use by the application.

construct_mapper() → Mapper[source]¶: Constructs a Mapper for use by the application.

construct_transcoder() → Transcoder[source]¶: Constructs a Transcoder for use by the application.

register_transcodings(transcoder: Transcoder) → None[source]¶: Registers Transcoding objects on given JSONTranscoder.

construct_recorder() → ApplicationRecorder[source]¶: Constructs an ApplicationRecorder for use by the application.

construct_event_store() → EventStore[source]¶: Constructs an EventStore for use by the application to store and retrieve aggregate AggregateEvent objects.

construct_snapshot_store() → EventStore[source]¶: Constructs an EventStore for use by the application to store and retrieve aggregate Snapshot objects.

construct_repository() → Repository[source]¶: Constructs a Repository for use by the application.

construct_notification_log() → LocalNotificationLog[source]¶: Constructs a LocalNotificationLog for use by the application.

save(*objs: ImmutableAggregateProtocol | MutableAggregateProtocol | DomainEventProtocol | None, **kwargs: Any) → List[Recording][source]¶: Collects pending events from given aggregates and puts them in the application’s event store.

take_snapshot(aggregate_id: UUID, version: int | None = None, projector_func: ProjectorFunction[TMutableOrImmutableAggregate, TDomainEvent] = <function project_aggregate>) → None[source]¶: Takes a snapshot of the recorded state of the aggregate, and puts the snapshot in the snapshot store.

notify(new_events: List[DomainEventProtocol]) → None[source]¶

Deprecated.

Called after new aggregate events have been saved. This method on this class doesn’t actually do anything, but this method may be implemented by subclasses that need to take action when new domain events have been saved.

exception eventsourcing.application.AggregateNotFoundError[source]¶

Bases: EventSourcingError

Raised when an Aggregate object is not found in a Repository.

class eventsourcing.application.EventSourcedLog(events: EventStore, originator_id: UUID, logged_cls: Type[TDomainEvent])[source]¶

Bases: Generic[TDomainEvent]

Constructs a sequence of domain events, like an aggregate. But unlike an aggregate the events can be triggered and selected for use in an application without reconstructing a current state from all the events.

This allows an indefinitely long sequence of events to be generated and used without the practical restrictions of projecting the events into a current state before they can be used, which is useful e.g. for logging and progressively discovering all the aggregate IDs of a particular type in an application.

__init__(events: EventStore, originator_id: UUID, logged_cls: Type[TDomainEvent])[source]¶

trigger_event(next_originator_version: int | None = None, **kwargs: Any) → TDomainEvent[source]¶: Constructs and returns a new log event.

get_first() → TDomainEvent | None[source]¶: Selects the first logged event.

get_last() → TDomainEvent | None[source]¶: Selects the last logged event.

get(*, gt: int | None = None, lte: int | None = None, desc: bool = False, limit: int | None = None) → Iterator[TDomainEvent][source]¶: Selects a range of logged events with limit, with ascending or descending order.

class eventsourcing.compressor.ZlibCompressor[source]¶

Bases: Compressor

compress(data: bytes) → bytes[source]¶: Compress bytes using zlib.

decompress(data: bytes) → bytes[source]¶: Decompress bytes using zlib.

application — Applications¶

Domain-driven design¶

Application objects¶

Simple example¶

Repository¶

Notification log¶

Outbox pattern¶

Selecting notifications from the log¶

Linked list of sections¶

Registering custom transcodings¶

Saving multiple aggregates¶

Event-sourced log¶

Application configuration¶

Configuring aggregate caching¶

Configuring persistence¶

Configuring compression¶

Configuring encryption¶

Snapshotting¶

Enabling snapshotting¶

Taking snapshots¶

Automatic snapshotting¶

Classes¶

`application` — Applications¶