---

Preface

Event traceability is an event centered method of writing business logic and persisting domain objects . Event traceability can eliminate some possible programming errors , Because this technology can ensure that events will be published when aggregations are created or updated .

This is a book about microservice architecture design , Here are my study notes . Some symbols are explained below ：

（） To supplement , Usually the content in books ;
[] The symbol is the author's note ;

---

1. Overview of using event traceability to develop business logic

Event traceability pattern ： Use a series of domain events called state changes to persist the aggregation .

1.1 The problem of traditional Persistence Technology

• Of objects and relationships “ Impedance offset ”： Table structure pattern of relational data , And the graphical structure of domain model and its complex relationship , There is a basic concept mismatch problem ;
• Lack of aggregation history ： After aggregate update , Its previous status will be lost ;
• Implementing the upgrade function will be cumbersome and error prone ： Time consuming , The code responsible for recording the audit log may deviate from the business logic code ;
• Event publishing is above business logic ： Cannot automatically publish messages as part of an update data transaction ;

1.2 Event traceability persistent aggregation through events

Event traceability adopts the concept of domain event to realize the persistence of aggregation ; It persists each aggregation into a series of events in the database , Called event storage .

The illustration ：

• Tracing events does not mean that every Order Stored as a row in ORDER In the table , Instead, each Order Aggregate persistence to EVENTS One or more rows in the table ;
• When an application creates or updates aggregations , It inserts the event emitted by the aggregation into EVENTS In the table ;
• The application loads the aggregate by retrieving and replaying events from the event store （ Such as Eventuate Client frame ）, To load an aggregate ：
  • Load aggregated Events ;
  • Create an aggregate instance using its default constructor ;
  • call apply() Method to traverse events ;
• Event traceability rebuilds the aggregated memory state by loading and replaying events ;

1.3 Event traceability is a new requirement for domain events

• Events represent changes in state ;
• Aggregation methods are event related ;

1.4 Events represent changes in state

• In case of event traceability , Aggregation mainly determines the event and its structure ;
• Every aggregation state change including creation , Are represented by domain events ;
• Whenever the state of aggregation changes , It must send an event ;
• The event must contain the data needed to aggregate the execution of state changes ;
• The state of the aggregation consists of the field values that make up the aggregation object ;

1.5 Aggregation methods are event related ;

• Command methods in event traceability based applications process requests for aggregate updates by generating events ;
• The result of calling the aggregate command method is a series of events , Indicates the status changes that must be made ;

• Generate events and apply （apply） Event approach will lead to refactoring of business logic ; Event traceability reconstructs command methods into two or more methods ;
  • The first method process() Receive command object parameters , This parameter represents the specific request , And determine what status changes you need to make ; It validates the parameters of the command object , And without changing the aggregation state , Returns a list of events representing a status change ; If the command cannot be executed , This method usually throws an exception ;
  • Other methods apply() All use a specific event type as a parameter to update the aggregate ; These methods correspond to the event types generated by aggregation one by one ; It is important to note that the execution of these methods will not fail , Because these events represent a state change that has occurred ; Each method updates the aggregation based on the event ;
  • An example is as follows ：

The illustration ：

• reviseOrder() Method is process() Methods and apply() Method substitution ;
• process() Methods will ReviseOrder Command as an argument ;
• process() Method either returns OrderRevisionProposed event , Or throw an exception ;
  • If the time is too late to modify the order or the proposed order revision does not meet the minimum value of the order ;
• OrderRevisionProposed The event apply() Methods will Order Is changed to REVISION_PENDING;

1.6 Steps to create and update aggregations

To create an aggregation ：

1. Instantiate the aggregation root using the default constructor of the aggregation ;
2. call process() To generate new events ;
3. Traverse the newly generated event and call apply() To update the status of the aggregation ;
4. Save the new event in the event Repository ;

Steps to update aggregation ：

1. Load aggregate events from the event Repository ;
2. Instantiate the aggregation root with its default constructor ;
3. Traverse the loaded Events , And call... On the aggregation root apply() Method ;
4. Call its process() Method to generate a new event ;
5. Traverse the newly generated event and call appply() To update the status of the aggregation ;
6. Save the new event in the event Repository ;

1.7 Event traceability based Order polymerization

• Business logic is implemented through commands , These commands issue events and apply events that update their status ;
• Create or update based on JPA Each method of aggregation , Such as createOrder() and reviseOrder(), In the event traceability version process() and apply() Method substitution ;

• be based on JPA The aggregated modify order business logic consists of three methods ：reviseOrder()、confirmRevision() and rejectRevision();
• The event traceability version uses three process() Methods and some apply() Method instead of these three methods ;

1.8 Use optimistic locks to handle concurrent updates

It refers to the situation where two or more requests update the same aggregation at the same time ;

• Optimistic locks usually use version Columns （ Mapping to VERSION Column ） To detect whether the aggregation has changed since reading ;
• Whenever an aggregate is updated ,VERSION The value of the column will increase ;
• Two things read the same aggregation , The first success , The second failed ; Because the version number has changed ;

1.9 Event traceability and release

• Use polling to publish Events ;
  • As shown in the figure below ：
    
• Use transaction log tailing technology to reliably publish Events ;
  • This article 2 Point explanation ;

1.10 Use snapshots to improve performance

Long life cycle aggregation may have a large number of events ; Over time , Loading and replaying these events will become more and more inefficient ; A common solution is to periodically persist snapshots of the aggregated state ;

1.11 Idempotent message processing

Call the message receiver safely multiple times with the same message , Then the message receiver is idempotent ; The specific implementation depends on whether the event repository is a relational database or NoSQL database ;

• Idempotent message processing based on relational database event repository ;
  • You can send a message ID Insert PROCESSED_MESSAGES surface , Insert as EVENTS Table events are part of the transaction [ When the same message is accepted , If it already exists in the database ID, Then ignore the message request ];
• Idempotent message processing based on non relational database event repository ;
  • Often with limited functionality , Different mechanisms need to be used to implement idempotent message processing ;
  • The message receiver must complete both event persistence and message logging in an atomic way ID;
  • Solution ： Message consumers put the message ID Stored in the event generated when processing it , It verifies whether the message is included in all events aggregated ID For repeated verification ;
  • Problems with this solution ： The processing of some messages may not generate any events ;
    • One solution ： Always post events ; If aggregation does not emit events , Then the application saves only logged messages ID Pseudo Events ; The event receiver must ignore these pseudo events ;

1.12 Evolution of domain events

The structure of the event traceability application is divided into three levels ：

• Consisting of one or more aggregates ;
• Define the events emitted by each aggregation ;
• Define the structure of the event ;

Different types of changes that can occur at each level ：

• The domain model of services evolves over time , These changes will happen naturally ;
• Changes to incompatible segments require consumers to change the event type ;

By switching up （Upcasting） To manage structural changes ：

• The event traceability framework does not migrate events to a new version , Instead, the transformation is performed when events are loaded from the event repository ;
• It is usually called “ Upward conversion ” The component of updates each event from the old version to the updated version ;

1.13 Advantages and disadvantages of event traceability

benefits ：

• Reliably publish domain events ;
• Keep the history of aggregation ;
• Avoid the of objects and relationships to the greatest extent “ Impedance offset ” problem （ Persisting events rather than aggregating itself ）;
• Provide developers with a “ Time machine ”;

disadvantages ：

• This kind of programming mode has a certain learning curve ;
• The complexity of messaging based applications ;
  • When dealing with non power events ;
  • resolvent ： Assign monotonically increasing... To each event ID;
• It's hard to deal with the evolution of events ;
  • The structure of events and snapshots becomes bloated over time ;
  • resolvent ： When loading events from the event Repository , Upgrade the event to the latest version ; Separate the event version processing from the aggregated code , Simplify aggregation ;
• There is some difficulty in deleting data ;
  • Europe's GDPR Give users the right to erase their data , User information may be embedded in the event structure , For example, the mailbox is used as the primary key of the aggregation , The application must clear specific user information without deleting Events ;
  • resolvent ： Use the encryption key ; Using pseudonym Technology （ Such as ：UUID Token instead of e-mail as aggregation ID）;
• Querying the event repository is very challenging ;
  • Refers to more complex and potentially inefficient queries that may use nesting ;
  • resolvent ： Reference resources 《 The first 7 Chapter CQRS Method to realize the query 》;

2. Implement event Repository

Applications that use event traceability store events in the event Repository ; The event repository is a combination of database and message broker capabilities ; It is represented as a database and a message broker ;

• There are several ways to implement an event Repository , One is to implement its own event repository and event traceability code framework ; The other is to use a dedicated event Repository ;
• A dedicated event repository usually provides a rich set of features 、 Better performance and scalability ;
• Such as ：Event Store、Lagom、Axon、Eventuate SaaS;

2.1 Eventuate Local How the event repository works

Eventuate Local Event database structure ：

• events： Store events （ At the core ）;
  • With this article 1.2 The graph of points is similar to ;
• entities： One line per entity ;
  • Store the current version of each entity ; Used to implement optimistic locks ;
• snapshots： Store snapshots ;
  • Store a snapshot of each entity ;
  • It supports find()、create()、update() Three actions ;

By subscribing Eventuate Local The event agent accepts events ：

• Services use events by subscribing to event proxies , The event broker has a topic for each aggregation type ;
• The topic is the message channel of the partition , Enables the receiver to expand horizontally while maintaining message ordering ;

Eventuate Local Event relay propagates events from the database to the message broker ：

• The event relay propagates the events inserted into the event database to the event agent ;
• It uses transaction log tailing whenever possible , Or poll other databases ;
• Event is deployed as a stand-alone process ;

2.2 in the light of Java Linguistic Eventuate Client The main classes and interfaces provided by the framework

Eventuate Client The framework enables developers to use Eventuate Local Event repository writes application programs based on event traceability ; It is for the development of aggregation based on event traceability 、 Services and event handlers provide the framework foundation ;

The illustration ：

• adopt ReflectiveMutableCommandProcessingAggregate Class defines aggregation ;
  • This class is the base class of aggregation , Is a generic class ;
  • There are two type parameters ： Concrete aggregate classes 、 Super class of aggregate command class ;
  • Use reflection to assign commands and events to process() and apply() Method ;
• Define aggregation commands ;
  • The aggregated command class must extend the aggregation specific base interface , The interface itself must be extended Command Interface ;
  • Such as ：Order The aggregated command extends Ordercommand;
• Define domain events ;
  • The aggregated event class must extend Event Interface , This is an identification interface without methods ;
• Use AggregateRepository Class creation 、 Find and update aggregations ;
  • This class is a generic class , The parameters it receives are the aggregate class and the aggregate base command class ;
  • Three overload methods are provided ：save() Create aggregations 、find() Find aggregation 、update() Update aggregation ;
  • Mainly used by services , Create and update aggregations when the service responds to external requests ;
• Subscribe to domain events ;
  • Eventuate Client The framework also provides tools for writing event handlers API, Such as ：
  • @EventSubscriber Annotation specifies the of the persistent subscriber ID;
  • @EventHandlerMethod The annotation will creditReserved() Method is identified as an event handler ;

3. Use at the same time Saga And trace the source of events

Event traceability can be easily used based on collaborative Saga; Business logic for event traceability and orchestration based Saga The combination is more challenging ;

3.1 Using event traceability to achieve collaborative Saga

• The event driven attribute of event traceability enables the implementation of Collaborative Saga It's simple ;
• When the aggregation is updated , It will send an event ; Different aggregated event handlers can accept this event , And update the aggregation ; The event traceability framework automatically makes each event handler idempotent ;
• Event traceability code provides Saga The mechanism required , Including interprocess communication based on message passing 、 The news is heavy , And atomization status update and message sending ;
• disadvantages ： The event embodies a dual purpose , That is, event traceability uses events to represent state changes , But use events to implement Saga synergy , If aggregation is required, events must be issued even if there is no state change ;
  • resolvent ： Use choreography to achieve complex Saga;

3.2 Create an orchestration Saga

Saga The choreographer is created by the method of the service , It will perform two operations: create and update aggregation , The service must ensure that the two operations are completed in the same thing ; Therefore, it depends on the type of event database used ;

When a relational database is used as an event Repository , How to create Saga Choreographer ：

• Relatively simple , Use @Transactional annotation , send Eventuate Local Framework and Eventuate Tram Saga The frame is in the same ACID Update the time repository in the transaction and create Saga The choreographer can ;

When a non relational database is used as an event Repository , How to create Saga Choreographer ：

• because NoSQL The transaction model function of database is limited , The application will not be able to create or update two different objects atomically ;
• The service must have an event handler , The event handler will create Saga Orchestrator to respond to domain events emitted by aggregation ;

3.3 Use event handlers to create Saga The case of choreographer

benefits ： Ensure loose coupling , because OrderService Such services are no longer explicitly instantiated Saga;

problem ： How to deal with repeated events to ensure idempotency , The solution is as follows ：

• Export... From the unique properties of the event Saga Of ID; There are many options , One is using aggregation that emits events ID As Saga Of ID, Applies to events created in response to aggregation Saga;
• （ It works ） Use the event ID As Saga ID; Because of the incident ID The only performance guarantee Saga ID It's the only one ;

3.4 Implement event traceability based Saga Participant

• Idempotent processing of imperative messages ;
  • It's easy to solve ：Saga The participant records the message in the event generated when processing the message ID; Before updating the aggregation ,Saga Participants look for messages in events ID To verify that it has processed the message before ;
• Send a reply message atomically ;
  • resolvent ： Give Way Saga Participants continue to Saga The reply channel of the choreographer sends a reply message ;
    • When Saga When the command handler creates or updates an aggregate , It will arrange to SagaReplyRequested Pseudo events are stored in the event repository together with the actual events emitted by the aggregation ;
    • SagaReplyRequested The event handler of the pseudo event uses the data contained in the event to construct the reply message , And then write it in Saga The reply channel of the choreographer ;
    • Such as 3.5 The point diagram shows ;

3.5 Event traceability based Saga Examples of participants

The image below shows Accounting Service How to deal with it Saga Sent Authorize Command;Accounting Service Use Eventuate Saga frame , This framework is used to write a program that uses event traceability Saga;

3.6 Implement event traceability based Saga Choreographer

• Use event traceability persistence Saga Choreographer ;
  • You can use the following event persistence Saga：
    • SagaOrchestratorCreated：Saga Choreographer created ;
    • SagaOrchestratorUpdated：Saga Choreographer updated ;
• Send imperative messages reliably ;
  • The key is how to update... Atomically Saga And send commands ;
  • about NoSQL Event Repository , The key is persistence SagaCommandEvent, It means to send a command ; Then the event handler subscribes SagaCommandEvents And send each imperative message to the appropriate channel ;
  • Please refer to the figure below for details ：

• Make sure that reply messages are processed only once ;
  • Similar to the mechanism described earlier ; The orchestrator will reply to the of the message ID Stored in the event sent when processing the reply ;

4. Summary of this chapter

• Event traceability preserves aggregation as a series of events . Each event represents the creation or state change of the aggregation . The application recreates the current state of the aggregation by replaying events . Event traceability preserves the history of domain objects , Provide accurate audit logs , And reliably publish domain events ;
• Snapshots improve performance by reducing the number of events that must be replayed ;
• Events are stored in the event Repository , The repository is a mixture of database and message broker . When the service saves events in the event Repository , It passes the event to the subscriber ;
• Eventuate Local It's based on MySQL and Apache Kafka Open source event Repository . Used by developers Eventuate Client Framework to write aggregation and event handlers ;
• One of the challenges of using event traceability is dealing with the evolution of events . An application may have to handle multiple event versions when replaying Events . A good solution is to use up conversion , When an event is loaded from the event Repository , It will upgrade the event to the latest version ;
• Deleting data in an event traceability application is tricky . The application must use technologies such as encryption and pseudonym , To comply with the EU GDPR And other regulations , Ensure that personal data is completely erased from the application ;
• Event traceability can be easily implemented based on coordination Saga. The service has event handlers , Used to listen to events published by aggregation based on event traceability ;
• We can also use event Traceability Technology to achieve Saga Choreographer . You can write applications that specifically use the event Repository ;

---

Last

New people make , If there is a mistake , Welcome to point out , Be deeply grateful ！ Welcome to the official account , Will share something more everyday ！ If you want to reprint , Please mark the source ！