What are Integration Patterns?

When designing an integration, it is worth considering the use of integration patterns.

Integration patterns are proven ways to solve a specific integration problem.

Just like architecture patterns or design patterns, they can help when designing a solution.

In the next part, I will present the most popular patterns that I have encountered during my professional work. These will be:

• request-response synchronous integration pattern,
• request-response asynchronous integration pattern,
• correlation identifier integration pattern,
• point-to-point integration pattern,
• publish-subscribe integration pattern,
• pipes and filters integration pattern,
• content-based router integration pattern,
• content enricher integration pattern,
• splitter integration pattern,
• aggregator integration pattern,
• message broker integration pattern,
• message bus integration pattern,
• canonical data model integration pattern.

Examples of integration patterns are presented in G. Hohpego i B. Woolfa publication and on the Enterprise Integration Patterns website.

Request-Response Synchronous Integration Pattern

The request-response synchronous integration pattern consists in the Consumer calling the function provided by the Supplier and waiting for a response. In this case, the Consumer waits for a response for a defined time limit (timeout). The request-response pattern is used when the Consumer must have the result of the function from the Supplier for further processing. This is an example of two-way integration.

An example of synchronous request-response calls is the use of SOAP operations or REST resources.

Request-Response Asynchronous Integration Pattern

The request-response asynchronous integration pattern consists in the Consumer transferring the request to the Supplier and not waiting for an immediate response. After the response has been prepared, it will be transferred from the Supplier to the Consumer.

We use this type of integration when the data preparation by the Supplier may take a long time, for example, due to complex calculations. This is an example of two-way integration. The request-response is transferred asynchronously. The pattern is an extension of one-way asynchronous integration.

An example of asynchronous request-response calls is the use of MQ queues.

Correlation Identifier Integration Pattern

In the asynchronous request-response integration, the message is transferred from the Consumer to the Supplier. The supplier prepares the result and asynchronously transfers the response. A correlation identifier is used so that the Consumer can determine which call does the transferred response concern. The correlation identifier integration pattern means to place a unique identifier in the request and then in the response to associate the request with the response.

The Consumer determines a message identifier and then submits it in the request. Message identifier must be unique. It can be generated using special UUID algorithms (Universally Unique ID) or by using the business fields from the request message, if they provide uniqueness.

Then the Supplier prepares the results. This is an asynchronous integration; therefore, the Consumer may perform other operations at the time when the Supplier is processing the transferred request. In response to the results prepared by the Supplier, a correlation identifier is added, which is the request message identifier. Then it is transferred to the Consumer. The correlation identifier will allow the Consumer to associate the request with the response.

Point-to-Point Integration Pattern

The point-to-point integration pattern consists in the direct integration of the Supplier and the Consumer. The Supplier transfers the message to the pipe. It is only received by one Consumer.

MQ queues can be used to implement point-to-point integration.

Publish-Subscribe Integration Pattern

The publish-subscribe integration pattern consists in integrating the Supplier with many Consumers. The Supplier transfers the message to the pipe to which the Consumers are subscribed. The same message is sent to many Consumers.

To implement the publish-subscribe integration we can use the topics in Apache Kafka.

Pipes and Filters Integration Pattern

The pipes and filters integration pattern consists in building a sequence of processors (filters) and connecting them using pipes. Instead of building a complex message-processing logic in one component, we can create a sequence of filters. Each filter has an input and output interface. All filters use the same interface definition. This ensures that filters can be combined with different pipes in any order. The connection between the filter and the pipe is called the port.

Content-Based Router Integration Pattern

The content-based router integration pattern consists in forwarding the message to the appropriate recipient based on the data from the message. Based on the content of the request, the router redirects the message to the appropriate recipients via pipes. Routing can be performed based on various criteria, such as verifying whether there is a specific field in the message or whether the field contains a specific value. A customizable rule engine can also be used to determine the recipient and destination pipe.

Content Enricher Integration Pattern

The Content Enricher integration pattern is used to extend the original message with the content required by the target component and unavailable in the source system. Missing information is extracted from other sources based on data provided by the source system. An example of using the content enricher pattern can be a use case where the source system transfers the address in the form of a postal code, while the target component also requires receiving the name of the province. For this purpose, the content enricher downloads information about the province based on a postal code from an external data source.

Splitter Integration Pattern

The splitter integration pattern is used to split the original message into multiple separate ones. The messages created within the division can be processed independently. When a message contains a placed order and needs to be processed by different systems, then a good solution is to use a splitter. It will enable the splitting of messages and independent processing in different systems.

Aggregator Integration Pattern

An aggregator integration pattern is a state-owned processor (filter) designed to collect and store individual messages to associate them. After receiving all the messages, the aggregator then completes the target message and transfers it to the Consumer. To complete the message, it is necessary to define an aggregation strategy. It is also necessary to specify the message’s correlation identifier (for example, the order identifier), the number of messages, or the timeout.

Message Broker Integration Pattern

The message broker integration pattern is to integrate Suppliers with Consumers using a central integration component. Suppliers transfer the message to one component responsible for transferring it to the relevant Consumer or Consumers. The message broker eliminates the problem of point-to-point integration between all Suppliers and Consumers. It can facilitate integration and establish uniform security rules. Message broker may be a bottleneck in the solution.

To implement the message broker, we can use Apache ActiveMQ or Apache Kafka.

Message Bus Integration Pattern

The message bus integration pattern consists in integrating Suppliers with Consumers using a central integration bus via shared interfaces. It allows to easily connect or disconnect the IT system through pipes and adapters. It uses a canonical data model that improves the management of the data model. The message bus eliminates the problem of point-to-point integration between all Suppliers and Consumers. It could be a bottleneck.

WebMethods Integration Server (webMethods ESB) or Mule ESB can be used to implement the message bus.

Canonical Data Model Integration Pattern

The canonical data model helps to simplify the integration of many systems by introducing a unified data model used in communication. Each system has its own internal data model. However, to integrate, it must communicate a message that conforms to a unified canonical model. Each system must perform a transformation between its internal data model and the canonical model. When a new system wants to integrate with any other, it must transform the message into a canonical data model.

Integration Architecture Principles

The integration architecture principles contain guidelines that we should follow when designing an integration. Organizations often define principles that should be considered when designing interfaces. They are called integration architecture principles or interface design principles. The principles may apply to security standards that should be applied, for example, by using secured protocols such as SSH, HTTPS, SFTP, etc., or authentication mechanisms (using X.509 certificates). They may contain requirements related to the naming of integration interfaces or the version of the integration protocol, for example, SOAP 1.2.

Thank you for reading the article. If you would like to share your comment with me, write to me at marcin@marcinziemek.com

If you want to develop your knowledge of IT systems integration design, check out my book available at: https://integration.marcinziemek.com