How to choose a systems integration approach

Article August 22, 2022

In this blog we provide you an overview of three system integration approaches and point out the benefits and drawbacks of the approaches. Integrating systems directly, also known as Point to Point, is compared with using a dedicated integration platform to connect systems. We also introduce an integration approach known as API-Led or a 3-tier API architecture.

Integration patterns

Three high-level application integration patterns have emerged over the past few decades. These three patterns hold true if we are talking about Web Services, REST, Database Connectivity, File Transfers, or Messaging systems like Rabbit or Kafka.

The patterns are Direct Point-to-Point Connections, using an ESB or an Integration Platform as an intermediary, and most recently, 3-Tier API Led. Let’s discuss what this means.

Direct connections between systems (point-to-point)

This approach involves inserting code into a system, initiating the integration by calling an API, accessing a database, producing an import/export file, or publishing an event. On top of this connectivity effort, things like data transformation, exception handling, and handling multithreading to provide scalability needs to be considered and built into the integration.

This effort needs to be duplicated in the integrated target system. This system implements the API that is called, reads the file created, or consumes the published event. Once again, data transformation, exception handling, and scalability need to be built into the receiving end of the integration. Moreover, security is also a concern for the integration of the system on the receiving side.

Direct Connectivity

When a pattern of direct connectivity between systems is used to integrate applications, it results in an exponential growth in the number of interconnections that need to be built and maintained individually.

The fastest to implement if the integration is simple
No learning curve outside of the knowledge of your existing Applications
Works well for File Exports and Imports
It may work when calling simple REST APIs

Cons of Direct Connection

Very hard for developers not familiar with the current systems to find or update the integration code
New integrations are done by copying/pasting existing code (if it can be found)
Data transformation is cumbersome in a procedural language
Complex scaffolding code is required to manage connections and security
Scalability is usually very limited (single threads, small result sets)

Point-to-Point Flows in an Integration Platform as a Service

This approach involves creating a single flow in an iPaaS to act as a pipe between an object or data source inside the system initiating the integration and an object in the target system. This moves aspects like system connectors, data transformation, process orchestration, threading, and exception handling into an integration platform designed specifically to solve these problems. However, implementing the end-to-end connectivity in a single flow results in initiator authentication, target system security, data transformation, and process orchestration combined in a single deployable component. This makes reuse nearly impossible to achieve in a realistic scenario. Teams using this overall pattern in ESB and SOA projects were more productive and efficient than teams directly connecting applications together, but integration service and API reuse continued below.

Point-to-Point Flows

Point-to-Point Flows Over Time

When a pattern of building individual flows to handle connectivity between two systems is used to integrate applications, it results in an exponential growth in the number of interconnections that need to be built and maintained individually. The complexity is the same as directly connecting applications, albeit with the advantage that all the integration complexity is located in the same integration platform instead of being scattered across multiple applications.

Pros of point-to-point flows

Tools are designed for Integration use cases
iPaaS platforms have drag-and-drop to speed up development
Code written is limited to Transformation Logic
No boilerplate scaffolding code required
Faster for medium and high complexity requirements
Easier for future developers to find and update integration code

Cons of point-to-point flows

The potential learning curve to get producing in iPaas
This pattern is essentially implementing Point to Point inside a tool. The iPaaS is simply hiding this fact
Reuse is very hard to accomplish because Integrations are hardwired to connect specific systems together.

API-led connectivity

API-led connectivity is an approach to integrating systems by building and reusing APIs. A fundamental tenet is to design the APIs following a specific pattern that groups the APIs into up to 3 types of layers. Generally speaking, all APIs in the design are implemented as a group of synchronous or asynchronous REST APIs. However, this pattern can also support database triggers, events, or scheduled batch processes. API-Led Connectivity has been proven to have higher reuse levels than creating a single API that is implemented as a direct connection between two systems, as described in the Point-to-Point Flow section above.

System layer: The System API Layer is the simplest to understand, but maybe the most complex to implement. A system API (SAPI) is designed to expose back-end applications and simplify connectivity, authentication, exception handling, retry mechanisms, and data translation from a native format (XML, CSV, Binary, SQL Resultset etc.) to JSON (or perhaps someday another common format).

Process layer: The Process API Layer is responsible for implementing processor data orchestration logic. In the case of a business transaction that involves coordinating updates to multiple objects, the transaction would be managed at the Process Layer. Any business rules for formulas should also be implemented at the process layer. For an API that calculates a tax value based on the cost of goods, the price should come from the system layer, and the execution of the formula ( for example, Tax = 5% of the cost) should occur in a process API (PAPI). The Process layer can be omitted in simple scenarios that don’t have business orchestration, transaction management, or custom business logic.

Experience layer: The Experience Layer acts as a proxy designed to give the initiating system the easiest and lowest friction API to access Process or Systems APIs that sit lower in the architecture. Its authentication method should sync with the system initiating the integration that will be calling it, and its data format should also match the initiator. This avoids custom-coding data transformation and translation inside the consuming application. Although reuse is a goal of API-Led, the reuse is not expected to be found at the Experience Layer. Each system that initiates an integration will generally have its own Experience Layer API (XAPI).

One way of thinking about API-Led is that the integration team focuses on building reusable API products that expose systems and processes instead of building bespoke pipes that directly connect systems together.

3 Tier API-Led (Building a layered set of Composable APIs that work together to integrate systems)

If no orchestration is required, omit the process layer.

Over time, the API-led approach results in an “API network” forming, which results in the reuse of the system and process layers.

Pros of API-Led Connectivity

The focus is on exposing IT products, not building pipes.
Microservice-based architecture
Supports Domain-Driven Design
Orchestration, transformation, security, data access, event processing, and connectivity details are separated into discrete layers.
Reuse is a proven outcome
Easy to find the APIs in a Portal
Easy to secure the APIs

Cons of API-Led connectivity

It can take longer to implement, especially when you are new to API-Led
More computing resources are used (especially with CloudHub)
More network hops can increase latency

API-Led case study

Organizing APIs into multiple standard tiers will allow developers to find and reuse the created APIs. Separating the Experience Layer, the Process Layer, and the System Layer will enable IT teams to move from building bespoke Pipes between systems to building reusable APIs that multiple systems can use over time.

Need Salesforce support? Let's talk.

Our contact form is currently blocked by your cookie preferences. Please change your preferences to continue.