LET'S GET STARTED
Are you new to API integrations? Maybe you're in need of a refresher. Either
LET'S GET STARTED
Are you new to API integrations? Maybe you're in need of a refresher. Either
Your developers are superheroes, but that doesn’t mean we have to expect them to have superpowers. When it comes to integration, there is so much below the surface that most of us don’t think about on a day-by-day case.
Let's dive in.
Integration is a means to an end; a connected experience where data flows seamlessly between your app and the other apps your customers are using.
In order to build the best user experience, pre-integration work (defining user stories, knowing your use cases, and selecting integration endpoints) is
Product managers are responsible for deciding on which integrations to build into their application to create a seamless experience with the services and applications used by their customers. To guide the development of these integrations, it’s helpful to write integration user stories that provide a clear picture of the experience you envision for your end-customers.
User stories can be defined from two different perspectives: a system persona or a user persona. We recommend user
Let’s use Mary Marketing as an administrator user example. A Marketing Administrator selects and authenticates their marketing automation application (e.g., HubSpot, Marketo) so that leads are automatically created from the email addresses of webinar attendees. This is a one-time setup that doesn’t need to be in the direct flow of the application for the end-users - a ‘set it and forget it’ function.
User stories can be defined from two different perspectives:
a system persona or user persona.
|SYSTEM PERSONA||RECOMMENDED: USER PERSONA|
By developing user stories in the system persona, Product Managers can become too focused on the technical aspects of the integration instead of the collaborative user experience.
System based integration stories don’t enable the Product Manager to fully visualize the workflow that the user will experience; they guide the integration as a technical solution but not as effectively as a business solution.
A great user persona is written from the point-of-view of an end user. That is contrary to a system persona, designed to specify technical interactions between systems. It’s important to get as much use out of these personas as possible to ensure that your stories relate back to an
The Webinar Management Application will automatically create leads and associate them with accounts in the selected Marketing Automation System so that the two systems will stay in-sync.
As Mary Marketing, I can select the Marketing Automation Service that I would like to connect to my Webinar Application so that I can create leads from webinar attendees.
In this example, and as a theme throughout the rest of the page, we will be discussing API integration design for SaaS (cloud-based) applications to other cloud services. This is an external integration use case, app-to-app.
Next, you will want to think about optimizing your app by selecting the API endpoints and data needed to integrate with your app. Your goal is to provide a natural experience for your users to
In the HubSpot example, the integration end-points are selected by marketing or office administrators. When setting up the integrations, the app admins (or end users) can map their data objects from HubSpot to a marketing webinar platform, like GoToWebinar.
Speaking of data, you want your end users to do as little work as possible when activating an integration, it is best to pre-map the data that your application will exchange with the endpoint so that your users only need to account for custom fields or fields that are specific to that user’s endpoint. Your customers should only have to validate and slightly modify data mappings to suit their individual needs, don’t make them start from scratch which is prone to error frustration - your app should already have the base fields locked in.
The best integration experience makes end users
forget they're even connecting your app with another service.
The best integration experience makes end users forget they’re even connecting your app with another service. End-point selection and authentication should happen within your application with your brand requesting access to the end-point. You manage the experience for the user from end-to-end.
Don’t let your users leave your app!
The last piece
Now, you are ready to dive deeper and integrate!
Pre-integration work is complete! But wait… Once you are ready to integrate, there are initially more questions than answers. API Integration is about building pre-built integration into your application. However, it’s more complex than point-to-point connection, and it’s bigger than simply writing directly to the API.
This section will walk you through all of these areas and break down steps to help you swan-dive into integration.
Securely and reliably authenticating to the cloud service endpoints that connect with your app is the foundation for your integration experience. You’ll need to determine the type of authentication mechanism used by the endpoint and the workflow required.
Once a user is presented with the UI to select the endpoint they wish to connect to, your application will open an authentication screen. Users will use their own login credentials to access the service and grant access to connect the apps.
Start by identifying the authentication mechanism. The authentication mechanism drives the workflow for each instance an endpoint connects to your app. There are four primary types of authentication mechanisms you can use to connect your app to a cloud service endpoint: Basic Authentication, OAuth, OAuth 2, and SAML.
Next, determine how to manage refresh tokens and make sure they are securely stored. If you want your user to authenticate once and then not have to authenticate again as they interact with the endpoint, your application will need to manage the refresh token from the endpoint (if available) in addition to the initial access token. Some access tokens expire in an hour, while others may last as long as a year or never expire. (See a related blog for details on the duration of cloud storage refresh tokens).
Tokens are storing very private user information. As a developer for the integration between your application and an endpoint, it’s your responsibility to protect this type of sensitive information. If your application stores tokens it is highly recommended that you encrypt with 256-bit encryption at rest within your data storage system with the key owned by the end user. Encrypted tokens stored with 256-bit encryptions are really tough to break, protecting your application and your
Once you have a strategy for managing access and refresh tokens, you will need to figure out how to manage permissions from the endpoint. This all depends on how that endpoint is structured. As you’re designing your API integration, consider whether or not your client will be authenticating an administrator account or an end-user account. How will you handle and pass permissions from the endpoint? An admin account at the endpoint will allow you to manage objects and data across your organization while protecting client data with specified user’s permissions.Lastly, you will want to track and log activity for each instance for support, monitoring and security purposes. When you embed integrations into your application, your customers will expect you to support the integration. Your support team will need access to API call activity logs, error reports, and alerts to provide the success experience for your customers. Tagging the API calls with an identifier for each customer’s instance of an integration will make it easier for your team to manage and support the specific user.
When integrating to application endpoints such as CRM, Marketing Automation or Finance systems, your app will need to discover the standard and custom data objects and fields used by your customer’s instance of the endpoint. This can be done by following a 3-step process.
Identify which of your application’s data objects are relevant to the data you’re targeting at the endpoint(s). In most cases, you will only be integrating a subset of the data objects and fields from the endpoint. Your application’s data model will provide the basis for determining the data you’re interested in integrating with. You will then identify the standard objects at the endpoint that you will be integrating
Your integration will likely need an automated means to discover custom objects and fields at the endpoint if you plan to support the integration and mapping of custom data. Since the majority of SaaS application endpoints make extensive use of custom data fields and objects, you will likely need to discover the data structure at the specific instance of the endpoint in addition to understanding the standard data objects. A standard object will persist across multiple instances of an endpoint, but custom objects and data fields will only exist for a specific instance of an endpoint.
The beauty of taking the extra effort to discover data objects your app is creating is that once you’ve collected enough data and evidence, you can start to figure out patterns your customers create. Those patterns are what you can use to automate the mapping and transformation process. From those
If you’re providing a Dynamic integration experience for your users, you’ll want to consider how your users will be able to map data fields and transform data values. With a UI for data mapping, you will enhance the experience to seamlessly synchronize applications.
A default mapping is a template that will define how standard data structures from an endpoint will map into your applications’ data model. This mapping establishes the association of standard fields within a standard object into the fields within your application. The primary goal here is to save your users’ time by pre-mapping standard data from the endpoint reducing the effort they need to undertake when connecting to your app.
Once your app Discovers the data structure at your
Custom data objects and custom fields within standard objects are a common feature of most SaaS applications. Invariably your customers will make extensive use of these capabilities. Therefore default mapping will only be the beginning for a majority of your customers. The challenge here is that you’re going to need to support unique data maps at the “Instance” level. The instance level data maps will be specific to that authenticated account of an endpoint.
There is no way to anticipate the type of data coming in, as it only exists specific to that “instance” of a service. Custom mappings can be done by an administrator or an end-user at your customer with a User Interface embedded within your app. The most dynamic user experience is to provide them full self-service control over these mappings, as their data structures will change regularly and it will reduce time and cost for you and your customer to make them self-sufficient.
There is no way to always anticipate how your users may use standard fields. Occasionally, users will use standard fields for a custom purpose and they will likely want to map this data to a field other than your default mapping. Just as you handle Custom Data on a dynamic self-service basis, it’s best to enable your users to override your default settings.
As discussed previously the optimal experience for your users is to have a user interface integrated with your app to facilitate the mapping of data from their cloud service apps into your data structure. By providing this facility, your customers can make their own mapping using their best
Mapping data is one piece. You have invested time and money into configuring the data to flow into your app in a very specific way. That doesn’t mean that the other apps that are on your integration roadmap have the data formatted in the same manner as you do. The next important step is data transformation. This is the processing of data from one format, such as XML or SOAP, or even currencies and time formats, into another.
That doesn't mean that the other apps that are on your integration
roadmap have the data formatted in the same manner as you do.
Once you’ve mapped the data and ensured the values can be transformed, the next step is to present the user with a way to search and/or browse the data that they wish to access.
There are a few areas to consider when planning and defining the API design to have
Another area of variability is how to page the results returned from each endpoint. When presenting the search results from the endpoint you will want to implement a common approach to pagination for your users. Paging is a big factor in how users can search and find the data they are looking for.
It is important to determine how you’ll consume events from the endpoint(s) you're integrating with.
Many endpoints support webhooks that can be used to track create, update and delete actions as they occur to objects notifying your application. Webhooks keep your app up-to-date with changes (events) at the endpoint.
If the endpoint does not support webhooks, then your application will need to poll the endpoint to identify changes as they occur. With polling, you’re querying the timestamps on objects at the endpoint to see if actions (create, update or delete) have occurred. Your app becomes responsible for reaching out on a periodic basis based on a frequency that you set such as every minute, hourly or daily.
Events are critical to giving you the “trigger” to sync data between multiple endpoints. RESTful methods initiate Create, Retrieve, Update and Delete actions to resources at endpoints using GET, POST, PUT, PATCH and DELETE methods.
Since the goal is to synchronize data when it’s changed, you will need to consume events that Create, Update and Delete data. Consuming events will enable your application to execute corresponding methods to Create, Update or Delete when notified that an event has occurred at the endpoint; thereby synchronizing actions that change data.
You’ve made it. You most certainly deserve a pat on the back. But let’s reserve that for the very end of these final and vital steps: logging/monitoring. Did you really think that simply deploying an integration was the end goal?
Did you really think that simply deploying an integration was the end goal?
Well, think again.
Integrations are mission critical. Your customers are depending on your integrations to be up and running, no matter what. We hear over and over that if an app’s integrations are not working, customers and end users are frustrated, reflecting poorly on your core application.
Integrations are mission critical. Your customers are depending
on your integrations to be up and running, no matter what.
We have compiled years of integration experience into useful advice to help you not only prepare