Becoming an e-commerce architect: 12 API integration skills for developers

This is a guest post written by Mike Sedzielewski, co-founder of Voucherify. 

“APIs changed their lives!” is a headline I hope to see when a major newspaper highlights the story of Voucherify’s product, an API-first promotion and loyalty engine. Ours is a story of how a bunch of ordinary coders, used to implementing everything from scratch, went out of their comfort zone and embraced external APIs to 10x their shipping speed and create a SaaS platform that developers around the world now trust.

Voucherify is 10 years old this year. On this momentous occasion, I want to share some learnings about integrating APIs into e-commerce projects. These concepts helped us become more proficient in business-oriented software architecture and let us grow a team of 50, with our platform becoming a part of the tech stacks of some of the biggest brands out there.

Early days of the e-commerce API

In 2014, in the early days of our company, we were contracted to help build the Uber-for-X marketplace. It was our first client, so we needed to be prepared. We summoned the best practices from every project we’d been involved in and created a shiny skeleton for our perfect marketplace app—one-click builds (not that obvious back then), AWS config test suites, and design patterns.

We quickly learned that the thing we’d built went to /dev/null. It turned out the client’s MVP at the time was working on Google Sheets wired to a simple landing page. This was it, and it worked just fine!

Our task was to migrate it to a proper CRM and then connect customer events to new marketing and support channels such as email, SMS, mobile apps, and other websites.

We learned the Salesforce CRM to create the proof of concept. It turned out that the Salesforce API encapsulated the CRM blocks behind a RESTful API, and it already simplified the integration by an order of magnitude. We discovered that it lets you define a custom data schema and can trigger a webhook when objects change their state. With an active Salesforce subscription, all you need to take care of is the frontend layer. We learned that this approach has a name—“headless.”

Next, went on an integration spree—we researched around 20 different headless platforms to ship more features to our high-growing customer. The sheer number of services to be connected caused data sync issues; they were extremely painful as we tested on production. Luckily, the early startup stage forgives a lot.

Within two years, we learned some hard-won lessons on making third-party commerce APIs talk to each other without data integrity problems. As a bonus, these lessons helped us build Voucherify, an API-first promotion and loyalty engine.

Today, since e-commerce APIs have become mainstream and standardization organizations like MACH Alliance have been brought to life by the industry, the following insights are more relevant to an aspiring e-commerce architect than ever.

1. Read the fancy manual

Most e-commerce APIs provide a detailed API reference. You can see example requests and responses, read how to authenticate, and browse through possible error codes. Everything you need to implement an integration. But if you do a deep dive, you can find they often offer much more than that. Vendors have special task forces to create goodies that can slash the integration time by half. What to look for:

• Check official SDKs: They might have some helper classes or UI widgets that make mundane tasks easier.
• Learn API query parameters: Some API providers will surprise you with various utility filters, sorting, and paging utilities.
• Learn how the platform identifies unique objects: In most cases, the job is done with a single field, but there are many exceptions.
• Browse through official examples: They might have 80% of your task implemented.
• Localize Postman Collection: You can easily connect to play around with the API without ramping up your IDE. The collection can also be used to turn requests into boilerplate code.
• Enable interactive documentation: When you provide API keys, you can now quickly test endpoint behavior right in the documentation.

Also, get to know the logs panel. With goodies such as free text search and time-based filters, it’s way easier to audit your connection. You can see what has been consumed or produced by the API without going through wizards and reports.

Equipped with knowledge and tools, we’re about to exchange data with a black box—a well-documented but still black box. Here are a few tips on how to ensure data integrity.

2. Check API version

While API-first vendors try to keep their endpoints backwards-compatible, they may release a completely remodeled version of the API. Make sure you’re working with the right one when learning the specs and later when connecting to endpoints. Check if the versioning is realized through a URL, query parameter, or an Accept/Custom Request Header. Also, watch out for the version sunset period and plan necessary upgrade tasks.

3. Double-check update guarantees and caching

API references, the platform’s architecture overview, and audit logs will also help you understand which endpoints have strong and eventual consistency. Some endpoints require heavy processing behind the scenes, and hence the expected result might not be immediately visible on the platform.

It’s a good practice for vendors to label these requests either by 202 status code or in the docs, but it’s a good idea to check the behavior yourself. This is because some endpoints might act as synchronous when testing and surprise you only when the traffic peaks—and these cases belong to the hard-to-debug family.

Response caching might be a member of the family, too. Make sure to vet the endpoint reference from top to bottom to learn about the caching policies. Then, push them to test to collect aggregate data on the endpoint behavior IRL.

4. Favor using idempotent endpoints

In contexts where you might be repeating requests to an external API, you may cause unwanted duplicates and more serious data-integrity issues.

Sometimes security concerns suggest using idempotent endpoints. An example would be if you’re a company that generates a unique gift card using the Amazon Incentives API, sends it directly to a customer, and then the customer loses it. To resend the gift card, your customer support would have to have access to the codes, which might be suboptimal. Fortunately, the generation endpoint is idempotent, and invoking it with the same arguments results in sending the existing gift card certificate instead of buying a new one. With this in place, it’s easy to implement the “I lost my gift card” functionality without customer support seeing the codes.

5. Scan the data-retention policy

Even if you took care of the proper data sync, there might be a case where an endpoint won’t return the expected result. One of the reasons this happens is that your data is simply not stored anymore. API-first platforms have data retention policies. They differ depending on the type and size of data, the frequency of reads, and, ultimately, your contract. Go through the docs and your terms to learn what to expect and to prepare the optional backup plan.

6. Examine API error handling

API errors are unavoidable. What you have control over is how you react to them. And this varies depending on the business outcome you want to achieve.

First, make sure to examine API error codes before integration. I guarantee you that this will reduce the number of surprises.

Let’s consider the situation when there’s no response at all—the service timeout. Every major API provider should list the average response times and recommended timeouts (for example, six seconds for the Amazon Incentives API). Knowing when to stop waiting for the response might have major implications for the overall performance of the process you implement. Read the recommended timeout for the services in use and simulate such occurrences in your tests.

You need to have a fallback procedure when your favorite API returns nothing more than 500s or becomes silent altogether. Assess the service from the business point of view—some endpoints aren’t critical and can be fixed manually, often by customer support, but the essential jobs (especially those with massive scale) should be put in the queue and wait for the API to be back live. For such scenarios, consider implementing a retry mechanism with exponential backoff so as not to overwork the services again. The increment between retried requests should be aligned with the business impact the job is responsible for. Trust me, your CTO will appreciate it if every fallback mode occurrence is monitored, as this will help them check the quality of the service against the SLA.

7. Meet an API guardian: the rate limiter

SaaS platforms use rate limiters to manage resources and costs and to protect themselves from distributed denial-of-service (DDoS) attacks.

The most common scenario to limit requests coming from a single user is to cap:

• Aggregate number of requests over a period of time (most often a month)
• Number of requests per minute, hour, or day

Sometimes, there are limits to the weight of the data ingress and egress, too.

With every response from the API, platforms inform you about your current usage so that you can take it as input for further processing. This is often done via response headers:

If you exceed the limit, you will most likely see the API return 429 instead of 200 OK.

There are three ways to react:

• Renegotiate the contract with the API provider
• Use bulk endpoints
• Queue requests  

Usually, the first one should be considered last. However, sometimes business requirements might suggest the other way. If the vendor offers a usage-based billing structure, the decision to temporarily lift the limits might be cheaper than any code changes.

Switching to bulk endpoints usually gives a good return on investment. Bulk endpoints are exposed for a reason. There’s probably a lot of data to be updated or queried. So, even if they have worse developer experience (using them needs more lines of code or async handling), using them will pay off.

The last one, request queueing, should often be the default mode of operation in the world of integrated services. Thinking ahead on how to control the frequency of requests helps mitigate the harm done by unexpected traffic spikes, like burning your monthly quota or, worse, generating disproportionately high bills.

This comes at a price, though. You need to take care of the load-balancing setup in place. And this is one of the biggest trade-offs you get in exchange for the freedom of a poly relationship with best-of-breed APIs instead of marrying a single vendor.

8. Map out the way from migration to production

The majority of API-first platforms were created by engineers who understand that it’s nice to have an environment to test the APIs without impacting production. That’s why they offer a sandbox—a separate account with more strict limits or limited functionality that a coder can use at any stage of development.

Before building a proof of concept in the sandbox, check what it takes to push your solution to the production environment. Some vendors have shortcuts for this, and some require you to virtually redo the work—for the latter scenario, you might want to consider writing a setup script along with crafting the PoC.

Sometimes the push to production entails the initial data migration, and this might require lifting the limits. Make estimates and check them with the vendor’s spec or support team before running your data migration script.

9. Master webhooks

We covered the case when you send stuff to an API platform and hope to get 200 OK. In the world of composable solutions, the reverse direction is essential, too. When you want to be notified by the platform about an event occurrence, you need to set up webhooks. You can think of them as a sort of “reverse API,” where your application is responsible for handling incoming requests. Let’s see what useful patterns and tips emerge here.

Most of the time, you will be the party receiving the webhook. It might happen that the webhook-sending platform will notify your service about a single event more than once. So, the very first step is to plan how to cope with duplicate events. One way of doing this is by logging the events you’ve processed and then building the logic for not processing already-logged events.

Sometimes, though, you will be tasked with creating a webhook-like engine in your application. Understanding webhook-failure scenarios comes in handy for this use case. Learn the difference between failure in synchronous processing mode (origin timeout, invalid request, unhealthy origin) and asynchronous one (origin dropped request), as explained in the fantastic article by Herman. This knowledge impacts what monitoring stack you should prepare to implement the recovery process for your webhook mechanism, but it’s also useful for your app being a receiving party—by using the right response codes to instruct the behavior of the webhook client.

Last but not least, your code needs to make sure the webhooks come from a trusted client. This can and should be implemented using various techniques, such as IP whitelisting and signing requests. The grandfather of API-first platforms, Stripe, created an exhaustive tutorial on how to consume their webhooks. There are also platforms that offer webhooks as a service; studying their code and ecosystem will give you a handful of tips.

10. Explore pre-built integrations

On their way to achieving the shortest integration time possible, API-first vendors build connectors between popular platforms. This allows you to skip API polling, webhooks, and building the mapping logic yourself. For instance, e-commerce platforms are usually paired with CMS, so most vendors have created plugins for the most popular CMS. If they fit your use case, why reinvent the wheel (the wheel you need to maintain)? A seasoned product engineer will research built-in capabilities to save as many code lines as possible.

One of the most essential concepts booming in the marketing and e-commerce world is a customer data platform (CDP). A CDP serves as a centralized customer database that keeps customer attributes in sync across various sources. Its biggest value is that it can offload the handling of:

• Data transfer traffic peak
• Connectors to the myriad of popular tools (CRM, analytics, email, e-commerce platforms, etc.)
• Data security and privacy policy management

The last feature gives you the ability to control schema, including advanced privacy and compliance features. In simple words, you get a single place where you control which platforms can read or write sensitive customer data. This becomes an issue for a typical e-commerce stack of more than 10 tools.

So before connecting popular API-first SaaS platforms together with your app, check if a CDP can do the heavy lifting for you.

11. Detect downtime with status monitoring

There will be times you’ve retried a request X times, your “back off” cycle comes to a stop, and the platform is still returning 500s. It might be the case that the service is simply down.

In the third-party API world, where almost every building block comes from a different provider, being notified about downtimes is a must.

Every composable platform offers a number of ways to send notifications. Most often, it’s as easy as subscribing to an email list. While you can build a suite of health checks yourself, there are also aggregators that allow you to collectively sign up for downtimes for any service.

12. Event-driven architecture and further reading

When the number of connected systems and the data volume grow, you should consider building a standardized management layer on top of the interacting systems. This topic has been coined “event-driven architecture,” and it exceeds most of what I wanted to fit in a short article. I recommend watching the “From Zero to Async API” video from Postman API Lifecycle Integration Specialist Kevin Swiber, Postman Director of Engineering Fran Mendez (founder of Async API), and Kin Lane (former chief evangelist at Postman).

As we reach the end of our journey through the maze of e-commerce APIs, it’s clear that mastering API-first platforms is not just a technical requirement—it’s a craft. Remember, as developers, we’re not just coding—we’re creating the digital backbone of businesses.

Embrace the challenge, stay curious, and keep experimenting. With these skills in your toolkit, you’re not just building applications; you’re architecting the future of e-commerce. So go ahead, code boldly, and let your API-first skills be the beacon that guides your path in this exciting, ever-evolving landscape!