Design and development for API-first organizations
As more organizations move to the cloud, they implement processes to deal with new microservices architectures, containerization, and continuous delivery. Whether they’re adopting cloud services or transitioning to a cloud infrastructure, an API-first approach can help you manage the complexity of working in the cloud.
The traditional code-first approach to app development sometimes results in delays, rework, or a disjointed frankenstein-esque experience for the developer, especially in this cloud-driven landscape.
An API-first approach assumes the design and development of an application programming interface (API) comes before the implementation. Your team begins by creating an interface for their application. After the API has been developed, the team will rely on this interface to build the rest of the application.
By introducing new features as an independent service accessed by API, the rest of the app can be stitched together, along with any other future apps.
In a world where speed-to-market holds a premium, why would you spend extra time to focus on the API first?
When the code comes first
With a code-first approach, you might start with your integrated development environment (IDE) to type out a few lines of code. Somewhere in the back of your mind, you know that eventually an interface will be created to deliver this service. However, your main focus is on what you call “core functionality” not the “interface”.
When it comes time to implement the interface (the API), the core service is mostly done so it drives the implementation of the API. In some cases, the API might need to be shoehorned into place to accommodate the behavior of the core service. For example, you can’t retrieve information the way you want to due to the way access is granted or the way a database is structured.
To the user who will ultimately use this API, it can feel tacked on, like an afterthought to the core service.
Besides a disjointed developer experience, this approach leaves your team vulnerable to bottlenecks.
Once a version of the API is completed, developers will commit the code to a shared repository and release it to a server environment. Only then can your dependent teams begin. At this point, there’s a host of other tools that you might use to consume the API, to develop test cases, write documentation, and provide feedback.
This is a synchronous flow that’s frequently encountered in the traditional waterfall software development process. Delays at any point in time will hold up dependent teams and push back the overall delivery of the product.
Feedback is gathered later in the development process, so changes are more costly since valuable time and resources have already been invested into this earlier version of the product.
When the API comes first
With an API-first approach, instead of starting with code, you could start with design, planning, mocks, and tests.
As you might already suspect, this process is aligned with the popular agile software development principle of iterating rapidly. This allows the team and any other stakeholders to weigh in on the proposed direction and functionality, early and often.
Gathering feedback at this early stage allows you to make changes more easily before a lot of time and effort is sunk into the project. Using mocks or an API development environment makes the current state of the project more accessible to both technical and non-technical team members.
By also separating the design of the API from its implementation, the architect is constrained only by the data model and business logic. Your API can now evolve unfettered by any existing user interface or legacy engineering frameworks.
Once the direction has been solidified, the design then serves as the contract that all teams can begin working towards in parallel, and only then does coding officially begin.
To drill down a little bit deeper on the API-first approach, let’s talk about API-first development and API-first design.
API-first development
This concept refers to developing the actual API first and foremost. When you’re developing new functionality, the functionality should first be exposed as an API in your organization. The developers responsible for the rest of the application will be the first consumers of this API. This ensures the quality, predictability, and stability to withstand web clients, mobile users, and other developer consumers. Other projects requiring this functionality can now independently consume the functionality via this API.
API-first design
This approach takes it a step further and requires planning the intended API’s functionality before building the API itself. What functionality will the API have? What data will it expose? What will the developer experience be like? How will it scale? How will we add new functionality in the future?
When people talk about API-first, sometimes they’re only referring to API-first development and sometimes they’re including API-first design as well. For the remainder of this article, API first will encompass both API-first development and API-first design.
Word of Caution: focusing on the design first does not mean ruminating endlessly on all the what-if scenarios. A good design should have the flexibility to adapt to changing circumstances and accommodate new insights.As some teams can attest, mandating that specifications drive development can backfire.
Most developers still prefer documenting existing code as specifications, rather than writing the specification first. An interface is intended to be an abstraction for interacting with a system, while the implementation is the way the system does its work.
Hyrum Wright observed that it’s problematic to fully separate an interface from its implementation, especially in large scale systems.
Hyrum’s Law differentiates between an explicitly documented interface that is planned for and an implicit interface that will become evident only with usage. Consumers will come to rely on behavior observed from both the explicit and implicit interface, at which point any changes to the implementation will violate their expectations.
With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.
- Hyrum’s Law
Why API first?
Clearly, pausing to flesh out the API delays valuable building time. Is it worth it? While not all organizations have the liberty to fully plan out their work, there are several benefits for choosing this approach that potentially outweigh delaying your time-to-market.
- Earlier validation: getting early feedback on the design allows the team to pivot or adapt to any new inputs while the cost of change is still relatively low. This reduces overall cost over the lifetime of the project.
- Clear abstraction layer: showing only the necessary details to the intended user hides internal complexity. This allows others to ramp up more quickly when implementing the new service.
- Decoupling dependencies: by adhering to a contract between internal services, dependencies are decoupled so that work can progress independent of other teams’ work. Working in parallel minimizes the project’s overall development time.
- Faster growth: building out the API design at an early stage takes future functionality into account, laying the groundwork for expansion, and accommodates the ability to quickly scale to other apps, devices, and platforms.
- Freedom from constraints: focusing first on the API instead of the code and implementation frees the design from legacy constraints.
What does API first look like in modern organizations?
Some organizations launched as API-only businesses, like Twilio or Algolia, who are both known for offering their services most importantly as an API. Twilio virtualized traditional telecommunications infrastructure allowing developers to replicate any communications experience using APIs. The business recently announced Twilio Build, a partner program designed with an API-first approach.
Twilio’s unique API-first cloud platform is tailor-made to support an ecosystem of partners that are differentiated by the innovations they deliver for their customers.
- Ron Huddleston, Twilio
As some organizations adopt agile software development practices, an API-first approach allows other companies to optimize their total development time. For example, Etsy switched to an API-first approach after facing the common challenge of implementing their logic twice.
All of the code that was built for the website then had to be rebuilt in our API to be used by our iOS and Android apps.
- Stephanie Schirmer, Etsy
Besides increasing extensibility for new devices and features, Etsy’s upgrade optimized their platform’s performance by enabling concurrent API calls previously limited by their PHP code base.
Another example of an API-first company is Netflix. Accounting for a large percent of total internet traffic, Netflix also re-configured their API architecture to allow concurrency. In the process, they distributed their API development so that each client application team was capable of implementing and operating their own endpoints.
A single team should not become a bottleneck nor need to have expertise on every client application to create optimized endpoints. Rapid innovation through fast, decoupled development cycles across a wide variety of device types and distributed ownership and expertise across teams should be enabled.
- Ben Christensen, Netflix
In this context, many teams treat APIs as a separate product or platform. Today, many engineering teams have a completely separate group for building new API functionality and maintaining scalability of their APIs.
Dedicating resources for designing and implementing the APIs demonstrates the value these organizations have for an API-first approach. This is in stark contrast with the code-first mentality which might propose building the application and then retrofitting an API as an afterthought.
A good read!