Generative AI and the impact on APIs and software development

Abhinav Asthana

May 8, 2023

Generative AI has captured the world’s imagination. Computers have surpassed humans in many ways in the past, but for the most part, they have excelled at repetitive and deterministic tasks. Programmers write algorithms that are intended to perform a task repeatedly and accurately unless there is a “bug.” When something goes wrong, skilled humans correct the bug in the computer’s programming. This paradigm is shifting with generative AI tools like ChatGPT, as computers seem to have made the leap from deterministic capabilities to creative thinking.

Generative AI’s output in the form of text, voice, or images can be pleasing to humans and augment our thinking, but if it is misused, it can be disruptive. We don’t yet know the ceiling here. However, we have been thinking a lot about the implications of generative AI for human-computer interaction, software development, and APIs.

AI bots are going to help advance the state of human-computer interaction. The most widespread tools that we use—Alexa and Siri—pale in comparison to what the current large language models (LLMs) are capable of doing now. The New York Times outlined some very interesting ways in which people are using these capabilities. As these bots get better, we can imagine that more complex use cases will emerge. These new use cases will create new habits, leading to another cycle of technological advancement.

As I see it, these new ways of interacting with computers will fundamentally change our relationship to software. Just like with the leaps in GUIs for personal computing, the internet browser, and mobile devices that we could carry anywhere, generative AI-driven interactions will make us rethink the ways in which we will build software for the future.

Going forward, all software interfaces will have creative companions that fetch information and data, execute actions on our behalf, and augment tasks that humans are performing. These bots will not be limited to a chat interface, though. I believe they will be deeply embedded in the existing workflows through which humans already interact with computers. For instance, bots will begin helping with intensive UI and data tasks, interacting through voice, and of course, interacting through a chat mechanism.

One area where I see a lot of potential is the simplification of complicated graphical user interfaces. For complex tasks, graphical user interfaces often become hard to use, and actions hide behind rows of buttons, menus, shortcuts, and procedures. They require years of training for people to become proficient, and even then, most people struggle with them. Generative AI trained on domain understanding has the potential to simplify those experiences.

However, these bots will only be able to do “actions” through APIs. APIs are the hands and legs that power the “thinking” that the AI is doing. APIs will connect these bots to data, as well as to verbs and nouns to get stuff done in the real world. Some of these bots might be completely autonomous. Auto-GPT, an experimental project that has been gaining popularity recently, can chain actions together towards a goal set by a human. If these bots continue to become popular, it will be important to ensure that the APIs they are calling are verified and tested for correct outcomes. Until now, we have primarily been designing APIs for applications that are used by humans, but designing APIs for machines will become an increasingly important area.

If you are the leader of an organization, what does this mean for you? Well, if your organization doesn’t have APIs or has poorly designed APIs, you are invisible to these bots.

We believe companies will need to double down on their API-first strategies in order to participate in the exchange of value in the software world. Companies were already leveraging multiple channels to interact with their consumers; this is one more way for them to harness the capabilities they have built.

So far, free LLMs for consumers have been trained on publicly available data sources, which means that most LLMs will be commoditized quickly. The organizations and individuals that create, gather, or verify high-value data are well-positioned to capture value on the data side of this equation (for instance, Bloomberg recently developed a generative AI model leveraging its proprietary financial data). If a company intends to monetize its data, it will need to establish clear guidelines on what is the right use case and what’s not. This requires designing and developing the right APIs for accessing data in this new world. Regulatory guidelines will also play a huge role, but it will take some time for them to catch up. Meanwhile, companies will also need to play catch up to protect the data they have sourced.

LLMs can’t create new facts, but they are good at making up facts (hallucinations). This is where humans will play a key role. Whether it is verifying text, verifying code, or “picking” the right sentence, the job of humans will continue to be “debugging” the output, rather than debugging the input. The input will still be the data that has been fed into the model, and while there may be ways to write a better prompt, there is currently no way (as far as I know) to know why the generative AI black box emitted what it did. This is a key limitation of these LLMs. What are the sources of data, what should be the right sources—these questions are all up for debate right now in various forums.

Finally, humans and human societies define what is valuable. Not every poem and every book written by generative AI has value. While it is fascinating to see a computer generate these artifacts, the value of that thing will ultimately depend on the amount of attention that will be given to it. This is unfortunately where there is a lot of potential for misuse of the technology, as these tools have the capability to unintentionally generate misinformation that can be indistinguishable from real facts. I believe verification through trusted sources and human collaboration will play a key role in solving these challenges.

Impact on APIs

So what does all of this mean for APIs? Let’s look at the possible impact on people and companies respectively.

For people:

• API debugging and API testing will become even more important to get right. These processes will be augmented by generative AI and will decrease the amount of time it takes for developers to get productive.
• API design and architecture will still very much be in the realm of skilled humans. Writing code will increasingly become more commoditized, but picking how to combine the right components for an integrated experience will be a key differentiating skill for developers.
• Collaborative API workspaces, used in conjunction with an AI-driven bot, will become a powerful way to work with APIs. Static documentation experiences and developer portals will feel even more outdated, and AI technology will only hasten their demise.
• API integration will become easier. Code-driven, point-to-point integrations or clunky integrations will feel rigid. AI-driven integrations will be able to incorporate new APIs—and heal faster if they break.
• Finally, generative AI will lower the barrier to entry for non-developers to build APIs.

For companies:

• Companies will start leveraging AI-powered software tools to drive productivity gains.
• Companies that don’t have APIs will be invisible to AI and will therefore fall further behind in the API economy. Companies that don’t have APIs but do expose data over the internet will not be able to capture the value of their data.
• Companies that have poor APIs will need to design better ones if they want AI models to interact with their data and actions in the right way.
• Companies will need to take an inventory of their known and unknown APIs. If they are exposing data that they want to harness later on, they need to act now.
• Companies will need to get smarter about identity and verification mechanisms for their APIs. As bots could be driving transactions, companies will need to ensure that they have put the right governance and security measures in place.

I am excited about the potential for generative AI and APIs. We will have a lot more to share in the coming days about how we are incorporating these ideas into the product.

Thank you to Sri Viswanath, Partner at Coatue and ex-CTO of Atlassian, for helping me review this blog post.