Software engineering in Generative AI projects

It is good to see lot of software engineering effort being put into the field of Artificial Intelligence.

Why do I say this?

The field was mostly, research driven where universities and organizations with deep pockets used to employ PhDs to churn out models to mimic various human-like tasks. Programming languages and tooling used, were not  friendly for deployment to production.

What has changed?

Now that the foundation models have matured and reached human level performance on specific tasks, software companies have started building frameworks and tools that leverage them to create applications for the end user. They have identified limitations of current stack at high volumes of data, concurrent users accessing the applications and unreliable behaviors' of models under certain conditions.

What opportunities lie ahead?

On the enterprise side, there is still lot of scope in helping enterprises, build use cases that leverages capabilities of AI. It will require, pipeline building for data to be fed in the model, choosing the right model, building the infrastructure, creating guardrails to prevent AI from hallucinating and deliver value keeping cost in check.

Identification of use case is mature and is specific to the industry vertical, some use cases cut across verticals as well.

 Data pipeline building exercise has been going on in enterprises to feed their data platforms for quite some time, same can be leveraged to pass data to AI. Certain additional processing needs to be done before data can be fed into the models and that is where software engineering can play a major role

Choosing the right model and building the infrastructure for the AI workload is a costly and time consuming activity. These drives the cost of implementing AI use cases high and hence lot of them are stopped as they are not financially viable. Software engineering can help bring the cost down by optimizing models to use less hardware

Within the construct of creating guardrails, there are multiple activities that needs to be performed, Prompt engineering, Data governance, Model governance, Prevent data leakage, IP protection and many others, software engineering principles can help organization in these as well.

There are few frameworks and design patterns that are emerging which helps organization with implementing the use cases, they are good but not as mature as enterprises would like them to be. There is room for many others to build those for the enterprises to consume. This is a big opportunity for Independent software vendors.

 In conclusion, now that the foundation models and AI use cases are here to stay, it is time for the ISVs and IT services companies to come with tools and framework for mature delivery of AI projects just like the software engineering projects.