The State of Vibe Coding

People have lots of opinions about vibe coding. Some refuse it. Some embrace it. Some seem scared of it, and others can't stop talking positively about it. The mainstreaming of GenAI will no doubt be remembered as one of largest shifts since the industrial revolution.

But with all the hype and with all the nay-saying, what's an individual to think? Are we ready to retire and let the bots do our work, or is that overly naive? I can't answer all these questions, but I can tell you my experience with using GenAI to do development tasks, and what my general impressions are.

The tldr here is that GenAI is amazing, but it's not quite at the point to replace jobs yet. Certain frustrations and limitations still exist, and with some of them, I have a feeling they'll exist for quite some time to come. That said, GenAI will be at the root of most innovation moving forward.

Here's are my general thoughts on vibe coding, GenAI, and how we should work with it.

AI is Great

I'm embarrassed to say this, but I was something of a late adopter to GenAI. I raged against the machine, refusing to help train my own replacement. If AI can really do my job, why would I possibly want to use it? I happen to like having a job!

But after a couple of years (and literally moving to a company that has AI at the core of it), I started dipping my toes in more, and really leaning into the tooling. To do so, I started with a few tools, but the major ones I used were Cursor, which has multiple foundation models built into it, and Databricks Assistant, which is built into the notebook experience on the Databricks platform.

Guess what? I liked it!

Not only did I like it, but it really did make my job much easier. Here are a few examples:

Defining Schemas

Writing out schemas is boring. It's a rote task where the output is pretty deterministic, and where repetition can't be fully avoided. For PySpark, this has meant writing the word "StructField" about 800 times, just with different parameters. Databricks Assistant made it much easier, though. I asked it to it analyze about 50 files and return back a JSON schema that makes sense.

It did this without trouble, and 20 seconds later, I had a well structured, perfectly indented, 81-line long schema definition of the files in my volume. Could I have done this by hand? Yes. Would it have taken longer? Yes. Would I have been bored doing it? Yes.

But with GenAI, I was able to delegate the boring part of the work to the computer, so I can think about the more complex parts of the application.

Normalizing Datasets

Using the schema defined above as context, I was then able to instruct the computer to build ingestion steps that will ingest the data appropriately, normalizing data as necessary. In one shot (almost... more on this later), the Databricks Assistant wrote out the code I needed to output multiple normalized data tables.

Again, this is the kind of thing that isn't hard per se, just boring.

My goal is to analyze, understand, and get value from data. Getting it loaded into something sensible is part of that task, but it's not central to it--it's preparatory. This is where GenAI can be very valuable.

Creating Starter UIs

In Cursor, along with Sonnet 4, I created the beginnings of a React UI for a POC I'm building. Not only was it able to create the general structure, but it was also able to pull stylings from Databricks' home page, and apply those stylings to the website so as to create brand continuity. It was able to do this without me having to find and copy the exact shade of orangish-red that our company uses. I simply told Cursor to review https://databricks.com and apply the stylings to the codebase I was working with.

My experience with building any kind of software, is that starting is the hardest part. Because it's not something that's done often, it's the kind of task that even we experienced techies forget how to execute, so we return back to the "getting started" pages on the framework we're working on, just so that we can bootstrap a project enough to have the general structure. As it turns out, GenAI is perfect for this kind of work.

Conclusion

All three of these are tasks that, while relatively straightforward, would have taken me much more time to accomplish than if I didn't use GenAI.

AI is Stupid

I might sound contradictory by saying so, but stick with me.

While GenAI is often great for getting started, it hits some limitations pretty quickly.

Misunderstandings Are Normal

While I was building the UI for my POC, I really wanted everything to fit on one page, without the need for the user to scroll. I asked Cursor to handle this for me, and it repeatedly failed at the task.

In fact, usually it would happily try to execute my request, and proudly tell me exactly how it had completed the task, but when I looked at the output, it was often suboptimal or just plain broken.

For the "full page no scroll" approach, it told me it accomplished the task, but all it really did was remove the vertical scrollbar from the page. Everything else remained the same size, so things that overflowed were just plain inaccessible. No matter what I asked, I couldn't seem to get the agent to correct the issue. Clearly, it didn't understand the current state of the page well enough to be able to make a proper adjustment.

Performance Is Often Lacking

GenAI did great in many cases getting something to technically work, but that doesn't mean the code it created is performant.

As an example, asking the Databricks Assistant to unzip a file into a directory on a volume will result in Assistant defaulting to using Python's ZipFile implementation, which, being written entirely in Python, isn't the most performant option. Assistant's first attempt would have resulted in nearly an entire day's worth of processing to extract the files--something that takes mere minutes on my local computer with an unzipping algorithm implemented in C or C++.

I was able to get it to create more performant code, but to do so required me to have some knowledge of how programming works in general, including how distributed compute and multithreading work. More on that in a moment.

It's Still Error-Prone

As we speak, I'm debugging one of the data transformations it built. It seems that it's struggling to use the posexplode function properly, and it's resulting in an error when it's outputting some of the data.

I tried asking the Assistant to fix the error based on the message, and while it gave a reasonable response, and a solid attempt to fix the issue, it still ultimately failed. This is where I'm glad I've worked with this kind of code before.

Despite the assistance of the GenAI tool, I still have to have the necessary skillset to fix any errors that do arise, and that skillset isn't one that comes overnight.

Forget Innovation

I've said this multiple times, and while it's sometimes unpopular, it is the truth.

AI is inherently non-innovative.

There's a simple reason for this--All AI and ML models are essentially nothing more than fancy averaging machines.

In case you're wondering, this is why the text you get back from ChatGPT can seem so lifeless and bland. The things that make a story interesting are usually the things that are unusual--rare turns of phrase, unusual plot twists, characters that never respond the way you think they should, but somehow still fit the overall vibe of the story.

GenAI models don't do innovation well because they're built on a foundation of discovering the road-most-travelled. While there's randomness built into the response algorithms, the randomness is never truly random, but rather a weighted randomness based on the likelihoods defined during training. For example, when deciding what the next word in a sentence should be, the model is given some options and told to pick, for example, the word "sentence" 60% of the time, "phrase" 30% of the time, and "story" 10% of the time. Any of these options would make sense in the context, but the scale is weighted towards picking the most obvious option.

And the most obvious option, is, by definition, not the most creative.

So What Should You Know?

Given those strengths and weaknesses, what should we keep in mind when working with these GenAI models to do vibe coding and other tasks?

You Still Need to Know Your Job

Sadly, this is where much of the hype cycle lives today. There are so many on social media saying that AI can fully replace humans in a variety of roles, but my experience has been the opposite. In many ways, I found that my use of GenAI has forced me to not only think critically about the things I want to do, but I also had to know a good deal about the internals of the GenAI output, in order for me to instruct GenAI appropriately.

One good example here was when I was asking GenAI to apply the stylings from Databricks' website onto the POC I was building. While it could certainly do so, I found that it's first few attempts were too much like the Databricks website. In fact, it even went so far as to add buttons to "learn more", even when I never explicitly told it I needed such a button. Why would it do that? Because it was on the Databricks Website.

So to work around this limitation, I had to be very specific. "Implement the stylings on Databricks' homepage at https://www.databricks.com/, but only implement them in CSS. Leave the general layout the same, but focus on the colors alone. Minimal changes to HTML should be made to accommodate the change."

My prompt required me to have a knowledge of CSS and HTML.

To get the AI agent to truly make the page full screen with no scrolling, I would have had to been pretty specific about using techniques such as flexbox on the front-end, which I doubt those without web development experience would even know about.

You still need to know your job.

Relying Solely on AI Is Dangerous

I've seen a pattern lately, especially among more junior devs. Many people rely so heavily on the AI tools, that they lack the skills necessary to fix things when they break.

Because of my experience with web UI development, I'm still very capable of making more directed and surgical tweaks to the generated code. While I love that GenAI can do so much for me, I have to be careful not to ask too much of it, as that could easily lead me into a situation where my skillset begins to decay due to underutilization.

I'm worried that, given the hype, many non-technical leaders will be short-sighted on this. Thinking that GenAI can do the job of a junior tech worker, they may be tempted to replace their junior workforce with GenAI, but this is likely to have some pretty negative latent effects on their organizations.

Juniors grow into seniors, and it's the seniors that understand the craft well enough to guide the AI agents and make corrections to output when necessary. It's those who understand the code well, and not GenAI, who are able to make innovative improvements, largely because GenAI is a model, and as we've discussed before, AI models, while being a very innovative technology, are themselves nothing more than fancy averaging machines not designed for innovation.

We need junior devs, because we're going to one day need their expertise.

Conclusion

In sum, my personal take is that AI should neither be feared nor worshipped, just as it shouldn't be in charge of everything or nothing at all. The key here is a happy medium. I see many rote tasks and boring repetitive tasks being delegated to GenAI, while the innovative tasks will still be performed by humans. This is good news for humans, as the history of trade and economics is all about innovation and novelty.

All that said, Generative AI is very important, and I definitely believe that companies and individuals who avoid it are going to ultimately be left behind. The ability of AI to remove from us the time-consuming tasks that once were part and parcel of our jobs will make it more possible than ever to innovate more quickly, and it's innovation that ultimately leads to value.

We just need to see it for what it is: an efficiency tool.

Efficiency tools rarely innovate anything themselves, but they do make it possible for others to do so, and this is where I think the real strength of GenAI lies: in the enablement of innovation.

So if you want to be innovative, use GenAI, just don't expect it to do the innovation for you. That part is still very much the realm of flesh and blood.