Using AI to Help Design a Language

Reading this back to myself, it sounds like science fiction, but over the last couple of weeks I have been using the new O1 LLM model from OpenAI to provide feedback on design decisions I have been making in the Plato language.

It's not like I am making all of the decisions using AI, but I am finding it to be an excellent sounding board for design decisions, and to explore the language design space.

About Plato

Plato is a programming language that I created to enable me to express 2D and 3D programming algorithms and data structures that can be ported easily to different languages and platforms without all of the boilerplate that is common required in similar libraries.

At Ara 3D we are writing a geometry kernel (Plato.Geometry). The primary short-term motivation is to enable us to write a very efficient and flexible IFC loading library. Longer term, it will be the basis of Ara 3D Studio: a design computation tool we are developing as an alternative to Grasshopper and Dynamo.

Designing the Language

When designing a programming language there are a lot of factors to balance, and sharing your ideas with someone who can give you balanced feedback is invaluable. However, finding someone with the interest, experience, training, and availability is virtually impossible unless you can pay a very large salary.

I do know a few experts in programming language design and implementation, but they are very busy with their own work and personal projects, and don't have extended amounts of time to debate and discuss boring details of various language decisions.

When the new O1 model came out, I was wrestling with some big philosophical questions about the design of the language. In Plato there is something called a "concept" which is related to interfaces and related abstractions found in other languages. It's hard to pinpoint exactly what a concept is, but through one of my conversations with O1, it generated the following table explaining various related concepts.

While I am very familiar with C#, Java, C++, and Python. I am not as familiar with the subtleties of some features found in other languages. This is where O1 proved to be invaluable, because it seems to have a very deep knowledge of a wide range of languages. Here is some additional explanation the LLM helped provide me:

Interfaces and Protocols:

• Similarities: Both define a set of method signatures that implementing types must fulfill.
• Differences: Protocols can be adopted by more types (e.g., structs, enums) and may offer default implementations (in Swift).

Traits and Mixins:

• Similarities: Both provide method implementations for code reuse and can be composed into classes.
• Differences: Traits often have more sophisticated conflict resolution mechanisms, while mixins might lead to the diamond problem if not managed properly.

Type Classes and Traits:

• Overlap in Rust: Rust's traits function both as a mechanism for code reuse (like traits) and as a way to define behavior for types (like type classes).

Type Classes and Concepts:

• Similarities: Both impose compile-time constraints on types.
• Differences: Type classes also enable ad-hoc polymorphism, whereas concepts primarily serve to constrain templates and improve generic programming.

Generic Concepts

The conversation actually started because I was questioning how generic functions work in Plato. I had made an early decision to avoid going down the explicit generics route, because it presents a very steep learning curve for newcomers to programming. I wanted a language that felt dynamic while still being theoretically sound and efficient.

Here is how the conversation went:

What the LLM proposed was of course a common solution used among various programming language, but I was surprised at how well it grasped the problem. This is much beyond the capability of most first or second year computer science student.

We had some really engaging back and forth where I proposed some additional examples to consider, and explained the rationale for the language design and the target audience. Eventually, after proposing many different alternatives, and explaining the pros and cons between each one, the LLM came to the same conclusion:

Today I started another discussion with O1-mini (I keep running up into my quota for using O1-preview). I wanted to discuss the pros and cons of using an implicit indexing syntax for arrays.

It was impressive to see the range of alternatives it suggested. In the end we arrived at the same conclusion that the convention approach would be easier to use and read for new programmers, and we just had to be careful about how it is implemented.

Summary

It feels that I am getting the kind of feedback that I would be getting from someone with a PhD and 10 years of professional experience, which is quite rare, and would be very expensive if I had to hire someone for this stuff. I do see a substantial improvement in reasoning and feedback between O1 and the 4o model.

I was amazed to see how quickly the model is able to quickly learn and adapt to the syntax of my language with just a couple of examples, and was able to balance complex design goals. One downside is that it does tend to stick to established patterns and conventions, so it can be a bit challenged to think outside of the box. This works fine for me, because I need the balance when I am getting feedback.

So far it seems I can guide the LLM to where I want with prompting and patience, which thankfully doesn't mean that it will always agree with me. At least it is patient, and doesn't get annoyed if I keep arguing with it.