AI, GPT, and all that Explainer

We hear AI all the time these days ...

Computer Vision, Robotics, Machine Learning, Deep Learning, Expert Systems, etc. Forget them for now - we are interested in the AI branch known as Natural Language Processing (NLP).

Why?

We humans easily understand and communicate with each other using our languages. However, when it comes to computers, we spend ages filling out fields, drop-down boxes, etc and then some code does further work to narrow our input into something the computer can understand. 

NLP, however, can take spoken instructions in our natural language and convert them into something the computer can understand and then provide its output in our language.

A key NLP tool is the Large Language Model (LLM), which has been around since the early 2010s and is used in everyday software like virtual assistants, translators, chatbots, etc. 

You know how car GPS helps you navigate through a city, finding the best route for you? Well, an LLM is like a GPS but for human language. It's an advanced AI that can read, write, and comprehend text with incredible accuracy.

Google Translate and Grammarly are good examples of LLM use.

One issue with LLMs used to be their limited ability to generate contextually relevant and coherent text based on the input received. They could understand the input language but when it came to the output they often struggled with context and coherency.

💞

Roger decided to wish his French actress friend good luck for her opening night. “Break a leg” he thought. Google Translate suggested "Casse une jambe" - literally break a leg. She never spoke to him again - it would have been better if he was given the phrase "Casser la baraque" (bring the house down). He was rather sweet on her, poor Roger.

💔

GPT, or Generative Pre-trained Transformer, came along and revolutionised LLMs. It's like a detective piecing together clues, using complex layers to analyse and understand patterns in human language. By doing so, it enabled more accurate text generation and improved AI-human interaction. 

The trick here was a feature known as "attention" - is like a spotlight on a stage, selectively focusing on the most relevant parts of the input data, while dimming the less significant information to create a meaningful and coherent output.

Attention also has other cool features:

• it can remember an entire "conversation" and retain the memory of what has been discussed before
• inputs can "train" it to develop mini-models in the context of this discussion - my code, our strategy, her history, their suggestions - and weigh them appropriately and override what its general knowledge and weighting are.

Here comes Harry

But AI is only good for discrete tasks.

AI and its child GPT are like the Sorting Hat in Harry Potter. Put the hat on a student and it allocates the student into a house. It can do nothing more. It is only one step in onboarding a new student to Hogwarts -

• Boarding the Hogwarts Express
• Sorting ceremony
• Welcoming Feast
• School robes
• House dormitories
• Class schedules etc.

The big productivity gains will come from being able to think like a human – analysing problems, formulating solution steps, and executing them effectively. Imagine if the computer could navigate a student through onboarding. What if it could even design the onboarding process, do up the documentation, and write an app to enable it?

In technical terms, people are calling this kind of tool "AGI", or Artificial General Intelligence. AGI aims to be a versatile autonomous creature, that can understand and perform any intellectual task a human can do. 

I treat Artificial General Intelligence as an aspirational name. Like Healthy Fast Food.

Well, AGI promises to be that versatile autonomous creature. The initial attempts at it seem more like Dobby, from Harry Potter. My weekend was like playing with a loyal, selfless creature prone to bouts of anxiety and erratic behaviour.

The AGI attempts that people are experimenting with right now (AutoGPT, BabyAGI) strive to be adaptable and skilled in multiple areas, not just one specific domain. Key features:

• It can plan solutions.
• It can read the web and mesh live info back into the "conversation" - something you have to do manually with chatGPT. Sure GPT has a snapshot of the web but versions prior to GPT4, in particular, had a nasty habit of making up really convincing-looking stuff, just to please you.
• It can spawn its own chats without needing your input - giving second- and third-order instructions like an army.
• It purports to execute code and test it but that is like the Leyland P76 section of this tool.

This is super-exciting because from what I see, these tools can be developed into productivity multipliers. As a portend of the future, the creators tell the developers "Whatever you do, don't turn this on in 100% autonomous mode".

An AGI tool is nominally superior to GPT because it has the potential to revolutionise business practices across various industries, solve complex challenges, and reshape the world as we know it.

I will write a later article on the potential use cases.

I call these tools "AGI attempts" as I don’t think we will ever arrive at true AGI. That does not matter - we will discover some amazing stuff along the way. Has off to the developers who built them and the army of following developers helping them improve it. What an age we live in.

In 1928, Alexander Fleming, a Scottish bacteriologist, was searching for ways to deal with the flu virus when he accidentally left a petri dish containing Staphylococcus bacteria open on his laboratory bench. A mould spore from a nearby lab contaminated the dish, and Fleming noticed that the bacteria around the mould had been killed. Hello Penicillin!

#AI #NLP #LLM #GPT #AGI #technology #innovation