Building an LLM wrapper prototype to develop my ML and coding skills...

It has been 7 months since I began up-leveling my technical skills through courses and projects. To apply these skills, I decided to develop a prototype that would demonstrate everything I've learned so far.

I leveraged my knowledge of algorithms, APIs, data cleaning and processing, large language models, machine learning, prompt engineering, and Python to create a system that understands the relationship between skills and job roles. This system allows job seekers to input their current skills and receive predicted and generated lists of potential career transition paths.

In this article, I will cover the steps I took to develop a solution I've named Skill(X).

System Overview

• Source files were pushed to a repository on Github.
• Streamlit Community Cloud pulls in the files from the repository, provisions an environment, processes the requirements files, installs dependencies, and then loads a Python script to launch the app, load the model, the label encoder, and the core model for inference.
• Streamlit hosts and deploys the app.
• Skill(X) is now online and available to the public.
• A user will input their current role and top 5 core skills into a Streamlit form.
• Once the skills have been provided they're converted into embeddings using the pre-trained SBERT model. The embeddings are then reshaped into Tensors for model input to pass them to the SBERT algorithm using the Keras API.
• The model generates predictions, and the script extracts the top 5 roles by probability, converts the labels back to role names, and stores them in a list.
• The list of predictions, the current job role, and the user's provided skills are mapped to a pre-built prompt so Google Gemini can analyze and assess the predictions from the model. The skills and current role are used to determine the relevancy and accuracy of the model predictions.
• Erroneous or inaccurate job role predictions are discarded and the LLM is asked to generate replacement role predictions for any discarded roles.
• The prompt is designed to output a response in the following structure:

"Roles to consider":
              "Matched Job Title #",
              "Job description": "Concise and informative job description (around 2-3 sentences). Focus on key responsibilities and required skills.",
              "Skill match analysis summary": "Overall analytical assessment of how well the user's *entire skillset* (all 5 skills combined) aligns with the requirements of this role. Consider the relative importance of each skill to the role. Examples: 'Strong overall match...', 'Partial match...', 'Weak overall match...'",
              "Skill match analysis explained":
                  "Skill": "Skill Name",
                  "Match": "Weak/Medium/Strong",
                  "Justification": "Brief explanation."
              "How your skills apply to this role": "Explain how the user's provided skills can be practically applied to this specific job. Provide concrete examples and focus on transferable skills. Aim for 3-4 sentences."
              "Skill Development": "Provide recommendations to the user to help them develop this skill."        

• From there, the prompt is passed to the Gemini Google API to request a response.
• While Google Gemini is developing the response, the user sees a status spinner.
• The response is then mapped to a message history container within the Streamlit app and presented to the user.
• The user has an option to download the recommendations in a .txt file format.

With that context in mind, for this next part, I shared my code with Google Gemini so it could generate a detailed description of the model creation, training, and testing process...

Here's what Gemini had to say about my code:

Step 1: Preparing the Data

The project begins by organizing data from a large database of job postings. Each job role (e.g., "Software Engineer") and skill set (e.g., "Python, SQL, Machine Learning") is extracted and prepped for analysis. To manage and manipulate this data efficiently, we use pandas, a Python library designed for data manipulation and analysis. By adding the data to a pandas DataFrame, we structure it in a tabular format, where each row represents a job posting, and columns contain details like job roles, skills, and other relevant information. This makes it easier to clean, transform, and analyze the data.

Step 2: Understanding Skills and Roles with SBERT

To help the computer "understand" text, we use a special tool called Sentence-BERT (SBERT). SBERT is a pre-trained language model that turns text into embeddings – numerical representations that capture the meaning of words and sentences. Think of it like creating a unique, detailed barcode for each role and skill set, enabling the system to recognize patterns and similarities between them. For example:

• The skill set "Python, SQL, Machine Learning" might be transformed into an embedding like [0.23, -0.17, 0.89, ..., 0.51], a list of numbers that represents the meaning of these skills.
• The job role "Software Engineer" might be transformed into an embedding like [0.42, -0.31, 0.76, ..., 0.14], a different list of numbers that captures the meaning of this role.

By converting both skills and job roles into embeddings, the system can calculate similarities between them and identify which roles align with which skill sets.

Step 3: Labeling Roles

Each job role is converted into a numerical label using Label Encoding from the sklearn library. This process assigns a unique number to each job role, making it easier for the system to process and classify them. For example:

• "Software Engineer" might be labeled as 0.
• "Data Scientist" might be labeled as 1.

Label encoding ensures that roles are represented in a consistent and machine-readable format, simplifying their use in the model.

# Extract roles and skill sets
print("Extracting roles and skills from dataset...")
roles = df['Role'].values
skills = df['skills'].values
print(f"Number of roles: {len(roles)} | Number of skill sets: {len(skills)}")

# Initialize the pre-trained SBERT model
print("Initializing pre-trained SBERT model...")
sbert_model = SentenceTransformer('all-MiniLM-L6-v2')

# Create embeddings for roles and skills
print("Creating embeddings for roles and skills...")
role_embeddings = sbert_model.encode(roles, show_progress_bar=True)
print("Role embeddings complete.")
print(f"Role Embeddings Shape: {role_embeddings.shape}") 
skill_embeddings = sbert_model.encode(skills, show_progress_bar=True)
print("Skill embeddings complete.")
print(f"Role Embeddings Shape: {skill_embeddings.shape}") 

# Encode the roles as labels
print("Encoding roles as labels...")
label_encoder = LabelEncoder()
role_labels = label_encoder.fit_transform(roles)
print("Role labels encoded. Sample labels:")
print(role_labels[:5])  # Display first 5 encoded labels        

Step 4: Splitting the Data

The dataset is divided into training and testing sets using the train_test_split function from sklearn. This step ensures that the model learns from one portion of the data (training set) and is tested on a different portion (testing set) to evaluate its performance. For example:

• Training Data: Used by the model to learn patterns (e.g., 80% of the dataset).
• Testing Data: Used to assess how well the model predicts unseen data (e.g., 20% of the dataset).

This separation prevents overfitting and ensures the model generalizes well to new data.

Step 5: Building the Brain (Neural Network)

A neural network acts as the brain of the system. Built using Keras, a high-level deep learning API in TensorFlow, this network is designed to find patterns in skill embeddings and map them to job roles. TensorFlow, the underlying framework, handles complex mathematical operations and optimization processes efficiently.

The network has multiple layers:

1. Input Layer: This takes the skill embeddings (348 numerical features).
2. Hidden Layers: These layers identify patterns and connections, like grouping technical skills into engineering roles or communication skills into managerial roles. These layers use an activation function called ReLU (Rectified Linear Unit). ReLU ensures that only meaningful signals (positive values) pass through the network, ignoring irrelevant information (negative values), which makes learning faster and more efficient. For example, if a neuron computes a value of -3, ReLU sets it to 0, while a value of 5 passes unchanged.

Output Layer: This predicts the job role, outputting probabilities for each role. For example, it might say there’s a 75% chance the role is "Data Scientist" and 25% chance it’s "Machine Learning Engineer."

# Build the neural network model
print("Building the neural network model...")
model = tf.keras.Sequential([
    tf.keras.layers.Dense(512, activation='relu', input_shape=(384,)),  # Explicit input shape
    tf.keras.layers.Dense(256, activation='relu'),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(376, activation='softmax') # softmax for classification
])        

Step 6: Training the System

The neural network is trained using the training data. During training, the system adjusts its internal settings to minimize errors in predicting roles. This is achieved using a process called optimization, where the model continuously improves by comparing its predictions to the actual roles and correcting itself.

The optimization process relies on an algorithm called the Adam Optimizer, short for "Adaptive Moment Estimation." Adam combines two key techniques:

1. Momentum – which helps smooth out updates by taking the past gradients into account, like remembering the speed and direction of a moving object.
2. Adaptive Learning Rates – which adjusts how fast the model learns based on how certain it is about the update. For example, if the model is making large errors, Adam takes bigger steps to correct it, and smaller steps when it’s close to a solution.

Step 7: Testing and Evaluating

After training, the model is evaluated on the testing data. It predicts roles for new skill sets, and we measure how accurate these predictions are. The accuracy score shows how well the system performs overall. For instance, if the model correctly predicts 90 out of 100 roles, the accuracy is 90%.

End of Gemini response.

If you'd like to review the solution in more detail feel free to review my Github repo for Skill(X).

What's next?

Although this was just a prototype, it was fun to test the solution and review its predictions.

From here, there are many opportunities for fine-tuning and further development:

• Developing a skill finder that analyzes a user's current and past roles to extract and identify their top skills. These skills could then be presented to the user to be ranked from strongest to weakest, which would help increase the matching success rate and relevancy.
• Identifying the industries in which they have the most experience.
• To help inform the recommendations, using their current and historical job levels: individual contributor, lead, manager, director, VP, etc.
• Enhancing skill development recommendations to recommend and provide web links to courses, certificates, experts, podcasts, and social media profiles to follow.
• Attaching salary range estimates to recommended job roles.

If you test the prototype, please note the following:

- Due to free hosting and usage limitations, the app might not be available 24/7. Limits on the number of concurrent users are also in place and rate limiting may occur.

- The app is optimized for desktops. While functional on mobile devices, the skill match analysis table may not be fully responsive extending beyond the screen.

- Like all models, the accuracy of its predictions depends on the quality and mix of the training data and the inputs you provide. Gaps in the roles and skills training data may exist, leading to poor matches. As a fallback, Gemini is asked to generate replacements when this occurs.

Thanks for reading.

David