Real-World Uses for Large Language Models

Large Language Models (LLMs) like ChatGPT have showcased their prowess and versatility across various industries, despite being introduced to the public just a year ago. This blog, authored by the Engineering team at Oscar Health, details their use of ChatGPT 4 in developing an insurance claim assistant function. This assistant is designed to answer customer queries about their claims effectively. In tackling this project, the team employed several unique strategies and solutions. Firstly, they translated complete claim information into a domain-specific language termed “Claim Trace,” enabling ChatGPT to convert structured data into natural language. To enhance the model's performance, they implemented a method akin to providing a table of contents, which aids ChatGPT in better understanding the structure of Claim Trace. Another strategy involved a chain-of-thought approach with function calling, directing ChatGPT to break down a complex problem into smaller, more manageable segments. Additionally, they incorporated an iterative retrieval function, prompting ChatGPT to seek further information in cases of high uncertainty, thereby ensuring more accurate responses. These three methodologies combined to yield great results. The team reported a 100% accuracy rate in simpler cases and over 80% accuracy in more complex scenarios. This achievement boosted the company's operational efficiency and demonstrated how to fine-tune LLMs like ChatGPT to effectively meet specific business objectives. – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Apple Podcast: https://lnkd.in/gj6aPBBY   -- Spotify: https://lnkd.in/gKgaMvbh #datascience #chatgpt #llm #finetuning #largelanguagemodels #engineering #healthcare https://lnkd.in/gRnf_KmV

Large language models (LLMs) are typically optimized to answer peoples’ questions. But there is a trend toward models also being optimized to fit into agentic workflows. This will give a huge boost to agentic performance! Following ChatGPT’s breakaway success at answering questions, a lot of LLM development focused on providing a good consumer experience. So LLMs were tuned to answer questions (“Why did Shakespeare write Macbeth?”) or follow human-provided instructions (“Explain why Shakespeare wrote Macbeth”). A large fraction of the datasets for instruction tuning guide models to provide more helpful responses to human-written questions and instructions of the sort one might ask a consumer-facing LLM like those offered by the web interfaces of ChatGPT, Claude, or Gemini. But agentic workloads call on different behaviors. Rather than directly generating responses for consumers, AI software may use a model in part of an iterative workflow to reflect on its own output, use tools, write plans, and collaborate in a multi-agent setting. Major model makers are increasingly optimizing models to be used in AI agents as well. Take tool use (or function calling). If an LLM is asked about the current weather, it won’t be able to derive the information needed from its training data. Instead, it might generate a request for an API call to get that information. Even before GPT-4 natively supported function calls, application developers were already using LLMs to generate function calls, but by writing more complex prompts (such as variations of ReAct prompts) that tell the LLM what functions are available and then have the LLM generate a string that a separate software routine parses (perhaps with regular expressions) to figure out if it wants to call a function. Generating such calls became much more reliable after GPT-4 and then many other models natively supported function calling. Today, LLMs can decide to call functions to search for information for retrieval augmented generation (RAG), execute code, send emails, place orders online, and much more. Recently, Anthropic released a version of its model that is capable of computer use, using mouse-clicks and keystrokes to operate a computer (usually a virtual machine). I’ve enjoyed playing with the demo. While other teams have been prompting LLMs to use computers to build a new generation of RPA (robotic process automation) applications, native support for computer use by a major LLM provider is a great step forward. This will help many developers! [Reached length limit; full text: https://lnkd.in/gHmiM3Tx ]

Can large language models be used in biotech? The short answer is yes. While LLMs are often associated with chatbots, their capabilities extend beyond that. In biotech, much of the data comes in the form of sequences – like nucleotides in DNA, or amino acids in proteins. Similar to sentences in natural language, these biological sequences have unique semantic meanings based on the arrangement of their components. When input data is fed into an LLM, a transformer converts these sequences into contextual vectors using its attention mechanism. This process allows the model to understand the context and relationships within the data, enabling it to predict subsequent elements. One such use case is prediction of neoantigens that enable targeting tumor cells in personalized cancer immunotherapies. Neoantigens are tumor-specific mutated peptides presented on the surface of tumor cells because they bind to human leukocyte antigen (HLA) molecules. LLMs can predict this binding affinity. This allows the development of personalized therapies that use the patient's own immune system to kill tumor cells without damaging healthy tissues.

Large language models: a primer for economists (https://lnkd.in/eJschCjr) & Systematic Interpretation of Central Bank Communication Large Language Models (LLMs) have revolutionized economic research by enabling advanced analysis of unstructured textual data such as policy statements, financial reports, and news articles. These models transform text into structured numerical representations, facilitating tasks like sentiment analysis, forecasting, and topic modeling. Their contextual understanding, enabled by transformer-based architectures, makes them particularly effective in analyzing economic narratives. For instance, LLMs can evaluate market sentiment or interpret the tone of central bank communications, offering valuable insights into monetary policy impacts. A study of US equity markets demonstrated this by analyzing over 60,000 news articles to identify key drivers such as fundamentals, monetary policy, and market sentiment, linking these themes to stock market movements. Before the explosion of LLMs, I conducted research with my colleagues at Morgan Stanley to systematically analyze central bank communication using earlier machine-learning techniques. Specifically, we trained a Convolutional Neural Network (CNN) to assess the degree of hawkishness or dovishness in FOMC communications. This effort led to the development of the MNLPFEDS Index, which proved to be a powerful tool for anticipating monetary policy actions up to a year in advance. The index provided valuable insights into potential inflection points in the monetary cycle and their effects on rates, the yield curve, and the USD. This work highlighted the predictive power of communication analysis, even before the advent of the sophisticated transformer models now driving advancements in LLMs. LLMs and earlier machine-learning approaches, like CNN-based analysis, each bring unique strengths to understanding monetary policy and market dynamics. While LLMs excel in processing vast and complex datasets with contextual depth, their capabilities can be further enhanced through fine-tuning for domain-specific tasks. This adaptability allows LLMs to specialize in areas like central bank communication, where nuances in tone and context are crucial. Combined with the foundational contributions of earlier models like the MNLPFEDS Index, fine-tuned LLMs provide economists with a comprehensive toolkit to analyze qualitative insights and integrate them into robust quantitative frameworks, enriching the understanding of policy effects and broader economic trends. #EconomicResearch #MonetaryPolicy #CentralBankCommunication #MachineLearning #ArtificialIntelligence #NaturalLanguageProcessing #LLMs #DeepLearning #EconomicForecasting #SentimentAnalysis #TextAnalysis #DataScience #MacroEconomics #QuantitativeResearch

🎉 Exciting news! Our latest article, “Using large language models to automate summarization of CT simulation orders in radiation oncology,” has been published in the Journal of Applied Clinical Medical Physics. In this study, we explored how an open source large language model (LLaMa 3.1 405B developed by Meta) can automate and standardize the summarization of CT simulation orders—an important but time-intensive step in radiation oncology workflows. 📊 Key findings: Over 98% accuracy compared to therapist-reviewed summaries Improved consistency and readability Potential to reduce workload and enhance efficiency for CT simulation therapists This work highlights how AI and LLMs can support clinical teams by streamlining documentation and improving workflow quality. 🔗 https://lnkd.in/dDbATQWP #AIinHealthcare #RadiationOncology #MedicalPhysics #ClinicalAI #LLM #HealthcareInnovation

Building your own large language model is difficult and expensive but can be very rewarding. Perhaps you will find buried treasure! Charlee.ai has analyzed 55 MILLION insurance claims. After building and training their own LLM they said, “We’ve got the smartest insurance claims adjuster, ever!” By comparing a new claim against the LLM they can predict chance of litigation, claim severity and send the claim to the right adjuster. A large hospital has 100 years of patient records moldering in a warehouse. They are digitizing and preparing to analyze them stating, “Why people were cured is buried in those records”. Another company I’ve met analyzed hundreds of thousands of customer sales chats and now watches agents chat in real-time. Detecting that it is time to close the sale it makes a suggestion of what the agent should say based on its vast experience with great success. I asked the company how they ingested the company’s product catalog. “We didn’t”, they said, “We just read all the chats!” Combining your data with public data can yield new and surprising results. A hospital combined patient, appointment, prescription, weather and pollen data to forecast when patients might have to be hospitalized when pollen levels soared. They reached out to those who had let prescriptions lapse, prompted patients to be proactive during the weather event and even encouraged some to come in to the hospital early to avoid emergency services. The most interesting AI uses may well be a combination of your data, industry data and public data deployed in a new recipe that creates a new product. What hidden gems are lying in your dusty data warehouse that AI could turn into revenue?

Simulating millions of AI agents to predict real-world outcomes 🌍 What if we could create a virtual world with millions of AI agents to predict the impact of different policies on society? That's the idea behind AgentTorch, a new open-source framework that combines agent-based modeling (ABM) with large language models (LLMs). ABMs simulate complex systems by modeling the behavior of individual agents. But capturing realistic dynamics and adaptive behavior while efficiently simulating millions of agents has been a challenge. AgentTorch addresses this by using LLMs as the "brains" of the agents, allowing for high-resolution individual behavior at scale. The researchers put this to the test with a case study on the COVID-19 pandemic in New York City. They simulated 8.4 million agents, each with its own characteristics and decision-making based on LLMs. With interesting results. AgentTorch was able to capture the impact of individual choices around isolation and employment on the overall trajectory of the pandemic and the economy. This opens up many possibilities for policy design. By modeling counterfactuals and potential future scenarios with adaptive agents, AgentTorch can help overcome the limitations of relying solely on historical data. The implications extend far beyond pandemics. According to the researchers, AgentTorch is already being used around the world for policy-making and scientific discovery in various domains. As the saying goes, "all models are wrong, but some are useful." And if you have millions of them, what then? ↓ If you enjoyed this post, give it a like and share it with your peers. 🤝

The Use of Large Language Models (LLM) for Cyber Threat Intelligence (CTI) in Cybercrime Forums Earlier this month, a paper highlighted the use of LLMs to analyze CTI data from cybercrime forums, examining a random sample of 500 daily conversations. The results were impressive: The LLM system achieved an average accuracy score of 98%. What can we learn: - Ambiguity in prompts can lead to hallucinations and inaccuracies. Even advanced models need clear guidance. - The LLM system is imperfect—just like humans. This underscores the importance of using LLMs to augment, not replace, human expertise. By handling the initial data review, categorizing events, and prioritizing the most relevant findings, LLMs can significantly reduce the manual workload on CTI teams. Link to paper: https://lnkd.in/e7mWaEzG