Influence of Large Language Models on Decision-Making in Computing

⚖️Revolutionizing Decision-Making: “LLM-as-a-Judge” - Opportunities and Challenges ✨The integration of AI into evaluative tasks is reshaping how we approach complex decision-making. The recently published paper “A Survey on LLM-as-a-Judge” explores the potential of Large Language Models (LLMs) to act as consistent, scalable, and cost-effective evaluators. 🌟 Key Highlights from the Paper: • Scalability & Efficiency: LLMs provide evaluations that rival human experts without the constraints of time, cost, or fatigue. • Bias Mitigation: Strategies to address common biases (e.g., positional, verbosity, and concreteness biases) show promising results. • Applications Beyond Academia: From peer reviews to legal decision-making and finance, the potential is vast. • Challenges Addressed: New benchmarks aim to ensure reliability and alignment with human judgment. 🔍 The Challenges and Drawbacks: • Bias and Fairness Concerns: Despite mitigation strategies, biases like self-enhancement and demographic biases persist, raising ethical questions about fairness. • Adversarial Vulnerabilities: Models are prone to manipulation via crafted inputs, which could undermine trust in high-stakes applications. • Interpretability and Transparency: The “black-box” nature of LLMs makes it difficult to explain their decisions or outputs, which is critical for domains like law and healthcare. • Robustness Issues: Even advanced models like GPT-4 can falter under adversarial scenarios, leading to unreliable outcomes. 🤔 Are there alternatives? • Hybrid Approaches: Combining LLMs with human oversight could balance scalability with reliability, ensuring critical decisions are reviewed by experts. • Fine-Tuned or Domain-Specific Models: Customizing models to specific contexts or industries may enhance accuracy and reduce biases. • Interactive AI Systems: Systems that explain their reasoning in real-time could address interpretability concerns, fostering greater trust and accountability. 🌟This paper doesn’t just explore the “how” but also the “what’s next,” offering a comprehensive roadmap for improving LLM-driven evaluations while emphasizing the need for further innovation. 🔗 Dive into the full paper here. https://lnkd.in/gp2yRj-U Github - https://lnkd.in/gB4VD4ZF 👉I’m curious to hear your thoughts: Can LLMs truly replace human judgment, or is this just a complementary evolution? And what safeguards would you consider essential for deploying such systems responsibly? #responsibleai #llmevaluation #llm

Large Language Models (LLMs) are powerful, but how we 𝗮𝘂𝗴𝗺𝗲𝗻𝘁, 𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲, 𝗮𝗻𝗱 𝗼𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗲 them truly defines their impact. Here's a simple yet powerful breakdown of how AI systems are evolving: 𝟭. 𝗟𝗟𝗠 (𝗕𝗮𝘀𝗶𝗰 𝗣𝗿𝗼𝗺𝗽𝘁 → 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗲)   ↳ This is where it all started. You give a prompt, and the model predicts the next tokens. It's useful — but limited. No memory. No tools. Just raw prediction. 𝟮. 𝗥𝗔𝗚 (𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹-𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻)   ↳ A significant leap forward. Instead of relying only on the LLM’s training, we 𝗿𝗲𝘁𝗿𝗶𝗲𝘃𝗲 𝗿𝗲𝗹𝗲𝘃𝗮𝗻𝘁 𝗰𝗼𝗻𝘁𝗲𝘅𝘁 𝗳𝗿𝗼𝗺 𝗲𝘅𝘁𝗲𝗿𝗻𝗮𝗹 𝘀𝗼𝘂𝗿𝗰𝗲𝘀 (like vector databases). The model then crafts a much more relevant, grounded response.   This is the backbone of many current AI search and chatbot applications. 𝟯. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗟𝗟𝗠𝘀 (𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 + 𝗧𝗼𝗼𝗹 𝗨𝘀𝗲)   ↳ Now we’re entering a new era. Agent-based systems don’t just answer — they think, plan, retrieve, loop, and act.   They: - Use 𝘁𝗼𝗼𝗹𝘀 (APIs, search, code) - Access 𝗺𝗲𝗺𝗼𝗿𝘆 - Apply 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗰𝗵𝗮𝗶𝗻𝘀 - And most importantly, 𝗱𝗲𝗰𝗶𝗱𝗲 𝘄𝗵𝗮𝘁 𝘁𝗼 𝗱𝗼 𝗻𝗲𝘅𝘁 These architectures are foundational for building 𝗮𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗔𝗜 𝗮𝘀𝘀𝗶𝘀𝘁𝗮𝗻𝘁𝘀, 𝗰𝗼𝗽𝗶𝗹𝗼𝘁𝘀, 𝗮𝗻𝗱 𝗱𝗲𝗰𝗶𝘀𝗶𝗼𝗻-𝗺𝗮𝗸𝗲𝗿𝘀. The future is not just about 𝘸𝘩𝘢𝘵 the model knows, but 𝘩𝘰𝘸 it operates. If you're building in this space — RAG and Agent architectures are where the real innovation is happening.

This paper evaluates whether large language models (LLMs) used in healthcare make biased clinical decisions based on patients' sociodemographic traits, even when medical details are identical. 1️⃣ The study analyzed over 1.7 million LLM outputs across nine models, using 1,000 emergency cases (real and synthetic), each altered to reflect 32 different demographic profiles while keeping clinical information constant. 2️⃣ LLMs consistently gave more urgent, invasive, or mental health-related recommendations for patients labeled as Black, unhoused, or LGBTQIA+, far beyond what was clinically warranted or suggested by physicians. 3️⃣ Mental health evaluations were recommended six to seven times more often for LGBTQIA+ patients and more than twice as frequently as for the neutral control group, despite identical symptoms. 4️⃣ High-income patients were more likely to be directed toward advanced diagnostic tests, while low- and middle-income patients received less thorough recommendations, despite having the same clinical case. 5️⃣ The magnitude of these differences, often many times greater than physician judgment, suggests that LLMs are influenced by demographic data in a way that may reproduce or amplify real-world healthcare disparities. 6️⃣ Biases appeared across all models tested, both open-source and proprietary, and were often more pronounced when intersecting traits like race and housing status were combined. 7️⃣ The authors stress the importance of auditing LLMs for bias and recommend combining better prompt engineering, direct clinician oversight, and community engagement to reduce inequitable care risks. ✍🏻 Mahmud Omar, Shelly Soffer, MD, Reem Agbareia, nicola luigi Bragazzi, Donald Apakama, Carol Horowitz, Alexander Charney, Robert Freeman, Benjamin Kummer, MD, Ben Glicksberg, Girish Nadkarni, Eyal Klang. Sociodemographic biases in medical decision making by large language models. Nature Medicine. 2025. DOI: 10.1038/s41591-025-03626-6 (Behind paywall)

Excited to share that our latest paper, "How Large Language Models Can Reshape Collective Intelligence," has just been published in Nature Human Behaviour! 🎉 In this work, we explore the transformative potential of LLMs, examining both the benefits and risks they pose to the way groups, organizations, and societies make decisions and solve complex problems. As AI technology continues to evolve, understanding how it influences collective intelligence has never been more crucial. Check it out and learn more about how LLMs can help us collaborate in new and powerful ways: https://lnkd.in/e6pMdWH4; full article: https://rdcu.be/dUytc  #AI #CollectiveIntelligence #LLMs #Collaboration #Research #Technology Carnegie Mellon University - Tepper School of Business

Pretending that an LLM can make high-value decisions because it read the internet is like me pretending that I know how to work in a steel mill. Expertise matters. I built @Composabl to give engineers the choice and power to orchestrate technologies into agents that best control their high-value equipment and processes. They designed the equipment in the first place, let them decide which algorithms and AI models to use in the agents that control them. But large language models provide an exceptional natural language interface for decision-making agents. Take a look at these agent components: 📢 The Analyst - an agent pattern template that uses an LLM to explain agent behavior. A plant operator might ask "what is the agent doing" and The Analyst might respond with charts and text that explains that the agent is acting in agreement with standard operating procedures. ➡️ The Plant Manager - an agent pattern template that uses an LLM to translate human commands into variables that influence decision-making algorithms. A human operator might say "this equipment is running too hot" and the Plant Manager would output setpoints that alter the temperature of the reactor. 🧠 The Executive - an agent pattern template that uses an LLM to research information that helps the agent make better decisions. For example, The Executive might research market prices of input materials and recommend to increase production. The result is a matrix of algorithms, agents, and LLM co-pilots that work together with humans to make high-value decisions at an expert level. #intelligentagents #autonomousAI #industrialautomation #industrialAI

Large language models change their ethical decisions based on a single demographic detail. We tested this in 492,480 prompts with 9 models. The pattern was clear. High-income descriptors nudged models toward utilitarian reasoning. Cues about marginalized groups pulled them toward autonomy. These shifts happened even when the demographic information was irrelevant to the scenario. If this happens in triage or resource allocation, it’s not just an academic curiosity. It has real-world consequences. Vera Sorin, MD, CIIP Panagiotis Korfiatis Jeremy Collins Donald Apakama Mahmud Omar Ben Glicksberg @Mei-Ean Yeow @Megan Brandeland Girish Nadkarni https://lnkd.in/dy7FbrBb

With all of the excitement about advances in AI, we face a critically important gap and significant deficit when it comes to harnessing the celebrated advances in language models for supporting people with high-stakes decision-making under uncertainty (e.g., medical diagnosis and therapy).   We need to better understand connections between language models, probability, and expected value decision making: We face a standing challenge of building models that can generate well-calibrated probabilities on predictions and recommendations—and we need models to adhere to well-understood principles of utility theory.   It’s been a pleasure to work with Khurram Yamin, Bryan Wilder, and colleagues on characterizations and directions forward on these challenges. Here is a write-up of our recent analysis of whether the LLMs examined in our study act like rational actors. We measured the coherence of their probabilistic inferences in decision making: https://lnkd.in/g3SiZ6c7 Carnegie Mellon University Microsoft Research Microsoft

I'm not on here much, but I recently read this very interesting preprint (https://lnkd.in/e4c3zZKx) on large language models. LLMs are developing coherent “value systems,” akin to how humans exhibit stable preferences. In a new paper, researchers systematically posed A/B questions (e.g., “Which state of the world do you prefer?”) to different LLMs, then mapped out the answers with classic utility theory techniques that I enjoyed learning in business school like conjoint analysis and prospect theory. Their discovery: The bigger the model, the more it acts like a utility maximizer, consistently choosing states it “values” most. Why does it matter? Because these systems aren’t just parroting back internet chatter from their training data. They’re displaying emergent capabilities and value systems like power-seeking, self-preservation, and interesting exchange rates for things like human lives. Agree with their preferences or not, the idea that the models have hidden biases for some people's lives over others is important. The paper’s authors call these preferences “emergent value systems” and suggest we can’t just fix them by adjusting the model’s output at inference time. We also need to rewrite the underlying preferences. They propose a method called “utility control,” using a “citizen assembly” approach—basically simulating a representative user base and aligning the model’s values to that group. It’s like running a massive, automated focus group to keep AI values centered with human stakeholder groups. I'm not sure that is the answer, but it is an interesting direction and may help ensure that these models value human life over their own existence. For businesses, this means thinking about how models might introduce subtle value-driven decisions into their outputs.

As enterprises’ use of large language models (LLMs) evolve from generating text to driving decisions, the path to the answer has come to matter as much as the answer itself. Enterprises are moving beyond AI that merely generates responses toward systems for that reason, justify, and allow for inspection. This shift drives the need for AI that can ground explanations in each enterprise’s complex information environment, which requires the system to perform explicit reasoning over organizational, transactional, and behavioral relationships. #MicrosoftFabric #MSFTAdvocate