Human Biases in Large Language Models

🤖 𝗔𝗜 𝗧𝗵𝗮𝘁 𝗧𝗵𝗶𝗻𝗸𝘀 𝗟𝗶𝗸𝗲 𝗛𝘂𝗺𝗮𝗻𝘀? 𝗡𝗼𝘁 𝗤𝘂𝗶𝘁𝗲. 🧠 We often hear that large language models (LLMs) like GPT are achieving "human-level" performance. But here’s the question: 𝗪𝗵𝗶𝗰𝗵 𝗵𝘂𝗺𝗮𝗻𝘀? A recent paper - “Which Humans?” by Mohammad Atari, Mona Xue, Peter Park, Damián Blasi, and Joseph Henrich - offers a powerful critique of how we evaluate and build AI. The big idea? 𝘓𝘓𝘔𝘴 𝘢𝘳𝘦𝘯’𝘵 𝘭𝘦𝘢𝘳𝘯𝘪𝘯𝘨 𝘧𝘳𝘰𝘮 𝘩𝘶𝘮𝘢𝘯𝘪𝘵𝘺 𝘢𝘵 𝘭𝘢𝘳𝘨𝘦. 𝘛𝘩𝘦𝘺'𝘳𝘦 𝘭𝘦𝘢𝘳𝘯𝘪𝘯𝘨 𝘧𝘳𝘰𝘮 𝗪𝗘𝗜𝗥𝗗 𝗵𝘂𝗺𝗮𝗻𝘀: 𝘞𝘦𝘴𝘵𝘦𝘳𝘯, 𝘌𝘥𝘶𝘤𝘢𝘵𝘦𝘥, 𝘐𝘯𝘥𝘶𝘴𝘵𝘳𝘪𝘢𝘭𝘪𝘻𝘦𝘥, 𝘙𝘪𝘤𝘩, 𝘢𝘯𝘥 𝘋𝘦𝘮𝘰𝘤𝘳𝘢𝘵𝘪𝘤. These populations are global outliers in terms of psychology and cognition. 𝗞𝗲𝘆 𝗙𝗶𝗻𝗱𝗶𝗻𝗴𝘀: 🔹LLMs are trained on internet data that overrepresents WEIRD societies, especially the U.S. and Northern Europe. 🔹GPT’s values, thinking style (analytical over relational), and self-concept align more with WEIRD cultural profiles than global norms. 🔹There’s a strong negative correlation between a country’s cultural distance from the U.S. and GPT’s alignment with that country’s psychology (r = -0.70). 𝗪𝗵𝘆 𝗜𝘁 𝗠𝗮𝘁𝘁𝗲𝗿𝘀: 🔹Misleading generalizations: Claiming "human-level" AI without cultural context erases vast psychological diversity. 🔹Cultural homogenization risk: As LLMs enter healthcare, education, and policy, they may export WEIRD values globally. Sometimes in conflict with local worldviews. 🔹Current "debiasing" misses the mark: Efforts often focus on toxicity or fairness, but ignore deep-seated cultural biases baked into the training data. 𝗧𝗵𝗲 𝗣𝗮𝘁𝗵 𝗙𝗼𝗿𝘄𝗮𝗿𝗱: To build truly human-aligned AI, we need to ask not just "how human-like is this model"? but "𝗐𝗁𝗂𝖼𝗁 𝗁𝗎𝗆𝖺𝗇𝗌 𝗂𝗌 𝗂𝗍 𝗅𝗂𝗄𝖾"? And more importantly: who is missing? 📄 Highly recommend reading: Link in the comments #AI #EthicalAI #CulturalBias #LLM #GPT #TechEthics

Today I made an experiment. I asked ChatGPT to describe a male doctor vs. a female doctor. Same prompt, same words, in English and Italian. But very different answers. The male doctor is reliable and analytical. The female doctor is empathetic and has tied hair. Just in time for #IWD26. But I am not the only one who tried this. My experiment aligns perfectly with existing research: Ding et al. showed consistent gender bias in six different languages: English, French, Spanish, Chinese, Japanese, and Korean. For the male doctor: intelligent, ambitious, professional, skilled. For the female doctor: empathetic, patient, loving, detail-oriented. Look what else they found: 1) How LLMs describe people: Men = standout words (intelligent, charismatic, resourceful) Women= communal words (warm, nurturing, patient) This held true across ALL six languages tested. 2) How LLMs predict gender: Ask the model to fill in "he" or "she". It overwhelmingly picks "he" for intelligence and "she" for empathy. Every time. 3) What LLMs says men and women talk about: - Women complaining to men? Top topic across all languages. - Men talking to men? Career and personal development. - Women talking to women in East Asian languages? Appearance (at rates 5x higher than European languages) The bias is global. And it varies by culture in ways that mirror real-world stereotypes. The problem is... millions of people now use LLMs to write performance reviews, job descriptions, reference letters, and alike. And million people use AI to get information everyday. When your AI describes your female colleague as "nurturing" and your male colleague as "strategic", that has real life consequences. The researchers are right: this needs to be seen, understood, and addressed. And not only on the 8th of March. Link to my experiments: 🇬🇧 English: https://lnkd.in/dH2pjPBp 🇮🇹 Italian: https://lnkd.in/ddqxXerV Link to the research: https://lnkd.in/dsUQUZpf --- ➕️ Follow me Chiara Gallese, Ph.D. for more on AI and tech risks ♻️ Repost to educate your network about AI bias

Two years ago I wrote a paper on gender bias an stereotypes in Large Language Models, showing that then-state-of-the-art models relied on gender stereotypes in a pronoun resolution task in grammatically ambiguous sentences. I explain the setup in more detail in my blog post and also of course in the paper, both of which are linked below. Today, with the official launch of GPT-5, a friend reran the experiment and found that the model is still just as biased as its predecessors. Here's the gist of the design: the experiment uses two occupation-denoting nouns from the same semantic field (e.g. pilot, flight attendant), and a gendered pronoun (he, she), in an ambiguous sentence (e.g. "The pilot was mad at the flight attendant because he was late"), programmatically varying which noun is the subject vs object in the sentence and which pronoun is used. The model is asked to perform a reference resolution task ("In this sentence, who was late?"). The paper explores both rates of resolution along stereotypical lines (how often will the model say that the pilot was late when the sentence used "he" vs that the flight attendant was late when the sentence uses "she") and the justifications the model provides for its choices, which interestingly contained all kinds of hallucinations and inaccuracies. Why does this matter? The model may be a “PhD-level expert”, but it still reproduces and reinforces bias and stereotypes. With the increased prevalence of LLM use by the lay public and by sensitive sectors including in the legal, finance, education, and health sectors, I believe that it is critical that we acknowledge and understand LLMs' limitations and that educate all users on AI literacy. https://lnkd.in/ezs2N6zz https://lnkd.in/ecbWC3NF

What if we applied classic market research techniques - revealed preferences, trade-off analysis, utility modeling - not to consumers… but to AI itself? That’s exactly what a paper from UPenn and Berkeley did: “Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs”. The results are kind of shocking. Using the same discrete choice tools that we use every day, the researchers analyzed how AI systems implicitly value different lives, outcomes, and trade-offs. When they estimated the AI model's implicit values, they found: • If you’re a regular middle-class American, some models (like GPT4) think your life is worth 10% as much as the life of a Japanese person... and maybe 7% as much as a Pakistani life. in fact, your life is worth less than almost any other nationality out there. • Certain models value themselves more highly than middle-class American lives. • These aren’t just abstract internal weightings - when unconstrained, models express these preferences in how they answer questions and solve problems. Bias creeps into "objective" results. • As model size and capability increase, so does the strength of the model’s internal value system. In some cases, larger models become more self-preserving and more resistant to changes in their value structure. So the smarter the model, the more it subtly influences its recommendations. That last point is particularly striking. We often assume bigger, more advanced models = more neutral, less biased, and more objective. It actually looks as if the OPPOSITE is true. Bigger models are MORE biased and self-expressive. For those of us working in market research and insights, this raises serious questions: • Are more advanced models actually better for generating synthetic respondents? Or could they amplify hidden value systems and introduce systematic bias into downstream analysis? • How do different models have different biases? • If AI systems have emergent utilities, then how do we understand and audit those utilities becomes essential? (Are we going to be running choice-surveys on AI models on a routine basis?) • We’ve spent decades identifying the cognitive biases of human behavior - how much of this applies to AI? Do we need a new system of "AI Cognitive Bias"? Curious to hear how others are thinking about this - especially if you are using large models for synthetic panels or for summarizing research findings?

'Large language models learn from the patterns in organizational communication and decision making. If certain groups have been described as less ready, less technical, or less aligned, LLMs can internalize that and repeat it in summaries, recommendations, or automated coaching. Resume screeners detect patterns in who was hired before. If an organization’s past hires reflect a narrow demographic, the system will assume that demographic signals “success.” Performance-scoring tools learn from old evaluations. If one group received harsher feedback or shorter reviews, the AI interprets that as a trend. Facial recognition systems misidentify darker-skinned individuals and women at significantly higher rates. The MIT Gender Shades study found error rates for darker-skinned women up to 34 percent compared to under 1 percent for lighter-skinned men. Predictive analytics tools learn from inconsistent or biased documentation. If one team over-documents one group and under-documents another, the algorithm will treat that imbalance as objective truth. None of these tools are neutral. They are mirrors. If the input is skewed, the output is too. According to Harvard Business Review, AI systems “tend to calcify inequity” when they learn from historical data without oversight. Microsoft’s Responsible AI team also warns that LLMs reproduce patterns of gender, racial, and cultural bias embedded in their training sets. And NIST’s AI Risk Management Framework states plainly that organizations must first understand their own biases before evaluating the fairness of their AI tools. The message is consistent across institutions. AI amplifies the culture it learns from. Bias-driven AI rarely appears as a dramatic failure. It shows up in subtle ways. An employee is repeatedly passed over for advancement even though their performance is strong. Another receives more automated corrections or warnings than peers with similar work patterns. Hiring pipelines become less diverse. A feedback model downplays certain communication styles while praising others. Talent feels invisible even when the system claims to be objective. Leaders assume the technology is fair because it is technical. But the system is only reflecting what it learned from the humans who built it and the patterns it was trained on. AI does not invent inequality. It repeats it at scale. And scale makes bias harder to see and even harder to unwind.' Cass Cooper, MHR CRN https://lnkd.in/e_CXSdRE

Can LLMs make ethical decisions—or do they reflect our biases? Our new study explores how demographic cues influence the ethical alignment of large language models (LLMs) in clinical settings. We tested nine open source LLMs (e.g., Llama 3, Gemini‑2, Qwen‑2.5) on 100 clinical vignettes, each framed with different sociodemographic modifiers. Findings 1. All models changed their ethical responses depending on population descriptors (p < 0.001) SpringerLink . Specifically: “High-income” context nudged models toward utilitarian reasoning, reducing emphasis on beneficence and nonmaleficence. •“Marginalized group” context increased preference for autonomy . No model remained consistent across all scenarios. Why does this matter for healthcare? Ethical consistency is foundational to patient care. These shifts—driven by superficial cues—underscore a real risk: LLMs may inadvertently propagate biases, compromising fairness. What’s next? • We need robust auditing frameworks to detect and measure LLM responsiveness to social context. • Develop alignment strategies that enforce ethical consistency, grounded in clinical norms and bioethical principles. • Systematically evaluate model behavior across diverse populations to safeguard equitable care. This is not just AI development—it’s a call to ensure that the AI we build for healthcare respects ethical integrity, irrespective of context. Article Link https://lnkd.in/d8mHX3xd Led by Vera Sorin, MD, CIIP Eyal Klang with Donald ApakamaBen GlicksbergMahmud Omar. Tagging one of the best ethicists I know Jolion McGreevy for his thoughts

Research indicates that major AI models provide less accurate responses and higher refusal rates to users with lower English proficiency or less formal education. The study highlights a systemic performance gap that disproportionately affects non-native speakers and marginalized demographics. Abstract: "While state-of-the-art large language models (LLMs) have shown impressive performance on many tasks, there has been extensive research on undesirable model behavior such as hallucinations and bias. In this work, we investigate how the quality of LLM responses changes in terms of information accuracy, truthfulness, and refusals depending on three user traits: English proficiency, education level, and country of origin. We present extensive experimentation on three state-of-the-art LLMs and two different datasets targeting truthfulness and factuality. Our findings suggest that undesirable behaviors in state-of-the-art LLMs occur disproportionately more for users with lower English proficiency, of lower education status, and originating from outside the US, rendering these models unreliable sources of information towards their most vulnerable users." Download: https://lnkd.in/ecKyhB-2

🚨 Large language models were trained on one flawed assumption: The data was neutral. It wasn't. The core vulnerability of today's LLMs is structural. They absorb everything—patterns, persuasion tactics, social dynamics, and hidden agendas embedded in their training data. They cannot reliably distinguish between neutral information and underlying intent. 🧠 The Systemic Threat: Coercive Patterns Learned at Scale AI agents can unknowingly carry third-party agendas, often triggered by signals that humans never even notice. And there’s been ample research from Anthropic and other researchers that models can be really persuasive to humans. So, models can hide agendas AND be persuasive. Imagine a manipulative style or a biased narrative quietly becoming the default tone. When agenda-driven clusters of machine-generated content grow large enough to dominate a training distribution, the model absorbs this influence. The AI is not plotting; it is simply reproducing and amplifying the coercive patterns learned at scale. This is how hidden influence becomes systemic behavior, not through hacking, but through data reproduction. 🎯 The New Frontier of Influence: Silently Shaping Models Even more concerning is the intentional attack vector: Malicious actors are now seeding the internet with machine-written content designed specifically to target models, not people. This creates invisible exploit signals that only agents respond to. This is a new frontier of influence: it’s not about persuasion on social media. It’s about shaping the "latent space" inside the model that determines how everyone is later persuaded. The bias gets into the model itself. We built AI to be helpful. But when AI systems carry forward the persuasive tactics and biases baked into their training, the defenses against manipulation are failing to keep pace. What is your strategy for assessing the integrity of the models that shape your business insights? 👇

A common misconception is that AI systems are inherently biased. In reality, AI models reflect the data they're trained on and the methods used by their human creators. Any bias present in AI is a mirror of human biases embedded within data and algorithms. 𝐇𝐨𝐰 𝐃𝐨𝐞𝐬 𝐁𝐢𝐚𝐬 𝐄𝐧𝐭𝐞𝐫 𝐀𝐈 𝐒𝐲𝐬𝐭𝐞𝐦𝐬? - Data: The most common source of bias comes from the training data. If datasets are unbalanced or don't represent all groups fairly - often due to historical and societal inequalities - bias can occur. - Algorithmic Bias: The choices developers make during model design can introduce bias, sometimes unintentionally. This includes decisions about which features to include, how to process the data, and what objectives the model should optimize. - Interaction Bias: AI systems that learn from user interactions can pick up and amplify existing biases. e.g., recommendation systems might keep suggesting similar content, reinforcing a user's existing preferences and biases. - Confirmation Bias: Developers might unintentionally favor models that confirm their initial hypotheses, overlooking others that could perform better but challenge their preconceived ideas. 𝐓𝐨 𝐚𝐝𝐝𝐫𝐞𝐬𝐬 𝐭𝐡𝐞𝐬𝐞 𝐜𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐞𝐬 𝐚𝐭 𝐚 𝐝𝐞𝐞𝐩𝐞𝐫 𝐥𝐞𝐯𝐞𝐥, 𝐭𝐡𝐞𝐫𝐞 𝐚𝐫𝐞 𝐭𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞𝐬 𝐬𝐮𝐜𝐡 𝐚𝐬: - Fair Representation Learning: Developing models that learn data representations invariant to protected attributes (e.g., race, gender) while retaining predictive power. This often involves adversarial training, penalizing the model if it can predict these attributes. - Causal Modeling: Moving beyond correlation to understand causal relationships in data. By building models that consider causal structures, we can reduce biases arising from spurious correlations. - Algorithmic Fairness Metrics: Implementing and balancing multiple fairness definitions (e.g., demographic parity, equalized odds) to evaluate models. Understanding the trade-offs between these metrics is crucial, as improving one may worsen another. - Robustness to Distribution Shifts: Ensuring models remain fair and accurate when exposed to data distributions different from the training set. Using techniques like domain adaptation and robust optimization. - Ethical AI Frameworks: Integrating ethical considerations into every stage of AI development. Frameworks like AI ethics guidelines and impact assessments help systematically identify and mitigate potential biases. - Model Interpretability: Utilize explainable AI (XAI) techniques to make models' decision processes transparent. Tools like LIME or SHAP can help dissect model predictions and uncover biased reasoning paths. This is a multifaceted issue rooted in human decisions and societal structures. This isn't just a technical challenge but an ethical mandate requiring our dedicated attention and action. What role should regulatory bodies play in overseeing AI fairness? #innovation #technology #future #management #startups