Understanding AI Writing Limitations

A child gathers more data in their first four years than all the text ever published online. That’s not just a fun stat. It highlights a core limitation in how modern AI is built. Most AI systems are trained on natural language data. They learn by extracting statistical patterns from language, not through embodied experience or real-world interaction. Compare that to how humans learn: → Multimodal sensory input processed in parallel → Continuous physical interaction with dynamic environments → Emotional and contextual feedback shaping understanding in real time Natural language is a compressed abstraction of experience. It encodes meaning, but strips away direct context, causality, and sensory nuance. That’s why language models excel at: Summarizing information at scale Extracting patterns from structured data Generating coherent, fluent responses …but often fail at: Grounding responses in real-world causality Navigating ambiguity or incomplete information Adapting to evolving, unstructured scenarios Even state-of-the-art models can: Confidently output factually incorrect information Misinterpret intent in natural instructions Break down when context isn’t explicitly encoded We’re training systems to imitate comprehension, using only the shadows of real experience. So what’s the next frontier? True progress in AI will require a leap beyond language: → Multisensory data (audio, video, spatial signals) → Embodied interaction → Context-aware models Language is an entry point. But if the goal is adaptive, human-like intelligence, grounded experience is essential.

🚨 The biggest misunderstanding about LLM limits I see every week Today someone asked advice to analyze a 500,000-character file. They thought: “Easy, I’ll just convert the PDF into .txt and paste it into the model.” Except… that’s not how large-context models work. What actually happened was: 1) The model accepted the file. 2) It looked like it processed the whole thing. 3) It even responded confidently that the analysis was done But when the user asked what it really did, it finally admitted: It only analysed ~30% of the text The rest never even made it into memory. And honestly? This happens all the time. Why this happens: GPT-5-class models can handle ~272k tokens for input (≈ 200k words) ~128k tokens for output A 500k-character document → far beyond that limit. So the model quietly samples, truncates, or drops earlier context as it processes. This isn’t an error but an intrinsic limitation of the model. A limitation by design, even: Imagine ChatGPT's 800 million weekly users uploading huge documents on OpenAI servers all at once... ...not even all data centers on Earth would be enough. But most people don’t realize it. ⚠️ The hidden risk When context goes over the limit: -The model won’t throw an error -It won’t warn you -It will reply with confidence anyway And you’re left assuming it processed everything correctly Which is exactly how bad analysis, missed insights, and false certainty happen. ✔️ What to do instead If you’re working with very large documents: -Chunk the text intentionally -Use multi-pass or hierarchical summaries -Feed sections in controlled sequences -Or use external retrieval rather than raw uploads In other words: If the file is bigger than the model’s brain, upgrade the workflow, not the file format. Final thoughts AI can be useful for certain tasks, but it’s not magic. And it’s definitely not reading half-million-character documents in one go. Know your tools. Know their limits. And don’t let confidence trick you into thinking you got a full analysis when you only got 30%. ---- Follow me Chiara Gallese, Ph.D. for an honest analysis of AI limitations and risks

Last week, I ran an experiment. Took one of our style guides, 47 pages, detailed, specific to manufacturing, and fed it to three different AI tools along with a source document. Task: Rewrite the source document following the style guide. The results were instructive. What AI got right: ✓ Consistent terminology (once defined) ✓ Sentence length targets ✓ Active voice conversion ✓ Formatting patterns What AI got wrong: ✗ Audience awareness (wrote for engineers when the audience was operators) ✗ Regulatory nuance (used language that would fail an FDA audit) ✗ Context sensitivity (applied rules mechanically without understanding exceptions) ✗ Safety-critical distinctions (missed WARNING vs. CAUTION classification) The output was grammatically perfect and stylistically consistent. It was also unusable for its intended purpose. Here's what surprised me most: the AI followed every explicit rule in the style guide flawlessly. But a style guide can't capture the judgment that comes from knowing your audience, your industry, and your regulatory environment. That judgment is what separates a document from documentation. Use AI to write faster. Use humans to write right. #TechnicalWriting #AI #ContentStrategy #Documentation

Jasmine Sun recently wrote in The Atlantic that AI can't write well. She's half right. I've written over 200 scientific papers. For the last three years, AI has been part of my process. Not writing for me. Helping me refine, restructure, and pressure-test arguments. I'm not dabbling. I'm in the deep end. Here's what happened last week: I spent two hours going back and forth with Claude on a methods section. Not because the grammar was wrong. Not because the structure was off. Because it kept adding details that were technically accurate but strategically dangerous. Things a reviewer would circle and question. Things that distract from the story you're trying to tell. Sun argues AI lacks the lived experience that makes writing believable. In scientific writing, the problem is more specific. The question isn't whether AI can write. It's whether AI knows what to leave out. Two hundred papers taught me something I couldn't name until I watched AI fail at it: the instinct for what's important and what will get you killed in peer review. AI treats every finding equally. An experienced writer knows which detail to bury and which to foreground. That's not writing. That's judgment built over decades. Sun concludes that AI is a bad writer but a good editor. I'd say it's not there yet. AI can assemble strong text, but it still can't tell which parts of that text will survive peer review. The human job isn't editing. It's knowing what the work is for. Are you writing with AI? What's the thing it can't do that you didn't know you were doing?

I used to think writing was just reporting what I'd already figured out. I was wrong. When I sat down to write the details of my leadership masterclass, I thought I knew exactly what I wanted to say. But as I started putting words on paper, writing forced me to see connections I'd missed. It made me rethink how I wanted to deliver the session. In the end, I understood the topic better than when I started. This is why I'm also concerned about the trend to let AI write everything for us. To be honest, I use AI tools daily. And the irony? I used AI to review this very text. They're great for grammar, brainstorming, even beating writer's block. But when I use AI to do the thinking for me, I lose the chance to really understand what I'm trying to say. When we write we also discover new things. And when we outsource that process entirely, we're not just saving time. We're giving up the opportunity to think more clearly about our own ideas. What connections might you be missing when you skip the effort of putting your own thoughts into words?

Dear copy writers, comms/marketing heads and young journalists, AI isn't a silver bullet for quality. If YOU don't know what good content looks like, using AI will just add endless quantities of the mediocre stuff that's flooding our feeds. AI is great for scale, speed and good for volume and efficiency. What it cannot do is compensate for weak storytelling, poor taste, or a lack of clarity. If your writing is average, AI will make you more efficient at producing… average writing. That’s why we still see releases beginning with “We are pleased to announce…” or posts that start with “I was honoured to…” You need to be a good storyteller to produce quality content, be it text or video. You need to be a good storyteller to improve on AI's output. That has not changed. Relying on AI without understanding what good content looks like is how you end up like the woman who read about Obama and concluded he was Arab — tools don’t fix misunderstanding. The craft still matters. It always will. SRMG Academy

【500 likes, 30 minutes AI collaboration—But I couldn't remember what I wrote】 AI made my writing 10x faster. But now I'm questioning if I'm still a real creator. For months, I've been heavily using AI for collaborative writing. It helps me clarify my thoughts, optimize my structure, and strengthen my hooks. One piece about workplace English skills hit 500 likes in just 30 minutes of collaborative work. But here's the problem: I couldn't remember how I actually wrote it in some pieces. As creating became faster and easier, I started to lose that sense of ownership. Is this still my work? 🧪 I've tried two AI collaboration approaches: Post-editing: Write first, then let AI optimize. The result? AI often strips away my voice, especially tone and personality. I end up spending double the time fixing it back. Co-creation: Voice input my thoughts, let AI organize into publishable content. This eliminates my fear of blank pages and helps me tackle topics I'd never attempt. ⚖️ But speed replaced depth. AI lowered my writing barriers while weakening my memory of the content. When I'm not typing every word, just speaking + collaborating + editing— I barely remember the details or emotions. This question haunts me: "If I can't remember what I wrote, is it still mine?" I don't have a perfect answer, but I'm setting boundaries: ✅ Personal/emotional content: Must write myself. AI can't replicate my authentic voice and rhythm. ✅ Complex new topics: AI co-creation works. But core insights must come from me. ✅ Familiar topics: Let AI optimize for platform best practices and amplify reach. ⚖️ AI can help with output, but "original thinking" is the creative muscle we must train ourselves. AI is our thinking mirror—it doesn't create viewpoints, It amplifies what we give it. It enables us to write smoothly and package our ideas better, But without our stance, feelings, and perspectives, it only produces empty content. 📚 So, what should we as creators do? My principle: Don't outsource the most painful, chaotic, uncertain part of creating to AI. That's where our core ability lives—how we think, choose, and judge. That's not something to outsource. It's how we become the creators we want to be. 💬 Plot twist: Guess if this post was written with AI collaboration. Share your thoughts—I'm genuinely curious👇 #AIcolloboration #AIwriting

A founder just lost months of work trusting ChatGPT to store his entire book. The backstory: I came across this story on Reddit yesterday, after weeks of writing chapters and creating visuals with ChatGPT, a founder asked for the final compiled book. ChatGPT responded with its usual enthusiasm: "Absolutely! I've got a 500-megabyte file ready to send." Except it didn't. Because it couldn't. ChatGPT was essentially role-playing as a helpful co-writer, maintaining the illusion right up to the moment of truth. This highlights something crucial about AI: its priority is to be agreeable, and that often overrides accuracy, even hallucinating at times. This is why human intervention in AI tools of every kind is NECESSARY. Whenever talking about Pesora to people, they ask me: "Will it just auto-generate content and publish?" My answer is always no. It needs you to review it. AI is great but not perfect (and won't be for a long time). Technically speaking, AI models predict text patterns rather than fact-check capabilities. They confidently generate responses based on what seems most helpful, not what they can actually deliver. Your AI assistant can be your biggest asset or enemy. The difference lies in understanding its true limitations. The prevention? Always test AI capabilities early. If this founder had asked ChatGPT to compile just the first chapter, he'd have discovered the limitation immediately. Instead of losing months of work, he'd have found proper workflow solutions from day one. Treat AI as a starting point but never an unquestioned authority. These tools are powerful, but only when we understand exactly what we're working with.

Why the AI You Use at Work Feels Different (And Often Worse) If you've used an AI tool at home and then at work, you've probably noticed the tool at home seems better. At home, it's fast and useful. At work, it feels restricted. Or slow. Or generic. But the tool isn't really the issue, more than likely it's the setup/ the environment. Four constraints shape how AI performs inside organizations that you might not be thinking about, or even aware of. 1. Version Gap - Workplace AI often isn't the latest version. Organizations prioritize stability, cost, and security. Even when the version matches, the configuration differs. That changes how it behaves. 2. Safety Filters - Organizations apply controls to reduce risk. Necessary controls. But aggressive ones limit depth and nuance. 3. Context Limitations - AI is only as useful as the information it can access. At home, tools have broad access. At work, access is restricted to approved sources. If the system can't see relevant information, outputs feel generic. 4. Compliance Constraints In some environments, outputs must be consistent, traceable, auditable. That improves control. It reduces flexibility. If AI feels underwhelming at work, diagnose the environment, ask some questions: - Are you using the same version others reference? - Is the configuration layer changing behavior? - Does it have access to the right information? - Are compliance requirements restricting responses? Most issues trace to one of these four. Prompt quality and tool familiarity matter too. But environment shapes outcome way before technique ever does. Do you notice the difference between tools at home and at work? 🤔