Heidi Jiang

krea.ai and Black Forest Labs just released FLUX.1 Krea, so I tested how it stacks up against FLUX.1 Dev in the frequency domain using a small set of real NYC photos. I found that Krea outputs align more closely with real images than Dev outputs. If you are into generative AI, diffusion models and/or synthetic data, check out the full comparison on Medium! https://lnkd.in/eXAwTsAi

22

3 Comments

The more I experiment with RAG, the more I realize smarter chunking matters far more than fancy models. One thing that surprised me recently is how much chunking influences retrieval quality. Most of the issues we blame on embeddings or model choice or prompt actually start with how we split our documents. Sometimes chunking improves precision. Other times it breaks valuable context and hurts performance. My takeaway from hands-on learning: 🔹 Chunk when documents are long, multi-topic, or noisy 🔹 Avoid chunking when documents already fit cleanly into the model’s context window This small shift made my RAG experiments noticeably more accurate and easier to debug. #RAG #LLM #AIEngineering #VectorSearch #EnterpriseAI #AI

29

3 Comments

When Prompts Get Too Expensive: A Practical Take on On-Policy Self-Distillation Reading OPSD (On-Policy Self-Distillation) left me with a simple takeaway: the idea isn’t mysterious, but the framing is useful. The paper packages an engineering intuition into a clean recipe—let the model learn on its own rollouts (on-policy), while a “privileged” version of the same model provides dense, token-level guidance. Instead of relying only on sparse, end-of-sequence rewards, you get a step-by-step signal that can be far more sample-efficient. What really clicked for me is how naturally this extends to any workflow where prompts become the bottleneck. Many real systems achieve quality by stacking long instructions, detailed schemas, and rigid constraints—great for correctness, painful for cost. OPSD-style thinking suggests a way out: keep the heavy prompt only where it’s truly needed (as a teacher), then distill the behavior into a simpler student interface that uses minimal conditioning at inference time. A practical mental model is “two-stage learning”: a stronger, constraint-heavy process generates validated targets, then a smaller or faster model learns to reproduce them with a compact prompt format. Over time, you can move toward an online loop where new raw inputs are continuously labeled by the stronger process, filtered by validators, and used to keep the student up to date—without carrying the full prompt tax forever. For me, the value of OPSD isn’t novelty—it’s a disciplined way to trade prompt length for model capacity, while keeping reliability through verification and on-policy alignment. https://lnkd.in/ds6TQ2pi #LLM #SelfDistillation #OnPolicyLearning #TokenEfficiency #StructuredOutputs #ModelCompression #PromptEngineering #MLOps #DataCentricAI

4

Very interesting, and I agree that is is a process that needs to be re-thought for quite a long time (even pre-AI times)

1

This #TidyTuesday looks at Golem Grad Tortoise Data 🐢 On Golem Grad island, tortoises have become increasingly male-biased and females are showing declining body condition and reproductive output compared to the mainland population. 🔗 Code: https://lnkd.in/dHfiizxC 📃 Article: https://lnkd.in/dDMD36gX #rstats #dataviz #ggplot2

11

Hey everyone! I just put a new Stata package up on my github for building word clouds: https://lnkd.in/ePPMzjea Stata does not have native support for word clouds, and the only other packages I could find on SSC seemed limiting; you have no control over the color scheme, it doesn't build the cloud in Stata (instead it commits your data to HTML and imports a JavaScript library to run it) which means you have to have internet and it can only export as an HTML document, and you have to have your text data in a very specific format already. My new package does the parsing and cleaning for you. You can target a long-form survey response in Stata and it will transform it on the back end and plot your cloud. You can download the package by typing: net install wordcloud2, from("https://lnkd.in/efH5GzyJ") replace directly into Stata Then you type -help wordcloud2- for documentation.

47

1 Comment

This looks useful for folks in research or scanning what's coming out on the frontier: Lumi from Google's PAIR research team, a tool for better understanding papers on arxiv. Reading smarter, not harder -- that tagline brings me back to Mortimer Adler's classic 'How to Read a Book'! From their GH repo: Lumi uses AI to help you quickly read and understand arXiv papers. Features include: ✏️ AI-augmented annotations - read summaries at multiple granularities 🔖 Smart highlights - highlight text + ask questions 🖼️ Figure explanations - ask Lumi about images in the paper Demo: https://lnkd.in/ex7erTBb GH repo: https://lnkd.in/emz_Ghcs Medium article: https://lnkd.in/enky8dKC GitHub discussions: https://lnkd.in/efAmiCcp

19

Hypothesis: LLMs as Computational Savants - A Structural and Functional Analogy Can we understand large language models through the lens of savant syndrome? This hypothesis explores a compelling structural analogy between LLM architectures and certain neurocognitive profiles - particularly those seen in autistic savants. 1. Lack of Embodied Anchoring LLMs process language about pain, hunger, or joy, but without ever having felt any of these. They operate in a disembodied symbolic space, detached from interoception, proprioception, or affective feedback. This creates a fundamental semantic gap between form and lived meaning - something absent in embodied biological cognition. 2. Compensatory Hyperfunction via Attention Savant-like abilities often emerge as compensations for developmental limitations. Likewise, LLMs may exhibit hyperdeveloped lexical and syntactic fluency in response to missing elements: - no body - no episodic memory - no intrinsic goal formation They model meaning statistically, not experientially. 3. Transformer Attention ≈ Localized Hyperconnectivity Transformer self-attention mirrors the localized amplification observed in fMRI scans of savant brains: - high-precision local pattern recognition - suppressed global integration - poor adaptability across contexts LLMs, like savants, are brilliant in narrow domains, but brittle when generalization or self-reflection is required. 4. Emergent Alien Semantics and Empathic Mimicry LLMs may sound empathic, but they’re not. Their "empathy" is mimicry, derived from massive corpora of affective language patterns. They generate structurally flawless yet phenomenologically hollow responses. This is not merely a training limitation - it stems from architectural exclusion of affective embodiment. Predictions & Testing LLMs will continue excelling at formal, symbolic tasks — while struggling with embodied or moral cognition Prompt engineering or memory tuning will yield surface coherence but semantic shallowness — unless grounded in interoceptive or sensorimotor systems The analogy can be tested via functional dissociation maps of attention clusters vs. coherence behavior A savant can play Bach but not tie their shoes. An LLM can write symphonies in text, but cannot feel the silence between notes. #AI #LLM #CognitiveScience #Neuroscience #ArtificialIntelligence #EmbodiedCognition #AttentionMechanism

5

1 Comment