Chih-Hui Wang

Most existing #lean4 datasets contain only correct proofs. Models learn error correction with RL, that's expensive. With UW Math AI lab we release a dataset of 260k erroneous Lean proofs with - compiler feedback - reasoning trace - corrected proof Improvements in Error Correction: - Goedel 8B: 2x - Kimina 8B: 3x Paper: https://lnkd.in/gaybt4bd

117

2 Comments

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

We're hiring three AI Analytics Engineers on the Data Science & Analytics team at Airtable. These are early to mid-career roles for builders who are excited about working at the intersection of data science, analytics, and AI. One of these roles will report directly to me as we spin up a new AI & Analytics Platform function. Things are moving fast. The most impactful work in data right now isn't building models — it's fundamentally changing how a company gets answers from its data. That's where my focus is, and it's why Airtable is investing in a new team to own it. Over the past six months, our team stopped experimenting with AI on the side and restructured around it: - Agent-powered development workflows — I built shared AI systems for the data team embedded in how we write SQL, build pipelines, document business logic, and debug. Team members are regularly seeing 2x to 10x speed improvements — not from shortcuts, but from AI woven into every step of how we build. - Conversational analytics — Ad-hoc questions that used to fill our queue from leadership, product, GTM, and finance are increasingly answered by an AI agent — in natural language, where people already work. -A context layer that makes it trustworthy — We built a system that gives the agent real business knowledge — which table is correct for a given analysis, how metrics differ across teams, why something spiked last quarter. Context that usually lives in an analyst's head. - A learning loop that compounds — Every user correction becomes a permanent improvement, not a one-off fix. — We're doubling down. Airtable is standing up an AI & Analytics Platform function within DS&A to keep building at the intersection of data science, ML, analytics, and AI — and I'm leading it. We need more builders. AI Analytics Engineer, AI & Analytics Platform First hire on this team. You'll build the context layer that makes AI tools accurate, design evaluation frameworks that make them trustworthy, and develop agent systems that let the org self-serve with confidence. You'll work directly with me to shape what this function becomes. https://lnkd.in/gFstQqQb We're also hiring two roles embedded in other parts of the business: AI Analytics Engineer, Marketing Analytics Own data infrastructure, dashboards, and AI-native tooling for Marketing. Build the canonical data sources leadership relies on. https://lnkd.in/gYHS55ma AI Analytics Engineer, Business Analytics Own financial data models powering ACV, ARR, billings, and revenue. Partner with a Finance team that's building their own AI agents and workflows. You'll build the data foundation they run on. https://lnkd.in/gV5hnJjG — I'm looking for people who build before they're asked. Who have a side project that started as "I wonder if I could..." and turned into something real. Who pick up a new tool and immediately try to break it. If that sounds like you — reach out or apply. I'd love to talk.

79

8 Comments

🎉 New paper from #NVIDIAResearch 🧩 Puzzle: Distillation-Based NAS for Inference-Optimized LLMs+ Puzzle is a distillation-based NAS approach, that transforms trained LLMs into far more efficient, inference-aware (heterogeneous) architectures.

1