Jing Chen

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

Excited to share our new paper published in Human Factors: “Inferring Hidden Attentional States in Driving: A Bayesian Approach to Modeling Distraction and Secondary Task Engagement.” This work was led by Lekhapriya Dheeraj Kashyap as part of her PhD research, and I’m grateful to have collaborated on it. Many real-world systems face the same challenge: the most important states are hidden. In driving, we can observe signals such as speed, eye movements, or pupil dilation — but not the driver’s true attentional state. In this work, we develop a Bayesian decision framework based on a Partially Observable Semi-Markov Decision Process (POSMDP) to infer latent attentional states and model how drivers allocate attention between competing tasks. The model detects distraction earlier than common heuristic rules and reveals substantial heterogeneity in attention strategies. Beyond driving safety, problems like this appear in many AI systems where human states are latent and decisions unfold sequentially. Great collaboration with Lekhapriya Kashyap, Yanling Chang, Maryam Zahabi, and Alfredo Garcia. 📄 Paper: https://lnkd.in/g23ubzht

9

Love this direction. Designing rewards that are dense while keeping benchmarks hard to hack is the right way to push LLM evaluation forward. I also appreciate how the paper introduces example-specific rubrics — making LLM judging aligned with actual user expectations instead of vague, artificial criteria. Clear, grounded rubrics are exactly what the evaluation ecosystem needs. Highly recommend reading. 👇

3

We at Haladir (YC W26) released a report on RLFR: Reinforcement Learning from Formally-Defined Rewards for Code Generation (https://lnkd.in/guCH7w7C). The core idea is simple: we present a simple offline and single-turn RL framework that uses formal verification as the primary reward signal for program synthesis in LLMs rather than unit test-based rewards (RLVR), showing improvements over a swath of different coding benchmarks. We fine-tune both Qwen2.5-Coder-7B-Instruct and Qwen3-8B using RLFR and Dafny, a specification language that utilizes Boogie (IVL) and Z3 (SMT solver) to check verification conditions. Despite solely training on Dafny, we show improvements on Python-native coding benchmarks, suggesting generalization past the formal verification domain.

90

8 Comments

🚀 Excited to share our latest work on advancing Retrieval-Augmented Generation (RAG) systems! Our paper --- Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decoding --- will be presented at the KDD 2025(W) on Inference Optimization for Generative AI. 📄 Paper: https://lnkd.in/dErdTDf7 What's the Challenge? Traditional RAG systems struggle with: Static retrieval strategies that don't adapt to context Inefficient memory usage in long-context applications Suboptimal balance between retrieval fidelity and response quality Computational bottlenecks during inference Our Solution: PORAG + ATLAS Framework We introduce two complementary techniques: PORAG (Policy-Optimized RAG): Extends Group Relative Policy Optimization (GRPO) to RAG settings Uses dual reward heads for retrieval fidelity AND response quality Employs QLoRA for parameter-efficient fine-tuning Prevents catastrophic forgetting while optimizing retrieval utilization ATLAS (Adaptive Token-Layer Attention Scoring): Multi-Layer Attention Gradient (MLAG) analysis detects information gaps Layerwise Representation Pooling (LRP) constructs targeted queries Dynamic retrieval scaling based on computational load Only retrieves when truly necessary CRITIC for Memory Optimization Our Cache Reduction via Importance-based Token Inclusion Criteria (CRITIC) technique: Reduces memory usage significantly Maintains performance with minimal quality trade-offs Enables longer context processing Test-Time Scaling Integration We demonstrate how inference techniques like Self-Consistency and Monte Carlo Tree Search further enhance performance on complex reasoning tasks. Looking forward to presenting at KDD 2025! Thanks to my amazing teammates Akash Das, SHIVAM GUPTA and Venkataramana Runkana for their incredible contributions! #KDD2025 #RAG #LLM #MachineLearning #AI #RetrievalAugmentedGeneration #DeepLearning #NLP #InferenceOptimization

35

We had great papers and demos on 12/9 at #ASRU. Here are some of them: 👉 Selection of Layers from Self-supervised Learning Models for Predicting Mean-Opinion-Score of Speech, Xinyu Liang, Fredrik Cumlin, Victor Ungureanu, Chandan K. A. Reddy, Christian Schuldt, Saikat Chatterjee. Investigate which layers of self-supervised speech models are most effective for speech quality assessment. Early layers of self-supervised models are better than final layers for predicting speech quality (MOS), leading to simpler, superior SQA models. 👉Transcribe, Translate, or Transliterate: An Investigation of Intermediate Representations in Spoken Language Models Tolúlopé Ògúnrèmí, Christopher Manning, Dan Jurafsky, Karen Livescu. Analysis shows Whisper-based SLMs form a semantic interlingua, while ASR-only encoders yield phonetic transliterations, revealing different audio-to-text linking methods. 👉Competitive Audio-Language Models with Data-Efficient Single-Stage Training on Public Data, Gokul Karthik Kumar, Rishabh Saraf, Ludovick Lepauloux, Abdul Muneer, Billel Mokeddem, Hakim Hacid. Falcon3-Audio, trained end-to-end in a single stage on fewer than 30K hours of fully public audio. The 7B model matches state-of-the-art open-weight performance on the MMAU benchmark, suggesting that complex training curricula and multi-encoder architectures are not necessary for competitive audio understanding. Also, the conversational AI demo by Tatiana Likhomanenko was amazing!!

31

1 Comment

Our latest paper has been accepted at AAAI 2026. We introduce Structural Alignment, a reinforcement learning approach that aligns large language models with human-like discourse structures, enabling more coherent and well-organized long-form text generation. Our paper, “Structural Alignment: Aligning Large Language Models with Structural Information,” is available here: https://lnkd.in/gx3ZRTZu

49

Reinforcement Pre-Training (Microsoft, June 2025) Paper: https://lnkd.in/eY4PCpUi Abstract: "In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling paradigm for large language models and reinforcement learning (RL). Specifically, we reframe next-token prediction as a reasoning task trained using RL, where it receives verifiable rewards for correctly predicting the next token for a given context. RPT offers a scalable method to leverage vast amounts of text data for general-purpose RL, rather than relying on domain-specific annotated answers. By incentivizing the capability of next-token reasoning, RPT significantly improves the language modeling accuracy of predicting the next tokens. Moreover, RPT provides a strong pre-trained foundation for further reinforcement fine-tuning. The scaling curves show that increased training compute consistently improves the next-token prediction accuracy. The results position RPT as an effective and promising scaling paradigm to advance language model pre-training."

7

1 Comment

GEPA over RL. "GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning" link: https://lnkd.in/gJfNJQZc "At the heart of this paper is a singular, transformative claim that challenges the RL paradigm for agent optimization. "The optimal set of instructions governing a compound Al system is discovered more efficiently through an evolutionary process of reflective, language-based self-correction than through numerical policy optimization guided by scalar rewards" #ai

4

💥 ICML-25 Paper Alert! Introducing the JETTS benchmark, Judge Evaluation for Test-Time Scaling ⏲️ LLM-judges specialize in evaluating responses generated by other LLMs. Test-time scaling requires multiple model responses to be evaluated for the best one. So, can LLM-judges help test-time scaling? Check this thread to find the answer and some interesting analysis👇 Repo: https://bit.ly/3YteE6Z X: https://lnkd.in/gXvQsd4C Paper: https://bit.ly/4jtRGVM

105