Ziqi Wang

Compressing LLMs but worried about accuracy? 🎯 EoRA uses eigenspace low-rank approximation to compensate for errors—no retraining needed. A promising direction for scalable, task-adaptive LLMs. New paper from #NVIDIAResearch.

1

Compressing LLMs but worried about accuracy? 🎯 EoRA uses eigenspace low-rank approximation to compensate for errors—no retraining needed. A promising direction for scalable, task-adaptive LLMs. New paper from #NVIDIAResearch.

1

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

Compressing LLMs but worried about accuracy? 🎯 EoRA uses eigenspace low-rank approximation to compensate for errors—no retraining needed. A promising direction for scalable, task-adaptive LLMs. New paper from #NVIDIAResearch.

3

A Chinese University just dropped a hardware-grounded coding model outperforming Claude 4.6! 🔥 Most LLMs excel at high-level programming but lack the grounding required for hardware-specific constraints and specialized industrial languages. Beihang University introduced InCoder-32B, a foundation model specifically architected for chip design, GPU optimization, and embedded systems. It utilizes a standard decoder-only Transformer architecture with 32 billion parameters, trained on a massive 15-trillion-token corpus using 4,096 GPUs. It is trained on a three-stage "Code-Flow" pipeline: • During Pre-training, the team used a multi-step strategy including OCR extraction from technical manuals and hardware books, to capture rare industrial data usually missing from public repositories. • Mid-training stage progressively extends the context window from 8K to 128K tokens to support complex industrial projects. This phase also introduces agentic trajectories using a Thought-Action-Observation cycle to improve the model's logical reasoning in troubleshooting scenarios. • Post-training moves beyond text, validating outputs through real industrial toolchains like Icarus Verilog, NVIDIA A100 nodes, and the Renode simulator. InCoder-32B establishes the strongest open-source results across all evaluated industrial domains. It even outperformed Claude-Sonnet-4.6 in specific domains like CAD-Coder IoU and KernelBench performance! Paper link in comments !! Follow for daily high-signal analysis of latest AI research! 🧠 #AI #ChipDesign #Transformer #MachineLearning #GPUOptimization #Engineering #NVIDIA #China #LLM #Coding

4

1 Comment

And incidentally, with this formulation, you end up using existing vendor-optimized numerical libraries efficiently, and ultimately, extracting more effectively the performance of your hardware. #matricization #HPC

26

Precisely. 8 years ago, Steve Brierley from Riverlane and I started a collaboration on #benchmarking #quantum systems, inspired by benchmarking #AI systems. Since then, there has been progress on both sides, notably the emergence of MLPerf as the de facto standard for the latter. Curious if quantum computing has reached a similar tipping point for building a strong community around benchmarking?

22

2 Comments