Weitong Liu

𝗧𝗵𝗲 𝗯𝗲𝘀𝘁 𝗔𝗜 𝗰𝗼𝗱𝗶𝗻𝗴 𝗹𝗲𝘀𝘀𝗼𝗻 𝗰𝗼𝘀𝘁 𝗼𝘂𝗿 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 $𝟲𝟬𝟬 𝗮𝗻𝗱 𝗮 𝗺𝗮𝗿𝗿𝗶𝗮𝗴𝗲 𝗮𝗿𝗴𝘂𝗺𝗲𝗻𝘁. Our VP of Engineering, Xiaofan(James) Luan, was supposed to buy his wife a Dior bag for their anniversary.  Instead, he bought three Claude Code subscriptions and spent the holiday trying to cross-compile 2 million lines of C++. Every fix on one platform broke two others.  $600 later, the only output was "git reset --hard" — and a very cold dinner table.😂 "Make it compile on Windows" is a trap. The real goal was "compile everywhere without hacks" — no AI is going to figure that out for you at 2 am. What worked: constraints before code, review tests not code, bottom-up, one layer at a time. Same task, two days. Then he ran six parallel Claude sessions across three machines with git worktree. The bottleneck stopped being intelligence and started being how fast one person can alt-tab. AI solves exactly the problem you give it. Engineering is in knowing which one to give. His wife is still waiting for that bag. Full story: https://lnkd.in/gtsW_Wvk ——— Follow Milvus, created by Zilliz, for everything related to unstructured data

9

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

RAG is becoming one of the most critical techniques for improving LLM accuracy — not just a research concept, but a practical system design pattern. It's exciting to see Googlers sharing real-world implementation experience and helping more developers build reliable AI

3

Google Research Director Denny Zhou, who founded the LLM Reasoning Team at DeepMind, recently gave a great talk at Stanford’s CS25 class: Large Language Model Reasoning. This was one of the most intuitive talks about reasoning model training and application. He outlined new directions for training models to handle questions where answers aren’t easily verifiable. My team is currently working on an enterprise LLM project focused on using reasoning to extract relevant context and generate responses for unverifiable cases. It’s a tough but promising area, and it’s an interesting time to be building. [Link to Denny Zhou's talk: https://lnkd.in/gdJ6Mzi8]

1,026

8 Comments

👉 My Key Takeaways from Chip Huyen's Recent Interview on Lenny's Podcast Chip Huyen is the author of the widely recognized "AI Engineering: Building Applications with Foundation Models". Link to the podcast: https://lnkd.in/dxZ-tFWX 💡 Importance of post-training.  Pre-training gives you raw capabilities (next token prediction on massive data), but post-training is what makes the model actually usable. SFT on high-quality examples + RLHF. Fine-tuning should be your last resort, not first. Most problems can be solved with better prompts, better data, or RAG. 💡 Evals.  You can't improve what you can't measure. Need multiple types: unit tests (does this specific prompt work?), integration tests (does the whole pipeline work?), regression tests (did we break something?), and user feedback loops. The hardest part isn't writing evals; it's maintaining them as your product evolves. 💡AI products.  Reliability and UX matter more than models. Most AI product failures aren't about bad models: they're about reliability (API limits, latency spikes, poor monitoring) and UX (users don't understand how to use it, doesn't fit workflow). Building reliable platforms and talking to users constantly beats chasing SOTA models. Most insights come from watching users, not from benchmarks. 💡How to improve AI-powered apps.  What people think improves apps: staying current on AI news, chasing newest agentic framework, obsessing over vector database choice, constantly evaluating model benchmarks, fine-tuning models. What actually improves apps: talking to users, building reliable platforms, preparing better data, optimizing end-to-end workflows, writing better prompts. Better prompt engineering beats switching models 90% of the time. A well-crafted system prompt, clear instructions, good examples (few-shot), and proper output formatting can transform a mediocre experience into a great one. 💡 Advice for builders.  Start with user problem, not with cool AI technique. Use the simplest solution that works (often that's a good prompt, not a fine-tuned model). Build evals early. Focus on end-to-end experience. Don't fine-tune unless you've exhausted everything else. Don't treat AI as deterministic (it's not, you need to handle variability). Don't ignore data quality (garbage in, garbage out).

1

1 Comment

Why does an E3 level SWE at Meta make only ~190k/year, while an E8 level engineer makes over ~$2M/year, even though both engineers are ICs and spend the same time at work? I have spent the last 5 years at Meta as an IC. I joined with a little over 15 years of experience, and I’ve worked with many solid engineers in this time. Here is how I think about that compensation jump. 1. Same hours, completely different “unit of work” An E3’s unit of work is usually a task or a ticket. An E8’s unit of work is a multi year problem for the company. E3: “Implement this service, fix this bug, write this feature.” E8: “How do we cut infra cost by 20 percent across this product” or “How do we make this platform safe to scale to 10x users.” One person is paid to execute. The other is paid to decide what is worth executing in the first place. 2. Radius of impact E3 usually impacts a file, a service, maybe a small team. E8 shapes whole orgs and product lines. If an E3 ships something great, the impact is great but local. If an E8 ships the right platform, hundreds of engineers become faster and the company saves or earns millions every year. Comp tracks the area of the circle you influence. 3. Risk and downside protection At junior levels, mistakes are usually contained and reversible. At senior staff levels, a bad call can burn tens of millions or damage the brand. E8s are paid for judgment under ambiguity. They decide which bets the company should not make, which migrations can wait, which “shiny idea” is going to kill reliability. You pay more to people whose good judgment protects you from very expensive failures.  4. They scale themselves This can happen in a few ways. 1. Delegation with ownership They define the shape of the problem, then hand large pieces to other senior and mid level engineers while keeping the bar and direction clear. 2. Knowledge that travels They write RFCs, public comments, FAQs, wikis, internal posts. One answer helps hundreds of people who will face the same issue next quarter. 3. Tools over heroics Instead of unblocking people manually all day, they build tools, libraries or guardrails so others can unblock themselves. One well designed tool can save thousands of engineering hours every year. This is what “scaling yourself” actually looks like. The company pays for that multiplier.  5. Ownership of the “uncomfortable problems” Junior engineers usually work inside a well defined box. E8s take ownership between the boxes. They pick up problems that:  Span many teams and no one really “owns”  Require aligning leaders who disagree  Have product, infra, legal and security angles at the same time Most people avoid those because they are messy, political and slow. Very senior ICs lean into them. That is where a lot of value sits. Continued ↓

201

15 Comments

Most existing #lean4 datasets contain only correct proofs. Models learn error correction with RL, that's expensive. With UW Math AI lab we release a dataset of 260k erroneous Lean proofs with - compiler feedback - reasoning trace - corrected proof Improvements in Error Correction: - Goedel 8B: 2x - Kimina 8B: 3x Paper: https://lnkd.in/gaybt4bd

117

2 Comments