Yangfei Li

𝗧𝗵𝗲 𝗯𝗲𝘀𝘁 𝗔𝗜 𝗰𝗼𝗱𝗶𝗻𝗴 𝗹𝗲𝘀𝘀𝗼𝗻 𝗰𝗼𝘀𝘁 𝗼𝘂𝗿 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 $𝟲𝟬𝟬 𝗮𝗻𝗱 𝗮 𝗺𝗮𝗿𝗿𝗶𝗮𝗴𝗲 𝗮𝗿𝗴𝘂𝗺𝗲𝗻𝘁. Our VP of Engineering, Xiaofan(James) Luan, was supposed to buy his wife a Dior bag for their anniversary.  Instead, he bought three Claude Code subscriptions and spent the holiday trying to cross-compile 2 million lines of C++. Every fix on one platform broke two others.  $600 later, the only output was "git reset --hard" — and a very cold dinner table.😂 "Make it compile on Windows" is a trap. The real goal was "compile everywhere without hacks" — no AI is going to figure that out for you at 2 am. What worked: constraints before code, review tests not code, bottom-up, one layer at a time. Same task, two days. Then he ran six parallel Claude sessions across three machines with git worktree. The bottleneck stopped being intelligence and started being how fast one person can alt-tab. AI solves exactly the problem you give it. Engineering is in knowing which one to give. His wife is still waiting for that bag. Full story: https://lnkd.in/gtsW_Wvk ——— Follow Milvus, created by Zilliz, for everything related to unstructured data

9

Most existing #lean4 datasets contain only correct proofs. Models learn error correction with RL, that's expensive. With UW Math AI lab we release a dataset of 260k erroneous Lean proofs with - compiler feedback - reasoning trace - corrected proof Improvements in Error Correction: - Goedel 8B: 2x - Kimina 8B: 3x Paper: https://lnkd.in/gaybt4bd

117

2 Comments

Want to process massive graphs super fast using your GPU? Here's something exciting! 🔹 Introducing CRAPH – my early attempt at building a lightning-fast C++/CUDA library to traverse big graphs on GPUs. Think of it like a super-efficient toolkit that lets your computer explore huge networks (like social media or road maps) faster than before. Right now, it can scan graphs using two smart methods (based on vertices or edges), with tricks to speed things up and save space. 🔹 Future attempts: Even faster searching (BFS) Smarter handling of what parts of the graph to explore A way to find shortest paths using label propagation Support for graphs too big to fit in GPU memory! Got ideas? Suggestions? I'd love to hear them — help me make CRAPH more powerful and practical for real-world, large-scale graph work! Github link: https://lnkd.in/ghTBU2qu

48

4 Comments

Just arXived: MUSTAFAR – Promoting Unstructured Sparsity for KV Cache Pruning in LLM Inference How do we shrink the memory-hogging key-value cache in large language models—without fine-tuning or accuracy degredation? Meet MUSTAFAR, our new end-to-end system that: 🔍 Prunes at the finest granularity with per-token magnitude-based unstructured sparsity: up to 70% pruning on both Key & Value caches with negligible quality drop. Notably, we can prune the Value cache (something structured methods can’t do) to very high sparsity! 📦 Compresses the resulting sparse cache into a bitmap-based format, packing the cache down to 45% of its original size. ⚡️ Accelerates decode-time attention via a custom sparse attention kernel, boosting tokens/sec throughput by up to 2.23× over dense inference. 🛠️ Integrates with orthogonal methods like token eviction and quantization for even richer compression. 👉 Check out the preprint & code: Paper: https://lnkd.in/eXeA99EZ Code & Kernel: https://lnkd.in/eZ9fyQP9 Huge thanks to my co-authors Helya Hosseini, Ramyad Hadidi and advisor Bahar Asgari for her guidance and insights 🚀

85

2 Comments

Run Claude Code Locally on Apple Silicon Using LM Studio and LiteLLM (Zero Cost) In this blog, Manjunath Janardhan walks us through how to run Claude Code locally on Apple Silicon using LM Studio + LiteLLM + Qwen3-Coder-30B — with zero API cost and no cloud dependency. The setup uses: - LM Studio for local inference - Qwen3-Coder-30B as the coding model - LiteLLM as a bridge between Claude Code and a local OpenAI-compatible endpoint This is a practical way to get the Claude Code experience fully offline on macOS, especially for M-series Macs where MLX models can make a real difference. Worth reading if you want to run agentic coding tools on your own machine without relying on the cloud. Read it from the comments!

58

3 Comments

Why does an E3 level SWE at Meta make only ~190k/year, while an E8 level engineer makes over ~$2M/year, even though both engineers are ICs and spend the same time at work? I have spent the last 5 years at Meta as an IC. I joined with a little over 15 years of experience, and I’ve worked with many solid engineers in this time. Here is how I think about that compensation jump. 1. Same hours, completely different “unit of work” An E3’s unit of work is usually a task or a ticket. An E8’s unit of work is a multi year problem for the company. E3: “Implement this service, fix this bug, write this feature.” E8: “How do we cut infra cost by 20 percent across this product” or “How do we make this platform safe to scale to 10x users.” One person is paid to execute. The other is paid to decide what is worth executing in the first place. 2. Radius of impact E3 usually impacts a file, a service, maybe a small team. E8 shapes whole orgs and product lines. If an E3 ships something great, the impact is great but local. If an E8 ships the right platform, hundreds of engineers become faster and the company saves or earns millions every year. Comp tracks the area of the circle you influence. 3. Risk and downside protection At junior levels, mistakes are usually contained and reversible. At senior staff levels, a bad call can burn tens of millions or damage the brand. E8s are paid for judgment under ambiguity. They decide which bets the company should not make, which migrations can wait, which “shiny idea” is going to kill reliability. You pay more to people whose good judgment protects you from very expensive failures.  4. They scale themselves This can happen in a few ways. 1. Delegation with ownership They define the shape of the problem, then hand large pieces to other senior and mid level engineers while keeping the bar and direction clear. 2. Knowledge that travels They write RFCs, public comments, FAQs, wikis, internal posts. One answer helps hundreds of people who will face the same issue next quarter. 3. Tools over heroics Instead of unblocking people manually all day, they build tools, libraries or guardrails so others can unblock themselves. One well designed tool can save thousands of engineering hours every year. This is what “scaling yourself” actually looks like. The company pays for that multiplier.  5. Ownership of the “uncomfortable problems” Junior engineers usually work inside a well defined box. E8s take ownership between the boxes. They pick up problems that:  Span many teams and no one really “owns”  Require aligning leaders who disagree  Have product, infra, legal and security angles at the same time Most people avoid those because they are messy, political and slow. Very senior ICs lean into them. That is where a lot of value sits. Continued ↓

201

15 Comments