Zhiwei Xin

If you're a Senior Engineer looking to move up, the next role will likely be a Staff Engineer. And in this guide, Shruti shares tips from her own experience of getting promoted to Staff Engineer at PayPal and Slack. She talks about what Staff Engineers do (and how it's different from Seniors), why you might not be getting promoted, and how to take that next step. https://lnkd.in/g58dnFEG

103

3 Comments

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

🎉 New paper from #NVIDIAResearch 🧩 Puzzle: Distillation-Based NAS for Inference-Optimized LLMs+ Puzzle is a distillation-based NAS approach, that transforms trained LLMs into far more efficient, inference-aware (heterogeneous) architectures.

1

𝗧𝗵𝗲 𝗯𝗲𝘀𝘁 𝗔𝗜 𝗰𝗼𝗱𝗶𝗻𝗴 𝗹𝗲𝘀𝘀𝗼𝗻 𝗰𝗼𝘀𝘁 𝗼𝘂𝗿 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 $𝟲𝟬𝟬 𝗮𝗻𝗱 𝗮 𝗺𝗮𝗿𝗿𝗶𝗮𝗴𝗲 𝗮𝗿𝗴𝘂𝗺𝗲𝗻𝘁. Our VP of Engineering, Xiaofan(James) Luan, was supposed to buy his wife a Dior bag for their anniversary.  Instead, he bought three Claude Code subscriptions and spent the holiday trying to cross-compile 2 million lines of C++. Every fix on one platform broke two others.  $600 later, the only output was "git reset --hard" — and a very cold dinner table.😂 "Make it compile on Windows" is a trap. The real goal was "compile everywhere without hacks" — no AI is going to figure that out for you at 2 am. What worked: constraints before code, review tests not code, bottom-up, one layer at a time. Same task, two days. Then he ran six parallel Claude sessions across three machines with git worktree. The bottleneck stopped being intelligence and started being how fast one person can alt-tab. AI solves exactly the problem you give it. Engineering is in knowing which one to give. His wife is still waiting for that bag. Full story: https://lnkd.in/gtsW_Wvk ——— Follow Milvus, created by Zilliz, for everything related to unstructured data

9

You are in a system design interview at Google for the L5 Senior Engineer role, and the interviewer leans in and asks: “Why does Spotify keep playing when I drive into a tunnel with no signal, but YouTube Music often stops or buffers? If you were designing a music streaming system, what different design choices would lead to these two behaviors?” Here is how you break it down. Btw, if you’re preparing for system design/coding interviews, check out our mock interview tool. You can use it for free here: https://lnkd.in/gpCn7t2T We’ve added new features as well:  -Company-specific interviews -In-built interview scheduler  -performance insights & trends — Both apps look like simple music players. Under the hood, they are optimized for very different priorities. [1] Spotify style – Cache first, stream second Idea: The client behaves like a smart offline player that happens to stream. The backend is built to support aggressive prefetching. What happens when you hit play - Client requests the track from a CDN or edge node. - Instead of tiny chunks, it downloads a big buffer ahead of the playhead. - In parallel, it starts pre downloading the next few tracks in the queue. - Data is written to an encrypted cache on disk, not just memory. - Playback reads from that local cache, not directly from the network socket. What this means for tunnels and bad networks - When the network drops, the player already has tens of seconds or entire tracks cached. - Because the next one or two songs are already downloaded, you can be offline for a while and never notice. - Cold start cost is a bit higher. First play might take slightly longer, but then everything feels smooth. - It burns more local storage and possibly more data, because not every prefetched song will be listened to fully. In a design answer, you can mention - Local disk cache with eviction policies (LRU per user, per device). - Background prefetch of N upcoming tracks based on queue. - Download manager that adapts how aggressively it prefetches based on network quality and user settings. - CDN tuned for larger object delivery and range requests. - Explicit offline mode that pins playlists into cache. [2] YouTube Music style – Stream first, cache is minimal Idea: Treat audio like video streams. Cost and bandwidth are optimized first. When you hit play - Player requests audio (and sometimes video) via HLS or DASH-style chunks. - Each chunk is only a few seconds long. - Client keeps a small rolling buffer in memory, not a large queue on disk. - Prefetch of future songs is limited, because video tracks are large and expensive to fetch speculatively. What this means for tunnels and bad networks - If your connection dies, the player only has a few seconds of buffered data. - As soon as those chunks are consumed, playback stalls. - Startup can feel snappy and data usage is controlled, especially for casual listeners. - Works very well on stable networks, feels fragile in spotty coverage.

1,219

38 Comments

Why does an E3 level SWE at Meta make only ~190k/year, while an E8 level engineer makes over ~$2M/year, even though both engineers are ICs and spend the same time at work? I have spent the last 5 years at Meta as an IC. I joined with a little over 15 years of experience, and I’ve worked with many solid engineers in this time. Here is how I think about that compensation jump. 1. Same hours, completely different “unit of work” An E3’s unit of work is usually a task or a ticket. An E8’s unit of work is a multi year problem for the company. E3: “Implement this service, fix this bug, write this feature.” E8: “How do we cut infra cost by 20 percent across this product” or “How do we make this platform safe to scale to 10x users.” One person is paid to execute. The other is paid to decide what is worth executing in the first place. 2. Radius of impact E3 usually impacts a file, a service, maybe a small team. E8 shapes whole orgs and product lines. If an E3 ships something great, the impact is great but local. If an E8 ships the right platform, hundreds of engineers become faster and the company saves or earns millions every year. Comp tracks the area of the circle you influence. 3. Risk and downside protection At junior levels, mistakes are usually contained and reversible. At senior staff levels, a bad call can burn tens of millions or damage the brand. E8s are paid for judgment under ambiguity. They decide which bets the company should not make, which migrations can wait, which “shiny idea” is going to kill reliability. You pay more to people whose good judgment protects you from very expensive failures.  4. They scale themselves This can happen in a few ways. 1. Delegation with ownership They define the shape of the problem, then hand large pieces to other senior and mid level engineers while keeping the bar and direction clear. 2. Knowledge that travels They write RFCs, public comments, FAQs, wikis, internal posts. One answer helps hundreds of people who will face the same issue next quarter. 3. Tools over heroics Instead of unblocking people manually all day, they build tools, libraries or guardrails so others can unblock themselves. One well designed tool can save thousands of engineering hours every year. This is what “scaling yourself” actually looks like. The company pays for that multiplier.  5. Ownership of the “uncomfortable problems” Junior engineers usually work inside a well defined box. E8s take ownership between the boxes. They pick up problems that:  Span many teams and no one really “owns”  Require aligning leaders who disagree  Have product, infra, legal and security angles at the same time Most people avoid those because they are messy, political and slow. Very senior ICs lean into them. That is where a lot of value sits. Continued ↓

201

15 Comments

RAG is becoming one of the most critical techniques for improving LLM accuracy — not just a research concept, but a practical system design pattern. It's exciting to see Googlers sharing real-world implementation experience and helping more developers build reliable AI

3

Do we get some a free DGX Spark depending on the number of companies we have worked in our career that showed up today during Jensen Huang's NVIDIA GTC keynote? 🤣

Felt cute, so I added 64-bit support for WASI preview1 in Wasmtime, the de facto standard WebAssembly runtime. https://lnkd.in/gtHxr34X Key points: 1. You can now compile a C program to wasm64-wasip1 using my fork of wasi-sdk (https://lnkd.in/gwqKX8NE) and run the resulting binary. 2. This feature is not standards-compliant so it will not be upstreamed. The WASI team is unlikely to agree to a memory64 preview1 ABI because they are focusing on preview2 and beyond. 3. With that said, this is useful if you want to run non-production workloads (e.g. for research purpose or to understand how memory64 can impact your application performance) High level steps: 1. Download the modified wasi-sdk toolchain (https://lnkd.in/g2YJt4Ys) 2. Compile your program targeting wasm64-wasip1 using the toolchain 3. Clone my fork of Wasmtime and checkout the yagehu/wasip1-memory64 branch 4. Build it with `cargo build --release` 5. Now you can run the wasm64 binary with `wasmtime run --wasm memory64 binary.wasm`

13

This was a good discussion about the Principal Engineer role at Amazon! I often say that the Principal Engineer role as it exists at Amazon simply doesn't make sense at smaller companies. To many, the title just means the next highest level of engineering skills. At Amazon, it requires dealing with high levels of ambiguity, business strategy, and organizational influence. If you're at a company with <50 people, the person with that level of influence should be leading the whole technical organization - it requires a significant amount of scale before it makes sense to have an IC role for it. I also personally appreciated Steve's points on internal level progression - or as Gergeley says, "Why being promoted from Senior to Principal at Amazon is one of the hardest jumps in tech." Steve: "Basically, to get from senior to principal, you have to do like two and a half level jump, from L6 to L7. Technically, it sounds like one level, but at some other companies, this might be like, you know, L8, L9 or L8 and a half. [...] I noticed that some of the best engineers that I'd ever worked with were having such problems getting to principal engineer that they ended up moving to Facebook or to Meta or to all these other places where the progression was sane. Now they're senior staff and, you know, principal and distinguished engineer at other companies. And so because we had high standards, we actually had this brain drain." In addition to the level itself being a big jump the promotion process is notoriously arduous and arbitrary. While I was L6, I had multiple managers over several years acknowledge that I was doing L7 work, and even agree that if I was being hired in from outside I would be hired at L7, and it was the internal promotion process that was the barrier. This was not the *only* reason I left Amazon, but it certainly made it very easy to do. Amazon has lost a lot of skilled engineers this way, but it's a difficult problem to fix, since the incentives are fundamentally badly aligned. If an L6 engineer is already doing the work that's needed, it's very hard for a manager to justify spending the 100+ hours of work it takes to advocate for their promotion. And while there have been attempts to make the process less arduous, no one wants to be seen as "lowering standards." I highly encourage engineers in that position to make it clear they will not be continuing to do work that way indefinitely. And I encourage leaders at other companies to learn from the example! https://lnkd.in/guC35n2B

3

4 Comments