Jiwen You

Most existing #lean4 datasets contain only correct proofs. Models learn error correction with RL, that's expensive. With UW Math AI lab we release a dataset of 260k erroneous Lean proofs with - compiler feedback - reasoning trace - corrected proof Improvements in Error Correction: - Goedel 8B: 2x - Kimina 8B: 3x Paper: https://lnkd.in/gaybt4bd

117

2 Comments

The 2025 State of AI (through Karpathy’s lens) - it’s not “the year of agents.” It’s the decade. Recently, Andrej Karpathy was interviewed on the Dwarkesh podcast (I'll put a link in the comments), and this is what he said on the current state of AI agents: 1) Creating AI agents that work reliably is a project that will take up to 2035 to complete. We don’t yet have the scaffolding for coworker-level agents: durable memory, reliable multimodal perception, continual learning, robust tools/OS control. This is a marathon of engineering, not a quarter’s roadmap. 2) Software progression from 1.0 to 2.0 to 3.0 1.0: humans write code 2.0: data writes weights 3.0: we program via language + LLMs Don’t skip the representation era. Strong “cognitive core” comes first, agents second. 3) We’re building “ghosts,” not animals. Models simulate human behavior from data. No instincts. No body. Expect brilliance in patterning, less so in grounded, embodied common sense (at least for now). 4) Reasoning lives in the context window. Weights are kind of like long-term memory. Context is kind of like working memory where thinking happens. Feed richer context, get sharper reasoning. Retrieval is way better than wishful prompting. 5) Only part of the ‘brain’ exists We have Cortex-like patterning. We don't have Hippocampus-like memory consolidation, emotion/instinct, coordination. That’s why current systems “think,” but don’t truly embed. 6) Building beats hand-wavy prompting Use AI to accelerate, not abdicate. Autocomplete helps but novel system design still needs humans who understand what they’re building. 7) RL is inefficient, process beats outcome-only rewards. Next step: process-based supervision and reflection loops. Teach models to review, self-correct, and generate useful training data. 8) Forgetting can improve generalization. Humans generalize because we forget. Models that memorize everything risk rigidity. Curate, prune, and reinforce abstraction, not trivia. 9) Bigger isn’t always better. Clean, high-quality data + smart architecture can outperform “just scale it.” A lean, well-trained cognitive core can feel startlingly capable. So, to summarize, here are a few practical takeaways: * Design for context: Retrieval + tool use + structured memory. * Instrument the process: Critique, reflect, and fine-tune steps, not just outputs. * Curate ruthlessly: Data hygiene is a moat. * Ship scaffolding: Memory, tool orchestration, reliability. Think in decades, execute weekly, and compound small real improvements. Let's not forget that we’re mid-build, not at the finish line. The transformer will likely be with us a while, just refined. The winners will be the ones who build patiently, instrument relentlessly, and curate obsessively. Let me know your thoughts in the comment section

1

1 Comment

🚀 Excited to share two of our recent papers on E-commerce Search & Retrieval: 📄 Beyond Text: Aligning Vision and Language for Multimodal E-Commerce Retrieval arXiv: https://lnkd.in/gUWgEGRx 📄 Unified Learning-to-Rank for Multi-Channel Retrieval in Large-Scale E-Commerce Search arXiv: https://lnkd.in/gtP-kM5T Both works explore multimodal retrieval and unified ranking frameworks for improving large-scale product search.

45

1 Comment

Bias in Causal Inference | Framing the problem When you can’t run an A/B test and rely on observational data to understand the impact of initiatives (e.g., promotions or new feature introductions), careful problem framing and awareness of bias are essential. Two common biases in causal inference are selection bias and confounding bias. Here’s a summary on the former, selection bias: Selection Bias | Conditioning on an Effect (Collider Bias) Selection bias from conditioning on an effect arises when we restrict the analysis to a variable influenced by treatment. For example, measuring a promotion’s impact on cancellations only among purchasers introduces bias. The promotion increases purchases, and purchase intent affects both purchases and cancellations, leading to misstated impact estimates. Problem framing depends on the outcome to influence. For example, to reduce overall churn, measure the total effect across all exposed users, including non-purchasers, capturing funnel-level influence. For post-purchase interventions, measure the conditional effect among purchasers. MVP causal representation: Promotion → Purchase ← Purchase Intent (unobserved) Purchase → Cancellation TLDR: Measure total cancellation impact for broad churn prevention strategies OR conditional impact among purchasers for tactical post-purchase interventions. Selection Bias | Conditioning on a Mediator Selection bias from conditioning on a mediator occurs when we control for a variable on the causal path from treatment to outcome. For example, notifications influence engagement, which impacts retention. Controlling for engagement removes part of the effect, giving an incomplete estimate. For attributing retention impact from notifications, focus on the total effect, capturing both direct impact and that flowing through engagement. To understand how much retention is driven by engagement, measure the mediated effect, which helps guide targeted engagement-driving initiatives without conflating total impact. MVP causal representation: Notification → Engagement (Mediator) → Retention Notification → Retention (direct) TLDR: Measure total impact on retention for measuring the success of feature introduction (e.g., notifications) OR the mediated impact from engagement to retention for targeted engagement strategies. Note: All opinions are my own and shared for data science community discussion.

99

Inference at scale is evolving. The future is composable systems where offline batch, online batch (near-realtime), and realtime traffic can all be served from a single, unified inference endpoint for most use-cases. This simplifies your architecture, reduces operational overhead, and increases throughput. Our new tutorial series will show you how to build it step by sep. In the first installment, Erik Saarenvirta walks you through creating a scalable image classification system on GKE that can be adapted to a variety of use-cases. Leave us feedback in comments or ask questions we are happy to answer. Kent Hua Ishmeet Mehta Erik Saarenvirta https://lnkd.in/dVNiUhwE

16