Jin Rou (JR) New

We developed an open-source AI tool to design concrete mixes that are stronger, more sustainable, and ready to build with faster—speeding up construction while reducing environmental impact. https://lnkd.in/gPCk8tCM But the impact of this AI tool is not just hypothetical! Amrize used Meta’s AI-based technologies to design a new low-carbon mix, and successfully deployed it in an at-scale slab-on-grade application at Meta's new data center in Rosemont, MN. Compared to the legacy mix, this new AI-designed mix is: 🦁 Stronger ⏱ Faster 🍃 Lower carbon ⏱ ️The ideal set time All this was achieved without needing any new materials, nor special equipment. Best of all, the AI is open-sourced. https://lnkd.in/g2KA7KZW This work was featured in a Meta engineering blog article published today! https://lnkd.in/gBU9HY8H

98

9 Comments

Inference at scale is evolving. The future is composable systems where offline batch, online batch (near-realtime), and realtime traffic can all be served from a single, unified inference endpoint for most use-cases. This simplifies your architecture, reduces operational overhead, and increases throughput. Our new tutorial series will show you how to build it step by sep. In the first installment, Erik Saarenvirta walks you through creating a scalable image classification system on GKE that can be adapted to a variety of use-cases. Leave us feedback in comments or ask questions we are happy to answer. Kent Hua Ishmeet Mehta Erik Saarenvirta https://lnkd.in/dVNiUhwE

16

Really appreciate this book - "Designing Machine Learning Systems" by Chip Huyen that talks about the iterative development lifecycle for AI/ML products. My key learnings are: 1. Transitioning from a prototype to a production-grade Machine Learning system requires a fundamental shift in mindset: we need stop treating the ML model as the entire solution and start viewing data, model, architecture and operation working as one holistic system. 2. True expertise in AI/ML Product Development lies in navigating the critical trade-offs between research objectives—like chasing state-of-the-art accuracy on static datasets—and production realities, where latency, throughput, and reliability often take precedence. 3. The real challenge begins after deployment, where we must rigorously monitor for data distribution shifts and train-serving skew, understanding that a model’s performance will inevitably degrade as real-world data diverges from training environments. 4. To build scalable and maintainable systems, we need to move beyond ad-hoc scripts and invest in robust infrastructure for data engineering, feature management, and continual learning, ensuring our systems adapt as dynamic business requirements evolve.

19

Another good one from Karpathy’s an 1-hour LLM primer (I missed to watch earlier) covers: • How LLMs are trained (and why the weights matter) • Fine-tuning into helpful assistants • Tool use & multimodality in action • Scaling laws driving progress • Security risks: jailbreaks, prompt injection, data poisoning https://lnkd.in/ev38636z

4

History is repeating itself. 10 years ago the recsys industry was migrating from linear model recsys to sparse neural network recsys. Back in 2016, many people in the field were skeptical that deep learning would deliver positive ROI for recsys, the fear being rooted in the perceived higher training and inference cost. I see people repeat the same fear, uncertainty, and doubt about this year's migration from sparse neural network recsys to LLM-native recsys. Those who do not lean into the LLM-recsys migration this year will fall behind.

78

1 Comment

If you’re an AI engineer or an AI leader looking to level up your skills in LangChain and LangGraph, this one is for you. I’m teaming up with Arvind Narayanamurthy for a beginner–intermediate, 1.5 hour hands-on workshop where you’ll learn how to move from simple prompts to real AI applications and agentic workflows. Why this matters - Most engineers know how to prompt. Few know how to orchestrate. - LangChain and LangGraph are becoming the backbone of production AI systems. - If you want to build or lead AI engineering teams, you need these mental models. What you’ll learn - Foundations of LLM orchestration and where LangChain fits - Essential building blocks: chat models, prompting patterns, structured outputs, LCEL - Tool calling with MCP adapters - RAG basics end to end - LangGraph workflows and agentic architectures - Two live labs: a micro-app and a simple agent - Quick dive into LangSmith for debugging The workshop is recorded, and everyone gets the replay. If you want practical, real-world skills to build modern AI systems, this is the one to sign up for. Grab your seats: https://lnkd.in/eW2WZdcf

328

36 Comments

Back in 2023, Kira Wang and I worked on using LLMs as a customer simulator. The idea was simple. Every year, Amazon Japan and our selling partners run countless promotions. The most canonical way in a vacuum to measure incrementality of these promotions are randomized experiments. However, in reality, there are always some other promotions running, always a slightly different variation you would like to test, and always the operational limitations to run an experiment. As it turns out, if you give them sufficient contexts (feature and outcome distribution, few-shot examples), right set of features, and even some finetuning on your seemingly unrelated historical data from previous promotions, LLMs can reasonably perform a simulation close to what predictive models can achieve. In our research, we benchmarked on a predictive task and a recommendation task based on randomized experiment data to show this. The work itself is quite dated (first draft in 2023) and limited to our internal, small datasets so I'm very curious how people can improve upon this approach both in terms of performance and scalability. Workshop: https://lnkd.in/gw2BprtD Paper: https://lnkd.in/gWEpwvGY Repo: https://lnkd.in/gZQqshVZ

50