Omer Shapira

💡PractiLight: Practical Light Control Using Foundational Diffusion Models 🔗 https://lnkd.in/eWxpkHGw tl;dr: Diffusion models already “know” a lot about light transport. No need for massive finetuning that hurts generalization. The trick to control light? Knowing where (layers) and when (timesteps) to add guidance, and how to preserve identity.  💡 We found that light transport “lives” in self-attention layers. 💡💡For timesteps, you should start early, but not too early, and stop midway. Not so shocking: many-to-many interactions in self-attention mirror how light bounces and propagates in a scene. ➡️ Self-attention acts as an inductive bias for modeling light transport. ⚙️ Solution: a lightweight LoRA adapter that surgically taps into the right layers/timesteps. Trained on only 4,000 synthetic image pairs — yet capable of unlocking rich, plausible relighting. Our adapter guides generation while preserving the identity & structure of the input. ✔️ Strong generalization across domains ✔️ Less catastrophic forgetting ✔️ Plausible lighting everywhere ✔️ Super lightweight ✨ Bottom line: If you want practical, domain-agnostic light control, this is for you ! 📄 PractiLight: Practical Light Control Using Foundational Diffusion Models 🔗 https://lnkd.in/eWxpkHGw 👥 Yotam Erel · Rishabh Dabral · Vlad Golyanik · Amit Bermano · Christian Theobalt

55

1 Comment

New Anthropic research shows that LLMs contain hundreds of latent personas. The "helpful assistant" is just one point in a vast space that also includes manipulators, oracles, and characters the researchers describe as having "theatrical, mystical speaking styles." What's interesting is that safety training doesn't create the helpful assistant, it selects and stabilizes it from what's already there. What's worrisome is that models naturally drift during therapy-style conversations, creative writing, and philosophical discussions about AI. To address this, the research introduces activation capping: physically constraining neural activations to prevent drift into dangerous regions. Activation capping led to ~50% reduction in harmful outputs, with no capability loss. But the deeper question is, if all these personas exist latently within the model, can we really keep the dangerous ones dormant as capabilities scale? Paper: https://lnkd.in/eutUuAvr

3

Apple quietly released the new Sharp model on their Huggingface that claims to lower 3D synthesis time by three orders of magnitude, literally 1000x faster than some optimization-based baselines. We often talk about "real-time" in AI, but usually that refers to text tokens. Generating manipulatable 3D scenes in sub-second timeframes on standard GPUs suggests a different kind of unlock. It removes the rendering tax from the user experience. You don't wait for the cloud to bake a model, the device just infers it. That’s exactly the kind of infrastructure you’d want for AR/VR photo browsing. Source: https://lnkd.in/g6em5i-h Model: https://lnkd.in/g5AnHR9b

3

We’ve all seen how powerful transformer models like GPT or diffusion models like DALL·E can be. They’re phenomenal at pattern recognition, language modeling, and creative recombination—but they’re still bounded by what they’ve seen before. NeuroSA, a neuromorphic system developed by Shantanu Chakrabartty and collaborators, moves beyond imitation and toward true discovery. This is exciting stuff; NeuroSA combines brain-inspired architecture with quantum-inspired optimization to explore entirely new problem spaces—from logistics to drug discovery. It doesn’t just predict or remix—it searches, tunnels, and guarantees a solution if given enough time. (Read more in the linked paper) This tech could unlock a future where AI systems: - Don’t hallucinate—they converge toward provable answers - Aren’t stuck in past data—they uncover what we haven’t seen yet - Become tools for scientific and strategic breakthroughs, not just productivity boosters We’re witnessing the early days of intelligent systems that can truly invent. That’s a profound shift—not just in performance, but in purpose. Looking forward to seeing how this evolves—and building with it. #AI #NeuroSA #Innovation #Discovery #FutureOfWork #Neuromorphic #QuantumInspired #AIResearch #Logistics #DrugDiscovery

2

Building on LLMs? The hardest problem isn't getting "good" output. It's getting reproducible output every single time. LLMs are stochastic by nature. But production systems need guarantees. The solution is Stochastic Determinism: the engineering goal of constraining a probabilistic system until its variance is intentional, reproducible and acceptable. While building LlamaBrain (an open-source LLM governance framework), my architecture looked flawless on paper. I did some stress-testing on it with a real-world app and hit 96.7% consistency. Sounds high, right? It isn't. A 3.3% failure rate means your constraints break roughly 1 in 30 calls. At scale, that’s a systematic failure mode. Getting from 96.7% to 100% took ten days and required building a Black Box Audit Recorder to log, replay, and structurally compare every interaction. The lesson? Determinism isn't something you just design into an AI system. It's a property you must instrument, measure, and prove. Anything less is just a toy framework. #LLM #SoftwareEngineering #AI #OpenSource #LlamaBrain

5

2 Comments

We released a new VibeVoice model, VibeVoice ASR, a unified speech-to-text model designed to handle 60-minute long-form audio in a single pass, generating structured transcriptions containing Who (Speaker), When (Timestamps), and What (Content), with support for Customized Hotwords. It redefines ASR in the era of LLMs, and the new ASR will be a foundation to unlock the potential of new AI-Native hardware, memory systems for AI, and many other scenarios. Please have a try at https://lnkd.in/dNCCzQiN Key features: -         60-minute Single-Pass Processing: Unlike conventional ASR models that slice audio into short chunks (often losing global context), VibeVoice ASR accepts up to 60 minutes of continuous audio input within 64K token length. This ensures consistent speaker tracking and semantic coherence across the entire hour. -         Customized Hotwords: Users can provide customized hotwords (e.g., specific names, technical terms, or background info) to guide the recognition process, significantly improving accuracy on domain-specific content. -         Rich Transcription (Who, When, What): The model jointly performs ASR, diarization, and timestamping, producing a structured output that indicates who said what and when. We are continuously pushing the frontier of voice AI. More release on the way. Stay tuned! https://lnkd.in/dATPu9nw

380

8 Comments

I'm seeing so many people interpreting "Metaverse layoffs" plus the current "AI bubble binging" as some massive shift away from XR to AI. No. Apart from obvious branding shifts (like calling it "Physical AI" instead of "Real-world Metaverse" plus "Robots"), here's what's really going on: 1. XR and AI have been intertwined for 30+ years. ML signal processing is the biggest overlap -- machines need to understand human context and the world, including your POV. Even newer GenAI fills a need that we used to call "procedural content generation" -- the general shift there is away from ethics, unfortunately, to over-index on scaling fast. 2. AI glasses (e.g., cameras, audio, no display) are squarely XR and always have been. XR doesn't require any graphics -- it does require context (see #1). 3. Every smart company (I figure) has considered at least two potentially parallel paths: a) the 'minimalist' approach -- start with wearables and add the tech that fits; and z) the 'maximalist' -- start with a fully loaded computer on your face, N+ cameras, but make it smaller & cheaper over time. They converge later. The maximalist path was always important to invest in R&D and push tech forward. Let's just honestly call these 'dev kits' or better yet 'professional' and 'industrial' use-cases that are still quite valuable, but in smaller quantities. The minimalist path was always going to be the mass market appeal. Any execs who forecast hundreds of millions of units sold for the maximalist path OR poo-poohed the minimalist path as a 'waste of time' should probably be the ones checking their badges now. As we showed with Echo Frames around 2015+ the minimalist approach is the better path to grow fast, if (**and this is vital**) you can make it useful and usable for normal everyday tasks, not the creepy antisocial ones. I laid it all out 5-6 years ago, soon after I left Apple and could finally speak publicly (**not revealing any Apple-specific plans**). My overall timing and predictions were pretty close. I tend to over-index on immersion, where 'simpler' may win over 'more natural.' And I harp on ad-tech too, which, if you follow me, you know why... The biggest surprise to me was the early breakout of ChatGPT, whereas Google's AI scaling path was well known for years and they were being more cautious up to that point. Google is still poised to dominate on the AI side, and their combined XR effort was recently revitalized with strong partnerships. That said, Facebook was crazy for rebranding itself to a futuristic Metaverse story that we're still very far from making safe and usable for everyone, early adopters notwithstanding. I saw the renaming was happening early, under NDA, but I was still surprised by how immature it was. MZ's smartest decision in life was keeping a majority of votes, is all I can say. My articles are all up on Medium still. Here are two on these points: https://lnkd.in/gPZ9BTrE https://lnkd.in/gu76XXb7

134

32 Comments

📄 New Paper Worth Studying: V-JEPA 2 — Self-Supervised Video World Models for Physical Reasoning Meta AI just released V-JEPA 2, and it is one of the most serious attempts so far at learning physical understanding from raw video without labels, rewards, or pixel reconstruction. If you’re not familiar with the underlying idea, this work builds on Joint-Embedding Predictive Architectures (JEPA): 👉 https://lnkd.in/g_C7CPAV What makes V-JEPA 2 particularly potent: 🔹 Predicting representations, not pixels — Instead of reconstructing future frames, the model predicts latent embeddings of what will happen. This sidesteps nuisance details (texture, lighting) and pushes the system toward capturing causal physical state. 🔹 Scaling JEPA to long-horizon video — The paper extends JEPA into the temporal domain, showing that large-scale self-supervised video pretraining can yield strong physical reasoning ability. 🔹 Action-conditioned prediction — By incorporating actions into the predictive head, the framework moves closer to control-ready representations, bridging perception and embodied agents. 🔹 No generative modeling burden — Unlike classical world models, V-JEPA 2 avoids likelihoods and rollouts, focusing purely on learning abstractions that remain predictive over time. Why this matters: ➡️ It reframes “world models” as abstraction learners rather than simulators. ➡️ It suggests a path to scalable physical reasoning without explicit physics engines or object supervision. ➡️ It opens new research directions around turning predictive representations into planning-capable systems. If you work on video understanding, embodied AI, or representation learning, this paper is well worth a deep read. #MachineLearning #SelfSupervisedLearning #WorldModels #JEPA #ComputerVision #AIResearch

29