Umnouy (Tee) Ponsukcharoen

My AAAI 2026 paper got accepted. Barely squeaked by with a begrudging "marginal pass"; along with expert reviewer confusion at basics (which I won't quote here). "fresh perspective on neural network interpretability"  "highly innovative" "experimental setup is rich" "concrete approach to deal with the problem by introducing specific architectures" "Solid behavioral validation" "Novelty: High" "Technical Quality: Good" "Clarity: Generally clear" "Significance: Potentially high" "novel and thought-provoking hypothesis with solid initial evidence" "conceptual and methodological contributions are significant enough to warrant presentation and discussion at AAAI." "paper is well written" "definitely interesting and important for the community" "reasoning is compelling and well-supported" etc. 5 reviewers, no review above a 6 out of 10 🏆 Basic gist of the paper is that: *LLMs appear to encode human language by also storing the relationships _between_ its decomposed features (concepts, words, patterns, etc); rather than encoding only just the independent decomposed features, as currently assumed.* Current SAE methodology presumes independence and linearity ('superposition") between features. After reading Anthropics blog, I spent a few weekends learning, coding, demonstrating (& then writing up) that you can use a nonlinear method to capture complex relationships between features, and read them in an interpretable way. This greatly improves decoding (better reconstruction accuracy ~20-40+%, reduced KL divergence ~+50%). Roughly, it explains why 'surprise' + 'diagnosis' != 'surprise' + 'party'. Demonstrated computationally, statistically, and behaviorally, in LLMs. Some critiques were have you demonstrated this on very large industry scale models? Or, how is this paper different from this other paper mentioning non-linearity might exist in LLM code (which had no characterization or modelling of said possible phenomena)? Can you relate this to concept XYZ? (literally in the 1st or 2nd paragraph of discussion section) 👍 🙏 Thank you AAAI for the anonymous submission process -- probably no way it would have been accepted as a non-field + unaffiliated + single-author, otherwise. And for their positive comments. It was an interesting project -- mechanistic interpretability has a lot of parallels to cog neuro and, in the near future, will probably import more value from cognition, ML, and statistics. 🙏 https://lnkd.in/dxCViN7J

15

1 Comment

Thrilled to share that I recently traveled to Suzhou to present our latest research at EMNLP 2025! Our work dives into the challenge of keeping LLMs updated in a fast-moving world: "Temporal Alignment of Time-Sensitive Facts with Activation Engineering." By investigating how we can steer model internal states, we’ve found promising ways to ensure LLMs stay chronologically accurate without the need for intensive retraining. Attending the sessions and connecting with the global NLP community in Suzhou was an incredible experience. Feeling deeply grateful to my co-authors and mentors who helped bring this paper to life. Check out the paper here: https://lnkd.in/ga_MVf-x

61

5 Comments

What happens if you scale Whisper to billions of parameters? Our #ICML2025 paper develops scaling laws for ASR/ST models, training models with up to 18B params and 360K hours of data, and 100+ languages. Our flagship model with 18B params and 360K hours of training is competitive with/out-performs Whisper, Seamless, and other state-of-the-art models. We show insights on never before seen emergent abilities in large ASR models, and open-source all models on HuggingFace: https://lnkd.in/gkNHwWGm Joint work between Carnegie Mellon University and NVIDIA, with Shinji Watanabe, Huck Yang, Brian Yan, and Jinchuan Tian. https://lnkd.in/gqAxYt3i

291

4 Comments

Some exciting results from our collaborators in the Lidar group / Quantum Elements: a demonstration of a quantum algorithm running on NISQ hardware with exponentially faster scaling than the best classical algorithm!

23

4 Comments

I'm pleased to share a major update from my summer internship! As a Machine Learning Researcher at the Qatar Computing Research Institute (QCRI), I had the opportunity to explore how we can evaluate and improve large language models in reasoning about people across cultures, languages, time periods, professions, and modalities. This work led to the development of ADAM (A Diverse Archive of Mankind), a framework for assessing and enhancing the biographical reasoning abilities of multimodal large language models (MLLMs). ADAM introduces three key components: AdamDB: A multilingual, multimodal dataset of over 4 million individuals spanning geography, time, and profession. AdamBench: A cognitively structured evaluation suite based on Bloom’s taxonomy, covering six levels of reasoning in both English and native languages. AdamRAG: A retrieval-augmented generation system tailored to biographical queries, designed to reduce hallucination and improve factual precision, especially for lesser-known individuals. In our experiments, we found that ADAMRAG significantly boosts biographical reasoning, with open-source models gaining 30–40 points and closed models exceeding 95% accuracy. Yet, popularity bias persists, with all models favoring well-known individuals. Higher-order reasoning remains challenging, and vision-based retrieval offers only modest gains compared to text-based retrieval. The full paper is now available here! Go check it out!: https://lnkd.in/gVJQATdS Grateful to the team at QCRI and my collaborators for such a fruitful summer. This experience strengthened my focus on designing ML systems that combine scalability with meaningful impact, by integrating multilingual knowledge, supporting fair and reliable reasoning, and enabling evaluation at global scale.

22

2 Comments

I’ve been following recent work coming out of the physics-based RL community, and Zen Luo’s dissertation defense on "Learning Universal Humanoid Control" is a great deep dive into how we move past "one-off" robotic skills toward a more general purpose motor intelligence. He frames the challenge of General Robot Intelligence along three axes: • Task Diversity: Moving from single-gait or single-game controllers to a broad library of human activities. • Complexity: Integrating high-DOF systems with vision and tactile feedback. • Realism: Ensuring the resulting motion isn't by just functional, but grounded in human-like physics. The core of the work involves scaling up motion tracking to build a foundational control prior. Instead of training a robot from scratch for every new environment, you leverage massive human motion datasets into a latent space (essentially giving the humanoid motor instincts). When you use this prior as a starting point, the sample efficiency for learning complex manipulation or locomotion tasks improves significantly. It’s a smart look at how scaling laws can be used to augment how we approach high-dimensional humanoid control. Check it out! Zhengyi (Zen) Luo Link to the defense: https://lnkd.in/gCB4RBeG #Robotics #Humanoids #ReinforcementLearning #ControlTheory

16

Just a thinking-out-loud rant from me on Dr Andrew Ng's letter about "How to Become a Multilingual Coder" enabled by LLMs: [Link: https://lnkd.in/gqBgubVB] --------- It's understandable to be concerned about the structural impact of LLMs on the job market. Many jobs will ultimately be phased out, but new job opportunities will arise as well. I believe the path forward is for individuals and society to proactively understand and utilize this breakthrough not to become passive consumers of AI, but to become more capable and intelligent collaborators with it. *The true opportunity lies in leveraging these powerful tools to amplify our capabilities and intelligence, not to foster complacency.* As highlighted in the article, LLMs can serve as incredible resources for learning new coding languages and adopting sophisticated code structures. However, this potential is unlocked *only* when we commit to critically reviewing the output, understanding the underlying patterns, and consciously integrating these best practices into our own workflow. This active process of learning from, evaluating, and synthesizing AI-generated content is crucial. It ensures we sharpen, rather than blunt, our critical thinking skills. We've navigated similar technological shifts before; the widespread adoption of search engines, for instance, ultimately expanded our access to information and problem-solving abilities. I am inclined to believe that by approaching AI as a powerful co-pilot and personalized tutor, we can continue to build upon our intellectual foundations! #DataScience #AI #MachineLearning #LLMs #Coding #ProfessionalDevelopment #FutureOfWork #LifelongLearning

24

I am very excited to share a new preprint from our group, led by Jingyi Chen and Xinyuan Zhang, titled "Unsupervised Learning for Solving the Elementary Shortest Path Problem (ESPP)." Since joining Rice one year ago, we have been working on developing learning-to-optimize techniques for transportation problems. This particular research addresses a fundamental issue that I have contemplated for some time, particularly when Shuocheng was working on his PhD dissertation. Unlike solving the traveling salesman problem (that needs to visit every node exactly once), learning to solve ESPP is very tricky: the length of ESPP can range from visiting two nodes to visiting every node, which is highly dependent on the underlying graph structure and the edge weight realization. We develop a method that can tackle this problem reasonably well, and here's a summary of our major contributions: - Modeled the ESPP problem via the probabilistic method, avoiding the need to generate an exponential number of subtour elimination constraints. - Developed an unsupervised learning pipeline to train the model without generating labels, and even better, embedding the well-known Bellman-Ford algorithm into the loss function. - Outstanding performances that (1) exceed best-known heuristics by a huge margin, and (2) demonstrate superior generalization capabilities on unseen graph structures and edge-weight distributions, attributable to the incorporation of algorithmic inductive biases. - The even better part: performance guaranteen. If you keep sampling, you will end up with the exact solution, with high probability. We appreciate the support from the Department of Energy for this work. Link to our preprint: https://lnkd.in/eXM2stjC

94

1 Comment

Structuring an Interpreter Expression Evaluator My current hobby project is writing a language interpreter and I got one working nicely. I'm writing an interpreter which employs a kind of compilation first step, which parses the program into evaluation trees. I decided to use a concept where each element of each expression (+,-,*,/ etc) is a "Function" but the interesting part is that the arguments of each function are all "Functions". In this way, the tree nature of an expression is built in during the "compilation" pass. Then, in runtime, I call the "root function", whose arguments are other "functions", whose arguments are...and so on. This forms an evaluation tree like a linked list built in. Now, the leaves (ends) of this tree are variables or constants, and their "functions" are "retrievers" which return the values. Then, the nested calls return upwards and the final value is returned by the root function call. I was inspired by the early BASIC language variants which actually had some compilation steps. I am exploring the concept of compiling into a structure (like AST), but then running along the AST as an interpreter...so that one program can run on multiple platforms that have the interpreter. Simple ideas but I wanted to share the concept of arguments as function pointers and how this builds an AST without an external storage for it (each function maintains its own argument "function" pointers initially set by the compiler.

24

7 Comments