Dependent Type Theory: Why Lean Can't Lie

One of the most impressive technical books I've come across recently. "My Adventures with Large Language Models" by Prathamesh Saraf builds five LLM architectures from scratch in PyTorch — from Transformers and GPT-2 all the way to Llama 3 and DeepSeek's Multi-Head Latent Attention and Mixture-of-Experts. What stands out: every chapter has runnable end-to-end code. This isn't theory. You build it, you run it, you understand it. If you work with LLMs and want to understand what's actually happening inside the models, this is the resource. The code is fully open source. Book (free sample available): https://lnkd.in/eA5QXkZU Code: https://lnkd.in/eQE46B8d

Full technical detail is on this blog page: https://lnkd.in/gcsdT72U but below is a summary I built a chess-playing AI from scratch — in a programming language I also built from scratch. No PyTorch. No NumPy. No Stockfish. Zero external dependencies. The language is CJC-Lang (Computational Jacobian Core), a numerical programming language I wrote in Rust — 21 crates, ~96K lines of code. It's designed for ML and scientific computing where reproducibility matters: the same input always produces the same output, enforced through careful floating-point handling, deterministic data structures, and explicit randomness control. The chess demo is ~1,715 lines of pure CJC-Lang. It includes: → A reinforcement learning agent (A2C + GAE) with 45,873 trainable parameters → A complete chess engine — castling, en passant, promotion, repetition detection, all of it → An Adam optimizer, self-play training loop, and evaluation suite with Elo ratings The language provides the building blocks: built-in tensors, an automatic differentiation engine for computing gradients, and native performance kernels that I added as bottlenecks emerged during development. That process was the most interesting part. Profiling showed 84% of training time in one function — so I wrote a native kernel (7.7× faster). Then the backward pass dominated instead, so I redesigned the autodiff engine: arena allocation, dead node elimination, fused layers. Total result: training went from 82 to 30 seconds per episode. The agent learns modestly — Elo +13.9 after 120 episodes. It won't beat Stockfish. But that's not the point. The point is that a language I built from the ground up can express, train, and evaluate a real ML agent — complete game rules, gradient computation, optimization, evaluation — with nothing borrowed. Every tensor op, every gradient, every optimizer step — code I own.

Distributed Machine Learning using deap #machinelearning #datascience #distributedmachinelearning #deap DEAP is a novel evolutionary computation framework for rapid prototyping and testing of ideas. It seeks to make algorithms explicit and data structures transparent. It works in perfect harmony with parallelisation mechanism such as multiprocessing and SCOOP. The following documentation presents the key concepts and many features to build your own evolutions. If you are used to any other evolutionary algorithm framework, you’ll notice we do things differently with DEAP. Instead of limiting you with predefined types, we provide ways of creating the appropriate ones. Instead of providing closed initializers, we enable you to customize them as you wish. Instead of suggesting unfit operators, we explicitly ask you to choose them wisely. Instead of implementing many sealed algorithms, we allow you to write the ones that fit all your needs. This tutorial will present a quick overview of what DEAP is all about along with what every DEAP program is made of. https://lnkd.in/gPCwmiEQ

I've have begun to read a really impressive (but technical) book I've come across in a while — "My Adventures with Large Language Models" by Prathamesh Saraf. While I'm not super technical, this book guided me to a deeper level than I thought I could reach — and there was a real primal joy in that. If you follow Karpathy and Andrew Ng’s lessons on starting from the basics of LLMs, then this is a great complement! Meant to help ML engineers and developers, he builds five LLM architectures from scratch in PyTorch. Starts with Transformers and GPT-2, then goes all the way to Llama 3 and DeepSeek's Multi-Head Latent Attention and Mixture-of-Experts.Every chapter comes with runnable end-to-end code. Hands on. You build the code and run the code. If you work with LLMs and want to understand what's actually happening inside the models, this is the resource. And the code is all out in the open. Book (free sample available): https://lnkd.in/gWvNQn-T  Code: https://lnkd.in/gPa6-zdK

Every programming language ever built answers the same question: What should the computer compute? Assembly. C. Java. Python. Rust. Different syntax. Different abstractions. Different performance profiles. Same foundational assumption — computation is the manipulation of state. None of them can answer the question that the next decade of software demands: What caused this output — and what will this action cause? This is not a missing feature. It is a missing paradigm. The history of software is a history of paradigm shifts: → Assembly → C: abstracted the machine → C → Objects: abstracted state and behaviour → Objects → Functional: abstracted effects and correctness → Functional → Probabilistic: abstracted uncertainty → Probabilistic → Causal: abstracts the generative structure of reality itself Each shift did not replace what came before. Each shift made the previously unsolvable — tractable. Causal programming languages are that shift. A language where: → do(X := value) is a native operator — intervention, not just assignment → Counterfactual reasoning is a language block — not a mathematical afterthought → The compiler rejects causally invalid code — correlation ≠ causation becomes a compile error → Every value carries its causal provenance — explainability is in the program, not bolted on afterwards → Causal authority governs what agents can cause — at the language level, not the policy level The AI safety crisis is a causal crisis. The explainability crisis is a causal crisis. The agentic AI governance crisis is a causal crisis. Every one of them is unsolvable in a causally blind language. The theoretical foundations exist — Pearl's causal calculus is complete. The demand is urgent and growing. The engineering effort is the only gap. The first causal programming language will do for AI reliability what Rust did for memory safety — eliminate an entire class of failures at compile time. That is not a distant frontier. That is the next language you will write production code in. #CausalAI #ProgrammingLanguages #FutureOfSoftware #CausalProgramming #SoftwareDevelopment #AIGovernance #AISafety #VectorPeak #CausalIntelligence #AgenticAI #SoftwareEngineering #CausalStack #Explainability #AIReliability #NextParadigm #TechInnovation #CausalComputing #FutureOfAI #DeveloperCommunity #AIInfrastructure

I didn't know until recently that all the top coding agents edit code differently. Claude Code finds and replaces exact chunks of text, Codex CLI generates diffs, Cursor rewrites the entire file. We tested 7 approaches across 4 models on a suite including some very large files and complex edits. The best approach isn't currently used by any major tool. It's called AST edit. Instead of making the model reproduce chunks of old code to show where the edit goes, you just let it say “replace this function” or “add a method to this class” and a parser figures out the rest. Three out of four models score 100% with AST edit. Nothing else comes close except rewriting the entire file, which takes 18x more tokens and 12x more time. The other formats all break in the same way: the model has to copy text perfectly from a file it saw once, and on big files it just can't. This work was done for Geometric. We use evolutionary algorithms to optimise CUDA kernels in complicated DSLs. AST edit means models can edit kernels without having to copy hundreds of lines of unfamiliar syntax. Full results and code here: https://lnkd.in/gNUgMgqR

Yep. Best way to refactor is by using AST transforms that are deterministic. eslint fixers and jscodeshift. Roslyn analyzers. etc. I added a nice diff tool to my toolbox earlier this year, difft (Difftastic) uses semantic diff (via treesitter I believe). Much nicer having that as well on the cli with side by side diffs too

It's this kind of post that leaves me stunned that: a) AI coding tools don't make more use of LSP servers - CC supports it, but I've never seen it seriously reach for it rather than grep. b) LSP servers haven't started making edits much safer/more targeted. Y'all need a LISP. (h/t Peter (PEZ) Strömberg for the link)

I miss learning for the sake of learning and going into rabbit holes. Reading research papers has always been aspirational for me but hard to be consistent with. What's different now is that the barrier to actually grokking hard concepts is so much lower. You can take a dense 1960s systems paper, work through it with an LLM, and come out the other side genuinely understanding it. You can customize the learning based on your skills, your experience, your knowledge. Starting a series on Turing Award winners and other CS papers I find interesting. Most of it will be AI generated (Skills FTW): the code, the write-ups, the structure. But every entry has one section written by me. Writing is how I think. It's what validates my understanding. First one is Alan Jay Perlis, 1966. He was the first to say humans shouldn't have to translate things into how computers understand them: write a simple equation, the compiler does the rest. He also got the best CS minds in the world to agree on a common way to think about languages. That's why Python, JavaScript, and Go all have similar concepts: it's ALGOL 60. And because all those languages ended up so similar, LLMs can understand them really well today. That's why we can program in English in this new world. Github in comments.

Me: That solution is so human, lol 😂 Claude: RIGHT?! 😂 That's literally the secret nobody tells you when you start coding — Programming is just human thinking, written formally. Like every algorithm ever made was just some person going: "How would I actually solve this with my own hands?" "Ok now let me write that down in a way a computer understands" The whole reason CS fundamentals feel hard at first is because we think code is this alien mathematical thing — but it's really just: Removing duplicates → "skip the ones that look the same" BFS level order → "finish this row before starting the next" Caching → "keep the stuff you use often nearby" Completely human intuitions dressed up in syntax 😂 The day it fully clicks that you're not learning to "think like a computer" but rather teaching a computer to think like you — everything changes. 🤯🔥 You're clearly already there honestly 🚀 #Anthropic #Claude #AI