David Comfort, DPhil

🚀 Introducing Synthora Synthetic Data for Everyone For years, generating realistic datasets meant writing scripts, learning libraries, or building custom tools. Not anymore. Today I'm launching Synthora — a simple, powerful way to generate synthetic datasets instantly. No setup. No login. No coding. Just data. Think of it like Postman — but for data generation. With Synthora, anyone can create realistic datasets using powerful templates in seconds. Whether you're: • A developer testing APIs • A data scientist prototyping models • A tester generating edge cases • A student learning analytics Synthora makes data generation effortless. ⚡ Key capabilities • Industry-ready dataset templates • One-click dataset generation • Download as CSV or JSON • Fully customizable data fields • Works instantly in the browser No friction. No accounts. Just start generating. 💡 Why I built Synthora Data is the fuel of modern software. But getting good test data is still surprisingly painful. Synthora removes that barrier and makes high-quality synthetic data accessible to everyone. 🌐 Try it here: https://lnkd.in/gddXRUsF I’d love feedback from builders, testers, and data folks. What datasets should Synthora generate next? #AI #SyntheticData #DataEngineering #TestAutomation #DeveloperTools #OpenSource #BuildInPublic

1

When NVIDIA's DGX Spark Benchmarks Meet Production Reality WARNING: Tech Heavy Post I spent 6+ days running intensive ML workloads on an NVIDIA DGX Spark—training multiple models, fine-tuning with LoRA, and benchmarking inference. Here's what NVIDIA's 82,739 tokens/sec benchmarks don't tell you: What Worked: ✅ Training performance matches claims (10-12hr for 7 LoRA experiments) ✅ Inference speed scales as expected (80 tok/sec via Ollama) ✅ 4-bit quantization quality is excellent (<1% degradation confirmed) What Didn't: ❌ GPU inference completely broken (inf/nan errors, empty responses) ❌ Memory fragmentation causes hard reboots after 3-8 hours ❌ llama.cpp direct invocation fails (ARM64 + Blackwell issues) The Plot Twist: I spent 15+ hours debugging "training failures" across 7 experiments. Turns out all the models trained perfectly—GPU inference was the problem. Testing on CPU revealed every model worked beautifully. Production Workarounds Required: - torch.cuda.empty_cache() every 50 steps - Maximum 2-3 hour training sessions - CPU inference for evaluation - Ollama for production serving My Verdict: NVIDIA's benchmarks are technically accurate. They're just not the whole truth. The hardware is powerful, but requires expert-level knowledge to navigate current limitations. Cautiously recommended for ML engineers willing to implement workarounds. Wait 6-12 months if you need stability. Read the full deep-dive with all experimental data, code samples, and production recommendations: https://bit.ly/43AGKQ8 #MachineLearning #AI #DGX #NVIDIA #ProductionML #LLM #FineTuning #MLOps #DeepLearning #GPU

87

7 Comments

I let an AI agent run 151 experiments on my GPU overnight. Karpathy's AutoResearch, two lines changed for my hardware. The result I didn't expect: the agent had 128 GB of memory available and chose to use 6.1 GB. Not because it couldn't use more. Because using more made things worse. Turns out my GPU is step-limited, not memory-limited. At 213 TFLOPS

23

2 Comments

Large Language Models (LLMs) in Drug Discovery & Clinical Trials 🔹 What Are Large Language Models? Large Language Models (LLMs) — like GPT, Claude, or Gemini — are neural networks trained on vast text corpora to learn the statistical relationships between words, phrases, and concepts. They don’t just memorize language — they learn to represent meaning, context, and reasoning across domains. Mathematically, they model the probability of a text sequence: n P(w1​,w2​,…,wn​) = ∏ P(wt​ ∣ w1:t−1​) t=1 At scale (hundreds of billions of parameters), this allows them to: Summarize and explain scientific literature Generate new hypotheses Interpret complex data narratives Interact in natural language 🔹 What Can LLMs Do in Drug Discovery? LLMs are revolutionizing how biomedical knowledge is accessed and synthesized. 1. Knowledge Mining from Biomedical Literature LLMs can read and summarize millions of PubMed abstracts, patents, and trial reports to: Identify potential drug–target interactions Detect repurposing opportunities Summarize mechanisms of action 🧠 Example: BioGPT (Microsoft) and Galactica (Meta) trained on biomedical text to generate mechanistic summaries and suggest novel hypotheses. 2. Target Discovery and Mechanistic Insights Integrate omics data and textual knowledge to uncover biological pathways. Link genetic variants (e.g., from GWAS) to functional targets. 📘 Example: LLMs like EVA Pharma’s BioNLP pipeline and Insilico Medicine’s PandaOmics use domain-tuned LLMs to rank and prioritize novel drug targets. 3. Clinical Trial Optimization LLMs can: Read prior clinical trial protocols to design new ones Extract inclusion/exclusion criteria Match patients to trials based on EHR narratives Predict dropout risks and adverse events 💡 Real-world example: Pfizer has piloted GPT-based tools for protocol writing and trial summarization, Medidata uses LLMs for automatic adverse event classification. 4. Regulatory and Safety Applications Automate generation of clinical study reports and lay summaries Interpret spontaneous safety reports Harmonize data to CDISC and ICH standards 🔹 How to Use LLMs in Practice Fine-tune foundation models on biomedical text (PubMed, CT.gov, EHRs). Use retrieval-augmented generation (RAG) to ensure traceability — cite real evidence. Combine with structured data (omics, assays, trial metadata) for hybrid reasoning. Apply human-in-the-loop review for safety and scientific accuracy. Large Language Models (LLMs) are transforming how we discover and test new drugs. 🚀 The future of drug discovery is not just data-driven — it’s language-driven.

5

Check out this R Shiny app for dimension reduction with UMAP and t-SNE. It allows you to transform and visualize complex data directly in your browser. You can: ✔️ Create interactive UMAP and t-SNE plots ✔️ Adjust parameters for each algorithm ✔️ Apply K-means or GMM clustering ✔️ Uncover patterns in high-dimensional data without coding You can upload your own CSV file or use a built-in R data set, and the tool will provide instant visual feedback along with clustering results. The image shows the app’s interface, including data upload, preprocessing, parameter tuning, and visualization panels. The example outputs feature UMAP and t-SNE plots with K-means and GMM cluster assignments. Thanks to Dean Smith for creating this app! Try it here: https://lnkd.in/ectxbyNz Subscribe to my newsletter for more practical tips on statistics, data science, R, and Python. Further details: http://eepurl.com/gH6myT #coding #analysisskills #dataviz

24

2 Comments

The Semantic Layer Needs a New Lens in the Age of LLMs For years, we treated the semantic layer as a static abstraction: ontologies, controlled vocabularies, curated taxonomies that are designed to make data “understandable” and queryable. That model is breaking down. LLMs have fundamentally changed how meaning is extracted, connected, and reasoned, especially for unstructured scientific and clinical data. The semantic layer can no longer be thought of as a dictionary. It needs to be seen as a reasoning fabric. A concrete example from recent work: In collaboration with Som Bandyopadhyay, Ph.D., recently used LLMs to extract rich clinical and molecular context from Gene Expression Omnibus data (SOFT files) that was never designed to be analysis-ready at scale. This included: • Cancer type • Disease stage and TNM staging • Progression-free survival signals • Drugs and therapies • Mutations • Clinical responses These signals were then normalized against OncoTree, NCI Thesaurus, and related reference standards, not just to standardize terms, but to align meaning across studies, cohorts, and narratives. What made this possible wasn’t just entity extraction. It was the ability of LLMs to: Interpret context when information was implicit Resolve partial or ambiguous staging language Connect molecular findings to clinical outcomes Preserve uncertainty rather than forcing false precision This is where the old semantic layer paradigm falls short. What’s changed more broadly? • Meaning is no longer binary (mapped vs not mapped); it’s probabilistic, its fuzzy • Relationships are inferred, not only declared • Ambiguity is unavoidable but is now manageable • Explanations matter as much as answers In practice, the semantic layer now operates across three modes simultaneously: 1️⃣ Vector semantics Embeddings surface latent similarity and candidate relationships across documents and datasets before structure exists. 2️⃣ Symbolic semantics Entities and concepts are grounded into ontologies and knowledge graphs but with confidence, provenance, and temporal awareness. 3️⃣ Interpretive semantics LLMs reason across vectors and graphs to construct justified narratives: why two things are related, how strong the evidence is, and where gaps remain. This shift is especially critical in biopharma and life sciences, where: • Data is heterogeneous and retrospective • Strategy and endpoints are often implicit • Language evolves faster than curated ontologies The future semantic layer won’t enforce truth, it will: Preserve uncertainty Track evidence Support disagreement Enable faster, more consistent decisions In the LLM era, the semantic layer is not a data product but a decision infrastructure. #AI #LLMs #SemanticLayer #KnowledgeGraphs #ClinicalData #RealWorldData #Ontologies

5

1 Comment

The Production RAG Pipeline project offers a well-structured framework for deploying RAG systems at scale, focusing on orchestration, observability, and performance optimization. As RAG becomes a key component for connecting LLMs with enterprise knowledge, solutions like this are essential for building robust, production-ready AI applications. 💡 How are you approaching RAG in your workflows? What strategies have worked best for scaling?

13

We're drowning in omics data but it’s the metadata design that’s killing us. Lately, I’ve been thinking about why some datasets become goldmines for discovery while others just… sit there. A recent Cell Genomics paper by Huang et al. helped organise my own thoughts. The issue here isn’t data generation as we’re producing more omics data than ever before. The problem is how poorly we organize it. Every time I’ve tried to integrate data from multiple studies, I spend more time decoding metadata than analyzing results. One study labels “tissue type,” another says “source,” a third buries it in a PDF supplement. It’s like trying to merge spreadsheets from parallel universes. The authors make a point I wish more teams absorbed early, that this mess starts at study design! If I don’t plan my metadata schema upfront • what fields matter, • how they connect, • which vocabularies I’ll use I am essentially building my data infrastructure on sand. Later, when someone wants to reuse it, they can’t. Or worse, they misinterpret it, thus revealing several pain points that feel all too familiar: • Metadata fields added as afterthoughts or inconsistently named within the same project • Everyone inventing their own terminology for the same concept • Raw data in one repository, annotations in another, clinical details in another sheet And for me it boils down to one critical organisational component: We give ZERO incentives for clean metadata. You might not have heard about anyone being promoted for maintaining data hygiene in successive projects. The fallout is predictable. Meta-analyses that should take weeks stretch into months. Reproducibility falters. Machine learning models choke on incompatible formats. And I shudder to think of what LLMs learn and teach our next generation of keyboard warriors. To sum up, its not just a compliance issue, it’s a design problem. Here the authors argue that that standards like FAIR, MIBBI, and MINSEQE should be treated as architectural blueprints and enforced by journals, DBs, for them to be embedded in training, and baked into workflows from day one. Because at this point, sequencing speed isn’t our bottleneck. The real limitation is whether we can find, integrate, and reuse what we already have. If we design our metadata schema with the same care of our experimental protocols, our future selves and every researcher who touches that data will thank us! https://lnkd.in/gzYWzxeY

23