Software and Hardware Co-design

Why Hardware-Software Co-Design Is Non-Negotiable? Dangerous assumption: Design them independently, then stitched together later. From my experience building scalable, field-tested industrial IoT solutions, I can confidently say this approach is flawed—and costly with cause of many failures in industrial deployments. Whether you're monitoring pressure in oil & gas pipelines or automating maintenance in a smart city infrastructure, the reliability, scalability, and total cost of ownership of an IoT system depend deeply on how well the hardware and software are integrated—side by side—from day one. Technical Reasons 1. Power efficiency and performance Battery-operated devices, especially in LPWAN and NB IoT environments, require tightly optimized firmware that aligns with hardware capabilities (sleep modes, sensor wake cycles, transmission windows, and many other factors). Designing software without a deep understanding of the hardware's physical and firmware limitations results in shorter lifespans, inconsistent data, or both. 2. Connectivity optimization Protocols like LoRaWAN, NB-IoT, or Cat-M1 are not just plug-and-play. Reliable transmission depends on antenna design, shielding, payload formatting, and retry mechanisms that must be embedded in both hardware specs and software logic—together. 3. Real-time fault detection and recovery Industrial environments are noisy—electrically, physically, and digitally. Integrating diagnostics, fallback strategies, and sensor validation into both firmware and cloud platform ensures that small glitches don’t turn into expensive field failures. 4. OTA updates and lifecycle management Without co-design, firmware updates become a logistical nightmare. A unified design ensures that remote updates are reliable, secure, and hardware-aware—so they don't brick your devices in the field. Non-Technical (But Just as Critical) Reasons 1. Lower long-term cost Reworking firmware or cloud APIs post-production is exponentially more expensive than doing it right upfront. Co-design reduces iteration cycles, deployment delays, and support overhead. 2. Faster time to market When teams work in silos, integration becomes a bottleneck. Side-by-side development removes surprises and streamlines validation—cutting months off your release timeline. 3. Better user experience From installation to data visualization, a co-designed solution feels cohesive. Installers don’t struggle with mismatched instructions. Platform users don’t question sensor data accuracy. Everyone wins. 4. Future-proofing the solution When hardware and software evolve in sync, scaling to new features or integrating with third-party platforms becomes a natural progression—not a painful migration. So, be assured hardware and software designed in the same room, by teams who speak the same language? If not, you're probably not building a solution. You're building a future problem. Let’s build smarter. #lpwan #IoT #lorawan #nbiot #ellenex

Last semester, I taught a graduate-level computer architecture class in which we read many accelerator design papers. After a dozen or so papers, it became clear to us that computer architects are not fully taking advantage of the optimization opportunities for accelerator memory system design. Please consider the following... Unlike CPUs, which strive to run anything well, accelerators typically only run a few specific kernels, allowing their memory system (and memory system interactions) to be significantly specialized and optimized. Below, I've made a design matrix that highlights some new opportunities for accelerator memory system design. When designing your accelerator memory system, ask these two additional design questions: 1) Is my address stream INPUT-DEPENDENT or INPUT-OBLIVIOUS? If input-OBLIVIOUS (as many kernels are), then your memory system design and interactions should be VERY SIMPLE, since you can anticipate addresses as early as needed, including in the compiler, allowing effective use of compiler prefetch. If addresses are input-dependent, then you may need to add speculative prefetch, caching, and cache coherence. 2) Is the data I am accessing DENSE or SPARSE? If dense, then invest in scratchpads and caches; otherwise, attempt to utilize blocking, expose parallelism in the memory system, and build a high-BW memory system. The more interesting design points arise when considering both of these questions in tandem: For example, if you are building a deterministic unpruned DNN inference accelerator (with dense data and input-oblivious addresses), you are locked into "easy design mode", so focus on compiler prefetching into scratchpad memories. (If you are adding a cache to this accelerator, you should ask yourself this question: "Why?" 😎) If you are building an accelerator for kNN search over a large dataset (with sparse data and input-dependent addresses), you are locked into "hard design mode", so go crazy and invest in caches, high B/W memory interfaces, parallel memory requests, speculative prefetch, and whatever other (effective) cleverness you can conjure. If you are up for a hardware-software co-design challenge, ask yourself this question: Can I move my accelerator kernel in the direction of a more simple (and likely more efficient) design by making its addresses input-oblivious and/or its data more dense? If you understand your kernel well, the answer may be "yes". Do these design considerations ring true for your designs? What other considerations should accelerator designers ponder? #computerarchitecture #memory #accelerators

Nine months ago, I published my paper on designing an AI hardware accelerator from scratch—a challenging yet rewarding journey. Unlike the common trends of binary computing, analog approaches, or neuromorphic designs, this project pushed the boundaries of digital logic by rethinking information theory. A deep integration of hardware-software co-optimization demanded versatility and proficiency in both high-level and low-level programming, and the creation of custom design automation to manage the vast complexity of CNN parameters. Recently, I came across one of my screenshots on MNIST CNN classification. It felt unreal at first but realizing that the results were on actual hardware turned that feeling into pride. It’s a glimpse into the potential of FPGA in future AI hardware by achieving CNN classification in nanoseconds. No AIE, no DSP, no BRAM, just pure LUT horsepower. https://lnkd.in/g8CrUJqX #AI #FPGA #Innovation

$2M co-location upgrade. FPGA deployment. New server racks. Microsecond-class NICs. Board-approved. The latency profile didn't move. My team profiled the stack. The bottleneck was not the switch, the NIC, or the fiber run. It was a sorted array in the order book. Specifically: inserts at non-best price levels were O(N). Every new order not at the top of the book triggered a linear scan to find its position. At 10,000 orders per minute — the volume Island ECN was handling by 1999 — that scan compounds into a wall. The hardware was waiting on the algorithm. This is not a new problem. Josh Levine solved it in 1996 running FoxPro on MS-DOS (I wrote about how he built it — link in comments). Island ECN's matching engine was not impressive hardware. It was a DOS machine at 50 Broad Street, co-founded with Jeff Citron at Datek Securities. Single-threaded, event-driven by deliberate design — no context switches, no lock contention. The order book used in-memory B-tree indexing via an ISAM storage engine. Zero disk access during matching. Every price level accessed in O(log N) time. The result: 2 milliseconds end-to-end latency when Instinet, the dominant venue, was delivering 2 seconds. 100x improvement on commodity hardware that cost less than a single month of co-location fees. By 2001, Island was clearing 350 million shares per day at 10,000+ orders per minute without slowdown. Levine later documented his own design philosophy: it is usually the architecture and algorithms that matter more than raw platform speed. That sentence does more work than most engineering teams give it credit for. When Binance upgraded its matching engine logic — not its hardware — order processing dropped from 10ms to 5ms and daily trade volume increased 15% in one week. No new servers. No FPGA refresh. A software change. Has the order book data structure been profiled at the price-level access layer — specifically insert and lookup complexity at non-best prices? If not, or if the answer is "sorted list," the hardware budget is premature. Modern tier-1 matching engines achieve sub-5-microsecond per-order processing on price-level hash maps with O(1) access, not on server rack upgrades. Island proved the architectural ceiling on a DOS machine. The lesson has been available for thirty years. >>> The data structure is the bottleneck. It almost always was. #HFT #MatchingEngine #TradingTechnology #MarketMicrostructure #OrderBook #ElectronicTrading #CapitalMarkets #TradingInfrastructure #LowLatency #IslandECN Nasdaq

Software-first design isn’t just for apps or websites anymore, it’s the new foundation for hardware as well. With AI making software 10x more powerful, hardware innovation can no longer rely on rigid, monolithic designs. Look at Palantir Technologies’s Titan project: for the first time, a software company acted as the prime contractor for a major hardware program. Titan is a modular, software-defined architecture that flips traditional military system design on its head. Built on Modular Open Systems Approach (MOSA) principles, it allows components to be added, removed, or replaced incrementally throughout the system’s lifecycle. New sensors, capabilities, or tech upgrades can be integrated rapidly, no full redesign needed. The system evolves continuously, adapting to new threats and technologies as they emerge. This isn’t unique to defense. Tesla's software-first thinking lets them push OTA updates, unlock new features, and optimize performance without touching a single bolt on the hardware. Their approach demonstrates that the real leverage is in the software that drives it. Most traditional manufacturing and hardware companies are still thinking in terms of fixed BOMs, rigid assembly lines, and multiyear update cycles. That mindset won’t survive in a world where software-first design accelerates innovation by orders of magnitude. Rethink hardware from a software-first perspective: modular architectures, AI-driven integration, and rapid evolution built in from day one. If you want to move from reactive upgrades to continuous innovation, software-first is the only way forward.

We’re no longer designing chips.  We’re engineering ecosystems—across die, data, and dimension. From AMD’s Zen5-based 3D V-Cache to UCIe 2.0 and TSMC’s AI-powered 3Dblox workflows, the chiplet era isn’t just here—it’s evolving fast. Here’s how the game is shifting: 🔹 Vertical isn’t just about stacking—it’s about performance density. The Ryzen 9800X3D isn’t just faster—it’s architecturally smarter. • +500 MHz base clocks • 3x L3 cache via vertical die • Uniform latency from equidistant cache layers Result? 15–23% uplift in CPU-bound gaming without increasing power draw. This isn’t just adding cache. It’s about bringing it closer to intent—matching compute paths to workloads. 🔹 UCIe 2.0 is making chiplets truly modular. Forget proprietary socket dances—this is plug-and-play at silicon scale. • <1μm bump pitch = 82% lower latency • Unified DFx = seamless cross-vendor integration • FLIT-based links = 3x energy efficiency Hybrid bonding, protocol-agnostic transport, and thermal/power telemetry are the real infrastructure for composable computing. 🔹 AI is now a co-designer. With 3Dblox 2.1, TSMC is running electrothermal-stress convergence during layout. • 19% thermal improvement in early floorplanning. • 12–15°C lower hotspots—before tapeout. This means AI isn’t optimizing for benchmarks. It’s optimizing for reliability, yield, and lifecycle from day zero. 🔹 This all converges at one truth: 200B+ transistor designs can’t scale with human heuristics alone. - You need AI. - You need interoperability. - You need an abstraction that respects physics. So here’s the real challenge for engineers today: Are you designing for specs? Or for systems? From TSMC to AMD, we’re moving from “how fast this chip can go” to “how robust this stack is at scale.” If you’re in silicon, architecture, or AI-hardware convergence, this moment isn’t optional. It’s defining. Curious: Where do you see the biggest bottlenecks in multi-die design today? Thermals, testing, yield, integration? Let’s trade notes.

Smart, connected, Software-Defined Products (SDP) are driving innovation in nearly every industry from medical devices to aircraft. And software and semiconductors are at the foundation of every one of these software-defined products.  Embracing the complexity this has introduced by optimizing semiconductors, software, electrical and mechanical systems in a Comprehensive Digital Twin (CDT) is the only way to gain a significant competitive advantage   Semiconductors are at the heart of these new products, so let's dig a bit more into how the CDT can accelerate semiconductor development.  But first, what is the CDT? ** A digital twin is a physics-based digital representation of an asset or process. To be comprehensive, the digital twin must include all the elements required to define a product, production process or business operations, ** incorporate information across all domains -- semiconductor, software, electrical and mechanical, ** and span across the lifecycle from engineering to manufacturing to deliver and support. Why is this important for the semiconductor industry?   First, semiconductors exist within the context of a product, such as an automobile, which means they should be designed and verified in the context of the entire product. This includes software, the wire harness and how they will connect to other systems of the car. The CDT is the only way to do this and in turn understand the performance characteristics of the semiconductor as well as how long it will take for the semiconductor and software together to interact with the car’s systems.    This interaction of the software and semiconductors is critical for SDP, which means companies can no longer afford to select an off-the-shelf processor and then build around it.  Due to rapidly advancing product complexity, it would result in a suboptimal solution that ultimately limits the features that can be added in the future or worse, creates a product not capable of handling all the software features. The CDT enables companies to codevelop the semiconductor and software architecture to deliver an optimized solution that meets the requirements of their product, today, and has room to upgrade with new software features in the future.   Finally, companies need to embrace new chip designs and architecture. 3D-IC helps accelerate the design of new chips so companies can focus on incorporating the most advanced nodes in a chiplet, and then build around it with existing solutions. This in turn can accelerate the design, testing and availability of new chip designs, but it does introduce new challenges for thermal management and the mechanical design of the chip, highlighting the need for the CDT and a multi-domain design environment.    If you are interested in learning more, I recently had an opportunity to discuss some of these challenges with my colleague Michael Munsey on a new podcast series. You can find the link to the series in the comments below. #digitaltransformation

We've flipped the script in semiconductor design. Used to be hardware first, software second. Not anymore. https://lnkd.in/eKBKXgYr Your iPhone getting better battery life and camera performance with each update? That's not just new apps - it's the hardware being optimized through software after you bought it. This fundamental shift means we're now starting with software requirements and building custom silicon around them. The challenge: how do you verify software when the hardware doesn't exist yet? We're using virtualization that evolves from virtual models to hybrid environments to full RTL. It's complex, but necessary when a processor lockup in automotive means lives at risk. What co-verification approaches are you seeing work in practice? #Semiconductors #SoftwareDefined #SiemensDigital

FPGAs offer unmatched flexibility and performance, but integrating them into complex systems can be daunting. The key to success lies in planning, collaboration, and leveraging best practices. 💡 Challenges in FPGA System Integration Ensuring seamless communication with other components (processors, memory, peripherals). Managing system-level timing constraints and resource allocation. Coordinating hardware and software development for optimal performance. 🛠 Strategies for Seamless FPGA Integration 1️⃣ Start with System Architecture Planning Define clear roles for the FPGA within the system (e.g., signal processing, co-processing, or interfacing). Collaborate across hardware and software teams to align on system requirements and constraints early on. 2️⃣ Use Industry-Standard Interfaces and Protocols Choose widely adopted protocols like AXI, PCIe, or Ethernet for compatibility and scalability. Leverage pre-built IP cores or previously developed modules to simplify integration and accelerate development. 3️⃣ Focus on Timing and Synchronization Perform system-level timing analysis early to ensure reliable communication between FPGA and other components. Use tools like timing constraint managers to handle complex clock domains and achieve design closure. 4️⃣ Co-Design Hardware and Software Implement a hardware/software co-design approach to balance workloads effectively. Develop FPGA drivers and APIs that simplify interaction with higher-level software. 5️⃣ Test and Validate at the System Level Simulate the system to identify integration issues before physical testing. Validate the FPGA’s functionality within the target system using test frameworks and real-world scenarios. 🔑 The Takeaway: Integrating FPGAs into complex systems isn’t just about hardware—it’s about aligning teams, tools, and strategies to create seamless, efficient solutions. What’s your biggest challenge in FPGA system integration? #fpgadesign #fpga #hardwaredesign #productdevelopment #innovation