Data-Driven Load Optimization

I took this report’s load time from 10-15 seconds to less than 1 second.. and reduced its model size from 192 MB to just 20 MB, approximately 90% reduction! For the Fabric User Group Nigeria September Challenge. The business problem was to optimize a slow-loading executive dashboard for Van Arsdel that was causing significant productivity and confidence issues. Leveraging Semantic Link Labs, my core actions were: 📍Streamlined Data Model & Query Steps: I used Power Query to disable unused tables and eliminate unreferenced columns, which was a key factor in reducing memory footprint. 📍Optimized Relationships: I replaced a problematic many-to-many relationship with an efficient one-to-many setup using a bridge table and switched to single-directional filters to improve query performance. 📍Disabled Auto Date/Time: This feature adds hidden, resource-intensive calendar tables. Turning it off immediately made the model leaner. 📍Refactored DAX: I replaced inefficient DAX measures that were forcing multiple table scans with streamlined, standard time intelligence functions like DATEADD, resulting in significant performance gains. Business Impact? The improvements I made directly addressed the business's pain points: ✅Increased Productivity: Executives now save 2-3 hours per week with a fast, responsive dashboard, allowing them to focus on strategic tasks rather than waiting for data to load. ✅Faster Decision-Making: The dashboard is now a reliable tool for quarterly planning, eliminating the delays that were affecting the business. ✅Restored Stakeholder Confidence: The dashboard now loads instantly, ensuring smooth, professional board presentations and reinforcing confidence in the data and the team behind it. For more detail, read repo: https://lnkd.in/dGBc4gCy

Setting up trending on a BAS (Building Automation System) network to minimize bandwidth consumption while providing real-time access to data involves a strategic approach to data collection, storage, and retrieval. Here are the steps to achieve this: 1. Adjust Polling Intervals: Set polling intervals based on the criticality and variability of the data. For less critical data, use longer intervals. Event-Driven Polling: Use change-of-value (COV) polling instead of periodic polling for data points that change infrequently. This means data is sent only when a change occurs. 2. Local Aggregation: Aggregate data locally at the field controllers before sending it to the central station. This reduces the amount of data sent over the network. Hierarchical Trending: Use a hierarchical trending approach where data is collected and stored at multiple levels, such as field controllers, supervisory controllers, and the central station. 3. Data Compression: Utilize data compression techniques to reduce the size of the data being transmitted. Niagara Framework supports various data compression methods. Delta Compression: Only send the difference (delta) between the last reported value and the current value. 4. Trend Only Essential Data: Identify and trend only the most critical data points. Avoid trending points that provide little value or insight. Trend Filtering: Apply filters to trend logs to limit data to specific ranges, times, or conditions. 5. Use Historical Databases: Store historical data in an optimized database designed for time-series data. Niagara typically uses the built-in history database, but you can also integrate with external databases. Data Archiving: Implement a data archiving strategy to move older data to long-term storage, reducing the load on the primary database. 6. Data Caching: Cache data locally on the client side to reduce the need for repeated data requests. WebSockets and Push Notifications: Use WebSockets or other push notification mechanisms to provide real-time updates to clients without constant polling. 7. Segment the Network: Use VLANs or other network segmentation techniques to separate BAS traffic from other network traffic, ensuring optimal performance. Quality of Service (QoS): Implement QoS policies to prioritize BAS traffic on the network. 8. Regularly Review and Adjust Trends: Periodically review the trends and adjust configurations as needed based on the usage patterns and network performance. Monitor Network and System Performance: Continuously monitor the network and system performance to identify and address any bottlenecks or issues. By implementing these strategies, you can ensure that the trending on your BAS network is efficient in terms of bandwidth consumption while providing real-time access to critical data for end users.

The data center buildout is about to create one of the most compelling economic cases for load flexibility the energy industry has ever seen — and most people aren’t talking about it yet. Tens of gigawatts of new AI-driven load are coming online backed by high heat rate reciprocating gas engines. These units are presented as a bridge — fast to permit, fast to deploy, fast to get load connected. But they are expensive to run. Once that load is live and those engines are the thing holding the lights on, operators are going to be staring at fuel bills that make demand flexibility look extraordinarily cheap by comparison. The merit order math is unforgiving: when your marginal unit costs $150/MWh or more to operate, any resource that can reduce net load at that margin has enormous value. This is where load flexibility wins — not as a sustainability story, but as a pure economics story. You don’t need to build anything to tap into it. The capacity is already embedded in the load itself. Shift a training job by two hours. Throttle a cooling system during a peak window. Sequence GPU workloads to avoid coincident peaks. The avoided cost of not running that reciprocating engine is the value signal, and it’s a strong one. At Octopus, we’ve long believed that load flexibility is the fastest and cheapest form of capacity — and the data center buildout is about to prove that at massive scale. What we are entering is a period of structural, not episodic, pressure to find alternatives to expensive marginal generation. The operators and utilities who build flexibility into their data center strategies now — rather than treating it as an afterthought — will have a significant cost advantage. The ones who don’t will spend the next decade paying for it, one expensive engine-hour at a time.

𝗛𝗼𝘄 𝘁𝗼 𝗔𝗽𝗽𝗹𝘆 𝗤𝘂𝗮𝗻𝘁𝘂𝗺-𝗜𝗻𝘀𝗽𝗶𝗿𝗲𝗱 𝗔𝗹𝗴𝗼𝗿𝗶𝘁𝗵𝗺𝘀 𝘁𝗼 𝗗𝗮𝘁𝗮 𝗖𝗲𝗻𝘁𝗲𝗿 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 (𝗔𝗜𝗢𝗽𝘀 𝗪𝗶𝘁𝗵𝗼𝘂𝘁 𝗮 𝗤𝘂𝗮𝗻𝘁𝘂𝗺 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝗿) Most leaders hear “quantum” and think of it as experimental, expensive, and years away. That’s a mistake. Quantum-inspired algorithms run on classical infrastructure today and solve the hardest problem you actually have: large-scale optimization under constraints. If you run data centers, this is immediately actionable. What they actually do They convert your environment into an energy minimization problem. Instead of brute forcing every possibility, they rapidly converge on high-quality solutions across massive decision spaces. Think: • Placement • Scheduling • Routing • Thermal balancing • Power allocation Where to apply first (high ROI use cases) 1. Rack and cluster placement Model racks, power domains, cooling zones, and network topology as constraints. Objective: minimize latency + cable length + thermal hotspots. 2. GPU scheduling and utilization: Encode job priority, SLA windows, GPU affinity, and network contention. Objective: maximize utilization while reducing idle burn and queue latency. 3. Thermal + power balancing: Integrate cooling capacity, airflow constraints, and power density. Objective: flatten hotspots without over-provisioning. 4. Network traffic shaping Model east-west traffic flows and oversubscription ratios. Objective: Reduce congestion and packet loss under peak load. How to implement (practical workflow) Step 1: Define variables • Binary: placement decisions, routing paths • Continuous: load, temperature, power draw Step 2: Define constraints • Power caps per rack and row • Cooling limits by zone • Network bandwidth ceilings • SLA requirements Step 3: Build the objective function. Combine into a weighted cost function: • Latency • Energy consumption • Thermal deviation • Resource fragmentation Step 4: Select a solver. Use simulated annealing or related heuristics to explore the solution space efficiently. Step 5: Iterate with real telemetry. Feed in live data: • DCIM • BMS • Scheduler metrics: Continuously refine the model. What “good” looks like • 10–25% improvement in GPU utilization • Lower east-west congestion without network upgrades • Reduced thermal excursions • Faster schedule generation cycles Where most teams fail • Overfitting the model before validating its impact • Ignoring real-time telemetry • Treating this as a one-time optimization instead of a continuous system Bottom line: You don’t need quantum hardware to get quantum-level thinking. You need a structured optimization model and the discipline to iterate it against real operating data. If you’re running >10MW environments and not doing this, you’re leaving efficiency and margin on the table. #DataCenters #AIInfrastructure #GPU #Optimization #HighPerformanceComputing #Cloud #Infrastructure #DigitalTransformation

There's 1 spot for a 100 MW data center in Belgium. Add 5% demand flexibility: suddenly there are 16. That’s the power of flexibility as visualized by Elia Group - Belgium’s transmission system operator. Looking at Elia’s grid capacity map for 2027, there’s literally one (new) spot in the country where you can connect a 100 MW data center. And 100 MW isn’t even that big. But if you allow for 5% of demand flexibility - which could mean shifting or reducing the load, or replacing it with on-site generation, there’s suddenly 16 free spots for new industrial loads, many up to 300 MW. If you go to 20% (which is a bit extreme), the grid pretty much stops becoming a bottleneck. Of course you wouldn’t want to close your data center for 2 months in a year. But actually, the yearly average load of data centers is 50%. So there’s clearly space for optimization. In fact, a recent trial by Nebius, Emerald AI, EPRI and National Grid showed that a test AI cluster in London could slash load by 30% in 40 seconds in response to sudden grid stress, while keeping critical jobs. Even better, it could sustain multi-hour load reductions of 10-40% and still deliver 99% performance on highest priority jobs. And you can optimize further: with onsite battery storage, you can shape the daily load profile to match the grid’s needs, potentially getting an accelerated connection agreement and saving money in the process. More in tomorrow's newsletter on data center flexibility. EDIT: a good point raised in the comments - I should clarify that the grid capacity map shows the connection capabilities for new projects, so on top projects that have already secured a grid connection.