Cloud Storage Cost Management

Over the last 1 year, we helped 15+ companies cut their cloud bills by 30-40% in 45 days (without a single new tool). Here’s what most cloud teams don’t realize: ❌ You don’t have a cost problem. ✅ You have a waste problem hidden in plain sight. We attacked the invisible waste buried deep in their Kubernetes clusters: 1. 𝐑𝐞𝐪𝐮𝐞𝐬𝐭𝐬 𝐚𝐧𝐝 𝐋𝐢𝐦𝐢𝐭𝐬 𝐖𝐞𝐫𝐞 𝐒𝐞𝐭… 𝐚𝐧𝐝 𝐅𝐨𝐫𝐠𝐨𝐭𝐭𝐞𝐧 Developers set inflated CPU/memory limits “just in case” and never revisited them. We ran real-time profiling using Prometheus + Grafana and recalibrated limits based on actual sustained usage. This alone brought down cluster size by 15-20%. 2. 𝐍𝐨𝐧-𝐏𝐫𝐨𝐝 𝐄𝐧𝐯𝐢𝐫𝐨𝐧𝐦𝐞𝐧𝐭𝐬 𝐖𝐞𝐫𝐞 𝐓𝐫𝐞𝐚𝐭𝐞𝐝 𝐋𝐢𝐤𝐞 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 Dev, QA, and Staging environments ran on on-demand instances (24/7). We moved them to spot instances with scheduled shutdowns during non-working hours. That delivered 18-22% savings instantly. 3. 𝐀𝐮𝐭𝐨𝐬𝐜𝐚𝐥𝐞𝐫𝐬 𝐖𝐞𝐫𝐞 𝐌𝐢𝐬𝐜𝐨𝐧𝐟𝐢𝐠𝐮𝐫𝐞𝐝 𝐨𝐫 𝐉𝐮𝐬𝐭 𝐈𝐝𝐥𝐞 Most teams rely purely on CPU-based HPA, which reacts too late. We introduced custom scaling triggers based on business KPIs like request queue lengths, job backlogs, and latency. The result? Clusters scaled proactively, not reactively. 4. 𝐙𝐨𝐦𝐛𝐢𝐞 𝐏𝐨𝐝𝐬 𝐚𝐧𝐝 𝐅𝐨𝐫𝐠𝐨𝐭𝐭𝐞𝐧 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞𝐬 𝐄𝐯𝐞𝐫𝐲𝐰𝐡𝐞𝐫𝐞 One client had 300+ idle pods running outdated builds (nobody knew why). We implemented automated cleanup jobs using lifecycle policies and kubectl prune scripts. That reduced node count immediately. 5. 𝐕𝐞𝐫𝐭𝐢𝐜𝐚𝐥 𝐏𝐨𝐝 𝐀𝐮𝐭𝐨𝐬𝐜𝐚𝐥𝐞𝐫 (𝐕𝐏𝐀) 𝐖𝐚𝐬𝐧’𝐭 𝐄𝐯𝐞𝐧 𝐄𝐧𝐚𝐛𝐥𝐞𝐝 VPA handled unpredictable workloads far better than manual tuning.   For stateful apps with variable patterns, this reduced over-provisioning by up to 25% while maintaining SLAs. 6. 𝐏𝐞𝐫𝐬𝐢𝐬𝐭𝐞𝐧𝐭 𝐕𝐨𝐥𝐮𝐦𝐞 𝐂𝐥𝐚𝐢𝐦𝐬 (𝐏𝐕𝐂𝐬) 𝐖𝐞𝐫𝐞 𝐚 𝐁𝐥𝐚𝐜𝐤 𝐇𝐨𝐥𝐞 Storage costs were silently draining budgets. We audited PVC usage, downgraded unnecessary high-IOPS gp2 volumes to gp3, and cleaned up stale volumes. For one client, this alone saved over $30,000 annually. Before you buy another cloud cost management tool, ask yourself… Have you really optimized what you already own? ♻️ 𝐑𝐄𝐏𝐎𝐒𝐓 𝐒𝐨 𝐎𝐭𝐡𝐞𝐫𝐬 𝐂𝐚𝐧 𝐋𝐞𝐚𝐫𝐧.

After more of a decade of cleaning up Azure enviroments! here are some of my tips (outside of RI and rightsize SKUs which everyone else talks about is the common tips..) ✅ Check for Orphaned Resources (Disks, App Service Plans, SQL, PIP) Use the Orphaned Resources Workbook in Azure Monitor. ) Do some manual checks to see which workloads are actually being used. ✅ Workloads that are stopped but not deallocated? That means that hardware is still reserved and the cost is still running. Either deallocate or delete them. ✅ Check for Old snapshots of disks (So many old test and lab enviroments here) ✅ Check for backup tier (Do you need GRS?) or is ZRS, which is 40% cheaper good enough? ✅ If you do not need lower then 24 hours RTO on Backup, use standard policy and not enhanced. (While enhanced is the only one supported for Premium v2 disks but is a lot more expensive) ✅ What kind of redundancy do you need on your storage? LRS/ZRS/GRS? Adding GRS provides higher redundancy but higher cost and latency. Use LRS on active storage and use higher-level on backup data. ✅ Check for disk type (Many disks can be configured with SSDv2 tier which can be cheaper and faster in many cases) ✅ Check for which logs are actually needed? A AKS Cluster alone can generate close to 23 GB a month without little workloads. Unfortunately few have a good strategy around logs. What is needed and why? Just by disabling kube-admin can save you much. This applies for all services and workloads you have. ✅ Check for which logs are needed for security? By enabling Sentinel on a Log Analytics workspace you are paying for the data ingested and not how many analytics rules you have. So much logs are just being collected without being used. ✅ Do you need all Defender for Cloud SKUs enabled? Defender for Storage has a cost of 10$ per account. Make sure you use Defender for Cloud service where it matters! production workloads) ✅ Can you use Workspaces with API Management instead of having multiple production API Management instances? ✅ DDoS on IP instead of Network. In most cases you require DDoS protection on certain external services but not everything. Using IP based protection is a lot cheaper compared to Network based protection. ✅ What kind of storage do you need? Azure has different NFS/SMB based storage options providing much of the same capabilities such as Azure Files and Azure NetApp files, but there is a high cost difference between them. ✅ Do you need Private Endpoints? (Cost per PE, Cost bandwidth cost, Cost VNET Peering) or can services (storage) be locked down using Service Endpoint with or without policies? ✅ Is LicenseType = "Windows_Client" set on AVD machines? It does it automatically using the portal, but not via Terraform/Bicep. This ensures that you do not pay for Windows license for AVD workloads. ✅ Standalone or centralized services? I see so often redundant instances APIM, WAF, NAT, Backup vaults, try and avoid redundant services

HubSpot saved millions in AWS S3 storage costs because of this simple shift by their backend performance team. Here’s exactly how they did it.  1. Identifying the Cost Problem - The Backend Performance Team at HubSpot focused on optimizing costs by analyzing cloud spending, specifically in AWS S3 storage.   - They discovered that S3 storage accounted for 45-50% of daily cloud costs.   - Two primary cost drivers:     1. Raw JSON logs (~31 petabytes of request logs).   2. Compaction lag: Only 30% of logs were being converted to ORC format due to bottlenecks.  2. Hypothesis for Savings - Compressing all logs to Optimized Row Columnar (ORC) format could reduce storage size by 95%.   - ORC was chosen because it provided better compression and was already supported by their existing infrastructure.   - They also identified TTL (time-to-live) discrepancies: Raw logs were stored for 730 days vs. ORC logs for 460 days, leaving room for optimization.  3. Redesigning the Logging Process - They reworked their pipeline to convert raw logs to ORC immediately during the staging phase to avoid JSON bloating. - Streaming conversion was implemented to process logs in real-time, ensuring better performance and reducing backlog.   - 140 workers were deployed to backfill existing 34.7 PB of JSON logs, converting them to 1.47 PB of ORC logs—achieving a 4.24% final storage size.  4. Execution & Results - The backfill process took 8 days, and 34.7 PB of logs were converted to ORC, reducing costs by seven figures (over $1 million).   - Monthly JSON log costs decreased by 55.7%, while ORC bucket costs increased by only 6.4% of the original JSON costs.   - Net Savings:     - One-time savings from the TTL reduction: 6 figures.     - Total yearly savings: 7 figures.   5. User Experience Impact - Engineers reported that query times dropped from 30 minutes to 36 seconds for high-throughput services due to ORC’s improved performance.   6. Key Takeaways - Cost-saving projects require revisiting assumptions and configurations (like TTL settings).   - HubSpot reduced storage costs and improved query performance, ensuring long-term scalability.   - Cost optimization isn’t just technical—regular audits of cloud usage can reveal hidden savings. This project shows how simple changes in data management like switching to ORC compression can yield massive financial and operational benefits.

If I were Head of FinOps of a SaaS company, here’s my 4-step playbook to cut up to 20% off our cloud costs, avoid expensive vendor lock-in, and align my entire company on cloud spending: This playbook is simple, but you’d be surprised how much the basics can help transform your bottom line. Here’s my playbook: 1. Understand your workloads You need to know what workloads you’re running and whether they’re predictable or dynamic. - Predictable If you have workloads that don’t change a lot – as in, you can forecast cloud costs accurately — lock in volume discounts like reserved instances or savings plans. - Dynamic If you have no idea what the resource profile of certain workloads will look like,  say you’re innovating, stick with on-demand capacity. You don’t want to risk overcommitting to enterprise discount pricing (EDP). For instance, if your actual spend is $70M but you commit to $250M, that’s a painful conversation with the CFO waiting to happen. 2. Stop running your engine overnight Instances running 24/7 without being used are a hidden cost killer. Implementing automated scheduling systems to power down these instances during periods of inactivity can significantly reduce costs. It’s like turning off your electric car overnight so you can drive it the next day without recharging. This may be straightforward. But at scale, this simple change can free up a significant budget. 3. Attached storage waste Storage utilization is often overlooked. One of our customers had a petabyte-sized S3 bucket costing $10k per month – yet no one knew what it was for. Right size your instances and audit storage usage regularly. Otherwise, you’re wasting resources like using a tank to kill a rat. 4. Make cost management a KPI Cloud cost visibility must be a company-wide priority – a top-level KPI so everyone knows they’re accountable. Focusing on this can lead to up to20% savings as people start paying attention to what’s being spent and why. Final thoughts: Cloud cost management is like fitness: every day counts. You won’t see the results immediately, but your expenses will balloon without consistent effort. Start today, focus on the basics, and watch your costs shrink over time. Pay now or pay later – the choice is yours.

Imagine you’re filling a bucket from what seems like a free-flowing stream, only to discover that the water is metered and every drop comes with a price tag. That’s how unmanaged cloud spending can feel. Scaling operations is exciting, but it often comes with a hidden challenge of increased cloud costs. Without a solid approach, these expenses can spiral out of control. Here are important strategies to manage your cloud spending: ✅ Implement Resource Tagging → Resource tagging, or labeling, is important to organize and manage cloud costs. → Tags help identify which teams, projects, or features are driving expenses, simplify audits, and enable faster troubleshooting. → Adopt a tagging strategy from day 1, categorizing resources based on usage and accountability. ✅ Control Autoscaling → Autoscaling can optimize performance, but if unmanaged, it may generate excessive costs. For instance, unexpected traffic spikes or bugs can trigger excessive resource allocation, leading to huge bills. → Set hard limits on autoscaling to prevent runaway resource usage. ✅ Leverage Discount Programs (reserved, spot, preemptible) → For predictable workloads, reserve resources upfront. For less critical processes, explore spot or preemptible Instances. ✅ Terminate Idle Resources → Unused resources, such as inactive development and test environments or abandoned virtual machines (VMs), are a common source of unnecessary spending. → Schedule automatic shutdowns for non-essential systems during off-hours. ✅ Monitor Spending Regularly → Track your expenses daily with cloud monitoring tools. → Set up alerts for unusual spending patterns, such as sudden usage spikes or exceeding your budgets. ✅ Optimize Architecture for Cost Efficiency → Every architectural decision impacts your costs. → Prioritize services that offer the best balance between performance and cost, and avoid over-engineering. Cloud cost management isn’t just about cutting back, it’s about optimizing your spending to align with your goals. Start with small, actionable steps, like implementing resource tagging and shutting down idle resources, and gradually develop a comprehensive, automated cost-control strategy. How do you manage your cloud expenses?

How I Cut Cloud Costs by $300K+ Annually: 3 Real FinOps Wins When leadership asked me to “figure out why our cloud bill keeps growing Here’s how I turned cost chaos into controlled savings: Case #1: The $45K Monthly Reality Check The Problem: Inherited a runaway AWS environment - $45K/month with zero oversight My Approach: ✅ 30-day CloudWatch deep dive revealed 40% of instances at <20% utilization ✅ Right-sized over-provisioned resources ✅ Implemented auto-scaling for variable workloads ✅ Strategic Reserved Instance purchases for predictable loads ✅ Automated dev/test environment scheduling (nights/weekends off) Impact: 35% cost reduction = $16K monthly savings Case #2: Multi-Cloud Mayhem The Problem: AWS + Azure teams spending independently = duplicate everything My Strategy: ✅ Unified cost allocation tagging across both platforms ✅ Centralized dashboards showing spend by department/project ✅ Monthly stakeholder cost reviews ✅ Eliminated duplicate services (why run 2 databases for 1 app?) ✅ Negotiated enterprise discounts through consolidated commitments Impact: 28% overall reduction while improving DR capabilities Case 3: Storage Spiral Control The Problem: 20% quarterly storage growth, 60% of data untouched for 90+ days in expensive hot storage My Solution: 1, Comprehensive data lifecycle analysis 2, Automated tiering policies (hot → warm → cold → archive) 3, Business-aligned data retention policies 4, CloudFront optimization for frequent access 5, Geographic workload repositioning 6, Monthly department storage reporting for accountability Impact: $8K monthly storage savings + 45% bandwidth cost reduction ----- The Meta-Lesson: Total Annual Savings: $300K+ The real win wasn’t just the money - it was building a cost-conscious culture** where: - Teams understand their cloud spend impact - Automated policies prevent cost drift - Business stakeholders make informed decisions - Performance actually improved through better resource allocation My Go-To FinOps Stack: - Monitoring: CloudWatch, Azure Monitor - Optimization: AWS Cost Explorer, Trusted Advisor - Automation: Lambda functions for policy enforcement - Reporting: Custom dashboards + monthly business reviews - Culture: Showback reports that make costs visible The biggest insight? Most “cloud cost problems” are actually visibility and accountability problems in disguise. What’s your biggest cloud cost challenge right now? Drop it in the comments - happy to share specific strategies! 👇 FinOps #CloudCosts #AWS #Azure #CostOptimization #DevOps #CloudEngineering P.S. : If your monthly cloud bill makes you nervous, you’re not alone. These strategies work at any scale.

Cloud costs are becoming the blind spot in digital transformation. A huge mistake is thinking cost control comes after deployment. Gartner, IDC, and regional surveys show the same thing: Cloud adoption is scaling, and so is waste. It raises hard questions for every delivery lead: How do we track value, not just spend? How do we forecast with accuracy? How do we stay cost-resilient across regions? It’s not about the cloud provider. It’s about the discipline behind it. And the reality: 94% of global organisations report cost overruns. Most common culprits? Idle compute. Unused storage. No tagging. No shutdown policies. Here’s why it keeps happening: → No unit cost ownership → No spend visibility at the service level → No roadmap alignment These aren’t random misses. They’re signs of a systemic problem: → Engineering owns infra ≫ not budgets → Finance owns totals ≫ not workloads → PMOs track milestones ≫ not consumption That’s why we use tools like: ⓘ AWS Cost Explorer to track EC2, S3, and Lambda usage ⓘ Azure Cost Management for daily anomaly alerts ⓘ GCP Billing for service-level granularity ⓘ CloudZero, Ternary, and nOps to push unit cost per job or user One UAE fintech cut idle compute by 37% in Q2 by tagging early, automating shutdowns, and publishing per-team cost scorecards. Cloud isn’t expensive. Lack of ownership is. الرؤية تسبق الوفورات. Savings follow visibility.

Controlling Cloud Costs: A Strategic Imperative The benefits of moving to the cloud are well-documented—agility, scalability, and the ability to deliver solutions rapidly. These are key drivers of modernization for many organizations. However, the financial realities can be surprising if not actively managed. Cloud adoption often begins organically and can quickly become a significant expense if left unchecked. Managing these costs is no small task, but it is critical to address them early and effectively. Here are some strategies to consider: 1️⃣ Establish a FinOps Practice: Tagging and monitoring expenses ensures visibility. Regularly audit your resources to identify and shut down unused services that contribute to unnecessary spending. 2️⃣ Leverage Reserved Instances and Savings Plans: To optimize your costs, understand the differences and benefits of these offerings compared to on-demand pricing. 3️⃣ Reevaluate Workloads: Overprovisioning or failing to reassess workloads post-deployment can lead to inefficiencies. Regular evaluations and adopting hybrid or cloud-agnostic architectures can yield substantial savings. 4️⃣ Engage Cross-Functional Teams: Collaboration between finance, procurement, and engineering is crucial. A shared understanding of cloud cost dynamics fosters better decision-making. With intentional strategies, organizations can regain control over cloud spending and achieve cost optimization without compromising innovation. How is your organization managing cloud costs? Let’s exchange ideas and best practices to navigate this ever-evolving landscape.

If your cloud bill feels overwhelming, you’re not alone. With its mix of data visualizations, summaries, and trends, deciphering your bill can be challenging. However, gaining clarity is key to managing cloud costs effectively—especially as expenses rise due to the high storage demands and processing power needed to support AI and GenAI technologies.    I recently shared some tips with Morning Brew's Billy Hurley around some of the common cloud billing challenges (https://deloi.tt/4esEO0z). In fact, taking a closer look at your bill can help pinpoint major cost drivers, such as high transfer fees or over-provisioned resources. Also using tools to monitor and analyze trends in computing, storage, and data transfer can help guide informed decision-making on resource allocation.    For example, developers might inadvertently run expensive prompts in loops or leave GPU-intensive workflows active longer than necessary. Implementing usage quotas and automated alerts can mitigate these issues. Additionally, matching storage tiers to specific workloads—reserving premium tiers for mission-critical tasks while opting for basic tiers for less demanding needs—can lead to substantial savings.    If you’re interested in optimizing your cloud resources or managing cloud costs, please reach out. We can help you make the most of your hybrid cloud investment! 

If you’re in cloud and not looking at optimization end-to-end, you’re missing out — here are the key strategies you should know.. → Compute ↳ Right-size instances, use auto-scaling/serverless, and leverage spot/preemptible VMs ↳ Consolidate workloads with Kubernetes/Fargate/Cloud Run → Storage ↳ Use lifecycle policies to move infrequently used data to cheaper tiers ↳ Deduplication, compression, and smart replication strategies reduce costs → Networking ↳ CDN for static content, private networking to cut egress, and traffic shaping with load balancers ↳ Always optimize data transfer (avoid unnecessary cross-region costs) → Databases ↳ Use managed services, read replicas, and caching ↳ Shard/partition for scale, and pick the right DB for the workload → Big Data ↳ Spot clusters for jobs, serverless analytics, and data partitioning ↳ Stream only what’s critical, batch the rest → Security ↳ Enforce least privilege IAM, encrypt in transit/at rest ↳ Automate threat detection and centralize secrets with KMS/Vault → AI/ML ↳ Track experiments, use AutoML/pre-trained APIs ↳ Share GPUs, and clean/optimize data before training Essential Note: Cloud optimization isn’t a one-time exercise. You have to keep at it — especially now, with AI workloads driving cloud costs to new highs. Start with one area → measure impact → repeat. What other strategies would you add? • • • If you found this useful.. 🔔 Follow me (Vishakha) for more Cloud & DevOps insights ♻️ Share so others can learn as well!