Joseph Bueno

As LLMs continue to show up across the federal space, one thing matters more than model size or hype: control. AWS quietly made an important move by adding response streaming support to Amazon API Gateway, and it directly addresses a challenge many of us hit when deploying AI in the federal arena. Streaming responses mean APIs no longer have to wait for a full LLM output before responding. Users can see results as they are generated, which improves perceived performance and reduces impatience waiting for a response. With streaming, teams gain: - More control over how quickly responses are delivered, including intentionally slower or paced outputs when overuse is detected. - Better user experience without bypassing security, logging, or gateway controls that come from workarounds. - Support teams that want to stay serveless and keep their architecture simple. The ability to shape how an AI responds is just as important as what it responds with. Serveless infrastructure is finally catching up to how LLMs are actually being used in production. https://lnkd.in/gxe_tkrp

9

4 Comments

Excited to see real-world proof of ARM’s impact on streaming. This collaboration between Amazon Web Services (AWS), Playtech, and the Wowza team makes this a must-watch for anyone focused on streaming performance and cost.

18

1 Comment

AWS just announced Graviton4 support for OpenSearch. My honest take? Stay where you are. I know that sounds counterintuitive. For years, upgrading to the new generation was the obvious FinOps move. When c5 came out, it was about 20% cheaper than c4, with improved performance. New generation meant automatic savings. That was the deal. That deal is gone. Here's what Graviton4 actually costs you on OpenSearch today vs Graviton3: m7g.xlarge → m8g.xlarge: $0.271 → $0.299 (+10%) r7g.xlarge → r8g.xlarge: $0.356 → $0.392 (+10%) c7g.xlarge → c8g.xlarge: $0.241 → $0.265 (+10%) AWS claims up to 30% better performance. Maybe true. But here's the question worth asking: do you actually need 30% more performance from your OpenSearch cluster right now for a 10% increase in costs? If the answer is no, skipping this upgrade is a double win. You keep paying the lower price, and you skip the migration work, testing cycles, and risk that comes with any infrastructure change. Cloud vendors used to compete on price between generations. Now they compete on performance - and they're charging you for it. That's a fundamental shift in how we should think about upgrades. The new default shouldn't be "upgrade when available." It should be "upgrade when your workload justifies it." Staying on Graviton3 isn't falling behind. It's just good FinOps. #FinOps #AWS #OpenSearch #CloudCostOptimization #Graviton https://lnkd.in/ezBqXJBt

28

Today I spoke with two frinds from Google AI and learned that the KPI for engineers at Google and many large tech companies has fundamentally changed. A key metric now is how many tokens you are able to use. The more tokens you use, the greater your potential contribution—one person can now accomplish the work that once required ten or even a hundred people. #HerPowerAI #Google #WorkModel #KPI

1

Yesterday, #Amazon shared the Q1-2025 results, with revenues of $155.7B (+9% YoY), #AWS growth at 17% YoY to $29.3B revenues. Further highlights below: 1- Began full-scale deployment of Project Kuiper’s satellite internet network with launch and activation of first satellites. 2- Introduced Alexa+, our next-generation Alexa personal assistant who’s meaningfully smarter and more capable than her prior self, can both answer virtually any question and take actions, and is free with Prime. 3- Announced Ocelot, Amazon’s first quantum computing chip prototype that reduces resources needed for quantum error correction by up to 90% 4- Ranked No. 2 on LinkedIn’s U.S. Top Companies list of the best workplaces for career growth. 5- New Nova capabilities, such as Nova Sonic (Speech-to-speech), Nova Act SDK (AI model trained to perform actions within a web browser), Nova Premier (Most capable of Amazon's multimodel models for complex tasks). 6- General availability of Amazon Nova models in Europe and Asia Pacific. https://lnkd.in/gfry8Xmt

43