Posts about reducing cloud costs and optimization strategies
सारे टैग देखें
Credits hide the bill. The day they end, the same usage shows up as cash. The useful move is to split that bill into a baseline you can commit to, and a variable line you refuse to commit to, then buy the commitment only after the baseline is real.
g6 was not there when the session started. Not on Spot, and not as a normal On-Demand launch either. Karpenter would sit on InsufficientInstanceCapacity while a user was already waiting. I used an On-Demand Capacity Reservation to hold the instance before business hours, then release it when the day was over.
The point was the GPU existing at the right moment. The smaller bill was a side effect.
Scale-to-zero is cheap until the first request waits on an image pull. I benchmarked NVMe instance storage, Bottlerocket, SOCI, AL2023 FastImagePull, and EBS throughput on a fresh node every time.
Setup: g4dn.xlarge, a 3.6 GB image, one run per configuration. Treat the numbers as indicative, not a paper.
The first Langfuse design on EKS was full HA and about $3,400 a month at the load I expected. Scenarios ran from about $1.8K to $7.7K. I re-checked it before buying the hardware. The estimate was about 3.5× too high. The tier that shipped is about $1,000 a month, roughly 71% below the original number.
Two embedding models, one physical GPU. That was the May design. By August each pod had its own L40S. Time-slicing had added sharing and no throughput.
रात भर idle पड़ा GPU node "available capacity" नहीं है। वह एक line item है। Intermittent inference के लिए g6e या g7e 24/7 चलाओ तो पहला token जाने से पहले ही लगभग $10k/माह जल सकता है। मैंने तीन महीने में AWS EKS पर एक event-driven MLOps platform बनाया जो GPU replicas को zero तक scale करता है, mission-critical sessions को Spot से दूर रखता है, और 15 GB+ LLM images पर sub-60-second cold starts का लक्ष्य रखता है।
यह GPU path है: architecture, hardware constraints, और scars।
मैंने एक पूरी तिमाही क्लाउड कॉस्ट को समझने और कम करने में लगाई — बिना reliability या developer velocity को नुकसान पहुँचाए। अंत तक नीचे दिए line items मिलाकर लगभग $1,800 USD प्रति माह — यानी सालाना $20k से ज़्यादा — AWS और GCP बिल से कट गए। Spot और GPU Reserved Instance discounts इसके ऊपर हैं, और उन्हें एक ही संख्या में बाँधना मुश्किल है।
यहाँ वो 8 चीज़ें हैं जिनसे असल में फ़र्क पड़ा।