3 ms·
$82K GCP bill in 48 hours – so I built an automatic API key kill switch
- daudmalik06 7mo agoOriginal incident: https://www.reddit.com/r/googlecloud/s/3S1KWpWRZm https://www.reddit.com/r/googlecloud/s/3S1KWpWRZm After reading about a 3-person startup that received an $82,000 Gemini API bill in 48 hours (normal monthly spend: $180), I started building CloudSentinel. The core problem: GCP has no native kill switch. Budget alerts send an email. Quota limits throttle requests. Neither revokes a key automatically. And billing data is delayed by hours — useless for real-time protection. The architecture: CloudSentinel monitors raw API request count — updated in near real-time. We create an Alerting Policy inside the user's own GCP project using MQL. When the request threshold is crossed, Google fires a Pub/Sub webhook to CloudSentinel. We receive it and revoke that exact key automatically. The security decision I'm most proud of: Revoke-Only IAM model. The Custom IAM Role has three responsibilities: - Read API key IDs and metadata (never key values) - Create monitoring rules inside the user's project - Revoke a specific key when a threshold is crossed The permission apikeys.create is not in the role. Not restricted — absent. Even if CloudSentinel is fully compromised, an attacker can only remove access, never create keys or touch anything else. Early access open at https://cloudsentinel.dev https://cloudsentinel.dev Have you ever dealt with a GCP billing surprise or a leaked key? Happy to hear your experience and discuss the architecture.