4 ms·

Basic metrics around platform API calls would have done the trick, no need for fancy container solutions. edit: I would have also assumed they would get this f

Basic metrics around platform API calls would have done the trick, no need for fancy container solutions. edit: I would have also assumed they would get this for free from GCP's billing breakdown, but I'm not familiar with it. My first intuition when facing unexpected billing would be to figure out what the major contributor to the bill is (in this case, massive reads from FireStore), not update my frontend packages.

dkoston 8y ago
Exactly. Not having proper tooling meant they didn’t know what to do. When your check engine light comes on, you don’t replace every system in your car, you get out a scanner and check the code. Google cloud has trace built in which could have shown them execution times and is dead simple to drop into most frameworks. The real story here is that hey didn’t have engineering leadership on the team who knew how to properly diagnose issues, put tooling in place before launch, and understand how their system is architected. Kudos to the engineers for solving this issue under pressure.
- chrisweekly 8y ago
  >"When your check engine light comes on, you don’t replace every system in your car, you get out a scanner and check the code." ^ Best response, hands down.
themarkn 8y ago
> My first intuition when facing unexpected billing would be to figure out what the major contributor to the bill is (in this case, massive reads from FireStore), not update my frontend packages. They didn’t upgrade packages to solve the mystery billing. They upgraded packages before they checked what was going on with the database. When they saw the high billing that pointed them to the problem and they fixed it. There was some questionable judgement shown by not checking db requests first, sure, but in no way did someone think “our google billing is high, we better upgrade angular”.

dkoston 8y ago

Exactly. Not having proper tooling meant they didn’t know what to do. When your check engine light comes on, you don’t replace every system in your car, you get out a scanner and check the code. Google cloud has trace built in which could have shown them execution times and is dead simple to drop into most frameworks. The real story here is that hey didn’t have engineering leadership on the team who knew how to properly diagnose issues, put tooling in place before launch, and understand how their system is architected. Kudos to the engineers for solving this issue under pressure.

chrisweekly 8y ago
>"When your check engine light comes on, you don’t replace every system in your car, you get out a scanner and check the code." ^ Best response, hands down.

themarkn 8y ago

> My first intuition when facing unexpected billing would be to figure out what the major contributor to the bill is (in this case, massive reads from FireStore), not update my frontend packages. They didn’t upgrade packages to solve the mystery billing. They upgraded packages before they checked what was going on with the database. When they saw the high billing that pointed them to the problem and they fixed it. There was some questionable judgement shown by not checking db requests first, sure, but in no way did someone think “our google billing is high, we better upgrade angular”.