3 ms·
> Reason through your system modern stacks have a huge opacity problem, everyone wants to be magic, and everyone fails. abstractions make reasoning harder, wha
by sitkack 8y ago
> Reason through your system
modern stacks have a huge opacity problem, everyone wants to be magic, and everyone fails. abstractions make reasoning harder, what tools and techniques would you suggest for doing this?
I'd probably run the application in some sort of sandbox and measure the outbound request load vs inbound request load, something a containerized deployment should be giving the end user (developer) as an affordance for application maintenance and visibility. Differential analysis and graphing built directly into the execution substrate.
Judge away!
- cloakandswagger 8y agoBasic metrics around platform API calls would have done the trick, no need for fancy container solutions. edit: I would have also assumed they would get this for free from GCP's billing breakdown, but I'm not familiar with it. My first intuition when facing unexpected billing would be to figure out what the major contributor to the bill is (in this case, massive reads from FireStore), not update my frontend packages.
- dkoston 8y agoExactly. Not having proper tooling meant they didn’t know what to do. When your check engine light comes on, you don’t replace every system in your car, you get out a scanner and check the code. Google cloud has trace built in which could have shown them execution times and is dead simple to drop into most frameworks. The real story here is that hey didn’t have engineering leadership on the team who knew how to properly diagnose issues, put tooling in place before launch, and understand how their system is architected. Kudos to the engineers for solving this issue under pressure.
- chrisweekly 8y ago>"When your check engine light comes on, you don’t replace every system in your car, you get out a scanner and check the code." ^ Best response, hands down.
- themarkn 8y ago> My first intuition when facing unexpected billing would be to figure out what the major contributor to the bill is (in this case, massive reads from FireStore), not update my frontend packages. They didn’t upgrade packages to solve the mystery billing. They upgraded packages before they checked what was going on with the database. When they saw the high billing that pointed them to the problem and they fixed it. There was some questionable judgement shown by not checking db requests first, sure, but in no way did someone think “our google billing is high, we better upgrade angular”.
- deleted 8y ago[deleted]
- TeMPOraL 8y ago> modern stacks have a huge opacity problem, everyone wants to be magic, and everyone fails. Also, lots of people are impatient and/or intellectually lazy. We have piled up a ton of abstraction layers, yes, but they aren't hard to pry apart. But people want immediate results without doing necessary cognitive work - understanding-guided exploration. It usually isn't hard to identify which component of your product is misbehaving. Before getting into complex magic of containers and sandboxes and stuff, I'd start with the easiest - looking at the Network tab, at your server's resource use, reading the logs, adding some log statements measuring times in suspected areas, actually profiling the backend code (e.g. with a statistical profiler). This should quickly help you identify where the problem is manifesting itself. Then the search for the cause begins.
- cm2187 8y agoThere is also a common sense problem. Not trying to play the blame game but the guy who ran a subtotal displayed on the front page by having to download every row in the database probably isn’t a great programmer. It’s easy to say don’t write bugs, but there are bugs and moronic design decisions. This to me doesn’t feel like a real bug.
- dmurray 8y agoThere's a good chance this is true, but this can happen even with competent developers. One person writes the unoptimized function figuring it can be improved later, someone else assumes this must be cheap and drops it on the front page. Or there was meant to be a caching layer, that didn't work because of some bug.
- voltagex_ 8y ago>Also, lots of people are impatient and/or intellectually lazy. We have piled up a ton of abstraction layers Sorry, the guy that wrote/implemented abstraction layers 2 and 3 left 2 years ago and didn't document anything. We've been understaffed for a year and we've been told not to hire any more staff until the new financial year. I've got a "technical debt" item on the backlog but business drives the priorities and it'll never get done.
- gitgud 8y agoWell a browser is basically a sandbox for a webapp. To debug something complex, I would use chrome devtools, which can measure all kinds of metrics, and the function "Audit > LightHouse" automates the process and ranks a webapp in several key categories. This case would appear to be related to network requests, so that issue should be fairly obvious in LightHouse.