lawrjone

9 ms·

91.
by lawrjone 3y ago
I work at https://incident.io/ where we build a product that aims to help with this. I'll callout some of the points you make that we can help with: 1. Our incidents are created automatically from triggers like PagerDu
92.
by lawrjone 3y ago
Author here, thanks for the share! Thought I'd write this as I think the "how easily can someone copy-and-paste?" is a fairly unknown principle by which people might evaluate their code, but one that I think can be extremely
93.
by lawrjone 3y ago
Feels a bit like the Amazon leadership principle "be right, a lot". If you have a manager who has been at the company a while and moved through several teams and all those teams felt like this: it's far more likely to be a re
94.
by lawrjone 3y ago
I think it's a really hard balance to strike. There's so much company politics/etc that isn't beneficial for a team to experience, but equally, without being exposed to that noise you won't develop an intuition that
95.
by lawrjone 3y ago
Respectfully, this is a really bad take. And one that flies in the face of the Go stdlib, given the default HTTP server will catch panics for you by default.
96.
by lawrjone 3y ago
No problem, easy to skim by it! In terms of how this reduces artifact size: Go statically compiles everything, and the majority of that weight is from dependencies. Compiling into three different binaries would: 1. Increase build time, beca
97.
by lawrjone 3y ago
It’s a Go app, not a Ruby one. Hence the Go code examples. And the rationale behind one binary rather than several is to reduce build time and deployment artefact size, along with benefits in development from one binary. All of which mean d
98.
by lawrjone 3y ago
You’ll find some libraries do this for you, such as HTTP servers. They do this because if your server code makes a mistake such as accessing a nil pointer, a segfault or panic would bring down the entire process. That’s why you want to reco
99.
by lawrjone 3y ago
The post-mortem we published about this outage explains the details of how this crashed the binary. It gives some code examples and explanations that should clear this up: https://incident.io/blog/intermittent-downtime#
100.
by lawrjone 3y ago
Yep, it's a very normal thing in the Rails world to separate things like this. For Django too. I'd encourage people to look a past just the splitting of workloads though. The other point about resource limits for shared resources
101.
by lawrjone 3y ago
Yep: this is definitely a valid alternative. The reasons we went with one binary are: - We can ensure common runtime setup by only having one codepath that runs the initialisation. It's nice that every deployment definitely instantiate
102.
by lawrjone 3y ago
Have you worked with Go codebases before? Standard practice is to wrap all your entrypoints - the start of a web request, the moment you begin to run a job - in a defer recover() which will catch panics. Sadly, recover won’t apply to any su
103.
by lawrjone 3y ago
Small note that the app in the post was not Rails, it’s a Golang app. You’re right in that PHP couldn’t fail like this though, as the isolation level is at a unix process, where the OS should contain all your errors.
104.
by lawrjone 3y ago
You’re right, that’s a very fair point. I should’ve made it clear that the middle ground of moderately-sized but logically separate services is very much a good world to be in. I’d actively encourage splitting a monolith into services when
105.
by lawrjone 3y ago
Glad you enjoyed the article! The experience I reference was from my time at GoCardless, where I was a Principal SRE leading the team who were on-call for the big Rails monolith. I’ll put the topic of “what does 10x eng team scaling feel li
106.
by lawrjone 3y ago
Yep, that article is about very similar concepts but grounded in Spring as the framework. I like what they do around package imports and it looks a lot like what we do at incident.io, with some rules about which packages can import what. Fo
107.
by lawrjone 3y ago
It’s ‘standard practice’ for many frameworks like Rails which encourage the pattern via a separate deployment of the async worker tier. But outside of these frameworks, you won’t often see it. And especially in languages like Go which tend
108.
by lawrjone 3y ago
Author here, thanks for sharing! This is a strategy I’ve used across many projects in several languages, from big Rails apps to the Go monolith we deploy at incident.io today. It’s just one way you can make a monolith much more robust witho
109.
by lawrjone 4y ago
On the team who built this. Always thought building a status page would be a fun technical challenge, and it was! You have to think about spiky customer traffic and how to provide a really great experience on first load. For us that meant S
110.
by lawrjone 4y ago
Author here! After seeing a lot of comments about the recent Slack outages, thought I'd write-up my thoughts about why status pages so often become a battleground for transparency, based on my experience working at companies that went
111.
Uptime, status pages, and transparency calculus(blog.lawrencejones.dev)
3 points by lawrjone 4y ago|1 comments
112.
by lawrjone 4y ago
Did I read this wrong or were the layoffs exclusively for GTM and G&A? So no R&D, which is who would know the intimate details?
113.
by lawrjone 4y ago
Wrote about my experience using ulids in Postgres if people are considering it: https://blog.lawrencejones.dev/ulid/
114.
by lawrjone 4y ago
Funny this keeps coming up! I wrote about my experience using ulids the other day, specifically with Postgres and some of the dis/advantages you get with it. It's a deeper dive into ulids than this article is, and shows some real
115.
by lawrjone 4y ago
Inspired by Mike Coutermarsh's post on Using NanoIDs at PlanetScale ( https://news.ycombinator.com/item?id=34172989 ), I've written up some of the trade-offs we've experienced using ULIDs at incident.io. It&#x2
116.
Using ULIDs at Incident.io(blog.lawrencejones.dev)
4 points by lawrjone 4y ago|1 comments
117.
by lawrjone 4y ago
Author here! This is a personal reflection on the last year, but shares a view into how incident.io has grown and my experience as an early engineer there. Figured it might be interesting to people in other start-ups, or as an insight into
118.
Looking back at 2022 as my first year at incident.io(blog.lawrencejones.dev)
2 points by lawrjone 4y ago|1 comments
119.
by lawrjone 4y ago
I think that depends on signal to noise ratio. If your snapshots change a lot normally and it's irrelevant to the code you're trying to alter, you'll get used to skipping by them. But if you use the snapshots as a means of un
120.
by lawrjone 4y ago
Author here! This is a short post about how to apply tests to problems outside the space of normal code. It covers two strategies – snapshot testing and verification of real production systems – using examples taken from my career, where in

More ›