dangoor

40 ms·

391.
by dangoor 7y ago
Does sound similar! We have established the rule that each service owns its own data. We've already got Stackdriver set up to give us distributed tracing and have set up standards around logging.
392.
by dangoor 7y ago
It looks like I need to write a followup with more details about how our migration looks! Since we're using GraphQL federation, we have a gateway which serves up our complete GraphQL schema, pulled together from a collection of service
393.
by dangoor 7y ago
Sorry if this wasn't clear: Go is 10x faster than Python, yes, but we know that we're not going to reduce our server count by 90%. 50% is quite possible, though, given Go's superior threading and its good resource use. Moving
394.
by dangoor 7y ago
> Of course the obvious thing missing in the article is how they expect to deliver new business features while recoding everything in a new language. For new features, the new parts of the GraphQL schema for those features will be writte
395.
by dangoor 7y ago
2020 is absolutely going to be an interesting year. One thing that might not have come across clearly in the blog post: we're already well on the path to using React everywhere (we started using React a week after it became public… 6 y
396.
by dangoor 7y ago
FWIW, our codebase is of similar size to that and we estimated it would take around a year to migrate, which is why spending a bit more time and ending up with a Go-based system on the other end was appealing.
397.
by dangoor 7y ago
We did do a bunch of refactoring a couple of years ago to draw boundaries within our monolith. We almost certainly wouldn't have made this choice had we not done that work. We're also doing work in Python to smooth the way for thi
398.
by dangoor 7y ago
We'll see how it plays out, but the situation that I've described isn't really different from the one we have today (because we have a monolith). Hopefully, it will be better because of how the Go project is trying to get lib
399.
by dangoor 7y ago
Given that this is mostly about our backend systems, we've got one big interface in front: our GraphQL schema. Our system isn't perfect, but a couple of years ago we spent a good deal of effort detangling our monolith. This plan w
400.
by dangoor 7y ago
What I meant by "the whole org is on board" isn't that there weren't other opinions. There have been multiple opinions on almost every decision we make (and we have an open process and document our decisions in a style v
401.
by dangoor 7y ago
That does sound pretty similar (though our services all have the luxury of serving nothing other than GraphQL!). Our current plan for local development is to continue cloning the monorepo and firing up all of the services. Go services don&#
402.
by dangoor 7y ago
We have some dataflow jobs written in Kotlin (we blogged about that in June 2018[1]). We also have an internal service written in Kotlin. We ideally want one language. But Apache Beam (which is behind Google Dataflow) doesn't yet hav
403.
by dangoor 7y ago
Indeed, that's true. I just think we have more to gain by talking about what we're doing and seeing what the community thinks about some of our approaches rather than keeping it all private for the next year.
404.
by dangoor 7y ago
Yes. Google Cloud now has multiple options for autoscaling servers (App Engine Standard, App Engine Flex, and Cloud Run) with the biggest differences being how they're deployed and specifics around the scaling. We _could_ manage our ow
405.
by dangoor 7y ago
I appreciate all of the links and the fact that, for a lot of folks, this is the state of Python 2 to 3 migration today. What you're suggesting is, imho, the best path for most (and I appreciate you mentioning the pickle incompatibilit
406.
by dangoor 7y ago
We essentially run in a serverless environment (App Engine), so fast startup does matter to avoid some unlucky users hitting the cold start.
407.
by dangoor 7y ago
I don't have recent numbers in front of me, but I believe our web servers are more like 40% of our hosting costs today. Over the past year, we've started leveraging our CDN (Fastly, who have been great) a lot more. That said, for
408.
by dangoor 7y ago
We're making a certain set of tradeoffs. For example, we're not adopting the "write code in whatever language you want" form of microservices that some folks adopt, because we don't feel like we're large enough
409.
by dangoor 7y ago
We have the good fortune of not having to be super secretive about our tech. We _want_ to talk about this project as we go along and share what we learn. We've already got more interesting stuff to talk about, and it'd be a lot le
410.
by dangoor 7y ago
I don't disagree with your pitfalls, but I do think we're working to avoid them. To your first point, "The decision seems to be primarily a software architecture one...", this project has had involvement of the whole eng
411.
by dangoor 7y ago
Our approach to services at Khan Academy is likely a bit different from most. We're sticking with a monorepo (the code for all services lives in one repository). We have a single go.mod file at the top of the repo, so all services use
412.
by dangoor 7y ago
You might be thinking of this excellent article by Joel Spolsky: https://www.joelonsoftware.com/2000/04/06/things-you-should-... (Ironically, that post is about Netscape and the rewrite, Mozilla, ended up gro
413.
by dangoor 7y ago
> I've worked with or known about too many places that have jumped on the microservices bandwagon and the only thing I've encountered is a lack of maturity (mine included) and a knee-jerk reaction against monolithic code, when
414.
by dangoor 7y ago
The Python 2 to 3 conversion was only part of it. The libraries were a huge difference for us. I will agree that porting to Python 3 would be less work than porting to Go, but we believe that the difference is less than people would suspect
415.
by dangoor 7y ago
FWIW, our tests showed Go as realistically being about 10x faster than Python for our tasks.
416.
by dangoor 7y ago
I _like_ Python. Even released a Python web framework. I think there are many projects it's a good fit for, which is why it's continuing to be quite successful, despite the pain of Python 3. But that doesn't mean it's a
417.
by dangoor 7y ago
> That said, yeah it's difficult to understand how KA's web server costs aren't already basically zero, and how their endpoints aren't already basically instantaneous. I find that this is the outsider's view of a
418.
by dangoor 7y ago
I have a lot of JavaScript history, but if I were optimizing for the things you want to optimize for, I would absolutely pick Kotlin above JS+TypeScript. It's got bunches of language features, including superior concurrency, and runs o
419.
by dangoor 7y ago
I've asked them about this. Their current thinking is something like $10/month service.
420.
Go and Services = One Goliath Project(engineering.khanacademy.org)
5 points by dangoor 7y ago|0 comments

More ›