4 ms·
As a counterpoint, I interned at Jane Street about a decade ago—when they were much smaller than now—and they also built everything in-house. I'm not just talki
by tikhonj 3y ago
As a counterpoint, I interned at Jane Street about a decade ago—when they were much smaller than now—and they also built everything in-house. I'm not just talking about the trading/finance-specific systems (I didn't interact with those as an intern), but general-purpose internal tools: a code review system, a pub-sub system, custom binary and text protocols, lots of developer tooling, their own build system, their own standard library and async runtime... All with <100 developers total.
And you know what? It worked amazingly well. I'm not just talking about scale, but also about how much any single developer seemed to be able to do. Their tools and APIs were tastefully designed, tailored to their own needs and fit together remarkably well. I remember that they were consistently building things with just one or a handful of people that would have taken multiple teams at other organizations I've seen.
Looking back, that was a remarkably productive engineering environment because of how much they were willing to do themselves. The key insight is that building something for yourself is qualitatively different from building something for external consumption or using something general-purpose that already exists. You naturally tailor the code and tools to what you need, which is much easier than building something for what you think other people might need—and, surprisingly often, easier than bending a different tool to your own ends.
But, perhaps more importantly, you develop a shared mental model of the system you're developing and how it fits in with everything else that you are doing. That shared mental model is, ultimately, worth far more than the code by itself... but it isn't legible to management, so it's hard to foster and preserve without a strong engineering culture backing it up.
The other lesson I took away from that experience (and others!) is that the high-level questions people focus on ("buy vs build", "monorepo vs polyrepo"... etc) are nowhere near determinative. Whether or not any given approach works out comes down far more to the little details and nuances of what you're doing and how you're doing it, as well as a healthy dollop of circumstances beyond your immediate control.
At the end of the day, if everything else was the same, but the startup you're talking about was using perfectly standard and popular tools, would you expect the outcome to be any different? I've seen far too many Enterprise-grade trash-fires to believe that. If the core dynamics that led to a mess don't change, you can have a "not invented here" mess or an "Enterprise Best Practices" mess or a "buy everything, build nothing" mess—but there's no simple strategic direction that won't get you some kind of mess.
- walterbell 3y agoWas Jane Street searching for product-market fit? > shared mental model of the system you're developing and how it fits in with everything else that you are doing. That shared mental model is, ultimately, worth far more than the code by itself... but it isn't legible to management, so it's hard to foster and preserve without a really strong engineering culture backing it up. Sounds awesome, did they have engineering leaders with long tenure?
- andrewjl 3y ago> Was Jane Street searching for product-market fit? In a manner of speaking, yes.
- zrail 3y agoJane Street started as and remains a proprietary trading firm. They added consulting services as some point as well, but that was after they proved their competence.
- EdwardDiego 3y agoI infer long term engineering leaders, just from how coherent their dev culture seems to be. They're big on FP and OCaml and do a lot of open source work around OCaml, and have done for years. Speaks to consistent leadership.
- benjaminwootton 3y agoGoldman Sachs are famously the same. Propriety language (Slang), proprietary object database (SecDB) and then a million and one proprietary apps on top of that where a commercial or open source alternative exists. It may have moved on since I worked there, but it seemed to be worth the investment in the early 2000s when I worked there.
- tikhonj 3y agoYeah. One of my friends and coworkers started his career at GS around that same time and had some (very!) positive stories about both the tools and the people building them. It sounded like the combination of Slang and SecDB was a real force multiplier for building and evaluating the kind of complex mathematical models he was working on. That's the kind of tool development that shows off just how high-leverage engineering work can be. My (100% outsider's) impression is that the focus, culture and teams at Goldman Sachs are totally different now thanks to market and regulatory changes, and that it is not a good place to work in a primarily technical sort of role any more.