6 ms·
Tour of our 250k line Clojure codebase (2021)
- dang 3y agoDiscussed at the time: Tour of our 250k line Clojure codebase - https://news.ycombinator.com/item?id=27385208 https://news.ycombinator.com/item?id=27385208 - June 2021 (227 comments)
- valcron1000 3y agoSounds like a great way to burn money: the company was founded in 2019 and they still haven't released anything, probably because they've spent most of their time building other stuff. Their devs sure are having a lot of fun though.
- _halgari 3y agoThat was my issue for a long time. I even talked with their founder several times on twitter back a few years ago. Each time I was greeted with buzzwords, that I knew the meaning of but I think they assumed I didn't. They would claim grand things like having solved the issues with continuations and delimited continuations, distributed process migration and a whole host of other very hard problems that haven't been solved in the past. I would ask their founder: "right, so you know that delimited continuations have problems with accidental captured scope, they run poorly on the JVM, how did you solve this, have any papers I can read?", and all I ever got was that Clojure, immutable data, X and Y would fix these issues and you just had to wait and see what they were cooking up. That's when I knew they had no clue what they were doing. I'm all for pushing the boundaries of tech, but if you're doing something attempted many times before, you at least need a good elevator pitch as to why it's solved now. I think a great example of this done right is Rich talking about Clojure. People would ask "isn't immutable data expensive to reclaim and allocate". And his reply was always that the JVM's GC was just that good that the benefits to be gained from immutable data outweighed the marginal performance penalty of the amount of garbage collected. What changed since the old lisp days? Well we now have GCs that are super fast and JITs that can optimize dynamic code well. That's the sort of laser focused vision I never saw from Red Planet labs. You gotta get that problem statement and the solution out early, refine the elevator pitch and be able to articulate to people who know what they're talking about how you're going to succeed where others have failed for decades.
- eduction 3y agoI see your overall point which is a good one and I know this is a nitpick, but I thought Rich’s core solution to the cost of immutable data structures was to find a way to get the cost down by extending some existing research by Phil Bagwell. From https://dl.acm.org/doi/pdf/10.1145/3386321 https://dl.acm.org/doi/pdf/10.1145/3386321 “I then set out to find a treelike implementation for hash maps which would be amenable to the path-copying with structural sharing approach for persistence. I found what I wanted in hash array mapped tries (HAMTs) [Bagwell 2001]. I built (in Java) a persistent implementation of HAMTs with branching factor of 32, using Java’s fast System.arrayCopy during path copying. The node arrays are freshly allocated and imperatively manipulated during node construction, and never mutated afterwards. Thus the implementation is not purely functional but the resulting data structures are immutable after construction. I designed and built persistent vectors on similar 32-way branching trees, with the path copying strategy. Performance was excellent, more akin to O(1) than the theoretical bounds of O(logN). This was the breakthrough moment for Clojure. Only after this did I feel like Clojure could be practical, and I moved forward with enthusiasm to release it later that year (2007).” Or you tell me what I’m missing. Big fan of your work in core.async if this is the same halgari.
- _halgari 3y agoSame halgari, yes :D. Yeah he took it from Bagwell, and adapted it, but in general there was a whole discussion way back in the day (~2012) questioning how creating this much garbage by boxing and throw away collections could ever be fast. Datomic is another example: making an immutable DB is a dumb idea right? Well what if storage was super cheap, and almost free? Well then maybe it's not such a bad idea. So a lot of the Clojure community is based on this idea of taking ideas from way back in the 70's and saying "Well everything has changed, what works now that didn't then"
- eduction 3y agoThat’s super interesting and makes sense - even with persistence of trunks and branches there will be leaves to throw away / GC. Thanks for explaining!
- jasonjackson 3y agoThe founder has released successful large projects in the past, e.g. Apache Storm used by Alibaba, Yahoo, Twitter, etc. Probably just need to wait a bit longer.
- adrianmsmith 3y ago> the company was founded in 2019 and they still haven't released anything According to this tweet the author has been working on the product/company for ten years! https://twitter.com/nathanmarz/status/1636050500553699330 https://twitter.com/nathanmarz/status/1636050500553699330 (And presumably still hasn't released anything in those 10 years?)
- chii 3y agoThe thing i want to know is how is this company funded? What puts food on the table? Or is it just living off personal wealth? Or is it investor funded, in which case, how come the investors are not clamering for a return on investment? Edit: just found https://medium.com/red-planet-labs/introducing-red-planet-labs-2a0304a67312 https://medium.com/red-planet-labs/introducing-red-planet-la... , so i guess they're using investor money.
- nathanmarz 3y agoWe're funded by top investors including Initialized, Naval Ravikant, and Max Levchin. They've seen firsthand what we're building and understand both how hard it is and how valuable it will be.
- deleted 3y ago[deleted]
- bokchoi 3y agoNow up to 345k: https://twitter.com/nathanmarz/status/1616577265139355655 https://twitter.com/nathanmarz/status/1616577265139355655?
- bschwindHN 3y agoGotta be honest, this sounds like a nightmare. Clojure felt like it got out of hand in my own _personal_ projects, I can't imagine going through 250k lines of it without a strong type system backing it up. And they're building a "programming tool". Hmmmm > We have about 400 macros defined through our codebase, 70% of which are part of source code and 30% of which are for test code only.
- jasonjackson 3y agoYou can write 'type signatures' easily in clojure code, there's dozens of options, they're just only checked at runtime (and usually relegated to developer builds). if you combine that with good code coverage, it stops being a weakness and starts being a strength, because you declare data types with far more specificity & precision than a static type system. As rich likes to say, the type signature of your stereotypical statically typed language, e.g. reverse :: [a] -> [a], doesn't have a much information content. I write clojure, and still 'think in types' & category theory.
- MrBuddyCasino 3y agoif I type ctrl+space, will it give me the correct auto-completion? despite what type-system galaxy brains might think, this is 95% of the value
- grumpyprole 3y ago> the type signature of your stereotypical statically typed language, e.g. reverse :: [a] -> [a], doesn't have a much information content. It's a little bit unfair to imply any alternative to bleeding-edge clojure is going to have a simple 70's Damas-Milner type system. Today, static types can describe a lot more. Refinement types (e.g liquid Haskell) and/or dependant types (e.g. Idris or Agda) can describe and enforce the desired properties for reverse statically.
- tome 3y agoI think you're downplaying the strength of types! In any relationally parametric type system `reverse :: [a] -> [a]` has a huge amount of content.
- riwsky 3y ago“Measuring programming progress by lines of code is like measuring aircraft building progress by weight“—Bill Gates
- 9659 3y agowhich is really a good way to do it. you design it to be so many pounds, when you get it all together you should be there (or close to it).
- conductr 3y agoYou start a project knowing how many LOC it will take? I want to assume you’re being sarcastic but it really doesn’t sound like it
- drekipus 3y agoIt's responding to the analogy.
- 9659 3y agonot knowing exactly, but with an estimate of the software size. sure. how else do you reliably estimate the projects cost and schedule. "engineering" not "programming"
- heyoni 3y agoDifficulty of the problem you’re trying to solve. Just like it could take 5 hours to come up with 5 lines of code, it could also take 5 minutes to produce another 5 lines. All depends on how complicated the task is.
- 9659 3y agoyep. and when you need to write 300K SLOC, the average takes care of the outliers.
- 3y ago
- samsquire 3y agoI feel that every codebase built by one person is understand by that person really well and nobody else gets to the same level of understanding, unless the codebase is embedded, it shall only exist as long as the original person is around.
- baby 3y agoI want to see a whole discussion based on that comment. This is a real and big issue, and the big reason why it’s hard to scale the team.
- samsquire 3y agoI don't believe code _is_ the documentation. I've thought of information systems for code, I once used OpenGrok to parse a large codebase and try work out how it works. But it's undirected learning. I think question answering would be useful: "Where should I change the code to insert behaviour that changes X, Y, Z behaviours"
- chii 3y ago> This is a real and big issue, and the big reason why it’s hard to scale the team. it's also why java as a language is good, not because the language is good, but that it's easy to have someone not originally involved in adding/maintaining the code base - you can't get too clever with java.
- baby 3y agoI would have said the same with Golang, Java though… not sure I understand considering all the crazy patterns that exist there and the huge mess codebases are due to how namespacing work and other reasons.
- kaba0 3y agoSee Naur’s Programming as Theory Building.
- lionkor 3y agoSo, does it do anything? Is it just pretty code that does nothing? I sure dont see it doing anything, and every sentence on the website is incredibly vague. The CEO just claims 100x more features on twitter, and things like that.
- layer8 3y agoHow many K parentheses might be more interesting.