5 ms·

Wheel Reinventor’s Principles (2024)

213 points by TobLoef 2y ago

austin-cheney 2y ago
An often unintended benefit of reinventing wheels is vastly superior performance improvements, most of which are entirely unintentional.
dazzawazza 2y ago
> Embrace the strengths of DYI. Create what you need and little more. Be wary of abstractions made for fabricated use cases. DYI?
- TobLoef 2y ago
  Meant to say "DIY" (Do it yourself). Thanks for letting me know, I've updated it now!
  - dazzawazza 2y ago
    Ah, that's cool. Thanks for writing the article.
amelius 2y ago
But should I tell my boss that I'm reinventing everything?
- chachacharge 2y ago
  colleague of mine wrote a json parser in sql when the rdbms already had a json parser... guess where all the errors came from
  - mdaniel 2y ago
    relevant: Parsing JSON Is a Minefield (2018) - https://news.ycombinator.com/item?id=40555431 https://news.ycombinator.com/item?id=40555431 et al <https://hn.algolia.com/?q=parsing+json+minefield https://hn.algolia.com/?q=parsing+json+minefield>
amelius 2y ago
Should mention NIH syndrome. https://en.wikipedia.org/wiki/Not_invented_here https://en.wikipedia.org/wiki/Not_invented_here
- TobLoef 2y ago
  Yes, it has to be balanced appropriately. I didn't emphasize this a lot in the blog post, but this is really important in a work setting, compared to when working on personal projects. I still think wheel-reinvention has its place in professional teams, but the related challenges should be taken more serious in that case.
JackC 2y ago
I'll add "reduce code size and complexity" to the list of benefits. A python library to calculate a simhash, or track changes on a django model, or auto generate test fixtures, will often be 90% configuration cruft for other usecases, and 10% the code your app actually cares about. Reading the library and extracting and finetuning the core logic makes you responsible for the bugs in the 10%, but no longer affected by bugs in the 90%.
- dkarl 2y ago
  Hard agree. A library should not inflict complex use cases' complexity on simple use cases, but sometimes they do, either because they're poorly designed or because they're overkill for your use case. But often I see pain and complexity excused with "this is the library that everybody else uses." Sometimes a simple bespoke solution minimizes costs compared to the complexity of using a massive hairball with a ton of power that you don't need. One big caveat to this: there's a tendency to underestimate the cost and complexity of a solution that you, personally, developed. If new developers coming onto the project disagree, they're probably right.
  - jofer 2y ago
    The big caveat is a big one. Choose your battles wisely! There are plenty of things that look simpler than an established library at first glance (I/O of specialized formats comes to mind quickly). However, a lot of the complexity of that established library can wind up being edge cases that you actually _do_ care about, you just don't realize it yet. It's easy to wind up blind to maintenance burden of "just a quick add to the in-house version" repeated over and over again until you wind up with something that has all of the complexities of the widely used library you were trying to avoid. With that said, I still agree that it's good to write things from scratch and avoid complex dependencies where possible! I just think choosing the right cases to do so can be a bit of an art. It's a good one to hone.
    - JackFr 2y ago
      > I/O of specialized formats comes to mind quickly The classic "I'll write my own csv parser - how hard can it be?"
the__alchemist 2y ago
I am a wheel re-inventor. Nice article. The _Specificity_ reason listed is usually the driving factor for me, with the others being downstream effects. In short, the wheels are often built for a different chassis than the one I'm using. Adapting these may be more difficult than making a new wheel.
- alankarmisra 2y ago
  This. I'm trying to set up a personal developer blog and I have a very specific set of requirements. Tried several static blogging frameworks. Apart from the software bloat, I found myself spending a gratituous amount of time trying to customize pieces to my needs. Finally got sick of it, landed up writing a python script and after a few days and < 200 lines of code - I have a working prototype that fits my current needs. I will be doing more of the wheel re-inventing for other projects I have in mind. I strongly agree with your observation that adapting generic frameworks to specific needs probably have longer learning curves than just building a new wheel.
- sunrunner 2y ago
  The specificity reason is interesting as it relates to what feels like an assumption in software that all software components are neatly shaped boxes that a) perfectly encapsulate an area of functionality and b) can be placed into neatly shaped holes of 'required functionality', neither of which ever seem to be true. The boxes are always weirdly shaped with odd edges, and the holes they have to fill are always oddly shaped. The code written to join the two is at minimum glue that seals the two edges, but also usually involves converting one shape to another.
roland35 2y ago
It's funny how often engineers say "it depends"! Even a basic axom like don't reinvent the wheel doesn't always apply. After all, we have entire industries dedicated to doing exactly that! Goodyear spends a lot of time investing in new wheel technology every year.
- austin-cheney 2y ago
  > It's funny how often engineers say "it depends" Then just wait until you speak with lawyers.
  - rambambram 2y ago
    I laughed out loud and can't agree more. If I have to boil down law school to one sentence, it's this one... it depends on the particular circumstances.
- deleted 2y ago
  [deleted]
ikanreed 2y ago
This is what I want to do as an engineer, but I know damn well it's a waste of time and money.
- 01HNNWZ0MV43FF 2y ago
  That's why I do it for fun on weekends
ozornin 2y ago
"I am not reinventing the wheel, I am disrupting the wheel industry" — TramSDK creator https://github.com/racenis/tram-sdk https://github.com/racenis/tram-sdk
- nkrisc 2y ago
  “Tramway Drifting and Dungeon Exploration Simulator” I thought I knew what those words meant but as I read the README I realize I don’t. I am clearly not the intended audience for whatever that is.
Vox_Leone 2y ago
Most importantly, roll your own crypto!* *https://www.schneier.com/blog/archives/2015/05/amateurs_produc.html https://www.schneier.com/blog/archives/2015/05/amateurs_prod...
jasonthorsness 2y ago
Creating a lighter, faster wheel that only works with sort of cart your company builds might invite accusations of “reinventing the wheel” but often it’s just “doing engineering”
- lnenad 2y ago
  But when you need other people to work with your wheel it's much harder to find those capable enough/that want to deal with that. Also when shit hits the fan and the wheel reinventor left the company you're gonna wish you had a standard wheel. (I am a wheel reinventor btw)
  - jasonthorsness 2y ago
    Yeah I’ve seen solo wheel reinvention lead to frustration with people leaving. It should be group effort or at least lots of “teach the wheel” (can we switch off this analogy yet :p)
bsoles 2y ago
The color contrast of the site, especially the background color and the font thickness (on android phone at least) is not good. Many visually impaired people would have very difficult time reading the content.
- badmintonbaseba 2y ago
  Both the dark and the light themes look perfectly readable to me, and I'm rather nitpicky on contrast.
- cratermoon 2y ago
  Agreed. There's something that causes the text to kind of vibrate. https://accessibility.psu.edu/color/brightcolors/ https://accessibility.psu.edu/color/brightcolors/
didgetmaster 2y ago
Reinventing the wheel often means breaking things. Innovation often requires getting rid of backwards compatibility. The status quo is promoted by those who have invested in it, so disrupting it can be met with fierce resistance. When I tell people that file systems are antiquated and need to be replaced with something much better; I often get strong push back. This is a wheel that I have been reinventing for some time. It's not something that can be fixed with minor tweaks.
- 01HNNWZ0MV43FF 2y ago
  Do you have a blog about replacing file systems with something much better? I'm curious what you've tried so far
  - didgetmaster 2y ago
    My blog is listed in my profile. Didgets.substack.com has many blog entries.
  - intalentive 2y ago
    I found this. There's also a demo. Good points. https://didgets.substack.com/p/where-did-i-put-that-file https://didgets.substack.com/p/where-did-i-put-that-file
- glitchc 2y ago
  We need content addressable FSes with bloom filters for fast lookups.
  - didgetmaster 2y ago
    That might be what we need, or not. But we do need to look at different approaches and figure out something better.
  - renox 2y ago
    I would already happy with FS based on 'tags' not trees.
mrbluecoat 2y ago
Missing point: only reinvent the wheel when you control all the wheels For large, complex systems with multiple developers this approach rarely works.
- hjadal 2y ago
  It is gonna be a very bumpy ride if not all the wheel makers are in agreement.
troman_dev 2y ago
I'm definitely a wheel re-inventor (in the educational and entrepreneur sense), and I've come across the same learning points myself. Recently, I just started blogging about my little wheels, and I think its been one of the most satisfying aspects of working on a project!
xipho 2y ago
In scientific software development "don't want to reinvent the wheel" is an oft-repeated mantra that I like to push back on when I hear it. To be fair it's often used in the context of "we'd rather/like to collaborate", rather than an appeal to use "that exact thing". Re-inventing things independently in parallel (parallel evolution analogies) is perhaps a strong indication that something interesting is going on. How do we know we got it "right" if we don't converge independently? If we invent a square wheel, and stopped because "wheel", we'd be in a horrible place. Science is a process, the process of reinventing is a great way to realize new things, and to train, at a low level, scientists. I suspect the process of re-inventing is also important in building out our (long term) ability to depend on our "gut feelings", thus providing the ability to nudge us to experiment along one path or another. [Edit ... all things the article mentions.]
- yummypaint 2y ago
  Reinventing certain wheels is arguably the only way to be sure you understand them. For example Monte Carlo sampling implementations. The logical conclusion of this mindset is mathematics, where people literally prove all of algebra and calculus to themselves as they learn it. There are good pedagogical reasons for doing this.
- fuzzfactor 2y ago
  I'm seeing scientific software being reinvented as kind of a scattered shitshow compared to when it was all drawn up in house using very few computer languages most of the time originally. OTOH, if you're going to invent something every day anyway, it's really not that bad to reinvent the same thing more than once, even in the same year ;) Can be more variety than watching the same movie more than once, plus you might even get more out of it the second time either way. Now what hits me is when you don't even know if your goal is possible but it would be good if it was, and it seems maybe within reach more than other impossible things, it takes a lot of efforts like this before any pay off. You eventually get the feeling of what to pursue, and when to wind down if a fruitless pursuit is the kind that cannot be brought to bear. Now with things like the wheel, you know it's possible to begin with, and quite popular already too, so that's a different starting point when you know it's a proven quantity of some kind, you're not taking the same kind of risks. Then you have to consider if you eventually do make something difficult possible, chances are it's barely possible and kind of touchy. But at least then you know it's possible, that can be one of the best milestones right there. Probably still need to keep on inventing so you can make it even more possible, though. I think it's most satisfying to have a mix of reinventions along with an ongoing effort on things that haven't been fully invented yet.
sunrunner 2y ago
Like every single software development principle, this phrase really needs to be explained with more context and considered with more subtlety than the usual "It's best practice" advice, for a number of reasons (some of which are stated in the article) of which I think the following two are the most important: Firstly, if you want to actually understand how the 'wheel' is invented then yes, you should re-invent it. The process of re-invention involves discovering what actually goes into some of the tools you use. Even if you never use your re-invented wheel in public (often advisable), the process of learning is invaluable in understanding the tools you do use. Secondly, however, what wheel are we even talking about? The wheel is a timeless design, seemingly perfectly suited its task. The software libraries and tools that are usually picked as targets for 'not re-invention' are not wheels. They're higher level abstractions that pre-suppose certain ways of working. There's no timeless design here, just a bunch of arbitrary desicions about how something should work at a higher level with some amount of the decision making you'd have to do without it already done. Is this a bad thing? Of course not. But understanding that all of the 'wheels' are just this and are not magical black boxes that can't be understoor or shouldn't be looked at is important. There are good times to not immediately go and re-implement *and publish* existing tools (emphasis on the publish, you should do things for learning), but understaning why you're choosing to do or NOT do a 'reinvention' is crucial.
- skvmb 2y ago
  Sometimes when reinventing the wheel, you realize that pot-holes suck. Learn to appreciate the wheel and reinvent the road. Learn to appreciate the road and reinvent the rocket. I guess I'm walking home.
  - bch 2y ago
    This is well-put. I think it speaks to "its the journey, not the destination", not learning to ski by only reading books, and Chesterson's Fence[0], off the top of my head. [0] https://fs.blog/chestertons-fence/ https://fs.blog/chestertons-fence/
- jstimpfle 2y ago
  The usual allegoric rebuttal is to show how many types of wheel there are, and how wheels have improved over time. The wheel as a basic concept is to mount a rotating circular shaped thing on a platform for moving. The first wheels were made of wood, at some point spokes were introduced, then other materials. Many more innovations (many of them mutually exclusive) are required to realize a formula 1 car, or a ralley car, or a bus, or a plane.
turnsout 2y ago
There's a big difference between "reinventing the wheel" and simply making your own wheel. People often conflate the two. e.g. making your own static site generator is not reinventing the wheel. It's making your own wheel, which is a perfectly cromulent use of time.
- convolvatron 2y ago
  there is an implicit assumption that we've all settled on what transportation looks like. its a car with 4 wheels, and windows, and a chassis and a glovebox. there are already a whole spectrum of gloveboxes we can but, open source, SaaS, etc. why would you make a new glovebox. its got a latch, and a hinge. maybe I dont even want to build a car.
- TobLoef 2y ago
  To me this is one of the differences between making a wheel for learning or a wheel for innovation, as I mention in the blog. The latter can truly be reinvention, while the former is indeed often more like simply making your own wheel.
Lerc 2y ago
A lot of time I find myself reinventing the wheel is because of some framework that has decided that inverted catenary flooring is the future and their provided wheels are excellent for going in the standard use case direction.
0xbadcafebee 2y ago
More thoughts about reinventing the wheel: - Did YOU invent the last wheel, or any before it? If not, then you will make the same mistakes the last inventor made. Until you make a bunch of wheels, you'll probably suck at it. - You learn more by studying old wheels than trying to bang one out yourself. Study the principles behind the designs rather than shooting from the hip. This is why we study medicine, science and engineering, and don't try to invent new medicines, sciences, and engineering disciplines from ignorance. - Novel-ness is only good when it fixes more problems than it introduces. Novel-ness introduces not only "bugs" from a new, untested design, but also the problems of changing people's expectations, requiring new training, and possibly requiring all the other parts to change to fit the new wheel (which is often more work than just dealing with the old shitty wheel!). New things are not inherently good. Incremental change is almost always better, even if it's harder because you have to struggle with all the existing limitations. Your job isn't to do something easy, it's to make a product better. - Only after you make your new wheel will you find out if it's good or not. Don't go into it assuming what you have is better just because you like the idea of it better. In fact, the more you like the idea, the more you should question it; ego is a dirty liar. Kill your darlings and be prepared to accept others' judgement of the thing, and the reality of it moving on the road.
- TobLoef 2y ago
  Good points, all of them. Especially the last one is just a painful reality of the process. I think it's somewhat similar to the scientific method in that regard. Often your hypothesis is just false, but that does not make the attempt less valid.
- naitgacem 2y ago
  I this this ought to be an iterative process, such as study principles so that you don't start from absolute scratch, then make a wheel that sucks, study some more and do more resarch yourself, rince and repeat until satisfied. There is so much nuance that doesn't get captured in all the study you can do about how a certain thing is made.
deleted 2y ago
[deleted]
rho4 2y ago
> Minimize third-party dependencies. Master the platform’s built-ins and accumulate your own toolbox over time. I would like to work with this person.
- aleph_minus_one 2y ago
  > > Minimize third-party dependencies. Master the platform’s built-ins and accumulate your own toolbox over time. > I would like to work with this person. Before you make such a bold claim, consider that this way of doing programming leads to an accumulation of knowledge that is barely transferable if you switch jobs. I would claim that a central reason why many programmers like third-party dependencies is that being knowledgeable in these is a set of skills that is better marketable and transferable to jobs at other companies. In other words: applying these approach to programming can easily result in a career trap.
  - rho4 2y ago
    You're probably right. Anyway, I will be eternally grateful to the creators of H2 Database for showing that it is very much possible to create an entire database including a web interface without forcing dozens of additional dependencies onto your users.
wcfrobert 2y ago
I find that for me to deeply understand something, I have to reinvent it. There's SO many nuance not captured in textbooks or papers. I reminds me of the feeling of attending lectures by great teachers. Everything makes so much sense until you start the homework assignment.
strongpigeon 2y ago
> [...] Be wary of abstractions made for fabricated use cases. Very well put and I would argue this applies to general software development. This is one of the biggest difference between my freshly-out-of-college self and me right now and something I try to teach engineer I'm trying to grow into "seniors". Too many time have I seen a lot of wasted efforts on trying to build hyper flexible components ("this can do anything!") to support potential future use cases, which often never come to be. This typically results in components that don't do that much and/or are hard to use. Design your components as simply as you need them, but no simpler. This typically gives more flexibility to grow rather than accounting for currently-not-needed use cases.
- add-sub-mul-div 2y ago
  Another good way I've seen it put is, the difference between underengineering and overengineering is that you can fix underengineering.
  - hinkley 2y ago
    Somewhere, at a tender age, I read a paean to the Chevy Straight Six engine block. One of the most heavily modified engines of all time. Later on when I read Zen and the Art of Motorcyle Maintenance it had a similar vibe and effect. I still sometimes use it as an allegory. As it originally shipped it had very low power density. It’s an unremarkable engine. But what it has in spades is potential. You can modify it to increase cylinder diameter, you can strap a giant header on it to improve volume and compression more. You can hang blowers and custom manifolds and more carbs off it to suck out more power. IIRC at the end of its reign they had people coaxing 3, almost 4 times the first gen OEM horsepower out of these things. They had turned it into a beast for that generation of “makers”.
- hinkley 2y ago
  I don’t think there’s an accepted set of concrete criteria for making software that can absorb major design changes later without a great deal of effort and stress. How you write code that can accept an abstraction layer at the last responsible moment. Some people have an intuition for it, but it’s sort of an ineffable quality, buried in Best Practices in a way that is not particularly actionable. So people having been scarred by past attempts to refactor code reach for the abstraction in fear, just in case, because they don’t know what else to do and it’s not written down anywhere, but abstractions are.
satiated_grue 2y ago
I think these folks, who reproduced the aircraft of the Wright Brothers using similar materials, methods, and tools, are a peak example of the value of reinventing as a path to deeper understanding. https://www.wrightexperience.com/ https://www.wrightexperience.com/
rikroots 2y ago
I decided to reinvent SVG Filters because Safari won't let them be used with HTML canvas elements[1] ... well, at least that's the official reason. Unofficially I just wanted a decent filter engine[2] that worked well with my canvas library, and there were things like SVG "filter chaining" which I really liked the idea of - but could they be done in a simpler way[3]? Also: proving to nobody that canvas filters can be done (fairly) efficiently without the need for WebGL shaders (because: why not?). And then I discovered I really like coding up filter functions and went a bit mad with them[4][5] and now the hole is so deep the only option left for me is to keep digging ... [1] - Though I think that's changing this year? May already have changed - but I'm not gonna un-reinvent my filters even if the Safari folks have shipped the fix! [2] - https://github.com/KaliedaRik/Scrawl-canvas/blob/v8/source/helper/filter-engine.js https://github.com/KaliedaRik/Scrawl-canvas/blob/v8/source/h... [3] - Why do the SVG filter primitives need to be so complicated to work with? [4] - https://scrawl-v8.rikweb.org.uk/demo/canvas-007.html https://scrawl-v8.rikweb.org.uk/demo/canvas-007.html [5] - https://scrawl-v8.rikweb.org.uk/demo/filters-103.html https://scrawl-v8.rikweb.org.uk/demo/filters-103.html
pizlonator 2y ago
Great post! If you're a PL/compiler/GC hacker, then here are wheels you should reinvent in order to even just have a basic idea of WTF is going on in the Big Serious Production Wheels that you might end up being gainfully employed to maintain: - Invent your own language, and write at least an interpreter for it, to get a feel for what makes a language work at all, or not. - Invent your own compiler IR. Don't worry if you make a bunch of mistakes. Don't worry about whether you follow my advice for how to do it, or anyone else's advice. Make it your own and learn from your mistakes. - Invent your own way of doing the major compiler optimizations. Of course there are established ways of doing SSA conversion, CSE, constant prop, regalloc, instruction selection, etc. But you won't know why they are that way unless you try to make your own, and then either succeed because you are smarter than everyone else (it's possible that you are), or succeed because you literally reinvented the wheel (then you understand the compiler's wheels better than your friends because you got there from first principles), or you'll fail (most likely outcome) but then you'll understand why the real wheels work the way that they do better than others. - Reinvent memory management. Write your own GC or whatever. That's how I learned the craft. Can't think of a better way to learn.
nuancebydefault 2y ago
The single biggest advantage of a self invented wheel is that you know how to use it. Most lines of code you write, because you know what they mean. Getting to know the pros and cons of somebody else's wheel is often quite an investment.
ian1321 2y ago
When I was a CS undergrad, I used to love to write string libraries.
jiggawatts 2y ago
I follow advice I heard decades ago (by I think John Carmack): Implement it yourself and then throw it away. This is a great way to learn why libraries and tools like compilers are the way they are. I practiced this in the late 90s by making my own 3D maths library and my own “standard” library. I ended up using the built-in 3D maths in DirectX and the C++ STL in the commercial game engine code I worked on later. But having practiced on my own libraries helped me understand the standard ones a lot better.
um1 2y ago
A little off-topic/tangential it seems the real discovery or invention was the axle, or probably more specifically the bearing.

213 points by TobLoef 2y ago

An often unintended benefit of reinventing wheels is vastly superior performance improvements, most of which are entirely unintentional.

dazzawazza 2y ago

> Embrace the strengths of DYI. Create what you need and little more. Be wary of abstractions made for fabricated use cases. DYI?

TobLoef 2y ago
Meant to say "DIY" (Do it yourself). Thanks for letting me know, I've updated it now!
- dazzawazza 2y ago
  Ah, that's cool. Thanks for writing the article.

amelius 2y ago

But should I tell my boss that I'm reinventing everything?

chachacharge 2y ago
colleague of mine wrote a json parser in sql when the rdbms already had a json parser... guess where all the errors came from
- mdaniel 2y ago
  relevant: Parsing JSON Is a Minefield (2018) - https://news.ycombinator.com/item?id=40555431 https://news.ycombinator.com/item?id=40555431 et al <https://hn.algolia.com/?q=parsing+json+minefield https://hn.algolia.com/?q=parsing+json+minefield>

amelius 2y ago

Should mention NIH syndrome. https://en.wikipedia.org/wiki/Not_invented_here https://en.wikipedia.org/wiki/Not_invented_here

TobLoef 2y ago
Yes, it has to be balanced appropriately. I didn't emphasize this a lot in the blog post, but this is really important in a work setting, compared to when working on personal projects. I still think wheel-reinvention has its place in professional teams, but the related challenges should be taken more serious in that case.

JackC 2y ago

I'll add "reduce code size and complexity" to the list of benefits. A python library to calculate a simhash, or track changes on a django model, or auto generate test fixtures, will often be 90% configuration cruft for other usecases, and 10% the code your app actually cares about. Reading the library and extracting and finetuning the core logic makes you responsible for the bugs in the 10%, but no longer affected by bugs in the 90%.

dkarl 2y ago
Hard agree. A library should not inflict complex use cases' complexity on simple use cases, but sometimes they do, either because they're poorly designed or because they're overkill for your use case. But often I see pain and complexity excused with "this is the library that everybody else uses." Sometimes a simple bespoke solution minimizes costs compared to the complexity of using a massive hairball with a ton of power that you don't need. One big caveat to this: there's a tendency to underestimate the cost and complexity of a solution that you, personally, developed. If new developers coming onto the project disagree, they're probably right.
- jofer 2y ago
  The big caveat is a big one. Choose your battles wisely! There are plenty of things that look simpler than an established library at first glance (I/O of specialized formats comes to mind quickly). However, a lot of the complexity of that established library can wind up being edge cases that you actually _do_ care about, you just don't realize it yet. It's easy to wind up blind to maintenance burden of "just a quick add to the in-house version" repeated over and over again until you wind up with something that has all of the complexities of the widely used library you were trying to avoid. With that said, I still agree that it's good to write things from scratch and avoid complex dependencies where possible! I just think choosing the right cases to do so can be a bit of an art. It's a good one to hone.
  - JackFr 2y ago
    > I/O of specialized formats comes to mind quickly The classic "I'll write my own csv parser - how hard can it be?"

the__alchemist 2y ago

I am a wheel re-inventor. Nice article. The _Specificity_ reason listed is usually the driving factor for me, with the others being downstream effects. In short, the wheels are often built for a different chassis than the one I'm using. Adapting these may be more difficult than making a new wheel.

alankarmisra 2y ago
This. I'm trying to set up a personal developer blog and I have a very specific set of requirements. Tried several static blogging frameworks. Apart from the software bloat, I found myself spending a gratituous amount of time trying to customize pieces to my needs. Finally got sick of it, landed up writing a python script and after a few days and < 200 lines of code - I have a working prototype that fits my current needs. I will be doing more of the wheel re-inventing for other projects I have in mind. I strongly agree with your observation that adapting generic frameworks to specific needs probably have longer learning curves than just building a new wheel.
sunrunner 2y ago
The specificity reason is interesting as it relates to what feels like an assumption in software that all software components are neatly shaped boxes that a) perfectly encapsulate an area of functionality and b) can be placed into neatly shaped holes of 'required functionality', neither of which ever seem to be true. The boxes are always weirdly shaped with odd edges, and the holes they have to fill are always oddly shaped. The code written to join the two is at minimum glue that seals the two edges, but also usually involves converting one shape to another.

roland35 2y ago

It's funny how often engineers say "it depends"! Even a basic axom like don't reinvent the wheel doesn't always apply. After all, we have entire industries dedicated to doing exactly that! Goodyear spends a lot of time investing in new wheel technology every year.

austin-cheney 2y ago
> It's funny how often engineers say "it depends" Then just wait until you speak with lawyers.
- rambambram 2y ago
  I laughed out loud and can't agree more. If I have to boil down law school to one sentence, it's this one... it depends on the particular circumstances.
deleted 2y ago
[deleted]

ikanreed 2y ago

This is what I want to do as an engineer, but I know damn well it's a waste of time and money.

01HNNWZ0MV43FF 2y ago
That's why I do it for fun on weekends

ozornin 2y ago

"I am not reinventing the wheel, I am disrupting the wheel industry" — TramSDK creator https://github.com/racenis/tram-sdk https://github.com/racenis/tram-sdk

nkrisc 2y ago
“Tramway Drifting and Dungeon Exploration Simulator” I thought I knew what those words meant but as I read the README I realize I don’t. I am clearly not the intended audience for whatever that is.

Vox_Leone 2y ago

Most importantly, roll your own crypto!* *https://www.schneier.com/blog/archives/2015/05/amateurs_produc.html https://www.schneier.com/blog/archives/2015/05/amateurs_prod...

jasonthorsness 2y ago

Creating a lighter, faster wheel that only works with sort of cart your company builds might invite accusations of “reinventing the wheel” but often it’s just “doing engineering”

lnenad 2y ago
But when you need other people to work with your wheel it's much harder to find those capable enough/that want to deal with that. Also when shit hits the fan and the wheel reinventor left the company you're gonna wish you had a standard wheel. (I am a wheel reinventor btw)
- jasonthorsness 2y ago
  Yeah I’ve seen solo wheel reinvention lead to frustration with people leaving. It should be group effort or at least lots of “teach the wheel” (can we switch off this analogy yet :p)

bsoles 2y ago

The color contrast of the site, especially the background color and the font thickness (on android phone at least) is not good. Many visually impaired people would have very difficult time reading the content.

badmintonbaseba 2y ago
Both the dark and the light themes look perfectly readable to me, and I'm rather nitpicky on contrast.
cratermoon 2y ago
Agreed. There's something that causes the text to kind of vibrate. https://accessibility.psu.edu/color/brightcolors/ https://accessibility.psu.edu/color/brightcolors/

didgetmaster 2y ago

Reinventing the wheel often means breaking things. Innovation often requires getting rid of backwards compatibility. The status quo is promoted by those who have invested in it, so disrupting it can be met with fierce resistance. When I tell people that file systems are antiquated and need to be replaced with something much better; I often get strong push back. This is a wheel that I have been reinventing for some time. It's not something that can be fixed with minor tweaks.

01HNNWZ0MV43FF 2y ago
Do you have a blog about replacing file systems with something much better? I'm curious what you've tried so far
- didgetmaster 2y ago
  My blog is listed in my profile. Didgets.substack.com has many blog entries.
- intalentive 2y ago
  I found this. There's also a demo. Good points. https://didgets.substack.com/p/where-did-i-put-that-file https://didgets.substack.com/p/where-did-i-put-that-file
glitchc 2y ago
We need content addressable FSes with bloom filters for fast lookups.
- didgetmaster 2y ago
  That might be what we need, or not. But we do need to look at different approaches and figure out something better.
- renox 2y ago
  I would already happy with FS based on 'tags' not trees.

mrbluecoat 2y ago

Missing point: only reinvent the wheel when you control all the wheels For large, complex systems with multiple developers this approach rarely works.

hjadal 2y ago
It is gonna be a very bumpy ride if not all the wheel makers are in agreement.

troman_dev 2y ago

I'm definitely a wheel re-inventor (in the educational and entrepreneur sense), and I've come across the same learning points myself. Recently, I just started blogging about my little wheels, and I think its been one of the most satisfying aspects of working on a project!

xipho 2y ago

In scientific software development "don't want to reinvent the wheel" is an oft-repeated mantra that I like to push back on when I hear it. To be fair it's often used in the context of "we'd rather/like to collaborate", rather than an appeal to use "that exact thing". Re-inventing things independently in parallel (parallel evolution analogies) is perhaps a strong indication that something interesting is going on. How do we know we got it "right" if we don't converge independently? If we invent a square wheel, and stopped because "wheel", we'd be in a horrible place. Science is a process, the process of reinventing is a great way to realize new things, and to train, at a low level, scientists. I suspect the process of re-inventing is also important in building out our (long term) ability to depend on our "gut feelings", thus providing the ability to nudge us to experiment along one path or another. [Edit ... all things the article mentions.]

yummypaint 2y ago
Reinventing certain wheels is arguably the only way to be sure you understand them. For example Monte Carlo sampling implementations. The logical conclusion of this mindset is mathematics, where people literally prove all of algebra and calculus to themselves as they learn it. There are good pedagogical reasons for doing this.
fuzzfactor 2y ago
I'm seeing scientific software being reinvented as kind of a scattered shitshow compared to when it was all drawn up in house using very few computer languages most of the time originally. OTOH, if you're going to invent something every day anyway, it's really not that bad to reinvent the same thing more than once, even in the same year ;) Can be more variety than watching the same movie more than once, plus you might even get more out of it the second time either way. Now what hits me is when you don't even know if your goal is possible but it would be good if it was, and it seems maybe within reach more than other impossible things, it takes a lot of efforts like this before any pay off. You eventually get the feeling of what to pursue, and when to wind down if a fruitless pursuit is the kind that cannot be brought to bear. Now with things like the wheel, you know it's possible to begin with, and quite popular already too, so that's a different starting point when you know it's a proven quantity of some kind, you're not taking the same kind of risks. Then you have to consider if you eventually do make something difficult possible, chances are it's barely possible and kind of touchy. But at least then you know it's possible, that can be one of the best milestones right there. Probably still need to keep on inventing so you can make it even more possible, though. I think it's most satisfying to have a mix of reinventions along with an ongoing effort on things that haven't been fully invented yet.

sunrunner 2y ago

Like every single software development principle, this phrase really needs to be explained with more context and considered with more subtlety than the usual "It's best practice" advice, for a number of reasons (some of which are stated in the article) of which I think the following two are the most important: Firstly, if you want to actually understand how the 'wheel' is invented then yes, you should re-invent it. The process of re-invention involves discovering what actually goes into some of the tools you use. Even if you never use your re-invented wheel in public (often advisable), the process of learning is invaluable in understanding the tools you do use. Secondly, however, what wheel are we even talking about? The wheel is a timeless design, seemingly perfectly suited its task. The software libraries and tools that are usually picked as targets for 'not re-invention' are not wheels. They're higher level abstractions that pre-suppose certain ways of working. There's no timeless design here, just a bunch of arbitrary desicions about how something should work at a higher level with some amount of the decision making you'd have to do without it already done. Is this a bad thing? Of course not. But understanding that all of the 'wheels' are just this and are not magical black boxes that can't be understoor or shouldn't be looked at is important. There are good times to not immediately go and re-implement *and publish* existing tools (emphasis on the publish, you should do things for learning), but understaning why you're choosing to do or NOT do a 'reinvention' is crucial.

skvmb 2y ago
Sometimes when reinventing the wheel, you realize that pot-holes suck. Learn to appreciate the wheel and reinvent the road. Learn to appreciate the road and reinvent the rocket. I guess I'm walking home.
- bch 2y ago
  This is well-put. I think it speaks to "its the journey, not the destination", not learning to ski by only reading books, and Chesterson's Fence[0], off the top of my head. [0] https://fs.blog/chestertons-fence/ https://fs.blog/chestertons-fence/
jstimpfle 2y ago
The usual allegoric rebuttal is to show how many types of wheel there are, and how wheels have improved over time. The wheel as a basic concept is to mount a rotating circular shaped thing on a platform for moving. The first wheels were made of wood, at some point spokes were introduced, then other materials. Many more innovations (many of them mutually exclusive) are required to realize a formula 1 car, or a ralley car, or a bus, or a plane.

turnsout 2y ago

There's a big difference between "reinventing the wheel" and simply making your own wheel. People often conflate the two. e.g. making your own static site generator is not reinventing the wheel. It's making your own wheel, which is a perfectly cromulent use of time.

convolvatron 2y ago
there is an implicit assumption that we've all settled on what transportation looks like. its a car with 4 wheels, and windows, and a chassis and a glovebox. there are already a whole spectrum of gloveboxes we can but, open source, SaaS, etc. why would you make a new glovebox. its got a latch, and a hinge. maybe I dont even want to build a car.
TobLoef 2y ago
To me this is one of the differences between making a wheel for learning or a wheel for innovation, as I mention in the blog. The latter can truly be reinvention, while the former is indeed often more like simply making your own wheel.

Lerc 2y ago

A lot of time I find myself reinventing the wheel is because of some framework that has decided that inverted catenary flooring is the future and their provided wheels are excellent for going in the standard use case direction.

0xbadcafebee 2y ago

More thoughts about reinventing the wheel: - Did YOU invent the last wheel, or any before it? If not, then you will make the same mistakes the last inventor made. Until you make a bunch of wheels, you'll probably suck at it. - You learn more by studying old wheels than trying to bang one out yourself. Study the principles behind the designs rather than shooting from the hip. This is why we study medicine, science and engineering, and don't try to invent new medicines, sciences, and engineering disciplines from ignorance. - Novel-ness is only good when it fixes more problems than it introduces. Novel-ness introduces not only "bugs" from a new, untested design, but also the problems of changing people's expectations, requiring new training, and possibly requiring all the other parts to change to fit the new wheel (which is often more work than just dealing with the old shitty wheel!). New things are not inherently good. Incremental change is almost always better, even if it's harder because you have to struggle with all the existing limitations. Your job isn't to do something easy, it's to make a product better. - Only after you make your new wheel will you find out if it's good or not. Don't go into it assuming what you have is better just because you like the idea of it better. In fact, the more you like the idea, the more you should question it; ego is a dirty liar. Kill your darlings and be prepared to accept others' judgement of the thing, and the reality of it moving on the road.

TobLoef 2y ago
Good points, all of them. Especially the last one is just a painful reality of the process. I think it's somewhat similar to the scientific method in that regard. Often your hypothesis is just false, but that does not make the attempt less valid.
naitgacem 2y ago
I this this ought to be an iterative process, such as study principles so that you don't start from absolute scratch, then make a wheel that sucks, study some more and do more resarch yourself, rince and repeat until satisfied. There is so much nuance that doesn't get captured in all the study you can do about how a certain thing is made.

deleted 2y ago

[deleted]

rho4 2y ago

> Minimize third-party dependencies. Master the platform’s built-ins and accumulate your own toolbox over time. I would like to work with this person.

aleph_minus_one 2y ago
> > Minimize third-party dependencies. Master the platform’s built-ins and accumulate your own toolbox over time. > I would like to work with this person. Before you make such a bold claim, consider that this way of doing programming leads to an accumulation of knowledge that is barely transferable if you switch jobs. I would claim that a central reason why many programmers like third-party dependencies is that being knowledgeable in these is a set of skills that is better marketable and transferable to jobs at other companies. In other words: applying these approach to programming can easily result in a career trap.
- rho4 2y ago
  You're probably right. Anyway, I will be eternally grateful to the creators of H2 Database for showing that it is very much possible to create an entire database including a web interface without forcing dozens of additional dependencies onto your users.

wcfrobert 2y ago

I find that for me to deeply understand something, I have to reinvent it. There's SO many nuance not captured in textbooks or papers. I reminds me of the feeling of attending lectures by great teachers. Everything makes so much sense until you start the homework assignment.

strongpigeon 2y ago

> [...] Be wary of abstractions made for fabricated use cases. Very well put and I would argue this applies to general software development. This is one of the biggest difference between my freshly-out-of-college self and me right now and something I try to teach engineer I'm trying to grow into "seniors". Too many time have I seen a lot of wasted efforts on trying to build hyper flexible components ("this can do anything!") to support potential future use cases, which often never come to be. This typically results in components that don't do that much and/or are hard to use. Design your components as simply as you need them, but no simpler. This typically gives more flexibility to grow rather than accounting for currently-not-needed use cases.

add-sub-mul-div 2y ago
Another good way I've seen it put is, the difference between underengineering and overengineering is that you can fix underengineering.
- hinkley 2y ago
  Somewhere, at a tender age, I read a paean to the Chevy Straight Six engine block. One of the most heavily modified engines of all time. Later on when I read Zen and the Art of Motorcyle Maintenance it had a similar vibe and effect. I still sometimes use it as an allegory. As it originally shipped it had very low power density. It’s an unremarkable engine. But what it has in spades is potential. You can modify it to increase cylinder diameter, you can strap a giant header on it to improve volume and compression more. You can hang blowers and custom manifolds and more carbs off it to suck out more power. IIRC at the end of its reign they had people coaxing 3, almost 4 times the first gen OEM horsepower out of these things. They had turned it into a beast for that generation of “makers”.
hinkley 2y ago
I don’t think there’s an accepted set of concrete criteria for making software that can absorb major design changes later without a great deal of effort and stress. How you write code that can accept an abstraction layer at the last responsible moment. Some people have an intuition for it, but it’s sort of an ineffable quality, buried in Best Practices in a way that is not particularly actionable. So people having been scarred by past attempts to refactor code reach for the abstraction in fear, just in case, because they don’t know what else to do and it’s not written down anywhere, but abstractions are.

satiated_grue 2y ago

I think these folks, who reproduced the aircraft of the Wright Brothers using similar materials, methods, and tools, are a peak example of the value of reinventing as a path to deeper understanding. https://www.wrightexperience.com/ https://www.wrightexperience.com/

rikroots 2y ago

I decided to reinvent SVG Filters because Safari won't let them be used with HTML canvas elements[1] ... well, at least that's the official reason. Unofficially I just wanted a decent filter engine[2] that worked well with my canvas library, and there were things like SVG "filter chaining" which I really liked the idea of - but could they be done in a simpler way[3]? Also: proving to nobody that canvas filters can be done (fairly) efficiently without the need for WebGL shaders (because: why not?). And then I discovered I really like coding up filter functions and went a bit mad with them[4][5] and now the hole is so deep the only option left for me is to keep digging ... [1] - Though I think that's changing this year? May already have changed - but I'm not gonna un-reinvent my filters even if the Safari folks have shipped the fix! [2] - https://github.com/KaliedaRik/Scrawl-canvas/blob/v8/source/helper/filter-engine.js https://github.com/KaliedaRik/Scrawl-canvas/blob/v8/source/h... [3] - Why do the SVG filter primitives need to be so complicated to work with? [4] - https://scrawl-v8.rikweb.org.uk/demo/canvas-007.html https://scrawl-v8.rikweb.org.uk/demo/canvas-007.html [5] - https://scrawl-v8.rikweb.org.uk/demo/filters-103.html https://scrawl-v8.rikweb.org.uk/demo/filters-103.html

pizlonator 2y ago

Great post! If you're a PL/compiler/GC hacker, then here are wheels you should reinvent in order to even just have a basic idea of WTF is going on in the Big Serious Production Wheels that you might end up being gainfully employed to maintain: - Invent your own language, and write at least an interpreter for it, to get a feel for what makes a language work at all, or not. - Invent your own compiler IR. Don't worry if you make a bunch of mistakes. Don't worry about whether you follow my advice for how to do it, or anyone else's advice. Make it your own and learn from your mistakes. - Invent your own way of doing the major compiler optimizations. Of course there are established ways of doing SSA conversion, CSE, constant prop, regalloc, instruction selection, etc. But you won't know why they are that way unless you try to make your own, and then either succeed because you are smarter than everyone else (it's possible that you are), or succeed because you literally reinvented the wheel (then you understand the compiler's wheels better than your friends because you got there from first principles), or you'll fail (most likely outcome) but then you'll understand why the real wheels work the way that they do better than others. - Reinvent memory management. Write your own GC or whatever. That's how I learned the craft. Can't think of a better way to learn.

nuancebydefault 2y ago

The single biggest advantage of a self invented wheel is that you know how to use it. Most lines of code you write, because you know what they mean. Getting to know the pros and cons of somebody else's wheel is often quite an investment.

ian1321 2y ago

When I was a CS undergrad, I used to love to write string libraries.

jiggawatts 2y ago

I follow advice I heard decades ago (by I think John Carmack): Implement it yourself and then throw it away. This is a great way to learn why libraries and tools like compilers are the way they are. I practiced this in the late 90s by making my own 3D maths library and my own “standard” library. I ended up using the built-in 3D maths in DirectX and the C++ STL in the commercial game engine code I worked on later. But having practiced on my own libraries helped me understand the standard ones a lot better.

um1 2y ago

A little off-topic/tangential it seems the real discovery or invention was the axle, or probably more specifically the bearing.