10 ms·

Stupid light software

97 points by pcr910303 6y ago

jlelse 6y ago
I wrote a reply to this on my blog (https://jlelse.blog/links/2020/11/stupid-light-software https://jlelse.blog/links/2020/11/stupid-light-software): With software I most often go from “heavy” to “light”. First, I try to realize the features I need and then I try to optimize where possible. First, I will use many libraries and after I got everything working, I will see if I really need all those libraries and may replace a whole library with a custom more simple implementation that still fits the needs. A case for “stupid light” software is probably trying to use a static website when an optimized dynamic site is much more appropriate. You start to over-engineer a complex build workflow, when all that’s really needed to speed up a site is better caching. I have some experiences especially related to the database point. When I developed Android apps, I tried to use flat text files too. But recently I discovered how awesome SQLite is. In some cases flat files are great, but especially when you need to parse those files or retrieve specific information from them, SQLite might be better than flat files. But in many cases SQLite could replace complex PostgreSQL or MariaDB databases, especially when concurrent writing isn’t needed.
- teraku 6y ago
  >With software I most often go from “heavy” to “light”. It works the same in hiking. You go first your first hike with _definetly_ all equipment you need. And afterwards you trim down for the next one, until you hit the sweet spot. And if you go too far, you are reaching the stupid light point.
EdwardDiego 6y ago
As an aside "Stupid light" hiking is a great term for it. I recall a case where a British hiker died a long and lonely death in our country when he broke a leg a few days into a 4 week journey. Locals had encouraged him go take a locator beacon (due to the rough terrain and remoteness), but he'd refused, because it weighed too much.
Geminidog 6y ago
Totally new information. Libraries and databases for general use cases can be bloated and bad when compared to rolling your own code that's specifically for your domain. This is genius level thinking. I had no idea. Nobody on the face of the earth ever knew about this concept until this article appeared on HN, so this is absolutely revolutionary and will likely change the industry forever. He gave it a metaphorical name from hiking too. "Stupid Light" and I'm just blown away at the incredible Einstein level imagination here.
- skipnup 6y ago
  What
makach 6y ago
This article, light, but not stupid light(!) cast a brilliant perspective on software development. Good read! Two thumbs from me!
1vuio0pswjnm7 6y ago
Is "stupid light hiking" really a proper comparison. In the case of hiking, the hiker is making the choices and the hiker is the one who will have to live (or die) with them. In the case of software, too often it is a commercially-oriented application developer making the choices and it is a different person, a user, who must live with them. The hiker has control over her choices. The user has no control over the application developer's choices. In some ways, the best programs are the ones I write for own use only. I know the user better than anyone and I have full control over the design choices. Obviously, choosing not to use someone else's software is ineffective as a means of controlling someone else's design choices. Perhaps unsolicited "advice" found in blogs might have some influence though.
- t0astbread 6y ago
  If the developer is marketing directly to the user then the user has a choice: They can decide if they wanna use the app. If the developer is marketing to a third party that is forcing the user to use the app (e.g. employers) then the user is powerless. The only exception is if the app locks the user in in some way.
- CJefferson 6y ago
  I feel there is an obvious missing comparision -- I'm often not going on a life-or-death hike, I'm just going on a pleasent afternoon walk with friends. The friend who brings all the serious hiking equipment, and even worse expects everyone else to bring such stuff, is really annoying in that situation.
- heelix 6y ago
  Stupid light does not have to mean life or death. I was shaving a few ounces from my pack - and switched to a hex tarp for my hammock. Got caught up in a rainstorm where wind shifted directions, and ended up getting soaked. I picked up a tarp with 'doors'. Similar mindset with code. We typically have to support what we write. When the war room hits because the dumb was stronger than expected... shortcuts can come back to bite one in the ass.
twic 6y ago
If i am ever in a situation where i am considering SQLite, i will try to use DuckDB instead. It's a similar weight (the all-inclusive JARs are 6.9 MB for SQLite and 8.7 for DuckDB), it also stores a database in a single file, it's different in many ways, but for me, the killer feature is that it uses the PostgreSQL SQL syntax (and parser). This should make it very easy to migrate from DuckDB to PostgreSQL if that becomes necessary. I currently work on a project based around SQLite, which suffers from that fact. Migrating it to PostgreSQL would be great, but would involve a lot of tedious work.
- gravypod 6y ago
  Something I've used similarly in these situations is LevelDB or RocksDB if you just need a really small key value store. It's a really tiny library that gets you really fast write performance and some light atomic operations.
- sradman 6y ago
  It is my understanding that DuckDB is an embedded in-memory OLAP ColumnStore designed to be primarily used with R/Python data frames. I noticed that they added JDBC support. Does this mean they've added persistence and other OLTP functionality? I thought their "SQLite for Analytics" tagline was confusing and I'm certainly questioning my recollection now.
  - twic 6y ago
    As far as i know, DuckDB has always supported files, and the full SQL standard you would need for OLTP, and Pandas is just one use case. People do seem to have a wide range of impressions of what DuckDB is for, so i think it's fair to conclude that their messaging has not been very effective!
    - sradman 6y ago
      The problem is my recall and laziness. I didn't look into the details of what parts of HyPer were borrowed and I missed the adoption of the DataBlocks [1] storage format. DuckDB is designed to be an embedded HTAP (Hybrid Transactional/Analytical Processing) engine. I think the project is immature and the documentation is sparse. I expect to see detailed documentation on default file locations and the recovery process for an OLTP/HTAP engine and I didn't see anything like that in the DuckDB docs. My bad, I should have checked my assumptions. I would not adopt DuckDB as an OLTP replacement for either SQLite or PostgreSQL without a strong OLAP component and some serious performance/load/integrity testing, but it sounds like you are confident in its abilities. Thanks for sharing your experience. [1] https://db.in.tum.de/downloads/publications/datablocks.pdf https://db.in.tum.de/downloads/publications/datablocks.pdf
brundolf 6y ago
I once heard someone say (paraphrased), "To some programmers RAM is like that couch in your grandmother's living room, with a plastic cover, that nobody's ever actually allowed to sit on."
- einr 6y ago
  I think it is prudent not to sit on other people's couches uninvited. Now what programmers do to their own couches or RAM I couldn't care less about, but generally they write software for other people and their RAM.
  - wruza 6y ago
    Programmers get paid (in money or respect) by other people like you who are free to use or to not use their software. Idk which percentage of modern stacks is free to us, and how much it would cost if "they" did it the "right" way. Blaming them for using your RAM to do the job is like blaming potato for using your enzymes to dissolve into nutrients.
- ThrowawayR2 6y ago
  Replace "RAM" with "electricity" and it becomes obvious why that quip is wrongheaded, at least for those of us who pay our own electric bill. "Developers" like that who don't care about externalities because they're not footing the bill waste millions of dollars worth of RAM, millions of dollars worth of electricity, etc. globally.
  - p1necone 6y ago
    RAM is not the same kind of resource as electricity. An application that uses twice as much RAM is not twice as expensive to run.
    - exmadscientist 6y ago
      You're right, it can easily be four times as expensive. Or even impossible.
xixixao 6y ago
I’ve been thinking about how extremes are practically never good (perhaps except for this rule?), for anything in life. This is a great example.
romwell 6y ago
This article, in the age where a chat app eats gigabytes of RAM, is almost insulting. And it's not like it's nothing because we have terabytes of RAM now. A 2020 laptop I'm eyeing only comes in 8GB/16GB soldered RAM config; and the 16GB version has been sold out since the release. Which is not surprising, since the browser alone easily needs 8GB in a medium-heavy session. Stupid light? Call me when common software isn't stupid heavy.
- black_puppydog 6y ago
  I'm not insulted. The concern is valid. Its range of application is just so fringe as to make this a near-academic discussion. I recently tried using a small JS module to identify a file's mime-type. It pulled in 68MB of dependencies. And that's just one of many things an actual application will need. W.T.F. So I agree, this is not the problem we need to tackle right now.
  - corytheboyd 6y ago
    Wow mind sharing that package name? That’s insane I’m curious what they’re even doing
    - bromuro 6y ago
      Probably he was installing the dev dependencies.
      - corytheboyd 6y ago
        Either that, the package legitimately has that many recursive dependencies, or the original commenter is hyperbolizing.
  - romwell 6y ago
    I agree with what you say; my choice of words was not perfect.
  - AussieWog93 6y ago

97 points by pcr910303 6y ago

I wrote a reply to this on my blog (https://jlelse.blog/links/2020/11/stupid-light-software https://jlelse.blog/links/2020/11/stupid-light-software): With software I most often go from “heavy” to “light”. First, I try to realize the features I need and then I try to optimize where possible. First, I will use many libraries and after I got everything working, I will see if I really need all those libraries and may replace a whole library with a custom more simple implementation that still fits the needs. A case for “stupid light” software is probably trying to use a static website when an optimized dynamic site is much more appropriate. You start to over-engineer a complex build workflow, when all that’s really needed to speed up a site is better caching. I have some experiences especially related to the database point. When I developed Android apps, I tried to use flat text files too. But recently I discovered how awesome SQLite is. In some cases flat files are great, but especially when you need to parse those files or retrieve specific information from them, SQLite might be better than flat files. But in many cases SQLite could replace complex PostgreSQL or MariaDB databases, especially when concurrent writing isn’t needed.

teraku 6y ago
>With software I most often go from “heavy” to “light”. It works the same in hiking. You go first your first hike with _definetly_ all equipment you need. And afterwards you trim down for the next one, until you hit the sweet spot. And if you go too far, you are reaching the stupid light point.

EdwardDiego 6y ago

As an aside "Stupid light" hiking is a great term for it. I recall a case where a British hiker died a long and lonely death in our country when he broke a leg a few days into a 4 week journey. Locals had encouraged him go take a locator beacon (due to the rough terrain and remoteness), but he'd refused, because it weighed too much.

Geminidog 6y ago

Totally new information. Libraries and databases for general use cases can be bloated and bad when compared to rolling your own code that's specifically for your domain. This is genius level thinking. I had no idea. Nobody on the face of the earth ever knew about this concept until this article appeared on HN, so this is absolutely revolutionary and will likely change the industry forever. He gave it a metaphorical name from hiking too. "Stupid Light" and I'm just blown away at the incredible Einstein level imagination here.

skipnup 6y ago
What

makach 6y ago

This article, light, but not stupid light(!) cast a brilliant perspective on software development. Good read! Two thumbs from me!

1vuio0pswjnm7 6y ago

Is "stupid light hiking" really a proper comparison. In the case of hiking, the hiker is making the choices and the hiker is the one who will have to live (or die) with them. In the case of software, too often it is a commercially-oriented application developer making the choices and it is a different person, a user, who must live with them. The hiker has control over her choices. The user has no control over the application developer's choices. In some ways, the best programs are the ones I write for own use only. I know the user better than anyone and I have full control over the design choices. Obviously, choosing not to use someone else's software is ineffective as a means of controlling someone else's design choices. Perhaps unsolicited "advice" found in blogs might have some influence though.

t0astbread 6y ago
If the developer is marketing directly to the user then the user has a choice: They can decide if they wanna use the app. If the developer is marketing to a third party that is forcing the user to use the app (e.g. employers) then the user is powerless. The only exception is if the app locks the user in in some way.
CJefferson 6y ago
I feel there is an obvious missing comparision -- I'm often not going on a life-or-death hike, I'm just going on a pleasent afternoon walk with friends. The friend who brings all the serious hiking equipment, and even worse expects everyone else to bring such stuff, is really annoying in that situation.
heelix 6y ago
Stupid light does not have to mean life or death. I was shaving a few ounces from my pack - and switched to a hex tarp for my hammock. Got caught up in a rainstorm where wind shifted directions, and ended up getting soaked. I picked up a tarp with 'doors'. Similar mindset with code. We typically have to support what we write. When the war room hits because the dumb was stronger than expected... shortcuts can come back to bite one in the ass.

twic 6y ago

If i am ever in a situation where i am considering SQLite, i will try to use DuckDB instead. It's a similar weight (the all-inclusive JARs are 6.9 MB for SQLite and 8.7 for DuckDB), it also stores a database in a single file, it's different in many ways, but for me, the killer feature is that it uses the PostgreSQL SQL syntax (and parser). This should make it very easy to migrate from DuckDB to PostgreSQL if that becomes necessary. I currently work on a project based around SQLite, which suffers from that fact. Migrating it to PostgreSQL would be great, but would involve a lot of tedious work.

gravypod 6y ago
Something I've used similarly in these situations is LevelDB or RocksDB if you just need a really small key value store. It's a really tiny library that gets you really fast write performance and some light atomic operations.
sradman 6y ago
It is my understanding that DuckDB is an embedded in-memory OLAP ColumnStore designed to be primarily used with R/Python data frames. I noticed that they added JDBC support. Does this mean they've added persistence and other OLTP functionality? I thought their "SQLite for Analytics" tagline was confusing and I'm certainly questioning my recollection now.
- twic 6y ago
  As far as i know, DuckDB has always supported files, and the full SQL standard you would need for OLTP, and Pandas is just one use case. People do seem to have a wide range of impressions of what DuckDB is for, so i think it's fair to conclude that their messaging has not been very effective!
  - sradman 6y ago
    The problem is my recall and laziness. I didn't look into the details of what parts of HyPer were borrowed and I missed the adoption of the DataBlocks [1] storage format. DuckDB is designed to be an embedded HTAP (Hybrid Transactional/Analytical Processing) engine. I think the project is immature and the documentation is sparse. I expect to see detailed documentation on default file locations and the recovery process for an OLTP/HTAP engine and I didn't see anything like that in the DuckDB docs. My bad, I should have checked my assumptions. I would not adopt DuckDB as an OLTP replacement for either SQLite or PostgreSQL without a strong OLAP component and some serious performance/load/integrity testing, but it sounds like you are confident in its abilities. Thanks for sharing your experience. [1] https://db.in.tum.de/downloads/publications/datablocks.pdf https://db.in.tum.de/downloads/publications/datablocks.pdf

brundolf 6y ago

I once heard someone say (paraphrased), "To some programmers RAM is like that couch in your grandmother's living room, with a plastic cover, that nobody's ever actually allowed to sit on."

einr 6y ago
I think it is prudent not to sit on other people's couches uninvited. Now what programmers do to their own couches or RAM I couldn't care less about, but generally they write software for other people and their RAM.
- wruza 6y ago
  Programmers get paid (in money or respect) by other people like you who are free to use or to not use their software. Idk which percentage of modern stacks is free to us, and how much it would cost if "they" did it the "right" way. Blaming them for using your RAM to do the job is like blaming potato for using your enzymes to dissolve into nutrients.
ThrowawayR2 6y ago
Replace "RAM" with "electricity" and it becomes obvious why that quip is wrongheaded, at least for those of us who pay our own electric bill. "Developers" like that who don't care about externalities because they're not footing the bill waste millions of dollars worth of RAM, millions of dollars worth of electricity, etc. globally.
- p1necone 6y ago
  RAM is not the same kind of resource as electricity. An application that uses twice as much RAM is not twice as expensive to run.
  - exmadscientist 6y ago
    You're right, it can easily be four times as expensive. Or even impossible.

xixixao 6y ago

I’ve been thinking about how extremes are practically never good (perhaps except for this rule?), for anything in life. This is a great example.

romwell 6y ago

This article, in the age where a chat app eats gigabytes of RAM, is almost insulting. And it's not like it's nothing because we have terabytes of RAM now. A 2020 laptop I'm eyeing only comes in 8GB/16GB soldered RAM config; and the 16GB version has been sold out since the release. Which is not surprising, since the browser alone easily needs 8GB in a medium-heavy session. Stupid light? Call me when common software isn't stupid heavy.

black_puppydog 6y ago
I'm not insulted. The concern is valid. Its range of application is just so fringe as to make this a near-academic discussion. I recently tried using a small JS module to identify a file's mime-type. It pulled in 68MB of dependencies. And that's just one of many things an actual application will need. W.T.F. So I agree, this is not the problem we need to tackle right now.
- corytheboyd 6y ago
  Wow mind sharing that package name? That’s insane I’m curious what they’re even doing
  - bromuro 6y ago
    Probably he was installing the dev dependencies.
    - corytheboyd 6y ago
      Either that, the package legitimately has that many recursive dependencies, or the original commenter is hyperbolizing.
- romwell 6y ago
  I agree with what you say; my choice of words was not perfect.
- AussieWog93 6y ago