7 ms·

On a side note inspired by this blog post. I'm wondering if humans are mostly incapable of producing great things without (artifical) restrictions. In this ca

by gnyman 3y ago

On a side note inspired by this blog post. I'm wondering if humans are mostly incapable of producing great things without (artifical) restrictions. In this case, marginalia is (ridiculously) efficient because Victor (the creator) is intentionally restricting what hardware it runs on and how much ram it has. If he just caved in and added another 32GiB it would work for a while, but the inefficient design would persist and the problem would just show it's head later and then there would be more complexity around that design and it might not be as easy to fix then. If the original thesis is correct, then I think it explains why most software is so bad (bloated, slow, buggy) nowadays. It's because very few individual pieces of software nowadays are hitting any limits (in isolation). So each individual piece is terribly inefficient but with the latest M2 Pro and GiB connection you can just keep ahead of the curve where it becomes a problem. Anyways, turned into a rant; but the conclusion might be to limit yourself, and you (and e everyone else) will be better off long term.

jancsika 3y ago
> I'm wondering if humans are mostly incapable of producing great things without (artifical) restrictions. I think the real issue is that there isn't a program language that produces a compiler error if the given code can exceed a maximum specified latency. Even working on a program with soft-realtime scheduling, I've had to constantly push back against patches that introduce some obscure convenience without having measured worst case latency. The problem is so bad I doubt most people realize it's there. I don't know what the answer is, but I have the feeling there's an intersection with timing attacks on software/hardware. Some kind of tooling that makes both worst case times and variance as visible as the computed CSS in devTools would probably help. Added to some kind of static analysis, perhaps devs to hack their way to decently responsive interfaces and services.
MichaelZuo 3y ago
American Airlines ran SABRE, a sizeable airline ticketing and reservation system, in the mid-1970s on two system/360 mainframes that could only process a few tens of millions of instructions per second. A raspberry pi 2 can do over 4 billion Dhrystone instructions per second, and a pi 4 over 10 billion per second. Of course by modern standards mid-1970s SABRE was pretty barebones for an airline's main system, but it's at least theoretically possible to run simplified systems for over a 100 airlines simultaneously on a single pi 2... So yes modern programs are very far from optimized. 1000x or 10 000x improvements are possible, less for math heavy stuff.
eh8 3y ago
This is especially true in UX design. When I see a website... - using a ridiculously thin or small font, - relying on a high-end monitor to provide sufficient color contrast - loading an unreasonable amount of resources only to provide laggy animations ...I'm wondering if the responsible designer(s) only have 32-in Retina displays and the latest Macbooks to work with. Because on any other combination of devices, the website looks and feels awful. And I know this because I was formerly guilty of it!
- rqtwteye 3y ago
  “ I'm wondering if the responsible designer(s) only have 32-in Retina displays and the latest Macbooks to work with. Because on any other combination of devices, the website looks and feels awful.” I think often it’s that they aren’t users themselves. They make it “pretty” but not functional.
  - pmontra 3y ago
    It's extremely rare that UX designers are also users of the products they develop. Maybe they use Figma or the order entry of Amazon even if they don't work for Figma or Amazon, but who of them is going to use again the order entry form for Random Customer N after they started working on the registration form of Random Customer N+1?
    - spaniard89277 3y ago
      My GF is a designer and she always says that the problem is that nobody test anything. I've been helping with a project for her client and pretty much everything went on the fly. I was able to spot obvious flaws in the design, she agreed but said that she had no time and such is life.
- famahar 3y ago
  There are so many tools to see how a website looks like on multiple screens / devices. Even full on emulation. I can see a designer making this oversight but a UX designer doing that kinda makes the UX part of the title irrelevant.
tkgally 3y ago
> I'm wondering if humans are mostly incapable of producing great things without (artifical) restrictions. Here is an opinion to support your hypothesis from a couple of different domains: poetry and music. While some people prefer free verse and avante-garde music, what stays most in my mind, and what seems to endure longest overall, are poetry with regular rhyme and meter and music that follows standard patterns of melody, rhythm, and harmony. Having to force their creativity into those sometimes rigid frameworks seems to enable many artists to produce better works.
- nottheengineer 3y ago
  Maybe a counterexample to it, not sure if it can be applied: I write software in ABAP, which is a weird and ridiculously complicated language that has inline SQL and type checking against the database and has never had a major version that breaks old stuff, so code from 30 years ago will (and does) still run. I used to have fun working around the quirks of it and finding solutions that work within the limitations, but now I'm just frustrated by having to solve problems that haven't been around for the past 15 years in the rest of world and looking at terrible code that can't be made any nicer because of those limitations or because customers don't give a shit as long as it works most of the time.
keyle 3y ago
This is very true. And as a developer or a team, you're bound by how long development takes, not by the required resources. You won't be asked by a business stakeholder "oh, and how much RAM does it take?" or "why is it $2,000 a month instead of $1,000?". These questions tend to come much later when profit needs to be ironed out.
- _cdl5 3y ago
  And later, when performance becomes important, it is often much harder to improve than early on. Especially with legacy db schemas with a lot of existing customer data.
cglee 3y ago
Interesting observation and aligns with my experience of really enjoying small focused tools and apps. This website is a good example. Further, it feels like there's a corollary here to companies, where financially constrained companies who are smaller and more focused provide better customer experience than cash-flush competitors.
nottheengineer 3y ago
Good point about not hitting limits individually. I think microsoft has a huge problem with this. Even 3000$ laptops from 5 years ago struggle with running a teams call, some office instances and a browser with 30 tabs at the same time without slowing down to unacceptable levels. They test stuff individually and running one thing alone is fine, but that's not what people do. I'd imagine that artificial limits in the form of run time on well-defined hardware that are only raised after an explicit decision could be the solution to this. But then again I only write business software where the performance aspect comes down to "don't do stupid shit with the database and don't worry about the rest because the client won't pay for those worries", so I might be on the wrong track entirely.
roughly 3y ago
> I'm wondering if humans are mostly incapable of producing great things without (artifical) restrictions. On the one hand, I agree with this. I think an awful lot of great art and great work comes from the enforced genius of operating within constraints, and there's a profound feeling that comes from recognizing that kind of brilliance. I'll also say, though, that there's also something about seeing the results of absolutely turning every nob to 11 - about seeing the absolute unfettered apex of what particularly talented people can actually do with no constraints whatsoever. It's a very different experience, and I deeply respect the genius of making art under constraint, but sometimes you've just gotta put a dude on the moon, you know?
- mfru 3y ago
  > sometimes you've just gotta put a dude on the moon, you know? in software development it looks like everyone and their grandmother is sending people to all moons known to mankind, though.
  - namaria 3y ago
    >in software development it looks like everyone and their grandmother is sending people to all moons known to mankind, though. And crash landing every time. To do a far reaching soft landing you need to have learned to reach LEO within constraints.
pjerem 3y ago
Oh yes ! That’s my pet theory too. I think it’s why old computers felt good and also why old games were so good. Maybe it have something to do with the complexity of the systems we deal with. When you have a restricted amount of some resource (RAM, physical space, food, materials, time, money …) you have to plan how you will use it. You are forced to be smart. When you have a virtually infinite resource, you can make whatever you feel making but you don’t have to really care about the final state, you just start and you’ll see when it will work. I’m not exactly a true gamer, but I’ve always been amazed by the fact humans were capable to store so much emotion, adventures and time to enjoy in the good old cartridges with some kb/mb of rom. I mean, Ocarina Of Time rom is just the size of the last 8 photos I took with my iPhone.
- prox 3y ago
  The guy who made Virtualdub (virtualdub.org) has a blog who said essentially that. His video program is supersmall because it doesn’t use 4 packaged libraries; he programmed everything to hardware / OS interfaces directly.
yuhong 3y ago
[dead]
crote 3y ago
It is mostly a matter of priorities. For most applications it simply does not make any sense to spend this much time on relatively small optimizations. If you can choose to either buy 32GiB of RAM for your server for less than $50 or spend probably over 40 hours of developer time at at least $20 / hour, it is quite obvious which one makes more sense from a business perspective. Not to mention that the website was offline for an entire week - that alone would've killed most businesses! A lot of tech people really like doing such deep dives and would happily spend years micro-optimizing even the most trivial code, but endless "yak shaving" isn't going to pay any bills. When the code runs on a trivial number of machines, it probably just isn't worth it. Not to mention that such optimizations often end up in code which is more difficult to maintain. In my opinion, a lot of "software bloat" we see these days for apps running on user machines comes from a mismatch between the developer machine and the user machine. The developer is often equipped with a high-end workstation as they simply need those resources to do their job, but they end up using the same machine to do basic testing. On the other hand, the user is running it on a five-year-old machine which was at best mid-range when they bought it. You can't really sell "we can save 150MB of memory" to your manager, but you can sell "saving 150MB of memory will make our app's performance go from terrible to borderline for 10% of users".
- vincnetas 3y ago
  So i guess that is the point GP is making. From practical perspective one would just spend 50$ on RAM and forget about it. But you miss opportunity to make something great in terms of algorithm improvements for example. Even if it costs you more. SO here artificial constraint is that "you cant have more RAM" and so you need to find other more creative solutions.
  - uoaei 3y ago
    It also costs everyone else more to run the inefficient code that you produce.
  - crote 3y ago
    But OP could have more RAM, and the end user doesn't care about how clever the algorithm is. They only care that the service is offline for a week while the developer is having fun implementing their new toy algorithm. Working on improvements like this makes a lot of sense in academia, when you are running thousands of servers, or when you need so much memory that you can't buy a big enough server. It's a nice backlog item for when there is literally nothing else to do. I have definitely fallen into this rabbit hole myself. Solving difficult problems with clever solutions is a lot of fun! But my manager - rightfully - chastised me for it because from a business PoV it was essentially a waste of time and money. Fine for a hobby project, not so much when you are trying to run a business.
marginalia_nu 3y ago
Yeah this aligns with my view. Limitations breed ingenuity, and that isn't limited to demo scene outputs. You're going to run into scaling problems sooner or later, and they're a lot easier to deal with early than late. If your software runs well on a raspberry pi[1], it's going to be absurdly performant on a real server. It's actually how we used to build software. It's why we could have an entire operating system perform well on a machine like a Pentium 1 with most of what you'd expect today, etc. while at the same time we have web pages that struggle to scroll smoothly on a smartphone with literally a thousand times more resources across all axes. The Word 95 team were constantly faced with limits and performance tradeoffs, and it very clearly worked or did not. If I had just gone and added more RAM (or whatever), I would still have been stuck with an inferior design, and soon enough I would need to buy even more RAM. The crazy part about this change is that it isn't just reducing the resource utilization, it's actually making the system more capable, and faster because free RAM means more disk caching. [1] e.g. this runs on a single pi, and is much faster than production wikipedia because it doesn't permit updates: https://encyclopedia.marginalia.nu/article/Hacker_News https://encyclopedia.marginalia.nu/article/Hacker_News
isaacremuant 3y ago
Without going about efficiency and priorities. I think it's easy enough to claim that a great way to spark creativity or great solutions is putting constraints. It's about specialization around the usage of few elements to achieve a goal vs a paradox of choice or going through common and known patterns. Jams can be great for this and people realize they can work so much more efficiently and focus on the core of their idea.
deleted 3y ago
[deleted]
maxweylandt 3y ago
There's definitely some artist who feel that way. Jack White comes to mind. He'll deliberately use restrictions(like writing music for only two instruments) and even physically obstruct things at live performances. See this (very good) interview with Conan: https://m.youtube.com/watch?t=890&v=AJgY9FtDLbs&feature=youtu.be https://m.youtube.com/watch?t=890&v=AJgY9FtDLbs&feature=yout...
uoaei 3y ago
I completely agree. Every creative I've ever trusted has the same philosophy: freedom through constraints. I've found in my life, too, I can focus more closely on elegant solutions when they become (perhaps artificially) necessary, not merely aesthetically pleasing. I'm actually having a similar experience of insane efficiency improvements in a personal project, much smaller in scope, that came down to using bit operations and as-branchless-as-possible methods for an Arduino Nano.
globular-toast 3y ago
It would probably help if the whole "developer spec" thing would go away. I never understood why people think they need 32GB of RAM and top of the line CPU to write code. If you're compiling a lot (especially C++ I guess) then you need a build server. I wonder how much better things would be if "developer spec" actually meant something close to median or representative spec.
boredumb 3y ago
I very often use my highspeed 3g option in the network tab when developing web UIs to give myself some serious constraints instead of assuming everyone is using a developer workstation.

by gnyman 3y ago

jancsika 3y ago

> I'm wondering if humans are mostly incapable of producing great things without (artifical) restrictions. I think the real issue is that there isn't a program language that produces a compiler error if the given code can exceed a maximum specified latency. Even working on a program with soft-realtime scheduling, I've had to constantly push back against patches that introduce some obscure convenience without having measured worst case latency. The problem is so bad I doubt most people realize it's there. I don't know what the answer is, but I have the feeling there's an intersection with timing attacks on software/hardware. Some kind of tooling that makes both worst case times and variance as visible as the computed CSS in devTools would probably help. Added to some kind of static analysis, perhaps devs to hack their way to decently responsive interfaces and services.

MichaelZuo 3y ago

American Airlines ran SABRE, a sizeable airline ticketing and reservation system, in the mid-1970s on two system/360 mainframes that could only process a few tens of millions of instructions per second. A raspberry pi 2 can do over 4 billion Dhrystone instructions per second, and a pi 4 over 10 billion per second. Of course by modern standards mid-1970s SABRE was pretty barebones for an airline's main system, but it's at least theoretically possible to run simplified systems for over a 100 airlines simultaneously on a single pi 2... So yes modern programs are very far from optimized. 1000x or 10 000x improvements are possible, less for math heavy stuff.

eh8 3y ago

This is especially true in UX design. When I see a website... - using a ridiculously thin or small font, - relying on a high-end monitor to provide sufficient color contrast - loading an unreasonable amount of resources only to provide laggy animations ...I'm wondering if the responsible designer(s) only have 32-in Retina displays and the latest Macbooks to work with. Because on any other combination of devices, the website looks and feels awful. And I know this because I was formerly guilty of it!

rqtwteye 3y ago
“ I'm wondering if the responsible designer(s) only have 32-in Retina displays and the latest Macbooks to work with. Because on any other combination of devices, the website looks and feels awful.” I think often it’s that they aren’t users themselves. They make it “pretty” but not functional.
- pmontra 3y ago
  It's extremely rare that UX designers are also users of the products they develop. Maybe they use Figma or the order entry of Amazon even if they don't work for Figma or Amazon, but who of them is going to use again the order entry form for Random Customer N after they started working on the registration form of Random Customer N+1?
  - spaniard89277 3y ago
    My GF is a designer and she always says that the problem is that nobody test anything. I've been helping with a project for her client and pretty much everything went on the fly. I was able to spot obvious flaws in the design, she agreed but said that she had no time and such is life.
famahar 3y ago
There are so many tools to see how a website looks like on multiple screens / devices. Even full on emulation. I can see a designer making this oversight but a UX designer doing that kinda makes the UX part of the title irrelevant.

tkgally 3y ago

> I'm wondering if humans are mostly incapable of producing great things without (artifical) restrictions. Here is an opinion to support your hypothesis from a couple of different domains: poetry and music. While some people prefer free verse and avante-garde music, what stays most in my mind, and what seems to endure longest overall, are poetry with regular rhyme and meter and music that follows standard patterns of melody, rhythm, and harmony. Having to force their creativity into those sometimes rigid frameworks seems to enable many artists to produce better works.

nottheengineer 3y ago
Maybe a counterexample to it, not sure if it can be applied: I write software in ABAP, which is a weird and ridiculously complicated language that has inline SQL and type checking against the database and has never had a major version that breaks old stuff, so code from 30 years ago will (and does) still run. I used to have fun working around the quirks of it and finding solutions that work within the limitations, but now I'm just frustrated by having to solve problems that haven't been around for the past 15 years in the rest of world and looking at terrible code that can't be made any nicer because of those limitations or because customers don't give a shit as long as it works most of the time.

keyle 3y ago

This is very true. And as a developer or a team, you're bound by how long development takes, not by the required resources. You won't be asked by a business stakeholder "oh, and how much RAM does it take?" or "why is it $2,000 a month instead of $1,000?". These questions tend to come much later when profit needs to be ironed out.

_cdl5 3y ago
And later, when performance becomes important, it is often much harder to improve than early on. Especially with legacy db schemas with a lot of existing customer data.

cglee 3y ago

Interesting observation and aligns with my experience of really enjoying small focused tools and apps. This website is a good example. Further, it feels like there's a corollary here to companies, where financially constrained companies who are smaller and more focused provide better customer experience than cash-flush competitors.

nottheengineer 3y ago

Good point about not hitting limits individually. I think microsoft has a huge problem with this. Even 3000$ laptops from 5 years ago struggle with running a teams call, some office instances and a browser with 30 tabs at the same time without slowing down to unacceptable levels. They test stuff individually and running one thing alone is fine, but that's not what people do. I'd imagine that artificial limits in the form of run time on well-defined hardware that are only raised after an explicit decision could be the solution to this. But then again I only write business software where the performance aspect comes down to "don't do stupid shit with the database and don't worry about the rest because the client won't pay for those worries", so I might be on the wrong track entirely.

roughly 3y ago

> I'm wondering if humans are mostly incapable of producing great things without (artifical) restrictions. On the one hand, I agree with this. I think an awful lot of great art and great work comes from the enforced genius of operating within constraints, and there's a profound feeling that comes from recognizing that kind of brilliance. I'll also say, though, that there's also something about seeing the results of absolutely turning every nob to 11 - about seeing the absolute unfettered apex of what particularly talented people can actually do with no constraints whatsoever. It's a very different experience, and I deeply respect the genius of making art under constraint, but sometimes you've just gotta put a dude on the moon, you know?

mfru 3y ago
> sometimes you've just gotta put a dude on the moon, you know? in software development it looks like everyone and their grandmother is sending people to all moons known to mankind, though.
- namaria 3y ago
  >in software development it looks like everyone and their grandmother is sending people to all moons known to mankind, though. And crash landing every time. To do a far reaching soft landing you need to have learned to reach LEO within constraints.

pjerem 3y ago

Oh yes ! That’s my pet theory too. I think it’s why old computers felt good and also why old games were so good. Maybe it have something to do with the complexity of the systems we deal with. When you have a restricted amount of some resource (RAM, physical space, food, materials, time, money …) you have to plan how you will use it. You are forced to be smart. When you have a virtually infinite resource, you can make whatever you feel making but you don’t have to really care about the final state, you just start and you’ll see when it will work. I’m not exactly a true gamer, but I’ve always been amazed by the fact humans were capable to store so much emotion, adventures and time to enjoy in the good old cartridges with some kb/mb of rom. I mean, Ocarina Of Time rom is just the size of the last 8 photos I took with my iPhone.

prox 3y ago
The guy who made Virtualdub (virtualdub.org) has a blog who said essentially that. His video program is supersmall because it doesn’t use 4 packaged libraries; he programmed everything to hardware / OS interfaces directly.

yuhong 3y ago

[dead]

crote 3y ago

It is mostly a matter of priorities. For most applications it simply does not make any sense to spend this much time on relatively small optimizations. If you can choose to either buy 32GiB of RAM for your server for less than $50 or spend probably over 40 hours of developer time at at least $20 / hour, it is quite obvious which one makes more sense from a business perspective. Not to mention that the website was offline for an entire week - that alone would've killed most businesses! A lot of tech people really like doing such deep dives and would happily spend years micro-optimizing even the most trivial code, but endless "yak shaving" isn't going to pay any bills. When the code runs on a trivial number of machines, it probably just isn't worth it. Not to mention that such optimizations often end up in code which is more difficult to maintain. In my opinion, a lot of "software bloat" we see these days for apps running on user machines comes from a mismatch between the developer machine and the user machine. The developer is often equipped with a high-end workstation as they simply need those resources to do their job, but they end up using the same machine to do basic testing. On the other hand, the user is running it on a five-year-old machine which was at best mid-range when they bought it. You can't really sell "we can save 150MB of memory" to your manager, but you can sell "saving 150MB of memory will make our app's performance go from terrible to borderline for 10% of users".

vincnetas 3y ago
So i guess that is the point GP is making. From practical perspective one would just spend 50$ on RAM and forget about it. But you miss opportunity to make something great in terms of algorithm improvements for example. Even if it costs you more. SO here artificial constraint is that "you cant have more RAM" and so you need to find other more creative solutions.
- uoaei 3y ago
  It also costs everyone else more to run the inefficient code that you produce.
- crote 3y ago
  But OP could have more RAM, and the end user doesn't care about how clever the algorithm is. They only care that the service is offline for a week while the developer is having fun implementing their new toy algorithm. Working on improvements like this makes a lot of sense in academia, when you are running thousands of servers, or when you need so much memory that you can't buy a big enough server. It's a nice backlog item for when there is literally nothing else to do. I have definitely fallen into this rabbit hole myself. Solving difficult problems with clever solutions is a lot of fun! But my manager - rightfully - chastised me for it because from a business PoV it was essentially a waste of time and money. Fine for a hobby project, not so much when you are trying to run a business.

marginalia_nu 3y ago

Yeah this aligns with my view. Limitations breed ingenuity, and that isn't limited to demo scene outputs. You're going to run into scaling problems sooner or later, and they're a lot easier to deal with early than late. If your software runs well on a raspberry pi[1], it's going to be absurdly performant on a real server. It's actually how we used to build software. It's why we could have an entire operating system perform well on a machine like a Pentium 1 with most of what you'd expect today, etc. while at the same time we have web pages that struggle to scroll smoothly on a smartphone with literally a thousand times more resources across all axes. The Word 95 team were constantly faced with limits and performance tradeoffs, and it very clearly worked or did not. If I had just gone and added more RAM (or whatever), I would still have been stuck with an inferior design, and soon enough I would need to buy even more RAM. The crazy part about this change is that it isn't just reducing the resource utilization, it's actually making the system more capable, and faster because free RAM means more disk caching. [1] e.g. this runs on a single pi, and is much faster than production wikipedia because it doesn't permit updates: https://encyclopedia.marginalia.nu/article/Hacker_News https://encyclopedia.marginalia.nu/article/Hacker_News

isaacremuant 3y ago

Without going about efficiency and priorities. I think it's easy enough to claim that a great way to spark creativity or great solutions is putting constraints. It's about specialization around the usage of few elements to achieve a goal vs a paradox of choice or going through common and known patterns. Jams can be great for this and people realize they can work so much more efficiently and focus on the core of their idea.

deleted 3y ago

[deleted]

maxweylandt 3y ago

There's definitely some artist who feel that way. Jack White comes to mind. He'll deliberately use restrictions(like writing music for only two instruments) and even physically obstruct things at live performances. See this (very good) interview with Conan: https://m.youtube.com/watch?t=890&v=AJgY9FtDLbs&feature=youtu.be https://m.youtube.com/watch?t=890&v=AJgY9FtDLbs&feature=yout...

uoaei 3y ago

I completely agree. Every creative I've ever trusted has the same philosophy: freedom through constraints. I've found in my life, too, I can focus more closely on elegant solutions when they become (perhaps artificially) necessary, not merely aesthetically pleasing. I'm actually having a similar experience of insane efficiency improvements in a personal project, much smaller in scope, that came down to using bit operations and as-branchless-as-possible methods for an Arduino Nano.

globular-toast 3y ago

It would probably help if the whole "developer spec" thing would go away. I never understood why people think they need 32GB of RAM and top of the line CPU to write code. If you're compiling a lot (especially C++ I guess) then you need a build server. I wonder how much better things would be if "developer spec" actually meant something close to median or representative spec.

boredumb 3y ago

I very often use my highspeed 3g option in the network tab when developing web UIs to give myself some serious constraints instead of assuming everyone is using a developer workstation.