8 ms·

Stone knives and bear skins – there is no money in tools

69 points by sizzle 3y ago

ggm 3y ago
There no money in tools: The Dutch company ASML would like to talk to you. The article does actually say that in highly specialised niches tool making is fantastically lucrative. So, don't mistake the headline for ground truth. Also there is good reason to believe tool making since flint knapping has been high status, and demands surplus production to liberate skilled labour to making tools which have immensely high use value: tools are magnifiers of human productivity and are worth a lot. The point is that when commodities are cheap, and tools like gdb can be considered a commodity, the relative advantage of "a better one" has to be weighed up with using a screwdriver as a hammer. Thus printf() debugging.
- JohnFen 3y ago
  Well said. I know there's money in tools because I've been making money making and selling tools for a long time. But not every kind of tool. A gdb replacement would be a hard sell unless it managed to actually perform miracles. But specialty tools of the sort that can save a relatively small group of devs major pain and/or time? There's good money in that for a small business. That the market is too small to be attractive to large companies is just gravy.
  - sanxiyn 3y ago
    Time-travel seems miraculous to me, but uptake seems very slow to me. I hope https://www.replay.io/ https://www.replay.io/ succeeds.
    - Veserv 3y ago
      Commercially available time travel debugging is over 20 years old at this point [1]. So yeah, the uptake is pretty slow. [1] http://jakob.engbloms.se/archives/1564 http://jakob.engbloms.se/archives/1564
  - coldtea 3y ago
    >Well said. I know there's money in tools because I've been making money making and selling tools for a long time Couldn't a succesful actor say the same? "I know there's money in acting, because I've been making money acting for a long time". And yet we'd instantly understand that doesn't refute the sense the statement was meant, and that acting isn't really the field you wanna pursue if you want to make money". Same for any other niche endeavor, where there are thousands more lucrative fields and niches, and much fewer succesful examples compared to other areas.
    - smallnamespace 3y ago
      > acting isn't really the field you wanna pursue if you want to make money That depends on the person. By the same token, software engineering isn’t the right field for the average person (because most people don’t have the knack for coding), and yet some people find they do have that talent. A career is like dating, it’s mostly about finding a good fit, which means you also need to know yourself. Once you do, you may find a ‘bad’ niche is actually the right one for you.
      - 7e 3y ago
        No, it depends on supply and demand. Lots of people want to act. Lots of people want to make tools.
        s1artibartfast 3y ago
        And some are successful.
        coldtea 3y ago
        Which is irrelevant when evaluating a job market. Some are succesful in any crap market, where the chances are that most will go under or utterly fail, and where the conditions and average salaries and profit margins and such are attrocious. "Some are succesful" as some kind of retort basically translate to "I don't care about market conditions, I have confidence in myself and I know I'll be one of the succesful ones". Which is all well and good, but it doesn't say anything about one's chances of success, or that job market's behavior in general. Besides, historically it's what every fresh-eyed kid fresh off the bus from Nebraska, still waiting tables 20 years later, believed too.
  - rhelz 3y ago
    All a gdb replacement has to do to succeed is not require the binary to be codesigned.
  - funcDropShadow 3y ago
    What kind of tools do you build? I'd be interested in your opinion on what are the preconditions for tools to profitable.
  - fmajid 3y ago
    IDA Pro competes with gdb and Ghidra and seems profitable enough: https://hex-rays.com/IDA-pro/ https://hex-rays.com/IDA-pro/ Then you have the people who sold shovels and pickaxes to prospectors during the Gold Rush
- 36870059 3y ago
  ASML's crown jewels are not mere "tools", they are machines. It is bleeding edge technology and entirely irrelevant to a general discussion of economics of 'tool makers'. https://www.youtube.com/watch?v=5Ge2RcvDlgw https://www.youtube.com/watch?v=5Ge2RcvDlgw
  - ggm 3y ago
    and GDB isn't a machine?
    - 36870059 3y ago
      Anyone can make a GDB. Haven't you noticed? Superpowers are fighting over ASML..
      - ggm 3y ago
        computers & software have been subject to ITAR regulations since decades. The implication is that superpowers are fighting over code.
  - nwiswell 3y ago
    In the industry they are very literally called "tools". No one calls them "machines". E.g., "the fab is ordering another six EUV tools". Similarly a lathe or vertical mill is "a machine tool". They're ideally suited for a very specific task in the process of making a thing. It's possible for something to be both a machine and a tool. If we're being pedantic, the ASML tools (and indeed any modern wafer fabrication equipment) are also robots. More related to the business of tool making broadly, however, the entire industry of semiconductor wafer fabrication equipment (WFE) is quite profitable and margins tend to be generous. Much of the industry is very technical and does involve "bleeding edge technology", but not all of it. The publicly traded Japanese company Disco makes tools for wafer grinding[1], which are not really any more complex than your typical computer-controlled metal cutting machine tools, yet they are quite profitable[2]. Just look at those headline margin figures! It seems to be a matter of niche and competition. [1]: https://www.disco.co.jp/eg/products/grinder/dfg8560.html https://www.disco.co.jp/eg/products/grinder/dfg8560.html [2]: https://www.disco.co.jp/eg/ir/library/doc/film/20230420.pdf https://www.disco.co.jp/eg/ir/library/doc/film/20230420.pdf
- trollerator23 3y ago
  In software tools. Hardware tools are hardware.
  - PostOnce 3y ago
    There are lots of profitable software tool companies, aren't there? Atlassian, JetBrains, Microsoft, most of them are smaller than this but still "money"; I live in open-source land, but everyone else I know works somewhere that spends money on all kinds of tools. It's just that the fruit in software tools doesn't hang so low, I think.
- evrimoztamur 3y ago
  If your tool is an integral part of a larger chain, as is the case for ASML's machines being in the chain of silicon chips, that's a different story than a tool which simply eases the process by a small factor. UV lithography is the only way to make advanced chips required for various other tools and appliances, but an OS debugger (as the author puts it) is just a nicer printf. There's big money in one, and small money in the other.
- coldtea 3y ago
  >There no money in tools: The Dutch company ASML would like to talk to you. "You made a general argument. Here's a single counter-example, on a very tiny niche, compared to thousands of examples in your favor, making much more money in other areas of IT".
  - Dalewyn 3y ago
    I mean, you're using a computer to shitpost here. Everything in that computer, let alone the computer as a whole, is a tool that made its manufacturer and any vendors involved lots of money.
    - coldtea 3y ago
      Yeah, I also wear clothes (kids go into garments/fashion), drink coffee (kids start coffee shops), and buy many other items besides. By this logic every market is a lucrative one, if people buy stuff from it. So, just go and build computers, companies make money selling them. Start the new Apple or Dell. It's almost as if this missed the point of TFA entirely, and as if "things are sold in that field and there are some player making lots of money in it" is useless by itself for evaluating a field/niche regarding to its money making opportunities. Let me introduce you to a wonderful concept: https://en.wikipedia.org/wiki/Opportunity_cost https://en.wikipedia.org/wiki/Opportunity_cost Not bad for shitposting eh?
  - PostOnce 3y ago
    Here's another multibillion dollar example https://en.wikipedia.org/wiki/Lam_Research https://en.wikipedia.org/wiki/Lam_Research
  - ggm 3y ago
    The article specifically reaches into the whole "spanners are too cheap to be worth making" space. The article didn't say "except of course, those gold plated NASA spanners" or "ok: so this is a broad brush but there are notable exceptions" I called out an exception. I like the article. I think the point is well made. I would say it a bit differently: commodities are very hard to make profitably once the exemplar is good enough, and cheap enough. If you seek higher utility then don't be surprised if the people who use your Tool/Machine/Robot balk at the price, when their goal is met by the lower utility commodity. If you don't care about lead and tolerance, those fake resistors are just fine for hobbyist use. If your project needs lead free and 0.001% then you really need to pay the premium for originals, not fakes. And ASML is a niche, but an example of a generic problem in their line of reasoning. For almost any "industry" you care to mention from coathanger bending to lobster fishing, there is somebody making good profit selling what the coathanger-wrangler, or fisherman would call "a tool" ie: cute argument, but actually... sorry.. niches are everywhere.
sanxiyn 3y ago
Being against a kernel debugger is one of the worst decision Linux ever made. Thankfully those days are behind us. https://lwn.net/Articles/270089/ https://lwn.net/Articles/270089/ Linus Torvalds, in paraphrase: > I happen to believe that stone knives and bear skins force people to think about their problem on a different level. This is exactly as ridiculous as it sounds.
- JohnFen 3y ago
  Yeah, I think he overstates the case. I do see people who have become overly dependent on debuggers and that has adversely affected their debugging skills -- but that isn't an indictment of debuggers.
- kbenson 3y ago
  It's ridiculous, but at the simplest level, it's not wrong. Your tools or the lack thereof will change how you think about things, for better or worse depending on the situation. If your goal is putting venison on the table tonight, vs understanding deer behavior and herd migration and when deer are even in the area in enough numbers and so can even be thought of as a viable food source, a stone knife and a bear skin as tools might promote (or force through necessity) better understanding than a hunting rifle and a large puffy jacket. That doesn't mean it's "better" or that everyone should be using shitty tools, but it is food for thought, and that perhaps doing it a few times is a learning experience that isn't all negative.
- m463 3y ago
  kernel debugging on linux sucks (IMHO) I've tried it and it is really hard to get even the most basic debug output - a symbol'd stack trace of a crash. If you can't get it at runtime, you need a kernel dump, and trying to get one is a mess. Once you have it, it is hard to match up the dump to the symbols. I believe some of the tools were kernel-version specific and I think when kernel 5 came out they didn't work.
  - worthless-trash 3y ago
    I have run "crash" against a live kernel, i have run systemtap against a live kernel, kprobes and written my own C debugging linux kernel issues. I can't speak for other distros, but you install the relevant debuginfo(symbol files) and then use your tooling, and its been pretty straight forward, at least for the last 10 years. Hit me up if you ever need assistance doing it on standard x86-64, I'll do what i can to help.
honeybadger1 3y ago
While this essay serves as an interesting critique of the current state of operating systems and their development, it perhaps falls prey to the very flaw it seeks to highlight - the tendency to oversimplify the complexities inherent in systems software development and to yearn for a 'perfect' solution that is largely a figment of the author's imagination.
m463 3y ago
I think there are a few things at work. There is no money in tools because employers (and to some extent employees) don't value them. Also, folks who make tools maybe value them too much? And maybe tools are too easy to copy? I remember decades ago when purify came out. I tried the demo at work on a project I was working on. Within a few hours, I found an enormous number memory errors. It was enormously helpful. Purify had a high per-seat license cost, and work wouldn't buy it. Many current employers are a little smarter about things, but I mainly see things like code coverage tools and version control systems, but no real spending on more personal/fashionable tools, even if employees are more efficient. I think tools that might work better would be those employees bought them themselves - think jetbrains - and priced accordingly. The analogy might be auto mechanics building up their collection from snap-on. also, warts and all, linux is a nice environment because you can get to the bottom of most things and tools might be less industrial quality, but there are plenty of them.
- neilv 3y ago
  Purify was glorious. As was Saber-C before it. My employer paid for both. But we sold even more expensive development software ourselves, and bought very expensive workstation and server hardware to run it on, so we were accustomed to the price points. I like that great software is now open source, and most any kid or adult around the world can run it on a cheap decade-old PC. But I do miss some things about that software development era.
- Veserv 3y ago
  Employers do not value tools because the employees say the cost-benefit is not worth it. The problem is that the employees do the cost-benefit calculations remembering their time as a poor student when their time was worthless. When your time is worth $0, a product must be 10x better to be worth paying 10x as much. When your time costs $200,000/yr, a 1% improvement is trivially worth $2,000/yr. People in software do not internalize this because we can build our own tools. We can solve the problem given time unlike say a chemist who can not build a reaction vessel. When we were poor, we built our own tools and we keep that habit even after we have money and our time is valuable. It is absolutely ridiculous; the average janitor spends more on tools than a software developer. Truck drivers, secretarys, artists, mechanics, all spend more on tools even though their time is significantly less costly on average. In every other engineering profession companys literally spend $30,000, $100,000, and more a year keeping their high value workers productive. Only in software do we think that $3,000/yr for Visual Studio, the tool that you literally use the entire time when doing Windows development, or $99 per year to do iOS development are “steep” costs. $99 is like one hour of time; it is ludicrously cheap. $25,000/yr for a 25% productivity improvement is a no-brainer. The problem is that software people think it is obviously wrong, but the business people think it is obviously right. We do not need to struggle with hand trowels, the smart business people want to give us excavators, we just need to stop telling them no.
version_five 3y ago
> A venture capitalist once told me, “There is no money in tools.” That suggests he's saying there is not upside in a portfolio of tool companies. Following average VC advice gives average VC results. If everyone listened to this kind of stuff we'd be awash is generic SaaS CRM stuff...oh
flappyeagle 3y ago
Tools matter where UX matters. Stuff that you set up once and a computer interfaces with in an automated fashion has no UX. That's almost the entire premium of every software tool in existence.
RagnarD 3y ago
Look at the catalog of ComponentSource (https://www.componentsource.com https://www.componentsource.com), which is mostly ... tools. Are Jetbrains and Progress (to name two offhand) doing badly?
thaumaturgy 3y ago
I think about this often because it feels like my peak ability and productivity s a developer happened back on classic MacOS (7.5 through 9). There was a program called MacsBug that you could install, and with just two keys on the keyboard, invoke a stop-the-world debugger, complete with disassembly, stepping, and heap inspection. You could twiddle a bit here and there, even in the operating system. MacsBug shipped with a Forth interpreter, if I remember right, so you could even automate some of the debugging process. I still have my MacsBug reference guide. I don't think there's a day that goes by that I don't wish I had those capabilities again. But, operating systems also have another important function: they have to defend against malicious software. In the days of Macsbug, every program had full read/write access to every other program's heap. That would be horrifying today. Every tactic that's been developed in the name of secure computing has made OS-level debugging significantly harder.
- dentalperson 3y ago
  Thanks for reminding me about this. MacsBug was great. Besides being a useful development tool, MacsBug was used extensively to break shareware as well by the mac warez community. I remember seeing some guides about it that were very illuminating about how program execution and object code actually worked that made things click for me. It was also a time when any consumer could issue a system interrupt with the same two keystrokes because it was built into mac OS, and see the cpu registers and enter hex to change the program flow. Since there was not as clean a system/program barrier as there was today and single threaded OS (I think?), before OS 8 the system froze all the time because of apps, and even non-expert users memorized the hex code for the system exit() trap.
  - thaumaturgy 3y ago
    0xA9F4, if memory serves.
deleted 3y ago
[deleted]
guidoism 3y ago
> The code used to build operating systems has only the most primitive of data types (lists, hashes, the occasional tree), while the libraries used in applications are a veritable cornucopia of modern data types. Really? Is this only Linux (which gets a lot of eyeballs)? Mac OS? Maybe he's only talking about the kernel?
- MobiusHorizons 3y ago
  He is definitely talking about the kernel (as well as things that run in kernel space like drivers). Userspace is all applications from this perspective, and can be debugged with a normal debugger.
n4te 3y ago
I like writing tools and make all my money doing that. It's a niche market, but it's mine and worked out amazingly well.
sky_rw 3y ago
It's worth noting that when a VC says "there's no money in X", their definition of what constitutes "money" is very different from a normal persons. It's more accurate to say "there are no wildly market-inefficient profit margins to capitalize on and make hundreds of millions from in tools".
Animats 3y ago
If you use an OS with a tiny, well-tested microkernel, you debug drivers and such with the user-level debug tools. I wrote FireWire camera drivers for QNX debugging them that way, and doing compiles on the same machine upon which I was testing. Without rebooting. That's not new. The first user-space driver development was for the Michigan Terminal System on the System/360 model 67. That was the first computer with virtualization hardware. People have been doing this in IBM land for over half a century.
rhelz 3y ago
The founders at a tool-making startup I worked for came back from a sales meeting one day looking very sad. I asked them why the long faces. They said they pitched the tool as something which would save their engineers a lot of pain. "Eh?" said the prospect. "What? Who cares how much pain the engineers are in? Its their job to endure pain." Clearly, the founders hadn't fully perfected their sales pitch....
AtlasBarfed 3y ago
This is very simple. Tools in the "real world" generally work on one protocol: physics. Yes.. there are standards and industries and the like in physical tools, but those are much more static and unchanging. Because physics and material world. A tool will generally be relevant until it wears out, and often/sometimes is useful for decades after "obsolescence". In software, where (looks at the Javascript ecosystem) things change in a year. Sometimes less. Version churn and feature churn means your tool is obsolete/useless (if you are lucky) within 3-4 years. Often less. In regular tools, it is pretty clear the distinction between a tool and the end product. But in software, a tool is SOFTWARE, with all the problems of software: version churn, feature drift, underlying OS support, and all the other quicksand. Tool markets are also inherently a fraction the endproduct market: The produced product volume is say 10x to 100x the value of the cost of the tool in the physical world before it needs replacement or wears out. Software being what it is, you can get 100000x - infinite the tool. That value proposition means the market for the tool is the inverse of the value factor of the tool. In software, you would get one tool and use it for 100000 products / views / etc. So the market for it is lower. Perplexingly to my point about software tools going out of support, is that REALLY established tools are long lived and never wear out. So a hot gee-whiz tool will be competing against well-featured tools (often free after a while because tool vendors go out of business and open source it), so there isn't much gee-whiz anymore. OpenAI is an interesting "tool". We'll see where it goes commercially. But most tools have reproducibility. AI tools do not, at least currently they don't. Free software tools seem more durable, because the user population sustains it generally with contributions, bug reports, feedback, and demand. And sometimes the minimal money to support the org. And sometimes marketing. Software is also generally about building a bandwagon. Gatekeeping tools are not good for bandwagon building. So software tools are generally free to make jumping on the bandwagon easier.

69 points by sizzle 3y ago

ggm 3y ago

There no money in tools: The Dutch company ASML would like to talk to you. The article does actually say that in highly specialised niches tool making is fantastically lucrative. So, don't mistake the headline for ground truth. Also there is good reason to believe tool making since flint knapping has been high status, and demands surplus production to liberate skilled labour to making tools which have immensely high use value: tools are magnifiers of human productivity and are worth a lot. The point is that when commodities are cheap, and tools like gdb can be considered a commodity, the relative advantage of "a better one" has to be weighed up with using a screwdriver as a hammer. Thus printf() debugging.

JohnFen 3y ago
Well said. I know there's money in tools because I've been making money making and selling tools for a long time. But not every kind of tool. A gdb replacement would be a hard sell unless it managed to actually perform miracles. But specialty tools of the sort that can save a relatively small group of devs major pain and/or time? There's good money in that for a small business. That the market is too small to be attractive to large companies is just gravy.
- sanxiyn 3y ago
  Time-travel seems miraculous to me, but uptake seems very slow to me. I hope https://www.replay.io/ https://www.replay.io/ succeeds.
  - Veserv 3y ago
    Commercially available time travel debugging is over 20 years old at this point [1]. So yeah, the uptake is pretty slow. [1] http://jakob.engbloms.se/archives/1564 http://jakob.engbloms.se/archives/1564
- coldtea 3y ago
  >Well said. I know there's money in tools because I've been making money making and selling tools for a long time Couldn't a succesful actor say the same? "I know there's money in acting, because I've been making money acting for a long time". And yet we'd instantly understand that doesn't refute the sense the statement was meant, and that acting isn't really the field you wanna pursue if you want to make money". Same for any other niche endeavor, where there are thousands more lucrative fields and niches, and much fewer succesful examples compared to other areas.
  - smallnamespace 3y ago
    > acting isn't really the field you wanna pursue if you want to make money That depends on the person. By the same token, software engineering isn’t the right field for the average person (because most people don’t have the knack for coding), and yet some people find they do have that talent. A career is like dating, it’s mostly about finding a good fit, which means you also need to know yourself. Once you do, you may find a ‘bad’ niche is actually the right one for you.
    - 7e 3y ago
      No, it depends on supply and demand. Lots of people want to act. Lots of people want to make tools.
      - s1artibartfast 3y ago
        And some are successful.
        coldtea 3y ago
        Which is irrelevant when evaluating a job market. Some are succesful in any crap market, where the chances are that most will go under or utterly fail, and where the conditions and average salaries and profit margins and such are attrocious. "Some are succesful" as some kind of retort basically translate to "I don't care about market conditions, I have confidence in myself and I know I'll be one of the succesful ones". Which is all well and good, but it doesn't say anything about one's chances of success, or that job market's behavior in general. Besides, historically it's what every fresh-eyed kid fresh off the bus from Nebraska, still waiting tables 20 years later, believed too.
- rhelz 3y ago
  All a gdb replacement has to do to succeed is not require the binary to be codesigned.
- funcDropShadow 3y ago
  What kind of tools do you build? I'd be interested in your opinion on what are the preconditions for tools to profitable.
- fmajid 3y ago
  IDA Pro competes with gdb and Ghidra and seems profitable enough: https://hex-rays.com/IDA-pro/ https://hex-rays.com/IDA-pro/ Then you have the people who sold shovels and pickaxes to prospectors during the Gold Rush
36870059 3y ago
ASML's crown jewels are not mere "tools", they are machines. It is bleeding edge technology and entirely irrelevant to a general discussion of economics of 'tool makers'. https://www.youtube.com/watch?v=5Ge2RcvDlgw https://www.youtube.com/watch?v=5Ge2RcvDlgw
- ggm 3y ago
  and GDB isn't a machine?
  - 36870059 3y ago
    Anyone can make a GDB. Haven't you noticed? Superpowers are fighting over ASML..
    - ggm 3y ago
      computers & software have been subject to ITAR regulations since decades. The implication is that superpowers are fighting over code.
- nwiswell 3y ago
  In the industry they are very literally called "tools". No one calls them "machines". E.g., "the fab is ordering another six EUV tools". Similarly a lathe or vertical mill is "a machine tool". They're ideally suited for a very specific task in the process of making a thing. It's possible for something to be both a machine and a tool. If we're being pedantic, the ASML tools (and indeed any modern wafer fabrication equipment) are also robots. More related to the business of tool making broadly, however, the entire industry of semiconductor wafer fabrication equipment (WFE) is quite profitable and margins tend to be generous. Much of the industry is very technical and does involve "bleeding edge technology", but not all of it. The publicly traded Japanese company Disco makes tools for wafer grinding[1], which are not really any more complex than your typical computer-controlled metal cutting machine tools, yet they are quite profitable[2]. Just look at those headline margin figures! It seems to be a matter of niche and competition. [1]: https://www.disco.co.jp/eg/products/grinder/dfg8560.html https://www.disco.co.jp/eg/products/grinder/dfg8560.html [2]: https://www.disco.co.jp/eg/ir/library/doc/film/20230420.pdf https://www.disco.co.jp/eg/ir/library/doc/film/20230420.pdf
trollerator23 3y ago
In software tools. Hardware tools are hardware.
- PostOnce 3y ago
  There are lots of profitable software tool companies, aren't there? Atlassian, JetBrains, Microsoft, most of them are smaller than this but still "money"; I live in open-source land, but everyone else I know works somewhere that spends money on all kinds of tools. It's just that the fruit in software tools doesn't hang so low, I think.
evrimoztamur 3y ago
If your tool is an integral part of a larger chain, as is the case for ASML's machines being in the chain of silicon chips, that's a different story than a tool which simply eases the process by a small factor. UV lithography is the only way to make advanced chips required for various other tools and appliances, but an OS debugger (as the author puts it) is just a nicer printf. There's big money in one, and small money in the other.
coldtea 3y ago
>There no money in tools: The Dutch company ASML would like to talk to you. "You made a general argument. Here's a single counter-example, on a very tiny niche, compared to thousands of examples in your favor, making much more money in other areas of IT".
- Dalewyn 3y ago
  I mean, you're using a computer to shitpost here. Everything in that computer, let alone the computer as a whole, is a tool that made its manufacturer and any vendors involved lots of money.
  - coldtea 3y ago
    Yeah, I also wear clothes (kids go into garments/fashion), drink coffee (kids start coffee shops), and buy many other items besides. By this logic every market is a lucrative one, if people buy stuff from it. So, just go and build computers, companies make money selling them. Start the new Apple or Dell. It's almost as if this missed the point of TFA entirely, and as if "things are sold in that field and there are some player making lots of money in it" is useless by itself for evaluating a field/niche regarding to its money making opportunities. Let me introduce you to a wonderful concept: https://en.wikipedia.org/wiki/Opportunity_cost https://en.wikipedia.org/wiki/Opportunity_cost Not bad for shitposting eh?
- PostOnce 3y ago
  Here's another multibillion dollar example https://en.wikipedia.org/wiki/Lam_Research https://en.wikipedia.org/wiki/Lam_Research
- ggm 3y ago
  The article specifically reaches into the whole "spanners are too cheap to be worth making" space. The article didn't say "except of course, those gold plated NASA spanners" or "ok: so this is a broad brush but there are notable exceptions" I called out an exception. I like the article. I think the point is well made. I would say it a bit differently: commodities are very hard to make profitably once the exemplar is good enough, and cheap enough. If you seek higher utility then don't be surprised if the people who use your Tool/Machine/Robot balk at the price, when their goal is met by the lower utility commodity. If you don't care about lead and tolerance, those fake resistors are just fine for hobbyist use. If your project needs lead free and 0.001% then you really need to pay the premium for originals, not fakes. And ASML is a niche, but an example of a generic problem in their line of reasoning. For almost any "industry" you care to mention from coathanger bending to lobster fishing, there is somebody making good profit selling what the coathanger-wrangler, or fisherman would call "a tool" ie: cute argument, but actually... sorry.. niches are everywhere.

sanxiyn 3y ago

Being against a kernel debugger is one of the worst decision Linux ever made. Thankfully those days are behind us. https://lwn.net/Articles/270089/ https://lwn.net/Articles/270089/ Linus Torvalds, in paraphrase: > I happen to believe that stone knives and bear skins force people to think about their problem on a different level. This is exactly as ridiculous as it sounds.

JohnFen 3y ago
Yeah, I think he overstates the case. I do see people who have become overly dependent on debuggers and that has adversely affected their debugging skills -- but that isn't an indictment of debuggers.
kbenson 3y ago
It's ridiculous, but at the simplest level, it's not wrong. Your tools or the lack thereof will change how you think about things, for better or worse depending on the situation. If your goal is putting venison on the table tonight, vs understanding deer behavior and herd migration and when deer are even in the area in enough numbers and so can even be thought of as a viable food source, a stone knife and a bear skin as tools might promote (or force through necessity) better understanding than a hunting rifle and a large puffy jacket. That doesn't mean it's "better" or that everyone should be using shitty tools, but it is food for thought, and that perhaps doing it a few times is a learning experience that isn't all negative.
m463 3y ago
kernel debugging on linux sucks (IMHO) I've tried it and it is really hard to get even the most basic debug output - a symbol'd stack trace of a crash. If you can't get it at runtime, you need a kernel dump, and trying to get one is a mess. Once you have it, it is hard to match up the dump to the symbols. I believe some of the tools were kernel-version specific and I think when kernel 5 came out they didn't work.
- worthless-trash 3y ago
  I have run "crash" against a live kernel, i have run systemtap against a live kernel, kprobes and written my own C debugging linux kernel issues. I can't speak for other distros, but you install the relevant debuginfo(symbol files) and then use your tooling, and its been pretty straight forward, at least for the last 10 years. Hit me up if you ever need assistance doing it on standard x86-64, I'll do what i can to help.

honeybadger1 3y ago

While this essay serves as an interesting critique of the current state of operating systems and their development, it perhaps falls prey to the very flaw it seeks to highlight - the tendency to oversimplify the complexities inherent in systems software development and to yearn for a 'perfect' solution that is largely a figment of the author's imagination.

m463 3y ago

I think there are a few things at work. There is no money in tools because employers (and to some extent employees) don't value them. Also, folks who make tools maybe value them too much? And maybe tools are too easy to copy? I remember decades ago when purify came out. I tried the demo at work on a project I was working on. Within a few hours, I found an enormous number memory errors. It was enormously helpful. Purify had a high per-seat license cost, and work wouldn't buy it. Many current employers are a little smarter about things, but I mainly see things like code coverage tools and version control systems, but no real spending on more personal/fashionable tools, even if employees are more efficient. I think tools that might work better would be those employees bought them themselves - think jetbrains - and priced accordingly. The analogy might be auto mechanics building up their collection from snap-on. also, warts and all, linux is a nice environment because you can get to the bottom of most things and tools might be less industrial quality, but there are plenty of them.

neilv 3y ago
Purify was glorious. As was Saber-C before it. My employer paid for both. But we sold even more expensive development software ourselves, and bought very expensive workstation and server hardware to run it on, so we were accustomed to the price points. I like that great software is now open source, and most any kid or adult around the world can run it on a cheap decade-old PC. But I do miss some things about that software development era.
Veserv 3y ago
Employers do not value tools because the employees say the cost-benefit is not worth it. The problem is that the employees do the cost-benefit calculations remembering their time as a poor student when their time was worthless. When your time is worth $0, a product must be 10x better to be worth paying 10x as much. When your time costs $200,000/yr, a 1% improvement is trivially worth $2,000/yr. People in software do not internalize this because we can build our own tools. We can solve the problem given time unlike say a chemist who can not build a reaction vessel. When we were poor, we built our own tools and we keep that habit even after we have money and our time is valuable. It is absolutely ridiculous; the average janitor spends more on tools than a software developer. Truck drivers, secretarys, artists, mechanics, all spend more on tools even though their time is significantly less costly on average. In every other engineering profession companys literally spend $30,000, $100,000, and more a year keeping their high value workers productive. Only in software do we think that $3,000/yr for Visual Studio, the tool that you literally use the entire time when doing Windows development, or $99 per year to do iOS development are “steep” costs. $99 is like one hour of time; it is ludicrously cheap. $25,000/yr for a 25% productivity improvement is a no-brainer. The problem is that software people think it is obviously wrong, but the business people think it is obviously right. We do not need to struggle with hand trowels, the smart business people want to give us excavators, we just need to stop telling them no.

version_five 3y ago

> A venture capitalist once told me, “There is no money in tools.” That suggests he's saying there is not upside in a portfolio of tool companies. Following average VC advice gives average VC results. If everyone listened to this kind of stuff we'd be awash is generic SaaS CRM stuff...oh

flappyeagle 3y ago

Tools matter where UX matters. Stuff that you set up once and a computer interfaces with in an automated fashion has no UX. That's almost the entire premium of every software tool in existence.

RagnarD 3y ago

Look at the catalog of ComponentSource (https://www.componentsource.com https://www.componentsource.com), which is mostly ... tools. Are Jetbrains and Progress (to name two offhand) doing badly?

thaumaturgy 3y ago

I think about this often because it feels like my peak ability and productivity s a developer happened back on classic MacOS (7.5 through 9). There was a program called MacsBug that you could install, and with just two keys on the keyboard, invoke a stop-the-world debugger, complete with disassembly, stepping, and heap inspection. You could twiddle a bit here and there, even in the operating system. MacsBug shipped with a Forth interpreter, if I remember right, so you could even automate some of the debugging process. I still have my MacsBug reference guide. I don't think there's a day that goes by that I don't wish I had those capabilities again. But, operating systems also have another important function: they have to defend against malicious software. In the days of Macsbug, every program had full read/write access to every other program's heap. That would be horrifying today. Every tactic that's been developed in the name of secure computing has made OS-level debugging significantly harder.

dentalperson 3y ago
Thanks for reminding me about this. MacsBug was great. Besides being a useful development tool, MacsBug was used extensively to break shareware as well by the mac warez community. I remember seeing some guides about it that were very illuminating about how program execution and object code actually worked that made things click for me. It was also a time when any consumer could issue a system interrupt with the same two keystrokes because it was built into mac OS, and see the cpu registers and enter hex to change the program flow. Since there was not as clean a system/program barrier as there was today and single threaded OS (I think?), before OS 8 the system froze all the time because of apps, and even non-expert users memorized the hex code for the system exit() trap.
- thaumaturgy 3y ago
  0xA9F4, if memory serves.

deleted 3y ago

[deleted]

guidoism 3y ago

> The code used to build operating systems has only the most primitive of data types (lists, hashes, the occasional tree), while the libraries used in applications are a veritable cornucopia of modern data types. Really? Is this only Linux (which gets a lot of eyeballs)? Mac OS? Maybe he's only talking about the kernel?

MobiusHorizons 3y ago
He is definitely talking about the kernel (as well as things that run in kernel space like drivers). Userspace is all applications from this perspective, and can be debugged with a normal debugger.

n4te 3y ago

I like writing tools and make all my money doing that. It's a niche market, but it's mine and worked out amazingly well.

sky_rw 3y ago

It's worth noting that when a VC says "there's no money in X", their definition of what constitutes "money" is very different from a normal persons. It's more accurate to say "there are no wildly market-inefficient profit margins to capitalize on and make hundreds of millions from in tools".

Animats 3y ago

If you use an OS with a tiny, well-tested microkernel, you debug drivers and such with the user-level debug tools. I wrote FireWire camera drivers for QNX debugging them that way, and doing compiles on the same machine upon which I was testing. Without rebooting. That's not new. The first user-space driver development was for the Michigan Terminal System on the System/360 model 67. That was the first computer with virtualization hardware. People have been doing this in IBM land for over half a century.

rhelz 3y ago

The founders at a tool-making startup I worked for came back from a sales meeting one day looking very sad. I asked them why the long faces. They said they pitched the tool as something which would save their engineers a lot of pain. "Eh?" said the prospect. "What? Who cares how much pain the engineers are in? Its their job to endure pain." Clearly, the founders hadn't fully perfected their sales pitch....

AtlasBarfed 3y ago

This is very simple. Tools in the "real world" generally work on one protocol: physics. Yes.. there are standards and industries and the like in physical tools, but those are much more static and unchanging. Because physics and material world. A tool will generally be relevant until it wears out, and often/sometimes is useful for decades after "obsolescence". In software, where (looks at the Javascript ecosystem) things change in a year. Sometimes less. Version churn and feature churn means your tool is obsolete/useless (if you are lucky) within 3-4 years. Often less. In regular tools, it is pretty clear the distinction between a tool and the end product. But in software, a tool is SOFTWARE, with all the problems of software: version churn, feature drift, underlying OS support, and all the other quicksand. Tool markets are also inherently a fraction the endproduct market: The produced product volume is say 10x to 100x the value of the cost of the tool in the physical world before it needs replacement or wears out. Software being what it is, you can get 100000x - infinite the tool. That value proposition means the market for the tool is the inverse of the value factor of the tool. In software, you would get one tool and use it for 100000 products / views / etc. So the market for it is lower. Perplexingly to my point about software tools going out of support, is that REALLY established tools are long lived and never wear out. So a hot gee-whiz tool will be competing against well-featured tools (often free after a while because tool vendors go out of business and open source it), so there isn't much gee-whiz anymore. OpenAI is an interesting "tool". We'll see where it goes commercially. But most tools have reproducibility. AI tools do not, at least currently they don't. Free software tools seem more durable, because the user population sustains it generally with contributions, bug reports, feedback, and demand. And sometimes the minimal money to support the org. And sometimes marketing. Software is also generally about building a bandwagon. Gatekeeping tools are not good for bandwagon building. So software tools are generally free to make jumping on the bandwagon easier.