3 ms·
> Really good advice - this is a hard-earned lesson for many folks. I've worked with quite a few data scientists who were brilliant at experimental design but n
by kragen 5y ago
> Really good advice - this is a hard-earned lesson for many folks. I've worked with quite a few data scientists who were brilliant at experimental design but not necessarily experts in the field of comp sci. Their relatively simple Python scripts would run nice and fast initially. As time passed and the organization grew,
The problem of "as the organization grew,..." is the problem you want to have. 90% of organizations instead have the problem "As we failed to get any interesting results or any funding, and our contributors started drifting away to other projects,..." and sometimes the reason they have that problem is precisely because the initial contributors were too concerned about doing things "the right way" and weren't satisfied with "relatively simple Python scripts [that] run nice and fast initially".
When your limiting factor is programmer effort, there are two kinds of systems: the kind that runs as slowly as the users and developers can tolerate, and the kind that people don't use because it lacks the functionality they want, functionality that some other system has. Probably a slower one.
Of course you should make things go as fast as you can, and performance can be critical not only for ergonomics but also for the ability to iterate rapidly, and sometimes the performance is the functionality, but when you find yourself facing down the shitty performance of "founder code" that locks up shared machines, don't fall into the trap of wishing that the people had done it right in the first place, before "time passed and the organization grew". In the startup graveyard there are nine other groups who tried to do the same thing, but couldn't quite get it right. Some of them probably had damn fast code, which you aren't looking at, because their organization never grew.
And I say that as someone who spent a fair bit of the last week looking at assembly language and thinking about compiler optimization tradeoffs.