3 ms·

If we take Python an example, how many man-centuries have been spent trying to speed it up enough so that it becomes usable for something it was never meant to

If we take Python an example, how many man-centuries have been spent trying to speed it up enough so that it becomes usable for something it was never meant to be used for?. If 'new science' was writing a small script to automate something instead of writing it in bash then yes I think this language is probably the best language for this task.

sam0x17 3y ago
Right but how many of those man-centuries could have been better spent just writing X in a suitable language where you get that for free, and how many subsequent man centuries will be wasted before everyone switches? Imagine where the AI community would be today if the second one of these languages had become stable and generally available, everything had been ported and we'd now be several years into iterating on things in a more suitable environment, and that environment would in turn be that much more advanced by this point?
- sseagull 3y ago
  There are two problems with this thinking. 1.) The rewrite never seems to have all the features of the original (for many reasons), so you end up keeping the original around because science can be very niche. Now you have two or more packages to deal with. 2.) There is always a better language. Science (or at least parts of science) have switched before - from Fortran to C to C++ to Python. Some of the gains have materialized (some safety, performance, borrowing stuff from outside science). But it has come at a cost (language fragmentation, and also packaging is an absolute shit show right now, partly because #1). But I'm sure the next batch of languages will finally solve all our problems once and for all, and we will never have to switch again. (In general, I am talking about non-AI type science. I am a computational chemist, and our code really does date back > 40 years at times. That is not always a bad thing).
  - sam0x17 3y ago
    > That is not always a bad thing Sure and that's reasonable, but in the cases where performance/throughput/training time/RAM usage is the main limiting factor of a field, suddenly this stuff matters a lot. There's plenty of useful code written in COBOL and Basic from back in the day, but you don't see people using those things to train LLMs Imagine if instead of letting Basic effectively die, we had improved it with a myriad of extensions to the point where you can run LLMs and GPU code and things in a performant-ish way on it. Now replace the word Basic with Python
  - ralphc 3y ago
    How do you, or "you" as the industry, decide that new code and new techniques are worth moving away from existing code?

sam0x17 3y ago

Right but how many of those man-centuries could have been better spent just writing X in a suitable language where you get that for free, and how many subsequent man centuries will be wasted before everyone switches? Imagine where the AI community would be today if the second one of these languages had become stable and generally available, everything had been ported and we'd now be several years into iterating on things in a more suitable environment, and that environment would in turn be that much more advanced by this point?

sseagull 3y ago
There are two problems with this thinking. 1.) The rewrite never seems to have all the features of the original (for many reasons), so you end up keeping the original around because science can be very niche. Now you have two or more packages to deal with. 2.) There is always a better language. Science (or at least parts of science) have switched before - from Fortran to C to C++ to Python. Some of the gains have materialized (some safety, performance, borrowing stuff from outside science). But it has come at a cost (language fragmentation, and also packaging is an absolute shit show right now, partly because #1). But I'm sure the next batch of languages will finally solve all our problems once and for all, and we will never have to switch again. (In general, I am talking about non-AI type science. I am a computational chemist, and our code really does date back > 40 years at times. That is not always a bad thing).
- sam0x17 3y ago
  > That is not always a bad thing Sure and that's reasonable, but in the cases where performance/throughput/training time/RAM usage is the main limiting factor of a field, suddenly this stuff matters a lot. There's plenty of useful code written in COBOL and Basic from back in the day, but you don't see people using those things to train LLMs Imagine if instead of letting Basic effectively die, we had improved it with a myriad of extensions to the point where you can run LLMs and GPU code and things in a performant-ish way on it. Now replace the word Basic with Python
- ralphc 3y ago
  How do you, or "you" as the industry, decide that new code and new techniques are worth moving away from existing code?