3 ms·

I mentioned this in another comment below, but scientific packages were often ignored when building web services and APIs and that's the context in which Django

I mentioned this in another comment below, but scientific packages were often ignored when building web services and APIs and that's the context in which Django/Flask were built. At the time the scientific packages were extremely heavy and best avoided in production environments. Some people did it anyway but removed it once they needed to scale. Simply installing these packages was a huge pain and blocked one's ability to provision new machines, for example. PyPI was indeed a big benefit. Ruby Gems are also what made Ruby so competitive at the time.

code_biologist 4y ago
Slapping some scipy, sklearn, or pandas stuff into Django was a pretty workable solution even back when. Just want to sing the the praises. I've slayed a lot of normal size (as opposed to big) data problems with Django/Flask. Rub some JS charting on top of it and people thought you were a wizard.
- 0xFACEFEED 4y ago
  Oh it was workable. Until it wasn't. I specifically remember extricating a high throughput web service from pandas. Having to scale made that stuff harder. If it was some app that a few thousand people were using, no problem.
  - mch82 4y ago
    > If it was some app that a few thousand people were using, no problem. You’re right, scale is absolutely an issue for companies that sell software or that are very large. Also important to consider that most companies use software internally instead of selling b2b or b2c. Fewer than 500 businesses in the US employ more than a few thousand employees* (https://fortune.com/fortune500/2020/search/?f500_%20employees=desc https://fortune.com/fortune500/2020/search/?f500_%20employee...) and there are over 17.5 million businesses (https://www.naics.com/business-lists/counts-by-company-size/ https://www.naics.com/business-lists/counts-by-company-size/). * I may be wrong in my assumption that the Fortune 500 companies are the largest employers
    - lambdadmitry 4y ago
      Even then, scaling is an almost self-solving issue, as it usually comes with money attached, which buys quite a lot of engineering hours spent on scaling things. Overscaling in advance on the other hand with all of its associated complexity and slowdown killed many, many companies.
      - 0xFACEFEED 4y ago
        Inability to scale because the application was fundamentally unscalable has also killed many companies. And for those it didn't kill it significantly hurt their earning potential (and ultimately everyone's payout). There's a balance and it's more delicate than people give it credit these days. Don't get me wrong though... over-engineering and/or premature optimization is just as bad. But there is an opposite extreme as well.

code_biologist 4y ago

Slapping some scipy, sklearn, or pandas stuff into Django was a pretty workable solution even back when. Just want to sing the the praises. I've slayed a lot of normal size (as opposed to big) data problems with Django/Flask. Rub some JS charting on top of it and people thought you were a wizard.

0xFACEFEED 4y ago
Oh it was workable. Until it wasn't. I specifically remember extricating a high throughput web service from pandas. Having to scale made that stuff harder. If it was some app that a few thousand people were using, no problem.
- mch82 4y ago
  > If it was some app that a few thousand people were using, no problem. You’re right, scale is absolutely an issue for companies that sell software or that are very large. Also important to consider that most companies use software internally instead of selling b2b or b2c. Fewer than 500 businesses in the US employ more than a few thousand employees* (https://fortune.com/fortune500/2020/search/?f500_%20employees=desc https://fortune.com/fortune500/2020/search/?f500_%20employee...) and there are over 17.5 million businesses (https://www.naics.com/business-lists/counts-by-company-size/ https://www.naics.com/business-lists/counts-by-company-size/). * I may be wrong in my assumption that the Fortune 500 companies are the largest employers
  - lambdadmitry 4y ago
    Even then, scaling is an almost self-solving issue, as it usually comes with money attached, which buys quite a lot of engineering hours spent on scaling things. Overscaling in advance on the other hand with all of its associated complexity and slowdown killed many, many companies.
    - 0xFACEFEED 4y ago
      Inability to scale because the application was fundamentally unscalable has also killed many companies. And for those it didn't kill it significantly hurt their earning potential (and ultimately everyone's payout). There's a balance and it's more delicate than people give it credit these days. Don't get me wrong though... over-engineering and/or premature optimization is just as bad. But there is an opposite extreme as well.