5 ms·

I was browsing the most popular pip packages recently and was also downright shocked how much AI stuff is up at the top. I will be very curious if much of thes

I was browsing the most popular pip packages recently and was also downright shocked how much AI stuff is up at the top. I will be very curious if much of these things are still popular or relevant in a year.

Spivak 3y ago
By volume of different projects probably not, but by usage probably. Once winners get decided and paths get treaded the ecosystem will probably be smaller but I bet the user count will be bigger. Look, even if you're the most biggest skeptic of all the AI stuff you can still do useful things with it that you couldn't before like present full-English interfaces and get embeddings for arbitrary text which is a leap compared to word2vec for search. I can dm my discord bot a picture of an appointment confirmation text and it with OCR + LLm it just schedules the event in my calendar. Also by gluing a v8 isolate to the LLm I can give it much more complicated date calculations. My favorite was "If f(n) is the nth Fibonacci number schedule an appointment in f(5) days."
- nwiswell 3y ago
  > you can still do useful things with it that you couldn't before like > ... > If f(n) is the nth Fibonacci number schedule an appointment in f(5) days hmmm
  - IanCal 3y ago
    That's absolutely not what they said. Let's keep hn better than that.
    - nwiswell 3y ago
      It's obviously tongue-in-cheek, but it's also not not what they said.
      - IanCal 3y ago
        It is not what they said. That was an example of complicated work with dates, which easily hits the level of "more complicated than you need". Before that, and before the cut in the quote was an obvious use case easily explained. I don't know why you want to go down this route but I won't engage further.
- axutio 3y ago
  Is this a setup you've already built? Could you outline what you've glued together for this bot?
  - Spivak 3y ago
    Oh sure! So the OCR is a statically linked build of tesseract based on [1] and pytesseract [2] which is a super thin wrapper but easier than writing it yourself. The I stole/modified the prompt from [3] to get the bot to write Python programs that do date calculations. Then I used [4] to take the output in case the Llm didn't use the format I asked. I run the Python code it generates in a rootless container that uses the Lambda RIE [5] because I was too lazy to make my own thing. So I'm mildly lied about the v8 isolate because everyone gets the implication and who wants to hear about RIE and seccomp profiles. [1] https://github.com/wingedrhino/static-tesseract https://github.com/wingedrhino/static-tesseract [2] https://pypi.org/project/pytesseract/ https://pypi.org/project/pytesseract/ [3] https://python.langchain.com/en/latest/modules/agents/tools/examples/python.html https://python.langchain.com/en/latest/modules/agents/tools/... [4] https://pypi.org/project/dateparser/ https://pypi.org/project/dateparser/ [5] https://docs.aws.amazon.com/lambda/latest/dg/images-test.html#images-test-AWSbase https://docs.aws.amazon.com/lambda/latest/dg/images-test.htm...
    - axutio 3y ago
      This is amazing, thank you for sharing!
mathisfun123 3y ago
So weird - you really think AI/ML is a flash in the pan when it comes to Python? You realize pytorch, tensorflow, pandas, numpy, scikit-learn are all approaching 10 years on pypi?
- qbasic_forever 3y ago
  Yeah all the stuff I was seeing seemed to be variations and light wrappers around openai APIs and such. We don't need a million versions of those things, one or two clear 'winners' should emerge. You can see them here, expand it to 50 per page and start browsing down after the usual suspects: https://awesomepython.org/ https://awesomepython.org/
  - mathisfun123 3y ago
    I'm sorry but I think you don't know what you're looking at - of the NN things in the top 50 there I see 4 such wrappers. The remainder are the NN frameworks and various perf motivated implementations. Also I don't know how this list is built but it seems to be literally measuring current popular things - so by definition no matter when i visit this list whatever is it at the top will probably wane in popularity shortly after I've visited.
  - Kwpolska 3y ago
    In a language that has a million incompatible and partial solutions to packaging?
- theaiquestion 3y ago
  I think this is more comparable to the bajillion JS frameworks that popped up between like 2015 and 2020. Same kinda deal with the big frameworks either surviving (likely with large rewrites for improvements) or succeeded by a replacement. But it feels like that's died down a bit. > pytorch, tensorflow, pandas, numpy, scikit-learn I think it's likely these will either survive or be replaced by a successor - but we wont be seeing 100 new ai packages a day.
forrestthewoods 3y ago
Python has exploded in popularity over the past ~7 years because of AI. Python’s ever increasingly relevance is directly tied to AI.
- tomrod 3y ago
  And Flask, and FastAPI, and pandas, and pola.rs, and numba, and as glue code, and.... "AI" is pretty narrow, but the python ecosystem is massive, which a strong incumbency in the data processing space. Weird, since it i̶s̶n̶'̶t̶ ̶a̶ ̶t̶y̶p̶e̶d̶ is a dynamically language, where sometimes placeholders like pandas `object` result in unexpected behavior, but it sure is simple to write something quick.
  - twic 3y ago
    > "AI" is pretty narrow, but the python ecosystem is massive, which a strong incumbency in the data processing space. Weird, since it isn't a typed language Nor is Excel, and that didn't stop it.
  - Tommstein 3y ago
    > "AI" is pretty narrow, but the python ecosystem is massive, which a strong incumbency in the data processing space. Weird, since it isn't a typed language . . . . Having used it for over 15 years, I assure you Python is a strongly typed language.
    - tomrod 3y ago
      Fair, I was wrong. It's a dynamically typed language. I meant to say it is not a statically typed language.
  - forrestthewoods 3y ago
    Sure. Python isn’t going away. But Python has tripled in popularity over the past 7 years and a very very large portion of that growth is due to AI, imho. [1] So, will a given AI Python package be popular in a year or three? Maybe, maybe not. Will the top Python packages be AI related in 3 to 5 years? Absolutely, imho. I’d definitely take that bet. [1] https://insights.stackoverflow.com/trends/?tags=java%2Cc%2Cc%2B%2B%2Cpython%2Cc%23%2Cvb.net%2Cjavascript%2Cassembly%2Cphp%2Cperl%2Cruby%2Cvb%2Cswift%2Cr%2Cobjective-c https://insights.stackoverflow.com/trends/?tags=java%2Cc%2Cc...

Spivak 3y ago

By volume of different projects probably not, but by usage probably. Once winners get decided and paths get treaded the ecosystem will probably be smaller but I bet the user count will be bigger. Look, even if you're the most biggest skeptic of all the AI stuff you can still do useful things with it that you couldn't before like present full-English interfaces and get embeddings for arbitrary text which is a leap compared to word2vec for search. I can dm my discord bot a picture of an appointment confirmation text and it with OCR + LLm it just schedules the event in my calendar. Also by gluing a v8 isolate to the LLm I can give it much more complicated date calculations. My favorite was "If f(n) is the nth Fibonacci number schedule an appointment in f(5) days."

nwiswell 3y ago
> you can still do useful things with it that you couldn't before like > ... > If f(n) is the nth Fibonacci number schedule an appointment in f(5) days hmmm
- IanCal 3y ago
  That's absolutely not what they said. Let's keep hn better than that.
  - nwiswell 3y ago
    It's obviously tongue-in-cheek, but it's also not not what they said.
    - IanCal 3y ago
      It is not what they said. That was an example of complicated work with dates, which easily hits the level of "more complicated than you need". Before that, and before the cut in the quote was an obvious use case easily explained. I don't know why you want to go down this route but I won't engage further.
axutio 3y ago
Is this a setup you've already built? Could you outline what you've glued together for this bot?
- Spivak 3y ago
  Oh sure! So the OCR is a statically linked build of tesseract based on [1] and pytesseract [2] which is a super thin wrapper but easier than writing it yourself. The I stole/modified the prompt from [3] to get the bot to write Python programs that do date calculations. Then I used [4] to take the output in case the Llm didn't use the format I asked. I run the Python code it generates in a rootless container that uses the Lambda RIE [5] because I was too lazy to make my own thing. So I'm mildly lied about the v8 isolate because everyone gets the implication and who wants to hear about RIE and seccomp profiles. [1] https://github.com/wingedrhino/static-tesseract https://github.com/wingedrhino/static-tesseract [2] https://pypi.org/project/pytesseract/ https://pypi.org/project/pytesseract/ [3] https://python.langchain.com/en/latest/modules/agents/tools/examples/python.html https://python.langchain.com/en/latest/modules/agents/tools/... [4] https://pypi.org/project/dateparser/ https://pypi.org/project/dateparser/ [5] https://docs.aws.amazon.com/lambda/latest/dg/images-test.html#images-test-AWSbase https://docs.aws.amazon.com/lambda/latest/dg/images-test.htm...
  - axutio 3y ago
    This is amazing, thank you for sharing!

mathisfun123 3y ago

So weird - you really think AI/ML is a flash in the pan when it comes to Python? You realize pytorch, tensorflow, pandas, numpy, scikit-learn are all approaching 10 years on pypi?

qbasic_forever 3y ago
Yeah all the stuff I was seeing seemed to be variations and light wrappers around openai APIs and such. We don't need a million versions of those things, one or two clear 'winners' should emerge. You can see them here, expand it to 50 per page and start browsing down after the usual suspects: https://awesomepython.org/ https://awesomepython.org/
- mathisfun123 3y ago
  I'm sorry but I think you don't know what you're looking at - of the NN things in the top 50 there I see 4 such wrappers. The remainder are the NN frameworks and various perf motivated implementations. Also I don't know how this list is built but it seems to be literally measuring current popular things - so by definition no matter when i visit this list whatever is it at the top will probably wane in popularity shortly after I've visited.
- Kwpolska 3y ago
  In a language that has a million incompatible and partial solutions to packaging?
theaiquestion 3y ago
I think this is more comparable to the bajillion JS frameworks that popped up between like 2015 and 2020. Same kinda deal with the big frameworks either surviving (likely with large rewrites for improvements) or succeeded by a replacement. But it feels like that's died down a bit. > pytorch, tensorflow, pandas, numpy, scikit-learn I think it's likely these will either survive or be replaced by a successor - but we wont be seeing 100 new ai packages a day.

forrestthewoods 3y ago

Python has exploded in popularity over the past ~7 years because of AI. Python’s ever increasingly relevance is directly tied to AI.

tomrod 3y ago
And Flask, and FastAPI, and pandas, and pola.rs, and numba, and as glue code, and.... "AI" is pretty narrow, but the python ecosystem is massive, which a strong incumbency in the data processing space. Weird, since it i̶s̶n̶'̶t̶ ̶a̶ ̶t̶y̶p̶e̶d̶ is a dynamically language, where sometimes placeholders like pandas `object` result in unexpected behavior, but it sure is simple to write something quick.
- twic 3y ago
  > "AI" is pretty narrow, but the python ecosystem is massive, which a strong incumbency in the data processing space. Weird, since it isn't a typed language Nor is Excel, and that didn't stop it.
- Tommstein 3y ago
  > "AI" is pretty narrow, but the python ecosystem is massive, which a strong incumbency in the data processing space. Weird, since it isn't a typed language . . . . Having used it for over 15 years, I assure you Python is a strongly typed language.
  - tomrod 3y ago
    Fair, I was wrong. It's a dynamically typed language. I meant to say it is not a statically typed language.
- forrestthewoods 3y ago
  Sure. Python isn’t going away. But Python has tripled in popularity over the past 7 years and a very very large portion of that growth is due to AI, imho. [1] So, will a given AI Python package be popular in a year or three? Maybe, maybe not. Will the top Python packages be AI related in 3 to 5 years? Absolutely, imho. I’d definitely take that bet. [1] https://insights.stackoverflow.com/trends/?tags=java%2Cc%2Cc%2B%2B%2Cpython%2Cc%23%2Cvb.net%2Cjavascript%2Cassembly%2Cphp%2Cperl%2Cruby%2Cvb%2Cswift%2Cr%2Cobjective-c https://insights.stackoverflow.com/trends/?tags=java%2Cc%2Cc...