6 ms·

Ritchie Vink (the creator of Polars) deliberately decided not to write a book so that he (and his team) can focus full time on Polars itself. Thijs Nieuwdorp a

Ritchie Vink (the creator of Polars) deliberately decided not to write a book so that he (and his team) can focus full time on Polars itself. Thijs Nieuwdorp and I are currently working on the O'Reilly book "Python Polars: The Definitive Guide" [1]. It'll be a while before it gets released, but the Early Release version on the O'Reilly platform gets updated regularly. We also post draft chapters on the Polars Discord server [2]. The Discord server is also a great place to ask questions and interact with the Polars team. [1] More information about the book: https://jeroenjanssens.com/pp/ https://jeroenjanssens.com/pp/ [2] Polars Discord server: https://discord.gg/fngBqDry https://discord.gg/fngBqDry

baq 3y ago
Slightly offtopic: it's a tragedy that projects like this use discord as the primary discussion forum. It's like slack in that knowledge goes to die there.
- axus 3y ago
  Shouldn't the knowledge go into a wiki anyways? Reading old discussions in Reddit can turn up interesting things, but it's time consuming and bad information doesn't get fixed later.
- Fuzzwah 3y ago
  I often see this comment, and every time I think; but having people come to the information AND the community is better for the project.
  - baq 3y ago
    Short term perhaps, but long term having a non-indexed community is inconvenient for newcomers.
    - thijsn 3y ago
      That's why Ritchie is very active on, and often refers to, Stackoverflow as well! Exactly to document frequent questions, instead of losing them to chat history.
    - amne 3y ago
      microsoft copilot can summarize discussions. with some orchestration it could extract even from past discussions question+answers and structure them in a stackoverflow-like format. source: we use this feature in beta as part of the enterprise copilot license to summarize Teams calls. Yes, it listens to us talking and spits out bullet points from our discussions at the end of the call. It's so good it feels like magic sometimes. note on copilot: any capable model could probably do it. I just said copilot because it does it today.
    - zknow 3y ago
      there are projects that you can use to index discord servers, unfortunately a lot of communities just don't use them.
      - esafak 3y ago
        Why would they? A person who picks Discord has no idea what knowledge discovery is.
  - throwmeback 3y ago
    by community do you mean all the people who make an account just to ask a question on the project's discord, only ever open it to check if someone answered and then never use discord again?
  - anigbrowl 3y ago
    Forums work really well for this. I personally avoid using Discord because chatrooms are too much of a time suck. There's far more chaff to sift through and trying to keep up with everything leads to FOMO.
- theLiminator 3y ago
  Yeah one thing that helps a bit is that they try to encourage that you post your questions to stack overflow and they'll answer it.
- jeroenjanssens 3y ago
  Luckily our book will also be available in hard copy so you can digest all that hard-won knowledge in an offline manner :)
  - tonyedgecombe 3y ago
    I'll wait until chatGPT can regurgitate it.
    - Xunjin 3y ago
      I do understand your "snarky" comment humoring, however do buy a copy if you want to support them, it's neither cheap or easy to make a book.
    - throwmeback 3y ago
      losing all nuance by virtue of getting dopamine quicker? count me in!
- ajhurliman 3y ago
  Disagree. Yes, forums with nested comment structure and voting are better for preserving and indexing data, but synchronous communication (like Discord) is better for having actual conversations and building community. Back and forth volleys of messages let people completely explore threads of thinking, rather than waiting days (or months) for a single response that misunderstands the question. There’s a reason people prefer it: it’s better.
  - diggan 3y ago
    > There’s a reason people prefer it: it’s better. That's a very "black and white" view, of any issue. It could be that it's better for that specific person(s) who is having the current conversation. But what about after that? It used to be that you could search Google and if it was answered in a forum post, you'd find it via Google. But since a lot of it are closed behind locked doors (like Discord), it becomes really hard to find, even with Discords own search. Everything being ephemeral could help someone right now, but what about the others in the future, wanting the exact same answer?
    - esafak 3y ago
      Just ask it again so you can burn- and drive out the existing members with repetitive questions!
    - ajhurliman 3y ago
      Okay I'll elaborate, it's better for community building and answering questions for nascent companies or organizations. A lot of the time there's a big disconnect between a community and how a product is intended to be used. I built a paper trading app for stocks and options and Discord was the primary place where the users talked. The subreddit was almost completely empty, nobody responded to tweets, the Instagram was thriving but there was no sense of community because you couldn't tell if anyone commenting actually used the product or a meme just showed up in their feed. Did I have to repeat myself a bunch? Yes, but that's fine, especially because the answer sometimes changed (rapidly), like "No we don't support multiple trading accounts per user" > "Yeah I implemented that last week, it's in the release notes, to add a second account..." For a mature product that's not changing as much and more or less has all its features built out, it makes sense to branch out into more structured forums that are easily searchable, especially as you progress through different versions and users are looking for answers to past versions.
bomewish 3y ago
Any plans to try fine tune an LLM specialised in polars? That would really be the killer feature to get major adoption IMO.
esel2k 3y ago
Is there a book that is even more basic for more junior people in regards to dataframe / storage solutions for ML applications to recommend? Thank you
lr1970 3y ago
There is a free Polars user guide [0] as a part of Polars project. It was known as "polars-book" before it has been was in-tree [1]. [0] https://docs.pola.rs/user-guide/ https://docs.pola.rs/user-guide/ [1] https://github.com/pola-rs/polars/tree/main/docs/user-guide https://github.com/pola-rs/polars/tree/main/docs/user-guide
anigbrowl 3y ago
I'm not suggesting people need to write books to introduce their projects, but that landing pages should be more accessible to newbies if you want to build a big user base. A lot of projects introduce themselves by ticking off a list of currently-desirable buzzwords ('performant', 'beautiful' etc.) but neglect to articulate clearly what their project is and why someone might want to use it.

baq 3y ago

Slightly offtopic: it's a tragedy that projects like this use discord as the primary discussion forum. It's like slack in that knowledge goes to die there.

axus 3y ago
Shouldn't the knowledge go into a wiki anyways? Reading old discussions in Reddit can turn up interesting things, but it's time consuming and bad information doesn't get fixed later.
Fuzzwah 3y ago
I often see this comment, and every time I think; but having people come to the information AND the community is better for the project.
- baq 3y ago
  Short term perhaps, but long term having a non-indexed community is inconvenient for newcomers.
  - thijsn 3y ago
    That's why Ritchie is very active on, and often refers to, Stackoverflow as well! Exactly to document frequent questions, instead of losing them to chat history.
  - amne 3y ago
    microsoft copilot can summarize discussions. with some orchestration it could extract even from past discussions question+answers and structure them in a stackoverflow-like format. source: we use this feature in beta as part of the enterprise copilot license to summarize Teams calls. Yes, it listens to us talking and spits out bullet points from our discussions at the end of the call. It's so good it feels like magic sometimes. note on copilot: any capable model could probably do it. I just said copilot because it does it today.
  - zknow 3y ago
    there are projects that you can use to index discord servers, unfortunately a lot of communities just don't use them.
    - esafak 3y ago
      Why would they? A person who picks Discord has no idea what knowledge discovery is.
- throwmeback 3y ago
  by community do you mean all the people who make an account just to ask a question on the project's discord, only ever open it to check if someone answered and then never use discord again?
- anigbrowl 3y ago
  Forums work really well for this. I personally avoid using Discord because chatrooms are too much of a time suck. There's far more chaff to sift through and trying to keep up with everything leads to FOMO.
theLiminator 3y ago
Yeah one thing that helps a bit is that they try to encourage that you post your questions to stack overflow and they'll answer it.
jeroenjanssens 3y ago
Luckily our book will also be available in hard copy so you can digest all that hard-won knowledge in an offline manner :)
- tonyedgecombe 3y ago
  I'll wait until chatGPT can regurgitate it.
  - Xunjin 3y ago
    I do understand your "snarky" comment humoring, however do buy a copy if you want to support them, it's neither cheap or easy to make a book.
  - throwmeback 3y ago
    losing all nuance by virtue of getting dopamine quicker? count me in!
ajhurliman 3y ago
Disagree. Yes, forums with nested comment structure and voting are better for preserving and indexing data, but synchronous communication (like Discord) is better for having actual conversations and building community. Back and forth volleys of messages let people completely explore threads of thinking, rather than waiting days (or months) for a single response that misunderstands the question. There’s a reason people prefer it: it’s better.
- diggan 3y ago
  > There’s a reason people prefer it: it’s better. That's a very "black and white" view, of any issue. It could be that it's better for that specific person(s) who is having the current conversation. But what about after that? It used to be that you could search Google and if it was answered in a forum post, you'd find it via Google. But since a lot of it are closed behind locked doors (like Discord), it becomes really hard to find, even with Discords own search. Everything being ephemeral could help someone right now, but what about the others in the future, wanting the exact same answer?
  - esafak 3y ago
    Just ask it again so you can burn- and drive out the existing members with repetitive questions!
  - ajhurliman 3y ago
    Okay I'll elaborate, it's better for community building and answering questions for nascent companies or organizations. A lot of the time there's a big disconnect between a community and how a product is intended to be used. I built a paper trading app for stocks and options and Discord was the primary place where the users talked. The subreddit was almost completely empty, nobody responded to tweets, the Instagram was thriving but there was no sense of community because you couldn't tell if anyone commenting actually used the product or a meme just showed up in their feed. Did I have to repeat myself a bunch? Yes, but that's fine, especially because the answer sometimes changed (rapidly), like "No we don't support multiple trading accounts per user" > "Yeah I implemented that last week, it's in the release notes, to add a second account..." For a mature product that's not changing as much and more or less has all its features built out, it makes sense to branch out into more structured forums that are easily searchable, especially as you progress through different versions and users are looking for answers to past versions.

bomewish 3y ago

Any plans to try fine tune an LLM specialised in polars? That would really be the killer feature to get major adoption IMO.

esel2k 3y ago

Is there a book that is even more basic for more junior people in regards to dataframe / storage solutions for ML applications to recommend? Thank you

lr1970 3y ago

There is a free Polars user guide [0] as a part of Polars project. It was known as "polars-book" before it has been was in-tree [1]. [0] https://docs.pola.rs/user-guide/ https://docs.pola.rs/user-guide/ [1] https://github.com/pola-rs/polars/tree/main/docs/user-guide https://github.com/pola-rs/polars/tree/main/docs/user-guide

anigbrowl 3y ago

I'm not suggesting people need to write books to introduce their projects, but that landing pages should be more accessible to newbies if you want to build a big user base. A lot of projects introduce themselves by ticking off a list of currently-desirable buzzwords ('performant', 'beautiful' etc.) but neglect to articulate clearly what their project is and why someone might want to use it.