4 ms·

I’ve been a user of SpaCy since 2016. I haven’t touched it in years and I just picked it up again to develop a new metric for RAG using part of speech coverage.

I’ve been a user of SpaCy since 2016. I haven’t touched it in years and I just picked it up again to develop a new metric for RAG using part of speech coverage. The API is one of the best ever, and really set the bar high for language tooling. I’m glad it’s still around and getting updates. I had a bit of trouble integrating it with uv, but nothing too bad. Thanks to the explosion team for making such an amazing project and keeping it going all these years. To the new “AI” people in the room: checkout SpaCy, and see how well it works and how fast it chews through text. You might find yourself in a situation where you don’t need to send your data to OpenAI for some small things. Edit: I almost forgot to add this little nugget of history: one of Huggingfaces first projects was a SpaCy extension for conference resolution. Built before their breakthrough with transformers https://github.com/huggingface/neuralcoref https://github.com/huggingface/neuralcoref

deleted 1y ago
[deleted]
jehejej 1y ago
*coreference resolution.
ok_dad 1y ago
What’s great about the API that you enjoy and do you have anything you hate about it? I’m writing a small library at work for some NLP tasks and I haven’t got a whole lot of experience in writing libraries for NLP, so I’m interested in what would make my library the best for the user.
- binarymax 1y ago
  The thing about spaCys API is that it perfectly aligns with how NLP worked at the time with actual programming paradigms and allows you to be very pythonic. For example, you can use list comprehension to get all the nouns from a document in a one liner. These days NLP is quite different, because we look for outcomes rather than iterating over tokens. What does your NLP library need to do? The way I design APIs is I write the calling code that I want to exist, and then I write the API to make it work. Here’s an example I’ve worked on for LLM integration. I just wanted to be able to get simple answers from an LLM and cast the answer to a type: https://www.npmjs.com/package/llm-primitives https://www.npmjs.com/package/llm-primitives

deleted 1y ago

[deleted]

jehejej 1y ago

*coreference resolution.

ok_dad 1y ago

What’s great about the API that you enjoy and do you have anything you hate about it? I’m writing a small library at work for some NLP tasks and I haven’t got a whole lot of experience in writing libraries for NLP, so I’m interested in what would make my library the best for the user.

binarymax 1y ago
The thing about spaCys API is that it perfectly aligns with how NLP worked at the time with actual programming paradigms and allows you to be very pythonic. For example, you can use list comprehension to get all the nouns from a document in a one liner. These days NLP is quite different, because we look for outcomes rather than iterating over tokens. What does your NLP library need to do? The way I design APIs is I write the calling code that I want to exist, and then I write the API to make it work. Here’s an example I’ve worked on for LLM integration. I just wanted to be able to get simple answers from an LLM and cast the answer to a type: https://www.npmjs.com/package/llm-primitives https://www.npmjs.com/package/llm-primitives