5 ms·

Very cool! I'm curious--I'd imagine that some long tail podcasts have transcripts that are too long to fit within a standard context window. Do you have some st

Very cool! I'm curious--I'd imagine that some long tail podcasts have transcripts that are too long to fit within a standard context window. Do you have some strategy for handling these?

victorbjorklund 3y ago
Not OP but I have seen several use cases where first summarising parts and then summarising the summaries have been used.
davepeck 3y ago
There are a few strategies in use today. All involve splitting the content to be summarized into chunks smaller than the context size, summarizing each, and building a full final summary from there (potentially in multiple steps). I wouldn’t necessarily recommend _using_ LangChain, but their summarization docs might be of interest: https://python.langchain.com/en/latest/modules/chains/index_examples/summarize.html https://python.langchain.com/en/latest/modules/chains/index_...
- BigElephant 3y ago
  What would you use besides LangChain?
  - thatcherthorn 3y ago
    I am also interested in the answer to this
  - davepeck 3y ago
    I’ve found it preferable to build directly on top of OpenAI’s API. (I’ve also written a simple API wrapper for llama.cpp hosted LLMs.) Over time I’ve built a small library of utilities, including for summarization. It’s not that much code. I don’t know if this is a spicy or a generally-agreed-upon take: my feeling is that, while LangChain was useful in that it helped the community codify some early intuitions about LLM invocation patterns, it’s basically a grab bag of partially complete somewhat disconnected utilities. It nods to composability but, in practice, its pieces often don’t fit together. On the Python side, it suffers from poor typing: when creating a chain, it’s often impossible to know what the full set of configuration options is without digging deep into LangChain’s code. It’s catch-as-can whether you can deeply configure specific sub-aspects of a chain. There are other things I want in my own code at the moment, including keeping track of how many input/output tokens each of my actions takes, etc. I dunno, maybe I’m the only one here. Curious what others think.
    - deleted 3y ago
      [deleted]
    - _fill 3y ago
      At the moment we're still using langchain but it is quite cumbersome in the long run. The library is developing quickly and a feature that you might expect to work one week might not the next. Have you had better luck with others?
_fill 3y ago
All the strategies below were ones we tried. You can check it out!

Very cool! I'm curious--I'd imagine that some long tail podcasts have transcripts that are too long to fit within a standard context window. Do you have some strategy for handling these?

victorbjorklund 3y ago

Not OP but I have seen several use cases where first summarising parts and then summarising the summaries have been used.

davepeck 3y ago

There are a few strategies in use today. All involve splitting the content to be summarized into chunks smaller than the context size, summarizing each, and building a full final summary from there (potentially in multiple steps). I wouldn’t necessarily recommend _using_ LangChain, but their summarization docs might be of interest: https://python.langchain.com/en/latest/modules/chains/index_examples/summarize.html https://python.langchain.com/en/latest/modules/chains/index_...

BigElephant 3y ago
What would you use besides LangChain?
- thatcherthorn 3y ago
  I am also interested in the answer to this
- davepeck 3y ago
  I’ve found it preferable to build directly on top of OpenAI’s API. (I’ve also written a simple API wrapper for llama.cpp hosted LLMs.) Over time I’ve built a small library of utilities, including for summarization. It’s not that much code. I don’t know if this is a spicy or a generally-agreed-upon take: my feeling is that, while LangChain was useful in that it helped the community codify some early intuitions about LLM invocation patterns, it’s basically a grab bag of partially complete somewhat disconnected utilities. It nods to composability but, in practice, its pieces often don’t fit together. On the Python side, it suffers from poor typing: when creating a chain, it’s often impossible to know what the full set of configuration options is without digging deep into LangChain’s code. It’s catch-as-can whether you can deeply configure specific sub-aspects of a chain. There are other things I want in my own code at the moment, including keeping track of how many input/output tokens each of my actions takes, etc. I dunno, maybe I’m the only one here. Curious what others think.
  - deleted 3y ago
    [deleted]
  - _fill 3y ago
    At the moment we're still using langchain but it is quite cumbersome in the long run. The library is developing quickly and a feature that you might expect to work one week might not the next. Have you had better luck with others?

_fill 3y ago

All the strategies below were ones we tried. You can check it out!