2 ms·

Pretty interesting stuff. I'm just wondering how you manage the limitation of context length.

• jasonjmcghee 3y ago
  For the "copy recent context"? The last 15 frames. It's a terrible approach! But I had to start somewhere. Actively experimenting with properly leveraging embedding search. But I've had a hard time finding CPU + RAM efficient vector indexing + search that meets my expectations. Been doing a lot of personal experimentation and research in this space. Is there a known approach to be able to maintain a large embedding space that you can insert into efficiently and search accurately without needing to load / maintain the entire thing into memory?