6 ms·

Using LangChainJS and Cloudflare Workers together

83 points by jgrahamc 3y ago

jasongill 3y ago
Looks like a new version of Wrangler was recently released, so the setup instructions in the article are outdated / incorrect already. Cloudflare is great and I love working with them, but Wrangler and way that there have been multiple major shifts in the way that Workers are set up over the years has become a bit frustrating at times.
- shakes 3y ago
  Hey Jason - Author of the post here. Thanks for the catch, updating the post now. If you drop me an email or twitter DM (details in my profile) and I'd love to send you a t-shirt as thanks for catching this.
  - latchkey 3y ago
    Why do you put CF tracking ref= turds in the 3rd party links in the article? Is that really necessary?
    - shakes 3y ago
      You're right, those aren't necessary. That was a setting on our blog CMS that was enabled with a recent upgrade. This is now disabled (it may take a bit for cache to clear and you see these disappears). Thank you for calling this out!
      - latchkey 3y ago
        What, no free t-shirt? ;-) Just kidding, thanks for fixing this so quickly.
        shakes 3y ago
        Drop me a DM or email!
theogravity 3y ago
This is awesome. I just implemented ChatGPT into a demo and am now looking into vector databases. I'd love to be able to swap between different models and databases and LangChainJS looks like a good abstraction to use to do this.
- shakes 3y ago
  Using LangChain definitely shifted how I was building with language models. It took me a couple days to really wrap my head around the possibilities, so if you feel similarly when you're getting started don't worry. If you have any specific questions, or want to chat with someone else using it, please don't be shy. My email and twitter handle are both in my profile. I truly love talking about all of this and hearing what other folks are working on.
  - theogravity 3y ago
    Thank you for the offer!
- rita3ko 3y ago
  we (cloudflare) have a great example of a vector database build on workers + kv here: https://ai.cloudflare.com/demos/vector-store/ https://ai.cloudflare.com/demos/vector-store/ (plan to talk about how we built it soon) edit: clarify i work at cloudflare
  - ignoramous 3y ago
    > plan to talk about how we built it soon Neat! Reminds me of using DynamoDB as a multi-dimensional store with (space-filling curve) z-order indexing [0], which isn't as fast (as native spatial databases, say) but it works. I wonder how performant retrofitting Workers KV is compared to a native vector database? Please do publish benchmarks, if possible. [0] https://archive.is/ZGY1T https://archive.is/ZGY1T / https://news.ycombinator.com/item?id=30009893 https://news.ycombinator.com/item?id=30009893
  - iib 3y ago
    Is this open source somewhere? It seems interesting to study
skybrian 3y ago
The article doesn’t say why someone would want to do this. Looks like this is about using edge computing to coordinate API calls to services like OpenAI’s ChatGPT that do the real work. So, it would make it easier to build a website that uses these services.
- shakes 3y ago
  Post author here. I think that's a great description of a "why" for this kind of setup. I personally feel like with a lot of these emerging AI tools I still find myself still in the exploration phase. Trying to understand how they work and what they do, to then wrap my ahead around why I use them. Some of this post is a look into how I've been wrapping my head around the what and how of LangChain. Doing a follow-up digging deeper into the "why" and what benefits folks may see from this kind of setup is a really good nudge for me. Thanks!
tmikaeld 3y ago
Tried the github code, looks correct according to the article, but produces: TypeError: Cannot read properties of undefined (reading 'replaceAll') Same error when tried in deno but with more details: TypeError: Cannot read properties of undefined (reading 'replaceAll') at R.embedQuery (https://esm.sh/v122/langchain@0.0.77/deno/embeddings/openai.js:4:319) at u.similaritySearch (https://esm.sh/v122/langchain@0.0.77/deno/vectorstores/memory.js:2:896)
- shakes 3y ago
  Hm.... Thanks for flagging. Taking a look now. Edit: Ok. I haven't been able to reproduce this issue (I know "it worked on my machine" is the worst answer. But I did do a fresh setup on two other machines to see if I can recreate so it worked on three of my machines?). If you want to drop me an email (my email is in my profile), I'm happy to help find time to debug live and I'd personally love to find out what the issue is. And/or hook you up with a t-shirt :) Digging in, this specific error looks like it is happening when LangChain is trying to strip out new lines (you can see the code in LangChain here: https://github.com/hwchase17/langchainjs/blob/main/langchain/src/embeddings/openai.ts#L164 https://github.com/hwchase17/langchainjs/blob/main/langchain...). For some reason your documents or embeddings are undefined. But I've been unable to create a scenario where that happens. I wondered if perhaps your OpenAI API Key wasn't set correctly and it was causing the Embeddings to fail, but I tried both deleting my key and setting to bogus key and the error was different. Any chance you switched out the URL we were pulling from? That didn't cause the same error for me either, but thinking through what else it could be. Sorry to not resolve, but please feel free to drop me an email. I'll keep trying to reproduce as well, and will jump back in this comment if I do (and find a fix).
  - tmikaeld 3y ago
    The reason was that I was using json for the input.url input.query - but for some reason, even thought it's just plain strings, it has this error, when i use query strings it just works.

83 points by jgrahamc 3y ago

Looks like a new version of Wrangler was recently released, so the setup instructions in the article are outdated / incorrect already. Cloudflare is great and I love working with them, but Wrangler and way that there have been multiple major shifts in the way that Workers are set up over the years has become a bit frustrating at times.

shakes 3y ago
Hey Jason - Author of the post here. Thanks for the catch, updating the post now. If you drop me an email or twitter DM (details in my profile) and I'd love to send you a t-shirt as thanks for catching this.
- latchkey 3y ago
  Why do you put CF tracking ref= turds in the 3rd party links in the article? Is that really necessary?
  - shakes 3y ago
    You're right, those aren't necessary. That was a setting on our blog CMS that was enabled with a recent upgrade. This is now disabled (it may take a bit for cache to clear and you see these disappears). Thank you for calling this out!
    - latchkey 3y ago
      What, no free t-shirt? ;-) Just kidding, thanks for fixing this so quickly.
      - shakes 3y ago
        Drop me a DM or email!

theogravity 3y ago

This is awesome. I just implemented ChatGPT into a demo and am now looking into vector databases. I'd love to be able to swap between different models and databases and LangChainJS looks like a good abstraction to use to do this.

shakes 3y ago
Using LangChain definitely shifted how I was building with language models. It took me a couple days to really wrap my head around the possibilities, so if you feel similarly when you're getting started don't worry. If you have any specific questions, or want to chat with someone else using it, please don't be shy. My email and twitter handle are both in my profile. I truly love talking about all of this and hearing what other folks are working on.
- theogravity 3y ago
  Thank you for the offer!
rita3ko 3y ago
we (cloudflare) have a great example of a vector database build on workers + kv here: https://ai.cloudflare.com/demos/vector-store/ https://ai.cloudflare.com/demos/vector-store/ (plan to talk about how we built it soon) edit: clarify i work at cloudflare
- ignoramous 3y ago
  > plan to talk about how we built it soon Neat! Reminds me of using DynamoDB as a multi-dimensional store with (space-filling curve) z-order indexing [0], which isn't as fast (as native spatial databases, say) but it works. I wonder how performant retrofitting Workers KV is compared to a native vector database? Please do publish benchmarks, if possible. [0] https://archive.is/ZGY1T https://archive.is/ZGY1T / https://news.ycombinator.com/item?id=30009893 https://news.ycombinator.com/item?id=30009893
- iib 3y ago
  Is this open source somewhere? It seems interesting to study

skybrian 3y ago

The article doesn’t say why someone would want to do this. Looks like this is about using edge computing to coordinate API calls to services like OpenAI’s ChatGPT that do the real work. So, it would make it easier to build a website that uses these services.

shakes 3y ago
Post author here. I think that's a great description of a "why" for this kind of setup. I personally feel like with a lot of these emerging AI tools I still find myself still in the exploration phase. Trying to understand how they work and what they do, to then wrap my ahead around why I use them. Some of this post is a look into how I've been wrapping my head around the what and how of LangChain. Doing a follow-up digging deeper into the "why" and what benefits folks may see from this kind of setup is a really good nudge for me. Thanks!

tmikaeld 3y ago

Tried the github code, looks correct according to the article, but produces: TypeError: Cannot read properties of undefined (reading 'replaceAll') Same error when tried in deno but with more details: TypeError: Cannot read properties of undefined (reading 'replaceAll') at R.embedQuery (https://esm.sh/v122/langchain@0.0.77/deno/embeddings/openai.js:4:319) at u.similaritySearch (https://esm.sh/v122/langchain@0.0.77/deno/vectorstores/memory.js:2:896)

shakes 3y ago
Hm.... Thanks for flagging. Taking a look now. Edit: Ok. I haven't been able to reproduce this issue (I know "it worked on my machine" is the worst answer. But I did do a fresh setup on two other machines to see if I can recreate so it worked on three of my machines?). If you want to drop me an email (my email is in my profile), I'm happy to help find time to debug live and I'd personally love to find out what the issue is. And/or hook you up with a t-shirt :) Digging in, this specific error looks like it is happening when LangChain is trying to strip out new lines (you can see the code in LangChain here: https://github.com/hwchase17/langchainjs/blob/main/langchain/src/embeddings/openai.ts#L164 https://github.com/hwchase17/langchainjs/blob/main/langchain...). For some reason your documents or embeddings are undefined. But I've been unable to create a scenario where that happens. I wondered if perhaps your OpenAI API Key wasn't set correctly and it was causing the Embeddings to fail, but I tried both deleting my key and setting to bogus key and the error was different. Any chance you switched out the URL we were pulling from? That didn't cause the same error for me either, but thinking through what else it could be. Sorry to not resolve, but please feel free to drop me an email. I'll keep trying to reproduce as well, and will jump back in this comment if I do (and find a fix).
- tmikaeld 3y ago
  The reason was that I was using json for the input.url input.query - but for some reason, even thought it's just plain strings, it has this error, when i use query strings it just works.