4 ms·

I tried it out and it's pretty pricey. My OpenAI API bill is $3.20 after using this on a few different pages to test it out. Not saying I wouldn't pay that for

I tried it out and it's pretty pricey. My OpenAI API bill is $3.20 after using this on a few different pages to test it out. Not saying I wouldn't pay that for some use cases, but it would limit me. One idea: making scrapers is a big pain. But once they are setup, they are cheap and fast to run... this is always going to be slower. What I'd love to see is a way to generate scrapers quickly. So you wouldn't be returning information from the New York City property registry... instead, you'd return Python code that I can use to scrape it in the future. edit: This is likely because it was struggling, so it had to make extra calls. What would be nice is a simple feature where you can input the maximum number of calls / tokens to use on the entire call. Or even better, do some math and put in a dollar cap. i.e., go fill out the Geico forms for me and don't spend more than $1.00 doing it.

tmountain 3y ago
Just piggybacking here, but this is a great suggestion. It makes the cost a one-time expense, and you get something material (source code) in return.
suchintan 3y ago
I love all of these ideas!! 1. You can set a "max steps" limit when you run it locally https://github.com/Skyvern-AI/skyvern/blob/d0935755963b017ed2d207cfd1087d2fed397d20/.env.example#L15 https://github.com/Skyvern-AI/skyvern/blob/d0935755963b017ed... We also spit out the cost for each step within the visualizer. Click on any task > Steps > there's a column that's dedicated to how much things cost to run https://github.com/Skyvern-AI/skyvern/issues/70 https://github.com/Skyvern-AI/skyvern/issues/70 2. We have a roadmap item to "cache" or "memorize" specific tasks, so you pay the cost once, and then just run it over and over again. We're going to get to it soon!!
- suchintan 3y ago
  https://github.com/Skyvern-AI/Skyvern/?tab=readme-ov-file#feature-roadmap https://github.com/Skyvern-AI/Skyvern/?tab=readme-ov-file#fe...
keremyilmaz 3y ago
You've raised valid points about the cost and efficiency of our approach, which aims to make the LLM function as closely as possible to a human user. We chose this approach primarily for its compatibility with various websites, as it aligns closely with a website's intended audience, which is typically human. Addressing complex website interactions is a key advantage of this approach. For instance, in the process of generating an auto insurance quote, the sequence of questions and their specifics can vary greatly depending on prior responses. A simple example is the choice of a foreign versus a California driver's license. Selecting a foreign license triggers additional queries about the country of issuance and expiry date, illustrating the complexity and branching nature of such web interactions. However, we recognize the concerns about cost and are actively working on strategies to reduce it: - Optimizing the context provided to the LLM - Implementing caching mechanisms for certain repeated actions and only use LLMs when there's a problem - Anticipating advancements in LLM efficiency and cost-effectiveness, with the hope of eventually finetuning our own models for greater efficiency
- dtnewman 3y ago
  I like this approach. Just as an example, if I'm getting a car insurance quote, I'd rather pay $1 to have the tool fill out the forms for me and be 90% that it filled them out correctly rather than pay $0.01 and only be 70% sure it did it correctly. And there are plenty of use cases like that.
  - amne 3y ago
    isn't that crazy rabbit thingy supposed to do just that? I hope you pre-ordered. I hear they're in great demand.
    - daniel_iversen 3y ago
      https://www.rabbit.tech/research https://www.rabbit.tech/research
  - Kerbonut 3y ago
    You would still be willing to pay $1 if it got it wrong 10% of the time, or if it got 10% of the information wrong every time?
jumploops 3y ago
Scrapers are one of the main use cases we're seeing for Magic Loops[0]. ...and you've hit the nail on the head in terms of our design philosophy: use LLMs to generate useful logic, then run that logic without needing to call an LLM/Agent. With that said, we don't support browser automation. Skyvern is very neat, it reminds me of VimGPT[1], but with a more robust planning implementation. [0] https://magicloops.dev https://magicloops.dev [1] https://github.com/ishan0102/vimGPT https://github.com/ishan0102/vimGPT
- suchintan 3y ago
  Nice! Thanks for sharing this. We tried approaches like VimGPT before but found the rate of hallucinations to be a bit too high to be used in production. The sweet spot definitely seems to be to combine the magic of Dom parsing AND vision We're going to definitely work on logic generation and execution, but we're taking it a bit more carefully. Many of the workflows we automate have changing workflow steps (ie I've never seen the exact same Geico flow twice), but this certainly isn't true for all workflows
- umaar 3y ago
  Really like the simplicity of your website. I think when you first announced it, you mentioned you might open source Magic Loops, might you do that?
  - jumploops 3y ago
    Yes! We’re in the middle of cleaning things up, just need to make the Loops a bit more portable/easy to run, but finally happy with the state of the tool.
    - anhner 3y ago
      This brings me so much joy! Thank you for considering this!
enlyth 3y ago
It's getting genuinely difficult these days with everything walled behind Cloudflare, various anti-bot protections and increasingly creative CAPTCHAs
kilroy123 3y ago
Yes, exactly what I want. I want to be able to have it code robust Cypress tests for e2e testing.
bigfatfrock 3y ago
> instead, you'd return Python code that I can use to scrape it in the future Bravo, I would pay for this one, or hopefully run it on my GPU - it would be so fast to even just shove out your selectors (xpath, css, dealer's choice) for point-by-point update after you had done an initial code gen, or perhaps it could just diff and update chunks of code for you! My local code model can already do the diff update stuff in nvim, but being able to pass it a URL and have it slam in all of the pertinent crawling code, wow.
daniel65464 3y ago
Interesting enough I made a chrome extension that does almost exactly what you are describing. It’s called automize and it lets you very quickly generate custom selectors and export the code to puppeteer, playwright, selenium etc. it handles all the verifications as well as provides a handy ui that shows what you are selecting
qwbfsa 3y ago
This requires AI to learn user behavior flow data.

tmountain 3y ago

Just piggybacking here, but this is a great suggestion. It makes the cost a one-time expense, and you get something material (source code) in return.

suchintan 3y ago

I love all of these ideas!! 1. You can set a "max steps" limit when you run it locally https://github.com/Skyvern-AI/skyvern/blob/d0935755963b017ed2d207cfd1087d2fed397d20/.env.example#L15 https://github.com/Skyvern-AI/skyvern/blob/d0935755963b017ed... We also spit out the cost for each step within the visualizer. Click on any task > Steps > there's a column that's dedicated to how much things cost to run https://github.com/Skyvern-AI/skyvern/issues/70 https://github.com/Skyvern-AI/skyvern/issues/70 2. We have a roadmap item to "cache" or "memorize" specific tasks, so you pay the cost once, and then just run it over and over again. We're going to get to it soon!!

suchintan 3y ago
https://github.com/Skyvern-AI/Skyvern/?tab=readme-ov-file#feature-roadmap https://github.com/Skyvern-AI/Skyvern/?tab=readme-ov-file#fe...

keremyilmaz 3y ago

You've raised valid points about the cost and efficiency of our approach, which aims to make the LLM function as closely as possible to a human user. We chose this approach primarily for its compatibility with various websites, as it aligns closely with a website's intended audience, which is typically human. Addressing complex website interactions is a key advantage of this approach. For instance, in the process of generating an auto insurance quote, the sequence of questions and their specifics can vary greatly depending on prior responses. A simple example is the choice of a foreign versus a California driver's license. Selecting a foreign license triggers additional queries about the country of issuance and expiry date, illustrating the complexity and branching nature of such web interactions. However, we recognize the concerns about cost and are actively working on strategies to reduce it: - Optimizing the context provided to the LLM - Implementing caching mechanisms for certain repeated actions and only use LLMs when there's a problem - Anticipating advancements in LLM efficiency and cost-effectiveness, with the hope of eventually finetuning our own models for greater efficiency

dtnewman 3y ago
I like this approach. Just as an example, if I'm getting a car insurance quote, I'd rather pay $1 to have the tool fill out the forms for me and be 90% that it filled them out correctly rather than pay $0.01 and only be 70% sure it did it correctly. And there are plenty of use cases like that.
- amne 3y ago
  isn't that crazy rabbit thingy supposed to do just that? I hope you pre-ordered. I hear they're in great demand.
  - daniel_iversen 3y ago
    https://www.rabbit.tech/research https://www.rabbit.tech/research
- Kerbonut 3y ago
  You would still be willing to pay $1 if it got it wrong 10% of the time, or if it got 10% of the information wrong every time?

jumploops 3y ago

Scrapers are one of the main use cases we're seeing for Magic Loops[0]. ...and you've hit the nail on the head in terms of our design philosophy: use LLMs to generate useful logic, then run that logic without needing to call an LLM/Agent. With that said, we don't support browser automation. Skyvern is very neat, it reminds me of VimGPT[1], but with a more robust planning implementation. [0] https://magicloops.dev https://magicloops.dev [1] https://github.com/ishan0102/vimGPT https://github.com/ishan0102/vimGPT

suchintan 3y ago
Nice! Thanks for sharing this. We tried approaches like VimGPT before but found the rate of hallucinations to be a bit too high to be used in production. The sweet spot definitely seems to be to combine the magic of Dom parsing AND vision We're going to definitely work on logic generation and execution, but we're taking it a bit more carefully. Many of the workflows we automate have changing workflow steps (ie I've never seen the exact same Geico flow twice), but this certainly isn't true for all workflows
umaar 3y ago
Really like the simplicity of your website. I think when you first announced it, you mentioned you might open source Magic Loops, might you do that?
- jumploops 3y ago
  Yes! We’re in the middle of cleaning things up, just need to make the Loops a bit more portable/easy to run, but finally happy with the state of the tool.
  - anhner 3y ago
    This brings me so much joy! Thank you for considering this!

enlyth 3y ago

It's getting genuinely difficult these days with everything walled behind Cloudflare, various anti-bot protections and increasingly creative CAPTCHAs

kilroy123 3y ago

Yes, exactly what I want. I want to be able to have it code robust Cypress tests for e2e testing.

bigfatfrock 3y ago

> instead, you'd return Python code that I can use to scrape it in the future Bravo, I would pay for this one, or hopefully run it on my GPU - it would be so fast to even just shove out your selectors (xpath, css, dealer's choice) for point-by-point update after you had done an initial code gen, or perhaps it could just diff and update chunks of code for you! My local code model can already do the diff update stuff in nvim, but being able to pass it a URL and have it slam in all of the pertinent crawling code, wow.

daniel65464 3y ago

Interesting enough I made a chrome extension that does almost exactly what you are describing. It’s called automize and it lets you very quickly generate custom selectors and export the code to puppeteer, playwright, selenium etc. it handles all the verifications as well as provides a handy ui that shows what you are selecting

qwbfsa 3y ago

This requires AI to learn user behavior flow data.