4 ms·

Hey, I helped make this! Happy to answer any questions.

by tasdfqwer0897 4y ago

aaaaaaaaaaab 4y ago
What was the training data?
- version_five 4y ago
  Also, are there benchmark tasks that you either created or that already exist that you evaluated the model on? PS - please don't let this me used at a way to prevent human interaction. Chatbots are a disaster and literally the worst possible application of ML, as a shitty interface to a menu system. I hope this will be used in a way that is not consumer-hostile and that the company actively resists ignorant business attempts to use it to avoid paying for customer support.
  - tasdfqwer0897 4y ago
    Yeah, we did have to custom-build our own benchmarks. And we are not building a chatbot, we're building something collaborative that you can work with to accomplish the stuff you want to do!
- tasdfqwer0897 4y ago
  We used a combination of human demonstrations and feedback data! You need custom software both to record the demonstrations and to represent the state of the Tool in a model-consumable way.
  - elcomet 4y ago
    How many demonstrations were used ? And was the feedback data used to train the model with reinforcement learning? Or did you request users to "correct" the action and get a supervised signal?
  - angrais 4y ago
    How were the demonstrations annotated? Did you use annotations?
tux3 4y ago
On an unrelated note, I imagine this can solve recaptchas and other simple non-visual challenges. Can I make ACT-1 Sybil a few thousand people on mechanical turk? Can I submit CVs with ACT-1 for entry-level full remote jobs and have it work for legacy companies, if those companies cannot setup ACT-1 themselves but provide a traditional human jobs interface? Can I put an interface that interracts with the real world through controls and text on a webpage and have ACT-1 take a physical presence?
- woojoo666 4y ago
  I thought captcha was designed specifically so that AI image recognition couldn't perform well at it
  - TuringTest 4y ago
    It was, for previous methods of image recognition. Generative adversarial networks and models trained on huge amounts of data might change that.
learndeeply 4y ago
Thanks for answering questions! Are the example given in the blog post considered zero-shot learning? Was the model trained on the websites in the examples given (e.g. on the Redfin site)? How much labeled data was used?
lee101 4y ago
I'd be interested to see if we can work together in some way, i'm founder of https://text-generator.io https://text-generator.io which also crawls links to the web/images to generate better text: https://text-generator.io/blog/text-generator-now-researches-via-crawling https://text-generator.io/blog/text-generator-now-researches... Some people self host it, so it can solve some subset of the questions, the image understanding part is important, both it can understand a question with image e.g. if a user drops a fridge to search by image, or multiple images (e.g. which of the 10 images is the nicest looking fridge in 1 API request) as well. Also supports getting shared embeddings for images/text/code, which can be important for the information retrieval/question answering example where it needs to first find the relevant context on wikipedia then feed to the reader model to read it out Also do other custom stuff like retraining etc. Thanks, Lee https://leepenkman.appspot.com/ https://leepenkman.appspot.com/
blueblimp 4y ago
In the Salesforce example, it's modifying database contents. Suppose the model misunderstands your request and modifies data in an unintended way (e.g. adding garbage data or, worse, deleting data). What's the recovery plan?
WiggleGuy 4y ago
This is super cool; I'd want to stay updated. Do you think you guys could add an rss feed to your blog? I'd want to add it to my rss feed aggregator.

Hey, I helped make this! Happy to answer any questions.

aaaaaaaaaaab 4y ago

What was the training data?

version_five 4y ago
Also, are there benchmark tasks that you either created or that already exist that you evaluated the model on? PS - please don't let this me used at a way to prevent human interaction. Chatbots are a disaster and literally the worst possible application of ML, as a shitty interface to a menu system. I hope this will be used in a way that is not consumer-hostile and that the company actively resists ignorant business attempts to use it to avoid paying for customer support.
- tasdfqwer0897 4y ago
  Yeah, we did have to custom-build our own benchmarks. And we are not building a chatbot, we're building something collaborative that you can work with to accomplish the stuff you want to do!
tasdfqwer0897 4y ago
We used a combination of human demonstrations and feedback data! You need custom software both to record the demonstrations and to represent the state of the Tool in a model-consumable way.
- elcomet 4y ago
  How many demonstrations were used ? And was the feedback data used to train the model with reinforcement learning? Or did you request users to "correct" the action and get a supervised signal?
- angrais 4y ago
  How were the demonstrations annotated? Did you use annotations?

tux3 4y ago

On an unrelated note, I imagine this can solve recaptchas and other simple non-visual challenges. Can I make ACT-1 Sybil a few thousand people on mechanical turk? Can I submit CVs with ACT-1 for entry-level full remote jobs and have it work for legacy companies, if those companies cannot setup ACT-1 themselves but provide a traditional human jobs interface? Can I put an interface that interracts with the real world through controls and text on a webpage and have ACT-1 take a physical presence?

woojoo666 4y ago
I thought captcha was designed specifically so that AI image recognition couldn't perform well at it
- TuringTest 4y ago
  It was, for previous methods of image recognition. Generative adversarial networks and models trained on huge amounts of data might change that.

learndeeply 4y ago

Thanks for answering questions! Are the example given in the blog post considered zero-shot learning? Was the model trained on the websites in the examples given (e.g. on the Redfin site)? How much labeled data was used?

lee101 4y ago

I'd be interested to see if we can work together in some way, i'm founder of https://text-generator.io https://text-generator.io which also crawls links to the web/images to generate better text: https://text-generator.io/blog/text-generator-now-researches-via-crawling https://text-generator.io/blog/text-generator-now-researches... Some people self host it, so it can solve some subset of the questions, the image understanding part is important, both it can understand a question with image e.g. if a user drops a fridge to search by image, or multiple images (e.g. which of the 10 images is the nicest looking fridge in 1 API request) as well. Also supports getting shared embeddings for images/text/code, which can be important for the information retrieval/question answering example where it needs to first find the relevant context on wikipedia then feed to the reader model to read it out Also do other custom stuff like retraining etc. Thanks, Lee https://leepenkman.appspot.com/ https://leepenkman.appspot.com/

blueblimp 4y ago

In the Salesforce example, it's modifying database contents. Suppose the model misunderstands your request and modifies data in an unintended way (e.g. adding garbage data or, worse, deleting data). What's the recovery plan?

WiggleGuy 4y ago

This is super cool; I'd want to stay updated. Do you think you guys could add an rss feed to your blog? I'd want to add it to my rss feed aggregator.