5 ms·
Related question: is this done on my machine or do I end up sending possible pii to a cloud service for evaluation?
by harvey9 4y ago
Related question: is this done on my machine or do I end up sending possible pii to a cloud service for evaluation?
- bluecoconut 4y agoThis is sending summary statistics to a cloud machine by default (for ease of immediate use. https://github.com/approximatelabs/sketch#sketch-currently-uses-promptsapproxdev-to-help-run-with-minimal-setup https://github.com/approximatelabs/sketch#sketch-currently-u... You can run using your own OpenAI key by setting 2 environment variables: (1) SKETCH_USE_REMOTE_LAMBDAPROMPT=False (2) OPENAI_API_KEY=YOUR_API_KEY To run entirely locally (using your own GPU and a model like Bloom) one would have to add a new prompt type to `lambdaprompt` (the package that this depends on), have a machine with enough GPU resources, and then add a slight modification to sketch.
- adabyron 4y agoNot sure if this is a business you're building out of this or an experiment. For real use for any of my customers, I would need to run this entirely locally. I think it's really awesome though! Curious what "enough GPU resources" looks like? Would a GeForce RTX 40 or 30 series card with 12-24GB of RAM be sufficient per user running locally on their machine?
- pklee 4y agoIf this is using OpenAI which it seems is what it is using, It is only sending column headers / column names. Not the data. If you are concerned about column names, you could also mask it on the way out and back in. If you are looking for an end to end database connect and query, please reach out to me.