4 ms·

No datas are sent to any 3rd party.YoBulk is self hosted.Your personal information is stored on your database.Feel free to ask any data security related questio

by yosai 4y ago

No datas are sent to any 3rd party.YoBulk is self hosted.Your personal information is stored on your database.Feel free to ask any data security related question.

counttheforks 4y ago
How are you running GPT locally? OpenAI is a third party.
- wstuartcl 4y ago
  it looks like the only thing openai does is generate a schema from a specific hand written prompt from cursory overview.
  - yosai 4y ago
    There are multiple use cases where we are using Open AI Example:uploaded CSV's column to template column string matching
    - jimlongton 4y ago
      That makes sense. It would be good to add this information to the README.
      - yosai 4y ago
        Sure We will add it in README..we captured it in our documentation.Please have a look. https://doc.yobulk.dev/YoBulk%20AI/AI%20usecases https://doc.yobulk.dev/YoBulk%20AI/AI%20usecases
- yosai 4y ago
  We are using OpenAI to only to create schema, column matching and Regex generation.No CSV data are sent to open AI..
  - nerdponx 4y ago
    So you ask OpenAI to generate a schema and computer code that will clean input data to conform to that schema, and then run the user's data through that program? Is it possible for users to obtain and audit the generated code for correctness, performance, etc.? How do you prevent things like the AI from generating catastrophically-backtracking regex, O(N!) algorithms, or outright mistakes?
    - yosai 4y ago
      @nerdponx Open AI only generate a validation schema.We have our own schema generation engine for any custom validation which can be used where Open AI is not able to understand or generate correct schema.Yes you are absolutely right.Open AIs output is not always right.We have integrated a JSON parser which validate Open AI output and currently developing a regex parser which validates Open AIs output.Hope it answers your query..Happy to understand your pain point more.
      - nerdponx 4y ago
        Interesting, thanks for explaining how it works. Would it also be possible to construct one of these JSON schemas by hand, without OpenAI? The core data cleaning system sounds like just as interesting a piece of technology as the AI schema generation.
        yosai 4y ago
        @nerdponx Yes YoBulk provides a way to write JSON schemas by hand..We have added some custom keyword like validate which is not defined in JSON standard..But our validation engine can understand that..You can pass a javascript function through validate keyword..It's a game changer.. "first_name":{ "type":"string", "format": "first_name", "validate": "(x) => {\r\n let regex = new RegExp(\"([a-z][a-zA-Z]*)\");\r\n return regex.test(x);\r\n }" },
        nerdponx 4y ago
- l33t233372 4y ago
  I understand that this product does use OpenAI’s API, but I just want to stress that OpenAI doesn’t own GPT, it did not create transformer models, and GPT is not something usable exclusively through OpenAI. A priori, there’s no reason they couldn’t include their own GPT model that also lives on your own server.
  - elevenoh 4y ago
    [dead]
  - olalonde 4y ago
    According to Wikipedia, OpenAI did create GPT[0] and the GPT-3 model[1], which this tool depends on, is exclusively available through the OpenAI API. It's true that some open alternatives to GPT-3 have been popping up but I believe they are still behind in terms of quality. [0] https://en.wikipedia.org/wiki/Generative_pre-trained_transformer#History https://en.wikipedia.org/wiki/Generative_pre-trained_transfo... [1] https://en.wikipedia.org/wiki/GPT-3 https://en.wikipedia.org/wiki/GPT-3
    - l33t233372 4y ago
      Thank you for adding a correction. I shouldn’t have been so flippant: OpenAI did not invent transformers, GPT-3 is an iteration of the BERT architecture, which is an encoder, whereas GPT is a decoder model. Regardless, a GPT model is not a proprietary technology you must get from OpenAI. You’re right that this tool uses OpenAI’s API, but this isn’t necessarily implied by the use of GPT.
      - coolspot 4y ago
        > Thank you for adding a correction. I shouldn’t have been so flippant: OpenAI did not invent transformers BTW you answer structure is very similar to ChatGPT’s when you point it to a mistake.
        l33t233372 4y ago

by yosai 4y ago

No datas are sent to any 3rd party.YoBulk is self hosted.Your personal information is stored on your database.Feel free to ask any data security related question.

counttheforks 4y ago

How are you running GPT locally? OpenAI is a third party.

wstuartcl 4y ago
it looks like the only thing openai does is generate a schema from a specific hand written prompt from cursory overview.
- yosai 4y ago
  There are multiple use cases where we are using Open AI Example:uploaded CSV's column to template column string matching
  - jimlongton 4y ago
    That makes sense. It would be good to add this information to the README.
    - yosai 4y ago
      Sure We will add it in README..we captured it in our documentation.Please have a look. https://doc.yobulk.dev/YoBulk%20AI/AI%20usecases https://doc.yobulk.dev/YoBulk%20AI/AI%20usecases
yosai 4y ago
We are using OpenAI to only to create schema, column matching and Regex generation.No CSV data are sent to open AI..
- nerdponx 4y ago
  So you ask OpenAI to generate a schema and computer code that will clean input data to conform to that schema, and then run the user's data through that program? Is it possible for users to obtain and audit the generated code for correctness, performance, etc.? How do you prevent things like the AI from generating catastrophically-backtracking regex, O(N!) algorithms, or outright mistakes?
  - yosai 4y ago
    @nerdponx Open AI only generate a validation schema.We have our own schema generation engine for any custom validation which can be used where Open AI is not able to understand or generate correct schema.Yes you are absolutely right.Open AIs output is not always right.We have integrated a JSON parser which validate Open AI output and currently developing a regex parser which validates Open AIs output.Hope it answers your query..Happy to understand your pain point more.
    - nerdponx 4y ago
      Interesting, thanks for explaining how it works. Would it also be possible to construct one of these JSON schemas by hand, without OpenAI? The core data cleaning system sounds like just as interesting a piece of technology as the AI schema generation.
      - yosai 4y ago
        @nerdponx Yes YoBulk provides a way to write JSON schemas by hand..We have added some custom keyword like validate which is not defined in JSON standard..But our validation engine can understand that..You can pass a javascript function through validate keyword..It's a game changer.. "first_name":{ "type":"string", "format": "first_name", "validate": "(x) => {\r\n let regex = new RegExp(\"([a-z][a-zA-Z]*)\");\r\n return regex.test(x);\r\n }" },
        nerdponx 4y ago
l33t233372 4y ago
I understand that this product does use OpenAI’s API, but I just want to stress that OpenAI doesn’t own GPT, it did not create transformer models, and GPT is not something usable exclusively through OpenAI. A priori, there’s no reason they couldn’t include their own GPT model that also lives on your own server.
- elevenoh 4y ago
  [dead]
- olalonde 4y ago
  According to Wikipedia, OpenAI did create GPT[0] and the GPT-3 model[1], which this tool depends on, is exclusively available through the OpenAI API. It's true that some open alternatives to GPT-3 have been popping up but I believe they are still behind in terms of quality. [0] https://en.wikipedia.org/wiki/Generative_pre-trained_transformer#History https://en.wikipedia.org/wiki/Generative_pre-trained_transfo... [1] https://en.wikipedia.org/wiki/GPT-3 https://en.wikipedia.org/wiki/GPT-3
  - l33t233372 4y ago
    Thank you for adding a correction. I shouldn’t have been so flippant: OpenAI did not invent transformers, GPT-3 is an iteration of the BERT architecture, which is an encoder, whereas GPT is a decoder model. Regardless, a GPT model is not a proprietary technology you must get from OpenAI. You’re right that this tool uses OpenAI’s API, but this isn’t necessarily implied by the use of GPT.
    - coolspot 4y ago
      > Thank you for adding a correction. I shouldn’t have been so flippant: OpenAI did not invent transformers BTW you answer structure is very similar to ChatGPT’s when you point it to a mistake.
      - l33t233372 4y ago