4 ms·

I'm waiting on my GPT-4 API access so I can use gpt-4-32k which maybe can soak up 10k LOC? Clearly this will break eventually, but I am playing around with som

by mpoon 4y ago

I'm waiting on my GPT-4 API access so I can use gpt-4-32k which maybe can soak up 10k LOC? Clearly this will break eventually, but I am playing around with some ideas to extend how much context I can give it. One is to do something like base64 encode file contents. I've seen some early success that GPT-4 knows how to decode it, so that'll allow me to stuff more characters into it. I'm also hoping that with the use of .gptignore, I can just selectively give the files I think are relevant for whatever prompt I'm writing.

Bjartr 4y ago
> GPT-4 knows how to decode it I wonder if you could teach it to understand a binary encoding using the raw bytestream, feed it compressed text, and just tell it to decompress it first.
- ozfive 4y ago
  Here is what GPT-4 says about it. "As an AI language model, I can understand and work with various text encoding schemes and compression algorithms. However, to work with a raw bytestream, you would need to provide specific details about the encoding and compression used. To teach me to understand a particular binary encoding and compressed text format, you should provide the following information: The binary encoding used (e.g., ASCII, UTF-8, UTF-16, etc.). The compression algorithm employed (e.g., gzip, Lempel-Ziv-Welch (LZW), Huffman coding, etc.). Once you provide these details, I can help you process the raw bytestream and decompress the text. However, keep in mind that my primary focus is on natural language understanding and generation, and I might not be as efficient at handling compressed data as a dedicated compression/decompression tool."
  - tablatom 4y ago
    When GPT gives an answer like that, is it actually a meaningful description of its capabilities? Does it have that kind of self-awareness? Or is it just a plausible answer based on the training corpus? Genuine question.
    - Olphs 4y ago
      My guess is that the training data includes things specifically about the GPT itself and its capabilities, so it would be somewhat correct. But it's also known to just make shit up when it feels like it, so you can't 100% trust it, same as with all other prompts/responses.
dc-programmer 4y ago
Base64 encoding increases the size of text by 4/3. Like the other commenter asked, I wonder if another encoding could work
WXLCKNO 4y ago
Have multiple instances of Gpt4 with different parts of the codebase interact with each other to write the whole thing. Probably doesn't work this way lol

by mpoon 4y ago

Bjartr 4y ago

> GPT-4 knows how to decode it I wonder if you could teach it to understand a binary encoding using the raw bytestream, feed it compressed text, and just tell it to decompress it first.

ozfive 4y ago
Here is what GPT-4 says about it. "As an AI language model, I can understand and work with various text encoding schemes and compression algorithms. However, to work with a raw bytestream, you would need to provide specific details about the encoding and compression used. To teach me to understand a particular binary encoding and compressed text format, you should provide the following information: The binary encoding used (e.g., ASCII, UTF-8, UTF-16, etc.). The compression algorithm employed (e.g., gzip, Lempel-Ziv-Welch (LZW), Huffman coding, etc.). Once you provide these details, I can help you process the raw bytestream and decompress the text. However, keep in mind that my primary focus is on natural language understanding and generation, and I might not be as efficient at handling compressed data as a dedicated compression/decompression tool."
- tablatom 4y ago
  When GPT gives an answer like that, is it actually a meaningful description of its capabilities? Does it have that kind of self-awareness? Or is it just a plausible answer based on the training corpus? Genuine question.
  - Olphs 4y ago
    My guess is that the training data includes things specifically about the GPT itself and its capabilities, so it would be somewhat correct. But it's also known to just make shit up when it feels like it, so you can't 100% trust it, same as with all other prompts/responses.

dc-programmer 4y ago

Base64 encoding increases the size of text by 4/3. Like the other commenter asked, I wonder if another encoding could work

WXLCKNO 4y ago

Have multiple instances of Gpt4 with different parts of the codebase interact with each other to write the whole thing. Probably doesn't work this way lol