3 ms·

I am a programmer and a blogger, like the author. Unlike the author, I do not monetize my blog in any way (or I assume he does on Substack). The author is both

I am a programmer and a blogger, like the author. Unlike the author, I do not monetize my blog in any way (or I assume he does on Substack). The author is both right and wrong, just about different things. First, he assumes GPT will get exponentially better. If that were true, all cars would be full self-driving already. GPT will have an S curve or Sigmoid function. Second, he assumes that GPT will always be able to produce things from differing voices well. However, as more data is used, these bots will only become more homogenous. You can see clues of this when it managed to make a Biggie Smalls rap, but did not with Woody Gunthrie, whose data is probably closer to the mainstream than Biggie's. (This is a gut feeling; could be wrong.) Third, as things become more of the same bot-like feeling, even from people, those who have their own voice and touch will stand out more. You can see this in his essay, versus the two written by the bot. The voice of the bot ones feels more stilted to me. His feels more natural. What people need to do is to develop their own voice and touch. In writing, develop your voice. Write without help. Write a lot. Create a blog. Put random stupid stuff there. Let yourself rant. That random stupid stuff and those rants will not be mainstream, and if it's stupid and ranty, you won't have to care about quality. That lack of care will let you have fun, and that fun will become your voice. This is how I developed my voice with my blog. In code, develop your touch. Don't just code; design. Think about concepts. Think about UX and workflows. Iterate until everything is great. Then design the implementation and iterate until everything fits. Then, and only then, code. But as you code, iterate the design to remove things that no longer fit and add things that do. Dogfood your software. Use it for everything you can, even things that don't fit well. Iterate and change some more to make your dogfooding easier. The end result will be software that is easy-to-use and fits the users' needs well. That fit will be your touch. This is how I developed my touch with my most well-known project: `bc`. [1] But you don't even need to stop there. With my `bc`, I spent time supporting users and implementing things for them. Obviously, I still designed changes before implementing them, but I listened to my users and gave them what they wanted within the constraints I had. The end result is a `bc` that can act like the GNU `bc` or the BSD `bc`, whichever you want. It can also be its own thing with extra features. Listening to users is another human touch. Make use of it. In fact, that's where I will make a living from my current project: my support for it will be best-in-class [2], and that's something GPT can never replicate because it cannot replicate my brain, my voice, and my touch, even when trained on my writing. In other words, the author is write that the 90% is useless, but those 90% were the ones where you were at least partially imitating a bot anyway. Those 10% are the human traits. If you want to have useful skills, be a human, not a fleshy bot. Footnote: I feel bad for anyone that gets my code or writing as output from GPT. My code fits my head and no one else's. It also uses custom API's that don't exist anywhere else, which means such output would be useless. The same goes for my writing, though to a lesser extent. [1]: https://github.com/gavinhoward/bc https://github.com/gavinhoward/bc [2]: https://github.com/gavinhoward/bc/issues/66 https://github.com/gavinhoward/bc/issues/66

slig 4y ago
>GPT will have an S curve or Sigmoid function How can you tell the difference between "we're in the beginning of the curve" vs "we're at the end of it" vs "there's no S curve"?
- gavinhoward 4y ago
  Several ways. First, diminishing returns. When people start talking about exponential amounts of resources required (data or compute) or the results are not growing as fast, you're approaching the top. Second, historical precedent. Self-driving cars have been around the corner for years now. I don't expect them anytime soon. Third, practical limits. These LLM's are trained on human text. Therefore, their output will only approach the best humans can do in the limit and in the best case. In the average case, they will only have average output, and their output will be the average of everything, making it flavorless. This is also how you know it's an S curve and not a hockey stick: it cannot be any better than the training data. Fourth, limits of the model. These LLM's predict the next sequences of words (or in the case of picture bots, they predict a sequence of pixels). That means they can only do things that have those inputs and require or accept those outputs. You're not going to get an LLM to drive a car, and you're not going to get a self-driving car to output a sequence of text. You may add an LLM to a self-driving car, but it cannot communicate with the driving code because they have mismatched inputs and outputs.
  - slig 4y ago
    Thank you!

slig 4y ago

>GPT will have an S curve or Sigmoid function How can you tell the difference between "we're in the beginning of the curve" vs "we're at the end of it" vs "there's no S curve"?

gavinhoward 4y ago
Several ways. First, diminishing returns. When people start talking about exponential amounts of resources required (data or compute) or the results are not growing as fast, you're approaching the top. Second, historical precedent. Self-driving cars have been around the corner for years now. I don't expect them anytime soon. Third, practical limits. These LLM's are trained on human text. Therefore, their output will only approach the best humans can do in the limit and in the best case. In the average case, they will only have average output, and their output will be the average of everything, making it flavorless. This is also how you know it's an S curve and not a hockey stick: it cannot be any better than the training data. Fourth, limits of the model. These LLM's predict the next sequences of words (or in the case of picture bots, they predict a sequence of pixels). That means they can only do things that have those inputs and require or accept those outputs. You're not going to get an LLM to drive a car, and you're not going to get a self-driving car to output a sequence of text. You may add an LLM to a self-driving car, but it cannot communicate with the driving code because they have mismatched inputs and outputs.
- slig 4y ago
  Thank you!