b44

3 ms·

1. 1.▲
   by b44 8mo ago

   the untrained model is literally just generating random characters, whereas your examples are at least pronouncable. you can add more layers to get progressively better results.
2. 2.▲
   by b44 8mo ago

   good catch - i intentionally cap node visualizations at 16 so it doesn't get super long, but the sidebar shouldn't have that
3. 3.▲
   by b44 8mo ago

   not many. diminishing returns start before 1000 and past that you should just add a second/third layer
4. 4.▲
   by b44 8mo ago

   hm. the way i see things, characters are the natural/obvious building blocks and tokenization is just an improvement on that. i do mention chatgpt et al. use tokens in the last q&a dropdown, though
5. 5.▲
   Show HN: Microgpt is a GPT you can visualize in the browser(microgpt.boratto.ca)

   282 points by b44 8mo ago|24 comments