5 ms·

Trainable Llama-like transformer (with backpropagation) in numpy only (~600 lines) https://github.com/joennlae/tensorli https://github.com/joennlae/tensorli

• Zambyte 2y ago
  The description says GPT-like, but is is just a GPT, right?
  • p1esk 2y ago
    GPT refers to the specific family of models developed at OpenAI.
    • Zambyte 2y ago
      It also stands for generative pretrained transformer, which this seems to be.
      • p1esk 2y ago
        It’s like saying SSD is a YOLO. Both are single shot object detectors, but only YOLO is “a YOLO”.