3 ms·

You must be joking, the title of this post literally says "build fast LLMs from scratch" but the code is neither an LLM, nor particularly fast. Python dicts ar

by mpeg 2y ago

You must be joking, the title of this post literally says "build fast LLMs from scratch" but the code is neither an LLM, nor particularly fast. Python dicts are actually different from JS objects, Python uses a hashmap behind the scenes while most fast js vms will apply some heuristics to decide whether to use a hashmap or to make the object as a static struct with a known offset for each key, this explains it better than I can: https://v8.dev/blog/fast-properties https://v8.dev/blog/fast-properties Nevertheless, for your specific use-case I would be really surprised if there was any significant difference between the performance of python and js – hashmaps are fast enough for this – plus with js objects you might be trading insertion performance for access performance in some cases, as there is overhead creating a v8-style "fast property" object

_akhe 2y ago
Not joking. You seem to be confused in thinking LLMs are built with other LLMs, but that's not the case. Otherwise why would you say "it says 'build fast LLMs from scratch' but the code is not an LLM" ?? Why would the code of a library to build LLMs need to also be an LLM? Getting off-topic but Python is incredibly slow at lookups (and most things) compared to JavaScript and it isn't even close, not all dynamic languages are the same. This is pretty widely known and a quick Google search yields plenty of benchmarks and articles! Give it a try. Python is used in AI/ML for its libraries (convenience) not because it's a fast language. There are 3 main reasons: 1) Time complexity of data structures is lower in JS, that's the primary exploit at play here 2) V8 compiles to machine code in less steps than Python and 3) Process forking - the concept that functions can run in parallel in the same thread. Thanks for your comment!
- mpeg 2y ago
  I feel you must be trolling now by how confidently incorrect you are, but in case you are not: > Time complexity of data structures is lower in JS What data structures? All of them? You know you can implement your own data structures if you need them to be optimal > V8 compiles to machine code in less steps than Python The "steps" or time it takes to compile has no bearing on runtime performance. v8 is generally faster on micro-benchmarks, but in python you spend most of your time calling out to libraries that are heavily optimised, the javascript library ecosystem is a complete joke for ML/AI compared to python's – for example there is nothing that can compare to numpy in js. > Process forking - the concept that functions can run in parallel in the same thread This is a OS feature and has nothing to do with js or python. I have to point out though that when you fork a process you are creating a new thread. This will be my last comment, it is clear to me now that you posted this thread to stroke your own ego and have no interest in actually learning anything. Good luck with your GPT-killer :)
  - _akhe 2y ago
    Not joking, not trolling, thought it was widely known that JavaScript is generally significantly faster than Python. Haven't had this debate in 15 years, but let me explain: > What data structures? Everything in JavaScript is an object, even Arrays. So even an Array lookup is O(1) in JavaScript - not true in Python where it has to search the Array. Only if you created key/val pairs in Python (via list/hash) could you exploit the O(1) lookup for accessing data, but I can't use Python list for a large model like an LLM without hitting memory errors (see: https://stackoverflow.com/questions/5537618/memory-errors-and-list-limits https://stackoverflow.com/questions/5537618/memory-errors-an...) > Python is run-time Both languages are dynamic (not compiled) lol, what are you trying to say here? The point is that the number of steps it takes to go from high-level dynamic code (scripts) to machine-readable instructions is 1 step in JS, but 2 steps in Python, that's literally why it's slower to RUN than JavaScript. Literally runs slower as in number of executions. > Multi-process is an OS feature that has nothing to do with JS or Python Not true in the slightest. It's a language feature. I'll use my favorite word of the day and say it's an "exploit" more than a feature, when you can run what would be blocking IO in parallel. Python on the other hand is "single flow" executing statements one-at-a-time. What a toxic comment, I said thanks to everyone else but not you. I retract my thank you! I hope you learned something today at least. This made me LOL: "JS uses some heuristics to decide whether to use a hashmap or to make the object as a static struct" neither hashmap nor struct exists in JS, just funny. There's ES6 Map, but that is really just an Object, not the other way around lol
    - selcuka 2y ago
      > Everything in JavaScript is an object, even Arrays. So even an Array lookup is O(1) in JavaScript - not true in Python where it has to search the Array. Huh? Array and list lookups are O(1) in Python too. Who told you that? Searching is different, and it's O(n) in both JS and Python. Can you really believe two mainstream programming languages can have such a drastic difference in time complexity? Also, not my comment, but: > JS uses some heuristics to decide whether to use a hashmap or to make the object as a static struct" neither hashmap nor struct exists in JS, just funny. I'm pretty sure they meant that Javascript internally uses either a hashmap or a struct (you can do something close in Python using __slots__) to represent an object. Those are standard data structure names, even though Javascript doesn't use the same terminology. Python doesn't call them hashmaps either.
- selcuka 2y ago
  Ok, not to belittle your work or anything, but I think you are either not using the words correctly, or trolling us here. If we used your "non-LLM" library to build an LLM, it wouldn't be "from scratch" would it? Therefore it is natural to assume that you meant that your code is supposed to be "an LLM written from scratch".
  - _akhe 2y ago
    Not trolling, though I feel the same way about you with this post haha. To answer your question, this library doesn't provide something like a `chat` method and doesn't provide anything like Stable Diffusion's `img2img` API either - that's not what it is - it's a library for predicting the next token (letter, word, pixel, etc.) based on data you train it on. The most typical use cases for this model are: Auto-completion, auto-correct, spell check, search, etc. However, if you watched the incredible video I shared in the original post you'd know that this completion concept can be applied to other interfaces: Chat, image generation, audio analysis/generation, etc. and in fact it is applied to LLMs like GPT. This library doesn't get into any chat-specific methods like `ask` (question & answer), it just completes token sequences. If your goal is to create a ChatGPT clone, you would have to add methods like `chat`, `ask`, etc. and provide code for ongoing conversations - probably using an NLP library to parse queries into cursors like I'm trying in another project, or use "MULTI-HEADED ATTENTION BLOCKS" (lmfao) if you fancy that. Or if you wanted to create a Stable Diffusion clone, you'd have to provide those image-related methods needed. As far as the meaning of "from scratch" - I mean compared to using Ollama etc. to run local models or using OpenAI - just trying to enable you to build whatever model suits your data and use case.

by mpeg 2y ago

_akhe 2y ago

Not joking. You seem to be confused in thinking LLMs are built with other LLMs, but that's not the case. Otherwise why would you say "it says 'build fast LLMs from scratch' but the code is not an LLM" ?? Why would the code of a library to build LLMs need to also be an LLM? Getting off-topic but Python is incredibly slow at lookups (and most things) compared to JavaScript and it isn't even close, not all dynamic languages are the same. This is pretty widely known and a quick Google search yields plenty of benchmarks and articles! Give it a try. Python is used in AI/ML for its libraries (convenience) not because it's a fast language. There are 3 main reasons: 1) Time complexity of data structures is lower in JS, that's the primary exploit at play here 2) V8 compiles to machine code in less steps than Python and 3) Process forking - the concept that functions can run in parallel in the same thread. Thanks for your comment!

mpeg 2y ago
I feel you must be trolling now by how confidently incorrect you are, but in case you are not: > Time complexity of data structures is lower in JS What data structures? All of them? You know you can implement your own data structures if you need them to be optimal > V8 compiles to machine code in less steps than Python The "steps" or time it takes to compile has no bearing on runtime performance. v8 is generally faster on micro-benchmarks, but in python you spend most of your time calling out to libraries that are heavily optimised, the javascript library ecosystem is a complete joke for ML/AI compared to python's – for example there is nothing that can compare to numpy in js. > Process forking - the concept that functions can run in parallel in the same thread This is a OS feature and has nothing to do with js or python. I have to point out though that when you fork a process you are creating a new thread. This will be my last comment, it is clear to me now that you posted this thread to stroke your own ego and have no interest in actually learning anything. Good luck with your GPT-killer :)
- _akhe 2y ago
  Not joking, not trolling, thought it was widely known that JavaScript is generally significantly faster than Python. Haven't had this debate in 15 years, but let me explain: > What data structures? Everything in JavaScript is an object, even Arrays. So even an Array lookup is O(1) in JavaScript - not true in Python where it has to search the Array. Only if you created key/val pairs in Python (via list/hash) could you exploit the O(1) lookup for accessing data, but I can't use Python list for a large model like an LLM without hitting memory errors (see: https://stackoverflow.com/questions/5537618/memory-errors-and-list-limits https://stackoverflow.com/questions/5537618/memory-errors-an...) > Python is run-time Both languages are dynamic (not compiled) lol, what are you trying to say here? The point is that the number of steps it takes to go from high-level dynamic code (scripts) to machine-readable instructions is 1 step in JS, but 2 steps in Python, that's literally why it's slower to RUN than JavaScript. Literally runs slower as in number of executions. > Multi-process is an OS feature that has nothing to do with JS or Python Not true in the slightest. It's a language feature. I'll use my favorite word of the day and say it's an "exploit" more than a feature, when you can run what would be blocking IO in parallel. Python on the other hand is "single flow" executing statements one-at-a-time. What a toxic comment, I said thanks to everyone else but not you. I retract my thank you! I hope you learned something today at least. This made me LOL: "JS uses some heuristics to decide whether to use a hashmap or to make the object as a static struct" neither hashmap nor struct exists in JS, just funny. There's ES6 Map, but that is really just an Object, not the other way around lol
  - selcuka 2y ago
    > Everything in JavaScript is an object, even Arrays. So even an Array lookup is O(1) in JavaScript - not true in Python where it has to search the Array. Huh? Array and list lookups are O(1) in Python too. Who told you that? Searching is different, and it's O(n) in both JS and Python. Can you really believe two mainstream programming languages can have such a drastic difference in time complexity? Also, not my comment, but: > JS uses some heuristics to decide whether to use a hashmap or to make the object as a static struct" neither hashmap nor struct exists in JS, just funny. I'm pretty sure they meant that Javascript internally uses either a hashmap or a struct (you can do something close in Python using __slots__) to represent an object. Those are standard data structure names, even though Javascript doesn't use the same terminology. Python doesn't call them hashmaps either.
selcuka 2y ago
Ok, not to belittle your work or anything, but I think you are either not using the words correctly, or trolling us here. If we used your "non-LLM" library to build an LLM, it wouldn't be "from scratch" would it? Therefore it is natural to assume that you meant that your code is supposed to be "an LLM written from scratch".
- _akhe 2y ago
  Not trolling, though I feel the same way about you with this post haha. To answer your question, this library doesn't provide something like a `chat` method and doesn't provide anything like Stable Diffusion's `img2img` API either - that's not what it is - it's a library for predicting the next token (letter, word, pixel, etc.) based on data you train it on. The most typical use cases for this model are: Auto-completion, auto-correct, spell check, search, etc. However, if you watched the incredible video I shared in the original post you'd know that this completion concept can be applied to other interfaces: Chat, image generation, audio analysis/generation, etc. and in fact it is applied to LLMs like GPT. This library doesn't get into any chat-specific methods like `ask` (question & answer), it just completes token sequences. If your goal is to create a ChatGPT clone, you would have to add methods like `chat`, `ask`, etc. and provide code for ongoing conversations - probably using an NLP library to parse queries into cursors like I'm trying in another project, or use "MULTI-HEADED ATTENTION BLOCKS" (lmfao) if you fancy that. Or if you wanted to create a Stable Diffusion clone, you'd have to provide those image-related methods needed. As far as the meaning of "from scratch" - I mean compared to using Ollama etc. to run local models or using OpenAI - just trying to enable you to build whatever model suits your data and use case.