kir-gadjello

6 ms·

31.
by kir-gadjello 4y ago
BLOOM, while being a useful model for underrepresented languages, underperforms openai's GPT-3 models of similar parameter count on english benchmarks. Which is a pity. And OpenAI's known models (at least the ones for which the ar
32.
by kir-gadjello 4y ago
There is something to be said here about complexity expanding like a gas and parallels with certain amoebas who evolved genomes of enormous size: https://en.wikipedia.org/wiki/Polychaos_dubium
33.
by kir-gadjello 4y ago
Positional embeddings are tricky - it very much depends on the specific embedding method chosen. Some advanced methods allow conserved or even slightly improved performance with context length increased beyond what was used for the main pre
34.
by kir-gadjello 4y ago
RAM read bandwidth of M1/M2 is still not really competitive with the large GPUs like RTX3090, but it's getting close, compared with the usual CPU setups.
35.
by kir-gadjello 4y ago
>How is the actual VRAM requirement calculated (says 175B on mid-range GPU) Back of the envelope calculation for GPT-like models is simple: just divide the number of parameters by number of layers (ignore the embeddings and lm head size
36.
by kir-gadjello 4y ago
Technically, this library is for loading a checkpoint one layer at a time and would work for any model you can load from a state_dict. To support the inference for the model of your choice, someone should write code that uses zipslicer to i
37.
Show HN: Zipslicer, a library for loading LLM checkpoints on consumer hardware(github.com)
38 points by kir-gadjello 4y ago|8 comments
38.
by kir-gadjello 4y ago
Can "something simpler", for example, code correct function bodies from comments describing functions in natural language? I think people are too quick to dismiss the power of these models.
39.
by kir-gadjello 4y ago
A few months ago I made a small library to sanitize pytorch checkpoints, here it is: https://github.com/kir-gadjello/safer_unpickle The usage boils down to import safer_unpickle from safer_unpickle safer_unpickle.patch
40.
by kir-gadjello 4y ago
Yes, it's strongly influenced by clip-interrogator, but I revamped the algorithm quite a bit. I think it could be improved even further without resorting to fine-tuning the BLIP model.
41.
by kir-gadjello 4y ago
Shameless plug: I have a similar open-source tool which uses locally executed pre-trained models, here https://github.com/kir-gadjello/extract_prompt
42.
by kir-gadjello 4y ago
Shameless plug - I've made a standalone version of this tool, dozens of times faster for a single run on CPU due to pre-cached text embeddings: https://github.com/kir-gadjello/extract_prompt I plan to enhance it w