lhl

37 ms·

181.
by lhl 3y ago
Most 4K models can use context window extension to get to 8K reasonably, but you're starting to see 16K, 32K, 128K (see YaRN for example) tunes become more common, or even a 200K version of Yi-34B.
182.
by lhl 3y ago
Haystack testing on GPT-4's 128K context suggests otherwise: https://twitter.com/SteveMoraco/status/1727370446788530236
183.
by lhl 3y ago
A couple big/strong open models that have just been released the past few days: * Qwen 72B (and 1.8B) - 32K context, trained on 3T tokens, <100M MAU commercial license, strong benchmark performance: https://twitter.com&#x
184.
by lhl 3y ago
I set up paperless-ngx w/ a scanner attached to my nas and a bit of scripting to get the scan button working a while back, but then forgot about it. For me, as someone who wants my docs on my own server, but well, doesn't care eno
185.
by lhl 3y ago
I switched from conda to mamba a while ago and never looked back (it's probably saved dozens of hours from waiting for conda's slow as molasses package resolution). I'm looking at the latest docs and it doesn't look like
186.
by lhl 3y ago
I don't think "base" works like that (while it can be a fallback for some dependencies, afaik, Python packages are isolated/not in path). But even if you could, don't do it. Different packages usually have differen
187.
by lhl 3y ago
So what I'm hearing is... no one should use "style" in its name anymore to describe style transfers because it's too closely associated with a set of models in a sub-field that uses a different concept to apply style tha
188.
by lhl 3y ago
It's not called a GAN TTS right? StyleGAN is called what it is because of a "style-based" approach and StyleTTS/2 seems to be doing the same (applying style transfer) through different method (and disentangling style fro
189.
by lhl 3y ago
I tested StyleTTS2 last month, my step-by-step notes that might be useful for people doing local setup (not too hard): https://llm-tracker.info/books/howto-guides/page/styletts-2 Also I did a little speed&#x2
190.
by lhl 3y ago
For specifically understanding transformers, this (w/ maybe GPT-4 by your side to unpack jargon/math) might be able to get you from lay-person to understanding enough to be dangerous pretty quickly: https://sebastianras
191.
by lhl 3y ago
https://www.deeplearningbook.org/ , PDF version: https://github.com/janishar/mit-deep-learning-book-pdf
192.
by lhl 3y ago
For those interested in this course, be sure to check out his Little Book of Deep Learning as well! https://fleuret.org/francois/lbdl.html
193.
by lhl 3y ago
I'd strongly second the suggestion to use the paid (GPT-4 based) ChatGPT Code Interpreter. I've found it to be incredibly strong/useful for interactive Python coding and it does a very good job explaining itself. You can use
194.
by lhl 3y ago
The Mechrevo Code 01 was a Tong Fang ODM chassis. XMG/Tuxedo (sister companies) have varying laptop models from both Tong Fang (Uniwill now?) and Clevo. Historically, a lot of devs using Linux laptops have preferred iGPU models both to
195.
by lhl 3y ago
It might come down to availability, although in general I prefer the higher wattage chips since they will usually have higher specced cooling and if you want to lower power consumption, you can use a tool like ryzenadj to lower power limits
196.
by lhl 3y ago
Pretty decent specs if you're OK w/ 32GB of (fast) soldered memory and don't need an ANSI keyboard (there is an EN-US ISO option). It's interesting to compare to the their Intel Infinity Book Pro Gen8 offering - that has
197.
by lhl 3y ago
While mining may not be what Nvidia made them for, it's what most of them were sold for, so I think the warning on their likely condition still applies. Beyond that, I'm pretty sure that the Techpowerup DB is wrong on the PCIe ban
198.
by lhl 3y ago
Just in case anyone gets tempted, the P106-100's were mining cards and locked to PCIe 1.1, so the bus bandwidth is terrible. Add to that the limited amount of memory (6GB), low memory bandwidth, Pascal (1/64 speed FP16), and that
199.
by lhl 3y ago
Just an FYI,the Yi license is a custom license an is not what I would describe an "open license": https://huggingface.co/01-ai/Yi-34B/blob/main/LICENSE It's non-commercial only: "If y
200.
by lhl 3y ago
For SRT, here are some front-ends: https://www.reddit.com/r/OpenAI/comments/163hzhe/recommended... Also I saw this thing called WhisperScript that looks pretty slick: https://github.com/o
201.
by lhl 3y ago
2 channels of DDR5-5600 = 89.6GB/s theoretical max bandwidth and a Q4_0 quant takes about 6.7GB of VRAM used, which is 13.4X, so that maps out pretty well for inferencing speed (I think the divided by parameters maybe only works for 4b
202.
by lhl 3y ago
Last month I did testing on the latest gen AMD APU (7940HS @ 65W - 8C Zen4 CPU, Radeon 780M RDNA3 GPU) on how llama2-7b (4-bit quants) performed w/ a variety of inferencers (llama.cpp, exllama v2, mlc w/ CPU, CLBlast, ROCm, and Vu
203.
by lhl 3y ago
Apple launched M2 MBPs only in January. I wonder if Qualcomm's Snapdragon X Elite (benchmarking now, but not in products until mid-2024) drove some of the timing on this launch: https://www.windowscentral.com/hardware&#
204.
by lhl 3y ago
The original patents expired years ago (2016/2017?) and even before them EInk had some legal setbacks (Trekstor/OED). I remember buying a Plastic Logic device that used a different tech w/ similar output (OTFT substrate), and
205.
by lhl 3y ago
The ACeP signage displays usually have a 30s refresh rate. There are Gallery 3 ebook devices which promise ~1s color refresh, but so far only one has come out (and it wasn't very good). That being said, if you don't need a lot of
206.
by lhl 3y ago
According to Notebookcheck, the latest speakers that Framework uses are actually pretty decent, about 85th percentile (MBP16 is 100th percentiled): https://www.notebookcheck.net/Framework-Laptop-13-5-Ryzen-7-...
207.
by lhl 3y ago
There's a small impact with larger capacities, but mostly it's memory speed and channels that have the biggest impact. Here's some testing posted in the Framework community forums from a couple years ago: https://c
208.
by lhl 3y ago
Yeah, Optimus was a huge PITA. I remember fighting with workarounds like bumblebee and prime for years. Also Nvidia dragged their feet on Wayland support for a few years too (and simultaneously was seemingly intent on sabotaging Nouveau).
209.
by lhl 3y ago
llama.cpp, ExLlama, and MLC LLM have all had ROCm inferencing for months (here are a bunch of setup instructions I've written up, for Linux and Windows: https://llm-tracker.info/books/howto-guides/page/am
210.
by lhl 3y ago
I had a similar experience recently looking some stuff for Starfield. Content farms are obviously switching to LLM-generated (mostly hallucinated) articles and Google seems to be ranking them pretty highly atm. Kagi's ability to manual

More ›