filterfiber

8 ms·

31.
by filterfiber 3y ago
I really hope AMD gains more ground on the software side. I know they are "technically supported" by onyx/pytorch etc. But almost every project I've seen there's a bunch of asterisks, a bunch of people needing diffe
32.
by filterfiber 3y ago
> Not enforceable. I'm not familiar with how lawsuits work in Europe/European countrys, but in the US anyone can sue anyone for any reason at any time (with very specific, few restrictions). If it's anything like the US, a
33.
by filterfiber 3y ago
Yeah I admittedly haven't used it much, our stuff was setup before they had async support at all, and our website doesn't benefit from it really. Async database queries would in theory be more efficient if you have many long runni
34.
by filterfiber 3y ago
The benefit of the LLMs is that the checkpoint already "understands" a lot by default. Finetuning is relatively cheap and makes many tasks such as this one perform decently well simply by shoving some data into it. The base checkp
35.
by filterfiber 3y ago
> It makes me hesitant to leave Typescript and even Rust (for web tooling). To be clear this is not what a "typical" python application/script looks like at all. While there's definitely an argument to be made about p
36.
by filterfiber 3y ago
I know it's not "shiny and new" but Django is still serving us very well were I work. The only two things it lacks great support of is typing and async. However the async is actively being worked on and does work (with some q
37.
by filterfiber 3y ago
I have to agree, RAG is effectively just compensating for data not fitting in the context window, and it has (potentially severe) limitations. Summarizing a book I think is a great example, RAG prevents LLMs from correlating all of the data
38.
by filterfiber 3y ago
Why are people surprised by them not using blue origin? Blue origin isn't scheduled to fly for some time yet. Amazon already announced agreements with 2 other companys as well. Amazon is a separate company with their own timelines. Wh
39.
by filterfiber 3y ago
The most likely "real solution" is going to be using various tricks and finetuning on higher context lengths to just extend the context window. However I will say knowledge graphs are becoming more popular. I made a comment the ot
40.
by filterfiber 3y ago
> I feel like the open source LLMs that are out there are never advertised/compared by their vocab list size. Increasing the vocab size increases training costs with little improvement in evaluation performance (how "smart&quot
41.
by filterfiber 3y ago
> Is the "parameters" the vocab list? The vocab list is the mapping of tokens to their utf encodings. for example, the token 680 is "ide", token 1049 is "land". > So from the 7B parameters, it picks whic
42.
by filterfiber 3y ago
> Parameter != token? They are two different things. A parameter is not what programmers call "parameters/arguments", it's the total number of weights from all of the layers in the network. A token is a number that re
43.
by filterfiber 3y ago
Yes! The obvious answer is to just increase your positions and train for that. This requires a ton of memory however (context length is squared) so most are currently training at 4k/8k and then finetuning higher similar to many of the
44.
by filterfiber 3y ago
> I feel like you mixed up context size with # of parameters? Respectfully I do not think I did :) The 7B is the parameter count. 32k is the vocab size, or list of tokens. I don't have access to the llama repo at the moment so I&#x2
45.
by filterfiber 3y ago
Yeah that's a fairly well studied one. Most of these techniques are rather "lossy" compared to extending the context window. The most likely "real solution" is going to be using various tricks and finetuning on high
46.
by filterfiber 3y ago
> are LLMs just a large way of spitting out which token it thinks is best based on scoring? aka it's guessing at best? They are 100% exactly this. More specifically they spit out a list of "logits" aka the probability of e
47.
by filterfiber 3y ago
> Of course, this entails the risk of occasional breakage. But for 99% of modules, this has no impact at runtime. Right so most projects end up with 100's (random one I have is 700+) modules. Which would mean multiple breakages. The
48.
by filterfiber 3y ago
A lot of public research is currently funded by them. It would really suck for the open access/research scene to lose them. For both LLM's and image generation Especially since meta doesn't seem interested in opening their im
49.
by filterfiber 3y ago
> I won't use Hue bulbs because I require that my automation setup fail gracefully back to plain old 'just works the way it used to' functionality. Maybe there's a setting that was changed? When I mess up my HA setup
50.
by filterfiber 3y ago
> When the bulbs are powered on and can't reach the hub they default to 100%. This is actually one of my favorite features. If I mess up my HA setup or anything like that, then worse case my "smart bulbs" are functionally
51.
by filterfiber 3y ago
As other's mentioned, they communicate via serial to the host, this is not a device I'd be greatly concerned about as a threat. Unless you took great care with sourcing your electronics then they're already packed with chips&
52.
by filterfiber 3y ago
Battery brands (or even skus in the brands) can matter a _lot_. Related, Panasonic's rechargable eneloop have worked very well for me, I'm kind of excited to see AAA be used in the IOT devices where it can fit. I have a lot less o
53.
by filterfiber 3y ago
Hopefully it becomes available soon. Oddly the C6 was announced before the C5 I think is only now making it's way onto devkits. > When I worked on WiFi for Google Fiber we tested it pretty extensively and didn't see any common
54.
by filterfiber 3y ago
Ah I've been out of the loop - thanks for correcting me!
55.
by filterfiber 3y ago
Are there any microcontrollers in the 1$ to 5$ range with 5GHz? And/or documented microcontrollers that are purchasable in the states? Won't 2.4GHz have better range anyway over 5GHZ?
56.
by filterfiber 3y ago
Most cheap wifi modules are all 2.4ghz, so printers, iot stuff, etc will almost certainly be 2.4ghz. Like 1$ cheap. > only 5ghz is performant Most of these barely need any bandwidth. A printer is possibly on the higher end of bandwidth,
57.
by filterfiber 3y ago
If anyone is interested, you can build your own adsb receiver for very cheap (under 50$), and you can forward the data to all the platforms - ADSB-exchange, airplanes.live, flightradar, flightaware, etc. You'll get the business tier pl
58.
by filterfiber 3y ago
> That's incredible, whats the catch!? They prioritize low-coverage areas (no idea how generous they are for covered areas). Nearly all of their position data is from these receivers, so you're providing them with more coverage
59.
by filterfiber 3y ago
Heads up - IIRC ADSB-exchange was sold out, an alternative might be adsb.fi Regardless you can send your ADS-B data to all of them - ADSB-exchange, adsb.fi, flightradar, flightaware, etc. That way you get all of the benefits and contribute
60.
by filterfiber 3y ago
I really dislike the word "hallucination" It's not a "hallucination" it's a "miss-prediction". And the more-classical methods of CV are still very capable of miss-predicting. No matter what method you

More ›