bradfox2

8 ms·

31.
by bradfox2 2y ago
I don't have the arxiv link bookmarked, but there was a paper written on pretraining with Lora. It involved merging adapters back every n steps with good results.
32.
by bradfox2 2y ago
We own and operate 40+ data center GPUs (v100s, a100s, and ax000s) in a private cluster and use vast to rent unused capacity. What would make you better than vast is extremely easy spot leasing and job prioritization. I want to be able to h
33.
by bradfox2 2y ago
The only experience I have is first hand, what my company is doing for our client base. We are doing continuous pretraining and the rest of the alignment stack training on about 10B private tokens + private customer data to produce private
34.
by bradfox2 2y ago
Qlora won't work well to add knowledge via private data. Parameter efficient methods are not useful for these cases at the 8b scale without a more complex training procedure that periodically merges back adapters. Maybe at the 70B scal
35.
by bradfox2 2y ago
Our 2013 carpet does it, very annoying. Happens under the accelerator pedal.
36.
by bradfox2 2y ago
Probably to avoid being arrested and/or retain the ability to pass a background check while finding alternate employment in the future.
37.
by bradfox2 2y ago
Then they should quit if their personal beliefs no longer align with the company. The expectation that they can do anything outside of working for the company on company objectives while on company time and not face consequences is insane
38.
by bradfox2 2y ago
What is a 'real' vacation? If it's laying on the beach for a week in Hawaii, or traveling through France, those are luxuries that should not be annual expectations. Vacation just means having time to periodically not work for
39.
by bradfox2 2y ago
So what? Plenty of model trainers are fine tuning to remove all the alignment and bias crap anyway. The concept of 'jailbreak' to a freely distributed open source model doesn't really apply.
40.
by bradfox2 2y ago
You are looking for Kairos. They are iterating and succeeding. Most of the hurdles are not engineering but licensing and regulatory.
41.
by bradfox2 3y ago
There's a long gh issues thread with technium struggling with Mistral 7 and loss spikes. Easy to find googling.
42.
by bradfox2 3y ago
My startup hosts our own training servers in a colo-ed space 10 min from our office. Took less than 40 hours to get moved in, with most of the time tinkering with fortigate network appliance settings. Cloudflare zero trust for free is a hu
43.
by bradfox2 3y ago
I think he means not needing to have a great search system to identify rag chunks. Just throw everything in.
44.
by bradfox2 3y ago
Tesla isn't janky compared to a Chinese EV with no reliable charging infrastructure.
45.
by bradfox2 3y ago
You want a specialized trade. HV Electrician at a utility or something like that. An off the shelf commercial or residential trade sounds like a nightmare, especially in an area with bad weather part of the year. That said, if you can deal
46.
by bradfox2 3y ago
We've been daily driving tesla exclusively since 2015. Too much uncertainty in a new area to rent one when on business travel with strict deadlines.
47.
by bradfox2 3y ago
This is just poor anti car attitude infecting actual judgement of the technology. Waymo doesn't require a human behind the wheel. They roam all over metro Phoenix without drivers and without issues. Tesla FSD works fine with appropria
48.
by bradfox2 3y ago
Whistlindiesel
49.
by bradfox2 3y ago
Are the 30x claims comparing full update training vs qlora/4bit weights?
50.
by bradfox2 3y ago
These are significant claims locked behind a paid 'pro' version. Red flags.
51.
by bradfox2 3y ago
Have you ever used either of the connectors? The advantage of NACS with water cooled cables is very obvious.
52.
by bradfox2 3y ago
This is insane when react query is available and so easy to use.
53.
by bradfox2 3y ago
This likely has a tangent to quantization research from Dettmers et al that corresponds to outlier weight values in models >6B params. Might allude to why the outliers emerge.
54.
by bradfox2 3y ago
4 miles per hour or so if you are referring to a nema 5-15.
55.
by bradfox2 3y ago
Someone else had the boxes, but that is what we were using!
56.
by bradfox2 3y ago
Have you ever connected with the folks behind XBConnect?
57.
by bradfox2 3y ago
I played competitive HaloCE through 3 at a professional level. XBConnect is what people playing competitive CE used prior to halo 2 and xbox live. Game was playable up to about 60ms, with <40 greatly preferred. Hosts had a large advanta
58.
by bradfox2 3y ago
You really want gpu rdma though. It's a bit of a pain to get setup but it's worth it.
59.
by bradfox2 3y ago
100gbe mellanox connect x cards are not actually that expensive though.
60.
by bradfox2 3y ago
My MSE dept was also colocated with mining engineering. Evolved from the metallurgy dept of a past era. We seemed to have a foot in manufacturing, computational materials, theory and chem lab.

More ›