kkielhofner

8 ms·

31.
by kkielhofner 2y ago
One of the things driving this crazy spend on AI infrastructure by big tech is excess capital. Big tech is getting increasing scrutiny by regulators, etc for monopolistic/anticompetitive practices/etc[0]. From S&P: "Regul
32.
by kkielhofner 2y ago
CPU, yes, but more importantly memory bandwidth. An RTX 3090 (as one example) has nearly 1TB/s of memory bandwidth. You'd need at least 12 channels of the fastest proof-of-concept DDR5 on the planet to equal that. If you have a di
33.
by kkielhofner 2y ago
Make sure to check out the existing Triton client libraries: https://github.com/triton-inference-server/client
34.
by kkielhofner 2y ago
> I don't think NCCL + Deepspeed/FSDP are really an alternative to Scuda I put "remote" in quotes because they're not direct equivalents but from a practical standpoint it's the alternate current approach. &
35.
by kkielhofner 2y ago
This is very interesting but many of the motivations listed are far better served with alternate approaches. For "remote" model training there is NCCL + Deepspeed/FSDP/etc. For remote inferencing there are solutions like
36.
by kkielhofner 2y ago
Hah, it actually gets worse. What I was describing was the Triton ONNX backend with the OpenVINO execution accelerator[0] (not the OpenVINO backend itself). Clear as mud, right? Your issue here is model performance with the additional chall
37.
by kkielhofner 2y ago
The model you linked is not an LLM either by architecture or size. A few thoughts: 1) TensorRT anything isn’t an option because it requires Nvidia GPUs. 2) The serving frameworks you linked likely don’t support the architecture of this mode
38.
by kkielhofner 2y ago
Do you feel like the overall volume level changes based on the general genre of film? As I understand it: Action movies, for example, have to factor in audio for everything from whispers to big explosions. In order to do that properly (avoi
39.
by kkielhofner 2y ago
> burning GPUs and pushing CO2 in the atmosphere My startup develops AI for the nuclear power industry to drive process, documentation, and regulatory efficiency. We like to say "AI needs nuclear and nuclear needs AI". Big tech
40.
by kkielhofner 2y ago
Synthetic data (data from some kind of generative AI) has been used in some form or another for quite some time[0]. The license for LLaMA 3.1 has been updated to specifically allow its use for generation of synthetic training data. Famously
41.
by kkielhofner 2y ago
Additionally, they've been on both sides. If they are looking to invest in a company when they do technical due diligence and bring in a source code auditing company like Synopsis Black Duck any AGPL you're using is so problematic
42.
by kkielhofner 2y ago
> just like bitcoin The problem with this comparison is Bitcoin has basically just been SHA256 for 15 years and likely will continue to be for some time. Transformers have been mostly dominant for at least several years but there are sti
43.
by kkielhofner 2y ago
Growing up my dad was a very well known PhD in his field (Occupational Therapy). There were quite a few people who insisted on using Doctor when referring to others, calling themselves Doctor, etc. He never did but I experienced it quite a
44.
by kkielhofner 2y ago
Hah, that’s amazing! Now I get to tell him this story next time I see him :).
45.
by kkielhofner 2y ago
I have a project on Frontier - happy to answer any questions! Funny story about Bronson Messer (quoted in the article): On my first trip to Oak Ridge we went on a tour of “The Machine”. Afterwards we were hanging out on the observation deck
46.
by kkielhofner 2y ago
> There is nothing about going to MIT that makes me think That makes you think (no shade)... Question is: does it bother you enough to not proceed in engaging? Likely not. The upside is other audiences that love that kind of stuff - inve
47.
by kkielhofner 2y ago
> I want to know what the current power requirements are, as well as the cost for the machine. "If you care and have to ask it's not for you". In all seriousness I've worked with and am familiar with Cerebras, Groq, e
48.
by kkielhofner 2y ago
Hard disagree. A solid chunk of the value in attending prestigious universities is being able to to drop names and logos in this exact case. Like it or not touting that your team has pedigreed educational backgrounds is something you put up
49.
by kkielhofner 2y ago
Omi has a pretty interesting architecture that has been completely open-sourced: - The device itself is a Nordic Semiconductor BLE chip with GPIO, mic, and BLE. - Audio is streamed (as OPUS) via BLE to the associated mobile device where som
50.
by kkielhofner 2y ago
Human medical errors have been one of the leading causes of death[0] since we started tracking it (at least decades). The healthcare system has always killed plenty of people because humans are notoriously unreliable, fallible, etc. It is s
51.
by kkielhofner 2y ago
Absolutely. For some reason inference seems to be overlooked. A lot of “ink” has been spilled over GPUs for training tasks but at the end of the day if you can’t do inference you can’t serve users and you can’t make money.
52.
by kkielhofner 2y ago
I remember seeing my first of these in a small beachside hotel (pousada) in Brasil. In this case the wiring was over-stripped and copper was exposed at the shower head. I used it but I was very, very careful not to splash…
53.
by kkielhofner 2y ago
> You could be out of a job with as little control of the whole situation as those rustbeltians you have such disregard for. I'll admit I was a bit angry and offended when I first read this. Did you miss the part about how horribly
54.
by kkielhofner 2y ago
> Or they have developed a good filter for BS marketing. Maybe. Or (maybe) yelling at the TV is a lot easier (and more visceral) than thinking "Hmm, this whole AI thing just might work out. Maybe I should treat the threat seriously
55.
by kkielhofner 2y ago
> It's good that people scrutinize and resist corporate greed. I agree. Problem is it didn't do anything when they were saying the exact same things that led to the closure or significant job loss of the manufacturing that was
56.
by kkielhofner 2y ago
I grew up and live in small town Wisconsin. Sounds very believable. I say this with the utmost respect and concern for my friends, family, and neighbors: This attitude is part of the reason why the rust belt exists - filled with towns and c
57.
by kkielhofner 2y ago
Naming anything is hard and I don’t have better suggestions but when you’re doing something that’s already poking at something a big corp holds dearly hitting on trademark while you’re at it makes it really easy for them.
58.
by kkielhofner 2y ago
“Weirdo” is an interesting take. Richard Heart has been charged by the SEC for the usual crypto scam stuff and spent many months dodging subpoenas. Charges and subpoenas need their day in court of course but it reads like the quintessential
59.
by kkielhofner 2y ago
ChatGPT hit 100 million users in two months (previous record was TikTok in nine). It’s currently estimated to be at least double that… In less than two years. Coinbase has been around since 2012 and has roughly 100 million users. Yes ChatGP
60.
by kkielhofner 2y ago
“AI” is much more than LLMs… Not only have all three of these companies released LLM model weights, Google and Microsoft especially have released nearly countless model architectures, weights, evaluation/training/inference/et

More ›