nodja

16 ms·

151.
by nodja 4y ago
15k is the price to buy a GPU, not to run it, you'd have to account for electricity costs which isn't so straight forward.
152.
by nodja 4y ago
For anyone curious, it took 2048 A100 GPUs to train LLaMa, each GPU costs roughly $15k, facebook probably gets some sort of discount. That's a $30Mil if you want to train at that scale. Also IIRC it took 23 days to train the biggest mo
153.
by nodja 4y ago
You can finetune with it. If you want a more generic framework you can use hivemind[1] which is what petals uses, but you'll have to create your own community for whatever model you're trying to train. https://github.co
154.
by nodja 4y ago
https://petals.ml/
155.
by nodja 4y ago
> Also, remote transcoding of videos seems like a huge win! Makes it so you won't need as beefy as a system to run PeerTube. You can technically do it if you're willing to hack around. Since it uses the ffmpeg binary you could
156.
by nodja 4y ago
Weird thing to say when runway and compvis made stable diffusion.
157.
by nodja 4y ago
They made T5 and released the weights, while T5 "only" goes up to 11B params, it's an efficient model for a wide variety of tasks and the base for a lot of other models/projects. Of note imagen[1] and open-assistant[2] u
158.
by nodja 4y ago
Yes, hivemind trained a gpt 6B model like this. General model training https://github.com/learning-at-home/hivemind Stable diffusion specific https://github.com/chavinlo/distributed-diffusion Infe
159.
by nodja 4y ago
Regardless of what the author of the repo did or did not do, I think that deleting his account and everything in it is a failure on the part of github and warrants a policy change. There's lots of good knowledge contained in the issues
160.
by nodja 4y ago
The pokemon bd/sp games suffer from a similar issue. https://www.youtube.com/watch?v=fB2C8q42X64
161.
by nodja 4y ago
Broadly speaking WSL1 is linux implemented in the windows kernel, think reverse wine. WSL2 is linux running on a thin VM with several bridge layers that can access windows resources (networking, files, GPU, etc.)
162.
by nodja 4y ago
https://www.youtube.com/watch?v=hHwjceFcF2Q
163.
by nodja 4y ago
Yes, there's a global setting that will mark newly uploaded videos as blocked automatically, they'll show up in a special moderation area where you can decide what to do with the videos. You can also mark users as trusted in which
164.
by nodja 4y ago
No, it's only true if you're selling steam keys for cheaper on other sites. Devs can generate steam keys for free and have other sites sell them, but they're not allowed to list them for cheaper than what's on steam. Ste
165.
by nodja 4y ago
http content can also be modified in flight by your ISP/VPN/Starbucks wifi hotspot spoofer, you could easily change the links on the page to ones where you would end up downloading malware/spyware versions of the products
166.
by nodja 4y ago
That is exactly how bittorrent works as well tho. Bittorrent hasn't needed any trackers for public torrents since 2005 and can work solely off magnet links using the DHT network. Just like how IPFS does it.
167.
by nodja 4y ago
https://www.youtube.com/watch?v=2rA4Ny-QQfg
168.
by nodja 4y ago
> A quick check of Wikipedia suggests Arabic, Portuguese and Vietnamese, all use number-based systems to name the days of the week, and they are indexed from Sunday = 1. But yes, the other is more common. Most of the Slavic languages, an
169.
by nodja 4y ago
Does this mean you could in theory make the adler32 and crc32 part of the md5 hash as well?
170.
by nodja 4y ago
Yup, that would be easily doable, the code would essentially be the same.
171.
by nodja 4y ago
This is not really what is being suggested, this repo just navigates the latent space between 2 prompts and generates an image at specific intervals. It create cool effects but will never be able to generate a coherent video of for example
172.
by nodja 4y ago
> but as anyone who has spent time doing prompt-based image generation actual control isn't there yet It's not, but there are many efforts to fix this and they solve most problems, although I admit they're not ideal. One w
173.
by nodja 4y ago
Depending on how you're measuring it, video generation will come in less than 12 months, or it's already here. - In the Video Diffusion paper[1] they generate 64x64 16 frame videos, model is not released, but an open source implem
174.
by nodja 4y ago
This is common knowledge, so common that someone that hasn't coded anything besides some basic linear regression model like me knows about it. It's like commenting on why you'd put parenthesis in some formula, it's just
175.
by nodja 4y ago
I see lots of people answering why it's faster, but not many saying why the engineers chose the slower version. As everyone said, this is more performant because x is being modified in place, the reason this was not done in place is be
176.
by nodja 4y ago
> Many times I have had to decide if my Python code would be more legible or get free performance. This is rarely an option that has presented itself to me. If there's a clear performance issue in my code then I probably picked the
177.
by nodja 4y ago
Written in plain readable C, open source, iconic game. Same with doom. This is also from an era where there wasn't that much abstraction and "software rendering" was the norm so the implementations tended to be much more stra
178.
by nodja 4y ago
> I think the main issue here is the computational cost, as - if I understand correctly - you basically have to do training for each concept you want to learn. Are pretrained embeddings available anywhere for common words? The basic SD m
179.
by nodja 4y ago
GPT is an autoregressive model, this means that it's a recurring prediction model that works one "word"[*] at a time until it predicts that the text should end, feeding it's own guesses as input for the next word. It&#x2
180.
by nodja 4y ago
> But can a 10-node-iphone-cluster really match an A100 for pure flops?! Only if the data is local, small, easily synchronizable, doesn't require fp32 precision and a dozen other caveats. Can a 10 iphone cluster match an A100 in cal

More ›