bkkaggle

4 ms·

1. 1.▲
   by bkkaggle 6y ago

   heh. I've been using jax for a couple of months and its been a pretty nice replacement of both pt and tf. it feels like what a ml framework would look like if it were built around easy scaling and dev friendliness.
2. 2.▲
   by bkkaggle 6y ago

   Yep i started off with trying to get it to work with pytorch ( https://github.com/bkkaggle/lm-training-research-project/blo... ) then with pt-lightning but the whole 1 user VM per TPU board limitation in pytorch-xla
3. 3.▲
   by bkkaggle 6y ago

   Hi, I love that you enjoyed it! Yeah I totally get your point about the title—the TPU quota that I got was close to about the equivalent of $20k—but in my defense I don't have any other access to compute beyond anything that I get thro
4. 4.▲
   Replicating GPT-2 at Home(bkkaggle.github.io)

   254 points by bkkaggle 6y ago|46 comments
5. 5.▲
   by bkkaggle 6y ago

   This is pretty interesting especially since I just finished an assignment a couple of days ago for my proofs course that was all about proving the divisibility of large integers and the sums of their digits. Modular arithmetic, pretty fun s
6. 6.▲
   by bkkaggle 6y ago

   i used pytorch lightning back in may when i was working on pretraining gpt2 on TPUs ( https://bkkaggle.github.io/blog/nlp-research-part-2/ ). it was really impressive how stable it was especially given how a lot of