5 ms·

> They're pretty rad. You can even get a few (dozen) for free: https://blog.gpt4.org/jaxtpu https://blog.gpt4.org/jaxtpu > Incidentally, that blog runs on a TPU

> They're pretty rad. You can even get a few (dozen) for free: https://blog.gpt4.org/jaxtpu https://blog.gpt4.org/jaxtpu > Incidentally, that blog runs on a TPU. You can just SSH into them. As someone without machine learning background, I assumed a TPU is something like a GPU. Aren't they used for ML as well? So I'm surprised you can run linux userspace on it?

manuel_w 5y ago
Oh, I read [1] too late. It's a (GNU/?)Linux VM with the special hardware already made available. [1] https://news.ycombinator.com/item?id=29129554 https://news.ycombinator.com/item?id=29129554
- sillysaurusx 5y ago
  > So I'm surprised you can run linux userspace on it? For what it's worth, I was equally shocked! Felt like a miracle. That special hardware also turns out to be miraculously-easy to use: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/xla/python/tpu_driver/client/libtpu_client.c#L19-L20 https://github.com/tensorflow/tensorflow/blob/master/tensorf... // To compile: gcc -o libtpu_client libtpu_client.c -ldl // To run: sudo ./libtpu_client I had to do a double-take, because compared to the uniquely hellacious experience of installing CUDA drivers on Ubuntu, this seemed to be... a single header file, and a single .c file. Turns out, it is that easy: https://twitter.com/theshawwn/status/1400749405356052483 https://twitter.com/theshawwn/status/1400749405356052483 And not because there's some SDK preinstalled -- it's because if you unmask libtpu like a scooby-doo villain, you'll discover libtpu is LuaJIT in disguise. You can even do the equivalent of lua's loadstring() function: https://github.com/tensorflow/tensorflow/blob/dd60c07888b6e7816f9a09a7b807de83c3cf73b3/tensorflow/compiler/xla/python/tpu_driver/client/libtpu_client.c#L63-L75 https://github.com/tensorflow/tensorflow/blob/dd60c07888b6e7... It's just called TpuDriver_CompileProgramFromText instead of loadstring, and you have to write in a quirky assembly language. But you don't need to write in that quirky assembly language, because you can just `import jax.jit`, jit a function, then dump it as HLO text. So you can just copy-paste it into your C file and run it :)
  - rrss 5y ago
    > compared to the uniquely hellacious experience of installing CUDA drivers on Ubuntu, this seemed to be... a single header file, and a single .c file. > Turns out, it is that easy: https://twitter.com/theshawwn/status/1400749405356052483 https://twitter.com/theshawwn/status/1400749405356052483 > And not because there's some SDK preinstalled Maybe not an SDK, but AFAICT it is that easy because all the drivers and stuff are already installed. If you start with the drivers already installed, this is exactly how GPGPU works too. You can write an almost-identical program using OpenCL (or CUDA) instead of libtpu, the only difference is that instead of this HLO the program is in SPIR-V (or OpenCL C, PTX, CUDA C). And you can start with all the drivers installed by using GCP or AMI images with all the GPU stuffs preinstalled. It is awesome that this is accessible for free.

manuel_w 5y ago

Oh, I read [1] too late. It's a (GNU/?)Linux VM with the special hardware already made available. [1] https://news.ycombinator.com/item?id=29129554 https://news.ycombinator.com/item?id=29129554

sillysaurusx 5y ago
> So I'm surprised you can run linux userspace on it? For what it's worth, I was equally shocked! Felt like a miracle. That special hardware also turns out to be miraculously-easy to use: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/xla/python/tpu_driver/client/libtpu_client.c#L19-L20 https://github.com/tensorflow/tensorflow/blob/master/tensorf... // To compile: gcc -o libtpu_client libtpu_client.c -ldl // To run: sudo ./libtpu_client I had to do a double-take, because compared to the uniquely hellacious experience of installing CUDA drivers on Ubuntu, this seemed to be... a single header file, and a single .c file. Turns out, it is that easy: https://twitter.com/theshawwn/status/1400749405356052483 https://twitter.com/theshawwn/status/1400749405356052483 And not because there's some SDK preinstalled -- it's because if you unmask libtpu like a scooby-doo villain, you'll discover libtpu is LuaJIT in disguise. You can even do the equivalent of lua's loadstring() function: https://github.com/tensorflow/tensorflow/blob/dd60c07888b6e7816f9a09a7b807de83c3cf73b3/tensorflow/compiler/xla/python/tpu_driver/client/libtpu_client.c#L63-L75 https://github.com/tensorflow/tensorflow/blob/dd60c07888b6e7... It's just called TpuDriver_CompileProgramFromText instead of loadstring, and you have to write in a quirky assembly language. But you don't need to write in that quirky assembly language, because you can just `import jax.jit`, jit a function, then dump it as HLO text. So you can just copy-paste it into your C file and run it :)
- rrss 5y ago
  > compared to the uniquely hellacious experience of installing CUDA drivers on Ubuntu, this seemed to be... a single header file, and a single .c file. > Turns out, it is that easy: https://twitter.com/theshawwn/status/1400749405356052483 https://twitter.com/theshawwn/status/1400749405356052483 > And not because there's some SDK preinstalled Maybe not an SDK, but AFAICT it is that easy because all the drivers and stuff are already installed. If you start with the drivers already installed, this is exactly how GPGPU works too. You can write an almost-identical program using OpenCL (or CUDA) instead of libtpu, the only difference is that instead of this HLO the program is in SPIR-V (or OpenCL C, PTX, CUDA C). And you can start with all the drivers installed by using GCP or AMI images with all the GPU stuffs preinstalled. It is awesome that this is accessible for free.