4 ms·

Was it George himself, or a person working for a bounty that was set up by tinycorp? Also, a question for those knowledgeable about the PCI subsys: it looked l

by jagrsw 3y ago

Was it George himself, or a person working for a bounty that was set up by tinycorp? Also, a question for those knowledgeable about the PCI subsys: it looked like something NVIDIA didn't care about, rather than something they actively wanted to prevent, no?

mtlynch 3y ago
Commits are by geohot, so it looks like George himself.
- throw101010 3y ago
  I've seen him work on tinygrad on his Twitch livestream couple times, so more than likely him indeed.
deleted 3y ago
[deleted]
squarra 3y ago
He also documented his progress on the tinygrad discord
- throwaway8481 3y ago
  I feel like I should say something about discord not being a suitable replacement for a forum or bugtracker.
  - guywhocodes 3y ago
    We are talking about a literal monologue while poking at a driver for a few hours, this wasn't a huge project.
toast0 3y ago
PCI devices have always been able to read and write to the shared address space (subject to IOMMU); most frequently used for DMA to system RAM, but not limited to it. So, poking around to configure the device to put the whole VRAM in the address space is reasonable, subject to support for resizable BAR or just having a fixed size large enough BAR. And telling one card to read/write from an address that happens to be mapped to a different card's VRAM is also reasonable. I'd be interested to know if PCI-e switching capacity will be a bottleneck, or if it'll just be the point to point links and VRAM that bottlenecks. Saving a bounce through system RAM should help in either case though.
- namibj 3y ago
  Fixed large bar exists in some older accelerator cards like e.g. iirc the MI50/MI60 from AMD (the data center variant of the Radeon Vega VII, the first GPU with PCIe 4.0, also famous for dominating memory bandwidth until the RTX 40-series took that claim back. It had 16GB of HBM delivering 1TB/s memory bandwidth). It's notably not compatible with some legacy boot processes and iirc also just 32bit kernels in general, so consumer cards had to wait for resizable BAR to get the benefits of large BAR (that being notably direct flat memory mapping of VRAM so CPUs and PCIe peers can directly read and write into all of VRAM, without dancing through a command interface with doorbell registers. AFAIK it allows a GPU to talk directly to NICs and NVMe drives by running the driver in GPU code (I'm not sure how/if they let you properly interact with doorbell registers, but polled io_uring as an ABI would be no problem (I wouldn't be surprised if some NIC firmware already allows offloading this).

by jagrsw 3y ago

mtlynch 3y ago

Commits are by geohot, so it looks like George himself.

throw101010 3y ago
I've seen him work on tinygrad on his Twitch livestream couple times, so more than likely him indeed.

deleted 3y ago

[deleted]

squarra 3y ago

He also documented his progress on the tinygrad discord

throwaway8481 3y ago
I feel like I should say something about discord not being a suitable replacement for a forum or bugtracker.
- guywhocodes 3y ago
  We are talking about a literal monologue while poking at a driver for a few hours, this wasn't a huge project.

toast0 3y ago

PCI devices have always been able to read and write to the shared address space (subject to IOMMU); most frequently used for DMA to system RAM, but not limited to it. So, poking around to configure the device to put the whole VRAM in the address space is reasonable, subject to support for resizable BAR or just having a fixed size large enough BAR. And telling one card to read/write from an address that happens to be mapped to a different card's VRAM is also reasonable. I'd be interested to know if PCI-e switching capacity will be a bottleneck, or if it'll just be the point to point links and VRAM that bottlenecks. Saving a bounce through system RAM should help in either case though.

namibj 3y ago
Fixed large bar exists in some older accelerator cards like e.g. iirc the MI50/MI60 from AMD (the data center variant of the Radeon Vega VII, the first GPU with PCIe 4.0, also famous for dominating memory bandwidth until the RTX 40-series took that claim back. It had 16GB of HBM delivering 1TB/s memory bandwidth). It's notably not compatible with some legacy boot processes and iirc also just 32bit kernels in general, so consumer cards had to wait for resizable BAR to get the benefits of large BAR (that being notably direct flat memory mapping of VRAM so CPUs and PCIe peers can directly read and write into all of VRAM, without dancing through a command interface with doorbell registers. AFAIK it allows a GPU to talk directly to NICs and NVMe drives by running the driver in GPU code (I'm not sure how/if they let you properly interact with doorbell registers, but polled io_uring as an ABI would be no problem (I wouldn't be surprised if some NIC firmware already allows offloading this).