4 ms·

I am seeking a first principles approach to graphics. From poking a chip to manipulating bits and drawing pixels on the screen in the most raw form. Anyone know

I am seeking a first principles approach to graphics. From poking a chip to manipulating bits and drawing pixels on the screen in the most raw form. Anyone know of any such books/tutorials? I need to study how the first terminals worked. Assembly as the highest level abstraction. Anything equivalent to assembly anyway - a forth would be fine.

el_oni 6y ago
Maybe something like Ben Eater's video card series? https://youtu.be/l7rce6IQDWs https://youtu.be/l7rce6IQDWs If i remember correctly he assembles a gfx card on a bread board. So its fairly fundamental
gruelsandwich 6y ago
I don't know how well it applies, but the Coursera course(s) NAND2TETRIS could be what you're looking for. You essentially learn how a computer works from the lowest level (logic gates), to the OS, where the final project is to make a little game. On the way, you also make a simple asslembly language, and a programming language that compiles to the assembly language.
f00zz 6y ago
Maybe check out this book: https://www.amazon.com/Designing-Video-Game-Hardware-Verilog/dp/1728619440 https://www.amazon.com/Designing-Video-Game-Hardware-Verilog...
dahart 6y ago
This one might be interesting to you: https://youtu.be/l7rce6IQDWs https://youtu.be/l7rce6IQDWs “The world’s worst video card?” Ben builds a framebuffer display device on a bread board from scratch, you’ll understand what every transistor does at a high level in an hour (don’t miss part 2). That is abstractly what first principles graphics looked like some time ago - pixels were represented by a block of main memory, and the video system simply scanned that memory at the same frequency as the CRT monitor’s beam. I’d say it’s pretty important to remember that what counts as “first principles” graphics has changed over time, and there was a time before frame buffers (the first Pong game was displayed on an oscilloscope) and the way we use framebuffers today is completely different - there is no more direct access to pixels in the same way old terminals used to work. The article here is detailing how we use a higher level API to draw higher level primitives, and the hardware takes care of the pixels. I’d love to hear a little more about your goal... what’s driving you to study the old terminals?
Jasper_ 6y ago
One difficulty is that the "principles" aren't really pixels anymore, but things like triangles, rasterization, and SIMD programs that run. Understanding the historical context of super old "graphics adapters" might not equip you for the (quite ingenious, imo) insights that underlie their transition into the "GPU".
- darknoon 6y ago
  Not even SIMD on recent GPUs, it's SIMT.
  - Jasper_ 6y ago
    SIMT has always just a fancy disguise on top of SIMD hardware, but don't tell the GPU manufacturers I said that :)
  - dragontamer 6y ago
    No one has ever been able to tell me the difference between SIMT and 1980s style SIMD on the Connection Machine 5 (CM-5). Researchers like Blelloch have been writing in SIMD-parallel style "parallel Lisp" or "C-Star" for decades before CUDA. https://www.cs.cmu.edu/~guyb/papers/Ble90.pdf https://www.cs.cmu.edu/~guyb/papers/Ble90.pdf
    - dahart 6y ago
      Thread masking is one of the big differences - threads in a warp today can be enabled or disabled independently to enable limited amounts of divergent flow control, and conversely super cheap syncronization. I don't know what the CM5 did - was it limited to always-on sets of vector instructions? Current GPUs also support certain amounts of fully independent thread scheduling, e.g., per-thread instruction counters, etc. Maybe that is beginning to leak out of the SIMT model.
      - dragontamer 6y ago
        > I don't know what the CM5 did - was it limited to always-on sets of vector instructions? EDIT: Woops, I mean CM2. CM5 was MIMD and a slightly different architecture. I've edited this post to say CM-2 instead, but my previous post about CM5 will remain in error. Individual thread masking was available on CM-2, for all 4096 cores that were executing in SIMD-parallel. CM2 was a 4096 x 1-bit SIMD processor. Very limited compared to modern GPUs, but the execution model that CM2 experimented with eventually became the modern GPU architecture. Yes, you can have effective execution masks even with only 1-bit cores. EDIT: See the C-Star manuals, a "per-thread if statement" required a "where" construct instead of if. But yes, the "where" statement performed similarly to "CUDA if": http://people.csail.mit.edu/bradley/cm5docs/AReferenceDescriptionoftheCStarLanguage.pdf http://people.csail.mit.edu/bradley/cm5docs/AReferenceDescri... The MIMD thing that CM-5 did didn't seem to stick. CM-2's SIMD execution seems to be a bigger influence. But C-Star ran on both... and it was the high-level languages like C-Star or Parallel-Lisp that influenced GPU languages like CUDA. > The where statement is involved with setting the context, a process known as contextualization. The context is a parallel boolean mask (i.e., each element of it is true or false) that controls the execution of parallel operations position by position. A different context is associated with each shape object, and the context associated with the current shape is always applied to operators. A CUDA-block was roughly equivalent to a CStar-Shape. A CUDA-if is roughly equivalent to a CStar-where. > Current GPUs also support certain amounts of fully independent thread scheduling, e.g., per-thread instruction counters, etc. Maybe that is beginning to leak out of the SIMT model. NVidia was calling SIMT back in 2010, long before per-thread instruction counters were available in Pascal (GTX 10xx series of GPUs)
ggambetta 6y ago
I've written and open-sourced a book (soon to be published by No Starch Press) that might be close to what you're looking for. It doesn't poke chips directly, but drawing a pixel to the screen is the highest-level abstraction it uses (and builds on top of it). http://gabrielgambetta.com/cgfs http://gabrielgambetta.com/cgfs
kbr2000 6y ago
You could also check out James Bowman's "Gameduino" line of graphics boards [0:2] (which includes a Forth-programmable CPU, the J1 [3], implemented on FPGA) [0] https://excamera.com/sphinx/gameduino/ https://excamera.com/sphinx/gameduino/ [1] https://excamera.com/sphinx/gameduino2/ https://excamera.com/sphinx/gameduino2/ [2] https://excamera.com/sphinx/gameduino3/ https://excamera.com/sphinx/gameduino3/ [3] https://excamera.com/sphinx/fpga-j1.html https://excamera.com/sphinx/fpga-j1.html
vivekseth 6y ago
Maybe this would be a good start! https://github.com/ssloy/tinyrenderer https://github.com/ssloy/tinyrenderer It covers how graphics libraries like OpenGL go from triangles to screen coordinates, and how they "shade" pixels in those triangles to create an image.
Impossible 6y ago
On the hardware side (there are plenty of public resources about implementing graphics in software for the ground up), Jaymin Kessler's Jaystation2 and Jaystation3 dev blogs where he makes a GPU that runs on a FPGA might be useful http://maisonikkoku.com/ http://maisonikkoku.com/. This course also looks very useful from the hardware design standpoint https://courses.cs.washington.edu/courses/cse467/15wi/ https://courses.cs.washington.edu/courses/cse467/15wi/. GPU hardware design, unlike CPUs, seems to be much more esoteric with very little shared info due to the key players in the space and possibly all the patents. For graphics programming in general Graphics Codex looks really good (I haven't done the course, but I know people that have). https://graphicscodex.com/ https://graphicscodex.com/

el_oni 6y ago

Maybe something like Ben Eater's video card series? https://youtu.be/l7rce6IQDWs https://youtu.be/l7rce6IQDWs If i remember correctly he assembles a gfx card on a bread board. So its fairly fundamental

gruelsandwich 6y ago

I don't know how well it applies, but the Coursera course(s) NAND2TETRIS could be what you're looking for. You essentially learn how a computer works from the lowest level (logic gates), to the OS, where the final project is to make a little game. On the way, you also make a simple asslembly language, and a programming language that compiles to the assembly language.

f00zz 6y ago

Maybe check out this book: https://www.amazon.com/Designing-Video-Game-Hardware-Verilog/dp/1728619440 https://www.amazon.com/Designing-Video-Game-Hardware-Verilog...

dahart 6y ago

This one might be interesting to you: https://youtu.be/l7rce6IQDWs https://youtu.be/l7rce6IQDWs “The world’s worst video card?” Ben builds a framebuffer display device on a bread board from scratch, you’ll understand what every transistor does at a high level in an hour (don’t miss part 2). That is abstractly what first principles graphics looked like some time ago - pixels were represented by a block of main memory, and the video system simply scanned that memory at the same frequency as the CRT monitor’s beam. I’d say it’s pretty important to remember that what counts as “first principles” graphics has changed over time, and there was a time before frame buffers (the first Pong game was displayed on an oscilloscope) and the way we use framebuffers today is completely different - there is no more direct access to pixels in the same way old terminals used to work. The article here is detailing how we use a higher level API to draw higher level primitives, and the hardware takes care of the pixels. I’d love to hear a little more about your goal... what’s driving you to study the old terminals?

Jasper_ 6y ago

One difficulty is that the "principles" aren't really pixels anymore, but things like triangles, rasterization, and SIMD programs that run. Understanding the historical context of super old "graphics adapters" might not equip you for the (quite ingenious, imo) insights that underlie their transition into the "GPU".

darknoon 6y ago
Not even SIMD on recent GPUs, it's SIMT.
- Jasper_ 6y ago
  SIMT has always just a fancy disguise on top of SIMD hardware, but don't tell the GPU manufacturers I said that :)
- dragontamer 6y ago
  No one has ever been able to tell me the difference between SIMT and 1980s style SIMD on the Connection Machine 5 (CM-5). Researchers like Blelloch have been writing in SIMD-parallel style "parallel Lisp" or "C-Star" for decades before CUDA. https://www.cs.cmu.edu/~guyb/papers/Ble90.pdf https://www.cs.cmu.edu/~guyb/papers/Ble90.pdf
  - dahart 6y ago
    Thread masking is one of the big differences - threads in a warp today can be enabled or disabled independently to enable limited amounts of divergent flow control, and conversely super cheap syncronization. I don't know what the CM5 did - was it limited to always-on sets of vector instructions? Current GPUs also support certain amounts of fully independent thread scheduling, e.g., per-thread instruction counters, etc. Maybe that is beginning to leak out of the SIMT model.
    - dragontamer 6y ago
      > I don't know what the CM5 did - was it limited to always-on sets of vector instructions? EDIT: Woops, I mean CM2. CM5 was MIMD and a slightly different architecture. I've edited this post to say CM-2 instead, but my previous post about CM5 will remain in error. Individual thread masking was available on CM-2, for all 4096 cores that were executing in SIMD-parallel. CM2 was a 4096 x 1-bit SIMD processor. Very limited compared to modern GPUs, but the execution model that CM2 experimented with eventually became the modern GPU architecture. Yes, you can have effective execution masks even with only 1-bit cores. EDIT: See the C-Star manuals, a "per-thread if statement" required a "where" construct instead of if. But yes, the "where" statement performed similarly to "CUDA if": http://people.csail.mit.edu/bradley/cm5docs/AReferenceDescriptionoftheCStarLanguage.pdf http://people.csail.mit.edu/bradley/cm5docs/AReferenceDescri... The MIMD thing that CM-5 did didn't seem to stick. CM-2's SIMD execution seems to be a bigger influence. But C-Star ran on both... and it was the high-level languages like C-Star or Parallel-Lisp that influenced GPU languages like CUDA. > The where statement is involved with setting the context, a process known as contextualization. The context is a parallel boolean mask (i.e., each element of it is true or false) that controls the execution of parallel operations position by position. A different context is associated with each shape object, and the context associated with the current shape is always applied to operators. A CUDA-block was roughly equivalent to a CStar-Shape. A CUDA-if is roughly equivalent to a CStar-where. > Current GPUs also support certain amounts of fully independent thread scheduling, e.g., per-thread instruction counters, etc. Maybe that is beginning to leak out of the SIMT model. NVidia was calling SIMT back in 2010, long before per-thread instruction counters were available in Pascal (GTX 10xx series of GPUs)

ggambetta 6y ago

I've written and open-sourced a book (soon to be published by No Starch Press) that might be close to what you're looking for. It doesn't poke chips directly, but drawing a pixel to the screen is the highest-level abstraction it uses (and builds on top of it). http://gabrielgambetta.com/cgfs http://gabrielgambetta.com/cgfs

kbr2000 6y ago

You could also check out James Bowman's "Gameduino" line of graphics boards [0:2] (which includes a Forth-programmable CPU, the J1 [3], implemented on FPGA) [0] https://excamera.com/sphinx/gameduino/ https://excamera.com/sphinx/gameduino/ [1] https://excamera.com/sphinx/gameduino2/ https://excamera.com/sphinx/gameduino2/ [2] https://excamera.com/sphinx/gameduino3/ https://excamera.com/sphinx/gameduino3/ [3] https://excamera.com/sphinx/fpga-j1.html https://excamera.com/sphinx/fpga-j1.html

vivekseth 6y ago

Maybe this would be a good start! https://github.com/ssloy/tinyrenderer https://github.com/ssloy/tinyrenderer It covers how graphics libraries like OpenGL go from triangles to screen coordinates, and how they "shade" pixels in those triangles to create an image.

Impossible 6y ago

On the hardware side (there are plenty of public resources about implementing graphics in software for the ground up), Jaymin Kessler's Jaystation2 and Jaystation3 dev blogs where he makes a GPU that runs on a FPGA might be useful http://maisonikkoku.com/ http://maisonikkoku.com/. This course also looks very useful from the hardware design standpoint https://courses.cs.washington.edu/courses/cse467/15wi/ https://courses.cs.washington.edu/courses/cse467/15wi/. GPU hardware design, unlike CPUs, seems to be much more esoteric with very little shared info due to the key players in the space and possibly all the patents. For graphics programming in general Graphics Codex looks really good (I haven't done the course, but I know people that have). https://graphicscodex.com/ https://graphicscodex.com/