4 ms·

Thanks for sharing. Is understanding how best to utilize computer hardware part of a classic CS program or is most learned on the job? I studied Econ in colleg

Thanks for sharing. Is understanding how best to utilize computer hardware part of a classic CS program or is most learned on the job? I studied Econ in college and been a python guy for last 8 years but have no knowledge of how the machine actually interacts with my code. Is there a formal name for understanding that?

tveita 4y ago
A course in high performance computing should include some material on computer architecture and SIMD, though the focus is more on parallel programming and GPU. To get into the right mindset for SIMD optimization, play Zachtronic games. You have a big pile of odd-shaped tools that take some input and output a result a few cycles later, you have limits on how many instructions can run at once, but maybe you can do something like a load each cycle for free. Run your benchmark, count the cycles, make a small tweak and test again.
billythemaniam 4y ago
CS program, but depends on the classes you take. For me, the main class was called "computer architecture". Classes about compilers, databases, servers, embedded programming, server clustering can all help to understand the machine. Of course, it can be learned on your own too. But generally this path is harder because it's hard to know what to study. The CS program will expose you to breadth of concepts.
mydriasis 4y ago
You should work through Nand2Tetris :) it will give you an understanding of how computers work at a very, very low level.
rektide 4y ago
It'd be fun to ask Daniel Lemire's undergrads if they learned about vector/simd instructions. It's hard for me to answer, because I was learning everything I could on the internet before undergrad, following each tiny micro-architectural tweak in minute detail. It makes it hard to remember what things college really provided & what was self-taught.
Palomides 4y ago
at my school (ordinary well-known state university), barely any CS students would graduate knowing how to approach this
qbasic_forever 4y ago
Yes, a BS in computer engineering is typically what you want for a 50/50 mix of software and hardware. Basically if you get that degree you understand and can build a machine from digital logic/basic circuits all the way up to it boots to an operating system (that you wrote) and its login prompt. It's a deep dive into machine architecture and system programming. An electrical engineering degree doesn't focus as much on the software side, and a regular CS degree doesn't focus as much on the hardware--computer engineering is in the middle.
Symmetry 4y ago
At MIT, at least, everybody in EECS (all one department) has to take 6.004 where you build a simple computer from the transistors up which at least gives you a framework for thinking about this stuff. The simple processor doesn't have caches, of course, so that's a big whole in terms of things you have to understand for optimization, but it's the basics at least. EDIT: Link to course in OpenCourseware if you want to virtually take it: https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/ https://ocw.mit.edu/courses/6-004-computation-structures-spr...
saagarjha 4y ago
I don’t really know of many computer science programs that focus on this kind of thing. The other comments talk about little hobby computers, which is not a bad thing to learn about, but trying to apply principles from an 8-bit toy computer to a modern superscalar, pipelined, and heavily OoO machine is like comparing a toy rocket to the Space Shuttle. If you’re interested in learning more about microarchitectural optimization I find that getting a basic understanding of how a modern processor has evolved and then reading posts like these (looking things up when necessary) can get you mostly up to speed with the area. Most people don’t use these details in their everyday code so if you’re just looking to make your code fast learning how to measure algorithmic complexity and use a profiler is likely to be much more valuable.

tveita 4y ago

A course in high performance computing should include some material on computer architecture and SIMD, though the focus is more on parallel programming and GPU. To get into the right mindset for SIMD optimization, play Zachtronic games. You have a big pile of odd-shaped tools that take some input and output a result a few cycles later, you have limits on how many instructions can run at once, but maybe you can do something like a load each cycle for free. Run your benchmark, count the cycles, make a small tweak and test again.

billythemaniam 4y ago

CS program, but depends on the classes you take. For me, the main class was called "computer architecture". Classes about compilers, databases, servers, embedded programming, server clustering can all help to understand the machine. Of course, it can be learned on your own too. But generally this path is harder because it's hard to know what to study. The CS program will expose you to breadth of concepts.

mydriasis 4y ago

You should work through Nand2Tetris :) it will give you an understanding of how computers work at a very, very low level.

rektide 4y ago

It'd be fun to ask Daniel Lemire's undergrads if they learned about vector/simd instructions. It's hard for me to answer, because I was learning everything I could on the internet before undergrad, following each tiny micro-architectural tweak in minute detail. It makes it hard to remember what things college really provided & what was self-taught.

Palomides 4y ago

at my school (ordinary well-known state university), barely any CS students would graduate knowing how to approach this

qbasic_forever 4y ago

Yes, a BS in computer engineering is typically what you want for a 50/50 mix of software and hardware. Basically if you get that degree you understand and can build a machine from digital logic/basic circuits all the way up to it boots to an operating system (that you wrote) and its login prompt. It's a deep dive into machine architecture and system programming. An electrical engineering degree doesn't focus as much on the software side, and a regular CS degree doesn't focus as much on the hardware--computer engineering is in the middle.

Symmetry 4y ago

At MIT, at least, everybody in EECS (all one department) has to take 6.004 where you build a simple computer from the transistors up which at least gives you a framework for thinking about this stuff. The simple processor doesn't have caches, of course, so that's a big whole in terms of things you have to understand for optimization, but it's the basics at least. EDIT: Link to course in OpenCourseware if you want to virtually take it: https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/ https://ocw.mit.edu/courses/6-004-computation-structures-spr...

saagarjha 4y ago

I don’t really know of many computer science programs that focus on this kind of thing. The other comments talk about little hobby computers, which is not a bad thing to learn about, but trying to apply principles from an 8-bit toy computer to a modern superscalar, pipelined, and heavily OoO machine is like comparing a toy rocket to the Space Shuttle. If you’re interested in learning more about microarchitectural optimization I find that getting a basic understanding of how a modern processor has evolved and then reading posts like these (looking things up when necessary) can get you mostly up to speed with the area. Most people don’t use these details in their everyday code so if you’re just looking to make your code fast learning how to measure algorithmic complexity and use a profiler is likely to be much more valuable.