20 ms·
MIT Hacker Tools: a lecture series on programmer tools
- Jonhoo 8y agoHi all! We (@anishathalye, @jjgo, and @jonhoo) have long felt that while university CS classes are great at teaching specific topics, they often leave it to students to figure out a lot of the common knowledge about how to actually use your computer. And in particular, how to use it efficiently. There’s just no class in the undergrad curriculum that teaches you how to become familiar with the system you’re working with! Students are expected to know about, or figure out, the shell, editors, remote access and file management, version control, debugging and profiling utilities, and all sorts of other useful tools on their own. Often times, they won’t even know that many of these tools exist, and instead do things in roundabout ways or simply be left frustrated about their development environment. To help mitigate this, we decided to run this short lecture series at MIT during the January Independent Activities Period that we called “Hacker Tools” (in reference to “hacker culture”, not hacking computers). Our hope was that through this class, and the resulting lecture materials and videos, we might be able to bootstrap students’ knowledge about the tools that are available to them, which they can then put to use throughout their time at university, and beyond. We’ve shared both the lecture notes and the recordings of the lectures in the hopes that people outside of MIT may also find these resources useful in making better use of their tools. If that turns out to be true, we’re also thinking of re-doing the videos in screen-cast style with live chat and a proper microphone when we get the time. If that sounds interesting to you, and if you have ideas about other things you’d like to see us cover, please leave a comment below; we’d love to hear from you! We’re sure there are also plenty of cool tools that we didn’t get to cover in this series that you all know and love. Please share them below along with a short description so we can all learn something new! Anish, Jose, and Jon
- gumby 8y ago> ...[universities] often leave it to students to figure out a lot of the common knowledge about how to actually use your computer. Now I'm out in the real world (MIT class of '86!) I think this is more of an MIT thing -- many graduates of other schools get explicit training in things that the MIT faculty, fairly or not, assumes students will simply pick up. And this is despite the heavy emphasis on actually doing stuff, especially in the school of engineering. Actually I've had hit or miss results hiring freshly-graduated MIT grads (maybe less so VI-A) for this reason, though once they are up to speed they've mostly been great. Really good use of IAP by the way!
- akhilcacharya 8y ago> many graduates of other schools get explicit training in things that the MIT faculty, fairly or not, assumes students will simply pick up. And this is despite the heavy emphasis on actually doing stuff, uh...I graduated from a low tier state school that was notable in our state for focusing on practical matters....and we didn't get 'explicitly trained' on this either. Most people do just pick it up.
- gumby 8y agoWell, I appears from my experience a surprising number of MIT students don’t, making this mini-course popular. As an MIT grad myself I should be biased to think otherwise, and I certainly hope the credential stands for something good. But I haven’t found much difference in programmers 5 years out of school — it’s been attitude that has mattered more.
- hinzed1127 8y agoI got a CS minor from a top-5 program, and while a lot of the core classes definitely covered some of these topics in various degrees, it's really nice to see all of these aggregated in a concise way. While you definitely do pick stuff up over time, this is arranged in a way that I could see myself preferring to reference this rather than find that one stack overflow answer that clicked with me at the time. Now that I at least have a better understanding of these concepts 3.5 years into real-world web development, this seems like a good resource to cover my bases better now.
- bubblethink 8y ago>many graduates of other schools get explicit training From my undergrad programming class, which was a while ago, I remember that the first class had a brief intro to vim. But something like vim is an acquired taste. I never used or appreciated vim until much later when I saw others at work use it. During school, I think I used geany since that's relatively easy to use and install on ubuntu. So maybe there is some merit to spending more time on tools. However, my experience is from before everything was so easily accessible on the internet (i.e., pre github, stackoverflow etc.). So maybe these days it's not such a big deal.
- alpb 8y agohttps://www.youtube.com/playlist?list=PLyzOVJj3bHQuiujH1lpn8cA9dsyulbYRv https://www.youtube.com/playlist?list=PLyzOVJj3bHQuiujH1lpn8... Thanks a lot for this! It appears like the recording doesn't really capture the screen well for multiple videos. I wish it was more professionally done, it could've been useful for years to come.
- anishathalye 8y agoFor most of the videos, we do have screen recordings (e.g. see https://www.youtube.com/watch?v=VW2jn9Okjhw https://www.youtube.com/watch?v=VW2jn9Okjhw). Sadly, we're missing screen recordings for a couple videos (such as the shell video, where we had some issues with the screen recording software). We're thinking of re-doing some of those videos in a screen-cast style so that they'll be more useful for viewers.
- idsout 8y agoThank you so much for sharing your knowledge with the world. I'm excited to follow along with this and I love your Rust streams.
- nmstoker 8y agoWill need to take a deeper look later but from a quick skim I really like the approach taken: a well curated list of time tested tools across each area, going deeper where applicable. For me the key point above is about a lack of tool knowledge causing "roundabout approaches" and frustration. Knowledge of these tools can do so much to demystify and make the seemingly arcane start to slot into place.
- acidburnNSA 8y agoThat's funny, I recently had the same revelation (that it's hard to discover these things) with engineering students in mind. I wrote an e-book called Digital Superpowers, covering many of the same topics you did. I added some general interest things on art and publishing too. I think I like your approach better. Anyway, my list of topics is as follows, maybe you want to consider adding a few of them to your list? Password Managers • 2-factor authentication • Virtual machines • Package managers • Command line • Routers • QR codes • Port forwarding • Python • Pandas • Onion routing • Ad-blocking • PDF reordering • Flow charts • Regular Expressions • Time-lapses • Encryption • Column edit • Digital photo editing in RAW • Audio noise reduction • Vector graphics • Making websites • DJing • LaTeX publishing • Reference management • pandoc • git version control • Programming • Web apps • Self-hosting • Home automation • Monte Carlo • VPNs, • ImageMagick • Linear regression • ffmpeg • Sphinx • Robotics
- freshcatch_ 8y agothis is insanely ambitious, and in a weird order
- acidburnNSA 8y agoYou're right. And "covering" is a strong term for the book. It only introduces them, offers a few exercises, and moves on. It's still 200 pages. It's intended to help engineering students (mechanical, civil/structural, nuclear, etc.) discover these wonders and then choose which ones to go deep with based on their needs and interests.
- bigmit37 8y agoThanks for sharing and seems it would be useful for non-engineering students as myself as well. Going to check it out =)
- TeMPOraL 8y agohttps://www.amazon.com/Digital-Superpowers-readily-available-turbocharge-productivity-ebook/dp/B07N67BD9T https://www.amazon.com/Digital-Superpowers-readily-available... That's the one? EDIT: I guess it is, via https://digitalsuperpowers.com/ https://digitalsuperpowers.com/. The list of topics is great, gonna check the book out!
- agumonkey 8y agoDo you like that kind of talks ? https://www.youtube.com/watch?v=sCZJblyT_XM https://www.youtube.com/watch?v=sCZJblyT_XM Gary Bernhardt - The Unix Chainsaw
- marzoev 8y agowow this is so cool!!!
- deleted 8y ago[deleted]
- melling 8y agoMIT went with the Berkeley editor. A lost opportunity to introduce programmers to the beauty of using Emacs and Lisp.
- rsfinn 8y agoCan't tell whether this is sarcasm or not. Perhaps the expectation is that those things will be acquired naturally; certainly I was immersed in both within two weeks of arriving at MIT. (Admittedly that was in the latter half of the Carter administration; things may be different these days.)
- snazz 8y agoI’m sure a not-insignificant portion will move on to Evil mode. Vim -> Emacs is a better direction than Emacs -> Vim, because you’ll learn the more efficient editing style, then have access to the better environment, instead of getting used to the great Emacs environment minus the Vim keybindings, and feeling constrained by losing this when moving on to Vim.
- joseph8th 8y agoI think this is true. I learned the Emacs keybindings and now they're muscle memory. I can't transfer that to Vim, but I could have transferred the other way and be proficient in two great editors. Nobody wants to die on Emacs Keybindings Hill, even diehards like me and my pinky.
- komali2 8y agoBut almost all IDEs and editors have a VIM mode - learning VIM is a good way to teach people how to keep their fingers on home row, and the skill is (because of what I stated previously) easily transferred to whatever editor you like. From my perspective, I tried to learn Emacs, and decided it's a great editor, once you spend weeks to months to get it actually "working good," and learn shitloads of commands (let alone keybindings). Given my initial goal was "keep hands on home row," it was far, far faster/better for me to spend 3 days learning/memorizing VIM commands, installing a vim plugin into VScode, and just doing that. Would love to use Emacs, but it's a nightmare to set up. I still haven't managed to get it feature parity with vscode.
- splittingTimes 8y agoSo one lecture covers window managers for tiling. I used dwm for years and love it, but am stuck on Windows at work. Is there anything for Windows 10 that has a similar approach of having a stack with your unused applications and a big window for the app that is in focus? And then you can easily move window from stack to focus and vice versa? I looked at the Wikipedia article [1], but none of the alternatives seem to do that. What can you recommend? === [1] https://en.m.wikipedia.org/wiki/Tiling_window_manager https://en.m.wikipedia.org/wiki/Tiling_window_manager
- snazz 8y agoYou could have a full-screen Xming session connected to dwm running in WSL, if you liked that better. Of course, you would still have to use the standard Windows DE/WM when using Win32 applications, but if most of your work is text editing, web browsing, etc, you might be well served by the full-screen Xming. You could start with this guide, which solves a slightly different but related problem: https://virtualizationreview.com/articles/2017/02/08/graphical-programs-on-windows-subsystem-on-linux.aspx?m=2 https://virtualizationreview.com/articles/2017/02/08/graphic...
- simcop2387 8y agoIt's no where near complete like dwm for that, but it is highly customizable, I use AquaSnap when I have to use windows for work. It is paid but I haven't had any issues with upgrades or redownloading the software when I haven't had to use a windows machine in a while. https://www.nurgo-software.com/products/aquasnap https://www.nurgo-software.com/products/aquasnap
- abeyer 8y agoHaven't tried it but I think HashTWM[1] claims to have a similar main/stack model for windows. [1] https://github.com/ZaneA/HashTWM https://github.com/ZaneA/HashTWM
- TeMPOraL 8y ago~Year ago, I had plenty of success with bug.n - https://github.com/fuhsjr00/bug.n https://github.com/fuhsjr00/bug.n. It's a DWM clone written in... AutoHotkey, of all things.
- flafla2 8y agoHere’s the CMU equivalent course, which interestingly was also designed by students: https://www.cs.cmu.edu/~15131/f17/ https://www.cs.cmu.edu/~15131/f17/
- sk5t 8y agoCMU did (maybe still does?) have a mandatory, once-a-week course to orient new students to logging in to *nix workstations, setting file permissions and copying files around on AFS, etc. Whether this was just for CIT or university-wide I cannot recall.
- jjoonathan 8y agoDidn't they recently nuke the unix infrastructure and replace it with cloud services? Or have reports been exaggerated?
- Shoop 8y agoThe unix infrastructure is still in place and widely used by nearly all cs courses.
- uasm 8y ago> https://hacker-tools.github.io/lectures/ https://hacker-tools.github.io/lectures/ Literally none of these have anything to do with proper information security/hacking/penetration testing. Will you please consider to stop abusing the term "hacker"?
- Jonhoo 8y agoSo, the term "hacker" and "hacking" has traditionally meant someone who is exploring the limits of systems (whether computer-based on otherwise) in a playful manner, often finding "clever" ways to make the system behave in unintended ways. The term is arguably even broader than that — see the Wikipedia entry on Hacker Culture[1] for more. The use of "hacker" to mean the exploitation of computer systems is more recent, and was traditionally referred to as "cracking". The two terms do have a decent amount of overlap, but the latter is much more commonly associated with malicious intent, whereas hacking really just means that a desire to push systems to their limits and see what kind of cleverness you can pull off! [1]: https://en.wikipedia.org/wiki/Hacker_culture https://en.wikipedia.org/wiki/Hacker_culture
- tgarv 8y ago"Hacker" and "hacking" at MIT have a couple of meanings, neither of which specifically means trying to "hack" into a system. "Hacking" either means 1) Working hard on a problem set or other project or 2) Pulling off a prank on the MIT campus without permission of the administration, e.g. turning the Great Dome into R2D2's head. So in the context of some people from MIT putting out a list of resources, originally for others at MIT doing #1 above, "hacker" is the right word to use.
- certifiedloud 8y agoThey specifically say in the course overview that they are using the term "hacker" in this sense: https://en.wikipedia.org/wiki/Hacker_culture https://en.wikipedia.org/wiki/Hacker_culture Not "hacker" in this sense: https://en.wikipedia.org/wiki/Security_hacker https://en.wikipedia.org/wiki/Security_hacker
- deleted 8y ago[deleted]
- andrewl 8y agoThis looks good. I would also recommend the Software Carpentry program: https://software-carpentry.org/ https://software-carpentry.org/
- taude 8y agoThis is a great series. I'm going to share with my team, just to make sure they can fill in any items they're missing, like using bash functions for shortcuts or aliasing. BTW, Stanford did an online training course 5 years ago or something called CS 184 Startup Engineering. They had some excellent tutorials on the CLI and Emacs [1]. -I wish I could find the materials, if anyone else can find them.- Actually, I found them online: [1] http://docshare.tips/cli-git-emacs-dotfiles_57592eb2b6d87f377c8b46af.html http://docshare.tips/cli-git-emacs-dotfiles_57592eb2b6d87f37... [2] Link to their dotfiles for beginner config: https://github.com/startup-class https://github.com/startup-class
- bhchiang 8y agoThe equivalent UCLA course is CS35L: Software Construction Laboratory (https://web.cs.ucla.edu/classes/winter19/cs35L/ https://web.cs.ucla.edu/classes/winter19/cs35L/). It's taught by Paul Eggert (big open source/coreutils/emacs contributor + author of diff/sort).
- mistrial9 8y agoPlease note that the Software Carpentry [0] project exists as well. All welcome . [0] https://software-carpentry.org/ https://software-carpentry.org/
- everetm 8y agoWe had a student run course at Purdue for a while to teach new students fundamental tools outside of class. https://github.com/Purdue-CSUSB https://github.com/Purdue-CSUSB
- flukus 8y agoForget university, most of these this should be the content of K-12 computing/programming classes. It's simpler and has much more real world value than a lot of the crap the tech, like graphical iPad programming or how to use a specific version of word.
- jointhefuture 8y agoThe article on editors is basically "here's the names of 5 editors, now learn vim" I know this will piss off vim users but in my experience vim users are seriously out date when it comes to dev tools. It's like they never left the 70s. vim might be great, it might be available everywhere but when I see a vim user I then see them use something like gdb and have clearly never experienced a modern debugger because if they had they'd be infuriated at how their debugging tools haven't improved in 30 years. I'm sure others could go into all the features a modern editor or IDE provide. The vim users will say they don't need that crap. They sound like some grandpa saying "back in my we had to walk to 5 miles to school, no one had invented bicycles or cars or buses. Don't need those new fangled things, now Get off my Lawn!"
- flukus 8y agoMy experience of IDE users (myself included much of the time) is that they don't know how limited their debugging experience is, just breakpoints and how to step through code. Things like breaking when a variable is set, debugging from a core dump, creating a core dump under certain conditions, viewing the assembly, setting variables on the fly etc, are features of a good debugger (often the one they're using) that are obscured from them. If you want that shallow level of debugging you can do it from inside vim though: https://github.com/idanarye/vim-vebugger https://github.com/idanarye/vim-vebugger
- jointhefuture 8y agoExact opposite of my experience. IDEs let me easily add conditional break points, toggle between source, disassembly, source+disassembly, set variables just by clicking on them, view multiple areas of memory in different tabs/windows. Set how that memory is viewed (bytes, words, dwords, floats, as a type of class/struct, array of classes/structs), show different contexts at once, have multiple windows per thread/process, never seen someone use one that didn't know those features exist as they are all obvious to discover in an IDE. sounds like you've never used one but then that's what I'd guess from watching most vim users. All of those have been standard features of visual IDE debuggers for over 30 years. Mean while instead of seeing every thing update live I watch the vim users type various gdb print and dump commands at the command line and then watch that data scroll off their terminal instead of being updated live as they progress like an IDE debugger.
- jackallis 8y agois this some sort of MIT class to undergrad or general public?
- c256 8y agoAt MIT, the month of January is a reserved time separate from the normal terms called IAP - Independent Activities Period. During IAP, there are many seminars, panels, talks, and short courses (and some long courses taught in an intensive fashion) on a huge variety of topics. Many are taught by students, faculty, staff, and area alumni. While a few classes do give credit, fulfill requirements, and the like, oat exist simply to learn cool, neat, useful, interesting, wacky, etc stuff.
- jackallis 8y agoohh. Only if it was to public, hop on the red line and walk 10 mins.
- jackallis 8y agocan't see anything on that projector :-(
- Jonhoo 8y agoYeah, sadly the screencasting software malfunctioned on some of the sessions. We're thinking of re-doing some of them with a proper microphone and screencast though (see also https://news.ycombinator.com/item?id=19081669 https://news.ycombinator.com/item?id=19081669), and will certainly announce when/if that happens! For what it's worth, the lecture notes should capture most of the actual contents, and in the same order.
- stebann 8y agoReally useful! I was looking something introductory for teaching, this will do.
- rollinDyno 8y agoI strongly believe man is a product of his tools, my slow but steady increase of their mastery is a source of pride. I've spent nights just mindelessly browsing the web in the search for a tool I didn't know about (it can be a service, a browser extension, a text editor plugin) so just looking at the title of this post made me ready for an injection of dopamine. I have a branch on my todo list solely focused on my tools, because there are so many things to configure continuously. I schedule these tasks for the occassion I feel like I'll have a couple of minutes to improve and their priority are often labeled as "Compounding Rewards" as opposed to "Critical".
- JeffRosenberg 8y agoLove it! Do you have any favorite tools you'd recommend, and/or favorite places for finding new ones?
- ehavener 8y agoI'm interested if you have a list of your favorite tooling resources?
- abhishekjain123 8y agoThank you Sir For This Amazing Post.It Really Helped Me To Buy new Tolls For My Industries.