16 ms·
Reading OpenBSD source code daily
- topspin 9y agoAnyone have a good exemplar React application for this purpose? I'm building small experimental stuff and now I need to level up and see how real applications are being built.
- lbill 9y agoThis is a smart thing to do! I might do it eventually... not right now though, because of <insert whatever reason you can think of here>, and stuff, you know.
- ianai 9y agoI wonder where best to start for people who haven't taken an OS class before?
- irundebian 9y agoThe best start would be to read a book about operating systems such as Tanenbaums. Reading the source of an OS has a really really low signal to noise ratio in getting important knowledge about operating systems due to implementation details an OS-specific peculiarities.
- klez 9y agoWhat about MINIX? Being a teaching operating system, do you think it may contain less nose and thus be more useful as reading material? (I mean, without the book)
- nickpsecurity 9y agoLinus Torvalds started out studying MINIX. He was a Finnish programmer into working on kernels. He ended up completing one.
- masklinn 9y agoCurrently they mostly seem to be doing utilities, not kernel stuff, so there should be fairly little OS stuff required.
- autoreleasepool 9y agoFor a good base, I recommend the book "The Design and Implementation of the FreeBSD Operating System" by Marshall Kirk McKusick. It's really in depth and I was able to follow it without previously taking an OS course.
- hackermailman 9y agoCS:APP 3rd (64bit) edition http://csapp.cs.cmu.edu/ http://csapp.cs.cmu.edu/ Lectures for it are here: https://scs.hosted.panopto.com/Panopto/Pages/Sessions/List.aspx#folderID=%22b96d90ae-9871-4fae-91e2-b1627b43e25e%22&maxResults=50 https://scs.hosted.panopto.com/Panopto/Pages/Sessions/List.a... This teaches you C, x86-64 arch, stuff like two's complement integers and floating point, how memory and CPUs work, how the C compiler works (linking stage, preprocessor), and how to write/understand signals and processes, file I/O, network code ect. After that you can just start reading the OpenBSD code and will figure it out or get an Andrew S Tanenbaum book on Operating Systems. Note, if you buy the Pearson Global Edition of CS:APP (it's only 10% the reg price) there's a lot of errata you will have to check. I once got stuck reversing an assembly program into C that did an even/odd parity check because of a print error returning the final XOR'd value & 0 instead of & 1.
- doody12 9y agoThat depends on what you want to learn. If you want to improve your code reading skills and/or C programming skills, then you can probably go ahead and start reading, for example, the OpenBSD source code, even though you don't have any operating systems knowledge.
- irundebian 9y agoHow about using a static security source code analyzer and going through all findings? The very good ones, the commercial ones, are free to use for open source projects. That would be real benefit to the project I think.
- masklinn 9y agoIt's missing the point entirely, their primary goal is to become better developer and to improve their C knowledge, hence the effort being mostly focused on reading and understanding a fair amount of code, including its context. Fixes and improvements are positive side-effects of the original effort, not goals in and of themselves.
- irundebian 9y agoIf you want to improve your C knowledge there are probably more efficient ways to do it instead of randomly reading OpenBSD sources such as reading more advanced C books or reading source codes of projects which are known for their good code quality (sqlite maybe?). One of problem of static source code analyzers are false positive. Soon or later you will have to reading code and understand the context. I assume it's better way to improve your C knowledge because you REALLY must understand the code . And besides that the positive effect are more valuable.
- insertnickname 9y ago>... such as ... reading source codes of projects which are known for their good code quality (sqlite maybe?). OpenBSD is one such project.
- doody12 9y agoI disagree, reading quality code, which I would argue that the OpenBSD source is, is one of the best ways to improve. Also, being able to read and understand code is an important skill in itself.
- masklinn 9y ago> more efficient ways to do it instead of randomly reading OpenBSD sources such as reading source codes of projects which are known for their good code quality Yes they could be reading the source of projects known for their code quality… > OpenSSL is not an OpenBSD project and the code quality is markedly different :-)[…] and yes, OpenSSL is a bit of a code quality difference than the OpenBSD norm. [nb: these comments were not praising OpenSSL's code quality] > OpenBSD has proven great at configuration, code quality, and minimalism. > OpenBSD's incredible code quality quite obviously doesn't apply to the ports tree (and that's not their fault) > OpenBSD […] has a slower evolution pace and a more carefully planned development model which leads to better code quality overall. Its well deserved reputation of being an ultra secure operating system is the byproduct of a no compromise attitude valuing simplicity, correctness, and most importantly proactivity. OpenBSD also deletes code, a lot of code. > After scouring the lists and other resources I've yet to find an official reason for OpenBSD dropping LKM support, but would wager it's due to security or code quality/openness ideals. > OpenBSD, a project that has a frankly psychotic focus on code quality. […] some examples of great code quality. OpenBSD is undoubtedly one of the pin-up projects of the Open Source world, featuring code that is almost supernaturally clean, consistent and direct. > SELinux, etc. is not that picky about audits and code quality as OpenBSD is. > “I think our code quality is higher, just because that’s really a big focus for us,” De Raadt says. such as OpenBSD.
- VMG 9y agoGreat idea. What other code bases are there that lend themselves to this? Some kind of curated genius.com for source code would be interesting.
- wolfgke 9y agoFor operating systems I would recommend xv6: > https://pdos.csail.mit.edu/6.828/2016/xv6.html https://pdos.csail.mit.edu/6.828/2016/xv6.html
- masklinn 9y agoWith the caveat that xv6 is more artificial, it's a codebase for teaching not a living one.
- wolfgke 9y agoIn the past every two years a new version was released - so I would not call it "not living". By personal correspondence with one of the xv6 authors concerning a somewhat larger patch for xv6 (which he liked), he told me that they consider to release a new version in fall 2017, which would be one year after the previous release.
- nickpsecurity 9y agomasklinn's claim still rings true. There's a difference between codebases designed mostly for being easy to understand and codebases designed to be optimal in performance, consistency with an existing style (esp if old), and portability. Getting used to reading one might not transfer well to the other kind since the patterns are different. Probably best to use one of its intended goal (learning how OS's work) and look at examples of the other kind when trying to absorb patterns seen in real-world codebases.
- noshbrinken 9y agoJohn Resig used Genius to annotate the jQuery source code. Seems to be broken now. https://genius.it/johnresig.com/files/jquery-original.html https://genius.it/johnresig.com/files/jquery-original.html
- peatmoss 9y agoI love this idea in part because it's the very opposite of the way I tend to work, which is to drive very hard to get a surface understanding of a thing in order to make a very targeted change. I learn lots along the way with this approach, but don't often get the deep, wholistic understanding of existing systems that only comes with repeated exposure over a long time. Some kinds of understanding involve a no shortcuts grind. That sort of a grind is a big commitment though.
- azhenley 9y agoWhat you're hinting at is Minimalist Learning Theory. You have a production bias to learn only what you need to immediately get your job done, rather than investing time to learn up front which could potentially be more efficient in the long run.
- erikb 9y agoYour process is very good at achieving results fast. But the problem is you always stay in the same "level" of achieving results, which of course in the beginning is a very low level. What do I mean with "level"? Let's look at transportation in that regards. At first we just had walking/running. Then we learned how to use horses. Then we developed the wheel and could use horse wagons. Then we discovered the walking bike, etc. If you are on a low level you may be the fastest on that level, but you may be dimensions slower than people on higher levels. Think horse riding vs car. But that is not the biggest problem if you only use the approach. On a higher level you'll also be able to solve problems that you didn't even know where solvable. For instance if everybody walks you won't even consider visiting other continents. But if you have airplanes you can get there in a few hours and it becomes something people do at least twice a year. In programming this "solving problems easily that you didn't even know that there were solvable" happens if you really learn software architecture from actual tools, apis, how standards work, etc. The biggest wow for me was when I started to put in the additional 20-50% overhead to becoming standard conform for a standardized API. In the end when I had a problem I didn't have to code anything, because the other tools were already working with the same API as my tool, and I could just connect them and be done. This way I solved 75% of a semester long software project in one weekend, and I wouldn't consider myself especially intelligent. I just put in the hours to become standard conform, because from learning open source tools I found out it's something that people really do and that it is possible to do that.
- sn41 9y agoI read the emacs lisp source code in site-lisp/ every day. The goal is to understand one file about a week or so. It has made my emacs and lisp knowledge better, and made me aware of several nice emacs features (align-regexp, for example).
- err4nt 9y agoThis is fantastic! I've recently decided to begin reading web browser source code, even though I understand very little of it at the moment. For now, what's been fun is to load up the same file in both Chromium and Firefox source, and compare the two and how both browsers work. Chromium source: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/core/ https://cs.chromium.org/chromium/src/third_party/WebKit/Sour... Firefox source: https://dxr.mozilla.org/mozilla-central/source/ https://dxr.mozilla.org/mozilla-central/source/
- FreeFull 9y agoCould also be fun to compare to the Servo source: https://github.com/servo/servo https://github.com/servo/servo
- rhelmer 9y agoServo source is also on DXR, which is a lot nicer than the github source viewer since it understands Rust (and other) languages (using an LLVM compiler plugin): https://dxr.mozilla.org/servo/source/ https://dxr.mozilla.org/servo/source/
- sqs 9y agoCompletely agree. But the tools don't make this very easy. Back in college I was working on patches to OpenSSL, Chrome, Firefox, Apache, etc., to add support for TLS-SRP, and it was a huge pain to jump into these massive codebases and try to understand them. I was using Emacs and had all of the various language support modes configured, but go-to-definition and cross-references barely worked. Searching was slow, and if I wanted to discuss a piece of code with my CS lab partners, I couldn't just share a link. A friend felt the same pain but then went to work at Google for a bit. At Google, they have some pretty amazing code reading/searching tools (see https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43835.pdf https://static.googleusercontent.com/media/research.google.c...), and these tools helped Google build a culture of thoroughly reading and reviewing code. The causality is bidirectional, but having good tools certainly played a role in Google's success. That friend and I ended up building a product, Sourcegraph, initially for ourselves to make code reading easier. We've now built a successful business out of it with the help of an amazing team. Here it is pulling in the OpenBSD sources: https://sourcegraph.com/github.com/openbsd/src/-/blob/lib/libutil/bcrypt_pbkdf.c?q=bcrypt_pbkdf#L98-98:13 https://sourcegraph.com/github.com/openbsd/src/-/blob/lib/li.... Sourcegraph has advanced features for several languages; see https://sourcegraph.com/github.com/mholt/caddy/-/blob/caddyhttp/httpserver/https.go#L22:23$references https://sourcegraph.com/github.com/mholt/caddy/-/blob/caddyh..., for example. If you love to read code (or want to), we hope you'll love our product. Email me if you have any feedback/requests.
- deleted 9y ago[deleted]
- eatbitseveryday 9y ago> Here it is pulling in the OpenBSD sources There is a note that shows up when viewing OpenBSD sources "C/C++ is not yet supported (beyond basic code browsing and text search)" In my experience, these two languages are the most difficult to find good tools for, to browse, jump, and manage large code bases. Yes, some exist, but I thought this was the point of "good tools matter"?
- e12e 9y agoI never got around to really testing it, but still remember when OpenGrok was announced, way back when - apparently it's still active. I've been wondering if it would make a good front-end/source-browser component for a trac[t]-like product (with something else for vcs and bug tracking etc): https://github.com/OpenGrok/OpenGrok https://github.com/OpenGrok/OpenGrok There's also a list of similar tools at Gnu.org: https://www.gnu.org/software/global/links.html https://www.gnu.org/software/global/links.html Thought it might be of interest for others looking at "source browsing" tools. [t] https://trac.edgewall.org/ https://trac.edgewall.org/
- carlmungz 9y agoI started doing this the other day for a JS framework I'm using. Cannot recommend this practice highly enough. You learn so much.
- jamie__k 9y agoThat sounds great idea. I thinking about reading the vuejs code. if any tips for reading, can you share it?
- carlmungz 9y agoThis Github repo has some very good tips on how to do it: https://github.com/aredridel/how-to-read-code/blob/master/how-to-read-code.md https://github.com/aredridel/how-to-read-code/blob/master/ho...
- z3t4 9y agoWhat I love about the web is that you can just right click and view source. See something cool ? Just click and see how they did it.
- woranl 9y agoSoon, you won't be able to if WebAssembly becomes popular.
- kfrzcode 9y agoI've been using Typing.io as a platform for reading source code (working my way through Gitlab now) and practicing typing with the right fingers. I have a few minor bad habits to correct, and I want to familiarize myself with the codebase, it's a good way to warm up for the day.
- aomix 9y agoI've fallen into doing something similar. I read the mailing lists regularly try to look over the source for something that gets a proposed patch. Because OpenBSD boils down their software to the essentials and tries to make their APIs impossible to misuse I find it pretty easy reading even though I'm not very experienced with C.
- brynet 9y agoHere's the latest daily chat transcript, from Jun 9th: The topic was OpenBSD nc(1) and libtls, but it wandered over to pledge(2) and other code fixes from new participants eager to contribute. https://junk.tintagel.pl/openbsd-daily-nc.txt https://junk.tintagel.pl/openbsd-daily-nc.txt
- andrestc 9y agoMay i suggest maintaining a repo on github with such material? Would be easier to keep up and might bring in some contributions
- brynet 9y agoIt might be worth reaching out to mulander, he already hosts his blog on github. https://github.com/mulander/blog https://github.com/mulander/blog
- vhhhggv 9y ago21:03 <@mulander> first of, hi to all 72 of you The meeting of the 72 virgins.