13 ms·

Getting better at Linux with mini-projects

1009 points by carltheperson 6y ago

code_scrapping 6y ago
Hi Carl, purely judging from your profile picture it seems that you're about half my age :) and I think it's excellent that you're doing a bunch of things - personal blog, linux, tutorial post... Just remember - most of the guys around here tend to do it for marketing reasons (i.e. we want to find a job and we're advertising our capabilities), and those projects might not be the most interesting things out there. Bottom line is - just because everybody else is doing it, might be because they have different motives than you, and I hope you'll do/build firstly what's interesting and fun, and secondly what's financially viable. Good luck, have fun!
- dj_mc_merlin 6y ago
  I got hired at (probably about his age?) just due to side projects like this. If nothing, it shows motivation. So it's got a marketing reason nonetheless.
  - CryptoGhost 6y ago
    Isn't this the programmer equivalent of one of those old stories "just go talk to the manager and give him a firm handshake." All the recent research in hiring seems to point to trends that white board / hackerrank / leetcode style filtering are up and someone with hiring authority actually looking at a portfolio or code repo at an all time low.
    - saagarjha 6y ago
      Getting things on the front page of Hacker News remains a good way to get people to contact you. Source: have had people contact me with job-related things for that reason.
- carltheperson 6y ago
  I'm 18 (: Thank you so much for the heads up. I'm only doing this because it's fun! I love using Linux and I feel like I learned a ton from this.
  - smlckz 6y ago
    being slightly younger than you, i can not but envy you: i have done zero projects or had fun like you have had. hmm.
    - log101 6y ago
      Envy = Desire, I'm sure you'll do similar side-projects soon!
    - saagarjha 6y ago
      Sounds like you've still got some time to do some projects for fun and be like him?
  - jvanderbot 6y ago
    At what point did the smiley face reverse? It seems anyone under 24-ish uses (: where as I know it as :)
    - saagarjha 6y ago
      I don't think I've ever seen anyone using (:, and I'm a bit younger than 24-ish :)
      - darkwater 6y ago
        I remember (: being used by left-handed in the late '90s, but maybe it was just the people I knew that used it for that reason.
    - thebruce87m 6y ago
      And when did the nose fall off too?
- mod 6y ago
  Just a couple days ago I was going through all kinds of things I made when I was 18. That was almost 20 years ago, and I had VB6 projects and a blog (we called them E/N sites, the word blog wasn't coined yet). The VB6 projects were AOL "progs"--things that interacted with AOL chat, mostly. For the blog, I started with html, then .shtml (server-side-includes), then discovered php. Built a whole CMS using flatfiles, which was the only thing I knew, then rebuild with a database... I have full source of ALL of that! I also discovered linux in that time, and one post mentioned running Mandrake 7.2--which I doubt was the first version I ran. Those skills eventually turned into jobs. And some of my peers who were doing the same thing turned into lifelong, important friendships. Anyway, just reminiscing. I had more fun computing in those few years than I've had in the whole time since. And even though it was for fun, it was easily some of the best-spent time in my life for $$$ in the future.
tyingq 6y ago
"Since this is just a reverse version of cat, I called the program recat." Curious if you compared the end result with "tac". Source: https://github.com/coreutils/coreutils/blob/master/src/tac.c https://github.com/coreutils/coreutils/blob/master/src/tac.c Walkthrough of how it works: http://www.maizure.org/projects/decoded-gnu-coreutils/tac.html http://www.maizure.org/projects/decoded-gnu-coreutils/tac.ht...
- bloak 6y ago
  See also "rev": https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/text-utils/rev.c https://git.kernel.org/pub/scm/utils/util-linux/util-linux.g...
  - tyingq 6y ago
    Ah, yes, though that reverses each line of text individually while maintaining line order, versus reversing the whole file.
  - bewuethr 6y ago
    The recat tool shown seems to have the same effect as tac | rev
    - tyingq 6y ago
      Ah, I see what it does now, thanks for that. It's not exactly the same as "tac | rev" though. For a typical text file with each line ending in a newline...this recat tool outputs a newline first, and the last line is missing a newline. So it's a true reverse printing of last byte to first.
  - _kst_ 6y ago
    Years ago, I wrote a filter that reversed lines and reversed characters within each line after padding with blanks. I ran a text file through it, printed the result, turned the paper upside down, and complained to a sysadmin that the printer was rotating each character in place. (I didn't keep them hanging for long.)
rootsudo 6y ago
Welp, he's done more than me. I kinda stopped writing C++ programs back a decade ago and Carl is making grep games and seashells.
unixhero 6y ago
Getting better at the shell Actually I recommend reading the entire man page for bash. Edit: https://linux.die.net/man/1/bash https://linux.die.net/man/1/bash
- cpach 6y ago
  I take it you mean the Info manual for Bash? ;-)
  - unixhero 6y ago
    Maybe I am! :) I think maybe you are referring to those numbers people used to quote after mentioning a manpage (1) (5) and so on. I totally missed the boat on that one, and never learnt what these designations meant.
    - lordgrenville 6y ago
      https://unix.stackexchange.com/questions/3586/what-do-the-numbers-in-a-man-page-mean https://unix.stackexchange.com/questions/3586/what-do-the-nu...
    - _kst_ 6y ago
      The numbers, as in "bash(1)" refer to sections of the manual. "Info" is a different documentation format, most commonly used for GNU tools (including bash). The section numbers don't generally apply to info documents. For some tools, the man and info documents have the same information. For some, the man page is a brief summary that directs you to the info documentation. For some, there's only a man page. (And for some there isn't even that.)
    - cpach 6y ago
      Nope :) The joke was referring to the fact that the GNU Project (and thus the Bash project) insists on writing their documentation in the Texinfo format. IIRC for a while they used to have man pages that basically said “just read the Info pages instead”. This is not the case anymore, in Linux distros these days even the GNU tools have decent man pages. Anyway, the Bash manual is actually written in the Texinfo format. It’s not so bad, to be honest :) https://www.gnu.org/software/bash/manual/ https://www.gnu.org/software/bash/manual/
AdmiralAsshat 6y ago
I always thought about writing a GNU/POSIX utility called 'pp' (prepend), that would simply take an input file and insert it above the first line of a target file. So, essentially, if I have a CSV file without a header and want to quickly insert the header file, I would run: pp header.txt spreadsheet.csv And it would be the equivalent of: cat header.txt spreadsheet.csv > temp mv temp spreadsheet.csv But also support all of the expected POSIX niceties, reading from stdin, etc. But I never got around to it, and it's been a few years since I had to use C at my day-job (which was never great to begin with), so it fell to the wayside.
- dj_mc_merlin 6y ago
  Why not just `pp() { echo -e "$(cat $1 $2)" > $2;}`? It can read from stdin just fine.
- jamespwilliams 6y ago
  sponge from moreutils is useful for this, eg cat a b | sponge a
- tyingq 6y ago
  The old line editor, ed, is good at this for files. I assume you could tinker a bit to get stdin support. echo -e "0r header.txt\nw" | ed spreadsheet.csv
  - tyingq 6y ago
    Replying to my own post, looks like ex is more straightforward. ex -sc '0r header.txt' -c wq spreadsheet.csv And ugly, but works for stdin: echo "whee" | ex -sc '0r /dev/stdin' -c wq somefile
geocrasher 6y ago
According to a post in the comments, the author is 18 years old which means that I've been using Linux longer than the author has been making use of the blood in his veins. But that does not mean that I didn't learn something from his post! While the technical accomplishments and understanding are commendable, I think that the main takeaway for me is the excellent approach to learning. Keeping it simple and taking an open-minded approach to learning (as exemplified by his attitude toward systemd) is an approach that more people could stand to take. So often we complicate things in our heads to the point where they seem unknowable and so we assume that they are too difficult and we don't try. I have seen this in myself at times and I see it others as well. I'll be sharing this post at work so that the people who I work with who don't really understand Linux so well can appreciate the approach that was taken and hopefully also glean something from the excellent write-up itself.
- abnry 6y ago
  I am not the author but this is such a lovely comment and a great thing to see on HN.
- carltheperson 6y ago
  Thank you so much for your awesome comment. You made my day!
  - geocrasher 6y ago
    /tips hat
  - gerdesj 6y ago
    I've also been a Linux user for cough quite a long time and I love your approach. You are also a good communicator - you clearly understand that a blog posting for general consumption should be short, pithy, focussed and short. The internets have a very short attention span ... oh look squirrel. Good skills Sir. At your age I couldn't find my arse with both hands 8)
- ghoomketu 6y ago
  > I think that the main takeaway for me is the excellent approach to learning As in the well-known dictum by Richard Feynman, “What I cannot create, I do not understand,” successful design is also a powerful way to show that a design principle has been understood.
dakna 6y ago
> The first thing I did was to read the entire Wikipedia page on UNIX. I also read this original paper written by Dennis Ritchie and Ken Thompson from 1974 We need more people like you. Kudos for having the attention span to do this. Enjoy your HN top spot, well deserved.
- RicoElectrico 6y ago
  I find good Wikipedia articles with competent copy-editing, pure fact and no nonsense to be a good read - at least much more so that many pseudo-intellectually-sounding articles from a few outlets that get posted on HN regularly.
  - moehm 6y ago
    If you find yourself one day without a good Wikipedia article to read, I collect (good) articles and their respective discussions posted on HN here: https://www.mostdiscussed.com/ https://www.mostdiscussed.com/
    - dakna 6y ago
      Nice work combining the two sources, saved!
- segmondy 6y ago
  When I got into Unix in the 90's, the first thing I did was read all the man page for every command and tried to run as many of them as I could with as many options. Decades later it still pays off. If you want to get good at sometime, just dive in.
  - carltheperson 6y ago
    Interesting how Unix/Linux knowledge stays relevant for so long. It feels like a lot of tech knowledge today gets obsolete every few years.
    - 6y ago
jvanderbot 6y ago
What a great bundle of projects. Kudos.
uyt 6y ago
Projects like these is exactly what university use for teaching too! You're way ahead of the curve. For example CMU also makes you write your own shell. The main focus is implementing job control like: jobs, fg, bg, and various signal handlers in order to understand how forking child processes work. I see that your shell doesn't handle much other than cd so this might be a nice next step if you want to continue working on it. (I can't find a good public link for it but you can probably follow along using: http://www.cs.cmu.edu/afs/cs/academic/class/15213-s02/www/applications/labs/lab5/shlab.html#SPECIFICATIONS http://www.cs.cmu.edu/afs/cs/academic/class/15213-s02/www/ap...)
- mav3rick 6y ago
  213 is a wonderful course
smlckz 6y ago
On 8. Processes - Stranger Danger: some improvements sudo ps -ef | awk -v user=$USER '$1 != user && $1 != "root"'
Farfignoggen 6y ago
Interesting but can there be a project to get OpenCL over Vulkan so maybe there can be out of the Box/ISO support for OpenCL for Linux on AMD's Ryzen APUs for OpenCL compute without having to install the OpenCL parts from the AMD Linux Pro driver stack. MESA appears to be All about OpenGL and Vulkan for Games but there's not much attention paid to OpenCL and GPGPU compute for Things Like Blender 3D's GPU Accelerated Cycles rendering, and other Applications like Dark Table, Gimp, etc. that need OpenCL for GPGPU/Compute acceleration workloads! There really needs to be a Primer for Linux and How that OpenCL is plumbed into the system and what files are used to get that all working on Both Integrated and Discrete GPU based laptops! So What's the Linux equivalent of MS's WDDM and how does the Linux Kernel get Graphics and the Graphics APIs and Drivers working properly for User Space Applications to make use of.
leephillips 6y ago
I’d like to echo some other comments and mention that this is the way to learn things like this. If I want to learn a new programming language, I write a program in it. I think the author’s Createservice script might be useful!
acomjean 6y ago
neat. Great way to explore the systems. For me out of university, it was a co-workers books (some of which I ended up buying) that help me understand the role of UNIX better (I used the alpha machines at university, but was thrown into HPUX/Solaris at work..) I think this was one: https://www.amazon.com/Advanced-Programming-UNIX-Environment-3rd/dp/0321637739 https://www.amazon.com/Advanced-Programming-UNIX-Environment... Of course a little out of date now, but a lot of the general concepts are the same. Unix Power Tools helped me a lot as well. https://www.oreilly.com/library/view/unix-power-tools/0596003307/ https://www.oreilly.com/library/view/unix-power-tools/059600... How did others learn this stuff?
freedomben 6y ago
Kind of a shameless plug, but you mentioned wanting to get better at Awk. I had that same desire and created a small course based on what I learned. The course got great feedback. I've gotten a lot of nice emails from people thanking me for making Awk understandable to them for the first time. I love getting those kinds of messages! There is a video presentation, and a set of "challenges" you can use to incrementally get more complex with awk, starting from super simple. The challenges repo is on github here: https://github.com/FreedomBen/awk-hack-the-planet https://github.com/FreedomBen/awk-hack-the-planet If you want to watch the videos, there are links in the github repo but for convenience: Presentation video: https://youtu.be/43BNFcOdBlY https://youtu.be/43BNFcOdBlY My solutions to exercises video: https://youtu.be/4UGLsRYDfo8 https://youtu.be/4UGLsRYDfo8
- Defenestresque 6y ago
  Hey, thanks for making the course free! I'm sure it took a lot of time and effort. I'll be definitely checking it out.
  - boneitis 6y ago
    I went through the video and exercises some time ago. I am _very_ glad that I did.
Arrath 6y ago
As someone who's only exposure to non-windows systems is 20 minutes with the raspberry pi I picked up the other week, I think this will be a great resource for me to follow along with as I familiarize myself with everything. Thanks!
- carltheperson 6y ago
  Let me know if you have any questions.
NexRebular 6y ago
What a fantastic idea! Should utilize something similar in teaching FreeBSD and illumos too.
fuball63 6y ago
I really like the idea of using games to teach command line concepts, like the author did with grep and find. There have been several other ones that have popped up about using vim [1] and navigating a filesystem via cd/ls[2]. I've always wanted to make a unix sandbox environment, using FreeBSD jails provisioned from a webapp, that has little challenges and games to teach the basics of the command line all the way up to hosting a basic HTML website with Apache. Is this something people would find useful? I've been thinking about implementations but I don't want to jump too far into it without validation that this would be something people would find helpful and interesting. [1] https://vim-adventures.com/ https://vim-adventures.com/ [2] https://gitlab.com/slackermedia/bashcrawl https://gitlab.com/slackermedia/bashcrawl
- krylon 6y ago
  It sounds like a brilliant idea. I remember learning my way around vim and emacs from interactive tutorials that were basically just text files. I had no idea what I was doing, but it was fun to experiment. If you implemented your game, I would definitely give it a try. More for fun than for the learning experience, but if I happen to learn something new along the way, I certainly would not complain. ;-)
RMPR 6y ago
> Can’t knock it till you try it, so here I am trying to learn Bash scripting. For bash scripting I found Julia Evans' zine bite size bash very interesting. Even though I had been scripting for quite some time I found hidden gems in there.
jlangemeier 6y ago
One of the biggest improvements I saw in my Linux usage/admin was when I started trying to challenge myself to use the mouse as little as possible; it was absolutely eye-opening when I could navigate around my machine, do all of my everyday processes, and never really leave "home row". It also made VM'ing into a windows server a total pain because it was fairly quickly back to "clicky-clicky" navigation and processing (although win+R, tabbing and arrow keys could get you by about 90% of the time). Great article, and great approach to learning the inner workings of the standard Linux suite!
FactCore 6y ago
I've been sniffing around for projects to do to improve my coding ability and understanding of different programs/languages. Your article really motivated me to start some small projects of my own! I like the idea of not getting yourself down with giant projects and instead just making small applications of new technologies you want to learn.
- carltheperson 6y ago
  It makes me so happy that I motivated you to create projects. Good luck!
jldugger 6y ago
> That thing ended up being a program that reverses the contents of a text file. Since this is just a reverse version of cat, I called the program recat. because `tac` was already taken ;)
__soter__ 6y ago
This kinda mirrors advice I would give my younger sibling when he was struggling to pick up programming, even though that is the direction he wanted his career to go. He thought he needed to know everything and saw that as an impossible mountain to climb. My advice to him was to find a small problem he cared about in the world or at home and develop a solution. When it's a problem you care about or want resolved, you are more invested and learning becomes fun. Then you can build on those skills with what ever you find to work on next.
mraza007 6y ago
What a great way to learn more about linux. really interesting article, I love how you implemented the programs in Golang. I had a very similar idea of implementing the Linux programs in python back then. Keep it up !!!
- carltheperson 6y ago
  The main reason behind why I choose Golang was that it compiles to binaries, but it is also a really simple and easy to learn language.
mqus 6y ago
Another more Linuxy-Thing to try: build a pam (authentication/authorization/etc) module to embed your own secret or come up with a fun/unique way to enter your password (e.g. you can not login unless you plugged your headphones in, etc :) ) This makes you very aware of the way login works in linux and teaches you how to compile your library as a shared object conforming to an interface. Of course, you should be aware that you just baked your own security mechanism, so use it that way and better avoid using it for anything sensitive :)
- LukeShu 6y ago
  Playing with PAM is definitely worth-while (and don't forget to have a root shell open the whole time so you can fix it when you break things!) It really drives home exactly what the kernel's view of users is, and what is just conventions in userspace. I think PAM was the last piece of GNU/Linux to transform from magic to understanding for me.
imhoguy 6y ago
Another programming exercise: writing own userspace VFS (Virtual File System).
trilinearnz 6y ago
Wholesome, and tangible results to boot. A great role model of 'growth mindset' for other people who are interested in technology.
jzer0cool 6y ago
When I took a programming class a student sitting next to me introduced me to Linux. I was like "what is that?" I honestly thought these kinds of machines were just in the movies. I loved how it boot in a black screen, text scrolling, and there was a penguin. Unlike today, there was no Ubuntu. I didn't have wireless at home. It took a while to get Disks burned to install. I had struggles moving back and forth between two machines across rooms to look up "help" documentation from the internet. When it was all installed, nothing worked. Network cards, sound, etc. When my network card worked, it felt like heaven. When my screen resolution was fixed (x11 settings), felt like heaven. Repeat. He had asked, do you really want to learn how it works? I said, yes, let's start there. I was very confused with all the Linux variants so he recommended one. Now I look back and unsure that is where I should have started. L.O.L.
- code_duck 6y ago
  That's roughly how I started, around 1999. I loved the all text console, multiple screens, and how the shell was essentially the same as my beloved Amiga OS. I went through countless distros - Corel, Suse, Mandrake, Debian, Red Hat, Caldera, Yellow Dog, Mandriva, and leagues of desktops and WMs... KDE, Gnome, IceWM, Xfce, xmonad, WindowMaker, Enlightenment, BlackBox, OpenBox, LXDE, Fluxbox. I'm not sure what effect this total lack of interface consistency has on my psyche. Every time I upgraded the kernel or distro there was a significant chance the audio or video would stop working and require 15 minutes through 2 days to restore. It turned out frequently the easiest way to get it to work again was reinstall. The positive aspect of this masochistic computing experience is that when I launched a startup 8 years later I was quite comfortable filling the sysadmin role.
temp0826 6y ago
Surprised that Linux From Scratch hasn’t been mentioned. Probably the best soup-to-nuts walkthrough out there for learning deeply about Linux. And I do agree with others that this is commendable- I remember my struggles with getting into *nix vividly. Keeping a beginner’s mindset and staying humble will take you far. http://linuxfromscratch.org/ http://linuxfromscratch.org/
bernardv 6y ago
Best post I’ve read today and nice to see so many positive comments here. Nothing like building something useful to learn a new tool or technology.
musicale 6y ago
> I settled on the name SeaShell, which I found way too funny. not to be confused with the C shell, csh/tcsh(1)
every 6y ago
One of my pastimes is randomly exploring the Unix Toolkit. It's basically, "What is this thing? What does it do? Do I have a use for it?" And even if the answer is "no", at least I now know it exists and where to find it should my needs change...
johnthescott 6y ago
welcome to the monkey house. the community behind unix has always been my favorite aspect. many, many years ago i met dennis ritchie at the computer literacy bookstore in san jose. in the Q&A he commented that when he realized a circuit board was essentially a small network, the entire design of unix came to him in a flash. unix is an i/o multiplexer with light security.
systematical 6y ago
And here I am just trying to learn sed and awk THIS YEAR. Granted I have other stuff on my to-learn-list, but still.
asicsp 6y ago
>Awk and sed are more of the UNIX magic that I have always thought was really cool, though I never really understood what they were used for. I was so put off by their syntax that I didn't even bother to try to learn them when I was a newbie. If I had a time machine, I would certainly go back and tell myself to learn these tools and save plenty of time instead of always using vim/perl for every text processing problem. I've now written my own books (https://github.com/learnbyexample/scripting_course#ebooks https://github.com/learnbyexample/scripting_course#ebooks) - might help you. For `bash`, I'd highly recommend these resources: * https://mywiki.wooledge.org/BashGuide https://mywiki.wooledge.org/BashGuide, https://mywiki.wooledge.org/BashFAQ https://mywiki.wooledge.org/BashFAQ, https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls, etc * https://www.shellcheck.net/ https://www.shellcheck.net/ * https://devmanual.gentoo.org/tools-reference/bash/index.html https://devmanual.gentoo.org/tools-reference/bash/index.html Also, https://linuxjourney.com/ https://linuxjourney.com/ is nice for many topics.
- arendtio 6y ago
  Actually, I am more of a POSIX shell person, so I would like to add https://shellhaters.org https://shellhaters.org It is a simple page with links to the corresponding parts of the POSIX standard which makes it a lot easier to find the correct documentation when you need it. For those who wonder what POSIX is: It is a standard for operating systems and one part is about shells. So if a shell whats to be POSIX compliant it has to support certain commands. And if you write POSIX commpliant scripts, they won't just run on Linux, but also on MacOS, other BSDs and other POSIX compliant operating systems. Bash supports POSIX, but if you use Bash extensions that are not part of POSIX, you might run into trouble with other shells (e.g. dash on Debian).
potato24 6y ago
I guess what we all learned today is if you want to do something break it into its most simple components and do it.
mitnk 6y ago
Another direction to learn Linux/Unix is spending time to learn from decent books like TLPI. https://man7.org/tlpi/ https://man7.org/tlpi/
zelphirkalt 6y ago
Getting better at "Linux"
florg 6y ago
A great book in the same vein is: Understanding UNIX/Linux-Programming by Bruce Molay.[0] It also delves into the basic tools and then writes them from scratch. I found it quite inspiring as to where to spend time while learning UNIX. [0] https://www.amazon.com/Understanding-UNIX-LINUX-Programming-Practice/dp/0130083968/ https://www.amazon.com/Understanding-UNIX-LINUX-Programming-...

1009 points by carltheperson 6y ago

Hi Carl, purely judging from your profile picture it seems that you're about half my age :) and I think it's excellent that you're doing a bunch of things - personal blog, linux, tutorial post... Just remember - most of the guys around here tend to do it for marketing reasons (i.e. we want to find a job and we're advertising our capabilities), and those projects might not be the most interesting things out there. Bottom line is - just because everybody else is doing it, might be because they have different motives than you, and I hope you'll do/build firstly what's interesting and fun, and secondly what's financially viable. Good luck, have fun!

dj_mc_merlin 6y ago
I got hired at (probably about his age?) just due to side projects like this. If nothing, it shows motivation. So it's got a marketing reason nonetheless.
- CryptoGhost 6y ago
  Isn't this the programmer equivalent of one of those old stories "just go talk to the manager and give him a firm handshake." All the recent research in hiring seems to point to trends that white board / hackerrank / leetcode style filtering are up and someone with hiring authority actually looking at a portfolio or code repo at an all time low.
  - saagarjha 6y ago
    Getting things on the front page of Hacker News remains a good way to get people to contact you. Source: have had people contact me with job-related things for that reason.
carltheperson 6y ago
I'm 18 (: Thank you so much for the heads up. I'm only doing this because it's fun! I love using Linux and I feel like I learned a ton from this.
- smlckz 6y ago
  being slightly younger than you, i can not but envy you: i have done zero projects or had fun like you have had. hmm.
  - log101 6y ago
    Envy = Desire, I'm sure you'll do similar side-projects soon!
  - saagarjha 6y ago
    Sounds like you've still got some time to do some projects for fun and be like him?
- jvanderbot 6y ago
  At what point did the smiley face reverse? It seems anyone under 24-ish uses (: where as I know it as :)
  - saagarjha 6y ago
    I don't think I've ever seen anyone using (:, and I'm a bit younger than 24-ish :)
    - darkwater 6y ago
      I remember (: being used by left-handed in the late '90s, but maybe it was just the people I knew that used it for that reason.
  - thebruce87m 6y ago
    And when did the nose fall off too?
mod 6y ago
Just a couple days ago I was going through all kinds of things I made when I was 18. That was almost 20 years ago, and I had VB6 projects and a blog (we called them E/N sites, the word blog wasn't coined yet). The VB6 projects were AOL "progs"--things that interacted with AOL chat, mostly. For the blog, I started with html, then .shtml (server-side-includes), then discovered php. Built a whole CMS using flatfiles, which was the only thing I knew, then rebuild with a database... I have full source of ALL of that! I also discovered linux in that time, and one post mentioned running Mandrake 7.2--which I doubt was the first version I ran. Those skills eventually turned into jobs. And some of my peers who were doing the same thing turned into lifelong, important friendships. Anyway, just reminiscing. I had more fun computing in those few years than I've had in the whole time since. And even though it was for fun, it was easily some of the best-spent time in my life for $$$ in the future.

tyingq 6y ago

"Since this is just a reverse version of cat, I called the program recat." Curious if you compared the end result with "tac". Source: https://github.com/coreutils/coreutils/blob/master/src/tac.c https://github.com/coreutils/coreutils/blob/master/src/tac.c Walkthrough of how it works: http://www.maizure.org/projects/decoded-gnu-coreutils/tac.html http://www.maizure.org/projects/decoded-gnu-coreutils/tac.ht...

bloak 6y ago
See also "rev": https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/text-utils/rev.c https://git.kernel.org/pub/scm/utils/util-linux/util-linux.g...
- tyingq 6y ago
  Ah, yes, though that reverses each line of text individually while maintaining line order, versus reversing the whole file.
- bewuethr 6y ago
  The recat tool shown seems to have the same effect as tac | rev
  - tyingq 6y ago
    Ah, I see what it does now, thanks for that. It's not exactly the same as "tac | rev" though. For a typical text file with each line ending in a newline...this recat tool outputs a newline first, and the last line is missing a newline. So it's a true reverse printing of last byte to first.
- _kst_ 6y ago
  Years ago, I wrote a filter that reversed lines and reversed characters within each line after padding with blanks. I ran a text file through it, printed the result, turned the paper upside down, and complained to a sysadmin that the printer was rotating each character in place. (I didn't keep them hanging for long.)

rootsudo 6y ago

Welp, he's done more than me. I kinda stopped writing C++ programs back a decade ago and Carl is making grep games and seashells.

unixhero 6y ago

Getting better at the shell Actually I recommend reading the entire man page for bash. Edit: https://linux.die.net/man/1/bash https://linux.die.net/man/1/bash

cpach 6y ago
I take it you mean the Info manual for Bash? ;-)
- unixhero 6y ago
  Maybe I am! :) I think maybe you are referring to those numbers people used to quote after mentioning a manpage (1) (5) and so on. I totally missed the boat on that one, and never learnt what these designations meant.
  - lordgrenville 6y ago
    https://unix.stackexchange.com/questions/3586/what-do-the-numbers-in-a-man-page-mean https://unix.stackexchange.com/questions/3586/what-do-the-nu...
  - _kst_ 6y ago
    The numbers, as in "bash(1)" refer to sections of the manual. "Info" is a different documentation format, most commonly used for GNU tools (including bash). The section numbers don't generally apply to info documents. For some tools, the man and info documents have the same information. For some, the man page is a brief summary that directs you to the info documentation. For some, there's only a man page. (And for some there isn't even that.)
  - cpach 6y ago
    Nope :) The joke was referring to the fact that the GNU Project (and thus the Bash project) insists on writing their documentation in the Texinfo format. IIRC for a while they used to have man pages that basically said “just read the Info pages instead”. This is not the case anymore, in Linux distros these days even the GNU tools have decent man pages. Anyway, the Bash manual is actually written in the Texinfo format. It’s not so bad, to be honest :) https://www.gnu.org/software/bash/manual/ https://www.gnu.org/software/bash/manual/

AdmiralAsshat 6y ago

I always thought about writing a GNU/POSIX utility called 'pp' (prepend), that would simply take an input file and insert it above the first line of a target file. So, essentially, if I have a CSV file without a header and want to quickly insert the header file, I would run: pp header.txt spreadsheet.csv And it would be the equivalent of: cat header.txt spreadsheet.csv > temp mv temp spreadsheet.csv But also support all of the expected POSIX niceties, reading from stdin, etc. But I never got around to it, and it's been a few years since I had to use C at my day-job (which was never great to begin with), so it fell to the wayside.

dj_mc_merlin 6y ago
Why not just `pp() { echo -e "$(cat $1 $2)" > $2;}`? It can read from stdin just fine.
jamespwilliams 6y ago
sponge from moreutils is useful for this, eg cat a b | sponge a
tyingq 6y ago
The old line editor, ed, is good at this for files. I assume you could tinker a bit to get stdin support. echo -e "0r header.txt\nw" | ed spreadsheet.csv
- tyingq 6y ago
  Replying to my own post, looks like ex is more straightforward. ex -sc '0r header.txt' -c wq spreadsheet.csv And ugly, but works for stdin: echo "whee" | ex -sc '0r /dev/stdin' -c wq somefile

geocrasher 6y ago

According to a post in the comments, the author is 18 years old which means that I've been using Linux longer than the author has been making use of the blood in his veins. But that does not mean that I didn't learn something from his post! While the technical accomplishments and understanding are commendable, I think that the main takeaway for me is the excellent approach to learning. Keeping it simple and taking an open-minded approach to learning (as exemplified by his attitude toward systemd) is an approach that more people could stand to take. So often we complicate things in our heads to the point where they seem unknowable and so we assume that they are too difficult and we don't try. I have seen this in myself at times and I see it others as well. I'll be sharing this post at work so that the people who I work with who don't really understand Linux so well can appreciate the approach that was taken and hopefully also glean something from the excellent write-up itself.

abnry 6y ago
I am not the author but this is such a lovely comment and a great thing to see on HN.
carltheperson 6y ago
Thank you so much for your awesome comment. You made my day!
- geocrasher 6y ago
  /tips hat
- gerdesj 6y ago
  I've also been a Linux user for cough quite a long time and I love your approach. You are also a good communicator - you clearly understand that a blog posting for general consumption should be short, pithy, focussed and short. The internets have a very short attention span ... oh look squirrel. Good skills Sir. At your age I couldn't find my arse with both hands 8)
ghoomketu 6y ago
> I think that the main takeaway for me is the excellent approach to learning As in the well-known dictum by Richard Feynman, “What I cannot create, I do not understand,” successful design is also a powerful way to show that a design principle has been understood.

dakna 6y ago

> The first thing I did was to read the entire Wikipedia page on UNIX. I also read this original paper written by Dennis Ritchie and Ken Thompson from 1974 We need more people like you. Kudos for having the attention span to do this. Enjoy your HN top spot, well deserved.

RicoElectrico 6y ago
I find good Wikipedia articles with competent copy-editing, pure fact and no nonsense to be a good read - at least much more so that many pseudo-intellectually-sounding articles from a few outlets that get posted on HN regularly.
- moehm 6y ago
  If you find yourself one day without a good Wikipedia article to read, I collect (good) articles and their respective discussions posted on HN here: https://www.mostdiscussed.com/ https://www.mostdiscussed.com/
  - dakna 6y ago
    Nice work combining the two sources, saved!
segmondy 6y ago
When I got into Unix in the 90's, the first thing I did was read all the man page for every command and tried to run as many of them as I could with as many options. Decades later it still pays off. If you want to get good at sometime, just dive in.
- carltheperson 6y ago
  Interesting how Unix/Linux knowledge stays relevant for so long. It feels like a lot of tech knowledge today gets obsolete every few years.
  - 6y ago

jvanderbot 6y ago

What a great bundle of projects. Kudos.

uyt 6y ago

Projects like these is exactly what university use for teaching too! You're way ahead of the curve. For example CMU also makes you write your own shell. The main focus is implementing job control like: jobs, fg, bg, and various signal handlers in order to understand how forking child processes work. I see that your shell doesn't handle much other than cd so this might be a nice next step if you want to continue working on it. (I can't find a good public link for it but you can probably follow along using: http://www.cs.cmu.edu/afs/cs/academic/class/15213-s02/www/applications/labs/lab5/shlab.html#SPECIFICATIONS http://www.cs.cmu.edu/afs/cs/academic/class/15213-s02/www/ap...)

mav3rick 6y ago
213 is a wonderful course

smlckz 6y ago

On 8. Processes - Stranger Danger: some improvements sudo ps -ef | awk -v user=$USER '$1 != user && $1 != "root"'

Farfignoggen 6y ago

Interesting but can there be a project to get OpenCL over Vulkan so maybe there can be out of the Box/ISO support for OpenCL for Linux on AMD's Ryzen APUs for OpenCL compute without having to install the OpenCL parts from the AMD Linux Pro driver stack. MESA appears to be All about OpenGL and Vulkan for Games but there's not much attention paid to OpenCL and GPGPU compute for Things Like Blender 3D's GPU Accelerated Cycles rendering, and other Applications like Dark Table, Gimp, etc. that need OpenCL for GPGPU/Compute acceleration workloads! There really needs to be a Primer for Linux and How that OpenCL is plumbed into the system and what files are used to get that all working on Both Integrated and Discrete GPU based laptops! So What's the Linux equivalent of MS's WDDM and how does the Linux Kernel get Graphics and the Graphics APIs and Drivers working properly for User Space Applications to make use of.

leephillips 6y ago

I’d like to echo some other comments and mention that this is the way to learn things like this. If I want to learn a new programming language, I write a program in it. I think the author’s Createservice script might be useful!

acomjean 6y ago

neat. Great way to explore the systems. For me out of university, it was a co-workers books (some of which I ended up buying) that help me understand the role of UNIX better (I used the alpha machines at university, but was thrown into HPUX/Solaris at work..) I think this was one: https://www.amazon.com/Advanced-Programming-UNIX-Environment-3rd/dp/0321637739 https://www.amazon.com/Advanced-Programming-UNIX-Environment... Of course a little out of date now, but a lot of the general concepts are the same. Unix Power Tools helped me a lot as well. https://www.oreilly.com/library/view/unix-power-tools/0596003307/ https://www.oreilly.com/library/view/unix-power-tools/059600... How did others learn this stuff?

freedomben 6y ago

Kind of a shameless plug, but you mentioned wanting to get better at Awk. I had that same desire and created a small course based on what I learned. The course got great feedback. I've gotten a lot of nice emails from people thanking me for making Awk understandable to them for the first time. I love getting those kinds of messages! There is a video presentation, and a set of "challenges" you can use to incrementally get more complex with awk, starting from super simple. The challenges repo is on github here: https://github.com/FreedomBen/awk-hack-the-planet https://github.com/FreedomBen/awk-hack-the-planet If you want to watch the videos, there are links in the github repo but for convenience: Presentation video: https://youtu.be/43BNFcOdBlY https://youtu.be/43BNFcOdBlY My solutions to exercises video: https://youtu.be/4UGLsRYDfo8 https://youtu.be/4UGLsRYDfo8

Defenestresque 6y ago
Hey, thanks for making the course free! I'm sure it took a lot of time and effort. I'll be definitely checking it out.
- boneitis 6y ago
  I went through the video and exercises some time ago. I am _very_ glad that I did.

Arrath 6y ago

As someone who's only exposure to non-windows systems is 20 minutes with the raspberry pi I picked up the other week, I think this will be a great resource for me to follow along with as I familiarize myself with everything. Thanks!

carltheperson 6y ago
Let me know if you have any questions.

NexRebular 6y ago

What a fantastic idea! Should utilize something similar in teaching FreeBSD and illumos too.

fuball63 6y ago

I really like the idea of using games to teach command line concepts, like the author did with grep and find. There have been several other ones that have popped up about using vim [1] and navigating a filesystem via cd/ls[2]. I've always wanted to make a unix sandbox environment, using FreeBSD jails provisioned from a webapp, that has little challenges and games to teach the basics of the command line all the way up to hosting a basic HTML website with Apache. Is this something people would find useful? I've been thinking about implementations but I don't want to jump too far into it without validation that this would be something people would find helpful and interesting. [1] https://vim-adventures.com/ https://vim-adventures.com/ [2] https://gitlab.com/slackermedia/bashcrawl https://gitlab.com/slackermedia/bashcrawl

krylon 6y ago
It sounds like a brilliant idea. I remember learning my way around vim and emacs from interactive tutorials that were basically just text files. I had no idea what I was doing, but it was fun to experiment. If you implemented your game, I would definitely give it a try. More for fun than for the learning experience, but if I happen to learn something new along the way, I certainly would not complain. ;-)

RMPR 6y ago

> Can’t knock it till you try it, so here I am trying to learn Bash scripting. For bash scripting I found Julia Evans' zine bite size bash very interesting. Even though I had been scripting for quite some time I found hidden gems in there.

jlangemeier 6y ago

One of the biggest improvements I saw in my Linux usage/admin was when I started trying to challenge myself to use the mouse as little as possible; it was absolutely eye-opening when I could navigate around my machine, do all of my everyday processes, and never really leave "home row". It also made VM'ing into a windows server a total pain because it was fairly quickly back to "clicky-clicky" navigation and processing (although win+R, tabbing and arrow keys could get you by about 90% of the time). Great article, and great approach to learning the inner workings of the standard Linux suite!

FactCore 6y ago

I've been sniffing around for projects to do to improve my coding ability and understanding of different programs/languages. Your article really motivated me to start some small projects of my own! I like the idea of not getting yourself down with giant projects and instead just making small applications of new technologies you want to learn.

carltheperson 6y ago
It makes me so happy that I motivated you to create projects. Good luck!

jldugger 6y ago

> That thing ended up being a program that reverses the contents of a text file. Since this is just a reverse version of cat, I called the program recat. because `tac` was already taken ;)

__soter__ 6y ago

This kinda mirrors advice I would give my younger sibling when he was struggling to pick up programming, even though that is the direction he wanted his career to go. He thought he needed to know everything and saw that as an impossible mountain to climb. My advice to him was to find a small problem he cared about in the world or at home and develop a solution. When it's a problem you care about or want resolved, you are more invested and learning becomes fun. Then you can build on those skills with what ever you find to work on next.

mraza007 6y ago

What a great way to learn more about linux. really interesting article, I love how you implemented the programs in Golang. I had a very similar idea of implementing the Linux programs in python back then. Keep it up !!!

carltheperson 6y ago
The main reason behind why I choose Golang was that it compiles to binaries, but it is also a really simple and easy to learn language.

mqus 6y ago

Another more Linuxy-Thing to try: build a pam (authentication/authorization/etc) module to embed your own secret or come up with a fun/unique way to enter your password (e.g. you can not login unless you plugged your headphones in, etc :) ) This makes you very aware of the way login works in linux and teaches you how to compile your library as a shared object conforming to an interface. Of course, you should be aware that you just baked your own security mechanism, so use it that way and better avoid using it for anything sensitive :)

LukeShu 6y ago
Playing with PAM is definitely worth-while (and don't forget to have a root shell open the whole time so you can fix it when you break things!) It really drives home exactly what the kernel's view of users is, and what is just conventions in userspace. I think PAM was the last piece of GNU/Linux to transform from magic to understanding for me.

imhoguy 6y ago

Another programming exercise: writing own userspace VFS (Virtual File System).

trilinearnz 6y ago

Wholesome, and tangible results to boot. A great role model of 'growth mindset' for other people who are interested in technology.

jzer0cool 6y ago

When I took a programming class a student sitting next to me introduced me to Linux. I was like "what is that?" I honestly thought these kinds of machines were just in the movies. I loved how it boot in a black screen, text scrolling, and there was a penguin. Unlike today, there was no Ubuntu. I didn't have wireless at home. It took a while to get Disks burned to install. I had struggles moving back and forth between two machines across rooms to look up "help" documentation from the internet. When it was all installed, nothing worked. Network cards, sound, etc. When my network card worked, it felt like heaven. When my screen resolution was fixed (x11 settings), felt like heaven. Repeat. He had asked, do you really want to learn how it works? I said, yes, let's start there. I was very confused with all the Linux variants so he recommended one. Now I look back and unsure that is where I should have started. L.O.L.

code_duck 6y ago
That's roughly how I started, around 1999. I loved the all text console, multiple screens, and how the shell was essentially the same as my beloved Amiga OS. I went through countless distros - Corel, Suse, Mandrake, Debian, Red Hat, Caldera, Yellow Dog, Mandriva, and leagues of desktops and WMs... KDE, Gnome, IceWM, Xfce, xmonad, WindowMaker, Enlightenment, BlackBox, OpenBox, LXDE, Fluxbox. I'm not sure what effect this total lack of interface consistency has on my psyche. Every time I upgraded the kernel or distro there was a significant chance the audio or video would stop working and require 15 minutes through 2 days to restore. It turned out frequently the easiest way to get it to work again was reinstall. The positive aspect of this masochistic computing experience is that when I launched a startup 8 years later I was quite comfortable filling the sysadmin role.

temp0826 6y ago

Surprised that Linux From Scratch hasn’t been mentioned. Probably the best soup-to-nuts walkthrough out there for learning deeply about Linux. And I do agree with others that this is commendable- I remember my struggles with getting into *nix vividly. Keeping a beginner’s mindset and staying humble will take you far. http://linuxfromscratch.org/ http://linuxfromscratch.org/

bernardv 6y ago

Best post I’ve read today and nice to see so many positive comments here. Nothing like building something useful to learn a new tool or technology.

musicale 6y ago

> I settled on the name SeaShell, which I found way too funny. not to be confused with the C shell, csh/tcsh(1)

every 6y ago

One of my pastimes is randomly exploring the Unix Toolkit. It's basically, "What is this thing? What does it do? Do I have a use for it?" And even if the answer is "no", at least I now know it exists and where to find it should my needs change...

johnthescott 6y ago

welcome to the monkey house. the community behind unix has always been my favorite aspect. many, many years ago i met dennis ritchie at the computer literacy bookstore in san jose. in the Q&A he commented that when he realized a circuit board was essentially a small network, the entire design of unix came to him in a flash. unix is an i/o multiplexer with light security.

systematical 6y ago

And here I am just trying to learn sed and awk THIS YEAR. Granted I have other stuff on my to-learn-list, but still.

asicsp 6y ago

>Awk and sed are more of the UNIX magic that I have always thought was really cool, though I never really understood what they were used for. I was so put off by their syntax that I didn't even bother to try to learn them when I was a newbie. If I had a time machine, I would certainly go back and tell myself to learn these tools and save plenty of time instead of always using vim/perl for every text processing problem. I've now written my own books (https://github.com/learnbyexample/scripting_course#ebooks https://github.com/learnbyexample/scripting_course#ebooks) - might help you. For `bash`, I'd highly recommend these resources: * https://mywiki.wooledge.org/BashGuide https://mywiki.wooledge.org/BashGuide, https://mywiki.wooledge.org/BashFAQ https://mywiki.wooledge.org/BashFAQ, https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls, etc * https://www.shellcheck.net/ https://www.shellcheck.net/ * https://devmanual.gentoo.org/tools-reference/bash/index.html https://devmanual.gentoo.org/tools-reference/bash/index.html Also, https://linuxjourney.com/ https://linuxjourney.com/ is nice for many topics.

arendtio 6y ago
Actually, I am more of a POSIX shell person, so I would like to add https://shellhaters.org https://shellhaters.org It is a simple page with links to the corresponding parts of the POSIX standard which makes it a lot easier to find the correct documentation when you need it. For those who wonder what POSIX is: It is a standard for operating systems and one part is about shells. So if a shell whats to be POSIX compliant it has to support certain commands. And if you write POSIX commpliant scripts, they won't just run on Linux, but also on MacOS, other BSDs and other POSIX compliant operating systems. Bash supports POSIX, but if you use Bash extensions that are not part of POSIX, you might run into trouble with other shells (e.g. dash on Debian).

potato24 6y ago

I guess what we all learned today is if you want to do something break it into its most simple components and do it.

mitnk 6y ago

Another direction to learn Linux/Unix is spending time to learn from decent books like TLPI. https://man7.org/tlpi/ https://man7.org/tlpi/

zelphirkalt 6y ago

Getting better at "Linux"

florg 6y ago

A great book in the same vein is: Understanding UNIX/Linux-Programming by Bruce Molay.[0] It also delves into the basic tools and then writes them from scratch. I found it quite inspiring as to where to spend time while learning UNIX. [0] https://www.amazon.com/Understanding-UNIX-LINUX-Programming-Practice/dp/0130083968/ https://www.amazon.com/Understanding-UNIX-LINUX-Programming-...