6 ms·
Build Your Own Redis with C/C++
- mattrighetti 4y agoI would really love to take a look at this book but I'm not really interested in C/C++, has anyone tried to follow along but with another language? How did you find it?
- jvans 4y agoI have done something similar with ruby, this is a great book: https://workingwithruby.com/downloads/Working%20With%20TCP%20Sockets.pdf https://workingwithruby.com/downloads/Working%20With%20TCP%2... I would recommend doing C at some point in your career, nothing comes close in terms of forcing you to understand the hardware you are using.
- Mike_12345 4y agoThat was true in the 70's but C doesn't require that you understand much about modern hardware. https://queue.acm.org/detail.cfm?id=3212479 https://queue.acm.org/detail.cfm?id=3212479
- doodlesdev 4y agoIt still is more low level than any other language currently (that can be run on multiple architectures) which I think is what the OP was saying. If you compile C without compiler optimizations, it will generate code that is exactly what you wrote. The argument the paper makes is that the instructions available in modern processors and the way we use them to optimize code (such as instruction level parallelism) is a consequence of C being so important despite being made for much older and much simpler machines. Because if the generated binaries from any C compiler without optimizations isn't low level then you might as well say the same about x86 assembly, and then you're basically out of options.
- Mike_12345 4y agoBut it's not. It has become a high level language. The idea that C is still a low level language is no longer true. It is highly abstracted from modern hardware. Edit: I just saw your edited comment. Yeah the point is you really don't need to know much about the hardware to write C. It doesn't force you to understand what's actually going on behind the scenes.
- doodlesdev 4y agoYeah sorry about the ninja edit. Very often I reply to a comment without actually finishing and then edit it. I thought Hacker News was delaying my comments by 1 minute though according to my configs? Anyways, I agree that you don't really need a deep understanding of the hardware nowadays to write C, but my point is that it's still useful to learn because you still need more understanding of the hardware than you need to write something such as Ruby. The point the OP was making is that it still forces you to learn more about the hardware than other languages. Arguably, it also forces you to learn more about the software itself with things being much more explicit than in a modern complex language such as Rust. I'm also just not aware of _any_ language that actually maps well to what the CPU is doing nowadays since they are such complex pieces of silicon.
- Mike_12345 4y agoBut does it really force you to learn more about the hardware versus other languages? I am challenging that assumption or belief. C has memory pointers, but those are pointers in a flat memory space abstracted over the physical memory hierarchy. So yes, you do need to understand that the hardware has memory addresses and that pointers can reference memory addresses. Aside from that I don't see much difference to Java or Python in terms of requiring a deeper understanding. Even Python has bitwise operators.
- Kamq 4y ago> Aside from that I don't see much difference to Java or Python in terms of requiring a deeper understanding. The big difference would be that it requires you to understand the difference between stack vs heap allocation.
- _gabe_ 4y agoThis whole article seems to be picking a lot of nits. I didn't read it too deeply, so feel free to correct me if I'm wrong, but the biggest complaints highlighted in the article are: 1. Modern CPUs use instruction level parallelism (ILP) 2. Memory isn't linear (you have separate caches, L1, L2, L3 and main memory) If you've ever debugged a C or C++ program in release, you've quickly found out about ILP. The code still maps relatively close to the hardware, it just won't run in the sequential order you've provided it in. Many C and C++ programmers know this and try to make the implicit assumptions in their code explicit to allow the compiler to reorder instructions more easily/reduce memory dependency chains[0]. And I'm sure you've heard several proponents over the past few years (Mike Acton comes to mind) espousing data oriented design. This is an entire code methodology intended to help the CPU caches, and it shows an implicit understanding that memory is not linear. Heck, most C/C++ programmers realize that they're using VRAM all the time, and the memory locations they get aren't necessarily backed by physical memory until the OS sorts it out. This is especially transparent if you've ever done any sort of memmapping with files or played with virtual memory. Anyways, C doesn't necessarily map directly to the hardware, but it's a heck of a lot easier to intuit what a C program will end up doing on the CPU vs what a Python program will do. And most C/C++ programmers realize this fact, and actively write code to utilize the fact that C does not map directly to hardware. [0]: https://johnnysswlab.com/instruction-level-parallelism-in-practice-speeding-up-memory-bound-programs-with-low-ilp/ https://johnnysswlab.com/instruction-level-parallelism-in-pr...
- josephg 4y agoYeah. Also, learning C is still probably the best way to learn how raw pointers work. And pointers underpin everything - even if you spend your life in Python or Java. When I was teaching programming, it was always a rite of passage for my students to implement a linked list in C. Once it clicked for them, the world of programming opened up. C is also still the universal glue language for FFI. Wanna call Swift from Rust? You can always reach for C.
- Mike_12345 4y agoLikewise it is possible to write optimally performing Java code if you know how what what's going on behind the scenes. https://lmax-exchange.github.io/disruptor/disruptor.html https://lmax-exchange.github.io/disruptor/disruptor.html
- sitzkrieg 4y agoid recommend assembly language to really achieve this. the further away you get from x86 the more enjoyable it seems to be fore a human. skilldricks easy6502 is a really good start
- iudqnolq 4y agoif you're interested in Rust this might help. It's intended to be an introduction to the tokio async runtime for Rust by having you implement a minimal redis. https://tokio.rs/tokio/tutorial/setup https://tokio.rs/tokio/tutorial/setup
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- pjmlp 4y agoC/C++ is two languages, C and C++.
- sbmthakur 4y agoCodecrafters has a similar exercise: https://app.codecrafters.io/courses/redis?track=c https://app.codecrafters.io/courses/redis?track=c
- intelVISA 4y agoNo C++? Hard pass from me.
- JTyQZSnP3cQGa8B 4y agoIt seems to be a nice project but they should remove "C++" from the name since both languages have nothing in common anymore, especially the use of smart pointers and Boost.Asio compared to "malloc and socket" that is used in the book.
- hewlett 4y agoYou can still use raw pointers and sockets in C++, which the code examples provided in the book do
- JTyQZSnP3cQGa8B 4y agoI agree but no one calls this C++ anymore especially since C++14/17, and I fail to see why you would use a C++ compiler for such a code.
- gpderetta 4y agoWell, if you are implementing something like ASIO, and of course c++ is a good language for that, you'll be dealing with sockets directly.
- deleted 4y ago[deleted]
- _gabe_ 4y ago> I agree but no one calls this C++ anymore especially since C++14/17 I do. Who's to say if you use C++, thou shalt always use: smart pointers, move semantics, OOP, templates, and more. Also, you're discounting a very large and very real industry of people writing code for embedded hardware that doesn't necessarily have the luxurious memory resources modern devices have. Why could I ever possibly want to use C++ if I don't want to use smart pointers and OOP and all that other stuff? Well, generic type safety is very nice. Templates don't exist in C. Constexpr and references are also quite nice. They allow me to annotate my code to make my intentions clearer. Operator overloading is also insanely helpful at times. Matrix/vector math is so much nicer to write/read using mathematical operators instead of function calls everywhere. This also isn't possible in C. Anyways, don't go making claims that if somebody doesn't use a language exactly the same way you do, then they're using it wrong. Especially when it's a language as massively overengineered as C++. There are plenty of reasons to prefer a subset of C++ to C, and plenty of valid reasons to prefer to use C instead of dealing with the complexities of C++.
- user5534762135 4y ago"Build your own Redis with Rust/Fortran" If the author actually can't understand the difference between C and C++, there's a pretty low chance of any good code being in the book. If they do, they should keep an eye on the editor next time they go into empty marketing mode for the title.
- noncoml 4y agoI kid you not I once had a recruiter asking someone with experience in C/C+/C++ Not a typo for C#. He really wanted someone with experience in C+
- heywhatupboys 4y agobenefit of the doubt: C with classes?
- noncoml 4y agoMaybe. Who knows what his client asked. To me he confirmed they were looking for C+…
- josephg 4y agoI’ve lost count of the number of recruiters who shorten Javascript to Java. Those are not the same thing! Arghhhhh
- sitzkrieg 4y agothis is also one of my biggest pet peeves too
- matttb 4y ago> The code is written as direct and straightforwardly as the author could. It’s mostly plain C with minimal C++ features. Sounds fair to me
- dang 4y ago"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html "Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- TheChaplain 4y agoFor giggles I was watching this thread waiting to see how long until the word "rust" appeared. Didn't take long :^) Someone on reddit joked that Rust is becoming the new Crossfit meme; "How can you tell when someone programs in Rust?" "Don't worry, they'll tell you."
- noncoml 4y agoYou know what? If I was more confident in my coding abilities, or some seriously good static analysis tool was available for C or C++, I would program in one of those languages. I’m really not big fan of Rust, but I love that’s it’s always looking over my shoulder for stupid mistakes.
- throwaway5959 4y agoIt’s OK. The truth is that even seasoned C/C++ developers can’t write safe C/C++ code. Otherwise exploits wouldn’t exist and static analysis wouldn’t be necessary. What are you building with Rust?
- 1024core 4y ago> The truth is that even seasoned C/C++ developers can’t write safe C/C++ code. Except DJB, of course.... ;-)
- tialaramex 4y agoNo, even DJB: https://lwn.net/Articles/820969/ https://lwn.net/Articles/820969/ > "This claim is denied. Nobody gives gigabytes of memory to each qmail-smtpd process, so there is no problem with qmail's assumption that allocated array lengths fit comfortably into 32 bits." Years later it turns out that well, "nobody" was lots of people. Not DJB of course, DJB can claim that DJB's software works exactly the way DJB intended, but for everybody else not so much.
- noncoml 4y agoI’ve been programming professionally in C since 2005 and still short myself in the foot all the time. I’m Rust I’m found various side projects. An all in one PiHole like dns server, a distributed key value store based on raft and an openwrt module. But nothing interesting or revolutionary enough to share. How about you?
- tylerhannan 4y agoI sort of want, one day, to read the same article targeted at Delphi or Turbo Pascal. lol.
- FpUser 4y agoAll styles of programming for implementing Redis are supported in Delphi. It'll work just fine. As for Turbo Pascal - replace it with FreePascal/Lazarus which together comprise opensource version of Delphi.
- andsoitis 4y ago> lol What’s the joke? I don’t get it.
- password4321 4y agoI've heard ChatGPT is a great programming language translator.
- boffinAudio 4y agoLua. In fact, I think I could do this in Turbo.LUA in about a day.
- heywhatupboys 4y ago> C/C++ there is NO SUCH THING
- Mesopropithecus 4y agoMy fist thought on reading the title and confirmed by looking at the article. Looks like they meant C.
- mzimbres 4y agoI haven't created Redis itself but a Redis client library. It all started with a chat server I wanted to write on top of Boost.Beast (and by consequence Boost.Asio) and at the time none of the clients available would fill my needs and wishes - Be asynchronous and based on Boost.Asio. - Perform automatic command pipelining [1]. All C++ libraries I looked at the time would open new connections for that, which results in unacceptable performance losses. - Parse Redis responses directly in their final data structures avoiding extra copies. This is useful for example to store json strings in Redis and read them back efficiently. With time I built more performance features - Full duplex communication. - Support for RESP3 and server pushes on the same connection that is being used for request/response. - Event demultiplexing: It can server thousands of requests (e.g. websocket sessions) on a single connection to Redis with back pressure. This is important to keep the number of connections to Redis low and avoid latency introduced by countermeasures like [3]. This client was proposed and accepted in Boost (but not yet integrated), interested readers can find links to the review here [2]. [1] https://redis.io/docs/manual/pipelining/ https://redis.io/docs/manual/pipelining/ [2] https://github.com/boostorg/redis/issues/53 https://github.com/boostorg/redis/issues/53 [3] https://github.com/twitter/twemproxy https://github.com/twitter/twemproxy
- spullara 4y agoIt is odd that they are building it in C when it is already written in C.
- actually_a_dog 4y agoThe point of the exercise seems to be to learn C by writing C. If it were to understand how Redis works, then you might as well write it in Python and get it done in ~200 (or whatever) LoC instead of ~1200 LoC. You can't learn C by writing some other language.
- elif 4y agoI will still trust antirez to write code more than myself. For expected behavior, for performance, for simplicity, I can't expect myself to do any of these things better than him. In fact, I find that fitting my code into patterns that work with redis actively improve these aspects of my own code. If I had a monkey wrench into redis (beyond lua), I imagine one or more of these aspects would rapidly deteriorate.
- c-smile 4y agoThe book should start from explaining what Redis is. It is a DB in the first place so "Learn network programming [by writing Redis]" is kind of questionable.
- hgs3 4y agoDoes the book itself have more prose than what's found on the webpages linked from the table of contents? Those webpages are very sparse on actual explanation; they are mostly code dumps. I would expect a book on a key-value store to be a lot more theory focused. You could write a whole book on hash tables alone, but the hash tables "chapter" is only a few paragraphs of actual explanation [1]. [1] https://build-your-own.org/redis/08_hashtables https://build-your-own.org/redis/08_hashtables
- heretoo 4y agoOriginal version was written TCL: https://gist.github.com/antirez/6ca04dd191bdb82aad9fb241013e88a8 https://gist.github.com/antirez/6ca04dd191bdb82aad9fb241013e...
- valbaca 4y agoFor the full "Build Your Own X" list: https://github.com/codecrafters-io/build-your-own-x https://github.com/codecrafters-io/build-your-own-x
- pjmlp 4y agoIt is schizophrenic? So is it C or C++?
- tejohnso 4y agoThis seems to be a common phrasing left over from maybe two or three decades ago. Whenever I see it I assume it's going to be written in terrible non idiomatic c++. Whatever it is, saying c/c++ doesn't not inspire confidence that I'm going to want to learn in that style.
- pjmlp 4y agoI usually argue for using C/C++ when the intent is talking about C and C++, after all we used to have "The C/C++ Users Journal". However programming a Redis clone in C/C++ is definitly not something that happens, it is either C or C++, and if it is in the common subset compiled by compilers of both languages, there are definitly better ways to express it.
- CoolCold 4y agofrom the very first chapter with code (chapter 2 if keep numbers): > There are several Linux system calls we need to learn before we can start socket programming. Have we reached the state where Linux by default is New Unix/New POSIX ? Isn't it alarming for say FreeBSD guys?
- gabrieledarrigo 4y agoI did something similar few years ago, while attending the Networking course at the university: https://github.com/gabrieledarrigo/ducky https://github.com/gabrieledarrigo/ducky I did choose UDP for the sake of simplicity, and I implemented a very naive protocol, using select to handle I/O. I also remember that In tried various benchmarks, but the hash implementation that I used failed after 500k inserts or something. Now, I started to program quite late (after 25 years old), and I mostly do web development, and even if I'm a complete C noob, it was a super funny project! And if someone would like to take a look at the code and share with me an opinion, it would be great!