40 ms·

Why is BIND 10 written in C++ and Python?

103 points by AndrewDucker 14y ago

przemoc 14y ago
For core services like BIND I don't see any good reason to write them in more than one language (actually anything other than C/C++) and introduce cumbersome dependencies because of that. I wouldn't complain if it was some another DIY name server toy created for whatever reason (how much time it would take..., I can do it!, etc.), but it's software that will become widely deployed in upcoming years (almost) without doubt. How much more work these 17% done in Python would take to be (re)written in C++? Would it make the code that much worse in terms of quality and managability? Having coherent, one-language codebase, is a good feature on its own too, often improving above mentioned factors.
- mattmanser 14y ago
  You can look at the source yourself. I don't understand either, randomly opening parts of both the Python and C++ and it's certainly not complicated and they write the Python in a C style anyway. I'm not a C++ coder and I only play with Python now and then, but can't say the code impressed me much. It's actually pretty hard to skim the code because it's massively over-commented and there are far too many tiny 2-line private functions that are called by exactly one other 2-line private function that is called by exactly one other 2-line function. Or Holographic code as John D. Cook called it[1]. And the copyright notice in every source file is extremely irritating. Far too high noise to code ratio for my tastes, so I got bored before I could really 'see' how the Python had helped. but I can't judge that well as I'm not sure if that's all just a bi-product of using C++ and being an open source project that comes out of committee. [1] http://www.johndcook.com/blog/2012/01/09/holographic-source-code/ http://www.johndcook.com/blog/2012/01/09/holographic-source-...
  - eru 14y ago
    In general, abstracting code out into functions can be useful, even when the new functions is only called once. That is because functions are a well understood abstraction. And having code separated into functions makes it easier for the reader to deduce the coupling points: two blocks of code after another can have all kinds of weird dependencies, e.g. the first blog might set some local variables that the second one relies on. But functions just have arguments and return-values.
    - mattmanser 14y ago
      Obviously. I've found in my experience working with other people's code and maintaining large code bases you find that overly nesting functions causes a lot of problems when you're trying to read or debug code. You often also see problems where the essentially dependant functions start to separate in the code as people accidentally add new functions between them. The article I linked is good, John describes it well. It's a nightmare to work with when you get triple or quadruple nesting of tiny functions, like in the code of this program. It's totally unnecessary.
      - eru 14y ago
        I guess it also depends a bit on the language you are working in. Some languages have an easier time dealing with functions. In, say Haskell, having quadruple nesting of tiny functions isn't too much of a problem---and if you are pedantic, is the only way to create a function with four arguments.
nicholassmith 14y ago
I think it's interesting that they did consider C and then decided it was too much of a risk. There's a constant theme of 'why use C++? C is faster and doesn't suck as much!' especially due to Linus' statements on the language, but often C is a technical risk as there's so much more that can go wrong. I'm not saying C++ is a better language than C, but it's definitely different and given the language is structured to be less of a headache I'm surprised it's often shot down over C anyway.
- justin66 14y ago
  It's a tangent, but: has Linus ever given a presentation that involved actually discussing code? Perhaps code projected on a screen, even part of a slideshow? It's odd that he's such an accomplished engineer but every presentation I've seen him give involves saying outrageous things while starry-eyed geeks stare at him adoringly. If it weren't for the fact that he's arguing from a position of (very great) authority, he would persuade very few people of anything.
  - nicholassmith 14y ago
    Well, I've not seen any but that doesn't preclude him actually giving them but I'm sure he has said in interviews he mostly just does email and patch wrangling rather than actual coding these days. He's loud and opinionated and one of the tech folk heroes, his opinion is going to be listened too and often parroted due to it, even if it's just his opinion rather than a technical fact.
  - zxcdw 14y ago
    > It's odd that he's such an accomplished engineer but every presentation I've seen him give involves saying outrageous things while starry-eyed geeks stare at him adoringly. Why is this? Because the man delivers.
  - qznc 14y ago
    Linus does not code much these days, at least not in the kernel. Writing emails and git-merging is most of his job. If Linus is coding, it is probably more on his scuba diving tool than the Linux kernel. https://github.com/torvalds/subsurface https://github.com/torvalds/subsurface
deleted 14y ago
[deleted]
fool 14y ago
Mirror: http://www.isc.org.nyud.net/wordpress/programming-languages-for-bind-10/ http://www.isc.org.nyud.net/wordpress/programming-languages-...
jejones3141 14y ago
Here's what I take from this: "C++ is by no means an easy language to work with, so the idea is that we will avoid its complexity when possible."
- alpatters 14y ago
  Avoiding its complexity does not mean avoiding it. E.g. you could avoid template metaprogramming due to its complexity whilst keeping to simpler areas of the language. C++ is a large, multi-paradigm language. It gives you choices and one is free to abuse those choices. But having more options gives you more power to express. I think with modern C++ style, boost and C++11 its "difficult-to-work-with" reputation is massively overstated. It is possible to write succinct code with good design and get massive performance benefits.
  - deleted 14y ago
    [deleted]
markwong 14y ago
based on the given criteria, wondering why Java wasn't considered.
- buster 14y ago
  Probably because of memory usage and garbage collection and the fact that java needs to be installed on most systems whereas C++ needs no dependencies and python comes preinstalled on the usual linux server (probably not python3 but i assume it will be some years before bind10 sees adoption for such critical infrastructure). Also he mentions specialized data structures and memory management. Atleast i wouldn't sleep well if i know that the heart of the internet was running Java :P
  - danieldk 14y ago
    Probably because of memory usage and garbage collection and the fact that java needs to be installed on most systems whereas C++ needs no dependencies Well, GCC (as in the compiler collection) has had a ahead-of-time compiler for ages: http://gcc.gnu.org/java/ http://gcc.gnu.org/java/ There is also work in the LLVM camp on AOT Java compilation: http://vmkit.llvm.org/ http://vmkit.llvm.org/ Atleast i wouldn't sleep well if i know that the heart of the internet was running Java :P Well lucky you, only many other vital body parts are running on the JVM via Java, JRuby, and increasingly Scala ;).
    - fdr_cs 14y ago
      Those implementations have quite weak garbage collection implementations (boehm conservative, but I'm not sure), which would just kill the performance. Hotspot JVM has a very sofisticated Incremental Generational garbage collector, which does have a _very_ good performance. I'm pretty sure they had their reasons for not using Java (I actually do have mine too), but, garbage collection is not one of those.
      - danieldk 14y ago
        Indeed. I was arguing against the 'you need the JVM as a dependency'. But then Go is all the hype these days, and people also use Go for network applications, which also has a weak GC ;).
btipling 14y ago
> "C lacks good memory management" C lets you manage memory without possibly inefficient or even broken magical black box automated processes or garbage collectors. Maybe what the author meant to say was "C lacks easy memory management."
- raverbashing 14y ago
  Funny you mention "black box automated process" because, what do you think malloc is? There are two ways of allocating memory directly from the kernel (IIRC), brk and mmap. There's some management malloc does, and if it's good or not depends on you application. For example, size, number and behaviour of your allocations. Depending on your situation you may want to do your own memory management.
  - simias 14y ago
    You're kind of reinforcing the OP's point though. C does lack easy memory management, unless you're ready to rely on external GC libraries.
  - qdog 14y ago
    When I worked on embedded systems, we didn't use malloc() at all, just some heap functions. Even after that, there have been times where I worked on projects where there was a single malloc() and that was turned into a memory heap. However, a call to malloc() is pretty straightforward, you expect it to return a pointer to the allotted memory, or not. Using a GC or the boots libraries is not quite as straightforward, and the black box is a lot bigger.
- CJefferson 14y ago
  Consider realloc. Seems like a sensible function, with its ability to extend memory blocks in-place. Except, you can't try to extend a memory block in-place, but not move it if it can't be extended. malloc and friends really are a quite poor way of managing a memory space if you want to do even slightly clever things I find, unfortunatly
  - 14y ago
StefanKarpinski 14y ago
Go seems like it would have been perfect here, but of course, the language wasn't even a glimmer in Robert Griesemer, Rob Pike, and Ken Thompson's collective eye yet.
- eropple 14y ago
  Obviously I can't speak for these guys, but if I were designing it the fact that Go uses a garbage collector is a pretty strong minus. I like GC'd languages for a lot of tasks, but if I care about performance to the extent that they need to with BIND, I'm not going near it.
  - rlpb 14y ago
    Interestingly, they actually wanted a garbage collector: "The language had to address most of the problems with C. Ideally this meant something with good string handling, garbage collection, exceptions, and that was object oriented."
    - eropple 14y ago
      Yeah, I saw that, and my WTF sensor lit up. DNS isn't trivial, but it seems like a sufficiently well-explored problem that memory lifecycles should be pretty well-understood.
belorn 14y ago
C/C++ and python is a nice combination, and from what I hear, even a best practice when dealing with python optimization. I strongly recollect hearing python developers promoting the idea of first analyze performance, and then rewrite the biggest resource hogs to C code and thus get the best from both worlds. I assume that c++ will have the same usefulness here. It would be very interesting to see the code paths being run in python vs C. I suspect that the 17% python code is actually is around 80% of all possible code paths, but that the 75% C code is just like 5-15% of possible code paths. A possible way to check that could be by looking at the test suites and see which one is bigger, python tree or the c++ tree.
rwmj 14y ago
Did they consider C with a pool allocator library like talloc? It's a good fit for servers (cf. Samba using talloc, and Apache which uses another pool allocator).
Scramblejams 14y ago
I'm finding that Cython[1] can be a very convenient way to speed up the critical parts of your Python. Anyone who considers porting parts of their app to C++ should give it a look. [1] http://www.cython.org http://www.cython.org
antirez 14y ago
Quoting from the article, with my replies: > String manipulation in C is a tedious chore. Use a dynamic strings library, like Postfix for instance, and everything else. > C lacks good memory management. So strange that you went for C++ for most of your code that is not immune of problems from this point of view. I could understand that point if you were opting for a language with GC support. With C you can easily get better (that is, safer) than C++ native MM just building a reference counting system on top of your C "objects". This is trivial and it is what Redis, Tcl, C-Python, and many others are doing. With Redis memory leaks or memory management never was a big issue. > Error handling is optional and cumbersome. Exceptions mostly suck, and in system software the only sane way to deal with errors is C-alike IMHO, that is, check the return value / error returned by every function and act accordingly. > Encapsulation and other object-oriented features must be emulated. This is not an objective point since many thing that this features are actually a problem. Weak points IMHO, and C++ and Python with the minority of Python looks like a design error.
- alpatters 14y ago
  > With C you can easily get better (that is, safer) than C++ native MM just building a reference counting system on top of your C "objects". Why build your own when C++ has std::shared_ptr.
  - antirez 14y ago
    I was referring about how to do it with C.
    - thwest 14y ago
      How is a C implementation of std::shared_ptr safer?
    - muks 14y ago
      Hi antirez C doesn't have a destructor that gets called when something goes out of scope. That's taken advantage of in C++ (RAII) to implement various things (like scoped_ptr that helps avoid leaks). Refcounting is also not perfect for every case. But I like and prefer C. ;) Just pointing out one thing different in C++.
wildchild 14y ago
I hope that OpenBSD team will fork it (BIND 9).
- tobiasu 14y ago
  I think the plan is to drop it entirely and replace it with nsd.
pschastain 14y ago
Obj-C anyone? Seems to meet all their requirements - OO, exception-handling, memory management via either ref counting or GC, and it's a C-superset so C-based optimizations would be easy. Maybe the run-time is to large and/or complicated? Although how it could be more complicated than a C++ runtime I don't know. Would Apple being the primary driving force behind it's development be a problem?
dottrap 14y ago
Sad. They didn't do their homework. They basically had the exact same requirements of the commercial video game industry which uses C/C++ and Lua to solve this problem. Lua had already long asserted its dominance by 2006 so it wasn't a secret. And they should have done a lot better than only 17% of the code in Python.
nodata 14y ago
A good overview of BIND 10's architecture: http://jpmens.net/2012/12/21/completely-different-bind-10/ http://jpmens.net/2012/12/21/completely-different-bind-10/ It seems that all apart from the performance critical parts are written in Python 3.1
- pixl97 14y ago
  Wow! Someone at the ISC has finally learned what Unix means. I've not used Bind for years after being burned by it's security issues back in the 90's. The 'one big monolithic' program was a terrible idea. I had used DJB dns for years, even with its shortcomings in areas, the modular design caused a bug or security issue in one area not to kill the entire system. Hopefully with bind 10 we'll be able to use user separation (or at least enough selinux) to keep each program in its own security 'domain'.
  - agwa 14y ago
    BIND 10 is not the Unix way. In BIND 10, the use of separate processes is just an implementation detail. The Unix way is about presenting small programs to the user that can be composed. To the user, BIND 10 is actually more monolithic than BIND 9 because it needlessly includes a DHCP server. The Unix way is also about human-editable text files - SQLite zone files and icky JSON config files are the antithesis of that. I'm not saying the separate-process design of BIND 10 is bad (to the contrary it's good for security), but using multiple processes internally is only Unix-y in a superficial way.
stefantalpalaru 14y ago
This doesn't look good for Python: "Whenever possible, we use Python" "When necessary, we use C++" "As of right now, it ends up that about 75% of our code is C++ and 17% is Python (link) since it turns out that a lot of BIND 10 is performance-critical."
- sigzero 14y ago
  I don't think you are interpreting that correctly. Python is never pushed as "fast". That is exactly how I would use it. Python wherever I could and C++ (or whatever) when performance is crucial.
- potkor 14y ago
  Generally you start with Python and 1) get the code correct (hardest part) and then 2) port the performance critical parts to C/C++. So even they ended up with 0% in Python, it wouldn't "look bad". You got the benefit in part 1. (And as others have said, 17% of LOC in Python might still mean majority of the functionality in Python).
- reidrac 14y ago
  It could mean that you need more C++ to do stuff (or less Python code to do stuff, whatever your like). What I'm saying is that "amount of code" is not a valid magnitude when comparing different languages. In my experience converting some unmaintained Perl tools to Python, the Python version was almost as long as the Perl one, but it included more functionality, comments and error handling (I'm not criticising Perl here but the unmaintained code that I had to convert).
- misnome 14y ago
  Why doesn't it look good for python? As far as I'm aware, Python has never been advertised as a top-performance language, and part of the advantage has always been that you can rewrite performance-critical paths in c. Adding to this, the fact that BIND is something pretty performance-intensive, I don't think this makes it look bad at all.
  - papsosouid 14y ago
deleted 14y ago
[deleted]
EwanToo 14y ago
There's a comment in the post: "As of right now, it ends up that about 75% of our code is C++ and 17% is Python (link) since it turns out that a lot of BIND 10 is performance-critical." Which could easily be taken the wrong way. I believe the right way to think about it is "How much _more_ C++ code would there be, if there wasn't that 17% in Python?".
- bnegreve 14y ago
  Hum, I don't think interpreting this comment as Python is not adequate for performance critical applications is misinterpretation. The author makes it very clear in some other part of the text: > [Python] has all of the features that we were looking for… except performance. Now your interpretation is also valid since they could have written everything in C++ but they didn't.
  - illumen 14y ago
    There is a python DNS server running on pypy that seems to get a pretty big speedup over CPython. Here is the benchmark result: http://speed.pypy.org/timeline/#/?exe=3,6,1,5&base=2+472&ben=twisted_names&env=1&revs=200&equid=off http://speed.pypy.org/timeline/#/?exe=3,6,1,5&base=2+472... 0.002 for pypy VS 0.015 for CPython, which is 6.66 times faster. So, for the last couple of years they have been wrong - since PyPy has shown pretty good performance in a DNS benchmark, so Python can have pretty good performance ;)
    - jerf 14y ago
      As much as I respect and appreciate Pypy, BIND is one of the core foundational pieces of the internet. It needs to be bulletproof and robust. I don't think Pypy itself qualifies as a mainstream enough language for them to build BIND on. This isn't a slight on Pypy, because the core problem is simply that it hasn't been around for long enough and subjected to enough stress to earn entry into the highest of the high tiers of reliable software. That's not a bad thing per se, and it can be fixed, but in the meantime, ISC has to take the situation as it is now, not how it might be in five years.
pjmlp 14y ago
Nice to see more projects moving into more expressive and safer languages.
- army 14y ago
  I'm not sure that I'd call dynamic languages like Python safer, it depends on context. Compared to C/C++ you lose static typechecking but gain a safer memory model.
  - pjmlp 14y ago
    Anything is better than C.
    - eru 14y ago
      You haven't used PHP much? Or MUMPS?
      - pjmlp 14y ago
        PHP is not a compiled language for systems programming. As for MUMPS I know it is something used only in US it seems.
        eru 14y ago
        Oh, I wasn't aware that you only talked about those. MUMPS is also not for systems programming, as far as I know. To nitpick a bit, compiled or not is more a property of the implementation than of the language itself. Of course, some languages are more commonly compiled than others. But, don't Facebook have a PHP compiler?
        pjmlp 14y ago
        > But, don't Facebook have a PHP compiler? Yes, but I don't see PHP as a possible systems programming language, even with a compiled implementation. Uhm, dreams of device drivers written in PHP...
denysonique 14y ago
Why isn't it written in Node.js?
- mahmud 14y ago
  Poe's law?
- eksith 14y ago
  After careful consideration, as the article mentions, ISC decided a combination of C++ and Python as being more appropriate to the goals of the project. That being safety, stability, established familiarity, speed and of course a level of guaranteed future-proof platform availability. Read: "This is one of the cornerstones of the internet that we didn't want to piss away on novelty". That sounded rude, and I'm sorry for that, but I don't know how else to make that point cogent.
  - afandian 14y ago
    I think it was a joke.
- zx2c4 14y ago
  obvious troll is obvious
- army 14y ago
  Because the performance would be so totally awesome that it would melt the servers. And we don't want the internet to melt.

103 points by AndrewDucker 14y ago

For core services like BIND I don't see any good reason to write them in more than one language (actually anything other than C/C++) and introduce cumbersome dependencies because of that. I wouldn't complain if it was some another DIY name server toy created for whatever reason (how much time it would take..., I can do it!, etc.), but it's software that will become widely deployed in upcoming years (almost) without doubt. How much more work these 17% done in Python would take to be (re)written in C++? Would it make the code that much worse in terms of quality and managability? Having coherent, one-language codebase, is a good feature on its own too, often improving above mentioned factors.

mattmanser 14y ago
You can look at the source yourself. I don't understand either, randomly opening parts of both the Python and C++ and it's certainly not complicated and they write the Python in a C style anyway. I'm not a C++ coder and I only play with Python now and then, but can't say the code impressed me much. It's actually pretty hard to skim the code because it's massively over-commented and there are far too many tiny 2-line private functions that are called by exactly one other 2-line private function that is called by exactly one other 2-line function. Or Holographic code as John D. Cook called it[1]. And the copyright notice in every source file is extremely irritating. Far too high noise to code ratio for my tastes, so I got bored before I could really 'see' how the Python had helped. but I can't judge that well as I'm not sure if that's all just a bi-product of using C++ and being an open source project that comes out of committee. [1] http://www.johndcook.com/blog/2012/01/09/holographic-source-code/ http://www.johndcook.com/blog/2012/01/09/holographic-source-...
- eru 14y ago
  In general, abstracting code out into functions can be useful, even when the new functions is only called once. That is because functions are a well understood abstraction. And having code separated into functions makes it easier for the reader to deduce the coupling points: two blocks of code after another can have all kinds of weird dependencies, e.g. the first blog might set some local variables that the second one relies on. But functions just have arguments and return-values.
  - mattmanser 14y ago
    Obviously. I've found in my experience working with other people's code and maintaining large code bases you find that overly nesting functions causes a lot of problems when you're trying to read or debug code. You often also see problems where the essentially dependant functions start to separate in the code as people accidentally add new functions between them. The article I linked is good, John describes it well. It's a nightmare to work with when you get triple or quadruple nesting of tiny functions, like in the code of this program. It's totally unnecessary.
    - eru 14y ago
      I guess it also depends a bit on the language you are working in. Some languages have an easier time dealing with functions. In, say Haskell, having quadruple nesting of tiny functions isn't too much of a problem---and if you are pedantic, is the only way to create a function with four arguments.

nicholassmith 14y ago

I think it's interesting that they did consider C and then decided it was too much of a risk. There's a constant theme of 'why use C++? C is faster and doesn't suck as much!' especially due to Linus' statements on the language, but often C is a technical risk as there's so much more that can go wrong. I'm not saying C++ is a better language than C, but it's definitely different and given the language is structured to be less of a headache I'm surprised it's often shot down over C anyway.

justin66 14y ago
It's a tangent, but: has Linus ever given a presentation that involved actually discussing code? Perhaps code projected on a screen, even part of a slideshow? It's odd that he's such an accomplished engineer but every presentation I've seen him give involves saying outrageous things while starry-eyed geeks stare at him adoringly. If it weren't for the fact that he's arguing from a position of (very great) authority, he would persuade very few people of anything.
- nicholassmith 14y ago
  Well, I've not seen any but that doesn't preclude him actually giving them but I'm sure he has said in interviews he mostly just does email and patch wrangling rather than actual coding these days. He's loud and opinionated and one of the tech folk heroes, his opinion is going to be listened too and often parroted due to it, even if it's just his opinion rather than a technical fact.
- zxcdw 14y ago
  > It's odd that he's such an accomplished engineer but every presentation I've seen him give involves saying outrageous things while starry-eyed geeks stare at him adoringly. Why is this? Because the man delivers.
- qznc 14y ago
  Linus does not code much these days, at least not in the kernel. Writing emails and git-merging is most of his job. If Linus is coding, it is probably more on his scuba diving tool than the Linux kernel. https://github.com/torvalds/subsurface https://github.com/torvalds/subsurface

deleted 14y ago

[deleted]

fool 14y ago

Mirror: http://www.isc.org.nyud.net/wordpress/programming-languages-for-bind-10/ http://www.isc.org.nyud.net/wordpress/programming-languages-...

jejones3141 14y ago

Here's what I take from this: "C++ is by no means an easy language to work with, so the idea is that we will avoid its complexity when possible."

alpatters 14y ago
Avoiding its complexity does not mean avoiding it. E.g. you could avoid template metaprogramming due to its complexity whilst keeping to simpler areas of the language. C++ is a large, multi-paradigm language. It gives you choices and one is free to abuse those choices. But having more options gives you more power to express. I think with modern C++ style, boost and C++11 its "difficult-to-work-with" reputation is massively overstated. It is possible to write succinct code with good design and get massive performance benefits.
- deleted 14y ago
  [deleted]

markwong 14y ago

based on the given criteria, wondering why Java wasn't considered.

buster 14y ago
Probably because of memory usage and garbage collection and the fact that java needs to be installed on most systems whereas C++ needs no dependencies and python comes preinstalled on the usual linux server (probably not python3 but i assume it will be some years before bind10 sees adoption for such critical infrastructure). Also he mentions specialized data structures and memory management. Atleast i wouldn't sleep well if i know that the heart of the internet was running Java :P
- danieldk 14y ago
  Probably because of memory usage and garbage collection and the fact that java needs to be installed on most systems whereas C++ needs no dependencies Well, GCC (as in the compiler collection) has had a ahead-of-time compiler for ages: http://gcc.gnu.org/java/ http://gcc.gnu.org/java/ There is also work in the LLVM camp on AOT Java compilation: http://vmkit.llvm.org/ http://vmkit.llvm.org/ Atleast i wouldn't sleep well if i know that the heart of the internet was running Java :P Well lucky you, only many other vital body parts are running on the JVM via Java, JRuby, and increasingly Scala ;).
  - fdr_cs 14y ago
    Those implementations have quite weak garbage collection implementations (boehm conservative, but I'm not sure), which would just kill the performance. Hotspot JVM has a very sofisticated Incremental Generational garbage collector, which does have a _very_ good performance. I'm pretty sure they had their reasons for not using Java (I actually do have mine too), but, garbage collection is not one of those.
    - danieldk 14y ago
      Indeed. I was arguing against the 'you need the JVM as a dependency'. But then Go is all the hype these days, and people also use Go for network applications, which also has a weak GC ;).

btipling 14y ago

> "C lacks good memory management" C lets you manage memory without possibly inefficient or even broken magical black box automated processes or garbage collectors. Maybe what the author meant to say was "C lacks easy memory management."

raverbashing 14y ago
Funny you mention "black box automated process" because, what do you think malloc is? There are two ways of allocating memory directly from the kernel (IIRC), brk and mmap. There's some management malloc does, and if it's good or not depends on you application. For example, size, number and behaviour of your allocations. Depending on your situation you may want to do your own memory management.
- simias 14y ago
  You're kind of reinforcing the OP's point though. C does lack easy memory management, unless you're ready to rely on external GC libraries.
- qdog 14y ago
  When I worked on embedded systems, we didn't use malloc() at all, just some heap functions. Even after that, there have been times where I worked on projects where there was a single malloc() and that was turned into a memory heap. However, a call to malloc() is pretty straightforward, you expect it to return a pointer to the allotted memory, or not. Using a GC or the boots libraries is not quite as straightforward, and the black box is a lot bigger.
CJefferson 14y ago
Consider realloc. Seems like a sensible function, with its ability to extend memory blocks in-place. Except, you can't try to extend a memory block in-place, but not move it if it can't be extended. malloc and friends really are a quite poor way of managing a memory space if you want to do even slightly clever things I find, unfortunatly
- 14y ago

StefanKarpinski 14y ago

Go seems like it would have been perfect here, but of course, the language wasn't even a glimmer in Robert Griesemer, Rob Pike, and Ken Thompson's collective eye yet.

eropple 14y ago
Obviously I can't speak for these guys, but if I were designing it the fact that Go uses a garbage collector is a pretty strong minus. I like GC'd languages for a lot of tasks, but if I care about performance to the extent that they need to with BIND, I'm not going near it.
- rlpb 14y ago
  Interestingly, they actually wanted a garbage collector: "The language had to address most of the problems with C. Ideally this meant something with good string handling, garbage collection, exceptions, and that was object oriented."
  - eropple 14y ago
    Yeah, I saw that, and my WTF sensor lit up. DNS isn't trivial, but it seems like a sufficiently well-explored problem that memory lifecycles should be pretty well-understood.

belorn 14y ago

C/C++ and python is a nice combination, and from what I hear, even a best practice when dealing with python optimization. I strongly recollect hearing python developers promoting the idea of first analyze performance, and then rewrite the biggest resource hogs to C code and thus get the best from both worlds. I assume that c++ will have the same usefulness here. It would be very interesting to see the code paths being run in python vs C. I suspect that the 17% python code is actually is around 80% of all possible code paths, but that the 75% C code is just like 5-15% of possible code paths. A possible way to check that could be by looking at the test suites and see which one is bigger, python tree or the c++ tree.

rwmj 14y ago

Did they consider C with a pool allocator library like talloc? It's a good fit for servers (cf. Samba using talloc, and Apache which uses another pool allocator).

Scramblejams 14y ago

I'm finding that Cython[1] can be a very convenient way to speed up the critical parts of your Python. Anyone who considers porting parts of their app to C++ should give it a look. [1] http://www.cython.org http://www.cython.org

antirez 14y ago

Quoting from the article, with my replies: > String manipulation in C is a tedious chore. Use a dynamic strings library, like Postfix for instance, and everything else. > C lacks good memory management. So strange that you went for C++ for most of your code that is not immune of problems from this point of view. I could understand that point if you were opting for a language with GC support. With C you can easily get better (that is, safer) than C++ native MM just building a reference counting system on top of your C "objects". This is trivial and it is what Redis, Tcl, C-Python, and many others are doing. With Redis memory leaks or memory management never was a big issue. > Error handling is optional and cumbersome. Exceptions mostly suck, and in system software the only sane way to deal with errors is C-alike IMHO, that is, check the return value / error returned by every function and act accordingly. > Encapsulation and other object-oriented features must be emulated. This is not an objective point since many thing that this features are actually a problem. Weak points IMHO, and C++ and Python with the minority of Python looks like a design error.

alpatters 14y ago
> With C you can easily get better (that is, safer) than C++ native MM just building a reference counting system on top of your C "objects". Why build your own when C++ has std::shared_ptr.
- antirez 14y ago
  I was referring about how to do it with C.
  - thwest 14y ago
    How is a C implementation of std::shared_ptr safer?
  - muks 14y ago
    Hi antirez C doesn't have a destructor that gets called when something goes out of scope. That's taken advantage of in C++ (RAII) to implement various things (like scoped_ptr that helps avoid leaks). Refcounting is also not perfect for every case. But I like and prefer C. ;) Just pointing out one thing different in C++.

wildchild 14y ago

I hope that OpenBSD team will fork it (BIND 9).

tobiasu 14y ago
I think the plan is to drop it entirely and replace it with nsd.

pschastain 14y ago

Obj-C anyone? Seems to meet all their requirements - OO, exception-handling, memory management via either ref counting or GC, and it's a C-superset so C-based optimizations would be easy. Maybe the run-time is to large and/or complicated? Although how it could be more complicated than a C++ runtime I don't know. Would Apple being the primary driving force behind it's development be a problem?

dottrap 14y ago

Sad. They didn't do their homework. They basically had the exact same requirements of the commercial video game industry which uses C/C++ and Lua to solve this problem. Lua had already long asserted its dominance by 2006 so it wasn't a secret. And they should have done a lot better than only 17% of the code in Python.

nodata 14y ago

A good overview of BIND 10's architecture: http://jpmens.net/2012/12/21/completely-different-bind-10/ http://jpmens.net/2012/12/21/completely-different-bind-10/ It seems that all apart from the performance critical parts are written in Python 3.1

pixl97 14y ago
Wow! Someone at the ISC has finally learned what Unix means. I've not used Bind for years after being burned by it's security issues back in the 90's. The 'one big monolithic' program was a terrible idea. I had used DJB dns for years, even with its shortcomings in areas, the modular design caused a bug or security issue in one area not to kill the entire system. Hopefully with bind 10 we'll be able to use user separation (or at least enough selinux) to keep each program in its own security 'domain'.
- agwa 14y ago
  BIND 10 is not the Unix way. In BIND 10, the use of separate processes is just an implementation detail. The Unix way is about presenting small programs to the user that can be composed. To the user, BIND 10 is actually more monolithic than BIND 9 because it needlessly includes a DHCP server. The Unix way is also about human-editable text files - SQLite zone files and icky JSON config files are the antithesis of that. I'm not saying the separate-process design of BIND 10 is bad (to the contrary it's good for security), but using multiple processes internally is only Unix-y in a superficial way.

stefantalpalaru 14y ago

This doesn't look good for Python: "Whenever possible, we use Python" "When necessary, we use C++" "As of right now, it ends up that about 75% of our code is C++ and 17% is Python (link) since it turns out that a lot of BIND 10 is performance-critical."

sigzero 14y ago
I don't think you are interpreting that correctly. Python is never pushed as "fast". That is exactly how I would use it. Python wherever I could and C++ (or whatever) when performance is crucial.
potkor 14y ago
Generally you start with Python and 1) get the code correct (hardest part) and then 2) port the performance critical parts to C/C++. So even they ended up with 0% in Python, it wouldn't "look bad". You got the benefit in part 1. (And as others have said, 17% of LOC in Python might still mean majority of the functionality in Python).
reidrac 14y ago
It could mean that you need more C++ to do stuff (or less Python code to do stuff, whatever your like). What I'm saying is that "amount of code" is not a valid magnitude when comparing different languages. In my experience converting some unmaintained Perl tools to Python, the Python version was almost as long as the Perl one, but it included more functionality, comments and error handling (I'm not criticising Perl here but the unmaintained code that I had to convert).
misnome 14y ago
Why doesn't it look good for python? As far as I'm aware, Python has never been advertised as a top-performance language, and part of the advantage has always been that you can rewrite performance-critical paths in c. Adding to this, the fact that BIND is something pretty performance-intensive, I don't think this makes it look bad at all.
- papsosouid 14y ago

deleted 14y ago

[deleted]

EwanToo 14y ago

There's a comment in the post: "As of right now, it ends up that about 75% of our code is C++ and 17% is Python (link) since it turns out that a lot of BIND 10 is performance-critical." Which could easily be taken the wrong way. I believe the right way to think about it is "How much _more_ C++ code would there be, if there wasn't that 17% in Python?".

bnegreve 14y ago
Hum, I don't think interpreting this comment as Python is not adequate for performance critical applications is misinterpretation. The author makes it very clear in some other part of the text: > [Python] has all of the features that we were looking for… except performance. Now your interpretation is also valid since they could have written everything in C++ but they didn't.
- illumen 14y ago
  There is a python DNS server running on pypy that seems to get a pretty big speedup over CPython. Here is the benchmark result: http://speed.pypy.org/timeline/#/?exe=3,6,1,5&base=2+472&ben=twisted_names&env=1&revs=200&equid=off http://speed.pypy.org/timeline/#/?exe=3,6,1,5&base=2+472... 0.002 for pypy VS 0.015 for CPython, which is 6.66 times faster. So, for the last couple of years they have been wrong - since PyPy has shown pretty good performance in a DNS benchmark, so Python can have pretty good performance ;)
  - jerf 14y ago
    As much as I respect and appreciate Pypy, BIND is one of the core foundational pieces of the internet. It needs to be bulletproof and robust. I don't think Pypy itself qualifies as a mainstream enough language for them to build BIND on. This isn't a slight on Pypy, because the core problem is simply that it hasn't been around for long enough and subjected to enough stress to earn entry into the highest of the high tiers of reliable software. That's not a bad thing per se, and it can be fixed, but in the meantime, ISC has to take the situation as it is now, not how it might be in five years.

pjmlp 14y ago

Nice to see more projects moving into more expressive and safer languages.

army 14y ago
I'm not sure that I'd call dynamic languages like Python safer, it depends on context. Compared to C/C++ you lose static typechecking but gain a safer memory model.
- pjmlp 14y ago
  Anything is better than C.
  - eru 14y ago
    You haven't used PHP much? Or MUMPS?
    - pjmlp 14y ago
      PHP is not a compiled language for systems programming. As for MUMPS I know it is something used only in US it seems.
      - eru 14y ago
        Oh, I wasn't aware that you only talked about those. MUMPS is also not for systems programming, as far as I know. To nitpick a bit, compiled or not is more a property of the implementation than of the language itself. Of course, some languages are more commonly compiled than others. But, don't Facebook have a PHP compiler?
        pjmlp 14y ago
        > But, don't Facebook have a PHP compiler? Yes, but I don't see PHP as a possible systems programming language, even with a compiled implementation. Uhm, dreams of device drivers written in PHP...

denysonique 14y ago

Why isn't it written in Node.js?

mahmud 14y ago
Poe's law?
eksith 14y ago
After careful consideration, as the article mentions, ISC decided a combination of C++ and Python as being more appropriate to the goals of the project. That being safety, stability, established familiarity, speed and of course a level of guaranteed future-proof platform availability. Read: "This is one of the cornerstones of the internet that we didn't want to piss away on novelty". That sounded rude, and I'm sorry for that, but I don't know how else to make that point cogent.
- afandian 14y ago
  I think it was a joke.
zx2c4 14y ago
obvious troll is obvious
army 14y ago
Because the performance would be so totally awesome that it would melt the servers. And we don't want the internet to melt.