2 ms·

> I do I write fast web servers and operating systems and text editors and language bindings and compilers... I didn't want to invoke any contest, but to remin

by vardump 11y ago

> I do I write fast web servers and operating systems and text editors and language bindings and compilers... I didn't want to invoke any contest, but to remind C is used a lot in those contexts and zero termination seems to be one of the bug magnets. A very common source for security vulnerabilities as well, pretty large portion of them are related to string processing. > Maybe memory buffer errors happen to other programmers often enough that it's worth worrying about for other people Yup. Maybe a large part of how I think of this to protect the other people. It's not rare someone messes up on an embedded system and writes a bit somewhere else in memory. Often it won't crash, but the bugs you get take ages to hunt down. There's just no way around it. I do want to delegate things to other people. Defensive coding tends to catch and prevent a lot of mistakes at that point. Many of them have pretty cowboy attitudes towards details such as parameter validation or error checking. Some of them will need to maintain it in the distant future. > I've observed a great deal of performance can be gained by controlling how my structure is laid out in memory, but in order to do that, the compiler needs to trust me. Yeah, performance is often about cache, locality of reference. Pointer chasing and other random access destroys performance. Applying SIMD without penalties requires at least 16 byte alignment. In 2016, 64 byte align is even better. SIMDs are getting pretty wide. Re 3)... I still think that 300GB string example is esoteric, even off topic. You're not going to put 300GB string in any standard string abstraction. There are probably a lot of other worries, such as running out of disk space, undetected data corruption on disk (happens pretty often), failure in the middle of the modification process, etc. > Sometimes I need to mutate, so I think enforcing immutability is simply not valuable. I don't think anyone wanted to enforce immutability. Just to have it default.

geocar 11y ago
> I didn't want to invoke any contest, but to remind C is used a lot in those contexts and zero termination seems to be one of the bug magnets. A very common source for security vulnerabilities as well, pretty large portion of them are related to string processing. I understand. I think the gross majority of this comes from the C standard library though, and not from the C language. qmail for example had no buffer overflow problems -- the first security vulnerabilities found 10 years after release wouldn't affect any system that qmail was developed to run on simply because nobody gave their mail server 4GB of ram! Surely the author should have predicted this problem, but it demonstrates methodology can protect against this issue. KDB takes an interesting approach of reference counting in all operators. Operators can then be optimised for situations where one argument (or both) have a reference-count of exactly one. This means writing C code with KDB's memory management library can make things very simple. Antirez also has an interesting string library that uses a combination of length+null-termination specifically for the purpose of finding bugs. > There's just no way around it. I do want to delegate things to other people. Defensive coding tends to catch and prevent a lot of mistakes at that point. Many of them have pretty cowboy attitudes towards details such as parameter validation or error checking. Some of them will need to maintain it in the distant future. I understand what you're saying, but I think technique can help a lot more than we think: Most of the software engineering industry wants to move to smarter and better tooling, and I'm just proposing we upgrade our brains some too. I don't have all the answers yet. > You're not going to put 300GB string in any standard string abstraction. In KDB this is actually pretty common, but it doesn't have a "standard string abstraction" because while it meaningfully supports byte-arrays that are in the 300GB range, the string-atoms (symbols) which are used similar to strings in python et al, never approach 1MB let alone 300GB. However in C I just did mmap() and updated the file. It took about 15 minutes to write and debug, and a few minutes to run on the server. > I don't think anyone wanted to enforce immutability. Just to have it default. I don't know. I parsed I don't know any good reason to deviate from these rules. as "enforce". I might be wrong, you'd have to ask "cousin_it" what he/she meant at the time. I know at least later they either were convinced, or clarified[1] this position. However what I meant was that I don't want to enforce those rules because I can think of an exception for each, and they understood that. Sorry I was unclear though. [1]: https://news.ycombinator.com/item?id=10842858 https://news.ycombinator.com/item?id=10842858

by vardump 11y ago

geocar 11y ago

> I didn't want to invoke any contest, but to remind C is used a lot in those contexts and zero termination seems to be one of the bug magnets. A very common source for security vulnerabilities as well, pretty large portion of them are related to string processing. I understand. I think the gross majority of this comes from the C standard library though, and not from the C language. qmail for example had no buffer overflow problems -- the first security vulnerabilities found 10 years after release wouldn't affect any system that qmail was developed to run on simply because nobody gave their mail server 4GB of ram! Surely the author should have predicted this problem, but it demonstrates methodology can protect against this issue. KDB takes an interesting approach of reference counting in all operators. Operators can then be optimised for situations where one argument (or both) have a reference-count of exactly one. This means writing C code with KDB's memory management library can make things very simple. Antirez also has an interesting string library that uses a combination of length+null-termination specifically for the purpose of finding bugs. > There's just no way around it. I do want to delegate things to other people. Defensive coding tends to catch and prevent a lot of mistakes at that point. Many of them have pretty cowboy attitudes towards details such as parameter validation or error checking. Some of them will need to maintain it in the distant future. I understand what you're saying, but I think technique can help a lot more than we think: Most of the software engineering industry wants to move to smarter and better tooling, and I'm just proposing we upgrade our brains some too. I don't have all the answers yet. > You're not going to put 300GB string in any standard string abstraction. In KDB this is actually pretty common, but it doesn't have a "standard string abstraction" because while it meaningfully supports byte-arrays that are in the 300GB range, the string-atoms (symbols) which are used similar to strings in python et al, never approach 1MB let alone 300GB. However in C I just did mmap() and updated the file. It took about 15 minutes to write and debug, and a few minutes to run on the server. > I don't think anyone wanted to enforce immutability. Just to have it default. I don't know. I parsed I don't know any good reason to deviate from these rules. as "enforce". I might be wrong, you'd have to ask "cousin_it" what he/she meant at the time. I know at least later they either were convinced, or clarified[1] this position. However what I meant was that I don't want to enforce those rules because I can think of an exception for each, and they understood that. Sorry I was unclear though. [1]: https://news.ycombinator.com/item?id=10842858 https://news.ycombinator.com/item?id=10842858