19 ms·

A Bit Overcomplicated

131 points by jjgreen 5y ago

codesections 5y ago
> And sure, `snowflake >> 22` would be shorter, clearer, and far more efficient, but it wouldn't let you show off all the cool language features you have at your disposal, and isn't that what's really important? I don't think the over-complicated code was motivated by a desire to show off — it was written by someone who doesn't understand how bit shifts work, which is still potentially troubling but very different
- kasey_junk 5y ago
  Or that simply has never had reason to use them so didn’t think to look for them.
  - PaulHoule 5y ago
    I never found C style operators for bit twiddling to be intuitive (compared to assembly) even if you use the left angle brace to shift left.
    - the_only_law 5y ago
      While this example is a pretty simple and I agree that a bit shift is a fine solution, I’ve recently been doing work on some bit-oriented HDLC-like protocols I’m convinced Erlang is the only language that can manipulate binary data very well at the bit level.
      - codesections 5y ago
        Out of curiosity, what makes the Erlang syntax so good?
        the_only_law 5y ago
        It's super easy to construct binaries and bitstrings even when you need to work with award sizes or set bits at weird positions and IMO looks a lot cleaner than a lot of shifting and boolean operators. For example constructing a 8-bit LAPB Information transfer control field might look like: build_info_control(SendSeq, RecvSeq, Poll) -> <<0:1, SendSeq/bitstring, Poll/bitstring, RecvSeq/bitstring>>. Where the `SendSeq` and `RecvSeq` are 3-bit sequence numbers... say one of them is `<<2:3>>` (0b010) and Poll is a single bit (you can see the format here: https://imgur.com/a/9bVBM0W https://imgur.com/a/9bVBM0W). This field would then be stuck into a binary representing the entire frame. This could probably be made a littler simpler, though, as I'm still rather new to Erlang. Note you also do get bit shift operators for cases when they’re an appropriate solution (`bsl` and `bsr`)
        voidnullnil 5y ago
        (As someone strongly against adding new features to languages,) I've never understood where there would be any need for anything more complicated than list concatenation when assembling bits. In some language with list notation, (++ being concatenation of two lists) you could do something like this: [0] ++ sendSeq ++ recvSeq ++ [poll] say sendSeq was one bit but we still wanted it to take 3 bits: [0] ++ pad 3 sendSeq ++ poll ++ recvSeq given some "pad" function defined in a library The only reason it's "hard" in C is because the operations are induced from what the machine can do efficiently and without "a sufficiently smart compiler".
        macintux 5y ago
        The bit syntax in Erlang is for deconstruction and assignment (pattern matching), not just assembling. The entire language is built around pattern matching to the point where it simply wouldn't exist otherwise, and I assume binary protocol handling was a core requirement when the language was designed since it was created for telecom switches.
        Jtsummers 5y ago
        https://erlang.org/doc/programming_examples/bit_syntax.html https://erlang.org/doc/programming_examples/bit_syntax.html > say sendSeq was one bit but we still wanted it to take 3 bits: > [0] ++ pad 3 sendSeq ++ poll ++ recvSeq In Erlang you could do this to get something padded (or truncated) to the correct size: << 2:3 >> ;; corresponds to the 3 bits <<010>> So you can do something like this to ensure the precise bit size of each field: << 0:1, SendSeq:3, Poll:3, RecvSeq:3>>
      - dnautics 5y ago
        elixir too, since the underlying technology is the same (arguably it's even easier to read in elixir than erlang). This was super helpful when I was writing a dhcp server.
      - kiadimoondi 5y ago
        Agreed. I was writing a port of the redis protocol to erlang for a personal project that's a server using said protocol as an interface for distributed MPSC "locking", and it was incredibly simple to implement because of erlang's binary strings and pattern matching. Same with base64/hex/etc. manipulations of strings into binary data. I've read a fair bit about how erlang is great for protocols, but hadn't experienced it myself (professionally or personally) until I decided to implement this project.
    - mdip 5y ago
      I learned these operators in a college course I took while in High School[0]. I remember I had to make flash cards with examples and memorize throughout the semester. I kept mixing up & and | in comparison/setting. It's still rare that I use anything outside of those two operators in work code, and when I do it tends to get commented as a premature optimization during code reviews. [0] That's not a humble-brag, context provided b/c it's a time, due to 7 hours a day spent taking tests/memorizing and spent also learning algebra through calculus, I was at my absolute peek for learning this sort of thing.
    - marcosdumay 5y ago
      The existence of a single pair of operators, instead of one for signed and another for unsigned shifts (with good names) make the operators impossible to remember to me. Ok, the article just made me remember they are not signed. But it's certain that the next time I need them, I won't remember it again.
    - CRConrad 5y ago
      > I never found C style operators for bit twiddling to be intuitive (compared to assembly) even if you use the left angle brace to shift left. What I don't get: Are those operators present in C++ too -- wouldn't they clash with the "streaming" thingamajigs, isn't that what "<<" means nowadays?
  - cm2187 5y ago
    Or perhaps for a rarely called function, and which was quicker to write via an inefficient but familiar method rather than spend any time figuring out how to do it in an optimal and correct way.
- growt 5y ago
  My guess is this was written as a joke. It's rust, so the person writing this has to have some programming knowledge.
  - codesections 5y ago
    "some programming knowledge" ≠ "knowledge of bit operators" Imo, Rust makes a very good second language for someone who first learned Javascript or Ruby — and someone who used those languages primarily for web dev may never have needed to learn about bit shifts
    - afandian 5y ago
      I find it hard to believe that someone simultaneously doesn't know the first thing about bit operations, but has come to the conclusion that they need to find the first 42 bits. Also, JavaScript's handling of numbers is so weird that it's one of those things that comes up just by virtue of its weirdness. For that reason I'd expect a JS programmer at least have come across but operations. (edit - I take the original post as a joke too, and that the humour is derived from the juxtaposition)
      - mdip 5y ago
        Both very good points -- I hadn't originally assumed it was a joke, mostly because I don't write `rust` code and I don't know how unlikely it would be to come to that solution. I know I've seen examples at least as odd as this in the past in languages I'm familiar with -- where, say, the developer was really good with LINQ or something and made five calls instead of just accessing an array by index -- that were meant "intentionally", but were bizarre to someone not sitting in their brain at that moment. I hadn't, however, thought about either of the points you mentioned. You're quite the detective. :)
      - voidnullnil 5y ago
        Most JS programmers are utterly unaware of how their "numbers" work.
  - 5y ago
- mdip 5y ago
  > which is still potentially troubling but very different I'm having a really difficult time remembering the last time I had to even use a bit operator at work[0]. For a lot of developers, the need to keep that operator in their head is going to rank way lower than, say, how to properly configure CORS in `ngninx`. You always have to judge a developer's gaps/knowledge given their context. We're dealing in technologies that have sub-sub-sub-sub-disciplines that a person may have expertise in which result in them missing huge portions of the languages' features. I've learned that this hyper-specialized developer can be as effective as anyone -- the ability to rapidly acquire, integrate and use new knowledge with whatever tooling is required is far more important to me than whether or not a developer has large gaps in the tools we use[1]. I'm nit-picking a little, but it's a habit of mine to turn around discussions at my office that go something like "How can any web developer not know (x)?"[2] Ask a developer on a conference call if they're familiar with some obscure technology, and you'll hear ranges of qualified "Yes" responses and a whole bunch of clacking while each Google's various bits of information. How much easier is it if everyone just says "No, I don't" and asks the person who brought it up to explain. It's really hard to get developers past "Imposter Syndrom" and open to admitting their gaps so they can be accounted for in planning. Even where I'm at, now, I see it from time to time and this is at a shop with excellent culture and a focus on projects/products that are new inventions/creations. And, unfortunately, it causes me to judge that individual unfairly, myself -- I've been writing code since I was 13 years old. I have written a lot of code. My goal is that the best code I wrote 6 months ago should at least slightly embarrass me, today.[3] And I guarantee, over the years, I've written code that is worthy of posting on a site like this. I can't imagine what kind of insanity my first "useful" rust program will end up looking like, but given the memory-safety rules imposed, I'm expecting to have to integrate a new mental model that will result in some comical implementations. For me -- this work has a way of either humbling you or down-right beating you up with how often you're reminded about what you don't know. I think it gets worse as you learn more almost feeding the whole "ageism" in the industry -- middle-aged developers are far more honest about what they don't know and some are just beaten to death by the treadmill. I have watched, over the years, as developer after developer has left for "management" or leaving a shop like the one I'm at for a global multi-national to work on the most boring internal enterprise-y waste -- consistent/easy, but pointless. For the few that I was close enough to that I got more than just "PR answers", the crux of it was burn out from never feeling like you're actually "knowing" less about what you're working on[4] [0] Outside of `enum`s in my language of choice, which perform best when flags are checked/appended using operators. But even that is an unnecessary optimization nearly always, there's a ".HasFlag()" method that I simply forget about because it was taught to me in my teens, it's implemented consistently in a lot of languages and it's muscle memory at times. [1] I've recommended the hire of individuals who, outside of "they have been writing C# code for a decade", they hadn't touched the part of the framework the job was for -- i.e. they were Desktop devs applying to Web, Web applying to mobile, and many candidates lack even basic Docker/shell knowledge, etc. All ended up picking up what they were missing on-the-job just like the rest of us. [2] Because, unlike every other field, when "(x)" became "old", it had been on its 5th version, 4th major API change and was born about three and a half minutes ago. [3] Sometimes I'm unfair -- the reason I'm embarrassed is because I didn't use a very obvious feature, but at the time I'm looking at the code I'm also forgetting that said feature didn't exist back then. But often it's just because I've continued to read/study/do. [4] These were not developers who were suffering from mental decline with age and were among the better I've worked with. I really think it's a case of "learning about something new" simultaneously adding to "discovering that your skillset does not include this massive body of knowledge around your small usage of 'something new'". You learn way more about how much you don't know than you add to your total knowledge making you feel 'net dumber'.
- voidnullnil 5y ago
  The author could have been unaware that it's possible to program computers without knowledge of binary arithmetic and bit level operations.
chrisseaton 5y ago
I'm hoping that the Rust compiler is good enough to compile this back to a bit-shift instruction?
- jjgreen 5y ago
  That would be impressive, could someone check?
  - SekstiNi 5y ago
    Unfortunately, but expectedly, it cannot reason about it [1]. Even if it could I'm not sure it would be allowed to elide the allocation(s). [1] https://godbolt.org/z/63xEcfaEE https://godbolt.org/z/63xEcfaEE
    - jjgreen 5y ago
      Pity, if it ever does then I'll put aside my distaste for the whole "curl <url> | sh" thing and learn the language.
    - h2odragon 5y ago
      The metaphysics of that are going to be interesting. "Do what I mean" programming meets all the stories about why one shouldn't accept bargains with fairies or djinn. "I can't let you use `strcpy()` there, Dave" is the least of it.
  - coldtea 5y ago
    No need to check, that would be almost impossible, unless they could read the programmer's mind for the intent...
    - AnIdiotOnTheNet 5y ago
      Which I think is the argument behind declarative languages. If we can encode the intent precisely, then at least in theory the compiler can find the optimal solution for the given architecture and other constraints. Of course in practice that probably requires strong AI to accomplish.
jarym 5y ago
There's how things can be done and how things ought to be done. This if nothing else demonstrates a learning path. And yes, I am shocked that the developer didn't know about bit shift operations... but hey, modern development is just different I suppose.
sudhirj 5y ago
At the risk of sounding stupid, I have no idea how bit-shifting and bitwise operators work, though I've worked in the field for over a decade and am now a Staff Engineer or whatever. There, I said it. It's just something I haven't gotten around to reading yet. I suspect some code would be made clearer by it, but even when doing something similar I wound up using the math textbook was to do it even though I suspect the bit operators would be much faster. https://github.com/sudhirj/shortuuid.rb/blob/master/lib/shortuuid.rb#L14 https://github.com/sudhirj/shortuuid.rb/blob/master/lib/shor... https://github.com/sudhirj/shortuuid.go/blob/master/shortuuid.go#L14 https://github.com/sudhirj/shortuuid.go/blob/master/shortuui...
- tomerv 5y ago
  That's like a carpenter who has never learned how to use a circular saw. Sure, you can work your way around it with all the other tools you have, but why would you? Just put 1 hour into studying it, and have a new tool under your belt.
  - sudhirj 5y ago
    Yeah, but I've filed along with "learn every regex operator". Might be interesting, but not sure where it would be useful these days.
    - codesections 5y ago
      Off topic, but I used to have "learn every regex operator" in that category but, after I actually did, I've been amazed at how often regex operators come in handy.
  - JohnBooty 5y ago
    Right, but we could also say this about thousands of other things in the world of software engineering. There is not enough time to learn everything, or even 1% of it. It's often best to have "wide, shallow" knowledge that is selectively deep in some places. I don't have much experience with time-series databases, but I know what they are and I'm familiar with their use cases. So if the need ever arises I'll be able to reach for the correct tool and invest my learning time wisely.
wyldfire 5y ago
If you've only ever worked with weak typed languages you might be tempted to think that 'everything is/should be/can be a character string.' it's likely that the author doesn't know about bitwise operators. This is similar to not knowing about modulus division (like in the classic 'fizzbuzz' test). Surprisingly, many developers just don't know about these things or if they do, they don't recall these features when designing a solution.
deleted 5y ago
[deleted]
BulgarianIdiot 5y ago
Everyone goes through this phase where we laugh at non-optimal solution to a problem, and proudly cut a line or ten, or hundred from someone's solution. But after 20 years in the industry, I find this trivial and expected even in code by people who are quite smart. Sometimes a solution evolves in a way where the initial variant wasn't subject to simplification, while the current version is. Sometimes there's an edge case that's non-obvious and solved by a complicated solution (comments help here). Sometimes the coder had their mind elsewhere so they approached the problem in a seemingly odd manner. But none of this matters in the big picture. What matters is all this code is wrapper in a function that goes like "getFirst42Bits(int)" and it returns that. And so the only thing that matters is if this function has the right input, output for the bigger problem to solve. The implementation is a detail. Of course, I'm excluding vulnerabilities and vast performance problems in hotspots, this matters. But a specific instance of inefficient solution doesn't matter. A pattern of inefficient solutions may be ground for being demoted or laid off. But any instance is irrelevant. WE ALL DO THIS from time to time, but we don't realize it. Heck, the primary reason why I hate most "web frameworks" is that they do in 2000 lines of boilerplate what I can do with my own APIs in 100.
- mdip 5y ago
  > Sometimes a solution evolves in a way Yeah, forgot all about that detail -- reminded me of one the other day. I had finished up a really painful feature add, involving altering code in like 7 different classes due to a data model change only to find out that four of the seven classes were unused, and the remaining provided small bits of unrelated data needed for previous versions. This feature had undergone six changes of the back-end that was responsible for providing the service it needed. Each time a new pipeline was added, the old pipeline was removed from the container but the code was left behind and I'd just added a 7th.
- codesections 5y ago
  > But none of this matters in the big picture. What matters is all this code is wrapper in a function that goes like "getFirst42Bits(int)" I disagree. Imo, `getFirst42Bits(snowflake)` is worse than `snowflake >> 22` — not only is it longer, but it also hides away the implementation in a way that could obscure bugs/require `jumpToDefinition` debugging. Abstraction is often useful, but this seems like a textbook case of over-abstraction.
kohlerm 5y ago
I am hoping this was meant to be a joke. If not fire him ;-)
stncls 5y ago
Slightly disturbed by the article's use of the term "first bits" for describing the most significant ones. [Sure they are the leftmost ones when writing down the number, but all CPUs are LSB-first now... Plus, I'm a loyalist to the Emperor of Lilliput :-).]
- mormegil 5y ago
  Dunno, RFC 791 states in its Appendix B: "Whenever an octet represents a numeric quantity the left most bit in the diagram is the high order or most significant bit. That is, the bit labeled 0 is the most significant bit." which I read as "left bits", numbered from 0 (i. e. the "first bits") are those most significant.
  - jtsiskin 5y ago
    Yes but that’s network byte order. It’s pretty natural to consider (var & 0b1) the “first bit”. (especially if coming from certain HW) Really “first bit” can mean whatever you want so the phrase just shouldn’t be used
    - mormegil 5y ago
      No, I am pointing out the _bit_ order inside a byte, not _byte_ order. (The mentioned Appendix describes both, I am quoting the intra-byte part.)
PaulDavisThe1st 5y ago
Having written an int62_t class in C++ last year, I now feel I should return to it and verify that the bitshift version I wrote is faster than this. I mean, it's written in Rust, which apparently is the new hotness, so I should really make sure, right? I agree with the comments that point out that not knowing about shift operators isn't really that surprising, but believing you have a reason to want 42 bits of a 64 bit value without knowing about shift operators is.
- CRConrad 5y ago
  > I agree with the comments that point out that not knowing about shift operators isn't really that surprising, but believing you have a reason to want 42 bits of a 64 bit value without knowing about shift operators is. Come to think of it, the article doesn't say why they wanted to do that in the first place. At least one comment mentions having stuff (booleans) encoded in bits and extracting those that go together, and of course that could be it, but we don't know. My own first thought, on seeing the variable name "snowflake", was something along the lines of "What, does the Snowflake database store integers as 42-bit values or something?"
TacticalCoder 5y ago
We cannot both lament that software has become so incredibly bloated and high-latency/unresponsive and yet applaud like there's no tomorrow that it's possible to develop for decades without knowing what bit-shifting is nor how to do it...
mmarq 5y ago
The snippet looks made up. It requires a programmer skilled enough to understand iterators and compose them in a rather complex operation, but not skilled enough to know about shifts or look for “bitwise operators rust” on Google.
- Zababa 5y ago
  That wouldn't be surprising, as you encounter iterators everywhere in Rust, but don't encounter bitwise operation unless you're working in a specific domain. What is surprising is to suddenly need the 42 first bits of a number.
deleted 5y ago
[deleted]
tijsvd 5y ago
Does Rust's `:b` formatter always print leading zeroes? Otherwise the code will not only be inefficient, but wrong for half the input space. Or perhaps that was the original goal, to find the most significant one and the 41 bits that trail it...
Apocryphon 5y ago
I've written shell scripts that had this level of convolution in string manipulation.
blunte 5y ago
Obviously the author of the code didn’t know or understand the bit shift operator and came up with a less optimal solution. I cannot believe that was done just to be cool.
- CRConrad 5y ago
  > I cannot believe that was done just to be cool. No, that was just the DWTF being sarcastic.
deleted 5y ago
[deleted]
HelloNurse 5y ago
Regarding practical applications, shift operators are only useful to slice and align masks and bitfields: something meaningful, simple and basic, but usually done only in specific domains (e.g. handling a certain style of compact file formats) that it is perfectly normal to avoid for a whole lifetime. On the other hand, in theory bit shift operators are more important: if you don't know what they do and why you would want to do it, and if you don't know that they are fast and cheap to implement in a microprocessor and why, you cannot possibly have a good grasp of how computers work, and you shouldn't be allowed to use any remotely "low-level" programming language.
MarkSweep 5y ago
In the past I would use “Five Essential Phone-Screen Questions”. The last question is on bit twiddling. A sample question might be “count how many bits in this byte are 1”. I don’t know if it is fair to assume that all developers know this stuff. When interviewing engineers who claimed experience with anything embedded (MCU programming or field busses like CAN or modbus), I don’t know how it would be possible to program anything at the lowest layers without but shifting. But even in an embedded world you spend most of your time writing application logic. Outside of the embedded world you can get away with repenting things without worrying about bits. Particularly in higher level languages it might be more natural to have a strict with several named bools or an array of bools. One language that makes working with bitfields much more pleasant is C#. You can apply a Flags attribute to an enum type. At runtime it represented by an int, so it’s fast. But it also has handy methods for converting to and from a string form that gives names to all the bits that are set. So it’s fairly friendly for logging and debugging. https://sites.google.com/site/steveyegge2/five-essential-phone-screen-questions https://sites.google.com/site/steveyegge2/five-essential-pho...
- dragontamer 5y ago
  Here's an issue I have with that question. > A sample question might be “count how many bits in this byte are 1” Use the Popcnt assembly instruction. > But what if the popcnt assembly instruction doesn't exist on your platform? Popcnt exists on POWER9, ARM, x86, NVidia PTX (GPU Assembly), AMD RDNA, AMD CDNA, AMD GCN. > But no, really this interview is about seeing if you're able to answer the question in a manner that's familiar to the interviewer... and not really how you'd solve the problem in practice. Fine fine fine. I'll open up my "Art of Computer Programming Volume 4A, Bitwise Tricks" and copy/paste the algorithm specified therein. I don't keep the particulars of how to implement popcount in my brain anymore because its implemented in most CPUs / GPUs these days, AND I can look it up if you happen to be on an uncommon platform in pretty standard bitwise-tricks based programming books. -------- But really, the interviewer would want to know "You do the weird "(x & 0x5555) + ((x & 0xAAAA) >> 1) (etc. etc.)" routine. But really, the practical answer (aka: use a single-cycle popcnt instruction available on all mainstream desktop-class CPUs and GPUs I can possibly imagine in today's world) is still the "wrong answer". --------- I think a better interview question would be: Demonstrate mastery of horizontal SIMD-programming, __such as__ implementing the popcnt instruction out of bitshifts, adds, and "AND" instructions. Or, demonstrate said parallel programming mastery over any other similar problem (be it lookahead carry addition, or kogge-stone bishop/rook moves in chess, or whatever). That's what you really are trying to figure out when you ask the question. Anyone can solve "popcnt" or look it up with a quickie search (either on the internet, or reading through a book). But the overall methodology generalizes to many problems in the SIMD-programming space, so its an important skill to have. But also, its important for the interviewer to demonstrate having this skill to the interviewee. Otherwise, you lose the interviewee. (Skilled individuals want their skills recognized. If you fail to recognize their skill during the interview, they'll seek someone else)
orblivion 5y ago
I was hoping that the punchline was going to be that the Rust compiler optimizes this down to a bit shift.
ultrablack 5y ago
Use AND with a const 1 42 times and 0s for the rest. Cant do it faster :)
deleted 5y ago
[deleted]
sly010 5y ago
No problem, we just have to improve our compilers until the above actually compiles to a single machine instruction... ;)
deleted 5y ago
[deleted]
phekunde 5y ago
> And sure, snowflake >> 22 would be shorter, clearer, and far more efficient, but it wouldn't let you show off all the cool language features you have at your disposal, and isn't that what's really important? And there goes the life of a software maintainer down the drain!
nyghtly 5y ago
"But I can do it in one line." Congratulations, you are the best coder.
rcurry 5y ago
We need to start a repo for Enterprise Right Shift.
majormajor 5y ago
There's a lot of discussion about "should most devs be expected to know this off the top of their head these days?" that I think misses the point entirely. A much more relevant question is: when you are working on code that eventually needs a "give me the first 42 bits of a 64-bit value" function, do you: (a) do some basic research into binary operators since you're at least aware that binary operations are a thing, or (b) just whip up the first loop implementation that comes to mind? If you're verifying your code anyway - which I sure hope you are when writing a delightful loop like the one in the linked article - it won't take you long to check out what the bitwise operators do.
- mypalmike 5y ago
  * deleted... in hindsight my silly comment doesn't add to the discussion.
  - rednerrus 5y ago
    Isn't this what engineers that build tangible things do?
- smitty1e 5y ago
  Consider a spectrum ranging from "new hire" to "understands Duff's Device[1] at a glance". The code needs to default toward the former, with well-socialized exceptions. [1] https://en.m.wikipedia.org/wiki/Duff's_device https://en.m.wikipedia.org/wiki/Duff's_device
ben7799 5y ago
What's really remarkable about this as everyone argues whether it's good for a "software engineer" to have even the most basic understanding of how a computer works is the magnitude of extra work being done here as the computer sees it versus what the coder sees. The coder sees one version being one line with an operator they don't know and the other version being 10 lines of code that look a little dense but it's all stuff they know. The computer sees one instruction versus thousands or maybe tens of thousands. Now is it silly to check if a multiplication is by 2 and then use shifts if you're writing a high level language and it's a one off/non-critical operation? Yes it is. But if you're decoding/encoding binary data as a lot of people have conjectured then you're probably doing a lot of it and this really matters.
robbrown451 5y ago
This is sort of a "if all you have is a hammer, everything looks like a nail" issue. I have a very smart friend who doesn't code, but is a math wiz and makes a ridiculously high salary at a big tech company. She'll do things with spreadsheets that seem like a ridiculously complicated and inefficient way to do things that I could efficiently do with a couple lines of code. But of course, she can also quickly do things with spreadsheets that would take me forever to code up. She earns that big salary. This is like that. I guess it is unfortunate that this developer never ran into bitwise operators, but honestly, I have hardly used them in decades (I learned to code in C, now I'm mostly JavaScript and Typescript). Obviously the developer has a lot of really useful skills, and in this case was able to solve the problem, if not optimally. I bet that developer is a lot more productive in general, than me from 25 years ago who knew bitwise operators but none of that other crazy stuff.
fungiblecog 5y ago
The problem here is not whether the programmer should know whether binary operators exist. It's that s/he couldn't ask someone or google it before implementing a terrible solution. Imagine all the other terrible things this programmer has done in this codebase
jurschreuder 5y ago
It's funny because we all did this the first time we actually needed <<
JJMcJ 5y ago
I assume "first" here means "highest order"? That's what the solution does. Assuming it's an unsigned shift, of course. If it's signed you will get the most significant bit replicated, and I doubt that's what you want.
yongjik 5y ago
That's... actually not that terrible for an enterprise code? Sure it's verbose and inefficient, but it does one thing, and it's trivial to fix, without rewriting anything else. An actual experienced enterprise developer would have created BitManipulator, BitManipulatorConfig, BitManipulatorFactory, ShiftBitManipulatorImpl, XorBitManipulatorImpl, and BitManipulatorRetryWrapper, and you will be required to specify which operations to use by adding a yaml config file, which includes thread pool options and has five different versions in the git repository and three more in k8s clusters.
wernsey 5y ago
Wow, the thedailywtf.com sure brings back memories. I used to read it daily (no pun intended) until I discovered HN, and I actually picked up a lot of knowledge about best practices and so on by reading the criticisms in the comments.
beardyw 5y ago
Others have mentioned microcontrollers and using shifts for efficiency, but also often because system services are often described and accessed by individual bits. So for instance (in C) xxx |= (1 << n) will set bit n and xxx &= ~(1 << n) will clear it.

131 points by jjgreen 5y ago

> And sure, `snowflake >> 22` would be shorter, clearer, and far more efficient, but it wouldn't let you show off all the cool language features you have at your disposal, and isn't that what's really important? I don't think the over-complicated code was motivated by a desire to show off — it was written by someone who doesn't understand how bit shifts work, which is still potentially troubling but very different

kasey_junk 5y ago
Or that simply has never had reason to use them so didn’t think to look for them.
- PaulHoule 5y ago
  I never found C style operators for bit twiddling to be intuitive (compared to assembly) even if you use the left angle brace to shift left.
  - the_only_law 5y ago
    While this example is a pretty simple and I agree that a bit shift is a fine solution, I’ve recently been doing work on some bit-oriented HDLC-like protocols I’m convinced Erlang is the only language that can manipulate binary data very well at the bit level.
    - codesections 5y ago
      Out of curiosity, what makes the Erlang syntax so good?
      - the_only_law 5y ago
        It's super easy to construct binaries and bitstrings even when you need to work with award sizes or set bits at weird positions and IMO looks a lot cleaner than a lot of shifting and boolean operators. For example constructing a 8-bit LAPB Information transfer control field might look like: build_info_control(SendSeq, RecvSeq, Poll) -> <<0:1, SendSeq/bitstring, Poll/bitstring, RecvSeq/bitstring>>. Where the `SendSeq` and `RecvSeq` are 3-bit sequence numbers... say one of them is `<<2:3>>` (0b010) and Poll is a single bit (you can see the format here: https://imgur.com/a/9bVBM0W https://imgur.com/a/9bVBM0W). This field would then be stuck into a binary representing the entire frame. This could probably be made a littler simpler, though, as I'm still rather new to Erlang. Note you also do get bit shift operators for cases when they’re an appropriate solution (`bsl` and `bsr`)
        voidnullnil 5y ago
        (As someone strongly against adding new features to languages,) I've never understood where there would be any need for anything more complicated than list concatenation when assembling bits. In some language with list notation, (++ being concatenation of two lists) you could do something like this: [0] ++ sendSeq ++ recvSeq ++ [poll] say sendSeq was one bit but we still wanted it to take 3 bits: [0] ++ pad 3 sendSeq ++ poll ++ recvSeq given some "pad" function defined in a library The only reason it's "hard" in C is because the operations are induced from what the machine can do efficiently and without "a sufficiently smart compiler".
        macintux 5y ago
        The bit syntax in Erlang is for deconstruction and assignment (pattern matching), not just assembling. The entire language is built around pattern matching to the point where it simply wouldn't exist otherwise, and I assume binary protocol handling was a core requirement when the language was designed since it was created for telecom switches.
        Jtsummers 5y ago
        https://erlang.org/doc/programming_examples/bit_syntax.html https://erlang.org/doc/programming_examples/bit_syntax.html > say sendSeq was one bit but we still wanted it to take 3 bits: > [0] ++ pad 3 sendSeq ++ poll ++ recvSeq In Erlang you could do this to get something padded (or truncated) to the correct size: << 2:3 >> ;; corresponds to the 3 bits <<010>> So you can do something like this to ensure the precise bit size of each field: << 0:1, SendSeq:3, Poll:3, RecvSeq:3>>
    - dnautics 5y ago
      elixir too, since the underlying technology is the same (arguably it's even easier to read in elixir than erlang). This was super helpful when I was writing a dhcp server.
    - kiadimoondi 5y ago
      Agreed. I was writing a port of the redis protocol to erlang for a personal project that's a server using said protocol as an interface for distributed MPSC "locking", and it was incredibly simple to implement because of erlang's binary strings and pattern matching. Same with base64/hex/etc. manipulations of strings into binary data. I've read a fair bit about how erlang is great for protocols, but hadn't experienced it myself (professionally or personally) until I decided to implement this project.
  - mdip 5y ago
    I learned these operators in a college course I took while in High School[0]. I remember I had to make flash cards with examples and memorize throughout the semester. I kept mixing up & and | in comparison/setting. It's still rare that I use anything outside of those two operators in work code, and when I do it tends to get commented as a premature optimization during code reviews. [0] That's not a humble-brag, context provided b/c it's a time, due to 7 hours a day spent taking tests/memorizing and spent also learning algebra through calculus, I was at my absolute peek for learning this sort of thing.
  - marcosdumay 5y ago
    The existence of a single pair of operators, instead of one for signed and another for unsigned shifts (with good names) make the operators impossible to remember to me. Ok, the article just made me remember they are not signed. But it's certain that the next time I need them, I won't remember it again.
  - CRConrad 5y ago
    > I never found C style operators for bit twiddling to be intuitive (compared to assembly) even if you use the left angle brace to shift left. What I don't get: Are those operators present in C++ too -- wouldn't they clash with the "streaming" thingamajigs, isn't that what "<<" means nowadays?
- cm2187 5y ago
  Or perhaps for a rarely called function, and which was quicker to write via an inefficient but familiar method rather than spend any time figuring out how to do it in an optimal and correct way.
growt 5y ago
My guess is this was written as a joke. It's rust, so the person writing this has to have some programming knowledge.
- codesections 5y ago
  "some programming knowledge" ≠ "knowledge of bit operators" Imo, Rust makes a very good second language for someone who first learned Javascript or Ruby — and someone who used those languages primarily for web dev may never have needed to learn about bit shifts
  - afandian 5y ago
    I find it hard to believe that someone simultaneously doesn't know the first thing about bit operations, but has come to the conclusion that they need to find the first 42 bits. Also, JavaScript's handling of numbers is so weird that it's one of those things that comes up just by virtue of its weirdness. For that reason I'd expect a JS programmer at least have come across but operations. (edit - I take the original post as a joke too, and that the humour is derived from the juxtaposition)
    - mdip 5y ago
      Both very good points -- I hadn't originally assumed it was a joke, mostly because I don't write `rust` code and I don't know how unlikely it would be to come to that solution. I know I've seen examples at least as odd as this in the past in languages I'm familiar with -- where, say, the developer was really good with LINQ or something and made five calls instead of just accessing an array by index -- that were meant "intentionally", but were bizarre to someone not sitting in their brain at that moment. I hadn't, however, thought about either of the points you mentioned. You're quite the detective. :)
    - voidnullnil 5y ago
      Most JS programmers are utterly unaware of how their "numbers" work.
- 5y ago
mdip 5y ago
> which is still potentially troubling but very different I'm having a really difficult time remembering the last time I had to even use a bit operator at work[0]. For a lot of developers, the need to keep that operator in their head is going to rank way lower than, say, how to properly configure CORS in `ngninx`. You always have to judge a developer's gaps/knowledge given their context. We're dealing in technologies that have sub-sub-sub-sub-disciplines that a person may have expertise in which result in them missing huge portions of the languages' features. I've learned that this hyper-specialized developer can be as effective as anyone -- the ability to rapidly acquire, integrate and use new knowledge with whatever tooling is required is far more important to me than whether or not a developer has large gaps in the tools we use[1]. I'm nit-picking a little, but it's a habit of mine to turn around discussions at my office that go something like "How can any web developer not know (x)?"[2] Ask a developer on a conference call if they're familiar with some obscure technology, and you'll hear ranges of qualified "Yes" responses and a whole bunch of clacking while each Google's various bits of information. How much easier is it if everyone just says "No, I don't" and asks the person who brought it up to explain. It's really hard to get developers past "Imposter Syndrom" and open to admitting their gaps so they can be accounted for in planning. Even where I'm at, now, I see it from time to time and this is at a shop with excellent culture and a focus on projects/products that are new inventions/creations. And, unfortunately, it causes me to judge that individual unfairly, myself -- I've been writing code since I was 13 years old. I have written a lot of code. My goal is that the best code I wrote 6 months ago should at least slightly embarrass me, today.[3] And I guarantee, over the years, I've written code that is worthy of posting on a site like this. I can't imagine what kind of insanity my first "useful" rust program will end up looking like, but given the memory-safety rules imposed, I'm expecting to have to integrate a new mental model that will result in some comical implementations. For me -- this work has a way of either humbling you or down-right beating you up with how often you're reminded about what you don't know. I think it gets worse as you learn more almost feeding the whole "ageism" in the industry -- middle-aged developers are far more honest about what they don't know and some are just beaten to death by the treadmill. I have watched, over the years, as developer after developer has left for "management" or leaving a shop like the one I'm at for a global multi-national to work on the most boring internal enterprise-y waste -- consistent/easy, but pointless. For the few that I was close enough to that I got more than just "PR answers", the crux of it was burn out from never feeling like you're actually "knowing" less about what you're working on[4] [0] Outside of `enum`s in my language of choice, which perform best when flags are checked/appended using operators. But even that is an unnecessary optimization nearly always, there's a ".HasFlag()" method that I simply forget about because it was taught to me in my teens, it's implemented consistently in a lot of languages and it's muscle memory at times. [1] I've recommended the hire of individuals who, outside of "they have been writing C# code for a decade", they hadn't touched the part of the framework the job was for -- i.e. they were Desktop devs applying to Web, Web applying to mobile, and many candidates lack even basic Docker/shell knowledge, etc. All ended up picking up what they were missing on-the-job just like the rest of us. [2] Because, unlike every other field, when "(x)" became "old", it had been on its 5th version, 4th major API change and was born about three and a half minutes ago. [3] Sometimes I'm unfair -- the reason I'm embarrassed is because I didn't use a very obvious feature, but at the time I'm looking at the code I'm also forgetting that said feature didn't exist back then. But often it's just because I've continued to read/study/do. [4] These were not developers who were suffering from mental decline with age and were among the better I've worked with. I really think it's a case of "learning about something new" simultaneously adding to "discovering that your skillset does not include this massive body of knowledge around your small usage of 'something new'". You learn way more about how much you don't know than you add to your total knowledge making you feel 'net dumber'.
voidnullnil 5y ago
The author could have been unaware that it's possible to program computers without knowledge of binary arithmetic and bit level operations.

chrisseaton 5y ago

I'm hoping that the Rust compiler is good enough to compile this back to a bit-shift instruction?

jjgreen 5y ago
That would be impressive, could someone check?
- SekstiNi 5y ago
  Unfortunately, but expectedly, it cannot reason about it [1]. Even if it could I'm not sure it would be allowed to elide the allocation(s). [1] https://godbolt.org/z/63xEcfaEE https://godbolt.org/z/63xEcfaEE
  - jjgreen 5y ago
    Pity, if it ever does then I'll put aside my distaste for the whole "curl <url> | sh" thing and learn the language.
  - h2odragon 5y ago
    The metaphysics of that are going to be interesting. "Do what I mean" programming meets all the stories about why one shouldn't accept bargains with fairies or djinn. "I can't let you use `strcpy()` there, Dave" is the least of it.
- coldtea 5y ago
  No need to check, that would be almost impossible, unless they could read the programmer's mind for the intent...
  - AnIdiotOnTheNet 5y ago
    Which I think is the argument behind declarative languages. If we can encode the intent precisely, then at least in theory the compiler can find the optimal solution for the given architecture and other constraints. Of course in practice that probably requires strong AI to accomplish.

jarym 5y ago

There's how things can be done and how things ought to be done. This if nothing else demonstrates a learning path. And yes, I am shocked that the developer didn't know about bit shift operations... but hey, modern development is just different I suppose.

sudhirj 5y ago

At the risk of sounding stupid, I have no idea how bit-shifting and bitwise operators work, though I've worked in the field for over a decade and am now a Staff Engineer or whatever. There, I said it. It's just something I haven't gotten around to reading yet. I suspect some code would be made clearer by it, but even when doing something similar I wound up using the math textbook was to do it even though I suspect the bit operators would be much faster. https://github.com/sudhirj/shortuuid.rb/blob/master/lib/shortuuid.rb#L14 https://github.com/sudhirj/shortuuid.rb/blob/master/lib/shor... https://github.com/sudhirj/shortuuid.go/blob/master/shortuuid.go#L14 https://github.com/sudhirj/shortuuid.go/blob/master/shortuui...

tomerv 5y ago
That's like a carpenter who has never learned how to use a circular saw. Sure, you can work your way around it with all the other tools you have, but why would you? Just put 1 hour into studying it, and have a new tool under your belt.
- sudhirj 5y ago
  Yeah, but I've filed along with "learn every regex operator". Might be interesting, but not sure where it would be useful these days.
  - codesections 5y ago
    Off topic, but I used to have "learn every regex operator" in that category but, after I actually did, I've been amazed at how often regex operators come in handy.
- JohnBooty 5y ago
  Right, but we could also say this about thousands of other things in the world of software engineering. There is not enough time to learn everything, or even 1% of it. It's often best to have "wide, shallow" knowledge that is selectively deep in some places. I don't have much experience with time-series databases, but I know what they are and I'm familiar with their use cases. So if the need ever arises I'll be able to reach for the correct tool and invest my learning time wisely.

wyldfire 5y ago

If you've only ever worked with weak typed languages you might be tempted to think that 'everything is/should be/can be a character string.' it's likely that the author doesn't know about bitwise operators. This is similar to not knowing about modulus division (like in the classic 'fizzbuzz' test). Surprisingly, many developers just don't know about these things or if they do, they don't recall these features when designing a solution.

deleted 5y ago

[deleted]

BulgarianIdiot 5y ago

Everyone goes through this phase where we laugh at non-optimal solution to a problem, and proudly cut a line or ten, or hundred from someone's solution. But after 20 years in the industry, I find this trivial and expected even in code by people who are quite smart. Sometimes a solution evolves in a way where the initial variant wasn't subject to simplification, while the current version is. Sometimes there's an edge case that's non-obvious and solved by a complicated solution (comments help here). Sometimes the coder had their mind elsewhere so they approached the problem in a seemingly odd manner. But none of this matters in the big picture. What matters is all this code is wrapper in a function that goes like "getFirst42Bits(int)" and it returns that. And so the only thing that matters is if this function has the right input, output for the bigger problem to solve. The implementation is a detail. Of course, I'm excluding vulnerabilities and vast performance problems in hotspots, this matters. But a specific instance of inefficient solution doesn't matter. A pattern of inefficient solutions may be ground for being demoted or laid off. But any instance is irrelevant. WE ALL DO THIS from time to time, but we don't realize it. Heck, the primary reason why I hate most "web frameworks" is that they do in 2000 lines of boilerplate what I can do with my own APIs in 100.

mdip 5y ago
> Sometimes a solution evolves in a way Yeah, forgot all about that detail -- reminded me of one the other day. I had finished up a really painful feature add, involving altering code in like 7 different classes due to a data model change only to find out that four of the seven classes were unused, and the remaining provided small bits of unrelated data needed for previous versions. This feature had undergone six changes of the back-end that was responsible for providing the service it needed. Each time a new pipeline was added, the old pipeline was removed from the container but the code was left behind and I'd just added a 7th.
codesections 5y ago
> But none of this matters in the big picture. What matters is all this code is wrapper in a function that goes like "getFirst42Bits(int)" I disagree. Imo, `getFirst42Bits(snowflake)` is worse than `snowflake >> 22` — not only is it longer, but it also hides away the implementation in a way that could obscure bugs/require `jumpToDefinition` debugging. Abstraction is often useful, but this seems like a textbook case of over-abstraction.

kohlerm 5y ago

I am hoping this was meant to be a joke. If not fire him ;-)

stncls 5y ago

Slightly disturbed by the article's use of the term "first bits" for describing the most significant ones. [Sure they are the leftmost ones when writing down the number, but all CPUs are LSB-first now... Plus, I'm a loyalist to the Emperor of Lilliput :-).]

mormegil 5y ago
Dunno, RFC 791 states in its Appendix B: "Whenever an octet represents a numeric quantity the left most bit in the diagram is the high order or most significant bit. That is, the bit labeled 0 is the most significant bit." which I read as "left bits", numbered from 0 (i. e. the "first bits") are those most significant.
- jtsiskin 5y ago
  Yes but that’s network byte order. It’s pretty natural to consider (var & 0b1) the “first bit”. (especially if coming from certain HW) Really “first bit” can mean whatever you want so the phrase just shouldn’t be used
  - mormegil 5y ago
    No, I am pointing out the _bit_ order inside a byte, not _byte_ order. (The mentioned Appendix describes both, I am quoting the intra-byte part.)

PaulDavisThe1st 5y ago

Having written an int62_t class in C++ last year, I now feel I should return to it and verify that the bitshift version I wrote is faster than this. I mean, it's written in Rust, which apparently is the new hotness, so I should really make sure, right? I agree with the comments that point out that not knowing about shift operators isn't really that surprising, but believing you have a reason to want 42 bits of a 64 bit value without knowing about shift operators is.

CRConrad 5y ago
> I agree with the comments that point out that not knowing about shift operators isn't really that surprising, but believing you have a reason to want 42 bits of a 64 bit value without knowing about shift operators is. Come to think of it, the article doesn't say why they wanted to do that in the first place. At least one comment mentions having stuff (booleans) encoded in bits and extracting those that go together, and of course that could be it, but we don't know. My own first thought, on seeing the variable name "snowflake", was something along the lines of "What, does the Snowflake database store integers as 42-bit values or something?"

TacticalCoder 5y ago

We cannot both lament that software has become so incredibly bloated and high-latency/unresponsive and yet applaud like there's no tomorrow that it's possible to develop for decades without knowing what bit-shifting is nor how to do it...

mmarq 5y ago

The snippet looks made up. It requires a programmer skilled enough to understand iterators and compose them in a rather complex operation, but not skilled enough to know about shifts or look for “bitwise operators rust” on Google.

Zababa 5y ago
That wouldn't be surprising, as you encounter iterators everywhere in Rust, but don't encounter bitwise operation unless you're working in a specific domain. What is surprising is to suddenly need the 42 first bits of a number.

deleted 5y ago

[deleted]

tijsvd 5y ago

Does Rust's `:b` formatter always print leading zeroes? Otherwise the code will not only be inefficient, but wrong for half the input space. Or perhaps that was the original goal, to find the most significant one and the 41 bits that trail it...

Apocryphon 5y ago

I've written shell scripts that had this level of convolution in string manipulation.

blunte 5y ago

Obviously the author of the code didn’t know or understand the bit shift operator and came up with a less optimal solution. I cannot believe that was done just to be cool.

CRConrad 5y ago
> I cannot believe that was done just to be cool. No, that was just the DWTF being sarcastic.

deleted 5y ago

[deleted]

HelloNurse 5y ago

Regarding practical applications, shift operators are only useful to slice and align masks and bitfields: something meaningful, simple and basic, but usually done only in specific domains (e.g. handling a certain style of compact file formats) that it is perfectly normal to avoid for a whole lifetime. On the other hand, in theory bit shift operators are more important: if you don't know what they do and why you would want to do it, and if you don't know that they are fast and cheap to implement in a microprocessor and why, you cannot possibly have a good grasp of how computers work, and you shouldn't be allowed to use any remotely "low-level" programming language.

MarkSweep 5y ago

In the past I would use “Five Essential Phone-Screen Questions”. The last question is on bit twiddling. A sample question might be “count how many bits in this byte are 1”. I don’t know if it is fair to assume that all developers know this stuff. When interviewing engineers who claimed experience with anything embedded (MCU programming or field busses like CAN or modbus), I don’t know how it would be possible to program anything at the lowest layers without but shifting. But even in an embedded world you spend most of your time writing application logic. Outside of the embedded world you can get away with repenting things without worrying about bits. Particularly in higher level languages it might be more natural to have a strict with several named bools or an array of bools. One language that makes working with bitfields much more pleasant is C#. You can apply a Flags attribute to an enum type. At runtime it represented by an int, so it’s fast. But it also has handy methods for converting to and from a string form that gives names to all the bits that are set. So it’s fairly friendly for logging and debugging. https://sites.google.com/site/steveyegge2/five-essential-phone-screen-questions https://sites.google.com/site/steveyegge2/five-essential-pho...

dragontamer 5y ago
Here's an issue I have with that question. > A sample question might be “count how many bits in this byte are 1” Use the Popcnt assembly instruction. > But what if the popcnt assembly instruction doesn't exist on your platform? Popcnt exists on POWER9, ARM, x86, NVidia PTX (GPU Assembly), AMD RDNA, AMD CDNA, AMD GCN. > But no, really this interview is about seeing if you're able to answer the question in a manner that's familiar to the interviewer... and not really how you'd solve the problem in practice. Fine fine fine. I'll open up my "Art of Computer Programming Volume 4A, Bitwise Tricks" and copy/paste the algorithm specified therein. I don't keep the particulars of how to implement popcount in my brain anymore because its implemented in most CPUs / GPUs these days, AND I can look it up if you happen to be on an uncommon platform in pretty standard bitwise-tricks based programming books. -------- But really, the interviewer would want to know "You do the weird "(x & 0x5555) + ((x & 0xAAAA) >> 1) (etc. etc.)" routine. But really, the practical answer (aka: use a single-cycle popcnt instruction available on all mainstream desktop-class CPUs and GPUs I can possibly imagine in today's world) is still the "wrong answer". --------- I think a better interview question would be: Demonstrate mastery of horizontal SIMD-programming, __such as__ implementing the popcnt instruction out of bitshifts, adds, and "AND" instructions. Or, demonstrate said parallel programming mastery over any other similar problem (be it lookahead carry addition, or kogge-stone bishop/rook moves in chess, or whatever). That's what you really are trying to figure out when you ask the question. Anyone can solve "popcnt" or look it up with a quickie search (either on the internet, or reading through a book). But the overall methodology generalizes to many problems in the SIMD-programming space, so its an important skill to have. But also, its important for the interviewer to demonstrate having this skill to the interviewee. Otherwise, you lose the interviewee. (Skilled individuals want their skills recognized. If you fail to recognize their skill during the interview, they'll seek someone else)

orblivion 5y ago

I was hoping that the punchline was going to be that the Rust compiler optimizes this down to a bit shift.

ultrablack 5y ago

Use AND with a const 1 42 times and 0s for the rest. Cant do it faster :)

deleted 5y ago

[deleted]

sly010 5y ago

No problem, we just have to improve our compilers until the above actually compiles to a single machine instruction... ;)

deleted 5y ago

[deleted]

phekunde 5y ago

> And sure, snowflake >> 22 would be shorter, clearer, and far more efficient, but it wouldn't let you show off all the cool language features you have at your disposal, and isn't that what's really important? And there goes the life of a software maintainer down the drain!

nyghtly 5y ago

"But I can do it in one line." Congratulations, you are the best coder.

rcurry 5y ago

We need to start a repo for Enterprise Right Shift.

majormajor 5y ago

There's a lot of discussion about "should most devs be expected to know this off the top of their head these days?" that I think misses the point entirely. A much more relevant question is: when you are working on code that eventually needs a "give me the first 42 bits of a 64-bit value" function, do you: (a) do some basic research into binary operators since you're at least aware that binary operations are a thing, or (b) just whip up the first loop implementation that comes to mind? If you're verifying your code anyway - which I sure hope you are when writing a delightful loop like the one in the linked article - it won't take you long to check out what the bitwise operators do.

mypalmike 5y ago
* deleted... in hindsight my silly comment doesn't add to the discussion.
- rednerrus 5y ago
  Isn't this what engineers that build tangible things do?
smitty1e 5y ago
Consider a spectrum ranging from "new hire" to "understands Duff's Device[1] at a glance". The code needs to default toward the former, with well-socialized exceptions. [1] https://en.m.wikipedia.org/wiki/Duff's_device https://en.m.wikipedia.org/wiki/Duff's_device

ben7799 5y ago

What's really remarkable about this as everyone argues whether it's good for a "software engineer" to have even the most basic understanding of how a computer works is the magnitude of extra work being done here as the computer sees it versus what the coder sees. The coder sees one version being one line with an operator they don't know and the other version being 10 lines of code that look a little dense but it's all stuff they know. The computer sees one instruction versus thousands or maybe tens of thousands. Now is it silly to check if a multiplication is by 2 and then use shifts if you're writing a high level language and it's a one off/non-critical operation? Yes it is. But if you're decoding/encoding binary data as a lot of people have conjectured then you're probably doing a lot of it and this really matters.

robbrown451 5y ago

This is sort of a "if all you have is a hammer, everything looks like a nail" issue. I have a very smart friend who doesn't code, but is a math wiz and makes a ridiculously high salary at a big tech company. She'll do things with spreadsheets that seem like a ridiculously complicated and inefficient way to do things that I could efficiently do with a couple lines of code. But of course, she can also quickly do things with spreadsheets that would take me forever to code up. She earns that big salary. This is like that. I guess it is unfortunate that this developer never ran into bitwise operators, but honestly, I have hardly used them in decades (I learned to code in C, now I'm mostly JavaScript and Typescript). Obviously the developer has a lot of really useful skills, and in this case was able to solve the problem, if not optimally. I bet that developer is a lot more productive in general, than me from 25 years ago who knew bitwise operators but none of that other crazy stuff.

fungiblecog 5y ago

The problem here is not whether the programmer should know whether binary operators exist. It's that s/he couldn't ask someone or google it before implementing a terrible solution. Imagine all the other terrible things this programmer has done in this codebase

jurschreuder 5y ago

It's funny because we all did this the first time we actually needed <<

JJMcJ 5y ago

I assume "first" here means "highest order"? That's what the solution does. Assuming it's an unsigned shift, of course. If it's signed you will get the most significant bit replicated, and I doubt that's what you want.

yongjik 5y ago

That's... actually not that terrible for an enterprise code? Sure it's verbose and inefficient, but it does one thing, and it's trivial to fix, without rewriting anything else. An actual experienced enterprise developer would have created BitManipulator, BitManipulatorConfig, BitManipulatorFactory, ShiftBitManipulatorImpl, XorBitManipulatorImpl, and BitManipulatorRetryWrapper, and you will be required to specify which operations to use by adding a yaml config file, which includes thread pool options and has five different versions in the git repository and three more in k8s clusters.

wernsey 5y ago

Wow, the thedailywtf.com sure brings back memories. I used to read it daily (no pun intended) until I discovered HN, and I actually picked up a lot of knowledge about best practices and so on by reading the criticisms in the comments.

beardyw 5y ago

Others have mentioned microcontrollers and using shifts for efficiency, but also often because system services are often described and accessed by individual bits. So for instance (in C) xxx |= (1 << n) will set bit n and xxx &= ~(1 << n) will clear it.