14 ms·

At the risk of sounding stupid, I have no idea how bit-shifting and bitwise operators work, though I've worked in the field for over a decade and am now a Staff

• tomerv 5y ago
  That's like a carpenter who has never learned how to use a circular saw. Sure, you can work your way around it with all the other tools you have, but why would you? Just put 1 hour into studying it, and have a new tool under your belt.
  • sudhirj 5y ago
    Yeah, but I've filed along with "learn every regex operator". Might be interesting, but not sure where it would be useful these days.
    • codesections 5y ago
      Off topic, but I used to have "learn every regex operator" in that category but, after I actually did, I've been amazed at how often regex operators come in handy.
  • JohnBooty 5y ago
    Right, but we could also say this about thousands of other things in the world of software engineering. There is not enough time to learn everything, or even 1% of it. It's often best to have "wide, shallow" knowledge that is selectively deep in some places. I don't have much experience with time-series databases, but I know what they are and I'm familiar with their use cases. So if the need ever arises I'll be able to reach for the correct tool and invest my learning time wisely.
    • magicalhippo 5y ago
      Then again, should take about 5 minutes tops to get bitshifting enough that you know where to look if you ever need it.
      • BenjiWiebe 5y ago
        Personally I'd disagree with the only 5 minutes. I'm comfortable with bit shifting, and I've done some ATmega projects without Arduino (just plain C), some of which were bitbanging. It took me several hours over the course of a couple days for the light to dawn. For some reason it just didn't want to click. And it still took practice to get comfortable with it.
        • magicalhippo 5y ago
          My point was that you don't need to be comfortable with it unless you need it. Learn enough to know what it does, so you know when you see a use-case and can come back and do the hard work.
  • h2odragon 5y ago
    > never learned how to use a circular saw. More like a hand plane, or perhaps a chainsaw. Sure they're tools that work on wood but you can do lots of woodwork without ever needing one. Once upon a time i diddled Grey codes and similar and the elegance of bit banging is lovely and foundational... but someone who makes ships in bottles need never know how to cut down a tree. To mash a metaphor.
• BulgarianIdiot 5y ago
  Bitwise shifting isn't that important, unless you happen to work with binary formats and protocols, which should be distinct low-level layer in any app, and ideally relegated to a library that does it better. Back in time it was known for being a fast way to multiply and divide by powers of 2. This hasn't been true for a long time now, and in the rare cases it is, most optimizing compilers will bitwise shift where in your source code you divide or multiply.
  • spaetzleesser 5y ago
    I think it’s important though to know that these operators exist.
    • majewsky 5y ago
      Which OP obviously does.
    • AnIdiotOnTheNet 5y ago
      More than that, I think it is important to have a certain level of understanding about how computers actually work if you're going to be programming them as a career, and boolean logic and bitwise operators are a very basic part of that.
      • BulgarianIdiot 5y ago
        How computers work is a very stretchy term. And what you need to know depends what you're working on. For example if you're scripting you don't need to know a thing about how computers work, your concerns are mostly at the software level. If you're writing a high-performance server or a game engine, you need to understand the topology of computers. Network performance and protocols, disk, memory and cache architecture. But you still don't need to understand every machine code on the CPU. We're risking falling into the trap of "more is more" like a chemistry teacher that is adamant that you need to absolutely understand covalent, ionic and so on bonds, because everything around you chemistry, including you. I mean how would you find a job as an accountant and start a family without understanding chemistry?
• spaetzleesser 5y ago
  I never have used a bitshift operator in production code but I think it’s important to at least know that these fundamental operators exist. Also, whenever I see code that takes a number, converts it to string, does some manipulation and then converts back to number , I always assume that this was a quick hack that can be done better.
  • CRConrad 5y ago
    > I think it’s important to at least know that these fundamental operators exist. And some inkling of what they're good for. I have that too, although I'd have to look up the exact syntax of the operators (i.e. even what symbols they're actually written as) in pretty much every language I know that has them. Heh, look up if it has them to begin with... > Also, whenever I see code that takes a number, converts it to string, does some manipulation and then converts back to number, I always assume that this was a quick hack It's usually the other way around, IME: You get what should be a number as a string, have to clean it up (thousands separators, different decimal "point" (comma, dammit!) symbols, etc), convert it to a number to do some calculation, then apply the formatting back onto the result. (Die besten Spätzle habe ich ironischerweise weder in Bayern noch im eigenen Ländle gegessen, sondern in der Dampfbäckerei in Berlin.)
• stncls 5y ago
  Your code is nowhere near as egregious as the OP's. Actually, I would even suspect your code is made better by not using hacks, because you deal with bases (radix) that are not necessarily 2. And using strings is a necessity since your output is a string. The original article, instead, is a straightforward bit extraction, from integer to integer. For people who do know bitwise operators, the simple shift reduces mental load (at least according to OP, and I strongly agree).
• AnIdiotOnTheNet 5y ago
  > At the risk of sounding stupid, I have no idea how bit-shifting and bitwise operators work, though I've worked in the field for over a decade and am now a Staff Engineer or whatever. There, I said it. I mean no offense, but honestly I'm a bit flabbergasted that anyone who has worked in programming computers for over a decade wouldn't have acquired this exceptionally basic bit of computing knowledge.
  • sudhirj 5y ago
    It simply hasn't come up. I've written accounting and billing software that processes millions without using bit shifting, logistics solvers that do multithreaded depth first search systems on lazy graphs without bit shifting, ticketing systems that sold millions of tickets correctly without bit shifting etc. I'm sure there are plenty of use cases out there, I just haven't worked on them.
    • thechao 5y ago
      The right algorithm beats out low-level every time. Think of bitwise operators as a vector unit, but the scalar is a bit. That’s it.
    • AnIdiotOnTheNet 5y ago
      It's not that they haven't come up in their job that's strange, but that they were never exposed to this level of base computing principle at any point in their career.
    • hnick 5y ago
      Would you have recognised the opportunity to use it, not knowing how it works? Sometimes when we learn a new technique we suddenly see all the places it can be used that we didn't know about before. There's the old (probably apocryphal) story about the landscaper who was able to fire one of his staff when a customer told him he can calculate the side of a right-angled triangle without measuring all 3 sides. What he did worked for him, he just didn't know there was a better way. Anyway I do agree in this case you are probably correct. I think most programmers these days use bitwise logical operators for flags and rarely if ever bit shift outside of system code.
• amelius 5y ago
  You don't need bit shifting. You can just divide or multiply by powers of 2. (And most of the time the compiler will emit the corresponding bit shifting code.)
  • phoyd 5y ago
    Wrong. (-1/2) is 0, (-1>>1) is -1 in C/C++/Java/C#...
    • amelius 5y ago
      Yeah, the signed case needs some care (and in fact has special shift operators in instruction sets).
    • kazinator 5y ago
      Wrong. -1 >> 1 produces an implementation-defined value in C. -1/2 used to also be implementation-defined; C99 started to require truncation toward zero. That actually makes division unsuitable for simulating bit shifting; you generally want a right shift of a negative quantity to have predictable behavior: either extend the sign, or bring in a zero. Truncating division toward zero doesn't do either; it propagates the sign, until zero is hit. If you can, avoid signed types if you're working with bit fields.
• the_only_law 5y ago
  I know how they work, but I’m really bad at arithmetic in general, so anything more than just basic operations (I do consider the above article pretty basic) and I’ve struggled to build stuff that required a lot of binary operations on values.
• JohnBooty 5y ago
  No shame in that. You're going to see lots of people scoffing at a failure to understand such "foundational" aspects of programming but I would not agree with them. In a "traditional" (let's say, up through the 90's? perhaps longer? maybe it's still that way today?) computer science education with lots of C and assembly, you do a lot of twiddling of individual bits. Either because you're trying to pack multiple values into a single integer (bits 0-3 are the player's score, bit 4 is the number of lives remaining, bits 5-8 are the time remaining, etc) or for certain kinds of low level math. So, folks who went through that sort of education will tend to view this as foundational knowledge. There will always be a need for such bit-twiddling, of course. Folks writing low level code, binary network protocols, etc. However, as you may already be thinking to yourself, most software engineering work today decidedly does not involve that sort of thing. Essentially, IMHO you just need to know that this stuff exists - to know that most languages give you ways to twiddle bits in case you actually need to twiddle bits somebody. Bitwise stuff is generally pretty simple, you can just get familiar with it you ever actually need it.
  • sudhirj 5y ago
    Exactly. The other thing I get dirty look for from some people is when I tell them I've never done memory management. Every language I've worked in professionally has garbage collection, and I've also never done pointer arithmetic.
  • AnIdiotOnTheNet 5y ago
    I disagree. Would you fly with a pilot who never learned the basics of lift and drag? Or hire a builder who didn't understand loads and support? But we call people professionals who build software with no understanding of computing fundamentals? Even getting an associates degree in networking required me to understand how binary operations work and do variable length subnetting by hand, even though subnet calculators are readily available. Maybe I'm being off base here in considering it a computing fundamental, but it is difficult for me to see things like this and not associate it with my super-computer phone routinely pausing for several seconds while I am typing.
    • 5y ago
• zomglings 5y ago
  It's not a big deal that you don't know about this stuff. You can get very far as, for example, a web programmer without ever having to explicitly shift a bit. If you are curious, bit shifting is essentially multiplication or division by 2. It is faster than doing the actual arithmetic. Bitwise operators just perform logical bitwise and, or, xor operations on the bit representations of two numbers. This is especially useful if you have mapped some data to binary 0-1 vectors and those operations are semantically meaningful for the representation. Just as an example, you can calculate the Hamming distance between two 0-1 vectors by taking an XOR of their integer representations and then accumulating the 1s digits for all shifts (towards 0) of the XOR-ed value.
  • karmanyaahm 5y ago
    One of the few positive and helpful posts in this thread. The anger and fighting seems to be invited much more. Welcome to the internet
• flerchin 5y ago
  Everything in a computer is binary. An 8 bit char has 8 bits. Bitwise operators operate on those bits. So a char with value 253 is represented in memory as 11111101. Bitwise operator >> would shift all those 1s to the right, resulting in 01111110. Bitwise & runs the AND operation between two values at a bit-level. Same for |, XOR, etc. Hope that helps. Here come the well actuallys.
  • sudhirj 5y ago
    That's clear enough, but I've never been able to wrap my head around why anyone would want to muck about with that. If I was working on the OS kernel or drivers or network implementations maybe? But even then if one wanted to do math with numbers wouldn't they do it with int and let the compiler optimise?
    • flerchin 5y ago
      It's super common for embedded devices, and kernel development. Yeah compilers can mostly figure it out, but if you're really looking at the bits of a value, like in the example, you should operate on them bitwise, it's more clear, and definitely leaves no room for the compiler to leave it in an inefficient state.
      • sudhirj 5y ago
        Yeah, I'm a little embarrassed to say I'd have used an array do what's in the example. I wouldn't have done all the unnecessary string conversions, but if I have a sequence of 64 things and I want the first 22 things that's a slice operation in my head, not a bit-shift, even if the things are bits.
    • edflsafoiewq 5y ago
      The usual reason is packing/unpacking bitfields. To extract the hours/minutes/seconds from a DOS timestamp hour = (timestamp >> 11) & 0x1f min = (timestamp >> 5) & 0x3f sec = (timestamp & 0x1f) * 2 To check if a byte is a UTF-8 continuation byte (x >> 6) == 0b10 To pack 5-bit r, g, b values into a 16-bit integer (uint16)r | ((uint16)g << 5) | ((uint16)b << 10) To swap the bytes of a uint32 (little endian <-> big endian conversion) (x >> 24) | ((x >> 8) & 0xff00) | ((x << 8) & 0xff0000) | (x << 24)
• ekster 5y ago
  Ben Eater’s YouTube videos are a great crash course on how computers work, and will make this kind of stuff (and much more) way more obvious if you watch.
• mateo411 5y ago
  I think you could learn how bit-shifting and bitwise operators work in an afternoon or two. The concepts aren't very difficult, but a lot of people don't know them, because they haven't take a Computer Architecture class or they haven't had to use them in their domain.
• gorgoiler 5y ago
  It’s worth learning about. Fundamentally, computer processors don’t do anything more than add, subtract and bit shift, with comparison, GOTO, and moving numbers to and from memory to make all the magic happen. The operators exist because they map directly to machine code.
• jareklupinski 5y ago
  they tend to come up when you're at the edge of your resource limitations, e.g. microcontrollers with kilobytes of ram, so you're stuffing data into custom bitfields, or quickly parsing certain bits out of a bitstream and taking atomic actions while ignoring the rest of the sample as fast as possible
• Grazester 5y ago
  I don't much about it either until I started bit banging old gaming console controller protocols with an Arduino and other microcontrollers. Here it becomes necessary since you don't have a whole lot of processing power available for the lower level Arduinos and I use a hell don't know Atmega assembly to help speed things along. Since learning there I have only used xor in a JavaScript frontend I wrote because I can. Otherwise I would have just used an if state to check for 1 and then change it to 0 and vice versa.
• athoscouto 5y ago
  Maybe you'll end up needing it someday and then we'll have the The Big Little Guide to Bitwise Operators. Will also take the opportunity to thank you for the message queues one, very good read!
• UglyToad 5y ago
  Wow, you really kicked the hornet's nest with this one! My hot take is that people not knowing this is a good thing and is a sign of progress. Our (range of) languages are just - better - now. We still have languages for situations where people need to squeeze the performance out in resource constrained environments. We also don't have people wrangling raw pointers and bits for web applications because we have performant abstractions over all that generally extraneous detail. And that enable productivity gains. For example C# has the [Flags] enum attribute which allows an enum variable to have multiple values. These enums are meant to have values which are powers of 2 like: [Flags] enum Options { IgnoreCase = 1, // 1 << 0 Multiline = 2, // 1 << 1 Global = 4, // 1 << 2 Foo = 8, // 1 << 3 Bar = 16, // 1 << 4 } And you can just use these things without having to really think about what they're doing, e.g: var myOpts = Options.Multiline | Options.Foo; var isGlobal = Enum.HasFlag(myOpts, Options.Global); This is all bitwise stuff under the hood. For example: var myOpts = Options.Global | Options.IgnoreCase; // 5. var isGlobal = myOpts & 4 > 0; I've ended up doing a lot of it at a low/raw level in C# when working with binary formats but most people just don't need to know it and that's great! Not working with that level of detail is good because it sure is easy to get it wrong. And it enables us to go faster and build quicker. Sure HasFlag is less performant than actually doing the bitwise operations in most cases but it's an irrelevant thing to try to optimise versus caching, the database, CDN, authentication, permissions checks, etc. People getting worked up about the whole thing smacks of premature optimisation.
• pwdisswordfish8 5y ago
  > At the risk of sounding stupid, I have no idea how bit-shifting and bitwise operators work, though I've worked in the field for over a decade and am now a Staff Engineer or whatever. There, I said it. That sounds stupid. There, I said it.
• QuadrupleA 5y ago
  Along the lines of AnIdiotOnTheNet's points, I'm surprised simple curiosity never led to browsing the bitwise operators - they're core features of every programming language (the languages themselves, not the standard libraries). Bitwise Or is used all the time for flags in higher level languages (e.g. regex flags in python). And dealing with any kind of efficient binary file format or network protocol usually requires their use. Guess these are things you could never encounter at a job, but it does worry me - modern software performance on our supercomputer phones and desktops is often atrocious, and needlessly so. Using compact binary representations has a lot of benefits - lets you do a lot more work in a single cache line (fetching from RAM is 100x slower), saves storage space (when you're dealing with a million or a billion of something) and is often easier to understand and maintain, surprisingly, than an over-abstracted class hierarchy that does everything it can to bury and hide the data your program works with.
• bradknowles 5y ago
  I learned how bit-shifting and bitwise operators worked while obtaining my BSCS degree. I even used it in a programming task in my Numerical Methods class, and my programs ran much, much faster than everyone else in that class. Like, orders of magnitude faster. But I’ve never used that knowledge outside of my degree, or in any of the programs I’ve worked on since 1989. So, interesting knowledge to have, but I can’t say it was actually useful in any professional setting.