17 ms·

In mobile, 62.7% of computation energy cost is spent on data movement [pdf]

200 points by bshanks 7y ago

bshanks 7y ago
One way in which the brain differs from conventional computers is that neurons appear to serve as both CPU and memory storage. Rather than having a few fast CPUs and a large bank of dedicated memory with a few busses (leading to "the von Neumann bottleneck" of data transfer between memory and CPU), the brain has many slow neurons each of which has many connections with other neurons. So, in-memory computing may be more brain-like. The brain appears to be very good at massive concurrency with low energy consumption, at the expense of slow serial computation and a high error rate.
- varjag 7y ago
  There is no CPU in brain, some analogies don't go very far.
- mikorym 7y ago
  From a mathematical view on complexity, I don't see exactly the difference between memory and computation. I wonder what would be suitable definitions for the two concepts.
  - shele 7y ago
    You can make sense of the differences abstractly considering the band, the state register and the instruction table of a Turing machine.
    - jacquesm 7y ago
      A Turing machine is just one of many ways in which you could make a computing device. It is one of the easiest to mechanize and program for, which is why we use them but there is absolutely no reason to believe it is the only way of doing things. GP was getting at a more basic form of computation than the specific one performed by Turing machines. From a mathematical perspective the tape, the registers and instruction tables are all implementation details.
      - shele 7y ago
        > absolutely no reason to believe it is the only way of doing things Well, I did not suggests that. > From a mathematical perspective the tape, the registers and instruction tables are all implementation details. No, they are used to define concepts such as space complexity for Turing machines and there is something to learn from Turing machines, that's why people are studying them.
        pdpi 7y ago
        You can think of the Church Turing thesis as providing a formalism for how the tape, registers and instruction tables are all implementation details.
    - mikorym 7y ago
      So the band and state register is memory? And the instruction table is computation?
      - gpm 7y ago
        The instruction table is a function. Computation is the act of applying that function to the state. Edit: Grammatically "computation" is "The act of computing" where computing is a verb. "Memory" is a noun, and not one about an act. It's more like saying "clothing".
        mikorym 7y ago
        This kind of touches on my original point. A function can be seen both as the set of all pairs of the function, or as an oracle that you feed input to and receive output. The first is to me the full memory space, the second is maybe computation.
  - wongarsu 7y ago
    Memory is the current state, computation is the permutation of state (your step function that takes you from one state to the next).
    - mikorym 7y ago
      This is assuming you have an automaton, I would guess? But what if you approach it via something like entropy? Edit: Doesn't that step function encode all of memory implicitly?
      - gpm 7y ago
        I have no clue what you mean by approaching computation from the point of view of entropy, entropy is a measure of the degree of disorder in a system, how is that a method of performing computation? The step function does not encode memory. The step function is a function that takes memory state 1 as input, and outputs memory state 2 as output. The step function itself doesn't change, it stores no information. In physical terms, the step function is the CPU, minus the caches, registers and other modifiable state. Memory is the disk and ram and all the CPUs caches and registers. You can't tell me anything about what the input to the CPU is, or what it is actually doing (other than that it is capable of running arbitrary x64 assembly), without looking at the state.
        mikorym 7y ago
        A function does store information. The common definition of a function is the set (R,(X,Y)) where R is the graph of the function (consisting of the pairs) and X and Y are domain and codomain, respectively. Computation can be seen as some act of carrying out logic. And in turn, logic can be approached via entropy. There are a lot of people working in QM, computation and entropy. But the simpler definition is just via partitions (and you especially don't have to insist on it being in the world of QM either). [1] [1] http://www.ellerman.org/intro-logical-entropy/ http://www.ellerman.org/intro-logical-entropy/
        gpm 7y ago
  - tel 7y ago
    If computation is the process of spending effort to receive information, then memory is partially paid-down computation. There are lots of ways to blur these lines.
  - arcticbull 7y ago
    This is actually reflected in modern FPGA devices. To your point, there's no material difference between "computation" and "retrieval" (they're just two different ways of mapping a set of inputs onto a set of outputs). FPGAs [1] are basically giant arrays of truth tables. These are implemented as look-up tables -- RAM -- paired with some dedicated, optimized, high-complexity logic elements like multipliers. Could you implement a 16x16 bit multiplier (yielding a 32-bit result) as a RAM or ROM, I mean, totally. It would be a 32 bit address space where each entry is also 32 bits. That would take 16GB. You'd have the fastest multiplier in the West, though, net of propagation delay. [1] From a combinational logic perspective.
- eggie 7y ago
  > neurons appear to serve as both CPU and memory storage I'm not sure this is really certain. But that uncertainty is more because it's hard to prove that there isn't another means of recording memory than something we know. There has been speculation that there might be some genetic or biopolymer based system in the hippocampus, but I'm not familiar with anything more than speculation on that topic.
  - eggie 7y ago
    It seems unlikely to me that biological systems wouldn't exploit digital (e.g. genomic/DNA/RNA) kinds of memory storage, given that the materials are there and the energy costs of maintaining and accessing that kind of storage can be fantastically low. Keeping memory alive in neural circuits would seem much more difficult. Surely that's happening to a large degree though. I'm just highlighting the possibility of an alternative basis for certain kinds of memory in the brain.
    - HPsquared 7y ago
      That method of storage doesn't even need to be in the brain - could be anywhere!
  - polyterative 7y ago
    Can it be that the neural structure itself is the memory? The weak/strong relationships maybe form the memories themselves, reconsructing a vivid image like seeds for procedural algorithms.
    - polyterative 7y ago
      This still doesn't explain our sense of time in memories. We always remember when other than what, even if some knowledge has no time positioning
  - MayeulC 7y ago
    While there might be other mechanisms at play, plasticity is a form of memory: If each synapse can store weights, read them, and change them, it could be seen as a memory. For me, an analogy that is quite adequate in computer science would be LUTs in an FPGA: they are memory+compute units, and can be used as both. To go a bit further, any memory could indeed be considered a computation unit, or vice-versa: consider a results cache, for instance. The difference I see between memory and computation is that memory accesses are, if not instantaneous, at least constant-time. If you want more precision, this can be computed: the necessary memory footprint is extended trough time as well as space (if you only use a results LUT, you'd have to make it bigger to gain precision).
- amelius 7y ago
  Yes but in a computer every processing unit can be used for widely different tasks. In contrast, the neurons in a brain are used for the same task over and over again.
  - didericis 7y ago
    > the neurons in a brain are used for the same task over and over again How does this square with the fact that people can get large portions of their brain scooped out and still retain many of their previous abilities? This is a naive assumption, but if there were purely a 1 to 1 correspondence between a neuron and any given task, I’d assume you wouldn’t be able to recover from massive brain damage. But it seems like people can retrain the functional parts of their brain after damage to other parts to pick up the slack somehow.
  - mojuba 7y ago
    But you could as well say, the same part of computer's memory where the execution binary is stored is used for the same thing again and again. I don't think these parallels between the computer and the brain are valid though. The brain is a highly parallel but slow machine, we have very fast and not-so-parallel computers (just a few cores). I think a lot of what the brain does can be expressed on computers without changing the architecture much, i.e. you don't need to compare the architectures, just the outcomes of computation.
    - TheFiend7 7y ago
      But surely we can improve modern computing devices architecture by taking a look at the computing advantages the brain takes in order to achieve its desired tasks. Don't forget humans are one of the most expensive resources and nothing can really replace them for even a vast majority of non-physical jobs. So if we could build computing devices that potentially solve some of these problems, needless to say, the productivity boosts would be plentiful. Secondly, we aren't even close to being able to model anything relatively close to the computational capabilities of the human brain because we don't even understand the human brain. So your comments on highly parallel but slow and fast but no so parallel don't make a lot of sense. For example take MapBox's new vision SDK. It's able to perform semi-decent feature extraction on the road while people drive via a camera. Well guess what I would absolute stomp the vision SDK on accuracy for every feature it thinks it identified, not only that, I am capable of identifying an order of magnitude more features than it can, not only that, but I'm able to identify new features on the fly and even guess with greater accuracy what they are. So yeah, there are a plethora of functional yields that we have yet to achieve with some of the most powerful computers in the world that are achieved by the human brain every day. Which could be indicative of maybe both a resource, but also an architecture problem.
- KoftaBob 7y ago
  Super fascinating, any recommendations on where I can do further reading on this?
- Richard_East 7y ago
  Low energy consumption checks out - a brain uses only 12 watts.
  - lm28469 7y ago
    You have to feed and train it for the first 15+ years before it's any good though.
    - TheFiend7 7y ago
      Brutal truth, that's also assuming 100% success rate. Lots of brains turn out to be shitty brains after 15+ years too...
      - 8bitsrule 7y ago
        "Insanity -- a perfectly rational adjustment to an insane world." — R. D. Laing
  - anticensor 7y ago
    Your brain uses 12 watts at rest and 120 watts under stress.
- tehjoker 7y ago
  High error rate?! I said precisely moving several complex muscles to vocalize a concept that requires simulating not only how I perceive it but how both the directed receiver and other listeners will as well while focusing my eyes rapidly on an area the size of a quarter.
  - LeifCarrotson 7y ago
    Try doing that a couple times in a row, recording the audio and video. Can you do it the same way twice, with a measure of "same-ness" that can be trivially computed? No, humans cannot do that. Error rate in the context of computing does not need to mean invalid, just different. Our astonishingly high abilities to compensate for variations in input data do not result from an ability to produce output data repeatably.
    - tehjoker 7y ago
      I don't think directly comparing humans and computers are that useful as they are "designed" for different tasks.
- SilasX 7y ago
  Could it be feasible to use the "brainlike" model for practical hardware and reap benefits from avoiding the vN bottleneck? My intuition is that such a system would be much harder to reason about -- and therefore harder for compilers to emit efficient machine code for -- but I'm assuming someone here knows the topic pretty well? (Before you give me "the lecture", yes, I'm aware that in general it's not a good idea to simply mimic the kludges that evolution came up with.)
  - lm28469 7y ago
    Unless we made some big advances I'm not aware of, as far as I know we really don't know much about how the brain works. I doubt anyone can answer your question.
    - SilasX 7y ago
      My question doesn't depend on knowing exactly how the brain works; it's a question about mixing memory and CPU, which happens to be similar to (how we believe) the brain works.
      - lm28469 7y ago
        > which happens to be similar to (how we believe) the brain works. Can you link sources supporting that ?
        derefr 7y ago
        The claim you quoted is actually a null hypothesis—the negation (that the brain separates memory and processing) would require us to posit new types of as-yet-undiscovered neurochemical interactions (which may certainly exist, but we can't say!), and thus the burden of evidence would fall on proving that claim. Or, un-negated, for this claim, the burden of evidence is on disproving it.
        lm28469 7y ago
- ggreer 7y ago
  The reason brains work the way they do is because evolution hill climbed to a solution, not because that solution is globally optimal. Compared to silicon, neurons are absurdly slow. Axons transmit information at 100 meters per second. Electronics transmit information 3,000,000 times faster. When your switching speed is 100Hz, the only way to get anything done in a reasonable amount of time is to be massively parallel. That's why brains are the way they are. Other points of comparison: Animal wings (birds, insects, bats) combine both lift and thrust, but we don't fly around in ornithopters. And while animal legs combine support and power for locomotion, we don't drive around in vehicles with legs. Wheels are far more efficient, but they're not something that evolution can discover easily.
  - __s 7y ago
    There's a few animals that have evolved wheels. Much harder to evolve an axle
  - kbenson 7y ago
    > Wheels are far more efficient Wheels are far more efficient if you have a very specific type of terrain to deal with. It's not immediately obvious to me that a wheel is more efficient overall on naturally occurring terrain. For example, wheels don't work well on steep and rough terrain, or in water (as opposed to under water). The trade-off might be better stated as wheels are very efficient for very select terrain types, and that efficiency tapers off fairly sharply for some terrains, reaching 0% in some cases, while legs are less efficient overall, but achieve some level of usefulness on almost all terrains (making them a better choice for organisms that have disparate terrain types to deal with). Similarly, a computer may be much faster in some types of operations, but we can't even get it to do some stuff that's fairly trivial for many animal brains, so that may also be a trade-off in efficiency somewhere (possibly in an area we don't sufficiently understand yet).
    - perl4ever 7y ago
      Possibly walking recaptures energy that a simple wheel mechanism wouldn't? I vaguely think I read something about how humans are fairly efficient at recycling the energy produced as you stride and gravity pulls you down.
kstenerud 7y ago
It's not just a problem with data movement inside the machine. Data movement between processes and machines is also horribly wasteful. Text encoding/decoding wastes massive amounts of energy, which is why I've been developing a format over the past 2 years with the benefits of binary (smaller, more efficient) as well as text (human readability/editability): https://github.com/kstenerud/concise-encoding https://github.com/kstenerud/concise-encoding
- aidos 7y ago
  Seems like a good effort. I couldn’t easily find an example of the text encoding looking at your readme though.
  - Reelin 7y ago
    Text: https://github.com/kstenerud/concise-encoding/blob/master/cte-specification.md#example https://github.com/kstenerud/concise-encoding/blob/master/ct... Binary: https://github.com/kstenerud/concise-encoding/blob/master/cbe-specification.md#decimal-floating-point https://github.com/kstenerud/concise-encoding/blob/master/cb...
- Reelin 7y ago
  How does this compare to MessagePack? (https://github.com/msgpack/msgpack https://github.com/msgpack/msgpack)
  - kstenerud 7y ago
    MessagePack: * Doesn't have a sister text format for human editing. * Uses raw uncompressed types for integers and floats, which wastes space * Doesn't support arbitrary sized/precision numeric types * Timestamp format uses seconds instead of Gregorian fields (which means that your implementation must keep a leap second database, and time conversions are complicated) * No time zones * Doesn't have metadata, comments, or references * Container types have an unnecessary length field * Stores data in big endian byte order (an unnecessary expense since all popular modern chips are little endian) * Wastes an extra bit for negative integers * 32-bit limitations on length (so max 4gb payload sizes) * No versioning of documents
alecco 7y ago
Slide 3. Paper https://people.inf.ethz.ch/omutlu/pub/Google-consumer-workloads-data-movement-and-PIM_asplos18.pdf https://people.inf.ethz.ch/omutlu/pub/Google-consumer-worklo... Still, this doesn't sound right. Screen should be dominant.
- segfaultbuserr 7y ago
  I expect baseband hardware to be dominant as well.
- marcosdumay 7y ago
  The paper is focused on "computation", that seems to be defined as what happens on the CPU and the things directly connected to it, and ignores everything else. I clicked expecting a paper about 4G networking.
- willvarfar 7y ago
  I don't think they are talking about the phone's total energy; the charts in that excellent paper you linked to have costs for CPU, L1, L2, DRAM etc but no screen, GPU, radio etc. A long time ago I was a technical product manager for, among other things, this stuff. At the time, the screen completely dominated and the GPU was a distant second, unless you were playing a 3D game, in which case it used almost as much as the backlight. CPU and radio were a long way behind them. My intuition is that this hasn't changed, as I can still get dramatically better battery life on my new iPhone in iBooks by playing with the brightness slider.
  - alecco 7y ago
    Yes, the title of this thread is not clear.
  - lonelappde 7y ago
    Per Carroll's results in An Analysis of Power Consumption in a Smartphone: The backlight is either the lowest power local component (10%, 40mW) or highest (50%, 400mW) depending on how bright it is turned up. The GSM radio is the highest power component (about 400mw), when it is working continuously (phone call). Related: a big contribution of smartphone OS optimizations and stuff like wifi 6 is about being smarter about turning off the radio for micro time slices when not needed. Laptops are similar, but it depends a lot on whether you are using a 15W CPU Ultrabook with no GPU, or 65W mobile workstation CPU plus a GPU.
Veedrac 7y ago
A major improvement to the energy cost of data movement will come from closer memories, like HBM and on-die stacked DRAM.
- pdimitar 7y ago
  What do you mean by "closer memories"?
  - Veedrac 7y ago
    Putting memory closer to the CPU, through dense and short-distance interconnects, allowing for much more efficient communication. Eg. https://www.youtube.com/watch?v=-besHp8HLxo https://www.youtube.com/watch?v=-besHp8HLxo.
extropy 7y ago
TLDR: let's put compute inside the memory. I'm very leaning towards calling this bullshit. Closer to memory compute is what we have been doing for last 10 years, and that's why we have ever increasing hierarchical caches. Yes the ALUs are a tiny part of the power budget because moving / syncing data is the hard problem. If you want massive parralelism, use a GPU. The in memory clone and zero could make some sense, but generally you want to start writing to that memory soon after that, meaning you still need to pull it into your cache and the benefit is negated.
- marcosdumay 7y ago
  > If you want massive parallelism, use a GPU. The GPU doesn't have all of your data stored in it. The clone and zero part have a very weak reasoning, focusing on bad targets because they look better than important ones, but take a look at the next section, where they do a simple database search.
- imtringued 7y ago
  PIM is already a reality it's just a matter of time until it sees broad adoption. PIM does not suffer from the crippling limitations of GPUs which only perform well with arithmetic bound problems, cannot run conventional non SIMD code efficiently and need a relatively large batch size. For instance try deserializing a million JSON strings with a GPU. The end result is a graph like memory structure which GPUs usually struggle with. GPUs cannot take advantage of sharing the instruction stream here because it will hit branches for almost every single character and diverge quickly. And finally if you only want parse one JSON object then the GPU is worthless. A PIM based solution would not struggle with heterogeneous non-batched workloads with arbitrary layout at all but still offer the same performance advantages that GPUs enjoy compared to CPUs and reduce energy usage at the same time.
aaxa 7y ago
I think this headline is very misleading. This is about computational energy cost and does not represent what the headline implies (that it's 62.7% of total energy consumption)
- brohee 7y ago
  Yeah on mobile screen and radio dominates the compute part by quite a bit. Not that it's not still worth optimizing.
  - lonelappde 7y ago
    Screen and radio are both data movement tools, so I don't understand the OP's classification process.
    - andai 7y ago
      The next step after in-memory computing: in-screen computing!
      - rubinelli 7y ago
        Maybe not computing, but cathode ray tubes were used as RAM in the forties: https://www.radiomuseum.org/forum/williams_kilburn_williams_kilburn_ram.html https://www.radiomuseum.org/forum/williams_kilburn_williams_...
      - ip26 7y ago
        We're already moving towards self refreshing panels and the like. And it's not a stretch to imagine in-screen cursor movement.
  - ksec 7y ago
    Yes, It is just lower prioritises. 100% of the Mobile Phone Energy Usage during Internet Browsing, 50% goes to Display, 30% goes to Radio Transmission and 5G Encode / Decoding. Your Three SoC + NAND + Memory only uses 20%, I just think title is a little misleading. Not to mention the Software we are running could be anywhere between 2 to 10x from maximum efficiency. It is just the human cost involves to get to the point might not be worth it. There is also Moore's Law, while we might be near or already at the end of it, we still have few more nodes to go and could reduce the energy usage of SoC + NAND + RAM by 30% to 50%. While on the Display and Radio is isn't so clear how far we could further reduces it.
pjc50 7y ago
Including a copy of Maslow's hierarchy of needs and some green field vs factory slides seems to be .. overselling it a bit? Fortunately it pulls back towards a "what are the easy wins" approach: in-memory initialising (rather than stream zeroes over the bus, ask the memory to zero itself) and in-memory copy. The three real problems are: - this is only useful if the whole stack "knows" about it, or can be transparently optimised to use it. Otherwise the person running a Javascript for loop to set values to zero will ruin it. - this relies on the data being in DRAM-row-size chunks, and moving it around within the same physical chip. That may also require re-architecting to make that common enough to be useful. Locality issues are the downside to distributed systems; the whole CPU architecture is oriented towards continuing to pretend that everything happens in a defined order in a single place. - possible security issues (rowhammer?): may seem far off, but if you don't think about it upfront it gets very expensive later. (Worth recapping: the silicon is fundamentally differently-processed for DRAM, such that implementing complex logic is slow and expensive.)
m0zg 7y ago
The entirety of acceleration hardware industry is based around this fact. Basically every single accelerator tries to move more memory towards significantly simpler (and wider) compute. At the extreme of this is compute-on-DRAM, not a new idea certainly, but one that is yet to materialize. Systolic architectures are also very efficient . GPUs are far less so, their main advantage is relatively good tooling and extensive programmability, not energy efficiency, per se. And CPUs utterly and completely suck at high throughput workloads, power efficiency wise. They do often have enough compute to do e.g. lightweight deep learning though. Years ago it cost 8pJ/mm to move a byte on-chip. It's probably closer to 5pJ/mm now, but ALUs consume a fraction of this energy to do something with that byte. And once you leave the chip and hit the memory bus, things get _really_ slow and expensive.
euroPoor 7y ago
ethz going _strong_ these days on HN
mojuba 7y ago
Some big part of the data movement can be eliminated by introducing persistent RAM, i.e. the memristor that never happened unfortunately. When the missing component is finally invented I think we'll have an opportinity to revisit some of the core OS concepts, such as a file. Just imagine for a second you have a persistent random-access memory device that doesn't require you to serialize/deserialize things, or move and resolve the binary modules. Your binary module/app will be executed right where it's stored, provided that it was resolved and prepared when it was first copied to your file system. Similarly, a JSON file will be stored as a memory structure rather than serialized JSON (converted to text as necessary, e.g. when viewed in a text editor) Etc. etc. There are some really interesting changes that can potentially happen when the memristor type of memory becomes available, possibly with its own problems too, but with the huge benefit of moving less data.
- golergka 7y ago
  There's already an experimental OS that is built on idea of persistent program memory: http://phantomos.org/ http://phantomos.org/
  - mojuba 7y ago
    That's very intriguing, thanks!
- wongarsu 7y ago
  > Similarly, a JSON file will be stored as a memory structure rather than serialized JSON You can do this in C++ right now: mmap a file to a memory region, and just create structs in that region. This has two problems: - normally you want to save at controlled points in time, otherwise you have to worry about recovering from states where some function updated part of the data and then crashed (phone ran out of power etc) - just writing a bunch of internal data structures into a file used to be moderately popular and has great performance, but it's a major headache when you ever update them. You end up implementing a versioning scheme and importers for migrating old files to your new application version. At that point a file format that is designed for data exchange is less headache. In general mmap already offers you a way to treat your disk like memory with decent performance (thanks to caching), and the number of good use cases turned out to be somewhat limited. I doubt just making that faster with new technology will change much.
aberforth123 7y ago
A small part of the solution: Let's block ads! Proof: https://webtest.app/?url=https://www.wowhead.com https://webtest.app/?url=https://www.wowhead.com
- the_pwner224 7y ago
  That link was shockingly fast. I was quite surprised by how quickly it loaded - you don't notice any loading time unless you specifically look for it. If only everything would be like that...
  - egdod 7y ago
    Disable JavaScript. Everything becomes fast (or doesn’t work at all).
    - jgalt212 7y ago
      I'm tired of this comment. 1. Pretty much every website needs JS to function as intended. 2. Delivering fully marked up HTML to the client is a waste of server resources and an under-utilization of 4 GHz client resources.
      - egdod 7y ago
        1. Actually no, most websites work just fine. I have YouTube, Twitter, and Facebook (I know) whitelisted. Pretty much everything else I use regularly is fantastic. 2. The server doesn’t have to run on a battery small enough to put in my pocket.
      - RussianCow 7y ago
        Mostly agree with #1, but I don't understand #2. You're saying delivering plain, pre-rendered HTML is too fast for the client, so we shouldn't use it (especially in a thread about energy use on mobile)?
        organsnyder 7y ago
Gravityloss 7y ago
Most of the data is useless anyway. Bloat factor over mostly text content and an adequate quality picture is huge, probably 10x.
localhost 7y ago
Cerebras's "wafer scale engine" [1] takes some of these ideas and applies them narrowly to deep learning training. 400,000 cores, 18GB of on-chip memory, 9.6 petabytes of memory bandwidth in a 1.2 trillion transistor package in a gigantic 46,225mm^2 package. [1] https://www.cerebras.net/cerebras-wafer-scale-engine-why-we-need-big-chips-for-deep-learning/ https://www.cerebras.net/cerebras-wafer-scale-engine-why-we-...
jedberg 7y ago
The number one cost of any distributed system is data movement. Mobile phones are essentially leaf nodes in a huge distributed system, where they interact with "downstream dependencies", ie. the servers that provide all the functionality of their apps. So it makes sense that moving data would use the most power.
stokedmartin 7y ago
"We observe that data movement between the main memory and conventional computation units is a major contributor to the total system energy consumption in consumer devices. On average, data movement accounts for 62.7% of the total energy consumed by Google consumer workloads." [0] [0] https://people.inf.ethz.ch/omutlu/pub/Google-consumer-workloads-data-movement-and-PIM_asplos18.pdf https://people.inf.ethz.ch/omutlu/pub/Google-consumer-worklo...
colechristensen 7y ago
With all of that ad overhead it's reached a point where the actual costs incurred by the user rival the ad revenue gained by the publisher.
deleted 7y ago
[deleted]

200 points by bshanks 7y ago

One way in which the brain differs from conventional computers is that neurons appear to serve as both CPU and memory storage. Rather than having a few fast CPUs and a large bank of dedicated memory with a few busses (leading to "the von Neumann bottleneck" of data transfer between memory and CPU), the brain has many slow neurons each of which has many connections with other neurons. So, in-memory computing may be more brain-like. The brain appears to be very good at massive concurrency with low energy consumption, at the expense of slow serial computation and a high error rate.

varjag 7y ago
There is no CPU in brain, some analogies don't go very far.
mikorym 7y ago
From a mathematical view on complexity, I don't see exactly the difference between memory and computation. I wonder what would be suitable definitions for the two concepts.
- shele 7y ago
  You can make sense of the differences abstractly considering the band, the state register and the instruction table of a Turing machine.
  - jacquesm 7y ago
    A Turing machine is just one of many ways in which you could make a computing device. It is one of the easiest to mechanize and program for, which is why we use them but there is absolutely no reason to believe it is the only way of doing things. GP was getting at a more basic form of computation than the specific one performed by Turing machines. From a mathematical perspective the tape, the registers and instruction tables are all implementation details.
    - shele 7y ago
      > absolutely no reason to believe it is the only way of doing things Well, I did not suggests that. > From a mathematical perspective the tape, the registers and instruction tables are all implementation details. No, they are used to define concepts such as space complexity for Turing machines and there is something to learn from Turing machines, that's why people are studying them.
      - pdpi 7y ago
        You can think of the Church Turing thesis as providing a formalism for how the tape, registers and instruction tables are all implementation details.
  - mikorym 7y ago
    So the band and state register is memory? And the instruction table is computation?
    - gpm 7y ago
      The instruction table is a function. Computation is the act of applying that function to the state. Edit: Grammatically "computation" is "The act of computing" where computing is a verb. "Memory" is a noun, and not one about an act. It's more like saying "clothing".
      - mikorym 7y ago
        This kind of touches on my original point. A function can be seen both as the set of all pairs of the function, or as an oracle that you feed input to and receive output. The first is to me the full memory space, the second is maybe computation.
- wongarsu 7y ago
  Memory is the current state, computation is the permutation of state (your step function that takes you from one state to the next).
  - mikorym 7y ago
    This is assuming you have an automaton, I would guess? But what if you approach it via something like entropy? Edit: Doesn't that step function encode all of memory implicitly?
    - gpm 7y ago
      I have no clue what you mean by approaching computation from the point of view of entropy, entropy is a measure of the degree of disorder in a system, how is that a method of performing computation? The step function does not encode memory. The step function is a function that takes memory state 1 as input, and outputs memory state 2 as output. The step function itself doesn't change, it stores no information. In physical terms, the step function is the CPU, minus the caches, registers and other modifiable state. Memory is the disk and ram and all the CPUs caches and registers. You can't tell me anything about what the input to the CPU is, or what it is actually doing (other than that it is capable of running arbitrary x64 assembly), without looking at the state.
      - mikorym 7y ago
        A function does store information. The common definition of a function is the set (R,(X,Y)) where R is the graph of the function (consisting of the pairs) and X and Y are domain and codomain, respectively. Computation can be seen as some act of carrying out logic. And in turn, logic can be approached via entropy. There are a lot of people working in QM, computation and entropy. But the simpler definition is just via partitions (and you especially don't have to insist on it being in the world of QM either). [1] [1] http://www.ellerman.org/intro-logical-entropy/ http://www.ellerman.org/intro-logical-entropy/
        gpm 7y ago
- tel 7y ago
  If computation is the process of spending effort to receive information, then memory is partially paid-down computation. There are lots of ways to blur these lines.
- arcticbull 7y ago
  This is actually reflected in modern FPGA devices. To your point, there's no material difference between "computation" and "retrieval" (they're just two different ways of mapping a set of inputs onto a set of outputs). FPGAs [1] are basically giant arrays of truth tables. These are implemented as look-up tables -- RAM -- paired with some dedicated, optimized, high-complexity logic elements like multipliers. Could you implement a 16x16 bit multiplier (yielding a 32-bit result) as a RAM or ROM, I mean, totally. It would be a 32 bit address space where each entry is also 32 bits. That would take 16GB. You'd have the fastest multiplier in the West, though, net of propagation delay. [1] From a combinational logic perspective.
eggie 7y ago
> neurons appear to serve as both CPU and memory storage I'm not sure this is really certain. But that uncertainty is more because it's hard to prove that there isn't another means of recording memory than something we know. There has been speculation that there might be some genetic or biopolymer based system in the hippocampus, but I'm not familiar with anything more than speculation on that topic.
- eggie 7y ago
  It seems unlikely to me that biological systems wouldn't exploit digital (e.g. genomic/DNA/RNA) kinds of memory storage, given that the materials are there and the energy costs of maintaining and accessing that kind of storage can be fantastically low. Keeping memory alive in neural circuits would seem much more difficult. Surely that's happening to a large degree though. I'm just highlighting the possibility of an alternative basis for certain kinds of memory in the brain.
  - HPsquared 7y ago
    That method of storage doesn't even need to be in the brain - could be anywhere!
- polyterative 7y ago
  Can it be that the neural structure itself is the memory? The weak/strong relationships maybe form the memories themselves, reconsructing a vivid image like seeds for procedural algorithms.
  - polyterative 7y ago
    This still doesn't explain our sense of time in memories. We always remember when other than what, even if some knowledge has no time positioning
- MayeulC 7y ago
  While there might be other mechanisms at play, plasticity is a form of memory: If each synapse can store weights, read them, and change them, it could be seen as a memory. For me, an analogy that is quite adequate in computer science would be LUTs in an FPGA: they are memory+compute units, and can be used as both. To go a bit further, any memory could indeed be considered a computation unit, or vice-versa: consider a results cache, for instance. The difference I see between memory and computation is that memory accesses are, if not instantaneous, at least constant-time. If you want more precision, this can be computed: the necessary memory footprint is extended trough time as well as space (if you only use a results LUT, you'd have to make it bigger to gain precision).
amelius 7y ago
Yes but in a computer every processing unit can be used for widely different tasks. In contrast, the neurons in a brain are used for the same task over and over again.
- didericis 7y ago
  > the neurons in a brain are used for the same task over and over again How does this square with the fact that people can get large portions of their brain scooped out and still retain many of their previous abilities? This is a naive assumption, but if there were purely a 1 to 1 correspondence between a neuron and any given task, I’d assume you wouldn’t be able to recover from massive brain damage. But it seems like people can retrain the functional parts of their brain after damage to other parts to pick up the slack somehow.
- mojuba 7y ago
  But you could as well say, the same part of computer's memory where the execution binary is stored is used for the same thing again and again. I don't think these parallels between the computer and the brain are valid though. The brain is a highly parallel but slow machine, we have very fast and not-so-parallel computers (just a few cores). I think a lot of what the brain does can be expressed on computers without changing the architecture much, i.e. you don't need to compare the architectures, just the outcomes of computation.
  - TheFiend7 7y ago
    But surely we can improve modern computing devices architecture by taking a look at the computing advantages the brain takes in order to achieve its desired tasks. Don't forget humans are one of the most expensive resources and nothing can really replace them for even a vast majority of non-physical jobs. So if we could build computing devices that potentially solve some of these problems, needless to say, the productivity boosts would be plentiful. Secondly, we aren't even close to being able to model anything relatively close to the computational capabilities of the human brain because we don't even understand the human brain. So your comments on highly parallel but slow and fast but no so parallel don't make a lot of sense. For example take MapBox's new vision SDK. It's able to perform semi-decent feature extraction on the road while people drive via a camera. Well guess what I would absolute stomp the vision SDK on accuracy for every feature it thinks it identified, not only that, I am capable of identifying an order of magnitude more features than it can, not only that, but I'm able to identify new features on the fly and even guess with greater accuracy what they are. So yeah, there are a plethora of functional yields that we have yet to achieve with some of the most powerful computers in the world that are achieved by the human brain every day. Which could be indicative of maybe both a resource, but also an architecture problem.
KoftaBob 7y ago
Super fascinating, any recommendations on where I can do further reading on this?
Richard_East 7y ago
Low energy consumption checks out - a brain uses only 12 watts.
- lm28469 7y ago
  You have to feed and train it for the first 15+ years before it's any good though.
  - TheFiend7 7y ago
    Brutal truth, that's also assuming 100% success rate. Lots of brains turn out to be shitty brains after 15+ years too...
    - 8bitsrule 7y ago
      "Insanity -- a perfectly rational adjustment to an insane world." — R. D. Laing
- anticensor 7y ago
  Your brain uses 12 watts at rest and 120 watts under stress.
tehjoker 7y ago
High error rate?! I said precisely moving several complex muscles to vocalize a concept that requires simulating not only how I perceive it but how both the directed receiver and other listeners will as well while focusing my eyes rapidly on an area the size of a quarter.
- LeifCarrotson 7y ago
  Try doing that a couple times in a row, recording the audio and video. Can you do it the same way twice, with a measure of "same-ness" that can be trivially computed? No, humans cannot do that. Error rate in the context of computing does not need to mean invalid, just different. Our astonishingly high abilities to compensate for variations in input data do not result from an ability to produce output data repeatably.
  - tehjoker 7y ago
    I don't think directly comparing humans and computers are that useful as they are "designed" for different tasks.
SilasX 7y ago
Could it be feasible to use the "brainlike" model for practical hardware and reap benefits from avoiding the vN bottleneck? My intuition is that such a system would be much harder to reason about -- and therefore harder for compilers to emit efficient machine code for -- but I'm assuming someone here knows the topic pretty well? (Before you give me "the lecture", yes, I'm aware that in general it's not a good idea to simply mimic the kludges that evolution came up with.)
- lm28469 7y ago
  Unless we made some big advances I'm not aware of, as far as I know we really don't know much about how the brain works. I doubt anyone can answer your question.
  - SilasX 7y ago
    My question doesn't depend on knowing exactly how the brain works; it's a question about mixing memory and CPU, which happens to be similar to (how we believe) the brain works.
    - lm28469 7y ago
      > which happens to be similar to (how we believe) the brain works. Can you link sources supporting that ?
      - derefr 7y ago
        The claim you quoted is actually a null hypothesis—the negation (that the brain separates memory and processing) would require us to posit new types of as-yet-undiscovered neurochemical interactions (which may certainly exist, but we can't say!), and thus the burden of evidence would fall on proving that claim. Or, un-negated, for this claim, the burden of evidence is on disproving it.
        lm28469 7y ago
ggreer 7y ago
The reason brains work the way they do is because evolution hill climbed to a solution, not because that solution is globally optimal. Compared to silicon, neurons are absurdly slow. Axons transmit information at 100 meters per second. Electronics transmit information 3,000,000 times faster. When your switching speed is 100Hz, the only way to get anything done in a reasonable amount of time is to be massively parallel. That's why brains are the way they are. Other points of comparison: Animal wings (birds, insects, bats) combine both lift and thrust, but we don't fly around in ornithopters. And while animal legs combine support and power for locomotion, we don't drive around in vehicles with legs. Wheels are far more efficient, but they're not something that evolution can discover easily.
- __s 7y ago
  There's a few animals that have evolved wheels. Much harder to evolve an axle
- kbenson 7y ago
  > Wheels are far more efficient Wheels are far more efficient if you have a very specific type of terrain to deal with. It's not immediately obvious to me that a wheel is more efficient overall on naturally occurring terrain. For example, wheels don't work well on steep and rough terrain, or in water (as opposed to under water). The trade-off might be better stated as wheels are very efficient for very select terrain types, and that efficiency tapers off fairly sharply for some terrains, reaching 0% in some cases, while legs are less efficient overall, but achieve some level of usefulness on almost all terrains (making them a better choice for organisms that have disparate terrain types to deal with). Similarly, a computer may be much faster in some types of operations, but we can't even get it to do some stuff that's fairly trivial for many animal brains, so that may also be a trade-off in efficiency somewhere (possibly in an area we don't sufficiently understand yet).
  - perl4ever 7y ago
    Possibly walking recaptures energy that a simple wheel mechanism wouldn't? I vaguely think I read something about how humans are fairly efficient at recycling the energy produced as you stride and gravity pulls you down.

kstenerud 7y ago

It's not just a problem with data movement inside the machine. Data movement between processes and machines is also horribly wasteful. Text encoding/decoding wastes massive amounts of energy, which is why I've been developing a format over the past 2 years with the benefits of binary (smaller, more efficient) as well as text (human readability/editability): https://github.com/kstenerud/concise-encoding https://github.com/kstenerud/concise-encoding

aidos 7y ago
Seems like a good effort. I couldn’t easily find an example of the text encoding looking at your readme though.
- Reelin 7y ago
  Text: https://github.com/kstenerud/concise-encoding/blob/master/cte-specification.md#example https://github.com/kstenerud/concise-encoding/blob/master/ct... Binary: https://github.com/kstenerud/concise-encoding/blob/master/cbe-specification.md#decimal-floating-point https://github.com/kstenerud/concise-encoding/blob/master/cb...
Reelin 7y ago
How does this compare to MessagePack? (https://github.com/msgpack/msgpack https://github.com/msgpack/msgpack)
- kstenerud 7y ago
  MessagePack: * Doesn't have a sister text format for human editing. * Uses raw uncompressed types for integers and floats, which wastes space * Doesn't support arbitrary sized/precision numeric types * Timestamp format uses seconds instead of Gregorian fields (which means that your implementation must keep a leap second database, and time conversions are complicated) * No time zones * Doesn't have metadata, comments, or references * Container types have an unnecessary length field * Stores data in big endian byte order (an unnecessary expense since all popular modern chips are little endian) * Wastes an extra bit for negative integers * 32-bit limitations on length (so max 4gb payload sizes) * No versioning of documents

alecco 7y ago

Slide 3. Paper https://people.inf.ethz.ch/omutlu/pub/Google-consumer-workloads-data-movement-and-PIM_asplos18.pdf https://people.inf.ethz.ch/omutlu/pub/Google-consumer-worklo... Still, this doesn't sound right. Screen should be dominant.

segfaultbuserr 7y ago
I expect baseband hardware to be dominant as well.
marcosdumay 7y ago
The paper is focused on "computation", that seems to be defined as what happens on the CPU and the things directly connected to it, and ignores everything else. I clicked expecting a paper about 4G networking.
willvarfar 7y ago
I don't think they are talking about the phone's total energy; the charts in that excellent paper you linked to have costs for CPU, L1, L2, DRAM etc but no screen, GPU, radio etc. A long time ago I was a technical product manager for, among other things, this stuff. At the time, the screen completely dominated and the GPU was a distant second, unless you were playing a 3D game, in which case it used almost as much as the backlight. CPU and radio were a long way behind them. My intuition is that this hasn't changed, as I can still get dramatically better battery life on my new iPhone in iBooks by playing with the brightness slider.
- alecco 7y ago
  Yes, the title of this thread is not clear.
- lonelappde 7y ago
  Per Carroll's results in An Analysis of Power Consumption in a Smartphone: The backlight is either the lowest power local component (10%, 40mW) or highest (50%, 400mW) depending on how bright it is turned up. The GSM radio is the highest power component (about 400mw), when it is working continuously (phone call). Related: a big contribution of smartphone OS optimizations and stuff like wifi 6 is about being smarter about turning off the radio for micro time slices when not needed. Laptops are similar, but it depends a lot on whether you are using a 15W CPU Ultrabook with no GPU, or 65W mobile workstation CPU plus a GPU.

Veedrac 7y ago

A major improvement to the energy cost of data movement will come from closer memories, like HBM and on-die stacked DRAM.

pdimitar 7y ago
What do you mean by "closer memories"?
- Veedrac 7y ago
  Putting memory closer to the CPU, through dense and short-distance interconnects, allowing for much more efficient communication. Eg. https://www.youtube.com/watch?v=-besHp8HLxo https://www.youtube.com/watch?v=-besHp8HLxo.

extropy 7y ago

TLDR: let's put compute inside the memory. I'm very leaning towards calling this bullshit. Closer to memory compute is what we have been doing for last 10 years, and that's why we have ever increasing hierarchical caches. Yes the ALUs are a tiny part of the power budget because moving / syncing data is the hard problem. If you want massive parralelism, use a GPU. The in memory clone and zero could make some sense, but generally you want to start writing to that memory soon after that, meaning you still need to pull it into your cache and the benefit is negated.

marcosdumay 7y ago
> If you want massive parallelism, use a GPU. The GPU doesn't have all of your data stored in it. The clone and zero part have a very weak reasoning, focusing on bad targets because they look better than important ones, but take a look at the next section, where they do a simple database search.
imtringued 7y ago
PIM is already a reality it's just a matter of time until it sees broad adoption. PIM does not suffer from the crippling limitations of GPUs which only perform well with arithmetic bound problems, cannot run conventional non SIMD code efficiently and need a relatively large batch size. For instance try deserializing a million JSON strings with a GPU. The end result is a graph like memory structure which GPUs usually struggle with. GPUs cannot take advantage of sharing the instruction stream here because it will hit branches for almost every single character and diverge quickly. And finally if you only want parse one JSON object then the GPU is worthless. A PIM based solution would not struggle with heterogeneous non-batched workloads with arbitrary layout at all but still offer the same performance advantages that GPUs enjoy compared to CPUs and reduce energy usage at the same time.

aaxa 7y ago

I think this headline is very misleading. This is about computational energy cost and does not represent what the headline implies (that it's 62.7% of total energy consumption)

brohee 7y ago
Yeah on mobile screen and radio dominates the compute part by quite a bit. Not that it's not still worth optimizing.
- lonelappde 7y ago
  Screen and radio are both data movement tools, so I don't understand the OP's classification process.
  - andai 7y ago
    The next step after in-memory computing: in-screen computing!
    - rubinelli 7y ago
      Maybe not computing, but cathode ray tubes were used as RAM in the forties: https://www.radiomuseum.org/forum/williams_kilburn_williams_kilburn_ram.html https://www.radiomuseum.org/forum/williams_kilburn_williams_...
    - ip26 7y ago
      We're already moving towards self refreshing panels and the like. And it's not a stretch to imagine in-screen cursor movement.
- ksec 7y ago
  Yes, It is just lower prioritises. 100% of the Mobile Phone Energy Usage during Internet Browsing, 50% goes to Display, 30% goes to Radio Transmission and 5G Encode / Decoding. Your Three SoC + NAND + Memory only uses 20%, I just think title is a little misleading. Not to mention the Software we are running could be anywhere between 2 to 10x from maximum efficiency. It is just the human cost involves to get to the point might not be worth it. There is also Moore's Law, while we might be near or already at the end of it, we still have few more nodes to go and could reduce the energy usage of SoC + NAND + RAM by 30% to 50%. While on the Display and Radio is isn't so clear how far we could further reduces it.

pjc50 7y ago

Including a copy of Maslow's hierarchy of needs and some green field vs factory slides seems to be .. overselling it a bit? Fortunately it pulls back towards a "what are the easy wins" approach: in-memory initialising (rather than stream zeroes over the bus, ask the memory to zero itself) and in-memory copy. The three real problems are: - this is only useful if the whole stack "knows" about it, or can be transparently optimised to use it. Otherwise the person running a Javascript for loop to set values to zero will ruin it. - this relies on the data being in DRAM-row-size chunks, and moving it around within the same physical chip. That may also require re-architecting to make that common enough to be useful. Locality issues are the downside to distributed systems; the whole CPU architecture is oriented towards continuing to pretend that everything happens in a defined order in a single place. - possible security issues (rowhammer?): may seem far off, but if you don't think about it upfront it gets very expensive later. (Worth recapping: the silicon is fundamentally differently-processed for DRAM, such that implementing complex logic is slow and expensive.)

m0zg 7y ago

The entirety of acceleration hardware industry is based around this fact. Basically every single accelerator tries to move more memory towards significantly simpler (and wider) compute. At the extreme of this is compute-on-DRAM, not a new idea certainly, but one that is yet to materialize. Systolic architectures are also very efficient . GPUs are far less so, their main advantage is relatively good tooling and extensive programmability, not energy efficiency, per se. And CPUs utterly and completely suck at high throughput workloads, power efficiency wise. They do often have enough compute to do e.g. lightweight deep learning though. Years ago it cost 8pJ/mm to move a byte on-chip. It's probably closer to 5pJ/mm now, but ALUs consume a fraction of this energy to do something with that byte. And once you leave the chip and hit the memory bus, things get _really_ slow and expensive.

euroPoor 7y ago

ethz going _strong_ these days on HN

mojuba 7y ago

Some big part of the data movement can be eliminated by introducing persistent RAM, i.e. the memristor that never happened unfortunately. When the missing component is finally invented I think we'll have an opportinity to revisit some of the core OS concepts, such as a file. Just imagine for a second you have a persistent random-access memory device that doesn't require you to serialize/deserialize things, or move and resolve the binary modules. Your binary module/app will be executed right where it's stored, provided that it was resolved and prepared when it was first copied to your file system. Similarly, a JSON file will be stored as a memory structure rather than serialized JSON (converted to text as necessary, e.g. when viewed in a text editor) Etc. etc. There are some really interesting changes that can potentially happen when the memristor type of memory becomes available, possibly with its own problems too, but with the huge benefit of moving less data.

golergka 7y ago
There's already an experimental OS that is built on idea of persistent program memory: http://phantomos.org/ http://phantomos.org/
- mojuba 7y ago
  That's very intriguing, thanks!
wongarsu 7y ago
> Similarly, a JSON file will be stored as a memory structure rather than serialized JSON You can do this in C++ right now: mmap a file to a memory region, and just create structs in that region. This has two problems: - normally you want to save at controlled points in time, otherwise you have to worry about recovering from states where some function updated part of the data and then crashed (phone ran out of power etc) - just writing a bunch of internal data structures into a file used to be moderately popular and has great performance, but it's a major headache when you ever update them. You end up implementing a versioning scheme and importers for migrating old files to your new application version. At that point a file format that is designed for data exchange is less headache. In general mmap already offers you a way to treat your disk like memory with decent performance (thanks to caching), and the number of good use cases turned out to be somewhat limited. I doubt just making that faster with new technology will change much.

aberforth123 7y ago

A small part of the solution: Let's block ads! Proof: https://webtest.app/?url=https://www.wowhead.com https://webtest.app/?url=https://www.wowhead.com

the_pwner224 7y ago
That link was shockingly fast. I was quite surprised by how quickly it loaded - you don't notice any loading time unless you specifically look for it. If only everything would be like that...
- egdod 7y ago
  Disable JavaScript. Everything becomes fast (or doesn’t work at all).
  - jgalt212 7y ago
    I'm tired of this comment. 1. Pretty much every website needs JS to function as intended. 2. Delivering fully marked up HTML to the client is a waste of server resources and an under-utilization of 4 GHz client resources.
    - egdod 7y ago
      1. Actually no, most websites work just fine. I have YouTube, Twitter, and Facebook (I know) whitelisted. Pretty much everything else I use regularly is fantastic. 2. The server doesn’t have to run on a battery small enough to put in my pocket.
    - RussianCow 7y ago
      Mostly agree with #1, but I don't understand #2. You're saying delivering plain, pre-rendered HTML is too fast for the client, so we shouldn't use it (especially in a thread about energy use on mobile)?
      - organsnyder 7y ago

Gravityloss 7y ago

Most of the data is useless anyway. Bloat factor over mostly text content and an adequate quality picture is huge, probably 10x.

localhost 7y ago

Cerebras's "wafer scale engine" [1] takes some of these ideas and applies them narrowly to deep learning training. 400,000 cores, 18GB of on-chip memory, 9.6 petabytes of memory bandwidth in a 1.2 trillion transistor package in a gigantic 46,225mm^2 package. [1] https://www.cerebras.net/cerebras-wafer-scale-engine-why-we-need-big-chips-for-deep-learning/ https://www.cerebras.net/cerebras-wafer-scale-engine-why-we-...

jedberg 7y ago

The number one cost of any distributed system is data movement. Mobile phones are essentially leaf nodes in a huge distributed system, where they interact with "downstream dependencies", ie. the servers that provide all the functionality of their apps. So it makes sense that moving data would use the most power.

stokedmartin 7y ago

"We observe that data movement between the main memory and conventional computation units is a major contributor to the total system energy consumption in consumer devices. On average, data movement accounts for 62.7% of the total energy consumed by Google consumer workloads." [0] [0] https://people.inf.ethz.ch/omutlu/pub/Google-consumer-workloads-data-movement-and-PIM_asplos18.pdf https://people.inf.ethz.ch/omutlu/pub/Google-consumer-worklo...

colechristensen 7y ago

With all of that ad overhead it's reached a point where the actual costs incurred by the user rival the ad revenue gained by the publisher.

deleted 7y ago

[deleted]