19 ms·
Neural network spotted deep inside Samsung's Galaxy S7 silicon brain
- dingo_bat 10y agoThis is literally insane. I've never heard of a neural net branch predictor. Edit: my layman knowledge about branch prediction and neural networks is showing its gross inadequacy :/
- nilved 10y agoe: was wrong about everything
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- xlayn 10y agoIf you have to pull an army of cores to fight two the really impressive arch belongs to Apple. OTOH I see this as the optimization of hardware to fix the cap that software, paradigms and devs mean.
- dingo_bat 10y agoAs long as the die area is same, I see it as a fair fight.
- pawadu 10y agoIn fact, for the same die size, Android hardware has been adapted to work better with the way Android works (many small processes working together). This is maybe not the best approach for certain games or benchmarks, but for normal use this has proven to be great.
- dingo_bat 10y agoI guess it's naive to assume that Apple isn't also doing the same thing. And apple's cores are bigger so they generally do more work per core per cycle. The exynos in GS7 has eight cores. But the die area is roughly same as apple's dual core chip.
- DannyBee 10y agoIt's actually really common these days. :) Search google for "hashed perceptron branch predictor"
- mbell 10y agoIntel (and probably AMD) have been using neural based approaches to branch prediction since the early 2000's. This isn't particularly new, it's just not a topic talked about much.
- gpderetta 10y ago[citation needed]. As far as I know, Intel is suspected to be using TAGE or some variant of it in its last few generations of chips.
- AstralStorm 10y agoGoogle for a PDF describing x86 architecture CPUs. There are ways to write a loop that will detect the kind of predictor in use.
- gpderetta 10y agoNo need, other people have already done the work: the paper 'Branch Prediction and the Performance of Interpreters' gave strong hints that Intel has been using ITTAGE at least since Haswell. Edit: also, Intel branch predictors up to Core2 were well characterized and were not believed to be using NN. [1] https://hal.inria.fr/hal-01100647/document https://hal.inria.fr/hal-01100647/document ,discussed last year at HN.
- gcr 10y agoBranch prediction strategies can be learned, just like any other model. I could certainly see something like this outperforming the huge bag of hand-tuned optimization strategies for branch prediction anyway. It could save some silicon. Even a simple dot product can be called a "neural network," albeit a small uninteresting one. Your features could be (say) the state of cache, the number of jumps, the number of recent stalls / stack size / returns, and so on. Put them into a 10-dimensional vector or whatever, take the dot product between that and a set of 10 learned weights, and preload the branch if the result is above some threshold. Even a simple model could perhaps work really well here. Of course it's not like they have some deep learning convolutional net running on the program input. I also bet the weights are set at the factory so it's not "learning" anything.
- medains 10y agoNothing in the article suggests that it's learning - so I suspect that the weights are indeed set in the factory. Learning on the fly would consume much more power as it back-propagates, so doesn't really make much sense as the potential performance gain would be marginal and would probably cause a performance hit when switching to a different process or thread.
- pawadu 10y agoBut if it is not learning, can you really call it NN?
- chas 10y agoYes. Most neural net applications separate training and inference so that once the net is in production it doesn't change, though it might be replaced with a new net rather frequently depending on how much the distribution of data it's operating on changes over time.
- pawadu 10y agonot an expert on this, but wouldn't what you mention be adaptive control rather than ANN?
- corysama 10y agoThe article indicates that academia has been writing about using them for branch prediction for a while and than Intel and AMD have been doing just that for a while. They just don't talk about it much. > AMD's Zen architect Mike Clark confirmed to us his microarchitecture uses a hashed perceptron system in its branch prediction. "Maybe I should have called it a neural net," he added. Neural networks can be very complicated, but they can also be very simple. What little I know about perceptrons is that they are very simple. If you google "hashed perceptron", many of the papers that come up mention branch prediction in their titles.
- verandaguy 10y agoNot necessarily. The cost of backtracking on a bad branch prediction on a device like a phone may be so great that implementing an ANN could be worth the effort to implement. I'm not as familiar with mobile phone architecture as with that of PCs, but the number and types of operations influenced by the device's various sensors (light sensor, GPS, accelerometer, gyroscope) could conceivably be more than normal, naïve branch prediction can handle effectively.
- gpderetta 10y agoWhy would speculation invalidation be more expensive on a phone? If anything, many phone CPUs tend to have shorter pipelines and less expensive branch prediction misses. Also I do not see how the number or quality of sensors could in any way affect the prediction rate.
- pawadu 10y agoIn addition, could this be worse than what we have today - on 64-bit ARM?
- flukus 10y agoFirst time I've heard of them in hardware. The JVM has been employing similar feats for many years now.
- zigzigzag 10y agoLike where? Even the best JVMs do basic counted profiling of branches only, as far as I know.
- pawadu 10y agoHave hotspot JVM optimization mechanisms ever been disclosed?
- zigzigzag 10y agoWhat? It's been open source for years!
- pawadu 10y agoI was under the impression that only some parts of the JVM was open and that the high performance server implementations have always been proprietary?
- pas 10y agoThe (old) Oracle JRockit is not really better. It was, but it was probably getting really troublesome to maintain, and with buying Sun they probably felt that it was rather redundant. (There was no need for a clean room implementation since they owned all relevant IP anyhow.)
- zigzigzag 10y agoNot at all.
- flukus 10y agoWith adaptive optimizing: https://en.wikipedia.org/wiki/Java_performance#Adaptive_optimizing https://en.wikipedia.org/wiki/Java_performance#Adaptive_opti...
- hcs 10y agoMy Computer Architecture professor Daniel Jimenez worked on (invented?) something like this: "Dynamic Branch Prediction with Perceptrons" (PDF) http://hpca23.cse.tamu.edu/taco/pdfs/hpca7_dist.pdf http://hpca23.cse.tamu.edu/taco/pdfs/hpca7_dist.pdf
- akash_m 10y agoI read the paper description and immediately jumped to conclusion that you are Calvin's student :) We had to study this paper during one of my classes at UTCS
- DannyBee 10y agoCalvin Lin is also no slouch :)
- strait 10y agoIncidentally, Daniel was my very first CS instructor about 20 years ago, when he taught my Intro to CS class at UT San Antonio. It was the vi editor and K&R book C programming on some ancient green monochrome terminals back then. I remember it as a fairly severe introduction, but he was always great at explaining things properly.
- goombastic 10y agoWhy is it still slower than oneplus3 and htc 10?
- Grazester 10y agoSamsung bloat?
- mrfusion 10y agoHmm now I'm wondering if the brain does something like this? Might the brain need branch prediction? Interesting line of thought.
- sbashyal 10y agoI have a background in Machine Learning and it is my belief that the mind does use branch prediction. Take for example a scenario where you are climbing down a stair in your house. Your brain already predicts that you are about to land on the next step. Now if the step height was lowered than the usual, the prediction fails and you immediately focus back on the climbing-down process to see what has happened and prosecute whatever fall-back actions are needed to minimize disruption.
- deleted 10y ago[deleted]
- randomacct44 10y agoReminds me of what happens when you step onto an escalator that isn't working (the broken escalator phenomenon): https://en.wikipedia.org/wiki/Broken_escalator_phenomenon https://en.wikipedia.org/wiki/Broken_escalator_phenomenon
- argonaut 10y agoNitpick: having a background in machine learning is only slightly relevant to belief on what the mind does. It's having a background in neuroscience that's important.
- Sharlin 10y agoAn analogy is basically every time you subconsciously predict something, but something else happens. You get surprised and confused which is the brain's analog of flushing the pipeline and calibrating the predictor. Indeed, the execution of most mundane tasks only bubbles up to the conscious level when something unexpected happens and you need to update your model of the world. Disclaimer: as I said, this is an analogy. Brains are not literally executing instructions in a pipeline.
- Jerry2 10y agoEven with all those neural net advantages, dual-core, 2GB RAM iPhone 6s beats octa-core, 4GB RAM Galaxy Note 7: http://www.redmondpie.com/galaxy-note-7-vs-iphone-6s-real-world-speed-test/ http://www.redmondpie.com/galaxy-note-7-vs-iphone-6s-real-wo... Video (3:29 sec): https://www.youtube.com/watch?v=3-61FFoJFy0 https://www.youtube.com/watch?v=3-61FFoJFy0 PS: I highly recommend you check out the video... iPhone completely obliterated Note 7.
- the_mitsuhiko 10y agoThat's not too surprising. There is a lot more that runs on an Android phone usually.
- matt_wulfeck 10y agoWhat more is running on an android phone then other OS? Seems like they have some fat to trim if there's more required to run their phone.
- pjmlp 10y agoFrameworks based on Java with GC and JIT for each process?
- zigzigzag 10y agoEven today, Android and iOS are not comparable in terms of what they allow apps to do, which is why iOS tends to win responsiveness comparisons and Android (still) tends to win feature comparisons, even years after Apple fell behind the features race and started simply duplicating Android's featureset (multi-tasking, notifications drawer, gradually allowing more and more background operations etc).
- digi_owl 10y agoYeah my first thought went to how Android will background anything when you hit center button, while iOS will basically give it a few seconds to pack up state and then evict it. Been that way since day one, and has very little to do with the hardware.
- sehugg 10y agoI don't see why branching is so hard -- it's two cycles to test the branch, one more if the branch is taken, one more if it crosses a page boundary! /6502joke
- pawadu 10y agowell... for the size of its pipeline (1?) and the amount of speculative execution going on (0) the non-existing branch prediction of 6502 was basically optimal.
- Someone 10y agoThe 6502 has a tiny bit of pipeline/speculative execution. The CPU fetches the next instruction before the current one has finished. http://www.atarihq.com/danb/files/64doc.txt http://www.atarihq.com/danb/files/64doc.txt: "If an instruction does not store data in memory on its last cycle, the processor can fetch the opcode of the next instruction while executing the last cycle." The 6502 also sometimes does excess reads and writes. For example, "Read-modify-write instructions (like INC) read the original data, write it back, and then write the modified data." (http://forum.6502.org/viewtopic.php?t=2 http://forum.6502.org/viewtopic.php?t=2). One could call that speculative execution, where the processor writes back the value, just in case adding one to it doesn't increase it, and then, having discovered that adding one changes the value, writes the right value :-)
- phire 10y agoThe 6502 had no concept of not doing a memory operation. Either it's doing a read or it's doing a write, there is no option to use the bus unused. So the 6502 spams dummy reads and occasionally dummy writes whenever it's doing something internally. Edit: And that's not the only interesting thing I've learned recently about the 6502, There is a very good reason why the stack is on page 1 and not some other constant page. There are only 3 constant values which can be pushed on the SB/ADH buses, 0xff, 0x00 and 0x01. Ox00 is loaded into upper address register for zero page instructions. For Push operations, it loads 0x01 into the upper address register and puts the stack pointer + 0xff into the adder, which subtracts one from the stack pointer. (The 0xff is just the default bus value, if nothing else is pulling it low, so you get it for free) For Pop operations, it pushes the same 0x01 into both the upper address register and the adder, at the same time. So if they wanted to put the stack page at another constant page, or at a variable page they would also need to provide more gates to provide another constant one somewhere.
- ForFreedom 10y agoiPhone excel because the OS and HW is designed by one company unlike Samsung where the HW and OS are from different companies, add to android Samsung bloatware. The iOS works well with the iPhone simply because it is fine tuned for the product. On the androids side the OS is tuned to add bloatware of the respective HW manufactures. To add the so called speed the HW manufactures put in more RAM.
- gambiting 10y agoYou do realise that samsung designs and makes most of Apple stuff, right?
- eknkc 10y agoWhat parts do Apple get exclusively from Samsung and depend on their expertise these days? It seems like they go for redundancy, use two suppliers etc for common parts and chips. Which means these are built based on the designs of Apple. Samsung just manufactures them. I don't follow closely though so I might be wrong.
- pawadu 10y agoI think that was the plan after the court fight with Samsung. But after some quality problems (IIRC with LG displays) Apple had to give up the dual-source approach for at least some components. Now the original iphone was probably 80% Samsung design and manufacturing, but these days the design part is closer to 0%.
- AstralStorm 10y agoThis is not new, AMD has been using perceptron-based predictors probably even before Bulldozer. They are good, but harder to optimise for than standard rule based heuristic ones.
- sitkack 10y agoThey should expose the NN coefficients to the end user so that one can at least start from a known-better value. This problem would seem analogous to JIT warmup.
- pepijndevos 10y agoThe latter half of the article makes me wonder to what extend the cycle count of instructions is specified, or up to a specific implementation.