6 ms·
Harnessing chaos in Cloudflare offices
- cojo 3y agoIt hadn't occurred to me that even a photo with the lens cap on still contains decent entropy, although it now seems somewhat obvious to me in hindsight after thinking about it. I really enjoyed how this article covered a variety of different "hacker-spirit" things; real-world entropy into "digital-world" meaningful use cases, plus a whole extra "one more thing" timelock encryption example at the end. As someone who uses Cloudflare Workers / Pages heavily these days whenever I can, it's quite fun to see both "how the sausage is made" as well as the culture (playfulness?) behind it. Kinda makes me want to go visit the Austin office since I'm local. Kudos and thanks to the Cloudflare team for writing stuff like this up! One of the more enjoyable tech pieces I've read in the past couple of weeks, and I learned multiple things along the way.
- jareklupinski 3y ago> a photo with the lens cap on still contains decent entropy https://en.wikipedia.org/wiki/Johnson%E2%80%93Nyquist_noise https://en.wikipedia.org/wiki/Johnson%E2%80%93Nyquist_noise spending so much time devising ways to mitigate or remove noise, it becomes difficult to think of it as a 'feature'...
- flumpcakes 3y agoI worked on research that built machine learning models to take that 'entropy' or sensor pattern noise and to match it against photographs, to trace image lineage when EXIF and similar are stripped out. For a practical application: as you can imagine, there are certain crimes where it really makes a difference if an image is just on a phone, or if it was verifiably taken by that phone. Possession-of vs. Production-of...
- jjeaff 3y agoThat's interesting. But, assuming the research found it possible to verify which device a photo came from based on the sensor noise, doesn't that kind of go against the idea that there is a lot of entropy in sensor pattern noise?
- OJFord 3y agoNo, because you could have it that the device is always identifiable, but nevertheless producing a randomly varying sequence of images. Take printers for example, with some known to print a signature. Clearly they can still print a sheet a solid colour (or a number 0-100, or something character or whatever) at random (given some random source & control for doing so I mean) despite the device being identifiable.
- pclmulqdq 3y agoI spend a lot of time thinking about randomness, and after running some tests on the entropy of dark images, I have started to believe that there is a lot less entropy in dark CCD images than people think, but there is still enough to get a useful entropy stream. A substantial portion of the "noise" from a CCD is definitely not random.
- code_biologist 3y agoI'm curious, how close to raw CCD data did you get from consumer cameras? It wouldn't surprise me if hard-wired camera internal postprocessing often almost immediately regularizes random noise, even with raw images and software postprocessing turned off. Just a wild-ass guess though.
- pclmulqdq 3y agoI didn't use consumer cameras to test this, and I assume cloudflare doesn't either.
- code_biologist 3y agoI was annoyed at the caginess of your answer until I saw you ran a randomness-as-a-service company and these are low grade trade secrets.
- everforward 3y agoIf I were going to take a stab at this, I would guess that most of the "camera" is really unnecessary and that you could do this using just the image sensor. A lot of the camera is just functionality to make actual pictures better that don't apply here. Eg you don't need to control exposure with shutter speed if it's in a black box. Having a whole camera might even be counterproductive. Eg actuating the shutter is predictable, so it might reduce entropy if actuating the shutter creates a signal that shows up in the randomness. Or maybe they just mean pro quality cameras, but I'm not sure why you'd want a whole camera instead of just the sensor. Reasons are not readily apparent, and I don't expect anyone to be immediately ready to correct me on trade secrets.
- KennyBlanken 3y agoThe sensor contains decent entropy, assuming you're actually getting raw samples out of the sensor, and your sensor isn't being affected by EMF either in the air or via its power supply. Noise reduction algorithms are going to affect the entropy if you can't get raw values, and the level of AI crap in cell phone cameras makes it even worse these days.
- paxys 3y agoI wonder what the added effectiveness is of such elaborate setups vs an off-the-shelf HRNG dongle plugged into the server (which I assume is what every other company with such requirements uses). Are the lava lamps and pendulums actually that much more functional or just marketing?
- mcherm 3y agoThe use of lava lamps as a source of randomness is, to use your term, "just marketing" -- it is not fundamentally more secure than other sources of randomness. The use of a group of trusted randomness generators, a majority of whom would have to collude in order to trick a consumer into thinking an input was random when it was actually staged, offers genuine functionality that cannot be dismissed as "just marketing".
- yellow_lead 3y agoIt's also the type of fun/cool marketing that potentially opens up new attack vectors
- lukevalenta 3y agoAs long as it's done correctly, mixing new entropy sources into an entropy pool will never _decrease_ the entropy. So in the case of LavaRand, even if it only ever returned a string of zeros, systems that mix it's output into their entropy pools wouldn't be any worse off than before. Perhaps we could have made this point more clearly in the post. (I'm one of the authors.)
- akira2501 3y agoSo, if your source of random data does not reduce entropy to the pool, but generating random numbers with it does reduce entropy from the pool, along a long enough time line, aren't you going to deplete the entropy anyways?
- 3y ago
- creeble 3y agoInteresting that they use a wall of pendulums, and even more interesting that it’s in London: https://www.forbes.com/sites/kevinknudson/2015/07/29/pendulum-pendulum-on-the-wall/ https://www.forbes.com/sites/kevinknudson/2015/07/29/pendulu... And Https://physicsworld.com/a/the-secret-of-the-synchronized-pendulums/ Still, probably plenty of entropy. But a weird choice if you know the context?
- chzblck 3y agowhat if we dont know the context? what should we googs?
- jgrahamc 3y agoThere are two links in the comment. They provide the context. The point being made is that pendulums connected to some support structure can exhibit synchornization caused by transmission of energy across the support. We have not observed synchronization in London which may be due to the chaotic nature of the double pendulums and/or the fact that they are very light compared to the structure they are on.
- yencabulator 3y agoThey're also on relatively long stalks, which makes the shared support not very rigid, and the individual stalks are seemingly random heights, which might desynchronize the effect.
- kypro 3y ago> Visible to visitors Right, so they've gone to all this effort to generate input as close as possible to random, then they undermine it by opening it up to the public? Would be interesting to see what the camera placement is like and if that's at least secure, otherwise someone's is just going to stick a still picture on the end of it... If you're going to go to all this effort to create randomness, at least secure it from physical interference... Sorry, I know I'm being a cynical here. This just feels like a marketing gimmick.
- jgrahamc 3y agoThe cameras themselves aren't accessible to visitors.
- deleted 3y ago[deleted]
- aarreedd 3y agoThe whole point is that it's chaotic. Knowing the the approximate current state cannot be used to predict future states. https://en.wikipedia.org/wiki/Chaos_theory https://en.wikipedia.org/wiki/Chaos_theory
- kypro 3y ago> Knowing the the approximate current state cannot be used to predict future states The reason these entropy sources are used is because there's no such thing as a perfectly random algorithm. If there's a way to remove the entropy from the system then the whole thing becomes pointless, and you may as well go back to using a pseudorandom algorithm. That's my point. If you care this much about ensuring true randomness then I'd argue the security of the system should be a primary consideration – perhaps the primary consideration. If you can't guarantee that you're entropy source is random, then you can't be confident of the randomness of the system generally. I'm not an expert on this though so if someone wants to explain why I'm wrong then please do so.
- jedberg 3y agoYou're getting downvoted for your cynicism, but your threat assessment is correct. It's possible for someone to put a still photo in front of the lens. That being said, the randomness on the sensor alone would probably defeat that, but also you could just check to make sure the previous image and the current image don't have the same hash, which I suspect Cloudflare does just as a basic error check.
- e28eta 3y agoI wouldn’t consider an image to be sufficiently random, because you don’t have a uniform probability of any specific value: the walls aren’t changing colors, I don’t see any blue lava lamps, camera probably spits out an image with some compression, it has predictable changes through the day if the lighting changes, etc So… I think it’s relying on a cryptographically secure hashing function to become random enough to rely on, correct?
- extraduder_ire 3y agoSomewhat related: https://www.random.org/faq/ https://www.random.org/faq/ Random.org generates random numbers by collecting static hiss from untuned radios around the world, selling them if they are required in bulk.
- senderista 3y agoLove the shout-out to "The Snail and the Whale"--it's one of our family's favorite books! (As is anything else by Julia Donaldson.)
- ThinkBeat 3y agoI dont have math or statistics knowledge. How hard is it to prove true randomness? My guess is that it is either incredibly difficult or impossible. Or at least one would need an enormous number of samples to start the analysis.
- charcircuit 3y agoIt's impossible as the seed can always be bigger.
- IncreasePosts 3y agoYou can never prove it absolutely just from the output, because there is always a possibility that a true random number generator would generate some non-random looking output, like a run of one thousand ones.
- pclmulqdq 3y agoI run a randomness company. It's impossible to prove it 100%. What you can do instead is run code audits, analyze the theoretical basis of the entropy source, and test large amounts of data for its statistical properties. That can get you to near certainty, but it's still empirical.
- ThinkBeat 3y agoThat is interesting. Do you offer randomness as a service? People contact your API to retrieve random data? or do you sell software / hardware to create random data?
- pclmulqdq 3y agoBoth, depending on the desires of customers, but it's currently leaning towards high-end hardware/software. A broad-market randomness-as-a-service offering is coming this month.
- charcircuit 3y agoThis is overengineered. We do not need that much entropy and entropy is easy to collect from within the chips themselves.
- notatoad 3y agolol of course it's overengineered. but it's also cool, and that's okay sometimes.
- deanCommie 3y agoThis is cool and an awesome tech demo and probably a great recruting technique, but... Is it actually practically/meaningfully more "Secure" than the standard PseudoRandom approaches that everyone relies on today? Short of "Nation States sharing secrets of alien technology", is this actually significant to any customer's practical applications?
- new23d 3y agoNo.
- lukevalenta 3y agoThe piece that helps security-wise is that we're mixing in entropy from a trusted external source, so not solely relying on the local random number generation from a machine in a data center somewhere. Is it likely that local random number generation would be compromised? No. But it does give us a little extra peace of mind.
- pclmulqdq 3y agoThis is more of a defense in depth for the paranoid, but cryptographic PRNGs (and even hardware RNGs) can be compromised in ways that are not easy to find. Since they generate your keys, a compromise of the RNG chain is very valuable for a threat actor.
- deanCommie 3y agoSo, no.