3 ms·
The exciting thing about GFS wasn't even the technology. It was the discovery you made as a new engineer starting at Google (in the 2000s) that from your works
by bborud 4y ago
The exciting thing about GFS wasn't even the technology. It was the discovery you made as a new engineer starting at Google (in the 2000s) that from your workstation (and from any job you ran on Borg) you had access to this enormous amount of data stored on GFS across different data centers.
This is the thing that people keep missing. The technology was only the enabler. What made the difference was to have thousands of developers with wide access to the biggest data lake in the world.
Almost nobody gets that when trying to make data lakes, or whatever they are called now, in the corporate world. They focus on vendors and technologies and whatnot. Then ignore that what makes data valuable is access to it by people who don't even know yet what they might do with it.
- Razengan 4y ago> data lakes, or whatever they are called now Clouds? Databergs? Smaug-hoards? No wait that's for anime collectors
- fh973 4y agoPlus all data is stored in the same format (protobufs in recordio or sstables), so was very easy to access.
- lazide 4y agoWith accessible schemas (due to single repo) and fast (performance wise). Of course now folks just complain that their job is shoveling protos haha.
- ethbr0 4y agoAbsolutely! IMHO, new engineers start with "What technology should we use to do this?", while experienced engineers start with "Assume we've already done this successfully with some (i.e. any) technology, what will the impact be?" And yes, if you have a hierarchical eng org then there's different job responsibilities at different levels. But at some point, new engineers end up making their own calls, and they usually don't pay enough attention to the forest. Some technically complex things just aren't worth doing, and some simple things are incredibly valuable to do. E.g. Porting the fundamentals of an MMU + append optimizations into a distributed storage system: not theoretically groundbreaking. But the impact for its users was!
- joekrill 4y agoHow does that work when it comes to private information like emails, search histories, etc?
- lazide 4y agoIt’s been a decade since open access like that was a thing at Google. Some high profile incidents made the company lock it down. It’s probably one of the most locked down environments now outside of a SCIF.
- deleted 4y ago[deleted]
- zaphar 4y agoYou encrypt or mask, usually mask, that data out of the data lake.
- jrockway 4y agoThe standard technique, at least when I was there, is to encrypt with a per-user public key whose decryption key is only present when your application has the user's login session. Basically, your application can't decrypt the data unless the user is making a request to your app. (There are a lot of potential attacks here, like modifying the code to exfiltrate data when the user is present. Hopefully those are easier to detect than just putting the data in a database unencrypted, though.) I worked on Google Fiber. We sent all the system logs from our WiFi routers (etc.) back to us, mostly to get to the bottom of the most obscure WiFi bugs. This could have been a privacy disaster, as Linux logs all sorts of PII like the MAC addresses of devices that are connected to the router. My team modified the Linux kernel to not do this, and added filtering before upload to XX anything that looked like a MAC address. That way, when some PM came up with a horrifyingly bad idea like "hey, let's make a social network based on who connected to your WiFi", that data didn't exist. (This predated widespread implementation of MAC address randomization. We took care with your PII, but nobody else did, so phone vendors simply stopped giving access points PII. Probably the right solution, honestly.) This wasn't just us that did things like this, it was institutionalized. There was a privacy team that had to review and sign off on this sort of thing. If you wanted to run something in production, or retain any sort of data, someone that actually cared about user privacy would be taking a look. HN kind of undervalues how much people actually cared; there is some trade off between usability and security, and nobody is going to use a product that can't be used, and no corporation can really stand up to the government the way they want. I thought it was a decent balance, though. Obviously, some things can't be redacted like we did, and you should be careful about who you share those things with. Location history comes to mind as the most obviously abused feature. I thought it was super cool to have "you visited this cafe 2 years ago" on Google Maps, but it comes at the cost of fishy warrants for "anyone within 1 mile of this minor property crime". I'd rather not be on those lists, so I don't use it. (But be careful, your cell phone provider surely has this information, even if the Vice article is about Google. They just don't use it for anything useful; it's ONLY a dragnet.)
- yanivbh1 4y agoCan’t agree more, but is the value of the data itself is important for the engineer like the analyst/executive?
- bborud 4y agoNot quite sure what you are asking. Could you clarify?
- bushbaba 4y agoThe ability to join datasets at google through BQ and 'Flink' like pipelines was really un-matched. No matter what dataset you wanted it was accessible and joinable using the same data-lake, query tools, and ACL system. Sure teams differed in how they queried it (tool wise), but GFS made it all readily accessible for joining. Most large orgs miss this when they make their data lake. Instead making a collection of silo'd data lakes
- joshuamorton 4y agoI am continually amazed by the accessibility and power of the query infrastructure, some of the queries I have written and seen are downright magic.