14 ms·
Amazon Quantum Ledger Database
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- wmf 8y agoHuge props to AWS for being honest and not blockchain-washing.
- sprt 8y agoMeet Amazon Managed Blockchain: https://aws.amazon.com/managed-blockchain/ https://aws.amazon.com/managed-blockchain/
- chrisco255 8y agoHa yeah it looks like Amazon is taking the grapeshot approach and doing it all.
- nixpulvis 8y agoYea, huge props to a magacorp looking for new ways to make money. <clap>
- m3kw9 8y agoIf centralized, why not just restrict the API to make history immutable? Lol
- pepr 8y agoThat doesn't give you the ability to cryptographically verify integrity of the database. Think git repository with pgp signature on every commit.
- m3kw9 8y agoI think once centralized, there is inherently something wrong with blockchains. There is always chance of bait and switch, even though you trust amazon.
- jonknee 8y agoWhen the largest cloud provider takes time to create and market a new product it's probably worth thinking about for more than a "lol".
- m3kw9 8y agoThis I’m that’s why I’m asking
- NathanKP 8y agoAWS employee here. QLDB has been used internally at AWS for quite some time. We decided to turn our internal service into an external facing product because we saw a demand for a ledger system with central authority. Many people were trying to force Blockchain to accomplish this, while we already had an internal ledger solution that is far more scalable while still being cryptographically verifiable.
- almstimplmntd 8y agoSo is “Quantum“ pure buzzword hype? Or is there a reasonable connection to the product?
- vvilliam0 8y agoQuantum just means something that can be counted or measured, usually in discrete amounts like whole number. Assuming the audits and entries are discrete and transactional, the word makes sense. But yeah, it just looks like they are trying to go for marketing buzzwords.
- reikonomusha 8y agoIn the context of computing technology, “quantum” is not the appropriate term, and I only agree it makes sense with the most generous interpretation of the term, whose meaning you describe finds itself mostly in certain physical theories. They could have used the word “discrete”, which is overwhelmingly more contextually accurate, if that’s what they mean.
- xkjkls 8y agoEh, it's a branding thing. I doubt many customers are confused.
- CiPHPerCoder 8y agoI predict many confused/rushed tech journalists will write articles that mention "Amazon already has a Quantum Computing Product" after this.
- adrianmonk 8y agoI don't see why it wouldn't be appropriate. Within the context of computing, there is no one set meaning of "quantum". In fact, there are at least three: 1. Quantum computing, as in quantum physics. 2. Time quantum, as in the time slices a multitasking scheduler gives to threads. 3. Quantum Corporation, the disk/tape storage company. Seems like there is room for one more meaning. They probably mean "quantum" as in "quantum leap" since this is a different paradigm for ledgers. It makes sense to me.
- deleted 8y ago[deleted]
- udpkts 8y agoThis is what I use protected git repo with a pgp signature for.
- ajkjk 8y agoWhat throughput do you get with that?
- DanielBMarkham 8y agoIs there a need for high throughput in some domain?
- ajkjk 8y agoYes? Name any domain and there's somebody who wants high-throughput auditable transactions in it.
- DanielBMarkham 8y agoFrom a centralized source? So sure. Voting. Wouldn't it be great to have an auditable trail? But why would you then go and put all of that in one spot? Wouldn't a better approach be reconfiguring the tech to work in a faster fashion?
- ajkjk 8y agoWhy wouldn't you put it all in one spot? The technical requirement of needing an audit table is ubiquitous and I bet almost every SQL database in history eventually ends up having a history table of some sort. This technical requirement is unrelated to any regulatory requirements about (not) storing data in the cloud, and certainly shouldn't be coupled to it.
- EGreg 8y agoI also don’t get it. Auditable trail? Use a Merkle DAG. Or something like this Permissionless Timestamping Network: https://intercoin.pdf/whitepaper.pdf https://intercoin.pdf/whitepaper.pdf I mean how hard is it? What are the real challenges?
- cloudhead 8y agoAs they pointed out, this requires me to trust Amazon. But since that's the case, what's the point of it being auditable? Amazon can easily drop the last N transactions on your ledger and tell you the digest you provided was "not found".
- jedberg 8y agoThey could, but why would they?
- cloudhead 8y agoIndeed, they wouldn't - but the product offering sounds like they are telling customers that they can audit their data and that it is cryptographically verifiable - but why does the customer care about this, unless they don't trust amazon to preserve data integrity?
- Filligree 8y ago"Trust, but verify."
- cheeze 8y agoAmazon could leak your secrets stored in Secrets Manager, or KMS. ACM could leak your private keys. EC2 hypervisor could have a 0day that allows China to steal all your data. CloudTrail could drop audit logs. At some point, you have to trust the company who's software you're using. Sure Amazon could easily drop the last N transactions, but what motivation do they have to do so? What behavior in the past makes you think this wuold happen?
- cloudhead 8y agoCompletely agree - what I'm wondering is why they are telling their customers that they can verify the data for themselves, when that is entirely pointless: "and using cryptography, you can easily verify that there have been no unintended modifications to your application’s data"
- CiPHPerCoder 8y agoDid they just ship Trillian, or is this an AWS-exclusive design? The webpage says they use SHA256, but it doesn't go much more into detail than that.
- cheeze 8y agoAWS-Exclusive from my understanding. Not in AWS but from reading other comments, it sounds like this has been widely used internally for some time.
- NathanKP 8y agoCorrect QLDB has been widely used within AWS: https://twitter.com/timbray/status/1067836869021327361 https://twitter.com/timbray/status/1067836869021327361 https://twitter.com/colmmacc/status/1067832198059970561 https://twitter.com/colmmacc/status/1067832198059970561
- politician 8y agoDo you know if the QLDB write API is more like an event store or more like a relational database? It sounds like the read API provides both models (read the journal, or read the snapshot).
- jonhohle 8y agoIt should work for both cases. It's an append-only ledger that captures any mutations on records previously on the ledger. It uses a recent snapshot primarily for efficiency to avoid having to read the entire ledger to enforce any transactional or schema constraints. So it's an event store in that each write transaction is an event. But it's a relational DB in that it (at least the internal version did) schemas, multi-record updates, and transactions.
- politician 8y agoThat's an understandable implementation, but a bit disappointing. It means that there's not going to be a simple way to process the journal since that log will consists of differently sized writes to the database. It would have been more interesting if QLDB was a CQRS+EventStore with built-in snapshots.
- dschuetz 8y agoAnother Amazon Ad?
- bpicolo 8y agoThe AWS conference is going on: https://reinvent.awsevents.com/ https://reinvent.awsevents.com/
- dschuetz 8y agoSo?
- codezero 8y agoSo a lot of people who work in that ecosystem are interested in the new announcements and as can be seen from a lot of the stories on HN, a lot of people are nerding out over them. This is kind of what HN is for.
- dang 8y agoPlease stop now.
- ralmeida 8y agoLots of announcements of cloud-related tech, which HN users find interesting.
- david-cako 8y agohttps://dilbert.com/strip/2018-11-26 https://dilbert.com/strip/2018-11-26
- ajkjk 8y agoI'm pretty sure this is what most people actually want when they say they want a blockchain, they just don't know it yet. It's got all the useful database features with none of the "well uh let's distribute it over people's computers and let people mine coins to support it" complexity, which adds nothing to 99% of usecases besides silly ICO potential.
- magnamerc 8y agoHow can you make something like MakerDAO on a centralized ledger?
- ajkjk 8y agoHey, I only said 99%.
- magnamerc 8y agoTrue, and I agree that most ICOs are just money grabs trying to sell 'blockchain' for services that don't need one. But decentralized ledgers offer possibilities that weren't possible before, and I suspect that in the next ten years these applications will start to surface.
- Filligree 8y agoI have no need for what MakerDAO offers. Do you?
- magnamerc 8y agoYes, I do. I trade derivatives without the need of a broker. I can also lend DAI to anyone in the world through a smart contract.
- lawrenceg 8y agoIt's still too early for most people to understand how amazing the concept around MakerDAO and DAI actually is. They will, but it will take a bit of time.
- jadiofan 8y agoI wonder if this is just based on Kafka with some native tamper-proof functionality added to it?
- mr_tristan 8y agoI had a similar thought - this is probably built on Kinesis or Kafka with additional crypto layer
- silentsea90 8y agoNewbie here. Questions: 1) Can somebody go over use cases and target customers, where one would prefer this over more traditional DBs. 2) Since this is centralized, it is in theory possible to go back to a previous state of the ledger. Is that blocked simply because the product doesn't support rolling back to a previous state?
- snissn 8y ago> build ecommerce site > have an inventory list and supply chain > want good audits of what happened when built into your database
- tlholaday 8y ago1: Use case. Traditional bookkeeping. Asset account balances start at zero initially, then increase with every debit and decrease with every credit. If someone mis-reads a check and enters a deposit as $200 instead of as $20, we do not change the $200 to $20. Instead, we do one of two things: reverse the difference with a $180 credit, or reverse the deposit with a $200 credit followed by a new $20 debit. Traditional DBs support all three approaches: reverse and repost, reverse differences, change the original. If you need to know why the original was changed, in a traditional DB you need an additional audit trail. Implementers can neglect to implement a n audit table. With QLDB, there is no risk that the implementers will choose to update the original deposit, because the database is write-once.
- Cieplak 8y agoIs it possible to comply with GDPR while using this to store data? Given that it operates like an append-only log, is it possible to actually remove data to comply with a GDPR request?
- philjohn 8y agoSeeing as it's immutable, it would appear not. Which is why you use it for specialised use cases and keep any PII out of there. It would be possible to replay into a new ledger, filtering out the pieces of data to be deleted, but that goes against having an immutable log in the first place.
- riffraff 8y agoyou probably would want to store the reserved data in a separate store, and just reference them from the immutable one.
- dangoor 8y agoYou can use cryptoshredding: have an encryption key for each user (stored outside of this ledger) and encrypt all PII with that key. Throw away the key if the user wants you to delete their data.
- viach 8y agoWhat if the key leaked before you have thrown it away?
- dangoor 8y agoThat's a good question! If your keys leaked, you'd probably have to assume you lost all of the data up to that point. To secure the data going forward, you'd need to generate a second key per user for all of the future data. Well, and hopefully shore up the security problems! I agree, though, that an immutable ledger like this complicates things in a way that you-shouldn't-mutate-but-can datastores do not.
- 8y ago
- tsuru 8y agoAm I right in believing this looks like a competitor to Datomic?
- Royalaid 8y agoI too am getting that vibe
- anonetal 8y agoDatomic doesn't have cryptographically-guaranteed immutability -- its logical data model is immutable and so you can query the past versions of data etc., but there isn't anything stopping one from altering the history. In a ledger like this, history cannot be modified (assuming reasonable computational limits). Datomic also has much richer data model and query languages.
- znpy 8y agoUhm... If some of my data is stored in one instance of that database and I then ask the owning company to delete such data in accordance to , say, GDPR... What happens ?
- deleted 8y ago[deleted]
- Scarbutt 8y agoYou delete it. You think there won't be a "I'm pretty sure I want to delete this?" option? you probably have to explicitly go out of your way to do it since the point is not to delete data. There's probably also an option for not keeping history for data that it doesn't make sense to have history for.
- bilate 8y agoIt seems to me the main reason people don't go for such a Blockchain is because they don't 'trust' Amazon. Fair enough - why not then have a blockchain owned by several different competing companies with huge data centers: Google, Amazon, IBM etc. This solves the computation and scalability issue with blockchain while also making it less likely one company gets too powerful (any new transaction will need to be confirmed by all the parties (Amazon, Google) providing a self regulating mechanism).
- wmf 8y agoMulti-company operation is the holy grail of private blockchain projects, but it's also too complex for people to manage.
- iblaine 8y agoQLDB = Quantum Ledger Database Seeing 'quantum' in that acronym is so cringeworthy. I'd prefer Immutable Ledger Database, or a word that shares more with the technology than media hyperbole.
- mabbo 8y ago> Adjective - quantum > Of a change, sudden or discrete, without intermediate stages. - https://en.wiktionary.org/wiki/quantum#Adjective https://en.wiktionary.org/wiki/quantum#Adjective It makes perfect sense if you know what the word means.
- Old_Thrashbarg 8y agoYes, but given the trend of quantum computers, it's confusing using that word freely. Case in point, the whole reason I clicked through on this was becausei thought it might be some earlier quantum database.
- joelg 8y agoCool, but I don't think I could get away with marketing a raspberry pi as a "quantum computer" because its bits change suddenly, discretely, and without intermediate steps. Like it or not, "quantum" has accumulated a) more connotations that make it less likely to be interpreted as "sudden or discrete" by most people, and b) an association with other pseudosciency buzzwords that don't have a very good reputation in product names.
- sqrt17 8y ago... it also makes perfect sense as a ruse if you figure all those buzzword-chasing QuantumComputinAIDeepLearningBlockchainOfThings will be flocking around something that is not just useful but also buzzword compatible. IBM has spent a bit of money drumming up all things Quantum (while being at most moderately more successful at actual useful QC implementation), it would be ironic if some of the buzz (in people who think Quantum is the "next big thing" without knowing how actual current QC would benefit them) now transfers to something useful and real but not-QC
- 8y ago
- sbhn 8y agoDoes quantum mean its thread safe?
- discodave 8y agoThe word quantum doesn't mean thread-safe. But it is thread-safe, since it's fundamentally based on a durable ACID Transaction journal. All the threads in your application can agree on state.
- code4tee 8y agoAWS is spot on here. Stop using blockchain as a database if you just want the cryptographic verification and immutability of assets et al. That’s lieterally all 95% of corporate use cases for blockchain wanted all along. Stop the hype and insanity and focus on a tool that actually does what you need.
- StreamBright 8y agoWow, this is exactly the sort of database I was dreaming about for a logging solution. I need to deep dive into pricing though.
- miguelmota 8y agoNot sure why there's so much hate on here. The article explicitly mentions that the database is Owned by a central trusted authority. This is distributed ledger technology within the AWS ecosystem, not decentralized blockchain technology
- onecool 8y agoWhy all of the Amazon spam? Three advertisements in 6 hours.
- trhway 8y agoUnder the guise of "immutable ledger" (basically just a honest scout promise by Amazon) Amazon gets into industrial provenance business. Genius. No sarcasm. Like that satellite ground station business yesterday. Basically "own the platform of the future" strategy the same way like spinning up the AWS 10 years ago. While companies like Google struggle to find any new business, Bezos just "printing" it non-stop (does he have a separate area in his brain labeled "Bezos X" ?).
- suddenstutter 8y agoThid is good stuff
- allnightowl 8y agoLooks like these guys were on a very similar path. https://m.youtube.com/watch?feature=youtu.be&v=3T2H4qO9_b0 https://m.youtube.com/watch?feature=youtu.be&v=3T2H4qO9_b0
- ineedasername 8y agoWhy are they using the word "quantum" here? With the advent of actual quantum computing this will just become increasingly confusing.
- nixpulvis 8y agoCame here to make the same complaint. Quantum science / computing is already confusing enough without things arbitrarily named "Quantum X". I wouldn't really be complaining if I didn't think Quantum Databases will probably be (are?) a thing.
- sixdimensional 8y agoThis reminds me of temporal database [1] concepts. I've been thinking that besides the hashing aspect of it, otherwise, it's actually basically the same as a temporal database, isn't it? I do like that they referred to it as a database rather than blockchain, as it more accurately captures the fact that it is a data store and I always felt like blockchain was more a reference to the way that the data is cryptographically linked together in the data store. [1] https://en.wikipedia.org/wiki/Temporal_database https://en.wikipedia.org/wiki/Temporal_database
- pmart123 8y agoI'm no expert, but there definitely are similarities. You could design a temporal table where a change in a customer's email always generates a new row versus an update, or you could just be capturing a snapshot of time-series data at a point in time. A ledger database is much more like double-entry bookkeeping, or git, where you are capturing each change.
- abrookewood 8y agoDo any immutable open source databases exist?
- kondro 8y agoNot good ones because everyone wants to solve the distributed store, proof-of-work & contract-as-software problems rather than the primary thing that makes blockchain useful for normal businesses.
- devxpy 8y agohttps://github.com/bigchaindb/bigchaindb https://github.com/bigchaindb/bigchaindb
- jillesvangurp 8y agoInteresting, I actually implemented a simple immutable ledger on top of mysql this year to track balance mutations of our InToken coin. The coin exists on Stellar as well but we track federated stellar accounts in our database for most of our users. This means that we represent them with a single stellar account. We allow users that clear our aml procedure to send/receive tokens to stellar. Having our own ledger means we can cheaply do internal transfers, micro rewards, and other incentives. Doing the bookkeeping correctly and in a tamper proof way is of course important for us, which is why we built our own ledger database to ensure we don't end up with corrupted data (either through bugs or malicious activity). I use content hashes as the id that include the id of the previous ledger entry and the key data stored in each row. The core design is pretty easy (it's basically a linked list) but the devil is in the details with this stuff since you indeed need to worry about auditing and making sure you don't end up losing transactions. Additional headaches include dealing with concurrent transactions and the fact that mysql does not do serializable transactions (at least not in sane way). Each row should only have 1 successor meaning that every new row involves looking up the previous row, and using it's id as a parent id. So we have a select and an insert happening in a transaction. We have a simple db constraint enforcing the parent is referred only once. We retry transactions when this constraint gets violated. This does actually happen when two concurrent transactions decide to use the same parent id. If transactions were serializable, this would not happen and the second transaction would end up using the id of the first. Another of the gotchas is that data migrations are kind of hard/impossible in an immutable data store. The only way to do it would be to effectively recreate a new database with new content hashes. So there are some things that I'd like to change that I can't actually change because it would break the content hashes. But by and large, this design is working quite well for us so far. So, in short, it's not rocket science but hard enough that having a well supported product that does this is worth having. We're actually considering open sourcing it at some point since it seems there are quite many projects out there that use a ledger primarily to have some tamper resistant immutable and auditable log of transactions.
- jacques_chester 8y agoGiven your problem with serializability, why not use a database which supports it better than MySQL?
- Purefan 8y agoOne thing I havent seen mentioned in the comments is talk about auditing challenges, how would journal auditing go? are there already tools for this? is it just another "table"?
- jaakl 8y agoThe promise and all the overhead of real distributed blockchains is that you should avoid any need to trust anyone, and if you're really paranoid then you should not trust even your own database copy. All this could be hacked just as Amazon can. Power of real distributed ledger is that as long as less than 50% of the data copies (consensus) are not hacked, then database as whole can be trusted. Now if you can hack also consensus, then even blockchain is hackable. With the "centralized ledger" you don't have the consensus protection level, therefore I see only marginal (if any) security improvement on top of just plain database with properly applied access control and auditing. You still have way more parties you need to blindly trust.
- godelmachine 8y agoDoes the name suggest it has Quantum Computing + Distributed Ledger?
- blessingmuseki 8y agoAmazon QLDB does solves some of the QDPR compliance shortfalls of blockchain, for example the physical location of the data could be guaranteed. But it doesn't solve the big one: the right to full erasure, also known as the right to be forgotten. The tactics suggested by other users here, such as 'losing' the key, do not fully qualify as full erasure, just pseudo-erasure. So far, to the best of my knowledge, nobody has as yet developed a GDPR-complaint, public-facing DAO application. It's a grey area. I am working on a blockchain wallet that allows contractors and gig economy workers to request, own and administer their own employment references. The main problem we're facing is to guarantee full erasure, and to guarantee the physical domicility of data (for example,in the EU region). I was excited about QLDB but I see many problems in its current state.
- cyphunk 8y agoThis doesn’t explain how amazon has removed capability for themselves to mitm the whole process. Or perhaps I missed that part somewhere?