5 ms·

Show HN: Pg_blkchain – Postgres blockchain extension

75 points by gtrubetskoy 9y ago

remcob 9y ago
From the name and the title, I was hoping it would be a prototype blockchain based distributed consensus extensions for Postgres. I have to admit that I was a little bit disappointed to discover that it is a collection of transaction parsing functions for the Bitcoin blockchain. Nevertheless, cool project! Definitely something I should have used a year ago instead of the slow cludge of Python scripts I build!
- gtrubetskoy 9y ago
  Apologies, the title should have contained POC. I am not sure what you mean by "blockchain based distributed consensus" - did you mean the the ability to group transactions into blocks and being able to verify those? It's possible, and that's the plan, it is, however, very complicated to build something that you can actually trust like the bitcoin core. I think that a better use for this sort of thing is the ability to analyze the blockchain. But something that would provide "full node" functionality I think is interesting as well. There would still need to be some sort of a program running in front of it to do all the network communication... Anyway - the point of this little project is to really get some feedback on the general approach and then see where to take it next. :)
  - NegativeLatency 9y ago
    Sounds like they were thinking of some kind of way of running Postgres on a blockchain, vs using Postgres to process the blockchain.
- misterdata 9y ago
  So something like Catena (https://github.com/pixelspark/catena https://github.com/pixelspark/catena, I wrote this)? It does SQL based on Transactions stored in a Blockchain (and coincidentally speaks the postgres pq client protocol).
  - gtrubetskoy 9y ago
    Cool stuff, interesting choice of language - are you not concerned that this limits you to mostly OSX?
    - unkown-unknowns 9y ago
      Readme has instructions for building on Linux so it's not limited to OS X.
- devrandomguy 9y ago
  I thought that it was going to log transactions to a blockchain, as a sabotage-resistant backup strategy.
sandGorgon 9y ago
This is awesome. I can't seem to under why most blockchains insist on using leveldb as their transactional data store. In fact, the bitcoin devs have admitted some of the attacks are specifically targeted towards the non ACID nature of leveldb. It would be so much better to use postgresql as the data store. But it seems there is a natural aversion to non-embedded databases.
- eklitzke 9y ago
  Why would it be better to use Postgres?
  - Woki 9y ago
    It implements the SQL standard very well It includes support for "advanced" SQL stuff like window functions or common table expressions It is very innovative in terms of how plpgsql interacts with SQL It supports lots of advanced data types, such as (multi-dimensional) arrays, user-defined types, etc. These are pretty good reasons :)
    - pavlov 9y ago
      Which of those are relevant to implementing a blockchain node?
  - sandGorgon 9y ago
    Two reasons : ACID and single node scalability. Postgresql is production tested with terabytes of Ram and disk usage on a single node. It has the jsonb column that can easily replicate the exact data model that you would use in leveldb. However there's another significant reason - devops if you are looking to do anything serious with blockchain. There is really no good way to build a meaningful production application on the blockchain without serious devops because of leveldb. What is funny is that everyone who wants to do something on the blockchain first pulls the data into postgresql and then runs something on it. Why have this problem in the first place? It just adds a few minutes to setup if you use postgresql instead of leveldb and get so so much more. And suddenly you can leverage all cloud hosted DB (rds, etc) and not have to focus on devops
0xCMP 9y ago
Can someone explain what this is for or could be used for? It seems like it allows you to store, process, and verify the blockchain inside postgres, right?
- rickycook 9y ago
  from my reading, the best use case i could think of would be to use it to have foreign keys to transactions and the like in your app database so that you can ensure their integrity
dna_polymerase 9y ago
Please rename this. This is not a pg-based blockchain or anything like that, just a Bitcoin extension to pg. And guess what Bitcoin is not the only blockchain out there...
- gtrubetskoy 9y ago
  It is a PostgreSQL Blockchain Extension, i.e.: a postgres extension [1] which allows one to process a blockchain [2]. The bitcoin blockchain is the first invented blockchain as we know it, all the other blockchains are borrowing the concepts from the bitcoin blockchain, which is why this extension, at least in its present initial form, is designed to work with the bitcoin blockchain. I stand by the name, it is exactly correct. [1] https://www.postgresql.org/docs/current/static/external-extensions.html https://www.postgresql.org/docs/current/static/external-exte... [2] "The first distributed blockchain was conceptualised by an anonymous person or group known as Satoshi Nakamoto" from https://en.wikipedia.org/wiki/Blockchain https://en.wikipedia.org/wiki/Blockchain
  - nosuchthing 9y ago
    Just because an Oldsmobile is one of the first mass produced automobiles in existence, doesn't mean anyone looking for a car would want an Oldsmobile. In regards to database technology, Bitcoin just pieced together technologies that already existed, i.e. hash cash, merkle trees, and distributed databases. It's worth acknowledging while Bitcoin was a great prototype for implementation of a blockchain, it's quiet stagnate and divided community and development teams has led to the current crisis of hard forks. Meanwhile other blockchain projects are iterating and developing user-friendly features.
epx 9y ago
A colleague was hard-pressing me about which advantages could Postgres have in relation to M(ySQL|ariaDB), these extensions (not this particular extension, but the concept of) seems to be a definitive advantage.
- nawgszy 9y ago
  Your comment only had one takeaway for me: len('M(ySQL|ariaDB)') > len('MySQL/MariaDB')
  - epx 9y ago
    As people say, once you turn to REs to solve a problem, you have two problems :)
ssutch3 9y ago
For a good C-bitcoin library you can use breadwallet-core: https://github.com/breadwallet/breadwallet-core https://github.com/breadwallet/breadwallet-core

75 points by gtrubetskoy 9y ago

From the name and the title, I was hoping it would be a prototype blockchain based distributed consensus extensions for Postgres. I have to admit that I was a little bit disappointed to discover that it is a collection of transaction parsing functions for the Bitcoin blockchain. Nevertheless, cool project! Definitely something I should have used a year ago instead of the slow cludge of Python scripts I build!

gtrubetskoy 9y ago
Apologies, the title should have contained POC. I am not sure what you mean by "blockchain based distributed consensus" - did you mean the the ability to group transactions into blocks and being able to verify those? It's possible, and that's the plan, it is, however, very complicated to build something that you can actually trust like the bitcoin core. I think that a better use for this sort of thing is the ability to analyze the blockchain. But something that would provide "full node" functionality I think is interesting as well. There would still need to be some sort of a program running in front of it to do all the network communication... Anyway - the point of this little project is to really get some feedback on the general approach and then see where to take it next. :)
- NegativeLatency 9y ago
  Sounds like they were thinking of some kind of way of running Postgres on a blockchain, vs using Postgres to process the blockchain.
misterdata 9y ago
So something like Catena (https://github.com/pixelspark/catena https://github.com/pixelspark/catena, I wrote this)? It does SQL based on Transactions stored in a Blockchain (and coincidentally speaks the postgres pq client protocol).
- gtrubetskoy 9y ago
  Cool stuff, interesting choice of language - are you not concerned that this limits you to mostly OSX?
  - unkown-unknowns 9y ago
    Readme has instructions for building on Linux so it's not limited to OS X.
devrandomguy 9y ago
I thought that it was going to log transactions to a blockchain, as a sabotage-resistant backup strategy.

sandGorgon 9y ago

This is awesome. I can't seem to under why most blockchains insist on using leveldb as their transactional data store. In fact, the bitcoin devs have admitted some of the attacks are specifically targeted towards the non ACID nature of leveldb. It would be so much better to use postgresql as the data store. But it seems there is a natural aversion to non-embedded databases.

eklitzke 9y ago
Why would it be better to use Postgres?
- Woki 9y ago
  It implements the SQL standard very well It includes support for "advanced" SQL stuff like window functions or common table expressions It is very innovative in terms of how plpgsql interacts with SQL It supports lots of advanced data types, such as (multi-dimensional) arrays, user-defined types, etc. These are pretty good reasons :)
  - pavlov 9y ago
    Which of those are relevant to implementing a blockchain node?
- sandGorgon 9y ago
  Two reasons : ACID and single node scalability. Postgresql is production tested with terabytes of Ram and disk usage on a single node. It has the jsonb column that can easily replicate the exact data model that you would use in leveldb. However there's another significant reason - devops if you are looking to do anything serious with blockchain. There is really no good way to build a meaningful production application on the blockchain without serious devops because of leveldb. What is funny is that everyone who wants to do something on the blockchain first pulls the data into postgresql and then runs something on it. Why have this problem in the first place? It just adds a few minutes to setup if you use postgresql instead of leveldb and get so so much more. And suddenly you can leverage all cloud hosted DB (rds, etc) and not have to focus on devops

0xCMP 9y ago

Can someone explain what this is for or could be used for? It seems like it allows you to store, process, and verify the blockchain inside postgres, right?

rickycook 9y ago
from my reading, the best use case i could think of would be to use it to have foreign keys to transactions and the like in your app database so that you can ensure their integrity

dna_polymerase 9y ago

Please rename this. This is not a pg-based blockchain or anything like that, just a Bitcoin extension to pg. And guess what Bitcoin is not the only blockchain out there...

gtrubetskoy 9y ago
It is a PostgreSQL Blockchain Extension, i.e.: a postgres extension [1] which allows one to process a blockchain [2]. The bitcoin blockchain is the first invented blockchain as we know it, all the other blockchains are borrowing the concepts from the bitcoin blockchain, which is why this extension, at least in its present initial form, is designed to work with the bitcoin blockchain. I stand by the name, it is exactly correct. [1] https://www.postgresql.org/docs/current/static/external-extensions.html https://www.postgresql.org/docs/current/static/external-exte... [2] "The first distributed blockchain was conceptualised by an anonymous person or group known as Satoshi Nakamoto" from https://en.wikipedia.org/wiki/Blockchain https://en.wikipedia.org/wiki/Blockchain
- nosuchthing 9y ago
  Just because an Oldsmobile is one of the first mass produced automobiles in existence, doesn't mean anyone looking for a car would want an Oldsmobile. In regards to database technology, Bitcoin just pieced together technologies that already existed, i.e. hash cash, merkle trees, and distributed databases. It's worth acknowledging while Bitcoin was a great prototype for implementation of a blockchain, it's quiet stagnate and divided community and development teams has led to the current crisis of hard forks. Meanwhile other blockchain projects are iterating and developing user-friendly features.

epx 9y ago

A colleague was hard-pressing me about which advantages could Postgres have in relation to M(ySQL|ariaDB), these extensions (not this particular extension, but the concept of) seems to be a definitive advantage.

nawgszy 9y ago
Your comment only had one takeaway for me: len('M(ySQL|ariaDB)') > len('MySQL/MariaDB')
- epx 9y ago
  As people say, once you turn to REs to solve a problem, you have two problems :)

ssutch3 9y ago

For a good C-bitcoin library you can use breadwallet-core: https://github.com/breadwallet/breadwallet-core https://github.com/breadwallet/breadwallet-core