4 ms·

This is awesome. I can't seem to under why most blockchains insist on using leveldb as their transactional data store. In fact, the bitcoin devs have admitted s

This is awesome. I can't seem to under why most blockchains insist on using leveldb as their transactional data store. In fact, the bitcoin devs have admitted some of the attacks are specifically targeted towards the non ACID nature of leveldb. It would be so much better to use postgresql as the data store. But it seems there is a natural aversion to non-embedded databases.

eklitzke 9y ago
Why would it be better to use Postgres?
- Woki 9y ago
  It implements the SQL standard very well It includes support for "advanced" SQL stuff like window functions or common table expressions It is very innovative in terms of how plpgsql interacts with SQL It supports lots of advanced data types, such as (multi-dimensional) arrays, user-defined types, etc. These are pretty good reasons :)
  - pavlov 9y ago
    Which of those are relevant to implementing a blockchain node?
- sandGorgon 9y ago
  Two reasons : ACID and single node scalability. Postgresql is production tested with terabytes of Ram and disk usage on a single node. It has the jsonb column that can easily replicate the exact data model that you would use in leveldb. However there's another significant reason - devops if you are looking to do anything serious with blockchain. There is really no good way to build a meaningful production application on the blockchain without serious devops because of leveldb. What is funny is that everyone who wants to do something on the blockchain first pulls the data into postgresql and then runs something on it. Why have this problem in the first place? It just adds a few minutes to setup if you use postgresql instead of leveldb and get so so much more. And suddenly you can leverage all cloud hosted DB (rds, etc) and not have to focus on devops
  - dozzie 9y ago
    What is this "devops" you're talking about?
comboy 9y ago
You want your client to be standalone, you can't expect users to install and configure postgresql and sqlite probably isn't gonna cut it. If you want blockchain stats, postgres is great, but for a node you just need to be able to look stuff up by its keys. Leveldb was made for that and it's good at doing it. I don't know these attacks, but it seems you don't really need ACID that badly, most stuff is append only and once appended it never gets deleted.
- AnkhMorporkian 9y ago
  I'm curious, why wouldn't sqlite cut it? Granted, I'm not a blockchain afficiando, but I have several sqlite DBs that run a couple hundred gigabytes a piece that work quite well.
  - comboy 9y ago
    I said probably because I thought about it myself while writing that comment. I also have good experience with it, but if sqlite only talks SQL then just creating string representation of binaries would already be prohibitive.
    - ComputerGuru 9y ago
      > but if sqlite only talks SQL What does that mean? Like any RDBMS, SQLite is interfaced with in SQL, but can store and process binary data just fine.
      - comboy 9y ago
        I mean that you need to serialize binary just to make an SQL statement that includes it, and that's very expensive. E.g. postgresql's COPY supports binary format which allows you to avoid that.
        ComputerGuru 9y ago
        The fact that SQLite is embedded means it can take full advantage of pointer passing. I don't think it serializes anything when you bind parameters to a blob.

eklitzke 9y ago

Why would it be better to use Postgres?

Woki 9y ago
It implements the SQL standard very well It includes support for "advanced" SQL stuff like window functions or common table expressions It is very innovative in terms of how plpgsql interacts with SQL It supports lots of advanced data types, such as (multi-dimensional) arrays, user-defined types, etc. These are pretty good reasons :)
- pavlov 9y ago
  Which of those are relevant to implementing a blockchain node?
sandGorgon 9y ago
Two reasons : ACID and single node scalability. Postgresql is production tested with terabytes of Ram and disk usage on a single node. It has the jsonb column that can easily replicate the exact data model that you would use in leveldb. However there's another significant reason - devops if you are looking to do anything serious with blockchain. There is really no good way to build a meaningful production application on the blockchain without serious devops because of leveldb. What is funny is that everyone who wants to do something on the blockchain first pulls the data into postgresql and then runs something on it. Why have this problem in the first place? It just adds a few minutes to setup if you use postgresql instead of leveldb and get so so much more. And suddenly you can leverage all cloud hosted DB (rds, etc) and not have to focus on devops
- dozzie 9y ago
  What is this "devops" you're talking about?

comboy 9y ago

You want your client to be standalone, you can't expect users to install and configure postgresql and sqlite probably isn't gonna cut it. If you want blockchain stats, postgres is great, but for a node you just need to be able to look stuff up by its keys. Leveldb was made for that and it's good at doing it. I don't know these attacks, but it seems you don't really need ACID that badly, most stuff is append only and once appended it never gets deleted.

AnkhMorporkian 9y ago
I'm curious, why wouldn't sqlite cut it? Granted, I'm not a blockchain afficiando, but I have several sqlite DBs that run a couple hundred gigabytes a piece that work quite well.
- comboy 9y ago
  I said probably because I thought about it myself while writing that comment. I also have good experience with it, but if sqlite only talks SQL then just creating string representation of binaries would already be prohibitive.
  - ComputerGuru 9y ago
    > but if sqlite only talks SQL What does that mean? Like any RDBMS, SQLite is interfaced with in SQL, but can store and process binary data just fine.
    - comboy 9y ago
      I mean that you need to serialize binary just to make an SQL statement that includes it, and that's very expensive. E.g. postgresql's COPY supports binary format which allows you to avoid that.
      - ComputerGuru 9y ago
        The fact that SQLite is embedded means it can take full advantage of pointer passing. I don't think it serializes anything when you bind parameters to a blob.