3 ms·

Joran from TigerBeetle here! Really stoked to see a bit of Jim Gray history on the front page and happy to dive into how TB's consensus and storage engine imple

Joran from TigerBeetle here! Really stoked to see a bit of Jim Gray history on the front page and happy to dive into how TB's consensus and storage engine implements these ideas (starting from main! https://github.com/tigerbeetle/tigerbeetle/blob/main/src/tigerbeetle/main.zig https://github.com/tigerbeetle/tigerbeetle/blob/main/src/tig...).

nickpeterson 2y ago
I’m curious if you ever came across any of the writings of Toon Koppelaars and the idea of ThickDB/SmartDB, it seems very in alignment.
- jorangreef 2y ago
  Thanks, I hadn't. Looking into it, I think Toon is asking the question: Should the DBMS expose the raw storage operations (insert, update, delete) for the application to take the burden of correctness for their physical composition into logical functionality? Or should the DBMS export that logical functionality? With TigerBeetle, we didn't want people to have to keep cobbling together their transaction processing system, to take that burden of correctness, but to solve this in open source, once and for all, for everyone to build around.
westurner 2y ago
TPS: Transactions Per Second: https://en.wikipedia.org/wiki/Transactions_per_second https://en.wikipedia.org/wiki/Transactions_per_second "A Measure of Transaction Processing Power" (1985) "A measure of transaction processing 20 years later" (2005) https://arxiv.org/abs/cs/0701162 https://arxiv.org/abs/cs/0701162 .. https://scholar.google.com/scholar?cluster=11019087883708435479&hl=en&as_sdt=5,43&sciodt=0,43 https://scholar.google.com/scholar?cluster=11019087883708435... About the cover sheet on those TPS reports. Max throughput, max efficiency, Network throughput: https://en.wikipedia.org/wiki/Network_throughput https://en.wikipedia.org/wiki/Network_throughput "Is measuring blockchain transactions per second (TPS) stupid in 2024? Big Questions" https://cointelegraph.com/magazine/blockchain-transactions-per-second-tps-stupid-big-questions/ https://cointelegraph.com/magazine/blockchain-transactions-p... : > focusing on the raw TPS number is a bit like “counting the number of bills in your wallet but ignoring that some are singles, some are twenties, and some are hundreds.” https://chainspect.app/dashboard https://chainspect.app/dashboard describes each of their metrics: Real-Time TPS (tx/s), Max Recorded TPS (tx/s), Max Theoretical TPS (tx/s), Block Time (s), Finality (s) USD/day probably doesn't predict TPS; because the Average transaction value is higher on networks with low TPS. Other metrics: FLOPS, FLOPS/WHr, TOPS, TOPS/WHr, $/OPS/WHr And then there's Uptime; or losses due to downtime (given SLA prorated costs)
- jorangreef 2y ago
  > "A measure of transaction processing 20 years later (2005)" This is one of my favorites. Along with https://www.microsoft.com/en-us/research/wp-content/uploads/2005/04/tr-2005-39.doc https://www.microsoft.com/en-us/research/wp-content/uploads/... (also 20 years later) which has more detail. > USD/day probably doesn't predict TPS; because the Average transaction value is higher on networks with low TPS. Exactly. If we only look at USD/day we might not see the trend in transaction volume. What's happened is like a TV set going from B&W to full color 4K. If you look at the dimensions of the TV, it's pretty much the same. But the number of pixels (txns) is increasing, with their size (value) decreasing, for significantly higher resolution across sectors. And this is directly valuable because higher resolution (i.e. transactionality) enables arbitrage/efficiency.
  - westurner 2y ago
    I don't know what the frontier is on TPS and value in financial systems. Does HFT or market-making really add that much value compared to direct capital investment and the trading frequency of an LTSE Long Term Stock Exchange? High tx fees (which are simply burnt) disincentive HFT, which may also be a waste of electricity like PoW, in terms of allocation. Low tx fees increase the likelihood of arbitrage that reduces the spread between prices (on multiple exchanges) and then what effect on volatility and real value? But those are market dynamics, not general high-TPS system dynamics.
    - jorangreef 2y ago
      You can see the trend better in the world of instant payments, if you look to India's UPI or Brazil's PIX, how they've taken over from cash and are disrupting credit card transactions, with this starting to spread to Europe and the US: https://www.npci.org.in/what-we-do/upi/product-statistics https://www.npci.org.in/what-we-do/upi/product-statistics However, it's not limited to fintech. For example, in some countries, energy used to be transacted once a month. Someone would come to your house or business, read your meter, and send you a bill for your energy usage. With the world moving away from coal to clean energy and solar, this is changing because the price of energy now follows the sun. So energy providers are starting to price energy, not once a month, but every 30 minutes. In other words, this 1440x increase in transactions volume enables cleaner more efficient energy pricing. You see the same trend also in content and streaming. You used to buy an album, then a song, and now you stream on Spotify and Apple. Same for movies and Netflix. The cloud is no different. Moving from colocation, to dedicated, to per-hour and then per-minute instance pricing, and now serverless. And then Uber and Airbnb have done much the same for car rentals or house rentals: take an existing business model and make it more transactional.

nickpeterson 2y ago

I’m curious if you ever came across any of the writings of Toon Koppelaars and the idea of ThickDB/SmartDB, it seems very in alignment.

jorangreef 2y ago
Thanks, I hadn't. Looking into it, I think Toon is asking the question: Should the DBMS expose the raw storage operations (insert, update, delete) for the application to take the burden of correctness for their physical composition into logical functionality? Or should the DBMS export that logical functionality? With TigerBeetle, we didn't want people to have to keep cobbling together their transaction processing system, to take that burden of correctness, but to solve this in open source, once and for all, for everyone to build around.

westurner 2y ago

TPS: Transactions Per Second: https://en.wikipedia.org/wiki/Transactions_per_second https://en.wikipedia.org/wiki/Transactions_per_second "A Measure of Transaction Processing Power" (1985) "A measure of transaction processing 20 years later" (2005) https://arxiv.org/abs/cs/0701162 https://arxiv.org/abs/cs/0701162 .. https://scholar.google.com/scholar?cluster=11019087883708435479&hl=en&as_sdt=5,43&sciodt=0,43 https://scholar.google.com/scholar?cluster=11019087883708435... About the cover sheet on those TPS reports. Max throughput, max efficiency, Network throughput: https://en.wikipedia.org/wiki/Network_throughput https://en.wikipedia.org/wiki/Network_throughput "Is measuring blockchain transactions per second (TPS) stupid in 2024? Big Questions" https://cointelegraph.com/magazine/blockchain-transactions-per-second-tps-stupid-big-questions/ https://cointelegraph.com/magazine/blockchain-transactions-p... : > focusing on the raw TPS number is a bit like “counting the number of bills in your wallet but ignoring that some are singles, some are twenties, and some are hundreds.” https://chainspect.app/dashboard https://chainspect.app/dashboard describes each of their metrics: Real-Time TPS (tx/s), Max Recorded TPS (tx/s), Max Theoretical TPS (tx/s), Block Time (s), Finality (s) USD/day probably doesn't predict TPS; because the Average transaction value is higher on networks with low TPS. Other metrics: FLOPS, FLOPS/WHr, TOPS, TOPS/WHr, $/OPS/WHr And then there's Uptime; or losses due to downtime (given SLA prorated costs)

jorangreef 2y ago
> "A measure of transaction processing 20 years later (2005)" This is one of my favorites. Along with https://www.microsoft.com/en-us/research/wp-content/uploads/2005/04/tr-2005-39.doc https://www.microsoft.com/en-us/research/wp-content/uploads/... (also 20 years later) which has more detail. > USD/day probably doesn't predict TPS; because the Average transaction value is higher on networks with low TPS. Exactly. If we only look at USD/day we might not see the trend in transaction volume. What's happened is like a TV set going from B&W to full color 4K. If you look at the dimensions of the TV, it's pretty much the same. But the number of pixels (txns) is increasing, with their size (value) decreasing, for significantly higher resolution across sectors. And this is directly valuable because higher resolution (i.e. transactionality) enables arbitrage/efficiency.
- westurner 2y ago
  I don't know what the frontier is on TPS and value in financial systems. Does HFT or market-making really add that much value compared to direct capital investment and the trading frequency of an LTSE Long Term Stock Exchange? High tx fees (which are simply burnt) disincentive HFT, which may also be a waste of electricity like PoW, in terms of allocation. Low tx fees increase the likelihood of arbitrage that reduces the spread between prices (on multiple exchanges) and then what effect on volatility and real value? But those are market dynamics, not general high-TPS system dynamics.
  - jorangreef 2y ago
    You can see the trend better in the world of instant payments, if you look to India's UPI or Brazil's PIX, how they've taken over from cash and are disrupting credit card transactions, with this starting to spread to Europe and the US: https://www.npci.org.in/what-we-do/upi/product-statistics https://www.npci.org.in/what-we-do/upi/product-statistics However, it's not limited to fintech. For example, in some countries, energy used to be transacted once a month. Someone would come to your house or business, read your meter, and send you a bill for your energy usage. With the world moving away from coal to clean energy and solar, this is changing because the price of energy now follows the sun. So energy providers are starting to price energy, not once a month, but every 30 minutes. In other words, this 1440x increase in transactions volume enables cleaner more efficient energy pricing. You see the same trend also in content and streaming. You used to buy an album, then a song, and now you stream on Spotify and Apple. Same for movies and Netflix. The cloud is no different. Moving from colocation, to dedicated, to per-hour and then per-minute instance pricing, and now serverless. And then Uber and Airbnb have done much the same for car rentals or house rentals: take an existing business model and make it more transactional.