11 ms·
Full Ethereum blockchain now available as a BigQuery public dataset
- r32a_ 8y agoThis is why Bitcoin blockchain is more concerned about keeping the block sizes small and efficient as possible than adding fancy smart contract features that just add bloat to the blockchain.
- bouncycastle 8y agoEthereum's block sizes are quite small, about 25kb per block right now. Actually, block sizes are not limited by bytes like in bitcoin, but by 'gas'. This gas limit can be dynamically adjusted by miner votes, and the way the incentives work, it keeps the blocks not too big, but also not too small. One feature of Ethereum is that it automatically discourages mining centralization using the 'uncle rewards' system. When the blocks increase in size (and thus put pressure on centralization as you noted), the uncle rate increases too, which is undesirable for miner profits. If the uncle rate gets too high, miner's interest is to vote down the block 'gas' limit, this ensures all the blocks can prpagate around the network fairly.
- ghthor 8y agoI didn't even know this. That's a great example of how interesting you can get with the incentives system, makes me so excited for the future.
- r32a_ 8y agocompare the size of the bitcoin blockchain and the ethereum blockchain. Now see when Bitcoin was launched and see when Ethereum was launched. Facts speak for themselves.
- ca98am79 8y agohaha, the size of the Ethereum blockchain is making it centralized. Less and less nodes can download and keep up with it.
- otoburb 8y agoFewer full nodes can download and keep up. But light weight ethereum clients[1] shouldn’t have an issue. [1] https://github.com/ethereum/wiki/wiki/Light-client-protocol https://github.com/ethereum/wiki/wiki/Light-client-protocol
- thinkmassive 8y agoPeople running light clients should also be running their own full nodes, otherwise the shortage of peers will only get worse https://github.com/ethereum/go-ethereum/issues/15454 https://github.com/ethereum/go-ethereum/issues/15454
- shazow 8y agoUltimately, all full nodes will also be light clients--that is the nature of the upcoming sharding changes. It does not make sense to have a full node for every light client. A full node is perfectly capable of safely serving hundreds or thousands of light clients. Also there are several initiatives towards incentivizing more light clients. I am working on one: https://vipnode.org/ https://vipnode.org/
- thinkmassive 8y agoI wasn't implying a full node for every light client, of course that doesn't make sense. People using light clients in their products, like a wallet or DApp, are probably adding tens to thousands of light clients. When you add that load to the network, you should also deploy a few full nodes, and probably add them as static peers if you want the application to work well.
- thinkmassive 8y ago> Also there are several initiatives towards incentivizing more light clients. I am working on one: https://vipnode.org/ https://vipnode.org/ This looks cool, I may have to give it a try!
- Barrin92 8y ago
- karp773 8y agoReads like a good April Fools' Day joke.
- srcreigh 8y agoThe public access to application data showcased here seems to strengthen my suspicion that Ethereum will spawn an era of machine learning innovation. It seems analogous to what happened with TCP/IP: the former lets everyone connect 1-1, while the latter lets everyone access and analyze entire application datasets. Of course it is possible to provide universal access to an application dataset with TCP/IP, but this is costly (effort, money) and incentives don't always align; organizations of today often work to prevent access to application datasets. The ethereum future, on other hand, is exciting: organizations having an incentive to share application datasets, making money with innovative analysis / applications of the datasets.
- jameslevy 8y agoThe incentive you speak of to share application datasets - is that a Layer 2 application on Ethereum where a token would be used for accessing data? Intriguing idea but not sure I completely follow what you are describing.
- shazow 8y agoIn case anyone is curious about the size of the Ethereum blockchain (which the linked article doesn't even mention but seems to inevitably come up in comments), I took some measurements recently: https://twitter.com/shazow/status/1004506114392838146 https://twitter.com/shazow/status/1004506114392838146 tl;dr: ~65GB of bandwidth is required to download all the necessary data. The default node indexing and denormalization of this data takes around 100GB after compacting.
- TTPrograms 8y agoFor practical interaction with the network this can be drastically reduced by using light sync, of course.
- CobrastanJorji 8y agoI don't follow cryptocurrency news, but the example query of cost transferred sums is interesting. What the heck happened around July 2nd?
- aldoushuxley001 8y agoA bunch of hungover Canadians?
- TTPrograms 8y agoThere was a Chinese decentralized exchange that (foolishly) decided they would include ERC20 coins for trading according to which had the most transfers from unique accounts to the exchange (I believe?). This incentivised individuals who wanted their coins included to distribute them among as many sock puppet accounts as possible and transfer them separately to the exchange. The resulting transaction traffic had a non-negligible impact on network capacity. In cases of load like that (or many individuals with strong incentives for their transactions to get in) people pay higher fees to increase the probability that their transaction is included in the next block.
- wslh 8y agoBut the problem is that using BigQuery is not cheap even when it is public.
- swerveonem 8y agoAnd you still have to implement your own "verify against actual blockchain node" function yourself before actually trusting any query results.
- fastball 8y agoIs the purpose not more for analysis than validation?
- zone411 8y agoIt's not that difficult, just a bit annoying, to get the full Ethereum and Bitcoin blockchains running locally and create a database-type access. You will probably spend more time waiting for the download to finish than actually writing code. Bitcoin is a little harder than Ethereum because of the format changes and that might require using an older version (at least that's what I did).
- Drdrdrq 8y agoIt's not trivial, that's for sure. And it is not download times that kill you, it is recomputing that is really slow, and you need it if you want full history. You really want to have SSDs. The export itself is quite fast though.
- buhrmi 8y agoWonder how they decide whether or not a contract is ERC20 or ERC721 compliant to set the `is_erc20` and `is_erc721` values in the `contracts` table.
- Drdrdrq 8y agoProbably by checking if contract supports the mandatory functions. (I did some eth blockchain analytics a while ago)