7 ms·
Database of Databases
- deleted 6y ago[deleted]
- chickenpotpie 6y agoReminds me of my favorite Wikipedia article: https://en.wikipedia.org/wiki/List_of_lists_of_lists https://en.wikipedia.org/wiki/List_of_lists_of_lists
- fegu 6y agoMy god that is a rabbit hole. Thanks for this! (But there goes my evening..)
- mitchtbaum 6y agohttps://youtu.be/vvp0r1T5ers https://youtu.be/vvp0r1T5ers
- rsecora 6y agoThis article itself is a list of lists, so it contains itself.
- abnry 6y agoFound the Russell Paradox.
- danpalmer 6y agoWell I guess that answers it, SQLite is the database of all databases. > https://github.com/cmu-db/dbdb.io/blob/master/dbdb/settings.py#L81 https://github.com/cmu-db/dbdb.io/blob/master/dbdb/settings....
- samsquire 6y agoWriting a simple query engine can be accomplished with a trie and a hash map to be a bit like Dynamo DB and SQL is straightforward to parse (but a pain to implement). https://github.com/samsquire/hash-db https://github.com/samsquire/hash-db https://github.com/samsquire/sql-database https://github.com/samsquire/sql-database These are my very straightforward codebases where I've been experimenting with SQL parsing (and execution) and dynamo db style querying. You need an efficient range operator to implement a database.
- overcast 6y agoAre these websites just completely unable to handle a 1000 simultaneous requests?
- etxm 6y agoThey should have used a JSON file as the backend.
- polote 6y agoThis is more in the range of 10s of requests per second at this position on HN
- mimischi 6y agoSomeone should make a database of caches or CDNs. Then combine knowledge! /s
- touisteur 6y agoThere you go https://www.splitgraph.com https://www.splitgraph.com
- samsquire 6y agoThey're using SQLite which is single threaded and single user by design. So this website will not be able to service that much traffic.
- tyingq 6y agoShould be fine for reads. Sqlite.org is dynamic, pulling from sqlite data for ~20% of the pages, and it does fine with HN piling on. The single threaded would be an issue for writes, but I don't see why they would be doing writes for a page view. See https://www.sqlite.org/whentouse.html https://www.sqlite.org/whentouse.html
- pfraze 6y agoAlso, caching layers. Presumably a service uses SQLite to simplify their ops. So long as the caching layer is equally simple to maintain, then SQLite continues to make sense to me
- tyingq 6y agoTheir repo shows the "DummyCache" (no caching) as the default for the Python/Django setup: https://github.com/cmu-db/dbdb.io/blob/master/dbdb/settings.py#L103 https://github.com/cmu-db/dbdb.io/blob/master/dbdb/settings.... I wonder if that's how it is in production.
- kevincox 6y agoFor a mostly static site you would want to cache "over" Django, probably in a caching proxy or CDN. Of course there are a lot of details, such as many CDNs will always go to the origin on a edge miss instead of locating another copy in the CDN. Of course running a caching nginx on the same box as the Django is probably way more performant than caching "under" Django
- tyingq 6y agoIt's intermittently working now. Shows the server as Apache 2.4.18 / Ubuntu. Apache can page cache, but I assume they aren't using it. I don't see any typical CDN headers either. Also, somewhat odd, they are bounding the browser-side cache to 10 minutes: Date: Wed, 16 Sep 2020 18:19:13 GMT Expires: Wed, 16 Sep 2020 18:29:14 GMT
- cocktailpeanuts 6y agoThought it was some groundbreaking new database technology.... Speaking of, What would an interesting version of "database of databases" look like?
- tyingq 6y agoThe closest I can think of would be the "Data Governance" space. Tools like Collibra: https://www.collibra.com/ https://www.collibra.com/ Screenshot that makes it easier to understand: https://www.collibra.com/wp-content/uploads/Blog-DataLineageDiagram-820x350.jpg https://www.collibra.com/wp-content/uploads/Blog-DataLineage...
- indymike 6y agoApparently SQLite. Who would have thought?
- LukeEF 6y agoTerminusHub is a database filled with databases (for collaboration): https://terminusdb.com/hub/ https://terminusdb.com/hub/
- ralusek 6y agoProbably CosmosDB or MarkLogic.
- chatmasta 6y agoThis is basically what we're building at Splitgraph [0]. We're calling it a "data delivery network." You connect to one SQL endpoint and can query (and join across) 40k+ different datasets. It's built on Postgres, and as far as your SQL client is concerned, it's talking to a Postgres database with 40k tables in it. Right now we forward queries to public data portals, but eventually you'll be able to connect live data sources to the DDN without writing any code. We want it to be as easy as configuring Cloudflare; you just upload a set of read-only credentials in the web UI and we take care of the rest. For more private use cases, we're planning to offer private deployments to AWS/GCP/Azure. Technically, this is database virtualization, which isn't really a new concept. We're implementing it as a database proxy, using PgBouncer instances to intercept queries and route them to Splitgraph engines. Within a Splitgraph engine (which is Postgres + some custom code), each "table" is either a "mounted" live database via a foreign database wrapper (FDW), or part of a point-in-time, versioned database snapshot called a "data image" that you can build with sgr. [0] https://www.splitgraph.com https://www.splitgraph.com
- hackbinary 6y agoThis article reminded me of this site: http://howfuckedismydatabase.com/ http://howfuckedismydatabase.com/
- damsdu78 6y agoI don't understand the goal of this site. :|
- eshyong 6y agoIt's satire written by a disgruntled engineer :P
- ssijak 6y agoAmazon affiliate links
- hackbinary 6y agoWhat's not to understand? The answers for access and Oracle are both brilliant and true.
- fizixer 6y agoAny good learning resources for implementing your own DBMS, and DB internals? (not DB theory, not SQL, not intro).
- macintux 6y agoYou might find useful guidance here: http://www.aosabook.org/en/ http://www.aosabook.org/en/
- otoolep 6y agohttps://github.com/rqlite/rqlite https://github.com/rqlite/rqlite https://github.com/rqlite/rqlite/blob/master/DOC/DESIGN.md https://github.com/rqlite/rqlite/blob/master/DOC/DESIGN.md I built a distributed database using Raft and SQLite, and I put a lot of thought into its design and implementation -- to make it as simple (but not simplistic) and clear as possible. One goal was clear separation between the consensus layer, the database access, and the HTTP API. I think it's a good resource to study (even if I do say so myself).
- gangulo 6y agoCMU DB Group has all their lectures on youtube. Checkout 15445 and 15721 https://www.youtube.com/channel/UCHnBsf2rH-K7pn09rb3qvkA https://www.youtube.com/channel/UCHnBsf2rH-K7pn09rb3qvkA
- cgh 6y agoYes, the Postgresql source and associated docs. It's a well-structured code base that is easy enough to dive into, if you don't try to understand the whole thing at once. I'm slowly reading it in bits and pieces. Here's a Github mirror so you can browse around: https://github.com/postgres/postgres https://github.com/postgres/postgres Note that the offical repo is at git.postgresql.org.
- agambrahma 6y agoShould mention the newer OReilly "Database Internals" book here: https://www.oreilly.com/library/view/database-internals/9781492040330/ https://www.oreilly.com/library/view/database-internals/9781...
- 6y ago
- hackbinary 6y agoAlso reminded me of the awesome list of awesome lists: https://github.com/sindresorhus/awesome https://github.com/sindresorhus/awesome
- altrunox 6y agoDamn, I don't think I've read about most of the site top database... Maybe I should try to use something different than PostgreSQL, at least in some side projects.
- llacb47 6y agoHN Hug of death.
- colonelpopcorn 6y agoDid anyone else pronounce this like dibby-dibby-yo?
- brummm 6y agoBut does it include itself?
- punnerud 6y agoArchived page (5days ago): https://web.archive.org/web/20200911140514/https://dbdb.io/ https://web.archive.org/web/20200911140514/https://dbdb.io/
- bird_monster 6y agoThe amount of databases I'd like to play with and explore greatly outweighs the amount of time or reasons I have to explore databases.
- joshdance 6y agoWebsite is down.
- asplake 6y agoSo this isn’t in fact instances of databases inside databases? Disappointed! Surely possible with SQLite, right? Has it been done?
- throwaway894345 6y agoI was really hoping it was a SQL interface for querying data stored in multiple databases; something like Presto.
- nerpderp82 6y agoThe lulz thing about the site is that you can't query it with SQL. Let me download the sqlite database directly! Andy?
- josefrichter 6y agoSo this is being reposted over and over again. Not saying it's not useful, but still..
- thrownaway954 6y agopretty cool. learned alot of new databases beyond the big 6. this site could benefit greatly from not running on a database and being statically generated. even the browse section could just be a vuejs app powered by a json collection.
- maxcx 6y agoupvote!
- deleted 6y ago[deleted]
- freeopinion 6y agohttps://dbdb.io/browse?foreign-keys=supported&query-interface=sql&stored-procedures=supported https://dbdb.io/browse?foreign-keys=supported&query-interfac... x Supports Foreign Keys x Supports Stored Procedures x Supports SQL Query Interface No Postgresql?