7 ms·

From a fun / teaching / learning point of view - I get it. As others are saying however, the secondary learning that you want is that sometimes a database is t

From a fun / teaching / learning point of view - I get it. As others are saying however, the secondary learning that you want is that sometimes a database is the right solution. For example, a student could learn that they need to use a API (with all of the API problems: latency, server down, throttling, blah blah blah) when all they really need to do is download a sqlite file, hook up sqlite to their application, run one query - and have a vastly simpler application. This could be as little as 10-50 lines of code. Never underestimate the power of shipping data via sqlite files.

alexchamberlain 4y ago
Files/dumps as an API is vastly underrated.
strogonoff 4y ago
Considering a set of relatively infrequently changing facts about external world, publishing them as RDF can be a good choice. Developers can use it in whatever way they need, including loading it into SQLite—and it’s self-documenting, so they don’t need some extra information such as a spec to explain what everything means. It can link to external data for things beyond its scope that someone else already took care of. Other data can link to it, too—so it benefits open information exchange. (Yes, RDF has got a bit of a bad rap in some circles, but I don’t think it’s entirely justified… More importantly, RDF is incredibly boring, so publishing such a dataset is likely to gather much less attention than releasing a free public API, at least here.)
medellin 4y ago
For this use case that works well but a big part of that learning is the data is fairly static and likely doesn’t update often
deleted 4y ago
[deleted]
Ao7bei3s 4y ago
I agree, but also don't underestimate the problems and risks in real systems that keep changing. I've shipped some data in a JSON file bundled with an application, because it didn't change a lot, and we had frequent (biweekly) releases anyway. I mistakenly assumed those two things would always be true. As time went on, the frequency of necessary changes increased but our overall products release cadence got worse and worse (many months) as the product grew. So customers had continually outdated data. An additional problem I didn't expect is that since we wanted to get customers the latest data (because the next release was so far out), we had to update the data set as close to release as possible. So always a change at the last minute, which is already a busy time and means it's not really tested well. And also it was extra toil; we had a script for data generation but it still needed a manual run and pull request. In the time I spent running this I could have easily written an API based version several times. Another problem was that this change had to be made for every release, including bugfix ones. This caused frequent discussion (do we need this?), was forgotten a few times, and caused merge conflicts later which were annoying because in this case the proper way to merge was to ignore both ancestors and rerun the script, which people weren't used to. Of course I rewrote it eventually (after convincing PM that I need time for this supposedly long done feature again), but then I had to deal with the extra problem of customers who had local customer specific modifications of the file (because that was the workaround our support staff came up with as a way to deal with the outdatedness). Thankfully, a problem I did not habe was binary data in git. The commits were ugly but at least still got ok diffs (once we sorted the JSON file properly). But you mentioned SQLite, which is not a good match for git. Of course one can work around it by merging the raw SQL, but still, yet another pitfall.

alexchamberlain 4y ago

Files/dumps as an API is vastly underrated.

strogonoff 4y ago

Considering a set of relatively infrequently changing facts about external world, publishing them as RDF can be a good choice. Developers can use it in whatever way they need, including loading it into SQLite—and it’s self-documenting, so they don’t need some extra information such as a spec to explain what everything means. It can link to external data for things beyond its scope that someone else already took care of. Other data can link to it, too—so it benefits open information exchange. (Yes, RDF has got a bit of a bad rap in some circles, but I don’t think it’s entirely justified… More importantly, RDF is incredibly boring, so publishing such a dataset is likely to gather much less attention than releasing a free public API, at least here.)

medellin 4y ago

For this use case that works well but a big part of that learning is the data is fairly static and likely doesn’t update often

deleted 4y ago

[deleted]

Ao7bei3s 4y ago

I agree, but also don't underestimate the problems and risks in real systems that keep changing. I've shipped some data in a JSON file bundled with an application, because it didn't change a lot, and we had frequent (biweekly) releases anyway. I mistakenly assumed those two things would always be true. As time went on, the frequency of necessary changes increased but our overall products release cadence got worse and worse (many months) as the product grew. So customers had continually outdated data. An additional problem I didn't expect is that since we wanted to get customers the latest data (because the next release was so far out), we had to update the data set as close to release as possible. So always a change at the last minute, which is already a busy time and means it's not really tested well. And also it was extra toil; we had a script for data generation but it still needed a manual run and pull request. In the time I spent running this I could have easily written an API based version several times. Another problem was that this change had to be made for every release, including bugfix ones. This caused frequent discussion (do we need this?), was forgotten a few times, and caused merge conflicts later which were annoying because in this case the proper way to merge was to ignore both ancestors and rerun the script, which people weren't used to. Of course I rewrote it eventually (after convincing PM that I need time for this supposedly long done feature again), but then I had to deal with the extra problem of customers who had local customer specific modifications of the file (because that was the workaround our support staff came up with as a way to deal with the outdatedness). Thankfully, a problem I did not habe was binary data in git. The commits were ugly but at least still got ok diffs (once we sorted the JSON file properly). But you mentioned SQLite, which is not a good match for git. Of course one can work around it by merging the raw SQL, but still, yet another pitfall.