3 ms·
Good question. The answer is that it depends on how frequently the data changes. Some data might change often, some might not. If the data falls into both ca
by gwu78 13y ago
Good question. The answer is that it depends on how frequently the data changes. Some data might change often, some might not. If the data falls into both categories, there will be a trade off if we treat all data as "dynamic" or all as "static" over a definite period. What might make more sense is to separate these two categories of data.
API calls (piecemeal data retrieval) makes better sense for "dynamic" data that changes frequently. Whereas periodically downloading a data dump makes better sense for "static" data that does not change very often.
For example, some of the data in a telephone book was subject to change from time to time. But for the majority of data, changes were not frequent enough to justify calling the operator (=API call) every time one needed to look up a telephone number, or printing new telephone books every hour. Distributing new telephone books periodically, once evry few months or once a year, was sufficient to keep up with changes to the data.
An analogous situation exists with the data in the DNS. At one time, before the DNS, the Internet's "telephone book" was the HOSTS file. Perhaps a majority of entries were likely to change frequently. Periodically downloading a new HOSTS file was reputedly insufficient to keep up with changes to the data.
Let's say, for discussion purposes, "frequently" means more than once a month.
Today, I would guess only a minority of the total data stored in the DNS changes "frequently". The sampling I've done over the years supports this assumption. In my research, the majority of data stays the same for at least a month at time. As such, downloading new data once a month would be sufficient to keep up with changes to _this portion_ of the data.
As stated above, one approach might be to split the database up into "entries that are subject to change frequently" and ones that are not: a "frequently changing" DNS (for those who need it) and a "regular" DNS (where all needed data is stored locally, not accessed remotely via a network) for those who are not changing IP addresses more than once a month. Based on my research, _most_ websites are not changing their IP address more than once a month.
Returning to the telephone book example, we know that _most_ persons and businesses did not change their telephone number very frequently. This is why the telephone was still useful even though we only updated the data (by getting a new copy of the book) once every few months or once a year.