4 ms·
What's the data store? And apologies if this is a dumb question, but given the number of astronomical objects in the universe that we might identify, is there s
by nowarninglabel 8y ago
What's the data store? And apologies if this is a dumb question, but given the number of astronomical objects in the universe that we might identify, is there some special consideration given to how large the amount of data stored might grow to?
- guillaumec 8y agoServer side we have a mongodb database that contains the list of all the searchable objects. It is generated mostly using common stars and nebulae catalogs. For streaming the data with good performances we also have an amazon S3 instance where we store compressed tiles of data. Each tile contains information about objects in a small region of the sky inside a given magnitude range, so when the user zoom in and out, we can download only the tiles of the visible portions of the sky. That's how we can render the whole Gaia catalog, containing about 1 billion stars. You can try it: zoom in, and you will see the stars being loaded on the fly.
- guillaumec 8y agoAs for the database size consideration, it will most likely be an issue once we start to support queries into the Gaia catalog (about 500GB of data). There are some ways to make it work though.
- arriu 8y agoHow are you guys constructing the DB from the catalog? Do you plan to grow it?
- guillaumec 8y agoFor Gaia, it would be impossible (or too expensive) to put the whole catalog into a conventional DB. The solution is probably going to be something similar to OpenDAP (https://www.opendap.org/ https://www.opendap.org/), but I haven't looked much into it myself.
- SiempreViernes 8y agoAny plans on integrating with the Virtual Observatory, or is that too slow to integrate nicely?
- xalioth 8y agoOne of the goal for the web version is to allow displaying sky observations yes (we are working on it). Not specifically from the VO, but it could as well.