3 ms·
> I see that it is in Memory, but are there any practical limits? Yes. The in-memory database is basically a big btree + a big rtree. These trees contain many
by tidwall 10y ago
> I see that it is in Memory, but are there any practical limits?
Yes. The in-memory database is basically a big btree + a big rtree. These trees contain many pointers to geo objects and depending on the complexity of the objects, it may take up a lot of memory. A 16GB machine can typically store 100 million+ points.
> How much memory is recommended for what size datasets?
Depends on the type of object and the length of the keys, but I would conservatively use 128-bytes per point as a general rule. So 16GB machine should support ~135 million points.
> And it doesn't look like the documentation has an examples as to how to load data in?
Please check out http://tile38.com http://tile38.com for extra info and docs on commands. Tile38 uses the Redis protocol (https://redis.io/topics/protocol https://redis.io/topics/protocol), so it's importing data is similar to https://redis.io/topics/mass-insert https://redis.io/topics/mass-insert.
> Do I need to convert to Web Mercator, or can I give it WGS84 and have it convert? Also, any plans to allow for pure WGS84?
All coordinates are in WGS84 lat/lon, so you should be fine. Under-the-hood the calculations and such are in EPSG:3857 / WGS 84 Web Mercator.
- Bedon292 10y agoThanks for the response. I see what I was actually looking for was on http://tile38.com/topics/object-types/ http://tile38.com/topics/object-types/ Right there on the front page, and not in the docs tab where I was looking. For the GeoJSON, does it support properties? Meaning can I store a a whole document in there? Or would it be best to just store an ID and lookup the ID in a DB or something?
- tidwall 10y ago> For the GeoJSON, does it support properties? Yes, Tile38 has full GeoJSON support including properties for Feature objects. Though there is one exception, the CRS key is ignored because WGS84 is assumed. > Meaning can I store a a whole document in there? Yes, as long as the document conforms to http://geojson.org/geojson-spec.html http://geojson.org/geojson-spec.html. Though JSON keys that are not defined in the spec will be stripped out of the document prior to storing in memory. For example, a "properties" key of a type that is not a "Feature" will be stripped, and requesting that object at later time will result in a JSON document that does not include the "properties".
- Opteron67 10y ago> but I would conservatively use 128-bytes per point as a general rule. why not store close items together in a small block and use offsets + delta encoding ?
- tidwall 10y agoThanks for the suggestion, I'll need to investigate further. Right now, under the hood Tile38 uses an 3d rtree. Each node clusters 8-16 objects together. I'm always looking for a better way.
- derefr 10y ago> Tile38 uses the Redis protocol Under the hood, is Tile38 a distribution of Redis with a Tile38 Redis module? If so, could the Tile38 module be used in "regular" Redis alongside other Redis modules?
- nrjdhsbsid 10y agoLol looks like this database needs its own database. Yeah seems like it should be a plugin or library not sure why it's standalone if it's just backed by redis.
- tidwall 10y agoIt's not backed by Redis. It's fully standalone. I wish I could have made a redis module back when I started the project, but I'm plenty happy how Tile38 has matured.
- nrjdhsbsid 10y agoEh, fair enough then. If you don't have to setup redis yourself it doesn't really matter
- tidwall 10y agoIt's custom. I explored a Redis fork months ago, prior to the ability for modules. https://github.com/tidwall/redis-gis https://github.com/tidwall/redis-gis I then tried to make a module but redis modules do not support hooking into the pubsub tooling, so Tile38 live geofencing and webhooks were not possible.