3 ms·

Show HN: Turn any data into a fast analytical API

26 points by caoxhua 5y ago

ghoshbishakh 5y ago
Cool. "All data will be loaded into our cloud cluster in distributed memory, encrypted under your credential." So before uploading I encrypt my data? Then how does the queries work? Or you encrypt it for me on your end?
- caoxhua 5y ago
  Hey, thanks for your comment! You don't encrypt, just leave your data where it is, what we need is a live connection, after loading your data in our distributed cluster in cloud, we have option to encrypt the data for further security if needed. So yes, we encrypt at our end if asked. Our API service doesn't store data, only load them into distributed memory for computing and API serving. Let me know if you're interested in this API service, would like to see if we can help.
  - pstuart 5y ago
    Interesting concept. Small typo: "will be splitted into multiple tasks" s/b "split"
    - caoxhua 5y ago
      Thank you for the correction, appreciate it! I'll update it.
  - onion2k 5y ago
    Our API service doesn't store data, only load them into distributed memory for computing and API serving. I think you're suggesting that a queryable in-memory cache isn't storage. I don't understand that. What is it if it's not short term, non-canonical storage?
kfk 5y ago
How is this different from Postgrest?
- caoxhua 5y ago
  Fundamentally they both need to do computation to answer query, but they are very different, a couple of highlights: 1. API service invest a lot on reliable data ingestion/loading for users, you specify data spec, data will be served with lease term reliably. It takes care of all the data sync maintenance for you. 2. It works seamlessly with real-time streaming such as Kafka for RTA. PG is not suitable for this. 3. It's distributed nature + In-memory store guarantees low-latency API service. Hope this describes a reasonable overview to think of it.
mritchie712 5y ago
What query engine are you using? It's worth checking out Tinybird (https://www.tinybird.co/ https://www.tinybird.co/) if you haven't seen them. Might give you some ideas.
- caoxhua 5y ago
  we use our in-house baked engine - open sourced here https://github.com/varchar-io/nebula https://github.com/varchar-io/nebula Yeah, Tinybird has lots of similarities, I will do more research on it, thanks for the reference.

26 points by caoxhua 5y ago

Cool. "All data will be loaded into our cloud cluster in distributed memory, encrypted under your credential." So before uploading I encrypt my data? Then how does the queries work? Or you encrypt it for me on your end?

caoxhua 5y ago
Hey, thanks for your comment! You don't encrypt, just leave your data where it is, what we need is a live connection, after loading your data in our distributed cluster in cloud, we have option to encrypt the data for further security if needed. So yes, we encrypt at our end if asked. Our API service doesn't store data, only load them into distributed memory for computing and API serving. Let me know if you're interested in this API service, would like to see if we can help.
- pstuart 5y ago
  Interesting concept. Small typo: "will be splitted into multiple tasks" s/b "split"
  - caoxhua 5y ago
    Thank you for the correction, appreciate it! I'll update it.
- onion2k 5y ago
  Our API service doesn't store data, only load them into distributed memory for computing and API serving. I think you're suggesting that a queryable in-memory cache isn't storage. I don't understand that. What is it if it's not short term, non-canonical storage?

kfk 5y ago

How is this different from Postgrest?

caoxhua 5y ago
Fundamentally they both need to do computation to answer query, but they are very different, a couple of highlights: 1. API service invest a lot on reliable data ingestion/loading for users, you specify data spec, data will be served with lease term reliably. It takes care of all the data sync maintenance for you. 2. It works seamlessly with real-time streaming such as Kafka for RTA. PG is not suitable for this. 3. It's distributed nature + In-memory store guarantees low-latency API service. Hope this describes a reasonable overview to think of it.

mritchie712 5y ago

What query engine are you using? It's worth checking out Tinybird (https://www.tinybird.co/ https://www.tinybird.co/) if you haven't seen them. Might give you some ideas.

caoxhua 5y ago
we use our in-house baked engine - open sourced here https://github.com/varchar-io/nebula https://github.com/varchar-io/nebula Yeah, Tinybird has lots of similarities, I will do more research on it, thanks for the reference.