5 ms·

- I am not really familiar with Anaplan. You could probably compare them, but as far as I understand Anaplan is a more specialized (planning) tool. - Egeria is

- I am not really familiar with Anaplan. You could probably compare them, but as far as I understand Anaplan is a more specialized (planning) tool. - Egeria is a database itself. I use Sqlite as a key value store for persistence now. I will probably use Mongo in production. - I will provide a command line tool to automate import and export with ETL tools in CSV format first. A better integration with relational databases would probably come later on. - All formulas are computed on the server, so users don't run them. Only superusers/designers will be able to change formulas. A normal user will only be able to enter values and change metadata. The concurrency is implemented using locks and in-memory data structures on the server. Open two tabs on the same sheet and try changing values in one of the tabs to see how it works.

kornish 8y ago
> I will probably use Mongo in production. Just curious: if one of the product differentiators is that its semantics more closely resemble a relational database, then why not power it with a relational database?
- deleted 8y ago
  [deleted]
- egeria_planning 8y ago
  Egeria is in-memory for performance reasons. Egeria can recompute millions cells per second. To do so cells are grouped into chunks and stored in binary form. Relational databases are much slower. It is possible to have a relational backend sometime in the future, but for now using a key value store also meant less implementation effort.
  - wenc 8y ago
    Multidimensional calculations typically require a lot of aggregation, rotations (pivots) and joins. Mongo is good for store-and-retrieve, and aggregation along a single dimension, and not as strong at the operations mentioned above. Have you benchmarked Mongo against a relational db for analytic operations on a multidimensional model? The results could be interesting and perhaps different from what you would expect.[1] Many new OLAP products are implemented on pure relational databases for performance reasons. Some databases with columnar indices are even faster for OLAP type operations. [1] That said, Mongo could be a good choice for latency reasons. If your spreadsheet is doing lots of small calculations, then it makes sense to use something that can return results quickly.
    - egeria_planning 8y ago
      Egeria is not OLAP in the sense of analytics. It does not aggregate data. It computes formulas. Relational databases do not support this.
      - deleted 8y ago
        [deleted]
  - ashrodan 8y ago
    Check out https://www.sigmacomputing.com https://www.sigmacomputing.com As they have done just this. Nice work though! Keen to test it out. Hopefully a mobile version is on the roadmap.
    - polskibus 8y ago
      Could you please elaborate on how sigma is similar to multidimensional modeling of arbitrary formulas and user writes? It seems that sigma is just a SQL generator + visualisations, so it only servers the read side. Spreadsheets are much more flexible as you operate more on cells than on entire columns, and can usually model an arbitrary-long formula chain.
  - barrystaes 8y ago
    > Egeria is in-memory for performance reasons I dont see how this relates, as a relational DB can run in-memory, and overflow to SSD. And cache values for the "views".
    - egeria_planning 8y ago
      Selecting rows from cache of a database is still 100-1000 times slower than reading numbers from an array. And dumping this array into a KV store is much easier than generating SQL.
go_prodev 8y ago
That's interesting. How do you represent a star schema in key value structure? Does every measure have a key in every dimension? And if users drill up/down does sqlite do the aggregation or is that done by your application? I haven't used kv stores before
- egeria_planning 8y ago
  Cells are grouped into chunks (similar to Minecraft) and stored in binary form. There is a complex algorithm which decides how the data is partitioned. All the operations are done in memory and the KV store is only used as data storage.
codeape 8y ago
Hi, just curious: could you say a bit more on the the backend and frontend tech in use?
- egeria_planning 8y ago
  Hi, it is: Angular 6, PrimeNG, TypeScript SPA frontend, ASP .NET Core application server (REST API only) and Sqlite as a key value data store

kornish 8y ago

> I will probably use Mongo in production. Just curious: if one of the product differentiators is that its semantics more closely resemble a relational database, then why not power it with a relational database?

deleted 8y ago
[deleted]
egeria_planning 8y ago
Egeria is in-memory for performance reasons. Egeria can recompute millions cells per second. To do so cells are grouped into chunks and stored in binary form. Relational databases are much slower. It is possible to have a relational backend sometime in the future, but for now using a key value store also meant less implementation effort.
- wenc 8y ago
  Multidimensional calculations typically require a lot of aggregation, rotations (pivots) and joins. Mongo is good for store-and-retrieve, and aggregation along a single dimension, and not as strong at the operations mentioned above. Have you benchmarked Mongo against a relational db for analytic operations on a multidimensional model? The results could be interesting and perhaps different from what you would expect.[1] Many new OLAP products are implemented on pure relational databases for performance reasons. Some databases with columnar indices are even faster for OLAP type operations. [1] That said, Mongo could be a good choice for latency reasons. If your spreadsheet is doing lots of small calculations, then it makes sense to use something that can return results quickly.
  - egeria_planning 8y ago
    Egeria is not OLAP in the sense of analytics. It does not aggregate data. It computes formulas. Relational databases do not support this.
    - deleted 8y ago
      [deleted]
- ashrodan 8y ago
  Check out https://www.sigmacomputing.com https://www.sigmacomputing.com As they have done just this. Nice work though! Keen to test it out. Hopefully a mobile version is on the roadmap.
  - polskibus 8y ago
    Could you please elaborate on how sigma is similar to multidimensional modeling of arbitrary formulas and user writes? It seems that sigma is just a SQL generator + visualisations, so it only servers the read side. Spreadsheets are much more flexible as you operate more on cells than on entire columns, and can usually model an arbitrary-long formula chain.
- barrystaes 8y ago
  > Egeria is in-memory for performance reasons I dont see how this relates, as a relational DB can run in-memory, and overflow to SSD. And cache values for the "views".
  - egeria_planning 8y ago
    Selecting rows from cache of a database is still 100-1000 times slower than reading numbers from an array. And dumping this array into a KV store is much easier than generating SQL.

go_prodev 8y ago

That's interesting. How do you represent a star schema in key value structure? Does every measure have a key in every dimension? And if users drill up/down does sqlite do the aggregation or is that done by your application? I haven't used kv stores before

egeria_planning 8y ago
Cells are grouped into chunks (similar to Minecraft) and stored in binary form. There is a complex algorithm which decides how the data is partitioned. All the operations are done in memory and the KV store is only used as data storage.

codeape 8y ago

Hi, just curious: could you say a bit more on the the backend and frontend tech in use?

egeria_planning 8y ago
Hi, it is: Angular 6, PrimeNG, TypeScript SPA frontend, ASP .NET Core application server (REST API only) and Sqlite as a key value data store