4 ms·

Bitmaps can also be used for computing funnels: https://vikramoberoi.com/using-bitmaps-to-run-interactive-retention-analyses-over-billions-of-events-for-less-t

Bitmaps can also be used for computing funnels: https://vikramoberoi.com/using-bitmaps-to-run-interactive-retention-analyses-over-billions-of-events-for-less-than-100-mo/ https://vikramoberoi.com/using-bitmaps-to-run-interactive-re...

JAA1337 4y ago
I think it's fair to say that the use cases for bitmaps have not yet been fully explored. I believe their use in areas of analytics and OLAP/RTOLAP will continue to grow as the desire to infer and target demo's across households increases.
- eurasiantiger 4y ago
  Isn’t Lucene already a household name?
  - clowen 4y ago
    Probably not, first thought I had was Lucerne dairy products.
  - JAA1337 4y ago
    Not sure I understand your point. Can you clarify some?
    - eurasiantiger 4y ago
      Lucene uses roaring bitmaps for filters.
      - JAA1337 4y ago
        Sure, I completely get it. However, does that mean bitmaps have been fully explored? For example, OLAP is only growing as a market as the need for realtime analysis on the most fresh data. I mean, we never going to have 'less' data nor are we ever going to seek results 'slower'. Where I am going with this, utilizing technologies like bitmaps in OLAP services is an area I believe will see continued growth. While Lucene is a known commodity, I think we can agree it hasn't solved the sector, right?
  - gthrone 4y ago
    I didn't realize Lucene used their own flavor of roaring bitmaps! Super Cool. I certainly think of search when it comes to Lucene, where as the cited material from Vikram & FeatureBase are targeting more general analytics.
  - kordlessagain 4y ago
    Lucene is fast, but probably can't handle analytical type questions on massive amounts of data as fast as a pure binary index can. I could be wrong about that though. I do know Solr has it because Lucene does...
    - eurasiantiger 4y ago
      Elasticsearch (another household name) is built on Lucene too, so I wouldn’t count it out.
skingkong 4y ago
100%, and are freaking fast...and pretty efficient, especially for high throughput applications tracking tons of changing data. We've found bitmaps handle updates, inserts, and schema changes really elegantly - and quickly (like sub-second). Depending on the use case, looking at cohorts MoM or YoY that freshness doesn't entirely matter, but computing cohorts in real-time hour-over-hour, or even minute-over-minute, for in-app actions/personalization is insanely valuable to keep users engaged (or what ever entity type you are analyzing and taking action on, whether it's devices, nodes across a network, etc).

JAA1337 4y ago

I think it's fair to say that the use cases for bitmaps have not yet been fully explored. I believe their use in areas of analytics and OLAP/RTOLAP will continue to grow as the desire to infer and target demo's across households increases.

eurasiantiger 4y ago
Isn’t Lucene already a household name?
- clowen 4y ago
  Probably not, first thought I had was Lucerne dairy products.
- JAA1337 4y ago
  Not sure I understand your point. Can you clarify some?
  - eurasiantiger 4y ago
    Lucene uses roaring bitmaps for filters.
    - JAA1337 4y ago
      Sure, I completely get it. However, does that mean bitmaps have been fully explored? For example, OLAP is only growing as a market as the need for realtime analysis on the most fresh data. I mean, we never going to have 'less' data nor are we ever going to seek results 'slower'. Where I am going with this, utilizing technologies like bitmaps in OLAP services is an area I believe will see continued growth. While Lucene is a known commodity, I think we can agree it hasn't solved the sector, right?
- gthrone 4y ago
  I didn't realize Lucene used their own flavor of roaring bitmaps! Super Cool. I certainly think of search when it comes to Lucene, where as the cited material from Vikram & FeatureBase are targeting more general analytics.
- kordlessagain 4y ago
  Lucene is fast, but probably can't handle analytical type questions on massive amounts of data as fast as a pure binary index can. I could be wrong about that though. I do know Solr has it because Lucene does...
  - eurasiantiger 4y ago
    Elasticsearch (another household name) is built on Lucene too, so I wouldn’t count it out.

skingkong 4y ago

100%, and are freaking fast...and pretty efficient, especially for high throughput applications tracking tons of changing data. We've found bitmaps handle updates, inserts, and schema changes really elegantly - and quickly (like sub-second). Depending on the use case, looking at cohorts MoM or YoY that freshness doesn't entirely matter, but computing cohorts in real-time hour-over-hour, or even minute-over-minute, for in-app actions/personalization is insanely valuable to keep users engaged (or what ever entity type you are analyzing and taking action on, whether it's devices, nodes across a network, etc).