3 ms·

Show HN: Delve, an open source (AGPL) enterprise-grade data analytics platform

21 points by ilovetux 1y ago

Hello, I am excited to announce the release of my project that I've been working on for quite a while. It has gone through many iterations. I believe that I now have a scalable platform that can efficiently ingest, search and report on large quantities of data. If you are at a very large scale, there will need to be some work to streamline your database access, but it is very possible to build a flexible and robust solution. Delve is built using Django, Django Rest Framework, webpack, JS and SCSS. Delve can be extended with apps, which are actually specially built Django apps. Delve has a unix-pipe-like search language that is powered by simple python scripts. The Delve Web UI has support for showing data in tables, line charts and bar charts with more visualizations on the way. I am excited to hear your feedback and suggestions. I will be trying to monitor the comments and reply quickly.

FuriouslyAdrift 1y ago
Up until fairly recently, Delve was a Microsoft product that profiled and analyzed corporate data... you might run into some confusion. https://supersimple365.com/microsoft-is-killing-delve-here-are-your-alternatives/ https://supersimple365.com/microsoft-is-killing-delve-here-a...
- ilovetux 1y ago
  Thank you for letting me know. I did not know about the Delve MS product. I just recently renamed from Flashlight to Delve. I like the name and MS retired it, so I'm going to try to stick with it.
atombender 1y ago
Delve is the name of a popular debugger: https://github.com/go-delve/delve https://github.com/go-delve/delve
- ilovetux 1y ago
  Thank you. I appreciate you pointing this out. It seems like Delve is a popular name for software, which makes sense. I am officially looking for a new name for this software. It used to be called Flashlight, as in "A portable tool to shine some light on your data", but I liked the name Delve and a naive google search didn't reveal any projects named delve. As an aside, Delve is not only popular in software naming, but it also appears in the English language writing quite a lot.
  - atombender 1y ago
    Wish I could help, but names are hard.
    - ilovetux 1y ago
      They really are.
  - arp242 1y ago
    DelveDeep, DeepDelve, DataDelve, DelveData, DelveLight, FlashDelve, DelveFlash
tuyguntn 1y ago
Name might get punished by SEO engines soon. Because of "Let's delve into details"
- ilovetux 1y ago
  Good point. because of this and a few other comments, I am planning on renaming the project soon. Any ideas you would like to share?
  - chatmasta 1y ago
    I like the name. If everyone renamed their product based on HN comments, the marketplace would look like the home directory of a mentally unstable data hoarder. It does sound like there are some collisions but that doesn’t really matter unless your CLI name clashes, in which case you have many options for adjusting it. The domain name can have suffix or prefix or just “analytics.” And realistically people are going to google “delve analytics” if they can’t find it.
ernestbro 1y ago
Do you have a demo video?
- ilovetux 1y ago
  I do have a video I made a while ago, but I dropped the ball on starting a youtube channel to publish it. Any recommendations on where I can upload and share a video, preferably one that I could embed in my README.md?
  - popalchemist 1y ago
    Just start the youtube.
  - mdaniel 1y ago
    Depends on the size, but I have seen a non-trivial number of GitHub projects that reference an .mp4 served by raw.githubuserassets.com in their readme I would suspect the cheapest way of doing that is to create an issue, drag the mp4 onto it, and then you have a stable reference to use from within the readme
popalchemist 1y ago
Is the name a reference to LOTR or ChatGPT?
- ilovetux 1y ago
  Neither. I'm using its English definition: To research or make inquiries into something
mdaniel 1y ago
Maybe it's an expected outcome in your system, but this is going to AttributeError if things don't go your way https://github.com/iLoveTux/delve/blob/v0.7.0-dev/src/home/events/parsers/apache.py#L23-L25 https://github.com/iLoveTux/delve/blob/v0.7.0-dev/src/home/e... AttributeError: 'NoneType' object has no attribute 'group'
vivzkestrel 1y ago
How does it compare to Plausible, Matomo and Umami
- ilovetux 1y ago
  Thank you for the question. Those platforms seem to be web analytics platforms, purpose built to track and analyze data from visitors to your web site. Delve, on the other hand, concentrates on ingesting large quantities of structured, semi-structured and unstructured text based data. Think log files, csv, json, xml, item descriptions, etc. The closest product to Delve would be something like Splunk or ELK. I would say Delve is easier to get started with than Splunk or ELK, but actually offers more control since you can easily write your own search commands and field extractions. Also, role based access control is implemented to give fine-grained access to data. Also configuration is all in one place, backend databases are configurable (sqlite, postgres, MySQL and any other database that works with the django ORM) and more. Please let me know if you'd like any more details.

21 points by ilovetux 1y ago

FuriouslyAdrift 1y ago

Up until fairly recently, Delve was a Microsoft product that profiled and analyzed corporate data... you might run into some confusion. https://supersimple365.com/microsoft-is-killing-delve-here-are-your-alternatives/ https://supersimple365.com/microsoft-is-killing-delve-here-a...

ilovetux 1y ago
Thank you for letting me know. I did not know about the Delve MS product. I just recently renamed from Flashlight to Delve. I like the name and MS retired it, so I'm going to try to stick with it.

atombender 1y ago

Delve is the name of a popular debugger: https://github.com/go-delve/delve https://github.com/go-delve/delve

ilovetux 1y ago
Thank you. I appreciate you pointing this out. It seems like Delve is a popular name for software, which makes sense. I am officially looking for a new name for this software. It used to be called Flashlight, as in "A portable tool to shine some light on your data", but I liked the name Delve and a naive google search didn't reveal any projects named delve. As an aside, Delve is not only popular in software naming, but it also appears in the English language writing quite a lot.
- atombender 1y ago
  Wish I could help, but names are hard.
  - ilovetux 1y ago
    They really are.
- arp242 1y ago
  DelveDeep, DeepDelve, DataDelve, DelveData, DelveLight, FlashDelve, DelveFlash

tuyguntn 1y ago

Name might get punished by SEO engines soon. Because of "Let's delve into details"

ilovetux 1y ago
Good point. because of this and a few other comments, I am planning on renaming the project soon. Any ideas you would like to share?
- chatmasta 1y ago
  I like the name. If everyone renamed their product based on HN comments, the marketplace would look like the home directory of a mentally unstable data hoarder. It does sound like there are some collisions but that doesn’t really matter unless your CLI name clashes, in which case you have many options for adjusting it. The domain name can have suffix or prefix or just “analytics.” And realistically people are going to google “delve analytics” if they can’t find it.

ernestbro 1y ago

Do you have a demo video?

ilovetux 1y ago
I do have a video I made a while ago, but I dropped the ball on starting a youtube channel to publish it. Any recommendations on where I can upload and share a video, preferably one that I could embed in my README.md?
- popalchemist 1y ago
  Just start the youtube.
- mdaniel 1y ago
  Depends on the size, but I have seen a non-trivial number of GitHub projects that reference an .mp4 served by raw.githubuserassets.com in their readme I would suspect the cheapest way of doing that is to create an issue, drag the mp4 onto it, and then you have a stable reference to use from within the readme

popalchemist 1y ago

Is the name a reference to LOTR or ChatGPT?

ilovetux 1y ago
Neither. I'm using its English definition: To research or make inquiries into something

mdaniel 1y ago

Maybe it's an expected outcome in your system, but this is going to AttributeError if things don't go your way https://github.com/iLoveTux/delve/blob/v0.7.0-dev/src/home/events/parsers/apache.py#L23-L25 https://github.com/iLoveTux/delve/blob/v0.7.0-dev/src/home/e... AttributeError: 'NoneType' object has no attribute 'group'

vivzkestrel 1y ago

How does it compare to Plausible, Matomo and Umami

ilovetux 1y ago
Thank you for the question. Those platforms seem to be web analytics platforms, purpose built to track and analyze data from visitors to your web site. Delve, on the other hand, concentrates on ingesting large quantities of structured, semi-structured and unstructured text based data. Think log files, csv, json, xml, item descriptions, etc. The closest product to Delve would be something like Splunk or ELK. I would say Delve is easier to get started with than Splunk or ELK, but actually offers more control since you can easily write your own search commands and field extractions. Also, role based access control is implemented to give fine-grained access to data. Also configuration is all in one place, backend databases are configurable (sqlite, postgres, MySQL and any other database that works with the django ORM) and more. Please let me know if you'd like any more details.