3 ms·

Show HN: Flashlight – A portable tool to shine a light on your data

2 points by ilovetux 2y ago

Hello, I am Cliff and I built Flashlight, a versatile data platform. Flashlight is meant to ingest arbitrary text-based data and allow filtering, transformation and visualization. Think logs, JSON/XML files, JSON REST responses and more. This is the Public Alpha, I invite everyone to try it out and tell me what you think. I expect to hear a lot about the lack of documentation. There is currently documentation, but it is not as comprehensive as I think is needed, so I am looking for pain-points and other issues that really NEED to be in the documentation. After that, I will be writing some tutorials to show off a lot of the features that might hide in plain sight. You can reach me by commenting here or by opening an Issue in the Github Repo.

ilovetux 2y ago
I thought I would provide more information here, as I left it pretty blank when posting: # Overview Flashlight is a versatile and powerful platform for ingesting, transforming, and searching through structured, unstructured, and semi-structured data. It allows for interactive searches, dashboards, alerts, and more. Key Features Ingest data from various sources Transform and normalize data Powerful search capabilities Interactive dashboards and visualizations Alerts and notifications Extensible and customizable # Use Cases Log analysis and monitoring Data exploration and visualization Real-time alerting and notifications API data crawling and analysis Troubleshooting and debugging # Project Goals Provide a user-friendly interface for data exploration Enable quick and efficient data ingestion and transformation Offer powerful search and filtering capabilities Support interactive and customizable dashboards Facilitate real-time alerting and notifications
kingsizebk 2y ago
This sounds awesome, I am going to try it out… It seems like it has features like other similar well known tools (e.g. Splunk, DataDog, etc.) Is that what you are going for?
- ilovetux 2y ago
  That's a good portion of what we are going for. I want it to be as simple as possible for users to make use of their data. Whether the data is coming from logs, APIs, files or any other source, it should be simple and straightforward to onboard, search, correlate and visualize. Once, the data is in Flashlight and you have crafted a couple of searches, then it is easy to put that search into a panel of a dashboard and share the dashboard with coworkers. So, in essence very similar to both Splunk and Datadog, but there are some important distinctions as well. * Flashlight will be much more affordable. * Flashlight is self-hosted, so your data stays safe. Built-in and custom permissions also make it easy to share information with only those people you want to have access. * Flashlight allows you to gradually model your data. Events are a nice, generic text-based data model, but once your data has special requirements, you can break out that data into a seperate model, which will allow you to specify only the specific fields you need, store the data in its own database table, apply custom permissions, generate CRUD API for the model and much more. * Search commands, in Flashlight, are meant to be a lot more versatile and extensible. For instance, I have a custom Flashlight search command to connect to a group of servers, pull backups and store those backups for later. There is a corresponding custom search command to restore a specific backup later, so I was able to use those search commands in a "System Backup" dashboard with buttons to retrieve or restore backups. Please let me know if you are having any issues with Flashlight by opening an Issue in the Github Repo.

2 points by ilovetux 2y ago

ilovetux 2y ago

I thought I would provide more information here, as I left it pretty blank when posting: # Overview Flashlight is a versatile and powerful platform for ingesting, transforming, and searching through structured, unstructured, and semi-structured data. It allows for interactive searches, dashboards, alerts, and more. Key Features Ingest data from various sources Transform and normalize data Powerful search capabilities Interactive dashboards and visualizations Alerts and notifications Extensible and customizable # Use Cases Log analysis and monitoring Data exploration and visualization Real-time alerting and notifications API data crawling and analysis Troubleshooting and debugging # Project Goals Provide a user-friendly interface for data exploration Enable quick and efficient data ingestion and transformation Offer powerful search and filtering capabilities Support interactive and customizable dashboards Facilitate real-time alerting and notifications

kingsizebk 2y ago

This sounds awesome, I am going to try it out… It seems like it has features like other similar well known tools (e.g. Splunk, DataDog, etc.) Is that what you are going for?

ilovetux 2y ago
That's a good portion of what we are going for. I want it to be as simple as possible for users to make use of their data. Whether the data is coming from logs, APIs, files or any other source, it should be simple and straightforward to onboard, search, correlate and visualize. Once, the data is in Flashlight and you have crafted a couple of searches, then it is easy to put that search into a panel of a dashboard and share the dashboard with coworkers. So, in essence very similar to both Splunk and Datadog, but there are some important distinctions as well. * Flashlight will be much more affordable. * Flashlight is self-hosted, so your data stays safe. Built-in and custom permissions also make it easy to share information with only those people you want to have access. * Flashlight allows you to gradually model your data. Events are a nice, generic text-based data model, but once your data has special requirements, you can break out that data into a seperate model, which will allow you to specify only the specific fields you need, store the data in its own database table, apply custom permissions, generate CRUD API for the model and much more. * Search commands, in Flashlight, are meant to be a lot more versatile and extensible. For instance, I have a custom Flashlight search command to connect to a group of servers, pull backups and store those backups for later. There is a corresponding custom search command to restore a specific backup later, so I was able to use those search commands in a "System Backup" dashboard with buttons to retrieve or restore backups. Please let me know if you are having any issues with Flashlight by opening an Issue in the Github Repo.