3 ms·

This is John from Gravwell; I'd be happy to answer any questions about Gravwell or Community Edition. We're rolling it out to the public today, and I thought ou

by floren 8y ago

This is John from Gravwell; I'd be happy to answer any questions about Gravwell or Community Edition. We're rolling it out to the public today, and I thought our network security focus might interest the HN crowd!

mdekkers 8y ago
Thanks. Here is a question about Gravwell, non-community edition: how much? I personally get really annoyed when I look at a tool or system that is possibly interesting, depending on cost, and don't find a price on the site. I don't have time to go through the whole sales-spiel for every tool of interest, and just want to quickly filter on "now, maybe later, no way" and cost is key factor in making that determination. And usually, if the answer is "it depends" it typically means "it depends on how far we think we can push". So thanks, but before I download a trial, community edition, or sign up to become an honoured member of your sales-funnel, please, lets' not waste anybodies time, and let me know "how much".
- eropple 8y ago
  Agreed. This looks interesting, but I just do not have it in me to care about a company's sales funnel. The last time I put up with this kind of thing I was on an hour-long call with a HashiCorp sales guy who didn't have the answers to any questions but did want to tell me that support would be $25,000/year for 9x5 access (lol).
  - cthuen 8y ago
    Shit, I know the frustration. This one's on me. I'm Corey, one of the co-founders. Pricing is a big "it depends" but average cost savings over Splunk are 30%. For starters, a single node Basic unlimited data license is $25k annually. Things obviously go up for larger enterprises that need bigger clusters but our pricing model is a step function rather than the bullshit "pay per GB" model that everyone else seems stuck on. Our view is, it's your hardware so use it. Every license is unlimited data. You only add more nodes if you need more phatty IOPs for fast searches - hence the it depends. Some people use it more like a black box and don't issue many searches so a single node with a shitload of storage is just fine. Others rely on active searching to monitor security incidents and KPIs so they want responsiveness and immediate insights.
    - eropple 8y ago
      Hey - no worries, and I appreciate the forthright (and detailed) response. I Show HN'd something the other day and got beat up a little too, I know how it goes. ;) I typically use AWS's built-in tools, but I've been looking for something for home, so I'll be checking this out. Thanks.
      - floren 8y ago
        Speaking of AWS, you may find our Kinesis stream integration useful: https://www.gravwell.io/blog/amazon-kinesis-streams-and-gravwell https://www.gravwell.io/blog/amazon-kinesis-streams-and-grav...
    - koolba 8y ago
      Website bug report: when visiting the trial request page[1] the header nav bar ("Home", "FAQ", etc) disappears. [1]: https://www.gravwell.io/local-trial-request https://www.gravwell.io/local-trial-request
jstarfish 8y ago
Definitely interested, but would like to see a demo or some screenshots of the UI or something featured more prominently.
- floren 8y ago
  There's some screenshots in the high-level quickstart (https://dev.gravwell.io/docs/#!quickstart/quickstart.md https://dev.gravwell.io/docs/#!quickstart/quickstart.md) but in general we do seem to be pretty demo-sparse on the main site. I'd also suggest checking out this older blog post about WiFi analytics (https://www.gravwell.io/blog/wifi-analytics-wild-west-hackin-fest-2017 https://www.gravwell.io/blog/wifi-analytics-wild-west-hackin...) with the caveat that some things may have been refined a bit since that time. Edit: we're putting screenshots into the blog post now
colonelxc 8y ago
I think this post (and the home page) would benefit greatly with some example dashboards and queries. You really have to dig deep into the docs to see what it is capable of. Splunk on the other hand has dashboard examples on almost every page (on their homepage, a carousel with five different examples of the things they purport to solve). (another blog post does a great job with showing off capabilities: https://www.gravwell.io/blog/gravwell-and-collectd https://www.gravwell.io/blog/gravwell-and-collectd )
- floren 8y ago
  It's a good point. What would you find particularly compelling as an example dashboard? We've built dashboards around hardware stats (cpu temp etc.), port scanning & brute-forcing attempts, even Reddit comments. All of the above? :)
  - colonelxc 8y ago
    I like the blog post I mentioned above, which covers cpu/disk/ram monitoring. I'd also like to see a fleshed out network analysis example.
jstarfish 8y ago
> Unlike regular Gravwell licenses, Community Edition licenses are restricted to 2GB of ingested data per day. I wanted to give this a chance but I see we're up from Splunk's 500MB to a mere 2GB, and I have to learn and deploy a completely new product and train everybody else on how to use it as well. I've got a team of users who are familiar with Splunk, have written code and parsers for Splunk and simply want to use Splunk. I can't get the funding for it though. If Splunk raises their cap to match yours, what is Gravwell's advantage (besides presumably licensing costs)?
- floren 8y ago
  It searches faster. It's easier to get data in--we store binary data and then crack/parse it at search time, so you can even ingest things like raw packet capture or video streams. It's easy to deploy. We intend Gravwell Community Edition as a way for home users to experiment on their own network, or to try things out at work to decide if they want a full license. We of course also do unrestricted evaluation licenses for interested parties.
  - deleted 8y ago
    [deleted]
floren 8y ago
Update: We went ahead and pushed out the first of our blog posts on building a home network monitoring center, to give a little more of an idea of what you could use it for: https://www.gravwell.io/blog/gravwell-and-collectd https://www.gravwell.io/blog/gravwell-and-collectd I'm particularly fond of this article because it showcases the Turing-complete scripting interface which I built. The blog post shows how to set up a script which runs on a schedule and emails you if your disks get too full. We'll be posting more articles over the coming days.
spike021 8y ago
I've noticed that Splunk tends to be quite crufty and bloated due to it likely just being added to over time, rather than refactored. It can be a pain to work with, especially when it comes to writing custom "apps" and other plugin type scripts for it. The included libraries aren't great and the system takes a fair bit of massaging to get things working at a usable level. Is Gravwell as extensible, and, if so, is it easier to work with? Thanks!
- floren 8y ago
  I'll offer up first the caveat that Gravwell is a pretty new product, but we've definitely been building with an eye to extensibility. It helps that we haven't been around long enough to build up too much cruft! The core of Gravwell is the search. You can run searches in the web GUI, or in the CLI client. You can schedule searches to run at certain times (specified with a cron spec, currently) so you could have search results ready first-thing every morning. More powerfully, you can write scripts to run searches. These scripts can be run on a schedule, or you can run them by hand using the CLI client. Check out our other blog post https://www.gravwell.io/blog/gravwell-and-collectd https://www.gravwell.io/blog/gravwell-and-collectd for an example script that runs a search on disk stats entries from collectd, checks if any machine is running out of disk space, and emails someone if there's a problem. Of course, scripts can be complex to write, so we're exploring options for simpler flowchart-like scripting within the GUI too. You can also run scripts within the pipeline, which we frequently do when existing search modules don't quite meet our needs. We've open-sourced the library that lets you ingest data, so you could pretty quickly ingest anything you want. Unlike Splunk, you don't have to massage every data source into a key-value sort of text format; we'll just take binary if you like, we don't care. Gravwell has a REST API, so you could also interface with it that way. We'll probably open-source our Go client library at some point, but we want to clean it up a little first and make things more idiomatic in places. I hope that answers your question a little? All the functionality I mentioned here is included in the Community Edition, of course!
0xdeadbeefbabe 8y ago
How does it compare to ELK stack (elastic logstash and kibana). Do you do zero copy in golang like kafka does?
- cthuen 8y ago
  Its a bit of a different paradigm. From a deployment perspective we are a few static binaries and we don't require that you fully understand your data prior to ingesting and operating on it. The storage system is a bit different too, in that it treats storage as a cost center (e.g. use expensive storage when you want speed and ageout and optimize when you need longevity. A short answer is that we are truly unstructured, and will handle a lot of data in its native form. To your second question, our entire system is built around processing local, copying is a HUGE no no in the platform until you absolutely have to. We're VERY happy with the performance we're getting out of this sucker. That's one of the primary reasons we were dumb enough to start from scratch on the storage and search architecture. We're just glad it paid off.
porker 8y ago
Two things: 1. Your docs site stays as a blank white screen for me. Some chrome extension is interfering, but uBlock Origin isn't flagging it as blocking anything and the site stays white even with it disabled. It displays in Chrome Incognito mode (Windows 10) 2. Is there a query language, or is this not designed to ingest textual/json logs? A let-down with a number of services is how opaque querying is (Scalyr for example). Examples would be good.

by floren 8y ago

mdekkers 8y ago

Thanks. Here is a question about Gravwell, non-community edition: how much? I personally get really annoyed when I look at a tool or system that is possibly interesting, depending on cost, and don't find a price on the site. I don't have time to go through the whole sales-spiel for every tool of interest, and just want to quickly filter on "now, maybe later, no way" and cost is key factor in making that determination. And usually, if the answer is "it depends" it typically means "it depends on how far we think we can push". So thanks, but before I download a trial, community edition, or sign up to become an honoured member of your sales-funnel, please, lets' not waste anybodies time, and let me know "how much".

eropple 8y ago
Agreed. This looks interesting, but I just do not have it in me to care about a company's sales funnel. The last time I put up with this kind of thing I was on an hour-long call with a HashiCorp sales guy who didn't have the answers to any questions but did want to tell me that support would be $25,000/year for 9x5 access (lol).
- cthuen 8y ago
  Shit, I know the frustration. This one's on me. I'm Corey, one of the co-founders. Pricing is a big "it depends" but average cost savings over Splunk are 30%. For starters, a single node Basic unlimited data license is $25k annually. Things obviously go up for larger enterprises that need bigger clusters but our pricing model is a step function rather than the bullshit "pay per GB" model that everyone else seems stuck on. Our view is, it's your hardware so use it. Every license is unlimited data. You only add more nodes if you need more phatty IOPs for fast searches - hence the it depends. Some people use it more like a black box and don't issue many searches so a single node with a shitload of storage is just fine. Others rely on active searching to monitor security incidents and KPIs so they want responsiveness and immediate insights.
  - eropple 8y ago
    Hey - no worries, and I appreciate the forthright (and detailed) response. I Show HN'd something the other day and got beat up a little too, I know how it goes. ;) I typically use AWS's built-in tools, but I've been looking for something for home, so I'll be checking this out. Thanks.
    - floren 8y ago
      Speaking of AWS, you may find our Kinesis stream integration useful: https://www.gravwell.io/blog/amazon-kinesis-streams-and-gravwell https://www.gravwell.io/blog/amazon-kinesis-streams-and-grav...
  - koolba 8y ago
    Website bug report: when visiting the trial request page[1] the header nav bar ("Home", "FAQ", etc) disappears. [1]: https://www.gravwell.io/local-trial-request https://www.gravwell.io/local-trial-request

jstarfish 8y ago

Definitely interested, but would like to see a demo or some screenshots of the UI or something featured more prominently.

floren 8y ago
There's some screenshots in the high-level quickstart (https://dev.gravwell.io/docs/#!quickstart/quickstart.md https://dev.gravwell.io/docs/#!quickstart/quickstart.md) but in general we do seem to be pretty demo-sparse on the main site. I'd also suggest checking out this older blog post about WiFi analytics (https://www.gravwell.io/blog/wifi-analytics-wild-west-hackin-fest-2017 https://www.gravwell.io/blog/wifi-analytics-wild-west-hackin...) with the caveat that some things may have been refined a bit since that time. Edit: we're putting screenshots into the blog post now

colonelxc 8y ago

I think this post (and the home page) would benefit greatly with some example dashboards and queries. You really have to dig deep into the docs to see what it is capable of. Splunk on the other hand has dashboard examples on almost every page (on their homepage, a carousel with five different examples of the things they purport to solve). (another blog post does a great job with showing off capabilities: https://www.gravwell.io/blog/gravwell-and-collectd https://www.gravwell.io/blog/gravwell-and-collectd )

floren 8y ago
It's a good point. What would you find particularly compelling as an example dashboard? We've built dashboards around hardware stats (cpu temp etc.), port scanning & brute-forcing attempts, even Reddit comments. All of the above? :)
- colonelxc 8y ago
  I like the blog post I mentioned above, which covers cpu/disk/ram monitoring. I'd also like to see a fleshed out network analysis example.

jstarfish 8y ago

> Unlike regular Gravwell licenses, Community Edition licenses are restricted to 2GB of ingested data per day. I wanted to give this a chance but I see we're up from Splunk's 500MB to a mere 2GB, and I have to learn and deploy a completely new product and train everybody else on how to use it as well. I've got a team of users who are familiar with Splunk, have written code and parsers for Splunk and simply want to use Splunk. I can't get the funding for it though. If Splunk raises their cap to match yours, what is Gravwell's advantage (besides presumably licensing costs)?

floren 8y ago
It searches faster. It's easier to get data in--we store binary data and then crack/parse it at search time, so you can even ingest things like raw packet capture or video streams. It's easy to deploy. We intend Gravwell Community Edition as a way for home users to experiment on their own network, or to try things out at work to decide if they want a full license. We of course also do unrestricted evaluation licenses for interested parties.
- deleted 8y ago
  [deleted]

floren 8y ago

Update: We went ahead and pushed out the first of our blog posts on building a home network monitoring center, to give a little more of an idea of what you could use it for: https://www.gravwell.io/blog/gravwell-and-collectd https://www.gravwell.io/blog/gravwell-and-collectd I'm particularly fond of this article because it showcases the Turing-complete scripting interface which I built. The blog post shows how to set up a script which runs on a schedule and emails you if your disks get too full. We'll be posting more articles over the coming days.

spike021 8y ago

I've noticed that Splunk tends to be quite crufty and bloated due to it likely just being added to over time, rather than refactored. It can be a pain to work with, especially when it comes to writing custom "apps" and other plugin type scripts for it. The included libraries aren't great and the system takes a fair bit of massaging to get things working at a usable level. Is Gravwell as extensible, and, if so, is it easier to work with? Thanks!

floren 8y ago
I'll offer up first the caveat that Gravwell is a pretty new product, but we've definitely been building with an eye to extensibility. It helps that we haven't been around long enough to build up too much cruft! The core of Gravwell is the search. You can run searches in the web GUI, or in the CLI client. You can schedule searches to run at certain times (specified with a cron spec, currently) so you could have search results ready first-thing every morning. More powerfully, you can write scripts to run searches. These scripts can be run on a schedule, or you can run them by hand using the CLI client. Check out our other blog post https://www.gravwell.io/blog/gravwell-and-collectd https://www.gravwell.io/blog/gravwell-and-collectd for an example script that runs a search on disk stats entries from collectd, checks if any machine is running out of disk space, and emails someone if there's a problem. Of course, scripts can be complex to write, so we're exploring options for simpler flowchart-like scripting within the GUI too. You can also run scripts within the pipeline, which we frequently do when existing search modules don't quite meet our needs. We've open-sourced the library that lets you ingest data, so you could pretty quickly ingest anything you want. Unlike Splunk, you don't have to massage every data source into a key-value sort of text format; we'll just take binary if you like, we don't care. Gravwell has a REST API, so you could also interface with it that way. We'll probably open-source our Go client library at some point, but we want to clean it up a little first and make things more idiomatic in places. I hope that answers your question a little? All the functionality I mentioned here is included in the Community Edition, of course!

0xdeadbeefbabe 8y ago

How does it compare to ELK stack (elastic logstash and kibana). Do you do zero copy in golang like kafka does?

cthuen 8y ago
Its a bit of a different paradigm. From a deployment perspective we are a few static binaries and we don't require that you fully understand your data prior to ingesting and operating on it. The storage system is a bit different too, in that it treats storage as a cost center (e.g. use expensive storage when you want speed and ageout and optimize when you need longevity. A short answer is that we are truly unstructured, and will handle a lot of data in its native form. To your second question, our entire system is built around processing local, copying is a HUGE no no in the platform until you absolutely have to. We're VERY happy with the performance we're getting out of this sucker. That's one of the primary reasons we were dumb enough to start from scratch on the storage and search architecture. We're just glad it paid off.

porker 8y ago

Two things: 1. Your docs site stays as a blank white screen for me. Some chrome extension is interfering, but uBlock Origin isn't flagging it as blocking anything and the site stays white even with it disabled. It displays in Chrome Incognito mode (Windows 10) 2. Is there a query language, or is this not designed to ingest textual/json logs? A let-down with a number of services is how opaque querying is (Scalyr for example). Examples would be good.