4 ms·

> If you have that level of hardware I don't know why you're running out of memory. Did you folks fix the "oops, I accidentally selected too much and OOMed the

> If you have that level of hardware I don't know why you're running out of memory. Did you folks fix the "oops, I accidentally selected too much and OOMed the machine" crash that was supposed to be resolved by the new query planner in .10? That was my immediate showstopper for InfluxDB for a very high-volume infrastructure, because I didn't feel like proxying InfluxDB just to enforce chunking, and the mere existence of the bug gave me pause. I hit that in 24 hours and shelved the system in 48. Even pre-1.0, I'm not encouraged when I have to "work around" pretty obvious oversights in reliability, so I put you back on the back burner to give you time to mature and transitioned back to a Kappa-style architecture for my needs. I might still use InfluxDB for low-volume aggregations out of my stream processing, but it's for sure out of the hot path for the foreseeable future.

SEJeff 11y ago
That is more or less how my employer worked around this glaring bug. That being said, we've worked together with Influx to fix some issues or add features to Telegraf and even have had their developers fix several of our issues found under excessive load (/me waves to Alex and Sebastian), Influx has a long way to go, but I've got faith that they're going in the right direction and things are getting a lot better. It is true this announcement is a bit too full of blatant marketing rhetoric, but the tech is good and if is getting better. If you've not tried it very recently, give it another shot.
pauldix 11y ago
The current version still has the problem that if you do a massive SELECT then it will fill up memory until the process gets killed. In the future we'll give controls to limit how much memory a query can take. I think it's common in databases that if you throw a massive query at it that the server doesn't have the resources to handle, things will go wrong. It'll thrash, or crash, or generally have poor performance. We'll be working on improving the failure conditions, but if you put a query to a database that is too big for the server to handle, some sort of failure scenario will be encountered. Just a question of how it's handled. Also, there's no new query planner in 0.10. We have a bunch of work getting merged in for the query engine for 0.11. But if you throw a huge query at the DB that the server can't handle, you'll still have great sadness. Which DB ended up solving your big query problems? Maybe we're just a poor fit? Or what kinds of queries should we be working on optimizing?
- u320 11y ago
  > I think it's common in databases that if you throw a massive query at it that the server doesn't have the resources to handle, things will go wrong. This has never ever happened to me in the last 20 years I've used SQL databases. Sure it might take a long time but that's it.
- thatsad 11y ago
  The current version still has the problem that if you do a massive SELECT then it will fill up memory until the process gets killed. That means any ad-hoc query can trivially crash your server. That's pretty serious. It's even worse if the ingest is using statsd-influxdb-backend with udp since during the crash you'll lose data. Do you really think it's ok for a novice user to be able to crash the server from your web gui if they make a minor mistake (e.g something like SELECT value FROM /series.*/ not realizing how many series their regex actually applies to until it's too late and the has gui stopped responding)? I think it's common in databases that if you throw a massive query at it that the server doesn't have the resources to handle, things will go wrong. This thinking is worse then wrong. Not only is it wrong - a simple unqualified select won't crash any "common" database - it makes anyone with any "common" database experience doubt your other claims.
- fweespeech 11y ago
  > Did you folks fix the "oops, I accidentally selected too much and OOMed the machine" crash that was supposed to be resolved by the new query planner in .10? > The current version still has the problem that if you do a massive SELECT then it will fill up memory until the process gets killed. In the future we'll give controls to limit how much memory a query can take. ... It locks on the query until its killed manually and/or kills it automatically. It doesn't crash.

SEJeff 11y ago

That is more or less how my employer worked around this glaring bug. That being said, we've worked together with Influx to fix some issues or add features to Telegraf and even have had their developers fix several of our issues found under excessive load (/me waves to Alex and Sebastian), Influx has a long way to go, but I've got faith that they're going in the right direction and things are getting a lot better. It is true this announcement is a bit too full of blatant marketing rhetoric, but the tech is good and if is getting better. If you've not tried it very recently, give it another shot.

pauldix 11y ago

The current version still has the problem that if you do a massive SELECT then it will fill up memory until the process gets killed. In the future we'll give controls to limit how much memory a query can take. I think it's common in databases that if you throw a massive query at it that the server doesn't have the resources to handle, things will go wrong. It'll thrash, or crash, or generally have poor performance. We'll be working on improving the failure conditions, but if you put a query to a database that is too big for the server to handle, some sort of failure scenario will be encountered. Just a question of how it's handled. Also, there's no new query planner in 0.10. We have a bunch of work getting merged in for the query engine for 0.11. But if you throw a huge query at the DB that the server can't handle, you'll still have great sadness. Which DB ended up solving your big query problems? Maybe we're just a poor fit? Or what kinds of queries should we be working on optimizing?

u320 11y ago
> I think it's common in databases that if you throw a massive query at it that the server doesn't have the resources to handle, things will go wrong. This has never ever happened to me in the last 20 years I've used SQL databases. Sure it might take a long time but that's it.
thatsad 11y ago
The current version still has the problem that if you do a massive SELECT then it will fill up memory until the process gets killed. That means any ad-hoc query can trivially crash your server. That's pretty serious. It's even worse if the ingest is using statsd-influxdb-backend with udp since during the crash you'll lose data. Do you really think it's ok for a novice user to be able to crash the server from your web gui if they make a minor mistake (e.g something like SELECT value FROM /series.*/ not realizing how many series their regex actually applies to until it's too late and the has gui stopped responding)? I think it's common in databases that if you throw a massive query at it that the server doesn't have the resources to handle, things will go wrong. This thinking is worse then wrong. Not only is it wrong - a simple unqualified select won't crash any "common" database - it makes anyone with any "common" database experience doubt your other claims.
fweespeech 11y ago
> Did you folks fix the "oops, I accidentally selected too much and OOMed the machine" crash that was supposed to be resolved by the new query planner in .10? > The current version still has the problem that if you do a massive SELECT then it will fill up memory until the process gets killed. In the future we'll give controls to limit how much memory a query can take. ... It locks on the query until its killed manually and/or kills it automatically. It doesn't crash.