4 ms·

I'm not quite sure what you're doing. If you have that level of hardware I don't know why you're running out of memory. Our tests with 100B points split across

by pauldix 11y ago

I'm not quite sure what you're doing. If you have that level of hardware I don't know why you're running out of memory. Our tests with 100B points split across 1M series ran on much more modest hardware than that and didn't have a problem. There's something different with what you're doing than with what we're testing. If you can give more detail about what your actual data looks like, I may be able to help. Are you doing this on v0.10 (beta1 or greater) and you're seeing this problem?

kev009 11y ago
I don't have a lot of time to spare on this and we've found a viable option so I won't be using influx any time soon. I can help you with access to my production system if you are interested and as you are watching arrange for between 1k and 10k hosts (for 2 - 20 values per second)
- monkpit 11y ago
  I'm sad that pauldix disappears when the comments get direct like this - I've used influxdb at smaller scales and I'm genuinely curious to see responses to a few different threads going on in this discussion. But they are left without a reply as of now.
  - pauldix 11y ago
    I disappeared because I can't spend all evening responding to every person on the internet that may be having trouble with our software. An offer to SSH into a system that isn't running Influx and trying to troubleshoot some ambiguous problem isn't direct. We have people that have given us useful troubleshooting information that we've helped (and have helped us improve InfluxDB). Constructive criticism with an offer to help us improve is the best. While I'd love to trace down every problem for everyone on the internet, my time on this planet is limited. As for your concerns about if it scales, we'll try to put out more benchmarks and test code over the coming weeks. For reference, we built a tool for stress testing that you can see here: https://github.com/influxdata/influxdb/tree/master/cmd/influx_stress https://github.com/influxdata/influxdb/tree/master/cmd/influ... You'll have to make your own decisions.
    - kev009 11y ago
      We would install Influx, as a go app that is fairly trivial and it speaks the same collection protocol... As a random problem on the internet, this is the workload you should expect from any midsize business and probably something your company would like to target for revenue. Good luck.
  - kev009 11y ago
    This is why I said they are great at marketing. There's really little value in working with me, but at least responding while the article is front page makes it look like they are responsive and looking into issues. Maybe they can sink a big user/customer that's willing to gut through all the pain of helping to make an operation data store.. someone in this thread mentioned mongo and it sounds very familiar. I'm wondering if I'm being too negative but the flip side of this is that it caused me and others a lot of stress, lost face, and time. Most developers and operators are overworked already so I am trying to save some pain since the blog post makes it seem like everything is fine and awesome.
  - kylemathews 11y ago
    He replied an hour ago...
    - monkpit 11y ago
      An hour after I said that, yeah. But not in the like 6 hours prior while he was answering other stuff.
  - JulianWasTaken 11y ago
    This is a very odd thread. I'm in no way involved in influx, we're just evaluating it, but this entire thread reads like complete FUD at the moment, so it's odd that you're calling out a maintainer or whoever pauldix is for "disappearing".
    - kev009 11y ago
      If you're just now evaluating it, you might not realize the history behind the project. It's more backlash than FUD.. infrastructure people don't like to be jerked around by marketing tactics any more than we already are. At various times each of the storage engines was promoted as the bee's knees LevelDB -> BoltDB -> bz1 -> tsm1. That kind of fumbling speaks for itself. It's strange for a high profile investment (including YC). Founded/funded around the same time contrast historically to FoundationDB.
      - misframer 11y ago
        > Founded/funded around the same time contrast historically to FoundationDB. FoundationDB was in stealth mode for at least three years before they launched their closed Alpha release, so perhaps this isn't a fair comparison. (I was an intern at FoundationDB during summer 2012.)
    - monkpit 11y ago
      At the time I wrote the comment, there were direct questions left unanswered for 6 hours, while replies were given to other less-interesting comments in the meantime - that to me looks like someone ignoring something. He came back, that's great!
- pauldix 11y ago
  If you found something else that works for you, that's great. I'm sorry that InfluxDB didn't work for you when you tried it, but we'll continue to improve and build. My feeling is that this release is a significant milestone in our evolution. We'll see how people react over the coming weeks after actually using it.
jsmthrowaway 11y ago
> If you have that level of hardware I don't know why you're running out of memory. Did you folks fix the "oops, I accidentally selected too much and OOMed the machine" crash that was supposed to be resolved by the new query planner in .10? That was my immediate showstopper for InfluxDB for a very high-volume infrastructure, because I didn't feel like proxying InfluxDB just to enforce chunking, and the mere existence of the bug gave me pause. I hit that in 24 hours and shelved the system in 48. Even pre-1.0, I'm not encouraged when I have to "work around" pretty obvious oversights in reliability, so I put you back on the back burner to give you time to mature and transitioned back to a Kappa-style architecture for my needs. I might still use InfluxDB for low-volume aggregations out of my stream processing, but it's for sure out of the hot path for the foreseeable future.
- SEJeff 11y ago
  That is more or less how my employer worked around this glaring bug. That being said, we've worked together with Influx to fix some issues or add features to Telegraf and even have had their developers fix several of our issues found under excessive load (/me waves to Alex and Sebastian), Influx has a long way to go, but I've got faith that they're going in the right direction and things are getting a lot better. It is true this announcement is a bit too full of blatant marketing rhetoric, but the tech is good and if is getting better. If you've not tried it very recently, give it another shot.
- pauldix 11y ago
  The current version still has the problem that if you do a massive SELECT then it will fill up memory until the process gets killed. In the future we'll give controls to limit how much memory a query can take. I think it's common in databases that if you throw a massive query at it that the server doesn't have the resources to handle, things will go wrong. It'll thrash, or crash, or generally have poor performance. We'll be working on improving the failure conditions, but if you put a query to a database that is too big for the server to handle, some sort of failure scenario will be encountered. Just a question of how it's handled. Also, there's no new query planner in 0.10. We have a bunch of work getting merged in for the query engine for 0.11. But if you throw a huge query at the DB that the server can't handle, you'll still have great sadness. Which DB ended up solving your big query problems? Maybe we're just a poor fit? Or what kinds of queries should we be working on optimizing?

by pauldix 11y ago

kev009 11y ago

I don't have a lot of time to spare on this and we've found a viable option so I won't be using influx any time soon. I can help you with access to my production system if you are interested and as you are watching arrange for between 1k and 10k hosts (for 2 - 20 values per second)

monkpit 11y ago
I'm sad that pauldix disappears when the comments get direct like this - I've used influxdb at smaller scales and I'm genuinely curious to see responses to a few different threads going on in this discussion. But they are left without a reply as of now.
- pauldix 11y ago
  I disappeared because I can't spend all evening responding to every person on the internet that may be having trouble with our software. An offer to SSH into a system that isn't running Influx and trying to troubleshoot some ambiguous problem isn't direct. We have people that have given us useful troubleshooting information that we've helped (and have helped us improve InfluxDB). Constructive criticism with an offer to help us improve is the best. While I'd love to trace down every problem for everyone on the internet, my time on this planet is limited. As for your concerns about if it scales, we'll try to put out more benchmarks and test code over the coming weeks. For reference, we built a tool for stress testing that you can see here: https://github.com/influxdata/influxdb/tree/master/cmd/influx_stress https://github.com/influxdata/influxdb/tree/master/cmd/influ... You'll have to make your own decisions.
  - kev009 11y ago
    We would install Influx, as a go app that is fairly trivial and it speaks the same collection protocol... As a random problem on the internet, this is the workload you should expect from any midsize business and probably something your company would like to target for revenue. Good luck.
- kev009 11y ago
  This is why I said they are great at marketing. There's really little value in working with me, but at least responding while the article is front page makes it look like they are responsive and looking into issues. Maybe they can sink a big user/customer that's willing to gut through all the pain of helping to make an operation data store.. someone in this thread mentioned mongo and it sounds very familiar. I'm wondering if I'm being too negative but the flip side of this is that it caused me and others a lot of stress, lost face, and time. Most developers and operators are overworked already so I am trying to save some pain since the blog post makes it seem like everything is fine and awesome.
- kylemathews 11y ago
  He replied an hour ago...
  - monkpit 11y ago
    An hour after I said that, yeah. But not in the like 6 hours prior while he was answering other stuff.
- JulianWasTaken 11y ago
  This is a very odd thread. I'm in no way involved in influx, we're just evaluating it, but this entire thread reads like complete FUD at the moment, so it's odd that you're calling out a maintainer or whoever pauldix is for "disappearing".
  - kev009 11y ago
    If you're just now evaluating it, you might not realize the history behind the project. It's more backlash than FUD.. infrastructure people don't like to be jerked around by marketing tactics any more than we already are. At various times each of the storage engines was promoted as the bee's knees LevelDB -> BoltDB -> bz1 -> tsm1. That kind of fumbling speaks for itself. It's strange for a high profile investment (including YC). Founded/funded around the same time contrast historically to FoundationDB.
    - misframer 11y ago
      > Founded/funded around the same time contrast historically to FoundationDB. FoundationDB was in stealth mode for at least three years before they launched their closed Alpha release, so perhaps this isn't a fair comparison. (I was an intern at FoundationDB during summer 2012.)
  - monkpit 11y ago
    At the time I wrote the comment, there were direct questions left unanswered for 6 hours, while replies were given to other less-interesting comments in the meantime - that to me looks like someone ignoring something. He came back, that's great!
pauldix 11y ago
If you found something else that works for you, that's great. I'm sorry that InfluxDB didn't work for you when you tried it, but we'll continue to improve and build. My feeling is that this release is a significant milestone in our evolution. We'll see how people react over the coming weeks after actually using it.

jsmthrowaway 11y ago

> If you have that level of hardware I don't know why you're running out of memory. Did you folks fix the "oops, I accidentally selected too much and OOMed the machine" crash that was supposed to be resolved by the new query planner in .10? That was my immediate showstopper for InfluxDB for a very high-volume infrastructure, because I didn't feel like proxying InfluxDB just to enforce chunking, and the mere existence of the bug gave me pause. I hit that in 24 hours and shelved the system in 48. Even pre-1.0, I'm not encouraged when I have to "work around" pretty obvious oversights in reliability, so I put you back on the back burner to give you time to mature and transitioned back to a Kappa-style architecture for my needs. I might still use InfluxDB for low-volume aggregations out of my stream processing, but it's for sure out of the hot path for the foreseeable future.

SEJeff 11y ago
That is more or less how my employer worked around this glaring bug. That being said, we've worked together with Influx to fix some issues or add features to Telegraf and even have had their developers fix several of our issues found under excessive load (/me waves to Alex and Sebastian), Influx has a long way to go, but I've got faith that they're going in the right direction and things are getting a lot better. It is true this announcement is a bit too full of blatant marketing rhetoric, but the tech is good and if is getting better. If you've not tried it very recently, give it another shot.
pauldix 11y ago
The current version still has the problem that if you do a massive SELECT then it will fill up memory until the process gets killed. In the future we'll give controls to limit how much memory a query can take. I think it's common in databases that if you throw a massive query at it that the server doesn't have the resources to handle, things will go wrong. It'll thrash, or crash, or generally have poor performance. We'll be working on improving the failure conditions, but if you put a query to a database that is too big for the server to handle, some sort of failure scenario will be encountered. Just a question of how it's handled. Also, there's no new query planner in 0.10. We have a bunch of work getting merged in for the query engine for 0.11. But if you throw a huge query at the DB that the server can't handle, you'll still have great sadness. Which DB ended up solving your big query problems? Maybe we're just a poor fit? Or what kinds of queries should we be working on optimizing?