6 ms·
A database without any test harness? While this could be a good toy or PoC I would never use it in production. Readers should be aware, just because it's on HN
by maxpert 5y ago
A database without any test harness? While this could be a good toy or PoC I would never use it in production. Readers should be aware, just because it's on HN doesn't mean it's production ready.
- corerman 5y agoProject tests have been added, from open source community partners, thanks for the initial attention.
- hughrr 5y agoIt uses Raft underneath as well which means there's a bunch of non-determinism and hell for anyone who invokes it as well from experience. The thing is cursed. Source: several years dealing with vault and consul.
- cortesoft 5y agoWhat is a better consensus protocol to use?
- hughrr 5y agoLook higher up the problem domain and solve it without requiring a consensus protocol.
- cortesoft 5y agoHmm, how do you have high availability consistent data without a consensus protocol? No matter where in the problem chain you move, you have to eventually solve that problem.
- hughrr 5y agoEasy you push the configuration to every target. Then they are logically consistent. We did this for 30 years fine before someone invented this stack on deployments much larger then the average consul or vault deployment these days. I had something running 15,000 dynamic rps on Apache about 15 years ago. People are blinded from simplicity by complexity. Eventually complexity owns you. You can only own simplicity. At the end of the day this is one way to solve a problem that doesn’t need to be solved that someone has convinced you is a problem.
- asadawadia 5y ago>Easy you push the configuration to every target. Then they are logically consistent. What does this mean? This doesn't mean anything? Are you saying to push the DB to the client?
- caffeine 5y agoI think he’s saying you have a cronjob with an rsync to every client
- cortesoft 5y agoPeople need distributed consistent data for a lot of things besides configuration.
- yencabulator 5y agoPeople say they do, but then they proceed to read stale data from replicas all the time...
- rubiquity 5y agoNot sure why you’re being down voted as this is definitely the path forward. The industry and research have both explored the polar opposites of weak consistency protocols and linearizibility and consensus on the other end. Understanding your domain and knowing how you can step down to something weaker than consensus will be critical for future applications.
- tempest_ 5y agoWe use consul a bit as some "light" service discovery and a KV store for a few things without much issue so far. What demons did you encounter with vault/consul?
- hughrr 5y agoWe had some massive problems including complete cluster collapse requiring rebuilds from scratch, eternal leadership elections and occasionally nodes would just entirely stop responding to KV requests causing cascading failures outside. Vault is a massive damage multiplier for these issues plus some other nasty ones like buggy barely supported plugins.
- zetsurin 5y agoWouldn't mind knowing what versions you were using/how long ago this was. We haven't seen anything like this in > 3 years. Possible a barely supported plugin caused issues?
- hughrr 5y agoThis is post 1.0 and without any plugins for consul. Vault has been a persistent point of failure. It’s a good idea in principle but it’s not got a good ROI
- chucky_z 5y agoHuh. I've been running Consul since 0.7 or so, and the only time any of this happened was my own fault. Most of the problems I've had with Vault have been around it's Terraform provider which they've improved enough that it's not an issue anymore. I think the only thing about Raft that folks don't realize is how disk hungry it gets, if you want fast write performance you gotta make dang sure all those fsyncs can keep up. Our largest Consul cluster today runs on storage-heavy boxes as it does ~500Mb/s of writes pretty much 24/7. I use Consul w/ Vault today instead of the internal storage for Vault just cause Consul has really nice monitoring around some stuff that Vault doesn't (path-based stuff for the most part), I think the internal storage is a really good option for 90% of use-cases.
- PYTHONDJANGO 5y agoPlease post URLs to bugs / issues to give your comment some cred. Thanks!
- tomnipotent 5y agoA database without any code, actually. It's less than a few hundred lines of Go that just wraps two other databases (syndtr/goleveldb and ledisdb/ledisdb) with a third library (tidwall/uhaha) that provides a Raft API.
- 3np 5y agoOooh that means I can form it to do Redis instead right? Because that could be a nice way to to Redis clustering
- tomnipotent 5y agoI suppose you could stick tidwall/uhaha directly in front of redis, but I'm not entirely certain what you'd call that... Here's the LSET code: https://github.com/gitsrc/IceFireDB/blob/main/lists.go#L232 https://github.com/gitsrc/IceFireDB/blob/main/lists.go#L232 func cmdLSET(m uhaha.Machine, args []string) (interface{}, error) { if len(args) != 4 { return nil, rafthub.ErrWrongNumArgs } index, err := ledis.StrInt64([]byte(args[2]), nil) if err != nil { return nil, err } if err := ldb.LSet([]byte(args[1]), int32(index), []byte(args[3])); err != nil { return nil, err } return redcon.SimpleString("OK"), nil } So what "IceFireDB" is: 1. tidwall/uhaha - Raft server (m uhaha.Machine, rafthub) 2. tidwall/redcon - Read/write redis protocol (redcon.SimpleString) 3. ledisdb/ledisdb - Redis-compatible with disk persistence via leveldb (ldb.LSet) 4. syndtr/goleveldb/leveldb - Provides snapshots, other scattered references throughout code It also includes this seemingly random file below, which seems to implement some string slice overloads using unsafe.Pointer: https://github.com/siddontang/go/blob/master/hack/hack.go https://github.com/siddontang/go/blob/master/hack/hack.go // no copy to change slice to string // use your own risk func String(b []byte) (s string) { pbytes := (*reflect.SliceHeader)(unsafe.Pointer(&b)) pstring := (*reflect.StringHeader)(unsafe.Pointer(&s)) pstring.Data = pbytes.Data pstring.Len = pbytes.Len return } // no copy to change string to slice // use your own risk func Slice(s string) (b []byte) { pbytes := (*reflect.SliceHeader)(unsafe.Pointer(&b)) pstring := (*reflect.StringHeader)(unsafe.Pointer(&s)) pbytes.Data = pstring.Data pbytes.Len = pstring.Len pbytes.Cap = pstring.Len return }