paddyforan

11 ms·

61.
by paddyforan 14y ago
And my application will certainly have its single points of failure. But they will be things that I could not architect around with new software.
62.
by paddyforan 14y ago
> thanks for asking me if I am "familiar with the basics" of a DHT. I get the impression that you feel I asked this in a demeaning manner. I had no such intent. But some of your statements seem to contradict the fundamentals of a DHT, s
63.
by paddyforan 14y ago
I mean this in the most neutral and unassuming way possible, but... Are you familiar with the basics of a distributed hash table? To say that "each machine is potentially a single point of failure" is... inaccurate. According to the algorit
64.
by paddyforan 14y ago
A consistent hash ring is a similar concept, I think. We're quickly approaching the out-of-my-depth line, though. It's about maintaining the property that you can route information from many different places and have that information end in
65.
by paddyforan 14y ago
That is what a distributed has table is. And the values and keys aren't lost--there's just some confusion over what the keys and values are. The keys and values, at least in Pastry's case, are the NodeID and the metadata about the Node, res
66.
by paddyforan 14y ago
This sounds like something I would have loved to come across. It didn't come up in any google searches or any conversations with other software people who I talked to about this problem, so I didn't evaluate it. Quite simply, I had no idea
67.
by paddyforan 14y ago
Yup. Totally could have made a choice I wasn't happy with. Opted to make a choice I was happy with instead.
68.
by paddyforan 14y ago
They both are the case. Well, sort of. The messages can go to any number of nodes as part of the routing, but they're only considered "delivered" at one node. However, nodes can die with minimal loss of data. The problem is you're confusing
69.
by paddyforan 14y ago
Hm. Very interesting, I'll have to take another look and get a better understanding of it.
70.
by paddyforan 14y ago
I was very excited when I saw NSQ. I thought it would solve my problem splendidly. It seemed, from my reading of the docs, that it did not support multicast however. :( I was very impressed with it, though. You guys did an awesome job, and
71.
by paddyforan 14y ago
I think we may have to agree to disagree here. You do not like the way I expressed my dissatisfaction with the previously available solutions, and I do not like they way you expressed dissatisfaction with my attempt to explain my predicamen
72.
by paddyforan 14y ago
It can, however, have bottlenecks. All of your solutions, based on the conversation below, separate the messaging into its own service. Which isn't really how I want to work with my software. I guess the best way I can describe it is as a d
73.
by paddyforan 14y ago
I thought you just missed this. Apparently I was wrong. I think you're missing the point a little.
74.
by paddyforan 14y ago
Er. This seems a little overly hostile for something that amounted to "Hey, here's this fun thing I made. Anyone else want to play with it?" Rather than arguing this point again, I'll just point at the other comment thread that addressed th
75.
by paddyforan 14y ago
Not off the top of my head, no. Sorry. :( I know there's http://tent.is , and there's a Go client for Tent out there, but that's really all I know about it.
76.
by paddyforan 14y ago
Hm. That's the closest I've got. How's your Go? You can build one on Pastry. :)
77.
by paddyforan 14y ago
http://tent.io/ ?
78.
by paddyforan 14y ago
No, I definitely appreciate the conversation, because I looked for forever for a pre-built solution to this, and couldn't figure out how nobody had needed a solution before now. So I'm glad there are solutions, and I'm not just crazy.
79.
by paddyforan 14y ago
Sounds like it would work, and have great availability and scalability. It just, conceptually, doesn't mesh with how I think about the system. An absolutely aesthetic distinction, I know.
80.
by paddyforan 14y ago
This reply link just appeared for me. Not cool -_- The lookup server is a single point of failure. And while it's not a hugely problematic one, it also is a dedicated machine whose sole purpose is keeping the other machines running. Which f
81.
by paddyforan 14y ago
Confusion is intentional. This is just an implementation of that algorithm, with a few tweaks (that are explained, along with their reasoning, in the README). I believe I even link to that Wikipedia page in the blog post and the README.
82.
by paddyforan 14y ago
That it is. I love redis. I just really didn't want to have a single point of failure. Really badly. Pastry has no single point of failure. [edit] Realised this thread isn't actually all about me. Oops! Sorry for answering a question not di
83.
by paddyforan 14y ago
Very cool. Where were you six months ago? ;) I was familiar with doozer, but doozer wasn't(/isn't) being actively maintained and it doesn't compile against the latest version of Go, so I'd have to bring it up to speed before I could use it
84.
by paddyforan 14y ago
I guess I was wrong about hard-coding the IPs. Maybe? It doesn't have many-to-many baked in, though. That I can see, at least. It looks like I may have been able to make this work, but the setup would be a little complex. ... because a dist
85.
by paddyforan 14y ago
I'll be using the Scribe algorithm, as defined by the authors of the Pastry algorithm: http://en.wikipedia.org/wiki/Pastry_(DHT)#SCRIBE The basic gist of it is similar to a phone tree. You route a message subscribing to a topic, basing th
86.
by paddyforan 14y ago
I believe I looked at that as well. It was six months ago, so forgive me if my memory is a bit off. Looking at the intro, it looks like I have to hardcode the IPs and ports of all of the machines I want to publish or subscribe to. Is that c
87.
by paddyforan 14y ago
I looked at it, I promise I did. Really wanted to avoid writing this if I could. From what I can see in the docs, RabbitMQ is a client/server relationship. Meaning there is a server. Meaning a single point of failure and a bottleneck. I hat
88.
by paddyforan 14y ago
Thanks! Fixed the typo. And I guess my point is, I've tested this. I can't seem to find any problems with it. But I do not have unit tests for the larger routing algorithm, only the routing within a Node. In my tests, it always worked out,
89.
by paddyforan 14y ago
Please do let me know how this works out for you. And I mention it in the blog post, and I mention it in the README, but I feel I should reinforce this: This. Is. Alpha. Please don't use it in your mission-critical code. I'd like to run a l
90.
Pastry: A distributed hash table in Go(secondbit.org)
97 points by paddyforan 14y ago|70 comments

More ›