hank777

8 ms·

31.
by hank777 18y ago
Yeah, certainly at the small scale there can be a myriad of reasons why a system doesn't perform. At a large scale its hard to imagine real world scenarios where storage has not been the bottleneck. Regarding whether twitter should be using
32.
by hank777 18y ago
Well perhaps its a matter of staring at something so long it becomes simple. But I really don't think what I am suggesting is that hard. The main thing that requires some thought is shard splitting. That is not a big piece of code, but it d
33.
by hank777 18y ago
Nevertheless you believe that there is no way to handle a distributed in-memory lookup system. It just goes to show you cant know everything.
34.
by hank777 18y ago
uhh... yes it requires a lookup. There is no problem with that since the lookup is not centralized but local. How... read about terracotta. No I dont work for the company just use their open source product.
35.
by hank777 18y ago
I think your math is off here. Basically you end up with this equation cost = ((NumOfUsers * avgMsgsRevceivedPerUser)/maxWritesPerCPU) * costOfCPU There is nothing quadratic here. Your are able to scale linearly with the number of messages
36.
by hank777 18y ago
I dont understand. I am saying that Terracotta, which is an open source product, handles the in memory table and synching stuff I was talking about and that you asked about. Its like you dont have to write a C compiler and you dont need to
37.
by hank777 18y ago
This is true. Though I can say that if one were building twitter and you didnt either have a design like the one I suggested, or a design with some kind of distributed caching then you are going to have exactly the kinds of scaling problems
38.
by hank777 18y ago
true, but it makes me curious what brilliance someone bring to the table when they have nothing specific to say other than you are wrong. Also given that much of what I am talking about is running code, I at least know that it solves a majo
39.
by hank777 18y ago
Well ok. I dont have to call them database problems, but I object to someone saying they are not database problems. I agree it is a bit of semantics and I would not have brought it up if he hadnt. Although I have to say I consider this
40.
by hank777 18y ago
Yes, I am defining database as "anything that stores data" It does not have to be SQL for me to consider it a database problem. And in fact my solution is well above the sql level.
41.
by hank777 18y ago
Actually, this is the beauty of the design: All of the writes happen in parallel . So there if you write to 2000 recipients, but all of the writes happen in parallel you have no bottleneck. And any time a given machine's load gets too heav
42.
by hank777 18y ago
I considered this, but given that the messages are small and the primary issue is scaling not storage this, it does not seem to me is a good solution. The entirety of everything ever written on twitter is probably not big enough to worry ab
43.
by hank777 18y ago
With all due respect andrew why dont you share your technical analysis of what exactly I am saying that is wrong here. Btw, have you ever actually worked on any database scaling problems? Just checking.
44.
by hank777 18y ago
Uh... yes it is. All scaling problems are database problems. ALL of them. They can be solved by different strategies such as caching, distributed design, etc. But at the end of the day the problem is always that you cant get to the data
45.
by hank777 18y ago
Its fascinating funny how you are so smart you can't even deign to share your wisdom on this. If I am wrong, tell me where. Given that I have spent the last several years studying/developing distributed databases if I just don't know what I
46.
by hank777 18y ago
Point taken. I really hadnt thought of things like linux distros. Was thinking more of movies tv shows etc.
47.
by hank777 18y ago
Actually, the two companies I mentioned - Kontiki and Red Swoosh have amazing QOS. Bit Torrent sucks for that, but it is an engineering issue, not an architectural or conceptual one. Akamai actually acquired red swoosh, and is integrating
48.
by hank777 18y ago
Huh? 1. This is not about replacing torrents. There wouldn't be much economic impact in that since that stuff is already stolen and in any case torrents are not on any CDN network. The point is that sites like youtube, hulu, and any other f
49.
by hank777 18y ago
with all due respect this is just not true. I just did the first few tests and it seemed to nail all of them in the first answer or two in the list.
50.
by hank777 18y ago
Indeed even the best search engines are only going to be able to answer that question if it is stated somewhere. Hakia can do that. However no search engine is going to be able to figure out the answer by, for example adding up all the time
51.
by hank777 18y ago
Excellent piece. The idea of the expensive hobbies is actually something that I had not thought of and is certainly another very interesting vector for offering services people will pay for.
52.
by hank777 18y ago
actually the scaling is absolutely an interesting problem. Its the reason they keep going down. because messages are not point to point it is a much harder problem than instant messaging. I have spent a fair amount of time considering the i
53.
by hank777 18y ago
I dont think it says anything about things people pay for changing the world or even that payment means something will change the world. In fact it says "Or if its free, at least figuring out how to do some stuff that has the potential to r
54.
by hank777 18y ago
actually no I am not ignorning his point. Its just not statistically meaningful. The truth is that of the people that click on ads , they are midwestern housewives. But there are statistically not enough of them to make many services t
55.
by hank777 18y ago
actually, as the commenter below indicated, according to jackob nielsen's research, I am absolutely right. http://www.useit.com/alertbox/banner-blindness.html Summary: Users rarely look at display advertisements on websites. Of the four d
56.
by hank777 19y ago
"But please don't say none of us is bold." I think you take things a bit too literally. This piece is about humanity in general, not "edw519"
57.
by hank777 19y ago
Life is about more than just business. And being bold isn't, at least in my context, about working harder. In fact it sounds like, for you, work is a default. I am talking about exploring things in life that you might not otherwise explore.
58.
by hank777 19y ago
Yes, but the question is whether any YC companies have the capacity to be the next google. I think it is a good question because it really is saying "if you can't find the gems why expect VCs to be able to". Its a people in glass houses thr
59.
by hank777 19y ago
Well, as I said, there is nothing that cant be expressed in an RDBMS - at least at first. But let me give an example of the kind of use cases we see. First, imagine having a database that allows one to freely create record types. One might
60.
by hank777 19y ago
I am the author of this article, and I must say, though I am not a regular reader of HN, I do find it refreshing that there are smart people here arguing real merits in an intelligent respectful way. I am sure it is not always the case, but

More ›