ankrgyl

12 ms·

91.
by ankrgyl 11y ago
Most relatively mature database implementations do in fact apply this optimization. These databases have a component which can estimate the cost and benefit of such operations before running the query. The optimization that you mentioned tu
92.
by ankrgyl 11y ago
Hi @superlogical, thank you for using it and for the kind feedback! CloudFormation by itself provides some very basic level of conditional/looping logic, but we found that it was not enough yet to provide a stellar experience. So, when
93.
by ankrgyl 12y ago
Interesting... it's actually associativity that matters for this class of distributed query execution problems (in particular, for AVG). While a/b != b/a indeed violates commutativity, the reason AVG doesn't distribute i
94.
by ankrgyl 12y ago
Happy to fill in the gaps here. Building a data structure that is lock free, performant at concurrent reads and writes, and fast at scanning data is very challenging. Skip lists enable us to implement this in a straightforward way, but are
95.
by ankrgyl 12y ago
This post is missing a key trade-off that you get with an in-memory row store (tl;dr ability to build lock-free), but also since 6/25/2012 several other innovations in MemSQL that have shipped over the past 2+ years. The main adva
96.
by ankrgyl 12y ago
Interestingly enough, in some ways it does. For example, during our nightly stress runs the fact that the system is under so much pressure helps induce race conditions that are otherwise pretty much impossible to reproduce.
97.
by ankrgyl 14y ago
Yes, we will fan out queries when necessary. In a hard core oltp workload this will affect latency across the system by flooding the network (if you send too many of these queries at once) but we expose knobs that let you limit these querie
98.
by ankrgyl 14y ago
http://developers.memsql.com/docs/2.0/2.0releasenotes.html
99.
by ankrgyl 14y ago
@Dhruv - The significant increase in number of rows is because of throughput. The experiment was run for the same length for all three databases, but MemSQL sustained a higher throughput, resulting in more data inserted. Refer to the simula
100.
by ankrgyl 14y ago
Referring to https://github.com/memsql/bench/blob/master/sql_database.py , we use _mysql, not mysqldb. Here's some code that demonstrates: In [1]: import _mysql In [2]: c = _mysql.connect(host='127.0.0.1', user='root') In
101.
by ankrgyl 14y ago
It has more to do with how much data is stored. MemSQL stores more than 30 times as many rows as MySQL by the end of the experiment.
102.
by ankrgyl 14y ago
Hey Dhruv, In the case of the 24 core machine... The database is initialised with 560000 players. For every action an event is written to the database, so at the end of the 10 minute run there are ~aps * 600 = 1.2 million rows for MySQL and
103.
by ankrgyl 14y ago
We used the latest stable versions (as mentioned in the article). It's now updated to include the exact versions: MongoDB 2.0.6, MySQL 5.5.25, MemSQL 1b
104.
by ankrgyl 14y ago
Good point- ours are currently unidirectional because they have to be lock free. We have some ideas on how to keep them lock free and make them bidirectional, but that's not part of the current release.
105.
by ankrgyl 14y ago
@calinet126 We have the durability interface and configuration options here http://developers.memsql.com/docs/1b/durability.html This should give you a good idea about how durability will react to tuning, but it doesn't dive deep into the
106.
by ankrgyl 14y ago
Sharded/replicated MemSQL is in the works. And yep, the replication will be compatible with MySQL replication to enable grouping MemSQL and MySQL nodes.
107.
by ankrgyl 15y ago
No, we ran an extensive benchmark against log data and found that simplejson was indeed 10x faster. Your benchmark assumes a different "shape" of json dictionary than ours, and I would recommend updating your methodology to use real data in
108.
by ankrgyl 15y ago
Author of the post here. I agree that 0 and 1 aren't the best choices for a response, but you have to keep backwards compatability in mind. We wanted this change to be transparent to people who have production code that interacts with the A
109.
by ankrgyl 15y ago
That's exactly the point we wanted to convey. Erlang and node are great, but we know Python really well and were able to write a performant server with the tools we're familiar with.
110.
How and Why Mixpanel Switched from Erlang to Python(code.mixpanel.com)
139 points by ankrgyl 15y ago|80 comments
111.
by ankrgyl 15y ago
That's really cool, and is the kind of good benchmark I was referring to. I'm curious to see how it compares to an iteratee approach. I suspect in this case that there wouldn't be much of a difference, because iteratees are suited especiall
112.
by ankrgyl 15y ago
Scala pulls it off. Here is one example: http://www.bluishcoder.co.nz/2006/07/scala-futures-and-lazy-...
113.
by ankrgyl 15y ago
Great explanation. Just a small point though, > "lazy" is the opposite of "imperative." should be: "lazy" is the opposite of "strict" or "eager". "Imperative", in this context, is the opposite of "functional".
114.
by ankrgyl 15y ago
This rant strives to dispel two myths: 1.) Haskell is "a bad language to write IO intensive programs in" 2.) "maintenance is more expensive as Haskell programmers are thin on the ground" I was convinced of neither. First, just because your
115.
Contrasting Haskell & Erlang in peer-to-peer protocol implementation(infoq.com)
2 points by ankrgyl 15y ago|0 comments
116.
by ankrgyl 15y ago
Erlang, and other similar efforts like Haskell's forkIO and Python's eventlet/gevent, are also built upon the same fundamentals as libev (and some actually just use libev). But using libev in the way that Node.js does boils down to user-spa
117.
by ankrgyl 15y ago
That works really well for handling requests for HTML pages, because they tend to render independently of one other. However, you run into trouble when you to make "Nodes" communicate, and the comment that the article is addressing specific
118.
by ankrgyl 15y ago
The distinction between parallelism and concurrency is extremely important. The guys who wrote Real World Haskell did a good job of explaining it here http://book.realworldhaskell.org/read/concurrent-and-multico... (explanation has noth
119.
Node.js is Backwards(blog.ankurgoyal.com)
144 points by ankrgyl 15y ago|85 comments
120.
by ankrgyl 15y ago
I'll add a couple of courses I've really enjoyed at CMU ( http://www.cs.cmu.edu ): 15-410 Operating System Design and Implementation: http://www.cs.cmu.edu/~410/ 15-251 Great Theoretical Ideas in Computer Science (webpage might be out of

More ›