3 ms·

Look not to sound harsh but since you're claiming the "80mm ops/sec" as a technical achievement "not possible with other databases" I found it to be fair to act

Look not to sound harsh but since you're claiming the "80mm ops/sec" as a technical achievement "not possible with other databases" I found it to be fair to actually review the benchmark you linked. What you're benchmarking for the "read" is this: benchmark(function(){ gun.val(ok); }); Looking at what the gun.val call does it seems to be more or less an identitiy/noop function (that simply returns its input). It appears your benchmark is basically testing how fast you can do a simple javascript function call which has nothing to do with any database operation whatsoever? How is that useful? The benchmark doesn't reflect the performance of your gun database more than it reflects the performance of _any javascript application_. The only thing that is being benchmarked here is the javascript interpreter. Seriously, this is like somebody claiming their database runs at 4 billions ops/sec because that is how many instructions the CPU can perform. Given how much effort you (clearly) put into your marketing and the way you worded this as a "cached read" I think you probably know all of this though -- so here's a plea: Please don't pull these kinds of completely dishonest stunts (you even put the BS numbers in the title of this submission). You just make others in the javascript [database] space look bad by association. Cheating on benchmarks will not convince anybody but the most junior web developers. source (hopefully I'm wrong ;) ): https://github.com/amark/gun/blob/master/test/ptsd/perf.js#L1999 https://github.com/amark/gun/blob/master/test/ptsd/perf.js#L...

marknadal 9y ago
The nitty gritty details are on this readthesource.io podcast: https://youtu.be/70dn1oZQFCk https://youtu.be/70dn1oZQFCk (watch on 2X speed). Each process does concurrency control and then has a centralized in-memory cache for the values (like what a lot of other in-memory databases do). So when `gun.val(cb)` is called, prototype context holds its value and is able to do an immediate read - without this, JS is so slow that every function call logarithmically decreases your performance (see the previously linked https://youtu.be/BEqH-oZ4UXI https://youtu.be/BEqH-oZ4UXI ). That is the advantage of the realtime/push-based model, you can cache most data before it is even read. Even in memory databases, like Redis and others, do this so there is no BS here, but I appreciate you trying to call it out. Additionally, not all in-memory reads are equal - my previous implementation of gun could only do a thousand or so reads/sec despite being cached. And please compare against other in-memory javascript database benchmarks: https://github.com/techfort/LokiJS/wiki/Indexing-and-Query-Performance https://github.com/techfort/LokiJS/wiki/Indexing-and-Query-P... (Joe's work is very good!) I sincerely wish it was as easy as leaving it up to the JS interpreter ;) but I've unfortunately found JS to be very slow. :P Hit me up with any other hard questions or if I missed anything. And please keep on trying hard to call out database vendors - I agree, it is extremely important to keep us open and honest. :) Cheers!
- ccommsxx 9y ago
  I don't really get what you're trying to say. Are you defending the fact that you benchmarked whats essentially a "function() { return 1; }", called that a "database operation" and then proceeded to claim to have achieved a level of performance "not possible with other databases"? Can you really not see how that is wrong on more than just one level? Or are you saying it's okay because other "javascript database" vendors are also cheating in their benchmarks?
  - marknadal 9y ago
    Woah woah, that is not true! `gun.val(cb)` performs a real dynamic read, you can even uncomment the `console.log(val)` to see it printed in the debugging console (although note: `console.log` has terrible performance, so don't have it on when you are actually benchmarking). Other javascript databases and even other regular in-memory databases do similar benchmarks. We (both them and us) note at the bottom of the article that: "Take all performance testing benchmarks with a huge grain of salt." Which is why we have our PANIC tests (see my other comments on where to find them and how to run them, let me know if you need any help). There is nothing unethical about our tests (and they are not the only tests), so /please/ call me out, but please don't misinform other readers that the op is a static function, that is misleading and damaging. Is there anything specific I can do to alleviate any concerns?

marknadal 9y ago

The nitty gritty details are on this readthesource.io podcast: https://youtu.be/70dn1oZQFCk https://youtu.be/70dn1oZQFCk (watch on 2X speed). Each process does concurrency control and then has a centralized in-memory cache for the values (like what a lot of other in-memory databases do). So when `gun.val(cb)` is called, prototype context holds its value and is able to do an immediate read - without this, JS is so slow that every function call logarithmically decreases your performance (see the previously linked https://youtu.be/BEqH-oZ4UXI https://youtu.be/BEqH-oZ4UXI ). That is the advantage of the realtime/push-based model, you can cache most data before it is even read. Even in memory databases, like Redis and others, do this so there is no BS here, but I appreciate you trying to call it out. Additionally, not all in-memory reads are equal - my previous implementation of gun could only do a thousand or so reads/sec despite being cached. And please compare against other in-memory javascript database benchmarks: https://github.com/techfort/LokiJS/wiki/Indexing-and-Query-Performance https://github.com/techfort/LokiJS/wiki/Indexing-and-Query-P... (Joe's work is very good!) I sincerely wish it was as easy as leaving it up to the JS interpreter ;) but I've unfortunately found JS to be very slow. :P Hit me up with any other hard questions or if I missed anything. And please keep on trying hard to call out database vendors - I agree, it is extremely important to keep us open and honest. :) Cheers!

ccommsxx 9y ago
I don't really get what you're trying to say. Are you defending the fact that you benchmarked whats essentially a "function() { return 1; }", called that a "database operation" and then proceeded to claim to have achieved a level of performance "not possible with other databases"? Can you really not see how that is wrong on more than just one level? Or are you saying it's okay because other "javascript database" vendors are also cheating in their benchmarks?
- marknadal 9y ago
  Woah woah, that is not true! `gun.val(cb)` performs a real dynamic read, you can even uncomment the `console.log(val)` to see it printed in the debugging console (although note: `console.log` has terrible performance, so don't have it on when you are actually benchmarking). Other javascript databases and even other regular in-memory databases do similar benchmarks. We (both them and us) note at the bottom of the article that: "Take all performance testing benchmarks with a huge grain of salt." Which is why we have our PANIC tests (see my other comments on where to find them and how to run them, let me know if you need any help). There is nothing unethical about our tests (and they are not the only tests), so /please/ call me out, but please don't misinform other readers that the op is a static function, that is misleading and damaging. Is there anything specific I can do to alleviate any concerns?