DmitryOlshansky

12 ms·

91.
by DmitryOlshansky 7y ago
If the dataset it fits on one node - good. You might still be missing out on reliability (should that node fail or just plain restart). Simple catalogs and/or web pages search usually fit in RAM, so the advice of disk:ram becomes less
92.
by DmitryOlshansky 7y ago
Indeed OpenDistro is a great option to have. Currently using ODFE Security plugin with X-Pack basic (minus X-Pack security module). Still waiting on these guys to review my perf patch though: https://github.com/opendistro-fo
93.
by DmitryOlshansky 7y ago
Last time I ran with ZGC the main problem was that ES OOM circuit-breakers are going nuts since they are not prepared to have memory mmaped 3 times. So their calculations are off by a factor of 3 blocking most of requests for no good reason
94.
by DmitryOlshansky 7y ago
I highly recommend trying out Java 11. G1 got quite a few improvements since 8. In particular full GC is parallel since 10: https://openjdk.java.net/jeps/307
95.
by DmitryOlshansky 7y ago
Certainly a problem with 100-s of nodes. One thing is that GC pauses in big clusters become more problematic as the chance to hit stop-the-world pause on at least one node for every request is getting higher. Going beyond dozens takes a f
96.
by DmitryOlshansky 7y ago
Currently we are running ES on big iron in production. There are both advantages and disadvantages. On plus side: - easier to manage several big machines than shitload of small instances potentially on top of virtualization solution (debugg
97.
by DmitryOlshansky 7y ago
And another note on shards - indexing a shard is a single writer process. If your drive tolerates parallel writes well (=SSD) having multiple primary shards per node helps scale indexing.
98.
by DmitryOlshansky 7y ago
Java compressed object pointers. Shipilev (JVM performance engineer) explains it in great detail here: https://shipilev.net/jvm/anatomy-quarks/23-compressed-refere... But the short of it: Java uses 32-bit numbers
99.
by DmitryOlshansky 7y ago
Mostly good stuff but a few comments: - article doesn’t clarify if it’s on hardware or VMs - 140 shards per node is certainly on the low side, one can easily scale to 500+ per node (if most shards are small, typically power law distribution
100.
by DmitryOlshansky 7y ago
Indeed. I think one of big problems in software development is that nobody measures the half-life of our assumptions. That is the amount of time it takes for half of the original assumptions to no longer hold. In my limited experience assum
101.
by DmitryOlshansky 7y ago
> Ted Glasser, who has a number of patents and is a very senior person in the computing industry, and is in Who's Who, once told people that the biggest team he could manage was one that could drive out to get pizza in a Volkswagen.
102.
by DmitryOlshansky 7y ago
Actually I’ve been through this but since we use Scala and our HTTP client wasn’t accounted for in default generators I went straight to writing a plugin for the ehm code-generator toolkit. Mostly same observations, but I actually made th
103.
by DmitryOlshansky 7y ago
Usually things like: 1u < -1 But more subtle when you have: a.size() - 1 < len
104.
by DmitryOlshansky 7y ago
I misread it as 12 minutes, and got really alert for a moment imagining some spooky mind-altering procedures. Damned age of velocity.
105.
by DmitryOlshansky 7y ago
It can’t be a dialect of a language if the compiler of said language doesn’t compile it.
106.
by DmitryOlshansky 7y ago
Invalidating cache lines is not that hard, it happens all the time. Refilling them from DRAM takes time.
107.
by DmitryOlshansky 7y ago
https://parallel.princeton.edu/papers/micro19-gao.pdf The amount of time wasted zeroing out memory pages in a typical OS is quite significant, also take into account that such an operation will trash perfectly good cac
108.
by DmitryOlshansky 7y ago
Adding an ability to zero-out arbitrary range directly to dram chip is very simple addition HW-wise. In fact all ones is not hard either, there were some experiments to give RAM ability to perform simple page-level computations with promisi
109.
by DmitryOlshansky 7y ago
Careful about the clouds, performance under virtualization has many pitfalls and in some workloads is very much unlike bare metal. The correlation of bare-metal cores vs cloud vCPUs is very much nonlinear.
110.
by DmitryOlshansky 7y ago
In a way each layer of abstraction could be seen as a virtual machine - a new set of “instructions” implemented via previous layer of instructions. The analogy holds as long as we don’t cross layers, which is quite often true. Therefore cou
111.
by DmitryOlshansky 7y ago
I used plain Sublime, but switched to VSCode with code-d plugin / dcd completion daemon. Usually works decently unless you tinker with std library / runtime where you are mostly on your own.
112.
by DmitryOlshansky 7y ago
512 / 8 = 64 bytes which is a cache line size in Intel’s (and AMD’s) CPUs.
113.
by DmitryOlshansky 7y ago
Can’t help but notice that most algorithms are implemented imperatively. Any chance for efficient pure functional string search algorithm?
114.
by DmitryOlshansky 7y ago
Whichever you know best
115.
by DmitryOlshansky 7y ago
The only good thing about time travel is that once we discover it we always had known it.
116.
by DmitryOlshansky 7y ago
Does not tell us the settings for text analysis done by two engines. Secondly on query side - again, scoring settings of RediSearch vs Elastic are not discussed. With that it’s just 2 points in space which gives us little information to ded
117.
by DmitryOlshansky 8y ago
I bet this heavily depends on what you consider an accurate definition.
118.
by DmitryOlshansky 8y ago
When it comes to biocomputing nothing beats humans challenged with “impossible” problems and decent amount of time.
119.
One reason I do not like Nginx that much
2 points by DmitryOlshansky 8y ago|0 comments
120.
I invented a nice brain trainer
3 points by DmitryOlshansky 8y ago|0 comments

More ›