jhgg

17 ms·

151.
by jhgg 5y ago
I would look into re-running the rust benchmarks w/ jemalloc or mimalloc instead of the system allocator and see how it performs.
152.
by jhgg 6y ago
We don't have any operational experience with Aerospike. We have a bunch of operational experience with C*/ScyllaDB.
153.
by jhgg 6y ago
The original implementation of this "service" was in Redis using lua scripts. We hit the performance ceiling of redis, and rather than deciding to go with redis-cluster, and dealing with the headache of managing persistent redis,
154.
by jhgg 6y ago
We kept dealing with GC pauses in go, and eventually after investing a bunch of time trying to fix this - we decided to research what it'd be like to rewrite it in another language. The service itself is pretty simple, so it wasn'
155.
by jhgg 6y ago
I think you're missing the point. We use Elixir to solve one class of problems which its excellent in doing (notably: real time distributed systems.) In this area, there is nothing even remotely close in terms of capability. So, we cho
156.
by jhgg 6y ago
No. React/TypeScript is too productive for our project engineers. But we do write our perf critical stuff in the client in C++/Rust! For example, our entire video capture pipeline is in C++/Rust (the meaty bits in rust, some
157.
by jhgg 6y ago
The only major cons to rust imo are the slow compile times, the steep learning curve and the immature ecosystem. You can solve the first hurdle with better hardware (for the same price as our usual specced macbook pro, I opted to custom bui
158.
by jhgg 6y ago
We will continue to use beam/otp where it makes sense. In fact some rust runs in BEAM now thanks to Rustler. We wrote a blog about this! https://blog.discord.com/using-rust-to-scale-elixir-for-11-m...
159.
by jhgg 6y ago
Elixir was not us chasing a trend - but rather betting on a language and runtime system we knew would be amazing if it worked. Discord would not exist in the form it does today if it was not for BEAM/OTP. The majority of the chat syste
160.
by jhgg 6y ago
Not a chance. We are going hard on rust in 2021 after some very successful projects in 2019 and 2020. our engineers have ramped up on the language - and we have good support internally (both in terms of tools, but also knowledge) to see its
161.
by jhgg 6y ago
We are hybrid. We use bare metal for our RTC workload (voice video streaming) and GCP for the rest.
162.
by jhgg 6y ago
We were experimenting with Go as a "fast" backend language to implement stuff like data services and high throughput systems in. Go has proven itself disappointing in many ways. Rust however has only continued to evolve and get be
163.
by jhgg 6y ago
A bit more literature on the N+1 situation in SQLite: https://www.sqlite.org/np1queryprob.html
164.
by jhgg 6y ago
An interpreter with a JIT is obviously faster than one without. Especially when dealing with CPU bound work. I'm not sure this is entirely noteworthy unless you somehow think CPython has a JIT. Would be much more interesting to compare
165.
by jhgg 6y ago
Oh real interesting! Didn't realize that was the case - and the source linked was just a new account O:
166.
by jhgg 6y ago
I don't think this is true, sorry. TS5 is definitely not built on Matrix. EDIT: Am wrong, see below! Maybe we can update the original post to one of the links below.
167.
by jhgg 6y ago
Our secret sauce is Elixir/BEAM and Rust :) Well for the real time side, I can't tell you how big a boon it's been to build our platform on top of Elixir/BEAM. Hands down the best runtime / VM for the job - and a bi
168.
by jhgg 6y ago
No I didn't but after looking at it it looks like its implementation falls short. - it only looks at system time at system startup, then relies on instant for monotonic clock source. unfortunately for setups that do leap-second smearin
169.
by jhgg 6y ago
The one that says "wip; use at your own risk" with no updates in the last 5 years? Or is there another? https://crates.io/crates/flake
170.
by jhgg 6y ago
Discord uses snowflake based IDs to great success. It's actually been one of the rock solid parts of our infrastructure and the fact that time stamps are embedded in the ID are great. Every channel, message, stream, attachment, user, s
171.
by jhgg 6y ago
Congratulations on 1.0! Tokio and the ecosystem surrounding it (tracing, hyper, prost, tower) are incredibly well thought out and a pleasure to use for building fast performant services with solid latencies.
172.
by jhgg 6y ago
To be fair, there is a cloud watch metric you could have set an alert on: "PercentIOLimit"
173.
by jhgg 6y ago
They do that as well - see "provisioned throughput" See the section "Specifying Throughput with Provisioned Mode" here: https://docs.aws.amazon.com/efs/latest/ug/performance.html
174.
by jhgg 6y ago
Prior art (rather similar, but instead of being json, it's a JPEG): https://news.ycombinator.com/item?id=12262470 tldr: the html is stuffed in the exif data!
175.
by jhgg 6y ago
It is a pretty high QPS. I work on a chat service that is quite popular (even more popular than Slack) and looked at our QPS to our persistent data stores. It is less than an order of magnitude compared to slack's when you normalize fo
176.
by jhgg 6y ago
We are not banning people for using Ripcord. We are banning people for using our API in ways that our official client doesn't. This is paramount to our spam detection strategy, which is detecting quirks in requests that try and emulate
177.
by jhgg 6y ago
Huge apologies on this one. Most of your tickets were still in the queue - and no one has gotten to them yet. Some other tickets were closed in error because it looked like the issue was resolved - as your original ticket had mentioned ther
178.
by jhgg 6y ago
No, this was not the case. A 3rd party client was very much involved here. > So i could imagine a user wants to do something that Discord makes it difficult to do (perhaps they want to search channel logs locally, or delete all of their
179.
by jhgg 6y ago
Hi, I lead the anti abuse team at Discord. I've looked into this case a bit more in-depth to see what was going on. Although I can't get into specifics, given the poster was kind enough to provide us with ticket IDs, I was able to
180.
by jhgg 6y ago
We needed to support arbitrary index access into the sorted set - which is why stdlib btree map doesn't work.

More ›