4 ms·

This really is just “new Map()” with a little bit of set dressing.

by jitl 4y ago

nbittich 4y ago
Exactly. Just an extra layer of abstraction, no idea what it tries to solve.
- IMTDb 4y ago
  Maybe... the need to have a map with some additional set of accessors and/or utility functions around it ? If you look at the code; it's literally just a map with some function around it. If you end up in a situation where you feel the need to wrap a map with some function to handle some typical needs (eg: access a random key; manage a ttl/expire for a key; ...) you might as well be using this. The fact that the name of the functions have been taken from redis means that if/when you move to an external KV store, the transition will be trivial. It also means you don't really need to learn it if you already know redis. I don't see becoming the #1 library on npm; with conferences and speakers debating the intricacies of its architecture. But hey it might solve someones need.
enjoylife 4y ago
Yep. This is a quintessential example of “I’m going to learn by recreating the essence of a complex thing”… taken to the end. And then a logo is slapped on top. I’m all for js libs but this is not performant. Just use redis.
- hem777 4y ago
  Can you run Redis in browser? You can’t. Is an in-process, in-memory database faster than making a query over network? Yes it is.
  - nbittich 4y ago
    no need an extra layer of abstraction to get an "in memory" database in javascript. no need, babel, webpack, ChatGpt, npm and 10000 libraries for this really. Let me show (works even on the browser): const KV = {}; KV["credentials"] = {access_key: "XXXX", secret: "YYYY"}; console.log(KV["credentials"].secret);
    - hem777 4y ago
      Maybe? Depends on the rest of the software you’d be developing and what features and abstractions you’d need. The point was that “just use redis” is a poor and incorrect advice.
    - hombre_fatal 4y ago
      Now show me the rest of the functionality of the OP library like TTL.
      - nbittich 4y ago
        Someone already replied to this in another comment.
  - thegeomaster 4y ago
    Run Redis on the same machine. No network latency. You only have a few memcpys and context switches versus an in-process solution, and if those make a difference you shouldn't be using JS anyways
  - gregors 4y ago
    I'm not saying anyone should actually use this btw https://narma.github.io/2015/redis-in-browser/ https://narma.github.io/2015/redis-in-browser/ https://medium.com/fluence-network/porting-redis-to-webassembly-with-clang-wasi-af99b264ca8 https://medium.com/fluence-network/porting-redis-to-webassem...
Etheryte 4y ago
I don't really agree, the logic around expiring values by itself is already a beneficial feature that isn't available out of the box.
- jitl 4y ago
  Yeah that’s true. But because it’s using the system hashmap it inherits those performance characteristics and maximum size. Map is hard limited to 2^24 = 16,777,216 keys in v8. Map can also have performance spikes that cause event loop delay when you start loading it heavily. If you expect higher load, using a two levels of map (eg Map<k1, Map<k2, V>>) can help with both issues. I went to look how the library accounted for these issues and it does not. I didn’t look to see if it uses timer coalescing or intervals, I wouldn’t want O(millions) of setTimout calls on my service either. I think it’s fine to use in the browser or a hobby server, but due to the limitations of Map I would not use it in my production server. Redis can do 2^32 = 4,294,967,296 keys (a few more), and you can do the “multi level map” by sharding your key space across multiple Redis or Memcached processes. Redis et al also has a large advantage of the cache surviving application deploy. Again, not everyone needs this but to me the main benefit of a remote cache is consistently lowering latencies, versus a in-memory cache where your latency will spike after every deploy as the cache refills.
- chrismorgan 4y ago
  The expiring part is very simple. For just get/set/delete and scheduled garbage collection, here’s an implementation that’s almost precisely equivalent to what KV.js offers (just with sane arguments and return types, fewer bugs, and markedly better performance due to not supporting stuff you certainly don’t need and removing unnecessary abstractions): const store = new Map(); const expiryTimes = new Map(); function get(key) { // (This works even for non-expiring keys because undefined is not less than any number.) if (expiryTimes.get(key) < Date.now()) { store.delete(key); } return store.get(key); } function set(key, value, expiry) { store.set(key, value); if (expiry) { expiryTimes.set(expiry); } } function del(key) { store.delete(key); expiryTimes.delete(key); } function flushExpired() { const now = Date.now(); for (const [key, time] of expiryTimes) { if (time < now) { del(key); } } } // Optional: automatically delete expired entries to free memory, once a second since that’s what KV.js does, but that’s probably waaaaay more often than you need. setInterval(flushExpired, 1000); Working directly with store and expiryTimes for more advanced stuff will genuinely regularly be easier than working with the KV.js abstraction.

This really is just “new Map()” with a little bit of set dressing.

by jitl 4y ago

This really is just “new Map()” with a little bit of set dressing.

nbittich 4y ago

Exactly. Just an extra layer of abstraction, no idea what it tries to solve.

IMTDb 4y ago
Maybe... the need to have a map with some additional set of accessors and/or utility functions around it ? If you look at the code; it's literally just a map with some function around it. If you end up in a situation where you feel the need to wrap a map with some function to handle some typical needs (eg: access a random key; manage a ttl/expire for a key; ...) you might as well be using this. The fact that the name of the functions have been taken from redis means that if/when you move to an external KV store, the transition will be trivial. It also means you don't really need to learn it if you already know redis. I don't see becoming the #1 library on npm; with conferences and speakers debating the intricacies of its architecture. But hey it might solve someones need.

enjoylife 4y ago

Yep. This is a quintessential example of “I’m going to learn by recreating the essence of a complex thing”… taken to the end. And then a logo is slapped on top. I’m all for js libs but this is not performant. Just use redis.

hem777 4y ago
Can you run Redis in browser? You can’t. Is an in-process, in-memory database faster than making a query over network? Yes it is.
- nbittich 4y ago
  no need an extra layer of abstraction to get an "in memory" database in javascript. no need, babel, webpack, ChatGpt, npm and 10000 libraries for this really. Let me show (works even on the browser): const KV = {}; KV["credentials"] = {access_key: "XXXX", secret: "YYYY"}; console.log(KV["credentials"].secret);
  - hem777 4y ago
    Maybe? Depends on the rest of the software you’d be developing and what features and abstractions you’d need. The point was that “just use redis” is a poor and incorrect advice.
  - hombre_fatal 4y ago
    Now show me the rest of the functionality of the OP library like TTL.
    - nbittich 4y ago
      Someone already replied to this in another comment.
- thegeomaster 4y ago
  Run Redis on the same machine. No network latency. You only have a few memcpys and context switches versus an in-process solution, and if those make a difference you shouldn't be using JS anyways
- gregors 4y ago
  I'm not saying anyone should actually use this btw https://narma.github.io/2015/redis-in-browser/ https://narma.github.io/2015/redis-in-browser/ https://medium.com/fluence-network/porting-redis-to-webassembly-with-clang-wasi-af99b264ca8 https://medium.com/fluence-network/porting-redis-to-webassem...

Etheryte 4y ago

I don't really agree, the logic around expiring values by itself is already a beneficial feature that isn't available out of the box.

jitl 4y ago
Yeah that’s true. But because it’s using the system hashmap it inherits those performance characteristics and maximum size. Map is hard limited to 2^24 = 16,777,216 keys in v8. Map can also have performance spikes that cause event loop delay when you start loading it heavily. If you expect higher load, using a two levels of map (eg Map<k1, Map<k2, V>>) can help with both issues. I went to look how the library accounted for these issues and it does not. I didn’t look to see if it uses timer coalescing or intervals, I wouldn’t want O(millions) of setTimout calls on my service either. I think it’s fine to use in the browser or a hobby server, but due to the limitations of Map I would not use it in my production server. Redis can do 2^32 = 4,294,967,296 keys (a few more), and you can do the “multi level map” by sharding your key space across multiple Redis or Memcached processes. Redis et al also has a large advantage of the cache surviving application deploy. Again, not everyone needs this but to me the main benefit of a remote cache is consistently lowering latencies, versus a in-memory cache where your latency will spike after every deploy as the cache refills.
chrismorgan 4y ago
The expiring part is very simple. For just get/set/delete and scheduled garbage collection, here’s an implementation that’s almost precisely equivalent to what KV.js offers (just with sane arguments and return types, fewer bugs, and markedly better performance due to not supporting stuff you certainly don’t need and removing unnecessary abstractions): const store = new Map(); const expiryTimes = new Map(); function get(key) { // (This works even for non-expiring keys because undefined is not less than any number.) if (expiryTimes.get(key) < Date.now()) { store.delete(key); } return store.get(key); } function set(key, value, expiry) { store.set(key, value); if (expiry) { expiryTimes.set(expiry); } } function del(key) { store.delete(key); expiryTimes.delete(key); } function flushExpired() { const now = Date.now(); for (const [key, time] of expiryTimes) { if (time < now) { del(key); } } } // Optional: automatically delete expired entries to free memory, once a second since that’s what KV.js does, but that’s probably waaaaay more often than you need. setInterval(flushExpired, 1000); Working directly with store and expiryTimes for more advanced stuff will genuinely regularly be easier than working with the KV.js abstraction.