3 ms·
We were looking at that (as well as riak) but processing the data would require pulling all the data into PHP. I guess you could do the processing in C but it's
by saulkw 15y ago
We were looking at that (as well as riak) but processing the data would require pulling all the data into PHP. I guess you could do the processing in C but it's then just as easy to store it there as well.
- Mikushi 15y agoHave you looked into the LUA scripting option for Redis? Allows for some processing to happen on the server side, and it's quite powerful.
- saulkw 15y agoThat sounds like a good option. Thanks, will note it.
- nknight 15y agoI'm not clear why you're worried about that. Is it the pulling, or the processing? The pulling shouldn't be an issue -- I don't know about PHP, but in pure Python, I can pull an arbitrary 10MB string from Redis in ~85-90ms. With hiredis (C extension), that falls to about 47ms. I can't speak to processing, since I don't know exactly what transformations you're performing.
- saulkw 15y agoIt's more the iteration of each pixel and it's neighbor (of which there are 8) making it around 9.6 million iterations. We will probably head towards redis in the future when precise backups are essential. Undecided what will do this processing though.
- nivertech 15y agoWe built GPU-accelerated NoSQL datastore. using it, this can be accelerated 100x, given you switch to binary pixel format.
- fsaintjacques 15y agoWhy would you use a GPU-accelerated storage when latency is the main goal?
- nivertech 15y agoGPU do not accelerate raw storage retrieval, but processing, like queries and map reduce. Use APU / HPU, if PCIe latency is a problem. I understood that they running something like convolution (I.e, each pixel calculated from surrounding pixels) - this will be fast using OpenCL model).