5 ms·
Neural-redis – Neural networks module for Redis
- tf2manu994 10y agoI started reading this thinking "Why not just hook into some python scripts that run tensorflow", then I got to this and IMO this makes it: >The main aim of Neural Redis, which is currently just a 48h personal hackatlon This was made in 2 days! Really intrigued to see what happens with this.
- stuartaxelowen 10y ago48 hours is 6 days in my book.
- antirez 10y agoSorry in the README I meant 48 hours of total time, and about 24-28 hours of total work. However now it's already more, but looking at the commit history says it all. Also note that nn.c was just modified, it's code I wrote before (years ago). The code I wrote in 2 days is the neuralredis.c file for a total of 1200 lines of code (including comments) plus most of the README.
- amix 10y agoMost dev teams probably run Redis in production and having neural networks as a datatype could be incredibly powerful -- it will make neural networks much more accessible. While this was a 48h personal hackathon (insane!) by Salvatore, I hope it makes it into Redis core. I will be spending my weekend on playing with this and trying it out with different internal datasets. Also, the documentation is a treat. It teaches the basics of NNs very well.
- wcummings 10y ago>Most dev teams probably run Redis in production Nope.
- fitzwatermellow 10y agoAgreed. For many prediction / classification tasks you don't need much feature engineering. Or much data for that matter ;)
- antirez 10y agoYep... nor there is locality in the inputs to exploit like it happens in CNNs.
- amix 10y agoI have made a Python interface to access neural-redis, ref: https://github.com/amix/neuralist https://github.com/amix/neuralist Should make it much easier to experiment with this stuff if you are using Python.
- Keyframe 10y agoI'm only on basic understanding of this, so this will be a cool learning tool! Can you, or more precisely did anyone try to create a decoder like this? For example, taking an input stream of mpeg4 or prores or whatever and giving that and the decoded output to the network and, after awhile, see what happens?
- amix 10y agoIn the current iteration, this only implements feed forward neural networks -- which are basic and can only resolve basic correlations. For more advanced stuff (like image or video processing) you have to move to recurrent neural networks.
- antirez 10y agoOr CNNs... However I see RNNs more in the future of Neural Redis than CNNs that are more domain-specific, while RNNs are an orthogonal tool for modeling algorithms and not functions.
- Keyframe 10y agoI didn't mean video specifically though. I mean decoding in general. Input and output fed in, and try to replicate blackbox encoder/decoder and see what happens. I'm sure it wouldn't work on (advanced) encryption because of some mathy reasons, but.. I don't know, xoring and RLE could work I guess? Only one way to find out! If it could work on more complicated codec systems, it would be an interesting approach regarding IP.
- toisanji 10y agowow, seems neural networks are going into everything
- antirez 10y agoYep, true... however note that the current Redis incarnation of NNs don't use RNNs nor convolutional NNs that are the big news in the area, it's mostly stuff that was accessible 10 years ago or more. Just computers are now a lot faster even without GPUs, so that small feed forward NNs can be trained in little time and show that they can be pretty useful, especially in a context where the data ingestion and training are encapsulated into an easy to use API. Nothing prevents us from incorporating more advanced NNs into Redis as a foundation for the API provided by Neural Redis, however with certain problems users face, they may not make much difference, but we'll try to explore new solutions.
- stuartaxelowen 10y agoI love this idea, but I'm also worried this is papering over a lot of the truly hard parts of building products on machine learning - the hard part isn't crunching the numbers.
- antirez 10y agoI totally agree, and indeed the help that Neural Redis attempts to provide is mainly in the data collection function and the simplicity to experiment. Actually for the number crunching part there are a lot of stellar alternatives (that we can just "connect" to Neural Redis as NN implementation later). The problem is to find a good API for the programmer that allows to easily make experiments.
- cheez 10y agoI've been doing this in my installable app for a bit now. Online training while effectively running blind has been a bit difficult... Machine learning is definitely going to make devs more productive in this kind of way.