6 ms·

My personal experience with trying to recreate tech from research papers (real-time mobile computer vision in my case): - assuming highly specialized Math skil

My personal experience with trying to recreate tech from research papers (real-time mobile computer vision in my case): - assuming highly specialized Math skills (e.g. manifolds), which wasn't me at the time, so it made testing and bug hunting harder - code missing, or code is a long C file vomited out by Matlab, with some pieces switched to (Desktop) SSE instructions for speed gains - papers were missing vital parameters to reproduce the experiment (e.g. exact setting for a variable that influences an optimization loop precision vs speed) - the experiment was very constrained and the whole algorithm would have never worked in real life, which is what I had supposed for the first few months (meh) - most papers, as the article says, are just a little bump over the previous ones, so now you have to read and implement a tree of papers - sometimes there would be the need of a dataset to train a model, but the dataset is closed source and incredibly expensive, so not a good avenue At the time I was also working with the first version of (Apple) Metal which, after going crazy on why my algo wasn't working, I discovered had a precision bug with the division operator. FML Still it was a very instructing experience, the biggest takeaway if you do something similar: don't be certain that once you implemented an algorithm, it will work as advertised. It's totally different from say, writing an API, it's not a well constrained problem.

mirsadm 6y ago
> - the experiment was very constrained and the whole algorithm would have never worked in real life, which is what I had supposed for the first few months (meh) This has been my experience with a large number of papers that make all sorts of wild claims. Whilst technically true they are more often than not are not actually very useful for any practical application.
de_watcher 6y ago
> working with the first version of (Apple) Metal How do they manage to push this non-portable waste of time on people?
- aledalgrande 6y ago
  Actually was pretty nice once used to it, especially for GPGPU, which didn't need a crazy mindwrap like with the old OpenGL ES. Obviously they can push it because of the platform. But developing with it is not very different from CUDA/OpenCL.
snovv_crash 6y ago
In my experience of the same field, the only lab you can consistently trust the outputs from is Microsoft Research. I don't know if they have some kind of blind QA or something, but everything from them has worked as advertised.
- aledalgrande 6y ago
  Yeah, Microsoft Research, Disney Research and ETH are the most reliable. You learn very fast to read only papers from the biggest universities and companies.

mirsadm 6y ago

> - the experiment was very constrained and the whole algorithm would have never worked in real life, which is what I had supposed for the first few months (meh) This has been my experience with a large number of papers that make all sorts of wild claims. Whilst technically true they are more often than not are not actually very useful for any practical application.

de_watcher 6y ago

> working with the first version of (Apple) Metal How do they manage to push this non-portable waste of time on people?

aledalgrande 6y ago
Actually was pretty nice once used to it, especially for GPGPU, which didn't need a crazy mindwrap like with the old OpenGL ES. Obviously they can push it because of the platform. But developing with it is not very different from CUDA/OpenCL.

snovv_crash 6y ago

In my experience of the same field, the only lab you can consistently trust the outputs from is Microsoft Research. I don't know if they have some kind of blind QA or something, but everything from them has worked as advertised.

aledalgrande 6y ago
Yeah, Microsoft Research, Disney Research and ETH are the most reliable. You learn very fast to read only papers from the biggest universities and companies.