5 ms·

Tracking.js － A modern approach for computer vision on the web

143 points by zenorocha 12y ago

joeyspn 12y ago
This is really cool and is going to help me A LOT with a detection script I was working on for a WebRTC video website. I was looking for something like this a couple of weeks ago and I decided to use python with Fisherfaces [0]. Now I'm really curious about Viola-Jones and its accuracy via JS... [0] https://github.com/bytefish/facerec https://github.com/bytefish/facerec
jbhatab 12y ago
I am very interested in computer vision libraries. How exactly does this stack up to OpenCV or is that even a fair comparison?
- pablocp 12y ago
  tracking.js is a new library that intends to bring the power of computer vision to the Web. OpenCV is far older and is a inspiration for our team regarding the number of available algorithms, robustness and speed. With the increasing power of JavaScript engines, we believe that we can achieve very good results using current state-of-the-art algorithms and low-level optimizations, so real-time CV applications can also be done on the Web. Currently tracking.js don't have as many algorithms as OpenCV, but we are making our best efforts to bring useful algorithms to the library while still keeping the API easy to use and to extend, so we can count on the community to help us to make this project a success.
  - jbhatab 12y ago
    Great. This is actually very relevant to a project I'm working on. It involves porting projecter/camera electronics from open cv over to the web with javascript. So how does this stack up vs jsfeat? Are those comparable at all? I do think your api is more web dev friendly from what it looks like, but I am curious about the capabilities.
    - pablocp 12y ago
      One of the biggest problems we find on most computer vision libraries is the fact that a lot of knowledge in the area is required to use them properly. We think the complex APIs that are exposed to developers that try to make CV apps can be the reason why we still don't see many apps exploring the potential of CV. So tracking.js was created with the goal of being as powerful as the best CV libraries, but without exposing the inherent complexity of the area to the developers that wish to use it. From what I saw of jsfeat, it looks very powerful, but targeted for people that have some knowledge of CV.
splintercell 12y ago
I can't get it to work(it detects my white shirt, the shoulder area as a face, but not my actual face). Is this a repetition of this: https://www.youtube.com/watch?v=t4DT3tQqgRM https://www.youtube.com/watch?v=t4DT3tQqgRM
- gravity13 12y ago
  I also cannot get it to work. The little bit of instructions there are make no sense so I don't know if I'm doing something wrong or if it's just broken on my computer.
  - jusuchin 12y ago
    Doesn't work for me either. It does nothing.
- pablocp 12y ago
  Face detection still being an open problem in computer vision area. While very good algorithms exist, they still fail on some cases or are not fast enough to run in real time. Our implementation is based on Viola-Jones algorithm, one of the most used algorithms for this problem (and also used by OpenCV). This algorithm can be used to track several types of objects just by changing the training data that is used as input. For the face detector, we used training data generated by OpenCV, but we don't know which faces were used to generate it. Perhaps this issue is a matter of changing the training data, or maybe this is a case where the algorithm is not robust enough to handle. We hope we can fix this problem soon, but we still don't know how (suggestions and contributions are appreciated ^.^)
  - drpgq 12y ago
    "While very good algorithms exist ... or are not fast enough to run in real time." As a research scientist at a company whose main product is face recognition software I have to disagree with this part of the statement.
    - 12y ago
moron4hire 12y ago
It's funny, I look almost exactly like the guy on the left in the "Face (Image)" example, and I have a very good quality webcam, yet it does a really terrible job of detecting my own, live face.
notastartup 12y ago
This looks pretty awesome. I know it's a bit offtopic but on this page http://trackingjs.com/examples/face_hello_world.html http://trackingjs.com/examples/face_hello_world.html where can you find such a nice 'chrome browser' graphic? it looks really solid. I want to use it on my website.
- joeyspn 12y ago
  Just google "flat browser frame" and pick the one you like...
hwswfun 12y ago
Any examples of all-javascript augmented reality using this yet? It seems like this could be done since feature detection is implemented.
- pablocp 12y ago
  We are currently working on a pose-estimation algorithm (EPnP) that will use the results we get from feature detection to find a camera pose, so we can make 3D augmentation. It will be very helpful to create augmented reality applications.

143 points by zenorocha 12y ago

This is really cool and is going to help me A LOT with a detection script I was working on for a WebRTC video website. I was looking for something like this a couple of weeks ago and I decided to use python with Fisherfaces [0]. Now I'm really curious about Viola-Jones and its accuracy via JS... [0] https://github.com/bytefish/facerec https://github.com/bytefish/facerec

jbhatab 12y ago

I am very interested in computer vision libraries. How exactly does this stack up to OpenCV or is that even a fair comparison?

pablocp 12y ago
tracking.js is a new library that intends to bring the power of computer vision to the Web. OpenCV is far older and is a inspiration for our team regarding the number of available algorithms, robustness and speed. With the increasing power of JavaScript engines, we believe that we can achieve very good results using current state-of-the-art algorithms and low-level optimizations, so real-time CV applications can also be done on the Web. Currently tracking.js don't have as many algorithms as OpenCV, but we are making our best efforts to bring useful algorithms to the library while still keeping the API easy to use and to extend, so we can count on the community to help us to make this project a success.
- jbhatab 12y ago
  Great. This is actually very relevant to a project I'm working on. It involves porting projecter/camera electronics from open cv over to the web with javascript. So how does this stack up vs jsfeat? Are those comparable at all? I do think your api is more web dev friendly from what it looks like, but I am curious about the capabilities.
  - pablocp 12y ago
    One of the biggest problems we find on most computer vision libraries is the fact that a lot of knowledge in the area is required to use them properly. We think the complex APIs that are exposed to developers that try to make CV apps can be the reason why we still don't see many apps exploring the potential of CV. So tracking.js was created with the goal of being as powerful as the best CV libraries, but without exposing the inherent complexity of the area to the developers that wish to use it. From what I saw of jsfeat, it looks very powerful, but targeted for people that have some knowledge of CV.

splintercell 12y ago

I can't get it to work(it detects my white shirt, the shoulder area as a face, but not my actual face). Is this a repetition of this: https://www.youtube.com/watch?v=t4DT3tQqgRM https://www.youtube.com/watch?v=t4DT3tQqgRM

gravity13 12y ago
I also cannot get it to work. The little bit of instructions there are make no sense so I don't know if I'm doing something wrong or if it's just broken on my computer.
- jusuchin 12y ago
  Doesn't work for me either. It does nothing.
pablocp 12y ago
Face detection still being an open problem in computer vision area. While very good algorithms exist, they still fail on some cases or are not fast enough to run in real time. Our implementation is based on Viola-Jones algorithm, one of the most used algorithms for this problem (and also used by OpenCV). This algorithm can be used to track several types of objects just by changing the training data that is used as input. For the face detector, we used training data generated by OpenCV, but we don't know which faces were used to generate it. Perhaps this issue is a matter of changing the training data, or maybe this is a case where the algorithm is not robust enough to handle. We hope we can fix this problem soon, but we still don't know how (suggestions and contributions are appreciated ^.^)
- drpgq 12y ago
  "While very good algorithms exist ... or are not fast enough to run in real time." As a research scientist at a company whose main product is face recognition software I have to disagree with this part of the statement.
  - 12y ago

moron4hire 12y ago

It's funny, I look almost exactly like the guy on the left in the "Face (Image)" example, and I have a very good quality webcam, yet it does a really terrible job of detecting my own, live face.

notastartup 12y ago

This looks pretty awesome. I know it's a bit offtopic but on this page http://trackingjs.com/examples/face_hello_world.html http://trackingjs.com/examples/face_hello_world.html where can you find such a nice 'chrome browser' graphic? it looks really solid. I want to use it on my website.

joeyspn 12y ago
Just google "flat browser frame" and pick the one you like...

hwswfun 12y ago

Any examples of all-javascript augmented reality using this yet? It seems like this could be done since feature detection is implemented.

pablocp 12y ago
We are currently working on a pose-estimation algorithm (EPnP) that will use the results we get from feature detection to find a camera pose, so we can make 3D augmentation. It will be very helpful to create augmented reality applications.