7 ms·

Square open-sources Tesseract: fast filtering for coordinated views

263 points by mbostock 15y ago

nodata 15y ago
Please don't call it tesseract, we already have a prominent open source project with that name: http://code.google.com/p/tesseract-ocr/ http://code.google.com/p/tesseract-ocr/
- dylanz 15y ago
  +1 ... The title totally confused me. I use Tesseract already, and it's a popular OCR project.
- knowtheory 15y ago
  I was just about to come here and lament this very fact. Because I work on DocumentCloud I very much do touch on both OCRing software and frontend JS libraries. It will only make things confusing to have these two projects fighting over namespace. And the Tesseract OCR engine is not going away. It's the defacto standard for FOSS OCRing. Edit: Having played around with the examples for this lib, it's awesome! I will have to find a project to work this into. Looks like fun.
  - Terretta 15y ago
    Concur -- fantastic tool, unfortunate name, due to the overlap. Cubic Prism is another name for tesseract and could also remind one of both SQL data rollup and looking at data in new lights.
- mbostock 15y ago
  Apologies; I wasn't aware of the Tesseract OCR project until very recently, and then I hoped there would not be much harm given that the two projects are so unrelated. (The name "Tesseract" was the natural progression from "Square" and "Cube".) What's the saying? There are only two hard things in Computer Science: cache invalidation and naming things.
  - jashkenas 15y ago
    If you're settled on "Tesseract", perhaps a small tweak would suffice to clear up the potential confusion. Call it "Tesseract.js".
    - fourneau 15y ago
      That would lead me to believe that it's 'Tesseract OCR, but in Javascript!'. I think a more substantial name change will be required because of how prominent Tesseract is.
      - ktizo 15y ago
        Tesseroct maybe.. Tesserect? ;p Mind you Tesseract OCR did miss a chance to be TesseractOCRus, which would be great, obviously.
  - sparky 15y ago
    Hypercube?
    - mattdeboard 15y ago
      n-dimensional hypercube
      - tesseract 15y ago
        hypercubes are already n-dimensional, that's not adding any extra information.
    - repsilat 15y ago
      Call it "4-square". Nobody will get confused then.
  - spaznode 15y ago
    http://en.wikipedia.org/wiki/Tesseract http://en.wikipedia.org/wiki/Tesseract "In geometry, the tesseract, also called an 8-cell or regular octachoron or cubic prism, is the four-dimensional analog of the cube. The tesseract is to the cube as the cube is to the square. Just as the surface of the cube consists of 6 square faces, the hypersurface of the tesseract consists of 8 cubical cells." Looks like the perfect name to me. Sounds like the "other" project already has an established name of "tesseract ocr", don't see any reason why this library would be confused with that. Lame for people to focus on this instead of the crazy beautiful api that comes with this thing: https://github.com/square/tesseract/wiki/API-Reference https://github.com/square/tesseract/wiki/API-Reference
- law 15y ago
  Calling it a "prominent open source project" is an understatement. Granted it's not perfect, it's the de facto standard in FOSS OCR software. When mentioning "tesseract" to just about anyone in the data mining/machine learning/artificial intelligence communities (which is pretty much the target user base), they will automatically think you're referring to the OCR software.
- zdw 15y ago
  Bug filed on their github issues page: https://github.com/square/tesseract/issues/1 https://github.com/square/tesseract/issues/1
- samstave 15y ago
  Would you prefer Ono-Sendai, or Ice-9?
- deleted 15y ago
  [deleted]
- tbe 15y ago
  I'm more confused about the name of the company. To me, Square was the maker of Final Fantasy and other role playing games in the golden years of 8- and 16-bit video game consoles. http://en.wikipedia.org/wiki/Square_(company) http://en.wikipedia.org/wiki/Square_(company)
drewda 15y ago
@mbostock: Why is there Dart code in the project? Just because that's where you borrowed one of the sort functions from? Thanks for another useful library!
- mbostock 15y ago
  Yep. Typed arrays don't have a built-in sort method, and even the built-in array.sort is extremely slow. I ported Dart's dual-pivot quicksort implementation, which reduced the time to sort 1M floats from ~2.5s to ~350ms (timed in Node v0.6.2). https://github.com/square/tesseract/blob/master/src/quicksort.js https://github.com/square/tesseract/blob/master/src/quicksor...
  - drewda 15y ago
    Good to know. I'll have to consider switching away from Underscore's sort, which is probably equally slow. By the way, quicksort.js is code you wrote, right? I assume that's too nice and concise to be from the Dart generator...
    - jashkenas 15y ago
      Underscore doesn't have a "sort" apart from the native Array.prototype.sort, but it does have a "sortBy", which is something else entirely.
      - drewda 15y ago
        The sortBy method is just producing an integer array that is then sorted using each browser's implementation of Array.prototype.sort. Is that a proper reading of the code?
        jashkenas 15y ago
        Bingo. You got it.
    - mbostock 15y ago
mnutt 15y ago
This is really great work, and I can't wait to see some of it filter into Square's Cube project.
troels 15y ago
That's timely. Recently I've been looking around for various widgets interfaces to explore multidimensional data. This looks quite useful - way up next to good old pivot table. What do you think of parallel coordinates as a widget? (F.ex. http://exposedata.com/parallel/veggie/ http://exposedata.com/parallel/veggie/)
huggyface 15y ago
Thank you for an release. I've been heavily promoting the web stack for over a decade and a half, yet still I'm surprised by what it is capable of. This also provides a clear demonstration of the power of algorithms even on an imperfect implementation. Excellent, clean API as well.
mukaiji 15y ago
I got a preview of this project from Square's CTO 3 weeks ago at Stanford. The fast-filtering is MIND-BLOWING.
swah 15y ago
So mbostock works for Square?
Tyr42 15y ago
It'd be cool if it could say, show the data for all weekends at once, skipping over the weekdays.
nodata 15y ago
Update: Square have changed the name to Crossfilter: "Renamed to Crossfilter, partly in homage to Chris Weaver's work on multidimensional visualization. It may not have the intrigue of "tesseract", but it does describe the library's function succinctly." -- https://github.com/square/crossfilter/issues/1#issuecomment-4706390 https://github.com/square/crossfilter/issues/1#issuecomment-...

263 points by mbostock 15y ago

Please don't call it tesseract, we already have a prominent open source project with that name: http://code.google.com/p/tesseract-ocr/ http://code.google.com/p/tesseract-ocr/

dylanz 15y ago
+1 ... The title totally confused me. I use Tesseract already, and it's a popular OCR project.
knowtheory 15y ago
I was just about to come here and lament this very fact. Because I work on DocumentCloud I very much do touch on both OCRing software and frontend JS libraries. It will only make things confusing to have these two projects fighting over namespace. And the Tesseract OCR engine is not going away. It's the defacto standard for FOSS OCRing. Edit: Having played around with the examples for this lib, it's awesome! I will have to find a project to work this into. Looks like fun.
- Terretta 15y ago
  Concur -- fantastic tool, unfortunate name, due to the overlap. Cubic Prism is another name for tesseract and could also remind one of both SQL data rollup and looking at data in new lights.
mbostock 15y ago
Apologies; I wasn't aware of the Tesseract OCR project until very recently, and then I hoped there would not be much harm given that the two projects are so unrelated. (The name "Tesseract" was the natural progression from "Square" and "Cube".) What's the saying? There are only two hard things in Computer Science: cache invalidation and naming things.
- jashkenas 15y ago
  If you're settled on "Tesseract", perhaps a small tweak would suffice to clear up the potential confusion. Call it "Tesseract.js".
  - fourneau 15y ago
    That would lead me to believe that it's 'Tesseract OCR, but in Javascript!'. I think a more substantial name change will be required because of how prominent Tesseract is.
    - ktizo 15y ago
      Tesseroct maybe.. Tesserect? ;p Mind you Tesseract OCR did miss a chance to be TesseractOCRus, which would be great, obviously.
- sparky 15y ago
  Hypercube?
  - mattdeboard 15y ago
    n-dimensional hypercube
    - tesseract 15y ago
      hypercubes are already n-dimensional, that's not adding any extra information.
  - repsilat 15y ago
    Call it "4-square". Nobody will get confused then.
- spaznode 15y ago
  http://en.wikipedia.org/wiki/Tesseract http://en.wikipedia.org/wiki/Tesseract "In geometry, the tesseract, also called an 8-cell or regular octachoron or cubic prism, is the four-dimensional analog of the cube. The tesseract is to the cube as the cube is to the square. Just as the surface of the cube consists of 6 square faces, the hypersurface of the tesseract consists of 8 cubical cells." Looks like the perfect name to me. Sounds like the "other" project already has an established name of "tesseract ocr", don't see any reason why this library would be confused with that. Lame for people to focus on this instead of the crazy beautiful api that comes with this thing: https://github.com/square/tesseract/wiki/API-Reference https://github.com/square/tesseract/wiki/API-Reference
law 15y ago
Calling it a "prominent open source project" is an understatement. Granted it's not perfect, it's the de facto standard in FOSS OCR software. When mentioning "tesseract" to just about anyone in the data mining/machine learning/artificial intelligence communities (which is pretty much the target user base), they will automatically think you're referring to the OCR software.
zdw 15y ago
Bug filed on their github issues page: https://github.com/square/tesseract/issues/1 https://github.com/square/tesseract/issues/1
samstave 15y ago
Would you prefer Ono-Sendai, or Ice-9?
deleted 15y ago
[deleted]
tbe 15y ago
I'm more confused about the name of the company. To me, Square was the maker of Final Fantasy and other role playing games in the golden years of 8- and 16-bit video game consoles. http://en.wikipedia.org/wiki/Square_(company) http://en.wikipedia.org/wiki/Square_(company)

drewda 15y ago

@mbostock: Why is there Dart code in the project? Just because that's where you borrowed one of the sort functions from? Thanks for another useful library!

mbostock 15y ago
Yep. Typed arrays don't have a built-in sort method, and even the built-in array.sort is extremely slow. I ported Dart's dual-pivot quicksort implementation, which reduced the time to sort 1M floats from ~2.5s to ~350ms (timed in Node v0.6.2). https://github.com/square/tesseract/blob/master/src/quicksort.js https://github.com/square/tesseract/blob/master/src/quicksor...
- drewda 15y ago
  Good to know. I'll have to consider switching away from Underscore's sort, which is probably equally slow. By the way, quicksort.js is code you wrote, right? I assume that's too nice and concise to be from the Dart generator...
  - jashkenas 15y ago
    Underscore doesn't have a "sort" apart from the native Array.prototype.sort, but it does have a "sortBy", which is something else entirely.
    - drewda 15y ago
      The sortBy method is just producing an integer array that is then sorted using each browser's implementation of Array.prototype.sort. Is that a proper reading of the code?
      - jashkenas 15y ago
        Bingo. You got it.
  - mbostock 15y ago

mnutt 15y ago

This is really great work, and I can't wait to see some of it filter into Square's Cube project.

troels 15y ago

That's timely. Recently I've been looking around for various widgets interfaces to explore multidimensional data. This looks quite useful - way up next to good old pivot table. What do you think of parallel coordinates as a widget? (F.ex. http://exposedata.com/parallel/veggie/ http://exposedata.com/parallel/veggie/)

huggyface 15y ago

Thank you for an release. I've been heavily promoting the web stack for over a decade and a half, yet still I'm surprised by what it is capable of. This also provides a clear demonstration of the power of algorithms even on an imperfect implementation. Excellent, clean API as well.

mukaiji 15y ago

I got a preview of this project from Square's CTO 3 weeks ago at Stanford. The fast-filtering is MIND-BLOWING.

swah 15y ago

So mbostock works for Square?

Tyr42 15y ago

It'd be cool if it could say, show the data for all weekends at once, skipping over the weekdays.

nodata 15y ago

Update: Square have changed the name to Crossfilter: "Renamed to Crossfilter, partly in homage to Chris Weaver's work on multidimensional visualization. It may not have the intrigue of "tesseract", but it does describe the library's function succinctly." -- https://github.com/square/crossfilter/issues/1#issuecomment-4706390 https://github.com/square/crossfilter/issues/1#issuecomment-...