8 ms·

Table-to-JSON

64 points by talhof8 13y ago

skram 13y ago
Awesome. Something else that would be nice is to be able to go from JSON to HTML table. There are a few libraries out there that do this but none seem to be updated or recent. Any suggestions would be greatly appreciated. Nice work!
- speeq 13y ago
  If you're using node.js/Express, you could do this using Jade: http://stackoverflow.com/questions/5204905/dynamic-html-pages-with-jade-nodejs-express http://stackoverflow.com/questions/5204905/dynamic-html-page...
- jwoah12 13y ago
  It might be overkill if you don't need interactive features, but my ATable[1] library takes a 2-dimensional array representation of the data, and turns it into an HTML table. I will add support for an array of JSON objects as the data source soon. [1] - http://jarwol.com/aTable/ http://jarwol.com/aTable/
charlysisto 13y ago
Nice ! Could be used in a progressive enhancement strategy : render your html then extract the json out of it to feed your MVWhatever (Angularjs, Emberjs, Backbonejs, nameitjs) and let javascript take on for the cool client side stuff.
rcfox 13y ago
It's a nice start to something that looks pretty useful. There are some things to consider though: What if you have headings as the first column in each row? What if you have headings in the first column, and the first row? Also, what is the use case for data overrides? In my opinion, it completely defeats the purpose when your intention is to take the contents of the table.
- chalst 13y ago
  The code does not try to be clever: var getHeadings = function(table) { var firstRow = table.find("tr:first").first(); return notNull(opts.headings) ? opts.headings : rowValues(firstRow); }; The code is also going to fall down if you have things like wide/high cells. But HTML tables are so flexible and so routinely abused, I guess the problem is too hard to be solved robustly with any generality.
  - JangoSteve 13y ago
    You could do what they're suggesting without being too clever. For instance, our dynatable plugin does it this way: table.find('thead tr').children('th,td') But it could easily be something like: var headings = table.find('thead th'); if (!headings.length) headings = table.find('th'); if (!headings.length) headings = table.find('tr:first td');
    - terrahawkes 13y ago
      The HTML specification's algorithm for determining the applicable headers for a table cell might be useful: http://www.whatwg.org/specs/web-apps/current-work/multipage/tabular-data.html#header-and-data-cell-semantics http://www.whatwg.org/specs/web-apps/current-work/multipage/... This was developed with reference to a corpus of real web tables.
      - JangoSteve 13y ago
        I was actually really excited to read that, thinking I was going to implement it straight away in our plugin. But it seems to require scanning up the table from the current cell, for each cell, to get its header, and it also requires knowing the rendered height of the cell, which I don't fully understand. I'm sure if I actually sat down and wrote it out, it'd make more sense. At any rate, it seems like trying to implement their algorithm verbatim in javascript would be pretty slow for large tables. Maybe it's better to just use it for inspiration. Also, their algorithm seems to come at if from the perspective of, "I have a cell, what is its header?" It would be more efficient in reverse, if we could instead answering the question, "I have a header, what are its cells?". What dynatable does now is sort of a hybrid approach (we find the header cells, make a dictionary, then loop through the rows, assigning each cell value to its attribute in the dictionary). The issue is, according to that article, any table cell could potentially be a header cell, not just those in the header row or in th elements.
talhof8 13y ago
FYI, I'm not the creator. I just saw it on G+ and thought it's worth sharing.
- lightswitch05 13y ago
  I am the creator, thanks for posting it on here! I had no idea so many people would have any interest it it, for me, it had a vary narrow use-case
JimmaDaRustla 13y ago
Cool. This would be cool for web scraping, but probably limited in number of scenarios you would use. Would have been nice for some Google Finance data I'm scraping in python.
zekenie 13y ago
How hard do you think this would be to rewrite in node w/o jquery? As was noted I could see a lot of scrapping potential for this.
nly 13y ago
The non-webdev part of me is rather depressed that we've resorted to storing N typically homogeneous records in a way which results in the serialization of field names N times. Wouldn't it be nicer to make the default format an object of arrays rather than an array of objects? In before "but gzip!"
- jimktrains2 13y ago
  TSV or CSV?
- cdcarter 13y ago
  But to do easy and ~beautiful~ work on the objects (and to work on them the way most JS libraries expect data) you'd have to zipper the arrays together anyway.
bdg 13y ago
I'm having a hard time understanding why everyone's jazzed about this. When is this a useful thing to do? I've literally never once encountered a problem where this would be useful. I'm not trying to sound harsh, I just don't understand when I'd ever need to do something like this.
- JangoSteve 13y ago
  This library is like a building block to e.g. building your own interactive table functionality. Let's say you have an HTML table, and you want to allow users to dynamically search or sort the table. Step 1 could be to translate the table to JSON, step 2 would be do all your JS magic on the JSON data set, and step 3 would be write it back to the table. This approach is cleaner than trying to do a bunch of logic on the DOM text. So, this plugin is step 1 done for you.
  - bdg 13y ago
    I suppose that's my issue. Dynamically searching or sorting wouldn't require converting to JSON, I'd rather work with the existing table object: tableName.rows[rowNum].cells[cellNum] Instead of loading all of that into a variable (a second time, it's already an object on your page), trying to keep it in sync, binding events in both directions to keep it in sync, the load of doing twice the work, the lag of redrawing the entire table every time you want to do something, etc.. I don't see why this is cleaner than working on the DOM. I've done this a number of times and only found headaches when trying to build something larger than a silly toy or a list of 30 items on a table. IIRC DataTables did something like this internally last time I looked at it, and that thing konks out and either slows down your page or crashes the browser depending on how many cells you load into it.
    - JangoSteve 13y ago
      I think the issue is that, if you are doing something like sorting, you actually are going to have to redraw the entire table. Also, for things like plain-text searching, it's faster to do it on a collection of objects detached from the DOM in JavaScript than to scan the DOM, and especially if you're doing non-text comparisons (like comparing a number with the parseFloat representation of the values in one of the table columns for all rows). And finally, if you are going to read and then redraw in the DOM, it's best to do it as grouped atomic operations due to the inefficiency making the browser reflow (https://developers.google.com/speed/articles/reflow https://developers.google.com/speed/articles/reflow). For what it's worth, this is how Dynatable works internally, though I haven't had any issues with it being slow, though it's possible I just haven't used it with large enough data sets yet (I've only used it on tables of probably 1000 rows or less so far).
kevinburke 13y ago
Ha, I've been working on something like this in a general case. https://github.com/kevinburke/hulk https://github.com/kevinburke/hulk
k2xl 13y ago
Can we go the other way? JSON to Table? That would make this a really useful library for saving table states.
- lightswitch05 13y ago
  I've had multiple people ask me about that. There are already tools out there that do it, so I didn't really have any motivation to make it
jasonlaster11 13y ago
Would be great as a chrome extension

64 points by talhof8 13y ago

skram 13y ago

Awesome. Something else that would be nice is to be able to go from JSON to HTML table. There are a few libraries out there that do this but none seem to be updated or recent. Any suggestions would be greatly appreciated. Nice work!

speeq 13y ago
If you're using node.js/Express, you could do this using Jade: http://stackoverflow.com/questions/5204905/dynamic-html-pages-with-jade-nodejs-express http://stackoverflow.com/questions/5204905/dynamic-html-page...
jwoah12 13y ago
It might be overkill if you don't need interactive features, but my ATable[1] library takes a 2-dimensional array representation of the data, and turns it into an HTML table. I will add support for an array of JSON objects as the data source soon. [1] - http://jarwol.com/aTable/ http://jarwol.com/aTable/

charlysisto 13y ago

Nice ! Could be used in a progressive enhancement strategy : render your html then extract the json out of it to feed your MVWhatever (Angularjs, Emberjs, Backbonejs, nameitjs) and let javascript take on for the cool client side stuff.

rcfox 13y ago

It's a nice start to something that looks pretty useful. There are some things to consider though: What if you have headings as the first column in each row? What if you have headings in the first column, and the first row? Also, what is the use case for data overrides? In my opinion, it completely defeats the purpose when your intention is to take the contents of the table.

chalst 13y ago
The code does not try to be clever: var getHeadings = function(table) { var firstRow = table.find("tr:first").first(); return notNull(opts.headings) ? opts.headings : rowValues(firstRow); }; The code is also going to fall down if you have things like wide/high cells. But HTML tables are so flexible and so routinely abused, I guess the problem is too hard to be solved robustly with any generality.
- JangoSteve 13y ago
  You could do what they're suggesting without being too clever. For instance, our dynatable plugin does it this way: table.find('thead tr').children('th,td') But it could easily be something like: var headings = table.find('thead th'); if (!headings.length) headings = table.find('th'); if (!headings.length) headings = table.find('tr:first td');
  - terrahawkes 13y ago
    The HTML specification's algorithm for determining the applicable headers for a table cell might be useful: http://www.whatwg.org/specs/web-apps/current-work/multipage/tabular-data.html#header-and-data-cell-semantics http://www.whatwg.org/specs/web-apps/current-work/multipage/... This was developed with reference to a corpus of real web tables.
    - JangoSteve 13y ago
      I was actually really excited to read that, thinking I was going to implement it straight away in our plugin. But it seems to require scanning up the table from the current cell, for each cell, to get its header, and it also requires knowing the rendered height of the cell, which I don't fully understand. I'm sure if I actually sat down and wrote it out, it'd make more sense. At any rate, it seems like trying to implement their algorithm verbatim in javascript would be pretty slow for large tables. Maybe it's better to just use it for inspiration. Also, their algorithm seems to come at if from the perspective of, "I have a cell, what is its header?" It would be more efficient in reverse, if we could instead answering the question, "I have a header, what are its cells?". What dynatable does now is sort of a hybrid approach (we find the header cells, make a dictionary, then loop through the rows, assigning each cell value to its attribute in the dictionary). The issue is, according to that article, any table cell could potentially be a header cell, not just those in the header row or in th elements.

talhof8 13y ago

FYI, I'm not the creator. I just saw it on G+ and thought it's worth sharing.

lightswitch05 13y ago
I am the creator, thanks for posting it on here! I had no idea so many people would have any interest it it, for me, it had a vary narrow use-case

JimmaDaRustla 13y ago

Cool. This would be cool for web scraping, but probably limited in number of scenarios you would use. Would have been nice for some Google Finance data I'm scraping in python.

zekenie 13y ago

How hard do you think this would be to rewrite in node w/o jquery? As was noted I could see a lot of scrapping potential for this.

nly 13y ago

The non-webdev part of me is rather depressed that we've resorted to storing N typically homogeneous records in a way which results in the serialization of field names N times. Wouldn't it be nicer to make the default format an object of arrays rather than an array of objects? In before "but gzip!"

jimktrains2 13y ago
TSV or CSV?
cdcarter 13y ago
But to do easy and ~beautiful~ work on the objects (and to work on them the way most JS libraries expect data) you'd have to zipper the arrays together anyway.

bdg 13y ago

I'm having a hard time understanding why everyone's jazzed about this. When is this a useful thing to do? I've literally never once encountered a problem where this would be useful. I'm not trying to sound harsh, I just don't understand when I'd ever need to do something like this.

JangoSteve 13y ago
This library is like a building block to e.g. building your own interactive table functionality. Let's say you have an HTML table, and you want to allow users to dynamically search or sort the table. Step 1 could be to translate the table to JSON, step 2 would be do all your JS magic on the JSON data set, and step 3 would be write it back to the table. This approach is cleaner than trying to do a bunch of logic on the DOM text. So, this plugin is step 1 done for you.
- bdg 13y ago
  I suppose that's my issue. Dynamically searching or sorting wouldn't require converting to JSON, I'd rather work with the existing table object: tableName.rows[rowNum].cells[cellNum] Instead of loading all of that into a variable (a second time, it's already an object on your page), trying to keep it in sync, binding events in both directions to keep it in sync, the load of doing twice the work, the lag of redrawing the entire table every time you want to do something, etc.. I don't see why this is cleaner than working on the DOM. I've done this a number of times and only found headaches when trying to build something larger than a silly toy or a list of 30 items on a table. IIRC DataTables did something like this internally last time I looked at it, and that thing konks out and either slows down your page or crashes the browser depending on how many cells you load into it.
  - JangoSteve 13y ago
    I think the issue is that, if you are doing something like sorting, you actually are going to have to redraw the entire table. Also, for things like plain-text searching, it's faster to do it on a collection of objects detached from the DOM in JavaScript than to scan the DOM, and especially if you're doing non-text comparisons (like comparing a number with the parseFloat representation of the values in one of the table columns for all rows). And finally, if you are going to read and then redraw in the DOM, it's best to do it as grouped atomic operations due to the inefficiency making the browser reflow (https://developers.google.com/speed/articles/reflow https://developers.google.com/speed/articles/reflow). For what it's worth, this is how Dynatable works internally, though I haven't had any issues with it being slow, though it's possible I just haven't used it with large enough data sets yet (I've only used it on tables of probably 1000 rows or less so far).

kevinburke 13y ago

Ha, I've been working on something like this in a general case. https://github.com/kevinburke/hulk https://github.com/kevinburke/hulk

k2xl 13y ago

Can we go the other way? JSON to Table? That would make this a really useful library for saving table states.

lightswitch05 13y ago
I've had multiple people ask me about that. There are already tools out there that do it, so I didn't really have any motivation to make it

jasonlaster11 13y ago

Would be great as a chrome extension