6 ms·
CSVJSON – Self Rise of an Online Tool
- HoyaSaxa 11y agoCoincidentally, I used this today for the first time. It was the 2nd result on Google for me, but was far superior to the first in ease of use. Thanks for your contribution and good luck going forward. My use case today was to create a fixture of initial data for a django app from csv. Oddly enough I wasn't able to find a specialized online tool to create the django fixture and was forced to use your tool along with some manual hacking.
- martin_drapeau 11y agoGlad it was helpful. And thanks for the nice comment.
- kawera 11y agoThanks, I've used your tool a few times and it works beautifully. Quite interesting article too from the SEO pov. BTW, does anyone know of a tool to do the reverse? I have some json files with deeply nested arrays and would like to output a csv file with one line for each "leaf". I can write it myself but would be happy to use an existing tool.
- martin_drapeau 11y agoI suggest you go to Github and open an issue: https://github.com/martindrapeau/csvjson-app/issues https://github.com/martindrapeau/csvjson-app/issues From there, I'll be happy to code it.
- matthewmacleod 11y agojq (https://stedolan.github.io/jq/ https://stedolan.github.io/jq/) is a great tool for general JSON manipulation. It includes a CSV export filter, so you can run commands like: jq -r '. | map(.key), map(.value) | @csv' which will output a JSON array as CSV.
- kawera 11y agoVery elegant solution, thank you. jq is indeed a powerful tool but it can get hairy with obnoxiously intricate json files/streams.
- jsnell 11y agoOne that I wrote a while ago, which might or might not do what you want: https://github.com/jsnell/json-to-multicsv https://github.com/jsnell/json-to-multicsv Note that the mapping from nested / hierarchical JSON file to CSV file(s) is not as obvious as it is the other way around. There's some configuration involved. I wrote a blog post about that particular aspect: https://www.snellman.net/blog/archive/2016-01-12-json-to-multicsv/ https://www.snellman.net/blog/archive/2016-01-12-json-to-mul...
- kawera 11y agoI tried but json-to-multicsv didn't really solve this particular case as I have some weird json files. Not been well versed in perl, I ended up writing a little script in python and, for that, your blog post was very helpful, thank you!
- yareally 11y ago> BTW, does anyone know of a tool to do the reverse? I created a java/scala library[1] (source is scala, but there's a jar with dependencies for java) a few years ago for an Android app I was hired to work on that converts nested JSON to CSV. I was allowed to open source the library later on and released it under the MIT License, so anyone can use it now. Never thought anyone else would ever need to convert json to csv, but I guess I was wrong :) I don't have a use case for it anymore, so I haven't maintained it, but if someone wants to update it, feel free to send a pull request. [1] https://github.com/yareally/json-to-csv https://github.com/yareally/json-to-csv
- EvanPlaice 11y agoYes. Use the built-in JSON.parse() to get the data into memory, then use my jquery-csv library's $.csv.fromArrays() to format the data as CSV. Guess I found a good candidate for a blog post. It's amazing How much I see others struggle with such a aimple concept.
- yareally 11y agoNot sure if you parsed my comment incorrectly or you meant to reply to someone else :)
- EvanPlaice 11y agoSomehow missed the 'java/scala part'. Either way, why do 2 data format transforms as one lib? It makes more sense to parse JSON into memory in one step then format it as CSV in a separate step. Ie, what about the 'Single Responsibility Principle'?
- yareally 11y ago1) The parsing and converting are done with separate methods. Thus single responsibility. 2) It's a very small library. Less than 200 lines (including comments). There's no reason to put this in 2 jars, classes or packages. Have you looked at the code at all? 3) The method call to parse JSON is in the method that converts the CSV, but it's trivial to move it out and run it explicitly. Simply remove the call to parse and explicitly run it separate. 4) Client wanted the ability to filter out JSON data they didn't want in the CSV. Thus, it's a little more complicated than a call to something like JSON.parse() in Scala/Java. 5) Code does what it needed to do and made the client happy. If you think you can do it better while keeping the code in Java/Scala, send a pull request to my repo and we'll discuss :)
- hayksaakian 11y agofrom 15000 quality page views you should expect at least $15 of revenue ($1 CPM) even higher numbers from developers (valuable users). Check out something like https://carbonads.net/ https://carbonads.net/ and it might at least pay for your hosting. It's invite only though, so you might have to ask for an invitation.
- d4nt 11y agoWe put ads on sqlizer.io for a while. The CPM numbers were terrible. I think developers all run Adblock.
- martin_drapeau 11y agoPersonally I hate it when ads get in my way. Like anyone else, I do like money though. So I'm struggling with trying to monetize this. I don't want to annoy developers. This being said, I will consider trying carbonads. Thanks for the tip.
- kawera 11y agoHave you thought about a long term sponsor? It may interest a toolmaker in the ETL space by offering to host a respectful ad yourself with a link and no tracking. You won't get rich but it's probably much better than using an ad network.
- martin_drapeau 11y agoI am open to this. Just haven't actively looked for one yet.
- homero 11y agoYou don't rank on beutifier, I've searched lots of times and have never ever seen you, I wish I did
- userbinator 11y agoHaving actually written something like this before --- and it remains on my machine as an executable of only a few kilobytes --- I'm curious about the thought process that could lead to someone uploading a considerable volume of potentially sensitive CSV data to this site, and then downloading a proportionally larger amount of JSON from it. The site doesn't even appear to support HTTPS.
- me_bx 11y agoTechnically, all the file processing can happen client-side now, thanks to the HTMl5 file API. Users' data doesn't have to leave the browser. [shameless plug] This is the approach I have taken creating a similar converter (which also works offline): http://mango-is.com/tools/csv-to-json/ http://mango-is.com/tools/csv-to-json/
- martin_drapeau 11y agoYes, that is a good idea. However I chose to upload it in order to allow sharing the conversion via a unique generated URL.
- plugnburn 11y agoWhy not use location.hash and URL shortener? That way no upload is needed.
- userbinator 11y agoWhat's the maximum length of a URL? Hint: a lot smaller than almost all the CSV files I've had to deal with. To the grandparent: you might want to make it more explicit that you're actually retaining the uploaded data and not just using it to generate the result page.
- martin_drapeau 11y agoIf the information is sensitive, I sure hope the person does not upload it to the internet.
- l_beka_l 11y agohttp://2json.net/ http://2json.net/
- duncans 11y agoFYI: The term I search for when needing something along these lines is "JSON prettify"
- martin_drapeau 11y agoInteresting. I had never considered that search term. The volumne is low (Google AdWords Keyword Planner shows a 2,400 search volume per month, 10% of JSON beautifer). Yet it is a keyword I will add. Thanks!
- EvanPlaice 11y agoThe funny thing is, very capable libraries already exist to handle CSV parsing. I know because I authored jquery-csv, te first RFC complete CSV parser implementation in vanilla JS. The transform is ridiculously simple and data can be fully loaded and processed client-side using the HTML5 File API. As soon as JS gets the :: function bind operator I plan to reimplement it as a pure async function.
- martin_drapeau 11y agoWow, I wish I had found your library when I wrote this. Looks like you published it around the same time I wrote it. I especially like the idea of client-side loading using the HTML5 file API. Things to consider for the future.
- EvanPlaice 11y agoUnfortunately. I made the mistake of hosting it on Google Code rather than GitHub. Everything has since been moved and it's gradually gaining traction again. The HTML5 part was very appealing to me because I wanted to build a client-side tool to load/save/edit database imports/exports. It unlocks a lot of potential for fully-featured client-side applications that work more like desktop apps. The bad part about it is, browser vendors, specifically MS, have been dragging their feet when it comes to fully supporting the File APIs.
- EvanPlaice 11y agoHonestly, I see JSON-to-CSV and the opposite tools as an anti-pattern. You're essentially applying 2 pure function transforms to data so it should be done in 2 steps. The built-in JSON.parse() provides a native implementation to handle the first transform. My jquery-csv lib handles the second. If you require stream processing and/or need to process large data sets client-side use the more fully featured PapaParse lib instead. The number of half-assed CSV parser implementations I've seen in the wild is what drove me to provide a fully-featured RFC implementation. To do it right requires a DFA, anything less will fail on many hard-to-identify edge cases. My project has been downloaded over half a million times, used on every platform/environment imaginable, and every technical limitation painfully identified. Seriously, I don't understand why devs feel the need to roll their own and miss out on benefiting from existing battle-tested solutions. Not to mention all the devs who embed a wholesale copy of an old/outdated version of my lib in their projects. Nowadays, half-assed CSV implementations have changed to half-assed JSON-to-CSV implementations. The same holds true for JSON and/or CSV to HTML transforms. Formatting a HTML table by iterating over data is so painfully sinple I don't know why devs feel te need to provide it as a standalone library.