karbarcca

5 ms·

1.
by karbarcca 6y ago
No worries, it doesn't come off that aggressive ;) Yes, I've definitely been involved in other projects where a single core developer can do a little too much "imposing" of their will. i.e. if they happen to have a per
2.
by karbarcca 6y ago
Unfortunately almost exclusively via http rest apis. It's not great, but it's the lowest common denominator between the vast "ingestion" service we've built (connectors to web apis, local application for local file
3.
by karbarcca 6y ago
The biggest difference in these benchmarks comes down to how multiple threads are leveraged; (disclaimer: primary CSV.jl author here). In CSV.jl, it was relatively straightforward to add multithreaded parsing support; we chunk up the file,
4.
by karbarcca 6y ago
I agree; if I needed to parse CSVs in python and could utilize the arrow format, I would definitely use pyarrow. I actually recently finished support for reading/writing the arrow format in Julia ( https://github.com/Jul
5.
by karbarcca 6y ago
Take a deep look at Julia! Adding multithreaded parsing capabilities to CSV.jl was really a joy; basically just chunking up the file and spawning threaded tasks to process each chunk with the existing parsing code. My favorite favorite thin
6.
by karbarcca 6y ago
It's always a transcendental experience to move data from CSV into any more structured/flexible format!
7.
by karbarcca 6y ago
One advantage I've noticed at least (having been a contributor to the Julia language itself), is that it at least makes contributing to the language very approachable. Granted you see lower quality proposals from time to time, but in g
8.
by karbarcca 6y ago
No worries. The CSV.jl package has just been around for 5 years now, has some 700 issues opened/resolved, has built up a pretty extensive test suite, and is used in production by a number of companies, so I just wanted to hopefully cla
9.
by karbarcca 6y ago
It can make huge amounts of difference in a production system; where I work, we process terabytes of csv data every day; saving minutes per file can add up to enormous differences in CPU cost/time for a production system running
10.
by karbarcca 6y ago
As the primary author of CSV.jl, I can help clarify on the posted issues: * #720: ended up not being an issue at all, but a misconfigured environment * #714: there was indeed a corner case when automatically detecting float values where the
11.
by karbarcca 6y ago
I made https://github.com/JuliaData/Strapping.jl , which is part of an ORM; i.e. it does the translating between Julia objects/vectors of objects and 2D tables. But it doesn't do some of the more "magical
12.
by karbarcca 10y ago
Nice lineup! Can't wait to nerd out for a week. Had a great time last year rubbing shoulders with a lot of smart people and the hack nights were great too.
13.
JuliaCon Optimization Presentation Videos(julialang.org)
6 points by karbarcca 12y ago|0 comments
14.
by karbarcca 12y ago
We're actually lucky it's this good! It was as good as our audio/visual dept could edit it; we definitely didn't prepare as much as we should in this area and next year will be much better.
15.
JuliaCon 2014 Opening Session Presentations(julialang.org)
68 points by karbarcca 12y ago|6 comments
16.
Data Structures as Code: The Joys of Meta-Programming(quinnj.github.io)
12 points by karbarcca 12y ago|0 comments