vegabook

94 ms·

841.
by vegabook 11y ago
Completely agreed. Problem is while you're exploring to find the correct dimensional reduction / chuckout strategy, you're typically rapidly scanning through lots of scatter plots / density plots / whatever in grids
842.
by vegabook 11y ago
31 libraries, zero for "Large Data Set". That tells you everything you need to know about Javascript/browser graphics for data science. A. The DOM cannot handle as many nodes as are needed for large datasets and/or B. ev
843.
by vegabook 11y ago
obviously I use do.call all day long because R is my primary weapon, but even if I say so myself, a happy R user, Python with Numpy is faster. I would invite you to show me a single instance where R is faster at bog-standard memory access,
844.
by vegabook 11y ago
> mm <- matrix(rnorm(1000000), 1000, 1000) > system.time(eigen(mm)) user system elapsed 5.26 0.00 5.25 IPy [1] >>> xx = np.random.rand(1000000).reshape(1000, 1000) IPy [2] >>> %
845.
by vegabook 11y ago
10k? Those sticks of RAM alone will cost you something like 75k USD. Then you'll need the processors, arguably 4 of the top of the line 18-core XEONs at 5000 USD each. Then you'll need to put it all together with software and a (p
846.
by vegabook 11y ago
Arguably Numpy/Pandas is just as performant as Scala/Java and it certainly beats R hands down when data becomes more than a say 10-20 gigabytes after which I find R slows to a crawl.
847.
by vegabook 11y ago
4-6 months and you'll have a Knight's Landing Xeon Phi with at least 72 cores and 288 hardware threads, with vector instructions, and you'll be able to stick 3 of them in single blade.
848.
by vegabook 11y ago
The Python 2 codebase is growing much faster than Python 3, because the Python core developers have lost touch with the fact that its most dedicated users are not web people, but scientific computing users for who the overwhelming choice
849.
by vegabook 11y ago
I use R extensively, even more than Python, but it is not a robust language when your problem moves from exploration to anything resembling production. The memory management appears to be the problem. Hugely inefficient and slow even with 3
850.
by vegabook 11y ago
right thanks for clarifying - I wasn't confused until Python 3 made it confusing. EDIT: Ah I remember. In one of my attempts to move to 3 I wrote a few modules which worked with strings, then I had to stick "u"s everywhere wh
851.
by vegabook 11y ago
First of all I don't think the print statement is a small issue. I bet 5% of my code is print statements. I love the print statement's unfussy "can do" mentality. Hey, the whole reason I got into Python was because it
852.
by vegabook 11y ago
For me, the only place where Python truly excels versus the competition is Numpy. The easy python scaffolding gets you to the heart of a problem in a friendly and flexible way, and then you can interface easily to this big V8 of a compute e
853.
by vegabook 11y ago
I am given to understand that some functionality is not compilable or redistributable. I work in a fairly specialized field (fixed income term structure optimization) and this page is not encouraging (see "cannot be compiled" colu
854.
by vegabook 11y ago
That is a decent counterpoint to my original comment I think. I work with clients who are very happy and very productive with Matlab, precisely for the reason you cite: fantastic docs and support. I just cannot justify the huge spend on eve
855.
by vegabook 11y ago
This seems a little hasty on the part of the Julia guys. This is a promising language and I for one was looking closely for alternatives to Python. Yet Julia hasn't built enough traction yet, and here we are with a commercial arm run b
856.
by vegabook 11y ago
That's like comparing grains of wheat to fully baked lasagnas, buddy. What matters is what the market is prepared to pay for the tech. No arguments here. Fifty:one. It's a verifiable fact just like my ipython example. Now that doe
857.
by vegabook 11y ago
Actually, dollar for dollar, Intel:ARM sales looks a lot like Intel:ARM performance. Something like 50:1.
858.
by vegabook 11y ago
Yes the RPi is a fantastic device and I am building a small minimilastic hardware terminal for financial visualization around it which will be dirt cheap and surprisingly powerful, mainly because its graphics are actually really fast enough
859.
by vegabook 11y ago
I think Gnome 3 doesn't quite "do" polish as much as Unity. I am very fussy about fonts looking right, and I'm afraid that at smaller sizes, the font scaling in the Gnome terminal just looked wrong, whereas Unity is perf
860.
by vegabook 11y ago
I love arm and own a cubietruck and a Pi. But my initial hopes of clustering up tons of arms together have been dashed by a very hard dose of reality: the bog-standard core i7 in my Dell M3800 is 50x faster. Try this on your pi in iPython
861.
by vegabook 12y ago
Consumer credit rating agencies already essentially do this, and a number of subprime lenders only make credit decisions if they can connect to social media accounts. There are specialists in "connecting the dots" of all your vari
862.
by vegabook 12y ago
I can't help thinking this whole trend started with the industry which epitomises the values that our working society holds, that is finance, and the trading floor. I worked on one of these for 15 years - it was fantastic for getting d
863.
by vegabook 12y ago
Wow I have to agree with this (even though the post is using plotly). I don't know where the culture difference comes from between the R and Python ecosystems that makes the latter so much worse than the former when it comes to graphic
864.
by vegabook 12y ago
I would love to see the tails of this distribution. My bet is that the top 20-2% of earners pay almost all of this 84% of income tax (adjusted for income), and the the "real money", that is, the last 2%, pay a vastly diminishing p
865.
by vegabook 12y ago
SQL is useful when you have a use case that will require randomly querying the dataset. This is a case of synchronous parsing in which any database (relational or not)'s random-access capabilities are not used at all, yet you're p
866.
by vegabook 12y ago
This is riddled with problems. a) I routinely load 6GB of financial data into R in less than 45 seconds, and it's complex and hierarchical to boot. Admittedly that's in R's native file format but even a flat CSV takes less th
867.
by vegabook 12y ago
Absolute nonsense indeed - in your own comment. Germany never had the euro "forced" upon it. If you recall it was called the "écu", until Germany called it the euro. Germany has benefited by far the most from euro in the
868.
by vegabook 12y ago
Concur. Hold onto her while she does the dirty work, but her days are numbered once Chrome is done. She's the hatchet (wo)man - and nobody will want her around post-cull. They'll want fresh untainted vision, buy-in from the surviv
869.
by vegabook 12y ago
I'm writing 20000 financial ticks per second into Mongo on commodity hardware, namely a bog-standard i7 with 32 gig costing less than 2k. I don't have a big budget. I don't have a lot of time to mess with ACID. There is no w
870.
by vegabook 12y ago
...though the idea that "engineers want to produce art, while management wants to sell stuff" has a lot of truth, and does have oblique relevance to the post, in the sense that Charles' "goofing off" was about think

More ›