cmollis

10 ms·

61.
by cmollis 4y ago
this is a major hassle when converting from avro (from kafka which uses a schema registry, so schemas are not shipped with the avro data) and storing in parquet which requires a schema in the file but you can 'upgrade' it with ano
62.
by cmollis 5y ago
I really only stopped caring what people thought after I got divorced (I was 43 at the time). Before that, it was 'the big house', 'the bmw', 'running the cool business'.. ostensibly, showing off to everyone e
63.
by cmollis 5y ago
I've run a 'crackpot consultant business' for like the past 30 years. except in the US it's called the 'Fortune 500'.
64.
by cmollis 5y ago
I should also qualify my jaundiced view of the future, with an actual belief that we'll eventually solve some of these problems.. particularly global warming, and energy creation. I have exactly zero faith that we are finished trying
65.
by cmollis 5y ago
I should add that, of course, there actually was terrorism.. just not in the US on the scale of 9/11.. (which I had the misfortune of experiencing firsthand in Manhattan).
66.
by cmollis 5y ago
I was born in 1967.. so I remember the 80's very well. First, let me say that it wasn't all great: AIDS was pretty scary (as someone else noted).. no one knew what the hell that was and it was killing a lot of people. There was
67.
by cmollis 5y ago
I can attest to that statement. I work with many contractors from Ukraine and they're tough, determined, and extremely skillful. They can handle any problem, in any condition, at any time. In my experience, they will not stop until
68.
by cmollis 5y ago
which, of course, is dependent on perspective, or zoom level.
69.
by cmollis 5y ago
I liked this one 'Chaos itself turns out to be stable'
70.
by cmollis 5y ago
I don't.
71.
by cmollis 5y ago
Moreover, storing a large binary file (in this case) in Dynamo is probably not the best use-case for it..most likely would have to convert to base64 in and out of it.
72.
by cmollis 5y ago
You're not going to get O(1) lookup using the S3 API. The APIs are basically O(n). So we we use DynamoDB to store the S3 URL.
73.
by cmollis 5y ago
currently reading Dr Strogatz' book 'Infinite Powers'.. fascinating journey through calculus' origin story. Actually reading about Newton's method of integration using infinite series right now..
74.
by cmollis 5y ago
we use DynamoDB like a big hash table of s3 file locations.. we look up these locations via a key (at the time, it sounded like a pretty good use-case for it). I suppose we could have used some other managed redis or memcached thing, but
75.
by cmollis 5y ago
I was at Princeton once (maybe 15 or 20 years ago) for a meeting with one of the startups that spun out of one of their graduate programs.. The guys asked to meet me at the CS building (which was pretty cool to actually be in because I cou
76.
by cmollis 5y ago
spark dataframe api..
77.
by cmollis 5y ago
agree.. I really like scala and Li's blog and book have helped me immensely. I use it mostly for spark, but I've found it really useful now for other things. The concurrency really is pretty nice (compared to other languages).
78.
by cmollis 5y ago
Yes it would.
79.
by cmollis 5y ago
I generally understand the construction.. (you know, the canonical example of splitting a string of words by spaces, which returns a collection, which is then flattened). But there seem to be so many ways to use it differently.. resolving
80.
Whenever I'm bored, I try to understand flatMap
5 points by cmollis 5y ago|5 comments
81.
by cmollis 5y ago
exactly. sounds really good.. but two guys in Seattle vs like the rest of the world's particle physicists? hmm..
82.
by cmollis 5y ago
another wonderful, honest, insightful post. Your book and blog posts have helped me immensely in the past two years deploying spark and 'big data' tools/processes at my company. I could not have done it without your help.
83.
by cmollis 5y ago
healthcare insurance is the best example of a state-sponsored scam I've ever seen.. and it just continues unabated. I can guarantee that almost no one involved in that transaction actually knows what the per-unit cost for a 1 hour ultr
84.
by cmollis 5y ago
it's all been said before here, but after working in IT for 30 years, I've literally forgotten more things that I've done than they could ever ask me. I've written threaded C++ code , built numerous IOS (objective C, Sw
85.
by cmollis 5y ago
I never understood how anyone who calls themselves a 'project manager' of a software project, could not actually be a technical contributor. The best projects that I've been associated with have always been led by technician
86.
by cmollis 5y ago
yeah.. I really don't see why everyone gets so bent about it. I can certainly understand the cognitive load of learning essentially all of the features so you can find the ones that work.. but really, picking a set of features that wo
87.
by cmollis 5y ago
"And everybody essentially picks a subset of C++ and writes in it ignoring the existence of other features.".. very true in my experience.
88.
by cmollis 6y ago
completely 'normal'.. I've literally never been trained on anything that I've actually worked on. The natural reaction to this is basically the standard 'google all the things' when you're working with so
89.
by cmollis 6y ago
I used to rail against this practice of using CS-y questions as hiring filters.. but I think now I get it. I just got a take home test from a start up where I had to find all bounding boxes in a 2d array of chars, filter out the overlapped
90.
by cmollis 6y ago
I switched my major from 'business' (yawn) to comp sci 35 years ago (1986).. at my school, it was previously a 'concentration' in the Math department, which meant you got a BS in Math, concentration in Computer Science.

More ›