mtrn

9 ms·

31.
by mtrn 10y ago
Very nice. It really took me just 10s seconds to sign up and clone a repo. I believe this kind of easy of setup should be much more common - across many domains and languages. That said, details will matter in the longer run. The markdown a
32.
by mtrn 10y ago
The git angle would be a huge step forward. What I found is that reproducability is not always on people's mind when they designing such systems, whereas I believe it's one of the most important properties. Pachyderm is on my TOD
33.
by mtrn 10y ago
Luigi is nice, because it is really simple to get started and it gradually allows you to do more complex things like (custom) parameter types, custom targets, enhanced super classes and dynamic dependencies, event hooks, task history and mo
34.
by mtrn 10y ago
You might like The Penultimate Truth by Philip K. Dick[1]. In this 1964 novel the world has shifted slightly: Instead of giving "just enough to quell the uprisings", large parts of the population are just kept eternally busy to su
35.
by mtrn 10y ago
> How more unavoidable is Facebook becoming over time? I do not use FB and I still have friends, family, fun and a happy life.
36.
by mtrn 10y ago
Spot on. Can you recommend some good articles or books dealing with these kind of problems? My current understanding is that a event-based (in a very broad sense) system are hard to "replay" in the case of a failure (error in data
37.
by mtrn 10y ago
There is the biological analogue, which has inspired neural networks: > Recall that in our general definition a feed-forward neural network is a computational graph whose nodes are computing units and whose directed edges transmit n
38.
by mtrn 10y ago
Glad to live in a country that has something close to universal health care. Everybody needs to contribute a monthly share (independent of their condition) which amounts to over 200B per year in total. This seems enough for modern infrast
39.
by mtrn 10y ago
> when the whole system is collectively planned and owned It's funny. It is not that capitalism does not tend towards central planning. When companies fuse and get bigger, they want to reap the profits of centralization and unified
40.
by mtrn 10y ago
On a related note, Canettis main work deals with the 20th century phenomenon of the "mass" and its constituents: https://en.wikipedia.org/wiki/Crowds_and_Power
41.
by mtrn 10y ago
James Hague, "a recovering programmer who has been designing video games since the 1980s...", http://prog21.dadgum.com
42.
by mtrn 10y ago
Thanks for the pachyderm pointer. I just installed it and will give it a try.
43.
by mtrn 10y ago
Nice, thanks for the pointer. It's nice to see templated shell calls, as these can be a powerful bridge between orchestration and execution.
44.
by mtrn 10y ago
Yes, that's ETL. Classic ETL dealt with databases, the modern variant has relaxed this constraint. As for the zip: We simply "unzip -p" and stream process it carefully (with a custom program reading XML and transforming it).
45.
by mtrn 10y ago
> I've been thinking about this space a lot Me too, for better or for worse. As for the issues, there are many. Just quickly a few: * Data provider has an FTP server, most files are automatically generated, some are hand-named (with
46.
by mtrn 10y ago
I've worked with and looked at a lot of data processing helpers. Tools, that try to help you build data pipelines, for the sake of performance, reproducibility or simply code uniformity. What I found so far: Most tools, that invent a n
47.
by mtrn 10y ago
> Git is certainly the best of the version control systems to date, and I'm happy that it won. Me too. What I found was that people do not recognize what tool git really is. People do not read the manuals and count on their world kn
48.
by mtrn 10y ago
> people shouldn't rely on them as a career Nobody thinks of that as a career, but 4.5 million people seem to rely on it for income nonetheless.
49.
by mtrn 10y ago
I believe it was this talk: https://www.youtube.com/watch?v=_1Cyyt-4-n8
50.
by mtrn 10y ago
In a recent talk LeCun said, in 2021 we can formulate natural language queries over a corpus and get back relevant snippets or even answers. His example was wikipedia, but of course this is more general. If this happens, this progress in th
51.
by mtrn 10y ago
I use jq daily and didn't know about rq, thanks! Quick check whether it is super fast for a simple task like extracting a single field from 1M JSON objects: $ time rq -j 'at "title"' < fixtures/1m.ldj > &
52.
by mtrn 10y ago
The road to hell is paved with good intentions.
53.
by mtrn 10y ago
In one way, this is sponsorship taken to a micro level. And sponsorship is fine - and usually open, for the benefit of both parties. But if I open a random influencer agency website, I can read something like this: "Influencers drive t
54.
by mtrn 10y ago
> Making the backend of Alexa and offering it to users is a killer feature. Same scheme that started AWS.
55.
by mtrn 10y ago
> I've been working on an app and spent a lot of time looking into frameworks to handle its complexity but ended up sticking to vanilla JS to keep more control. On a recent hackathon, I wanted to implement instant search and ended u
56.
by mtrn 10y ago
While not new, I find the discussion around immutable storage important and interesting. Datomic[1] comes to mind. There are many scenarios and applications, where it would be nice to just write things to a structured log and build an appli
57.
by mtrn 10y ago
I've written my first LaTeX documents with LyX [1], a wonderful application, if you're just getting started. [1] https://www.lyx.org/
58.
by mtrn 10y ago
Technology is one challenge. I wonder what other kinds of scaling must take place to sustain such industries? Like, how do you scale learning, so that a single person or a small team is able to have a good overview plus detailed know-how on
59.
by mtrn 10y ago
If we are currently at 35B transistors per person on earth produced per year, back-of-the-envelope calculated, in 2025, we will probably have 200B transistors per person running on this planet. Or about two DGX-1 for every single human.
60.
Transistor Production Has Reached Astronomical Scales (2015)(spectrum.ieee.org)
81 points by mtrn 10y ago|40 comments

More ›