slotrans

18 ms·

151.
by slotrans 4y ago
In short, no, no, and no. If my DAG tool says "I can't handle this, use Spark" I'm throwing that tool away. This is not an exotic Big Data problem, it is extremely pedestrian and common. It's just not reasonable to
152.
by slotrans 4y ago
> "spicy" take: allowing users to write imperative code (e.g. using loops) that dynamically generates DAGs are never a good idea. Said with all love: you're definitely wrong and it's not hard to demonstrate. 1) so
153.
by slotrans 4y ago
Ah, it seems like you're imagining Hamilton to be used to structure sort of "small" pieces, which might then be orchestrated into a Big Picture thing by another tool, something in the Airflow/Dagster/Argo/Flyte
154.
by slotrans 4y ago
As a long-time fan of DAG-oriented tools, congrats on the launch. Maybe you can get added here https://github.com/pditommaso/awesome-pipeline now or in the future... This is a problem space I've worked in and been
155.
by slotrans 4y ago
On the object level, Roko's Basilisk is absurd. However, it's a very useful thing nonetheless, in that professing to believe in it is a Giant Red Flag. Any such person can be safely dismissed as a crank.
156.
by slotrans 4y ago
It's extremely accurate, though. He described 4 of the last 6 jobs I've had, as if he had been there with me.
157.
by slotrans 4y ago
Since the data is not under your control -- it belongs to the "real" engineering team, which has vastly greater status -- you will not be able to. You will complain and they will ignore you. You will offer to help and they will re
158.
by slotrans 4y ago
Oh for sure. But doesn't that increase the returns to making the code more efficient? I don't remember which software it was but one of them would show, in the footer, a count of MySQL queries run. My memory tells me it would us
159.
by slotrans 4y ago
I honestly don't know why I have to explain this, it is, as I said, shockingly obvious. First, the mega obvious: if I leave a PR comment, and you ignore it for several days , resolving the conversation will take several days! If I wal
160.
by slotrans 4y ago
Yes. These companies will die without Google and Facebook ads. Not immediately mind you, but after 6-12 months of near-zero new customer acquisition they will be out of business.
161.
by slotrans 4y ago
I'm generally pasting the results of a SQL query so this is, while possible, massively annoying to do
162.
by slotrans 4y ago
I love how fast it is. I used to hang out on a lot of phpBB and vBulletin forums, and one thing that puzzles me in retrospect is why their performance was so terrible. I can certainly understand how the first versions would be not so great,
163.
by slotrans 4y ago
This is cool, but I think it might be missing what the big problems with spreadsheets are. I was complaining just today because I pasted some records into Excel and it decided to treat 933349234275230104 as a number (it's a hash), conv
164.
by slotrans 4y ago
I (mainly) don't want to aggregate them, I want to look at them. Don't forget "Filter". It's very easy in a spreadsheet to take a huge number of rows and cut them down -- interactively! -- to a subset that's of
165.
by slotrans 4y ago
No one loves SQL more than me but the effort jump between "paste into a spreadsheet" and "load into SQLite" is at least 100x Computers are fast, I have gigabytes of RAM, I should be able to look at 10M rows
166.
by slotrans 4y ago
can it handle a million rows? 10M? believe it or not this is my biggest problem with Sheets, and while Excel does better it's still capped at 1M
167.
by slotrans 4y ago
A typical ecommerce business that advertises on Facebook might bring in 30% or more of their new customers through that channel. That's "vital". EVERY direct-to-consumer ecommerce company buys Facebook and Google ads. Every
168.
by slotrans 4y ago
> That's really not my experience. I'm always very confused when I see this statement being thrown around as if it was a universal truth. Because it's obviously true. Communicating through pull requests is shockingly ineff
169.
by slotrans 4y ago
> Also anecdotally, in 10+ years of office work, I've never observed a single "watercooler conversation" leading to a particularly interesting idea. I have. Happens all the time. > The famous "whiteboarding session
170.
by slotrans 4y ago
Why are refreshes assumed to be bad?
171.
by slotrans 4y ago
It screams "I'm about to quit"
172.
by slotrans 4y ago
Ah yes, the old "if FAANG does it, it must be right" argument. You love to see it.
173.
by slotrans 4y ago
> Is there a way to get that speed back without the same sort of practice? No. Practice is the ONLY way to get good at anything. Doesn't matter what it is.
174.
by slotrans 4y ago
Dependencies.
175.
by slotrans 4y ago
in 17 years of professional software development I've used a binary tree exactly zero times
176.
by slotrans 4y ago
For me the big one was relational modeling & normalization. I took a database class in college and I thought normalization was idiotic. It seemed obviously foolish and wasteful and I couldn't imagine why anyone would do this to the
177.
by slotrans 4y ago
This comes up periodically, but it's never interesting. SQL is good. What people don't like about it is all rooted in their unwillingness to learn it, on its own terms. They invariably want to import ideas from imperative language
178.
by slotrans 4y ago
> What hopefully isn’t true in your business is that no-one will ever care about this issue and that it will never be fixed. Instead, I hope you periodically address technical issues brought up during prioritized work and you fix bugs li
179.
by slotrans 4y ago
> New languages like Rust/Ocaml/Nim.. if yes then which? Completely irrelevant. DE is SQL, Python, sometimes Scala/Java. Get really good at SQL. Learn relational fundamentals. Learn row- and column-store internals. Underst
180.
by slotrans 4y ago
Office upgrades were never forced. I pay for an Office365 subscription so I can have it on Mac, but on my Windows PC I use Office 2010, which I paid for when it was current. It is lightning fast compared to the current release of Office,

More ›