snidane

8 ms·

31.
by snidane 4y ago
Shell and SQL make you 10x productive over any alternative. Nothing even comes close. I've seen people scrambling for 1 hours to write some data munging, then spend another 1 hour to run it through a thread pool to utilize those cores
32.
by snidane 4y ago
If you need to maintain some sort of deductive correctness - ie. my inputs are correct and my code is correct therefore my outputs are also correct - you're gonna expose yourself to only a tiny amount of real world problems. Data engin
33.
by snidane 4y ago
That sounds like infra ops, not data engineering. It's a common theme, though, for DEs to slip into maintaining infra and analysts taking on the role of engineering data pipelines.
34.
by snidane 4y ago
Engineers demanding unit test for data is a perfect test to weed out the SWEs who are bit DEs. Ask about experience with data quality and data testing when you interview candidates and you'll distinguish the people who will solve a pro
35.
by snidane 4y ago
Technology can be both good and bad, depends on the holder. The government clearly has a goal they want to achieve and cleaning up their data with AI seems like a solution more likely to work than throwing a small army of bureaucrats at it.
36.
by snidane 4y ago
My Viagogo experience in 2019. After an hour waiting in line for the event I get my ticket scanned only to realize they sold it to somebody else at the same time. Another 30 mins waiting at the security trying to get somebody from their cal
37.
by snidane 4y ago
Another useful json explorer TUI - jless. https://jless.io/
38.
by snidane 4y ago
"Quantum computing as a field is obvious bullshit" https://scottlocklin.wordpress.com/2019/01/15/quantum-comput...
39.
by snidane 4y ago
The world without Softbank is not infested by monopolies, bot at all. There never were taxi medallions worth hundreds of thousand of dollars, there is no monopolistic ponzi game in real estate, the world is a perfect competition as the econ
40.
by snidane 4y ago
As academic world starts to act more like a religious church, and losing grip on being the good pastors of science, the actual scientific community will detach themselves and leave the academic dogmatists with a label of "scientism&quo
41.
by snidane 4y ago
What is the use case? I can't imagine anything besides getting a real time view on the frontpage or likes of my own comments that a monthly static data dump wouldn't solve.
42.
by snidane 4y ago
Solving the unaffordability of housing crisis is the step needed to unblock fertility. The solution exist, yet it cannot be pushed forward in a democratic system as it creates a generational conflict. Hint: LVT
43.
by snidane 4y ago
> So, what instead? You want to build more housing in NYC? Great, that should work. But you have to make it cheap. > because that's the only way a developer can recoup their enormous land acquisition, There you go - in order to m
44.
by snidane 4y ago
The employer needs the worker's labor and ghe employee needs the employer's concentration of power. One needs the other and vice versa. Or are you suggesting the directionality is only one way? Marxists do think that way and they
45.
by snidane 4y ago
Another Marxist in a new coat. Good analysis, wrong prescription. Been there, done that, ended up badly. Look into Georgism, that's where the cool kids are now. You don't need a revolution, you need a minor shift in tax code that
46.
by snidane 4y ago
All of these visual ETL tools are nothing more yhan a patch to the new breed of analytical databases because their support for loading files sucks so badly. Even the praised dbt is just a hack to make declarative incremental because the SQL
47.
by snidane 4y ago
Europeans would rather go Japan - ie. keep their national identity and downsize, than America - healthy demographics, but constant cultural clashes and crime waves.
48.
by snidane 4y ago
Your goal as a manager is to maintain flow - provided that you hired competent people. When you lose it and you feel the blame starts shifting your way your tendency is to set up more control and visibility. Here comes the daily standup and
49.
by snidane 4y ago
select product , sum(price) as price from table where price<1000 having price>10000 You can refer to the aliased price column before or after aggregation using where or having. Depending on the sql
50.
by snidane 4y ago
Except that according to the same website he stated the opposite of that quote. > It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth.” – W. Edwards Deming, The New Economics. https://demin
51.
by snidane 4y ago
The outcome for citizens is the same as in soviet style communism, except in communism the state enterprises were supposed to serve the commons. In this case they don't even have to pretend to.
52.
by snidane 4y ago
Abolish and replace with what? How do you ensure that the outcome doesn't resemble post-soviet privatisations that are impossible to get undone within 100 years, if even that. It looks great on paper, until it doesn't in practice
53.
by snidane 4y ago
World government makes sense so that coverage of responsibility would be extended over all common areas such as air and oceans. With decentralised govt there is no owner of oceans, it's a common pasture, so nobody prevents certain coun
54.
by snidane 4y ago
The company doesn't have the power to change politics like that, so there is no point in getting the company involved or pick sides. The only neutral solution is to ban political discussions in the workplace, same as it should be in sc
55.
by snidane 4y ago
> Why would I write a separate schema when the object itself knows what it will accept, what is optional, and what range the values should be in? Because data is just data and the meaning to it is given at the time of application. If you
56.
by snidane 4y ago
The article is not explaining the point, which I believe is: type your dicts if you want to provide strict guarantees to your downstream about data shape. If you know precisely what the data is used for - great, go ahead - type system is yo
57.
by snidane 4y ago
Snowflake doesn't support this kind of except, which is needed for the diff to work seamlessly: select * except (column1, column2, ...)
58.
by snidane 4y ago
What benefit does it give me over running table diff query in SQL? select * exclude (date_uploaded) from dev_table except select * exclude (date uploaded) from prod_table or select * from (select * exclude (date_uploaded) f
59.
by snidane 4y ago
No code is a sales tactic to circumvent engineering folks who would correctly point out that the tool helps with 80% basic use cases and for the remaining 20% engineers have to be involved, but now instead of using python or something appro
60.
by snidane 4y ago
A bit clunky, but works. ps | awk '$1=$1' OFS=, | duckdb :memory: "select PID,TTY,TIME from read_csv_auto('/dev/stdin')"

More ›