dancrystalbeach

9 ms·

91.
by dancrystalbeach 2y ago
I’ve been hacking around with tools and programming since Perl was a thing. I’ve worked the gambit of Data Platforms from large organizations to tiny startups, and all those in between. I’ve worked on Data Platforms that dropped ungodly amo
92.
by dancrystalbeach 2y ago
Weap and moan all you keyboard jockeys, you simple peddlers of code. Your doom and what you feared greatly is coming to pass. You’ve been replaced.
93.
Rust for the small things? but what about Python?(dataengineeringcentral.substack.com)
37 points by dancrystalbeach 2y ago|87 comments
94.
by dancrystalbeach 2y ago
It’s been a while since I kicked the tires on a new tool, you know I love to pick and poke at things. Also, in my never-ending attempt to keep on top of the changing landscape and keep the wolves (newer, smarter, younger engineers) from sna
95.
by dancrystalbeach 2y ago
Recently, for some unknown reason, I was pursuing the new Stackoverflow … called Reddit, for Data Engineering … and I ran across an interesting question … more or less it was related to “what makes a good Software Engineer … in a Data Engin
96.
by dancrystalbeach 2y ago
I feel like I’m sitting on the edge of the ocean, not a nice wonderful sandy beach where the gentle waves are lapping at my feet and the breeze blowing through my non-existent hair, but rather a boiling and roiling ocean pounding the shorel
97.
by dancrystalbeach 2y ago
I’ve had something rattling around in the old noggin for a while; it’s just another strange idea that I can’t quite shake out. We all keep hearing about Arrow this and Arrow that … seems every new tool built today for Data Engineering seems
98.
by dancrystalbeach 2y ago
Kubernetes. K8s. Man, I remember those days when it was hot hot hot, and you were an idiot if you weren’t deploying something onto Kubernetes. If you didn’t know how to use kubectl then you were a prehistoric human, almost as bad as those S
99.
by dancrystalbeach 2y ago
When I was young and full of myself, writing Perl and PHP, while your ma was still reading you a bedtime story and giving you a stuffy to fall asleep with, I had to program uphill, both ways, in the rain and snow. Not like you milk toast Da
100.
Databricks Buys Tabular – 1B Dollar Deal Iceberg vs. Delta Lake?(confessionsofadataguy.com)
2 points by dancrystalbeach 2y ago|1 comments
101.
by dancrystalbeach 3y ago
Ever since the days of yore, SQL has been the bread and butter of everyone from the old school DBAs to the new school Data Engineers. Of course, Python is probably the bread bag if SQL is the sandwich, or is it the other way around?
102.
by dancrystalbeach 3y ago
Using Rust to write Spark? Lord save us.
103.
by dancrystalbeach 3y ago
Are we all going to lose our jobs to AI?
104.
by dancrystalbeach 3y ago
You probably think this is another internet clickbait title uh? Just trying to get you to clickty clickty and sell you some Google Ads. Two problems. I don’t have Google Ads, and I know a small percentage of people will actually listen to t
105.
by dancrystalbeach 3y ago
16GB on 4GB machine Challenge.
106.
by dancrystalbeach 3y ago
Rust is fun to write, Python is required.
107.
by dancrystalbeach 3y ago
Rust is the future, but wrapped in Python
108.
by dancrystalbeach 3y ago
Programmers Abstract Too Much
109.
by dancrystalbeach 3y ago
Be good at software. Be better at everything else.
110.
by dancrystalbeach 3y ago
Dive into SQL indexes.
111.
by dancrystalbeach 3y ago
Simple steps for how to fine-tune an LLM on a GPU.
112.
by dancrystalbeach 3y ago
What’s the one thing that never changes in the Data World?
113.
by dancrystalbeach 3y ago
Commands your mom should have taught you.
114.
by dancrystalbeach 3y ago
Demystify semi-structured data.
115.
by dancrystalbeach 3y ago
I write an article about naughty SQL, https://www.confessionsofadataguy.com/sparksql-is-destroying... , I get destroyed on Reddit, I do a YouTube video. Life is good.
116.
by dancrystalbeach 3y ago
Learning is doing little projects that use multiple technologies. The only way to get better is to write code.
117.
by dancrystalbeach 3y ago
Two Pillars of Data Engineering
118.
by dancrystalbeach 3y ago
The Age of Containers
119.
by dancrystalbeach 3y ago
The harsh truth about writing software
120.
by dancrystalbeach 3y ago
Apache Spark vs Polars. Reducing costs.

More ›