3 ms·
Oh yes, it happened to me. It was with Palantir software, an interesting experience, really, but it is a good thing it was a relatively short mission. It was s
by GuB-42 3y ago
Oh yes, it happened to me. It was with Palantir software, an interesting experience, really, but it is a good thing it was a relatively short mission.
It was some kind of big data processing, probably called that because the entire database doesn't fit in a single floppy disk. So for some reason, they had to use an overly complex distributed architecture. Absolutely terrible, it took tens of minutes to get a result because was a queued job and the queue was filled with other jobs that were so poorly optimized that they took hours instead of seconds. I don't really blame the authors of these jobs since there is no profiling of any kind so you can't optimize properly, you don't even know how long your job actually took to run because it may be stuck waiting for a resource and it still counts as runtime.
In the end, they were essentially SQL queries that any decent database (ex: Postgres) could support, and our biggest tables where in the order of a few GB, which would be absolutely no trouble for them.
For dev that went beyond your typical JOIN query, I ended up getting the open source software the framework was built on, downloaded the data as csv files, and tried my stuff locally, and when I was done, copy-pasted the result in the editor. I also made a few tools using the REST API to work around some front-end problems.
The most "fun" part was when my manager went to explain me what I had to do. He imported the data into Excel, and in a few minutes of Excel magic, he did everything, just to show me. It took me 3 days to implement it in the framework. In other words, my manager was 100x more productive with Excel than I was with the framework, and I was rather "efficient" among my coworkers. My manager was really good with Excel, but still...
I think I got a glimpse of how it felt to work with punch cards.