4 ms·

Can you give examples of these "hacky domain specific scripts"? I love hearing about niche custom computing.

by cptaj 5y ago

montenegrohugo 5y ago
As an intern at my first job a few years ago I basically did something like that. We had a data hunting that would go out into supermarkets and collect data, but oftentimes with errors. Supervisors would sift through all of them and spend 4-5 hours a day removing/fixing/joining faulty entries. I saw that, wrote two Python scripts to automate it (~2 weeks from start to deployment). All that workload automated immediately, and to my knowledge it's still in use today.
bityard 5y ago
Not the one you're replying to, but I'm my experience this means writing your own tools and automation for the job you do when none exist. (Or are too expensive.) I've seen many shops where the amount of code for dev and QA tools significantly exceeds whatever product they ship.
AussieWog93 5y ago
Yeah, specifically in my business (e-commerce, selling used video games) we have a set of scripts that can create a full eBay listing simply by scanning a game's barcode, another one that generates sales reports and breaks them down with exactly the information that I want, one that interfaces with some hardware to automatically test the save function in Game Boy games, and some more to come in future (we're new!). I saw some similar things in the accounting department at my previous employer, and on the flip side I've met many people who seem to spend an inordinate amount of time doing steps that a computer could do, but worse. I think the important bit is to have tight integration between the person writing the code and the person with the domain-specific knowledge that the code is performing - ideally they should be the same person. As we move away from seeing computer programming as a specialised skill for a class of elite engineers and towards something just generally useful like reading and writing, I can see this yielding huge dividends economically.
- fijiaarone 5y ago
  Being able to code puts you in the top 10% of the IQ bell curve. Being able to see the value in automating something and being able to implement it puts you in the top 1%. That’s not meant as flattery, it’s just pointing out that the efficiency gains of universal computer literacy will not be realized.
  - lifeisstillgood 5y ago
    "They used to say you could not teach women to read" I disagree - we spend years from age two teaching literacy. It's a skill that needs practise, but it's a relatively new invention that has enormous benefits. Software is too. (Mind blowing fact: The walls of jericho were built 2,000 years before writing was invented)
  - diffeomorphism 5y ago
    At a high level maybe, but basic coding is succesfully taught in middle schools. If you needed to be "10%" for that, those classes would fail horribly. > will not be realized. Not quickly, but give it a few decades. In particular, you will probably have to wait for another generation of teachers.
    - AussieWog93 5y ago
      >At a high level maybe, but basic coding is succesfully taught in middle schools. Exactly this. Not everyone can work for Google (or even for a B-grade software firm!), but in many instances all you need to do is string a couple of libraries together in order to get order-of-magnitude productivity gains.
- lifeisstillgood 5y ago
  Yes ! I call it software literacy. welcome to the dark side, we have cookies :-)
firecall 5y ago
Not Python, but Ruby and HTML/CSS/JS examples: I worked at an international company with its own terrible email marketing system. I wrote my own email builder based on their templates that allowed me to visually build emails and then just copy and paste in the code! I also wrote a Ruby script that iterated over a CSV/Excel spreadsheet file supplied on a regular basis to generate HTML page, which I then pasted into their horrible CMS! The previous person in my job used to have to do all those things manually and it took forever! The key win for me was convincing the head of IT to enable WSL on my work supplied Windows 10 Laptop! Having access to Linux made everything possible!
NegativeLatency 5y ago
Wrote a script to crawl databases by url and export their table schema into a format that could be consumed by an early big data processing language. There was a whole team of people doing this job and this one script got through what a whole team of engineers were doing in about 30 seconds
prova_modena 5y ago
Wrote a python script to extract data and produce spreadsheets/reports from the database of an ancient, custom, closed source FoxPro-based ERP software. By being able to create custom reports, I easily reduced the workload during tax time by a factor of 10. This replaced a process that involved manual review of hundreds of sheets of dot-matrix printouts for certain edge cases.
Cyphase 5y ago
I often use this story as an example for people of how programming can save them time and make them more efficient. It's from the introduction to Automate the Boring Stuff with Python, which is also a good resource for them if they decide they do want to pursue such benefits: https://automatetheboringstuff.com/2e/chapter0/#calibre_link-1610 https://automatetheboringstuff.com/2e/chapter0/#calibre_link... --- “You’ve just done in two hours what it takes the three of us two days to do.” My college roommate was working at a retail electronics store in the early 2000s. Occasionally, the store would receive a spreadsheet of thousands of product prices from other stores. A team of three employees would print the spreadsheet onto a thick stack of paper and split it among themselves. For each product price, they would look up their store’s price and note all the products that their competitors sold for less. It usually took a couple of days. “You know, I could write a program to do that if you have the original file for the printouts,” my roommate told them, when he saw them sitting on the floor with papers scattered and stacked all around. After a couple of hours, he had a short program that read a competitor’s price from a file, found the product in the store’s database, and noted whether the competitor was cheaper. He was still new to programming, so he spent most of his time looking up documentation in a programming book. The actual program took only a few seconds to run. My roommate and his co-workers took an extra-long lunch that day. This is the power of computer programming. A computer is like a Swiss Army knife that you can configure for countless tasks. Many people spend hours clicking and typing to perform repetitive tasks, unaware that the machine they’re using could do their job in seconds if they gave it the right instructions.
- AussieWog93 5y ago
  >My roommate and his co-workers took an extra-long lunch that day. If only that was what actually happened when people took initiative in this way...

Can you give examples of these "hacky domain specific scripts"? I love hearing about niche custom computing.

by cptaj 5y ago

Can you give examples of these "hacky domain specific scripts"? I love hearing about niche custom computing.

As an intern at my first job a few years ago I basically did something like that. We had a data hunting that would go out into supermarkets and collect data, but oftentimes with errors. Supervisors would sift through all of them and spend 4-5 hours a day removing/fixing/joining faulty entries. I saw that, wrote two Python scripts to automate it (~2 weeks from start to deployment). All that workload automated immediately, and to my knowledge it's still in use today.

bityard 5y ago

Not the one you're replying to, but I'm my experience this means writing your own tools and automation for the job you do when none exist. (Or are too expensive.) I've seen many shops where the amount of code for dev and QA tools significantly exceeds whatever product they ship.

AussieWog93 5y ago

Yeah, specifically in my business (e-commerce, selling used video games) we have a set of scripts that can create a full eBay listing simply by scanning a game's barcode, another one that generates sales reports and breaks them down with exactly the information that I want, one that interfaces with some hardware to automatically test the save function in Game Boy games, and some more to come in future (we're new!). I saw some similar things in the accounting department at my previous employer, and on the flip side I've met many people who seem to spend an inordinate amount of time doing steps that a computer could do, but worse. I think the important bit is to have tight integration between the person writing the code and the person with the domain-specific knowledge that the code is performing - ideally they should be the same person. As we move away from seeing computer programming as a specialised skill for a class of elite engineers and towards something just generally useful like reading and writing, I can see this yielding huge dividends economically.

fijiaarone 5y ago
Being able to code puts you in the top 10% of the IQ bell curve. Being able to see the value in automating something and being able to implement it puts you in the top 1%. That’s not meant as flattery, it’s just pointing out that the efficiency gains of universal computer literacy will not be realized.
- lifeisstillgood 5y ago
  "They used to say you could not teach women to read" I disagree - we spend years from age two teaching literacy. It's a skill that needs practise, but it's a relatively new invention that has enormous benefits. Software is too. (Mind blowing fact: The walls of jericho were built 2,000 years before writing was invented)
- diffeomorphism 5y ago
  At a high level maybe, but basic coding is succesfully taught in middle schools. If you needed to be "10%" for that, those classes would fail horribly. > will not be realized. Not quickly, but give it a few decades. In particular, you will probably have to wait for another generation of teachers.
  - AussieWog93 5y ago
    >At a high level maybe, but basic coding is succesfully taught in middle schools. Exactly this. Not everyone can work for Google (or even for a B-grade software firm!), but in many instances all you need to do is string a couple of libraries together in order to get order-of-magnitude productivity gains.
lifeisstillgood 5y ago
Yes ! I call it software literacy. welcome to the dark side, we have cookies :-)

firecall 5y ago

Not Python, but Ruby and HTML/CSS/JS examples: I worked at an international company with its own terrible email marketing system. I wrote my own email builder based on their templates that allowed me to visually build emails and then just copy and paste in the code! I also wrote a Ruby script that iterated over a CSV/Excel spreadsheet file supplied on a regular basis to generate HTML page, which I then pasted into their horrible CMS! The previous person in my job used to have to do all those things manually and it took forever! The key win for me was convincing the head of IT to enable WSL on my work supplied Windows 10 Laptop! Having access to Linux made everything possible!

NegativeLatency 5y ago

Wrote a script to crawl databases by url and export their table schema into a format that could be consumed by an early big data processing language. There was a whole team of people doing this job and this one script got through what a whole team of engineers were doing in about 30 seconds

prova_modena 5y ago

Wrote a python script to extract data and produce spreadsheets/reports from the database of an ancient, custom, closed source FoxPro-based ERP software. By being able to create custom reports, I easily reduced the workload during tax time by a factor of 10. This replaced a process that involved manual review of hundreds of sheets of dot-matrix printouts for certain edge cases.

Cyphase 5y ago

I often use this story as an example for people of how programming can save them time and make them more efficient. It's from the introduction to Automate the Boring Stuff with Python, which is also a good resource for them if they decide they do want to pursue such benefits: https://automatetheboringstuff.com/2e/chapter0/#calibre_link-1610 https://automatetheboringstuff.com/2e/chapter0/#calibre_link... --- “You’ve just done in two hours what it takes the three of us two days to do.” My college roommate was working at a retail electronics store in the early 2000s. Occasionally, the store would receive a spreadsheet of thousands of product prices from other stores. A team of three employees would print the spreadsheet onto a thick stack of paper and split it among themselves. For each product price, they would look up their store’s price and note all the products that their competitors sold for less. It usually took a couple of days. “You know, I could write a program to do that if you have the original file for the printouts,” my roommate told them, when he saw them sitting on the floor with papers scattered and stacked all around. After a couple of hours, he had a short program that read a competitor’s price from a file, found the product in the store’s database, and noted whether the competitor was cheaper. He was still new to programming, so he spent most of his time looking up documentation in a programming book. The actual program took only a few seconds to run. My roommate and his co-workers took an extra-long lunch that day. This is the power of computer programming. A computer is like a Swiss Army knife that you can configure for countless tasks. Many people spend hours clicking and typing to perform repetitive tasks, unaware that the machine they’re using could do their job in seconds if they gave it the right instructions.

AussieWog93 5y ago
>My roommate and his co-workers took an extra-long lunch that day. If only that was what actually happened when people took initiative in this way...