10 ms·

The solution is to do more with less. Devs are taking advantage of a wealth of abstraction and being buried under it. But they actually don't need it. You do n

The solution is to do more with less. Devs are taking advantage of a wealth of abstraction and being buried under it. But they actually don't need it. You do not need some 3rd party library to write a command line interface for your python code. You do not need three different projects just to maintain your list of installed packages or virtual environment. And you don't need a package to wrap around loading environment variables. Just learn how to do these things in a simpler way, without the extra deps. Will there be some more boilerplate code? Yes. But it will be your code, and it will be so simple that you will learn to write it very well. In addition: make your code loosely coupled and composeable, and overload existing code with additional functions when feasible, rather than starting a new project from scratch. Press your company to let you contribute back to OSS, as this lets somebody else do more of the maintenance, and has the side effect of reducing the total number of software projects. (It immediately saves you money and time. Kind of stupid that more of us don't do this already.) Simplifying is hard. But when you get good at it, you find you're much more buoyant.

chasd00 4y ago
> Simplifying is hard. But when you get good at it, you find you're much more buoyant. it's not finished when there's nothing left to add, it's finished when there's nothing left to take away.
- Supermancho 4y ago
  Code can always be improved. How you evaluate tradeoffs both varies between individuals and your own views from day to day. Code is never finished, until it's forgotten or feared.
drewcoo 4y ago
> You do not need some 3rd party library to write a command line interface for your python code. > Just learn how to do these things in a simpler way, without the extra deps. That's known as reinventing the wheel. Commonly espoused by people with NIH syndrome. https://www.techopedia.com/definition/3848/not-invented-here-syndrome-nihs https://www.techopedia.com/definition/3848/not-invented-here...
- gyulai 4y ago
  Imagine some point in history where the wheel hasn't yet been invented. But someone figured out that an octagon works kind of reasonably well as something that you might want to put under your chariot, and the whole world of engineering is under a culture where "reinventing the octagon" is some kind of cardinal sin. Where would we be then? Also: Your reasoning is logically a fallacy. From "people making mistake X tend to do Y" it does not follow that "doing Y is a mistake".
  - thombat 4y ago
    But now imagine the world of tomorrow, where despite the convenient availability of a range of wheels well optimized for specific roles you get told "that just adds dependencies: any artisan builds their own from scratch" and so you and your customers get to learn the unique quirks and paper-cuts of each wheel of every chariot you sell thereafter.
    - gyulai 4y ago
      I merely pointed out that the "don't reinvent the wheel" knee-jerk reflex that some people are exhibiting in a broad range of situations is an absurd absolute. Negating something and then stating it as an absurd absolute does not prove nor contradict the original proposition. Sometimes it's more economical to reinvent some wheel than not to, sometimes it's not. Sometimes reinventing a wheel leads to a net-improvement in the state of the art of engineering, sometimes it doesn't. That sort of relativism makes for boring reading, but it also happens to be the truth.
      - TuringTest 4y ago
        That's why the better approach is: learn how to build wheels and how they work, then use off-the-shelf wheels - unless you know that you need some peculiarities that are not found on the market; only then modify those wheels or build your own.
        4y ago
gyulai 4y ago
> You do not need some 3rd party library to write a command line interface for your python code. You do not need three different projects just to maintain your list of installed packages or virtual environment. And you don't need a package to wrap around loading environment variables. Couldn't agree more. I'm going through the experience now of switching from Python to Lua and this experience of trying to do more with less is really eye-opening. I find that lots of things where Python has trained me to expect them to be one-liners are now idioms of half a dozen lines of code. ...but as I use them, I find myself introducing a lot of variation around each half-dozen-loc-idiom. Then the thought occurs to me: Should I make this into a utility function? And the answer is usually no: Because it would have to support all this variation and create a complex interface. Having to understand the interface would introduce as much complexity for the person trying to read/maintain my code as having to tackle half-dozen-loc-idioms where there could be one-liners. Languages that default to simplicity and make-your-own-batteries force that thought process on developers, making them better developers in the process, able to produce more maintainable code. Languages that default to batteries-included try to spare developers the work of having to think about such things, and the end result is way more messy.
- moviewatcher333 4y ago
  My favorite aspect of lua is that it gives you all the tools to implement anything and do it simply. Python, C#, and many languages push you to find the “right” way to do things, and spend time searching through documentation to find the shortcomings of each option and which method may be optimal for your situation. It’s like walking into a hardware store with a specialized tool for everything—you spend your time worrying if you’re really using the right thing. Lua feels like a garage workshop with all the basic tools you need to quickly implement something on your own—master a few and only go hunting for something new when you’ve got no other choice.
  - pferde 4y ago
    That is an odd take on Python particularly, which has been widely lauded as a good language for exploratory coding - easy and quick to put something together and iterate on it. You seem to be claiming the opposite here.
tonto 4y ago
>overload existing code with additional functions when feasible, rather than starting a new project from scratch. why? are you making a jab at small packages? ill just rant on that and say i think small packages are great. they are extremely purpose built and lovely to use. they do become transitive dependencies often leading to the "issue" in this blogpost but im in the yolo upgrade camp
- avidphantasm 4y ago
  The churn required to keep up with frequent updates to many dependencies can work for some projects, but I would argue only for projects with continuous funding. If a project is only funded every couple of years and has many dependencies, odds are, the next time it is funded, there will be an inordinate amount of work to patch/update the code. For such projects, fewer or no dependencies may be better.
  - tonto 4y ago
    this one hits home for me because I do work in academia where funding goes kaput frequently and I do have to agree on some level. Some of the most stable and popular tools in my field have few to zero dependencies and are written in c... bioinformaticians will know what I mean.
oweiler 4y ago
And because every dev has a different idea about how to simplify things, you throw any sort of standardization out of the window.
- gyulai 4y ago
  Standardization and pluralism both have their respective merits. It's a "cathedral vs bazaar" thing or corporate culture vs hacker culture.
- mysterydip 4y ago
  As opposed to the standardization of a random collection of github libraries?
amelius 4y ago
> But it will be your code And your colleagues will have great difficulty figuring out how it works because there is no documentation and no community support.
- fileeditview 4y ago
  Which is true for most libraries pulled from Github also.. but additionally you cannot just ask your colleague.
  - KptMarchewa 4y ago
    But most people don't pull 100s of random libraries, just the few most popular and needed. The strawman where people download tens of libraries just not to write oneliners mostly does not exist, besides javascript land.
  - PeterisP 4y ago
    Having to ask your colleague is an extra cost - for common tasks (like the example of parsing of command line arguments used in the parent comment) you should be using the exact same code as your colleagues unless there is a good reason to make an exception (which there rarely is), so you should be able to read and maintain others' code just as yours because it's written in the same way as you would, making the same style of calls to the exact same functions. And furthermore, the third party libraries (at least for Python) are so standard that a new hire would have used the exact same library if they ever had to do the same task - for most things there's going to be just a few reasonable ways (and often just a single idiomatic one) to do that thing in the language, which sometimes used a built-in library and sometimes uses a popular third-party library. If your company has a dozen tools which need the same functionality, "rolling your own" for each codebase (or worse, copy/pasting it without being able to update them from a single place) is obviously not a good solution - the choice is between making and using an internally maintained library, or using an open source one.
- avidphantasm 4y ago
  Then you flog mercilessly people in your organization who write code like this. When I was coming up, the greybeards were bastards, but they taught me the right way to go about things.
deleted 4y ago
[deleted]
annowiki 4y ago
I come from a Python background so libraries were always just a pip install away. In the past year, however, I've been working on a large C++ codebase (couple million lines) and the result has been a considerably greater amount of "roll your own." This has filtered back into my Python. If it's not in the standard library, I don't install it unless I really need it. My boss always tells me that you can probably write it faster than a library, something I never used to believe. Until I tried it. I needed to check which of two version strings, of XX.XX.XX.XXXX format, were bigger. I tried the most recommended version number library, then I tried writing my own solution, the simplest solution I could think of: def version_compare(v1: str, v2: str) -> int: """ Compares two versions. Parameters ---------- v1: str v2: str Returns ------- int 1 if v1 is greater, -1 if v2 is greater, 0 if equal. """ for el1, el2 in zip(v1.split('.'), v2.split('.')): if el1 != el2: return 1 if int(el1) > int(el2) else -1 return 0 My code was faster by like 20x. Libraries are bloated and you probably only need a small subset of the functionality, so write your own code has become my mantra.
- thyrsus 4y ago
  TIL python type hints: https://docs.python.org/3/library/typing.html https://docs.python.org/3/library/typing.html Thanks.
- pc86 4y ago
  Another benefit, that IMO is as big if not bigger, is that you can easily make changes to this code whereas changes on the library side may be hidden behind configuration, impossible at all, or god forbid even require a wrapper anyway. In the above example, if you wanted this instead to return the version number string that is larger, that's an easy change and one that's very obvious from the PR what it is doing.
- bruce343434 4y ago
  Little bit off topic perhaps but why encode the types a second time in the comment when you already have the annotations?
fragmede 4y ago
Back in my day, we coded in assembly and looked up opcodes in paper books. And we liked it! At least we weren't still on punch cards. Eg, I use the python docopt library because it makes life easier and more straight-forwards once you understand it. Which, to be fair, not everyone wants to put in the time to learn. But I like it more than having to code up an argument parser from scratch using string matching, and then having to do that every single time... I don't need it, but I also don't anything more than nasm, anything more than that is luxury! Kids these days...
- commandlinefan 4y ago
  > we coded in assembly and looked up opcodes in paper books Don't threaten me with a good time.
- P5fRxh5kUvp2th 4y ago
  Most people use abstractions so they don't have to. There's a difference between someone using an ORM who knows SQL and someone using an ORM so they don't have to learn. The fact is, most people use these abstractions so they don't have to understand it. --- Back in my day ... it was accepted that you needed to understand atleast 1 level of abstraction below where you're working at. Nowadays, that attitude is only used by older developers.
  - ziml77 4y ago
    > The fact is, most people use these abstractions so they don't have to understand it. How do people just accept that there's magic going on? That would drive me crazy. I don't expect people to know all the details, but there should be some level of understanding at least one layer down.
collyw 4y ago
I try to convince them to keep stuff simple at my work, but we still go creating a shitshow of epic proportions. If our app was done the traditional way it could probably run on a Rasperry Pi. Instead they are pushing for microservicces, one database per service has been mentioned. A SPA app for the frontend. All for what amounts to a standard crud app that sends some jobs to the cloud.
lake_vincent 4y ago
Just a quick shout-out to Python Poetry (https://python-poetry.org/ https://python-poetry.org/) which makes package, version, and virtual environment management extremely simple! I love it so much that I can't imagine having to work on a python project without Poetry now, lol.

chasd00 4y ago

> Simplifying is hard. But when you get good at it, you find you're much more buoyant. it's not finished when there's nothing left to add, it's finished when there's nothing left to take away.

Supermancho 4y ago
Code can always be improved. How you evaluate tradeoffs both varies between individuals and your own views from day to day. Code is never finished, until it's forgotten or feared.

drewcoo 4y ago

> You do not need some 3rd party library to write a command line interface for your python code. > Just learn how to do these things in a simpler way, without the extra deps. That's known as reinventing the wheel. Commonly espoused by people with NIH syndrome. https://www.techopedia.com/definition/3848/not-invented-here-syndrome-nihs https://www.techopedia.com/definition/3848/not-invented-here...

gyulai 4y ago
Imagine some point in history where the wheel hasn't yet been invented. But someone figured out that an octagon works kind of reasonably well as something that you might want to put under your chariot, and the whole world of engineering is under a culture where "reinventing the octagon" is some kind of cardinal sin. Where would we be then? Also: Your reasoning is logically a fallacy. From "people making mistake X tend to do Y" it does not follow that "doing Y is a mistake".
- thombat 4y ago
  But now imagine the world of tomorrow, where despite the convenient availability of a range of wheels well optimized for specific roles you get told "that just adds dependencies: any artisan builds their own from scratch" and so you and your customers get to learn the unique quirks and paper-cuts of each wheel of every chariot you sell thereafter.
  - gyulai 4y ago
    I merely pointed out that the "don't reinvent the wheel" knee-jerk reflex that some people are exhibiting in a broad range of situations is an absurd absolute. Negating something and then stating it as an absurd absolute does not prove nor contradict the original proposition. Sometimes it's more economical to reinvent some wheel than not to, sometimes it's not. Sometimes reinventing a wheel leads to a net-improvement in the state of the art of engineering, sometimes it doesn't. That sort of relativism makes for boring reading, but it also happens to be the truth.
    - TuringTest 4y ago
      That's why the better approach is: learn how to build wheels and how they work, then use off-the-shelf wheels - unless you know that you need some peculiarities that are not found on the market; only then modify those wheels or build your own.
      - 4y ago

gyulai 4y ago

> You do not need some 3rd party library to write a command line interface for your python code. You do not need three different projects just to maintain your list of installed packages or virtual environment. And you don't need a package to wrap around loading environment variables. Couldn't agree more. I'm going through the experience now of switching from Python to Lua and this experience of trying to do more with less is really eye-opening. I find that lots of things where Python has trained me to expect them to be one-liners are now idioms of half a dozen lines of code. ...but as I use them, I find myself introducing a lot of variation around each half-dozen-loc-idiom. Then the thought occurs to me: Should I make this into a utility function? And the answer is usually no: Because it would have to support all this variation and create a complex interface. Having to understand the interface would introduce as much complexity for the person trying to read/maintain my code as having to tackle half-dozen-loc-idioms where there could be one-liners. Languages that default to simplicity and make-your-own-batteries force that thought process on developers, making them better developers in the process, able to produce more maintainable code. Languages that default to batteries-included try to spare developers the work of having to think about such things, and the end result is way more messy.

moviewatcher333 4y ago
My favorite aspect of lua is that it gives you all the tools to implement anything and do it simply. Python, C#, and many languages push you to find the “right” way to do things, and spend time searching through documentation to find the shortcomings of each option and which method may be optimal for your situation. It’s like walking into a hardware store with a specialized tool for everything—you spend your time worrying if you’re really using the right thing. Lua feels like a garage workshop with all the basic tools you need to quickly implement something on your own—master a few and only go hunting for something new when you’ve got no other choice.
- pferde 4y ago
  That is an odd take on Python particularly, which has been widely lauded as a good language for exploratory coding - easy and quick to put something together and iterate on it. You seem to be claiming the opposite here.

tonto 4y ago

>overload existing code with additional functions when feasible, rather than starting a new project from scratch. why? are you making a jab at small packages? ill just rant on that and say i think small packages are great. they are extremely purpose built and lovely to use. they do become transitive dependencies often leading to the "issue" in this blogpost but im in the yolo upgrade camp

avidphantasm 4y ago
The churn required to keep up with frequent updates to many dependencies can work for some projects, but I would argue only for projects with continuous funding. If a project is only funded every couple of years and has many dependencies, odds are, the next time it is funded, there will be an inordinate amount of work to patch/update the code. For such projects, fewer or no dependencies may be better.
- tonto 4y ago
  this one hits home for me because I do work in academia where funding goes kaput frequently and I do have to agree on some level. Some of the most stable and popular tools in my field have few to zero dependencies and are written in c... bioinformaticians will know what I mean.

oweiler 4y ago

And because every dev has a different idea about how to simplify things, you throw any sort of standardization out of the window.

gyulai 4y ago
Standardization and pluralism both have their respective merits. It's a "cathedral vs bazaar" thing or corporate culture vs hacker culture.
mysterydip 4y ago
As opposed to the standardization of a random collection of github libraries?

amelius 4y ago

> But it will be your code And your colleagues will have great difficulty figuring out how it works because there is no documentation and no community support.

fileeditview 4y ago
Which is true for most libraries pulled from Github also.. but additionally you cannot just ask your colleague.
- KptMarchewa 4y ago
  But most people don't pull 100s of random libraries, just the few most popular and needed. The strawman where people download tens of libraries just not to write oneliners mostly does not exist, besides javascript land.
- PeterisP 4y ago
  Having to ask your colleague is an extra cost - for common tasks (like the example of parsing of command line arguments used in the parent comment) you should be using the exact same code as your colleagues unless there is a good reason to make an exception (which there rarely is), so you should be able to read and maintain others' code just as yours because it's written in the same way as you would, making the same style of calls to the exact same functions. And furthermore, the third party libraries (at least for Python) are so standard that a new hire would have used the exact same library if they ever had to do the same task - for most things there's going to be just a few reasonable ways (and often just a single idiomatic one) to do that thing in the language, which sometimes used a built-in library and sometimes uses a popular third-party library. If your company has a dozen tools which need the same functionality, "rolling your own" for each codebase (or worse, copy/pasting it without being able to update them from a single place) is obviously not a good solution - the choice is between making and using an internally maintained library, or using an open source one.
avidphantasm 4y ago
Then you flog mercilessly people in your organization who write code like this. When I was coming up, the greybeards were bastards, but they taught me the right way to go about things.

deleted 4y ago

[deleted]

annowiki 4y ago

I come from a Python background so libraries were always just a pip install away. In the past year, however, I've been working on a large C++ codebase (couple million lines) and the result has been a considerably greater amount of "roll your own." This has filtered back into my Python. If it's not in the standard library, I don't install it unless I really need it. My boss always tells me that you can probably write it faster than a library, something I never used to believe. Until I tried it. I needed to check which of two version strings, of XX.XX.XX.XXXX format, were bigger. I tried the most recommended version number library, then I tried writing my own solution, the simplest solution I could think of: def version_compare(v1: str, v2: str) -> int: """ Compares two versions. Parameters ---------- v1: str v2: str Returns ------- int 1 if v1 is greater, -1 if v2 is greater, 0 if equal. """ for el1, el2 in zip(v1.split('.'), v2.split('.')): if el1 != el2: return 1 if int(el1) > int(el2) else -1 return 0 My code was faster by like 20x. Libraries are bloated and you probably only need a small subset of the functionality, so write your own code has become my mantra.

thyrsus 4y ago
TIL python type hints: https://docs.python.org/3/library/typing.html https://docs.python.org/3/library/typing.html Thanks.
pc86 4y ago
Another benefit, that IMO is as big if not bigger, is that you can easily make changes to this code whereas changes on the library side may be hidden behind configuration, impossible at all, or god forbid even require a wrapper anyway. In the above example, if you wanted this instead to return the version number string that is larger, that's an easy change and one that's very obvious from the PR what it is doing.
bruce343434 4y ago
Little bit off topic perhaps but why encode the types a second time in the comment when you already have the annotations?

fragmede 4y ago

Back in my day, we coded in assembly and looked up opcodes in paper books. And we liked it! At least we weren't still on punch cards. Eg, I use the python docopt library because it makes life easier and more straight-forwards once you understand it. Which, to be fair, not everyone wants to put in the time to learn. But I like it more than having to code up an argument parser from scratch using string matching, and then having to do that every single time... I don't need it, but I also don't anything more than nasm, anything more than that is luxury! Kids these days...

commandlinefan 4y ago
> we coded in assembly and looked up opcodes in paper books Don't threaten me with a good time.
P5fRxh5kUvp2th 4y ago
Most people use abstractions so they don't have to. There's a difference between someone using an ORM who knows SQL and someone using an ORM so they don't have to learn. The fact is, most people use these abstractions so they don't have to understand it. --- Back in my day ... it was accepted that you needed to understand atleast 1 level of abstraction below where you're working at. Nowadays, that attitude is only used by older developers.
- ziml77 4y ago
  > The fact is, most people use these abstractions so they don't have to understand it. How do people just accept that there's magic going on? That would drive me crazy. I don't expect people to know all the details, but there should be some level of understanding at least one layer down.

collyw 4y ago

I try to convince them to keep stuff simple at my work, but we still go creating a shitshow of epic proportions. If our app was done the traditional way it could probably run on a Rasperry Pi. Instead they are pushing for microservicces, one database per service has been mentioned. A SPA app for the frontend. All for what amounts to a standard crud app that sends some jobs to the cloud.

lake_vincent 4y ago

Just a quick shout-out to Python Poetry (https://python-poetry.org/ https://python-poetry.org/) which makes package, version, and virtual environment management extremely simple! I love it so much that I can't imagine having to work on a python project without Poetry now, lol.