22 ms·

The art of over-engineering your side projects

869 points by elsyms 9y ago

forthelove 9y ago
There's probably another reason hidden in the first sentence, which is that there's a transition from building the product (engineering) to shipping the product (marketing, customer development, sales) that requires different skill sets and experiences. Was just discussing this with a software engineer friend of mine who realizes that he "has no clue how to market my product the right ways."
klokoman 9y ago
But that's all the fun
- wyager 9y ago
  Agreed. The best part of side projects is that I can take as long as I want to build it to my satisfaction, rather than to the minimal acceptable state for business purposes.
ropman76 9y ago
If your goal is to make money on something, then yes I agree with the author's points. However the whole goal of side projects is to have fun with new languages, frameworks etc.
auserperson 9y ago
I think the majority of engineers only do side projects to learn things they couldn't at work or explore architectures they wouldn't be able to in a work environment.
mixedmath 9y ago
When I first read the title, I had thought that this would be a humorous example of a delightfully over-engineered side project. It turns out that this is instead a list of 5 common mistakes of over-engineering with some commentary.
caleblloyd 9y ago
Lots of times I use side projects as a way to explore new stacks, deployment approaches, etc. The goal of the project is not to deliver anything, it's to learn so that when I go to do my next "real" project at work, I know what works well and what doesn't.
- mrisoli 9y ago
  Me too, I think there are two types of side projects, this article focuses on those which I believe people will try to generate a business out of, or at least some form of passive income. But there are also some side projects where you just want to get your hands dirty with some different tech, if you do deliver something then you can showcase it on your portfolio. I know when the app store gold rush first started a lot of people did this as experience to get iOs dev jobs.
  - dcminter 9y ago
    Exactly - I enjoy side projects precisely because they're not bound by pragmatic concerns. I can delve as deep as I like into getting the CI pipeline working just right, making sure the servers and cacheing are set up ok and all those other things that I usually have to leave to colleagues. More often than not the skills I learn doing this kind of thing end up being very useful to me in the day job. I'd like it if the side project was successful enough to justify all of that yak shaving, but I don't really expect it to be and I'm not that fussed when it isn't. But if your primary goal is that your side project should become your day job then the article is pretty good advice. Leave your yak unshaven :)
    - TeMPOraL 9y ago
      > they're not bound by pragmatic concerns. 99% of the time, if you want to go pragmatic, you should buy a ready-made solution (and then possibly adapt it to your use case). It's efficient (especially if you need the problem solved to make money) - but it's no fun.
  - mgolawala 9y ago
    I agree, but there are good reasons why it is often good to combine both types. Considering this is only a side project, you have a high likelihood of abandoning it out of boredom, lack of traction or simply burn out. However, if your side project is "interesting" to you because you are experimenting with various tech, you come back to it because it is fun and it will keep you motivated. It is totally cool to build something to learn and throwaway, but why not build something to learn that might also generate some income in the future?
jlebrech 9y ago
this should apply to the start of any project.
xwvvvvwx 9y ago
For me it's very important that I don't discuss projects with friends / family / colleagues etc. until I really have something to show. I have always found it very easy to talk all the energy out of a good idea before I've even got any real work done.
- hectorlorenzo 9y ago
  That's a thing. Expressing in public your desire to achieve something makes you less likely to deliver. http://www.psych.nyu.edu/gollwitzer/09_Gollwitzer_Sheeran_Seifert_Michalski_When_Intentions_.pdf http://www.psych.nyu.edu/gollwitzer/09_Gollwitzer_Sheeran_Se...
  - owebmaster 9y ago
    I don't agree with this reasoning. Expressing in public your projects is a compromise and it can help you keep developing it when people ask if it is ready. If you are going to abandon your projects, not telling people will not help you succeed, it will only spare you the shame and effort explaining why you droped another project.
    - hectorlorenzo 9y ago
      A little bit of pop psychology from my end... I don't think people start projects with the conscious intention of building identity, but with a genuine intention of delivering. I do think, though, that there's a small (or big!) part of your unconscious brain that binds these efforts to an overarching persona-building (both inwards and outwards) project. Although unconscious, it contributes to this net energy. I guess that this communicating your intentions satisfies this portion of your brain, and reduces the amount of energy involved in the project.
      - owebmaster 9y ago
        Sure, this makes sense. I just don't think (using only myself as a reference, which is totally biased) that this effect has a bigger impact than the social pressure the public compromise creates.
myhf 9y ago
> 4. Use frameworks and customise them – only refactor/build your own when absolutely neccessary. > 5. Build your project first – then worry about continuous delivery. These contradict each other. Building your own deployment system is just as much of a distraction as building your own CSS framework.
- elsyms 9y ago
  Why do you need a deployment system if you have nothing to deploy? Build the project locally, then use continuous delivery when you have something to deliver.
  - myhf 9y ago
    "Build the project locally" implies some kind of pipeline system, even if it's not "deployment" or "delivery". Triggering that pipeline from a commit trigger instead of manually just means copying the build command into a config file. It pays for itself in the first hour.
    - vorpalhex 9y ago
      `git push deploy` You can get quite far without any kind of "pipeline system" nor does building imply a pipeline.
- MichaelBurge 9y ago
  Your "deployment system" could be a 20-line systemd service and a git checkout. You deploy with a "git pull" and "sudo service restart".
  - staticelf 9y ago
    You could even skip the systemd and just rely that the server will never be restarted in the very beginning.
- Allower 9y ago
  Who said anything about building a deployment system?? thats not the project..
quadcore 9y ago
“I know: I’ll create a triple zoned redundant architecture with pub/sub database replication, a 32 node Kubernetes cluster and private networking across all regions – that way, I can handle anything!” The good news is, once you've done that, you can get a pretty decent job. The majority of software engineers seem to be under the illusion that potential customers care about the stack they are running. This is not entirely true. The technology do surface in terms of user experience. It does in the details: app start, ui responsiveness, etc. I'm writing a game using go right now, and the end user can tell it's different because the user's cpu seems more powerful. All in all, I think those are good mistakes, not bad ones. I wish more engineers could spot over-engineering, and they don't until they did the mistake a couple times. Side projects are, I believe, how one really learn to write programs. I think the right advice is: do whatever you want, do all the mistakes. How can you write a successful side project anyway if you never learn how to write a program? Also, the worse type of programmers are the ones telling you about over-engineering because they read about it in a blog. Code more, read less, I guess should be a good motto.
- elsyms 9y ago
  "A customer will not know or care if you are using Ruby, Go, PHP or any other language as long as what you have written is performant and is fit for purpose (which all modern languages are)." Where games are concerned I'd tend to agree (though Minecraft was built in Java and was TERRIBLE for the first couple of years, now look at it).
  - majewsky 9y ago
    > Minecraft was built in Java and was TERRIBLE for the first couple of years, now look at it Well, it still takes absurd amounts of memory for what it's doing, which I guess is on par for Java. (On my notebook with 4GB RAM, modpacks fall into one of three categories: "works well", "have to shut off as many OS services as possible before starting" and "forget it".)
    - merb 9y ago
      well only because java has no value types/struct like data structure. the reason why this makes memory so heavy is because even a simple int list, needs boxing. So you end up with a big chunk of garbage which you don't need. On the server, this is not a problem because most of the time the list is a young gen object and die really fast, so G1GC solves most problems. The problem of course is really big, because arrays are a pain to work with and not many people do that. However as soon as java 10 hits, we might get value types and maybe this changes the game (but we will see, since it would've taken way to long for that thing). Also there are more and more types inside the standard library who would be great value types, i.e. LocalDate/LocalDateTime, but at the moment they take way more memory than needed, especially since they only contain 2 immutable shorts and 1 immutable int.. in the perfect case it would be something like 8 byte, however it takes way way way way more. (basically a simple class at least takes 16 bytes, which is already double the field size, so you end up with 24 bytes [probably more due to various other stuff]) btw. for a game like minecraft you probably have a lot of types that follow the same stuff like LocalDate, small class with immutable int's/short's for position, etc. these have the same problem and will fill the memory more quickly than needed
dtzur 9y ago
"Continuously Delivering Nothing" is a mission statement for several companies I know.
- bpicolo 9y ago
  Tar it up, cat it to devnull
  - sleepychu 9y ago
    You shouldn't use cat for a single tar. Less will work just as well.
  - cryptos 9y ago
    It is important to compress it with bzip2 or your even better custom algorithm before finally archiving it in /dev/null
- le-mark 9y ago
  I'm working on v2 of an app that's been under development for 5 years with no production users. They're still on v1. Pitiful.
GhostVII 9y ago
I always overengineer my side projects, because it's way more interesting that way. I do side projects for fun and to learn something, both of which happen more easily when the project is overengineered.
- majewsky 9y ago
  I tend to underengineer my side projects because the stuff that I touch at work is frequently much too overengineered to begin with (cough OpenStack cough).
- kentt 9y ago
  Agreed. Part of the fun of my side project is that it's well built. At work I have to deal with poor quality code practices from artificially tight timelines, contractors, etc. From one perspective it's very wasteful to have higher quality code for an app that generates $150/mo. vs multimillion dollar projects I manage at my day job, but I enjoy that $150, the code and customers more than what I get from my day job. I wouldn't if it was an MVP.
mikegerwitz 9y ago
The article has good points. But the term "side project" implies working on something that isn't part of your job. I think looking at your project from that perspective can also be a step in the wrong direction---if you enjoy what you do, work is just one such application of your knowledge and ability. Projects you do outside of work are another, and may even be more important to you than work. A number of commenters point out that many side-projects to them are to explore new concepts/libraries, or as a means of learning. If it's for the sake of learning for your job, then yes, it'd be a side project. But if you're looking to take a project to completion (the author talks about products; my projects aren't), I find that using it as a learning environment extremely detrimental. Yes, you do learn things over the course of development---through research, struggle, and growth. But if too much of your time is spent on things you don't have a good foundation on, you may burn out too early, wind up frustrated, and wind up with a mess that leads to refactoring. Use a foundation that you have experience with and know well, and learn the parts that are necessary. (I do distinguish, though, a research project with a project for playful learning. The former is much more formal and disciplined. But you can still burn out early.)
twelvedogs 9y ago
heh, most of this stuff is extra wastes of time past the very first hurdle that you should be working on: make something that works and is useful to people. sure it would be great to have scaling to 10,000 users already sorted by the time you get there but odds are your project will never get there, when you get to 5k users you need to start worrying about that if you're just doing it to learn or for fun then obviously this article or my comment don't apply
wyager 9y ago
> A customer will not know or care if you are using Ruby, Go, PHP or any other language Users will absolutely notice if your product crashes or breaks, which is more likely with some languages than others. The single most likely reason for me to stop using a nominally useful product is that it doesn't work reliably. I hate the fact that so much technology today is broken, partly because people have this cavalier attitude towards choosing technologies that help to keep things working.
pgm8705 9y ago
#3 hits home. I've been "working" on my side project for 3 years now with almost no progress because I'm constantly intrigued by new frameworks and BaaS. I spend all my time tinkering with them until I get bored with it all together. If I had just built the thing in Rails from the start I'd probably have a solid app under my belt by now.
- sleepychu 9y ago
  Time for a new branch with rails then!
- 0xfeba 9y ago
  Yeah I've been working on a game for 5+ years that went thru C++/DirectX, then C# and Managed DirectX, and now Java and LibGDX. Thinking about using Unity now...
  - Kiro 9y ago
    I'm in the same position. I've built my own game engine from scratch and while it was a really good learning exercise I recently started looking at Unity and wow... So many things I've always wanted in my engine that I will get for free. I don't regret a thing though.
yigitozkavci 9y ago
I think side projects and hobby projects need to be thought of separately. In the context of hobby projects, over-engineering can even serve purpose of learning new things you otherwise could not. For side projects, or startup candidates, however, you just ship it
viach 9y ago
If only it was applied to side projects exclusively...
CJefferson 9y ago
I recently had this with a html5 project. Thought it would be nice to use es6 and transpile, then disappeared down a hole of gulp and webpack. Eventually backed up and used Make to run Babel and then cat the files together.
Mojah 9y ago
Related to this, I've been following a practice I call "Release Notes Driven Development", which is perfect for side projects. Don't worry too much about technical debt (although, keep it under control), but focus on the little time you have and how to make that as efficient as possible. I wrote about it here; https://ma.ttias.be/release-notes-driven-development-rndd/ https://ma.ttias.be/release-notes-driven-development-rndd/
- dispo001 9y ago
  Sounds good. Early on, a quick list of bragables should be able to tell you how awesome the idea is. Have new bragables for the next version. If its not awesome at some stage (and the bugs are fixed) you are probably done? It reminds me of many release notes describing a ton of work without any woah.
dudul 9y ago
Oddly enough, I would apply some of these practices to real projects as well.
hamvocke 9y ago
Good points in there. I have a little trouble with "mistake #5" though. I found that worrying about deployment automation early on is immensely valuable, especially for side projects. I don't always find time or interest to keep working on side projects after a regular day of work. Quite often there are weeks or even months where I abandon a side project only to come back at a later point when motivation is back or things have settled down and leave more time for my side projects. If I didn't take the time to automate testing and deployments (which makes a lot of what continuous delivery is about) I find myself struggling with getting things up and running and become frustrated before I even get started. Then there's only one way to make sure that my test and deployment automation keeps working: running it regularly, e.g. in a pipeline. So, yes, don't over optimise your deployment pipeline early on. But having one in place early can pay off really soon -- and maybe even keep your side project alive.
- monksy 9y ago
  This. I'm a big fan of doing the hard work early on. When you push that step off it makes dev later slow and frustrating.
- bitwisebob 9y ago
  Agreed. Having a deployment pipeline in place dramatically reduces the friction of returning to an old project. I have a back-burner project that touch very infrequently, but I can jump in, make a quick change, and have it deployed to "production" in just a few seconds.
- juanuys 9y ago
  I definitely automate locally by way of Makefiles, running tests then using the GCloud/aws CLI to deploy from local. Things like TravisCI, webhooks, etc come much later if at all.
shortoncash 9y ago
I can tell this article was written by a very wise man who wasted time on lots of projects or saw a lot of other people waste time on projects. He's described behaviors I have dabbled in many, many times. In fact, I have a side project currently where I just needed to fabricate a tiny fixture for a physical device. I caught myself over-engineering the prototype, went to sleep, woke up, saw this article and came to my senses. Consider this article bookmarked!
redm 9y ago
Man, I think this article is dead on, except it doesn't identify the original reason, for me at least. In a normal environment, there are constraints on time, money, goals, resources, etc. Suddenly, when there are no constraints, a side project, you try to do everything perfectly, which leads to the issues in this article.
aetherspawn 9y ago
Doing #5 is chronic because I'm too lazy to handle my own deployments and it's easier to copy paste a generic firebase deploy. Then you can send the link to anyone and it always just works.
ratsimihah 9y ago
"Continuously delivering nothing" – exactly what I'm doing, building a product no one will use.
jondubois 9y ago
Side projects are meant to be challenging because they are a learning opportunity so it's possible that they end up over-engineered sometimes. You can always refactor the parts that are over-engineered. There's a very fine line between highly-engineered and over-engineered. One of my favourite projects is Kubernetes; some might consider it over-engineered but it's extremely powerful. "Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away"
- zimpenfish 9y ago
  > "[...] when there is nothing left to take away" I've always taken that quote to apply only when you have been taking things away; not when you've ended up with a colossal complex monstrosity where removal of any single strand would cause total collapse.
esseti 9y ago
For the record (it's not mentioned but a lot of time it happens): security (e.g. password storage) should not be something left for later. Do it or use a framework that does it for you.
breatheoften 9y ago
This makes a lot of sense -- and makes me feel like I'm guilty of over engineering a non-side project I started working on at work ... I'm always looking to make the code as easy to change as possible -- the idea being that change is a precondition for any form of improvement ... Reading this article makes me wonder if I've turned that pursuit into an anti-goal
darod 9y ago
Setting up a CI on AWS using git, CodePipeline, Code Build, and elastic beanstalk takes no more than a few hours. It's worth the few hours. Otherwise you're manually pushing builds out which wastes more time in the long. In the beginning your CI should be nothing more than ==> I check in code, it deploys to environment. Later on you can worry about triggering tests and preventing builds for deploying if they fail. Also if you use elastic beanstalk, you can scale as necessarily with a few configuration changes.
hashkb 9y ago
I have a personal Jenkins that's been running for years. Adding new jobs isn't yak shaving; it's a two minute task that gives me CD. Same with my AWS "framework". And my CSS "framework" and my React component superclass and lots of other stuff I carry from one side project to another. Some would (and have) accused me of over engineering; but I can discover my project is pointless very quickly.
datpuz 9y ago
This article ignores the fact that many of us work on side projects for fun. PHP? Nah.
FatAmericanDev 9y ago
Every enterprise client I have worked with has asked about our tech stack. I will agree that your groupon clone users do not care about your stack, but respond to an enterprise SaaS RFP and stack, security, DR will all be discussed.
polote 9y ago
This post should be renamed : The art of over-engineering "those software engineers often have the tendency to over-engineer their side projects" When your first sentence is false, so do is the rest of the article !
- tw1010 9y ago
  "When your first sentence is false, so do is the rest of the article !" Not necessarily.
scadge 9y ago
I believed over-engineering IS the point of personal side-projects.
- zimpenfish 9y ago
  I'm in the opposite camp - personal side-projects are where you can safely implement the MVP and/or ruthlessly chop things/ideas that never* get used without people harassing you about bells and whistles.
khana 9y ago
The takeaway is ARCHITECTURE. thank you
SonicSoul 9y ago
guilty of all of the above. my thinking is always "i'll kill 2 birds with one stone, side project and learning the latest framework!" and yah.. after struggling with the framework for 2 weeks the project loses steam and then just thinking about a stupid challenge i have to solve BEFORE getting back to my project is crippling. it does have a nice side benefit however of actually learning a new framework :)
awjr 9y ago
A lot of time this can be applied to any new project not just side projects.
rthomas6 9y ago
Triggered. Leave me alone and let me make my Hugo blog migration with my custom theme using bulma.io and webpack. Never mind that the WordPress setup I have now could handle 100x the traffic I get.
wreath 9y ago
I think deferring the CD part until the end can be really painful when you are "ready". I think it's a worth upfront investment even if you have few users, will make your life much easier and less stressful.
zhuzhu 9y ago
the best real joke of month!
masukomi 9y ago
> "Develop your project. Then, put it on the bare minimum architecture that it can run on (be that a 512MB instance from DigitalOcean or a medium instance on AWS). As you get more users, monitor, and modify the infrastructure appropriately to account for load and redundancy." That's a pretty ignorant and potentially short sighted statement. Look at what happened to Instagram. They almost lost it all because they hadn't planned for the possibility of handling scale quickly Sometimes you simply don't have the luxury of monitoring and modifying to account for load and redundancy. Sometimes you're monitoring says "hey traffic" shortly followed by "OMG YOU'RE SCREWED" because you can't possibly think up and implement an appropriate scalable architecture, and modify the codebase to handle it and and and while your servers are melting and everyone is yelling at you. Yes, this is very unlikely to happen, but when it does it can cost you your business if you're not prepared. Obviously apps with no networking and no viral component are far less likely to have this problem. Got a macApp that creates icon sets for XCode? Yeah, you can ignore this scaling issue. Writing the next Instagram? You're a fool if you don't have plans in place, in advance of launch, for . handling the onslaught of people that will make your business successful.
- tarr11 9y ago
  > Look at what happened to Instagram. They almost lost it all because they hadn't planned for the possibility of handling scale quickly I had never heard this version before. The story I heard was that they pivoted from burbn, which they felt was to cluttered and unfocused. [1] If they followed your advice, they would have built a scaled up version of burbn, which nobody would have used. [1] http://www.makeuseof.com/tag/1-billion-in-2-years-the-story-of-instagram-as-told-by-its-founders/ http://www.makeuseof.com/tag/1-billion-in-2-years-the-story-...
- lucideer 9y ago
  It's interesting that you choose Instagram - a highly successful company/product - as an example to warn people against. It may be harder to come up with examples of people who failed, for the obvious reason - they failed and are therefore not well known, but I really don't find the statement that Instagram "nearly lost it all" very compelling. Did they? Really? If they had had even more problems scaling, would they really have lost users? Look at Twitter, they took years to resolve issues. I actually think the importance of stability and uptime are overblown in the early stages. If I think about the times downtime has significantly impacted my view of a company or product, it's when large established services I'd come to rely on professionally went down. You need to establish a reputation for reliability before most people expect it of you. That's not to say it's not important, but you need to look at context. Noone relies on Instagram minute by minute for their professional work, so the priorities are different. UX, marketing, network effects, etc. are going to trump stability.
thecodemonkey 9y ago
We just did a talk on this exact thing at Laracon last month. And raised many of the same points. We however also talked a lot about the business side of things. For those who are interested: Video: https://streamacon.com/video/laracon-us-2017/day-2-mathias-and-michele-hansen https://streamacon.com/video/laracon-us-2017/day-2-mathias-a... Slides: https://speakerdeck.com/minicodemonkey/launching-and-scaling-a-side-project https://speakerdeck.com/minicodemonkey/launching-and-scaling...
bojanvidanovic 9y ago
I'm pretty sure if I wasn't over-engineering my side projects I wouldn't be technically competitive as I am now. So over-engineering is a very good strategy for learning and mastering any field. You can't really lose anything, either your project will succeed or you will acquire/strengthen your skills.
1hkogreendragon 9y ago
Where would Hacker News be without over-engineered garbage software?
ThomPete 9y ago
When I do side projects it's normally because I want to explore something which is too risky for someone to do (or it would most likely have been done). In my experience that normally requires some sort of overengineering because your problem and solution isn't common. A few of them turned into great side businesses but most of the times it help me explore technologies and projects while still solving real problems.
trollopTheJope 9y ago
I didn't think they'd be a word I dislike more than "architecting". Hats off to the author for using "over-architecting" in his insightful blogpost.
seanwilson 9y ago
A major roadblock I find is that when you want to release your side project and start charging for it, I feel it becomes much harder to make changes after. This means you get caught in a loop of trying to polish it more and more before biting the bullet and releasing it. If you change the interface a lot or how features work, you'll annoy users that don't like the change and you'll have to be extra careful not to introduce any bugs. If users can save data, the new version will have to work with the old data format. If you want to move features from a free version to the paid version you'll annoy users. So to me, releasing a MVP you charge for that might change a lot later might lead to more headaches than developing the idea further. Are there ways around this? Side projects you release for free are one thing but when you want to charge for them the amount of polish and attention to detail you have to pay to marketing jumps by an order of magnitude. People will tolerate bugs and rough edges for free projects but not for paid ones. I agree with the advice about not wasting time with CI, switching frameworks etc. though.
thinbeige 9y ago
IDK, I over-engineer my side projects every single time. Like tweaking for days something which is so clearly premature optimization. But sometimes I need that knowledge again for the next side project. And this over-engineering is often a way to simplify a reference architecture and be much faster at the end. So for me over-engineering equals learning but is also a big part of the motivation.
- wvenable 9y ago
  I have similar experiences. I was working with an embedded platform that uses C exclusively and I decided I wanted to use C++. I ran into mountains of hurdles, had to stare at disassembled code, tweak stuff, build a class library, etc. I never actually finished the project I was working on but doing all that stuff was the fun part.
bitwize 9y ago
If I don't overengineer my side projects -- if I stick to what's simple and practical -- how am I going to gain marketable skills with the modern frameworks?
- parmesan 9y ago
  Set your goals high for the project instead, achieving your goals can never be considered over-engineering. Don't let the scope creep into "oh I should totally spend 30 hours on this form design"
EGreg 9y ago
I think https://qbix.com/platform https://qbix.com/platform is the epitome of this. It came out of me wanting to abstract away once and for all everything that made developing social networks hard. And I just kept going lol.
pg_is_a_butt 9y ago
more like the art of over-authoring.... writing this article would also fit in the list of ways to waste your time while you're trying to accomplish something else. you're all idiots.
AngeloAnolin 9y ago
Bottomline is, you need to deliver something usable - regardless of its size or the technology behind it.
tmaly 9y ago
I have committed almost all of these mistakes with my food side project https://bestfoodnearme.com https://bestfoodnearme.com . Probably the best thing I have come to understand in building side projects is that you need to get it out in front of people as soon as possible. After that, just get feedback, use analytics, and iterate quickly to fix issues. Even more important, go look for existing problems, whether it be in online forums or talking to people in person. If you making a unicorn detector and you spend several years with your nose down, its a rough lesson to learn that no one wants a unicorn detector.
- kamikaz1k 9y ago
  Nice! I actually wanted to do something like for my own food obsessed self. Were you able to get good traction with your project? Or did you just do it for fun? My biggest hurdle was friction between using instagram and my app. But may that's not what you're targeting. Either way, I was just curious...
  - tmaly 9y ago
    I am still iterating to improve it, but I was doing it for fun and to scratch my own itch. There is not much traction, the 1% internet rule is real in terms of the number of people that contribute verse those that consume content.
yummybear 9y ago
I'm going to try another approach with my next project - not sure if it's a good idea, but I'm going with the Minimum Unviable Product. As soon as the project is bootstrapped and compilable, I'm going to release a version. I figure this way at least I can say I released something, even if it's not that feature rich.
old_chap 9y ago
As an amateur coder/noob I'm happy I'm not unique in this idea. I constantly start projects in Flask/Python and just try to make them as complex as possible (to the limit of my current capabilities). When I'm at work (im an Incident Manager for a NOC/SOC) I can somewhat hold my own in conversations about infrastructure issues. I'm also hoping to go into the security field soon (OSCP cert).
jcadam 9y ago
I absolutely use my side projects as a vehicle to learn new things... though I always try to pursue ideas that at least appear to have a possible path toward monetization at some point (though I haven't had much success with turning much of a profit from any projects thus far) :) For my current side project, which I've been working on for a year (and is probably a month or two away from being "beta-able"), I've learned Clojure, Elasticsearch, ES6, mithril.js, Ansible, and have expanded/refined my skills with RabbitMQ, Postgres (or SQL in general), and distributed architecture (not to mention getting to figure out how systemd service and timer configuration files work). I've really been working hard on it because my current day job doesn't involve any programming, and by the time I get home in the evening I'm really jonesing for some code. Have my side projects made me a better developer? Without a doubt. Am I any more employable? No, not really - I still haven't figured out how to improve my 'Cultural Fit' via side projects.
- rochak 9y ago
  Could you tell me how you go about knowing what all you might be needing.
godshatter 9y ago
I spent many years before I learned not to over-engineer side projects. I'm a c guy, mostly, so I don't have the stack issues (make and the standard c library and I'm g2g). Now I just start writing code to do the first basic feature, then refactor if needed to handle the second one, rinse and repeat. Once the project takes shape, it gets easier to predict what changes might be needed in the future, and what concessions I can make in the code for them now without slowing me down. Working this way is much nicer for me. It's more organic. I end up with code that evolves, and takes shape of it's own accord. It makes me feel more creative. The best thing is, I have code that does things. It's not out in the wild, but when I come home from work I have a couple of projects that are coming along nicely to work on and play with.
dev360 9y ago
I've stopped the urge to create repos, buy domain names etc, and started just diving into Sketch to do mockups to flesh out ideas instead. It gives me pause to think about features and to show people to get feedback. You can also use it as a sales tool to sell an app you are thinking of building without investing weeks and months. Nowadays its so simple to do nice mockups its so easy to find stencils for SemanticUI, icons etc. Makes it a breeze to get an interface going and I have never been this productive in the 'idea' stage before.
funnyenough 9y ago
on point! cannot agree more - I have been down this road before with my last project - Jabid.com. We tried to boil the ocean even though that was not the goal of the project. This time around, for https://veganfutura.com https://veganfutura.com, I am taking a complete opposite approach, that closely aligns with the list you have provided. Keep it simple, focus on the problem you are trying to solve and the user. Marketing and User acquisition > Tech initially.
beeforpork 9y ago
The best thing about side project is that I may be wonderfully inefficient, so I can do it the way I want and enjoy the way. Even mentioning the word 'deadline' or 'planing' or 'efficient' or 'customer' in the same sentence as 'side project' is contradictory to me. E.g., I love to program in assembler. Preferably 8 bit SoCs like AVR. The point is not to finish the project fast or even at all by using a more efficient language that may have libraries for everything, but to enjoy programming in 8 bit assembler. Reading the absurdly detailed manuals. Comparing chips at register level. Obviously, I need to write my own libraries/frameworks, because, I want to do that myself, in assembler. Even if not using assembly, I may still want to write the library myself in order to, well, do that. Wood workers often seem to have similar ideas about side projects, like when they build a project without power tools, without glue, from old scrap wood, etc., just because. But for some reason, I seldom meet hackers who understand the fun in doing stuff from scratch.
- cortesoft 9y ago
  Yeah, I read the title and I thought it was going to be an ode to the joy of trying new things with your side projects, and spending a ton of time working something just because it is fun. I guess the author and I have a different purpose for side projects. It sounds like the author is trying to actually start a business while still employed, while I am just trying to mess around and have fun.
- TeMPOraL 9y ago
  > But for some reason, I seldom meet hackers who understand the fun in doing stuff from scratch. Go visit your local hackerspace :). That said, it's probably because in tech, there's much more vertical space for one to pick their definition of "from scratch" from. You go low, to assembly level, but you don't fab your own chips :).
outworlder 9y ago
> I know: I’ll create a triple zoned redundant architecture with pub/sub database replication, a 32 node Kubernetes cluster and private networking across all regions – that way, I can handle anything! Well, maybe not a 32 node k8s cluster. However, spinning up K8s clusters is so easy now, that it costs almost no time to deploy stuff there. And so many things are taken care for you: configuration, secret storage, networking, versioning, keeping services up, etc. If on AWS, there's Kops. If on GCP, that's even easier, a couple of clicks and you have a cluster.
arrty88 9y ago
Depends on the goal. Do you want to learn a lot or launch a MVP in the least amount of time possible.
nathan_f77 9y ago
I'm currently working on a side-project, and I think I've figured out how to beat over-engineering and just launch a MVP which is actually a "minimum viable product". I'm building the product for my own use, and I'm just building the bare minimum that I actually need to complete the task. After that, I'm planning to bring on my first 10 customers and be available 24/7 to answer any questions and build the features that they need. But only if they're going to pay me money to use those features. I've often fallen in the trap of "oh that would be cool", but this time I'm going to try to only build the necessary features that actually bring in customers, instead of wasting time on a bunch of useless stuff. > Over-architecting infrastructure Yep, I think it's a good idea to just put everything on Heroku until you're spending at least $100 per month. It's just so easy to get up and running.
GigabyteCoin 9y ago
>You’re so hyped for this project and confident that it will succeed that you start thinking about the future: “how will I scale my application for the millions of users it will have?” That's one of my side project vices! Every time I write a new login database... I wonder if I should make the ID a MEDIUMINT or INT.
- imhoguy 9y ago
  I go full and make them BIGINT straight away. One less thing to worry once I will become billionaire.
fundabulousrIII 9y ago
Back in the 90's and early oughts no one thought of writing code this way. It was counter productive. I'd suggest that it is still counter productive to consider SDLC concepts when writing pet code: great idea or not. This whole agile sdlc idea is a distraction from writing good code.
binthere 9y ago
I have been successfully applying Jonathan Blow principles into my side projects. It's been working great. If you are not familiar I highly suggest you to listen to a few of his presentations about software development.
renas 9y ago
No over engineering, release early and release often, for example the open source https://lifelottery.github.io/ https://lifelottery.github.io/
pfarnsworth 9y ago
Just like in real life, the art of engineering is iteration. So I always get quick and dirty results with my side projects, and iterate if it's worth iterating. But my side projects are just for myself, not for others, so it's things like writing a multi-thread stock strategy backtester, a dashboard for my IoT devices, etc. It's never that I think will make money.
legulere 9y ago
I write a SVG path string optimizer for fun on the side. So far I wrote a custom decimal floating point type because of the 0.1+0.2 problem which I threw out afterwards, a zero allocation filter for printing rounded f32 without leading zeroes (.1 instead of 0.1) and dropping of unneeded spaces (.1.1 is valid and equal to 0.1 0.1), which I now want to replace with a custom float formatting routine. All I did I did for fun and maybe learning something. So I think over-engineering side projects is fine.
jmkni 9y ago
Duck it, it's your side project. You own it, nobody it paying you to work on it, over-engineer the shit out of it if you feel like it.
CarrotCodes 9y ago
I thoroughly enjoy over-engineering side projects, and writing about them (linked below). The trick for me is in managing expectations. It's completely possible for a learning project to evolve in to something widely used, just don't go in with that expectation. https://blog.skywelch.io/2017/06/over-engineering-bunnies-io/ https://blog.skywelch.io/2017/06/over-engineering-bunnies-io...
Spartan-S63 9y ago
I actually find that unless I do some amount of project management (i.e. use Trello or Pivotal Tracker), I easily get myself sidetracked. I'll use other tools for longer running timelines, but splitting things out into small stories actually helps me stay focused without getting bogged down in considering the implications to every design decision I make. For a lot of software, making the best decision today without worrying about tomorrow, while still following tried and true engineering practices (i.e. test driving, etc), leads to a project that continues to progress.
czep 9y ago
The whole point of a side project is to over-engineer it! The thought of holding myself to deadlines and driving myself to "ship" is totally antithetical to the entire purpose of a hobby project. It's a welcome relief from the 9-to-5 constant necessity to cut corners and finish things without ever properly understanding them because there's no time. For my side projects, I actually want to go down rabbit holes. I will try the same thing 7 different ways and benchmark the hell out of them. I'll read all the documentation and every blog post I can find. Learn a new language just to see if its features make things ever so slightly easier. Learn a new framework, write a new framework, throw it away and write a new one. The point isn't to ship code, it's to learn. Honestly I don't care if it ever ships. And even when I do release one, I don't care if nobody uses it except me.
- andmarios 9y ago
  Totally agree. The author seems to write about a specific subset of side projects: SaaS with a possibility to commercialize.
- krptos 9y ago
  Agreed. Generally, at any point of time I have two types of side projects going on: #1. One that I am very clear about how to implement. One that I want build, ship and grab users as fast as I can. The real goal of the project is to get it out their in the market and make a tiny dent in the universe ;) #2. One that I am not very clear about what it's gonna become. This one I want to take it slow. Experiment. Play with tech that I haven't had a chance to flirt with. The goal is to enhance my toolbelt and learn new concepts/tech. Clearly keeping these projects separate helps me keep things clear. I quickly take decisions and move fast with one. I play and experiment a lot, no matter what the outcome is, with the other one.
  - collinmanderson 9y ago
    Exactly. Side projects are often the best time to try out new tech. It's low risk.
  - deleted 9y ago
    [deleted]
Outrageous 9y ago
I feel like it's all related to your passion for the product. If you really love your side project, it's pretty likely it will be over-engineered. Whereas if you aren't that into it, it's much more likely to be under-enginered.
rubenbe 9y ago
#Mistake 6: not writing tests. In case I want to actually deliver a side-project, I usually write some basic tests. As such I can focus more on adding new (fun) features instead of hunting down regressions.
partycoder 9y ago
Depends what the goal of the project is. If the side project is for learning purposes then taking time to explore different approaches is OK. If your side project goal is to build a product, then you should approach it as a more traditional project, budgeting your time, and focusing on time-sensitive tradeoffs, etc.
jgable 9y ago
If you are doing hobby side projects that are solely for learning, then this article's advice is (mostly) not intended for you. If your goal is to explore new tech, try new approaches, go down rabbit holes, then by all means, go for it! The author's use of "side project" is referring to "I'm building something outside of my day job that I hope will turn into a business." IF your goal is to make money, then the author's advice is spot on. Get to MVP. Get users. Listen to them. Iterate. All the advice we hear that applies to startups, applies here.
SadWebDeveloper 9y ago
> Then, put it on the bare minimum architecture that it can run on (be that a 512MB instance from DigitalOcean or a medium instance on AWS). Or just test it in a VM and stop wasting money on side-projects.
Walkman 9y ago
I recently was thinking about 1). At my current company, I got to know agile methodologies for the first time, I liked it and started planning my side project, but it instantly started to felt like work! I just realized if I write the parts whatever interests me the most, I will be more inclined to finish it!
cyberferret 9y ago
At risk of a tar and feathering by the 'lean' crowd, I am going to declare that I don't do MVP's at all, and never really have, with any of my projects. Why? Well, because almost all my projects are designed for businesses and enterprises, and with that audience - things better work well and they better work first time during the demo or you don't stand a chance of getting a customer. Perhaps if I was writing a social media app for sharing cat pictures, then yeah, I would slap together an MVP and show it to cat lovers. After all - what is the worst that can happen if your cute cat picture doesn't upload? Just try it again or delete the app and walk away, and there are no consequences. But what if my HR app doesn't send that crucial reminder email to a department manager that one of your electrical tradesmen's license expires tomorrow, and you end up sending him out to a job next week and he makes a mistake that ends up getting your company sued and eventually shut down, putting hundreds of others out of work? I am sure as sh*t going to ensure that my reminder email system is absolutely over engineered to ensure reliable email delivery and tracking. You don't get ANY chances to blow that kind of stuff with a business of enterprise audience.
- jamesmishra 9y ago
  I fully agree with this. Big enterprise MVPs are often about cutting scope -- not cutting corners. Of course, by the very nature of enterprise customers, sometimes there isn't a whole lot of scope that you can cut.
spott 9y ago
As in everything: figure out what you are trying to get out of your side project. If you want something to generate a nice supplementary income stream, or practice for starting a company when you get "that great idea" then this article is very good advice. On the other hand, if you are trying to learn a new technology stack, develop experience in building scalable architectures or even beefing up your project management skills then this is terrible advice. So, the real art is figuring out what you want out of your side project.
n1vz3r 9y ago
I really like over-engineering my side projects. I have three frameworks, one in PHP, one in JS, and one in PHP/JS (a bit abandoned) that no one really uses. But you know what? Because no one uses them, I'm not burdened by any opinion except mine. When I change something, I don't read death threats because someone's solution stopped working. And since I don't have any deadlines, I can spend as much time as I want to get the exact solution as I planned. I don't need to compromise. That's quite opposite thing to what I do at worktime and get paid for.
treyhuffine 9y ago
If you want to release your project and maybe turn it into a start up, then these are all great points. However, many times the purpose of a side project is to just learn new things.
greggman 9y ago
I feel like there should be a solution that covers 99% of online projects. out of the box it scales, deploys, upgrades, does continuous integration, testing, logging, supports all major logins, has an admin console, metric console, etc... maybe it starts with the traditional Todo app and you just start modding. I find it incredibly frustrating that much of this isn't a solved problem that I can then just insert my side project into. hacking is fun until I have to maintain it and then I start wishing for all of those features instead of having to spend days or weeks implementing each of them from scratch
mmphosis 9y ago
Minimum viable product (MVP) https://en.wikipedia.org/wiki/Minimum_viable_product https://en.wikipedia.org/wiki/Minimum_viable_product
hoodoof 9y ago
I have built projects with a friend in the past. He so aggressively avoids overengineering that he goes the other way - if you make ANY effort to properly engineer something he gets really annoyed and says time is being wasted. It's almost like he feels software MUST be engineered to NOT scale in order to be an "MVP". It's strange because for me, thinking about how something is built means that it can be built right the first time, without extra effort, just by thinking it through.
- robertlagrant 9y ago
  This is my attitude to the React PATENTS file - yeah okay I probably won't be big enough to worry about it, but just in case I am, why take the risk with React?
p4lindromica 9y ago
One of the ways I learn is via code review. In a side project I'm often doing new things that I am not already an expert in. Any ideas on how to get feedback that things are ok vs not?
kyberias 9y ago
These apply to every project.
real-hacker 9y ago
The definition of a side project is not clear. Is it just for fun/learning about new tech, and can be thrown away? Or is it a serious endeavor, and supposed to be the next big thing? These require completely different mindsets.
dejv 9y ago
All those rules applies to our new non-side projects as well. I can't even count how many times I seen projects with awesome infrastructure and basically zero functionality after depleting whole budget (and usually some more).
elderK 9y ago
I can only speak for myself here but I feel that belonging too strongly in either camp is a bad thing. Ideally, you'd want to understand both worlds. As such, I feel that the projects I work on in my own time and the crazy things I do in them help me to better understand what kind of things really do need to happen when working under tight deadlines in the workplace. I.e. It's easier to produce an effective product quickly if you know from personal experience exactly which approaches or things really are essential. Also, note I haven't read the actual article. I'm replying mostly in response to the comments :)
mehh 9y ago
Love the title of this post. My side projects have all been over engineered, an unsuccessful. They have also helped my career enormously :)

869 points by elsyms 9y ago

There's probably another reason hidden in the first sentence, which is that there's a transition from building the product (engineering) to shipping the product (marketing, customer development, sales) that requires different skill sets and experiences. Was just discussing this with a software engineer friend of mine who realizes that he "has no clue how to market my product the right ways."

klokoman 9y ago

But that's all the fun

wyager 9y ago
Agreed. The best part of side projects is that I can take as long as I want to build it to my satisfaction, rather than to the minimal acceptable state for business purposes.

ropman76 9y ago

If your goal is to make money on something, then yes I agree with the author's points. However the whole goal of side projects is to have fun with new languages, frameworks etc.

auserperson 9y ago

I think the majority of engineers only do side projects to learn things they couldn't at work or explore architectures they wouldn't be able to in a work environment.

mixedmath 9y ago

When I first read the title, I had thought that this would be a humorous example of a delightfully over-engineered side project. It turns out that this is instead a list of 5 common mistakes of over-engineering with some commentary.

caleblloyd 9y ago

Lots of times I use side projects as a way to explore new stacks, deployment approaches, etc. The goal of the project is not to deliver anything, it's to learn so that when I go to do my next "real" project at work, I know what works well and what doesn't.

mrisoli 9y ago
Me too, I think there are two types of side projects, this article focuses on those which I believe people will try to generate a business out of, or at least some form of passive income. But there are also some side projects where you just want to get your hands dirty with some different tech, if you do deliver something then you can showcase it on your portfolio. I know when the app store gold rush first started a lot of people did this as experience to get iOs dev jobs.
- dcminter 9y ago
  Exactly - I enjoy side projects precisely because they're not bound by pragmatic concerns. I can delve as deep as I like into getting the CI pipeline working just right, making sure the servers and cacheing are set up ok and all those other things that I usually have to leave to colleagues. More often than not the skills I learn doing this kind of thing end up being very useful to me in the day job. I'd like it if the side project was successful enough to justify all of that yak shaving, but I don't really expect it to be and I'm not that fussed when it isn't. But if your primary goal is that your side project should become your day job then the article is pretty good advice. Leave your yak unshaven :)
  - TeMPOraL 9y ago
    > they're not bound by pragmatic concerns. 99% of the time, if you want to go pragmatic, you should buy a ready-made solution (and then possibly adapt it to your use case). It's efficient (especially if you need the problem solved to make money) - but it's no fun.
- mgolawala 9y ago
  I agree, but there are good reasons why it is often good to combine both types. Considering this is only a side project, you have a high likelihood of abandoning it out of boredom, lack of traction or simply burn out. However, if your side project is "interesting" to you because you are experimenting with various tech, you come back to it because it is fun and it will keep you motivated. It is totally cool to build something to learn and throwaway, but why not build something to learn that might also generate some income in the future?

jlebrech 9y ago

this should apply to the start of any project.

xwvvvvwx 9y ago

For me it's very important that I don't discuss projects with friends / family / colleagues etc. until I really have something to show. I have always found it very easy to talk all the energy out of a good idea before I've even got any real work done.

hectorlorenzo 9y ago
That's a thing. Expressing in public your desire to achieve something makes you less likely to deliver. http://www.psych.nyu.edu/gollwitzer/09_Gollwitzer_Sheeran_Seifert_Michalski_When_Intentions_.pdf http://www.psych.nyu.edu/gollwitzer/09_Gollwitzer_Sheeran_Se...
- owebmaster 9y ago
  I don't agree with this reasoning. Expressing in public your projects is a compromise and it can help you keep developing it when people ask if it is ready. If you are going to abandon your projects, not telling people will not help you succeed, it will only spare you the shame and effort explaining why you droped another project.
  - hectorlorenzo 9y ago
    A little bit of pop psychology from my end... I don't think people start projects with the conscious intention of building identity, but with a genuine intention of delivering. I do think, though, that there's a small (or big!) part of your unconscious brain that binds these efforts to an overarching persona-building (both inwards and outwards) project. Although unconscious, it contributes to this net energy. I guess that this communicating your intentions satisfies this portion of your brain, and reduces the amount of energy involved in the project.
    - owebmaster 9y ago
      Sure, this makes sense. I just don't think (using only myself as a reference, which is totally biased) that this effect has a bigger impact than the social pressure the public compromise creates.

myhf 9y ago

> 4. Use frameworks and customise them – only refactor/build your own when absolutely neccessary. > 5. Build your project first – then worry about continuous delivery. These contradict each other. Building your own deployment system is just as much of a distraction as building your own CSS framework.

elsyms 9y ago
Why do you need a deployment system if you have nothing to deploy? Build the project locally, then use continuous delivery when you have something to deliver.
- myhf 9y ago
  "Build the project locally" implies some kind of pipeline system, even if it's not "deployment" or "delivery". Triggering that pipeline from a commit trigger instead of manually just means copying the build command into a config file. It pays for itself in the first hour.
  - vorpalhex 9y ago
    `git push deploy` You can get quite far without any kind of "pipeline system" nor does building imply a pipeline.
MichaelBurge 9y ago
Your "deployment system" could be a 20-line systemd service and a git checkout. You deploy with a "git pull" and "sudo service restart".
- staticelf 9y ago
  You could even skip the systemd and just rely that the server will never be restarted in the very beginning.
Allower 9y ago
Who said anything about building a deployment system?? thats not the project..

quadcore 9y ago

“I know: I’ll create a triple zoned redundant architecture with pub/sub database replication, a 32 node Kubernetes cluster and private networking across all regions – that way, I can handle anything!” The good news is, once you've done that, you can get a pretty decent job. The majority of software engineers seem to be under the illusion that potential customers care about the stack they are running. This is not entirely true. The technology do surface in terms of user experience. It does in the details: app start, ui responsiveness, etc. I'm writing a game using go right now, and the end user can tell it's different because the user's cpu seems more powerful. All in all, I think those are good mistakes, not bad ones. I wish more engineers could spot over-engineering, and they don't until they did the mistake a couple times. Side projects are, I believe, how one really learn to write programs. I think the right advice is: do whatever you want, do all the mistakes. How can you write a successful side project anyway if you never learn how to write a program? Also, the worse type of programmers are the ones telling you about over-engineering because they read about it in a blog. Code more, read less, I guess should be a good motto.

elsyms 9y ago
"A customer will not know or care if you are using Ruby, Go, PHP or any other language as long as what you have written is performant and is fit for purpose (which all modern languages are)." Where games are concerned I'd tend to agree (though Minecraft was built in Java and was TERRIBLE for the first couple of years, now look at it).
- majewsky 9y ago
  > Minecraft was built in Java and was TERRIBLE for the first couple of years, now look at it Well, it still takes absurd amounts of memory for what it's doing, which I guess is on par for Java. (On my notebook with 4GB RAM, modpacks fall into one of three categories: "works well", "have to shut off as many OS services as possible before starting" and "forget it".)
  - merb 9y ago
    well only because java has no value types/struct like data structure. the reason why this makes memory so heavy is because even a simple int list, needs boxing. So you end up with a big chunk of garbage which you don't need. On the server, this is not a problem because most of the time the list is a young gen object and die really fast, so G1GC solves most problems. The problem of course is really big, because arrays are a pain to work with and not many people do that. However as soon as java 10 hits, we might get value types and maybe this changes the game (but we will see, since it would've taken way to long for that thing). Also there are more and more types inside the standard library who would be great value types, i.e. LocalDate/LocalDateTime, but at the moment they take way more memory than needed, especially since they only contain 2 immutable shorts and 1 immutable int.. in the perfect case it would be something like 8 byte, however it takes way way way way more. (basically a simple class at least takes 16 bytes, which is already double the field size, so you end up with 24 bytes [probably more due to various other stuff]) btw. for a game like minecraft you probably have a lot of types that follow the same stuff like LocalDate, small class with immutable int's/short's for position, etc. these have the same problem and will fill the memory more quickly than needed

dtzur 9y ago

"Continuously Delivering Nothing" is a mission statement for several companies I know.

bpicolo 9y ago
Tar it up, cat it to devnull
- sleepychu 9y ago
  You shouldn't use cat for a single tar. Less will work just as well.
- cryptos 9y ago
  It is important to compress it with bzip2 or your even better custom algorithm before finally archiving it in /dev/null
le-mark 9y ago
I'm working on v2 of an app that's been under development for 5 years with no production users. They're still on v1. Pitiful.

GhostVII 9y ago

I always overengineer my side projects, because it's way more interesting that way. I do side projects for fun and to learn something, both of which happen more easily when the project is overengineered.

majewsky 9y ago
I tend to underengineer my side projects because the stuff that I touch at work is frequently much too overengineered to begin with (cough OpenStack cough).
kentt 9y ago
Agreed. Part of the fun of my side project is that it's well built. At work I have to deal with poor quality code practices from artificially tight timelines, contractors, etc. From one perspective it's very wasteful to have higher quality code for an app that generates $150/mo. vs multimillion dollar projects I manage at my day job, but I enjoy that $150, the code and customers more than what I get from my day job. I wouldn't if it was an MVP.

mikegerwitz 9y ago

The article has good points. But the term "side project" implies working on something that isn't part of your job. I think looking at your project from that perspective can also be a step in the wrong direction---if you enjoy what you do, work is just one such application of your knowledge and ability. Projects you do outside of work are another, and may even be more important to you than work. A number of commenters point out that many side-projects to them are to explore new concepts/libraries, or as a means of learning. If it's for the sake of learning for your job, then yes, it'd be a side project. But if you're looking to take a project to completion (the author talks about products; my projects aren't), I find that using it as a learning environment extremely detrimental. Yes, you do learn things over the course of development---through research, struggle, and growth. But if too much of your time is spent on things you don't have a good foundation on, you may burn out too early, wind up frustrated, and wind up with a mess that leads to refactoring. Use a foundation that you have experience with and know well, and learn the parts that are necessary. (I do distinguish, though, a research project with a project for playful learning. The former is much more formal and disciplined. But you can still burn out early.)

twelvedogs 9y ago

heh, most of this stuff is extra wastes of time past the very first hurdle that you should be working on: make something that works and is useful to people. sure it would be great to have scaling to 10,000 users already sorted by the time you get there but odds are your project will never get there, when you get to 5k users you need to start worrying about that if you're just doing it to learn or for fun then obviously this article or my comment don't apply

wyager 9y ago

> A customer will not know or care if you are using Ruby, Go, PHP or any other language Users will absolutely notice if your product crashes or breaks, which is more likely with some languages than others. The single most likely reason for me to stop using a nominally useful product is that it doesn't work reliably. I hate the fact that so much technology today is broken, partly because people have this cavalier attitude towards choosing technologies that help to keep things working.

pgm8705 9y ago

#3 hits home. I've been "working" on my side project for 3 years now with almost no progress because I'm constantly intrigued by new frameworks and BaaS. I spend all my time tinkering with them until I get bored with it all together. If I had just built the thing in Rails from the start I'd probably have a solid app under my belt by now.

sleepychu 9y ago
Time for a new branch with rails then!
0xfeba 9y ago
Yeah I've been working on a game for 5+ years that went thru C++/DirectX, then C# and Managed DirectX, and now Java and LibGDX. Thinking about using Unity now...
- Kiro 9y ago
  I'm in the same position. I've built my own game engine from scratch and while it was a really good learning exercise I recently started looking at Unity and wow... So many things I've always wanted in my engine that I will get for free. I don't regret a thing though.

yigitozkavci 9y ago

I think side projects and hobby projects need to be thought of separately. In the context of hobby projects, over-engineering can even serve purpose of learning new things you otherwise could not. For side projects, or startup candidates, however, you just ship it

viach 9y ago

If only it was applied to side projects exclusively...

CJefferson 9y ago

I recently had this with a html5 project. Thought it would be nice to use es6 and transpile, then disappeared down a hole of gulp and webpack. Eventually backed up and used Make to run Babel and then cat the files together.

Mojah 9y ago

Related to this, I've been following a practice I call "Release Notes Driven Development", which is perfect for side projects. Don't worry too much about technical debt (although, keep it under control), but focus on the little time you have and how to make that as efficient as possible. I wrote about it here; https://ma.ttias.be/release-notes-driven-development-rndd/ https://ma.ttias.be/release-notes-driven-development-rndd/

dispo001 9y ago
Sounds good. Early on, a quick list of bragables should be able to tell you how awesome the idea is. Have new bragables for the next version. If its not awesome at some stage (and the bugs are fixed) you are probably done? It reminds me of many release notes describing a ton of work without any woah.

dudul 9y ago

Oddly enough, I would apply some of these practices to real projects as well.

hamvocke 9y ago

Good points in there. I have a little trouble with "mistake #5" though. I found that worrying about deployment automation early on is immensely valuable, especially for side projects. I don't always find time or interest to keep working on side projects after a regular day of work. Quite often there are weeks or even months where I abandon a side project only to come back at a later point when motivation is back or things have settled down and leave more time for my side projects. If I didn't take the time to automate testing and deployments (which makes a lot of what continuous delivery is about) I find myself struggling with getting things up and running and become frustrated before I even get started. Then there's only one way to make sure that my test and deployment automation keeps working: running it regularly, e.g. in a pipeline. So, yes, don't over optimise your deployment pipeline early on. But having one in place early can pay off really soon -- and maybe even keep your side project alive.

monksy 9y ago
This. I'm a big fan of doing the hard work early on. When you push that step off it makes dev later slow and frustrating.
bitwisebob 9y ago
Agreed. Having a deployment pipeline in place dramatically reduces the friction of returning to an old project. I have a back-burner project that touch very infrequently, but I can jump in, make a quick change, and have it deployed to "production" in just a few seconds.
juanuys 9y ago
I definitely automate locally by way of Makefiles, running tests then using the GCloud/aws CLI to deploy from local. Things like TravisCI, webhooks, etc come much later if at all.

shortoncash 9y ago

I can tell this article was written by a very wise man who wasted time on lots of projects or saw a lot of other people waste time on projects. He's described behaviors I have dabbled in many, many times. In fact, I have a side project currently where I just needed to fabricate a tiny fixture for a physical device. I caught myself over-engineering the prototype, went to sleep, woke up, saw this article and came to my senses. Consider this article bookmarked!

redm 9y ago

Man, I think this article is dead on, except it doesn't identify the original reason, for me at least. In a normal environment, there are constraints on time, money, goals, resources, etc. Suddenly, when there are no constraints, a side project, you try to do everything perfectly, which leads to the issues in this article.

aetherspawn 9y ago

Doing #5 is chronic because I'm too lazy to handle my own deployments and it's easier to copy paste a generic firebase deploy. Then you can send the link to anyone and it always just works.

ratsimihah 9y ago

"Continuously delivering nothing" – exactly what I'm doing, building a product no one will use.

jondubois 9y ago

Side projects are meant to be challenging because they are a learning opportunity so it's possible that they end up over-engineered sometimes. You can always refactor the parts that are over-engineered. There's a very fine line between highly-engineered and over-engineered. One of my favourite projects is Kubernetes; some might consider it over-engineered but it's extremely powerful. "Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away"

zimpenfish 9y ago
> "[...] when there is nothing left to take away" I've always taken that quote to apply only when you have been taking things away; not when you've ended up with a colossal complex monstrosity where removal of any single strand would cause total collapse.

esseti 9y ago

For the record (it's not mentioned but a lot of time it happens): security (e.g. password storage) should not be something left for later. Do it or use a framework that does it for you.

breatheoften 9y ago

This makes a lot of sense -- and makes me feel like I'm guilty of over engineering a non-side project I started working on at work ... I'm always looking to make the code as easy to change as possible -- the idea being that change is a precondition for any form of improvement ... Reading this article makes me wonder if I've turned that pursuit into an anti-goal

darod 9y ago

Setting up a CI on AWS using git, CodePipeline, Code Build, and elastic beanstalk takes no more than a few hours. It's worth the few hours. Otherwise you're manually pushing builds out which wastes more time in the long. In the beginning your CI should be nothing more than ==> I check in code, it deploys to environment. Later on you can worry about triggering tests and preventing builds for deploying if they fail. Also if you use elastic beanstalk, you can scale as necessarily with a few configuration changes.

hashkb 9y ago

I have a personal Jenkins that's been running for years. Adding new jobs isn't yak shaving; it's a two minute task that gives me CD. Same with my AWS "framework". And my CSS "framework" and my React component superclass and lots of other stuff I carry from one side project to another. Some would (and have) accused me of over engineering; but I can discover my project is pointless very quickly.

datpuz 9y ago

This article ignores the fact that many of us work on side projects for fun. PHP? Nah.

FatAmericanDev 9y ago

Every enterprise client I have worked with has asked about our tech stack. I will agree that your groupon clone users do not care about your stack, but respond to an enterprise SaaS RFP and stack, security, DR will all be discussed.

polote 9y ago

This post should be renamed : The art of over-engineering "those software engineers often have the tendency to over-engineer their side projects" When your first sentence is false, so do is the rest of the article !

tw1010 9y ago
"When your first sentence is false, so do is the rest of the article !" Not necessarily.

scadge 9y ago

I believed over-engineering IS the point of personal side-projects.

zimpenfish 9y ago
I'm in the opposite camp - personal side-projects are where you can safely implement the MVP and/or ruthlessly chop things/ideas that never* get used without people harassing you about bells and whistles.

khana 9y ago

The takeaway is ARCHITECTURE. thank you

SonicSoul 9y ago

guilty of all of the above. my thinking is always "i'll kill 2 birds with one stone, side project and learning the latest framework!" and yah.. after struggling with the framework for 2 weeks the project loses steam and then just thinking about a stupid challenge i have to solve BEFORE getting back to my project is crippling. it does have a nice side benefit however of actually learning a new framework :)

awjr 9y ago

A lot of time this can be applied to any new project not just side projects.

rthomas6 9y ago

Triggered. Leave me alone and let me make my Hugo blog migration with my custom theme using bulma.io and webpack. Never mind that the WordPress setup I have now could handle 100x the traffic I get.

wreath 9y ago

I think deferring the CD part until the end can be really painful when you are "ready". I think it's a worth upfront investment even if you have few users, will make your life much easier and less stressful.

zhuzhu 9y ago

the best real joke of month!

masukomi 9y ago

> "Develop your project. Then, put it on the bare minimum architecture that it can run on (be that a 512MB instance from DigitalOcean or a medium instance on AWS). As you get more users, monitor, and modify the infrastructure appropriately to account for load and redundancy." That's a pretty ignorant and potentially short sighted statement. Look at what happened to Instagram. They almost lost it all because they hadn't planned for the possibility of handling scale quickly Sometimes you simply don't have the luxury of monitoring and modifying to account for load and redundancy. Sometimes you're monitoring says "hey traffic" shortly followed by "OMG YOU'RE SCREWED" because you can't possibly think up and implement an appropriate scalable architecture, and modify the codebase to handle it and and and while your servers are melting and everyone is yelling at you. Yes, this is very unlikely to happen, but when it does it can cost you your business if you're not prepared. Obviously apps with no networking and no viral component are far less likely to have this problem. Got a macApp that creates icon sets for XCode? Yeah, you can ignore this scaling issue. Writing the next Instagram? You're a fool if you don't have plans in place, in advance of launch, for . handling the onslaught of people that will make your business successful.

tarr11 9y ago
> Look at what happened to Instagram. They almost lost it all because they hadn't planned for the possibility of handling scale quickly I had never heard this version before. The story I heard was that they pivoted from burbn, which they felt was to cluttered and unfocused. [1] If they followed your advice, they would have built a scaled up version of burbn, which nobody would have used. [1] http://www.makeuseof.com/tag/1-billion-in-2-years-the-story-of-instagram-as-told-by-its-founders/ http://www.makeuseof.com/tag/1-billion-in-2-years-the-story-...
lucideer 9y ago
It's interesting that you choose Instagram - a highly successful company/product - as an example to warn people against. It may be harder to come up with examples of people who failed, for the obvious reason - they failed and are therefore not well known, but I really don't find the statement that Instagram "nearly lost it all" very compelling. Did they? Really? If they had had even more problems scaling, would they really have lost users? Look at Twitter, they took years to resolve issues. I actually think the importance of stability and uptime are overblown in the early stages. If I think about the times downtime has significantly impacted my view of a company or product, it's when large established services I'd come to rely on professionally went down. You need to establish a reputation for reliability before most people expect it of you. That's not to say it's not important, but you need to look at context. Noone relies on Instagram minute by minute for their professional work, so the priorities are different. UX, marketing, network effects, etc. are going to trump stability.

thecodemonkey 9y ago

We just did a talk on this exact thing at Laracon last month. And raised many of the same points. We however also talked a lot about the business side of things. For those who are interested: Video: https://streamacon.com/video/laracon-us-2017/day-2-mathias-and-michele-hansen https://streamacon.com/video/laracon-us-2017/day-2-mathias-a... Slides: https://speakerdeck.com/minicodemonkey/launching-and-scaling-a-side-project https://speakerdeck.com/minicodemonkey/launching-and-scaling...

bojanvidanovic 9y ago

I'm pretty sure if I wasn't over-engineering my side projects I wouldn't be technically competitive as I am now. So over-engineering is a very good strategy for learning and mastering any field. You can't really lose anything, either your project will succeed or you will acquire/strengthen your skills.

1hkogreendragon 9y ago

Where would Hacker News be without over-engineered garbage software?

ThomPete 9y ago

When I do side projects it's normally because I want to explore something which is too risky for someone to do (or it would most likely have been done). In my experience that normally requires some sort of overengineering because your problem and solution isn't common. A few of them turned into great side businesses but most of the times it help me explore technologies and projects while still solving real problems.

trollopTheJope 9y ago

I didn't think they'd be a word I dislike more than "architecting". Hats off to the author for using "over-architecting" in his insightful blogpost.

seanwilson 9y ago

A major roadblock I find is that when you want to release your side project and start charging for it, I feel it becomes much harder to make changes after. This means you get caught in a loop of trying to polish it more and more before biting the bullet and releasing it. If you change the interface a lot or how features work, you'll annoy users that don't like the change and you'll have to be extra careful not to introduce any bugs. If users can save data, the new version will have to work with the old data format. If you want to move features from a free version to the paid version you'll annoy users. So to me, releasing a MVP you charge for that might change a lot later might lead to more headaches than developing the idea further. Are there ways around this? Side projects you release for free are one thing but when you want to charge for them the amount of polish and attention to detail you have to pay to marketing jumps by an order of magnitude. People will tolerate bugs and rough edges for free projects but not for paid ones. I agree with the advice about not wasting time with CI, switching frameworks etc. though.

thinbeige 9y ago

IDK, I over-engineer my side projects every single time. Like tweaking for days something which is so clearly premature optimization. But sometimes I need that knowledge again for the next side project. And this over-engineering is often a way to simplify a reference architecture and be much faster at the end. So for me over-engineering equals learning but is also a big part of the motivation.

wvenable 9y ago
I have similar experiences. I was working with an embedded platform that uses C exclusively and I decided I wanted to use C++. I ran into mountains of hurdles, had to stare at disassembled code, tweak stuff, build a class library, etc. I never actually finished the project I was working on but doing all that stuff was the fun part.

bitwize 9y ago

If I don't overengineer my side projects -- if I stick to what's simple and practical -- how am I going to gain marketable skills with the modern frameworks?

parmesan 9y ago
Set your goals high for the project instead, achieving your goals can never be considered over-engineering. Don't let the scope creep into "oh I should totally spend 30 hours on this form design"

EGreg 9y ago

I think https://qbix.com/platform https://qbix.com/platform is the epitome of this. It came out of me wanting to abstract away once and for all everything that made developing social networks hard. And I just kept going lol.

pg_is_a_butt 9y ago

more like the art of over-authoring.... writing this article would also fit in the list of ways to waste your time while you're trying to accomplish something else. you're all idiots.

AngeloAnolin 9y ago

Bottomline is, you need to deliver something usable - regardless of its size or the technology behind it.

tmaly 9y ago

I have committed almost all of these mistakes with my food side project https://bestfoodnearme.com https://bestfoodnearme.com . Probably the best thing I have come to understand in building side projects is that you need to get it out in front of people as soon as possible. After that, just get feedback, use analytics, and iterate quickly to fix issues. Even more important, go look for existing problems, whether it be in online forums or talking to people in person. If you making a unicorn detector and you spend several years with your nose down, its a rough lesson to learn that no one wants a unicorn detector.

kamikaz1k 9y ago
Nice! I actually wanted to do something like for my own food obsessed self. Were you able to get good traction with your project? Or did you just do it for fun? My biggest hurdle was friction between using instagram and my app. But may that's not what you're targeting. Either way, I was just curious...
- tmaly 9y ago
  I am still iterating to improve it, but I was doing it for fun and to scratch my own itch. There is not much traction, the 1% internet rule is real in terms of the number of people that contribute verse those that consume content.

yummybear 9y ago

I'm going to try another approach with my next project - not sure if it's a good idea, but I'm going with the Minimum Unviable Product. As soon as the project is bootstrapped and compilable, I'm going to release a version. I figure this way at least I can say I released something, even if it's not that feature rich.

old_chap 9y ago

As an amateur coder/noob I'm happy I'm not unique in this idea. I constantly start projects in Flask/Python and just try to make them as complex as possible (to the limit of my current capabilities). When I'm at work (im an Incident Manager for a NOC/SOC) I can somewhat hold my own in conversations about infrastructure issues. I'm also hoping to go into the security field soon (OSCP cert).

jcadam 9y ago

I absolutely use my side projects as a vehicle to learn new things... though I always try to pursue ideas that at least appear to have a possible path toward monetization at some point (though I haven't had much success with turning much of a profit from any projects thus far) :) For my current side project, which I've been working on for a year (and is probably a month or two away from being "beta-able"), I've learned Clojure, Elasticsearch, ES6, mithril.js, Ansible, and have expanded/refined my skills with RabbitMQ, Postgres (or SQL in general), and distributed architecture (not to mention getting to figure out how systemd service and timer configuration files work). I've really been working hard on it because my current day job doesn't involve any programming, and by the time I get home in the evening I'm really jonesing for some code. Have my side projects made me a better developer? Without a doubt. Am I any more employable? No, not really - I still haven't figured out how to improve my 'Cultural Fit' via side projects.

rochak 9y ago
Could you tell me how you go about knowing what all you might be needing.

godshatter 9y ago

I spent many years before I learned not to over-engineer side projects. I'm a c guy, mostly, so I don't have the stack issues (make and the standard c library and I'm g2g). Now I just start writing code to do the first basic feature, then refactor if needed to handle the second one, rinse and repeat. Once the project takes shape, it gets easier to predict what changes might be needed in the future, and what concessions I can make in the code for them now without slowing me down. Working this way is much nicer for me. It's more organic. I end up with code that evolves, and takes shape of it's own accord. It makes me feel more creative. The best thing is, I have code that does things. It's not out in the wild, but when I come home from work I have a couple of projects that are coming along nicely to work on and play with.

dev360 9y ago

I've stopped the urge to create repos, buy domain names etc, and started just diving into Sketch to do mockups to flesh out ideas instead. It gives me pause to think about features and to show people to get feedback. You can also use it as a sales tool to sell an app you are thinking of building without investing weeks and months. Nowadays its so simple to do nice mockups its so easy to find stencils for SemanticUI, icons etc. Makes it a breeze to get an interface going and I have never been this productive in the 'idea' stage before.

funnyenough 9y ago

on point! cannot agree more - I have been down this road before with my last project - Jabid.com. We tried to boil the ocean even though that was not the goal of the project. This time around, for https://veganfutura.com https://veganfutura.com, I am taking a complete opposite approach, that closely aligns with the list you have provided. Keep it simple, focus on the problem you are trying to solve and the user. Marketing and User acquisition > Tech initially.

beeforpork 9y ago

The best thing about side project is that I may be wonderfully inefficient, so I can do it the way I want and enjoy the way. Even mentioning the word 'deadline' or 'planing' or 'efficient' or 'customer' in the same sentence as 'side project' is contradictory to me. E.g., I love to program in assembler. Preferably 8 bit SoCs like AVR. The point is not to finish the project fast or even at all by using a more efficient language that may have libraries for everything, but to enjoy programming in 8 bit assembler. Reading the absurdly detailed manuals. Comparing chips at register level. Obviously, I need to write my own libraries/frameworks, because, I want to do that myself, in assembler. Even if not using assembly, I may still want to write the library myself in order to, well, do that. Wood workers often seem to have similar ideas about side projects, like when they build a project without power tools, without glue, from old scrap wood, etc., just because. But for some reason, I seldom meet hackers who understand the fun in doing stuff from scratch.

cortesoft 9y ago
Yeah, I read the title and I thought it was going to be an ode to the joy of trying new things with your side projects, and spending a ton of time working something just because it is fun. I guess the author and I have a different purpose for side projects. It sounds like the author is trying to actually start a business while still employed, while I am just trying to mess around and have fun.
TeMPOraL 9y ago
> But for some reason, I seldom meet hackers who understand the fun in doing stuff from scratch. Go visit your local hackerspace :). That said, it's probably because in tech, there's much more vertical space for one to pick their definition of "from scratch" from. You go low, to assembly level, but you don't fab your own chips :).

outworlder 9y ago

> I know: I’ll create a triple zoned redundant architecture with pub/sub database replication, a 32 node Kubernetes cluster and private networking across all regions – that way, I can handle anything! Well, maybe not a 32 node k8s cluster. However, spinning up K8s clusters is so easy now, that it costs almost no time to deploy stuff there. And so many things are taken care for you: configuration, secret storage, networking, versioning, keeping services up, etc. If on AWS, there's Kops. If on GCP, that's even easier, a couple of clicks and you have a cluster.

arrty88 9y ago

Depends on the goal. Do you want to learn a lot or launch a MVP in the least amount of time possible.

nathan_f77 9y ago

I'm currently working on a side-project, and I think I've figured out how to beat over-engineering and just launch a MVP which is actually a "minimum viable product". I'm building the product for my own use, and I'm just building the bare minimum that I actually need to complete the task. After that, I'm planning to bring on my first 10 customers and be available 24/7 to answer any questions and build the features that they need. But only if they're going to pay me money to use those features. I've often fallen in the trap of "oh that would be cool", but this time I'm going to try to only build the necessary features that actually bring in customers, instead of wasting time on a bunch of useless stuff. > Over-architecting infrastructure Yep, I think it's a good idea to just put everything on Heroku until you're spending at least $100 per month. It's just so easy to get up and running.

GigabyteCoin 9y ago

>You’re so hyped for this project and confident that it will succeed that you start thinking about the future: “how will I scale my application for the millions of users it will have?” That's one of my side project vices! Every time I write a new login database... I wonder if I should make the ID a MEDIUMINT or INT.

imhoguy 9y ago
I go full and make them BIGINT straight away. One less thing to worry once I will become billionaire.

fundabulousrIII 9y ago

Back in the 90's and early oughts no one thought of writing code this way. It was counter productive. I'd suggest that it is still counter productive to consider SDLC concepts when writing pet code: great idea or not. This whole agile sdlc idea is a distraction from writing good code.

binthere 9y ago

I have been successfully applying Jonathan Blow principles into my side projects. It's been working great. If you are not familiar I highly suggest you to listen to a few of his presentations about software development.

renas 9y ago

No over engineering, release early and release often, for example the open source https://lifelottery.github.io/ https://lifelottery.github.io/

pfarnsworth 9y ago

Just like in real life, the art of engineering is iteration. So I always get quick and dirty results with my side projects, and iterate if it's worth iterating. But my side projects are just for myself, not for others, so it's things like writing a multi-thread stock strategy backtester, a dashboard for my IoT devices, etc. It's never that I think will make money.

legulere 9y ago

I write a SVG path string optimizer for fun on the side. So far I wrote a custom decimal floating point type because of the 0.1+0.2 problem which I threw out afterwards, a zero allocation filter for printing rounded f32 without leading zeroes (.1 instead of 0.1) and dropping of unneeded spaces (.1.1 is valid and equal to 0.1 0.1), which I now want to replace with a custom float formatting routine. All I did I did for fun and maybe learning something. So I think over-engineering side projects is fine.

jmkni 9y ago

Duck it, it's your side project. You own it, nobody it paying you to work on it, over-engineer the shit out of it if you feel like it.

CarrotCodes 9y ago

I thoroughly enjoy over-engineering side projects, and writing about them (linked below). The trick for me is in managing expectations. It's completely possible for a learning project to evolve in to something widely used, just don't go in with that expectation. https://blog.skywelch.io/2017/06/over-engineering-bunnies-io/ https://blog.skywelch.io/2017/06/over-engineering-bunnies-io...

Spartan-S63 9y ago

I actually find that unless I do some amount of project management (i.e. use Trello or Pivotal Tracker), I easily get myself sidetracked. I'll use other tools for longer running timelines, but splitting things out into small stories actually helps me stay focused without getting bogged down in considering the implications to every design decision I make. For a lot of software, making the best decision today without worrying about tomorrow, while still following tried and true engineering practices (i.e. test driving, etc), leads to a project that continues to progress.

czep 9y ago

The whole point of a side project is to over-engineer it! The thought of holding myself to deadlines and driving myself to "ship" is totally antithetical to the entire purpose of a hobby project. It's a welcome relief from the 9-to-5 constant necessity to cut corners and finish things without ever properly understanding them because there's no time. For my side projects, I actually want to go down rabbit holes. I will try the same thing 7 different ways and benchmark the hell out of them. I'll read all the documentation and every blog post I can find. Learn a new language just to see if its features make things ever so slightly easier. Learn a new framework, write a new framework, throw it away and write a new one. The point isn't to ship code, it's to learn. Honestly I don't care if it ever ships. And even when I do release one, I don't care if nobody uses it except me.

andmarios 9y ago
Totally agree. The author seems to write about a specific subset of side projects: SaaS with a possibility to commercialize.
krptos 9y ago
Agreed. Generally, at any point of time I have two types of side projects going on: #1. One that I am very clear about how to implement. One that I want build, ship and grab users as fast as I can. The real goal of the project is to get it out their in the market and make a tiny dent in the universe ;) #2. One that I am not very clear about what it's gonna become. This one I want to take it slow. Experiment. Play with tech that I haven't had a chance to flirt with. The goal is to enhance my toolbelt and learn new concepts/tech. Clearly keeping these projects separate helps me keep things clear. I quickly take decisions and move fast with one. I play and experiment a lot, no matter what the outcome is, with the other one.
- collinmanderson 9y ago
  Exactly. Side projects are often the best time to try out new tech. It's low risk.
- deleted 9y ago
  [deleted]

Outrageous 9y ago

I feel like it's all related to your passion for the product. If you really love your side project, it's pretty likely it will be over-engineered. Whereas if you aren't that into it, it's much more likely to be under-enginered.

rubenbe 9y ago

#Mistake 6: not writing tests. In case I want to actually deliver a side-project, I usually write some basic tests. As such I can focus more on adding new (fun) features instead of hunting down regressions.

partycoder 9y ago

Depends what the goal of the project is. If the side project is for learning purposes then taking time to explore different approaches is OK. If your side project goal is to build a product, then you should approach it as a more traditional project, budgeting your time, and focusing on time-sensitive tradeoffs, etc.

jgable 9y ago

If you are doing hobby side projects that are solely for learning, then this article's advice is (mostly) not intended for you. If your goal is to explore new tech, try new approaches, go down rabbit holes, then by all means, go for it! The author's use of "side project" is referring to "I'm building something outside of my day job that I hope will turn into a business." IF your goal is to make money, then the author's advice is spot on. Get to MVP. Get users. Listen to them. Iterate. All the advice we hear that applies to startups, applies here.

SadWebDeveloper 9y ago

> Then, put it on the bare minimum architecture that it can run on (be that a 512MB instance from DigitalOcean or a medium instance on AWS). Or just test it in a VM and stop wasting money on side-projects.

Walkman 9y ago

I recently was thinking about 1). At my current company, I got to know agile methodologies for the first time, I liked it and started planning my side project, but it instantly started to felt like work! I just realized if I write the parts whatever interests me the most, I will be more inclined to finish it!

cyberferret 9y ago

At risk of a tar and feathering by the 'lean' crowd, I am going to declare that I don't do MVP's at all, and never really have, with any of my projects. Why? Well, because almost all my projects are designed for businesses and enterprises, and with that audience - things better work well and they better work first time during the demo or you don't stand a chance of getting a customer. Perhaps if I was writing a social media app for sharing cat pictures, then yeah, I would slap together an MVP and show it to cat lovers. After all - what is the worst that can happen if your cute cat picture doesn't upload? Just try it again or delete the app and walk away, and there are no consequences. But what if my HR app doesn't send that crucial reminder email to a department manager that one of your electrical tradesmen's license expires tomorrow, and you end up sending him out to a job next week and he makes a mistake that ends up getting your company sued and eventually shut down, putting hundreds of others out of work? I am sure as sh*t going to ensure that my reminder email system is absolutely over engineered to ensure reliable email delivery and tracking. You don't get ANY chances to blow that kind of stuff with a business of enterprise audience.

jamesmishra 9y ago
I fully agree with this. Big enterprise MVPs are often about cutting scope -- not cutting corners. Of course, by the very nature of enterprise customers, sometimes there isn't a whole lot of scope that you can cut.

spott 9y ago

As in everything: figure out what you are trying to get out of your side project. If you want something to generate a nice supplementary income stream, or practice for starting a company when you get "that great idea" then this article is very good advice. On the other hand, if you are trying to learn a new technology stack, develop experience in building scalable architectures or even beefing up your project management skills then this is terrible advice. So, the real art is figuring out what you want out of your side project.

n1vz3r 9y ago

I really like over-engineering my side projects. I have three frameworks, one in PHP, one in JS, and one in PHP/JS (a bit abandoned) that no one really uses. But you know what? Because no one uses them, I'm not burdened by any opinion except mine. When I change something, I don't read death threats because someone's solution stopped working. And since I don't have any deadlines, I can spend as much time as I want to get the exact solution as I planned. I don't need to compromise. That's quite opposite thing to what I do at worktime and get paid for.

treyhuffine 9y ago

If you want to release your project and maybe turn it into a start up, then these are all great points. However, many times the purpose of a side project is to just learn new things.

greggman 9y ago

I feel like there should be a solution that covers 99% of online projects. out of the box it scales, deploys, upgrades, does continuous integration, testing, logging, supports all major logins, has an admin console, metric console, etc... maybe it starts with the traditional Todo app and you just start modding. I find it incredibly frustrating that much of this isn't a solved problem that I can then just insert my side project into. hacking is fun until I have to maintain it and then I start wishing for all of those features instead of having to spend days or weeks implementing each of them from scratch

mmphosis 9y ago

Minimum viable product (MVP) https://en.wikipedia.org/wiki/Minimum_viable_product https://en.wikipedia.org/wiki/Minimum_viable_product

hoodoof 9y ago

I have built projects with a friend in the past. He so aggressively avoids overengineering that he goes the other way - if you make ANY effort to properly engineer something he gets really annoyed and says time is being wasted. It's almost like he feels software MUST be engineered to NOT scale in order to be an "MVP". It's strange because for me, thinking about how something is built means that it can be built right the first time, without extra effort, just by thinking it through.

robertlagrant 9y ago
This is my attitude to the React PATENTS file - yeah okay I probably won't be big enough to worry about it, but just in case I am, why take the risk with React?

p4lindromica 9y ago

One of the ways I learn is via code review. In a side project I'm often doing new things that I am not already an expert in. Any ideas on how to get feedback that things are ok vs not?

kyberias 9y ago

These apply to every project.

real-hacker 9y ago

The definition of a side project is not clear. Is it just for fun/learning about new tech, and can be thrown away? Or is it a serious endeavor, and supposed to be the next big thing? These require completely different mindsets.

dejv 9y ago

All those rules applies to our new non-side projects as well. I can't even count how many times I seen projects with awesome infrastructure and basically zero functionality after depleting whole budget (and usually some more).

elderK 9y ago

I can only speak for myself here but I feel that belonging too strongly in either camp is a bad thing. Ideally, you'd want to understand both worlds. As such, I feel that the projects I work on in my own time and the crazy things I do in them help me to better understand what kind of things really do need to happen when working under tight deadlines in the workplace. I.e. It's easier to produce an effective product quickly if you know from personal experience exactly which approaches or things really are essential. Also, note I haven't read the actual article. I'm replying mostly in response to the comments :)

mehh 9y ago

Love the title of this post. My side projects have all been over engineered, an unsuccessful. They have also helped my career enormously :)