22 ms·

What I wish I knew when I became CTO

519 points by edmack 9y ago

ripberge 9y ago
Enjoyed reading this article, all valid points. However, the one thing that stood out to me in this area was how light I was in effective principles of management and leadership. As a CTO of an organization of more than a handful of people you eventually "get things done" largely via other people rather than being hands on yourself. Had to read a lot of Harvard Business Review to gain the skills and confidence for that. Just like programming, there are indeed tangible skills to learn. It's not just common sense and you're not just born with it.
- redler 9y ago
  It’s funny how, as you progress through a career and gain responsibility, those HBR articles go from seeming like a bunch of Markov chain corporate-speak to being on-target for that exact problem you had last month with the leadership team.
  - derefr 9y ago
    Are you sure they really have any more meaning, or are you just ascribing them meaning that exists more in your "evaluation context" than in the text itself? Compare: the way meditation is usually taught. There is something "there" to communicate, but meditation teachers mostly fail to communicate it. To use an old phrase, they are "pointing at the moon"—but, to stretch the analogy a bit, they're doing this pointing indoors, where the sight-picture you get by following the tangent of their finger does not, in fact, contain a visible moon. You have to imagine taking the thing they're doing (pointing), and reframe it in a context where there is a hypothetical moon to see. Whether that helps you find the moon is more about what you know about the sky and fingers and angles, than it is about how well the meditation teacher can point. And this is why the teachers end up failing to communicate: they did not, themselves, figure out how to "reach enlightenment" by absorbing a verbalized lesson, but rather by pondering a gestalt mess of ideas that have little in the way of words associated—so they can't just turn that gestalt mess back into words. So: are HBR writers pointing at a visible moon, or are their words Markov-chain-speak because they're trying to backwards-chain the gestalt mess of their own mostly wordless understanding into a verbal lesson?
    - darkerside 9y ago
      What is up with the disrespect I constantly hear for wordless understanding? Not everything is best communicated verbally. There's a reason traditional education is often described as a series of falsehoods.
      - derefr 9y ago
        There's nothing wrong with wordless understanding per se; the thing that's "wrong" is thinking that you have words (i.e. a teaching) that can effectively, repeatably communicate a concept, when you actually just have a wordless understanding. The problem of meditation teaching is false positives: people experience enlightenment while pondering some koan, so they think that that koan actually helped, and pass it on. It's superstition. Anything could have helped. Something that truly helps, should help more people than average, more often than chance—and if you've got that, you've got words.
        darkerside 9y ago
        False dichotomy. Understandings aren't completely wordless or wordable. They fall along a scale. > Anything could have helped. If something helped a person, and they want to pass it along, even if it's difficult to communicate in a tangible fashion, I'm not going to stand in their way.
        derefr 9y ago
        Sure, but If I want to learn a difficult-to-communicate lesson, I would hope that the people who have a wordless understanding would keep their communicating to themselves—unless-and-until they come up with some coherent words to match their thoughts, that they can be sure can be used to reconstruct those thoughts without their brain there to help. People don't yet know what they don't know, until they know it—so it can't be the learner's task to preemptively avoid vacuous lessons. That responsibility has to fall to the teacher.
        9y ago
  - michaelchisari 9y ago
    Same goes for philosophy. However, a good writer should be able to convey even the most advanced topics in accessible ways. Often when I see someone relying on jargon and insider language too much, they strike me as a poor writer, regardless of their grasp of the source material.
  - tinymollusk 9y ago
    The risk you run here is internally over-emphasizing events that happened more recently as more important. That could make them more relevant to your recent experience but not necessarily something of more value than the problems you were solving earlier in your career. I have also found increased risk of bikeshedding. The higher you go, the more likely you're working cross-disciplinary with ego-intellects. That also leads to suppressing dissent (hierarchy relationships more than experience-based), leading to worse decisions. Please don't listen to the HBR articles, they're generally very terrible and often can be summed up by survivorship bias.
- mooreds 9y ago
  I loved "High Output Management" for a concrete handbook on a lot of these topics: https://www.goodreads.com/book/show/324750.High_Output_Management https://www.goodreads.com/book/show/324750.High_Output_Manag...
majormajor 9y ago
> Don’t hire someone to do something you’ve not yet figured out (some exceptional candidates can bring new capabilities to companies, but often the most reliable route is for some “founder magic” to re-assemble the company until it can perform the new thing) I'm curious what this "founder magic" bit means. Is this advice largely because of the difficulty of trying to find a qualified expert to bring new capabilities to your company when you personally aren't familiar with that area? E.g., it's hard to not get the wool pulled over your eyes by someone who talks well but can't deliver?
- cardine 9y ago
  Unless you are hiring someone for a VP type role, your new hire likely isn't going to step in and know exactly what they should be doing everyday to achieve the goals you have laid out for them. So you have to try it all out yourself first and figure out what makes someone in this role successful, what makes them not successful, and how to create a process or blueprint that your new hire can follow to success.
  - ramses0 9y ago
    Specifically: in a company of five, don't hire a customer service person if you haven't done customer support yourself at least a bit. Don't hire a database person if you yourself (or somebody internal) hasn't already tried [and presumably failed]. Experience and Failure are important guide-posts to help you look for the right person to fill that role. Where are they better than you? Then you have to mentor them so they get to be better than themselves so they can make your next hire(s).
- stephensonsco 9y ago
  The problem is that a founder who hasn't already filled that role themselves doesn't know two things: 1) what the key parts of doing the job are and 2) which skills/personality the hire must possess. There are many many ways to fail in a position and only a few key parts that matter. The "founder magic" is taking your unique perspective as a domain expert in your business and finding out what the role really needs. You do it by executing in that role for real. After you do that for a while (weeks/months) then you know what will make a candidate successful (and now you have a 50/50 chance of hiring the right person rather than a 10/90).
perfmode 9y ago
Yo, why are you doing BI queries on MySQL?
- benhoyt 9y ago
  I'm no fan of MySQL either, but probably because they were already using MySQL and they had some Business questions they wanted answered Intelligently without setting up a bunch of new infrastructure. Sometimes at a startup you just need to get things done, and fix it later when (if) it becomes a pain point.
  - perfmode 9y ago
    Sounds like they hit a paint point.
  - bpicolo 9y ago
    Definitely the kind of thing you use a dedicated replica for
    - deleted 9y ago
      [deleted]
    - edmack 9y ago
      We had a little MySQL db, and both the data and the different systems consuming it grew quite rapidly, faster than we could get ahead of given company priorities. We have a read-replica for the BI dashboarding system, and this keeps our world relatively stable and reliable.
    - reilly3000 9y ago
      Really not a replica. That is a fine shim for early days but it means you are severely limited to what you can do with reporting based on the data structure prod has. A better pattern is prodDB->Kafka/kinesis streams to a reporting DB like redshift/snowflake/big query. That way you can shape the data however you need, and it lets data teams avoid bogging down engineering.
dustedrob 9y ago
Great article!
hartator 9y ago
> Don’t hire someone to do something you’ve not yet figured out Hum. I would say the reverse. Bring people that are smarter and know more than you.
- gboudrias 9y ago
  There is a balance I guess. You don't need to do all the shooting yourself but you need to know where to point the gun, so to speak.
- Negitivefrags 9y ago
  If you don't know how to do something yourself, you wont even be able to identify someone who is better than you in that field. I've seen people who don't know how to market their product go out and try to hire a marketing guy. You might luck out and get someone perfect for you, but I've never seen it. Usually they just end up wasting a lot of money and learning some hard lessons.
  - eleusive 9y ago
    So how do you hire effectively as a CEO? There are too many areas for you to be knowledgeable, yet you need to be able to hire top talent across a variety of areas.
    - hippich 9y ago
      You hire CTO and let him do his job. How you hire CTO? You learn intimately how to hire for such positions, what their day to day jobs are, etc. Only then you can evaluate canidate for CTO position.
    - mathattack 9y ago
      The method I've found is to ask people I respect a lot, "Who is the best X that you know?" Then call/email them, saying, "I'm the CEO of Y, and I'm trying to find out what a good X looks like. So and so said you're the best she knows. Can I have 30 minutes of your time?" Do 5 of these, and you'll have a good idea of what someone good looks like. (And those 5 may give you some candidates) This is very difficult though because things like "organizes the team to hit quota every quarter" can come in many different manners.
staunch 9y ago
> ...it’s a blessing that my predilection for hipster technologies has not caused any serious problems. It's entirely possible that this was the primary source of his problems with hiring, firing, testing, and a lot more. The technology you choose determines which technologists you attract. And it's not a superficial thing, it actually says everything about the CTO's own technical skill, judgement, and experience.
- sidlls 9y ago
  I was thinking that most of the problems he notes are the result of that litany of tech. How much of that was really necessary or appropriate?
- laythea 9y ago
  Couldn't agree more. It does not seem like a wise thing to do.
mkarazin 9y ago
> I’ve found it a real struggle to get our team to adopt writing tests. I find this hard to believe. Do others CTOs / team leads find this to be the case? I've been a CTO of two small startups with 3-7 developers. We've had resistance to tests at some points (myself included). We've solved it fairly simply. All pull requests (PRs) require tests. PRs are rejected immediately without tests. If a PR doesn't have tests and it is critical to get in, we open a new ticket to track adding tests. It isn't fool proof, but it does result in a high degree of test coverage. And once developers understand how and where to write tests, they usually see the benefit quickly and want to write more tests.
- edmack 9y ago
  I totally agree that requiring PRs to have tests is a good way to solve this - it's what we've adopted after trying a few approaches
- apocalyptic0n3 9y ago
  I'm not a CTO but I do lead the dev team at our agency (was previously 16 devs, but we've slimmed down to 7 currently). I want to preface this by saying that at an agency, your biggest enemy is always time; sales teams have to sell projects for the absolute minimum in order to get a contract, so you can't waste time on non-essentials for most projects. That said, the biggest resistance I have found is "this feature is due in three days, I need two and a half to finish, and then we have another half to review and find bugs." In the end, the biggest issue is that we have time to test on the spot or write tests, but not both. You can scrape by with just manual testing, but I don't think anyone would ever rely on automated tests 100%. Our larger projects are test-backed, and our largest even reaches 90% coverage, but the only reasons we wrote tests for those was because we knew we would be working on them for 2-3 years and it was worth the time in that case. I wish this wasn't the case, but I've found it's always the argument against automated tests in my corner of the market
  - muttech 9y ago
    In my previous agency life, this was something that I experienced as well. A short lived product that was due in less time than any sane dev would estimate. We all knew that we "should" write tests, but there just wasn't time. And in 6 weeks the project would be relegated to living in source control because the campaign was over. It made hiring devs fun. Trying to explain to people why it was that way, and their insistence that software development doesn't work that way.
dumbfounder 9y ago
"Only hire when you feel you’re completely desperate for the role". Maybe for a tiny, extremely lean startup. But for anyone else if you wait until you are desperate you will end up hiring the first person that you think might do the job. That doesn't sound like the right way to go to me. But maybe "desperate" is relative.
- paladin314159 9y ago
  Although I'm not a founder, the rule I espoused in the early days (<20 people) was to not hire for a role unless 50% of a person's time was collectively being spent on that role across the company. This was a great way to be disciplined in determining what was actually a bottleneck for our growth.
thrrr 9y ago
If your engineers don‘t write tests you hired the wrong people. Testing is vital. Make a rule: Every change needs to be tested (you can even set up a pre-commit hook for this. If a class has no test, one has to be written. If tests can not be written easily for a class, it has to be refactored.
- tomnipotent 9y ago
  So basically you've never worked at an early stage start-up.
- k_sh 9y ago
  > If your engineers don‘t write tests you hired the wrong people. Disagree - if your engineers don't write tests, you need to clearly state to them that tests are table stakes, and create an environment conducive to the outcome you want (set up CI, make it fast, set aside time for test-writing hackathons). If your engineers don't want to _follow_ that leadership after it's given, then yeah, you hired the wrong people - but don't demonize employees for not doing something they weren't told they need to do.
  - saalweachter 9y ago
    If your engineers don't write tests, it also probably means that they are not being rewarded for writing tests or punished for not writing tests; indeed, if they are rewarded for doing things that are not writing tests (such as pushing new features) and they can do those things without writing tests, they are being rewarded for not writing tests. Just telling engineers, "write tests" and then promoting the ones that don't is bad leadership: you need to create an environment where the behaviors you desire are the ones that are promoted.
- Clubber 9y ago
  It's a matter of cost. Adding good covering unit tests basically doubles your development time. You are paying now for dividends later. In terms of business, you are trying to prove your business model. If your business model is bad, it doesn't matter how well your software is written. You need to prove your business model before you run out of funds. It's a give and take. You really need to understand both the technical aspects and the business aspects to understand why entities might do certain things. Also, people have been writing software without unit tests for decades.
mberning 9y ago
I have quit jobs because we kept bad hires too long and then didn’t fight to keep good hires from walking away. I think grooming and retaining talent is just as important as providing technical leadership. You need to be strong in both areas.
- joallard 9y ago
  I've seen exactly this in a local well-regarded startup. Incompetent hires with problematic behaviors thriving and being protected, and competent hires being unprotected, not cared about, and almost pushed out. They would hire almost anyone, and then not take active action in maintaining a healthy staff. Needless to say, it's not going very well over there, regardless of the CTO being quite technically proficient.
  - ihsw2 9y ago
    Incompetent PMs can be an issue too, between inaccurate/incomplete feature planning and shoving their responsibilities onto unwitting developers. I'd argue that a great PM is worth as much as the much-vaunted 10x developers, if not worth much more.
zeeshanm 9y ago
I like reading posts like this one. May be they serve as a form of therapy for me that I’m not in this alone. There are others in a similar boat, fighting the good fight, making similar mistakes, and having the same realizations. Very well; now, I can go back to work with my head up high. :)
JarlUlvi 9y ago
CTO.... MySQL... I think not!
cyberferret 9y ago
> "I appreciate now that technologies have a surprisingly short lifespan" This fact alone makes me so glad that I stuck with older tech that has withstood the test of time for our own SaaS. I know that we have users from bleeding edge tech companies sign up for our service, then run away when they glean the 'ancient' tech that it runs on - but then again, I think we have outlasted many other new tech frameworks/languages that have rocketed on high, then fizzled out into obscurity in that same time.
- wu-ikkyu 9y ago
  What was the stack?
  - cyberferret 9y ago
    Front end is basically Bootstrap + jQuery (o_O) Back end is Ruby, but built using Padrino, based on the Sinatra framework instead of Rails. Not exactly 'old' tech there, but not nearly as cool or fast moving as Rails, Go, Rust etc.
latch 9y ago
> I’ve found it a real struggle to get our team to adopt writing tests. If you're struggling to judge the engineering culture of a company that you're considering joining, consider this indicative of a poor one. It isn't definitive, but it's something you should ask about and probe further. Ask to see their CI dashboard and PR comments over the last few days. When they talk about Agile, ask what _engineering_ techniques (not process!) they leverage. These things will tell you if you're joining a GM or a Toyota; a company that sees quality and efficiency as opposing forces, or one that sees them as inseparable. When it comes to tests, there are two types of people: those who know how to write tests, and those who think they're inefficient. If I had to guess what happened here, I'd say: the company had a lack of people who knew how to write effective tests combined with a lack of mentoring. That's why you ask to see recent PR comments and find out if they do pair programming. Because these two things are decisive factors in a good engineering culture.
- doubleocherry 9y ago
  > Ask to see their CI dashboard and PR comments over the last few days This is fantastic advice.
- darkerside 9y ago
  > a company that sees quality and efficiency as opposing forces, or one that sees them as inseparable. I just wanted to say that this was beautifully stated. I've been looking for better words to explain this concept to the people around me.
  - corpMaverick 9y ago
    I imagine it is thinking along this lines... https://totalqualitymanagement.wordpress.com/2008/09/12/cost-of-quality/ https://totalqualitymanagement.wordpress.com/2008/09/12/cost... Definition of cost of Quality It's a term that's widely used – and widely misunderstood. The “cost of quality” isn't the price of creating a quality product or service. It's the cost of NOT creating a quality product or service. Every time work is redone, the cost of quality increases. Obvious examples include: The ...
- 9y ago
mbesto 9y ago
Much of this is summed up to be: CTO positions are much more about technology vision (e.g. choosing frameworks/technologies that can last + serve your needs today and tomorrow) and hiring/retaining talent. Everything else is gravy.
- dpeck 9y ago
  Is the choosing frameworks and technologies really a thing that CTOs do? That seems like more of a tech lead/architect job to choose the right tool for the job. I could see the cto pushing back on those choices from time to time if something is being drastically over engineered but declaring what technology is being used seems like a job far below a cto.
  - NickBusey 9y ago
    This article seems focused towards startups, and the chances of most startups even having a tech lead/architect are pretty slim. So yes. Many CTOs do pretty much 'everything' for quite a while until the funding is there to build a real team.
  - speby 9y ago
    In a word, no. But in startup-land, where the total number of people on the engineering team is, say, less than 10, chances are good that the CTO will also play a lead engineer and/or architect sort of role, in which case they will play a part in designing the architecture, selecting frameworks, and so forth. CTO of, say, US Foods? No, of course not.
    - ecshafer 9y ago
      Why call the role a CTO then? If the role is closer to a tech lead or architect, just call them an architect. This has always confused me in start up land. There will be a full c suite in a company of 10 people, despite that those c suite folks day to day would look nothing like a corporate position. Just call it what it is instead of inflating titles.
akurilin 9y ago
As time goes on, the CTO becomes a pretty flexible position, somewhat analogous to that of a COO. This article was useful for me to figure out the kind of options I had as a CTO, in terms of specializing, as the company got progressively bigger: https://www.linkedin.com/pulse/five-flavors-being-cto-matt-tucker/ https://www.linkedin.com/pulse/five-flavors-being-cto-matt-t... Early on, like OP discovered, you pretty much have to do it all, but you slowly remove yourself away from a lot of those tasks as you find better people to replace you in those areas.
senoroink 9y ago
It's funny you mention that you had difficulty having your team write tests. At my company, the CTO has difficulty writing tests and the team has consistently written adequate test coverage. I fixed this in a new project by starting with jest [1] and failing the CI if the test coverage wasn't at 100%. [1] : https://facebook.github.io/jest/ https://facebook.github.io/jest/
- tomnipotent 9y ago
  > failing the CI if the test coverage wasn't at 100%. This is horrible advice and should never be followed.
  - senoroink 9y ago
    Why? It's not hard to do if you start a fresh project.
    - baconomatic 9y ago
      Just because you have coverage doesn't necessarily mean that you have written good tests. That being said, we do something similar where we require 80% coverage.
      - tehlike 9y ago
        The difference between 80% coverage and 100% coverage is overrated. 80% is more than sufficient, i'd even go ahead and say 70% is better. 100% goes into "change detecting test" territory. There's also the time aspect: going from 0-70 is not hard, 70-100 is extremely time consuming, and often not worth the effort. Monitoring is a way more efficient tool at catching issues.
        baconomatic 9y ago
        While I agree with 70% is about the sweet spot, it really depends on the tools you're using. We've found that with using Jest and just doing snapshots you can get to 70% without actually testing any of your others methods, hence the 80% coverage requirement.
WhitneyLand 9y ago
What always stands out about startup reflections like these is how utterly undefined, freeform, and rapidly evolving the roles can be. The old fire hose saying is true, but it’s not just that you’re drinking from a fire hose, it’s that you often don’t know what’s coming out of the hose next. One minute deep technical decisions, the next minute helping to establish hiring philosophy, and cashflow and growth always on a background thread. After a few years of this I think my experience is not uncommon. If you exit and through whatever circumstance (success or failure), come back inside an F500 company, you realize that trial by fire has force fed you a vast amount of new skills without even realizing it. On one hand, the realization is really empowering, the realization you feel comfortable taking on various high impact tasks without much thought that you could have never jumped right into before. On the other hand, it can feel limiting, because F500 companies tend not to encourage even the most talented technical people to cross roles and help define company wide hiring practices. It’s an invaluable education, but I don’t know if MBA is quite the correct ananlogy, not sure what a better comparison is.
siliconc0w 9y ago
So hiring is pretty hard but I kinda disagree with most of the points there.. * only hire when desperate Strong talent is so hard to get you should probably always be hiring. If you're hiring too many people your bar is probably too low. * only hire to keep up with growth You need to be at least a little preemptive. The hiring process itself can take months, plus the time to train even good new hires is at least a few months, AND you need your most sr. engineers to help interview so that is time they aren't writing features when you're trying to hit that critical milestone. * Don’t hire someone to do something you’ve not yet figured out This is probably also a mistake as software engineering has become pretty specialized. Specialized Frontend, Devops, or Data engineers can bang out solutions even a strong generalist would take ten times longer to even approximate (and most likely anything they build will be throw-away). There is huge low hanging fruit in engineering productivity /business value to getting at least a decent 80% solution for most of these areas that it's worth hiring at least one strong specialist to help Shepard development.
- nine_k 9y ago
  This is good advise when you have a lot of money to spare. Startups sometimes are more strained.
- Oras 9y ago
  In startups, specialised engineer would not necessarily add a good value due to how fast things are changing and mostly lack of proper spec. Specialised engineer can find a good solution with a proper structure which startups lack.
- chatmasta 9y ago
  > Don’t hire someone to do something you’ve not yet figured out I think this is not an indictment of hiring for something you do not know how to do, so much as it is of hiring someone before you have a defined job for them to do. When you’re hiring an engineer, presumably you’ll be placing them onto a team that is responsible for some well-defined part of the stack. So you should know what skills you’re looking for when you’re interviewing. This should make interviewing easier; if you know what capabilities you need a new hire to have, then you know exactly what to test for in new candidates. (This is yet another reason why generic whiteboard interviews make no sense. They’re optimizing for solving problems that could be wholly unrelated to the problems your company faces on a daily basis. I’m surprised more companies do not give interviews that focus more specifically on their relevant problem domains.) If you don’t know what the new hire is going to do when he or she starts work, then you have no idea what skills to measure in the interview, and end up settling for the “least common denominator” of whiteboard coding ability.
tribby 9y ago
> You accept long-term “technical debt” with the adoption of any technology. how long have they been using perl5 over at craiglist?
hota_mazi 9y ago
> I appreciate now that technologies have a surprisingly short lifespan That's pretty much only true in the Javascript ecosystem. Every other areas of the technological stack usually see lifetimes in the decades. > Stepping aside from pure technical decisions, the life-blood of being a CTO is people management Not really, no. That's the job of a CTO at a start up, not at a larger company. I'm not sure the author of the article has actually learned the right lessons from his experience. At the end of the day, CTO of a start up is not really a CTO role in my opinion. It's a technical co founder. You just happened to be the most senior person of the team at a point in time and you inherited a few leadership responsibilities in the process. I've seen a lot of start ups fail because they fail to recognize that fact and didn't realize that after a few years, they needed a different CTO than the co founder, someone who understands that role at scale and the many tasks it implies that are not necessarily relevant to the early years of the company.
fergie 9y ago
> hence why cloud providers can offer $100,000 initial credit Is this a thing? How can my company get $100,000 of AWS on credit?
- joncrane 9y ago
  Probably proof that you have over $1m in seed money.
- lozenge 9y ago
  Apply for AWS Activate.
- GordonS 9y ago
  That kind of offer is generally only available to startups that are in an accelerator program
robinwarren 9y ago
Re: getting your team more interested in testing. This is not an easy thing to get momentum on if people aren't used to it. Yes to getting the test time down (and keeping it down) Also, try defining (maybe in collaboration with the team) the tests you want people to write rather than leaving it up to them or (hopefully not) expecting 100% coverage. I wrote this on my thoughts a while back https://getcorrello.com/blog/2015/11/20/how-much-automated-testing-is-enough-automated-testing/ https://getcorrello.com/blog/2015/11/20/how-much-automated-t... We had some success with increasing testing using that and code review so others could check tests were being written. Still not total buy in to be honest but a big move in the right direction :) One surprising thing was that after years of thinking I was encouraging my team to write tests, the main feedback on why they didn't was that the didn't have time. Making it an explicit part of the process and importantly defining what tests didn't need to be maintained forever really helped.
macca321 9y ago
I just write the gherkin in comments, interleaved with the test code. No messing with regexes
shenli3514 9y ago
>Of the list, AngularJS and MySQL have been the only ones to give us scaling problems. Our monolithic AngularJS code-bundle has got too big and the initial download takes quite a while and the application is a bit too slow. MySQL (in RDS) crashes and restarts due to growing BI query complexity and it’s been hard to fix this. Maybe they should try TiDB(https://github.com/pingcap/tidb https://github.com/pingcap/tidb). It is a MySQL drop-in replacement that scales.
mychael 9y ago
Why are so many self-appointed startup CTOs so anxious to share startup advice?
orginal__idear 9y ago
I recently interviewed with these guys. Was not impressed.

519 points by edmack 9y ago

Enjoyed reading this article, all valid points. However, the one thing that stood out to me in this area was how light I was in effective principles of management and leadership. As a CTO of an organization of more than a handful of people you eventually "get things done" largely via other people rather than being hands on yourself. Had to read a lot of Harvard Business Review to gain the skills and confidence for that. Just like programming, there are indeed tangible skills to learn. It's not just common sense and you're not just born with it.

redler 9y ago
It’s funny how, as you progress through a career and gain responsibility, those HBR articles go from seeming like a bunch of Markov chain corporate-speak to being on-target for that exact problem you had last month with the leadership team.
- derefr 9y ago
  Are you sure they really have any more meaning, or are you just ascribing them meaning that exists more in your "evaluation context" than in the text itself? Compare: the way meditation is usually taught. There is something "there" to communicate, but meditation teachers mostly fail to communicate it. To use an old phrase, they are "pointing at the moon"—but, to stretch the analogy a bit, they're doing this pointing indoors, where the sight-picture you get by following the tangent of their finger does not, in fact, contain a visible moon. You have to imagine taking the thing they're doing (pointing), and reframe it in a context where there is a hypothetical moon to see. Whether that helps you find the moon is more about what you know about the sky and fingers and angles, than it is about how well the meditation teacher can point. And this is why the teachers end up failing to communicate: they did not, themselves, figure out how to "reach enlightenment" by absorbing a verbalized lesson, but rather by pondering a gestalt mess of ideas that have little in the way of words associated—so they can't just turn that gestalt mess back into words. So: are HBR writers pointing at a visible moon, or are their words Markov-chain-speak because they're trying to backwards-chain the gestalt mess of their own mostly wordless understanding into a verbal lesson?
  - darkerside 9y ago
    What is up with the disrespect I constantly hear for wordless understanding? Not everything is best communicated verbally. There's a reason traditional education is often described as a series of falsehoods.
    - derefr 9y ago
      There's nothing wrong with wordless understanding per se; the thing that's "wrong" is thinking that you have words (i.e. a teaching) that can effectively, repeatably communicate a concept, when you actually just have a wordless understanding. The problem of meditation teaching is false positives: people experience enlightenment while pondering some koan, so they think that that koan actually helped, and pass it on. It's superstition. Anything could have helped. Something that truly helps, should help more people than average, more often than chance—and if you've got that, you've got words.
      - darkerside 9y ago
        False dichotomy. Understandings aren't completely wordless or wordable. They fall along a scale. > Anything could have helped. If something helped a person, and they want to pass it along, even if it's difficult to communicate in a tangible fashion, I'm not going to stand in their way.
        derefr 9y ago
        Sure, but If I want to learn a difficult-to-communicate lesson, I would hope that the people who have a wordless understanding would keep their communicating to themselves—unless-and-until they come up with some coherent words to match their thoughts, that they can be sure can be used to reconstruct those thoughts without their brain there to help. People don't yet know what they don't know, until they know it—so it can't be the learner's task to preemptively avoid vacuous lessons. That responsibility has to fall to the teacher.
        9y ago
- michaelchisari 9y ago
  Same goes for philosophy. However, a good writer should be able to convey even the most advanced topics in accessible ways. Often when I see someone relying on jargon and insider language too much, they strike me as a poor writer, regardless of their grasp of the source material.
- tinymollusk 9y ago
  The risk you run here is internally over-emphasizing events that happened more recently as more important. That could make them more relevant to your recent experience but not necessarily something of more value than the problems you were solving earlier in your career. I have also found increased risk of bikeshedding. The higher you go, the more likely you're working cross-disciplinary with ego-intellects. That also leads to suppressing dissent (hierarchy relationships more than experience-based), leading to worse decisions. Please don't listen to the HBR articles, they're generally very terrible and often can be summed up by survivorship bias.
mooreds 9y ago
I loved "High Output Management" for a concrete handbook on a lot of these topics: https://www.goodreads.com/book/show/324750.High_Output_Management https://www.goodreads.com/book/show/324750.High_Output_Manag...

majormajor 9y ago

> Don’t hire someone to do something you’ve not yet figured out (some exceptional candidates can bring new capabilities to companies, but often the most reliable route is for some “founder magic” to re-assemble the company until it can perform the new thing) I'm curious what this "founder magic" bit means. Is this advice largely because of the difficulty of trying to find a qualified expert to bring new capabilities to your company when you personally aren't familiar with that area? E.g., it's hard to not get the wool pulled over your eyes by someone who talks well but can't deliver?

cardine 9y ago
Unless you are hiring someone for a VP type role, your new hire likely isn't going to step in and know exactly what they should be doing everyday to achieve the goals you have laid out for them. So you have to try it all out yourself first and figure out what makes someone in this role successful, what makes them not successful, and how to create a process or blueprint that your new hire can follow to success.
- ramses0 9y ago
  Specifically: in a company of five, don't hire a customer service person if you haven't done customer support yourself at least a bit. Don't hire a database person if you yourself (or somebody internal) hasn't already tried [and presumably failed]. Experience and Failure are important guide-posts to help you look for the right person to fill that role. Where are they better than you? Then you have to mentor them so they get to be better than themselves so they can make your next hire(s).
stephensonsco 9y ago
The problem is that a founder who hasn't already filled that role themselves doesn't know two things: 1) what the key parts of doing the job are and 2) which skills/personality the hire must possess. There are many many ways to fail in a position and only a few key parts that matter. The "founder magic" is taking your unique perspective as a domain expert in your business and finding out what the role really needs. You do it by executing in that role for real. After you do that for a while (weeks/months) then you know what will make a candidate successful (and now you have a 50/50 chance of hiring the right person rather than a 10/90).

perfmode 9y ago

Yo, why are you doing BI queries on MySQL?

benhoyt 9y ago
I'm no fan of MySQL either, but probably because they were already using MySQL and they had some Business questions they wanted answered Intelligently without setting up a bunch of new infrastructure. Sometimes at a startup you just need to get things done, and fix it later when (if) it becomes a pain point.
- perfmode 9y ago
  Sounds like they hit a paint point.
- bpicolo 9y ago
  Definitely the kind of thing you use a dedicated replica for
  - deleted 9y ago
    [deleted]
  - edmack 9y ago
    We had a little MySQL db, and both the data and the different systems consuming it grew quite rapidly, faster than we could get ahead of given company priorities. We have a read-replica for the BI dashboarding system, and this keeps our world relatively stable and reliable.
  - reilly3000 9y ago
    Really not a replica. That is a fine shim for early days but it means you are severely limited to what you can do with reporting based on the data structure prod has. A better pattern is prodDB->Kafka/kinesis streams to a reporting DB like redshift/snowflake/big query. That way you can shape the data however you need, and it lets data teams avoid bogging down engineering.

dustedrob 9y ago

Great article!

hartator 9y ago

> Don’t hire someone to do something you’ve not yet figured out Hum. I would say the reverse. Bring people that are smarter and know more than you.

gboudrias 9y ago
There is a balance I guess. You don't need to do all the shooting yourself but you need to know where to point the gun, so to speak.
Negitivefrags 9y ago
If you don't know how to do something yourself, you wont even be able to identify someone who is better than you in that field. I've seen people who don't know how to market their product go out and try to hire a marketing guy. You might luck out and get someone perfect for you, but I've never seen it. Usually they just end up wasting a lot of money and learning some hard lessons.
- eleusive 9y ago
  So how do you hire effectively as a CEO? There are too many areas for you to be knowledgeable, yet you need to be able to hire top talent across a variety of areas.
  - hippich 9y ago
    You hire CTO and let him do his job. How you hire CTO? You learn intimately how to hire for such positions, what their day to day jobs are, etc. Only then you can evaluate canidate for CTO position.
  - mathattack 9y ago
    The method I've found is to ask people I respect a lot, "Who is the best X that you know?" Then call/email them, saying, "I'm the CEO of Y, and I'm trying to find out what a good X looks like. So and so said you're the best she knows. Can I have 30 minutes of your time?" Do 5 of these, and you'll have a good idea of what someone good looks like. (And those 5 may give you some candidates) This is very difficult though because things like "organizes the team to hit quota every quarter" can come in many different manners.

staunch 9y ago

> ...it’s a blessing that my predilection for hipster technologies has not caused any serious problems. It's entirely possible that this was the primary source of his problems with hiring, firing, testing, and a lot more. The technology you choose determines which technologists you attract. And it's not a superficial thing, it actually says everything about the CTO's own technical skill, judgement, and experience.

sidlls 9y ago
I was thinking that most of the problems he notes are the result of that litany of tech. How much of that was really necessary or appropriate?
laythea 9y ago
Couldn't agree more. It does not seem like a wise thing to do.

mkarazin 9y ago

> I’ve found it a real struggle to get our team to adopt writing tests. I find this hard to believe. Do others CTOs / team leads find this to be the case? I've been a CTO of two small startups with 3-7 developers. We've had resistance to tests at some points (myself included). We've solved it fairly simply. All pull requests (PRs) require tests. PRs are rejected immediately without tests. If a PR doesn't have tests and it is critical to get in, we open a new ticket to track adding tests. It isn't fool proof, but it does result in a high degree of test coverage. And once developers understand how and where to write tests, they usually see the benefit quickly and want to write more tests.

edmack 9y ago
I totally agree that requiring PRs to have tests is a good way to solve this - it's what we've adopted after trying a few approaches
apocalyptic0n3 9y ago
I'm not a CTO but I do lead the dev team at our agency (was previously 16 devs, but we've slimmed down to 7 currently). I want to preface this by saying that at an agency, your biggest enemy is always time; sales teams have to sell projects for the absolute minimum in order to get a contract, so you can't waste time on non-essentials for most projects. That said, the biggest resistance I have found is "this feature is due in three days, I need two and a half to finish, and then we have another half to review and find bugs." In the end, the biggest issue is that we have time to test on the spot or write tests, but not both. You can scrape by with just manual testing, but I don't think anyone would ever rely on automated tests 100%. Our larger projects are test-backed, and our largest even reaches 90% coverage, but the only reasons we wrote tests for those was because we knew we would be working on them for 2-3 years and it was worth the time in that case. I wish this wasn't the case, but I've found it's always the argument against automated tests in my corner of the market
- muttech 9y ago
  In my previous agency life, this was something that I experienced as well. A short lived product that was due in less time than any sane dev would estimate. We all knew that we "should" write tests, but there just wasn't time. And in 6 weeks the project would be relegated to living in source control because the campaign was over. It made hiring devs fun. Trying to explain to people why it was that way, and their insistence that software development doesn't work that way.

dumbfounder 9y ago

"Only hire when you feel you’re completely desperate for the role". Maybe for a tiny, extremely lean startup. But for anyone else if you wait until you are desperate you will end up hiring the first person that you think might do the job. That doesn't sound like the right way to go to me. But maybe "desperate" is relative.

paladin314159 9y ago
Although I'm not a founder, the rule I espoused in the early days (<20 people) was to not hire for a role unless 50% of a person's time was collectively being spent on that role across the company. This was a great way to be disciplined in determining what was actually a bottleneck for our growth.

thrrr 9y ago

If your engineers don‘t write tests you hired the wrong people. Testing is vital. Make a rule: Every change needs to be tested (you can even set up a pre-commit hook for this. If a class has no test, one has to be written. If tests can not be written easily for a class, it has to be refactored.

tomnipotent 9y ago
So basically you've never worked at an early stage start-up.
k_sh 9y ago
> If your engineers don‘t write tests you hired the wrong people. Disagree - if your engineers don't write tests, you need to clearly state to them that tests are table stakes, and create an environment conducive to the outcome you want (set up CI, make it fast, set aside time for test-writing hackathons). If your engineers don't want to _follow_ that leadership after it's given, then yeah, you hired the wrong people - but don't demonize employees for not doing something they weren't told they need to do.
- saalweachter 9y ago
  If your engineers don't write tests, it also probably means that they are not being rewarded for writing tests or punished for not writing tests; indeed, if they are rewarded for doing things that are not writing tests (such as pushing new features) and they can do those things without writing tests, they are being rewarded for not writing tests. Just telling engineers, "write tests" and then promoting the ones that don't is bad leadership: you need to create an environment where the behaviors you desire are the ones that are promoted.
Clubber 9y ago
It's a matter of cost. Adding good covering unit tests basically doubles your development time. You are paying now for dividends later. In terms of business, you are trying to prove your business model. If your business model is bad, it doesn't matter how well your software is written. You need to prove your business model before you run out of funds. It's a give and take. You really need to understand both the technical aspects and the business aspects to understand why entities might do certain things. Also, people have been writing software without unit tests for decades.

mberning 9y ago

I have quit jobs because we kept bad hires too long and then didn’t fight to keep good hires from walking away. I think grooming and retaining talent is just as important as providing technical leadership. You need to be strong in both areas.

joallard 9y ago
I've seen exactly this in a local well-regarded startup. Incompetent hires with problematic behaviors thriving and being protected, and competent hires being unprotected, not cared about, and almost pushed out. They would hire almost anyone, and then not take active action in maintaining a healthy staff. Needless to say, it's not going very well over there, regardless of the CTO being quite technically proficient.
- ihsw2 9y ago
  Incompetent PMs can be an issue too, between inaccurate/incomplete feature planning and shoving their responsibilities onto unwitting developers. I'd argue that a great PM is worth as much as the much-vaunted 10x developers, if not worth much more.

zeeshanm 9y ago

I like reading posts like this one. May be they serve as a form of therapy for me that I’m not in this alone. There are others in a similar boat, fighting the good fight, making similar mistakes, and having the same realizations. Very well; now, I can go back to work with my head up high. :)

JarlUlvi 9y ago

CTO.... MySQL... I think not!

cyberferret 9y ago

> "I appreciate now that technologies have a surprisingly short lifespan" This fact alone makes me so glad that I stuck with older tech that has withstood the test of time for our own SaaS. I know that we have users from bleeding edge tech companies sign up for our service, then run away when they glean the 'ancient' tech that it runs on - but then again, I think we have outlasted many other new tech frameworks/languages that have rocketed on high, then fizzled out into obscurity in that same time.

wu-ikkyu 9y ago
What was the stack?
- cyberferret 9y ago
  Front end is basically Bootstrap + jQuery (o_O) Back end is Ruby, but built using Padrino, based on the Sinatra framework instead of Rails. Not exactly 'old' tech there, but not nearly as cool or fast moving as Rails, Go, Rust etc.

latch 9y ago

> I’ve found it a real struggle to get our team to adopt writing tests. If you're struggling to judge the engineering culture of a company that you're considering joining, consider this indicative of a poor one. It isn't definitive, but it's something you should ask about and probe further. Ask to see their CI dashboard and PR comments over the last few days. When they talk about Agile, ask what _engineering_ techniques (not process!) they leverage. These things will tell you if you're joining a GM or a Toyota; a company that sees quality and efficiency as opposing forces, or one that sees them as inseparable. When it comes to tests, there are two types of people: those who know how to write tests, and those who think they're inefficient. If I had to guess what happened here, I'd say: the company had a lack of people who knew how to write effective tests combined with a lack of mentoring. That's why you ask to see recent PR comments and find out if they do pair programming. Because these two things are decisive factors in a good engineering culture.

doubleocherry 9y ago
> Ask to see their CI dashboard and PR comments over the last few days This is fantastic advice.
darkerside 9y ago
> a company that sees quality and efficiency as opposing forces, or one that sees them as inseparable. I just wanted to say that this was beautifully stated. I've been looking for better words to explain this concept to the people around me.
- corpMaverick 9y ago
  I imagine it is thinking along this lines... https://totalqualitymanagement.wordpress.com/2008/09/12/cost-of-quality/ https://totalqualitymanagement.wordpress.com/2008/09/12/cost... Definition of cost of Quality It's a term that's widely used – and widely misunderstood. The “cost of quality” isn't the price of creating a quality product or service. It's the cost of NOT creating a quality product or service. Every time work is redone, the cost of quality increases. Obvious examples include: The ...
9y ago

mbesto 9y ago

Much of this is summed up to be: CTO positions are much more about technology vision (e.g. choosing frameworks/technologies that can last + serve your needs today and tomorrow) and hiring/retaining talent. Everything else is gravy.

dpeck 9y ago
Is the choosing frameworks and technologies really a thing that CTOs do? That seems like more of a tech lead/architect job to choose the right tool for the job. I could see the cto pushing back on those choices from time to time if something is being drastically over engineered but declaring what technology is being used seems like a job far below a cto.
- NickBusey 9y ago
  This article seems focused towards startups, and the chances of most startups even having a tech lead/architect are pretty slim. So yes. Many CTOs do pretty much 'everything' for quite a while until the funding is there to build a real team.
- speby 9y ago
  In a word, no. But in startup-land, where the total number of people on the engineering team is, say, less than 10, chances are good that the CTO will also play a lead engineer and/or architect sort of role, in which case they will play a part in designing the architecture, selecting frameworks, and so forth. CTO of, say, US Foods? No, of course not.
  - ecshafer 9y ago
    Why call the role a CTO then? If the role is closer to a tech lead or architect, just call them an architect. This has always confused me in start up land. There will be a full c suite in a company of 10 people, despite that those c suite folks day to day would look nothing like a corporate position. Just call it what it is instead of inflating titles.

akurilin 9y ago

As time goes on, the CTO becomes a pretty flexible position, somewhat analogous to that of a COO. This article was useful for me to figure out the kind of options I had as a CTO, in terms of specializing, as the company got progressively bigger: https://www.linkedin.com/pulse/five-flavors-being-cto-matt-tucker/ https://www.linkedin.com/pulse/five-flavors-being-cto-matt-t... Early on, like OP discovered, you pretty much have to do it all, but you slowly remove yourself away from a lot of those tasks as you find better people to replace you in those areas.

senoroink 9y ago

It's funny you mention that you had difficulty having your team write tests. At my company, the CTO has difficulty writing tests and the team has consistently written adequate test coverage. I fixed this in a new project by starting with jest [1] and failing the CI if the test coverage wasn't at 100%. [1] : https://facebook.github.io/jest/ https://facebook.github.io/jest/

tomnipotent 9y ago
> failing the CI if the test coverage wasn't at 100%. This is horrible advice and should never be followed.
- senoroink 9y ago
  Why? It's not hard to do if you start a fresh project.
  - baconomatic 9y ago
    Just because you have coverage doesn't necessarily mean that you have written good tests. That being said, we do something similar where we require 80% coverage.
    - tehlike 9y ago
      The difference between 80% coverage and 100% coverage is overrated. 80% is more than sufficient, i'd even go ahead and say 70% is better. 100% goes into "change detecting test" territory. There's also the time aspect: going from 0-70 is not hard, 70-100 is extremely time consuming, and often not worth the effort. Monitoring is a way more efficient tool at catching issues.
      - baconomatic 9y ago
        While I agree with 70% is about the sweet spot, it really depends on the tools you're using. We've found that with using Jest and just doing snapshots you can get to 70% without actually testing any of your others methods, hence the 80% coverage requirement.

WhitneyLand 9y ago

What always stands out about startup reflections like these is how utterly undefined, freeform, and rapidly evolving the roles can be. The old fire hose saying is true, but it’s not just that you’re drinking from a fire hose, it’s that you often don’t know what’s coming out of the hose next. One minute deep technical decisions, the next minute helping to establish hiring philosophy, and cashflow and growth always on a background thread. After a few years of this I think my experience is not uncommon. If you exit and through whatever circumstance (success or failure), come back inside an F500 company, you realize that trial by fire has force fed you a vast amount of new skills without even realizing it. On one hand, the realization is really empowering, the realization you feel comfortable taking on various high impact tasks without much thought that you could have never jumped right into before. On the other hand, it can feel limiting, because F500 companies tend not to encourage even the most talented technical people to cross roles and help define company wide hiring practices. It’s an invaluable education, but I don’t know if MBA is quite the correct ananlogy, not sure what a better comparison is.

siliconc0w 9y ago

So hiring is pretty hard but I kinda disagree with most of the points there.. * only hire when desperate Strong talent is so hard to get you should probably always be hiring. If you're hiring too many people your bar is probably too low. * only hire to keep up with growth You need to be at least a little preemptive. The hiring process itself can take months, plus the time to train even good new hires is at least a few months, AND you need your most sr. engineers to help interview so that is time they aren't writing features when you're trying to hit that critical milestone. * Don’t hire someone to do something you’ve not yet figured out This is probably also a mistake as software engineering has become pretty specialized. Specialized Frontend, Devops, or Data engineers can bang out solutions even a strong generalist would take ten times longer to even approximate (and most likely anything they build will be throw-away). There is huge low hanging fruit in engineering productivity /business value to getting at least a decent 80% solution for most of these areas that it's worth hiring at least one strong specialist to help Shepard development.

nine_k 9y ago
This is good advise when you have a lot of money to spare. Startups sometimes are more strained.
Oras 9y ago
In startups, specialised engineer would not necessarily add a good value due to how fast things are changing and mostly lack of proper spec. Specialised engineer can find a good solution with a proper structure which startups lack.
chatmasta 9y ago
> Don’t hire someone to do something you’ve not yet figured out I think this is not an indictment of hiring for something you do not know how to do, so much as it is of hiring someone before you have a defined job for them to do. When you’re hiring an engineer, presumably you’ll be placing them onto a team that is responsible for some well-defined part of the stack. So you should know what skills you’re looking for when you’re interviewing. This should make interviewing easier; if you know what capabilities you need a new hire to have, then you know exactly what to test for in new candidates. (This is yet another reason why generic whiteboard interviews make no sense. They’re optimizing for solving problems that could be wholly unrelated to the problems your company faces on a daily basis. I’m surprised more companies do not give interviews that focus more specifically on their relevant problem domains.) If you don’t know what the new hire is going to do when he or she starts work, then you have no idea what skills to measure in the interview, and end up settling for the “least common denominator” of whiteboard coding ability.

tribby 9y ago

> You accept long-term “technical debt” with the adoption of any technology. how long have they been using perl5 over at craiglist?

hota_mazi 9y ago

> I appreciate now that technologies have a surprisingly short lifespan That's pretty much only true in the Javascript ecosystem. Every other areas of the technological stack usually see lifetimes in the decades. > Stepping aside from pure technical decisions, the life-blood of being a CTO is people management Not really, no. That's the job of a CTO at a start up, not at a larger company. I'm not sure the author of the article has actually learned the right lessons from his experience. At the end of the day, CTO of a start up is not really a CTO role in my opinion. It's a technical co founder. You just happened to be the most senior person of the team at a point in time and you inherited a few leadership responsibilities in the process. I've seen a lot of start ups fail because they fail to recognize that fact and didn't realize that after a few years, they needed a different CTO than the co founder, someone who understands that role at scale and the many tasks it implies that are not necessarily relevant to the early years of the company.

fergie 9y ago

> hence why cloud providers can offer $100,000 initial credit Is this a thing? How can my company get $100,000 of AWS on credit?

joncrane 9y ago
Probably proof that you have over $1m in seed money.
lozenge 9y ago
Apply for AWS Activate.
GordonS 9y ago
That kind of offer is generally only available to startups that are in an accelerator program

robinwarren 9y ago

Re: getting your team more interested in testing. This is not an easy thing to get momentum on if people aren't used to it. Yes to getting the test time down (and keeping it down) Also, try defining (maybe in collaboration with the team) the tests you want people to write rather than leaving it up to them or (hopefully not) expecting 100% coverage. I wrote this on my thoughts a while back https://getcorrello.com/blog/2015/11/20/how-much-automated-testing-is-enough-automated-testing/ https://getcorrello.com/blog/2015/11/20/how-much-automated-t... We had some success with increasing testing using that and code review so others could check tests were being written. Still not total buy in to be honest but a big move in the right direction :) One surprising thing was that after years of thinking I was encouraging my team to write tests, the main feedback on why they didn't was that the didn't have time. Making it an explicit part of the process and importantly defining what tests didn't need to be maintained forever really helped.

macca321 9y ago

I just write the gherkin in comments, interleaved with the test code. No messing with regexes

shenli3514 9y ago

>Of the list, AngularJS and MySQL have been the only ones to give us scaling problems. Our monolithic AngularJS code-bundle has got too big and the initial download takes quite a while and the application is a bit too slow. MySQL (in RDS) crashes and restarts due to growing BI query complexity and it’s been hard to fix this. Maybe they should try TiDB(https://github.com/pingcap/tidb https://github.com/pingcap/tidb). It is a MySQL drop-in replacement that scales.

mychael 9y ago

Why are so many self-appointed startup CTOs so anxious to share startup advice?

orginal__idear 9y ago

I recently interviewed with these guys. Was not impressed.