7 ms·

The Tools We Use To Stay Afloat

113 points by mxstbr 9y ago

cpr 9y ago
Matches more or less what we do as a 4-man team. I'm curious why they used appear.in instead of Slack's audio/video conferencing? The latter is too new?
- badosu 9y ago
  > It’s quite reliable and even offers screen sharing without any plugins on Firefox. Since they bothered writing this, I guess the lack of support for Firefox.
- evadne 9y ago
  Slack’s audio/video conferencing feature sometimes double-connects an user and frequently connects without sending audio traffic either way. My team has been using ScreenHero prior to Slack buying them out. ScreenHero also supports audio calls without screen-sharing, and its audio calls have not been problematic for us.
- anselmh 9y ago
  It’s a combination of a few things. First, when we started doing calls together Slack calls weren’t available yet for multiple people. In our experience, Slack calls often fail for a variety of reasons. Other than that, we don’t want to shift too much data to Slack and since it’s pretty unclear how their calling technology works and appear.in uses WebRTC, we rather use another service here than a service where we don’t know much about it but only that calls aren’t too reliable. But as said, we haven’t bothered to check the group-video calling yet there. We’re pragmatic. Appear.in works in any browser (Slack calls don’t) and therefore… why should we change a well working system?
kingbirdy 9y ago
The branch deploy sounds pretty interesting, I wish there was more detail on it than just a footnote
- e1g 9y ago
  We do the same - it was surprisingly simple to implement - - CircleCI listens across all GIT branches and injects $CIRCLE_BRANCH into ENV - A build step populates a vhost template to point "$CIRCLE_BRANCH.qa.bofh.com" to "/var/www/$CIRCLE_BRANCH/current" - Another build step creates "manifest.json" with $CIRCLE_BRANCH and $CIRCLE_BUILD_NUM - The vhost config and the manifest are added to the release tarball/Docker image - The tarball/image gets shipped to the staging server - (we ship to S3/Docker, then pull because of security concerns. But can ship inside the build process) - In staging, a script reads manifest.json and moves content to /var/www/$CIRCLE_BRANCH/$CIRCLE_BRANCH - The provided nginx vhost is copied into a standard dir with other vhosts - A symlink of "/var/www/$CIRCLE_BRANCH/current" is forced to point to the fresh release - nginx -s reload && profit You can use a wildcard SSL to cover subdomains or add a step to trigger LetsEncrypt certbot as required. This method works equally well with Docker images - just requires an interim step to launch the container. In staging, we use a naming convention to launch on predictable ports for every app. First 2 numbers are app specific, second 3 numbers are build-specific. For example, "25$CIRCLE_BUILD_NUM" (where $CIRCLE_BUILD_NUM is just the last 3 numbers of the build).
  - vmasto 9y ago
    How do you handle the database? Do you reuse the one on staging or create a new instance with prepopulated data? If the former how do you deal with migrations and schema edits?
    - e1g 9y ago
      Yes, the db has been the most challenging aspect for us. We have 3 situations - 1. "Common baseline". With a relatively stable product, most branches (as in ~51%) do not impact the schema. For testing / QA purposes, these share one central QA db and pollute each out. Turns out, a lot of the times this is quite ok because the PR is about how the data is displayed, or improved logging, or UX change, or a security layer or anything else other than core domain knowledge - they don't care for the data that much. 2. "I'm special". Some branches do modify data (whether the format or the structure). To handle these, the manifest.json file has an option to request a separate database. If present, the rollout script will do "pg_dump + copy" of the shared staging DB, and duplicate it into "qa_$BRANCH", then update the config file (or .env for Docker) with the appropriate connection value. Additionally, it will all *sql files in a dir specified in manifest.json against the clone DB. This is done on every release, which does get annoying by resetting the qa data (we could add another manifest switch here I). On the upside, it forces you to codify all data migration rules from the start. 3. "I am very special". Some changes transform data in a way that requires business processing and cannot be done with easy SQL. Sorry, out of luck - we don't automate special cases yet. The developer has to pull the QA database to localhost, do his magic, and push it back. Not ideal, but hasn't caused any problems yet. If ain't broke...
hgdsraj 9y ago
Why don't you guys use pivotal tracker? Way better ticket management, integrates with github
- bdcravens 9y ago
  We've use Pivotal Tracker before, and it always felt challenging to have more than a basic description/discussion (maybe that's the point), but most tools like Jira, Github Issues, Trello, etc, facilitate a larger view.
  - wpietri 9y ago
    It is definitely the point. One of the basic notions in the school of thought that Tracker comes out of is that team interaction is valuable and to be encouraged. So making a tool that makes it easy for people to not talk undermines the broader goal. One of the basic insights of the Agile movement (R.I.P.) about Waterfall is that processes structured around documentation rather than collaboration have a lot of subtle bad effects that gradually destroy characteristics you'd like your teams to have. E.g., responsiveness to change, systemic effectiveness, resilience to failure, low overhead, ability to ship frequently, ability to deliver customer value. That's why when I set things up I generally drive things off of index cards. [1] Those are obviously insufficient, which forces people to discuss and collaborate. (That's in contrast to more voluminous documentation, which is subtly insufficient.) [1] E.g.: http://williampietri.com/writing/2015/the-big-board/ http://williampietri.com/writing/2015/the-big-board/
  - jacques_chester 9y ago
    Tracker is very opinionated and is deliberately trying to not be the be-all and end-all for software projects. It's really closely tuned to how Pivotal works: XP with Lean trimmings in small teams with a product manager, designer and engineers. The downside is that because it is deliberately limited, now and then you will find that you sorely miss something. Often those missing features are recreated with conventions around tagging or release markers. But tagging conventions are not the same as a first-class feature, for good or ill. JIRA is massively more featuresome and flexible. These days it's grown into a workflow middleware which comes with a bug tracker as the first application installed. For some organisations that will make more sense. Disclosure: I work for Pivotal, but not on Tracker.
dbg31415 9y ago
ZenHub is free for teams under 5 people. And it's great. * ZenHub - Agile GitHub Project Management || https://www.zenhub.com/ https://www.zenhub.com/ Also I use a tool that keeps labels in sync across GitHub repos. * github-label-sync || https://www.npmjs.com/package/github-label-sync https://www.npmjs.com/package/github-label-sync Harvest for time tracking. * Simple Online Time Tracking Software - Harvest || https://www.getharvest.com/ https://www.getharvest.com/ Red Pen for annotations. (Does't integrate with anything and that pisses me off -- how freakin' hard would it be to build web hooks so it could tie into Slack?! But on the whole it's got an easy to use interface.) * Red Pen || https://redpen.io/ https://redpen.io/ While I like Slack a lot, one client I have uses Discord... and it's not bad. * Discord - Free Voice and Text Chat for Gamers || https://discordapp.com/ https://discordapp.com/
- anselmh 9y ago
  Thanks for the list, it’s quite good! However, our point is to not use third-parties for things where we don’t need it. The point of the article is to show that you can use Github solely for project management without a third-party service. But I completely agree with you: If you don’t like the little extra-work we do to achieve that with just github.com, it’s probably right to add another tool like ZenHub or waffle.io on top of it. Making your toolset work for your team is the most important thing. For us, it’s enough — and maybe for a few others as well. That’s why we shared our approach :)
  - dbg31415 9y ago
    Gonna rant a bit... GitHub boards cost time. No one should use them. They're just an inferior option to ZenHub (or even Waffle or Asana or any number of other "board" interfaces you can tack on to GitHub). I've wasted my team's time on GitHub boards... everyone quickly demanded we go back to ZenHub. Most projects have multiple repositories, right? But GitHub boards have it so that each board is based on one repo. Stupid. You want to see a project view of all your repos at once... front-end, back-end, deployment, whatever... not 4 or 5 separate views on it. I feel like GitHub REALLY dropped the ball on not having a board forever, then putting out such a "beta-feeling" board. They should have just bought ZenHub -- still time... but literally any tool out there makes the boards work better than default GitHub. ZenHub is free for small teams, and I'd argue that anyone can afford $5 / month / user. Budget $150 / month / user for any team for tools -- seems about right.
awinter-py 9y ago
I wish this were an article about some hot new documentation tool instead of another misguided psalm to slack & kanban. Making docs sexy would be a big quality of life boost for programmers and the people who love them (i.e. project managers and users of software).
- hobofan 9y ago
  Best tip for docs in small teams: Have a server running in the local network that pulls the latest develop branch, renders it and serves it on the network for each project. For microservices I would recommend API Blueprint, for normal libraries, whatever is the language standard. Really cuts down the friction of getting to the docs, since you don't have to do all those steps on you local machine anymore whenever you need to access the documentation of an internal project. Next best thing would be building Dash docfeeds for internal projects.
  - awinter-py 9y ago
    hadn't heard of API blueprint -- pretty cool. in addition to tools for documenting interfaces (i.e. APIs), I'm interested in tools that document behavior (both desired & observed) and integrate it back to codebases. In type safety terms, this is like the difference between function signatures that verify inputs & outputs vs something like coq/TLA that can verify more advanced properties of your program. Configuration is particularly difficult to document in 2017 -- if you solve that problem you fix a lot of migraines.
    - hobofan 9y ago
      > I'm interested in tools that document behavior (both desired & observed) and integrate it back to codebases. > Configuration is particularly difficult to document in 2017 -- if you solve that problem you fix a lot of migraines. Could you expand on those points a bit more? As for the first one, it sounds vaguely like https://deckard.ai https://deckard.ai, but I feel you could also mean something entirely different. (Disclamier: I know the founders of Deckard very well.)

113 points by mxstbr 9y ago

cpr 9y ago

Matches more or less what we do as a 4-man team. I'm curious why they used appear.in instead of Slack's audio/video conferencing? The latter is too new?

badosu 9y ago
> It’s quite reliable and even offers screen sharing without any plugins on Firefox. Since they bothered writing this, I guess the lack of support for Firefox.
evadne 9y ago
Slack’s audio/video conferencing feature sometimes double-connects an user and frequently connects without sending audio traffic either way. My team has been using ScreenHero prior to Slack buying them out. ScreenHero also supports audio calls without screen-sharing, and its audio calls have not been problematic for us.
anselmh 9y ago
It’s a combination of a few things. First, when we started doing calls together Slack calls weren’t available yet for multiple people. In our experience, Slack calls often fail for a variety of reasons. Other than that, we don’t want to shift too much data to Slack and since it’s pretty unclear how their calling technology works and appear.in uses WebRTC, we rather use another service here than a service where we don’t know much about it but only that calls aren’t too reliable. But as said, we haven’t bothered to check the group-video calling yet there. We’re pragmatic. Appear.in works in any browser (Slack calls don’t) and therefore… why should we change a well working system?

kingbirdy 9y ago

The branch deploy sounds pretty interesting, I wish there was more detail on it than just a footnote

e1g 9y ago
We do the same - it was surprisingly simple to implement - - CircleCI listens across all GIT branches and injects $CIRCLE_BRANCH into ENV - A build step populates a vhost template to point "$CIRCLE_BRANCH.qa.bofh.com" to "/var/www/$CIRCLE_BRANCH/current" - Another build step creates "manifest.json" with $CIRCLE_BRANCH and $CIRCLE_BUILD_NUM - The vhost config and the manifest are added to the release tarball/Docker image - The tarball/image gets shipped to the staging server - (we ship to S3/Docker, then pull because of security concerns. But can ship inside the build process) - In staging, a script reads manifest.json and moves content to /var/www/$CIRCLE_BRANCH/$CIRCLE_BRANCH - The provided nginx vhost is copied into a standard dir with other vhosts - A symlink of "/var/www/$CIRCLE_BRANCH/current" is forced to point to the fresh release - nginx -s reload && profit You can use a wildcard SSL to cover subdomains or add a step to trigger LetsEncrypt certbot as required. This method works equally well with Docker images - just requires an interim step to launch the container. In staging, we use a naming convention to launch on predictable ports for every app. First 2 numbers are app specific, second 3 numbers are build-specific. For example, "25$CIRCLE_BUILD_NUM" (where $CIRCLE_BUILD_NUM is just the last 3 numbers of the build).
- vmasto 9y ago
  How do you handle the database? Do you reuse the one on staging or create a new instance with prepopulated data? If the former how do you deal with migrations and schema edits?
  - e1g 9y ago
    Yes, the db has been the most challenging aspect for us. We have 3 situations - 1. "Common baseline". With a relatively stable product, most branches (as in ~51%) do not impact the schema. For testing / QA purposes, these share one central QA db and pollute each out. Turns out, a lot of the times this is quite ok because the PR is about how the data is displayed, or improved logging, or UX change, or a security layer or anything else other than core domain knowledge - they don't care for the data that much. 2. "I'm special". Some branches do modify data (whether the format or the structure). To handle these, the manifest.json file has an option to request a separate database. If present, the rollout script will do "pg_dump + copy" of the shared staging DB, and duplicate it into "qa_$BRANCH", then update the config file (or .env for Docker) with the appropriate connection value. Additionally, it will all *sql files in a dir specified in manifest.json against the clone DB. This is done on every release, which does get annoying by resetting the qa data (we could add another manifest switch here I). On the upside, it forces you to codify all data migration rules from the start. 3. "I am very special". Some changes transform data in a way that requires business processing and cannot be done with easy SQL. Sorry, out of luck - we don't automate special cases yet. The developer has to pull the QA database to localhost, do his magic, and push it back. Not ideal, but hasn't caused any problems yet. If ain't broke...

hgdsraj 9y ago

Why don't you guys use pivotal tracker? Way better ticket management, integrates with github

bdcravens 9y ago
We've use Pivotal Tracker before, and it always felt challenging to have more than a basic description/discussion (maybe that's the point), but most tools like Jira, Github Issues, Trello, etc, facilitate a larger view.
- wpietri 9y ago
  It is definitely the point. One of the basic notions in the school of thought that Tracker comes out of is that team interaction is valuable and to be encouraged. So making a tool that makes it easy for people to not talk undermines the broader goal. One of the basic insights of the Agile movement (R.I.P.) about Waterfall is that processes structured around documentation rather than collaboration have a lot of subtle bad effects that gradually destroy characteristics you'd like your teams to have. E.g., responsiveness to change, systemic effectiveness, resilience to failure, low overhead, ability to ship frequently, ability to deliver customer value. That's why when I set things up I generally drive things off of index cards. [1] Those are obviously insufficient, which forces people to discuss and collaborate. (That's in contrast to more voluminous documentation, which is subtly insufficient.) [1] E.g.: http://williampietri.com/writing/2015/the-big-board/ http://williampietri.com/writing/2015/the-big-board/
- jacques_chester 9y ago
  Tracker is very opinionated and is deliberately trying to not be the be-all and end-all for software projects. It's really closely tuned to how Pivotal works: XP with Lean trimmings in small teams with a product manager, designer and engineers. The downside is that because it is deliberately limited, now and then you will find that you sorely miss something. Often those missing features are recreated with conventions around tagging or release markers. But tagging conventions are not the same as a first-class feature, for good or ill. JIRA is massively more featuresome and flexible. These days it's grown into a workflow middleware which comes with a bug tracker as the first application installed. For some organisations that will make more sense. Disclosure: I work for Pivotal, but not on Tracker.

dbg31415 9y ago

ZenHub is free for teams under 5 people. And it's great. * ZenHub - Agile GitHub Project Management || https://www.zenhub.com/ https://www.zenhub.com/ Also I use a tool that keeps labels in sync across GitHub repos. * github-label-sync || https://www.npmjs.com/package/github-label-sync https://www.npmjs.com/package/github-label-sync Harvest for time tracking. * Simple Online Time Tracking Software - Harvest || https://www.getharvest.com/ https://www.getharvest.com/ Red Pen for annotations. (Does't integrate with anything and that pisses me off -- how freakin' hard would it be to build web hooks so it could tie into Slack?! But on the whole it's got an easy to use interface.) * Red Pen || https://redpen.io/ https://redpen.io/ While I like Slack a lot, one client I have uses Discord... and it's not bad. * Discord - Free Voice and Text Chat for Gamers || https://discordapp.com/ https://discordapp.com/

anselmh 9y ago
Thanks for the list, it’s quite good! However, our point is to not use third-parties for things where we don’t need it. The point of the article is to show that you can use Github solely for project management without a third-party service. But I completely agree with you: If you don’t like the little extra-work we do to achieve that with just github.com, it’s probably right to add another tool like ZenHub or waffle.io on top of it. Making your toolset work for your team is the most important thing. For us, it’s enough — and maybe for a few others as well. That’s why we shared our approach :)
- dbg31415 9y ago
  Gonna rant a bit... GitHub boards cost time. No one should use them. They're just an inferior option to ZenHub (or even Waffle or Asana or any number of other "board" interfaces you can tack on to GitHub). I've wasted my team's time on GitHub boards... everyone quickly demanded we go back to ZenHub. Most projects have multiple repositories, right? But GitHub boards have it so that each board is based on one repo. Stupid. You want to see a project view of all your repos at once... front-end, back-end, deployment, whatever... not 4 or 5 separate views on it. I feel like GitHub REALLY dropped the ball on not having a board forever, then putting out such a "beta-feeling" board. They should have just bought ZenHub -- still time... but literally any tool out there makes the boards work better than default GitHub. ZenHub is free for small teams, and I'd argue that anyone can afford $5 / month / user. Budget $150 / month / user for any team for tools -- seems about right.

awinter-py 9y ago

I wish this were an article about some hot new documentation tool instead of another misguided psalm to slack & kanban. Making docs sexy would be a big quality of life boost for programmers and the people who love them (i.e. project managers and users of software).

hobofan 9y ago
Best tip for docs in small teams: Have a server running in the local network that pulls the latest develop branch, renders it and serves it on the network for each project. For microservices I would recommend API Blueprint, for normal libraries, whatever is the language standard. Really cuts down the friction of getting to the docs, since you don't have to do all those steps on you local machine anymore whenever you need to access the documentation of an internal project. Next best thing would be building Dash docfeeds for internal projects.
- awinter-py 9y ago
  hadn't heard of API blueprint -- pretty cool. in addition to tools for documenting interfaces (i.e. APIs), I'm interested in tools that document behavior (both desired & observed) and integrate it back to codebases. In type safety terms, this is like the difference between function signatures that verify inputs & outputs vs something like coq/TLA that can verify more advanced properties of your program. Configuration is particularly difficult to document in 2017 -- if you solve that problem you fix a lot of migraines.
  - hobofan 9y ago
    > I'm interested in tools that document behavior (both desired & observed) and integrate it back to codebases. > Configuration is particularly difficult to document in 2017 -- if you solve that problem you fix a lot of migraines. Could you expand on those points a bit more? As for the first one, it sounds vaguely like https://deckard.ai https://deckard.ai, but I feel you could also mean something entirely different. (Disclamier: I know the founders of Deckard very well.)