13 ms·

Ask HN: What are the best open source tools you use in your organization?

52 points by devcheese 11y ago

bigdubs 11y ago
git
jasondecastro 11y ago
docker
engi_nerd 11y ago
<sigh> Not a single tool we use is open source. Mainly because with an open source application there's no one source for security to go to and ask for a letter certifying the safety of the software. I wish I were kidding.
- LesZedCB 11y ago
  no linux servers anywhere? No open source network infrastructure?
  - amlgsmsn 11y ago
    Perhaps they use commercial distros, like RedHat and buy a support contract.
    - LesZedCB 11y ago
      ah, yeah that would make sense, thanks!
    - nizmow 11y ago
      Red Hat is still open source.
    - engi_nerd 11y ago
      It's an entirely Windows shop.
  - mikekchar 11y ago
    I'm not the original poster, but I find it slightly humorous at the incredulity of not running open source software anywhere. 15 years ago nobody would have batted an eyelid. Back in the "dark days", I worked for a couple of Microsoft Certified Partners. One of the things they all had in common was a "no open source" policy. I wasn't even allowed to run emacs as my editor. GPL was explicitly "evil" and even more permissive licenses were "evil enough". Hell, I had to use Visual Source Safe for version control! I would have killed for RCS, let alone something like Subversion. And, no, not a single Linux box, or even BSD system to be found. Every single server was running Windows. I remember noticing strange network traffic one time and, lacking the tools in Windows, booted up Knoppix and diagnosed the problem -- a former employee had set up a program on an idle machine to forward the source code for our software outside the company. I couldn't tell the network admins how I had found the problem because it was more than my job was worth to admit to having used GPL software on the premises. Of course, I don't work for companies like that any more. In fact, I'm gratified to discover that they are so rare that nobody believes they could possibly exist. In my current job, apart from 3 SaaS products (Github, Trello and Slack), every single tool I use is open source. I sometimes have to pinch myself to make sure I'm not dreaming...
  - engi_nerd 11y ago
    Nope, not that I'm aware of. Everything is Windows Server. We had an application from one of our system vendors that incorporated FFmpeg for playing video streams within the app. We had to go and ask them to remove FFmpeg and replace it with some other video package that IT security approved of. This is normal for my industry. It's kind of depressing. I hear about all these nice open source tools (pandas! numpy! for a couple of examples) and have to make do with closed source equivalents I like much less (MATLAB).
yrro 11y ago
GNU Make.
mfincham 11y ago
RT from BestPractical https://www.bestpractical.com/rt/ https://www.bestpractical.com/rt/. Extremely extensible and robust, can be used for all kinds of things beyond just ticket tracking (change control approval workflows, knowledge bases, asset tracking, CRM and automation plugins). The king of ticket trackers, could be at the heart of any organisation. Prosody https://prosody.im/ https://prosody.im/. Open source XMPP server with support for MUCs and all the usual XMPP stuff. Best way for teams to communicate. Extend it with plugins for your NMS and so on to get alerts in realtime. Nagios https://www.nagios.org/ https://www.nagios.org/. Solid (if a little fiddly) monitoring / alerting solution. Works best if you write your own plugins and use check_by_ssh instead of NRPE. bcfg2 http://bcfg2.org/ http://bcfg2.org/. Configuration management that makes sense. Tidy and well defined "model" for how configuration should be managed, sensible defaults and a bunch of useful powerful features. I could go on but there are some of my all-time favourites.
vacri 11y ago
ssh, openvpn, ansible, git... bash :)
mfincham 11y ago
Come on folks, don't just list the command line tools that come with your Linux distro :) Let's talk about the big packages that bring value to organisations.
LesZedCB 11y ago
Rails ElasticSearch (And Kibana, holy shit) MySQL React Ractive Ubuntu NGINX Redis We stand on the backs of giants. Yet, for some reason, corporations feel entitled to not give back. Modern computing lives by open source. Companies should be obliged to contribute similarly.
- DougN7 11y ago
  If only there were some mechanism whereby value was traded fairly... :)
- mooreds 11y ago
  I love what kibana gives us in terms of search and visualizations, but have found es and kibana to be quite fiddly and needy with respect to devops attention.
amlgsmsn 11y ago
It has been my observation that when many folks ask for open source software, what they really want and mean is free(as in beer) software.
pvorb 11y ago
GitLab
codemac 11y ago
GCC
drewbug01 11y ago
HAProxy: http://www.haproxy.org/ http://www.haproxy.org/ The configuration language leaves much to be desired ... but it's otherwise a fantastic piece of software. It's the most configurable, full-featured, stable, and fast proxy server I've ever worked with.
- Scarbutt 11y ago
  What does it have over nginx?
  - drewbug01 11y ago
    Depending on what you care about, either everything or nothing. Some interesting things I find useful: * Multiple load-balancing schemes (round robin, least conn, first, balance-on-src-ip, balance-on-uri, balance-on-uri-param, balance-on-header-value, etc) * Advanced health-checking (out of band, script-based, remote socket based, HTTP send/expect, etc) * Really, really easy HTTP request and response manipulation * Generic counter support - think something like "rate limit request based on the average number of requests over the last 5 minutes that all had this random URL param". That's a contrived example, but it's easy in HAProxy. * Fantastic ACL support. You can even dynamically update them on the fly based on a response from an upstream server, if you so wanted. Not that you should, but you could. Honestly, you can do most things you'd need in nginx these days. But HAProxy tends to do it just as well, and in a much, much more flexible manner. And in some ways, the ACL support and terseness of the configuration allows you to write out really complex behavior in ways that would be annoying or difficult in nginx.
bigger_cheese 11y ago
GCC GDB VTK OpenCV I work for big monolithic company (not a software company). Most of the libraries and dev tools the engineers in my dept use are open source unfortunately the environment they run on isn't.
fixxer 11y ago
Spark has improved our ETL jobs by orders of magnitude, both with respect to performance and ability to engage our workforce (mostly Python programmers). Previous tools that improved workflow: docker, nginx.
- mfincham 11y ago
  What is Spark? Google turns up a bunch of projects called that.
  - tuckerman 11y ago
    I assume they are talking about Apache Spark http://spark.apache.org/ http://spark.apache.org/
- xfax 11y ago
  Mind if I ask how you use Spark for your ETL jobs?
  - fixxer 11y ago
    Feature engineering. Transfers about 3.5b records into features that go into a variety of models. Previously was a hadoop streaming job (~40 hours); now about 6.
gardnr 11y ago
loomio & cobudget
- mfincham 11y ago
  Do you run your own loomio or use the hosted service?
mooreds 11y ago
Git Eclipse Vagrant Bash Pmwiki Ubuntu Nginx Apache MySQL Maven Puppet
robgibbons 11y ago
Lots of Node & NPM. Also Grunt, Brew, Ant, Maven. Not to mention the rest of the stacks.
sprobertson 11y ago
ZeroMQ, React, Node.js, Nginx, Redis, PostgreSQL, and of course Linux. Together they make a great platform for building quality apps very quickly (which is the point of our organization).
ddw 11y ago
Ansible, Redis, Postgres, Django, Linux, Nginx, Spark, Git, AngularJS, Elasticsearch. Thanks!
JasonCEC 11y ago
Phabricator for task managment, project management, and code reviews. It's open source, great software, and has a fun personality great for internal use.
- thomas-b 11y ago
  Same here, really great as it packs a lot. The new Gitlab 8 with the CI integrated is great as well but we preferred Phabricator for the extended feature. Ultimately we decided not to embrace fully yet the Phabricator CI (harbormaster/drydock) as it's been changing a lot (it's a prototype after all) and breaking frequently so we have a mix of Harbormaster and Jenkins which works very well.
- toupeira 11y ago
  The main thing that's confusing me about Phabricator is that it seems to be centered around a one-repository workflow (which I gather FB uses). Does it work well when you have different projects that are unrelated to each other, or is the idea to set up dedicated instances for each project?
grhmc 11y ago
A bit different from the rest, is `repo`, from AOSP: https://source.android.com/source/downloading.html https://source.android.com/source/downloading.html We run dozens of microservices and have nearly 100 git repositories. Using `repo`, every dev has every repo in consistent (relative) path from each other. It also greatly simplifies cross-cutting changes across repositories, and staying in sync (`repo sync` updates all repos all at once.) I also use it to manage my dotfiles: github.com/grahamc/manifest Second: Go.CD (https://www.go.cd/ https://www.go.cd/) -- applying build templates to our 100 repositories has greatly simplified our management of the repositories. Using pipelines has added deep visibility into what is deployed where. I love Go.CD.
RobbyMcCullough 11y ago
WordPress
nbclark 11y ago
Node, React/Native, Angular, Spring, RabbitMQ, PostgreSQL, Docker, Vagrant, Consul, Spark, Kafka, Too much to name...
dineshp2 11y ago
Git,Debian,Ruby on Rails,PostegreSql,Redis,nginx primarily.
imaginenore 11y ago
Blender. For fully programmable 3D rendering, including command line only.
markbnj 11y ago
For us the list is very long, beginning with debian and ubuntu. Elasticsearch, redis, haproxy, mongodb, postgressql ... really too many to list. Of these certainly elasticsearch, redis, and haproxy are best in class in my opinion, and all the tools we use are proven performers with solid community support.
imsofuture 11y ago
Ubuntu, HAProxy, Ejabberd, Redis are the solid standouts.
0xCMP 11y ago
Oh man sentry is pretty awesome. http://getsentry.com http://getsentry.com
gtaylor 11y ago
Kubernetes has been amazing for us. Getting better every month. More recently, Drone for CI has been a quantum leap in usability for us over Jenkins. Built on top of Docker, has service container support built in, and has the unique feature of plugins-as-Docker-containers (so you can write them in any language): https://github.com/drone/drone https://github.com/drone/drone Ansible, ramping up with some Terraform and really loving it.
DrPhish 11y ago
mrtg/smokeping, netdisco2, nfdump/nfsen and previously request tracker, Nagios and racktables
dexterbt1 11y ago
The question needs expansion. What is the OP's definition of best? Why does it matter to the OP that it's open source (would your org contribute back)? Would specific family of licenses matter? And in what problem domain do the tools be used for? I'm guessing most of HN's audience would be using some form of OSS. The coverage of the comments here is large: devops tools, programming tools, libraries, productivity, etc. Our own org uses a lot of open source components, from desktops, office suite, dev environments, server OSes, programming platforms, libraries, databases, CMSes, heck, does Android count? More context is better.
packetized 11y ago
If you haven't heard of Heka (http://hekad.rtfd.org http://hekad.rtfd.org), you should check it out.

52 points by devcheese 11y ago

bigdubs 11y ago

git

jasondecastro 11y ago

docker

engi_nerd 11y ago

<sigh> Not a single tool we use is open source. Mainly because with an open source application there's no one source for security to go to and ask for a letter certifying the safety of the software. I wish I were kidding.

LesZedCB 11y ago
no linux servers anywhere? No open source network infrastructure?
- amlgsmsn 11y ago
  Perhaps they use commercial distros, like RedHat and buy a support contract.
  - LesZedCB 11y ago
    ah, yeah that would make sense, thanks!
  - nizmow 11y ago
    Red Hat is still open source.
  - engi_nerd 11y ago
    It's an entirely Windows shop.
- mikekchar 11y ago
  I'm not the original poster, but I find it slightly humorous at the incredulity of not running open source software anywhere. 15 years ago nobody would have batted an eyelid. Back in the "dark days", I worked for a couple of Microsoft Certified Partners. One of the things they all had in common was a "no open source" policy. I wasn't even allowed to run emacs as my editor. GPL was explicitly "evil" and even more permissive licenses were "evil enough". Hell, I had to use Visual Source Safe for version control! I would have killed for RCS, let alone something like Subversion. And, no, not a single Linux box, or even BSD system to be found. Every single server was running Windows. I remember noticing strange network traffic one time and, lacking the tools in Windows, booted up Knoppix and diagnosed the problem -- a former employee had set up a program on an idle machine to forward the source code for our software outside the company. I couldn't tell the network admins how I had found the problem because it was more than my job was worth to admit to having used GPL software on the premises. Of course, I don't work for companies like that any more. In fact, I'm gratified to discover that they are so rare that nobody believes they could possibly exist. In my current job, apart from 3 SaaS products (Github, Trello and Slack), every single tool I use is open source. I sometimes have to pinch myself to make sure I'm not dreaming...
- engi_nerd 11y ago
  Nope, not that I'm aware of. Everything is Windows Server. We had an application from one of our system vendors that incorporated FFmpeg for playing video streams within the app. We had to go and ask them to remove FFmpeg and replace it with some other video package that IT security approved of. This is normal for my industry. It's kind of depressing. I hear about all these nice open source tools (pandas! numpy! for a couple of examples) and have to make do with closed source equivalents I like much less (MATLAB).

yrro 11y ago

GNU Make.

mfincham 11y ago

RT from BestPractical https://www.bestpractical.com/rt/ https://www.bestpractical.com/rt/. Extremely extensible and robust, can be used for all kinds of things beyond just ticket tracking (change control approval workflows, knowledge bases, asset tracking, CRM and automation plugins). The king of ticket trackers, could be at the heart of any organisation. Prosody https://prosody.im/ https://prosody.im/. Open source XMPP server with support for MUCs and all the usual XMPP stuff. Best way for teams to communicate. Extend it with plugins for your NMS and so on to get alerts in realtime. Nagios https://www.nagios.org/ https://www.nagios.org/. Solid (if a little fiddly) monitoring / alerting solution. Works best if you write your own plugins and use check_by_ssh instead of NRPE. bcfg2 http://bcfg2.org/ http://bcfg2.org/. Configuration management that makes sense. Tidy and well defined "model" for how configuration should be managed, sensible defaults and a bunch of useful powerful features. I could go on but there are some of my all-time favourites.

vacri 11y ago

ssh, openvpn, ansible, git... bash :)

mfincham 11y ago

Come on folks, don't just list the command line tools that come with your Linux distro :) Let's talk about the big packages that bring value to organisations.

LesZedCB 11y ago

Rails ElasticSearch (And Kibana, holy shit) MySQL React Ractive Ubuntu NGINX Redis We stand on the backs of giants. Yet, for some reason, corporations feel entitled to not give back. Modern computing lives by open source. Companies should be obliged to contribute similarly.

DougN7 11y ago
If only there were some mechanism whereby value was traded fairly... :)
mooreds 11y ago
I love what kibana gives us in terms of search and visualizations, but have found es and kibana to be quite fiddly and needy with respect to devops attention.

amlgsmsn 11y ago

It has been my observation that when many folks ask for open source software, what they really want and mean is free(as in beer) software.

pvorb 11y ago

GitLab

codemac 11y ago

GCC

drewbug01 11y ago

HAProxy: http://www.haproxy.org/ http://www.haproxy.org/ The configuration language leaves much to be desired ... but it's otherwise a fantastic piece of software. It's the most configurable, full-featured, stable, and fast proxy server I've ever worked with.

Scarbutt 11y ago
What does it have over nginx?
- drewbug01 11y ago
  Depending on what you care about, either everything or nothing. Some interesting things I find useful: * Multiple load-balancing schemes (round robin, least conn, first, balance-on-src-ip, balance-on-uri, balance-on-uri-param, balance-on-header-value, etc) * Advanced health-checking (out of band, script-based, remote socket based, HTTP send/expect, etc) * Really, really easy HTTP request and response manipulation * Generic counter support - think something like "rate limit request based on the average number of requests over the last 5 minutes that all had this random URL param". That's a contrived example, but it's easy in HAProxy. * Fantastic ACL support. You can even dynamically update them on the fly based on a response from an upstream server, if you so wanted. Not that you should, but you could. Honestly, you can do most things you'd need in nginx these days. But HAProxy tends to do it just as well, and in a much, much more flexible manner. And in some ways, the ACL support and terseness of the configuration allows you to write out really complex behavior in ways that would be annoying or difficult in nginx.

bigger_cheese 11y ago

GCC GDB VTK OpenCV I work for big monolithic company (not a software company). Most of the libraries and dev tools the engineers in my dept use are open source unfortunately the environment they run on isn't.

fixxer 11y ago

Spark has improved our ETL jobs by orders of magnitude, both with respect to performance and ability to engage our workforce (mostly Python programmers). Previous tools that improved workflow: docker, nginx.

mfincham 11y ago
What is Spark? Google turns up a bunch of projects called that.
- tuckerman 11y ago
  I assume they are talking about Apache Spark http://spark.apache.org/ http://spark.apache.org/
xfax 11y ago
Mind if I ask how you use Spark for your ETL jobs?
- fixxer 11y ago
  Feature engineering. Transfers about 3.5b records into features that go into a variety of models. Previously was a hadoop streaming job (~40 hours); now about 6.

gardnr 11y ago

loomio & cobudget

mfincham 11y ago
Do you run your own loomio or use the hosted service?

mooreds 11y ago

Git Eclipse Vagrant Bash Pmwiki Ubuntu Nginx Apache MySQL Maven Puppet

robgibbons 11y ago

Lots of Node & NPM. Also Grunt, Brew, Ant, Maven. Not to mention the rest of the stacks.

sprobertson 11y ago

ZeroMQ, React, Node.js, Nginx, Redis, PostgreSQL, and of course Linux. Together they make a great platform for building quality apps very quickly (which is the point of our organization).

ddw 11y ago

Ansible, Redis, Postgres, Django, Linux, Nginx, Spark, Git, AngularJS, Elasticsearch. Thanks!

JasonCEC 11y ago

Phabricator for task managment, project management, and code reviews. It's open source, great software, and has a fun personality great for internal use.

thomas-b 11y ago
Same here, really great as it packs a lot. The new Gitlab 8 with the CI integrated is great as well but we preferred Phabricator for the extended feature. Ultimately we decided not to embrace fully yet the Phabricator CI (harbormaster/drydock) as it's been changing a lot (it's a prototype after all) and breaking frequently so we have a mix of Harbormaster and Jenkins which works very well.
toupeira 11y ago
The main thing that's confusing me about Phabricator is that it seems to be centered around a one-repository workflow (which I gather FB uses). Does it work well when you have different projects that are unrelated to each other, or is the idea to set up dedicated instances for each project?

grhmc 11y ago

A bit different from the rest, is `repo`, from AOSP: https://source.android.com/source/downloading.html https://source.android.com/source/downloading.html We run dozens of microservices and have nearly 100 git repositories. Using `repo`, every dev has every repo in consistent (relative) path from each other. It also greatly simplifies cross-cutting changes across repositories, and staying in sync (`repo sync` updates all repos all at once.) I also use it to manage my dotfiles: github.com/grahamc/manifest Second: Go.CD (https://www.go.cd/ https://www.go.cd/) -- applying build templates to our 100 repositories has greatly simplified our management of the repositories. Using pipelines has added deep visibility into what is deployed where. I love Go.CD.

RobbyMcCullough 11y ago

WordPress

nbclark 11y ago

Node, React/Native, Angular, Spring, RabbitMQ, PostgreSQL, Docker, Vagrant, Consul, Spark, Kafka, Too much to name...

dineshp2 11y ago

Git,Debian,Ruby on Rails,PostegreSql,Redis,nginx primarily.

imaginenore 11y ago

Blender. For fully programmable 3D rendering, including command line only.

markbnj 11y ago

For us the list is very long, beginning with debian and ubuntu. Elasticsearch, redis, haproxy, mongodb, postgressql ... really too many to list. Of these certainly elasticsearch, redis, and haproxy are best in class in my opinion, and all the tools we use are proven performers with solid community support.

imsofuture 11y ago

Ubuntu, HAProxy, Ejabberd, Redis are the solid standouts.

0xCMP 11y ago

Oh man sentry is pretty awesome. http://getsentry.com http://getsentry.com

gtaylor 11y ago

Kubernetes has been amazing for us. Getting better every month. More recently, Drone for CI has been a quantum leap in usability for us over Jenkins. Built on top of Docker, has service container support built in, and has the unique feature of plugins-as-Docker-containers (so you can write them in any language): https://github.com/drone/drone https://github.com/drone/drone Ansible, ramping up with some Terraform and really loving it.

DrPhish 11y ago

mrtg/smokeping, netdisco2, nfdump/nfsen and previously request tracker, Nagios and racktables

dexterbt1 11y ago

The question needs expansion. What is the OP's definition of best? Why does it matter to the OP that it's open source (would your org contribute back)? Would specific family of licenses matter? And in what problem domain do the tools be used for? I'm guessing most of HN's audience would be using some form of OSS. The coverage of the comments here is large: devops tools, programming tools, libraries, productivity, etc. Our own org uses a lot of open source components, from desktops, office suite, dev environments, server OSes, programming platforms, libraries, databases, CMSes, heck, does Android count? More context is better.

packetized 11y ago

If you haven't heard of Heka (http://hekad.rtfd.org http://hekad.rtfd.org), you should check it out.