4 ms·

Docker, CoreOS, fleet, and etcd have completely changed how I build projects. It's made me much more productive. I'm working on Strata, which is a building man

Docker, CoreOS, fleet, and etcd have completely changed how I build projects. It's made me much more productive. I'm working on Strata, which is a building management & commissioning system for property owners of high-rise smart buildings. It's currently deployed in a single building in downtown Toronto, and it's pulling in data from thousands of devices, and presenting it in real-time via an API and a dashboard. So in this building, I have a massive server. 2 CPUs, 10 cores each, 128GB of RAM, the works. It came with VMWare ESXi. I have 10 instances of CoreOS running, each identical, but with multiple NICs provisioned for each so that they can communicate with the building subsystems. I built every "app" in its own Docker container. That means PostgreSQL, Redis, RabbitMQ, my Django app, my websocket server, even Nginx, all run in their own containers. They advertise themselves into etcd, and any dependencies are pulled from etcd. That means that the Django app gets the addresses for the PostgreSQL and Redis servers from etcd, and connects that way. If these values change, each container restarts itself as needed. I also have a number of workers to crawl the network and pull in data. Deployment is just a matter of running 'fleetctl start overlord@{1..9}.service', and it's deployed across every machine in my cluster. With this setup, adding machines, or adding containers is straightforward and flexible. Furthermore, for development, I run the same Docker containers locally via Vagrant, building and pushing as needed. And when I applied for YC, I spun up 3 CoreOS instances on DigitalOcean and ran the fleet files there. As I said, I've been able to streamline development and make it super agile with Docker & CoreOS. Oh, and I'm the only one working on this. I figure if I can do it on my own, imagine what a team of engineers can do. Very powerful stuff.

ownagefool 12y ago
Sounds like it works for you, but you should probably think about redundancy in your hardware there.
- HorizonXP 12y ago
  I've said it many times, but I inherited the hardware. My current plan is to virtualize a couple of other servers, and then create a VMWare cluster using the extra machines to allow a sort of "auto-failover" setup.
jorlow 12y ago
> They advertise themselves into etcd, and any dependencies are pulled from etcd. That means that the Django app gets the addresses for the PostgreSQL and Redis servers from etcd, and connects that way. If these values change, each container restarts itself as needed. Could you explain a bit more how that works?
- HorizonXP 12y ago
  Take Postgres for example. I use the official Docker image, and run it via a fleet .service file, having it store its data in a Docker data-only container, also launched from a fleet unit file. Finally, there's a third unit file that runs a slightly customized version of the docker-register image from CoreOS's polvi. This Dockerized app polls the Docker socket every 10s or so to get the IP and port information for the Docker container it's monitoring, in this case Postgres. It PUTs this value into etcd. Afterwards, when my Django containers start, they pull their configuration from etcd using python-etcd. It will get the IPs and ports for the services, including Postgres, from etcd, and configure itself using that. Finally, it will keep these values in a dictionary. If these values ever differ from what's currently in etcd, then my Django app will send a signal to cause uwsgi to gracefully restart the Django app, so that it can reload itself with the new configuration. For other Dockerized apps like nginx, I use kelseyhightower's confd-watch to monitor the etcd keys I'm interested in. confd-watch is great because it will use golang's text/template package to allow me to generate configuration files, like nginx.conf, with the values from etcd. So as these values get updated, so will nginx as confd-watch will force a graceful reload. Hope that helps.
  - leonidlm 12y ago
    "If these values ever differ from what's currently in etcd, then my Django app will send a signal to cause uwsgi to gracefully restart the Django app, so that it can reload itself with the new configuration" You could probably avoid the restart/reload by using skydns (https://github.com/skynetservices/skydns https://github.com/skynetservices/skydns), it is a local dns service built on top of values in etcd. You can configure your django app to access the other service containers by dns name and it will be translated to the correct ip all the time.
acous 12y ago
Does Strata have a website?
- HorizonXP 12y ago
  It does not, currently. Consequences of being a one-person show is that things like that are left on the todo list. Soon though!
tga 12y ago
I'd love to get in touch and have a chat with you on what you're doing, I'm working on similar stuff but don't see much HVAC/energy around HN. Could you drop me a line (contact details in my profile)? Thanks!

ownagefool 12y ago

Sounds like it works for you, but you should probably think about redundancy in your hardware there.

HorizonXP 12y ago
I've said it many times, but I inherited the hardware. My current plan is to virtualize a couple of other servers, and then create a VMWare cluster using the extra machines to allow a sort of "auto-failover" setup.

jorlow 12y ago

> They advertise themselves into etcd, and any dependencies are pulled from etcd. That means that the Django app gets the addresses for the PostgreSQL and Redis servers from etcd, and connects that way. If these values change, each container restarts itself as needed. Could you explain a bit more how that works?

HorizonXP 12y ago
Take Postgres for example. I use the official Docker image, and run it via a fleet .service file, having it store its data in a Docker data-only container, also launched from a fleet unit file. Finally, there's a third unit file that runs a slightly customized version of the docker-register image from CoreOS's polvi. This Dockerized app polls the Docker socket every 10s or so to get the IP and port information for the Docker container it's monitoring, in this case Postgres. It PUTs this value into etcd. Afterwards, when my Django containers start, they pull their configuration from etcd using python-etcd. It will get the IPs and ports for the services, including Postgres, from etcd, and configure itself using that. Finally, it will keep these values in a dictionary. If these values ever differ from what's currently in etcd, then my Django app will send a signal to cause uwsgi to gracefully restart the Django app, so that it can reload itself with the new configuration. For other Dockerized apps like nginx, I use kelseyhightower's confd-watch to monitor the etcd keys I'm interested in. confd-watch is great because it will use golang's text/template package to allow me to generate configuration files, like nginx.conf, with the values from etcd. So as these values get updated, so will nginx as confd-watch will force a graceful reload. Hope that helps.
- leonidlm 12y ago
  "If these values ever differ from what's currently in etcd, then my Django app will send a signal to cause uwsgi to gracefully restart the Django app, so that it can reload itself with the new configuration" You could probably avoid the restart/reload by using skydns (https://github.com/skynetservices/skydns https://github.com/skynetservices/skydns), it is a local dns service built on top of values in etcd. You can configure your django app to access the other service containers by dns name and it will be translated to the correct ip all the time.

acous 12y ago

Does Strata have a website?

HorizonXP 12y ago
It does not, currently. Consequences of being a one-person show is that things like that are left on the todo list. Soon though!

tga 12y ago

I'd love to get in touch and have a chat with you on what you're doing, I'm working on similar stuff but don't see much HVAC/energy around HN. Could you drop me a line (contact details in my profile)? Thanks!