6 ms·

Aptly: Owning Your Debian Distribution

172 points by smira 12y ago

dozzie 12y ago
Would be sensible, if only the installation procedure didn't cluelessly combine downloading source with compiling it. Hey, some people want to build binary packages out of the code out there!
ossreality 12y ago
Hm, this could benefit from an edit. I'm definitely stumbling in places here on the text.
pan69 12y ago
I'm no grammar Nazi but the first sentence on this website had me completely tripped up: >> Linux distribution is well-tested collection of packages carefully organized and supported by the vendor. wat?
- smira 12y ago
  Sorry for my bad English. Does this sound better: "Linux distribution is a well-tested collection of packages that are carefully organized and supported by the vendor."
  - deleted 12y ago
    [deleted]
  - codeflo 12y ago
    The missing article at the start of the sentence makes it very hard to decipher if you're talking about Linux distributions in general or a specific one. "A Linux distrubution is ..." is probably what you intended to say, but I'm not sure.
    - smira 12y ago
      Thanks for your suggestions!
      - robinson-wall 12y ago
        Have you considered making the website / documentation source available in github? That would make it very easy for others to contribute suggestions.
        smira 12y ago
        It is actually available: https://github.com/aptly-dev/aptly-dev.github.io/tree/trunk https://github.com/aptly-dev/aptly-dev.github.io/tree/trunk In trunk branch, the markdown under content/ folder.
        padraic7a 12y ago
        I see you've just added changes, are you looking for anyone to proof read at this stage?
  - pluma 12y ago
    Grammar aside, I'd say the plural feels more natural in this case: "Linux distributions are well-tested collections of packages …" But for singular you need to qualify "Linux distribution": "A Linux distribution is a well-tested collection of packages …" And I'd probably say "by a vendor" instead of "by the vendor" as it's not clear you're talking about a specific vendor. Also, the "a" is wrong in this: "There’s a support for …" This should be: "There is support for …", in other words: "Support exists for …". Support is non-countable (like water, peace or terrorism) in English. As a rule of thumb: * if it's non-countable, just leave it bare: "I bought water", "It has support", "This is murder". * if you're referring to a specific instance of something the audience knows about or you're going to qualify, use "the": "I bought the house [I previously told you about]", "I shot the gopher that kept digging up my lawn". * otherwise use "a": "I bought a house" (this is news to you), "I ran over a cat" (if you say "the cat", the listener would instantly think "which one?"). English is "fun".
    - dllthomas 12y ago
      English has a countable "support" - it can be used to refer to several physical objects in various structural contexts (architecture being one). That is not the sense in which it's being used here, though.
    - pbhjpbhj 12y ago
      In English (en-gb) colloquialism "I ran over the cat" would be perfectly fine as "the cat" is understood to mean "our/my cat". "Your dinner's in the dog", "I locked the keys in the car", et cetera. Similarly in a sentence like "Why don't you and the family come up to the house sometime for tea.". "The house" substitutes for "our house". Now all you need to work out is whether tea is afternoon tea, a cup of tea, or an evening meal and if the offer is sincere. I gather the response is supposed to be "That would be lovely." rather than actually trying to arrange the proffered social meeting.
      - pluma 12y ago
        Sure, but presume neither the speaker nor the recipient owns a cat. Without an obvious context ("the cat"? what cats could we both know of?), it's not entirely clear. Saying "the" instead of "a" generally assumes a context that has either been established before during the conversation or outside of it. Imagine a stranger telling you they "accidentally killed the koala". You'd be confused. Not that strangers telling you about exotic animals they accidentally killed wouldn't be confusing enough.
- deathanatos 12y ago
  > I'm no grammar Nazi but Our author seems to consistently make the same mistake: there's a distinct lack of articles in the writing. Sprinkle some in, and things start to make sense: > [A] Linux distribution is [a] well-tested collection of packages > we would like to have [an] internal mirror of [a] package repository > some packages are missing in [the] official distribution > we would like to own [a] package repository, that is to have [a] package repository > There are many tools to create [a] mirror of [the] Debian package repositories … and so on. At any rate, this consistency suggests (to me, at least) that he's probably not a native English speaker; in fact, the writer is almost certainly Russian. There's a link to a sponsor, which has a Russian phone number, and if you click around enough, you can find his CV quite easily. Not everyone on the Internet speaks English, however, enough people do that I could easily see it being beneficial to put your content in English. Who knows?
  - eLobato 12y ago
    For some reason, native speakers of Slavic languages who learn English as a second language, often omit some prepositions in English. It's 100% understandable, but I'm really curious what's the actual reason why this happens.
    - heinz 12y ago
      Don't you mean articles instead of prepositions? If so, take into consideration that most of all Slavic languages don't feature articles at all.
    - szabba 12y ago
      It probably has something to do with out languages having more grammatical cases than English (which, arguably, has two, though it's possible to only use one).
    - groovy2shoes 12y ago
      Not sure about prepositions, but the articles are often omitted because most Slavic languages don't have articles. http://en.wikipedia.org/wiki/Article_%28grammar%29#Variation_among_languages http://en.wikipedia.org/wiki/Article_%28grammar%29#Variation...
- pessimizer 12y ago
  It's just article-dropping; get to know a few Eastern Europeans well and you might start doing it yourself. One of my old bosses was Bosnian Croat, dropped articles like crazy, and added them in the wrong places. She would run her important emails by me for article-fixing before sending them. She was still smarter than me and better at her job than I was. Turns out that mastery of the arbitrary rules of English article placement and choice isn't a very good indicator of intelligence or competence.
  - ianlevesque 12y ago
    No, but still a useful skill when communicating in English.
_qc3o 12y ago
This is really cool. If you're using Ubuntu or Debian as your base OS then you really should be managing your software with something like this and offloading your deployment to it as well. The benefits of a setup like that are dead-simple rollbacks and deployment scripts.
- chrisan 12y ago
  I like the concept, but wouldnt it be better to offload deployment/rollbacks to docker?
  - hosay123 12y ago
    Aptly is a repository manager, it works in terms of tightly defined package versions and dependencies. Docker is for deployment, it works in terms of graphs of filesystem images. The tools do slightly overlap in use case, but they operate in entirely different domains. It wouldn't be unreasonable to mix both in the same project, but there is no hard rule for when this would be appropriate.
    - qwerta 12y ago
      I think I will stick with a Docker. DEB packages are very complicated and this tool does not really handle dependency conflits and so on that well. Plus it does not work with RPM and other packages systems.
      - icebraining 12y ago
        DEB packages are very complicated Not really, it's just a tar file with some metadata. Using fpm¹, making packages from a directory is extremely simple. I've been building internal packages from our different components, and the build script only has three or four lines. And besides, even Dockerfiles often use apt/yum. ¹ https://github.com/jordansissel/fpm https://github.com/jordansissel/fpm
        smira 12y ago
Havvy 12y ago
What does this give you over using NixOS?
- smira 12y ago
  NixOS is aiming for the same goal, I believe. NixOS is built on top of different package manager, Nix. With aptly you're still using your favorite Debian/Ubuntu distro, installing packages with apt-get and so on.
  - Dewie 12y ago
    How about using Nix on Debian/Ubuntu?
    - vertex-four 12y ago
      The issue with this setup is that you can't define "services" from within Nix without NixOS. A service basically ties together a set of packages and configuration and puts them in systemd. At the moment, you'd have to use an external configuration management system for actually defining your systemd services, etc.
      - pmahoney 12y ago
        I deploy a couple things via Nix package manger to non-NixOS systems, and I manage my own system configs, etc. within Nix. My "external configuration management system" is pared down to a small script that symlinks everything in a directory into another directory (such as /etc/init). One thing missing from Nix is ability to store passwords and the like in the nix store, and from what I understand this applies to NixOS as well as using Nix outside NixOS. For example, every file in the nix store is world readable. https://github.com/NixOS/nix/issues/8 https://github.com/NixOS/nix/issues/8
        vertex-four 12y ago
        NixOps gets around this to some extent by allowing you to define keys to be copied to /run/keys when configuring a remote system. They thus never touch the remote store.
jvdb 12y ago
This sounds really nice. I've been using apt as a deployment mechanism not unlike the use-cases describe. While the whole thing's been rock solid, the repo management could do with a cleaner interface. Looking forward to giving it a go!
dzderic 12y ago
For anyone who has ever battled with maintaining their own apt/yum repos, this seems like a godsend. The most-used tools for getting a package into your repo involve scp'ing the file to the repository server and running a command to update its index. It's nice to have a proper toolset to do this, but it's too bad I spend most of my time with YUM nowadays.
- dima55 12y ago
  man dput
oblio 12y ago
So, basically Nexus/Artifactory for deb. Cool.
Dewie 12y ago
Half of the posts in this thread are about the author's English. I know that clear communication and constructive feedback on one's command of a language is good, but... come on.
robinson-wall 12y ago
Props to smira + other contributors to aptly, it looks like it has come a long way in a short time.
helper 12y ago
We use deb-s3[1] to maintain a custom apt repository in s3. It supports signed release files and does all the hard work for you. [1]: https://github.com/krobertson/deb-s3 https://github.com/krobertson/deb-s3
- smira 12y ago
  aptly can do more much more than just S3 publishing: for example snapshotting, combining packages from different sources.
lamby 12y ago
https://apt-captain.readthedocs.org/en/latest/ https://apt-captain.readthedocs.org/en/latest/ Here's my solution to this. Love using .debs for distribution.
- smira 12y ago
  Thanks, looks really interesting! At the moment working on REST API for aptly which should allow similar simple publishing scenarios.
- grosskur 12y ago
  Looks awesome! I'm actually working on a SaaS product along similar lines. Right now it can build RPM and Debian packages directly from GitHub sources: https://www.packagelab.com/ https://www.packagelab.com/ Sort of like a cross between Launchpad and Travis CI. If anyone's interested in trying it out, I'd love feedback.
jcapote 12y ago
This looks neat, just in time for the weekend :) I've been using the free plan at https://packagecloud.io https://packagecloud.io for my debs (handles my rpms and gems too), and it's been working well so far.
zwilliamson 12y ago
Aptly rocks. I put this together for those who use Vagrant and want to test out some of Aptly's commands and functionality. https://github.com/sepulworld/aptly-vagrant https://github.com/sepulworld/aptly-vagrant It also sets up a Jenkins server so you can test out some build jobs that interact with Aptly. I am looking forward to Aptly's REST API. Also on the roadmap is the ability to manage Yum repositories.
vacri 12y ago
I just finished putting together an aptly repo system on Friday, with a 'testing', 'staging', and 'production' repo. Buildbot dumps unsigned builds into testing, and a meatspace process copies those builds into staging or production when appropriate. Aptly is a godsend to people who want to run .deb repos, as previously the software was basically "run a full mirror" or "sucks to be you". Jordan Sissel (fpm author) once remarked that 'there is a lot of silly ceremony in managing .debs'. We were previously using reprepro, which was alright, but had the glaring flaw of only allowing one version of a package to live in the repo. No easy rollbacks or machines on different versions there. Aptly is really flexible, and although it does have some corner cases or slightly unintuitive behaviour (to me), it has excellent documentation and is in heavy development. Kudos to you, smira.

172 points by smira 12y ago

Would be sensible, if only the installation procedure didn't cluelessly combine downloading source with compiling it. Hey, some people want to build binary packages out of the code out there!

ossreality 12y ago

Hm, this could benefit from an edit. I'm definitely stumbling in places here on the text.

pan69 12y ago

I'm no grammar Nazi but the first sentence on this website had me completely tripped up: >> Linux distribution is well-tested collection of packages carefully organized and supported by the vendor. wat?

smira 12y ago
Sorry for my bad English. Does this sound better: "Linux distribution is a well-tested collection of packages that are carefully organized and supported by the vendor."
- deleted 12y ago
  [deleted]
- codeflo 12y ago
  The missing article at the start of the sentence makes it very hard to decipher if you're talking about Linux distributions in general or a specific one. "A Linux distrubution is ..." is probably what you intended to say, but I'm not sure.
  - smira 12y ago
    Thanks for your suggestions!
    - robinson-wall 12y ago
      Have you considered making the website / documentation source available in github? That would make it very easy for others to contribute suggestions.
      - smira 12y ago
        It is actually available: https://github.com/aptly-dev/aptly-dev.github.io/tree/trunk https://github.com/aptly-dev/aptly-dev.github.io/tree/trunk In trunk branch, the markdown under content/ folder.
        padraic7a 12y ago
        I see you've just added changes, are you looking for anyone to proof read at this stage?
- pluma 12y ago
  Grammar aside, I'd say the plural feels more natural in this case: "Linux distributions are well-tested collections of packages …" But for singular you need to qualify "Linux distribution": "A Linux distribution is a well-tested collection of packages …" And I'd probably say "by a vendor" instead of "by the vendor" as it's not clear you're talking about a specific vendor. Also, the "a" is wrong in this: "There’s a support for …" This should be: "There is support for …", in other words: "Support exists for …". Support is non-countable (like water, peace or terrorism) in English. As a rule of thumb: * if it's non-countable, just leave it bare: "I bought water", "It has support", "This is murder". * if you're referring to a specific instance of something the audience knows about or you're going to qualify, use "the": "I bought the house [I previously told you about]", "I shot the gopher that kept digging up my lawn". * otherwise use "a": "I bought a house" (this is news to you), "I ran over a cat" (if you say "the cat", the listener would instantly think "which one?"). English is "fun".
  - dllthomas 12y ago
    English has a countable "support" - it can be used to refer to several physical objects in various structural contexts (architecture being one). That is not the sense in which it's being used here, though.
  - pbhjpbhj 12y ago
    In English (en-gb) colloquialism "I ran over the cat" would be perfectly fine as "the cat" is understood to mean "our/my cat". "Your dinner's in the dog", "I locked the keys in the car", et cetera. Similarly in a sentence like "Why don't you and the family come up to the house sometime for tea.". "The house" substitutes for "our house". Now all you need to work out is whether tea is afternoon tea, a cup of tea, or an evening meal and if the offer is sincere. I gather the response is supposed to be "That would be lovely." rather than actually trying to arrange the proffered social meeting.
    - pluma 12y ago
      Sure, but presume neither the speaker nor the recipient owns a cat. Without an obvious context ("the cat"? what cats could we both know of?), it's not entirely clear. Saying "the" instead of "a" generally assumes a context that has either been established before during the conversation or outside of it. Imagine a stranger telling you they "accidentally killed the koala". You'd be confused. Not that strangers telling you about exotic animals they accidentally killed wouldn't be confusing enough.
deathanatos 12y ago
> I'm no grammar Nazi but Our author seems to consistently make the same mistake: there's a distinct lack of articles in the writing. Sprinkle some in, and things start to make sense: > [A] Linux distribution is [a] well-tested collection of packages > we would like to have [an] internal mirror of [a] package repository > some packages are missing in [the] official distribution > we would like to own [a] package repository, that is to have [a] package repository > There are many tools to create [a] mirror of [the] Debian package repositories … and so on. At any rate, this consistency suggests (to me, at least) that he's probably not a native English speaker; in fact, the writer is almost certainly Russian. There's a link to a sponsor, which has a Russian phone number, and if you click around enough, you can find his CV quite easily. Not everyone on the Internet speaks English, however, enough people do that I could easily see it being beneficial to put your content in English. Who knows?
- eLobato 12y ago
  For some reason, native speakers of Slavic languages who learn English as a second language, often omit some prepositions in English. It's 100% understandable, but I'm really curious what's the actual reason why this happens.
  - heinz 12y ago
    Don't you mean articles instead of prepositions? If so, take into consideration that most of all Slavic languages don't feature articles at all.
  - szabba 12y ago
    It probably has something to do with out languages having more grammatical cases than English (which, arguably, has two, though it's possible to only use one).
  - groovy2shoes 12y ago
    Not sure about prepositions, but the articles are often omitted because most Slavic languages don't have articles. http://en.wikipedia.org/wiki/Article_%28grammar%29#Variation_among_languages http://en.wikipedia.org/wiki/Article_%28grammar%29#Variation...
pessimizer 12y ago
It's just article-dropping; get to know a few Eastern Europeans well and you might start doing it yourself. One of my old bosses was Bosnian Croat, dropped articles like crazy, and added them in the wrong places. She would run her important emails by me for article-fixing before sending them. She was still smarter than me and better at her job than I was. Turns out that mastery of the arbitrary rules of English article placement and choice isn't a very good indicator of intelligence or competence.
- ianlevesque 12y ago
  No, but still a useful skill when communicating in English.

_qc3o 12y ago

This is really cool. If you're using Ubuntu or Debian as your base OS then you really should be managing your software with something like this and offloading your deployment to it as well. The benefits of a setup like that are dead-simple rollbacks and deployment scripts.

chrisan 12y ago
I like the concept, but wouldnt it be better to offload deployment/rollbacks to docker?
- hosay123 12y ago
  Aptly is a repository manager, it works in terms of tightly defined package versions and dependencies. Docker is for deployment, it works in terms of graphs of filesystem images. The tools do slightly overlap in use case, but they operate in entirely different domains. It wouldn't be unreasonable to mix both in the same project, but there is no hard rule for when this would be appropriate.
  - qwerta 12y ago
    I think I will stick with a Docker. DEB packages are very complicated and this tool does not really handle dependency conflits and so on that well. Plus it does not work with RPM and other packages systems.
    - icebraining 12y ago
      DEB packages are very complicated Not really, it's just a tar file with some metadata. Using fpm¹, making packages from a directory is extremely simple. I've been building internal packages from our different components, and the build script only has three or four lines. And besides, even Dockerfiles often use apt/yum. ¹ https://github.com/jordansissel/fpm https://github.com/jordansissel/fpm
      - smira 12y ago

Havvy 12y ago

What does this give you over using NixOS?

smira 12y ago
NixOS is aiming for the same goal, I believe. NixOS is built on top of different package manager, Nix. With aptly you're still using your favorite Debian/Ubuntu distro, installing packages with apt-get and so on.
- Dewie 12y ago
  How about using Nix on Debian/Ubuntu?
  - vertex-four 12y ago
    The issue with this setup is that you can't define "services" from within Nix without NixOS. A service basically ties together a set of packages and configuration and puts them in systemd. At the moment, you'd have to use an external configuration management system for actually defining your systemd services, etc.
    - pmahoney 12y ago
      I deploy a couple things via Nix package manger to non-NixOS systems, and I manage my own system configs, etc. within Nix. My "external configuration management system" is pared down to a small script that symlinks everything in a directory into another directory (such as /etc/init). One thing missing from Nix is ability to store passwords and the like in the nix store, and from what I understand this applies to NixOS as well as using Nix outside NixOS. For example, every file in the nix store is world readable. https://github.com/NixOS/nix/issues/8 https://github.com/NixOS/nix/issues/8
      - vertex-four 12y ago
        NixOps gets around this to some extent by allowing you to define keys to be copied to /run/keys when configuring a remote system. They thus never touch the remote store.

jvdb 12y ago

This sounds really nice. I've been using apt as a deployment mechanism not unlike the use-cases describe. While the whole thing's been rock solid, the repo management could do with a cleaner interface. Looking forward to giving it a go!

dzderic 12y ago

For anyone who has ever battled with maintaining their own apt/yum repos, this seems like a godsend. The most-used tools for getting a package into your repo involve scp'ing the file to the repository server and running a command to update its index. It's nice to have a proper toolset to do this, but it's too bad I spend most of my time with YUM nowadays.

dima55 12y ago
man dput

oblio 12y ago

So, basically Nexus/Artifactory for deb. Cool.

Dewie 12y ago

Half of the posts in this thread are about the author's English. I know that clear communication and constructive feedback on one's command of a language is good, but... come on.

robinson-wall 12y ago

Props to smira + other contributors to aptly, it looks like it has come a long way in a short time.

helper 12y ago

We use deb-s3[1] to maintain a custom apt repository in s3. It supports signed release files and does all the hard work for you. [1]: https://github.com/krobertson/deb-s3 https://github.com/krobertson/deb-s3

smira 12y ago
aptly can do more much more than just S3 publishing: for example snapshotting, combining packages from different sources.

lamby 12y ago

https://apt-captain.readthedocs.org/en/latest/ https://apt-captain.readthedocs.org/en/latest/ Here's my solution to this. Love using .debs for distribution.

smira 12y ago
Thanks, looks really interesting! At the moment working on REST API for aptly which should allow similar simple publishing scenarios.
grosskur 12y ago
Looks awesome! I'm actually working on a SaaS product along similar lines. Right now it can build RPM and Debian packages directly from GitHub sources: https://www.packagelab.com/ https://www.packagelab.com/ Sort of like a cross between Launchpad and Travis CI. If anyone's interested in trying it out, I'd love feedback.

jcapote 12y ago

This looks neat, just in time for the weekend :) I've been using the free plan at https://packagecloud.io https://packagecloud.io for my debs (handles my rpms and gems too), and it's been working well so far.

zwilliamson 12y ago

Aptly rocks. I put this together for those who use Vagrant and want to test out some of Aptly's commands and functionality. https://github.com/sepulworld/aptly-vagrant https://github.com/sepulworld/aptly-vagrant It also sets up a Jenkins server so you can test out some build jobs that interact with Aptly. I am looking forward to Aptly's REST API. Also on the roadmap is the ability to manage Yum repositories.

vacri 12y ago

I just finished putting together an aptly repo system on Friday, with a 'testing', 'staging', and 'production' repo. Buildbot dumps unsigned builds into testing, and a meatspace process copies those builds into staging or production when appropriate. Aptly is a godsend to people who want to run .deb repos, as previously the software was basically "run a full mirror" or "sucks to be you". Jordan Sissel (fpm author) once remarked that 'there is a lot of silly ceremony in managing .debs'. We were previously using reprepro, which was alright, but had the glaring flaw of only allowing one version of a package to live in the repo. No easy rollbacks or machines on different versions there. Aptly is really flexible, and although it does have some corner cases or slightly unintuitive behaviour (to me), it has excellent documentation and is in heavy development. Kudos to you, smira.