8 ms·
AWS CodeArtifact: A fully managed software artifact repository service
- tkinz27 6y agoIt’s frustrating to not see more system package management (deb, rpm) from these new services (github and gitlab for instance). Are others not packaging their code in intermediate packages before packing them into containers?
- wmf 6y agoThat sounds like double work.
- manigandham 6y agoWhat's the purpose of intermediate packages if you're already using containers?
- tkinz27 6y agoVery large c++/python/cuda application that is packed into various different images (squashfs images, but functionally the same). We end up having a lot of libraries that are shared across multiple images.
- manigandham 6y agoWould it not be easier to just pack into different base images? Docker is very efficient with reusing these layers.
- Jtsummers 6y agoIntermediate packages permit you to choose different deployment situations later, with minimal additional cost now. Tying everything to Docker images ties you to Docker and removes your ability to transition to other systems. It may not be worth the cost now, but as soon as you want to deploy on more than one platform it can become critical to maintaining momentum (vice having to hand tailor deployment for each new environment).
- deleted 6y ago[deleted]
- blaisio 6y agoMost people don't need to do that. You can build things you need as part of the image build. No need to setup a deb or rpm package unless you're also installing it that way somewhere else.
- asguy 6y agoWe've been going that direction. Packages integrate better into multiple use cases (e.g. VM images, containers). Running a properly signed apt repo is easy these days, so why not? For people that disagree with this model: where do you think the the software comes from when you apt/apk install things inside your Dockerfile?
- secondcoming 6y agoWe use jfrog. One jenkins job builds our code into a .deb and pushes it there. Another job builds the VM image which is then deployed once testing passes.
- FrenchTouch42 6y agoI'd like really like to see more support added (Ruby, etc). It could be a great alternative to Artifactory.
- WatchDog 6y agoThis has been a fairly obvious service that has been missing for a while, nice to see them provide a solution. Most dependency management tools have some kind of hacky support for using S3 directly. Full fledged artifact management tools like Artifactory and Nexus support S3 backed storage. Interesting to see that the pricing is approximately double that of S3, for what I imagine is not much more than a thin layer on top of it.
- djhaskin987 6y agoTo add to your list of Artifactory and Nexus, Pulp[1] is also a cool project in this space, and is fully open source. Honestly the fact that they only support javascript, Python and Java is pretty bare bones compared to what the others on the above list support, and again as you say, for a fairly high price. 1: https://pulpproject.org/ https://pulpproject.org/
- entee 6y ago> Interesting to see that the pricing is approximately double that of S3, for what I imagine is not much more than a thin layer on top of it. Haven’t looked carefully, but is there a difference in the guarantees it provides? Might be a performance or SLA difference.
- thayne 6y agoIt looks like the SLAs are about the same (https://aws.amazon.com/s3/sla/ https://aws.amazon.com/s3/sla/ and https://aws.amazon.com/codeartifact/sla/ https://aws.amazon.com/codeartifact/sla/). I haven't seen any documentation on garantees for performance for either service, but I'm skeptical this will perform any better than s3.
- StreamBright 6y agoWe have used S3 successfully several times. You can create a Maven repository, use it as RPM repo and many other use cases to host artifacts. I am not sure what functionality is missing that cannot be implemented on the top of S3 and requires CodeArtifact.
- dahfizz 6y agoI don't get it. The git server you use supports artifacts already. You could also just put all of your artifacts on an S3 bucket if you needed somewhere to put them, which is exactly what this is but more expensive. I don't understand when this would save you money or simplify devops.
- dmlittle 6y agoThe benefit is being able to keep your existing maven/npm/pip workflows as well as use the same workflow for both internal and public dependencies.
- dahfizz 6y agoI still don't see what's different. I can configure pip to look at my git server, so that all I have to do is `pip install my_thing` and it will automatically download all public and private deps. I don't know what you mean by "workflow" in this context but this is just about as simple as can be.
- deleted 6y ago[deleted]
- code4tee 6y agoYou’re not the target user here. In highly secure environments you can’t just “pip install your-thing”.
- baq 6y agoLooks like you’re assuming you have some kind of access to any part of the internet you please. I envy you because most tools just work in this case. Not so on enterprise networks.
- code4tee 6y agoCan occur in a VPC without direct internet access. For the average developer this isn’t usually an issue but in highly secure corporate environments this helps a lot. Can’t just do pip install X in such situations. Even the S3 proxy solutions often require many hoops from the security Jedi council before you can use any packages there. A lot of people won’t find this useful but for some it’s a big blessing.
- saxonww 6y agoAppears to support ivy/gradle/maven, npm/yarn, and pip/twine only.
- antoncohen 6y agoThe login credentials expire after 12 hours (or less)[1], just like with their Docker registry (ECR). That makes it pretty annoying to use, especially on developer laptops. GCP has a similar offering[2]. And GitHub[3]. [1] https://docs.aws.amazon.com/codeartifact/latest/ug/python-configure.html https://docs.aws.amazon.com/codeartifact/latest/ug/python-co... [2] https://cloud.google.com/artifact-registry https://cloud.google.com/artifact-registry [3] https://github.com/features/packages https://github.com/features/packages
- toomuchtodo 6y agoYou should have a shell alias to rapidly top up your auth token, just like with the Docker ECR. Short lived tokens are best practice, and a 12 hour TTL is reasonable. That’s no more than two auths in a day as a dev.
- antoncohen 6y agoAnd every developer needs to have that alias. And all automation needs to be changed to call that command before trying to use pip, or mvn, or whatever. It sucks. No other hosted artifact repository does this.
- toomuchtodo 6y agoIt’s roughly a dozen lines of bash (error handling and all), speaking as someone who has had to maintain dev tooling for an org where Docker ECR was used, and can be checked into your project’s repo. It’s not onerous at all, either on devs or your build and deployment pipelines/runners.
- cle 6y agoIf only devs had a way to share code with one another...
- deleted 6y ago[deleted]
- blaisio 6y ago
- lflux 6y agoYou know it's an AWS service when you look at it and go "Huh, it's only 2x the price of S3, what a bargain!"
- scarface74 6y agoNo C#/Nuget support? Really?
- politelemon 6y agoThat is strange, I wonder if that's coming later but I didn't see anything to that effect. I'd also have liked to see docker image support (despite ecr) and raw binaries too.
- jen20 6y agoMy guess (purely a guess though) is that this is a good proportion of the platforms AWS use internally, and that this service will expand to other ecosystems less used internally in response to customer demand.
- StreamBright 6y agoWeird when you can just do it using S3 for 50% of the price. https://github.com/emgarten/sleet https://github.com/emgarten/sleet
- scarface74 6y agoStatic feeds are much slower than one that use a real server.
- StreamBright 6y agoAny reason why? Could we not make it faster?
- scarface74 6y agoIt’s been awhile since I tried a static feed. But basically, the client NuGet command had to read the directory structure to find all of the NuGet packages and versions instead of using an API where the server had everything indexed already.
- pskinner 6y agoIs it just me or is this missing plain artifacts - those that are not packaged for a specific tool? I'm thinking of plain binaries and resources required for things like db build tools and automated testing tools - just files really. How do I publish a tarball up to this, for example? Also the lack of nuget is a major issue.
- greyskull 6y agoI think CodeArtifact loses value when you aren't using a package manager; the benefit is an api-compatible service with various controls and audits built on top. Out of curiosity, what would you want from this service for the "plain binary" use-case when S3 already exists?
- ec109685 6y agoIt’s nice having the metadata around the push available versus raw blobs to s3.
- bostik 6y agoObjects in S3 can have custom metadata associated with them. Look at the returned data for the HeadObject call.[0] It's not advertised in the documentation, but HeadObject(Bucket, Key)['Metadata'] is a neat dictionary of custom values. 0: https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObje...
- camhart 6y agoS3 supports metadata (see https://docs.aws.amazon.com/AmazonS3/latest/user-guide/add-object-metadata.html https://docs.aws.amazon.com/AmazonS3/latest/user-guide/add-o...). Perhaps I don't understand what you're saying fully though--as I don't fully understand your comment.
- StreamBright 6y agoYou mean like Object Metadata for S3? https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadat...
- StreamBright 6y agoWhat is wrong with S3?
- soygul 6y agoSeems like a direct competitor for Artifactory and Nexus. I wonder if it is profitable for them to create an inferior alternative to fully flagged artifact managers. Or if they are doing this for product-completeness of AWS.
- andycowley 6y agoNo deb, RPM, or nuget. Half a product really. As annoying and expensive as Nexus and Artifactory are, at least they're more fully featured.
- doliveira 6y agoI'd wait a few years to be ready, AWS developer tools are really crude. Last year I had to build a Lambda to be able to spit multiple output artifacts in CodePipeline.