4 ms·

Migrating old artifacts from JCenter to MavenCentral

34 points by prof18 6y ago

jillesvangurp 6y ago
I hope a more convenient way is created. I've deployed some projects via Sonatype in the past. And while they are friendly and responsive, it's a complicated process that involves manual approvals by them (for your first release), setting up gpg keys, etc. It takes days for this to happen. It's a big reason jCenter became so popular: it removed a lot of the friction for publishing a simple jar. One thing with jcenter is that you can actually propagate your jars to maven central if they meet minimum requirements (like having javadoc and sources and the right meta data). Many popular jars are actually propagated to maven central that way. I've done that with one of my projects and it already is in maven central. Once there, it will stay there. The issue here is not Sonatype but the fact that the Apache Foundation is understandably very selective about which companies they allow jars to propagate to maven central. Unlike python, ruby, node.js, etc. there is no public repository where you can sign up and publish your jars. Instead you publish to one of a select few of repositories they replicate from, which effectively means Sonatype or JCenter. For a long time, Sonatype's repository for OSS jars was the only way to get into maven central. Basically you publish your jars on their repository and then some time after that, maven central picks up the changes. I expect the Sonatype people will be very busy in the next months dealing with the fallout of JCenter disappearing. So, expect some delays. That may be a good moment for them to reconsider some of their processes around this. A more modern/user friendly way would be helpful. Maybe Amazon, Microsoft/Github, or Google could step up and provide something. Or all of them. Or maybe they can even work together to make that happen. After all, MS bought npm as well.
- prof18 6y ago
  Yeah, I agree. The process is not so easy and this can cause barriers to entry. And that's why 5 years ago I've published my first libraries on JCenter. Back then I was still at the university, never worked yet in the field and JCenter seemed to me the least painful path.
- habosa 6y ago
  +100 I've been doing Android development (incl. publishing libraries) since 2010 and the Android community is way behind every other community in the tools available for library publishers. iOS has Cocoapods and SPM, Web has NPM (and Yarn), Flutter has pub, etc. JCenter/Bintray made it possible for an independent dev to figure out how to share their utilities with others but even then it was too hard. Everyone has their own hacked together Gradle publishing script to assemble the right AARs, JARs, POMs, etc. But the Android community has a really bad habit of accepting things that are too hard because simplicity is for Bad Programmers. So we will accept this, and 10 years from now we'll continue to be in a place where Google and Square publish 90% of the good Android libraries and the community is still hugely underdeveloped.
- deleted 6y ago
  [deleted]
- KingOfCoders 6y ago
  I'd agree, same experience. But see the dependency confusion hack.
- brabel 6y ago
  If you think you don't need all the checks Sonatype performs for you even after this week's security exploit came up on npm and other less secure repos, you can use this for JVM libs: https://jitpack.io/ https://jitpack.io/ EDIT: notice that I am not criticizing jitpack, it's great to be able to point at a git repository and get its jar as a simple dependency... but it's not as secure as Maven Central IMO because it requires you to fully trust `jitpack.io` to not insert malicious code in the jars it serves, as well as the GitHub/BitBucket/GitLab account owner to not have been hacked or "become" malicious... still much better than npm's ridiculous resolution that will just trust anything at all.
  - jermo 6y ago
    A lot of projects publish to Central from cloud CIs. That means that private keys are stored in config/secrets. It is debatable whether that makes the artifacts more reliable since you have to trust the CI.
- merb 6y ago
  github did? https://github.com/features/packages https://github.com/features/packages
  - jillesvangurp 6y ago
    That does not propagate to maven central (at least not that I know). So you have to add a custom repository. You can also use jitpack.io or rsync artifacts to your websiter (I've done both).
vbezhenar 6y ago
I wish Java would move to more distributed artifact storage. Just specify URL and let your build tool download library there. Why does it have to be some kind of central storage? It's not like they perform some curating or static analysis or something like that.
- flurdy 6y ago
  Mostly to avoid a situation like this https://www.bleepingcomputer.com/news/security/researcher-hacks-over-35-tech-firms-in-novel-supply-chain-attack/ https://www.bleepingcomputer.com/news/security/researcher-ha... (This week's hack using npm, gems etc to trick non-java build tools to not use internal repos but the hacker's compromised packages instead)
  - kuschku 6y ago
    And with signed artifacts that wouldn't have happened. https://docs.gradle.org/current/userguide/dependency_verification.html#sec:trusting-keys-globally https://docs.gradle.org/current/userguide/dependency_verific... > Note that a variation of a compromised library is often name squatting, when a hacker would use GAV coordinates which look legit but are actually different by one character, or repository shadowing, when a dependency with the official GAV coordinates is published in a malicious repository which comes first in your build. Best part is that the documentation literally describes the attack used here. Another example where npm and co are reinventing the same issues that Java has already solved long ago.
- stepbeek 6y ago
  You have to prove ownership of a domain to publish to maven central fwiw.
- hocuspocus 6y ago
  > Just specify URL and let your build tool download library there. All build tools let you do just that. Many companies set up internal mirrors and encourage/force employees not to use Maven central and third party repositories. In fact, Maven central will blacklist IPs that generate too much traffic.
kasperni 6y ago
I hear a lot of complains about the Sonatype process being not trivial. Here is a background article about why: https://blog.sonatype.com/why-namespacing-matters-in-public-open-source-repositories https://blog.sonatype.com/why-namespacing-matters-in-public-...
- deleted 6y ago
  [deleted]
- jillesvangurp 6y ago
  And yet world + dog manages fine with a lot less complexity in the Rust, Php, Python, Javscript, Ruby, C++, docker, etc. communities. They all have tools for publishing packages and using them is straightforward and easy. JCenter managed to work around some of the maven limitations and make the process similarly usable (but you need their gradle or maven plugin). And of course they were allowed to sync packages to maven central. I've done this. It works and was very easy to setup. So, Sonatype can do better and they should consider doing so. Unless they look forward to manually approving gazillions of requests from stranded JCenter users in the next weeks. Maybe business is that good that they don't care about the overhead. Or alternatively, JFRog's move has prompted them to reconsider whether they want to keep on doing this at all. They are a for profit company after all and they must be similarly affected by big cloud providers putting up their own repositories. In any case, their process is expensive and convoluted; it always was. It suffers from them having had no competition for being gatekeepers for maven central. There are multiple alternate ways of verifying identity and ownership that don't involve manual approvals, keypairs, etc. It's what everybody else does. Including JCenter and whatever they did seems to have been acceptable to Apache (who own and run maven central).
  - kuschku 6y ago
    > Unless they look forward to manually approving gazillions of requests from stranded JCenter users in the next weeks. Maybe business is that good that they don't care about the overhead. Getting approved for a groupId took me literally 2 minutes. It's really not that complicated. Not sure yet about the process for actually getting releases approved, I'll see about that tomorrow.

34 points by prof18 6y ago

I hope a more convenient way is created. I've deployed some projects via Sonatype in the past. And while they are friendly and responsive, it's a complicated process that involves manual approvals by them (for your first release), setting up gpg keys, etc. It takes days for this to happen. It's a big reason jCenter became so popular: it removed a lot of the friction for publishing a simple jar. One thing with jcenter is that you can actually propagate your jars to maven central if they meet minimum requirements (like having javadoc and sources and the right meta data). Many popular jars are actually propagated to maven central that way. I've done that with one of my projects and it already is in maven central. Once there, it will stay there. The issue here is not Sonatype but the fact that the Apache Foundation is understandably very selective about which companies they allow jars to propagate to maven central. Unlike python, ruby, node.js, etc. there is no public repository where you can sign up and publish your jars. Instead you publish to one of a select few of repositories they replicate from, which effectively means Sonatype or JCenter. For a long time, Sonatype's repository for OSS jars was the only way to get into maven central. Basically you publish your jars on their repository and then some time after that, maven central picks up the changes. I expect the Sonatype people will be very busy in the next months dealing with the fallout of JCenter disappearing. So, expect some delays. That may be a good moment for them to reconsider some of their processes around this. A more modern/user friendly way would be helpful. Maybe Amazon, Microsoft/Github, or Google could step up and provide something. Or all of them. Or maybe they can even work together to make that happen. After all, MS bought npm as well.

prof18 6y ago
Yeah, I agree. The process is not so easy and this can cause barriers to entry. And that's why 5 years ago I've published my first libraries on JCenter. Back then I was still at the university, never worked yet in the field and JCenter seemed to me the least painful path.
habosa 6y ago
+100 I've been doing Android development (incl. publishing libraries) since 2010 and the Android community is way behind every other community in the tools available for library publishers. iOS has Cocoapods and SPM, Web has NPM (and Yarn), Flutter has pub, etc. JCenter/Bintray made it possible for an independent dev to figure out how to share their utilities with others but even then it was too hard. Everyone has their own hacked together Gradle publishing script to assemble the right AARs, JARs, POMs, etc. But the Android community has a really bad habit of accepting things that are too hard because simplicity is for Bad Programmers. So we will accept this, and 10 years from now we'll continue to be in a place where Google and Square publish 90% of the good Android libraries and the community is still hugely underdeveloped.
deleted 6y ago
[deleted]
KingOfCoders 6y ago
I'd agree, same experience. But see the dependency confusion hack.
brabel 6y ago
If you think you don't need all the checks Sonatype performs for you even after this week's security exploit came up on npm and other less secure repos, you can use this for JVM libs: https://jitpack.io/ https://jitpack.io/ EDIT: notice that I am not criticizing jitpack, it's great to be able to point at a git repository and get its jar as a simple dependency... but it's not as secure as Maven Central IMO because it requires you to fully trust `jitpack.io` to not insert malicious code in the jars it serves, as well as the GitHub/BitBucket/GitLab account owner to not have been hacked or "become" malicious... still much better than npm's ridiculous resolution that will just trust anything at all.
- jermo 6y ago
  A lot of projects publish to Central from cloud CIs. That means that private keys are stored in config/secrets. It is debatable whether that makes the artifacts more reliable since you have to trust the CI.
merb 6y ago
github did? https://github.com/features/packages https://github.com/features/packages
- jillesvangurp 6y ago
  That does not propagate to maven central (at least not that I know). So you have to add a custom repository. You can also use jitpack.io or rsync artifacts to your websiter (I've done both).

vbezhenar 6y ago

I wish Java would move to more distributed artifact storage. Just specify URL and let your build tool download library there. Why does it have to be some kind of central storage? It's not like they perform some curating or static analysis or something like that.

flurdy 6y ago
Mostly to avoid a situation like this https://www.bleepingcomputer.com/news/security/researcher-hacks-over-35-tech-firms-in-novel-supply-chain-attack/ https://www.bleepingcomputer.com/news/security/researcher-ha... (This week's hack using npm, gems etc to trick non-java build tools to not use internal repos but the hacker's compromised packages instead)
- kuschku 6y ago
  And with signed artifacts that wouldn't have happened. https://docs.gradle.org/current/userguide/dependency_verification.html#sec:trusting-keys-globally https://docs.gradle.org/current/userguide/dependency_verific... > Note that a variation of a compromised library is often name squatting, when a hacker would use GAV coordinates which look legit but are actually different by one character, or repository shadowing, when a dependency with the official GAV coordinates is published in a malicious repository which comes first in your build. Best part is that the documentation literally describes the attack used here. Another example where npm and co are reinventing the same issues that Java has already solved long ago.
stepbeek 6y ago
You have to prove ownership of a domain to publish to maven central fwiw.
hocuspocus 6y ago
> Just specify URL and let your build tool download library there. All build tools let you do just that. Many companies set up internal mirrors and encourage/force employees not to use Maven central and third party repositories. In fact, Maven central will blacklist IPs that generate too much traffic.

kasperni 6y ago

I hear a lot of complains about the Sonatype process being not trivial. Here is a background article about why: https://blog.sonatype.com/why-namespacing-matters-in-public-open-source-repositories https://blog.sonatype.com/why-namespacing-matters-in-public-...

deleted 6y ago
[deleted]
jillesvangurp 6y ago
And yet world + dog manages fine with a lot less complexity in the Rust, Php, Python, Javscript, Ruby, C++, docker, etc. communities. They all have tools for publishing packages and using them is straightforward and easy. JCenter managed to work around some of the maven limitations and make the process similarly usable (but you need their gradle or maven plugin). And of course they were allowed to sync packages to maven central. I've done this. It works and was very easy to setup. So, Sonatype can do better and they should consider doing so. Unless they look forward to manually approving gazillions of requests from stranded JCenter users in the next weeks. Maybe business is that good that they don't care about the overhead. Or alternatively, JFRog's move has prompted them to reconsider whether they want to keep on doing this at all. They are a for profit company after all and they must be similarly affected by big cloud providers putting up their own repositories. In any case, their process is expensive and convoluted; it always was. It suffers from them having had no competition for being gatekeepers for maven central. There are multiple alternate ways of verifying identity and ownership that don't involve manual approvals, keypairs, etc. It's what everybody else does. Including JCenter and whatever they did seems to have been acceptable to Apache (who own and run maven central).
- kuschku 6y ago
  > Unless they look forward to manually approving gazillions of requests from stranded JCenter users in the next weeks. Maybe business is that good that they don't care about the overhead. Getting approved for a groupId took me literally 2 minutes. It's really not that complicated. Not sure yet about the process for actually getting releases approved, I'll see about that tomorrow.