9 ms·

The new contribution workflow for Gnome

284 points by bpierre 10y ago

gyger 10y ago
Awesome work done by the community and the Flatpak/Builder guys
hashhar 10y ago
I'm so happy. The last time I tried setting up a GNOME dev environment I ran into two major issues. Using jhbuild to download dependencies doesn't work if your network blocks git:// URLs. I had to hunt a lot to learn to tell jhbuild to use https:// https:// or ssh:// URLs instead. It still broke when updating a dependency and only worked for toplevel explicit dependencies. The second was the extremely long process of getting the dev environment set up compared to the ease with which Mozilla does it. Both of those issues look fixed now and make the ecosystem all the more inviting for me to hack on. I even think you guys bested Mozilla on that second point. I'll try it out and report back. PS: Is this tied in some way to Ubuntu or can other systems get in on the fun as well? Arch Linux does have a gnome-unstable repo so I can grab the latest GNOME Builder.
- snuxoll 10y ago
  Assuming this will be part of GNOME 3.24 it should be in Fedora 26 and Ubuntu 17.04. Other distributions will depend on when GNOME 3.24 packages land.
  - hashhar 10y ago
    Cool. Arch's gnome-unstable repo ships git master builds for GNOME 3.24. I'm gonna go set it up now.
    - ronjouch 10y ago
      Also, for those wondering, 3.24 is coming in stable Arch repos after the rebuild-the-universe move to OpenSSL 1.1.0. Source: r/arch, https://www.reddit.com/r/archlinux/comments/633jgj/whats_the_current_progress_on_gnome_324_in_arch/ https://www.reddit.com/r/archlinux/comments/633jgj/whats_the... and https://www.reddit.com/r/archlinux/comments/640wl0/why_isnt_gnome_324_available/ https://www.reddit.com/r/archlinux/comments/640wl0/why_isnt_...
- deleted 10y ago
  [deleted]
- avar 10y ago
  Unrelated to your jhbuild issues, you can often solve this class of issue in git by teaching git itself to search-replace the URL with "url.<base>.insteadOf". See the git-config manpage.
  - hashhar 10y ago
    I tried that at first but the issue was that URLs didn't map perfectly. A url like git://freedesktop.org/harfbuzz.git mapped to https://git.freedesktop.org/projects/harfbuzz.git https://git.freedesktop.org/projects/harfbuzz.git or something to that end.
    - avar 10y ago
      It doesn't have to map perfectly as long as both end in the same suffix. Test with: git -c url.https://git.freedesktop.org/projects/.insteadOf=git://freedesktop.org/ clone git://freedesktop.org/harfbuzz.git Cloning into 'harfbuzz'... fatal: unable to access 'https://git.freedesktop.org/projects/harfbuzz.git/': Failed to connect to git.freedesktop.org port 443: Connection refused Then set it in your config: git config --global url.https://git.freedesktop.org/projects/.insteadOf git://freedesktop.org/ And now anything on your system cloning or fetching git://freedesktop.org/harfbuzz.git will use https://git.freedesktop.org/projects/harfbuzz.git https://git.freedesktop.org/projects/harfbuzz.git instead.
      - hashhar 10y ago
        Wow. Thanks. I guess it will work for all git commands (including the clone --recursive) ones? EDIT: Yup. Works great. I'll check with jhbuild and add to the wiki if it works.
    - jwilk 10y ago
      You can copy&paste config for freedesktop.org (and other popular git repos) from here: https://github.com/jwilk/https-everygit/blob/master/gitconfig https://github.com/jwilk/https-everygit/blob/master/gitconfi...
deleted 10y ago
[deleted]
rukittenme 10y ago
Flatpak, builder, Ubuntu contributing to Gnome. 2017 is a great year for Gnome. I think these improved devtools will really help adoption. There are tons of developers (like myself) who don't make apps for linux because its too much effort. The easier it is the less excuses I have.
- chrisper 10y ago
  Won't Ubuntu still use Snap though?
  - johnny22 10y ago
    the builder folks said they would accept a snap plugin, so snaps will work fine with this setup.
    - audidude 10y ago
      The flatpak support is provided as a plugin. So assuming they do the same work and commit to maintaining it upstream, I'll accept it.
      - sandGorgon 10y ago
        @christian - what do you personally think is the long term future of snap vs flatpack. Is it going to be another 30 years of 2 different packaging formats.
        AsyncAwait 10y ago
        Not OP, but Flatpack I think Snaps will go the way of Mir in due time, Flatpack is integrated into GNOME.
        sandGorgon 10y ago
        The opportunity for Linux is now - unify the future packaging format. It doesnt matter to me which one wins, as long as there is one. life will be infinitely easier for people who will package linux software.
tux1968 10y ago
A minor nitpick to be sure, but it's odd that you must close down all other copies of your application before your development copy will launch. There is probably a way to work around this, but the limitation is hard enough to get around that it is mentioned in the Builder instructions. This seems like an unnecessary limitation of the Gnome environment. It would be nice to be able to compare your existing release to your changed copy and run them side by side.
- snuxoll 10y ago
  > This seems like an unnecessary limitation of the Gnome environment. It would be nice to be able to compare your existing release to your changed copy and run them side by side. This is because GNOME applications have a hybrid of Windows and OS X behaviors now, due to the addition of D-Bus activation. Applications still exit when you close the last window, but each application has a unique identifier (org.gnome.gedit, etc) and when the binary starts it will search for a running instance and tell it to activate if present.
- audidude 10y ago
  The problem at hand is that applications are "single instance" by default. For Builder 3.26 we'll get something in place that starts a dbus-session to separate the built application from your host system. Another option is to finish the simulator, but I have a debugger to finish first.
haddr 10y ago
I'm curious what is the trick in building the whole Nautilus in under 5 minutes? I mean how do this new build tool accomplish that?
- mwleeds 10y ago
  Flatpak lets applications share a common set of libraries, called a runtime. So org.gnome.Platform has all the libraries like glib and gtk+ that pretty much all GNOME applications use, and all that's left to build are the ones that the application bundles that aren't in the runtime.
- csoriano 10y ago
  As audidude mentioned, but to add to it a little, is basically we have an SDK now. So you download (with delta update etc and sharing Sdk code) what's needed instead of compiling etc.
Gudin 10y ago
There are no requirements to start development. It's an advantage if you know a bit of object oriented programming and git. Reading this sound so encouraging. I love open-source, but as junior dev, I feel like it's super hard to contribute and everyone expects that you already know all the advanced level stuff.
- audidude 10y ago
  The GNOME project is a great group of people to develop your skill-set with. We can help find you projects to work on and there is pretty much everything you can imagine that needs doing. #newcomers on irc.gnome.org is a great place to start.
haidermalik12 10y ago
Better ways to write asynchronous code in Node.js http://fullstackhour.com/better-ways-to-write-asynchronous-code-in-node-js/ http://fullstackhour.com/better-ways-to-write-asynchronous-c...
xfs 10y ago
I'm not quite sure what's going on with this website as it is flooded with ads. http://i.imgur.com/R6ICEgA.png http://i.imgur.com/R6ICEgA.png I've been quite tolerant of ads myself without using an ad blocker for a long time but recently this kind of stuff with the whole page being clogged up by loading ads is getting out of hand.
- notatoad 10y ago
  wow, that's really terrible - i went and turned off my adblocker just to see how bad it is, and it appears to only be loading two ad units at the bottom of the article - as far as ads go, it's pretty unintrusive. but somehow they manage to load those two ads in a way that completely throttles the CPU on my poor little chromebook.
  - csoriano 10y ago
    Wait what? Is my blog full of ads?? I was not aware. How can I see them? Any alternative to Wordpress?
    - notatoad 10y ago
      it's not full of ads, it's actually not that bad: http://imgur.com/a/JQp7r http://imgur.com/a/JQp7r but when i turn off my adblocker, it's just constantly making network requests for seemingly no reason: http://imgur.com/a/QCQSU http://imgur.com/a/QCQSU if it isn't something you've set up, it must be wordpress.com's ad network.
msl09 10y ago
The most impressive fact about this is that it actually works as advertised, though it took a little longer to build nautilus, more like 15 minutes. This is so cool that I'm actually tempted to use it for non-gnome apps.
- audidude 10y ago
  If you have any issues (no doubt we still have bugs) feel free to ping us on irc.gnome.org in #gnome-builder. I'm usually around hacking on things.
- csoriano 10y ago
  Right, I should had clarify from where the 5 minutes come from and what kind of machine. I upgraded few months ago to a new machine (200MB/s internet, latest intel CPU + SSD, quite clearly not the most common set up)
gue5t 10y ago
Is it possible to use this to contribute to Gnome Shell or GTK+? Or is "contribution" limited to applications?
- audidude 10y ago
  I use it regularly to hack on GLib, Gtk+, Builder, Shell, Mutter, Sysprof, etc. The primary issue is that for things that are libraries or system services, we don't have a way to install that and run some sort of test suite. We have a GSoC/Outreachy project for the unit testing integration, so that is not far off. My eventual goal is to tie in GNOME continuous so that we can have all the base OS bits and just overlay the systems-level project (be it gtk, shell, boot services, etc). And then one keyboard shortcut to build/deploy/launch/debug. Should work for remote devices too.
- csoriano 10y ago
  The Flatpak version is not ready for that, no. That's why we link to JHBuild too, for the time being. Of course Christian is working on trying to make that work. In any case, most probably newcomers (the target audience of the initiative for now) won't probably hack on those anyway. In the future we would like to have a generic guide for contribution, apart of just newcomers apps, but first we had to figure out the base. Now that it's done, we can move up and up in difficulty.
uiri 10y ago
All with an UI and integrated, no terminal required. What if I just want to git clone something and run make (or similar) ? I don't see any documentation for that workflow at all. My usual development workflow is Emacs + git + commandline tools. Perhaps I'm simply not the target audience.
- audidude 10y ago
  git clone $project_url flatpak-builder builddir org.gnome.Foo.json
- mwleeds 10y ago
  Builder does have support for building projects that use make without autotools, yes.
- csoriano 10y ago
  It's in the first sentence of https://wiki.gnome.org/Newcomers/BuildProject https://wiki.gnome.org/Newcomers/BuildProject .
josteink 10y ago
From 6 hours distro specific bootstrapping to a 5 minutes generic flow? That's just extremely impressive. Even as a Fedora and Gnome user I've never even considered contributing to Gnome. I've just assumed (so far rightly) that it's just too big and heavy to work with. Things like this may certainly change that. Great job!
tmsldd 10y ago
Thanks guys!! Specially to Alex and Christian.. Gnome project just got one more contributor ;)
chris_wot 10y ago
Hmmm... I wonder if I could get LibreOffice to build like that... Actually, I wonder if anyone has ever tried to edit, compile and build LibreOffice in Builder? I know it can be done in KDevelop...
- csoriano 10y ago
  LibreOffice uses Flatpak too, so it should be as doable as any of the ones we have. But I didn't try myself :)
stuaxo 10y ago
This is great. Every few years I've got it into my head I want to contribute, and each time getting everything working on Ubuntu has flummoxed me after a few hours.

284 points by bpierre 10y ago

gyger 10y ago

Awesome work done by the community and the Flatpak/Builder guys

I'm so happy. The last time I tried setting up a GNOME dev environment I ran into two major issues. Using jhbuild to download dependencies doesn't work if your network blocks git:// URLs. I had to hunt a lot to learn to tell jhbuild to use https:// https:// or ssh:// URLs instead. It still broke when updating a dependency and only worked for toplevel explicit dependencies. The second was the extremely long process of getting the dev environment set up compared to the ease with which Mozilla does it. Both of those issues look fixed now and make the ecosystem all the more inviting for me to hack on. I even think you guys bested Mozilla on that second point. I'll try it out and report back. PS: Is this tied in some way to Ubuntu or can other systems get in on the fun as well? Arch Linux does have a gnome-unstable repo so I can grab the latest GNOME Builder.

snuxoll 10y ago
Assuming this will be part of GNOME 3.24 it should be in Fedora 26 and Ubuntu 17.04. Other distributions will depend on when GNOME 3.24 packages land.
- hashhar 10y ago
  Cool. Arch's gnome-unstable repo ships git master builds for GNOME 3.24. I'm gonna go set it up now.
  - ronjouch 10y ago
    Also, for those wondering, 3.24 is coming in stable Arch repos after the rebuild-the-universe move to OpenSSL 1.1.0. Source: r/arch, https://www.reddit.com/r/archlinux/comments/633jgj/whats_the_current_progress_on_gnome_324_in_arch/ https://www.reddit.com/r/archlinux/comments/633jgj/whats_the... and https://www.reddit.com/r/archlinux/comments/640wl0/why_isnt_gnome_324_available/ https://www.reddit.com/r/archlinux/comments/640wl0/why_isnt_...
deleted 10y ago
[deleted]
avar 10y ago
Unrelated to your jhbuild issues, you can often solve this class of issue in git by teaching git itself to search-replace the URL with "url.<base>.insteadOf". See the git-config manpage.
- hashhar 10y ago
  I tried that at first but the issue was that URLs didn't map perfectly. A url like git://freedesktop.org/harfbuzz.git mapped to https://git.freedesktop.org/projects/harfbuzz.git https://git.freedesktop.org/projects/harfbuzz.git or something to that end.
  - avar 10y ago
    It doesn't have to map perfectly as long as both end in the same suffix. Test with: git -c url.https://git.freedesktop.org/projects/.insteadOf=git://freedesktop.org/ clone git://freedesktop.org/harfbuzz.git Cloning into 'harfbuzz'... fatal: unable to access 'https://git.freedesktop.org/projects/harfbuzz.git/': Failed to connect to git.freedesktop.org port 443: Connection refused Then set it in your config: git config --global url.https://git.freedesktop.org/projects/.insteadOf git://freedesktop.org/ And now anything on your system cloning or fetching git://freedesktop.org/harfbuzz.git will use https://git.freedesktop.org/projects/harfbuzz.git https://git.freedesktop.org/projects/harfbuzz.git instead.
    - hashhar 10y ago
      Wow. Thanks. I guess it will work for all git commands (including the clone --recursive) ones? EDIT: Yup. Works great. I'll check with jhbuild and add to the wiki if it works.
  - jwilk 10y ago
    You can copy&paste config for freedesktop.org (and other popular git repos) from here: https://github.com/jwilk/https-everygit/blob/master/gitconfig https://github.com/jwilk/https-everygit/blob/master/gitconfi...

deleted 10y ago

[deleted]

rukittenme 10y ago

Flatpak, builder, Ubuntu contributing to Gnome. 2017 is a great year for Gnome. I think these improved devtools will really help adoption. There are tons of developers (like myself) who don't make apps for linux because its too much effort. The easier it is the less excuses I have.

chrisper 10y ago
Won't Ubuntu still use Snap though?
- johnny22 10y ago
  the builder folks said they would accept a snap plugin, so snaps will work fine with this setup.
  - audidude 10y ago
    The flatpak support is provided as a plugin. So assuming they do the same work and commit to maintaining it upstream, I'll accept it.
    - sandGorgon 10y ago
      @christian - what do you personally think is the long term future of snap vs flatpack. Is it going to be another 30 years of 2 different packaging formats.
      - AsyncAwait 10y ago
        Not OP, but Flatpack I think Snaps will go the way of Mir in due time, Flatpack is integrated into GNOME.
        sandGorgon 10y ago
        The opportunity for Linux is now - unify the future packaging format. It doesnt matter to me which one wins, as long as there is one. life will be infinitely easier for people who will package linux software.

tux1968 10y ago

A minor nitpick to be sure, but it's odd that you must close down all other copies of your application before your development copy will launch. There is probably a way to work around this, but the limitation is hard enough to get around that it is mentioned in the Builder instructions. This seems like an unnecessary limitation of the Gnome environment. It would be nice to be able to compare your existing release to your changed copy and run them side by side.

snuxoll 10y ago
> This seems like an unnecessary limitation of the Gnome environment. It would be nice to be able to compare your existing release to your changed copy and run them side by side. This is because GNOME applications have a hybrid of Windows and OS X behaviors now, due to the addition of D-Bus activation. Applications still exit when you close the last window, but each application has a unique identifier (org.gnome.gedit, etc) and when the binary starts it will search for a running instance and tell it to activate if present.
audidude 10y ago
The problem at hand is that applications are "single instance" by default. For Builder 3.26 we'll get something in place that starts a dbus-session to separate the built application from your host system. Another option is to finish the simulator, but I have a debugger to finish first.

haddr 10y ago

I'm curious what is the trick in building the whole Nautilus in under 5 minutes? I mean how do this new build tool accomplish that?

mwleeds 10y ago
Flatpak lets applications share a common set of libraries, called a runtime. So org.gnome.Platform has all the libraries like glib and gtk+ that pretty much all GNOME applications use, and all that's left to build are the ones that the application bundles that aren't in the runtime.
csoriano 10y ago
As audidude mentioned, but to add to it a little, is basically we have an SDK now. So you download (with delta update etc and sharing Sdk code) what's needed instead of compiling etc.

Gudin 10y ago

There are no requirements to start development. It's an advantage if you know a bit of object oriented programming and git. Reading this sound so encouraging. I love open-source, but as junior dev, I feel like it's super hard to contribute and everyone expects that you already know all the advanced level stuff.

audidude 10y ago
The GNOME project is a great group of people to develop your skill-set with. We can help find you projects to work on and there is pretty much everything you can imagine that needs doing. #newcomers on irc.gnome.org is a great place to start.

haidermalik12 10y ago

Better ways to write asynchronous code in Node.js http://fullstackhour.com/better-ways-to-write-asynchronous-code-in-node-js/ http://fullstackhour.com/better-ways-to-write-asynchronous-c...

xfs 10y ago

I'm not quite sure what's going on with this website as it is flooded with ads. http://i.imgur.com/R6ICEgA.png http://i.imgur.com/R6ICEgA.png I've been quite tolerant of ads myself without using an ad blocker for a long time but recently this kind of stuff with the whole page being clogged up by loading ads is getting out of hand.

notatoad 10y ago
wow, that's really terrible - i went and turned off my adblocker just to see how bad it is, and it appears to only be loading two ad units at the bottom of the article - as far as ads go, it's pretty unintrusive. but somehow they manage to load those two ads in a way that completely throttles the CPU on my poor little chromebook.
- csoriano 10y ago
  Wait what? Is my blog full of ads?? I was not aware. How can I see them? Any alternative to Wordpress?
  - notatoad 10y ago
    it's not full of ads, it's actually not that bad: http://imgur.com/a/JQp7r http://imgur.com/a/JQp7r but when i turn off my adblocker, it's just constantly making network requests for seemingly no reason: http://imgur.com/a/QCQSU http://imgur.com/a/QCQSU if it isn't something you've set up, it must be wordpress.com's ad network.

msl09 10y ago

The most impressive fact about this is that it actually works as advertised, though it took a little longer to build nautilus, more like 15 minutes. This is so cool that I'm actually tempted to use it for non-gnome apps.

audidude 10y ago
If you have any issues (no doubt we still have bugs) feel free to ping us on irc.gnome.org in #gnome-builder. I'm usually around hacking on things.
csoriano 10y ago
Right, I should had clarify from where the 5 minutes come from and what kind of machine. I upgraded few months ago to a new machine (200MB/s internet, latest intel CPU + SSD, quite clearly not the most common set up)

gue5t 10y ago

Is it possible to use this to contribute to Gnome Shell or GTK+? Or is "contribution" limited to applications?

audidude 10y ago
I use it regularly to hack on GLib, Gtk+, Builder, Shell, Mutter, Sysprof, etc. The primary issue is that for things that are libraries or system services, we don't have a way to install that and run some sort of test suite. We have a GSoC/Outreachy project for the unit testing integration, so that is not far off. My eventual goal is to tie in GNOME continuous so that we can have all the base OS bits and just overlay the systems-level project (be it gtk, shell, boot services, etc). And then one keyboard shortcut to build/deploy/launch/debug. Should work for remote devices too.
csoriano 10y ago
The Flatpak version is not ready for that, no. That's why we link to JHBuild too, for the time being. Of course Christian is working on trying to make that work. In any case, most probably newcomers (the target audience of the initiative for now) won't probably hack on those anyway. In the future we would like to have a generic guide for contribution, apart of just newcomers apps, but first we had to figure out the base. Now that it's done, we can move up and up in difficulty.

uiri 10y ago

All with an UI and integrated, no terminal required. What if I just want to git clone something and run make (or similar) ? I don't see any documentation for that workflow at all. My usual development workflow is Emacs + git + commandline tools. Perhaps I'm simply not the target audience.

audidude 10y ago
git clone $project_url flatpak-builder builddir org.gnome.Foo.json
mwleeds 10y ago
Builder does have support for building projects that use make without autotools, yes.
csoriano 10y ago
It's in the first sentence of https://wiki.gnome.org/Newcomers/BuildProject https://wiki.gnome.org/Newcomers/BuildProject .

josteink 10y ago

From 6 hours distro specific bootstrapping to a 5 minutes generic flow? That's just extremely impressive. Even as a Fedora and Gnome user I've never even considered contributing to Gnome. I've just assumed (so far rightly) that it's just too big and heavy to work with. Things like this may certainly change that. Great job!

tmsldd 10y ago

Thanks guys!! Specially to Alex and Christian.. Gnome project just got one more contributor ;)

chris_wot 10y ago

Hmmm... I wonder if I could get LibreOffice to build like that... Actually, I wonder if anyone has ever tried to edit, compile and build LibreOffice in Builder? I know it can be done in KDevelop...

csoriano 10y ago
LibreOffice uses Flatpak too, so it should be as doable as any of the ones we have. But I didn't try myself :)

stuaxo 10y ago

This is great. Every few years I've got it into my head I want to contribute, and each time getting everything working on Ubuntu has flummoxed me after a few hours.