8 ms·

Native Mac APIs for Go

480 points by maydemir 6y ago

coetry 6y ago
This looks really cool! Is there any overhead with using these bindings as opposed to using Swift and calling the APIs directly?
- axaxs 6y ago
  > Is there any overhead with using these bindings Not the author, but yes. Go calling C is expensive, and that's precisely how this works. Whether or not that matters for your use case is another topic.
  - lsllc 6y ago
    There's a bit of a write-up here: https://www.cockroachlabs.com/blog/the-cost-and-complexity-of-cgo/ https://www.cockroachlabs.com/blog/the-cost-and-complexity-o... In their benchmark, calling a `func() {}` in Go vs a `void foo() {}` in C (via CGo) is almost 100x faster. $ go test -bench . -gcflags '-l' # disable inlining for fairness BenchmarkCGO-8 10000000 171 ns/op BenchmarkGo-8 2000000000 1.83 ns/op EDIT: And then you still have the extra overhead when using `C.CString` and `C.GoBytes` if you're passing those sorts of arguments to C.
    - _ph_ 6y ago
      But if the called function takes more than 200ns to execute, the overhead is less than 50%. Which probably is true for most relevant functions in an API
  - _ph_ 6y ago
    Go calling C is expensive compared to the normal function call overhead, but if the called function does significant work, should not introduce too much overhead.
  - MaxBarraclough 6y ago
    > Go calling C is expensive Is this true even if only primitives are passed and returned? If so, why? For comparison, Java is fast at that these days, if I understand correctly. (Interesting related reading regarding Java: https://web.archive.org/web/20160304055443/http://nerds-central.blogspot.co.uk/2012/04/extreme-jni-performance.html https://web.archive.org/web/20160304055443/http://nerds-cent... )
- progrium 6y ago
  This is a good question that should be added in the readme. Would love an issue for it to also collect data
tectonic 6y ago
Most of Jeff Lindsay's stuff is worth checking out. He also started dokku and coined the term webhook.
- 52-6F-62 6y ago
  We use Dokku on the regular for a number of small services at work. It was a little strange being new to it, but it's been great. Kudos to him.
  - asaddhamani 6y ago
    I know it's kind of going against the idea of Dokku, but can it be made to scale across servers? I love the simplicity of Heroku but the cost can be a barrier for some use cases. But k8s is too complicated. Anything like Dokku that scales horizontally?
    - turtlebits 6y ago
      flynn.io was the product I used found when I wanted a little better scalability than Dokku. This was a few years ago when I tried it, so unsure of the current landscape.
      - asaddhamani 6y ago
        I had tried flynn in the past (a few years ago just as you) but couldn't get it to work, but that was with Meteor which has very specific deployment requirements. I'll try out flynn when I need to scale beyond Dokku.
        goliatone 6y ago
        For my hobby servers I have tried dokku, Flynn and countless other similar solutions. Settled with caprover[1] and very happy with it [1] https://caprover.com https://caprover.com
    - cweagans 6y ago
JacobCarlborg 6y ago
D has native support for calling Objective-C. No need to integrate with the Objective-C runtime directly. Everything is written using familiar D syntax. No extra overhead compared to Objective-C. The generated D code is the same as the Objective-C compiler would generate. https://dlang.org/spec/objc_interface.html https://dlang.org/spec/objc_interface.html
- hu3 6y ago
  Although it's a bit out of the etiquete to post tangents like this, I find it really cool that D has such native support and will be reading more about it because you shared. So thanks.
  - andyfleming 6y ago
    I don’t think it’s out of line. Oftentimes comments like this do come with jabs at the language being discussed, but this one was pleasantly free of snark. It makes me more curious about D.
w0mbat 6y ago
I am freaked out by the line, "Retain and Release methods for working with Objective-C garbage collection". Do they mean the abandoned GC system Apple tried, or the obsolete manual pool-based system Apple used to use? Either way, I want ARC instead.
- hctaw 6y ago
  Last I tried to bind to ObjC APIs through C you needed to use NSAutoReleasePool at some point, and to interact with the GC as the APIs expect. There was no option to use anything else. Has that changed?
  - w0mbat 6y ago
    ARC is built on top of the retain/release pool system, but it automates everything from the programmer's point of view so you almost never have to explicitly interact with it.
    - hctaw 6y ago
      This works across FFI? How?
      - favorited 6y ago
        It doesn't. ARC is a compile-time operation (in Objective-C/Swift)– if you're not using clang or swiftc, then ARC isn't a thing.
- JonathonW 6y ago
  Retain/Release methods would be for manual reference counting, as is traditional in Obj-C/Cocoa. ARC is a compiler trick in the clang objective-c and swift compilers-- essentially injecting the appropriate retain/release calls for you. You won't get that for free in Go (because, again, it's a compiler feature-- compiled ObjC/Swift code is still calling retain/release just like you would manually), but I'd expect some integration with Go's memory management system so you're not having to make those calls manually. (It looks like that is not here in this particular release, though-- they have retain/release calls in some of their examples.) Objective-C garbage collection is not a thing any more, and hasn't been for quite some time. Deprecated back in macOS 10.8, and outright removed in more recent versions of the runtime (it was never available on iOS, and was taken out sometime around macOS 10.12 on desktop).
casey 6y ago
This is great, I've done a few one-off go wrappers of objective C libraries [1][2] for an OSX menuapp framework I built [3]. This seems like a much more general and useful solution, excited to switch some things over to it! [1] https://github.com/caseymrm/go-pmset https://github.com/caseymrm/go-pmset [2] https://github.com/caseymrm/go-smc https://github.com/caseymrm/go-smc [3] https://github.com/caseymrm/menuet https://github.com/caseymrm/menuet
Klonoar 6y ago
This is seriously cool. I feel slightly validated that I've also been wrapping these by hand, since it's kind of quirky to try and auto-generate these while keeping the same usability and such as their original incarnations. For those interested in a Rust variant, I've been hacking on one for awhile: https://github.com/ryanmcgrath/cacao https://github.com/ryanmcgrath/cacao I wouldn't say it's yet ready for production use, but so far it's working pretty well for me. Been dogfooding it by building an app I've wanted for a bit - a proper magic-wormhole macOS app. One of the things I've really wanted to keep to is the delegate pattern, since I think it actually works really well for Rust's model. A fun example I finished yesterday is ListView cell reuse: https://twitter.com/ryanmcgrath/status/1357097991081844737/photo/1 https://twitter.com/ryanmcgrath/status/1357097991081844737/p... Ultimately I view this as one of the last pieces needed for a cross-platform Rust UI framework to actually work.
- deleted 6y ago
  [deleted]
lunixbochs 6y ago
I don't see an @autoreleasepool anywhere in the source, which I believe means calling some AppKit APIs from Go-managed threads will silently leak memory (due to memory allocated internally by the APIs that will never get released). In my own projects, I had to wrap calls from foreign threads into AppKit APIs with an @autoreleasepool {} block or my app would leak memory. [1] [1] https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/MemoryMgmt/Articles/mmAutoreleasePools.html https://developer.apple.com/library/archive/documentation/Co...
- progrium 6y ago
  I thought I wrapped NSAutoreleasePool but maybe I just used it directly dynamically. If something is not wrapped in the source that doesn't mean you can't use it. objc.Get("NSAutoreleasePool").Alloc().Init() Unfortunately I can afford some leaks at the moment, so if that's critical to anybody else and I'm doing something wrong just submit a PR
  - lunixbochs 6y ago
    Ah, I don't just mean the class, I mean you need to have an autoreleasepool block active before calling into AppKit otherwise you can leak memory on every call. It doesn't look like you're using pools yet, or documenting that users of your library should use them. See here: https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/MemoryMgmt/Articles/mmAutoreleasePools.html https://developer.apple.com/library/archive/documentation/Co... > Cocoa always expects code to be executed within an autorelease pool block, otherwise autoreleased objects do not get released and your application leaks memory
    - progrium 6y ago
      I guess I will look into this as that really sounds like syntactic sugar for something more basic. Like using the class. I have a hard time keeping up with their changes but you might be right: https://developer.apple.com/documentation/foundation/nsautoreleasepool https://developer.apple.com/documentation/foundation/nsautor... Oddly it says you cannot use them directly, but later implies maybe they are just less efficient. It would be nice if somebody made an issue for this.
revskill 6y ago
So we could use Go to develop iPhone apps ?
- gspq 6y ago
  You can use Gomobile for that.
gspq 6y ago
Looks cool. What about Swift support?
- favorited 6y ago
  That would be a lot more difficult (unless you limited support to the subset of Swift which is shared by Objective-C). Objective-C is an extremely simple extension to C, and has a rich runtime API. So any language which can interface with C can interface with Objective-C by using the runtime interfaces.
  - gspq 6y ago
    I successfully used the other way - calling Golang functions from iOS/Swift using Gomobile. It is working now for the new M1 Apple desktop computers too.
jfb 6y ago
We used a lot of PyObjC BITD at Apple to explore frameworks that we couldn't get documentation for. That was simultaneously awesome and so so so broken, from an organizational perspective.
dvt 6y ago
Great work, OP. However, I think it's problematic that modern languages (Go, Rust) don't provide these kinds of bindings out of the box. I honestly think it's probably starting to be "in-scope" for standard libraries. I'm working on a side-project now which needs a desktop app, and I'm actually using Fyne[1] because it makes cross-platform development semi-easy (even though it doesn't look native, nor particularly great). The alternative is using either Electron or using something like this for OSX, using something else for Windows, and using something else for Linux. Window/widget/notification/taskbar APIs are stable for all major operating systems, and it seems like we keep reinventing the wheel here. [1] https://fyne.io/ https://fyne.io/
- moldavi 6y ago
  Do you mean we should have OS-specific methods in languages' standard libraries? Or that we should add cross-platform frameworks to languages' standard libraries?
nkozyra 6y ago
Not going to do a GH issue as it says this area is very much in flux but it looks like the bridge documentation is outdated. Otherwise this worked really well and intuitively for a golang dev!
Camillo 6y ago
Bindings for Apple's APIs for other languages come out every once in a while. For example, PyObjC for Python, RubyCocoa and MacRuby, Apple's own old Java bindings, etc. Usually these garner early interest, but no significant programs come out using them, and they tend to be abandoned within a few years. I don't mean to discourage the author, but I wonder if he's aware of the history and if he has thoughts on why Go may fare better.
barelyusable 6y ago
It's a little sad that after 50 years since Microsoft and Apple were started, there still is no easy way to write a natively looking cross-platform GUI app, even with a language like Golang. Recently I looked at couple of options of writing some Golang to make a simple GUI app that would look natively on Windows/MacOS (text area, several checkboxes/buttons), and there's no much there to be honest.
- smoldesu 6y ago
  Unironically, your best option is GTK. I've found that it generates consistent looking UIs for Linux and Windows (my main build targets) without totally ruling out MacOS
- vmception 6y ago
  Fortunately you can go four levels of virtualization in and do it pretty easily Users won’t care, there’s no award
- tadfisher 6y ago
  Sure there is: Write a separate UI for each platform. Seriously, this is the only option if you care not just about looks but also about feel, because no cross-platform toolkit gets things like keyboard shortcuts, scrolling inertia, menu mnemonics, etc. correct on all platforms they target. Not to mention platform accessibility features. Surely most of the work in your app is platform-independent, right? Plumbing data into platform-specific UIs should not be a hard endeavor.
- pjmlp 6y ago
  Sure there is, my C++ code has worked just fine all these years.
- Gibbon1 6y ago
  I think Microsoft wants to address that in .net 6

480 points by maydemir 6y ago

This looks really cool! Is there any overhead with using these bindings as opposed to using Swift and calling the APIs directly?

axaxs 6y ago
> Is there any overhead with using these bindings Not the author, but yes. Go calling C is expensive, and that's precisely how this works. Whether or not that matters for your use case is another topic.
- lsllc 6y ago
  There's a bit of a write-up here: https://www.cockroachlabs.com/blog/the-cost-and-complexity-of-cgo/ https://www.cockroachlabs.com/blog/the-cost-and-complexity-o... In their benchmark, calling a `func() {}` in Go vs a `void foo() {}` in C (via CGo) is almost 100x faster. $ go test -bench . -gcflags '-l' # disable inlining for fairness BenchmarkCGO-8 10000000 171 ns/op BenchmarkGo-8 2000000000 1.83 ns/op EDIT: And then you still have the extra overhead when using `C.CString` and `C.GoBytes` if you're passing those sorts of arguments to C.
  - _ph_ 6y ago
    But if the called function takes more than 200ns to execute, the overhead is less than 50%. Which probably is true for most relevant functions in an API
- _ph_ 6y ago
  Go calling C is expensive compared to the normal function call overhead, but if the called function does significant work, should not introduce too much overhead.
- MaxBarraclough 6y ago
  > Go calling C is expensive Is this true even if only primitives are passed and returned? If so, why? For comparison, Java is fast at that these days, if I understand correctly. (Interesting related reading regarding Java: https://web.archive.org/web/20160304055443/http://nerds-central.blogspot.co.uk/2012/04/extreme-jni-performance.html https://web.archive.org/web/20160304055443/http://nerds-cent... )
progrium 6y ago
This is a good question that should be added in the readme. Would love an issue for it to also collect data

tectonic 6y ago

Most of Jeff Lindsay's stuff is worth checking out. He also started dokku and coined the term webhook.

52-6F-62 6y ago
We use Dokku on the regular for a number of small services at work. It was a little strange being new to it, but it's been great. Kudos to him.
- asaddhamani 6y ago
  I know it's kind of going against the idea of Dokku, but can it be made to scale across servers? I love the simplicity of Heroku but the cost can be a barrier for some use cases. But k8s is too complicated. Anything like Dokku that scales horizontally?
  - turtlebits 6y ago
    flynn.io was the product I used found when I wanted a little better scalability than Dokku. This was a few years ago when I tried it, so unsure of the current landscape.
    - asaddhamani 6y ago
      I had tried flynn in the past (a few years ago just as you) but couldn't get it to work, but that was with Meteor which has very specific deployment requirements. I'll try out flynn when I need to scale beyond Dokku.
      - goliatone 6y ago
        For my hobby servers I have tried dokku, Flynn and countless other similar solutions. Settled with caprover[1] and very happy with it [1] https://caprover.com https://caprover.com
  - cweagans 6y ago

JacobCarlborg 6y ago

D has native support for calling Objective-C. No need to integrate with the Objective-C runtime directly. Everything is written using familiar D syntax. No extra overhead compared to Objective-C. The generated D code is the same as the Objective-C compiler would generate. https://dlang.org/spec/objc_interface.html https://dlang.org/spec/objc_interface.html

hu3 6y ago
Although it's a bit out of the etiquete to post tangents like this, I find it really cool that D has such native support and will be reading more about it because you shared. So thanks.
- andyfleming 6y ago
  I don’t think it’s out of line. Oftentimes comments like this do come with jabs at the language being discussed, but this one was pleasantly free of snark. It makes me more curious about D.

w0mbat 6y ago

I am freaked out by the line, "Retain and Release methods for working with Objective-C garbage collection". Do they mean the abandoned GC system Apple tried, or the obsolete manual pool-based system Apple used to use? Either way, I want ARC instead.

hctaw 6y ago
Last I tried to bind to ObjC APIs through C you needed to use NSAutoReleasePool at some point, and to interact with the GC as the APIs expect. There was no option to use anything else. Has that changed?
- w0mbat 6y ago
  ARC is built on top of the retain/release pool system, but it automates everything from the programmer's point of view so you almost never have to explicitly interact with it.
  - hctaw 6y ago
    This works across FFI? How?
    - favorited 6y ago
      It doesn't. ARC is a compile-time operation (in Objective-C/Swift)– if you're not using clang or swiftc, then ARC isn't a thing.
JonathonW 6y ago
Retain/Release methods would be for manual reference counting, as is traditional in Obj-C/Cocoa. ARC is a compiler trick in the clang objective-c and swift compilers-- essentially injecting the appropriate retain/release calls for you. You won't get that for free in Go (because, again, it's a compiler feature-- compiled ObjC/Swift code is still calling retain/release just like you would manually), but I'd expect some integration with Go's memory management system so you're not having to make those calls manually. (It looks like that is not here in this particular release, though-- they have retain/release calls in some of their examples.) Objective-C garbage collection is not a thing any more, and hasn't been for quite some time. Deprecated back in macOS 10.8, and outright removed in more recent versions of the runtime (it was never available on iOS, and was taken out sometime around macOS 10.12 on desktop).

casey 6y ago

This is great, I've done a few one-off go wrappers of objective C libraries [1][2] for an OSX menuapp framework I built [3]. This seems like a much more general and useful solution, excited to switch some things over to it! [1] https://github.com/caseymrm/go-pmset https://github.com/caseymrm/go-pmset [2] https://github.com/caseymrm/go-smc https://github.com/caseymrm/go-smc [3] https://github.com/caseymrm/menuet https://github.com/caseymrm/menuet

Klonoar 6y ago

This is seriously cool. I feel slightly validated that I've also been wrapping these by hand, since it's kind of quirky to try and auto-generate these while keeping the same usability and such as their original incarnations. For those interested in a Rust variant, I've been hacking on one for awhile: https://github.com/ryanmcgrath/cacao https://github.com/ryanmcgrath/cacao I wouldn't say it's yet ready for production use, but so far it's working pretty well for me. Been dogfooding it by building an app I've wanted for a bit - a proper magic-wormhole macOS app. One of the things I've really wanted to keep to is the delegate pattern, since I think it actually works really well for Rust's model. A fun example I finished yesterday is ListView cell reuse: https://twitter.com/ryanmcgrath/status/1357097991081844737/photo/1 https://twitter.com/ryanmcgrath/status/1357097991081844737/p... Ultimately I view this as one of the last pieces needed for a cross-platform Rust UI framework to actually work.

deleted 6y ago
[deleted]

lunixbochs 6y ago

I don't see an @autoreleasepool anywhere in the source, which I believe means calling some AppKit APIs from Go-managed threads will silently leak memory (due to memory allocated internally by the APIs that will never get released). In my own projects, I had to wrap calls from foreign threads into AppKit APIs with an @autoreleasepool {} block or my app would leak memory. [1] [1] https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/MemoryMgmt/Articles/mmAutoreleasePools.html https://developer.apple.com/library/archive/documentation/Co...

progrium 6y ago
I thought I wrapped NSAutoreleasePool but maybe I just used it directly dynamically. If something is not wrapped in the source that doesn't mean you can't use it. objc.Get("NSAutoreleasePool").Alloc().Init() Unfortunately I can afford some leaks at the moment, so if that's critical to anybody else and I'm doing something wrong just submit a PR
- lunixbochs 6y ago
  Ah, I don't just mean the class, I mean you need to have an autoreleasepool block active before calling into AppKit otherwise you can leak memory on every call. It doesn't look like you're using pools yet, or documenting that users of your library should use them. See here: https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/MemoryMgmt/Articles/mmAutoreleasePools.html https://developer.apple.com/library/archive/documentation/Co... > Cocoa always expects code to be executed within an autorelease pool block, otherwise autoreleased objects do not get released and your application leaks memory
  - progrium 6y ago
    I guess I will look into this as that really sounds like syntactic sugar for something more basic. Like using the class. I have a hard time keeping up with their changes but you might be right: https://developer.apple.com/documentation/foundation/nsautoreleasepool https://developer.apple.com/documentation/foundation/nsautor... Oddly it says you cannot use them directly, but later implies maybe they are just less efficient. It would be nice if somebody made an issue for this.

revskill 6y ago

So we could use Go to develop iPhone apps ?

gspq 6y ago
You can use Gomobile for that.

gspq 6y ago

Looks cool. What about Swift support?

favorited 6y ago
That would be a lot more difficult (unless you limited support to the subset of Swift which is shared by Objective-C). Objective-C is an extremely simple extension to C, and has a rich runtime API. So any language which can interface with C can interface with Objective-C by using the runtime interfaces.
- gspq 6y ago
  I successfully used the other way - calling Golang functions from iOS/Swift using Gomobile. It is working now for the new M1 Apple desktop computers too.

jfb 6y ago

We used a lot of PyObjC BITD at Apple to explore frameworks that we couldn't get documentation for. That was simultaneously awesome and so so so broken, from an organizational perspective.

dvt 6y ago

Great work, OP. However, I think it's problematic that modern languages (Go, Rust) don't provide these kinds of bindings out of the box. I honestly think it's probably starting to be "in-scope" for standard libraries. I'm working on a side-project now which needs a desktop app, and I'm actually using Fyne[1] because it makes cross-platform development semi-easy (even though it doesn't look native, nor particularly great). The alternative is using either Electron or using something like this for OSX, using something else for Windows, and using something else for Linux. Window/widget/notification/taskbar APIs are stable for all major operating systems, and it seems like we keep reinventing the wheel here. [1] https://fyne.io/ https://fyne.io/

moldavi 6y ago
Do you mean we should have OS-specific methods in languages' standard libraries? Or that we should add cross-platform frameworks to languages' standard libraries?

nkozyra 6y ago

Not going to do a GH issue as it says this area is very much in flux but it looks like the bridge documentation is outdated. Otherwise this worked really well and intuitively for a golang dev!

Camillo 6y ago

Bindings for Apple's APIs for other languages come out every once in a while. For example, PyObjC for Python, RubyCocoa and MacRuby, Apple's own old Java bindings, etc. Usually these garner early interest, but no significant programs come out using them, and they tend to be abandoned within a few years. I don't mean to discourage the author, but I wonder if he's aware of the history and if he has thoughts on why Go may fare better.

barelyusable 6y ago

It's a little sad that after 50 years since Microsoft and Apple were started, there still is no easy way to write a natively looking cross-platform GUI app, even with a language like Golang. Recently I looked at couple of options of writing some Golang to make a simple GUI app that would look natively on Windows/MacOS (text area, several checkboxes/buttons), and there's no much there to be honest.

smoldesu 6y ago
Unironically, your best option is GTK. I've found that it generates consistent looking UIs for Linux and Windows (my main build targets) without totally ruling out MacOS
vmception 6y ago
Fortunately you can go four levels of virtualization in and do it pretty easily Users won’t care, there’s no award
tadfisher 6y ago
Sure there is: Write a separate UI for each platform. Seriously, this is the only option if you care not just about looks but also about feel, because no cross-platform toolkit gets things like keyboard shortcuts, scrolling inertia, menu mnemonics, etc. correct on all platforms they target. Not to mention platform accessibility features. Surely most of the work in your app is platform-independent, right? Plumbing data into platform-specific UIs should not be a hard endeavor.
pjmlp 6y ago
Sure there is, my C++ code has worked just fine all these years.
Gibbon1 6y ago
I think Microsoft wants to address that in .net 6