8 ms·

GoPlus – The Go+ language for data science

143 points by dx034 6y ago

cpgeier 6y ago
I have a feeling languages like this have their niche cases however I'm not getting a good reason for why this is need when compared to Python for data science. Maybe I'm missing something but this is essentially a wrapper for Golang code to make it feel more like Python.
- DataJunkie 6y ago
  Agreed, but it might be useful for a full stack data scientist that is forced to work in a Go systems environment. That's why Python+PyData has had so much success. There are packages to support data science, but the language itself can also be used to implement a system, so integration is rather seamless. That's not true for, say, R.
  - techwizrd 6y ago
    I use Python and R for data science, and I've never had any issue with R. In fact, I find that many tasks are much simpler in R than in Python.
    - livvelupp 6y ago
      > never had any issue
    - DataJunkie 6y ago
      I am referring to using R to build systems. It's not common.
      - cepth 6y ago
        What would you consider a system? Python definitely has more market share than R, but there's still name brand companies of various sizes that use an R stack for data science. RStudio lists dozens of example clients here: https://rstudio.com/about/customer-stories/ https://rstudio.com/about/customer-stories/. Use cases include collaborative model development, EDA tools, dashboarding, printed report generation (PDFs and HTML), public facing websites, etc.
  - cepth 6y ago
    I guess it depends on what you’re trying to accomplish (I’ve worked heavily with both R and Python). If you’re trying to create ETL pipelines that integrate with BigQuery, Mongo, or whatever other database, I think it’s fair to say that the Python packages are generally better documented than their R counterparts. For most other things, IMO it’s hard to really separate the two languages. Is standing up a Flask API really easier than in plumber? For dashboarding, it’s is as quick (if not much quicker) to create a decent prototype with Shiny vs Plotly Dash or bokeh. For simple linear and logistic model training, R’s built-in stats package has much more interpretable outputs vs sklearn, and directly inspired statsmodel. Wes McKinney has acknowledged that pandas draws heavily from R’s native dataframe. And so on and so on. EDIT: Also forgot to mention that with R packages like reticulate, you can also directly run Python code within an R environment now. So if there happens to be some Python package that doesn’t have an R equivalent, you can still work in R (though I’ve found the opposite situation to be far more common).
- devy 6y ago
  The official Github repo of Go+ is hosted on a PaaS company called Qiniu's team repo with their founder & CEO as the top contributor[1][2]. He is a Golang enthusiast[3] and my guess is that this is probably his favorite pet project ;) [1]: https://github.com/qiniu/goplus/graphs/contributors https://github.com/qiniu/goplus/graphs/contributors [2]: https://github.com/xushiwei https://github.com/xushiwei [3]: https://twitter.com/xushiwei https://twitter.com/xushiwei
rhacker 6y ago
I'm not a Go user, but is the important idea here that it can be run from a scriptingish environment like Jupyter or something?
- sk0g 6y ago
  It's significant syntactic sugar that brings it closer to Python, but no notebook support from what I can see. Do any notebooks support a compiled language?
  - cjohnson318 6y ago
    The Jupyter project has a notebook called Xeus for C++.
  - rolls-reus 6y ago
    There is a Go kernel for jupyter/nteract. https://github.com/gopherdata/gophernotes https://github.com/gopherdata/gophernotes
  - pasokan 6y ago
    There is support for many languages Haskell :: https://github.com/gibiansky/IHaskell https://github.com/gibiansky/IHaskell Rust :: https://github.com/google/evcxr https://github.com/google/evcxr https://github.com/jupyter/jupyter/wiki/Jupyter-kernels https://github.com/jupyter/jupyter/wiki/Jupyter-kernels will give you a more detailed list
  - zmmmmm 6y ago
    Absolutely, see eg: BeakerX [1] [1] http://beakerx.com/ http://beakerx.com/
  - andylynch 6y ago
    R notebooks support a bunch. I count about 41 languages in my install, including Go, C++, Fortran and Julia, as well as Python, SQL, Bash, etc.
  - xushiwei 6y ago
    Go+ is planed to support jupyter and nteract.
yahyaheee 6y ago
I feel where they are trying to go with this, I wrote https://github.com/aunum/gold https://github.com/aunum/gold in Go because of all the nice parts of Go. This solves some of the pain points but is still fatally flawed as any other Go ML tool in that it can’t accelerate due to the Go<->C FFI. Until that issue is resolved Go simply won’t be broadly accepted in data science.
- mleonhard 6y ago
  Would you please explain?
  - yahyaheee 6y ago
    So due to how Go handles memory, it needs to do something called "trampoline", over to the C stack. This causes Go->C calls to have a substantial overhead 70-200ns/op. Most other languages that value is closer to 1ns. This means that every call to a GPU suffers from this latency. This isn't too much of a problem if your doing all supervised learning with batch operations because the speedup of a GPU over a bigger operation outweighs the FFI latency. However, it's a problem that doesn't appear to have a solution due to Go's memory management choices, and will hamper it ever being used for accelerated computing problems. This is one of the reasons Rust moved to using ownership rules. You can read a bit more at https://dave.cheney.net/2016/01/18/cgo-is-not-go https://dave.cheney.net/2016/01/18/cgo-is-not-go
WoodenChair 6y ago
IMHO these conveniences should just be in the language. List comprehensions, dictionary comprehensions, and short-hand for literals are the kind of syntactic sugar that have almost no downsides, save you some RSI, and make code more readable. I'm surprised comprehensions in particular haven't spread to more modern languages.
- meddlepal 6y ago
  It doesn't really help that the Go developers are stuck in the early 80's still. I hope this succeeds and forces some changes.
- pansa2 6y ago
  > I'm surprised comprehensions in particular haven't spread to more modern languages. Comprehensions become unreadable quite quickly - even simple ones have to be read inside-out, and complex (especially nested) ones are much worse. Most modern languages seem to prefer functional-style methods like `map`, `filter` etc, which are more readable in all but the simplest cases.
  - kbd 6y ago
    > which are more readable in all but the simplest cases. Except in practice, the vast majority of comprehensions are simple.
- zmmmmm 6y ago
  > I'm surprised comprehensions in particular haven't spread to more modern languages. I've come to dislike list comprehensions. Simple ones are ok but they do not handle incremental complexity well. Invariably it means code slowly becomes really unreadable as time goes by because nobody wants to rewrite the list comprehension when one more little tweak stuffed in there will do the job. I much prefer object-functional chaining style ala Scala, Groovy, Ruby etc: some_giant_list .findAll { it.foo > 20 } .groupBy { it.bar } .countBy { it.value.size() > 5 } They scale much better over time as new constructs get inserted as functional additions in the sequential pipeline.
sk0g 6y ago
This is very cool! I wonder what the chances of this being pulled into Go proper is, and whether the lack of cgo is an intractable issue, which would rule it out entirely.
- maxmcd 6y ago
  The error handing is similar enough to a go proposal that was rejected: https://github.com/qiniu/goplus/wiki/Error-Handling https://github.com/qiniu/goplus/wiki/Error-Handling foo()? // for goplus try(foo()) // the go proposal https://github.com/golang/go/issues/32437 https://github.com/golang/go/issues/32437
waheoo 6y ago
Why not gosci? Anything with plus in the name is awful to google. Then again, go is a similarly awful thing to google so I guess youre in good company.
bumbledraven 6y ago
We introduce rational number as native Go+ types. We use -r suffix as a rational constant. For example, (1r << 200) means a big int whose value is equal to 2200. And 4/5r means the rational constant 4/5. Nice! Coming from Python, I miss being able to use the normal arithmetic operators on bigints in Go.
maddyboo 6y ago
I love the idea of main-less Go scripts for times where you just want to do something simple, fast. Neugram [1] aimed to make Go a better scripting tool, but unfortunately it seems the project is dead. Although Go+ says its focus is on data science, I think it could fill this niche too. By the way: does Go+ have shebang support? 1: https://github.com/neugram/ng https://github.com/neugram/ng
- deleted 6y ago
  [deleted]
- sa46 6y ago
  Not main-less but you can run go scripts using a shebang with some elbow grease. https://blog.cloudflare.com/using-go-as-a-scripting-language-in-linux/ https://blog.cloudflare.com/using-go-as-a-scripting-language...
- dgellow 6y ago
  > I love the idea of main-less Go scripts for times where you just want to do something simple, fast. What do you mean by a main-less script? And what would be "simple and fast" about it? If I want to try out something fast, I just do everything in a main.go and run it with "go run main.go", and that works well as a scripting language.
  - uryga 6y ago
    > What do you mean by a main-less script? probably something that's common in "scripting" languages, where you don't have to wrap your code in main(), it just executes from the top: myscript.py ----------- print('hi') vs myscript.go ----------- func main() { fmt.PrintLn('hi') }
- stevekemp 6y ago
  Go doesn't need to have shebang support, the kernel will provide that. But for the record it will work: frodo ~ $ cat t.go //opt/go/bin/go run $0 $@ ; exit package main import( "fmt" ) func main() { fmt.Printf("Hello, world\n" ); } frodo ~ $ ./t.go Hello, world Otherwise there are a bunch of go-interpreters out there, which can be used for adhoc scripting. I'm sure that in some circumstances they can be useful, but I've only used them for providing extensions / scripting to host-applications, rather than trying to use them interactively.
johnmc408 6y ago
Is there any talk on changing garbage collection from standard go? Is that needed in the data science world?
zactato 6y ago
Can we try our best to not fork languages? Someone in 8 years is going to have to refactor a bunch of this GoPlus code to be compatible the Go2 or Go3 and it’s going to suck. We already have Python and Scala
- hombre_fatal 6y ago
  You're barking up the wrong tree.
pjmlp 6y ago
Nice improvement, but given the community's receptions regarding such kind of changes, why not just make use of Julia, Ocaml, F#?
svntid 6y ago
I love it!
scared2 6y ago
It will end up being like Google plus
- xushiweizh 6y ago
  This project is not owned by Google.
zshbleaker 6y ago
Surprisingly to see my former employer on HN lol. Better than qiniulang in many ways.
xushiwei 6y ago
I'm founder of the Go+ project. And thanks for your attention. We run this project very seriously. It will be released weekly until version 1.0 is released.
- dx034 6y ago
  Thank you so much for this! I submitted it here because I stumbled across it and was really surprised that it hadn't been on HN yet! If we somehow get Go+ in a Jupyter Kernel that would be awesome!

143 points by dx034 6y ago

I have a feeling languages like this have their niche cases however I'm not getting a good reason for why this is need when compared to Python for data science. Maybe I'm missing something but this is essentially a wrapper for Golang code to make it feel more like Python.

DataJunkie 6y ago
Agreed, but it might be useful for a full stack data scientist that is forced to work in a Go systems environment. That's why Python+PyData has had so much success. There are packages to support data science, but the language itself can also be used to implement a system, so integration is rather seamless. That's not true for, say, R.
- techwizrd 6y ago
  I use Python and R for data science, and I've never had any issue with R. In fact, I find that many tasks are much simpler in R than in Python.
  - livvelupp 6y ago
    > never had any issue
  - DataJunkie 6y ago
    I am referring to using R to build systems. It's not common.
    - cepth 6y ago
      What would you consider a system? Python definitely has more market share than R, but there's still name brand companies of various sizes that use an R stack for data science. RStudio lists dozens of example clients here: https://rstudio.com/about/customer-stories/ https://rstudio.com/about/customer-stories/. Use cases include collaborative model development, EDA tools, dashboarding, printed report generation (PDFs and HTML), public facing websites, etc.
- cepth 6y ago
  I guess it depends on what you’re trying to accomplish (I’ve worked heavily with both R and Python). If you’re trying to create ETL pipelines that integrate with BigQuery, Mongo, or whatever other database, I think it’s fair to say that the Python packages are generally better documented than their R counterparts. For most other things, IMO it’s hard to really separate the two languages. Is standing up a Flask API really easier than in plumber? For dashboarding, it’s is as quick (if not much quicker) to create a decent prototype with Shiny vs Plotly Dash or bokeh. For simple linear and logistic model training, R’s built-in stats package has much more interpretable outputs vs sklearn, and directly inspired statsmodel. Wes McKinney has acknowledged that pandas draws heavily from R’s native dataframe. And so on and so on. EDIT: Also forgot to mention that with R packages like reticulate, you can also directly run Python code within an R environment now. So if there happens to be some Python package that doesn’t have an R equivalent, you can still work in R (though I’ve found the opposite situation to be far more common).
devy 6y ago
The official Github repo of Go+ is hosted on a PaaS company called Qiniu's team repo with their founder & CEO as the top contributor[1][2]. He is a Golang enthusiast[3] and my guess is that this is probably his favorite pet project ;) [1]: https://github.com/qiniu/goplus/graphs/contributors https://github.com/qiniu/goplus/graphs/contributors [2]: https://github.com/xushiwei https://github.com/xushiwei [3]: https://twitter.com/xushiwei https://twitter.com/xushiwei

rhacker 6y ago

I'm not a Go user, but is the important idea here that it can be run from a scriptingish environment like Jupyter or something?

sk0g 6y ago
It's significant syntactic sugar that brings it closer to Python, but no notebook support from what I can see. Do any notebooks support a compiled language?
- cjohnson318 6y ago
  The Jupyter project has a notebook called Xeus for C++.
- rolls-reus 6y ago
  There is a Go kernel for jupyter/nteract. https://github.com/gopherdata/gophernotes https://github.com/gopherdata/gophernotes
- pasokan 6y ago
  There is support for many languages Haskell :: https://github.com/gibiansky/IHaskell https://github.com/gibiansky/IHaskell Rust :: https://github.com/google/evcxr https://github.com/google/evcxr https://github.com/jupyter/jupyter/wiki/Jupyter-kernels https://github.com/jupyter/jupyter/wiki/Jupyter-kernels will give you a more detailed list
- zmmmmm 6y ago
  Absolutely, see eg: BeakerX [1] [1] http://beakerx.com/ http://beakerx.com/
- andylynch 6y ago
  R notebooks support a bunch. I count about 41 languages in my install, including Go, C++, Fortran and Julia, as well as Python, SQL, Bash, etc.
- xushiwei 6y ago
  Go+ is planed to support jupyter and nteract.

yahyaheee 6y ago

I feel where they are trying to go with this, I wrote https://github.com/aunum/gold https://github.com/aunum/gold in Go because of all the nice parts of Go. This solves some of the pain points but is still fatally flawed as any other Go ML tool in that it can’t accelerate due to the Go<->C FFI. Until that issue is resolved Go simply won’t be broadly accepted in data science.

mleonhard 6y ago
Would you please explain?
- yahyaheee 6y ago
  So due to how Go handles memory, it needs to do something called "trampoline", over to the C stack. This causes Go->C calls to have a substantial overhead 70-200ns/op. Most other languages that value is closer to 1ns. This means that every call to a GPU suffers from this latency. This isn't too much of a problem if your doing all supervised learning with batch operations because the speedup of a GPU over a bigger operation outweighs the FFI latency. However, it's a problem that doesn't appear to have a solution due to Go's memory management choices, and will hamper it ever being used for accelerated computing problems. This is one of the reasons Rust moved to using ownership rules. You can read a bit more at https://dave.cheney.net/2016/01/18/cgo-is-not-go https://dave.cheney.net/2016/01/18/cgo-is-not-go

WoodenChair 6y ago

IMHO these conveniences should just be in the language. List comprehensions, dictionary comprehensions, and short-hand for literals are the kind of syntactic sugar that have almost no downsides, save you some RSI, and make code more readable. I'm surprised comprehensions in particular haven't spread to more modern languages.

meddlepal 6y ago
It doesn't really help that the Go developers are stuck in the early 80's still. I hope this succeeds and forces some changes.
pansa2 6y ago
> I'm surprised comprehensions in particular haven't spread to more modern languages. Comprehensions become unreadable quite quickly - even simple ones have to be read inside-out, and complex (especially nested) ones are much worse. Most modern languages seem to prefer functional-style methods like `map`, `filter` etc, which are more readable in all but the simplest cases.
- kbd 6y ago
  > which are more readable in all but the simplest cases. Except in practice, the vast majority of comprehensions are simple.
zmmmmm 6y ago
> I'm surprised comprehensions in particular haven't spread to more modern languages. I've come to dislike list comprehensions. Simple ones are ok but they do not handle incremental complexity well. Invariably it means code slowly becomes really unreadable as time goes by because nobody wants to rewrite the list comprehension when one more little tweak stuffed in there will do the job. I much prefer object-functional chaining style ala Scala, Groovy, Ruby etc: some_giant_list .findAll { it.foo > 20 } .groupBy { it.bar } .countBy { it.value.size() > 5 } They scale much better over time as new constructs get inserted as functional additions in the sequential pipeline.

sk0g 6y ago

This is very cool! I wonder what the chances of this being pulled into Go proper is, and whether the lack of cgo is an intractable issue, which would rule it out entirely.

maxmcd 6y ago
The error handing is similar enough to a go proposal that was rejected: https://github.com/qiniu/goplus/wiki/Error-Handling https://github.com/qiniu/goplus/wiki/Error-Handling foo()? // for goplus try(foo()) // the go proposal https://github.com/golang/go/issues/32437 https://github.com/golang/go/issues/32437

waheoo 6y ago

Why not gosci? Anything with plus in the name is awful to google. Then again, go is a similarly awful thing to google so I guess youre in good company.

bumbledraven 6y ago

We introduce rational number as native Go+ types. We use -r suffix as a rational constant. For example, (1r << 200) means a big int whose value is equal to 2200. And 4/5r means the rational constant 4/5. Nice! Coming from Python, I miss being able to use the normal arithmetic operators on bigints in Go.

maddyboo 6y ago

I love the idea of main-less Go scripts for times where you just want to do something simple, fast. Neugram [1] aimed to make Go a better scripting tool, but unfortunately it seems the project is dead. Although Go+ says its focus is on data science, I think it could fill this niche too. By the way: does Go+ have shebang support? 1: https://github.com/neugram/ng https://github.com/neugram/ng

deleted 6y ago
[deleted]
sa46 6y ago
Not main-less but you can run go scripts using a shebang with some elbow grease. https://blog.cloudflare.com/using-go-as-a-scripting-language-in-linux/ https://blog.cloudflare.com/using-go-as-a-scripting-language...
dgellow 6y ago
> I love the idea of main-less Go scripts for times where you just want to do something simple, fast. What do you mean by a main-less script? And what would be "simple and fast" about it? If I want to try out something fast, I just do everything in a main.go and run it with "go run main.go", and that works well as a scripting language.
- uryga 6y ago
  > What do you mean by a main-less script? probably something that's common in "scripting" languages, where you don't have to wrap your code in main(), it just executes from the top: myscript.py ----------- print('hi') vs myscript.go ----------- func main() { fmt.PrintLn('hi') }
stevekemp 6y ago
Go doesn't need to have shebang support, the kernel will provide that. But for the record it will work: frodo ~ $ cat t.go //opt/go/bin/go run $0 $@ ; exit package main import( "fmt" ) func main() { fmt.Printf("Hello, world\n" ); } frodo ~ $ ./t.go Hello, world Otherwise there are a bunch of go-interpreters out there, which can be used for adhoc scripting. I'm sure that in some circumstances they can be useful, but I've only used them for providing extensions / scripting to host-applications, rather than trying to use them interactively.

johnmc408 6y ago

Is there any talk on changing garbage collection from standard go? Is that needed in the data science world?

zactato 6y ago

Can we try our best to not fork languages? Someone in 8 years is going to have to refactor a bunch of this GoPlus code to be compatible the Go2 or Go3 and it’s going to suck. We already have Python and Scala

hombre_fatal 6y ago
You're barking up the wrong tree.

pjmlp 6y ago

Nice improvement, but given the community's receptions regarding such kind of changes, why not just make use of Julia, Ocaml, F#?

svntid 6y ago

I love it!

scared2 6y ago

It will end up being like Google plus

xushiweizh 6y ago
This project is not owned by Google.

zshbleaker 6y ago

Surprisingly to see my former employer on HN lol. Better than qiniulang in many ways.

xushiwei 6y ago

I'm founder of the Go+ project. And thanks for your attention. We run this project very seriously. It will be released weekly until version 1.0 is released.

dx034 6y ago
Thank you so much for this! I submitted it here because I stumbled across it and was really surprised that it hadn't been on HN yet! If we somehow get Go+ in a Jupyter Kernel that would be awesome!