4 ms·

I find myself reaching for it over other languages when I want to build small servers with a bit of in-memory state or a bit of heavy processing. For little sea

I find myself reaching for it over other languages when I want to build small servers with a bit of in-memory state or a bit of heavy processing. For little search-engines, Go is perfect. While writing servers in Flask + Python is much more convenient, I still prefer Go because I don't run into the limits that Python has. The development process is fluid enough that I wish the language was suited to more usecases. When I need to handle complicated data (e.g. abstract syntax trees), I use Rust or Haskell because of their rich data types. But it tends to be much harder to get things running in those languages (borrow-checker and monads, respectively). I want to reach for Go to make those problems go away, but then realize that it would be really painful to try to express the program in Go. I understand that other people are unhappy with Go's development process, particularly if you deal with package versioning. The language design itself may have been backed into a corner where adding generics will create an ugly mess (what will the standard library look like if it tries to maintain backwards compatibility?). Time will tell. The tooling is mostly pretty good, but many things feel half-baked (compared to more mature ecosystems). On the spectrum of 'written in a weekend' to 'dozens of developer-years of work', the package "net/http/pprof" feels closer to the weekend side. There are bright sides, like having the production parser available as a library. It's a language that is frustrating in different ways from other languages. Instead of fighting with Cabal/Stack, you have to write a million `if err != nil {}` statements. Part of what makes it frustrating is seeing how good it could have been.

mylons 6y ago
What limitations do you run into with the flask/python setup? (I’m not asking what ARE the limitations, I’m asking which ones specifically you run into)
- piinbinary 6y ago
  For large data sets, it sometimes spends all day in GC while just loading the data into memory (this is actually more of a problem in Ruby than in Python, but it still exists here). It's also not very efficient about using memory, so it can be easier to get a dataset to fit in RAM in Go vs. Python. Python also makes it hard enough to make use of multiple cores that it becomes easier to just use Go. The same goes for making it fast. It can be done in Python, but I tend to be happy with the performance of my Go code out of the box.
  - leaveapennyman 6y ago
    I’m interested in learning more about Ruby’s garbage collection issues. Could you recommend any resources on this? Or any tools in the Ruby language that would allow me to test this myself?
    - themantri 6y ago
      https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html https://www.joyfulbikeshedding.com/blog/2019-03-14-what-caus... is a pretty interesting resource about this.
  - deleted 6y ago
    [deleted]
  - chucky_z 6y ago
    Have you ever tried Pypy? I know when I had issues with GC and some memory usage (specifically dealing with large datasets as well, I was parsing large MySQL query results -- many millions of rows) Pypy didn't really drop memory usage much, but it did give me an almost 10x speedup. Enough that I went "oh, guess I don't need to use a different language."
    - lanstin 6y ago
      I had a python server written in Python with greenlets to hat I switched to PyPy and the load it could handle went up maybe 4x. (5k per second to 20k per second. I took two weeks to switch to go and it went up another 10x to 100k per second and had less bugs (in fact a parsing bug in the python caused messages greater than 64k to be dropped; the Go handled partial reads properly and starting sending 2M msgs (to Kafka). The subscribers started getting the long messages and barfing making the system go wonky. Oddly all the giant messages were corrupt data anyways. Code size was about the same due to stuff being implemented in Python manually being handled by Go runtime.
jen20 6y ago
What is it that you find more convenient about Python/Flask?
- piinbinary 6y ago
  Flask is better in the little mechanics of interacting with HTTP - parsing a parameter out of a URL (in Flask, it's a function argument; in Go, it's at least one extra line per arg), returning JSON (just return a dictionary in Flask), handling errors. Flask also is able to auto-reload the code when it changes (turn on development mode) which is really convenient. Plus all the ways that Python is more convenient than Go.
  - andreygrehov 6y ago
    But...you are comparing a web framework with a Go's standard library. I mean, it's not a fair comparison. I'm pretty sure there are Go modules that support those handy things.
    - dnautics 6y ago
      doesn't go stdlib contain an http server library "that you are supposed to use in prod"?
      - andreygrehov 6y ago
        It does. It's rock solid and most people use it in production. But the parent is more concerned about quick and handy functions. IMHO, such functionality is usually part of third-party libraries and is certainly an option in Go. It's just idiomatic Go favors KISS principles, less external dependancies and a more self-contained software. The benefit is that pretty much all the Go source code is more-or-less identical, regardless of an author. It makes it super simple to understand and deep dive into a random complex piece of software. Go is not about abstractions, Go is about solving problems here and now.
    - barumi 6y ago
      > But...you are comparing a web framework with a Go's standard library. I mean, it's not a fair comparison. Why do you feel that the comparison isn't fair? Aren't we talking about modules that provide the features necessary to implement a type of application? Would it make any difference if tomorrow there was a PEP making Flask a standard component? Unless you are aware of any Go module that addresses the downsides of Go with regards to Python, your observation is mute and unhelpful.
thiht 6y ago
> While writing servers in Flask + Python is much more convenient The only time I had to use Flask, it was a nightmare to deploy. To this day I still fail to understand why I had to mess with Nginx, gunicorn, WSGI or whatever just for a basic backend. The dev experience was fine, but I don't remember it being more convenient than using Go
- pydry 6y ago
  Because web servers look simple from the outset but are, underneath it all, very complicated. And it's not a web server. If you want something basic, gunicorn can run flask with one line.
  - thiht 6y ago
    But I still need gunicorn, don't I? My point is that with Go, I just have to execute my single binary and I'm good to go. It doesn't prevent me from adding a reverse proxy and a loaf balancer if I want to, but I don't need gunicorn or other runners. Also if a single line is enough, maybe adding it clearly to the doc would be nice.
    - BiteCode_dev 6y ago
      You don't. Some frameworks are stand alone. It just happens flask uses wsgi, and so you need gunicorn.
    - barumi 6y ago
      > But I still need gunicorn, don't I? My point is that with Go, I just have to execute my single binary and I'm good to go. I don't see how your point is relevant. I mean, with interpreted languages you need to run the interpreter, but if having to write an additional word or line of code bothers you that much then you should not worry about it if you dockerize the app or, god forbid, use a launcher script. I mean, in non land no one ever complained about how hard it is to add scripts to mom's start and prestarte targets to launch a server by doing mpm start. So why is this suddenly relevant with Go?
  - 6y ago
dhekir 6y ago
OCaml is good for handling abstract syntax trees (strong typing, pattern matching) and its mix of imperative and functional programming don't require using borrow-checker nor monads. You can also consider Reason, if you prefer a more C-like syntax.
- rienbdj 6y ago
  Also consider F#! In some ways it is more limited (no first class modules) but in others it is arguably better (operator overload, C# ecosystem, a slick take on do-notation).
- piinbinary 6y ago
  I really ought to learn OCaml (besides just the toy programs I've written). I think if I got past the messy ecosystem (is tuareg the right thing to use, or is that the old thing?) I'd really like it.
  - blandflakes 6y ago
    I always wonder if I just showed up late when I hear about the ecosystem! I just get opam for packages, dune for building, and merlin for editor integration (I did pay the complexity task using a language server for neovim, but merlin integrates with vim pretty trivially). It was easier than setting up Java + mvn|sbt|whatever + Intellij, easier than Python... there's maybe more going on than Go because the tools aren't all in one distribution, but man are the returns over Go massive (for the things I value).
  - loxs 6y ago
    I just migrated fully from OCaml to Rust. Couldn't be happier. I cannot even start to explain how big a difference a stable and mature ecosystem (including tooling) makes. Language-wise, Rust leaves some (small) things to be desired with regards of ergonomics, but this isn't even close to offsetting the general feeling of developer friendliness and fitness to the problems it aims to solve. The borrow mechanics/lifetimes are godsend and one starts to wonder why all languages are not like this. Rust completely exhausted the problem domains of these languages (for me): Java, Go, OCaml, even Python for something that would require more than 50 lines of code. Once you "get it", it's not harder to write than these. OCaml still has some very advanced features (types) like polymorphic variants, GADTs and others, which I have used in the past to elegantly solve (in a type safe way) problems. But at least for my use-cases (and the software that I migrated) these were mostly problems created by the functional, garbage collected nature of OCaml. With Rust's imperative (but still very type-safe) nature I just don't need to apply such complicated idioms - it seems to fit perfectly to the "real world problems". And on top of that I get 3-4x speedup "for free" (for the program that I migrated). I added some very "lightweight" (as in "easy to do") parallelism and I got another x3 speedup (for the most frequent use case). This kind of speedup makes a dramatic difference. It enables me to do things that were previously not possible in the OCaml version. And yes, on top of that I get to use Cargo, which feels decades ahead of every other package manager I have used and the IDE support (at least in IntelliJ/CLion) is pretty good (OCaml is nowhere near). PS. And Traits are extremely cool
cercatrova 6y ago
TypeScript might be good for your use case, expressing programs in terms of their types. You could also try OCaml/ReasonML.
cik 6y ago
Which python limits do you run into, for which you prefer Go?

mylons 6y ago

What limitations do you run into with the flask/python setup? (I’m not asking what ARE the limitations, I’m asking which ones specifically you run into)

piinbinary 6y ago
For large data sets, it sometimes spends all day in GC while just loading the data into memory (this is actually more of a problem in Ruby than in Python, but it still exists here). It's also not very efficient about using memory, so it can be easier to get a dataset to fit in RAM in Go vs. Python. Python also makes it hard enough to make use of multiple cores that it becomes easier to just use Go. The same goes for making it fast. It can be done in Python, but I tend to be happy with the performance of my Go code out of the box.
- leaveapennyman 6y ago
  I’m interested in learning more about Ruby’s garbage collection issues. Could you recommend any resources on this? Or any tools in the Ruby language that would allow me to test this myself?
  - themantri 6y ago
    https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html https://www.joyfulbikeshedding.com/blog/2019-03-14-what-caus... is a pretty interesting resource about this.
- deleted 6y ago
  [deleted]
- chucky_z 6y ago
  Have you ever tried Pypy? I know when I had issues with GC and some memory usage (specifically dealing with large datasets as well, I was parsing large MySQL query results -- many millions of rows) Pypy didn't really drop memory usage much, but it did give me an almost 10x speedup. Enough that I went "oh, guess I don't need to use a different language."
  - lanstin 6y ago
    I had a python server written in Python with greenlets to hat I switched to PyPy and the load it could handle went up maybe 4x. (5k per second to 20k per second. I took two weeks to switch to go and it went up another 10x to 100k per second and had less bugs (in fact a parsing bug in the python caused messages greater than 64k to be dropped; the Go handled partial reads properly and starting sending 2M msgs (to Kafka). The subscribers started getting the long messages and barfing making the system go wonky. Oddly all the giant messages were corrupt data anyways. Code size was about the same due to stuff being implemented in Python manually being handled by Go runtime.

jen20 6y ago

What is it that you find more convenient about Python/Flask?

piinbinary 6y ago
Flask is better in the little mechanics of interacting with HTTP - parsing a parameter out of a URL (in Flask, it's a function argument; in Go, it's at least one extra line per arg), returning JSON (just return a dictionary in Flask), handling errors. Flask also is able to auto-reload the code when it changes (turn on development mode) which is really convenient. Plus all the ways that Python is more convenient than Go.
- andreygrehov 6y ago
  But...you are comparing a web framework with a Go's standard library. I mean, it's not a fair comparison. I'm pretty sure there are Go modules that support those handy things.
  - dnautics 6y ago
    doesn't go stdlib contain an http server library "that you are supposed to use in prod"?
    - andreygrehov 6y ago
      It does. It's rock solid and most people use it in production. But the parent is more concerned about quick and handy functions. IMHO, such functionality is usually part of third-party libraries and is certainly an option in Go. It's just idiomatic Go favors KISS principles, less external dependancies and a more self-contained software. The benefit is that pretty much all the Go source code is more-or-less identical, regardless of an author. It makes it super simple to understand and deep dive into a random complex piece of software. Go is not about abstractions, Go is about solving problems here and now.
  - barumi 6y ago
    > But...you are comparing a web framework with a Go's standard library. I mean, it's not a fair comparison. Why do you feel that the comparison isn't fair? Aren't we talking about modules that provide the features necessary to implement a type of application? Would it make any difference if tomorrow there was a PEP making Flask a standard component? Unless you are aware of any Go module that addresses the downsides of Go with regards to Python, your observation is mute and unhelpful.

thiht 6y ago

> While writing servers in Flask + Python is much more convenient The only time I had to use Flask, it was a nightmare to deploy. To this day I still fail to understand why I had to mess with Nginx, gunicorn, WSGI or whatever just for a basic backend. The dev experience was fine, but I don't remember it being more convenient than using Go

pydry 6y ago
Because web servers look simple from the outset but are, underneath it all, very complicated. And it's not a web server. If you want something basic, gunicorn can run flask with one line.
- thiht 6y ago
  But I still need gunicorn, don't I? My point is that with Go, I just have to execute my single binary and I'm good to go. It doesn't prevent me from adding a reverse proxy and a loaf balancer if I want to, but I don't need gunicorn or other runners. Also if a single line is enough, maybe adding it clearly to the doc would be nice.
  - BiteCode_dev 6y ago
    You don't. Some frameworks are stand alone. It just happens flask uses wsgi, and so you need gunicorn.
  - barumi 6y ago
    > But I still need gunicorn, don't I? My point is that with Go, I just have to execute my single binary and I'm good to go. I don't see how your point is relevant. I mean, with interpreted languages you need to run the interpreter, but if having to write an additional word or line of code bothers you that much then you should not worry about it if you dockerize the app or, god forbid, use a launcher script. I mean, in non land no one ever complained about how hard it is to add scripts to mom's start and prestarte targets to launch a server by doing mpm start. So why is this suddenly relevant with Go?
- 6y ago

dhekir 6y ago

OCaml is good for handling abstract syntax trees (strong typing, pattern matching) and its mix of imperative and functional programming don't require using borrow-checker nor monads. You can also consider Reason, if you prefer a more C-like syntax.

rienbdj 6y ago
Also consider F#! In some ways it is more limited (no first class modules) but in others it is arguably better (operator overload, C# ecosystem, a slick take on do-notation).
piinbinary 6y ago
I really ought to learn OCaml (besides just the toy programs I've written). I think if I got past the messy ecosystem (is tuareg the right thing to use, or is that the old thing?) I'd really like it.
- blandflakes 6y ago
  I always wonder if I just showed up late when I hear about the ecosystem! I just get opam for packages, dune for building, and merlin for editor integration (I did pay the complexity task using a language server for neovim, but merlin integrates with vim pretty trivially). It was easier than setting up Java + mvn|sbt|whatever + Intellij, easier than Python... there's maybe more going on than Go because the tools aren't all in one distribution, but man are the returns over Go massive (for the things I value).
- loxs 6y ago
  I just migrated fully from OCaml to Rust. Couldn't be happier. I cannot even start to explain how big a difference a stable and mature ecosystem (including tooling) makes. Language-wise, Rust leaves some (small) things to be desired with regards of ergonomics, but this isn't even close to offsetting the general feeling of developer friendliness and fitness to the problems it aims to solve. The borrow mechanics/lifetimes are godsend and one starts to wonder why all languages are not like this. Rust completely exhausted the problem domains of these languages (for me): Java, Go, OCaml, even Python for something that would require more than 50 lines of code. Once you "get it", it's not harder to write than these. OCaml still has some very advanced features (types) like polymorphic variants, GADTs and others, which I have used in the past to elegantly solve (in a type safe way) problems. But at least for my use-cases (and the software that I migrated) these were mostly problems created by the functional, garbage collected nature of OCaml. With Rust's imperative (but still very type-safe) nature I just don't need to apply such complicated idioms - it seems to fit perfectly to the "real world problems". And on top of that I get 3-4x speedup "for free" (for the program that I migrated). I added some very "lightweight" (as in "easy to do") parallelism and I got another x3 speedup (for the most frequent use case). This kind of speedup makes a dramatic difference. It enables me to do things that were previously not possible in the OCaml version. And yes, on top of that I get to use Cargo, which feels decades ahead of every other package manager I have used and the IDE support (at least in IntelliJ/CLion) is pretty good (OCaml is nowhere near). PS. And Traits are extremely cool

cercatrova 6y ago

TypeScript might be good for your use case, expressing programs in terms of their types. You could also try OCaml/ReasonML.

cik 6y ago

Which python limits do you run into, for which you prefer Go?