10 ms·

Go's definitely picking up momentum. I know that Mozilla is shifting much of its services infrastructure to Go. We're porting over our Arabic sentiment engine,

Go's definitely picking up momentum. I know that Mozilla is shifting much of its services infrastructure to Go. We're porting over our Arabic sentiment engine, currently in Python/Cython to Go. If you're dealing with simple data structures, going from Python to Go is almost a line-for-line port, but the performance benefits are, of course, massive. Our benchmarks show a 40x speed improvement so far. Lastly, for anyone thinking about taking the plunge, use Go tip (from their source code repo) - don't use their "stable" releases. They fix bugs so fast you'll always want to be current.

DASD 14y ago
Do you have a citation for Mozilla using Go? I would be a bit surprised given their(Mozilla) development of Rust.
- waterside81 14y ago
  Sure, check out their github pages (here's a few): * https://github.com/mozilla-services/heka-mozsvc-plugins https://github.com/mozilla-services/heka-mozsvc-plugins * https://github.com/mozilla-services/heka https://github.com/mozilla-services/heka My friend works for Mozilla - that's how I knew.
  - DASD 14y ago
    Thanks waterside81. Can anyone explain the difference between Mozilla and Mozilla services? A cursory glance seems some of the Github repositories overlap.
    - rfk 14y ago
      Mozilla Services is the team within Mozilla that builds/runs much of the backend infrastructure, e.g. the firefox sync servers, marketplace servers etc. The existence of separate "mozilla" and "mozilla-services" github projects is largely a historical accident, since different teams started moving to github organically at different times. (Source: I work for Mozilla, on the Services team)
      - mh- 14y ago
        Source: I work for Mozilla-Services
- azakai 14y ago
  Rust and Go are not similar or competing. They are both new, but aside from that the differences are huge. So if a project uses one, that doesn't mean the other was a possibility for that project too.
  - DASD 14y ago
    Indeed they are not similar but wildly different tools are often used in a similar or competing use cases(eg. the blog's post of the company using Ruby and conversion to Go). I had asked because if you were to reverse the usage do you think Google is likely using much Rust since it is developing Go? Just to clarify, I see a bright future for both languages.
    - tmzt 14y ago
      It would be interesting to see Rust used in Android in some of the parts that are now C or C++.
  - steveklabnik 14y ago
    The Rust team agrees: https://github.com/mozilla/rust/wiki/Doc-language-FAQ https://github.com/mozilla/rust/wiki/Doc-language-FAQ (control-f for 'this Google language')
  - stcredzero 14y ago
    > Rust and Go are not similar or competing. Rust is more "kitchen sink" (done as elegantly and cleanly as possible) and Go is more minimalist.
    - pcwalton 14y ago
      Being "kitchen sink" isn't a design goal of Rust. Rather, Rust is a lower-level, safer language. Go is higher-level. Rust strives to be as minimalist as possible without sacrificing the goals of low-level control over memory and C++ performance (optional GC), memory safety, race-free concurrency, and type safety (no null pointers).
cpprototypes 14y ago
Just curious, what were the tradeoffs for profiling the Python code and rewriting slow parts in C vs total rewrite in Go? For high level languages, the usual argument has been to rewrite just the slow parts in C or some other low level language.
- codygman 14y ago
  One thing you don't get in the C/Python combo is total static typing. The other big win of Go is elimination of a lot of runtime errors. So it comes down to which will be more of a win for you and your product. Go has a big advantage here IMO though because of static typing AND performance increase.
  - pjmlp 14y ago
    Like any other language with static typing and available compiler implementations.
    - codygman 14y ago
      Is that relevant though? The parent is asking about a comparison between python+c and Go for the OP's software, so doesn't it make sense to answer to that? Of course, you are correct. Rewriting in any other language with static typing or an available compiler implementation would also provide the benefits of static typing and detection of runtime errors.
      - pjmlp 14y ago
        Because those benefits are not some special feature of Go. On the other hand I think youth generation of programmers lack exposition to programming languages, like we used to have in the old days, hence they make quite limited comparisons.
- waterside81 14y ago
  And that's what we did during our first pass: identify the slow parts and port them to C. This gave us about 30% better performance. Without getting too much into the nitty gritty of our specific case, this wasn't enough. We needed some massive speed improvements, I'm talking in the order of 100X faster. The nature of our algorithms was such that they could be done in parallel (i.e map/reduce) - an ideal candidate for Go's goroutines. We actually tried to make it parallelizable at first in Python, using gevent (and even just multiprocessing) and the results were not great. One other aspect that really guided us towards Go was memory usage. Python was just sucking up so much memory whereas our Go implementation thus far is so much thinner.
beatpanda 14y ago
Hi, I'm in the market* for an Arabic sentiment engine for some academic research, can you contact me? My email is in my profile. *Note that "in the market" doesn't mean I have a lot of money to spend :D
- adamnemecek 14y ago
  What is an "Arabic sentiment engine" :-D? Like textual analysis as to whether a piece of Arabic text is positive or negative?

DASD 14y ago

Do you have a citation for Mozilla using Go? I would be a bit surprised given their(Mozilla) development of Rust.

waterside81 14y ago
Sure, check out their github pages (here's a few): * https://github.com/mozilla-services/heka-mozsvc-plugins https://github.com/mozilla-services/heka-mozsvc-plugins * https://github.com/mozilla-services/heka https://github.com/mozilla-services/heka My friend works for Mozilla - that's how I knew.
- DASD 14y ago
  Thanks waterside81. Can anyone explain the difference between Mozilla and Mozilla services? A cursory glance seems some of the Github repositories overlap.
  - rfk 14y ago
    Mozilla Services is the team within Mozilla that builds/runs much of the backend infrastructure, e.g. the firefox sync servers, marketplace servers etc. The existence of separate "mozilla" and "mozilla-services" github projects is largely a historical accident, since different teams started moving to github organically at different times. (Source: I work for Mozilla, on the Services team)
    - mh- 14y ago
      Source: I work for Mozilla-Services
azakai 14y ago
Rust and Go are not similar or competing. They are both new, but aside from that the differences are huge. So if a project uses one, that doesn't mean the other was a possibility for that project too.
- DASD 14y ago
  Indeed they are not similar but wildly different tools are often used in a similar or competing use cases(eg. the blog's post of the company using Ruby and conversion to Go). I had asked because if you were to reverse the usage do you think Google is likely using much Rust since it is developing Go? Just to clarify, I see a bright future for both languages.
  - tmzt 14y ago
    It would be interesting to see Rust used in Android in some of the parts that are now C or C++.
- steveklabnik 14y ago
  The Rust team agrees: https://github.com/mozilla/rust/wiki/Doc-language-FAQ https://github.com/mozilla/rust/wiki/Doc-language-FAQ (control-f for 'this Google language')
- stcredzero 14y ago
  > Rust and Go are not similar or competing. Rust is more "kitchen sink" (done as elegantly and cleanly as possible) and Go is more minimalist.
  - pcwalton 14y ago
    Being "kitchen sink" isn't a design goal of Rust. Rather, Rust is a lower-level, safer language. Go is higher-level. Rust strives to be as minimalist as possible without sacrificing the goals of low-level control over memory and C++ performance (optional GC), memory safety, race-free concurrency, and type safety (no null pointers).

cpprototypes 14y ago

Just curious, what were the tradeoffs for profiling the Python code and rewriting slow parts in C vs total rewrite in Go? For high level languages, the usual argument has been to rewrite just the slow parts in C or some other low level language.

codygman 14y ago
One thing you don't get in the C/Python combo is total static typing. The other big win of Go is elimination of a lot of runtime errors. So it comes down to which will be more of a win for you and your product. Go has a big advantage here IMO though because of static typing AND performance increase.
- pjmlp 14y ago
  Like any other language with static typing and available compiler implementations.
  - codygman 14y ago
    Is that relevant though? The parent is asking about a comparison between python+c and Go for the OP's software, so doesn't it make sense to answer to that? Of course, you are correct. Rewriting in any other language with static typing or an available compiler implementation would also provide the benefits of static typing and detection of runtime errors.
    - pjmlp 14y ago
      Because those benefits are not some special feature of Go. On the other hand I think youth generation of programmers lack exposition to programming languages, like we used to have in the old days, hence they make quite limited comparisons.
waterside81 14y ago
And that's what we did during our first pass: identify the slow parts and port them to C. This gave us about 30% better performance. Without getting too much into the nitty gritty of our specific case, this wasn't enough. We needed some massive speed improvements, I'm talking in the order of 100X faster. The nature of our algorithms was such that they could be done in parallel (i.e map/reduce) - an ideal candidate for Go's goroutines. We actually tried to make it parallelizable at first in Python, using gevent (and even just multiprocessing) and the results were not great. One other aspect that really guided us towards Go was memory usage. Python was just sucking up so much memory whereas our Go implementation thus far is so much thinner.

beatpanda 14y ago

Hi, I'm in the market* for an Arabic sentiment engine for some academic research, can you contact me? My email is in my profile. *Note that "in the market" doesn't mean I have a lot of money to spend :D

adamnemecek 14y ago
What is an "Arabic sentiment engine" :-D? Like textual analysis as to whether a piece of Arabic text is positive or negative?