13 ms·
Space Monkey dumps Python for Go
- impostervt 12y agoIs anyone else getting SSL errors on the site?
- jtolds 12y agoWe're getting slammed :( We're spinning up a new webserver.
- phaedryx 12y agoI'm curious what the memory usage was like before and after.
- jtolds 12y agoDoh, I totally should have mentioned that, cause yeah that was big too. We went from about 80-95mb resident memory (and our cgroups config doesn't allow for much swap) to about 50-60mb on average, which is a metric we weren't even trying to get lower. So, yeah, that's another win we should have mentioned.
- jessaustin 12y agoAfter all of this optimization, we got up to 1.2 MB/s. Ouch. That was a lot of work for 0.2 MB/s. The next 2.8 MB/s was also a lot of work, but it seems conceptually more straightforward.
- wspeirs 12y agoSwitching to Go though they were able to get 4MB/s. Even though they claimed to do "line-by-line" translation, I wonder how much of that speed-up was language/libraries vs rearchitecting.
- robotresearcher 12y agoUnless there is some reason to disbelieve the line-by-line claim, can't we assume it's all due to the more performant implementation? Python is a joy when it's fast enough, but it's not surprising that it isn't good at throughput on small devices.
- wspeirs 12y agoI didn't mean to imply they were lying in the post, more that it's probably hard to do a line-by-line translation from Python to Go. The nature of having to convert to things like channels in Go might help to re-architect the program even at a micro-level.
- melling 12y ago"So in the bottom of the ninth, we decided to transliterate our 90k lines of Python directly to Go, line by line. It took us about 4 weeks." How many people worked on the project? What's the build time on a program of this size?
- jtolds 12y agoWe had about 3 or 4 guys full time. On my silly little i3 laptop, the whole codebase builds in 18 seconds - I suspect most of that is our C extensions.
- yukichan 12y agoC extensions because something you wanted wasn't in go? Are you using cgo to build?
- wisty 12y agoPresumably, because they were doing it line by line, and the Python version used C extensions (for performance reasons).
- jtolds 12y agoYeah, we actually have a few cgo packages. We have OpenSSL bindings (we need to use our hardware accelerated crypto, go's crypto/tls lib doesn't support that (yet)), a faster CRC thing, we needed monotonic clocks and couldn't wait for Go 1.3, etc. We'll be releasing a bunch of these. Particularly the OpenSSL bindings, which even without hardware acceleration are faster than Go's TLS library, and despite Heartbleed, OpenSSL is not vulnerable to timing attacks like crypto/tls is.
- yukichan 12y agoI wonder how many lines of code they ended up with afterward, although admittedly lines of code is a pretty silly metric. I think it is useful in orders of magnitude when describing the size of a project, 100 lines, 1,000 lines, 10,000 lines, 100,000 lines all speak volumes about effort. So I guess what I'm wondering if you got about a roughly similar sized code base or significant change difference?
- sandGorgon 12y agoWe rewrote Twisted Deferred handling to be 30% faster. We optimized the Twisted Reactor event loop to not only do less constant work, but to prefer incoming I/O. ...We spent long nights poring over profiling readouts and traces, wrote our own monitoring system, and wrote our own benchmarking tools ... We enabled cgroups on our Python process to tightly control memory usage and swap contention without requiring a different Python memory allocator scheme. This is very interesting for the rest of the Python/Twisted community - now that they are not using this (and so is'nt part of the secret sauce), I wonder if they are willing to push this out as a patch/blog post ?
- jtolds 12y agoWe're planning on open sourcing a bunch of stuff. We'll have some follow up posts.
- sandGorgon 12y agothanks ! really look forward to this..
- stuaxo 12y agoDid you release your faster version of the Protocol Buffer library ?
- jtolds 12y agoWhile we're planning on open sourcing some things, I'm not sure we'll ever open source that one. Even though it was much faster, it didn't see much time in production due to the less than desirable impact it made (go Amdahl's law), and it still has some bugs and was highly customized to our specific protocol buffer definitions. Internally, Google has something faster for Python, but they haven't released it yet. You might try putting pressure on them here: https://code.google.com/p/protobuf/issues/detail?id=434 https://code.google.com/p/protobuf/issues/detail?id=434 http://yz.mit.edu/wp/fast-native-c-protocol-buffers-from-python/ http://yz.mit.edu/wp/fast-native-c-protocol-buffers-from-pyt... talks about the issues some more with some links to other rewrite attempts.
- sandGorgon 12y agowould you have gone with thrift if you had to do it all over again ? I ask because I saw the bug description and the glacial response ;)
- jtolds 12y agoNo, I think we're honestly even more sold on protocol buffers' wire format than before. That said, if I did want to go looking elsewhere, I think the main serialization format I am really interested in is Cap'n Proto: http://kentonv.github.io/capnproto/ http://kentonv.github.io/capnproto/
- leccine 12y agoWell it is kind of obvious that a compiled language is going to be faster than an interpreted one, especially the way how these interpreters work (https://wiki.python.org/moin/GlobalInterpreterLock https://wiki.python.org/moin/GlobalInterpreterLock). You can fool yourself with Twisted (or in Ruby with EventMachine, Goliath etc.) but it gets so just a bit ahead. The surprising fact for me is that Go is not as much faster. I was expecting a bigger gap in the performance between Go and Python. Understanding where your bottlenecks are is crucial and it is not super hard in Go. https://www.datadoghq.com/2014/04/go-performance-tales/ https://www.datadoghq.com/2014/04/go-performance-tales/ I guess the SpaceMonkey guys might further improve the performance just by doing a thorough analysis on their code.
- MBlume 12y agoThey wrote a naive, line-by-line translation of their python code in Go. There's probably still a lot of low-hanging optimization to be had.
- leccine 12y agoExactly. I am kind of wondering why this 1:1 translation came up why not just follow best practices and implement a functionality instead...
- jessaustin 12y agoThe path they chose was much more predictable than your suggestion. I.e., after their team spent three days at it, their estimate for the total translation was probably within 50% of the month it actually took them. I don't expect they could have been as accurate in estimating an entirely new implementation.
- jtolds 12y agoRight on. We immediately had not totally horrible estimates for how long it would take. Further, we have always really been inspired by Joel Spolsky's article on rewrites: http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html In fact, I can't help but wonder if we would have attempted a Go rewrite sooner if not for that article.
- wisty 12y agoThat's actually kind of shocking. I thought it would take way more work to port from Python to Go, and a 400% speedup is way less than I'd have expected (though this is string handling, and the Python was already brutally optimised with C modules where it counted).
- jtolds 12y agoWe're I/O bound now (as we should have been to begin with), so speedup isn't the linear function you might expect here.
- chimeracoder 12y agoThis article doesn't address one of the biggest benefits I've noticed in switching from Python to Go. I've been using Go for almost two years now. Like OP, I am/was primarily a Python developer. Like OP, my first Go project was a time-sensitive rewrite[0] of a project from Python (tornado) to Go. Even though I was an experienced Python developer, the Go rewrite was marginally (~20%) faster[1]. But the real benefit came from the subsequent development - refactoring, rearchitecting, and maintaining the project on an ongoing basis. Go was designed to make it easy to scale the maintenance[2] of software, and on this one axis, it absolutely blows every other language and environment I've used out of the water. For a fresh project, I'd say Go is about 10% slower to write than the equivalent Python[3] for the average case. But the time/cost savings are very quick to come thereafter. [0] I would absolutely not recommend doing time-sensitive rewrites in general, but that decision was a separate matter. [1] Some of this is due to the nature of rewrites in general, but the fact that it wasn't slower to use a language I'd never used before says something about the language. [2] Scaling software development as teams grow is very different from scaling software as users grow. [3] Assuming comparable familiarity with both languages, which is rarely the case, of course
- aturek 12y agoThis sounds really interesting, but can you elaborate on how Go scales better with maintenance? (disclosure: I haven't fiddled with Go at all yet, so I know nothing about the language itself)
- SpikeGronim 12y agoSeveral things make it easier to maintain: - super fast compile times for fast developer iterations - you can create an interface (set of methods) that your module owns, and apply it to objects created by other modules without recompiling. If you only need one method you can take that one. It encourages decoupling you from your dependencies. - the code has one correct formatting convention and a tool that will auto-format your code - many complex rules for numeric conversions that are implicit in C and cause no end of trouble are explicit and much simpler in Go. - treating concurrency as a series of sequential process connected with channels makes it MUCH easier to reason about.
- andybak 12y agoI've got a completely data-free feeling that Ruby people hitting performance bottlenecks seem to reach for node whilst Python people have a tendency to look at Go. Does anyone else think that this rings true? And if so - is it a cultural thing?
- steveklabnik 12y agoThere are a _lot_ of Rubyists interested in Go, I'm not sure we can draw any real conclusions about the relative popularity of the move at this time.
- AYBABTME 12y agoI have the feeling that both camps are going for Go instead of Node. My only data-point is the shop I work for, and folks in the community that I know come from a Ruby background, such as Dominik Honnef, Ben Johnson, Mitchell Hashimoto.
- chc 12y agoMy impression is that people writing Web interfaces reach for Node while people writing CPU-heavy programs reach for Go. It so happens that Ruby people are more often writing simple Web apps while Python is somewhat more likely to be used for more computationally intensive tasks, but I think that's about as far as the correlation goes.
- bdcravens 12y agoI think that depends on whether they're more of a "Rubyist" or primarily Rails developers who know "just enough Ruby". I think Node (and especially a framework like Meteor or Express) is more approachable for those who spend most of their time in a framework like Rails.
- _random_ 12y agoJavaScript people go for Node.js - they have nothing to loose.
- deleted 12y ago[deleted]
- sigil 12y agoIt sounds like the cost of context switching between these very different workloads (crypto, disk I/O, network I/O, protocol buffer parsing and formatting) could be improved in Twisted. Any idea where the overhead comes from? Twisted, or the Python interpreter itself? Is this a GIL performance issue? Or perhaps even lower -- something here is really hostile to CPU cache? I realize this is a matter of taste, but my favorite async framework is still the kernel. Write small programs that do one thing well (and thus have pretty uniform workloads) and then let the kernel balance resource usage. After continuing to hit walls with the standard Python Protocol Buffer library, we wrote our own that was 5x faster than the barely-documented C++ Python Protocol Buffer compiled module support. Ugh yeah, the standard Python protobuf library is pure python and horribly slow. And it requires code generation -- in a dynamic language! The C++ one is faster, but also requires code generation and is just nasty to work with. Not that this matters much to you at this point, but I have a small C/Python protobuf library that's 10x faster than the standard Python protobuf: https://github.com/acg/lwpb https://github.com/acg/lwpb PS. I see you're in SLC area -- me too. We should talk tech shop in person sometime!
- jtolds 12y agoHi! Yes, you should totally swing by. Shoot us an email :) I think the sort of performance issues we were hitting were honestly related to just Python runtime overhead. A Python function call is actually really expensive, and with Twisted Deferred handling, it's really hard to eliminate the massive amount of function calls each I/O event does. We just had a really slow CPU so we did our best to eliminate as many Python function calls in hotspots as possible, but yeah that was challenging.
- sigil 12y agoA Python function call is actually really expensive, and with Twisted Deferred handling, it's really hard to eliminate the massive amount of function calls each I/O event does. Ah, makes total sense. Any particular reason you chose to use Twisted in a hardware appliance? Is there a web interface that's supposed to be super responsive?
- hyp0 12y agoWouldn't java be the natural rewrite, for performance speed up? Java is betwen Python and C, in terms of ease-of-use and performance. They mention a C rewrite, but not a java one... I guess the concept here is that Go is between Python and Java: (much of) the ease of use of Python + (much of) the performance of Java. The type system literally straddles the dynamic and static. [ I also detect an enthusiasm for cool, new tech for its own sake (despite risks and cost, or whether it's actually better or not - hell let's find out!) ]
- untothebreach 12y agoSince they are targeting an embedded ARM system, maybe they had concerns about running the JVM on such constrained hardware?
- sagichmal 12y ago> The type system literally straddles the dynamic and > static The Go type system is fully static...
- nostrademons 12y agoGo has an escape hatch (interface{} and downcasts) that basically give it an embedding of a dynamic type system inside the static one. You end up having to use this a lot if you're doing things like decoding complex JSON data structures or working with containers, since Go doesn't have generics.
- flavoie 12y agoI'm curious if they tried Cython. I've read that you can achieve up to 35% speedup just by compiling python code, plus you can type pyx file to get near C performance. I never tried it, this would have been a nice use case.
- jtolds 12y agoWe did. We used Cython where we could, but Cython doesn't work with Twisted (or at least inline deferreds, which holy we had everywhere), due to different generator semantics. Edit: in fact, our internal protobuf lib was all generated Cython. We had a code-gen inception.
- sfk 12y agoAnd the website takes more than 30s to load...
- bananas 12y agoAt least it does load. A lot of sites fall over on a good HN'ing.
- yelnatz 12y ago> Our website is Apache + Python
- jtolds 12y agoWe just doubled our instance count, we should be good now.
- sfk 12y agoLGTM, and your response time to "support" requests is certainly great! I'll leave my grumpy comment, so that the context does not get lost.
- jlafon 12y agoIf you did a line for line transliteration, did you use any of Go's concurrency features such goroutines or channels? I ask because I find that when I translate a program from Python to Go that it's beneficial to structure the program differently so that I can use goroutines.
- jtolds 12y agoWith Twisted (and with our threadpool worker pools), many patterns translated directly into Goroutine usage in a much cleaner way. Where with Python we were using our own helper libraries, Go's stdlib and the language itself were often more than enough. We didn't end up using channels too much. Deferreds got translated into Futures that we wrote a small library for. Many of our Go-specific utility classes do use channels heavily though.
- deleted 12y ago[deleted]
- jpatel3 12y agofyi..login link fails on the site.
- wting 12y agoI think people are rediscovering the benefits of static typing for performance and correctness reasons. The latest generation of statically typed languages[0] all have some level of type inferencing which helps. If Python is fast enough for you, it’s a fantastic language. The problem is once performance or codebase demands scale, dynamic typing rears its ugly head and there are no simple solutions. At work we sidestep this issue by writing a plethora of tests, but now dynamic typing productivity gains are offset and we spin up a lot of AWS instances for performance. [0] Go, Rust, Scala. Haskell and OCaml have had it for a while.
- diminish 12y agoWhile web 2.0. and mobile is reaching maturity with winners are consolidating the market we see the trading of productivity with performance and reduction of operational costs... I m more curious of upcoming booms and which languages and frameworks they are going to bring.
- femngi 12y agoC++ isn't exactly a new language but has also had type inference for a few years. I think people still think of C++ as that dusty old thing in the corner and forget that it is actually capable of a lot of the things users of these new languages get excited about.
- masklinn 12y agoC++'s auto is younger than most of these languages (let alone than Haskell or MLs), to say nothing of actual tooling support.
- dilap 12y agogcc has had auto under a different name (typeof) for years and years, though i don't think it got much use outside of programming competitions.
- decasteve 12y agoC++11 is almost a new language. Coming back to C++ now after almost a decade away is exciting.
- micro_cam 12y agoI really hope the go compilers will soon support shared object output so we can write python extensions in go instead of having to fully dump one for the other. I know it was a maybe for go 1.3 but I haven't heard much chatter and fear it will get pushed backed.
- dalek2point3 12y agoCan someone tell me how much control GOOG has over the development of Go? I'm interested, but worry that is this another GOOG "open source" project that they have a huge say in.
- GhotiFish 12y agoLooks pretty open to me: here's the current bleeding edge: https://code.google.com/p/go/source/browse https://code.google.com/p/go/source/browse here's the issue tracker: https://code.google.com/p/go/issues/list https://code.google.com/p/go/issues/list looks like it's being used by the dev team. If you would like to fork the golang project, run this command: hg clone https://code.google.com/p/go This is pretty clearly not Android.
- ansible 12y agoNote also that Googlers have stated that they use the public version like everyone else.
- wmf 12y agoEven if Go is completely controlled by Google today, there are enough companies using it that they can fork if Google turns evil.
- dilap 12y agoI'm curious as to how they did a line-by-line port of the twisted program -- twisted's async (plus coroutines-light w/ via generators) seems like a significantly different model than goroutines + channels. I wonder if they actually implemented Deferred in Go?
- jtolds 12y agoWe implemented a future class that had very similar functionality to a deferred. We used goroutines heavily but channels not so much. I suspect we can improve go runtime scheduler performance by starting to go through and replace pieces with channels where appropriate.
- adem 12y ago"So in the bottom of the ninth, we decided to transliterate our 90k lines of Python directly to Go, line by line." A question here: Is it always a practical approach to transliterate line by line when transitioning to another language or because "Go [is] semantically very similar to Python"?
- avenger123 12y agoUsually it's too much work to do a refactor with the new language and a conversion. Much easier to convert first (with whatever warts the existing code base has) then do a refactor. So I doubt their approach had too much to do with Go per se and more with wanting to get something out sooner rather than later.
- kazagistar 12y agoBoth language semantics and language best practices are pretty similar in Python and Go, and I am sure this helps a lot. Transliterating to something like Java would be hard because their interfaces work differently enough from duck typing to force a lot of refactoring. And switching to something even more different, like Haskell or Clojure, would just make transliteration impossible.
- saurik 12y agoI once consulted for a company whose previous consultant had tried to help them do a line-for-line translation (using off-site random-people-in-a-sweatshop online programmers for hire) from ASP to Rails. They pitched it as "transcoding". I will argue that this was not practical (though I can appreciate the idea that at first glance it almost sounds amazingly scalable ;P).
- deleted 12y ago[deleted]
- the_paul 12y agoIt's a female? How can you tell?
- TD-Linux 12y agoI wrote some Python to run on an OpenWRT (MIPS-based, 400MHz) router that interfaced with an Xbee radio and ran a web interface. I quickly discovered that what I expected to be IO-bound was in fact CPU-bound - reformatting the data and all the related text operations were fast enough after optimization, but only by a hair. Our solution was to switch to a dedicated ARM board (beaglebone) attached to the router. But I'm definitely going to take a look at using a compiled language now, as the codebase is still very small.
- cgag 12y agoI like seeing people embrace static typing, but it makes me sad that it's a language where you can't even write a container (a map/dictionry/linked-list/etc) without giving up type safety. I recognize that Golang is filling a gap, letting people get static typing without having to learn much new stuff, and there's value in that, but I think Go enthusiasts should read these great posts by Tikhon Jelvis with an open mind: https://www.quora.com/Googles-programming-language-Go-seems-to-be-the-most-modern-and-well-featured-language-in-existence-today-Is-that-the-case-or-are-there-major-drawbacks-that-Im-not-seeing https://www.quora.com/Googles-programming-language-Go-seems-... https://www.quora.com/Go-programming-language/Do-you-feel-that-golang-is-ugly https://www.quora.com/Go-programming-language/Do-you-feel-th...
- burntsushi 12y agoQuite frankly, those articles are absolutely terrible. Firstly, it's clear that the author assumes there exists some correct way to design a language. (I reject that.) Secondly, the author never acknowledges any of the trade offs that come from adding additional features to a language. (Hint: one of those trade offs is... there are more features!) This isn't a revolutionary thought. More compile time safety generally means more complexity in the type system. We, as programmers, must decide when, where and how much we're willing to pay for that type safety. You can go balls-to-the-wall with Idris or Agda or ATS and encode a whole mess of things directly into your types. Maybe you can completely remove out-of-bounds errors! Surely, this is better than the alternative, which admits less safety. This isn't a fair comparison because all three of those languages are ridiculously hard to use for practical things. (Well, it's hard for me, anyway.) Then there are other, more practical languages like Rust, Java, Haskell, OCaml, etc., that all provide a helluva lot more compile time type safety than Go does. (Or, to be precise, a lot more expressive power with respect to compile time safe polymorphism.) Some of the languages go even further; Rust eliminates data races and Haskell isolates "side effects" with monads. All of these things are wonderful, because we get a lot more guarantees about ways in which our program cannot fail once it's compiled. But they aren't free. They come with a cost. Typically the cost is in greater language complexity. The programmer must write code that the compiler accepts. If you're in that language's wheelhouse, maybe the complexity isn't so bad. But when you want to write a program that isn't easily accepted by the compiler, you run into complexity. Maybe it's not that bad---maybe you just need to figure out how to write a monad transformer. Or maybe you just need to think a bit more carefully about the lifetime of a borrowed pointer. But other times, you need type families and multi-parameter type classes before you can make an unboxed vector with your own type: http://hackage.haskell.org/package/vector-0.10.9.1/docs/Data-Vector-Unboxed.html http://hackage.haskell.org/package/vector-0.10.9.1/docs/Data... --- The safety is great, but dammit, you're going to pay for it. On some days and for some projects, I'm more than happy to pay the price for safety. It is immensely satisfying to write a program and be able to make all sorts of claims about its safety. (I just wrote a regexp implementation in Rust. Guaranteed no memory errors. No GC. Totally type safe. It's awesome.) But other times, I don't feel like paying the cost. And that's OK. It's not sad. It's not "ugly." It's a legitimate and reasonable choice given a trade off. Saying a language is "badly designed" just because it didn't include your preferred set of features is bad juju IMO.
- dekhn 12y agoOne thing to mention, since the OP talks about developing for ARM: I was surprised recently: first that Go itself bundles a cross compiler (you can easily download Go on x86 and compile ARM binaries with it), second that this made developing for Raspberry Pi and similar platforms very productive, and third that many Linux distributions have QEMU and set up binfmt translation so ARM-compiled binaries run on x86 with no extra effort.
- jtolds 12y agoThe qemu binfmt extensions inside a chroot are so great. For the uninitiated: https://wiki.debian.org/EmDebian/CrossDebootstrap#QEMU.2Fdebootstrap_approach https://wiki.debian.org/EmDebian/CrossDebootstrap#QEMU.2Fdeb...
- azth 12y ago> provided that your problems are not algorithmic complexity What does that even mean?
- gjm11 12y agoIt means "provided the reason your code is too slow isn't that its performance scales very badly as a function of a parameter that, in practice, is not small". If your code takes time proportional to 2^2^n then for n=4 that's 2^16 x constant, which is probably fine for any reasonable constant, and for n=6 that's 2^64 x constant, which is probably too much for any reasonable constant, so if you're running into problems with n>=6 then moving to a different language or faster hardware or more-micro-optimized code is not going to help you. On the other hand, if your code takes time of order n^2 and typical values of n are in the thousands or millions, then the difference between C and Python, or Core i7 and Z80, or sloppily written code and highly-bummed code with an inner loop in hand-tuned assembler, may make a big difference.
- azth 12y agoThanks for the explanation. I am aware of that; however, the sentence I quoted does not make sense grammatically; "algorithmic complexity" is not an adjective.
- gjm11 12y agoI did consider adding a final paragraph saying something like "I am assuming here that you were sincerely asking a question rather than merely being rude and snarky, but of course that may not be true", but decided to play it straight. So, having established that you were merely complaining about a grammatical slip, and that despite asking "what does ... even mean?" you knew perfectly well what it meant: Yeah, I agree, it would have been more correct had they written something like "provided that your problems are not a matter of algorithmic complexity" or "provided that your problem is not algorithmic complexity". Congratulations, you found a mistake. (Though not, I think, anything to do with thinking "algorithmic complexity" an adjective.) But (1) so what? and (2) if I'm right in detecting a subtext of "man, if they can make that mistake, why should I take anything else they say seriously?", then I strongly disagree; anyone can perpetrate the occasional grammatical error. (Just out of curiosity, I looked at some of your recent comments, and found one having looked through considerably less text than the length of the article in which you found an error. I dare say one could do the same for mine. To err is human.)
- azth 12y agoWas the CPU pegged at 100% when using Python then?
- jtolds 12y agoyep
- aet 12y agoCan someone explain why SpaceMonkey is better than Dropbox?
- jtolds 12y agoSpace Monkey is cheaper for more space - $49/yr for 1TB. Further, because there's a device in your home, getting your data synced off your laptop is a much faster process - you aren't bottlenecked by whatever Dropbox is throttling your upstream at. Not to mention all of the DLNA, Samba, home media system integration we're working on.
- shiven 12y agoAnd lack of oligarchs, who signed-off on torture, on the company board. Now that can't be a bad thing! SpaceMonkey needs to use it as a selling point.
- ternaryoperator 12y agoIt would be good if you explained this more on your site. I couldn't figure out what the device was for, nor whether it was required, etc. I'd suggest a home page that clearly explains what you do and how it works. It's also not clear what happens price-wise after the first year. Is there some fee to pay for the device or just the $49/yr for the storage? Etc.
- bsder 12y agoWhy not just bump your processing power? No offense, but that was a hideous amount of effort for just 20% in one area. An extra dollar or two spent on a hardware upgrade wins for everything.
- azatris 12y agoWhat are the disadvantages of using Go instead of Python?
- NateDad 12y agoTesting can be more difficult in some cases, if you haven't carefully structured your code. In python you can always monkey patch whatever outside dependencies you have, whereas in Go, you can't do that without structuring your code to allow it. Careful use of interfaces and dependency injection can let you do it, but if you don't know to use that style, you can get big blocks of code that become hard to test because they expect the whole operating environment to exist, which may be hard to do during testing. If you want to write code in a Domain Specific Language (even if that domain is "math"), it's nearly impossible in Go. You can't do operator overloading or all the really crazy hacking on the functionality of basic parts of the language to make it work in fundamentally different ways (the way you can with numpy etc). Go will always look like types, functions, and methods. Right now, Go has no user-defined generics.. this is actually not a problem for many projects, but if you happen to need a lot of specialized containers for different types (like red black trees, graphs, etc), then it can be a barrier. There are ways around it, probably the best way is just code generation to duplicate the container code per type that you need. The other option is just using interface{}, which is basically void* or "Object" for Go, and casting in and out of it... but that can be slow. There's probably some stuff I'm missing, but those are the major points that I can think of.
- jtolds 12y agoMy guess is that if you've written code that's hard to test in Go, it's going to be hard to write good, non-brittle tests in Python. It's not so much that you need to write code amenable to dependency-injection to write good tests, you just need to write modular code. As a fan of DSLs, I actually do appreciate how easy it is to understand any Go that anyone has written, since there aren't any cutesy DSL tricks anywhere. So, yeah, it's hard/impossible to do DSLs, but that might not be a Bad Thing. Generics is a (frequently discussed) downside. It's definitely a tradeoff. For every container type we had, we had to make a copy of it or a typesafe wrapper around it for every possible instantiation of the container with different types. If you're used to coding by making flexible containers and so on, that's definitely much more brittle and challenging in Go. Go peeps will tell you that's part of a tradeoff they're willing to take.