7 ms·

Everyone keeps saying "Use Python and write the slow parts in C"..... except that writing GOOD (secure, correct, safe) C is really hard. That's actually the mai

by NateDad 14y ago

Everyone keeps saying "Use Python and write the slow parts in C"..... except that writing GOOD (secure, correct, safe) C is really hard. That's actually the main reason why the Go authors came up with Go. Not to replace Python etc, but to replace C/C++. I'm kind of surprised that all the "Python: let's get stuff done" people are suggesting writing a whole bunch of code in C, one of the least "Getting stuff done" languages there is. For those who have asked what the productivity of Go is like vs. Python + C: Go is very productive. When newbies to Go say it's "verbose" they mean "I have to write out some 3 line loops that in other languages are one line". If that's the major cause of slowness in my development cycle, I'd be ecstatic. Go is also very easy to write "correct" code in... no buffer overflows, no memory leaks, no access of random memory, no unnecessary allocations, and a very clear and defined behavior from clear, concise code.

pekk 14y ago
Nobody said you have to write any C. If you are freaking out about performance, the option to write some parts in C is valuable.
gsnedders 14y ago
Given the original presenter is one of the major contributors to PyPy, I'm fairly certain that wouldn't be his first advice: I'd expect it'd be to try PyPy (and file a bug if it's slower than CPython!). Also note on the last slide: "Stop writing C extensions, use something like cffi". There's two (really) good reasons you should do this: firstly, your code is no longer tied tied to CPython, so using alternative Python implementations (like PyPy) becomes viable; secondly, you don't have to use C — you can trivially use anything that exports functions using the C ABI.
officemonkey 14y ago
>"I have to write out some 3 line loops that in other languages are one line" I think every vim, emacs and decent IDE user would say "I has a text editor. Your argument is invalid."
- stouset 14y ago
  On the flip side of this argument, code should be optimized for reading, not writing. Short, idiomatic code IMHO optimizes for reading. Code like array.map(&:to_s) parses instantaneously. Whereas the procedural equivalent strings = [] for (element in array) strings.push element.to_s end requires scanning through the loop to extract out the logic specific to this case. And it's easy to accidentally overlook something when mentally ignoring the "boilerplate". Not to make a mountain out of a molehill, and not commenting on Go specifically, but the extra "cost" of this style of code adds up when the language itself gets in the way of using high-level abstractions. Methods that are three meaningful lines long are inherently more understandable than equivalent methods that are twelve. Multiply this cost across an entire class, file, and/or project, and it can have a significant impact on one's ability to grok the expected behavior.
  - pacala 14y ago
    On the flip side, I had to scratch my head on what "&:" is supposed to mean. Or perhaps it's "&" ":". I know, I'll Google it </yeahright>. More seriously, "for ..." maps into "map", "foldl", "foldl'", "foldr" and perhaps other constructs. They do pretty much the same thing, but I don't necessarily see a real cognitive advantage of either style.
    - epochwolf 14y ago
      The language is ruby. array.map(&:to_s) eventually means array.map{|str| str.to_s } As I understand it, the & means convert a variable to a code block. If the object already is a callable object (proc or lambda) ruby doesn't need to do anything to pass it through. Symbols (:to_s) aren't callable so ruby calls the to_proc method on it. The rails framework addd this for Ruby 1.8. The code is below. # File activesupport/lib/active_support/core_ext/symbol.rb, line 10 def to_proc Proc.new { |*args| args.shift.__send__(self, *args) } end
    - Falling3 14y ago
      stouset is obviously not arguing that the ruby version with "&:" is immediately obvious to any programmer. The point is if you know ruby, you will understand that line a lot quicker than the second example.
MostAwesomeDude 14y ago
Please don't say that "all" of us endorse writing C. Many of us gave up on writing C a long time ago and want to see pure-Python solutions to things.
klrr 14y ago
Go is by far a replacement for C. C was made as a language for writing operating systems. C++ tried add more abstraction making it a better choice for more regular applications. Go was made for writing distributed systems.
hogu 14y ago
I've had pretty easy success moving my bottlenecks into Cython. With the development of Numba, https://github.com/numba/numba https://github.com/numba/numba, this is going to become even easier(I work for the company that produces Numba, but I do not work on Numba). I think one thing the scientific community does right, is that we keep our data in NumPy arrays. This makes it really easy to write extensions in other langauges because NumPy arrays are very transparent and easy to access in other languages (they're just chunks of memory with dtypes, and a shape)
- quantgsm 14y ago
  > I work for the company that produces Numba, but I do not work on Numba In experience Cython and C++ extensions are still the best way to speed up existing Python algorithms. Initially Numba showed great promise but after attempting to jit compile several simple hotspots ( cryptography and hashing functions primarily ) in my application I discovered lot of bugs and apparent numeric instability particularly in the trigonometric and bit functions. Have these issues been resolved? For instance for Diffie–Hellman calculations?
  - freyrs3 14y ago
    I also work for the company that produces Numba. We do have an issue tracker for Numba, if you're having problems then it might help to post your code. https://github.com/numba/numba/issues https://github.com/numba/numba/issues
  - travisoliphant 14y ago
    Right now Numba is still pre 1.0 and so there will be issues for specific cases. The best way to get them solved is to provide your test-case so that we can grow the test suite. Right now, I agree that Cython and pure C++ is still the best approach if you have a case which Numba does not yet quite solve. However, I expect this to change over the next 6-12 months as Numba is getting a lot of attention both inside of Continuum and externally at several industrial-strength locations. Our goal is that Numba can replace (almost all) Cython and raw C-extensions or even Go-extensions faster than other language approach will be able to build a run-time and ecosystem the equivalent of what is available with Python. The roadmap (which includes Go-like channels and go-routines) as well as compiled class-support is here: http://numba.pydata.org/numba-doc/dev/roadmap.html http://numba.pydata.org/numba-doc/dev/roadmap.html The project is open-source and contributions and contributors are welcome to join.

by NateDad 14y ago

pekk 14y ago

Nobody said you have to write any C. If you are freaking out about performance, the option to write some parts in C is valuable.

gsnedders 14y ago

Given the original presenter is one of the major contributors to PyPy, I'm fairly certain that wouldn't be his first advice: I'd expect it'd be to try PyPy (and file a bug if it's slower than CPython!). Also note on the last slide: "Stop writing C extensions, use something like cffi". There's two (really) good reasons you should do this: firstly, your code is no longer tied tied to CPython, so using alternative Python implementations (like PyPy) becomes viable; secondly, you don't have to use C — you can trivially use anything that exports functions using the C ABI.

officemonkey 14y ago

>"I have to write out some 3 line loops that in other languages are one line" I think every vim, emacs and decent IDE user would say "I has a text editor. Your argument is invalid."

stouset 14y ago
On the flip side of this argument, code should be optimized for reading, not writing. Short, idiomatic code IMHO optimizes for reading. Code like array.map(&:to_s) parses instantaneously. Whereas the procedural equivalent strings = [] for (element in array) strings.push element.to_s end requires scanning through the loop to extract out the logic specific to this case. And it's easy to accidentally overlook something when mentally ignoring the "boilerplate". Not to make a mountain out of a molehill, and not commenting on Go specifically, but the extra "cost" of this style of code adds up when the language itself gets in the way of using high-level abstractions. Methods that are three meaningful lines long are inherently more understandable than equivalent methods that are twelve. Multiply this cost across an entire class, file, and/or project, and it can have a significant impact on one's ability to grok the expected behavior.
- pacala 14y ago
  On the flip side, I had to scratch my head on what "&:" is supposed to mean. Or perhaps it's "&" ":". I know, I'll Google it </yeahright>. More seriously, "for ..." maps into "map", "foldl", "foldl'", "foldr" and perhaps other constructs. They do pretty much the same thing, but I don't necessarily see a real cognitive advantage of either style.
  - epochwolf 14y ago
    The language is ruby. array.map(&:to_s) eventually means array.map{|str| str.to_s } As I understand it, the & means convert a variable to a code block. If the object already is a callable object (proc or lambda) ruby doesn't need to do anything to pass it through. Symbols (:to_s) aren't callable so ruby calls the to_proc method on it. The rails framework addd this for Ruby 1.8. The code is below. # File activesupport/lib/active_support/core_ext/symbol.rb, line 10 def to_proc Proc.new { |*args| args.shift.__send__(self, *args) } end
  - Falling3 14y ago
    stouset is obviously not arguing that the ruby version with "&:" is immediately obvious to any programmer. The point is if you know ruby, you will understand that line a lot quicker than the second example.

MostAwesomeDude 14y ago

Please don't say that "all" of us endorse writing C. Many of us gave up on writing C a long time ago and want to see pure-Python solutions to things.

klrr 14y ago

Go is by far a replacement for C. C was made as a language for writing operating systems. C++ tried add more abstraction making it a better choice for more regular applications. Go was made for writing distributed systems.

hogu 14y ago

I've had pretty easy success moving my bottlenecks into Cython. With the development of Numba, https://github.com/numba/numba https://github.com/numba/numba, this is going to become even easier(I work for the company that produces Numba, but I do not work on Numba). I think one thing the scientific community does right, is that we keep our data in NumPy arrays. This makes it really easy to write extensions in other langauges because NumPy arrays are very transparent and easy to access in other languages (they're just chunks of memory with dtypes, and a shape)

quantgsm 14y ago
> I work for the company that produces Numba, but I do not work on Numba In experience Cython and C++ extensions are still the best way to speed up existing Python algorithms. Initially Numba showed great promise but after attempting to jit compile several simple hotspots ( cryptography and hashing functions primarily ) in my application I discovered lot of bugs and apparent numeric instability particularly in the trigonometric and bit functions. Have these issues been resolved? For instance for Diffie–Hellman calculations?
- freyrs3 14y ago
  I also work for the company that produces Numba. We do have an issue tracker for Numba, if you're having problems then it might help to post your code. https://github.com/numba/numba/issues https://github.com/numba/numba/issues
- travisoliphant 14y ago
  Right now Numba is still pre 1.0 and so there will be issues for specific cases. The best way to get them solved is to provide your test-case so that we can grow the test suite. Right now, I agree that Cython and pure C++ is still the best approach if you have a case which Numba does not yet quite solve. However, I expect this to change over the next 6-12 months as Numba is getting a lot of attention both inside of Continuum and externally at several industrial-strength locations. Our goal is that Numba can replace (almost all) Cython and raw C-extensions or even Go-extensions faster than other language approach will be able to build a run-time and ecosystem the equivalent of what is available with Python. The roadmap (which includes Go-like channels and go-routines) as well as compiled class-support is here: http://numba.pydata.org/numba-doc/dev/roadmap.html http://numba.pydata.org/numba-doc/dev/roadmap.html The project is open-source and contributions and contributors are welcome to join.