8 ms·

A 100x speedup with unsafe Python

318 points by ingve 3y ago

idkdotcom 3y ago
Safety is one of Python's greatest advantages over C or C++. Why would anyone use unsafe Python when P0ython doesn't have other features such as type safety and all the debugging tools that have been built for C and C++ over time is beyond me. Python is a great language, but just as the generation of kids who got out of computer science programs in the 2000s were clueless about anything that wasn't Java, it seems this generation is clueless about anything that isn't Python. There was life before Python and there will be life after Python!
- jandrese 3y ago
  Seems to me if you could do most of the work in Python and then just make the critical loop unsafe and 100x faster then that would certainly have some appeal.
  - gibolt 3y ago
    Plenty of people would gladly not have to learn another language (especially C). You could also benefit from testing blocks of code with safety enabled to have more confidence when safety is removed.
    - KeplerBoy 3y ago
      It was explained pretty well in the blog: Installing the OpenCV python package is easy, fast and painless as long as you're happy with the default binary. Building OpenCV from source to get it into a C/C++ program can quickly turn into a multi-hour adventure, if you're not familiar with building sizeable C++ projects.
      - idkdotcom 3y ago
        [dead]
    - TylerE 3y ago
      Or if you’re going to learn another language might as well learn nim. Keep most of the python syntax, ditch the performance and the packaging insanity.
  - idkdotcom 3y ago
    [dead]
- eru 3y ago
  > Why would anyone use unsafe Python when Python doesn't have other features such as type safety and all the debugging tools that have been built for C and C++ over time is beyond me. C and C++ 'type safety' is barely there (compared to more sane languages like OCaml or Rust or even Java etc). As to why would anyone do that? The question is 'why not?' It's fun.
  - idkdotcom 3y ago
    [dead]
- gibolt 3y ago
  "Why would anyone use" -> "when" usage generally means lots of use cases are being ignored / swept under the rug. >1 billion people exist. Each has a unique opinion / viewpoint.
- bongodongobob 3y ago
  I don't use Python because of type safety, I use it because it's easy to write. I couldn't give a single fuck about some missed pointers that weren't cleaned up in the 2.5 seconds my program ran. I'm not deploying public code. I tend to prototype in Python and then just rewrite the whole thing in C with classes if I need the 10000x speedup.
  - SunlitCat 3y ago
    That easy to write part is something, I am not so sure about. It took me ages to understand why my first try at writing a blender plugin didn't work. It was because of white spaces not lining up. I was like, really? Using white spaces as a way to denote the body of a function? Really?
    - nottorp 3y ago
      I'm curious... what rock have you been living under in the past 33 years since python was launched?
      - SunlitCat 3y ago
        Let's see, I was living under the: - Amiga E - Assembler (on Amiga) - perl - tcl/tk - javascript - vbscript - java - vba - c - c++ - python rock. Granted at some of those rocks, I just took a quick peek (more like a glance), but a rock with whitespaces being important was, at that moment, new to me!
        nottorp 3y ago
        Oh cmon, I can still write some z80 assembly from memory and remember the zx spectrum memory layout somewhat, but I check new programming languages now and then :)
        SunlitCat 3y ago
        :) I just meant that for someone coming from a language(s) where different kinds of delimiters are used to denote function bodies, a language that puts such a huge emphasis on whitespace was kinda throwing me off to get an error message slapped at me even though I was replicating the example letter by letter (although, obviously, I missed the importance of whitespace).
- omoikane 3y ago
  The "unsafe" in the title appears to be used in the sense of "exposing memory layout details", but not in the sense of "direct unbounded access to memory". It's probably not the memory safety issue you are thinking of.
  - yosefk 3y ago
    You can absolutely get direct unbounded access to memory with ctypes, with all the bugs that come from this. I just think/hope the code I show in TFA happens to have no such bugs.
- SunlitCat 3y ago
  Thing about academia (like computer science and the like) is, you need a programming language you can get across easily, quickly and can get people to produce satisfying results in no time. Back then it might have been java, then python, till the next language comes around. The java thing was so apparent, that when you looked at c++ code, you were able to spot the former java user pretty easily. :) About python, sometime ago, I was looking into arudino programming in c and found someone offering a library (I think) in c (or c++, can't remember). What i remember is, that this person stopped working on it, because they got many requests and questions regarding to be able to use their library in python.
  - pjmlp 3y ago
    > The java thing was so apparent, that when you looked at c++ code, you were able to spot the former java user pretty easily. :) Well, Java was designed based on C++ GUI development patterns typical in the 10 years of C++ GUI frameworks that predated Java. Yet, people keep using these kind of remarks.
- kragen 3y ago
  while python is eminently criticable, yosef kreinin has designed his own cpu instruction set and gotten it fabbed in a shipping hardware product, and is the author of the c++fqa, which is by far the best-informed criticism of c++; criticizing him with 'this generation is clueless about anything that isn't Python' seems off the mark
intelVISA 3y ago
Isn't all Python, by design, unsafe?
- eru 3y ago
  Is this supposed to be a joke? Have a look at the linked article to see what meaning of 'unsafe' the author has in mind.
  - intelVISA 3y ago
    Safety is catching (more) errors ahead of time ...for which Python is grossly unsuitable imo. Fun lang though, just not one that ever comes to mind when I hear 'safe'.
    - eru 3y ago
      That's one definition of safety. But it's not the one the author uses in this case. The generic snide about Python that has nothing to do with the article ain't all that informative. You could make the same remark you just made about all Python being unsafe about Rust: 'Isn't all Rust, by design, unsafe?'. And compared to eg Agda or Idris that would be true, but it wouldn't be a very useful nor interesting comment when talking about specifically 'unsafe' Rust vs normal Rust.
      - intelVISA 3y ago
        I would agree, Rust is not safe. We need to encourage more formal rigor in our craft and avoid misconceptions like 'safe Rust' or 'safe Python'. Thus my original comment :P
        eru 3y ago
        Eh, even Agda is only safe in the sense of 'does what you've proven it to do'. That doesn't mean that if you eg write a machine learning library in Agda, your AI won't come and turn us all into paperclips. So it doesn't really make sense to pretend there's some single meaning of 'safe' vs 'unsafe' that's appropriate in all contexts.
comex 3y ago
Do you actually need ctypes here, or can you just use np.reshape? That would cut out the unsafety.
- Retr0id 3y ago
  You can exploit memory unsafety in cpython using only built-in methods, no imports needed at all: https://github.com/DavidBuchanan314/unsafe-python/ https://github.com/DavidBuchanan314/unsafe-python/
  - yosefk 3y ago
    You... probably shouldn't, but, very interesting stuff in that link! I didn't know CPython had no bounds checks in the load_const bytecode op
- im3w1l 3y ago
  I think what you want is a combination of swapaxes and flip (reshape does not turn rows into columns, it turns m input rows into n output rows), but yeah. Actually let me include a little figure original reshape swapaxes abc ab ad def cd be ef cf
- yosefk 3y ago
  I think you'll have a problem with BGR data absent ctypes, since you have a numpy array with the base pointing at the first red channel value, and you want an array with the base pointing 2 bytes before the first red channel value. This is almost definitely "unsafe"?.. or does numpy have functions knowing that due to the negative z stride, the red value before the blue value is within the bounds of the original array? I somehow doubt it though it would be very impressive. And I think the last A value is hopelessly out of reach absent ctypes since a safe API has no information that this last value is within the array bounds; more strictly speaking all the alpha values are out of bounds according to the shape and strides of the BGRA array.
qaq 3y ago
Well with mojo it looks like you will have both safety and performance
- grandma_tea 3y ago
  Closed source and not relevant.
  - ianbutler 3y ago
    Mojo was open sourced months ago iirc https://github.com/modularml/mojo https://github.com/modularml/mojo
    - viraptor 3y ago
      Those are examples and docs. Mojo is still closed.
      - ianbutler 3y ago
        Their standard lib is in there, seems more open to me Theres the announcement where they open sourced the modules in their standard lib: https://www.modular.com/blog/the-next-big-step-in-mojo-open-source https://www.modular.com/blog/the-next-big-step-in-mojo-open-... There's the standard lib: https://github.com/modularml/mojo/blob/nightly/stdlib/README.md https://github.com/modularml/mojo/blob/nightly/stdlib/README...
        viraptor 3y ago
        Yeah, that's some minimal progress, but really not that interesting. (As in, it's cool that it exists, but given we've got Python stdlib and numpy already open, it's not really new/exciting) And doesn't allow you to port to platforms they don't care enough about.
        qaq 3y ago
        It might be not interesting to you. Having a lot of Rust features with much smarter compiler and Python syntax is pretty interesting to me.
Animats 3y ago
Oh, array striding. This is a classic bikeshedding issue. When Go and Rust were first being designed, I brought up support for multidimensional arrays. For both cases, that became lost in discussions over what features arrays should have. Subarrays, like slices but multidimensional? That's the main use case for striding. Flattening in more than one dimension? And some people want sparse arrays. Stuff like that. So the problem gets pushed off onto collection classes, there's no standard, and everybody using such arrays spends time doing format conversion. This is why FORTRAN and Matlab still have strong positions in number-crunching.
- lifthrasiir 3y ago
  But format conversion is inevitable because C and FORTRAN has a different axis ordering anyway, isn't it?
  - blt 3y ago
    Not really, you can always change the indexing to account for it. For example, the GEMM matrix multiplication subroutines from BLAS can transpose their arguments [1]. So if you have A (m x n) and B (n x p) stored row-major, but you want to use a column-major BLAS to compute A*B, you can instead tell BLAS that A is n x m, B is p x n, and you want to compute A' * B'. As the article mentions, NumPy can handle both and do all the bookkeeping. So can Eigen in C++. [1] https://www.math.utah.edu/software/lapack/lapack-blas/dgemm.html https://www.math.utah.edu/software/lapack/lapack-blas/dgemm....
    - lifthrasiir 3y ago
      That's conceptually still a format conversion, though the actual conversion might not happen. Users have to track which format is being used for which matrix and I believe that was what the GP was originally complaining for.
      - yosefk 3y ago
        Agreed, realistically you're going to either convert the data or give up on using the function rather than reimplementing it to support another data layout efficiently; while converting the code instead of the data, so to speak, is more efficient from the machine resource use POV, it can be very rough on human resources. (The article is basically about not having to give up in some cases where you can tweak the input parameters and make things work without rewriting the function; but it's not always possible)
gugagore 3y ago
Technically, image resizing does in general care about the color channel ordering, because color spaces are in general not linear. https://www.alanzucconi.com/2016/01/06/colour-interpolation/ https://www.alanzucconi.com/2016/01/06/colour-interpolation/
- quietbritishjim 3y ago
  That article fails to touch on the fundamental issue with its title "The Secrets of Colour Interpolation": RGB values are a nonlinear function of the light emitted (because we are better at distinguishing dark colours, so it's better to allow representing more of those), so to interpolate properly you need to invert that function, interpolate, then reapply. The difference that makes to colour gradients is really striking.
  - topherclay 3y ago
    Do you mean converting the RGB value to LAB values in the CIELAB color space and doing the interpellation there? Is there a better way to do it?
    - CarVac 3y ago
      No, you just need to linearize the brightness.
      - drjasonharrison 3y ago
        Typically this is done by using a look-up table to convert the 8-bit gamma encoded intensities to 10-bit (or more) linear intensities. You can use the same look-up table for R, G, and B. Alpha should be linear.
- yosefk 3y ago
  While this is a valid point, cv2.resize doesn't actually implement color conversion in a way addressing this issue; at least in my testing I get identical results whether I interpret the data as RGB or BGR. So if you want to use cv2.resize, AFAIK you can count on it not caring which channel is which. And if you need fast resizing, you're quite likely to settle for the straightforward interpolation implemented by cv2.resize.
ggm 3y ago
Why does numpy do column order data? Is it because in much of the maths domain it turns out you manipulate columns more often than rows? (not a mathematician but I could believe if the columns represent "dimensions" or "qualities" of some dataset, and you want to apply a function to a given dimension, having data in column-natural order makes that faster.) Obviously you want to believe naievely there is no difference to the X and Y in the X,Y plane, but machines are not naieve and sometimes there IS a difference to doing things to the set of verticals, and the set of horizontals.
- sdeer 3y ago
  Probably because Fortran stores matrices and other multidimensional arrays in column order. Traditionally most numerical computation software was written in Fortran and numpy calls those under the hood. Storing in row order would have meant copying the data to column major order and back for any call to Fortran.
  - Thrymr 3y ago
    NumPy can _store_ arrays in either row-major (C-style) or column-major (Fortran-style) order. Row-major is the default. Index order is always row, column.
- thayne 3y ago
  Numpy was originally designed for mathematicians and scientists, and in those domains convention often lines up better with column major order. For example, vectors are a single column, and the column index comes before the row index in many notations. So using column major order meant familar formulas and algorithms (including translating from fortran code which is column major, or matlab for that matter) are easy to translate to numpy code. Also, numpy is built on some fortran libraries, like BLAS and LAPACK that assume a column major order. If it took row major input, it would need to transpose matrices in some cases change the order, or rewrite those libraries to use row major form.
  - kolbusa 3y ago
    Copying is usually not necessary. Often times you can swap data and/or shape arguments and get a transposed result out. While it is true that Fortran BLAS only supports col-major, CBLAS supports both row- and col-major. Internally, all the libraries I worked on use col-major, but that is just a convention.
ruined 3y ago
oh THIS is why image byte order and dimensions are so confusing every time i fuck with opencv and pygame well, half of why. for some reason i keep doing everything directly on /dev/fb0
- TeMPOraL 3y ago
  Image byte order, axis directions, coordinate system handedness when in 3D... after enough trying, you eventually figure out the order of looping at any given stage of your program, and then it Just Works, and then you never touch it again.
Too 3y ago
Clickbait title. The speedup only applies to one particular interaction between SDL and numpy.
C4stor 3y ago
All of this seems unnecessary, and easily replaced in the provided benchmark by : i2 = np.ascontiguousarray(pg.surfarray.pixels3d(isurf)) Which does the 100x speedup too and is a "safe" way to adjust memory access to numpy strides. Whether the output is correct or not is left as an exercise to the author, since the provided benchmark only use np.zeros() it's kind of hard to verify
- yosefk 3y ago
  I just measured this with the np.ascontiguousarray call (inside the loop of course, if you do it outside the loop when i2 is assigned to, then ofc you don't measure the overhead of np.ascontiguousarray) and the runtime is about the same as without this call, which I would expect given everything TFA explains. So you still have the 100x slowdown. TFA links to a GitHub repo with code resizing non-zero images, which you could use both to easily benchmark your suggested change, and to check whether the output is still correct.
londons_explore 3y ago
> What can we do about this? We can't change the layout of pygame Surface data. And we seriously don't want to copy the C++ code of cv2.resize, with its various platform-specific optimizations, Or... you could have sent a ~25 line pull request to opencv to fix this performance problem not just for you, but for thousands of other developers and millions of users. I think your fix would go here: https://github.com/opencv/opencv/blob/ba65d2eb0d83e6c9567a2534f8994cd4484a1635/modules/imgproc/src/resize.cpp#L3410 https://github.com/opencv/opencv/blob/ba65d2eb0d83e6c9567a25... And you could have tracked down that slow code easily by running your slow code in gdb, hitting Ctrl+C while it's doing the slow thing, and then "bt" to get a stack trace of what it's doing and you'd see it constructing this new image copy because the format isn't correct.
- yosefk 3y ago
  25 line pull request doing what? Supporting this format efficiently is probably more LOC and unlikely to get merged as few need this and it complicates the code. Doing the thing I did in Python inside OpenCV C++ code instead (reinterpreting the data) is not really possible since you have less knowledge about the input data at this point (like, you're going to assume it's an RGBA image and access past the area defined as the allocated array data range for the last A value? The user gives a 3D array and you blithely work on the 4th dimension?) And what about the other examples I measured, including pure numpy code? More patches to submit?
  - londons_explore 3y ago
    inside the python opencv bindings? Or just special case resize, since there the channel order doesn't matter. If you interpret R as A you'll still get the right answer.
    - yosefk 3y ago
      The pygame API gives you a 3D RGB array and a separate 2D alpha array, which happen to reference the same RGBA/BGRA memory. Are you suggesting that OpenCV Python bindings (in most of their functions, and likewise other libraries including numpy operators themselves) should have code assuming that a WxHx3 RGB array really references the memory of a WxHx4 RGBA array?.. If you really want a solution which is a patch rather than a function in your own code, I guess the way to go would be to persuade the maintainers of pygame and separately, the maintainers of its fork, pygame-ce, to add a pixels4d_maybe_rgba_and_maybe_bgra() function returning a WxHx4 array. Whether they would want to add something like that I don't know. But in any case, I think that since closed source projects, open source projects, and code by sister teams exist that will not accept patches, or will not accept them in a timely manner, it's interesting what you can do without changing someone else's code.
antirez 3y ago
MicroPython allows to do even more than that, and is fantastic and I wish it was part of standard Python as well. Using a decorator you can write functions in Viper, a subset of Python that has pointers to bytearrays() and only fast integer types that have a behavior similar to C (fixed size, they wrap around and so forth). It's a very nice way to speedup things 10 or 100x without resorting to C extensions.
- yosefk 3y ago
  I dunno if it's "more than that" since they're not directly comparable? What you describe sounds more like numba or Cython than what TFA describes which is a different use case?..
  - antirez 3y ago
    Indeed, I don't mean they are directly comparable, but it's a great incarnation of the "Unsafe Python" idea, because it does not go too far with the subset of Python that you can use, like introducing a complete different syntax or alike. There are strict rules, but if you follow them you still kinda write Python that goes super fast.
    - 3abiton 3y ago
      Iam curious about the quantification of "super fast", do you have some pointers?
      - antirez 3y ago
        Viper is normally half the speed of using the MicroPython decorator to directly write assembly.
Joker_vD 3y ago
Yep, welcome to the world of RGBA and ARGB storage and memory representation formats, with little- and big-endianness thrown in for the mix, it's all very bloody annoying for very little gain.
sylware 3y ago
I wonder how it would perform with the latest openstreetmap tile renderer which was released not long ago.
jofer 3y ago
I'm confused on the need for cytpes/etc here. You can directly modify somearr.strides and somearr.shape. And if you need to set them both together, then there's numpy.lib.stride_tricks.as_strided. Unless I'm missing something, you can do the same assignment that ctypes is used for here directly in numpy. But I might also be missing a lot - I'm still pre-coffee. On a different note, I'm surprised no one has mentioned Cython in this context. This is distinct from, but broadly related to things like @cython.boundscheck(False). Cython is _really_ handy, and it's a shame that it's kinda fallen out of favor in some ways lately.
- yosefk 3y ago
  You need to modify base pointer in this example, specifically to point 2 bytes before the current base pointer (moving back from the first red pixel value to the first blue value.) I don't think you can do it without ctypes, maybe I'm wrong. What does Cython do better than numbs except static compilation? Honest q, I know little about both
  - jofer 3y ago
    You actually can do the "offset by 2 bytes back" with a reshape + indexing + reshape back. But I suspect I'm still missing something. I need to read things in more depth and try it out locally. By "numbs" here, I'm assuming you mean numba. If that's not correct, I apologize in advance! There are several things where Cython is a better fit than numba (and to be fair, several cases where the opposite is true). There are two that stick out in my mind: First off, the optimizations for cython are explicit and a matter of how you implement the actual code. It's a separate language (technically a superset of python). There's no "magic". That's often a significant advantage in and of itself. Personally, I often find larger speedups with Cython, but then again, I'm more familiar with it, and understand what settings to turn on/off in different situations. Numba is much more of a black box given than it's a JIT compiler. With that said, it can also do things that Cython can't _because_ it's a JIT compiler. If you want to run python code as-is, then yeah, numba is the better choice. You won't see a speedup at all with Cython unless you change the way you've written things. The second key thing is one that's likely the most overlooked. Cython is arguably the best way to write a python wrapper around C code where you want to expose things as numpy arrays. It takes a _huge_ amount of the boilerplate out. That alone makes it worth learning, i.m.o.

318 points by ingve 3y ago

Safety is one of Python's greatest advantages over C or C++. Why would anyone use unsafe Python when P0ython doesn't have other features such as type safety and all the debugging tools that have been built for C and C++ over time is beyond me. Python is a great language, but just as the generation of kids who got out of computer science programs in the 2000s were clueless about anything that wasn't Java, it seems this generation is clueless about anything that isn't Python. There was life before Python and there will be life after Python!

jandrese 3y ago
Seems to me if you could do most of the work in Python and then just make the critical loop unsafe and 100x faster then that would certainly have some appeal.
- gibolt 3y ago
  Plenty of people would gladly not have to learn another language (especially C). You could also benefit from testing blocks of code with safety enabled to have more confidence when safety is removed.
  - KeplerBoy 3y ago
    It was explained pretty well in the blog: Installing the OpenCV python package is easy, fast and painless as long as you're happy with the default binary. Building OpenCV from source to get it into a C/C++ program can quickly turn into a multi-hour adventure, if you're not familiar with building sizeable C++ projects.
    - idkdotcom 3y ago
      [dead]
  - TylerE 3y ago
    Or if you’re going to learn another language might as well learn nim. Keep most of the python syntax, ditch the performance and the packaging insanity.
- idkdotcom 3y ago
  [dead]
eru 3y ago
> Why would anyone use unsafe Python when Python doesn't have other features such as type safety and all the debugging tools that have been built for C and C++ over time is beyond me. C and C++ 'type safety' is barely there (compared to more sane languages like OCaml or Rust or even Java etc). As to why would anyone do that? The question is 'why not?' It's fun.
- idkdotcom 3y ago
  [dead]
gibolt 3y ago
"Why would anyone use" -> "when" usage generally means lots of use cases are being ignored / swept under the rug. >1 billion people exist. Each has a unique opinion / viewpoint.
bongodongobob 3y ago
I don't use Python because of type safety, I use it because it's easy to write. I couldn't give a single fuck about some missed pointers that weren't cleaned up in the 2.5 seconds my program ran. I'm not deploying public code. I tend to prototype in Python and then just rewrite the whole thing in C with classes if I need the 10000x speedup.
- SunlitCat 3y ago
  That easy to write part is something, I am not so sure about. It took me ages to understand why my first try at writing a blender plugin didn't work. It was because of white spaces not lining up. I was like, really? Using white spaces as a way to denote the body of a function? Really?
  - nottorp 3y ago
    I'm curious... what rock have you been living under in the past 33 years since python was launched?
    - SunlitCat 3y ago
      Let's see, I was living under the: - Amiga E - Assembler (on Amiga) - perl - tcl/tk - javascript - vbscript - java - vba - c - c++ - python rock. Granted at some of those rocks, I just took a quick peek (more like a glance), but a rock with whitespaces being important was, at that moment, new to me!
      - nottorp 3y ago
        Oh cmon, I can still write some z80 assembly from memory and remember the zx spectrum memory layout somewhat, but I check new programming languages now and then :)
        SunlitCat 3y ago
        :) I just meant that for someone coming from a language(s) where different kinds of delimiters are used to denote function bodies, a language that puts such a huge emphasis on whitespace was kinda throwing me off to get an error message slapped at me even though I was replicating the example letter by letter (although, obviously, I missed the importance of whitespace).
omoikane 3y ago
The "unsafe" in the title appears to be used in the sense of "exposing memory layout details", but not in the sense of "direct unbounded access to memory". It's probably not the memory safety issue you are thinking of.
- yosefk 3y ago
  You can absolutely get direct unbounded access to memory with ctypes, with all the bugs that come from this. I just think/hope the code I show in TFA happens to have no such bugs.
SunlitCat 3y ago
Thing about academia (like computer science and the like) is, you need a programming language you can get across easily, quickly and can get people to produce satisfying results in no time. Back then it might have been java, then python, till the next language comes around. The java thing was so apparent, that when you looked at c++ code, you were able to spot the former java user pretty easily. :) About python, sometime ago, I was looking into arudino programming in c and found someone offering a library (I think) in c (or c++, can't remember). What i remember is, that this person stopped working on it, because they got many requests and questions regarding to be able to use their library in python.
- pjmlp 3y ago
  > The java thing was so apparent, that when you looked at c++ code, you were able to spot the former java user pretty easily. :) Well, Java was designed based on C++ GUI development patterns typical in the 10 years of C++ GUI frameworks that predated Java. Yet, people keep using these kind of remarks.
kragen 3y ago
while python is eminently criticable, yosef kreinin has designed his own cpu instruction set and gotten it fabbed in a shipping hardware product, and is the author of the c++fqa, which is by far the best-informed criticism of c++; criticizing him with 'this generation is clueless about anything that isn't Python' seems off the mark

intelVISA 3y ago

Isn't all Python, by design, unsafe?

eru 3y ago
Is this supposed to be a joke? Have a look at the linked article to see what meaning of 'unsafe' the author has in mind.
- intelVISA 3y ago
  Safety is catching (more) errors ahead of time ...for which Python is grossly unsuitable imo. Fun lang though, just not one that ever comes to mind when I hear 'safe'.
  - eru 3y ago
    That's one definition of safety. But it's not the one the author uses in this case. The generic snide about Python that has nothing to do with the article ain't all that informative. You could make the same remark you just made about all Python being unsafe about Rust: 'Isn't all Rust, by design, unsafe?'. And compared to eg Agda or Idris that would be true, but it wouldn't be a very useful nor interesting comment when talking about specifically 'unsafe' Rust vs normal Rust.
    - intelVISA 3y ago
      I would agree, Rust is not safe. We need to encourage more formal rigor in our craft and avoid misconceptions like 'safe Rust' or 'safe Python'. Thus my original comment :P
      - eru 3y ago
        Eh, even Agda is only safe in the sense of 'does what you've proven it to do'. That doesn't mean that if you eg write a machine learning library in Agda, your AI won't come and turn us all into paperclips. So it doesn't really make sense to pretend there's some single meaning of 'safe' vs 'unsafe' that's appropriate in all contexts.

comex 3y ago

Do you actually need ctypes here, or can you just use np.reshape? That would cut out the unsafety.

Retr0id 3y ago
You can exploit memory unsafety in cpython using only built-in methods, no imports needed at all: https://github.com/DavidBuchanan314/unsafe-python/ https://github.com/DavidBuchanan314/unsafe-python/
- yosefk 3y ago
  You... probably shouldn't, but, very interesting stuff in that link! I didn't know CPython had no bounds checks in the load_const bytecode op
im3w1l 3y ago
I think what you want is a combination of swapaxes and flip (reshape does not turn rows into columns, it turns m input rows into n output rows), but yeah. Actually let me include a little figure original reshape swapaxes abc ab ad def cd be ef cf
yosefk 3y ago
I think you'll have a problem with BGR data absent ctypes, since you have a numpy array with the base pointing at the first red channel value, and you want an array with the base pointing 2 bytes before the first red channel value. This is almost definitely "unsafe"?.. or does numpy have functions knowing that due to the negative z stride, the red value before the blue value is within the bounds of the original array? I somehow doubt it though it would be very impressive. And I think the last A value is hopelessly out of reach absent ctypes since a safe API has no information that this last value is within the array bounds; more strictly speaking all the alpha values are out of bounds according to the shape and strides of the BGRA array.

qaq 3y ago

Well with mojo it looks like you will have both safety and performance

grandma_tea 3y ago
Closed source and not relevant.
- ianbutler 3y ago
  Mojo was open sourced months ago iirc https://github.com/modularml/mojo https://github.com/modularml/mojo
  - viraptor 3y ago
    Those are examples and docs. Mojo is still closed.
    - ianbutler 3y ago
      Their standard lib is in there, seems more open to me Theres the announcement where they open sourced the modules in their standard lib: https://www.modular.com/blog/the-next-big-step-in-mojo-open-source https://www.modular.com/blog/the-next-big-step-in-mojo-open-... There's the standard lib: https://github.com/modularml/mojo/blob/nightly/stdlib/README.md https://github.com/modularml/mojo/blob/nightly/stdlib/README...
      - viraptor 3y ago
        Yeah, that's some minimal progress, but really not that interesting. (As in, it's cool that it exists, but given we've got Python stdlib and numpy already open, it's not really new/exciting) And doesn't allow you to port to platforms they don't care enough about.
        qaq 3y ago
        It might be not interesting to you. Having a lot of Rust features with much smarter compiler and Python syntax is pretty interesting to me.

Animats 3y ago

Oh, array striding. This is a classic bikeshedding issue. When Go and Rust were first being designed, I brought up support for multidimensional arrays. For both cases, that became lost in discussions over what features arrays should have. Subarrays, like slices but multidimensional? That's the main use case for striding. Flattening in more than one dimension? And some people want sparse arrays. Stuff like that. So the problem gets pushed off onto collection classes, there's no standard, and everybody using such arrays spends time doing format conversion. This is why FORTRAN and Matlab still have strong positions in number-crunching.

lifthrasiir 3y ago
But format conversion is inevitable because C and FORTRAN has a different axis ordering anyway, isn't it?
- blt 3y ago
  Not really, you can always change the indexing to account for it. For example, the GEMM matrix multiplication subroutines from BLAS can transpose their arguments [1]. So if you have A (m x n) and B (n x p) stored row-major, but you want to use a column-major BLAS to compute A*B, you can instead tell BLAS that A is n x m, B is p x n, and you want to compute A' * B'. As the article mentions, NumPy can handle both and do all the bookkeeping. So can Eigen in C++. [1] https://www.math.utah.edu/software/lapack/lapack-blas/dgemm.html https://www.math.utah.edu/software/lapack/lapack-blas/dgemm....
  - lifthrasiir 3y ago
    That's conceptually still a format conversion, though the actual conversion might not happen. Users have to track which format is being used for which matrix and I believe that was what the GP was originally complaining for.
    - yosefk 3y ago
      Agreed, realistically you're going to either convert the data or give up on using the function rather than reimplementing it to support another data layout efficiently; while converting the code instead of the data, so to speak, is more efficient from the machine resource use POV, it can be very rough on human resources. (The article is basically about not having to give up in some cases where you can tweak the input parameters and make things work without rewriting the function; but it's not always possible)

gugagore 3y ago

Technically, image resizing does in general care about the color channel ordering, because color spaces are in general not linear. https://www.alanzucconi.com/2016/01/06/colour-interpolation/ https://www.alanzucconi.com/2016/01/06/colour-interpolation/

quietbritishjim 3y ago
That article fails to touch on the fundamental issue with its title "The Secrets of Colour Interpolation": RGB values are a nonlinear function of the light emitted (because we are better at distinguishing dark colours, so it's better to allow representing more of those), so to interpolate properly you need to invert that function, interpolate, then reapply. The difference that makes to colour gradients is really striking.
- topherclay 3y ago
  Do you mean converting the RGB value to LAB values in the CIELAB color space and doing the interpellation there? Is there a better way to do it?
  - CarVac 3y ago
    No, you just need to linearize the brightness.
    - drjasonharrison 3y ago
      Typically this is done by using a look-up table to convert the 8-bit gamma encoded intensities to 10-bit (or more) linear intensities. You can use the same look-up table for R, G, and B. Alpha should be linear.
yosefk 3y ago
While this is a valid point, cv2.resize doesn't actually implement color conversion in a way addressing this issue; at least in my testing I get identical results whether I interpret the data as RGB or BGR. So if you want to use cv2.resize, AFAIK you can count on it not caring which channel is which. And if you need fast resizing, you're quite likely to settle for the straightforward interpolation implemented by cv2.resize.

ggm 3y ago

Why does numpy do column order data? Is it because in much of the maths domain it turns out you manipulate columns more often than rows? (not a mathematician but I could believe if the columns represent "dimensions" or "qualities" of some dataset, and you want to apply a function to a given dimension, having data in column-natural order makes that faster.) Obviously you want to believe naievely there is no difference to the X and Y in the X,Y plane, but machines are not naieve and sometimes there IS a difference to doing things to the set of verticals, and the set of horizontals.

sdeer 3y ago
Probably because Fortran stores matrices and other multidimensional arrays in column order. Traditionally most numerical computation software was written in Fortran and numpy calls those under the hood. Storing in row order would have meant copying the data to column major order and back for any call to Fortran.
- Thrymr 3y ago
  NumPy can _store_ arrays in either row-major (C-style) or column-major (Fortran-style) order. Row-major is the default. Index order is always row, column.
thayne 3y ago
Numpy was originally designed for mathematicians and scientists, and in those domains convention often lines up better with column major order. For example, vectors are a single column, and the column index comes before the row index in many notations. So using column major order meant familar formulas and algorithms (including translating from fortran code which is column major, or matlab for that matter) are easy to translate to numpy code. Also, numpy is built on some fortran libraries, like BLAS and LAPACK that assume a column major order. If it took row major input, it would need to transpose matrices in some cases change the order, or rewrite those libraries to use row major form.
- kolbusa 3y ago
  Copying is usually not necessary. Often times you can swap data and/or shape arguments and get a transposed result out. While it is true that Fortran BLAS only supports col-major, CBLAS supports both row- and col-major. Internally, all the libraries I worked on use col-major, but that is just a convention.

ruined 3y ago

oh THIS is why image byte order and dimensions are so confusing every time i fuck with opencv and pygame well, half of why. for some reason i keep doing everything directly on /dev/fb0

TeMPOraL 3y ago
Image byte order, axis directions, coordinate system handedness when in 3D... after enough trying, you eventually figure out the order of looping at any given stage of your program, and then it Just Works, and then you never touch it again.

Too 3y ago

Clickbait title. The speedup only applies to one particular interaction between SDL and numpy.

C4stor 3y ago

All of this seems unnecessary, and easily replaced in the provided benchmark by : i2 = np.ascontiguousarray(pg.surfarray.pixels3d(isurf)) Which does the 100x speedup too and is a "safe" way to adjust memory access to numpy strides. Whether the output is correct or not is left as an exercise to the author, since the provided benchmark only use np.zeros() it's kind of hard to verify

yosefk 3y ago
I just measured this with the np.ascontiguousarray call (inside the loop of course, if you do it outside the loop when i2 is assigned to, then ofc you don't measure the overhead of np.ascontiguousarray) and the runtime is about the same as without this call, which I would expect given everything TFA explains. So you still have the 100x slowdown. TFA links to a GitHub repo with code resizing non-zero images, which you could use both to easily benchmark your suggested change, and to check whether the output is still correct.

londons_explore 3y ago

> What can we do about this? We can't change the layout of pygame Surface data. And we seriously don't want to copy the C++ code of cv2.resize, with its various platform-specific optimizations, Or... you could have sent a ~25 line pull request to opencv to fix this performance problem not just for you, but for thousands of other developers and millions of users. I think your fix would go here: https://github.com/opencv/opencv/blob/ba65d2eb0d83e6c9567a2534f8994cd4484a1635/modules/imgproc/src/resize.cpp#L3410 https://github.com/opencv/opencv/blob/ba65d2eb0d83e6c9567a25... And you could have tracked down that slow code easily by running your slow code in gdb, hitting Ctrl+C while it's doing the slow thing, and then "bt" to get a stack trace of what it's doing and you'd see it constructing this new image copy because the format isn't correct.

yosefk 3y ago
25 line pull request doing what? Supporting this format efficiently is probably more LOC and unlikely to get merged as few need this and it complicates the code. Doing the thing I did in Python inside OpenCV C++ code instead (reinterpreting the data) is not really possible since you have less knowledge about the input data at this point (like, you're going to assume it's an RGBA image and access past the area defined as the allocated array data range for the last A value? The user gives a 3D array and you blithely work on the 4th dimension?) And what about the other examples I measured, including pure numpy code? More patches to submit?
- londons_explore 3y ago
  inside the python opencv bindings? Or just special case resize, since there the channel order doesn't matter. If you interpret R as A you'll still get the right answer.
  - yosefk 3y ago
    The pygame API gives you a 3D RGB array and a separate 2D alpha array, which happen to reference the same RGBA/BGRA memory. Are you suggesting that OpenCV Python bindings (in most of their functions, and likewise other libraries including numpy operators themselves) should have code assuming that a WxHx3 RGB array really references the memory of a WxHx4 RGBA array?.. If you really want a solution which is a patch rather than a function in your own code, I guess the way to go would be to persuade the maintainers of pygame and separately, the maintainers of its fork, pygame-ce, to add a pixels4d_maybe_rgba_and_maybe_bgra() function returning a WxHx4 array. Whether they would want to add something like that I don't know. But in any case, I think that since closed source projects, open source projects, and code by sister teams exist that will not accept patches, or will not accept them in a timely manner, it's interesting what you can do without changing someone else's code.

antirez 3y ago

MicroPython allows to do even more than that, and is fantastic and I wish it was part of standard Python as well. Using a decorator you can write functions in Viper, a subset of Python that has pointers to bytearrays() and only fast integer types that have a behavior similar to C (fixed size, they wrap around and so forth). It's a very nice way to speedup things 10 or 100x without resorting to C extensions.

yosefk 3y ago
I dunno if it's "more than that" since they're not directly comparable? What you describe sounds more like numba or Cython than what TFA describes which is a different use case?..
- antirez 3y ago
  Indeed, I don't mean they are directly comparable, but it's a great incarnation of the "Unsafe Python" idea, because it does not go too far with the subset of Python that you can use, like introducing a complete different syntax or alike. There are strict rules, but if you follow them you still kinda write Python that goes super fast.
  - 3abiton 3y ago
    Iam curious about the quantification of "super fast", do you have some pointers?
    - antirez 3y ago
      Viper is normally half the speed of using the MicroPython decorator to directly write assembly.

Joker_vD 3y ago

Yep, welcome to the world of RGBA and ARGB storage and memory representation formats, with little- and big-endianness thrown in for the mix, it's all very bloody annoying for very little gain.

sylware 3y ago

I wonder how it would perform with the latest openstreetmap tile renderer which was released not long ago.

jofer 3y ago

I'm confused on the need for cytpes/etc here. You can directly modify somearr.strides and somearr.shape. And if you need to set them both together, then there's numpy.lib.stride_tricks.as_strided. Unless I'm missing something, you can do the same assignment that ctypes is used for here directly in numpy. But I might also be missing a lot - I'm still pre-coffee. On a different note, I'm surprised no one has mentioned Cython in this context. This is distinct from, but broadly related to things like @cython.boundscheck(False). Cython is _really_ handy, and it's a shame that it's kinda fallen out of favor in some ways lately.

yosefk 3y ago
You need to modify base pointer in this example, specifically to point 2 bytes before the current base pointer (moving back from the first red pixel value to the first blue value.) I don't think you can do it without ctypes, maybe I'm wrong. What does Cython do better than numbs except static compilation? Honest q, I know little about both
- jofer 3y ago
  You actually can do the "offset by 2 bytes back" with a reshape + indexing + reshape back. But I suspect I'm still missing something. I need to read things in more depth and try it out locally. By "numbs" here, I'm assuming you mean numba. If that's not correct, I apologize in advance! There are several things where Cython is a better fit than numba (and to be fair, several cases where the opposite is true). There are two that stick out in my mind: First off, the optimizations for cython are explicit and a matter of how you implement the actual code. It's a separate language (technically a superset of python). There's no "magic". That's often a significant advantage in and of itself. Personally, I often find larger speedups with Cython, but then again, I'm more familiar with it, and understand what settings to turn on/off in different situations. Numba is much more of a black box given than it's a JIT compiler. With that said, it can also do things that Cython can't _because_ it's a JIT compiler. If you want to run python code as-is, then yeah, numba is the better choice. You won't see a speedup at all with Cython unless you change the way you've written things. The second key thing is one that's likely the most overlooked. Cython is arguably the best way to write a python wrapper around C code where you want to expose things as numpy arrays. It takes a _huge_ amount of the boilerplate out. That alone makes it worth learning, i.m.o.