18 ms·

Writing Pythonic Rust

218 points by zbentley 5y ago

adamnemecek 5y ago
You probably don’t want to do that.
lalaithion 5y ago
> python expects you to share references to things. When you create a binding to a point in a contour, that binding refers to the same data as the original point, and modifying the binding modifies the collection. Honestly, as someone who has written way more Python than Rust, this seems like bad API design regardless of language. This screams "it's impossible to take a font, make two separate modifications to it, and then work with those separate modifications at the same time", because deepcopying objects is usually very difficult.
- Blikkentrekker 5y ago
  Why would deep copying be difficult? Even in Rust, the `Clone` trait produces a deep copy. Copying only up to a specific depth in a general way is what is difficult. I would argue that shallow copies, id est the depth being 1, are far more difficult to realize that copies of unbounded depth until a value o trait `Copy` is reached, id est a type that is purely encoded by the data it takes up on the stack and owns no further data and can thus be fully be `Clone`ed by copying it's stack bits.
  - kzrdude 5y ago
    > Even in Rust, the `Clone` trait produces a deep copy. Clone is not really a deep copy. I like the description that says it is "deep enough to give you an object of identical/independent ownership (of what you started with), but no deeper". Example: Rc<String> when cloned only gives you another handle to the Rc, the string data is not duplicated (for this reason we don't call it a real deep copy). You get a new rc handle that is on equal footing with the old handle. There is plenty of Rust types that consist of tree-shaped ownership relations with no frills - in these cases clone and deep copy are identical. Let's say for example a HashMap<i32, String>.
    - oconnor663 5y ago
      I think every "deep" copy ultimately has some limitation like this. Even in Python, an object can implement __deepcopy__() to prevent copy.deepcopy() from "copying too much", and things like file objects, classes, and modules get special treatment.
      - kzrdude 5y ago
        I found that in Rust, you don't have to think about the distinction so much - not like you have to in for example Python and C# (pervasive/transparent object reference languages). In Rust it's predictable and mostly obvious which data is shared ownership or shared, and which is not. We don't need to talk about deep vs shallow copy semantics much in Rust. :)
    - Blikkentrekker 5y ago
      You are correct; my original definition that only went till `copy` was encountered is only the most common case of your more general definition. The principles are still quite similar and it still seems easier to implement to me than shallow copies as a generic trait. I don't know how I would fit shallow copies into Rust's trait system.
      - kzrdude 5y ago
        Clone seems to be the shallowest kind copy permitted by the type and ownership system.
        Blikkentrekker 5y ago
        Which is why I find deep copies to be easier to implement than shallow. It is of course possible to express a shallower copy, but not in the same generic way as a universal trait, that I know of, in any case.
  - wk_end 5y ago
    It's not intrinsically difficult, but in Python it's arguably un-Pythonic and somewhat awkward. Like, sharing references has built-in syntax that's taught to most Python programmers within their first hour with the language, whereas copying - without writing a tedious manual implementation - requires importing the `copy` module [0], which outlines a number of pitfalls, quirks, and caveats with the process, and that I've literally never used so far as I can recall in a decade of professional Python development. [0] https://docs.python.org/3/library/copy.html https://docs.python.org/3/library/copy.html
  - adrianN 5y ago
    You have to pass around the whole object you want to copy for that to work. For example in a compiler, if you function only sees an AstNode, it can't make a deep copy of the whole AST easily. You also have to be careful with things like objects that contain refcounted pointers.
- xapata 5y ago
  Passing pointers around is a pretty common pattern. It avoids allocating memory and copying data unnecessarily. Seemed like a decent choice in the late 80's and early 90's, probably all the way up through mid-2000's. It's still fine as the default for many situations.
  - TheCoelacanth 5y ago
    > It avoids allocating memory and copying data unnecessarily My experience is the exact opposite. Pervasive shared mutability leads to developers making lots of unnecessary copies out of fear that some other part of the code will change values out from under them.
    - skrtskrt 5y ago
      Maybe it's just the kind of work I have done (web services, where objects don't live longer than a request), but I have never had a problem of "some other part of the code changing values out from under me" in Python or Go.
    - xapata 5y ago
      I go with the "we're all adults here" philosophy and trust my colleagues not to inappropriately mutate the things they receive. It's worked well enough so far. I understand why it might not work so well for other people.
- Orou 5y ago
  Naive question: in Python, isn't this solved by .copy and .deepcopy if you really have to get a copy of an object rather than create a new instance of something? I'm curious where the bad API design is, unless you're saying that Python's assignment behavior itself is bad and everything should be immutable by default.
adsharma 5y ago
If you have python code and trying to generate equivalent rust for performance, please give py2many a try. Looking for feedback. Also experimental support for pyo3 extensions. But really the main idea of the project is static python and complete recompilation (not interoperability with dynamic python code).
amelius 5y ago
I wonder what Rustaceous Python would look like.
- Twisol 5y ago
  Aggressively shared-xor-mutable. If an object A has a reference to some other object B, either B is (deeply) immutable or A has the only reference to B. This means that in general, data is only held by some code as long as is needed to operate on it, at which point the data is relinquished. Personally, I find OO designs to be enhanced by this principle, so I don't think it's only something one does in Rust. I certainly learned it from Rust though.
  - nerdponx 5y ago
    For what it's worth, following this pattern informally is usually a good idea anyway in Python (and Lua and Ruby and Javascript etc). Even if B is technically mutable (which 99% of the time it is because almost everything in Python is mutable), just don't mutate it and pretend like you're not allowed to do so.
    - Twisol 5y ago
      > Even if B is technically mutable (which 99% of the time it is because almost everything in Python is mutable), just don't mutate it and pretend like you're not allowed to do so. This gets you pretty far, but it's hard to ensure that nobody else ever mutates B. If B is technically mutable, and there are multiple mutable references to B, then when you invoke some other object in the course of your work, control might come back to you with B mutated without you realizing it. This is why the XOR is so important: if B is shared (with you) and mutable (not by you), B could change under you while you've passed control temporarily to someone else. This is certainly less problematic than having multiple mutable references, but it's still a source of complexity. As a very small toy example, consider iterating forward over an array while deleting elements.
      - macintux 5y ago
loeg 5y ago
I think OP's first approach (with some combination of Arc/Rc and Mutex or just plain RefCell) was probably on the right track and I'm not sure why they chose something else. They don't really elaborate on this? > This was my initial approach, but it started to become pretty verbose, pretty quickly. In hindsight it may have ultimately been simpler than where I did end up, but, well, that’s hindsight.
whalesalad 5y ago
I don’t agree with a lot of this post but I think it does a good job pointing out that porting a library from ecosystem A to ecosystem B is not always a good idea. A good API design in one language can be a horrible one in another. Python doesn’t expect you to share references to things - it’s just that the API that was being copied follows an imperative style that mutates objects.
- Spivak 5y ago
  But the predominant concern in a lot of porting efforts is interoperability with developers who are already familiar with the current API. Porting the API to feel very Rusty would be a mistake when your audience is people who already know the Python API. Python itself has adopted some decidedly non-pythonic APIs for the same reason. In the Python world if this bothers you you're supposed to use the adapter pattern to Pythonify the mechanically ported API.
  - tomnipotent 5y ago
    > interoperability with developers who are already familiar I wouldn't say it's "predominant". Some instances are code authors trying to bring their library to new audiences (c/c++ frameworks with multiple language bindings), or developers unfamiliar with the language but want that specific API in their own (Python's requests library is a great example, cloned in many languages now).
  - redrobein 5y ago
    Arguably you'd want to write in idiomatic style for the ecosystem you're working in. It doesn't help when the standard library plus any other libraries you're using offer rusty apis and this one library is pythonic.
  - whalesalad 5y ago
    I would first start by asking - why am I porting an identical API? Why am I not going back to first principles and rethinking from ground zero? Usually there is a compelling reason to switch from A to B, likely for performance in this case (going from py -> rust). Any programmer worth a salt knows that nothing in life is free - and everything has a tradeoff. In this case the tradeoff (should) be that the API might change, but the performance benefits will be worhtwhile. For instance - lets look at sequential operations against a DB versus batches. In a sequential style, you might iterate your items and just write them one by one. Alternatively, in a batched style you might need to do things like prepare queries or store your queries alongside the actual data values, then hand it all to a batch mechanism that will perform the write. The ergonomics are completely different but at the end of the day the result (rows in the table) will be the same. So even in the same language you will see totally different ways to go about solving the same problem. This is why I do not really see this as a python/rust argument and more of a generic program architecture argument. In the above db case - the programmer isn't disappointed that they need to rewrite their iterative loop approach because the know that the batching approach is going to be much faster and will achieve their goals. I think the same parallel can be drawn here: it is not a bad thing to have a slightly different API. Especially in the context of going from a dynamic language to one that is compiled and more performant. All of this is kind of a moot point though because the Rust lib in question doesn't have any instructions/documentation or design docs so it is hard to say for certain what the intention is here other than a port for the sake of porting.
xvedejas 5y ago
What is the idiomatic way to mimic inheritance in Rust, when writing bindings for OO languages? I'm new to Rust but I imagine there are ways to mimic upcasting/downcasting its types?
- rav 5y ago
  You can take a look at how the Gtk bindings handle the inheritance hierarchy in Gtk objects: https://gtk-rs.org/docs-src/tutorial/object_oriented https://gtk-rs.org/docs-src/tutorial/object_oriented
- andrewzah 5y ago
  With the 'as' keyword you can cast between types [0]. Instead of inheritance, you can implement Traits for objects [1], which works more like composition. [0]: https://doc.rust-lang.org/reference/expressions/operator-expr.html#type-cast-expressions https://doc.rust-lang.org/reference/expressions/operator-exp... [1]: https://doc.rust-lang.org/rust-by-example/trait.html https://doc.rust-lang.org/rust-by-example/trait.html
  - ChrisSD 5y ago
    Note that `from` and `try_from` are generally preferred to using `as`, where possible. This also allows casting between non-primitive types. https://doc.rust-lang.org/std/convert/trait.TryFrom.html#examples https://doc.rust-lang.org/std/convert/trait.TryFrom.html#exa...
    - xvedejas 5y ago
      How expensive is this kind of casting in Rust? Like, if I have a struct which has a superset of attributes that I need for the type I'm casting to, do I need to construct a new struct instance just to have something to treat as my target type, or is there a simpler way that doesn't involve new allocation?
      - msbarnett 5y ago
        You have options. Given Superset struct A, some other struct B that happens to have a subset of A’s fields, and some function foo which takes an argument of type B, you could 1) implement From<A> for B, and use that to convert instances of A when you want to call foo. This would probably involve some copying and/or allocation. 2) back up and turn foo into something that takes a trait argument instead. Now instead of taking instances of A, what it needs struct to provide is defined by a trait, which you can provide implementations of for both A and B. Now you can pass either to foo. 3) potentially, depending on the layout of structs, create a union and do some type punning to convert between them. this requires unsafe and you’d better be right about the struct layouts. 4) just make foo take a tagged union of the two structs. this isn’t unsafe but has different storage tradeoffs. also foo would need to handle each case separately. 5) find some other way to skin this cat. there’s plenty #2 would be idiomatic in most circumstances I think?
aazaa 5y ago
The example code is: font = Font.open("MyFont.ufo") glyphA = font.layers.defaultLayer["A"] point = glyphA.contours[0].points[0] point.x = 404; assert point.x == glyphA.contours[0].points[0].x The author apparently wants to be able to perform ad hoc modifications to an existing font. Unfortunately, it's not clear from the article what the use case is. Most of the time I've worked with fonts, they're treated as immutable values. If the idea is that a font needs to be built from a serialization format, then an alternative approach would be to eliminate the python mutable interface and replace it with a build function that calls the underlying Rust to build a Font. That way Python never needs to deal with mutating a font. Rust lets you take unlimited numbers of references to an immutable value. That's not a problem, other than defining lifetimes if you pass those references around and/or hold them in structs. The problem arises the instant you want to mutate the value behind a reference. Hold just one active immutable reference at the same time, and that won't be possible. Note that mutable and immutable references can in some situations exist in the same block thanks to non-lexical lifetimes.
- mjw1007 5y ago
  I think the use case for modifications is letting people write custom extensions to a font editor.
- cmyr 5y ago
  There are a lot of existing python scripts that run on top of the existing python `ufoLib2` library, and the initial motivation for this work was seeing what would be involved in creating a wrapper API that would allow all of these existing scripts to work transparently on top of an existing rust crate.
- alkonaut 5y ago
  Exactly. If I do glyphA.contours[0].points[0].x = 1 what happened to someone else holding a reference to the same glyph? To a python programmer it might not be a surprise to see such an API (a shared mutable value), but to most OO programmers I hope this API looks very strange.
  - 5y ago
sgt 5y ago
Speaking of.. is there an article about writing Pythonic python? I think that would be extremely useful to various team members. Sometimes the official documentation is too verbose.
- fnord123 5y ago
  Pythonic guide, not too verbose? >>> import this The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those!
  - throwaway894345 5y ago
    I've often wished Python indexed more on "simple is better than complex" and "There should be one-- and preferably only one --obvious way to do it". Python has become quite complex over the years, including its object system, its tooling, ecosystem, and even many of the standard library APIs (thinking of you, `subprocess`!).
    - richardwhiuk 5y ago
      Or format strings.
      - CornCobs 5y ago
        I think f strings are great - obvious, unambiguous and an instant huge qol improvement
simias 5y ago
This article demonstrates well something I've seen a lot with people coming from GC languages getting into Rust: they just write the code the way they're used to and work around the borrow checker by slapping `Arc<Mutex<_>>` all over the place. Then that leads to frustration and wondering why you even got rid of the GC in the first place if you end up with a crappier non-transparent reference-counted garbage collection with all these Arc/Rc. Some devs even seem to think that non-GC devs are silly poseurs who refuse to have a garbage collector only to reimplement it manually with reference counting. We are silly poseurs alright, but we're more efficient than that! I share the author's conclusions: don't do that. If you find yourself slapping Mutexes and Arc/Rc all over the place it probably means that there's something messed up with the way you modeled data ownership within your program.
- cogman10 5y ago
  Yup, definitely a frustration I have reading through rust tutorials as well. So many of them will get out of memory problems by "Oh, just add Arc around it".
- PragmaticPulp 5y ago
  This is also a problem with Rust conversations and tutorials online: There's a lot of talk about what not to do or what you can't do in Rust, but it's rarely followed up with an explanation of what users should do. Mutexes aren't an entirely foreign concept to many programmers. Obviously if someone can architect their system in a way that doesn't require Arc<Mutex<_>> then go for it, but we need to be careful about giving blanket advice without alternatives.
  - rmdashrfstar 5y ago
    Indeed, this blanket advice does not apply in situations where shared resources that require mutable access need to be synchronized. If my async HTTP router is serving two requests that are using the same database client to persist changes, that needs to be synchronized. How does one do this without a mutex?
    - 5y ago
granduh 5y ago
can someone comment on the OP’s website platform? it looks like a plain ftp explorer but it seems to support inline pages on mobile.
- scoopertrooper 5y ago
  Looks like a custom job.
waffletower 5y ago
The world needs fewer python idioms rather than more. Python is a sprawling language with an identity crisis stemming from its orthogonal object oriented and functional features. Why impose this mindset on Rust? Perhaps python, or at the very minimum the python developer, has much to gain from other languages instead.
- cmyr 5y ago
  This question has been answered at least twice in comments in this thread.
oconnor663 5y ago
The `Arc<Mutex<T>>` approach here would make me worried about reference cycles, unless the type relationships here look like a tree/DAG. Out of curiosity, if I store references to other things as opaque `pyo3::PyObject` objects, would the CPython cycle collector be able to see these references and collect their cycles?
phantom_oracle 5y ago
I checked this thread and the first thing I searched for was nim-lang[1]. Looks like nobody mentioned it yet. Nim/Nim-lang makes it much easier to write Pythonic code. Minus all the complex things that only smart(er) programmers get(like GC, etc.), with Nim you can write code that is much more readable. Another win for it is that it has very limited support for OOP, which IMHO sucks in terms of readability and understanding what the code is doing(as compared to imperative code). [1]https://nim-lang.org/ https://nim-lang.org/
batterylow 5y ago
Coming from Python, I was interested in using Rust for data analysis within Jupyter notebooks... Evcxr made that possible, and I've self-published a book on the topic https://datacrayon.com/shop/product/data-analysis-with-rust-notebooks/ https://datacrayon.com/shop/product/data-analysis-with-rust-...
lilyball 5y ago
Honestly, the Arc<Mutex<T>> approach is the right one here. We need to hand out shared references, and that’s what that is. Otherwise there’s a ton of bookkeeping, and throwing exceptions in code that could otherwise reasonably work. This approach assumes of course that there are no reference cycles in the graph. If there are, you’ll need some way to clean them up if Weak isn’t sufficient. Alternatively, is there a way to ask Python to allocate memory for you, which is then subject to Python’s GC? If so, you could make a smart pointer type for this and use that everywhere. Though you’ll still need to figure out how thread safety works. So this really would only be for handling reference cycles.
willyt 5y ago
Can someone explain why they need to implement this in Rust rather than python? What operation is the author doing to a font that is compute intensive enough and happens often enough that you would gain anything by implementing this in a complicated low level language like Rust?
- mmmrk 5y ago
  Font projects (especially multi-script ones) can amass (tens of) thousands of small XML files in the UFO format. Rust can load and save through them in seconds, Python can't :)

218 points by zbentley 5y ago

You probably don’t want to do that.

> python expects you to share references to things. When you create a binding to a point in a contour, that binding refers to the same data as the original point, and modifying the binding modifies the collection. Honestly, as someone who has written way more Python than Rust, this seems like bad API design regardless of language. This screams "it's impossible to take a font, make two separate modifications to it, and then work with those separate modifications at the same time", because deepcopying objects is usually very difficult.

Blikkentrekker 5y ago
Why would deep copying be difficult? Even in Rust, the `Clone` trait produces a deep copy. Copying only up to a specific depth in a general way is what is difficult. I would argue that shallow copies, id est the depth being 1, are far more difficult to realize that copies of unbounded depth until a value o trait `Copy` is reached, id est a type that is purely encoded by the data it takes up on the stack and owns no further data and can thus be fully be `Clone`ed by copying it's stack bits.
- kzrdude 5y ago
  > Even in Rust, the `Clone` trait produces a deep copy. Clone is not really a deep copy. I like the description that says it is "deep enough to give you an object of identical/independent ownership (of what you started with), but no deeper". Example: Rc<String> when cloned only gives you another handle to the Rc, the string data is not duplicated (for this reason we don't call it a real deep copy). You get a new rc handle that is on equal footing with the old handle. There is plenty of Rust types that consist of tree-shaped ownership relations with no frills - in these cases clone and deep copy are identical. Let's say for example a HashMap<i32, String>.
  - oconnor663 5y ago
    I think every "deep" copy ultimately has some limitation like this. Even in Python, an object can implement __deepcopy__() to prevent copy.deepcopy() from "copying too much", and things like file objects, classes, and modules get special treatment.
    - kzrdude 5y ago
      I found that in Rust, you don't have to think about the distinction so much - not like you have to in for example Python and C# (pervasive/transparent object reference languages). In Rust it's predictable and mostly obvious which data is shared ownership or shared, and which is not. We don't need to talk about deep vs shallow copy semantics much in Rust. :)
  - Blikkentrekker 5y ago
    You are correct; my original definition that only went till `copy` was encountered is only the most common case of your more general definition. The principles are still quite similar and it still seems easier to implement to me than shallow copies as a generic trait. I don't know how I would fit shallow copies into Rust's trait system.
    - kzrdude 5y ago
      Clone seems to be the shallowest kind copy permitted by the type and ownership system.
      - Blikkentrekker 5y ago
        Which is why I find deep copies to be easier to implement than shallow. It is of course possible to express a shallower copy, but not in the same generic way as a universal trait, that I know of, in any case.
- wk_end 5y ago
  It's not intrinsically difficult, but in Python it's arguably un-Pythonic and somewhat awkward. Like, sharing references has built-in syntax that's taught to most Python programmers within their first hour with the language, whereas copying - without writing a tedious manual implementation - requires importing the `copy` module [0], which outlines a number of pitfalls, quirks, and caveats with the process, and that I've literally never used so far as I can recall in a decade of professional Python development. [0] https://docs.python.org/3/library/copy.html https://docs.python.org/3/library/copy.html
- adrianN 5y ago
  You have to pass around the whole object you want to copy for that to work. For example in a compiler, if you function only sees an AstNode, it can't make a deep copy of the whole AST easily. You also have to be careful with things like objects that contain refcounted pointers.
xapata 5y ago
Passing pointers around is a pretty common pattern. It avoids allocating memory and copying data unnecessarily. Seemed like a decent choice in the late 80's and early 90's, probably all the way up through mid-2000's. It's still fine as the default for many situations.
- TheCoelacanth 5y ago
  > It avoids allocating memory and copying data unnecessarily My experience is the exact opposite. Pervasive shared mutability leads to developers making lots of unnecessary copies out of fear that some other part of the code will change values out from under them.
  - skrtskrt 5y ago
    Maybe it's just the kind of work I have done (web services, where objects don't live longer than a request), but I have never had a problem of "some other part of the code changing values out from under me" in Python or Go.
  - xapata 5y ago
    I go with the "we're all adults here" philosophy and trust my colleagues not to inappropriately mutate the things they receive. It's worked well enough so far. I understand why it might not work so well for other people.
Orou 5y ago
Naive question: in Python, isn't this solved by .copy and .deepcopy if you really have to get a copy of an object rather than create a new instance of something? I'm curious where the bad API design is, unless you're saying that Python's assignment behavior itself is bad and everything should be immutable by default.

adsharma 5y ago

If you have python code and trying to generate equivalent rust for performance, please give py2many a try. Looking for feedback. Also experimental support for pyo3 extensions. But really the main idea of the project is static python and complete recompilation (not interoperability with dynamic python code).

amelius 5y ago

I wonder what Rustaceous Python would look like.

Twisol 5y ago
Aggressively shared-xor-mutable. If an object A has a reference to some other object B, either B is (deeply) immutable or A has the only reference to B. This means that in general, data is only held by some code as long as is needed to operate on it, at which point the data is relinquished. Personally, I find OO designs to be enhanced by this principle, so I don't think it's only something one does in Rust. I certainly learned it from Rust though.
- nerdponx 5y ago
  For what it's worth, following this pattern informally is usually a good idea anyway in Python (and Lua and Ruby and Javascript etc). Even if B is technically mutable (which 99% of the time it is because almost everything in Python is mutable), just don't mutate it and pretend like you're not allowed to do so.
  - Twisol 5y ago
    > Even if B is technically mutable (which 99% of the time it is because almost everything in Python is mutable), just don't mutate it and pretend like you're not allowed to do so. This gets you pretty far, but it's hard to ensure that nobody else ever mutates B. If B is technically mutable, and there are multiple mutable references to B, then when you invoke some other object in the course of your work, control might come back to you with B mutated without you realizing it. This is why the XOR is so important: if B is shared (with you) and mutable (not by you), B could change under you while you've passed control temporarily to someone else. This is certainly less problematic than having multiple mutable references, but it's still a source of complexity. As a very small toy example, consider iterating forward over an array while deleting elements.
    - macintux 5y ago

loeg 5y ago

I think OP's first approach (with some combination of Arc/Rc and Mutex or just plain RefCell) was probably on the right track and I'm not sure why they chose something else. They don't really elaborate on this? > This was my initial approach, but it started to become pretty verbose, pretty quickly. In hindsight it may have ultimately been simpler than where I did end up, but, well, that’s hindsight.

whalesalad 5y ago

I don’t agree with a lot of this post but I think it does a good job pointing out that porting a library from ecosystem A to ecosystem B is not always a good idea. A good API design in one language can be a horrible one in another. Python doesn’t expect you to share references to things - it’s just that the API that was being copied follows an imperative style that mutates objects.

Spivak 5y ago
But the predominant concern in a lot of porting efforts is interoperability with developers who are already familiar with the current API. Porting the API to feel very Rusty would be a mistake when your audience is people who already know the Python API. Python itself has adopted some decidedly non-pythonic APIs for the same reason. In the Python world if this bothers you you're supposed to use the adapter pattern to Pythonify the mechanically ported API.
- tomnipotent 5y ago
  > interoperability with developers who are already familiar I wouldn't say it's "predominant". Some instances are code authors trying to bring their library to new audiences (c/c++ frameworks with multiple language bindings), or developers unfamiliar with the language but want that specific API in their own (Python's requests library is a great example, cloned in many languages now).
- redrobein 5y ago
  Arguably you'd want to write in idiomatic style for the ecosystem you're working in. It doesn't help when the standard library plus any other libraries you're using offer rusty apis and this one library is pythonic.
- whalesalad 5y ago
  I would first start by asking - why am I porting an identical API? Why am I not going back to first principles and rethinking from ground zero? Usually there is a compelling reason to switch from A to B, likely for performance in this case (going from py -> rust). Any programmer worth a salt knows that nothing in life is free - and everything has a tradeoff. In this case the tradeoff (should) be that the API might change, but the performance benefits will be worhtwhile. For instance - lets look at sequential operations against a DB versus batches. In a sequential style, you might iterate your items and just write them one by one. Alternatively, in a batched style you might need to do things like prepare queries or store your queries alongside the actual data values, then hand it all to a batch mechanism that will perform the write. The ergonomics are completely different but at the end of the day the result (rows in the table) will be the same. So even in the same language you will see totally different ways to go about solving the same problem. This is why I do not really see this as a python/rust argument and more of a generic program architecture argument. In the above db case - the programmer isn't disappointed that they need to rewrite their iterative loop approach because the know that the batching approach is going to be much faster and will achieve their goals. I think the same parallel can be drawn here: it is not a bad thing to have a slightly different API. Especially in the context of going from a dynamic language to one that is compiled and more performant. All of this is kind of a moot point though because the Rust lib in question doesn't have any instructions/documentation or design docs so it is hard to say for certain what the intention is here other than a port for the sake of porting.

xvedejas 5y ago

What is the idiomatic way to mimic inheritance in Rust, when writing bindings for OO languages? I'm new to Rust but I imagine there are ways to mimic upcasting/downcasting its types?

rav 5y ago
You can take a look at how the Gtk bindings handle the inheritance hierarchy in Gtk objects: https://gtk-rs.org/docs-src/tutorial/object_oriented https://gtk-rs.org/docs-src/tutorial/object_oriented
andrewzah 5y ago
With the 'as' keyword you can cast between types [0]. Instead of inheritance, you can implement Traits for objects [1], which works more like composition. [0]: https://doc.rust-lang.org/reference/expressions/operator-expr.html#type-cast-expressions https://doc.rust-lang.org/reference/expressions/operator-exp... [1]: https://doc.rust-lang.org/rust-by-example/trait.html https://doc.rust-lang.org/rust-by-example/trait.html
- ChrisSD 5y ago
  Note that `from` and `try_from` are generally preferred to using `as`, where possible. This also allows casting between non-primitive types. https://doc.rust-lang.org/std/convert/trait.TryFrom.html#examples https://doc.rust-lang.org/std/convert/trait.TryFrom.html#exa...
  - xvedejas 5y ago
    How expensive is this kind of casting in Rust? Like, if I have a struct which has a superset of attributes that I need for the type I'm casting to, do I need to construct a new struct instance just to have something to treat as my target type, or is there a simpler way that doesn't involve new allocation?
    - msbarnett 5y ago
      You have options. Given Superset struct A, some other struct B that happens to have a subset of A’s fields, and some function foo which takes an argument of type B, you could 1) implement From<A> for B, and use that to convert instances of A when you want to call foo. This would probably involve some copying and/or allocation. 2) back up and turn foo into something that takes a trait argument instead. Now instead of taking instances of A, what it needs struct to provide is defined by a trait, which you can provide implementations of for both A and B. Now you can pass either to foo. 3) potentially, depending on the layout of structs, create a union and do some type punning to convert between them. this requires unsafe and you’d better be right about the struct layouts. 4) just make foo take a tagged union of the two structs. this isn’t unsafe but has different storage tradeoffs. also foo would need to handle each case separately. 5) find some other way to skin this cat. there’s plenty #2 would be idiomatic in most circumstances I think?

aazaa 5y ago

The example code is: font = Font.open("MyFont.ufo") glyphA = font.layers.defaultLayer["A"] point = glyphA.contours[0].points[0] point.x = 404; assert point.x == glyphA.contours[0].points[0].x The author apparently wants to be able to perform ad hoc modifications to an existing font. Unfortunately, it's not clear from the article what the use case is. Most of the time I've worked with fonts, they're treated as immutable values. If the idea is that a font needs to be built from a serialization format, then an alternative approach would be to eliminate the python mutable interface and replace it with a build function that calls the underlying Rust to build a Font. That way Python never needs to deal with mutating a font. Rust lets you take unlimited numbers of references to an immutable value. That's not a problem, other than defining lifetimes if you pass those references around and/or hold them in structs. The problem arises the instant you want to mutate the value behind a reference. Hold just one active immutable reference at the same time, and that won't be possible. Note that mutable and immutable references can in some situations exist in the same block thanks to non-lexical lifetimes.

mjw1007 5y ago
I think the use case for modifications is letting people write custom extensions to a font editor.
cmyr 5y ago
There are a lot of existing python scripts that run on top of the existing python `ufoLib2` library, and the initial motivation for this work was seeing what would be involved in creating a wrapper API that would allow all of these existing scripts to work transparently on top of an existing rust crate.
alkonaut 5y ago
Exactly. If I do glyphA.contours[0].points[0].x = 1 what happened to someone else holding a reference to the same glyph? To a python programmer it might not be a surprise to see such an API (a shared mutable value), but to most OO programmers I hope this API looks very strange.
- 5y ago

sgt 5y ago

Speaking of.. is there an article about writing Pythonic python? I think that would be extremely useful to various team members. Sometimes the official documentation is too verbose.

fnord123 5y ago
Pythonic guide, not too verbose? >>> import this The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those!
- throwaway894345 5y ago
  I've often wished Python indexed more on "simple is better than complex" and "There should be one-- and preferably only one --obvious way to do it". Python has become quite complex over the years, including its object system, its tooling, ecosystem, and even many of the standard library APIs (thinking of you, `subprocess`!).
  - richardwhiuk 5y ago
    Or format strings.
    - CornCobs 5y ago
      I think f strings are great - obvious, unambiguous and an instant huge qol improvement

simias 5y ago

This article demonstrates well something I've seen a lot with people coming from GC languages getting into Rust: they just write the code the way they're used to and work around the borrow checker by slapping `Arc<Mutex<_>>` all over the place. Then that leads to frustration and wondering why you even got rid of the GC in the first place if you end up with a crappier non-transparent reference-counted garbage collection with all these Arc/Rc. Some devs even seem to think that non-GC devs are silly poseurs who refuse to have a garbage collector only to reimplement it manually with reference counting. We are silly poseurs alright, but we're more efficient than that! I share the author's conclusions: don't do that. If you find yourself slapping Mutexes and Arc/Rc all over the place it probably means that there's something messed up with the way you modeled data ownership within your program.

cogman10 5y ago
Yup, definitely a frustration I have reading through rust tutorials as well. So many of them will get out of memory problems by "Oh, just add Arc around it".
PragmaticPulp 5y ago
This is also a problem with Rust conversations and tutorials online: There's a lot of talk about what not to do or what you can't do in Rust, but it's rarely followed up with an explanation of what users should do. Mutexes aren't an entirely foreign concept to many programmers. Obviously if someone can architect their system in a way that doesn't require Arc<Mutex<_>> then go for it, but we need to be careful about giving blanket advice without alternatives.
- rmdashrfstar 5y ago
  Indeed, this blanket advice does not apply in situations where shared resources that require mutable access need to be synchronized. If my async HTTP router is serving two requests that are using the same database client to persist changes, that needs to be synchronized. How does one do this without a mutex?
  - 5y ago

granduh 5y ago

can someone comment on the OP’s website platform? it looks like a plain ftp explorer but it seems to support inline pages on mobile.

scoopertrooper 5y ago
Looks like a custom job.

waffletower 5y ago

The world needs fewer python idioms rather than more. Python is a sprawling language with an identity crisis stemming from its orthogonal object oriented and functional features. Why impose this mindset on Rust? Perhaps python, or at the very minimum the python developer, has much to gain from other languages instead.

cmyr 5y ago
This question has been answered at least twice in comments in this thread.

oconnor663 5y ago

The `Arc<Mutex<T>>` approach here would make me worried about reference cycles, unless the type relationships here look like a tree/DAG. Out of curiosity, if I store references to other things as opaque `pyo3::PyObject` objects, would the CPython cycle collector be able to see these references and collect their cycles?

phantom_oracle 5y ago

I checked this thread and the first thing I searched for was nim-lang[1]. Looks like nobody mentioned it yet. Nim/Nim-lang makes it much easier to write Pythonic code. Minus all the complex things that only smart(er) programmers get(like GC, etc.), with Nim you can write code that is much more readable. Another win for it is that it has very limited support for OOP, which IMHO sucks in terms of readability and understanding what the code is doing(as compared to imperative code). [1]https://nim-lang.org/ https://nim-lang.org/

batterylow 5y ago

Coming from Python, I was interested in using Rust for data analysis within Jupyter notebooks... Evcxr made that possible, and I've self-published a book on the topic https://datacrayon.com/shop/product/data-analysis-with-rust-notebooks/ https://datacrayon.com/shop/product/data-analysis-with-rust-...

lilyball 5y ago

Honestly, the Arc<Mutex<T>> approach is the right one here. We need to hand out shared references, and that’s what that is. Otherwise there’s a ton of bookkeeping, and throwing exceptions in code that could otherwise reasonably work. This approach assumes of course that there are no reference cycles in the graph. If there are, you’ll need some way to clean them up if Weak isn’t sufficient. Alternatively, is there a way to ask Python to allocate memory for you, which is then subject to Python’s GC? If so, you could make a smart pointer type for this and use that everywhere. Though you’ll still need to figure out how thread safety works. So this really would only be for handling reference cycles.

willyt 5y ago

Can someone explain why they need to implement this in Rust rather than python? What operation is the author doing to a font that is compute intensive enough and happens often enough that you would gain anything by implementing this in a complicated low level language like Rust?

mmmrk 5y ago
Font projects (especially multi-script ones) can amass (tens of) thousands of small XML files in the UFO format. Rust can load and save through them in seconds, Python can't :)