17 ms·

We moved from Pony to Rust

201 points by aturley 5y ago

latenightcoding 5y ago
There are people commenting that Pony was the wrong choice from the get go, but many of us have only heard of this startup (which operates in a saturated market) because they were using Pony.
- spooneybarger 5y ago
  As the head of engineering at the time that we made the decision, that had nothing to do with it, but I suppose it can't hurt. That said, I don't think we ever got any business as a result of folks having heard of Wallaroo from any of the promotion we tried to do around technical decisions. We did however attract interest from investors who would notice when our technical content did well on HN. It certainly didn't hurt during fundraising. Our decision to go with Pony for the original product we were building (high performance, finance focused, stream processing) was in a large part driven by time to market. We didn't have runway to build our own high-performance runtime for the product and using Pony got us that runtime. You can hear me talking more about the decision process here: https://corecursive.com/055-unproven-with-sean-allen/ https://corecursive.com/055-unproven-with-sean-allen/ There was more to it than that, but the "we get a runtime that can meet our needs" was a huge, huge part of why we picked Pony. I have often told people that we didn't so much pick a language as we picked the runtime and the language came along with it. That's not really true, but it's not really wrong either.
pjmlp 5y ago
I guess it makes sense from business point of view, although it is a pity that Pony is losing what is probably the only commercial user (that I am aware of).
- runevault 5y ago
  I remember becoming curious about Pony because of how much they talked it up, though I never got around to trying it. Feels weird to see the pivot, though I've felt for a while Rust or Rust-inspired languages will become a larger and larger part of the future of software so I guess I shouldn't be surprised.
  - spooneybarger 5y ago
    Hey, there's still plenty of time to try Pony. We are a little volunteer driven project and will be here whenever you find the time. Stop by the Zulip sometime and chat! https://ponylang.zulipchat.com/ https://ponylang.zulipchat.com/ -Sean-
Thaxll 5y ago
Shocking that using a language that no one use was a mistake on the long term. It made 0 sense what so ever to use it in the first place.
- _6hmp 5y ago
  OTOH an interesting language can be a recruiting draw. It probably helped them recruit engineers who were interested in the distributed systems and concurrency problems they were trying to solve. See for example Jane Street with OCaml. [edit: oops, thanks for the heads up on the spelling :)]
  - Zababa 5y ago
    OCaml is old and reliable, and I think it was already proven to work in the industry when Jane Street chose it. Even if it wasn't the best programming language ever, it was still a strong choice at the time, and still is. Pony on the other hand was and still is very young. I'm not saying it's a bad choice, but it's definitely not the same risk profile as OCaml. It's just OCaml by the way, for Objective Caml. Unless you're talking about the secret Irish fork /s. There's a nice page on the official website about the history: https://ocaml.org/learn/history.html https://ocaml.org/learn/history.html.
    - openfuture 5y ago
      Maybe they should have used a horse, they are much more reliable than camels and more mature than ponies.
      - Zababa 5y ago
        I'm not sure, AoE2 taught me that camels always win against horses. And if you consider Rust to be OCaml's child (which is kind of true if you really stretch things), it seems like young camels win against young horses too.
        shpongled 5y ago
shrubble 5y ago
Wasn't there a big brouhaha, many years ago, when Reddit switched from Lisp to Python?
Fiahil 5y ago
> However, in the meantime data science and machine learning continued to explode, and we found that customer interest in deploying machine learning models far outstripped demand for more conventional stream processing data algorithms. My experience is quite similar : I either end up working with clients who are already using Spark etc, or they don’t have a mature data engineering lab. In both cases, deploying a model to production is always the most challenging. I hope their MLOps strategy is better, than just binding to MLFlow. This tool is absolutely not ready for prime time, and plagued by bad product decisions.
Zababa 5y ago
This feels like the same pattern as Dark leaving OCaml for F#: https://blog.darklang.com/leaving-ocaml// https://blog.darklang.com/leaving-ocaml//. Ecosystem matters a lot these days. Outside of these two specific cases, I wonder if we're, as an industry, too afraid of writing this kind of stuff now I feel like it was done a lot before, and not at all these days. Sure, NIH syndrome is a fallacy, but having to write one library may not be so bad. I would be glad to hear about any experience with that.
- wyager 5y ago
  I used to work at an OCaml company and it wasn't nearly as much of an issue as one might predict. You can (it turns out) build a very successful business even if there aren't a lot of existing libraries, or if the language lacks certain basic features like native multithreading (same with Python of course). I don't have a great model for why this isn't devastatingly expensive, but it's probably some combination of * Most existing libraries are kind of bad anyway so you're not missing out much by not using them * If you write everything yourself you get system expertise "for free", and gaining expertise in something that already exists is hard * You can tailor everything to your business requirements * Writing libraries is "fun work" so it's easier to be productive while doing it
  - Zababa 5y ago
    That's interesting, thanks for sharing your experience. Sometimes I wonder how much interesting experience I miss by mostly using things people already built. Sure, if you're just trying to get something done it's probably faster, but on the other hand spending more time gives you experience.
  - pbiggar 5y ago
    I don't agree with this at all. While it's true that there are lots of bad libraries and many libraries are easy, you really isolate yourself from the broader ecosystem by doing this. Vendors and 3rd party solutions are now much harder to use, and when you use them you'll probably only use them at a surface level instead of using all the features. And some things are so mature and vast you don't have a chance of building them yourself. If what you are doing can be done well in very mature ecosystems like React or PyTorch, the effort to recreate them will dwarf the time spend on important work.
guu 5y ago
Last year they discussed why they didn’t choose rust (predicted 12-18 months to build a runtime that met their requirements) or erlang (performance too slow): https://corecursive.com/055-unproven-with-sean-allen/#considering-rust https://corecursive.com/055-unproven-with-sean-allen/#consid...
- spooneybarger 5y ago
  That's me. I discussed why we (I am no longer at Wallaroo) made the decision about 5 years ago to use Pony to build a product that is no longer what Wallaroo is selling.
ZeroCool2u 5y ago
The product Wallaroo is building sounds really interesting, (It's so appealing I honestly checked out their open positions), but it's really difficult to get a feel for what exactly it's like to use their product now. The description and outcome statistics make it sound _excellent_ and their technology stack makes their claims seem viable. However, and I say this as someone that is currently running an MLOps RFP, after looking at every page of their site and all their open job postings I have no idea what it's actually like to use their product.
- aturley 5y ago
  Thanks for checking us out. We're working improving our website and our explainer materials. Would you be interested in a quick conversation? I'd be happy to do a demo and get some feedback on ways we could improve the first impression. you can reach me at `andy at wallaroo dot ai`.
debarshri 5y ago
Just a thought. Rewriting the core platform from one language to another just takes huge resources for orgs who are in high growth path. It feels like a side mission, derailed from core value that is solving the problem statement. Rewrites are generally a red flag for me, especially early in an orgs history.
- spooneybarger 5y ago
  This article is about how Wallaroo when they started a NEW product chose a different language than what was used to write the original product. There's no rewrite here.
secondcoming 5y ago
> Our new Rust-based platform recently handled millions of inferences a second of a sophisticated ad-targeting model with a median latency of 5 milliseconds, and a yearly infrastructure cost of $130K. Were these run on CPUs or GPUs? How many of them? Last I looked at running Tensorflow models on CPUs it was really slow, so slow we had to abandon it.
btheshoe 5y ago
That font is so thin it's almost uncomfortable to read. I've never heard of Pony before - is it any good, or worth playing around with?
- spooneybarger 5y ago
  I'm a Pony core team member. I think its quite good in the most general sense, but "is it good for your problem" is entirely different. I think at worst, you would get something out of learning Pony (although that can take some time, it's one of the harder languages to learn). Feel free to stop by our Zulip and chat more as folks from the Pony community are rarely on HN, but are on the Zulip all the time. https://ponylang.zulipchat.com/ https://ponylang.zulipchat.com/
rackjack 5y ago
I just wish Pony would reach 1.0 already. I'm very hesitant to spend my time on a language that might change under me.
awb 5y ago
One of the hardest things at any company is recruiting and retaining top talent. I’m sure some devs would love to learn a new language at a new job, but many would see it as a career mistake. Your career income increases the more you master a certain discipline like a certain language. If you’re a jack of all trades (and adding Pony developer to your resume), you’re probably limiting your future income potential.
smallstepforman 5y ago
I evaluated Pony 4 years ago, and walked away due to 2 technical issues: - garbage collection - no mechanism for synchronous access to actors We ended up building a C++ actor model, with it's associated headaches. Yes, we still have race conditions and some developers invoke the function directly instead of using messages, and yes, sometimes we will grab a locking mechanism to do synchronous access, but at the end of the day, the performance of real code (embedded gaming system) is all that matters, and we did meet our performance goals. Having said that, I've given up looking for an actor programming language and started building my own. Essentially, C++ like with Actors and compiler validated tracking of resources across actors (so the compiler knows about locks and actors). Still working on the compiler (compiles to C++20) and maybe 18 months away from public reveal. But the initial batch of test apps look very nice (but without compiler validation of resource sharing at this point in time). Very terse code, and implementing a compiler that outputs C++ was much easier than I originally feared. And no, there are no forward declarations or header files, compilers are run on modern workstations that have the grunt to produce the necessary scaffolding (from the project dependency script).
- devit 5y ago
  Why not use Rust? It can do what you want.
  - Karrot_Kream 5y ago
    +1 actix.rs is pretty well-known and gives you the benefit of a full programming language if you need to do other things. Rust should be really easy to onboard to if your devs already know C++ and while the compiler is a lot slower, there's enough posts online about Actix that you shouldn't have issues searching around.
  - smallstepforman 5y ago
    With 21st century developer workstations having >= 8 cores, >= 2Ghz, >= 16Gb memory, it is ridiculous to have the scaffolding that Rust, Zig and other modern languages have. Source files dont live on their own and shouldn't be compiled as isolated units, they live in projects and need to be compiled as a batch. I'm stunned to see that so many developers are used to needing scaffolding, they just accept it as being necessary, but when you step back and think about it, it no longer makes any sense to require it. C and C++ get a pass here since they were designed in the 70/80's, but languages that have been designed from 2010 onwards must be ridiculed for targetting a PDP-11 era development machine where each compiler unit has limited access to resources. The end language I'm designing has smaller source files, however the compiler is a lot more complex since it auto generates the scaffolding (header files, forward declarations, etc). It's still a fair bit away, I just got the proof of concept compiler which outputs C++ working (and the compiler is written in the language itself, so I'm dog fooding the project).
eatonphil 5y ago
I was under the impression that Wallaroo was the (sole) organization behind Pony. But I guess that's not true the way this is written.
forgotpwd16 5y ago
Pony is a younger than Rust and has far smaller exposure. It isn't strange that it didn't worked out. Using the shiny new thing is rarely a good choice.
- spooneybarger 5y ago
  Technical issues were never the problem with the first product we built at Wallaroo. Pony met all of our technical needs for that product. We never got product market fit with it. Rightly, around the time I left the company, they pivoted and started a new product that they thought would have better product market fit and they started over with a new codebase as it is a rather different product than what we first built.
  - forgotpwd16 5y ago
    Arguably the non-technical issues are equally important and, according to the essay, those were the reasons they ended up choosing Rust going forwards. >Pony [...] has a smaller community, and as a small startup we were better off not having to solve problems outside our core domain >wealth of available libraries >access to a large community that will guarantee ongoing future support >a wide variety of tools >much larger pool of engineers who are eager to learn and work in Rust, or who already have significant Rust experience >more resources available for learning Rust, and more opportunities for participation in conferences, etc. Didn't say that Pony was bad for some technical reason. It was/is bad choice for a startup to use due to being a relatively new, niche language.
mbrodersen 5y ago
It is absolutely batshit crazy to start a business with new but immature tools like Pony. Unless investors are throwing money at you and you don’t care about building a long term successful business. All developer tools take years to become solid and well supported. And most of them fail to get any major developer mindshare. If you think your unicorn snowflake project needs a new immature but oh so shiny programming language to be successful then you are simply wrong. Not even “likely to be wrong” but 100% guaranteed to be wrong.

201 points by aturley 5y ago

There are people commenting that Pony was the wrong choice from the get go, but many of us have only heard of this startup (which operates in a saturated market) because they were using Pony.

spooneybarger 5y ago
As the head of engineering at the time that we made the decision, that had nothing to do with it, but I suppose it can't hurt. That said, I don't think we ever got any business as a result of folks having heard of Wallaroo from any of the promotion we tried to do around technical decisions. We did however attract interest from investors who would notice when our technical content did well on HN. It certainly didn't hurt during fundraising. Our decision to go with Pony for the original product we were building (high performance, finance focused, stream processing) was in a large part driven by time to market. We didn't have runway to build our own high-performance runtime for the product and using Pony got us that runtime. You can hear me talking more about the decision process here: https://corecursive.com/055-unproven-with-sean-allen/ https://corecursive.com/055-unproven-with-sean-allen/ There was more to it than that, but the "we get a runtime that can meet our needs" was a huge, huge part of why we picked Pony. I have often told people that we didn't so much pick a language as we picked the runtime and the language came along with it. That's not really true, but it's not really wrong either.

pjmlp 5y ago

I guess it makes sense from business point of view, although it is a pity that Pony is losing what is probably the only commercial user (that I am aware of).

runevault 5y ago
I remember becoming curious about Pony because of how much they talked it up, though I never got around to trying it. Feels weird to see the pivot, though I've felt for a while Rust or Rust-inspired languages will become a larger and larger part of the future of software so I guess I shouldn't be surprised.
- spooneybarger 5y ago
  Hey, there's still plenty of time to try Pony. We are a little volunteer driven project and will be here whenever you find the time. Stop by the Zulip sometime and chat! https://ponylang.zulipchat.com/ https://ponylang.zulipchat.com/ -Sean-

Thaxll 5y ago

Shocking that using a language that no one use was a mistake on the long term. It made 0 sense what so ever to use it in the first place.

_6hmp 5y ago
OTOH an interesting language can be a recruiting draw. It probably helped them recruit engineers who were interested in the distributed systems and concurrency problems they were trying to solve. See for example Jane Street with OCaml. [edit: oops, thanks for the heads up on the spelling :)]
- Zababa 5y ago
  OCaml is old and reliable, and I think it was already proven to work in the industry when Jane Street chose it. Even if it wasn't the best programming language ever, it was still a strong choice at the time, and still is. Pony on the other hand was and still is very young. I'm not saying it's a bad choice, but it's definitely not the same risk profile as OCaml. It's just OCaml by the way, for Objective Caml. Unless you're talking about the secret Irish fork /s. There's a nice page on the official website about the history: https://ocaml.org/learn/history.html https://ocaml.org/learn/history.html.
  - openfuture 5y ago
    Maybe they should have used a horse, they are much more reliable than camels and more mature than ponies.
    - Zababa 5y ago
      I'm not sure, AoE2 taught me that camels always win against horses. And if you consider Rust to be OCaml's child (which is kind of true if you really stretch things), it seems like young camels win against young horses too.
      - shpongled 5y ago

shrubble 5y ago

Wasn't there a big brouhaha, many years ago, when Reddit switched from Lisp to Python?

Fiahil 5y ago

> However, in the meantime data science and machine learning continued to explode, and we found that customer interest in deploying machine learning models far outstripped demand for more conventional stream processing data algorithms. My experience is quite similar : I either end up working with clients who are already using Spark etc, or they don’t have a mature data engineering lab. In both cases, deploying a model to production is always the most challenging. I hope their MLOps strategy is better, than just binding to MLFlow. This tool is absolutely not ready for prime time, and plagued by bad product decisions.

Zababa 5y ago

This feels like the same pattern as Dark leaving OCaml for F#: https://blog.darklang.com/leaving-ocaml// https://blog.darklang.com/leaving-ocaml//. Ecosystem matters a lot these days. Outside of these two specific cases, I wonder if we're, as an industry, too afraid of writing this kind of stuff now I feel like it was done a lot before, and not at all these days. Sure, NIH syndrome is a fallacy, but having to write one library may not be so bad. I would be glad to hear about any experience with that.

wyager 5y ago
I used to work at an OCaml company and it wasn't nearly as much of an issue as one might predict. You can (it turns out) build a very successful business even if there aren't a lot of existing libraries, or if the language lacks certain basic features like native multithreading (same with Python of course). I don't have a great model for why this isn't devastatingly expensive, but it's probably some combination of * Most existing libraries are kind of bad anyway so you're not missing out much by not using them * If you write everything yourself you get system expertise "for free", and gaining expertise in something that already exists is hard * You can tailor everything to your business requirements * Writing libraries is "fun work" so it's easier to be productive while doing it
- Zababa 5y ago
  That's interesting, thanks for sharing your experience. Sometimes I wonder how much interesting experience I miss by mostly using things people already built. Sure, if you're just trying to get something done it's probably faster, but on the other hand spending more time gives you experience.
- pbiggar 5y ago
  I don't agree with this at all. While it's true that there are lots of bad libraries and many libraries are easy, you really isolate yourself from the broader ecosystem by doing this. Vendors and 3rd party solutions are now much harder to use, and when you use them you'll probably only use them at a surface level instead of using all the features. And some things are so mature and vast you don't have a chance of building them yourself. If what you are doing can be done well in very mature ecosystems like React or PyTorch, the effort to recreate them will dwarf the time spend on important work.

guu 5y ago

Last year they discussed why they didn’t choose rust (predicted 12-18 months to build a runtime that met their requirements) or erlang (performance too slow): https://corecursive.com/055-unproven-with-sean-allen/#considering-rust https://corecursive.com/055-unproven-with-sean-allen/#consid...

spooneybarger 5y ago
That's me. I discussed why we (I am no longer at Wallaroo) made the decision about 5 years ago to use Pony to build a product that is no longer what Wallaroo is selling.

ZeroCool2u 5y ago

The product Wallaroo is building sounds really interesting, (It's so appealing I honestly checked out their open positions), but it's really difficult to get a feel for what exactly it's like to use their product now. The description and outcome statistics make it sound _excellent_ and their technology stack makes their claims seem viable. However, and I say this as someone that is currently running an MLOps RFP, after looking at every page of their site and all their open job postings I have no idea what it's actually like to use their product.

aturley 5y ago
Thanks for checking us out. We're working improving our website and our explainer materials. Would you be interested in a quick conversation? I'd be happy to do a demo and get some feedback on ways we could improve the first impression. you can reach me at `andy at wallaroo dot ai`.

debarshri 5y ago

Just a thought. Rewriting the core platform from one language to another just takes huge resources for orgs who are in high growth path. It feels like a side mission, derailed from core value that is solving the problem statement. Rewrites are generally a red flag for me, especially early in an orgs history.

spooneybarger 5y ago
This article is about how Wallaroo when they started a NEW product chose a different language than what was used to write the original product. There's no rewrite here.

secondcoming 5y ago

> Our new Rust-based platform recently handled millions of inferences a second of a sophisticated ad-targeting model with a median latency of 5 milliseconds, and a yearly infrastructure cost of $130K. Were these run on CPUs or GPUs? How many of them? Last I looked at running Tensorflow models on CPUs it was really slow, so slow we had to abandon it.

btheshoe 5y ago

That font is so thin it's almost uncomfortable to read. I've never heard of Pony before - is it any good, or worth playing around with?

spooneybarger 5y ago
I'm a Pony core team member. I think its quite good in the most general sense, but "is it good for your problem" is entirely different. I think at worst, you would get something out of learning Pony (although that can take some time, it's one of the harder languages to learn). Feel free to stop by our Zulip and chat more as folks from the Pony community are rarely on HN, but are on the Zulip all the time. https://ponylang.zulipchat.com/ https://ponylang.zulipchat.com/

rackjack 5y ago

I just wish Pony would reach 1.0 already. I'm very hesitant to spend my time on a language that might change under me.

awb 5y ago

One of the hardest things at any company is recruiting and retaining top talent. I’m sure some devs would love to learn a new language at a new job, but many would see it as a career mistake. Your career income increases the more you master a certain discipline like a certain language. If you’re a jack of all trades (and adding Pony developer to your resume), you’re probably limiting your future income potential.

smallstepforman 5y ago

I evaluated Pony 4 years ago, and walked away due to 2 technical issues: - garbage collection - no mechanism for synchronous access to actors We ended up building a C++ actor model, with it's associated headaches. Yes, we still have race conditions and some developers invoke the function directly instead of using messages, and yes, sometimes we will grab a locking mechanism to do synchronous access, but at the end of the day, the performance of real code (embedded gaming system) is all that matters, and we did meet our performance goals. Having said that, I've given up looking for an actor programming language and started building my own. Essentially, C++ like with Actors and compiler validated tracking of resources across actors (so the compiler knows about locks and actors). Still working on the compiler (compiles to C++20) and maybe 18 months away from public reveal. But the initial batch of test apps look very nice (but without compiler validation of resource sharing at this point in time). Very terse code, and implementing a compiler that outputs C++ was much easier than I originally feared. And no, there are no forward declarations or header files, compilers are run on modern workstations that have the grunt to produce the necessary scaffolding (from the project dependency script).

devit 5y ago
Why not use Rust? It can do what you want.
- Karrot_Kream 5y ago
  +1 actix.rs is pretty well-known and gives you the benefit of a full programming language if you need to do other things. Rust should be really easy to onboard to if your devs already know C++ and while the compiler is a lot slower, there's enough posts online about Actix that you shouldn't have issues searching around.
- smallstepforman 5y ago
  With 21st century developer workstations having >= 8 cores, >= 2Ghz, >= 16Gb memory, it is ridiculous to have the scaffolding that Rust, Zig and other modern languages have. Source files dont live on their own and shouldn't be compiled as isolated units, they live in projects and need to be compiled as a batch. I'm stunned to see that so many developers are used to needing scaffolding, they just accept it as being necessary, but when you step back and think about it, it no longer makes any sense to require it. C and C++ get a pass here since they were designed in the 70/80's, but languages that have been designed from 2010 onwards must be ridiculed for targetting a PDP-11 era development machine where each compiler unit has limited access to resources. The end language I'm designing has smaller source files, however the compiler is a lot more complex since it auto generates the scaffolding (header files, forward declarations, etc). It's still a fair bit away, I just got the proof of concept compiler which outputs C++ working (and the compiler is written in the language itself, so I'm dog fooding the project).

eatonphil 5y ago

I was under the impression that Wallaroo was the (sole) organization behind Pony. But I guess that's not true the way this is written.

forgotpwd16 5y ago

Pony is a younger than Rust and has far smaller exposure. It isn't strange that it didn't worked out. Using the shiny new thing is rarely a good choice.

spooneybarger 5y ago
Technical issues were never the problem with the first product we built at Wallaroo. Pony met all of our technical needs for that product. We never got product market fit with it. Rightly, around the time I left the company, they pivoted and started a new product that they thought would have better product market fit and they started over with a new codebase as it is a rather different product than what we first built.
- forgotpwd16 5y ago
  Arguably the non-technical issues are equally important and, according to the essay, those were the reasons they ended up choosing Rust going forwards. >Pony [...] has a smaller community, and as a small startup we were better off not having to solve problems outside our core domain >wealth of available libraries >access to a large community that will guarantee ongoing future support >a wide variety of tools >much larger pool of engineers who are eager to learn and work in Rust, or who already have significant Rust experience >more resources available for learning Rust, and more opportunities for participation in conferences, etc. Didn't say that Pony was bad for some technical reason. It was/is bad choice for a startup to use due to being a relatively new, niche language.

mbrodersen 5y ago

It is absolutely batshit crazy to start a business with new but immature tools like Pony. Unless investors are throwing money at you and you don’t care about building a long term successful business. All developer tools take years to become solid and well supported. And most of them fail to get any major developer mindshare. If you think your unicorn snowflake project needs a new immature but oh so shiny programming language to be successful then you are simply wrong. Not even “likely to be wrong” but 100% guaranteed to be wrong.