4 ms·

Author here: I'm happy to elaborate on any of this, except where I've signed NDAs about specific projects. Also an obligatory "we are hiring" on behalf of G-Re

Author here: I'm happy to elaborate on any of this, except where I've signed NDAs about specific projects. Also an obligatory "we are hiring" on behalf of G-Research; feel free to get in touch at patrick.stevens@gresearch.co.uk or patrick+hn@patrickstevens.co.uk if you're interested in quant finance research/development in central London.

sriku 7y ago
This is a technique worth talking/writing about a bit more. Sometime ago, Jimmy Koppel also wrote somewhat extensively about defunctionalization and refunctionalization as "refactoring" techniques - http://www.pathsensitive.com/2019/07/the-best-refactoring-youve-never-heard.html http://www.pathsensitive.com/2019/07/the-best-refactoring-yo...
- Smaug123 7y ago
  That's a cool talk which I'd definitely recommend to people! Is there something in particular that you'd like to see "a bit more"? Are you imagining e.g. a more extended walkthrough of how to realise the benefits of defunctionalisation on a real-life program, or some more theoretical background, or some of how it's used in compilers, or anything in particular?
  - sriku 7y ago
    More real-life cases. The classic cases like compilers are covered enough, but I think we could use more examples in a wider range of domains. For those who use it, I think it feels like second nature and perhaps even "it's not a big deal really", but the duality is critical for building flexible systems. It's only a feeling I have at this point though. I can offer a small example to start with. Steller [1] - is a sound modeling library that's written primarily as higher order functions. It however exposes a "specFromJSON" function [2] that offers a serialization through (informal) defunctionalization. [1] https://github.com/srikumarks/steller https://github.com/srikumarks/steller [2] https://github.com/srikumarks/steller/blob/master/src/steller/scheduler.js#L1156 https://github.com/srikumarks/steller/blob/master/src/stelle...
ToJans 7y ago
Isn't this strongly related to monads?
- Smaug123 7y ago
  Insofar as everything is related to monads, yes ;) In fact you don't need to use monads at all here. I've used this pattern on merely applicative non-monadic initial algebras before; in fact, the article's "arithmetic expressions" example isn't what I would call "monadic", because it doesn't readily admit Bind as currently phrased. Worse, the "writeSmallOddToFile" example isn't even generic at all, so I don't think there's a reasonable sense in which it could even be said to be functorial.
- dack 7y ago
  Somewhat. A monad allows for representing an effectful program as a data structure, but it's not very "introspectable" by itself (which is what this article is advocating). For example, you can pass a monad around, but you can't tell what it will do. All you can really do with a monad is execute it and find out (similar to how a regular function is "opaque" - you can run it with various inputs but can't know ahead of time what it plans to do with them). However, if you were to use "free monads", you get closer to what the author is talking about. Free monads are a special implementation of monad that don't actually _run_ any effects - they just build up a data benign structure that can be interpreted later (possibly as an effect). To do defunctionalization, you don't need monads though - you just need to define data that represents operations. Monads can just provide convenient syntax to do this while looking like a sequential program.
  - kkdaemas 7y ago
    You could draw an analogy that monads are to text as defunctionalization is to an AST.
hhmc 7y ago
G-Research should come with the health warning of just how litigous and paranoid around IP theft they are. My understanding is that you can't have a personal phone whilst at work, you're weighed on the way in and out, and they sent one of their former quants to jail for several years. One might suspect that the periodic renames (De Putron, Glouster Research, G Research) are mostly a tactic to distance themselves from the negative image. 1. https://www.bloomberg.com/news/features/2018-11-19/the-triple-jeopardy-of-ke-xu-a-chinese-hedge-fund-quant https://www.bloomberg.com/news/features/2018-11-19/the-tripl...
- hermitdev 7y ago
  Its quite common in the quantitative finance world. Unscrupulous people steal a model, try and peddle it to a competitor. Thing is most models only work if noone (or only a few others) are doing the same. Usually when approached, most firms stay above board and report it to the person's employer. Citadel, for instance sued a former employee for steeling a model (he emailed the source for a model to his personal email. He was sued in federal at 8am on a Monday, fired at noon. Criminal charges came months later. Tried dispose g of the evidence by tossing hard drives in the Chicago River. Dive teams were involved to recover the drives. But, Citadel already had all they needed because they monitored all outgoing and internal communication, including MITMing SSL email services.
  - hhmc 7y ago
    I'm fully aware of the sensitivites of the quant finance world, but there are plenty of high quality workplaces out there that don't have the extremely invasive approach to security that e.g. g research do.
- senderista 7y ago
  So it looks like the whole purpose of this company is to make one (very unpleasant) person richer. What a reason to get up in the morning.
infogulch 7y ago
The second definition of module Expr appears to have a typo: | TwoArg of OneArgFunction * Expr * Expr Should this instead be: | TwoArg of TwoArgFunction * Expr * Expr
- Smaug123 7y ago
  You're quite right, thanks. I'll try and get this fixed up tomorrow.
  - Jtsummers 7y ago
    The "Testability" example also seems to have a typo in the definition of describe. Specifically the first error case, it uses the constructor Odd, when Even is the one defined (and the error case itself).

sriku 7y ago

This is a technique worth talking/writing about a bit more. Sometime ago, Jimmy Koppel also wrote somewhat extensively about defunctionalization and refunctionalization as "refactoring" techniques - http://www.pathsensitive.com/2019/07/the-best-refactoring-youve-never-heard.html http://www.pathsensitive.com/2019/07/the-best-refactoring-yo...

Smaug123 7y ago
That's a cool talk which I'd definitely recommend to people! Is there something in particular that you'd like to see "a bit more"? Are you imagining e.g. a more extended walkthrough of how to realise the benefits of defunctionalisation on a real-life program, or some more theoretical background, or some of how it's used in compilers, or anything in particular?
- sriku 7y ago
  More real-life cases. The classic cases like compilers are covered enough, but I think we could use more examples in a wider range of domains. For those who use it, I think it feels like second nature and perhaps even "it's not a big deal really", but the duality is critical for building flexible systems. It's only a feeling I have at this point though. I can offer a small example to start with. Steller [1] - is a sound modeling library that's written primarily as higher order functions. It however exposes a "specFromJSON" function [2] that offers a serialization through (informal) defunctionalization. [1] https://github.com/srikumarks/steller https://github.com/srikumarks/steller [2] https://github.com/srikumarks/steller/blob/master/src/steller/scheduler.js#L1156 https://github.com/srikumarks/steller/blob/master/src/stelle...

ToJans 7y ago

Isn't this strongly related to monads?

Smaug123 7y ago
Insofar as everything is related to monads, yes ;) In fact you don't need to use monads at all here. I've used this pattern on merely applicative non-monadic initial algebras before; in fact, the article's "arithmetic expressions" example isn't what I would call "monadic", because it doesn't readily admit Bind as currently phrased. Worse, the "writeSmallOddToFile" example isn't even generic at all, so I don't think there's a reasonable sense in which it could even be said to be functorial.
dack 7y ago
Somewhat. A monad allows for representing an effectful program as a data structure, but it's not very "introspectable" by itself (which is what this article is advocating). For example, you can pass a monad around, but you can't tell what it will do. All you can really do with a monad is execute it and find out (similar to how a regular function is "opaque" - you can run it with various inputs but can't know ahead of time what it plans to do with them). However, if you were to use "free monads", you get closer to what the author is talking about. Free monads are a special implementation of monad that don't actually _run_ any effects - they just build up a data benign structure that can be interpreted later (possibly as an effect). To do defunctionalization, you don't need monads though - you just need to define data that represents operations. Monads can just provide convenient syntax to do this while looking like a sequential program.
- kkdaemas 7y ago
  You could draw an analogy that monads are to text as defunctionalization is to an AST.

hhmc 7y ago

G-Research should come with the health warning of just how litigous and paranoid around IP theft they are. My understanding is that you can't have a personal phone whilst at work, you're weighed on the way in and out, and they sent one of their former quants to jail for several years. One might suspect that the periodic renames (De Putron, Glouster Research, G Research) are mostly a tactic to distance themselves from the negative image. 1. https://www.bloomberg.com/news/features/2018-11-19/the-triple-jeopardy-of-ke-xu-a-chinese-hedge-fund-quant https://www.bloomberg.com/news/features/2018-11-19/the-tripl...

hermitdev 7y ago
Its quite common in the quantitative finance world. Unscrupulous people steal a model, try and peddle it to a competitor. Thing is most models only work if noone (or only a few others) are doing the same. Usually when approached, most firms stay above board and report it to the person's employer. Citadel, for instance sued a former employee for steeling a model (he emailed the source for a model to his personal email. He was sued in federal at 8am on a Monday, fired at noon. Criminal charges came months later. Tried dispose g of the evidence by tossing hard drives in the Chicago River. Dive teams were involved to recover the drives. But, Citadel already had all they needed because they monitored all outgoing and internal communication, including MITMing SSL email services.
- hhmc 7y ago
  I'm fully aware of the sensitivites of the quant finance world, but there are plenty of high quality workplaces out there that don't have the extremely invasive approach to security that e.g. g research do.
senderista 7y ago
So it looks like the whole purpose of this company is to make one (very unpleasant) person richer. What a reason to get up in the morning.

infogulch 7y ago

The second definition of module Expr appears to have a typo: | TwoArg of OneArgFunction * Expr * Expr Should this instead be: | TwoArg of TwoArgFunction * Expr * Expr

Smaug123 7y ago
You're quite right, thanks. I'll try and get this fixed up tomorrow.
- Jtsummers 7y ago
  The "Testability" example also seems to have a typo in the definition of describe. Specifically the first error case, it uses the constructor Odd, when Even is the one defined (and the error case itself).