3 ms·
Yes, I think you've basically summed it up very well! One of the reasons we use this pattern internally is because we have quite specific performance requireme
by Smaug123 7y ago
Yes, I think you've basically summed it up very well!
One of the reasons we use this pattern internally is because we have quite specific performance requirements which are not well-served by your standard language compiler/runtime. If we handle some of this compilation workload, we can make sure we emit constructs in the underlying runtime which have the right performance properties. We also want to make sure our users don't really need to think about this sort of low-level mucking around with performance; so we use the initial algebra to expose natural data-driven abstractions to them, which we then carefully manipulate into the right forms for the .NET runtime to have our desired performance properties.
This is an area where F# really shines. Its "computation expressions" make it really easy to construct DSLs embedded in F#. We offer our users this DSL, and we promise that anything written using this DSL will be appropriately fast and safe; but we do sometimes give them an escape hatch. By encouraging the user to stick to this heavily restricted DSL, we make it easier for them to write code that we guarantee will perform well. If the code doesn't perform well, that's a big problem, but crucially it's a problem for the devs to solve, not for the quant researchers to waste hours digging into.
Entirely separately from the above, we can also offer certain safety guarantees by restricting to our DSLs. One project in particular has involved taking something that previously existed but required a lot of manual procedural bookkeeping on the part of the user, and extracting the "intent" of the library in a purely data-oriented DSL. As long as the user sticks to our DSL, they don't need to consider how their computations are sequenced; we'll do that for them, in the process of converting from the DSL into the underlying system.
- 6gvONxR4sf7o 7y agoAre you effectively interpreting it, or are you compiling it to something else? I'm really curious how you do this in a performant way.
- Smaug123 7y agoWe have places where we ultimately emit IL (the bytecode of .NET). However, usually we just emit F#, restricted to certain constructs which are allocation-free and so forth. We aren't producing F# in the sense that we have to invoke the F# compiler ourselves, though. In the calculator example, given the description `Add (Negate (Const 5)) (Const 3)`, we might ultimately end up having assembled an F# function in memory like `fun () -> -5 + 3`; when this function is invoked, the value `-2` will be calculated. Ultimately we usually put together the functions in the usual way you make a function: by composition of smaller functions. The very simplest expressions like `Const 5` have a simple template like `fun () -> 5` which we can just directly produce; more complex expressions have to be interpreted recursively into F# function objects. I agree it's a bit hard to explain unless you're trying to solve a less trivial problem :(
- jhgb 7y ago> we have quite specific performance requirements which are not well-served by your standard language compiler/runtime. If we handle some of this compilation workload, we can make sure we emit constructs in the underlying runtime which have the right performance properties. Is it just me or does this sound a bit like Common Lisp's compiler macros (http://www.lispworks.com/documentation/HyperSpec/Body/03_bbaa.htm http://www.lispworks.com/documentation/HyperSpec/Body/03_bba...)?