4 ms·

Defunctionalisation. Instead of writing code that does what you want to do, have it return a description of what you want to do. Then write an simple executor

Defunctionalisation. Instead of writing code that does what you want to do, have it return a description of what you want to do. Then write an simple executor for such descriptions. Why do this? Well it allows you to manipulate, test, store and inspect the description.

throw1234651234 4y ago
What? How?
- flipflip 4y ago
  You write a simple interpreter basicly. Instead of directly manipulating the data , you build an ast of how to manipulate the data then you execute it. It also allows for nifty optimizations later on.
- chriswarbo 4y ago
  Here's a "functionalised" API: GET https://example.com/users?postprocess=result.sort("signup-date").reverse This is easy to implement: just send the 'postprocess' parameter through 'eval()'. It's completely general, allowing the data to be sorted, filtered, transformed, etc. in any way the caller likes. It's also completely insecure, since it executes arbitrary code. It's also hard to refactor, since there's always a chance that some caller's postprocess parameter is relying on e.g. a particular variable name, etc. Here's a "defunctionalised" alternative: GET https://example.com/users?sort=signup-date&order=descending Instead of allowing arbitrary code, we only accept a description of what to do (sort by "signup-date" in descending order). This requires we implement all the logic ourselves, as well as parsing and branching-on the input to determine what was requested. Defunctionalised APIs are the norm on the Web (except for those who accept raw SQL...); whilst functionalised APIs are the norm inside applications. For example, `myList.map(myArbitraryFunction)`. Defunctionalising is a good way to break systems into modules, with the descriptions acting as the interface.
xyzal 4y ago
Would you please provide an example or direct towards some explanation? I can't seem to grasp what you mean exactly.
- assbuttbuttass 4y ago
  A simple example might be a function that makes an HTTP request to a server. You could separate out a function that builds the request from a function that sends the request to the server. Then it's easy to unit test that the request is what you expect, without coding up a realistic fake for the server.
- the_gipsy 4y ago
  You can't send a function/closure like data over the wire. You have to limit yourself to what kind of code you want executed in the future, but you can send or store it limitless.
- substation13 4y ago
  Suppose I have a procedure that plays a piano: const playTwinkleTwinkleLittleStar = (piano) => { piano.play('c'); piano.play('c'); piano.play('g'); piano.play('g'); piano.play('a'); piano.play('a'); piano.play('g'); }; Now I want to have every note played twice. How can I do that? Usual OOP solution is to have a wrapper on piano: const playItTwicePiano = (piano) => { return { play: (note) => { piano.play(note); piano.play(note); }, }; }; playTwinkleTwinkleLittleStar(playItTwicePiano(piano)); However, if we instead return a description of what to do, such manipulations compose far more easily: const twinkleTwinkleLittleStar = () => { return [ 'c', 'c', 'g', 'g', 'a', 'a', 'g' ]; }; const playItTwice = (sequence) => { const xs = []; for (const x of sequence) { xs.push(x); xs.push(x); } return xs; }; And it's easier to test: it('should start with c', () => { const xs = twinkleTwinkleLittleStar(); assert(xs[0] === 'c'); }); Wherever you are building a complex mock for testing consider defunctionalization instead.
  - b0afc375b5 4y ago
kalecserk 4y ago
I also recommend this! I usually use the error handling of the language of choice for that purpose and implement the entire functionality returning errors/throwing Exceptions for the not implemented parts, and asserting upon in on unit tests to implement each pending functionality. I write the description on the assert message itself. Edit: example use Psr\Http\Message\ResponseInterface; use Psr\Http\Message\ServerRequestInterface; class TransferHandler extends BaseHandler { public function __construct(Generator $faker, DataStorage $storage) { parent::__construct($faker, $storage); } public function handle(ServerRequestInterface $http_request): ResponseInterface { $resource_token = $http_request->getHeader('X-Resource-Token')[0]; $account_id = $this->storage->account_id_by_resource_token[$resource_token]; if (empty($account_id)) { return $this->responseInvalidResourceToken($http_request); } throw new \Exception('WIP -> store transaction data here for subsequent query, also issue notification'); } } With the following test: public function testTransfer() { $juno = JunoAPISimulator::initialize($this->faker); $repo = AccountCreationRepoResolver::resolve(); $account_id = $this->createAccountAndGetId( $this->merchant, fn(DigitalAccountRequestOperator $op) => $op->withProvider(DigitalAccountProviderSelector::JUNO()) ); try { Executor::execute(new IssueTransferByBankDetails( $account_id->toHex(), "BRL 12.42", $_ispb = '21018182', $_agencyNumber = '0001', $_accountNumber = '10000368021', $_accountType = 'CHECKING', )); $this->fail('should have thrown exception'); } catch (\Throwable $e) { $this->assertEquals('WIP -> store transaction data here for subsequent query, also issue notification', $e->getPrevious()->getMessage()); } }
pca006132 4y ago
Somehow reminds me of monad... btw defunctionalization usually means converting higher order functions into statically known function calls
- substation13 4y ago
  Haskell monadic IO is basically this pattern, yes. Term from this blog post: https://www.gresearch.co.uk/blog/article/defunctionalisation/ https://www.gresearch.co.uk/blog/article/defunctionalisation...
zvsmith 4y ago
So much this, really helps breakup complex tasks. Some examples off the top of my head: If you have a function handling user sign up it often needs to build and write to multiple db tables. Split this into two functions a pure one that outputs the entities, and a second which takes these and handles writing them to a db inside a transaction. First the code is super simple to read, but more importantly it's easy to unit test since you don't need to mock anything, just make a list of input combinations paired with a list of expected outputs. Spend more time testing your business logic, less time verifying that postgres is working. Transforming HTML into a pdf is trivial if you first iterate over the html and output a list of instructions that then get fed into a writer function one at a time, similar to how 3d printer instructions work. Anytime I've done analysis work on a state that gets updated by a complex series of events it's always been worthwhile to build a discrete handler for each event type, then feed the events into it like a woodchipper. Complex problem solved with simple functions and a case statement.
vitorbaptistaa 4y ago
+100 I first heard of this in the great Functional Core, Imperative Shell [1] screencast of Destroy All Software. It allows you to easily unit test your functional core (as they're just pure functions). [1] https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell https://www.destroyallsoftware.com/screencasts/catalog/funct...
marginalia_nu 4y ago
Yeah I love this pattern. I'm using it for the data loading for my search engine. Instead of having the code that processes website data insert the data into the database and index directly, it essentially writes a program to a file, which is interpreted later to actually create the desired effects. Means I can just grab some random subset of the search engine index, copy it over to my workstation, and load it up over there. It's also amazing for testing and inspection.
btschaegg 4y ago
I'll add this: If you have to deal with state (e.g.: say you write something akin to `git checkout` that should rearrange the file system): Combine this strategy with the question of "can I roll this step back?". I.e. for a checkout: - Make a temporary folder on the target disk (rollback: delete it) - Create new files for those you have to change (rollback: delete them) - Fill these files with their target content - Move old files away into the temp folder - Move new files into place - If everything worked, delete the leftovers (the only thing that can't be rolled back) Your disk fills up during checkout? Roll back! You're on Windows and some file is blocked by another process? Roll back! …etc. :)
swat535 4y ago
Can you give an example of this?

throw1234651234 4y ago

What? How?

flipflip 4y ago
You write a simple interpreter basicly. Instead of directly manipulating the data , you build an ast of how to manipulate the data then you execute it. It also allows for nifty optimizations later on.
chriswarbo 4y ago
Here's a "functionalised" API: GET https://example.com/users?postprocess=result.sort("signup-date").reverse This is easy to implement: just send the 'postprocess' parameter through 'eval()'. It's completely general, allowing the data to be sorted, filtered, transformed, etc. in any way the caller likes. It's also completely insecure, since it executes arbitrary code. It's also hard to refactor, since there's always a chance that some caller's postprocess parameter is relying on e.g. a particular variable name, etc. Here's a "defunctionalised" alternative: GET https://example.com/users?sort=signup-date&order=descending Instead of allowing arbitrary code, we only accept a description of what to do (sort by "signup-date" in descending order). This requires we implement all the logic ourselves, as well as parsing and branching-on the input to determine what was requested. Defunctionalised APIs are the norm on the Web (except for those who accept raw SQL...); whilst functionalised APIs are the norm inside applications. For example, `myList.map(myArbitraryFunction)`. Defunctionalising is a good way to break systems into modules, with the descriptions acting as the interface.

xyzal 4y ago

Would you please provide an example or direct towards some explanation? I can't seem to grasp what you mean exactly.

assbuttbuttass 4y ago
A simple example might be a function that makes an HTTP request to a server. You could separate out a function that builds the request from a function that sends the request to the server. Then it's easy to unit test that the request is what you expect, without coding up a realistic fake for the server.
the_gipsy 4y ago
You can't send a function/closure like data over the wire. You have to limit yourself to what kind of code you want executed in the future, but you can send or store it limitless.
substation13 4y ago
Suppose I have a procedure that plays a piano: const playTwinkleTwinkleLittleStar = (piano) => { piano.play('c'); piano.play('c'); piano.play('g'); piano.play('g'); piano.play('a'); piano.play('a'); piano.play('g'); }; Now I want to have every note played twice. How can I do that? Usual OOP solution is to have a wrapper on piano: const playItTwicePiano = (piano) => { return { play: (note) => { piano.play(note); piano.play(note); }, }; }; playTwinkleTwinkleLittleStar(playItTwicePiano(piano)); However, if we instead return a description of what to do, such manipulations compose far more easily: const twinkleTwinkleLittleStar = () => { return [ 'c', 'c', 'g', 'g', 'a', 'a', 'g' ]; }; const playItTwice = (sequence) => { const xs = []; for (const x of sequence) { xs.push(x); xs.push(x); } return xs; }; And it's easier to test: it('should start with c', () => { const xs = twinkleTwinkleLittleStar(); assert(xs[0] === 'c'); }); Wherever you are building a complex mock for testing consider defunctionalization instead.
- b0afc375b5 4y ago

kalecserk 4y ago

I also recommend this! I usually use the error handling of the language of choice for that purpose and implement the entire functionality returning errors/throwing Exceptions for the not implemented parts, and asserting upon in on unit tests to implement each pending functionality. I write the description on the assert message itself. Edit: example use Psr\Http\Message\ResponseInterface; use Psr\Http\Message\ServerRequestInterface; class TransferHandler extends BaseHandler { public function __construct(Generator $faker, DataStorage $storage) { parent::__construct($faker, $storage); } public function handle(ServerRequestInterface $http_request): ResponseInterface { $resource_token = $http_request->getHeader('X-Resource-Token')[0]; $account_id = $this->storage->account_id_by_resource_token[$resource_token]; if (empty($account_id)) { return $this->responseInvalidResourceToken($http_request); } throw new \Exception('WIP -> store transaction data here for subsequent query, also issue notification'); } } With the following test: public function testTransfer() { $juno = JunoAPISimulator::initialize($this->faker); $repo = AccountCreationRepoResolver::resolve(); $account_id = $this->createAccountAndGetId( $this->merchant, fn(DigitalAccountRequestOperator $op) => $op->withProvider(DigitalAccountProviderSelector::JUNO()) ); try { Executor::execute(new IssueTransferByBankDetails( $account_id->toHex(), "BRL 12.42", $_ispb = '21018182', $_agencyNumber = '0001', $_accountNumber = '10000368021', $_accountType = 'CHECKING', )); $this->fail('should have thrown exception'); } catch (\Throwable $e) { $this->assertEquals('WIP -> store transaction data here for subsequent query, also issue notification', $e->getPrevious()->getMessage()); } }

pca006132 4y ago

Somehow reminds me of monad... btw defunctionalization usually means converting higher order functions into statically known function calls

substation13 4y ago
Haskell monadic IO is basically this pattern, yes. Term from this blog post: https://www.gresearch.co.uk/blog/article/defunctionalisation/ https://www.gresearch.co.uk/blog/article/defunctionalisation...

zvsmith 4y ago

So much this, really helps breakup complex tasks. Some examples off the top of my head: If you have a function handling user sign up it often needs to build and write to multiple db tables. Split this into two functions a pure one that outputs the entities, and a second which takes these and handles writing them to a db inside a transaction. First the code is super simple to read, but more importantly it's easy to unit test since you don't need to mock anything, just make a list of input combinations paired with a list of expected outputs. Spend more time testing your business logic, less time verifying that postgres is working. Transforming HTML into a pdf is trivial if you first iterate over the html and output a list of instructions that then get fed into a writer function one at a time, similar to how 3d printer instructions work. Anytime I've done analysis work on a state that gets updated by a complex series of events it's always been worthwhile to build a discrete handler for each event type, then feed the events into it like a woodchipper. Complex problem solved with simple functions and a case statement.

vitorbaptistaa 4y ago

+100 I first heard of this in the great Functional Core, Imperative Shell [1] screencast of Destroy All Software. It allows you to easily unit test your functional core (as they're just pure functions). [1] https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell https://www.destroyallsoftware.com/screencasts/catalog/funct...

marginalia_nu 4y ago

Yeah I love this pattern. I'm using it for the data loading for my search engine. Instead of having the code that processes website data insert the data into the database and index directly, it essentially writes a program to a file, which is interpreted later to actually create the desired effects. Means I can just grab some random subset of the search engine index, copy it over to my workstation, and load it up over there. It's also amazing for testing and inspection.

btschaegg 4y ago

I'll add this: If you have to deal with state (e.g.: say you write something akin to `git checkout` that should rearrange the file system): Combine this strategy with the question of "can I roll this step back?". I.e. for a checkout: - Make a temporary folder on the target disk (rollback: delete it) - Create new files for those you have to change (rollback: delete them) - Fill these files with their target content - Move old files away into the temp folder - Move new files into place - If everything worked, delete the leftovers (the only thing that can't be rolled back) Your disk fills up during checkout? Roll back! You're on Windows and some file is blocked by another process? Roll back! …etc. :)

swat535 4y ago

Can you give an example of this?