JackHopkins

7 ms·

1. 31.▲
   by JackHopkins 3y ago

   Cheers!
2. 32.▲
   by JackHopkins 3y ago

   Cool! Thanks for sharing. What do you mean that Ts transformers aren’t supported by default? Is this like a runtime modification of types?
3. 33.▲
   by JackHopkins 3y ago

   Hey! There are 2 main similarities to Marvin, namely: (a) functions that act as APIs to the LLM backend, and (b) type coercion to ensure that the responses fit into the data model of your application. However, there are a couple of big addi
4. 34.▲
   by JackHopkins 3y ago

   Currently the distillation happens automatically in the background for all functions but we're aiming to implement ways for the user to be able to turn it off if they wish to keep using the teacher models. Good to know that this'd
5. 35.▲
   by JackHopkins 3y ago

   Great to know! We're working on extending MonkeyPatch to typescript, the work-in-progress repository can be found here https://github.com/monkeypatch/monkey-patch.ts We will keep you posted on when it'll be r
6. 36.▲
   by JackHopkins 3y ago

   Currently we distill the general GPT-4 down to function specific GPT3.5 turbo model using pseudo-labelling. The input-output pairs from the aligned few-shot GPT-4 are saved and this dataset is used to finetune a function-specific GPT3.5 mod
7. 37.▲
   by JackHopkins 3y ago

   Thanks! A big part of MonkeyPatch, which Langchain or OpenAI are lacking, is the model distillation aspect, which can reduce costs up to 10x and latency up to 6x in some of the tests we've been running. This means the more you use Monk
8. 38.▲
   by JackHopkins 3y ago

   Thanks! I find the enforced typed outputs and structured object creation from unstructured inputs very useful, for instance we created a use-case around creating structured support-ticket objects that could be processed in downstream applic
9. 39.▲
   by JackHopkins 3y ago

   The type constraints are indeed enforced but not by the tests but by the type-hints you give to the patched functions. The constraints and enforced structure are followed, there is also a repair feedback loop in place if the original LLM ou
10. 40.▲
    by JackHopkins 3y ago

    Great questions! The tests act as few-shot examples for the LLMs, which has been shown to guide the style and accuracy of model outputs and improve performance quite well. For instance we’ve seen accuracy go from <70% to 93%+ vs without
11. 41.▲
    Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python(github.com)

    95 points by JackHopkins 3y ago|71 comments