3 ms·

Would be interesting if someone could detail the approach to decode the pre-post processing steps before it enters the model, and how to find the correct input

Would be interesting if someone could detail the approach to decode the pre-post processing steps before it enters the model, and how to find the correct input encoding.

refulgentis 2y ago
Boils down to "use Frida to find the arguments to the TensorFlow call beyond the model file" Key here is, a binary model is just a bag-of-floats with primitively typed inputs and outputs. It's ~impossible to write up more than what's here because either: A) you understand reverse engineering and model basics, and thus the current content is clear you'd use Frida to figure out how the arguments are passed to TensorFlow or B) you don't understand this is a binary reverse engineering problem, even when shown Frida. If more content was provided, you'd see it as specific to a particular problem. Which it has to be. You'd also need a walkthrough by hand about batching, tokenization, so on and so forth, too much for a write up, and it'd be too confusing to follow for another model. TL;Dr a request for more content is asking for a reverse engineering article to give you a full education on modal inference
- deleted 2y ago
  [deleted]
- littlestymaar 2y ago
  > TL;Dr a request for more content is asking for a reverse engineering article to give you a full education on modal inference I don't understand what you mean: I have no clue about anything related to reverse engineering, but I ported the mistral tokenizer to Rust and also wrote a basic CPU Llama training and inference implementation in Rust, so I definitely wouldn't need an intro to model inference…
  - refulgentis 2y ago
    You're also not the person I'm replying to, nor do you appear in any of this comment chain, so I've definitely not implied you need an intro to inference, so I'm even more confused than you :)
    - littlestymaar 2y ago
      I share the sentiment of the person you're responding to, and I didn't understand your response, that's it.
- TeMPOraL 2y ago
  > It's ~impossible to write up more than what's here Except you just did - or at least you wrote an outline for it, which is 80% of the value already.
  - refulgentis 2y ago
    The more impolite version of this basically says "If you can't figure out you're supposed to also use Frida to check the other arguments, you have no business trying." I agree, though, wrote a more polite version.

Would be interesting if someone could detail the approach to decode the pre-post processing steps before it enters the model, and how to find the correct input encoding.

refulgentis 2y ago

Boils down to "use Frida to find the arguments to the TensorFlow call beyond the model file" Key here is, a binary model is just a bag-of-floats with primitively typed inputs and outputs. It's ~impossible to write up more than what's here because either: A) you understand reverse engineering and model basics, and thus the current content is clear you'd use Frida to figure out how the arguments are passed to TensorFlow or B) you don't understand this is a binary reverse engineering problem, even when shown Frida. If more content was provided, you'd see it as specific to a particular problem. Which it has to be. You'd also need a walkthrough by hand about batching, tokenization, so on and so forth, too much for a write up, and it'd be too confusing to follow for another model. TL;Dr a request for more content is asking for a reverse engineering article to give you a full education on modal inference

deleted 2y ago
[deleted]
littlestymaar 2y ago
> TL;Dr a request for more content is asking for a reverse engineering article to give you a full education on modal inference I don't understand what you mean: I have no clue about anything related to reverse engineering, but I ported the mistral tokenizer to Rust and also wrote a basic CPU Llama training and inference implementation in Rust, so I definitely wouldn't need an intro to model inference…
- refulgentis 2y ago
  You're also not the person I'm replying to, nor do you appear in any of this comment chain, so I've definitely not implied you need an intro to inference, so I'm even more confused than you :)
  - littlestymaar 2y ago
    I share the sentiment of the person you're responding to, and I didn't understand your response, that's it.
TeMPOraL 2y ago
> It's ~impossible to write up more than what's here Except you just did - or at least you wrote an outline for it, which is 80% of the value already.
- refulgentis 2y ago
  The more impolite version of this basically says "If you can't figure out you're supposed to also use Frida to check the other arguments, you have no business trying." I agree, though, wrote a more polite version.