3 ms·
How an inference provider can prove they're not serving a quantized model
- 45dsilicon 8mo ago[dead]
- arcanemachiner 8mo agoCall me an old fuddy-duddy, but my faith in the quality of your reporting really fell through the floor when I saw that the first image showed Spongebob Squarepants swearing at the worst-performing numbers. EDUT: I read through the article, and it's a little over my head, but I'm intrigued. Does this actually work?
- exceptione 8mo agoThe idea is that you run a workload at a model provider, that might cheat on you by altering the model they offer, right? So how does this help? If the provider wants to cheat (they apparently do), wouldn't they be able to swap the modelwrap container, or maybe even do some shenanigans with the filesystem? I am ignorant about this ecosystem, so I might be missing something obvious.
- FrasiertheLion 8mo agoThe committed weights are open source and pinned to a transparency log, along with the full system image running in the enclave. At runtime, the client SDK (also open source: https://docs.tinfoil.sh/sdk/overview https://docs.tinfoil.sh/sdk/overview) fetches the pinned measurement from Sigstore, and compares it to the attestation from the running enclave, and checks that they’re equal. This previous blog explains it in more detail: https://tinfoil.sh/blog/2025-01-13-how-tinfoil-builds-trust https://tinfoil.sh/blog/2025-01-13-how-tinfoil-builds-trust
- rhodey 8mo agoIn my opinion this is very well written Two comments so far suggesting otherwise and I guess idk what their deal is Attestation is taking off
- deleted 8mo ago[deleted]
- viraptor 8mo agoThe title here seems very different from the post. All that verification happens locally only. There's no remote validation at any point. So I'm not sure what's the reason to even apply this check. If you're running the model yourself, you know what you're downloading and can check the hash once for transfer problems. Then you can do different things for preventing storage bitrot. But you're not proving anything to your users this way. You'd need to run a full, public system image with known attestation keys and return some kind of signed response with every request to do that. Which is not impossible, but the remote part seems to be completely missing from the description.
- FrasiertheLion 8mo agoThe verification is not happening locally only. The client SDKs fetch the measurement of the weights (+ system software, inference engine) that are pinned to Sigstore, then grabs the same measurement (aka remote attestation of the full, public system image) from the running enclave, and checks that the two are exactly equal. Our previous blog explains this in more detail: https://tinfoil.sh/blog/2025-01-13-how-tinfoil-builds-trust https://tinfoil.sh/blog/2025-01-13-how-tinfoil-builds-trust Sorry it wasn’t clear from the post!
- arboles 8mo agoWhat prevents the provider from sending to the client an attestation of hardware state and actually running another?
- julesdrean 8mo agoThe provider cannot chose the attestation that is sent, the hardware assembles the attestation through mechanisms that it cannot control. That why it's called "trusted hardware" technology, you only need to trust the hardware (how it was implemented), and you don't need to trust the provider operating it.
- FrasiertheLion 8mo agoWhen the enclave boots, two things happen: 1. An HPKE (https://www.rfc-editor.org/rfc/rfc9180.html https://www.rfc-editor.org/rfc/rfc9180.html ) key is generated. This is the key that encrypts communication to the model. 2. The enclave is provisioned a certificate The certificate is embedded with the HPKE key accessible only inside the enclave. The code for all this is open source and part of the measurement that is being checked against by the client. So if the provider attempts to send a different attestation or even route to a different enclave, this client side check would fail.
- wongarsu 8mo agoI'm somehow more convinced by the method shown in the introduction of the article: run a number of evals across model providers, see how they compare. This also catches all other configuration changes an inference provider can make, like KV-cache quantization. And it's easy to understand, talk about, and the threat model is fairly clear (be wary of fixed answers to your benchmark if you're really distrustful) Of course conceptually attestation is neat and wastes less compute with repeated benchmarks. It definitely has its place
- Aurornis 8mo agoThis comes up so frequent that I’ve seen at least 3-4 different websites running daily benchmarks on providers and plotting their performance. The last one I bookmarked has already disappeared. I think they’re generally vibe coded by developers who think they’re going to prove something but then realize it’s expensive to spend that money on tokens every day. They also use limited subsets of big benchmarks because to keep costs down, which increases the noise of the results. The last time someone linked to one of the sites claiming a decline in quality looked like a noisy mostly flat graph that someone had put a regression line on that was very slightly sloping downward.
- hleszek 8mo agoWhy not allow the user to provide the seed used for the generation. That way at least we can detect if the model has changed if the same prompt with the same seed suddenly gives a new answer (assuming they don't cache answers), you could compare different providers which supposedly use the same model, and if the model is open-weight you could even compare yourself on your own hardware or on rented gpus.
- bthornbury 8mo agoAFAIK seed determinism can't really be relied upon between two machines, maybe not even between two different gpus.
- whatsupdog 8mo agoThat doesn't seem correct. It's just matrix multiplications at the end. Doesn't matter if it's a different computer, GPU or even math on a napkin. Same seed, input and weights should give the same output. Please correct me if I'm wrong.
- jashulma 8mo agohttps://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ https://thinkingmachines.ai/blog/defeating-nondeterminism-in... A nice write up explaining how it’s not as simple as it sounds
- measurablefunc 8mo agoYou're assuming consistent hardware & software profiles. The way these things work at scale is essentially a compiler/instruction scheduling problem where you can think of different CPU/GPU combinations as the pipelines for what is basically a data center scale computer. The function graph is broken up into parts, compiled for different hardware profiles w/ different kernels, & then deployed & stitched together to maximize hardware utilization while minimizing cost. Service providers are not doing this b/c they want to but b/c they want to be profitable so every hardware cycle that is not used for querying or optimization is basically wasted money. You'll never get agreement from any major companies on your proposal b/c that would mean they'd have to provide a real SLA for all of their customers & they'll never agree to that.
- LoganDark 8mo agoI don't understand what stops an inference provider from giving you a hash of whatever they want. None of this proves that's what they're running, it only proves they know the correct answer. I can know the correct answer all I want, and then just do something different.
- FrasiertheLion 8mo agoThere’s a few components that are necessary to make it work: 1. The provider open sources the code running in the enclave and pins the measurement to a transparency log such as Sigstore 2. On each connection, the client SDK fetches the measurement of the code actually running (through a process known as remote attestation) 3. The client checks that the measurement that the provider claimed to be running exactly matches the one fetched at runtime. We explain this more in a previous blog: https://tinfoil.sh/blog/2025-01-13-how-tinfoil-builds-trust https://tinfoil.sh/blog/2025-01-13-how-tinfoil-builds-trust
- LoganDark 8mo agoWhat enclave are you using? Is it hardware-backed? Edit: I found https://github.com/tinfoilsh/cvmimage https://github.com/tinfoilsh/cvmimage which says AMD SEV-SNP / Intel TDX, which seems almost trustworthy.
- FrasiertheLion 8mo agoYes, we use Intel TDX/AMD SEV-SNP with H200/B200 GPUs configured to run in Nvidia Confidential Computing mode
- LoganDark 8mo agoI would be interested to see Apple Silicon in the future, given its much stronger isolation and integrity guarantees. But that is an entirely different tech stack.
- 8mo ago
- bthornbury 8mo agoIs modelwrap running on arbitrary clients? I'm not following the whole post, but how are you able to maintain confidence in client-owned hardware/disks following the secure model the method seems to depdend on?
- FrasiertheLion 8mo agoThe disk isn’t client owned, but anyone can run modelwrap on any device and reproduce the root measurement that is being attested against.
- jMyles 8mo agoRelated but distinct: Is there an ELI5 about determinism in inference? In other words, when will the same prompt lead to the same output, and when not? And why not?
- FrasiertheLion 8mo agojashulma above has a great link: https://news.ycombinator.com/item?id=47105315 https://news.ycombinator.com/item?id=47105315
- measurablefunc 8mo agoEven if you reduce all the non-determinism you still will not get consistent results b/c of floating point rounding & instruction scheduling in the GPU. There is no way to guarantee that the GPU pipelines will execute your instructions exactly in the order you want it to be executed b/c GPUs are now essentially equivalent to sufficiently smart compilers & perform all sorts of clever instruction re-ordering behind the scenes. Expecting complete reproducibility at scale is a pipe dream.
- cmrx64 8mo agohttps://hellas.ai https://hellas.ai is building out their category theoretic compiler and protocol for solving this issue
- tripplyons 8mo agoZKML is a very exciting emerging field, but the math is no where near efficient enough to prove an inference result for an LLM yet. They are probably just trying to sell their crypto token.
- robrenaud 8mo agoPlease serve well quantized models. If you can get 99 percent of the quality for 50 percent of the cost, that is most times a good tradeoff.