FrasiertheLion

5 ms·

31.
by FrasiertheLion 8mo ago
When the enclave boots, two things happen: 1. An HPKE ( https://www.rfc-editor.org/rfc/rfc9180.html ) key is generated. This is the key that encrypts communication to the model. 2. The enclave is provisioned a certifica
32.
by FrasiertheLion 8mo ago
Yes, we use Intel TDX/AMD SEV-SNP with H200/B200 GPUs configured to run in Nvidia Confidential Computing mode
33.
by FrasiertheLion 8mo ago
There’s a few components that are necessary to make it work: 1. The provider open sources the code running in the enclave and pins the measurement to a transparency log such as Sigstore 2. On each connection, the client SDK fetches the meas
34.
by FrasiertheLion 8mo ago
The committed weights are open source and pinned to a transparency log, along with the full system image running in the enclave. At runtime, the client SDK (also open source: https://docs.tinfoil.sh/sdk/overview ) fetch
35.
by FrasiertheLion 8mo ago
The verification is not happening locally only. The client SDKs fetch the measurement of the weights (+ system software, inference engine) that are pinned to Sigstore, then grabs the same measurement (aka remote attestation of the full, pub
36.
How an inference provider can prove they're not serving a quantized model(tinfoil.sh)
69 points by FrasiertheLion 8mo ago|48 comments
37.
Google Trust Services is having an outage(status.pki.goog)
3 points by FrasiertheLion 8mo ago|1 comments
38.
by FrasiertheLion 10mo ago
Try elevenlabs
39.
by FrasiertheLion 11mo ago
I would argue personal blogs are back and Substack is the medium of choice this time around
40.
by FrasiertheLion 1y ago
Ha we didn't think of that, thanks for the tip!
41.
by FrasiertheLion 1y ago
Ollama does heavily quantize models and has a very short context window by default, but this has not been my experience with unquantized, full context versions of Llama3.3 70B and particularly, Deepseek R1, and that is reflected in the benc
42.
by FrasiertheLion 1y ago
Yeah not the case for FHE. But yes, not practically viable. We would be happy to switch as soon as it is.
43.
by FrasiertheLion 1y ago
We’re already using vllm as our inference server for our standard models. We can run whatever inference server for custom deployments.
44.
by FrasiertheLion 1y ago
Aye comrade, shot you an email :)
45.
by FrasiertheLion 1y ago
That's the best part, you don't. You only need to trust NVIDIA and AMD/Intel. Modulo difficult to mount physical attacks and side channels, which we wrote more about here: https://tinfoil.sh/blog/2025-05
46.
by FrasiertheLion 1y ago
Yes, in the US right now. We don't run our own datacenters, though we sometimes consider it in a moment of frustration when the provider is not able to get the correct hardware configuration and firmware versions. Currently renting bar
47.
by FrasiertheLion 1y ago
That's exactly our plan for Defcon next year as Nate just mentioned: https://news.ycombinator.com/item?id=44000103 But making it a public competition is a fantastic idea.
48.
by FrasiertheLion 1y ago
We're not competing with Gemini or OpenAI or the big cloud providers. For instance, Google is partnering with NVIDIA to ship Gemini on-prem to regulated industries in a CC environment to protect their model weights as well as for addit
49.
by FrasiertheLion 1y ago
>Since you rightly open-sourced the code (AGPL) is there anything stopping the cloud vendors from running and selling access to their own instances of your server-side magic? Sure they can do that. Despite being open source, CC-mode on G
50.
by FrasiertheLion 1y ago
Bedrock has strong contractual guarantees, but it's still only a legal contract and runs on AWS infra. This is certainly okay for many use cases, we're trying to build for users who want verifiable privacy guarantees beyond legal
51.
by FrasiertheLion 1y ago
Ah we don't have support@tinfoil.sh set up yet. Can you try contact@tinfoil.sh?
52.
by FrasiertheLion 1y ago
Yes, we have a customer who is indeed interested in having a frozen client for their app, which we're making possible. We currently have not frozen our client because we're in the early days and want to be able to iterate quickly
53.
by FrasiertheLion 1y ago
Yeah, totally agree with you. We would love to use FHE as soon as it's practical. And if you have the money and infra expertise to deploy air gapped LLMs locally, you should absolutely do that. We're trying to do the best we can w
54.
by FrasiertheLion 1y ago
Yes, excited to connect, scheduled a call! We used Nitro back in December when we were prototyping but moved to NVIDIA CC because we wanted to support LLMs.
55.
by FrasiertheLion 1y ago
CPU-based TEEs (AWS Nitro Enclaves, AMD SEV, Intel TDX) have been around for a few years, but aren’t widely used because they are more akin to primitives than fully baked security solutions. We are trying to make this as user friendly and s
56.
by FrasiertheLion 1y ago
>You're not terminating the TLS connection from the client anywhere besides the enclave? Yes. >How do you load balance or front end all of this effectively? We don't, atleast not yet. That's why all our model endpoints
57.
by FrasiertheLion 1y ago
We actually run into the rate limit issue often particularly while spinning up new enclaves while debugging. We plan on moving to HPKE: https://www.rfc-editor.org/rfc/rfc9180.html over the next couple months. This will
58.
by FrasiertheLion 1y ago
Not yet, we're about one week away from SOC2, will pursue HIPAA which is arguably easier next.
59.
Launch HN: Tinfoil (YC P25): Verifiable Privacy for Cloud AI
146 points by FrasiertheLion 1y ago|100 comments
60.
by FrasiertheLion 2y ago
Hey, it's Tanya (co-founder of Tinfoil) here! Happy to answer any questions.

More ›