FrasiertheLion

5 ms·

1.
Architecting Secure Prompt Caching(tinfoil.sh)
1 points by FrasiertheLion 3mo ago|1 comments
2.
by FrasiertheLion 4mo ago
That's basically what we built at Tinfoil. We run open source models inside secure enclaves (also using Intel TDX/AMD SEV-SNP + NVIDIA Confidential Computing). All the code running inside the enclave is open source and the client
3.
by FrasiertheLion 5mo ago
Another option is verifiably private inference with open source models running inside secure enclaves on the cloud (using NVIDIA confidential computing), and the enclave code is open source and verified via remote attestation upon connectio
4.
by FrasiertheLion 5mo ago
Overall I'm bullish on standardized local APIs that ship with the browser or platform. Far more tractable than expecting end users to stand up their own local model instances, though r/LocalLLaMA is a fantastic community to follow
5.
by FrasiertheLion 5mo ago
Very reasonable if you have the resources to run it locally and certainly the best option. But we created Tinfoil because not everyone has that capability especially when it comes to larger models, and it still doesn’t solve for the situati
6.
by FrasiertheLion 5mo ago
Unfortunately we don’t support crypto payments at this time as we use Stripe. We try to add models selectively as we have to be mindful about our compute allocation. Is there a specific reason why you need those two models (and our models s
7.
by FrasiertheLion 5mo ago
Yes we do, but the load balancer also runs inside the enclave and is attested: https://github.com/tinfoilsh/confidential-model-router In turn, that attests the model enclaves, for instance, see https://githu
8.
by FrasiertheLion 5mo ago
Yeah everything is open source if you’re good at reversing. Models are increasingly capable of converting binaries into source, and excellent at implementing systems when there’s a finite and constrained end state to validate against, which
9.
by FrasiertheLion 5mo ago
This was largely true before. But AI reduces the cost of comprehension and finding vulnerabilities en-masse to zero, so this no longer holds, and I’m increasingly convinced that hiding in noise and complexity is no longer a valid strategy.
10.
by FrasiertheLion 5mo ago
Oh that's quite interesting and hasn't been my experience with regular backend code specifically with respect to tool calling. However that could be because the tool calling format in vllm for Deepseek v4 was broken until a few da
11.
by FrasiertheLion 5mo ago
Because V4 doesn't even beat Kimi K2.6 and GLM 5.1, which have been out longer. It's only talked about as much as it is because it's Deepseek and R1 was the first open source reasoning model. V4 isn't even multimodal (un
12.
by FrasiertheLion 5mo ago
You can use Tinfoil for inference, which lets you use the model in the cloud while getting similar privacy as running locally: https://tinfoil.sh/inference . Disclaimer I'm the cofounder. This works by running the model
13.
by FrasiertheLion 5mo ago
Have you given GLM 5.1 or Kimi K2.6 a shot for coding? They outperform Deepseek v4 pro.
14.
by FrasiertheLion 5mo ago
This is the canonical use case for Tinfoil: https://tinfoil.sh/inference . It provides verifiably private AI inference with frontier open source models: https://docs.tinfoil.sh/models/overview Disclaime
15.
by FrasiertheLion 5mo ago
It's 80 minutes, not 80 hours.
16.
The Closing of the Frontier(tanyaverma.sh)
4 points by FrasiertheLion 6mo ago|0 comments
17.
by FrasiertheLion 6mo ago
Yes they absolutely care and have been doing serious work to migrate PKI to PQC. This was the first of several articles coming out of Google: https://blog.google/innovation-and-ai/technology/safety-secu... And the
18.
by FrasiertheLion 6mo ago
It's unfortunate that we're past the point where all quantum computing progress is public. Between this and the unbearable secrecy of AI labs, balkanization of knowledge is in full force.
19.
by FrasiertheLion 7mo ago
Tanya from the Tinfoil team that worked on the confidential computing and security substrate here. Also around to answer any questions!
20.
by FrasiertheLion 7mo ago
We don't have reproducible builds because we attest the full OS image that we run, which is the Ubuntu image. Unfortunately bit-by-bit reproducible binaries for OS images is kind of an unsolved problem, because it requires the hundreds
21.
by FrasiertheLion 7mo ago
Enclaves have a property that allows the hardware to compute a measurement (a cryptographic hash) of everything running inside it, such as the firmware, system software such as the operating system and drivers, the application code, the sec
22.
by FrasiertheLion 7mo ago
The model is running in a secure enclave that spans the GPU using NVIDIA Confidential Computing: https://www.nvidia.com/en-us/data-center/solutions/confident... . The connection is encrypted with a key that is
23.
by FrasiertheLion 7mo ago
Arguably this is less useful for consumer hardware in the first place. This is mostly useful when I don’t trust the service provider with my data but still need to use their services (casting my vote, encrypted inference, and so forth)
24.
by FrasiertheLion 7mo ago
Most people outside of a narrow set of cryptography engineers are unfamiliar with the term anonymous credentials, while age and identity verification are two privacy-invasive requirements that are being heavily discussed and rapidly being w
25.
Privacy-preserving age and identity verification via anonymous credentials(blog.cryptographyengineering.com)
90 points by FrasiertheLion 7mo ago|61 comments
26.
by FrasiertheLion 7mo ago
AI has normalized single 9's of availability, even for non-AI companies such as Github that have to rapidly adapt to AI aided scaleups in patterns of use. Understandably, because GPU capacity is pre-allocated months to years in advance
27.
by FrasiertheLion 7mo ago
Recent paper by Nicholas Carlinini and others really showcases how little it takes to deanonymize users across platforms with LLMs: https://arxiv.org/abs/2602.16800
28.
by FrasiertheLion 8mo ago
Yes, it is a TLS certificate generated by the enclave on boot (the code responsible for doing this is open source and the attestation is also included in the certificate so you can check that this is exactly what’s happening). We go into mo
29.
by FrasiertheLion 8mo ago
jashulma above has a great link: https://news.ycombinator.com/item?id=47105315
30.
by FrasiertheLion 8mo ago
The disk isn’t client owned, but anyone can run modelwrap on any device and reproduce the root measurement that is being attested against.

More ›