ekzhang

5 ms·

31.
by ekzhang 2y ago
Yeah, you hit the nail on the head. We considered NAT64 as well and looked at some implementations including eBPF-based ones like Cilium. Glad to know that IPv6-only is working well for you. "In a perfect world…" :)
32.
by ekzhang 2y ago
Yep! This is something we have internal tests for haha, you have good instincts that it can be tricky. Here's an example of using that for multi-GPU training https://modal.com/docs/examples/llm-finetuning
33.
by ekzhang 2y ago
Neat, thanks for sharing! Glad to know we're in good company here.
34.
by ekzhang 2y ago
Thanks. We did check out Tailscale, but they didn't quite have what we were looking for: some high-availability custom component that plugs into a low-level container runtime. (Which makes sense, it's pretty different from their i
35.
by ekzhang 2y ago
We don't use Kubernetes to run user workloads, we do use gVisor. We don't use MIG (multi-instance GPU) or MPS. If you run a container on Modal using N GPUs, you get the entire N GPUs. If you'd like to learn more, you can chec
36.
by ekzhang 2y ago
Yeah, great question. This came up at the beginning of design. A lot of our customers specifically needed IPv4 whitelisting. For example, MongoDB Atlas (a very popular database vendor) only supports IPv4. https://www.mongodb.com&
37.
by ekzhang 2y ago
I mentioned this in another comment thread, but we use gVisor to enforce isolation. https://gvisor.dev/users/ It's also used by Google Kubernetes Engine, OpenAI, and Cloudflare among others to run untrusted code.
38.
by ekzhang 2y ago
That makes sense, mTLS is great. Some services like Google Cloud SQL are really good about support for it. https://cloud.google.com/sql/docs/mysql/configure-ssl-instan... It's not quite a zero-trust solu
39.
by ekzhang 2y ago
What would you suggest as an alternative?
40.
by ekzhang 2y ago
Thanks for sharing. I'm interested in seeing what a global control plane might look like, seems like authentication might be tricky to get right! Controlling our worker environment (like `net.ipv4.conf.all.rp_filter` sysctl) is a big h
41.
by ekzhang 2y ago
We use gVisor! It's an open-source application security sandbox spun off from Google. We work with the gVisor team to get the features we need (notably GPUs / CUDA support) and also help test gVisor upstream https://gvi
42.
by ekzhang 2y ago
Hi! This is a blog post sharing some low-level Linux networking we're doing at Modal with WireGuard. As a serverless platform we hit a bit of a tricky tradeoff: we run multi-tenant user workloads on machines around the world, and each
43.
Static IPs for Serverless Containers(modal.com)
125 points by ekzhang 2y ago|66 comments
44.
The Color of Noise (2014)(caseymuratori.com)
80 points by ekzhang 2y ago|12 comments
45.
Scaling Automatic Neuron Description (Describing Every Neuron in Llama 3)(transluce.org)
8 points by ekzhang 2y ago|1 comments
46.
by ekzhang 2y ago
This is really neat! It looks like you implemented the parser and encoder entirely in Python, how does that compare to an approach where you load an extension module in C or Rust? (Possibly with the actual V8 source)
47.
Fast Container Filesystems on macOS(orbstack.dev)
3 points by ekzhang 2y ago|0 comments
48.
Best Practices for Speeding Up Your Web Site (2006)(developer.yahoo.com)
2 points by ekzhang 2y ago|0 comments
49.
Lord of the io_uring: A guide to Linux io_uring programming(unixism.net)
3 points by ekzhang 3y ago|0 comments
50.
by ekzhang 3y ago
(Author) Modal tackles how to make FaaS work, but for actual _function calls_, and also with containers that have much higher resource caps (see article: 3 CPUs vs 64 CPUs, or 10 GB RAM vs 336 GB RAM). EC2 isn't the same compute shape.
51.
by ekzhang 3y ago
(Author) HTTP is the de facto protocol / "narrow waist" of the Internet. To prove the point further, as other commenters mention, gRPC is also HTTP.
52.
by ekzhang 3y ago
(Author) Yes, that's exactly right
53.
by ekzhang 3y ago
(Author) Thanks for the feedback! Yes, there's still a network load balancer involved — we debated whether we should include the last section of the post but decided it's still worth writing out. The idea is that the load balancer
54.
by ekzhang 3y ago
(Author) Yes, it’s in the page linked in the section — and the maximum memory is 10240 MB, so that’s 5.8 vCPU < 3 physical CPUs. Good point about 2 Mbps vs 2 MBps, I’ll update that. Forgive me for the typo!
55.
Lambda on hard mode: Inside Modal's web infrastructure(modal.com)
3 points by ekzhang 3y ago|0 comments
56.
by ekzhang 3y ago
We made a small web app for SDXL Lightning, running your own Python code on GPUs. It generates images in real time. We know there was a fal.ai post yesterday, and that garnered a lot of interest. We also made this quick demo yesterday but d
57.
Show HN: Deploy your own real-time SDXL Lightning API(potatoes.ai)
2 points by ekzhang 3y ago|1 comments
58.
by ekzhang 3y ago
I'm not sure about this criticism — the script is written to be as simple as possible and also has a comment about this if you do inspect it as you suggest. # This is a short script to install the latest version of the sshx binar
59.
by ekzhang 3y ago
I'm not sure honestly, if you could raise an issue I'd appreciate it. I didn't go out of my way to do this; it seems to be some default of SvelteKit or the server code that I don't understand?
60.
by ekzhang 3y ago
Thanks, yes as you suggested, I already responded to their specific points and explained my rationale for the security model in detail.

More ›