3 ms·

Thanks for letting me know. Overall, we use them for high availability and non mission critical tasks anyways but now that you say this, it gives me more confid

Thanks for letting me know. Overall, we use them for high availability and non mission critical tasks anyways but now that you say this, it gives me more confidence in recommending it. It was strange, it only affected one of the two GKE cluster (both had preemptive nodes). I tried debugging and found that it wasn’t able to even ping any external ips (which is why I thought maybe preemtible nodes also might have networking issues). There weren’t any notices about general networking outages within the last +\- 48 hours (I subscribe to the RSS GCP outages via slack). I ended up manually rebuilding the nodes (non preemtible nodes temp) and it went away. As soon as healthchecks were failing, I created a debug container and kubectl exec -it bashed in and saw that it wasn’t able to connect to the internet. It could ping/telnet internal IP databases just fine. It does warn me that it’s in beta so, again maybe you’re working through some things but thought you should know in case other people have experienced this.

dirtydroog 7y ago
Did your instances have external IPs? We had connectivity issues when using NAT due to it dropping SYN packets. We just gave everything that needed to talk to external IPs their own external IPs.
- boulos 7y ago
  Glad you found a workaround, but please file a support ticket for this. If you don’t have support: (a) consider it and/or (b) send me an email with your info (contact in profile) and I’ll file a bug.
  - dirtydroog 7y ago
    We actually did have a dedicated Googler who oversaw our move from AWS to GCP. I asked him about it but never got a reply. Anyway, yes. It caused us problems. Attempts to connect to machines outside of GCP would fail repeatedly due to the NA dropping the SYN. This would mean our connect() call would timeout repeatedly. There was an ICMP response I could see with tcpdump: ICMP host aaa.bbb.ccc.ddd unreachable - admin prohibited filter, length 68

dirtydroog 7y ago

Did your instances have external IPs? We had connectivity issues when using NAT due to it dropping SYN packets. We just gave everything that needed to talk to external IPs their own external IPs.

boulos 7y ago
Glad you found a workaround, but please file a support ticket for this. If you don’t have support: (a) consider it and/or (b) send me an email with your info (contact in profile) and I’ll file a bug.
- dirtydroog 7y ago
  We actually did have a dedicated Googler who oversaw our move from AWS to GCP. I asked him about it but never got a reply. Anyway, yes. It caused us problems. Attempts to connect to machines outside of GCP would fail repeatedly due to the NA dropping the SYN. This would mean our connect() call would timeout repeatedly. There was an ICMP response I could see with tcpdump: ICMP host aaa.bbb.ccc.ddd unreachable - admin prohibited filter, length 68