7 ms·

There is a very simple way to get them to stop sending the .5 qps that is described as a ddos. The linked bug show another user successfully saying "please opt

There is a very simple way to get them to stop sending the .5 qps that is described as a ddos. The linked bug show another user successfully saying "please opt me out", and Google building the feature to do that in a week. Drew has for some reason chosen not to ask for an opt out, even though it appears trivial and would probably be fixed by the weekend if he asked for it.

Rantenki 4y ago
Calling it .5 QPS to downplay the severity is willfully ignoring the complaint in the article. It's not just a query, it's a full Git clone of the entire repo with it's history. That's a huge difference.
- tptacek 4y ago
  That's what a "Q" means in this context. It's a git hosting service.
  - CaliforniaKarl 4y ago
    True, though I don't think a "Q" from a client doing a fetch involves as much work as a "Q" from a client doing a full clone. It seems like this Git hosting service's workload-per-Q has increased due to the Go proxy change, which (IMO) is a cause for concern.
mig39 4y ago
I think he's been banned from the issue tracker.
- joshuamorton 4y ago
  Possibly, but he was actively participating in https://github.com/golang/go/issues/44577 https://github.com/golang/go/issues/44577 up until the week it was fixed. If that's the root of the issue though, most of the article ("no one will get promoted for prioritizing that at Google.", "the go team has not prioritized it", etc.) is wrong. They may not have addressed the issue in the precise way he wanted, but I also think it's rather unreasonable to expect someone to not only address your issue, but address it in the precise way you suggest. That's rarely globally optimal and feels rather entitled.
  - 8organicbits 4y ago
    I think the problem with the proposed solution (https://github.com/golang/go/issues/44577#issuecomment-857200479 https://github.com/golang/go/issues/44577#issuecomment-85720...) is that the refresh is used to keep the cache from getting too stale. Would it be reasonable for golang modules on sourcehub to be served from a stale cache? The problem is that the refresh does a full clone which is too heavyweight. The other user who accepted that proposed fix had a single module that had a single user, so they can tolerate some weird caching behavior for their toy project.
    - joshuamorton 4y ago
      If I'm reading correctly, it would not be stale, but be directly proxied, which would mean that a popular module could result in a higher overall qps (but this is also true if people clone directly from sourcehut). On the other hand, Drew's suggestion would actually result in stale caches. There comes a point (and it isn't a particularly far-off point) where disabling this would result in significantly increased traffic to sourcehut. That's a problem, but it's a problem with sourcehut's scalability.
      - 8organicbits 4y ago
        > This may impact the freshness of your domain's data which users receive from our servers, since we need to have some caching on our end to prevent too frequent fetches. They mention that requests still are served from the cache when refresh is disabled.
  - cycomanic 4y ago
    So the solution is that everyone who does not want that behavior should request that they turn of refresh traffic for their domain? How long would it take until they would complain about people spamming their issue tracker if everyone who self-hosts a go module would start requesting this? Maybe they should request 500 times a day because that's what the proxy is doing.
    - 4y ago

Rantenki 4y ago

Calling it .5 QPS to downplay the severity is willfully ignoring the complaint in the article. It's not just a query, it's a full Git clone of the entire repo with it's history. That's a huge difference.

tptacek 4y ago
That's what a "Q" means in this context. It's a git hosting service.
- CaliforniaKarl 4y ago
  True, though I don't think a "Q" from a client doing a fetch involves as much work as a "Q" from a client doing a full clone. It seems like this Git hosting service's workload-per-Q has increased due to the Go proxy change, which (IMO) is a cause for concern.

mig39 4y ago

I think he's been banned from the issue tracker.

joshuamorton 4y ago
Possibly, but he was actively participating in https://github.com/golang/go/issues/44577 https://github.com/golang/go/issues/44577 up until the week it was fixed. If that's the root of the issue though, most of the article ("no one will get promoted for prioritizing that at Google.", "the go team has not prioritized it", etc.) is wrong. They may not have addressed the issue in the precise way he wanted, but I also think it's rather unreasonable to expect someone to not only address your issue, but address it in the precise way you suggest. That's rarely globally optimal and feels rather entitled.
- 8organicbits 4y ago
  I think the problem with the proposed solution (https://github.com/golang/go/issues/44577#issuecomment-857200479 https://github.com/golang/go/issues/44577#issuecomment-85720...) is that the refresh is used to keep the cache from getting too stale. Would it be reasonable for golang modules on sourcehub to be served from a stale cache? The problem is that the refresh does a full clone which is too heavyweight. The other user who accepted that proposed fix had a single module that had a single user, so they can tolerate some weird caching behavior for their toy project.
  - joshuamorton 4y ago
    If I'm reading correctly, it would not be stale, but be directly proxied, which would mean that a popular module could result in a higher overall qps (but this is also true if people clone directly from sourcehut). On the other hand, Drew's suggestion would actually result in stale caches. There comes a point (and it isn't a particularly far-off point) where disabling this would result in significantly increased traffic to sourcehut. That's a problem, but it's a problem with sourcehut's scalability.
    - 8organicbits 4y ago
      > This may impact the freshness of your domain's data which users receive from our servers, since we need to have some caching on our end to prevent too frequent fetches. They mention that requests still are served from the cache when refresh is disabled.
- cycomanic 4y ago
  So the solution is that everyone who does not want that behavior should request that they turn of refresh traffic for their domain? How long would it take until they would complain about people spamming their issue tracker if everyone who self-hosts a go module would start requesting this? Maybe they should request 500 times a day because that's what the proxy is doing.
  - 4y ago