37 ms·
I wonder why someone would equate "instance launch time" with "reliability"... I won't go as far as calling it "clickbait" but wouldn't some other noun ("startu
by dark-star 4y ago
I wonder why someone would equate "instance launch time" with "reliability"... I won't go as far as calling it "clickbait" but wouldn't some other noun ("startup performance is wildly different") have made more sense?
- xmonkee 4y agoGCP also had 84 errors compared to 1 for AWS
- danielmarkbruce 4y agoIf not a 4xx, what should they return for instance not available?
- eurasiantiger 4y ago503 service unavailable?
- dheera 4y agoUsing HTTP error codes for non-REST things is cringe. 503 would mean the IaaS API calls themselves are unavailable. Very different from the API working perfectly fine but the instances not being available.
- devmunchies 4y agoWhat? REST is just some API philosophy, its doesn't even have to be on top of HTTP. Why would you think HTTP status codes are made for REST? They are made for HTTP to describe the response of the resource you are requesting, and the AWS API uses HTTP so it makes sense to use HTTP status codes.
- eurasiantiger 4y ago
- sn0wf1re 4y agoThat would be confusing. The HTTP response code should not be conflated with the application's state.
- eurasiantiger 4y agoThere will come a moment in time when you realize exactly what you have stated here and why it is not a good mental palace to live in.
- yjftsjthsd-h 4y agoYeah, 4xx is client error, 5xx is server error.
- pdpi 4y agoYes, and trying to create duplicate resources is a client error.
- eurasiantiger 4y agoStill, 409 seems inappropriate, as it is meant to signal a version conflict, i.e. someone else changed something, and user tried to uplod a stale version. ”10.4.10 409 Conflict The request could not be completed due to a conflict with the current state of the resource. This code is only allowed in situations where it is expected that the user might be able to resolve the conflict and resubmit the request. The response body SHOULD include enough information for the user to recognize the source of the conflict. Ideally, the response entity would include enough information for the user or user agent to fix the problem; however, that might not be possible and is not required. Conflicts are most likely to occur in response to a PUT request. For example, if versioning were being used and the entity being PUT included changes to a resource which conflict with those made by an earlier (third-party) request, the server might use the 409 response to indicate that it can't complete the request. In this case, the response entity would likely contain a list of the differences between the two versions in a format defined by the response Content-Type.” Then again, perhaps it is the service itself making that state change.
- danielmarkbruce 4y agoIt's not the service that's unavailable. The resource isn't available. The service is running just fine.
- verdverm 4y agoGCP error messages will indicate if resources were not available, if you reached your quota, or if it was some other error. Tests like OP can differentiate these situations
- sheeshkebab 4y agoMaybe 1 reported. Not saying aws reliability is bad, but the number of various glitches that crop up in various aws services and not reflected on their status page is quite high.
- theamk 4y agothat was measured from API call return codes, not by looking at overall service status page Amazon is pretty good about this, if their API says machine is ready, it usually is.
- mcqueenjordan 4y agoErrors returned from APIs and the status page are completely separate topics in this context.
- antonvs 4y agoAnother comment on this thread pointed out they had a potential collision in their instance name generation which may have caused this. That would mean this was user error, not a reliability issue. AWS doesn’t require instance names to be unique.
- runeks 4y agoThey were almost exclusively user errors (HTTP 4xx). They are supposed to indicate that the API is being used incorrectly. Although, it seems the author couldn't find out why they occurred, which points to poor error messages and/or lacking documentation.
- chrismarlow9 4y agoI mean if you're talking about worst case systems you assume everything is gone except your infra code and backups. In that case your instance launch time would ultimately define what your downtime looks like assuming all else is equal. It does seem a little weird to define it that way but in a strict sense maybe not.
- santoshalper 4y agoI won't go so far as saying "you didn't read the article", but I think you missed something.
- mikewave 4y agoWell, if your system elastically uses GPU compute and needs to be able to spin up, run compute on a GPU, and spin down in a predictable amount of time to provide reasonable UX, launch time would definitely be a factor in terms of customer-perceived reliability.
- rco8786 4y agoSure but not anywhere remotely near clearing the bar to simply calling that “reliability”.
- VWWHFSfQ 4y agoI would still call it "reliability". If the instance takes too long to launch then it doesn't matter if it's "reliable" once it's running. It took too long to even get started.
- endisneigh 4y agothat's not what reliability means
- VWWHFSfQ 4y ago> that's not what reliability means What is your definition of reliability?
- endisneigh 4y agounfortunately cloud computing and marketing have conflated reliability, availability and fault tolerance so it's hard to give you a definition everyone would agree to, but in general I'd say reliability is referring to your ability to use the system without errors or significant decreases in throughput, such that it's not usable for the stated purpose. in other words, reliability is that it does what you expect it to. GCP does not have any particular guarantees around being able to spin up VMs fast, so its inability to do so wouldn't make it unreliable. it would be like me saying that you're unreliable for not doing something when you never said you were going to. if this were comparing Lambda vs Cloud Functions, who both have stated SLAs around cold start times, and there were significant discrepancies, sure.
- iLoveOncall 4y agoIt is clickbait, the real title should be "AWS vs. GCP on-demand provisioning of GPU resources performance is wildly different". That said, while I agree that launch time and provisioning error rate are not sufficient to define reliability, they are definitely a part of it.
- tl_donson 4y ago“ AWS vs. GCP on-demand provisioning of GPU resources performance is wildly different” yeah i guess it does make sense that one didn’t win the a/b test
- deleted 4y ago[deleted]
- lelandfe 4y ago> wildly different For this, I'd prefer a title that lets me draw my own conclusions. 84 errors out of 3000 doesn't sound awful to me...? But what do I know – maybe just give me the data: "1 in 3000 GPUs fail to spawn on AWS. GCP: 84" "Time to provision GPU with AWS: 11.4s. GCP: 42.6s" "GCP >4x avg. time to provision GPU than AWS" "Provisioning on GCP both slower and more error-prone than AWS"
- esrauch 4y ago84 of 3000 failed is only "one nine"
- deleted 4y ago[deleted]
- rmah 4y agoThey are talking about the reliability of AWS vs GCP. As a user of both, I'd categorize predictable startup times under reliability because if it took more than a minute or so, we'd consider it broken. I suspect many others would have even tighter constraints.
- thesuperbigfrog 4y ago"AWS encountered one valid launch error in these two weeks whereas GCP had 84." 84 times more launch errors seems like a valid definition for "less reliable".
- irjustin 4y agoI'll say it is valid to use reliability. If I depend on some performance metric, startup, speed, etc, my dependance on it equates to reliability. Not just on/off but the spectrum that it produces. If a CPU doesn't operate at its 2GHz setting 60% of the time, I would say that's not reliable. When my bus shows up on time only 40% of the time - I can't rely on that bus to get me where I need to go consistently. If the GPU took 1 hour to boot, but still booted, is it reliable? What about 1 year? At some point it tips over an "personal" metric of reliability. The comparison to AWS which consistently out-performs GCP, while not explicitly, implicitly turns that into a reliability metric by setting the AWS boot time as "the standard".
- RajT88 4y agoReliability is a fair term, with an asterix. It is a specific flavor of reliability: deployment or scaling or net-new or allocation or whatever you want to call it.
- thayne 4y agoWell, I mean it is measuring how reliably you can get a GPU instance. But it certainly isn't the overall reliability. And depending on your workflow, it might not even be a very interesting measure. I would be more interested in seeing a comparison of how long regular non-GPU instances can run without having to be rebooted, and maybe how long it takes to allocate a regular VM.
- hericium 4y agoCloud reliability is not the same as a reliability of already spawned VM. Here it's the possibility to launch new VMs to satisfy dynamic projects' needs. Cloud provider should allow you to scale-up in a predictable way. When it doesn't - it can be called unreliable. Also, "unreliable" is basically a synonym for "Google" these days.
- DonHopkins 4y agoLet me unreliable that for you.
- ReptileMan 4y agoTo be fair their search is so crap lately, throwing the dice is not the worst option in the world to find a result that will be actually useful.