4 ms·

I do want to make this as easy as possible. As part of that effort I am experimenting[1] with making filing an ICE report be as easy as "open ticket and attach

I do want to make this as easy as possible. As part of that effort I am experimenting[1] with making filing an ICE report be as easy as "open ticket and attach this file". We do require a GitHub login because that is where we keep our issue tracker, and given the user base size I would have to expect that any alternative service would require some kind of login to cut down on spam. [1]: https://github.com/rust-lang/rust/pull/114128 https://github.com/rust-lang/rust/pull/114128

yjftsjthsd-h 3y ago
> as easy as "open ticket and attach this file" Does the "open a ticket" step expand into "create an issue and then fill out an elaborate questionnaire"? For instance, https://github.com/rust-lang/rust/blob/master/.github/ISSUE_TEMPLATE/ice.yaml https://github.com/rust-lang/rust/blob/master/.github/ISSUE_... isn't... bad for what it is, but it's definitely asking for enough work that it falls back into "yeah no maybe someday when I'm not trying to get my code to work" territory. (Asking to clarify; I think you might be suggesting to reduce to the form just being "got a ICE? paste your error here and hit submit" which would be good)
- estebank 3y ago
  The only mandatory fields in that form are the rustc version (that's printed as part of the error) and the backtrace. The file being generated includes both of those things. > Asking to clarify; I think you might be suggesting to reduce to the form just being "got a ICE? paste your error here and hit submit" which would be good I would do that once the new ICE storage mechanism is exercised enough for everyone to be confident in its behavior and enabled in the stable channel.
yorwba 3y ago
I've never had to log in to send a Firefox or Thunderbird crash report to Mozilla. A look at what they're doing might be fruitful.
amluto 3y ago
I wonder if splitting the telemetry upload into a separate command could make sense. So rustc could panic and say “Run 'rustup report' to report this issue”, and the user could send the report at their option. (Or ‘rustup report 12345' where 12345 identifies this panic.) This would be the ultimate opt-in - the user has to specifically request each report.
- Groxx 3y ago
  Honestly this sounds pretty good to me. Unlikely to be accidentally triggered, easy to explain and run, lets you do it later rather than having to save things before continuing, and provides a few ways to preview the data before submitting (e.g. temporarily run a webserver to locally view and possibly clean up the report, which can easily include large files, and just click submit there to send it with CORS... or bundle it into a file that can be dropped somewhere if that doesn't actually work).
Yoric 3y ago
The Element Web Matrix client does that. I'm convinced that we're losing 99% of bug reports by making it so complicated. That being said, the Rust crowd is very different, so it's definitely worth a try!
davidhyde 3y ago
< "We do require a GitHub login because that is where we keep our issue tracker" The problem with this is that you are de-anonymizing the user when you ask for this and the user may decide that proceeding is not worth it. Also, they may find that logging in is a pain. Think of those companies who ask you to login before you can unsubscribe from their news letter. It is unpleasant for the user who is already upset. I think that the GP made some really good points and may well have adequately explained the following point you made. Re-quoting: < "Whatever the reason, nightly and beta failed to be an effective canary." I think its worth the effort to figure out the reason why nightly is not an effective canary and if it's as simple as the user requiring a github login and not wanting to go that far then that should be fairly simple to solve. I have a suggestion that may or may not be feasible. Logins are one way to cut down on spam but not the only way. You don't have to have users directly posting to your issue tracker, you could use a website. The concerns you have about telemetry are absolutely on point so this service could simply be an anonymous dumping ground for copy-paste ICE compiler output that anyone, including bots could use. No automation from the user's machine, no telemetry, no metrics. The service would be responsible for despamming / deduplicating the submissions and posting to the github issue tracker using an account not associated with the original poster. Anyway, you guys do superb work and really think that articles like this are important. Thanks for posting it.
- LightFog 3y ago
  ‘Whatever the reason, nightly and beta failed to be an effective canary’ - Im also curious about this? It is typical to try to move quality ‘left’ - which would imply understanding this issue and fixing it. But the focus seems to be on adding more logging - which will by definition have less impact on the quality of public releases than improving the quality of beta/canary releases will.
JohnFen 3y ago
Why require them to open a ticket, though? Just allow an easy way for people to upload the file and be done with it. You should take care of whatever bookkeeping your process requires yourself. I have no github account, for instance, and there's no chance that I'd create one just to provide this data.
Groxx 3y ago
Frankly I think GitHub Issues is possibly the biggest barrier, and not just in UX terms, and it astonishes me that it's often the only (visible) option for so many major projects. I've watched multiple coworkers run across issues in other languages and systems (we don't use Rust, sadly), and when they get to the GitHub report they stop cold. Submitting it there means making it literally public, and relatively highly visible. Can they do that with [internal software X]? Do they need to hide their internal username, which is all over the report? Are they publicly acting in the company's name by sticking it in a world-visible-and-google-indexed location with their work GitHub account? A good number don't even have a separate personal account - GitHub is big, but extremely far from ubiquitous. These are not hypothetical examples either, I've had newer engs ask me about ^ multiple of those. I've seen more-senior-than-me people asking our legal department similar questions before clicking submit, and mostly getting non-answers two days later. I see almost every case including manual scrubbing of dozens or hundreds of lines of output to try to avoid these questions. With the increasing public pile-ons in GitHub lately, this is all likely to worsen. All of that is coming from highly motivated people trying to improve things for themselves and possibly dozens or hundreds of others (else they just try to work around the crash). And they frequently stop. Meanwhile everyone just clicks "report and clear" in Intellij error reports.

yjftsjthsd-h 3y ago

> as easy as "open ticket and attach this file" Does the "open a ticket" step expand into "create an issue and then fill out an elaborate questionnaire"? For instance, https://github.com/rust-lang/rust/blob/master/.github/ISSUE_TEMPLATE/ice.yaml https://github.com/rust-lang/rust/blob/master/.github/ISSUE_... isn't... bad for what it is, but it's definitely asking for enough work that it falls back into "yeah no maybe someday when I'm not trying to get my code to work" territory. (Asking to clarify; I think you might be suggesting to reduce to the form just being "got a ICE? paste your error here and hit submit" which would be good)

estebank 3y ago
The only mandatory fields in that form are the rustc version (that's printed as part of the error) and the backtrace. The file being generated includes both of those things. > Asking to clarify; I think you might be suggesting to reduce to the form just being "got a ICE? paste your error here and hit submit" which would be good I would do that once the new ICE storage mechanism is exercised enough for everyone to be confident in its behavior and enabled in the stable channel.

yorwba 3y ago

I've never had to log in to send a Firefox or Thunderbird crash report to Mozilla. A look at what they're doing might be fruitful.

amluto 3y ago

I wonder if splitting the telemetry upload into a separate command could make sense. So rustc could panic and say “Run 'rustup report' to report this issue”, and the user could send the report at their option. (Or ‘rustup report 12345' where 12345 identifies this panic.) This would be the ultimate opt-in - the user has to specifically request each report.

Groxx 3y ago
Honestly this sounds pretty good to me. Unlikely to be accidentally triggered, easy to explain and run, lets you do it later rather than having to save things before continuing, and provides a few ways to preview the data before submitting (e.g. temporarily run a webserver to locally view and possibly clean up the report, which can easily include large files, and just click submit there to send it with CORS... or bundle it into a file that can be dropped somewhere if that doesn't actually work).

Yoric 3y ago

The Element Web Matrix client does that. I'm convinced that we're losing 99% of bug reports by making it so complicated. That being said, the Rust crowd is very different, so it's definitely worth a try!

davidhyde 3y ago

< "We do require a GitHub login because that is where we keep our issue tracker" The problem with this is that you are de-anonymizing the user when you ask for this and the user may decide that proceeding is not worth it. Also, they may find that logging in is a pain. Think of those companies who ask you to login before you can unsubscribe from their news letter. It is unpleasant for the user who is already upset. I think that the GP made some really good points and may well have adequately explained the following point you made. Re-quoting: < "Whatever the reason, nightly and beta failed to be an effective canary." I think its worth the effort to figure out the reason why nightly is not an effective canary and if it's as simple as the user requiring a github login and not wanting to go that far then that should be fairly simple to solve. I have a suggestion that may or may not be feasible. Logins are one way to cut down on spam but not the only way. You don't have to have users directly posting to your issue tracker, you could use a website. The concerns you have about telemetry are absolutely on point so this service could simply be an anonymous dumping ground for copy-paste ICE compiler output that anyone, including bots could use. No automation from the user's machine, no telemetry, no metrics. The service would be responsible for despamming / deduplicating the submissions and posting to the github issue tracker using an account not associated with the original poster. Anyway, you guys do superb work and really think that articles like this are important. Thanks for posting it.

LightFog 3y ago
‘Whatever the reason, nightly and beta failed to be an effective canary’ - Im also curious about this? It is typical to try to move quality ‘left’ - which would imply understanding this issue and fixing it. But the focus seems to be on adding more logging - which will by definition have less impact on the quality of public releases than improving the quality of beta/canary releases will.

JohnFen 3y ago

Why require them to open a ticket, though? Just allow an easy way for people to upload the file and be done with it. You should take care of whatever bookkeeping your process requires yourself. I have no github account, for instance, and there's no chance that I'd create one just to provide this data.

Groxx 3y ago

Frankly I think GitHub Issues is possibly the biggest barrier, and not just in UX terms, and it astonishes me that it's often the only (visible) option for so many major projects. I've watched multiple coworkers run across issues in other languages and systems (we don't use Rust, sadly), and when they get to the GitHub report they stop cold. Submitting it there means making it literally public, and relatively highly visible. Can they do that with [internal software X]? Do they need to hide their internal username, which is all over the report? Are they publicly acting in the company's name by sticking it in a world-visible-and-google-indexed location with their work GitHub account? A good number don't even have a separate personal account - GitHub is big, but extremely far from ubiquitous. These are not hypothetical examples either, I've had newer engs ask me about ^ multiple of those. I've seen more-senior-than-me people asking our legal department similar questions before clicking submit, and mostly getting non-answers two days later. I see almost every case including manual scrubbing of dozens or hundreds of lines of output to try to avoid these questions. With the increasing public pile-ons in GitHub lately, this is all likely to worsen. All of that is coming from highly motivated people trying to improve things for themselves and possibly dozens or hundreds of others (else they just try to work around the crash). And they frequently stop. Meanwhile everyone just clicks "report and clear" in Intellij error reports.