So, in the blocker meeting today we discussed having specific criteria for the functionality of automated bug handling tools (currently, that would be abrt and setroubleshoot, and I think KDE has its own tool similar to abrt as well).
I'd like to propose the following two criteria:
* Automated bug handling tools intended to be present in each build should be present and functional to the point of generating valid error reports and storing them. (Alpha)
* Automated bug handling tools present in each build must be able to generate valid bug reports and report them to the appropriate bug tracking tool. (Beta)
I don't think we need Alpha to ship with reporting capabilities intact, it's something we can fix post-Alpha release and not lose much, since people will be able to submit stored reports once the bug is fixed. Or is this again too complex and we should just go with the more stringent requirement at Alpha stage?
Why would we have to make it submit reports now? Unless the steps needed to implement this are very few, there are plenty of other things that have a higher priority right now, no?
John Dulaney
Subject: Release criteria proposal: automated bug checking tools functionality From: awilliam@redhat.com To: test@lists.fedoraproject.org Date: Fri, 30 Jul 2010 15:29:00 -0700
I don't think we need Alpha to ship with reporting capabilities intact, it's something we can fix post-Alpha release and not lose much, since people will be able to submit stored reports once the bug is fixed. Or is this again too complex and we should just go with the more stringent requirement at Alpha stage? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net
On Fri, 2010-07-30 at 21:14 -0400, John Dulaney wrote:
Why would we have to make it submit reports now? Unless the steps needed to implement this are very few, there are plenty of other things that have a higher priority right now, no?
The tools should always be capable of submitting reports. When they can't, it's a bug. There's no special work that needs to happen to enable it.
Having reporting working is a reasonably high priority because the whole point of having pre-releases is to catch bugs, and the point of automated bug reporting tools is to make catching and reporting bugs easier. Writing tools for that purpose and then not having them work in pre-release phase would be quite a waste :)
On Fri, 2010-07-30 at 15:29 -0700, Adam Williamson wrote:
I'd like to propose the following two criteria:
- Automated bug handling tools intended to be present in each build
should be present and functional to the point of generating valid error reports and storing them. (Alpha)
- Automated bug handling tools present in each build must be able to
generate valid bug reports and report them to the appropriate bug tracking tool. (Beta)
I don't think we need Alpha to ship with reporting capabilities intact, it's something we can fix post-Alpha release and not lose much, since people will be able to submit stored reports once the bug is fixed. Or is this again too complex and we should just go with the more stringent requirement at Alpha stage?
Apologies for the delay. I like the above proposed additions, and their recommended milestone.
I was about to suggest combining the first criteria with the existing Alpha criteria "The installer must be able to report failures to Bugzilla, with appropriate information included". I like combining them from a KISS standpoint. However, I think there is value in requiring that the installer be able to submit bug reports directly into bugzilla for the Alpha, and all subsequent milestones. That's the first contact most testers will have with Fedora, and to me, it's tremendously valuable to have it as easy as possible to receive good bug reports (including failure logs etc...). In summary: In addition to your proposed Alpha addition, keep the existing Alpha installer bug reporting criteria as well.
I tweaked the Alpha criteria slightly to add a bit more detail around expectations. Thoughts?
- Automated bug handling tools intended to be present in each build
must be able to detect, notify the user and generate valid error reports for manual submission to the appropriate bug tracking tool. (Alpha)
Thanks, James
On Mon, 30 Aug 2010 08:25:50 -0400 James Laska wrote:
Alpha criteria "The installer must be able to report failures to Bugzilla, with appropriate information included".
I'm always defeated by that when it asks for my redhat bugzilla password which is not stored in my brain, but in an encrypted file in another partition of the same system I'm currently installing on :-). It would be nice if it could save up the bugzilla somewhere and send me mail about providing a good password later when I can find it.
-
Adam Williamson -
James Laska -
John Dulaney -
Tom Horsley