Hi, everyone. So we partly used the proposed new nice-to-have bug tracking system during the F14 Beta process, and it seemed to go well. In a quick burst of airport productivity, I've quickly written up a bunch of proposed new wiki pages and modifications to existing ones to document the nice-to-have process (and, incidentally, extend documentation of the blocker process, since we don't seem to have much of it beyond the blocker meeting SOP right now). All the pages can be found here:
https://fedoraproject.org/wiki/Category:NTH_adjustment_drafts
it should be pretty obvious which are new and which are modifications of existing pages. I hope this is mostly straightforward and non-controversial.
A quick five-minute summary is that we will now have (in fact, we already do) trackers for nice-to-have bugs for Alpha, Beta and Final releases as well as trackers for blocker bugs. Bugs on these NTH lists will be given priority after blocker bugs for QA, devel and releng work for releases. Fixes for these bugs - and *only* these bugs, if a fix is to be taken through a freeze, there must be a matching accepted NTH bug - will be taken through release freezes. Proposing, reviewing and monitoring NTH bugs will work exactly as it does for blocker bugs, and mostly happen in the blocker meetings, but of course after consideration of the blocker lists.
In practice this is a formalization of existing procedure - until F14 Beta, QA and releng did much the same process but entirely informally, we just kept lists of bugs we'd take fixes for either in our heads or in the RC creation trac tickets. This process is meant to be more robust, documented and discoverable.
Some releng SOP pages may require minor updates, I figured I'd leave that to releng. The process for creating blocker trackers should also be updated to cover creating NTH trackers (I couldn't find that; poelcat, where is it?)
Comments, questions, suggestions welcome!
On Thu, 2010-09-23 at 12:58 +0100, Adam Williamson wrote:
Hi, everyone. So we partly used the proposed new nice-to-have bug tracking system during the F14 Beta process, and it seemed to go well. In a quick burst of airport productivity, I've quickly written up a bunch of proposed new wiki pages and modifications to existing ones to document the nice-to-have process (and, incidentally, extend documentation of the blocker process, since we don't seem to have much of it beyond the blocker meeting SOP right now). All the pages can be found here:
https://fedoraproject.org/wiki/Category:NTH_adjustment_drafts
Thanks for providing draft wiki edits with the proposal. I've made a few _minor_ tweaks to the wiki pages.
it should be pretty obvious which are new and which are modifications of existing pages. I hope this is mostly straightforward and non-controversial.
A quick five-minute summary is that we will now have (in fact, we already do) trackers for nice-to-have bugs for Alpha, Beta and Final releases as well as trackers for blocker bugs. Bugs on these NTH lists will be given priority after blocker bugs for QA, devel and releng work for releases. Fixes for these bugs - and *only* these bugs, if a fix is to be taken through a freeze, there must be a matching accepted NTH bug
- will be taken through release freezes. Proposing, reviewing and
monitoring NTH bugs will work exactly as it does for blocker bugs, and mostly happen in the blocker meetings, but of course after consideration of the blocker lists.
I like the idea of formalizing the 'nice-to-have' process. I recall tension during the F-13-RC phase regarding the decision making process that allowed a selection of non-Blocker fixes into the release. I hope that this process helps clarify the acceptable exceptions to the blocker process.
In practice this is a formalization of existing procedure - until F14 Beta, QA and releng did much the same process but entirely informally, we just kept lists of bugs we'd take fixes for either in our heads or in the RC creation trac tickets. This process is meant to be more robust, documented and discoverable.
Perhaps the pending rel-eng SOP documents would cover this, but I'm unclear how FXX-accepted bugs are treated during at compose time. For our current Blocker bug process, it's established that if the appropriate blocker bug list is not empty, we can't compose. With non-blocker nice-to-have (NTH) bugs, how would that fix get into a compose?
Guessing ... * The fix would have to be packaged and available in bodhi updates-testing * The bodhi update has received the required karma * The accepted bug is in VERIFIED state?
To summarize, what instructions can we provide maintainers with so they can be confident an tested, packaged and approved nice-to-have fix will be pulled into any (re)compose? Perhaps, more of a question for the release-engineering team.
Different topic ...
In the days of using *only* Blocker bugs, it's now somewhat confusing to look at the F14Beta-accepted tracker, after we started mirroring F14Beta, and see 12 open bugs (some trackers) [1]. I don't think this is a bad thing, but perhaps another item to manage based on the result of the go/no-go meeting ... move any pending FXX-accepted bugs into the next milestone (so open FXXBeta-accepted bugs would move to FXX-accepted)?
[1] https://bugzilla.redhat.com/showdependencytree.cgi?id=F14Beta-accepted&h...
Some releng SOP pages may require minor updates, I figured I'd leave that to releng. The process for creating blocker trackers should also be updated to cover creating NTH trackers (I couldn't find that; poelcat, where is it?)
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers
I assume "User:Adamwill/QA:SOP_blocker_bug_meeting_nth_draft" is intended to replace "QA:SOP_Blocker_Bug_Meeting", and not define an additional blocker review meeting?
I've queued https://fedoraproject.org/wiki/User:Adamwill/QA:SOP_nth_process_nth_draft up for some additional reading, I'll reply later with any additional comments.
Thanks, James
On Fri, 2010-09-24 at 16:22 -0400, James Laska wrote:
In practice this is a formalization of existing procedure - until F14 Beta, QA and releng did much the same process but entirely informally, we just kept lists of bugs we'd take fixes for either in our heads or in the RC creation trac tickets. This process is meant to be more robust, documented and discoverable.
Perhaps the pending rel-eng SOP documents would cover this, but I'm unclear how FXX-accepted bugs are treated during at compose time. For our current Blocker bug process, it's established that if the appropriate blocker bug list is not empty, we can't compose. With non-blocker nice-to-have (NTH) bugs, how would that fix get into a compose?
Guessing ... * The fix would have to be packaged and available in bodhi updates-testing * The bodhi update has received the required karma * The accepted bug is in VERIFIED state?
To summarize, what instructions can we provide maintainers with so they can be confident an tested, packaged and approved nice-to-have fix will be pulled into any (re)compose? Perhaps, more of a question for the release-engineering team.
So far, we haven't been that strict; the process has been that I've listed any builds currently available in Koji which address nth issues in the RC compose request ticket, and then it's been up to rel-eng to take them or not. I agree we should nail this down a little better, and your list seems reasonable (though I'd probably say it's not necessary that the build be *available* in updates-testing, just that it have been *submitted* there; waiting for an updates-testing compose is an arbitrary limitation). Indeed this is probably something to co-ordinate with releng's SOPs on.
Different topic ...
In the days of using *only* Blocker bugs, it's now somewhat confusing to look at the F14Beta-accepted tracker, after we started mirroring F14Beta, and see 12 open bugs (some trackers) [1]. I don't think this is a bad thing, but perhaps another item to manage based on the result of the go/no-go meeting ... move any pending FXX-accepted bugs into the next milestone (so open FXXBeta-accepted bugs would move to FXX-accepted)?
Yes, I meant to include that and forgot about it. Indeed this should be the process.
[1] https://bugzilla.redhat.com/showdependencytree.cgi?id=F14Beta-accepted&h...
Some releng SOP pages may require minor updates, I figured I'd leave that to releng. The process for creating blocker trackers should also be updated to cover creating NTH trackers (I couldn't find that; poelcat, where is it?)
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers
I assume "User:Adamwill/QA:SOP_blocker_bug_meeting_nth_draft" is intended to replace "QA:SOP_Blocker_Bug_Meeting", and not define an additional blocker review meeting?
Yes. I considered creating a separate page detailing a 'nice-to-have review meeting' and noting that in practice it could be combined with the blocker meeting, but that just felt like over-engineering.
I've queued https://fedoraproject.org/wiki/User:Adamwill/QA:SOP_nth_process_nth_draft up for some additional reading, I'll reply later with any additional comments.
Thanks!
On Thu, 2010-09-23 at 12:58 +0100, Adam Williamson wrote:
Hi, everyone. So we partly used the proposed new nice-to-have bug tracking system during the F14 Beta process, and it seemed to go well. In a quick burst of airport productivity, I've quickly written up a bunch of proposed new wiki pages and modifications to existing ones to document the nice-to-have process (and, incidentally, extend documentation of the blocker process, since we don't seem to have much of it beyond the blocker meeting SOP right now). All the pages can be found here:
https://fedoraproject.org/wiki/Category:NTH_adjustment_drafts
Thanks to James for his feedback on this. I haven't had much feedback from anyone else. However, given that in practice everyone involved in the release review process has been happy using the NTH system drafted here so far, I intend to make the draft changes final (with modifications to reflect James' feedback) by the end of the week, so if you have any feedback you've been sitting on, now would be the perfect time to send it :) Thanks everyone!