On Wed, 2023-04-12 at 09:00 -0400, Ben Cotton wrote:
Footnote - "What does this cover?": Toolbox's general functionality as covered by this test cases https://fedoraproject.org/wiki/QA:Testcase_toolbox
The test case seems to be much narrower than the text of the criterion implies. Is the test case insufficient or is the criterion too broad?
Also, this isn't really how we do things. The criterion should explain itself. The test case covers the criterion. It's not really procedurally sound for the criterion definition to reference the test case.
I think this criterion could target F39 Beta Release Criteria or Basic Release Criteria [2].
I'd say this is a Beta (or possibly Final, although I'm not sure if I believe that) criterion for now. When/if ostree variants become our primary desktop deliverable, moving it to Basic would make sense to me.
So aside from the good notes Ben provided, there's another problem: how exactly does toolbox functionality intersect with the release process?
We don't produce the toolbox container image as part of the compose process, so what exactly are we covering here? Is this a criterion for the toolbox container image itself? For running toolbox from the media produced as part of the compose?
The latter seems more practicable. In that case I'd probably write the criterion to state that, i.e. something like "For any release-blocking deliverable whose default deployment includes toolbox, it must be possible to..." or something along those lines.