V Tue, Aug 30, 2022 at 09:39:27AM +0300, Alexander Bokovoy napsal(a):
On ma, 29 elo 2022, Adam Williamson wrote:
> On Tue, 2022-08-30 at 00:32 -0400, DJ Delorie wrote:
> > It sounds to me like the problem is "how do we best use the available
> > automated test resources?" so I'll answer accordingly. Ignore me if
I
> > misunderstood ;-)
>
> No, not really, sorry if I didn't explain clearly enough :D It's more
> just a 'what's the best way to work what I want to do around the
> existing definitions' question. What I want to do is have updates of
> FreeIPA-related packages gated. The awkwardness is that right now the
> definition of what gets gated is "critical path packages", and those
> clearly don't meet the current definition of "critical path".
Not all of them meet the current definition of 'critical path' but many
do. In fact, almost all crypto-implementing packages should be on that
list, if not already.
Ideally, use of gating tests should be defined in the component itself.
Do we have a mechanism to trigger OpenQA runs from the gating.yaml in
the specific component? If we do, then we really should move it there.
Another part of the problem is that most of these runs should probably
be consulting, not gating in the sense that failing them should give you
a clue that things might be wrong but there needs to be a human overview
of the failures. It is what we have right now with OpenQA-reported
failures for Bodhi updates in most cases, the test results aren't really
preventing the submissions from the going forward unless a maintainer
defined the update configuration to be so.
Ideally, we should have reverse dependency updates to trigger FreeIPA
tests and give their results a visibility but don't block on failures.
Exactly. If a package needs gating, it can enable it in its dist-git. At the
same time I agree with Kevin, that if a distribution/edition needs to
gate a set of packages, then it should also have it's own way to define the
set.
If a package needs to run tests from a group of related packages, the package
can either manually define the group, or there should be an out-of-band way of
defining the group. (TMT calls it a test plan.)
A reverse dependency is a possible, IMHO good, way of the definition. Alas,
Fedora TMT does not support it
<
https://pagure.io/fedora-ci/general/issue/251>.
I feel Adam is aware of it and tries to work around this TMT deficiency
with an allowedlist.
-- Petr