On Tue, May 28, 2024 at 7:24 AM Sumantro Mukherjee sumukher@redhat.com wrote:
Hello Testers,
Hi Sumantro. Just a note, I'd send proposals to the test list only. (And even after it is decided, I don't think wiki categories are something that most subscribers are that interested in ;-)).
Fedora QA has been hosting loads of test days for sometime (ie 15 test days/release) which is why we author test cases. These test cases are sometimes for features which may never see the light of day (at least not in that particular release). Sometimes, these test cases are integration test cases and are very hard to categorize in package-specific test cases. Many times, due to their non-blocking nature, they are not associated with "Release Criterion". This makes test cases authored for a particular test day, extremely hard to find and check for bitrot.
I propose the following:
a) Add a new category Category:CoreOS Test Cases
That exists already: https://fedoraproject.org/wiki/Category:CoreOS_Test_Cases
b) Add new Category TestDay:NAME
Do you mean e.g. "TestDay:I18N" and "TestDay:Kernel", or do you mean "Test Day:2024-03-05 I18N Test Day" and "Test Day:2023-11-12 Kernel 6.6 Test Week", i.e. specific to a particular date?
Please note that we already have this: https://fedoraproject.org/wiki/Category:Test_Days_Test_Cases
But it's not seeing much use. My idea was that we assign all test days-specific test cases into it, so that we have one place to find them all. But we could have subcategories in there.
c) We already track the release cycle, Category:Fedora 41 Test Days
Yes, that exists already: https://fedoraproject.org/wiki/Category:Fedora_41_Test_Days
Benefits:
a) In the long run add more filtering functions to testday-stats on test cases added b) Easier navigation of test cases, written just for a test day and updation of the same c) Establishing scope for blocking criteria for upcoming features that get tested in Rawhide Test Days
I guess my choice would be to have "Category:Test day kernel test cases" [1], and "Category:Test day podman test cases", which follows the naming convention of "Category:Package kernel test cases" for Bodhi-related test cases. It also enables us to have generic "Category:Kernel test cases" (or CoreOS, see above) if we want to have test cases categorized for a certain package/theme, but not belonging to neither Bodhi nor Test days use case.
These new "Category:Test day XXX test cases" would be subcategories of "Category:Test_Days_Test_Cases", so that it's easy to find them all.
WDYT?