On Thu, Mar 12, 2020 at 10:59 AM Kamil Paral kparal@redhat.com wrote:
It has been 7 days since the last response to the thread and I received no further complaints, so I've put the criterion into effect:
https://fedoraproject.org/w/index.php?title=Fedora_32_Final_Release_Criteria...
I've done slight tweaks compared to the original proposal:
- Help viewer has been included in the covered application types.
- Primary/default application is the one that must satisfy requirements in
case of multiple same-type applications present (per the proposal by KDE folks mentioned in this thread).
- "only one of said applications must satisfy the requirements" has been
changed to "at least one of said applications...", which has the same meaning but is clearer (I hope)
- "file browser" was renamed to "file manager", which seems to be the more
official term (when looking at wikipedia)
- application type examples were put into a footnote, to make the
criterion itself more readable (and avoid some potential confusion when people see exact application names in there and miss the "e.g.")
I'll work on adjusting associated test cases, to put them in line with the new criterion.
(Replying just to test list now, no longer across lists)
The adjusted criterion seems to affect only this test case: https://fedoraproject.org/wiki/QA:Testcase_desktop_menus
My current plan is to redo it somewhat. I want to:
1. Drop all instructions to run applications from Testcase_desktop_menus and make it Optional. If will once again reflect its title and only care about menus and icons in menus.
2. Create the following new test cases: Testcase_desktop_app_web_browser Testcase_desktop_app_file_manager Testcase_desktop_app_package_manager Testcase_desktop_app_image_viewer Testcase_desktop_app_document_viewer Testcase_desktop_app_text_editor Testcase_desktop_app_archive_manager Testcase_desktop_app_terminal_emulator Testcase_desktop_app_problem_reporter Testcase_desktop_app_help_viewer Each of these will instruct people to start and test basic functionality of an application of that type, provided by the desktop. It will also explain what to do when there are multiple such applications. These test cases will be mandatory for all desktops.
3. Create the following new test case: Testcase_desktop_app_others This will include the same instructions as in 2), but it will instruct people to test all applications not covered by specific test cases in the same matrix. This test case will be mandatory for Workstation x86_64 and Optional for others.
This will have the benefit of multiple people being able to easily collaborate on desktop testing. It will definitely be less off-putting than a single "do it all" test case. It allows people to participate in desktop testing even if they have limited time. And it allows us to write OpenQA tests for certain applications and fill out the results on the wiki.
Note: In point 2), we don't actually need 10 different test cases (templated or not). We might just create Testcase_desktop_app_single instead of those 10 and differentiate them on the matrix level the same way as we do with e.g. "Testcase_Anaconda_save_traceback_to_bugzilla" [1]. So like this: Testcase_desktop_app_single (web browser) Testcase_desktop_app_single (file manager) ... I don't know if it complicates OpenQA results submission, though.
Please give me your thoughts, thanks. Kamil
[1] https://fedoraproject.org/wiki/Test_Results:Fedora_32_Beta_1.2_Installation#...