[Fedora QA] #173: Obsoleting the tests for verifying http/nfs/ftp/hard disk install sources
#173: Obsoleting the tests for verifying http/nfs/ftp/hard disk install sources -------------------------+-------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Keywords: | -------------------------+-------------------------------------------------- = problem = Now that install.img is included in initrd.img, it can no longer be loaded from somewhere else other than media, therefore Using 'askmethod' is actually the same as 'repo=' which is to direct anaconda using a specific package repo.
= analysis = So I propose to obsolete the install source test cases of install sources from http, nfs, ftp, hd, and only test the package repositories of these methods.
= enhancement recommendation =
The following tests will be kept: - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceBootIso InstallSourceBootIso] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd InstallSourceDvd] - [https://fedoraproject.org/wiki/QA:TestCases/Install_Source_Live_Image Install_Source_Live_Image]
The following will be obsoleted: - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceHttp InstallSourceHttp] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceNfs InstallSourceNfs] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceNfsIso InstallSourceNfsIso] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceFtpAnonymous InstallSourceFtpAnonymous] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceFtpNonAnonymous InstallSourceFtpNonAnonymous] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceHardDrive InstallSourceHardDrive]
Current package repository tests are listed at: [https://fedoraproject.org/wiki/Category:Repository Category:Repository]
Hard disk and nfs iso repo tests need be added, then they'll cover all package install methods.
clumens, how do you think of these? do the remaining tests(install source+repo) cover the common use cases?
#173: Obsoleting the tests for verifying http/nfs/ftp/hard disk install sources --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):
Thanks for raising this topic! We'll need to confirm with clumens of course, but I don't know if there are plans to add graphical repository configuration for NFS ISO, HD and HD ISO installation methods. I don't believe there are plans to remove loader handling for package repository access methods. Which means, Fedora will still support traditional installation methods available by way of ''askmethod''.
Perhaps we need to rename/rethink these different tests?
The two functions that need verification are ... 1. the loader handling/setup of a given installation repository. I'm thinking these are the operations that dictate how the 'Installation repo' is setup. For some boot methods, this is handled for us (e.g. DVD creates a CDROM repo, boot.iso and pxeboot don't add any 'installation repo') 2. the graphical repository dialog allowing for additional repos, repo removal, and existing repo edits. These are mostly manual (except for kickstart) functions.
As for loader handling/setup of a given installation repository, we have the following scenarios (most of which are already tests, but perhaps need adjustment/rename):
1. Installation repository - http * default scenario 1. Boot and install using a netinst.iso or pxeboot images * variation(s): 1. Boot with: 'askmethod' and manually input HTTP repo 2. Boot with: repo=http://server/path/to/repo 3. Use ftp://server/path/to/repo 4. Use ftp://user:pass@server/path/to/repo 5. Configure proxy=[protocol://][username[:password]@]host[:port] 2. Installation repository - NFS * default scenario 1. Use graphical repository dialog to input NFS Server and PATH * variation(s): 1. Boot with: repo=nfs:<server>:/<path> 3. Installation repository - NFS ISO * default scenario 1. Boot with 'askmethod' and manually enter NFS ISO server + path * variation(s): 1. Boot with: repo=nfsiso[:options]:<server>:/<path> 4. Install repository - HD ISO * default scenario 1. Boot with 'askmethod' and manually enter HD device and path * variations: 1. repo=hd:<device>:/<path> 2. repo=hd:LABEL=<label>:/<path> 3. repo=hd:UUID=<uuid>:/<path> 5. Install repository - CD/DVD ISO * default scenario 1. Boot and install from a CD or DVD ISO image * variations: 1. Boot with 'askmethod' and selects CD-ROM/DVD-ROM 2. Boot with repo=cdrom:<device>
If we include the FTP* tests you list as a variation of the HTTP test, we can remove them altogether.
As for graphical repository edits, we have those covered already right?
https://fedoraproject.org/wiki/Category:Repository
#173: Obsoleting the tests for verifying http/nfs/ftp/hard disk install sources --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by rhe):
Replying to [comment:1 jlaska]:
Thanks for raising this topic! We'll need to confirm with clumens of
course, but I don't know if there are plans to add graphical repository configuration for NFS ISO, HD and HD ISO installation methods. I don't believe there are plans to remove loader handling for package repository access methods. Which means, Fedora will still support traditional installation methods available by way of ''askmethod''. Ok, if it's still supported, there should be tests covering it.
Perhaps we need to rename/rethink these different tests?
Yeah, and your grouping idea reminds me a way to do it: Combine current [https://fedoraproject.org/wiki/Category:Installation_Source Install source](except install source on Media tests) and [https://fedoraproject.org/wiki/Category:Repository Package repository] together, naming it as 'Installation repository'. In this category, cases are sub-categoried by http, NFS, NFSISO and CD/DVD. How does this sound?
The two functions that need verification are ...
- the loader handling/setup of a given installation repository. I'm
thinking these are the operations that dictate how the 'Installation repo' is setup. For some boot methods, this is handled for us (e.g. DVD creates a CDROM repo, boot.iso and pxeboot don't add any 'installation repo')
- the graphical repository dialog allowing for additional repos, repo
removal, and existing repo edits. These are mostly manual (except for kickstart) functions.
As for loader handling/setup of a given installation repository, we have
the following scenarios (most of which are already tests, but perhaps need adjustment/rename):
- Installation repository - http
- default scenario
- Boot and install using a netinst.iso or pxeboot images
Current test is: [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceBootIso InstallSourceBootIso] (using default retrieved mirror repo)
- variation(s):
- Boot with: 'askmethod' and manually input HTTP repo
Current test is: [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceHttp InstallSourceHttp], need rename it and modify the content about 'install source' part.
- Boot with: repo=http://server/path/to/repo
Current test is: [https://fedoraproject.org/wiki/QA:Testcase_Http_Repository QA:Testcase_Http_Repository]
- Use ftp://server/path/to/repo
- Use ftp://user:pass@server/path/to/repo
If we include FTP tests in HTTP tests, these two can be included in the above two http cases as a choice to test?
- Configure proxy=[protocol://][username[:password]@]host[:port]
No test case yet. I think this can be included as an optional step in the http repo tests?
- Installation repository - NFS
- default scenario
- Use graphical repository dialog to input NFS Server and PATH
Current test is: [https://fedoraproject.org/wiki/QA:Testcase_Additional_NFS_Repository QA:Testcase_Additional_NFS_Repository]
- variation(s):
- Boot with: repo=nfs:<server>:/<path>
Current test is: [https://fedoraproject.org/wiki/QA:Testcase_Nfs_Repository QA:Testcase_Nfs_Repository]
and 2. Boot with 'askmethod'? [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceNfs InstallSourceNfs] is the test to be adjusted.
- Installation repository - NFS ISO
- default scenario
- Boot with 'askmethod' and manually enter NFS ISO server + path
Current test to be adjusted: [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceNfsIso InstallSourceNfsIso]
- variation(s):
- Boot with: repo=nfsiso[:options]:<server>:/<path>
No test case covering it yet. Need to create it.
- Install repository - HD ISO
- default scenario
- Boot with 'askmethod' and manually enter HD device and path
[https://fedoraproject.org/wiki/QA/TestCases/InstallSourceHardDrive InstallSourceHardDrive]
- variations:
- repo=hd:<device>:/<path>
- repo=hd:LABEL=<label>:/<path>
- repo=hd:UUID=<uuid>:/<path>
No test case yet. The above three can be combined into one test in my opinion.
- Install repository - CD/DVD ISO
- default scenario
- Boot and install from a CD or DVD ISO image
Current test is [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd InstallSourceDvd].
- variations:
- Boot with 'askmethod' and selects CD-ROM/DVD-ROM
- Boot with repo=cdrom:<device>
No test case yet. Do we need cover these two? I don't think they are common use cases.
If we include the FTP* tests you list as a variation of the HTTP test,
we can remove them altogether.
Can you clarify this? do you mean ftp can be a option in http test cases?
As for graphical repository edits, we have those covered already right?
Yes. They may need to rename then.
#173: Obsoleting the tests for verifying http/nfs/ftp/hard disk install sources --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):
I like your idea of creating/renaming tests to be ''Installation repository''. In addition to that, some test rewording may be required. I think all the same installation sources are still supported, but the phrasing, and the procedure, has changed slightly.
An attempt to identify the methods for testing the different ''Installation repository'' scenarios. This doesn't include the ''Additional repository'' tests. As you noted, those may need to be run also :(
{{{ #!html
<table border="1"> <tr> <th>Installation Repository <th>Boot media used <th>Default test scenario <th>Additional test variations </tr> <tr> <td>DVD <td>DVD.iso <td>Boot DVD with no additional arguments <td> <ul> <li>askmethod, and select CDROM/DVD <li>repo=cdrom <li>repo=cdrom:/dev/sr0 </ul> </tr> <tr> <td>Mirrorlist <td>netinst.iso or pxeboot <td>Boot netinst.iso with no additional arguments <td> </tr> <tr> <td>HTTP/FTP <td>netinst.iso, pxeboot or DVD <td>Boot netinst.iso with ''askmethod'', select URL <td> <ul> <li>repo=http://host/path <li>repo=https://host/path <li>proxy=[protocol://][username[:password]@]host[:port] </ul> </tr> <tr> <td>NFS <td>netinst.iso, pxeboot or DVD <td>Boot with ''askmethod'', select NFS <td> <ul> <li>repo=nfs:server:path </ul> </tr> <tr> <td>NFS ISO <td>netinst.iso, pxeboot or DVD <td>Boot with ''askmethod'', select NFS <td> <ul> <li>repo=nfs[:options]:server:path <li>repo=nfsiso[:options]:server:path </ul> </tr> <tr> <td>HD ISO <td>netinst.iso, pxeboot or DVD <td>Boot with ''askmethod'', select ''Hard drive'' <td> <ul> <li>repo=hd:device:/path <li>repo=hd:LABEL=:/path <li>repo=hd:UUID=:/path </ul> </tr> </table>
}}}
What do you think about the idea of including additional variations inside a test case. My thought here is that the test could still pass using the default test scenario, but would result in a warning if any of the optional test variations fail. Is this confusing, should we just create distinct test cases for each of the optional test variations? I was hoping to avoid that, but not if it's too confusing.
Can you clarify this? do you mean ftp can be a option in http test
cases?
Yeah, since pycurl is used now for HTTP and FTP, I'm wondering if we need to explicitly call out FTP in our tests. Should we leave that as a variation?
#173: Obsoleting the tests for verifying http/nfs/ftp/hard disk install sources --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by rhe):
Replying to [comment:3 jlaska]:
I like your idea of creating/renaming tests to be ''Installation
repository''. In addition to that, some test rewording may be required. I think all the same installation sources are still supported, but the phrasing, and the procedure, has changed slightly.
An attempt to identify the methods for testing the different
''Installation repository'' scenarios. This doesn't include the ''Additional repository'' tests. As you noted, those may need to be run also :(
No worries, then they'll be included in ''Installation repository''.
{{{ #!html
<table border="1"> <tr> <th>Installation Repository <th>Boot media used <th>Default test scenario <th>Additional test variations </tr> <tr> <td>DVD <td>DVD.iso <td>Boot DVD with no additional arguments <td> <ul> <li>askmethod, and select CDROM/DVD <li>repo=cdrom <li>repo=cdrom:/dev/sr0 </ul> </tr> <tr> <td>Mirrorlist <td>netinst.iso or pxeboot <td>Boot netinst.iso with no additional arguments <td> </tr> <tr> <td>HTTP/FTP <td>netinst.iso, pxeboot or DVD <td>Boot netinst.iso with ''askmethod'', select URL <td> <ul> <li>repo=http://host/path <li>repo=https://host/path <li>proxy=[protocol://][username[:password]@]host[:port] </ul> </tr> <tr> <td>NFS <td>netinst.iso, pxeboot or DVD <td>Boot with ''askmethod'', select NFS <td> <ul> <li>repo=nfs:server:path </ul> </tr> <tr> <td>NFS ISO <td>netinst.iso, pxeboot or DVD <td>Boot with ''askmethod'', select NFS <td> <ul> <li>repo=nfs[:options]:server:path <li>repo=nfsiso[:options]:server:path </ul> </tr> <tr> <td>HD ISO <td>netinst.iso, pxeboot or DVD <td>Boot with ''askmethod'', select ''Hard drive'' <td> <ul> <li>repo=hd:device:/path <li>repo=hd:LABEL=:/path <li>repo=hd:UUID=:/path </ul> </tr> </table>
}}}
Wow, I'm impressed by your table format here, which inspired me to learn trac syntax.:)
What do you think about the idea of including additional variations
inside a test case. My thought here is that the test could still pass using the default test scenario, but would result in a warning if any of the optional test variations fail. Is this confusing, should we just create distinct test cases for each of the optional test variations? I was hoping to avoid that, but not if it's too confusing.
It also depends on our test coverage IMO. First, the default test scenario must be covered in every run. For the additional test variations, should all of them or at least one of them be tested for each source? if the latter case, I prefer to write all Additional variations of each source in one case, then each source has two tests(except graphical): default one, and anyone from the variations. How do you think?
Can you clarify this? do you mean ftp can be a option in http test
cases?
Yeah, since pycurl is used now for HTTP and FTP, I'm wondering if we
need to explicitly call out FTP in our tests. Should we leave that as a variation?
Yeah, since ftp is still a use case, we can leave that as a variation of HTTP/FTP source.
#173: Obsoleting the tests for verifying http/nfs/ftp/hard disk install sources --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):
Note, while testing, hansg reminded us that many of our tests refer to stage1 and stage2. With Fedora 15, the previous distinction between stage1 and stage2 no longer applies. There is still a difference, but the difference now is between loader (old stage1) and anaconda (old stage2).
Are stage1/stage2 changes to test cases included in this ticket, or should I track this as a separate effort?
#173: Improve the install sources tests to adapt F-15 anaconda --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by rhe):
* summary: Obsoleting the tests for verifying http/nfs/ftp/hard disk install sources => Improve the install sources tests to adapt F-15 anaconda
Comment:
Replying to [comment:5 jlaska]:
Note, while testing, hansg reminded us that many of our tests refer to
stage1 and stage2. With Fedora 15, the previous distinction between stage1 and stage2 no longer applies. There is still a difference, but the difference now is between loader (old stage1) and anaconda (old stage2).
Are stage1/stage2 changes to test cases included in this ticket, or
should I track this as a separate effort?
Never mind, this is already in part of the plan, I've also renamed the ticket, thanks for reminding.：）
#173: Improve the install sources tests to adapt F-15 anaconda --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by rhe):
I've updated the tests based on the table in comment:4 by either creating uncovered new tests or moving/renaming some install source tests to install repo ones with rewordings and adjustments. Now all the tests have been integrated into the [http://fedoraproject.org/wiki/Category:Installation_Repository install repository category] sub-categoried by DVD, HTTP/FTP, Mirrorlist, NFS, NFSISO and HardDrive. In each subcategory, there are generally three cases: default scenario, additional variation, and Graphical repo dialog. Feel free to review and feedback is welcomed! Thanks.
#173: Improve the install sources tests to adapt F-15 anaconda --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by jlaska):
Nice work indeed! Some minor comments ... * When you say ''default'' you mean the default method for testing that particular function, not the installers default method for handling it?
With the modified tests, and the boot method tests ... do we have the following scenarios covered for each media? 1. boot media and accept defaults (different behavior based on media used) 2. boot media with askmethod, and choose appropriate installation source 3. boot media with appropriate repo= boot parameter 4. boot media and graphically choose installation source
#173: Improve the install sources tests to adapt F-15 anaconda --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by rhe):
Replying to [comment:8 jlaska]:
Nice work indeed! Some minor comments ...
- When you say ''default'' you mean the default method for testing that
particular function, not the installers default method for handling it?
I mean the default method for testing that function. The 'default', 'variation' was named according to the table in comment:3
With the modified tests, and the boot method tests ... do we have the
following scenarios covered for each media?
- boot media and accept defaults (different behavior based on media
used)
Boot media tests have covered all the media, so I prefer not to modify them a lot but will reword the descriptions about anaconda stage#1.
For install repo tests, the followings are the default install repo cases for each media: * DVD: http://fedoraproject.org/wiki/QA:Testcase_install_repository_DVD_default * netinst.iso: http://fedoraproject.org/wiki/QA:Testcase_install_repository_Mirrorlist_defa... * pxeboot: no default install repo * live: https://fedoraproject.org/wiki/QA:Testcase_install_repository_Live_Image
- boot media with askmethod, and choose appropriate installation
source
- boot media with appropriate repo= boot parameter
Now there are two tests which are 'askmethod' and 'repo=' for every method: HTTP/FTP, Harddrive, NFS, NFSISO and DVD(askmethod and repo= are contained in one test for DVD). IMO, they don't depend on any media type and should also be put in the [https://fedoraproject.org/wiki/QA:Fedora_15_Install_Results_Template#General... 'Install Variations & General Tests']. What do you think?
- boot media and graphically choose installation source
These tests have been integrated to install repository category, expect NFS ISO and Harddrive which are not supported. Previously, graphical tests were put into [https://fedoraproject.org/wiki/QA:Fedora_15_Install_Results_Template#DVD DVD media tests] to verify the network enabling ability at that step. For fedora-16, do you think if these graphical tests should be put to [https://fedoraproject.org/wiki/QA:Fedora_15_Install_Results_Template#General... 'Install Variations & General Tests'] instead?[[BR]] [[BR]] In addition, These repos can also be specified in kickstart, is it an alternative way to test 'repo=' method?
#173: Improve the install sources tests to adapt F-15 anaconda --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by rhe):
I've modified the statements about stage#1 and stage#2 for the following tests:
https://fedoraproject.org/wiki/QA/TestCases/BootMethodsBootIso https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd https://fedoraproject.org/wiki/QA/TestCases/BootMethodsPxeboot https://fedoraproject.org/wiki/QA/TestCases/KickstartKsFilePathKsCfg https://fedoraproject.org/wiki/QA:Testcase_Anaconda_rescue_mode https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_URL https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_installa...
#173: Improve the install sources tests to adapt F-15 anaconda --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by rhe):
Correct the format
Replying to [comment:10 rhe]: I've modified the statements about stage#1 and stage#2 for the following
tests:
- https://fedoraproject.org/wiki/QA/TestCases/BootMethodsBootIso - https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd - https://fedoraproject.org/wiki/QA/TestCases/BootMethodsPxeboot - https://fedoraproject.org/wiki/QA/TestCases/KickstartKsFilePathKsCfg - https://fedoraproject.org/wiki/QA:Testcase_Anaconda_rescue_mode - https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_URL - https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_installa...
#173: Improve the install sources tests to adapt F-15 anaconda --------------------------+------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: closed Priority: major | Milestone: Fedora 15 Component: Wiki | Version: Resolution: fixed | Keywords: --------------------------+------------------------------------------------- Changes (by rhe):
* status: new => closed * resolution: => fixed
Comment:
As discussed above, I've improved the install source cases and re- categorized them to [https://fedoraproject.org/wiki/Category:Installation_Repository Category:Installation_Repository]. I also updated [https://fedoraproject.org/wiki/Category:Installer_Boot_Methods Category:Installer_Boot_Methods]. I think the previous requirements have been met. Close this ticket for now, but feel free to comment and reopen it.
-
fedora-badges