= problem = Now that install.img is included in initrd.img, it can no longer be loaded from somewhere else other than media, therefore Using 'askmethod' is actually the same as 'repo=' which is to direct anaconda using a specific package repo.
= analysis = So I propose to obsolete the install source test cases of install sources from http, nfs, ftp, hd, and only test the package repositories of these methods.
= enhancement recommendation =
The following tests will be kept: - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceBootIso InstallSourceBootIso] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd InstallSourceDvd] - [https://fedoraproject.org/wiki/QA:TestCases/Install_Source_Live_Image Install_Source_Live_Image]
The following will be obsoleted: - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceHttp InstallSourceHttp] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceNfs InstallSourceNfs] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceNfsIso InstallSourceNfsIso] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceFtpAnonymous InstallSourceFtpAnonymous] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceFtpNonAnonymous InstallSourceFtpNonAnonymous] - [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceHardDrive InstallSourceHardDrive]
Current package repository tests are listed at: [https://fedoraproject.org/wiki/Category:Repository Category:Repository]
Hard disk and nfs iso repo tests need be added, then they'll cover all package install methods.
clumens, how do you think of these? do the remaining tests(install source+repo) cover the common use cases?
Thanks for raising this topic! We'll need to confirm with clumens of course, but I don't know if there are plans to add graphical repository configuration for NFS ISO, HD and HD ISO installation methods. I don't believe there are plans to remove loader handling for package repository access methods. Which means, Fedora will still support traditional installation methods available by way of ''askmethod''.
Perhaps we need to rename/rethink these different tests?
The two functions that need verification are ... 1. the loader handling/setup of a given installation repository. I'm thinking these are the operations that dictate how the 'Installation repo' is setup. For some boot methods, this is handled for us (e.g. DVD creates a CDROM repo, boot.iso and pxeboot don't add any 'installation repo') 2. the graphical repository dialog allowing for additional repos, repo removal, and existing repo edits. These are mostly manual (except for kickstart) functions.
As for loader handling/setup of a given installation repository, we have the following scenarios (most of which are already tests, but perhaps need adjustment/rename):
If we include the FTP* tests you list as a variation of the HTTP test, we can remove them altogether.
As for graphical repository edits, we have those covered already right?
https://fedoraproject.org/wiki/Category:Repository
Replying to [comment:1 jlaska]:
Thanks for raising this topic! We'll need to confirm with clumens of course, but I don't know if there are plans to add graphical repository configuration for NFS ISO, HD and HD ISO installation methods. I don't believe there are plans to remove loader handling for package repository access methods. Which means, Fedora will still support traditional installation methods available by way of ''askmethod''. Ok, if it's still supported, there should be tests covering it. Perhaps we need to rename/rethink these different tests?
Thanks for raising this topic! We'll need to confirm with clumens of course, but I don't know if there are plans to add graphical repository configuration for NFS ISO, HD and HD ISO installation methods. I don't believe there are plans to remove loader handling for package repository access methods. Which means, Fedora will still support traditional installation methods available by way of ''askmethod''. Ok, if it's still supported, there should be tests covering it.
Yeah, and your grouping idea reminds me a way to do it: Combine current https://fedoraproject.org/wiki/Category:Installation_Source Install source and [https://fedoraproject.org/wiki/Category:Repository Package repository] together, naming it as 'Installation repository'. In this category, cases are sub-categoried by http, NFS, NFSISO and CD/DVD. How does this sound?
The two functions that need verification are ... 1. the loader handling/setup of a given installation repository. I'm thinking these are the operations that dictate how the 'Installation repo' is setup. For some boot methods, this is handled for us (e.g. DVD creates a CDROM repo, boot.iso and pxeboot don't add any 'installation repo') 2. the graphical repository dialog allowing for additional repos, repo removal, and existing repo edits. These are mostly manual (except for kickstart) functions. As for loader handling/setup of a given installation repository, we have the following scenarios (most of which are already tests, but perhaps need adjustment/rename): Installation repository - http * default scenario Boot and install using a netinst.iso or pxeboot images
Current test is: [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceBootIso InstallSourceBootIso] (using default retrieved mirror repo)
variation(s): 1. Boot with: 'askmethod' and manually input HTTP repo Current test is: [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceHttp InstallSourceHttp], need rename it and modify the content about 'install source' part. 2. Boot with: repo=http://server/path/to/repo Current test is: [https://fedoraproject.org/wiki/QA:Testcase_Http_Repository QA:Testcase_Http_Repository] 3. Use ftp://server/path/to/repo 4. Use ftp://user:pass@server/path/to/repo
If we include FTP tests in HTTP tests, these two can be included in the above two http cases as a choice to test?
Configure proxy=[protocol://][username[:password]@]host[:port] No test case yet. I think this can be included as an optional step in the http repo tests? Installation repository - NFS * default scenario Use graphical repository dialog to input NFS Server and PATH
Configure proxy=[protocol://][username[:password]@]host[:port] No test case yet. I think this can be included as an optional step in the http repo tests?
Installation repository - NFS * default scenario
Current test is: [https://fedoraproject.org/wiki/QA:Testcase_Additional_NFS_Repository QA:Testcase_Additional_NFS_Repository]
variation(s): 1. Boot with: repo=nfs:<server>:/<path> Current test is: [https://fedoraproject.org/wiki/QA:Testcase_Nfs_Repository QA:Testcase_Nfs_Repository]
and 2. Boot with 'askmethod'? [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceNfs InstallSourceNfs] is the test to be adjusted.
Installation repository - NFS ISO * default scenario Boot with 'askmethod' and manually enter NFS ISO server + path Current test to be adjusted: [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceNfsIso InstallSourceNfsIso] variation(s): 1. Boot with: repo=nfsiso[:options]:<server>:/<path> No test case covering it yet. Need to create it. Install repository - HD ISO * default scenario Boot with 'askmethod' and manually enter HD device and path
[https://fedoraproject.org/wiki/QA/TestCases/InstallSourceHardDrive InstallSourceHardDrive]
variations: 1. repo=hd:<device>:/<path> 2. repo=hd:LABEL=<label>:/<path> 3. repo=hd:UUID=<uuid>:/<path>
No test case yet. The above three can be combined into one test in my opinion.
Install repository - CD/DVD ISO * default scenario Boot and install from a CD or DVD ISO image Current test is [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd InstallSourceDvd]. * variations: Boot with 'askmethod' and selects CD-ROM/DVD-ROM Boot with repo=cdrom:<device>
No test case yet. Do we need cover these two? I don't think they are common use cases.
Can you clarify this? do you mean ftp can be a option in http test cases?
As for graphical repository edits, we have those covered already right? Yes. They may need to rename then.
Yes. They may need to rename then.
I like your idea of creating/renaming tests to be ''Installation repository''. In addition to that, some test rewording may be required. I think all the same installation sources are still supported, but the phrasing, and the procedure, has changed slightly.
An attempt to identify the methods for testing the different ''Installation repository'' scenarios. This doesn't include the ''Additional repository'' tests. As you noted, those may need to be run also :(
{{{
}}}
What do you think about the idea of including additional variations inside a test case. My thought here is that the test could still pass using the default test scenario, but would result in a warning if any of the optional test variations fail. Is this confusing, should we just create distinct test cases for each of the optional test variations? I was hoping to avoid that, but not if it's too confusing.
Yeah, since pycurl is used now for HTTP and FTP, I'm wondering if we need to explicitly call out FTP in our tests. Should we leave that as a variation?
Replying to [comment:3 jlaska]:
I like your idea of creating/renaming tests to be ''Installation repository''. In addition to that, some test rewording may be required. I think all the same installation sources are still supported, but the phrasing, and the procedure, has changed slightly. An attempt to identify the methods for testing the different ''Installation repository'' scenarios. This doesn't include the ''Additional repository'' tests. As you noted, those may need to be run also :(
No worries, then they'll be included in ''Installation repository''.
{{{ !html Installation Repository Boot media used Default test scenario Additional test variations DVD DVD.iso Boot DVD with no additional arguments askmethod, and select CDROM/DVD repo=cdrom repo=cdrom:/dev/sr0 Mirrorlist netinst.iso or pxeboot Boot netinst.iso with no additional arguments HTTP/FTP netinst.iso, pxeboot or DVD Boot netinst.iso with ''askmethod'', select URL repo=http://host/path repo=https://host/path proxy=[protocol://][username[:password]@]host[:port] NFS netinst.iso, pxeboot or DVD Boot with ''askmethod'', select NFS repo=nfs:server:path NFS ISO netinst.iso, pxeboot or DVD Boot with ''askmethod'', select NFS repo=nfs[:options]:server:path repo=nfsiso[:options]:server:path HD ISO netinst.iso, pxeboot or DVD Boot with ''askmethod'', select ''Hard drive'' repo=hd:device:/path repo=hd:LABEL=:/path repo=hd:UUID=:/path }}}
Wow, I'm impressed by your table format here, which inspired me to learn trac syntax.:)
It also depends on our test coverage IMO. First, the default test scenario must be covered in every run. For the additional test variations, should all of them or at least one of them be tested for each source? if the latter case, I prefer to write all Additional variations of each source in one case, then each source has two tests(except graphical): default one, and anyone from the variations. How do you think?
Can you clarify this? do you mean ftp can be a option in http test cases? Yeah, since pycurl is used now for HTTP and FTP, I'm wondering if we need to explicitly call out FTP in our tests. Should we leave that as a variation?
Yeah, since ftp is still a use case, we can leave that as a variation of HTTP/FTP source.
Note, while testing, hansg reminded us that many of our tests refer to stage1 and stage2. With Fedora 15, the previous distinction between stage1 and stage2 no longer applies. There is still a difference, but the difference now is between loader (old stage1) and anaconda (old stage2).
Are stage1/stage2 changes to test cases included in this ticket, or should I track this as a separate effort?
Replying to [comment:5 jlaska]:
Note, while testing, hansg reminded us that many of our tests refer to stage1 and stage2. With Fedora 15, the previous distinction between stage1 and stage2 no longer applies. There is still a difference, but the difference now is between loader (old stage1) and anaconda (old stage2). Are stage1/stage2 changes to test cases included in this ticket, or should I track this as a separate effort?
Never mind, this is already in part of the plan, I've also renamed the ticket, thanks for reminding.：）
I've updated the tests based on the table in comment:4 by either creating uncovered new tests or moving/renaming some install source tests to install repo ones with rewordings and adjustments. Now all the tests have been integrated into the [http://fedoraproject.org/wiki/Category:Installation_Repository install repository category] sub-categoried by DVD, HTTP/FTP, Mirrorlist, NFS, NFSISO and HardDrive. In each subcategory, there are generally three cases: default scenario, additional variation, and Graphical repo dialog. Feel free to review and feedback is welcomed! Thanks.
Nice work indeed! Some minor comments ... * When you say ''default'' you mean the default method for testing that particular function, not the installers default method for handling it?
With the modified tests, and the boot method tests ... do we have the following scenarios covered for each media? 1. boot media and accept defaults (different behavior based on media used) 2. boot media with askmethod, and choose appropriate installation source 3. boot media with appropriate repo= boot parameter 4. boot media and graphically choose installation source
Replying to [comment:8 jlaska]:
I mean the default method for testing that function. The 'default', 'variation' was named according to the table in comment:3
With the modified tests, and the boot method tests ... do we have the following scenarios covered for each media? 1. boot media and accept defaults (different behavior based on media used)
Boot media tests have covered all the media, so I prefer not to modify them a lot but will reword the descriptions about anaconda stage#1.
For install repo tests, the followings are the default install repo cases for each media: * DVD: http://fedoraproject.org/wiki/QA:Testcase_install_repository_DVD_default * netinst.iso: http://fedoraproject.org/wiki/QA:Testcase_install_repository_Mirrorlist_default * pxeboot: no default install repo * live: https://fedoraproject.org/wiki/QA:Testcase_install_repository_Live_Image
boot media with askmethod, and choose appropriate installation source boot media with appropriate repo= boot parameter
Now there are two tests which are 'askmethod' and 'repo=' for every method: HTTP/FTP, Harddrive, NFS, NFSISO and DVD(askmethod and repo= are contained in one test for DVD). IMO, they don't depend on any media type and should also be put in the [https://fedoraproject.org/wiki/QA:Fedora_15_Install_Results_Template#General_Tests 'Install Variations & General Tests']. What do you think?
boot media and graphically choose installation source
These tests have been integrated to install repository category, expect NFS ISO and Harddrive which are not supported. Previously, graphical tests were put into [https://fedoraproject.org/wiki/QA:Fedora_15_Install_Results_Template#DVD DVD media tests] to verify the network enabling ability at that step. For fedora-16, do you think if these graphical tests should be put to [https://fedoraproject.org/wiki/QA:Fedora_15_Install_Results_Template#General_Tests 'Install Variations & General Tests'] instead?[[BR]] [[BR]] In addition, These repos can also be specified in kickstart, is it an alternative way to test 'repo=' method?
I've modified the statements about stage#1 and stage#2 for the following tests:
https://fedoraproject.org/wiki/QA/TestCases/BootMethodsBootIso https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd https://fedoraproject.org/wiki/QA/TestCases/BootMethodsPxeboot https://fedoraproject.org/wiki/QA/TestCases/KickstartKsFilePathKsCfg https://fedoraproject.org/wiki/QA:Testcase_Anaconda_rescue_mode https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_URL https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_installation_source
Correct the format
Replying to [comment:10 rhe]: I've modified the statements about stage#1 and stage#2 for the following tests:
As discussed above, I've improved the install source cases and re-categorized them to [https://fedoraproject.org/wiki/Category:Installation_Repository Category:Installation_Repository]. I also updated [https://fedoraproject.org/wiki/Category:Installer_Boot_Methods Category:Installer_Boot_Methods]. I think the previous requirements have been met. Close this ticket for now, but feel free to comment and reopen it.
Log in to comment on this ticket.