Fedora 34 final will ship with newer versions of shim and GRUB than will be available for Fedora 34 beta. It's a significant, but unavoidable, change due to numerous security improvements primarily related to UEFI Secure Boot.
This poses risks for regressions whether or not UEFI Secure Boot is enabled; and while BIOS systems don't use shim at all, they will use a much newer GRUB. While this is a significant change that will be getting on-going testing before and after test days, we really need a wide assortment of firmware to discover regressions.
There's some consideration that shim and GRUB will also be updated in Fedora 33. If F33 and F34 will use GRUB 2.06, it might make sense to have combined test days? If F33 will have backports to GRUB, it might make sense to split up test days.
@javierm Do you have a preferred date? Or a "no earlier/later than" date range? Realistically this has to happen, and since it's very likely to get blocker status (as it did for F33 but we ended up punting to now) both beta and final freezes won't apply.
systemd-oomd test week is 03-18 to 03-26 which I'm part of; but I can do double duty. I defer to @sumantrom and @kparal if that's realistic, or should be avoided. Oomd may not need that whole week, and could be shortened by one or two days, if it's optimal to work in shim/grub test day(s) that week.
Draft idea that needs some critical feedback. I'm thinking the test page needs to explain some prerequisite step so that the user knows what firmware type (UEFI/BIOS) they have, and whether or not UEFI Secure Boot is enabled, and then to run the test case for their specific situation.
NOTE: If we're combining F33 and F34 testers, we'll need to be careful to warn users against stepping on /boot/efi/EFI/fedora/grub.cfg on F34 with grub2-mkconfig. Off hand I don't think that command will be needed (?) on Fedora 34 at all.
I am willing to have it in that week also will be great if we can have the exact days nailed down. I am gonna create test day pages and put them here
Metadata Update from @sumantrom: - Issue assigned to sumantrom - Issue set to the milestone: Fedora 34 - Issue tagged with: test cases, test days
@chrismurphy I am thinking to have a test day on 25th for the Grub and 29th for shim ? What do you think?
Both dates work for me. We probably could do separate BIOS and UEFI days? @javierm any opinion on that? Since BIOS will only use GRUB, and it 's nearly ready now, we could do that for March 25th.
Whereas with UEFI the idea is to test both new shim and GRUB. But, I think we need to hold off on setting a date until there's word on when a signed shim will be available.
Both dates work for me. We probably could do separate BIOS and UEFI days? @javierm any opinion on that? Since BIOS will only use GRUB, and it 's nearly ready now, we could do that for March 25th. Whereas with UEFI the idea is to test both new shim and GRUB. But, I think we need to hold off on setting a date until there's word on when a signed shim will be available.
If 25th is fixed, I would like to create the pages and fix else we can give it a few days more and do it around 29th? How is OOMd going on?
Both dates work for me as well. Agreed that BIOS could be tested but it's better to wait for the new shim for EFI.
I'm good with March 29th for a tentative date. I'm only 1/2 expecting signed shim to be ready by then, but we can make a decision on March 26th whether to push it back.
In the meantime I'll see about writing up some draft test cases.
Hello friends, any updates on this?
@geraldosimiao No signed shim yet; once there's a heads up, or it lands, we can schedule the test day.
@chrismurphy I will create a stub wiki for this event along with some test cases, lets keep the bits ready, change the dates and run on D-day.. sounds good?
Sounds good. The most minimalist test case that I'd like to see get widespread use is "make a USB stick and try to boot". They don't need to install. If they don't even get to a GRUB menu, it might be a shim problem. If they don't get a boot scroll it might be a GRUB problem. if there's some failure later on, it's probably neither shim nor GRUB.
Awesome!
Signed shim has landed in bodhi. https://bodhi.fedoraproject.org/updates/FEDORA-2021-cab258a413
booking the 12 and 13 for Bootloader test day. On top of this one right now.
Wiki : https://fedoraproject.org/wiki/Test_Day:2021-04-12_Grub_and_Shim_Test_Day
@chrismurphy would you be kind enough to review the BIOS and UEFI test cases and the wiki page?
@javierm Any other info we should ask users to collect and include in results?
Shim related: Maybe dmesg | grep DMI: to track make/model/firmware? Is mokutil --sb-state equivalent to kernel secureboot: message or is one preferred over the other? (I guess it'd be a bug if they conflict.)
dmesg | grep DMI:
mokutil --sb-state
secureboot:
I think the tests are pretty much: (a) can you boot and (b) if SB was enabled before upgrading is it still enabled? Anything else?
@adamwill is having a problem with his XPS 13 (9360) and new shim. So for now new shim is being held in updates-testing. We can still make a test day specific image with new shim. I think we need to consider a low impact first test:
Create a USB stick with the test day F34 image, and just try booting it.
If that works then the user can consider updating shim on their installed system.
Otherwise, it could be difficult for users to revert if shim prevents boot, and I'd rather not recommend that they disable Secure Boot as a work around.
The results page link goes to https://testdays.fedoraproject.org/events/111 and gives an Internal Server Error.
@sumantrom , did you create an event in testdays? I just checked and don't see it in the database. (Error 500 is ofc not ideal where it should be 404). I'd go ahead and do that myself, but I don't see https://fedoraproject.org/wiki/Test_Day:2021-04-12_Grub_and_Shim_Test_Day_TestdayApp_Metadata either.
It doesn't. I am trying to figure out the test cases. I tried and running with the update and I didn't face any issues except once when the secure boot is enabled. I will finish up the test case and wire up by tonight.
The meta has been created long back https://fedoraproject.org/wiki/Test_Day:2021-04-12_Grub_and_Shim_TestdayApp_Metadata
https://testdays.fedoraproject.org/events/111
@chrismurphy do you want to build the image? or have people use latest and install the update?
I don't know how to do a test day compose, but Adam said it's possible so I'm guessing if no one else does it, he'll do it. :rabbit:
https://fedorapeople.org/groups/qa/test_days/
Thanks for the compose, I will wire up things in the test day page in a better way !
(Deleted my earlier comment)
I think it'll look like this:
What I realized is that even in the non-secure boot case, shim is being pointed to out of the box, so even though it should be a no-op, it may not be and we should test UEFI the same across the board. It is not important for UEFI testers to update their installed shim at this time.
For what it's worth, recommending https://fedorapeople.org/groups/qa/test_days/01136096-FEDORA-2021-cab258a413-netinst-x86_64.iso is gentler on the tester's download requirements. We don't care about the ensuing environment at all. Just that they get to GRUB really...
@sumantrom Is the test day really today? I don't see it in the calendar.
Yes My bad. I forgot to update it over the weekend. It's added now :)
@chrismurphy I have transferred the results to https://fedoraproject.org/wiki/Test_Day:2021-04-12_Grub_and_Shim_Test_Day#Test_Results
Do you have any closing comments ??
Thanks for all the help and co-ordination!!
It went well, we found some bugs and they got fixed before release. Thanks!
Thanks! I look forward to work with you on the upcoming release cycle and many many more to come :)
Metadata Update from @sumantrom: - Issue close_status updated to: Fixed - Issue set to the milestone: None (was: Fedora 34) - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.