I've had this on my personal task list for a while now, but it turns out to be rather subtler than I thought, at least for Rawhide.
We can do it quite straightforwardly for Branched, but we can't really do it yet (except blind) as we don't have a Branched right now.
The problem for Rawhide is that Rawhide doesn't really work like other releases. The fact that our console upgrade test works for Rawhide is actually kind of odd, and relies on a messy hack someone put in place at some point.
So here's the skinny. There are various Fedora repo definitions. The 'stock' ones are in fedora-repos. There are special Rawhide repo definitions in the package fedora-repos-rawhide.
fedora-repos
fedora-repos-rawhide
According to Dennis Gilmore, the 'official' way to use Rawhide is to install fedora-repos-rawhide and then enable the repository rawhide and disable the repos fedora, updates and updates-testing. To upgrade to Rawhide, you're supposed to do that, and then run dnf system-upgrade download --releasever=26 --nogpgcheck (at present - you have to know the 'right' release number for Rawhide and use it).
rawhide
fedora
updates
updates-testing
dnf system-upgrade download --releasever=26 --nogpgcheck
We don't do anything like that. Instead we just leave the repo definitions as they are (we don't install fedora-repos-rawhide at all) and do dnf system-upgrade download --releasever=rawhide --nogpgcheck.
dnf system-upgrade download --releasever=rawhide --nogpgcheck
...and this works. Which Dennis was quite surprised about, so I went and looked into why. Here's why!
When you pass --releasever to dnf you're basically telling it 'just go ahead and use whatever repos are configured (or otherwise specified on the command line), but when you see $releasever, don't replace it with whatever you detected the current installed system's release version is. Replace it with this value instead'. So when we install Fedora - say Fedora 25 - and use the default repos from fedora-repos and pass --releasever=rawhide, we're telling dnf to use the repos fedora and updates - since those are the ones that are enabled by default - but to replace $releasever in their values with rawhide, not 25.
--releasever
$releasever
--releasever=rawhide
25
OK, so what do the repos actually say? Well, the interesting bits of fedora look like this:
[fedora] #baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/os/ metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$basearch gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$releasever-$basearch skip_if_unavailable=False
the interesting bits of updates look like this:
[updates] #baseurl=http://download.fedoraproject.org/pub/fedora/linux/updates/$releasever/$basearch/ metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-f$releasever&arch=$basearch gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$releasever-$basearch skip_if_unavailable=False
so basically we're going to wind up hitting these URLs, on x86_64:
the first of those is fine, it's what the rawhide repo definition from fedora-repos-rawhide says anyway, if you use that repo. The second one, however, is kinda crazy, and it's what Dennis was expecting to trip us up, because Rawhide has no updates repository. However, at some point, someone has put some kinda hack in place: if you hit that URL, you'll see that it works, and it's actually set up to work exactly like the fedora-rawhide target. This is also the case for the updates-testing-frawhide target. Basically, someone at some point told MirrorManager to treat updates-testing-frawhide and updates-released-frawhide just the same as fedora-rawhide, which makes using dnf --releasever=rawhide with the default repos 'work' (though it's a bit of a hack as you wind up using two or three identical copies of the same repo, depending on whether you have u-t enabled).
fedora-rawhide
updates-testing-frawhide
updates-released-frawhide
dnf --releasever=rawhide
Why does all this matter? Well, because now we more or less know how dnf behaves if you just tell it "I want to upgrade to 'rawhide'", but I don't know what GNOME Software will do. I've poked into a bit and so far made it as far as pk_task_upgrade_system_async in PackageKit. Right now GNOME Software won't even offer an upgrade to Rawhide, ever; that's fairly trivial to fix, and I might patch it tomorrow and play around just to see what happens next. But I wanted to jot all this down somewhere so we have the notes available while we look at what G-S / PK actually do if you say 'upgrade me to Rawhide, please'. We should probably test both what happens if you leave the 'default' Fedora repos in place, and what happens if you disable fedora and updates and enable rawhide.
pk_task_upgrade_system_async
Right now, Dennis still maintains that just using the default repos and dnf --releasever=rawhide is wrong and we shouldn't do that, we should switch the repo definitions before running a Rawhide upgrade. I'm kinda of the opinion that it'd be nice to just bless the --releasever=rawhide approach, coming up with a slightly more elegant hack for what MirrorManager does for the updates and updates-testing repos in that case, because it'd prevent us having to have a variant repo definition for rawhide and tell people to manually change the repo configuration when they want to use Rawhide (and having the fedora and updates repos disabled by default in the Rawhide fedora-repos package, and remembering to keep flipping that back to enabled when we branch). But I could be convinced otherwise, I want to kick it around with Dennis. If we do decide the separate rawhide repo config is the one and only true way to do it, we should adjust the console tests to do the repository reconfiguration when upgrading to Rawhide.
This ticket is a duplicate of https://pagure.io/fedora-qa/fedora_openqa/issue/48
The dupe status is intentional; this task requires changes to both projects.
One rather obvious point I didn't note here before is: we can potentially suppress the test for Rawhide, but this depends a bit on openQA. In the templates you can specify which distro versions each test should run for. However, I'm not sure if it supports any nice ways to say "anything but Rawhide". If it does, this should be pretty easy. If not, maybe we should patch it.
Metadata Update from @adamwill: - Issue tagged with: newtest
I have submitted this issue as a project for Red Hat Open Source contest. Let us see, what it brings to us.
Yep. In our meeting I forgot I'd already written up the issues in this ticket :)
The write-up above is still pretty much correct and explains the 'unique' problems we have here. The ideas I've had for dealing with this so far basically boil down to:
version
product
*
https://gitlab.gnome.org/GNOME/gnome-software/issues/124 is a ticket where I requested GNOME Software upstream make upgrading to Rawhide possible, though it seems that so far the answer is 'nope'.
So, would there be an option to create the images for two previous versions and have them upgraded to the current version using the gsettings set org.gnome.software show-upgrade-prerelease true? when we cannot make Gnome to upgrade to Rawhide.
gsettings set org.gnome.software show-upgrade-prerelease true
Last time when we discussed this, we had space issues, but maybe we would have some space available nowadays? @adamwill
We still have "the Rawhide problem" here, AFAIK. Nothing has been done to resolve that.
OK, let's resolve it today! 🙂
We can easily convince GNOME Software to upgrade to Rawhide by editing ~/.cache/gnome-software/fedora-pkgdb-collections/fedora.json (this file is created once gnome-software process starts, and then re-downloaded when it refreshes the repos periodically or manually), finding this section:
~/.cache/gnome-software/fedora-pkgdb-collections/fedora.json
gnome-software
{ "allow_retire": true, "branchname": "master", "date_created": "2014-05-14 12:36:15", "date_updated": "2023-08-08 17:07:23", "dist_tag": ".fc40", "koji_name": "rawhide", "name": "Fedora Linux", "status": "Under Development", "version": "devel" },
and then changing "version": "devel" to "version": "40" (if needed, that number can be inferred from "dist_tag": ".fc40"). You don't even need to restart gnome-software process (but it can't hurt), and F40 is now visible in the Updates tab, yay!
"version": "devel"
"version": "40"
"dist_tag": ".fc40"
I just tested F39->F40 upgrade using GNOME Software in a VM, worked without an issue. 🎉
I can help you to write a jq or python command to adjust the json, if you want.
jq
python
Now, can we please finally get the GNOME Software test cases automated? It's a lot of manual work each release. Thanks! 😎 🍹 🥥 🌴
thanks for that, yes, that should be possible I think. I can try and get to it at some point, or lruzicka can...or you can take a shot...:D
note this probably relies on mirrormanager behaviour which you're technically not supposed to rely on, but....mehhh. it's worked for years, and I think we possibly rely on it in other places anyway.
I will take it, @adamwill, and start working on it.
Metadata Update from @lruzicka: - Issue assigned to lruzicka
Fix available as #312.
I have uploaded the changes to staging to see if it works without issues.
staging
In Staging, I was able to run three of the four tests:
The last one ended up somewhere else, I guess I will need to revisit some needle. Please, do not wipe any stored needles beginning in upgrade_ as I did not have time to copy them over.
upgrade_
This is now done! Great job, @lruzicka . https://pagure.io/fedora-qa/os-autoinst-distri-fedora/pull-request/312
Metadata Update from @adamwill: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
🎆 🎉 🥳 Thanks, Lukas!
Log in to comment on this ticket.