So... this is the f39/40 oci (images/format?)? or the direct ostree refs? or both?

On the oci side I think thats because we stopped doing that every night in the container build cron and we need to fix that.

The ostree refs I do see updating correctly... so I am not sure whats going on there.

Sorry, I wasn't clear. This is about the ostree remote refs only (the official ones).

$ ostree remote summary fedora
...
* fedora/40/x86_64/silverblue
    Latest Commit (62,7 kB):
      762d22937621dae35f1ecd66d3d4b0a4cc80890fd2fe203a49d6a19a65e338e7
    Version (ostree.commit.version): 40.20240419.n.0
    Timestamp (ostree.commit.timestamp): 2024-04-19T10:09:50+02

* fedora/40/x86_64/testing/silverblue
    Latest Commit (66,0 kB):
      e00e91c7d12dcf1d68158598b3c9b2ac7f39fec92e45063b28a98780ad19adc0
    Version (ostree.commit.version): 40.20240422.0
    Timestamp (ostree.commit.timestamp): 2024-04-22T04:09:18+02

* fedora/40/x86_64/updates/silverblue
    Latest Commit (62,5 kB):
      e84ef63581f1c237ff4d7911bba78b7c0b69cfd5da49810653ab16b84659695f
    Version (ostree.commit.version): 40.20240422.0
    Timestamp (ostree.commit.timestamp): 2024-04-22T02:32:45+02

testing & updates are good. The "stable" one is not linked to the updates one.

It looks a lot like https://docs.pagure.org/releng/sop_composing_fedora.html#update-silverblue-refs (from https://pagure.io/releng/issue/11773#comment-883316 and https://pagure.io/releng/issue/11773#comment-883611)

Yeah... @jnsamyak Did you get to/do that part of the SOP?

also, its out of date only mentioning silverblue... and of course the release is wrong/should be replaced.

Okay so me and @kevin just discussed this on the releng channel and based on that below is the script - the requirement was to only delete the base reference and then alias the update reference for different artifacts, considering that not all artifacts have support for all architectures

PR for same: https://pagure.io/releng/pull-request/12073

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

#!/bin/bash

# Define the base directory
REPO_DIR="/mnt/koji/ostree/repo"

# Define the artifacts and the architectures they support
declare -A artifacts
artifacts["silverblue"]="x86_64 aarch64 ppc64le"
artifacts["kionite"]="x86_64 aarch64"
artifacts["onyx"]="x86_64"
artifacts["sericea"]="x86_64 aarch64"

# Define the Fedora version
read -p "Enter the Fedora version (e.g., 31): " fedora_version

# Function to update refs for a given artifact and architecture
update_refs() {
  local artifact=$1
  local fedora_version=$2
  local arch=$3

  # Navigate to the repository directory
  cd $REPO_DIR

  # Delete the old ref
  sudo ostree refs --delete fedora/${fedora_version}/${arch}/${artifact}

  # Create alias
  sudo -u ftpsync ostree refs --alias --create=fedora/${fedora_version}/${arch}/${artifact} fedora/${fedora_version}/${arch}/updates/${artifact}
}

# Loop through each artifact and its supported architectures
for artifact in "${!artifacts[@]}"; do
  archs=${artifacts[$artifact]}
  for arch in $archs; do
    update_refs "${artifact}" "$fedora_version" "$arch"
  done
done

# Update the ostree summary
sudo ostree summary -u

echo "OSTree refs update completed for all specified artifacts on Fedora $fedora_version."

It is a matter of fact if @dustymabe or @siosm confirm this behavior so we can proceed with this and be double sure about things?

That looks good to me but I've not tried it nor done it before. I'd recommend doing it in steps / with a pause to verify the result.

Turns out we have:

https://pagure.io/releng/blob/main/f/scripts/update_ostree_refs.sh

already, which does this same thing?

But I don't understand the first step of making a updates ref from the base one? Is that for when updates hasn't been composed yet, because we already do have updates refs right?

But I don't understand the first step of making a updates ref from the base one? Is that for when updates hasn't been composed yet, because we already do have updates refs right?

I think, if executed at the right time, there would be no updates ref because no updates composes would have happened yet and thus the first step would be to create the updates ref. Creating the updates ref based on the existing main ref would ensure we had continuity of OSTree history (think like git commit history) all the way back to when the candidate composes were created.

So we should update the branching SOP to include this step early on, before we turn on the updates repos.

If we don't want to loose the history for F40, we can still run the script and then make sure to trigger an update compose so that we get it updated to the last packages. We'll loose a little bit of history but that should be fine.

We'll need to add the update ref deletion to the current script

Hopefully this all goes away once we've completed the transition to containers but we're not there yet :/

@siosm you mention to add "update ref deletion" to the current script but aren't
we already doing that, check below [1]. Or do you mean something else?

[1] https://pagure.io/releng/blob/main/f/scripts/update_ostree_refs.sh#_15

No, this deletes the main ref. The command before creates an update ref based on the main ref, but in this case as we already have an update ref, it will likely fail.

So we need to delete the update ref (losing its history, but that's just a few days), create the update one from the main one to keep the history, then delete the main one as the history is now safe in the update ref, then re-create the main one from the update one as it's this one that is going to be updated by update composes in the future.

At least that's my understanding.

Ideally in the future the script works as is if we run it at the right time. As a one shot for this release, we need to change it or run a few manual commands before.

Hum, ok. I guess that makes sense.

Would anyone care to propose a script (or mod to the existing one) for this? I can look at it more later in the morning after release stuff is flowing out ok...

I have a quick PR based on this ready someone can review it? https://pagure.io/releng/pull-request/12076

CC: @siosm

Okay the script ran and the new stable compose was requested, hopefully it should fix this now!

Thanks!

ok, updates push finished.

Please confirm that things look correct now? :)

Everything looks good to me now. Thanks!

$ +ostree-remotes
silverblue  38               39               40               rawhide
x86_64      38.20240423.0    39.20240423.0    40.20240423.1    Rawhide.20240423.n.0
aarch64     38.20240423.0    39.20240423.0    40.20240423.1    Rawhide.20240423.n.0
ppc64le     38.20240423.0    39.20240423.0    40.20240423.1    Rawhide.20240423.n.0

kinoite     38               39               40               rawhide
x86_64      38.20240423.0    39.20240423.0    40.20240423.1    Rawhide.20240423.n.0
aarch64     38.20240423.0    39.20240423.0    40.20240423.1    Rawhide.20240423.n.0
ppc64le     38.20240423.0    39.20240423.0    40.20240423.1    Rawhide.20240423.n.0

sericea     38               39               40               rawhide
x86_64      38.20240423.0    39.20240423.0    40.20240423.1    Rawhide.20240423.n.0
aarch64     38.20240423.0    39.20240423.0    40.20240423.1    Rawhide.20240423.n.0

onyx        38               39               40               rawhide
x86_64                       39.20240423.0    40.20240423.1    Rawhide.20240423.n.0

$ +ostree-remotes updates
silverblue  38               39               40               rawhide
x86_64      38.20240423.0    39.20240423.0    40.20240423.1                   
aarch64     38.20240423.0    39.20240423.0    40.20240423.1                   
ppc64le     38.20240423.0    39.20240423.0    40.20240423.1                   

kinoite     38               39               40               rawhide
x86_64      38.20240423.0    39.20240423.0    40.20240423.1                   
aarch64     38.20240423.0    39.20240423.0    40.20240423.1                   
ppc64le     38.20240423.0    39.20240423.0    40.20240423.1                   

sericea     38               39               40               rawhide
x86_64      38.20240423.0    39.20240423.0    40.20240423.1                   
aarch64     38.20240423.0    39.20240423.0    40.20240423.1                   

onyx        38               39               40               rawhide
x86_64                       39.20240423.0    40.20240423.1