From 977d05c7cea796ba5f2191eb1d8f8db3cdf3c001 Mon Sep 17 00:00:00 2001
From: Michal Konecny <mkonecny@redhat.com>
Date: Oct 02 2024 10:50:19 +0000
Subject: [PATCH 1/3] Add read-only troubleshooting guide to koji SOP


As this is happening relatively often lately it would be good to add guide.

Signed-off-by: Michal Konecny <mkonecny@redhat.com>

---

diff --git a/modules/sysadmin_guide/pages/koji.adoc b/modules/sysadmin_guide/pages/koji.adoc
index 6a6601b..4de068f 100644
--- a/modules/sysadmin_guide/pages/koji.adoc
+++ b/modules/sysadmin_guide/pages/koji.adoc
@@ -99,3 +99,22 @@ After restarting any `s390x` machine in `inventory/builders` `[runroot]` group s
 You need to have access to Bitwarden Vault for the password prompt.
 ====
 
+=== OSError: [Errno 30] Read-only file system: "/var/tmp/koji/tasks/xxx"
+
+For more information about this issue see link:<https://bugzilla.redhat.com/show_bug.cgi?id=2312886>[relevant bugzilla ticket].
+The issue is also reported on parent task, make sure you found the exact task that failed to find the failing host.
+Only way to deal with it currently is to disable the builder with issue in koji and reinstall it again.
+
+==== Disabling builder in koji
+
+. Generate a kerberos ticket
+. koji disable <builder_with_issue>
+
+==== Reinstalling the builder
+
+. Run `groups/buildvm.yml` ansible playbook with `--limit <builder_hostname>`
+
+[NOTE]
+====
+You need to be koji admin.
+====

From a00e7fc8286c092e72bc50081c97ec1bf4bc8f8c Mon Sep 17 00:00:00 2001
From: Michal Konecny <mkonecny@redhat.com>
Date: Oct 02 2024 10:50:19 +0000
Subject: [PATCH 2/3] Add another option of disabling builder


Signed-off-by: Michal Konecny <mkonecny@redhat.com>

---

diff --git a/modules/sysadmin_guide/pages/koji.adoc b/modules/sysadmin_guide/pages/koji.adoc
index 4de068f..8db08bb 100644
--- a/modules/sysadmin_guide/pages/koji.adoc
+++ b/modules/sysadmin_guide/pages/koji.adoc
@@ -108,13 +108,14 @@ Only way to deal with it currently is to disable the builder with issue in koji 
 ==== Disabling builder in koji
 
 . Generate a kerberos ticket
-. koji disable <builder_with_issue>
-
-==== Reinstalling the builder
-
-. Run `groups/buildvm.yml` ansible playbook with `--limit <builder_hostname>`
+. `koji disable <builder_with_issue>`
 
 [NOTE]
 ====
-You need to be koji admin.
+If you don't have permissions to disable builder using `koji disable` command. Ssh to the machine and stop
+kojid service by running `systemctl stop kojid`. This will prevent the machine to serve any more builds.
 ====
+
+==== Reinstalling the builder
+
+. Run `groups/buildvm.yml` ansible playbook with `--limit <builder_hostname>`

From a7f7b78d6823e6751dfd9c569b62c262fdc8c5a6 Mon Sep 17 00:00:00 2001
From: Michal Konecny <mkonecny@redhat.com>
Date: Oct 02 2024 10:50:19 +0000
Subject: [PATCH 3/3] Add guide for destroying the virt instance


Signed-off-by: Michal Konecny <mkonecny@redhat.com>

---

diff --git a/modules/sysadmin_guide/pages/koji.adoc b/modules/sysadmin_guide/pages/koji.adoc
index 8db08bb..8700320 100644
--- a/modules/sysadmin_guide/pages/koji.adoc
+++ b/modules/sysadmin_guide/pages/koji.adoc
@@ -118,4 +118,5 @@ kojid service by running `systemctl stop kojid`. This will prevent the machine t
 
 ==== Reinstalling the builder
 
+. Destroy the VM instance by following xref:howtos:destroy_a_virt_instance.adoc[How to destroy a virt instance]
 . Run `groups/buildvm.yml` ansible playbook with `--limit <builder_hostname>`