When deploying 9-stream in the CI infra , I noticed an issue :
DNF version: 4.7.0 Command: dnf -d9 makecache Installroot: / Releasever: 9 cachedir: /var/cache/dnf Base command: makecache Extra commands: ['-d9', 'makecache'] Making cache files for all metadata files. baseos: has expired and will be refreshed. appstream: has expired and will be refreshed. repo: downloading from remote: baseos countme: no event for baseos: window already counted error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (https://download.cf.centos.org/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (http://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (http://mirror.0xem.ma/stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (https://mirror.0xem.ma/stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (http://mirrors.up.pt/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (https://mirrors.up.pt/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (https://mirror1.hs-esslingen.de/pub/Mirrors/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (http://mirror1.hs-esslingen.de/pub/Mirrors/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (http://mirror.netsite.dk/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (https://mirror.netsite.dk/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (http://mirror1.cl.netactuate.com/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (https://mirror1.cl.netactuate.com/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (http://linuxsoft.cern.ch/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (https://linuxsoft.cern.ch/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (http://mirror.2degrees.nz/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) (https://mirror.2degrees.nz/centos-stream/9-stream/BaseOS/x86_64/os/repodata/repomd.xml). CentOS Stream 9 - BaseOS 584 B/s | 3.9 kB 00:06 Errors during downloading metadata for repository 'baseos': - Downloading successful, but checksum doesn't match. Calculated: dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b(sha512) Expected: 8cbc4c85dca2f889c3df2d8efe73a1f287618afed2ce9b18c560d6bd9ca53a231f07a73ac70100578e7da0a7d23bbec30e48cf52cad7b2b0ad8014df3bc29d0d(sha512) Cleaning up. Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/dnf/repo.py", line 574, in load ret = self._repo.load() File "/usr/lib64/python3.9/site-packages/libdnf/repo.py", line 331, in load return _repo.Repo_load(self) libdnf._error.Error: Failed to download metadata for repo 'baseos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/dnf/cli/main.py", line 67, in main return _main(base, args, cli_class, option_parser_class) File "/usr/lib/python3.9/site-packages/dnf/cli/main.py", line 106, in _main return cli_run(cli, base) File "/usr/lib/python3.9/site-packages/dnf/cli/main.py", line 122, in cli_run cli.run() File "/usr/lib/python3.9/site-packages/dnf/cli/cli.py", line 1055, in run return self.command.run() File "/usr/lib/python3.9/site-packages/dnf/cli/commands/makecache.py", line 50, in run return self.base.update_cache(timer) File "/usr/lib/python3.9/site-packages/dnf/base.py", line 370, in update_cache self.fill_sack(load_system_repo=False, load_available_repos=True) # performs the md sync File "/usr/lib/python3.9/site-packages/dnf/base.py", line 399, in fill_sack self._add_repo_to_sack(r) File "/usr/lib/python3.9/site-packages/dnf/base.py", line 139, in _add_repo_to_sack repo.load() File "/usr/lib/python3.9/site-packages/dnf/repo.py", line 581, in load raise dnf.exceptions.RepoError(str(e)) dnf.exceptions.RepoError: Failed to download metadata for repo 'baseos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried Error: Failed to download metadata for repo 'baseos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
But
When I check the authoritative repomd.xml on mirror.stream.centos.org it's up2date :
curl --silent http://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/repodata/repomd.xml|sha512sum dd496ae5f01df5b27da7fa98cf7509c03f8db34b982e79d320cd9ecbacd4504a80e1fe5cc893447d2d5b66fe624a4d1a06b2eeeb39cddb5be9a7c1e824e8fd4b -
Also the file for repomd.xml is from oct 14th and I notice the issue still on oct 16th, so it looks like if mirrormanager crawler was stuck on 9-stream and probably with previous repomd.xml, meaning that it's invalidating all real up2date mirrors, and so no dnf transaction is possible
adding @bstinson @adrian @carlgeorge in the loop for awareness
Should be fixed now. Changes should be live in one hour. This is the same as https://pagure.io/centos-infra/issue/482
if you have a potential fix we can implement it but all nodes behind mirror.stream.centos.org are up2date and repomd.xml from two days ago so how is that possible that mirrormanager couldn't see the newer repomd.xml earlier ? Also, is that possible to always return something that dnf users can use instead of just having empty list ? for mirrorlist.centos.org we had previous entry in cache but also because we aren't using metalink . So let's assume that if crawler finds a new repomd.xml but then it would invalidate (don't know the crawler time) all mirrors until some have it (in the next crawl time) ?
Metadata Update from @zlopez: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: dev, medium-gain, medium-trouble
I think this has been solved. If there's anything further to do, feel free to reopen.
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Happening again
Linux hostname 5.14.0-105.el9.aarch64 #1 SMP PREEMPT_DYNAMIC Fri Jun 3 10:17:21 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
error: Downloading successful, but checksum doesn't match. Calculated: 92f44febb5fbf0f731af498feefb653d5b516e7184a23ab6e63b71c1faa43f18cb10a39bea249ee9467dfc010a51acd9c0314dcd69ea5e1d1e86c5bd48ed94db(sha512) Expected: 7ce2adbd012d321cabb2bf28fb3fa67e4b31f8f1f21e91643b8b79d8ea18215711e6eadb72f792e1eeff2a39bb22346eb9594078b824ebc45599962a431995e2(sha512) (http://mirror.siena.edu/centos-stream/9-stream/BaseOS/aarch64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: 92f44febb5fbf0f731af498feefb653d5b516e7184a23ab6e63b71c1faa43f18cb10a39bea249ee9467dfc010a51acd9c0314dcd69ea5e1d1e86c5bd48ed94db(sha512) Expected: 7ce2adbd012d321cabb2bf28fb3fa67e4b31f8f1f21e91643b8b79d8ea18215711e6eadb72f792e1eeff2a39bb22346eb9594078b824ebc45599962a431995e2(sha512) (http://mirror.shastacoe.net/centos-stream/9-stream/BaseOS/aarch64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: 92f44febb5fbf0f731af498feefb653d5b516e7184a23ab6e63b71c1faa43f18cb10a39bea249ee9467dfc010a51acd9c0314dcd69ea5e1d1e86c5bd48ed94db(sha512) Expected: 7ce2adbd012d321cabb2bf28fb3fa67e4b31f8f1f21e91643b8b79d8ea18215711e6eadb72f792e1eeff2a39bb22346eb9594078b824ebc45599962a431995e2(sha512) (https://mirror.shastacoe.net/centos-stream/9-stream/BaseOS/aarch64/os/repodata/repomd.xml). error: Downloading successful, but checksum doesn't match. Calculated: 92f44febb5fbf0f731af498feefb653d5b516e7184a23ab6e63b71c1faa43f18cb10a39bea249ee9467dfc010a51acd9c0314dcd69ea5e1d1e86c5bd48ed94db(sha512) Expected: 7ce2adbd012d321cabb2bf28fb3fa67e4b31f8f1f21e91643b8b79d8ea18215711e6eadb72f792e1eeff2a39bb22346eb9594078b824ebc45599962a431995e2(sha512) (http://mirror.net.cen.ct.gov/centos-stream/9-stream/BaseOS/aarch64/os/repodata/repomd.xml).
+1 for experiencing this issue again
Login to comment on this ticket.