we might want to improve STG and PROD website automation/publication

5 days ago, I changed footer in https://pagure.io/fedora-l10n/fedora-localization-statistics/c/ea3c22004abfc99a7ec4f6f882ffa010836adb89?branch=main to add generation dates

8 days ago, I changed footer in https://pagure.io/fedora-l10n/fedora-localization-statistics/c/393475499983e5d569dd6ef75c72aa35dafebbdd?branch=main to add link to this repository

https://languages.stg.fedoraproject.org/ don't show any of these changes
https://languages.fedoraproject.org/ shows everything

open questions:

• what should be the correct rhythm to publish a new version?
• what should be the purpose of staging in the context of this website?

our tasks look like to be organized globally

stats-latest: runs à 4AM every days:

    ./build.py --results f$(RELEASEVER)
    ./build_language_list.py --results f$(RELEASEVER) --refresh &&
    ./build_language_list.py --results f$(RELEASEVER) --analyzealllang &&
    ./build_tm.py --results f$(RELEASEVER) --compress &&
    ./build_stats.py --results f$(RELEASEVER)

build-website: runs every two hours:

./build_website.py for each of our results folders
build static files with hugo and publish it

all of this takes a huge amount of resources and delay

proposal for staging

• should allow us to test the whole publication system works
• should test all new extraction & build files on a limited set of packages (ie: packages_regex="blue.*)
• should be done accross multiple releases to test new content results
• should run on its own code branch staging

proposal for production

• build.py - the script that goes through all srpm packages - should only be run again in case of a Fedora package changes
• no need to execute this on old fedora releases if the python script have no evolution
• run once a week is good enough
• ./build_language_list.py, ./build_tm.py and ./build_stats.py - should only be run if there is a package change or a source code change
• ./build_website.py and hugo can be run once a day
• published website should explicitly tell us last content extraction and content were generated
• published website should provides a view on functional errors to be fixed
• if a task fails, previous content should remain intact, and log should allow to understand what happened

conclusion

I have no idea how to make this possible with Ansible
but I'll be happy to help by making the required changes in code to make this possible