#11552 Allow CentOS Integration SIG to sent Gitlab events to the message bus
Closed: Fixed with Explanation 3 months ago by kevin. Opened 8 months ago by bookwar.

On behalf of CentOS Integration SIG (sig-integration in CentOS accounts) I'd like to resurrect the service which posts gitlab events to Fedora Message Bus (see https://gitlab.com/CentOS/Integration/general/-/issues/2)

For that we need credentials and topic to send messages to.

Describe what you would like us to do:

  • Create a topic

I am not sure if there is a convention for SIG messages already, something like org.centos.sig.integration.gitlab ?

Example of the message which was sent previously before decommissioning of the service:
https://apps.fedoraproject.org/datagrepper/v2/id?id=0c2dcfcd-8908-4c1b-94ed-b7ef03c4c77d&is_raw=true&size=extra-large

It used org.centos.prod.gitlab.* namespace, but we would like to shift it under CentOS Integration SIG umbrella

  • Provide credentials for sending messages to Fedora message bus to that topic

When do you need this to be done by? (YYYY/MM/DD)

This is not specifically urgent, so let's say a 2023/10/20 ?


Metadata Update from @phsmoura:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: low-gain, low-trouble, ops

8 months ago

So, you need a cert/username and will need to make a message schema...

I can generate the name/cert, but I am not sure the details on the message schema. @abompard or @zlopez might be able to point to more info there.

There seem to be a user for that service already: gitlab-centos (and gitlab-centos.stg). I'll allow the proposed topic to the allowed topics.

I don't know however how the service is running, since it's running on CentOS' openshift instance. I'm happy to help any centos sysadmin with the connection to the Fedora message bus when they're ready.

Shouldn't this be filed with centos-infra? https://pagure.io/centos-infra

Let's get back to this.

There seem to be a user for that service already: gitlab-centos (and gitlab-centos.stg).

@abompard That service is dead, and afaik there are no plans in CPE to resurrect it (at least for now). See https://issues.redhat.com/browse/CS-1524 That's why we are trying to create the alternative on the CentOS Integration SIG platform through a community initiative.

I want to reuse the code from https://github.com/CentOS/gitlab-webhooks/ but I will create a completely new deployment for it in CentOS SIG Infra at https://console-openshift-console.apps.ocp.cloud.ci.centos.org/topology/ns/cs-gitlab-gate?view

The user you mention was originally created as being owned by CPE admins. I am asking for a new account to be owned by the CentOS Integration SIG.

I don't think reusing the existing account is a good idea, as who knows where those credentials were used and which access they had.

Shouldn't this be filed with centos-infra? https://pagure.io/centos-infra

@ngompa I am not requesting anything from the CentOS Infra, I think? I need access to post messages to a certain topic of the Fedora Message bus. The rest is being tracked and discussed in CS-1524 and https://gitlab.com/CentOS/Integration/general/-/issues/2

So, you need a cert/username and will need to make a message schema...

@kevin i think I need just certificates and a topic

The message schema already is implemented in https://github.com/CentOS/gitlab-webhooks/blob/master/images/gitlab-webhooks/gitlab-webhook/gitlab_webhook.py#L104 I would just reuse it.

But for the topic I don't want to reuse the one from https://github.com/CentOS/gitlab-webhooks/blob/master/docker-compose.yaml#L14 for the same reason as I want a new user. This is the SIG infra and I think it shouldn't claim the "official" name of org.centos.prod.gitlab.*. Rather it should use something marked as owned by the SIG, like in org.centos.sig.integration.*

I'd love to have this thing running before Feb 1st (CentOS Connect and FOSDEM) so that I can show and describe how it works at the CentOS Integration SIG Meetup at CentOS Connect.

@abompard any thoughts on what topic could be used here?

I'm happy to create the cert / queue once we settle on names? (or you can! :)

Using the org.centos.sig.integration.* is fine, do you want to have the .prod. prefix in there as well? (before .sig.) I personally don't think it's necessary, we've had that in Fedora for historic reasons but they shouldn't be necessary with the new rabbitmq-based bus.

I'll delete the gitlab-centos{,.stg} users if the service is dead. I can create a centos-integration-gitlab user instead and allow it to publish to the topic.

With regards to schemas, the file you pointed to is using the default Fedora Messaging schema, which will work, but will not let you generate notifications, set severities, declare a format for your data and help you avoid sending incompatible data to your consumers. I can help bootstrap or review a specific schema if the docs or the cookiecutter template are not enough.

@abompard Thanks, I now understand the schema part better. We definitely need it, but it will need some work. I'am going to try to use the cookiecutter as a starting point.

I do want to clear the credentials part out of my subconscious though, so that we can then work on the schema at our own pace.

I think we finalized it now:
we use org.centos.sig.integration.* without prod, and we need a new user to be able to post there and nowhere else.

@kevin Let's just do this! :)

Sure. Can you point me to or email me a gpg key I can use to encrypt things? Or another secure way to get you the certs?

Please use the GPG key on my Fedora account: Key ID 5770831D8B19E198

Huh. I can't seem to find it on keys.openpgp.org... Can you check the fingerprint?

Sorry, I use gpg keys so rarely, I always mess up when I do :)

Should be ok now:

$ gpg2 --send-keys --keyserver keys.openpgp.org E86A391503746B23E740E8995770831D8B19E198
gpg: sending key 5770831D8B19E198 to hkp://keys.openpgp.org

Sorry, this dropped off my radar. ;(

@abompard I don't see the user here? Did you already create it? It doesn't seem to be in ansible?

Ooops, dropped off mine as well. The great FOSDEM purge I suppose.

  • deleted the old gitlab-centos RabbitMQ user
  • created the centos-integration user (and centos-integration.stg in staging)
  • created their certificates in the private repo
  • allowed them publication to topics matching org.centos.sig.integration.*

I have mailed the certs to @bookwar

I think this can be closed now... feel free to re-open if I missed anything.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed with Explanation
- Issue status updated to: Closed (was: Open)

3 months ago

Login to comment on this ticket.

Metadata
Boards 1
ops Status: Backlog