releng

#9695 kerberos error in coreos-koji-tagger
Closed: Fixed by mohanboddu. Opened by dustymabe.

Closed: Fixed
Reopen Issue
  • Describe the issue

I've seen this a few times over the past few weeks where coreos-koji-tagger gets in a weird state and the ERROR requests_gssapi.gssapi_ - generate_request_header(): acquiring credentials failed: message is presented:

2020-08-18 18:53:14,756 INFO builtins - Watching commits against branches ['refs/heads/testing-devel', 'refs/heads/next-devel'] of coreos/fedora-coreos-config repo
2020-08-18 18:53:14,756 INFO builtins - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2020-08-18 18:53:14,761 INFO builtins - Found principal coreos-koji-tagger/coreos-koji-tagger.fedoraproject.org@FEDORAPROJECT.ORG in keytab
2020-08-18 18:53:14,781 ERROR requests_gssapi.gssapi_ - generate_request_header(): acquiring credentials failed:
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/requests_gssapi/compat.py", line 50, in generate_request_header
    self.creds = gssapi.Credentials(name=name, usage="initiate")
  File "/usr/lib64/python3.8/site-packages/gssapi/creds.py", line 63, in __new__
    res = cls.acquire(name, lifetime, mechs, usage,
  File "/usr/lib64/python3.8/site-packages/gssapi/creds.py", line 136, in acquire
    res = rcreds.acquire_cred(name, lifetime,
  File "gssapi/raw/creds.pyx", line 158, in gssapi.raw.creds.acquire_cred
gssapi.raw.misc.GSSError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529639053): Can't find client principal coreos-koji-tagger\/coreos-koji-tagger.fedoraproject.org/fedoraproject.org@FEDORAPROJECT.ORG in cache collection
2020-08-18 18:53:14,782 ERROR requests_gssapi.gssapi_ - Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529639053): Can't find client principal coreos-koji-tagger\/coreos-koji-tagger.fedoraproject.org/fedoraproject.org@FEDORAPROJECT.ORG in cache collection
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/requests_gssapi/compat.py", line 50, in generate_request_header
    self.creds = gssapi.Credentials(name=name, usage="initiate")
  File "/usr/lib64/python3.8/site-packages/gssapi/creds.py", line 63, in __new__
    res = cls.acquire(name, lifetime, mechs, usage,
  File "/usr/lib64/python3.8/site-packages/gssapi/creds.py", line 136, in acquire
    res = rcreds.acquire_cred(name, lifetime,
  File "gssapi/raw/creds.pyx", line 158, in gssapi.raw.creds.acquire_cred
gssapi.raw.misc.GSSError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529639053): Can't find client principal coreos-koji-tagger\/coreos-koji-tagger.fedoraproject.org/fedoraproject.org@FEDORAPROJECT.ORG in cache collection
2020-08-18 18:53:14,783 ERROR requests_gssapi.gssapi_ - generate_request_header(): acquiring credentials failed:
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/requests_gssapi/compat.py", line 50, in generate_request_header
    self.creds = gssapi.Credentials(name=name, usage="initiate")
  File "/usr/lib64/python3.8/site-packages/gssapi/creds.py", line 63, in __new__
    res = cls.acquire(name, lifetime, mechs, usage,
  File "/usr/lib64/python3.8/site-packages/gssapi/creds.py", line 136, in acquire
    res = rcreds.acquire_cred(name, lifetime,
  File "gssapi/raw/creds.pyx", line 158, in gssapi.raw.creds.acquire_cred
gssapi.raw.misc.GSSError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529639053): Can't find client principal coreos-koji-tagger\/coreos-koji-tagger.fedoraproject.org/fedoraproject.org@FEDORAPROJECT.ORG in cache collection
2020-08-18 18:53:14,783 ERROR requests_gssapi.gssapi_ - Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529639053): Can't find client principal coreos-koji-tagger\/coreos-koji-tagger.fedoraproject.org/fedoraproject.org@FEDORAPROJECT.ORG in cache collection
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/requests_gssapi/compat.py", line 50, in generate_request_header
    self.creds = gssapi.Credentials(name=name, usage="initiate")
  File "/usr/lib64/python3.8/site-packages/gssapi/creds.py", line 63, in __new__
    res = cls.acquire(name, lifetime, mechs, usage,
  File "/usr/lib64/python3.8/site-packages/gssapi/creds.py", line 136, in acquire
    res = rcreds.acquire_cred(name, lifetime,
  File "gssapi/raw/creds.pyx", line 158, in gssapi.raw.creds.acquire_cred
gssapi.raw.misc.GSSError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529639053): Can't find client principal coreos-koji-tagger\/coreos-koji-tagger.fedoraproject.org/fedoraproject.org@FEDORAPROJECT.ORG in cache collection
Traceback (most recent call last):
  File "/usr/bin/fedora-messaging", line 11, in <module>
    load_entry_point('fedora-messaging==2.0.1', 'console_scripts', 'fedora-messaging')()
  File "/usr/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/fedora_messaging/cli.py", line 120, in consume
    _consume(exchange, queue_name, routing_key, callback, app_name)
  File "/usr/lib/python3.8/site-packages/fedora_messaging/cli.py", line 163, in _consume
    deferred_consumers = api.twisted_consume(
  File "/usr/lib/python3.8/site-packages/fedora_messaging/api.py", line 150, in twisted_consume
    callback = _check_callback(callback)
  File "/usr/lib/python3.8/site-packages/fedora_messaging/api.py", line 74, in _check_callback
    callback_object = callback()
  File "/work/coreos_koji_tagger.py", line 262, in __init__
    self.koji_login()
  File "/work/coreos_koji_tagger.py", line 458, in koji_login
    self.koji_client.gssapi_login(principal, self.keytab_file)
  File "/usr/lib/python3.8/site-packages/koji/__init__.py", line 2521, in gssapi_login
    raise AuthError('unable to obtain a session')
koji.AuthError: unable to obtain a session

Any ideas?

https://os.fedoraproject.org/console/project/coreos-koji-tagger/browse/pods

  • When do you need this? (YYYY/MM/DD)

  • When is this no longer needed or useful? (YYYY/MM/DD)

  • If we cannot complete your request, what is the impact?

This error from the above paste seems relevant:

Can't find client principal coreos-koji-tagger\/coreos-koji-tagger.fedoraproject.org/fedoraproject.org@FEDORAPROJECT.ORG in cache collection

I deleted and re-created the app in openshift. The error seems to have gone away.

I hope I don't see it again, will report back here if it happens again.

For now closing the ticket.

And as always:

https://giphy.com/gifs/the-it-crowd-DUtVdGeIU8lmo

:smile:

Metadata Update from @mohanboddu:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

Seeing this again today https://os.fedoraproject.org/console/project/coreos-koji-tagger/browse/pods/coreos-koji-tagger-7-65nbh?tab=logs

Turned out to be an issue because openshift was caching the build and it was still using python3-requests-gssapi-1.2.1-1.fc32.noarch. After we forced a build with no cache it works again.

Metadata
None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.