Login to Bugzilla via FAS fails with "400 - Bad Request"

Btw: The login to pagure.io worked via fas in the same private browsing window. I wonder if that is a me-specific issue or if I'm just the first one to report it.

I'm in with the same problem pagure , ask , etc works execpt bugzilla

400 - Bad Request

Invalid SAML request token

If the error persists and if you think this is an error, contact Fedora Infra to resolve the problem.

Regards.,

yeah, can duplicate easily. ;(

I am not sure whats going on. I refreshed saml2 data from bugzilla, but that didn't help any.

I'll continue to look. Perhaps @abompard could take a look?

It's worth noting that it's only bugzilla thats not working. SAML2 auth for aws or other things is working normally. :(

I'm seeing this in the logs:

2024-12-04 00:37:03 (tools.c/:816) EVP_DigestVerifyInit failed

Maybe some cert has expired, I'll investigate.

Alright I've printed the error, it seems to be an invalid signature error.

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/ipsilon/providers/saml2/auth.py", line 80, in _parse_request
    login.processAuthnRequestMsg(message)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/site-packages/lasso.py", line 2284, in processAuthnRequestMsg
    Error.raise_on_rc(rc)
    ~~~~~~~~~~~~~~~~~^^^^
  File "/usr/lib64/python3.13/site-packages/lasso.py", line 62, in raise_on_rc
    raise exception
lasso.DsInvalidSignatureError: <lasso.DsInvalidSignatureError(102): Invalid signature.>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/ipsilon/providers/saml2/auth.py", line 169, in saml2login
    login = self._parse_request(request)
  File "/usr/lib/python3.13/site-packages/ipsilon/providers/saml2/auth.py", line 107, in _parse_request
    raise InvalidRequest(msg)
ipsilon.providers.common.InvalidRequest: "Invalid SAML Request: <lasso.Samlp2AuthnRequest object at 0x7f45f4090d70> (DsInvalidSignatureError() ['SAMLRequest=hZJPbxMxEMXv%2FRSW79l%2FhJC1sqnSRohIhVYJcOCCJvZs4sprL%2FZsoP309W4bFYlV8dGe9%2BY3frO4%2FNMYdkIftLMVz5OMs0BgFRhnseIPGPjl8mIRoDGtWHV0tFv81WEgFoU2iOGh4p23wkHQQVhoMAiSYrf6fCOKJBOtd%2BSkM%2FyCjZxXm7ddIAT0FCnHbe68O2mF%2FksUVnyLin0CYlfd4VEbA%2BOadRxDW6Bh9CNRG0SaapXUqJyHiH2PkhLnD2mPV6S73W0anbWP1%2BOO388fGYnHK1bnMa6dDV2Dfof%2BpCV%2B2968MuxfsBOP6giUSNc8I%2FwEGRJ50JxtQuhwY%2FusKLbLiukkLybZ9GueiWkp3s9%2FjPffrCteFwgfAIt5XchZti9LWeZ5PcvL%2FezdHHPFl1E6RC6GNn75f7AGCRQQ9HSL9G%2Ft2asVfTab9Z0zWj6wlTHu97VHoJhXPg7L2EfnG6C3N6O%2F0WpSD6Wi7SMIhJY4S%2BPmpv%2Bu7vIJ&RelayState=https%3A%2F%2Fbugzilla.redhat.com%2Findex.cgi&SigAlg=http%3A%2F%2Fwww.w3.org%2F2000%2F09%2Fxmldsig%23rsa-sha1&Signature=GKzGGzPtmJqYui5pHnoslF0Um87%2BC7Ym9GDcntWGGiBNj4CjJvTtN%2F1Q%2F5qWDDbftPXWD2CwMicWEvj13njhG%2B3WpFlrzH7iIWmxUSC1SQZMFgppDfxXTeBIL7payF29rpPlLm1fdDbVlfxHu04dfNSs3OKN5eLva60ZNvmsZEbPnjYEf6PIKB4fEm4%2BV3AImkQZNMrx6%2BWr7SM0ZYSxGSJDpHf7dob%2FsYKNQbdfyxKSX18Qgj8rsXQx0KqAJfM7q8YgxjuboG93Ymp8nhucp5VfWKwwSKwO9eJOHcCpPtlcy1Njn089SulalKW9g1xXnwK%2BZ1%2FF3GkBOz%2FNQ3c2FMqasjr3B4VW2AiZ1JrWtMmkNixMAyRLrVizAKTZ8qfzveABw8tIOWb2%2FPWm5es%2BLvX%2FAwDG92upSRAFoIfk7vBvIICZ8UdmZzBytZcWRmxrwIqJbfJZ0A57Z%2BwNtw9%2BdJ%2Ffx9hG5ZjggSw88PKr7UXsLF3TW0uq34nEzz%2BK4ieZNAwjgAypaMJ8UTM4c1S7dZJhTmY61EfhDupLy5VIq2FwGUuTjh2VexbT%2BatdnAzTmhMMmdk%2BGUWZvxFkMw1KYpaizneunwWONxLzX3X6NoHZHabPE0FmSG9PYzEJcwdzOS%2B44wQkKGnCkNohkNOU9V37uimZfSUQuYMrGEEpaKw%3D'])"

I wonder if it could be on Bugzilla's side. I don't see any expired cert on our side but the SAML2 metadata for bugzilla includes validUntil="2024-12-17T18:57:27Z" and it's suspiciously close to now. I'm not sure. The metadata does embed a cert that is valid until Jan 1st, so it should be fine.

Or maybe it's related to the machine being updated to F41 and a cipher being forbidden or whatnot. I've seen those 400 errors in the logs before this ticket was opened but they only go as far as Dec 02.

I did update the metadata, but I don't think that's it because it broke before that. ;(

I tried allowing SHA1, but that didn't help. ;(

@jfearn anything obvious stand out here to you?

So one extra piece of data:

It fails for me in the same way if I try to login on F40 with FAS (with an OTP account).

...now, there's a tiny chance this isn't related because I always login to bugzilla with RH SSO ... and there are other variables that might make it a me problem. But I thought it was worth letting everyone know.

I'm pretty sure that affects everyone using Bugzilla with FAS (possibly delayed until their current session runs out). I get this error on a private browsing instance in Firefox and Chromium as well as on my mobile Firefox/Fennec without even entering any personal data.

In my browser I see this in the URL:

SigAlg=http://www.w3.org/2000/09/xmldsig#rsa-sha1

Looks to me, like Bugzilla is using a pretty outdated signature algorithm.

Some searching led me to these threads which suggest that the sha1 algorithm could have been removed from lasso, which means @kevin 's attempt to allow it could have been without effect.

https://github.com/latchset/mod_auth_mellon/issues/115
https://forum.checkmk.com/t/saml-2-0-failed-to-verify-signature/36630

I'd love to dig deeper now but I guess this is just as much as I can do without admin access to the involved systems.

Thanks for the detective work.

It turns out I set DEFAULT:SHA1, which didn't fix it, but trying again with LEGACY seems to?

ie, it's working now for me with LEGACY. obviously we don't want to keep doing that, but at least it's working until we can sort it out.

Also, our/fedora lasso seems to be compiled with min sha1 still allowed? So, not sure it's sha1 or something else in the chain.

Thanks a lot @kevin ! Good to have a quick workaround for now, but I agree that this should be properly fixed. Shall we rename this issue or close it and create a new one?

Lets just keep this issue for a bit... I might also file a bugzilla ticket.

I filed an internal ticket. Watching for some response there.

hmm, is the issue back? After being able to log in via FAS in the past days, I can't any more ...

This or a different issue with the same symptoms. One 400 looks pretty much like the other so someone who has the power to do so has to look into the logs to be sure.

I also see the 400 - Bad Request response when trying to authenticate to BZ using FAS link.

Sorry about that. I inadvertently messed up the workaround.

It should be working again now.

We get a similar error when trying to login into Copr (and for example Bodhi as well).

400 - Bad Request
Invalid transaction id
If the error persists and if you think this is an error, contact Fedora Infra to resolve the problem.

100% reproducible when trying to login in a private window. Once I get the error, I can refresh the page and I can then see I am logged in.

Is this the same issue or should I file a separate ticket?

Copr, bugzilla and bodhi all works for me currently (including login via fas), so if it persists for you I'd say that is a different issue. This one is fixed temporarily but from the information I see here I'd say it is still waiting for a long term fix and hence still open.

Right, this is only about the bugzilla thing.

We also do still have https://pagure.io/fedora-infrastructure/issue/12092 (but I doublechecked and httpd wasn't upgraded again, so I guess this is not that either)

Hey, you are right. The Copr issue is gone now, sorry for hijacking this ticket.

Hi, as part of Bug 2330639 I am updating the Net::SAML2 module being used, which now uses SHA256. Testing it from the internal development server, this works with the Red Hat Customer SSO, but Fedora SSO generates a 500.

Can we get some log details about this transaction?

https://id.fedoraproject.org/saml2/SSO/Continue?ipsilon_transaction_id=0d7599a1-30ac-4948-8a8f-28c634c17f7b

Yes, I'm seeing this in the logs for this transaction:

lasso.LoginFederationNotFoundError: <lasso.LoginFederationNotFoundError(601): Federation not found on login>

Do you need more info?

@abompard apparently that is enough ...

It was sending:

<samlp:NameIDPolicy Format="urn:oasis:names:tc:SAML:2.0:nameid-format:persistent" />

But apparently it needs:

<samlp:NameIDPolicy AllowCreate="1" Format="urn:oasis:names:tc:SAML:2.0:nameid-format:persistent" />

FYI the fix for this has been deployed to production RHBZ.

SigAlg: http://www.w3.org/2001/04/xmldsig-more#rsa-sha256

Then I believe this issue is resolved and can be closed :) Thanks everyone!

Then I believe this issue is resolved and can be closed :) Thanks everyone!