#9600 login fails with longer passwords after enabling TOTP
Opened by ryanlerch. Modified

We have been hitting this issue in Fedora Accounts for a while where users with longer passwords ( over 128 chars or so) are unable to login once they enable OTP (TOTP specifically)

I have been trying to track this down in the Fedora Accounts (noggin) code, but i have been able to reproduce the issue on https://ipa.demo1.freeipa.org/

The steps i did were:

  1. Log in to demo1 as admin with the Secret123 password, and create a new user mytestuser, with a 130 char password.
  2. Log out and log in as mytestuser, the password is expired, so i just changed it to the same password.
  3. Log in as mytestuser with the 130char password success
  4. Create a TOTP OTP token, and set it up in my authenticator.
  5. Log out, and try to log back in with the 130char password and the 6 digit OTP at the end FAILURE The password or username you entered is incorrect
  6. Log back in as admin, and change the password for mytestuser to Secret123.
  7. Log back in as mytestuser with Secret123+6 digit OTP
  8. Success, but password is expired (becasue admin set it), so have to change it, once changed, can log in as mytestuser+ 6 digit OTP no worries.

In my testing on the Noggin side, even with the long password, i was unable to login, but the change_password endpoint worked with the long password+OTP.

Just FYI, the noggin issue we have open for this is:

https://github.com/fedora-infra/noggin/issues/1103

with a bit more info in this closed duplicate issue here:

https://github.com/fedora-infra/noggin/issues/1394


I went through the code. We only have a hard limit for passwords at 1000 characters. This is because of MIT Kerberos limiting the input buffer in tools like kinit to 1024 characters. See more in https://pagure.io/freeipa/issue/8268

I went the whole path from LDAP server to our password-checking plugin. We parse ASN.1 input from the LDAP client as an octet string. This means no inherent limit -- whatever client provided will be taken as a credential. Then this credential unchanged is provided to IPA plugin that identifies whether this credential needs OTP handling. If it does, then last numbers of it treated as a token value depending on properties of a specific token it processes (there might be multiple tokens associated with a user). Once that token authentication is successful, the last numbers of the credential get cut off and the rest is treated as a password.

At any point, there are no limitations to the password's size beyond that 1000 characters one.

Did some more digging, and the password+otp is intact up until the kinit call that is done here:

https://pagure.io/freeipa/blob/master/f/ipaserver/rpcserver.py#_1016

Note too, that the change_password endpoint here does work, no matter how long the password:

https://pagure.io/freeipa/blob/master/f/ipaserver/rpcserver.py#_1162

Not sure if this anything, but i noticed in the /var/log/krb5kdc.log that when i try to log in with a too-long password + OTP, i get the following message in the logs:

Jun 03 05:14:43 ipa.tinystage.test krb5kdc[14434](info): preauth (otp) verify failure: Message too long

Not sure where that message is coming from though.

For context, all the output is pasted below for the two cases.

Failure case: trying to log in with a 150char password + OTP

Jun 03 05:14:43 ipa.tinystage.test krb5kdc[14435](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 127.0.0.1: NEEDED_PREAUTH: WELLKNOWN/ANONYMOUS@TINYSTAGE.TEST for krbtgt/TINYSTAGE.TEST@TINYSTAGE.TEST, Additional pre-authentication required
Jun 03 05:14:43 ipa.tinystage.test krb5kdc[14435](info): closing down fd 11
Jun 03 05:14:43 ipa.tinystage.test krb5kdc[14435](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 127.0.0.1: ISSUE: authtime 1717391683, etypes {rep=aes256-cts-hmac-sha384-192(20), tkt=aes256-cts-hmac-sha384-192(20), ses=aes256-cts-hmac-sha384-192(20)}, WELLKNOWN/ANONYMOUS@TINYSTAGE.TEST for krbtgt/TINYSTAGE.TEST@TINYSTAGE.TEST
Jun 03 05:14:43 ipa.tinystage.test krb5kdc[14435](info): closing down fd 11
Jun 03 05:14:43 ipa.tinystage.test krb5kdc[14434](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 127.0.0.1: NEEDED_PREAUTH: adamnelson@TINYSTAGE.TEST for krbtgt/TINYSTAGE.TEST@TINYSTAGE.TEST, Additional pre-authentication required
Jun 03 05:14:43 ipa.tinystage.test krb5kdc[14434](info): closing down fd 11
Jun 03 05:14:43 ipa.tinystage.test krb5kdc[14434](info): preauth (otp) verify failure: Message too long
Jun 03 05:14:43 ipa.tinystage.test krb5kdc[14434](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 127.0.0.1: PREAUTH_FAILED: adamnelson@TINYSTAGE.TEST for krbtgt/TINYSTAGE.TEST@TINYSTAGE.TEST, Preauthentication failed
Jun 03 05:14:43 ipa.tinystage.test krb5kdc[14434](info): closing down fd 11

Success case: logging in with an 8 char password + OTP

Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14435](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 127.0.0.1: NEEDED_PREAUTH: WELLKNOWN/ANONYMOUS@TINYSTAGE.TEST for krbtgt/TINYSTAGE.TEST@TINYSTAGE.TEST, Additional pre-authentication required
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14435](info): closing down fd 11
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14435](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 127.0.0.1: ISSUE: authtime 1717387878, etypes {rep=aes256-cts-hmac-sha384-192(20), tkt=aes256-cts-hmac-sha384-192(20), ses=aes256-cts-hmac-sha384-192(20)}, WELLKNOWN/ANONYMOUS@TINYSTAGE.TEST for krbtgt/TINYSTAGE.TEST@TINYSTAGE.TEST
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14435](info): closing down fd 11
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14434](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 127.0.0.1: NEEDED_PREAUTH: adamnelson@TINYSTAGE.TEST for krbtgt/TINYSTAGE.TEST@TINYSTAGE.TEST, Additional pre-authentication required
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14434](info): closing down fd 11
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14435](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 127.0.0.1: ISSUE: authtime 1717387878, etypes {rep=aes256-cts-hmac-sha384-192(20), tkt=aes256-cts-hmac-sha384-192(20), ses=aes256-cts-hmac-sha384-192(20)}, adamnelson@TINYSTAGE.TEST for krbtgt/TINYSTAGE.TEST@TINYSTAGE.TEST
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14435](info): closing down fd 11
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14435](info): TGS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 127.0.0.1: ISSUE: authtime 1717387878, etypes {rep=aes256-cts-hmac-sha384-192(20), tkt=aes256-cts-hmac-sha384-192(20), ses=aes256-cts-hmac-sha384-192(20)}, adamnelson@TINYSTAGE.TEST for HTTP/ipa.tinystage.test@TINYSTAGE.TEST
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14435](info): closing down fd 11
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14434](info): TGS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 127.0.0.1: ISSUE: authtime 1717387878, etypes {rep=aes256-cts-hmac-sha384-192(20), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha384-192(20)}, HTTP/ipa.tinystage.test@TINYSTAGE.TEST for ldap/ipa.tinystage.test@TINYSTAGE.TEST
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14434](info): ... CONSTRAINED-DELEGATION s4u-client=adamnelson@TINYSTAGE.TEST
Jun 03 04:11:18 ipa.tinystage.test krb5kdc[14434](info): closing down fd 11

Thanks. I guess now I know where this might come from. OTP pre-authentication plugin in MIT Kerberos is using RADIUS to communicate with the backend. It sends Access-Request and expects Access-Response from the RADIUS backend. The problem here, however, that the size of Access-Request message attributes can be no longer than 254 bytes. We stumbled upon this already when implementing OAuth2 device authorization flow, as described in https://freeipa.readthedocs.io/en/latest/designs/external-idp/external-idp.html#authentication-flow-for-oauth2-proxying-over-kerberos-protocol:

Original design for idp pre-authentication mechanism used RADIUS attribute State to pass through the state of OAuth 2.0 flow. In RADIUS packets, size of RADIUS attributes is limited up to 254 bytes. For many IdPs, the state can be larger than 253 bytes. MIT Kerberos krad library supports concatenating multiple appearances of the same attribute in the packet when retrieving the value. State attribute semantics are defined in RFC 2865, where State attribute can only be present at most once. Therefore, was decided to utilize Proxy-State attribute instead. Proxy-State allows multiple appearances in a single RADIUS packet.

Additionally, a bug was found in krad library implementation which prevented to use RADIUS attribute values larger than 127 bytes. As of now, the fix for MIT Kerberos is expected to be released with MIT Kerberos 1.20. Those fixes include commits from upstream pull requests krb5#1229 and krb5#1230. They were backported to RHEL 8.7, RHEL 9.1, and Fedora 34-37 releases ahead of MIT Kerberos 1.20 release.

Since for OAuth2 device authorization flow we controlled both client and KDC sides of the pre-authentication plugin (provided by SSSD), we switched the exchange to use Proxy-State RADIUS attribute but for OTP we cannot do that. That plugin is part of MIT Kerberos upstream and if any change to be done, it needs a change there. FreeIPA backend (ipa-otpd daemon) would need modifications as well, so it is not a simple update, as both the KDC side of the plugin and ipa-otpd has to use the same logic.

Individual attributes would support up to 253 characters with krb5 1.21+. As I said, RHEL 8/9 have the following patch backported: https://gitlab.com/redhat/centos-stream/rpms/krb5/-/blob/c8s/krb5-krad-larger-attrs.patch?ref_type=heads, so it should work there.

However, 'Message too long' is EMSGSIZE error. Looking through the krad library implementation, the attribute encoder enforces attribute size limits based on RFC 2865 where User-Password is defined as having 128 characters at most: https://www.rfc-editor.org/rfc/rfc2865#section-5.2

A summary of the User-Password Attribute format is shown below.  The
   fields are transmitted from left to right.
    0                   1                   2
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
   |     Type      |    Length     |  String ...
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
   Type
      2 for User-Password.
   Length
      At least 18 and no larger than 130.
   String
      The String field is between 16 and 128 octets long, inclusive.

So, basically we cannot change the length of User-Password beyond what is defined in the RFC 2865 without breaking compiance with RADIUS protocol spec.

I'm inclined to update documentation with required limits for various supported situations. We have a problem here that the same 'otp' mechanism is used for proxying RADIUS authentication to remote RADIUS servers as well, so we cannot really know how ipa-otpd backend would be using the User-Password field: will it process that within FreeIPA only or would it attempt to pass through to a remote RADIUS server. All of those servers will limit User-Password field length to the same 128 characters as well, so perhaps we should document that OTP and RADIUS authentication methods only support credentials which aren't larger than 128 characters (including the token value in case of OTP).

Thanks for digging into this so much!

We can pretty easily add this restriction on the Noggin / Fedora Accounts side so we don't hit this.

Thanks again.

Metadata