#12017 New SSH keys for fedorapeople.org
Closed: Fixed with Explanation 8 months ago by kevin. Opened 8 months ago by ppisar.

It seems you have reinstalled fedorapeople.org server (#12008) and RSA SSH of the server has changed. Where can I verify the new SSH keys?


The same question. Maybe display fingerprints somewhere at fedorapeople.org?

So using ssh with VerifyKeyHost DNS kind of works

$ ssh -o 'VerifyHostKeyDNS=yes' fedorapeople.org
The authenticity of host 'fedorapeople.org (2600:2701:4000:5211:dead:beef:a7:9475)' can't be established.
ED25519 key fingerprint is SHA256:rWjv2pnT4nWaH6Xud/ePK2CnVnnJoo7iUlBla0iT5LM.
Matching host key fingerprint found in DNS.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? 

However if you have the older key, it does not seem to check that the DNS changed and tell you the new key is verifiable by DNS. Also if you are going to something like ssh ppisar.fedorapeople.org it will fail because those keys do not seem to match the ones for the server.

Metadata Update from @zlopez:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: high-gain, medium-trouble, ops

8 months ago

Btw looks like DNSSEC isn't available for fedorapeople. According to this blog-post if DNS-record has both SSHFP added, and DNSSEC enabled then SSH will not generate any prompts at all.

Yes, we reinstalled fedorapeople.org

You can be sure of the ssh host keys by using our ssh ca, as all our ssh host keys are signed by it. Add this to your .ssh/known_hosts or the like:
https://admin.fedoraproject.org/ssh_known_hosts

I did miss updating the sshfp wildcard records. So if you sshed to say 'peter.fedorapeople.org' the sshfp records wouldn't have matched. I have pushed a fix so this should now work.

dnssec is definitely enabled for fedorapeople.org: https://dnsviz.net/d/fedorapeople.org/dnssec/
Perhaps you are using a non dnssec aware dns server (like systemd-resolved by default).

I am not sure why DNSSEC isn't available:
The named.conf for it uses the signed version:

zone "fedorapeople.org" {
        type master;
        file "/var/named/master/built/fedorapeople.org.signed";
};

And the zone contains RRSIG entries for the SSHFP. I am going to have to let someone with sysadmin powers to help figure that out.

Thanks Kevin for the certificates. I updated https://fedoraproject.org/wiki/Infrastructure/fedorapeople.org#Accessing_your_fedorapeople.org_space documentation. I confirm the certificates works for me.

I am not sure why DNSSEC isn't available:

That's my mistake - it is available. I just need to setup systemd-resolved to enable it.

@kevin thanks for the tip!

Metadata Update from @kevin:
- Issue assigned to kevin

8 months ago

Thanks. I sent a devel-announce post about the ssh host key changing.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed with Explanation
- Issue status updated to: Closed (was: Open)

8 months ago

I'm trying to conect using the new certificates (I added them to my known_hosts file, but I'm facing one of the two folowing problems:
If I delete all other previous key for the fedorapeople.org (for example people.fedoraproject.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEwhjvbtMsSnbvk6lEM7+ZVZpXVOcRXLFhPdKXQ9+u5T) ssh ask for confirmation if I trust the server, I answer yes, give my key password, and after a while he closes the conection, here's the log for this:

debug1: Server host certificate: ssh-ed25519-cert-v01@openssh.com SHA256:Tv9lb6xLTnHhDm7sc0Bd07SlON0/g7gXHzgGcZWxXpk, serial 1727208284 ID "proxy40.fedoraproject.org" CA ssh-rsa SHA256:IPuhCSNXqj4m2eq6UKYE1jHFglLgLCbBzINft+OxUMA valid from 2024-09-24T16:04:44 to 2025-09-23T17:04:44
debug1: load_hostkeys: fopen /home/geraldo/.ssh/known_hosts2: No such file or directory
debug1: Host 'people.fedoraproject.org' is known and matches the ED25519-CERT host certificate.
debug1: Found CA key in /home/geraldo/.ssh/known_hosts:67
Certificate invalid: name is not a listed principal
debug1: No matching CA found. Retry with plain key
debug1: Host 'people.fedoraproject.org' is known and matches the ED25519 host key.
debug1: Found key in /home/geraldo/.ssh/known_hosts:64
debug1: ssh_packet_send2_wrapped: resetting send seqnr 3
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: Sending SSH2_MSG_EXT_INFO
debug1: expecting SSH2_MSG_NEWKEYS
debug1: ssh_packet_read_poll2: resetting read seqnr 3
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 4294967296 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_ext_info_client_parse: server-sig-algs=<ecdsa-sha2-nistp256,sk-ecdsa-sha2-nistp256@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,sk-ssh-ed25519@openssh.com,rsa-sha2-256,rsa-sha2-512>
debug1: kex_ext_info_check_ver: publickey-hostbound@openssh.com=<0>
debug1: kex_ext_info_check_ver: ping@openssh.com=<0>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_ext_info_client_parse: server-sig-algs=<ecdsa-sha2-nistp256,sk-ecdsa-sha2-nistp256@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,sk-ssh-ed25519@openssh.com,rsa-sha2-256,rsa-sha2-512>
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: No credentials were supplied, or the credentials were unavailable or inaccessible
No Kerberos credentials available (default cache: KCM:)


debug1: No credentials were supplied, or the credentials were unavailable or inaccessible
No Kerberos credentials available (default cache: KCM:)


debug1: Next authentication method: publickey
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: ssh_fetch_identitylist: agent contains no identities
debug1: Will attempt key: /home/geraldo/.ssh/geraldosimiao RSA SHA256:9rt6VwHMJcNOlyGg0VmFPcW7+WEKvu7fF+ZKZr/afmA explicit
debug1: Will attempt key: /home/geraldo/.ssh/geraldosimiao RSA SHA256:9rt6VwHMJcNOlyGg0VmFPcW7+WEKvu7fF+ZKZr/afmA explicit
debug1: Offering public key: /home/geraldo/.ssh/geraldosimiao RSA SHA256:9rt6VwHMJcNOlyGg0VmFPcW7+WEKvu7fF+ZKZr/afmA explicit
debug1: Server accepts key: /home/geraldo/.ssh/geraldosimiao RSA SHA256:9rt6VwHMJcNOlyGg0VmFPcW7+WEKvu7fF+ZKZr/afmA explicit
Enter passphrase for key '/home/geraldo/.ssh/geraldosimiao':
Connection closed by 2600:1f1e:fa1:6501:ef6e:a389:440a:2fb6 port 22

After that, if I retry, it says this:

debug1: Server host certificate: ssh-ed25519-cert-v01@openssh.com SHA256:L3zxnair9gK4k9gY//82NBPi7db6Cwu4GhLRvJpvhGA, serial 1727208283 ID "proxy34.fedoraproject.org" CA ssh-rsa SHA256:IPuhCSNXqj4m2eq6UKYE1jHFglLgLCbBzINft+OxUMA valid from 2024-09-24T16:04:43 to 2025-09-23T17:04:43
debug1: load_hostkeys: fopen /home/geraldo/.ssh/known_hosts2: No such file or directory
debug1: Host 'people.fedoraproject.org' is known and matches the ED25519-CERT host certificate.
debug1: Found CA key in /home/geraldo/.ssh/known_hosts:63
Certificate invalid: name is not a listed principal
debug1: No matching CA found. Retry with plain key
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:L3zxnair9gK4k9gY//82NBPi7db6Cwu4GhLRvJpvhGA.
Please contact your system administrator.
Add correct host key in /home/geraldo/.ssh/known_hosts to get rid of this message.
Offending ED25519 key in /home/geraldo/.ssh/known_hosts:69
Host key for people.fedoraproject.org has changed and you have requested strict checking.
Host key verification failed.

If I try just check the server, it gives me this:

geraldo@rivotril ~> ssh -o 'VerifyHostKeyDNS=yes' fedorapeople.org
Certificate invalid: name is not a listed principal
Enter passphrase for key '/home/geraldo/.ssh/geraldosimiao': 
Enter passphrase for key '/home/geraldo/.ssh/geraldosimiao': 
Connection closed by 2620:52:3:1:dead:beef:cafe:fed6 port 22
geraldo@rivotril ~ [255]> ssh -o 'VerifyHostKeyDNS=yes' fedorapeople.org
Certificate invalid: name is not a listed principal
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:L3zxnair9gK4k9gY//82NBPi7db6Cwu4GhLRvJpvhGA.
Please contact your system administrator.
Add correct host key in /home/geraldo/.ssh/known_hosts to get rid of this message.
Offending ED25519 key in /home/geraldo/.ssh/known_hosts:69
Host key for people.fedoraproject.org has changed and you have requested strict checking.
Host key verification failed.

you don't want people.fedoraproject.org... thats a web redirect using our proxy network.

do you have some config for this in your ~/.ssh/config? you shouldn't need any there...

you don't want people.fedoraproject.org... thats a web redirect using our proxy network.

do you have some config for this in your ~/.ssh/config? you shouldn't need any there...

Yeah, I have it on the config file as HostName people.fedoraproject.org.
Anyway, now I deleted it and can connect without problems.
Thanks for the help :)

Log in to comment on this ticket.

Metadata
Boards 1
ops Status: Backlog