Login into Fedora services takes too long. Just tried to log in into the Bodhi and realized that login takes ages, I hit the stop watch and it took additional 30 seconds to finish the login. So overall, it could have been e.g. 45 seconds? I observe this behavior with all services recently. I don't think login should take that much time.
In order to debug this, the infrastructure team will need to know: 1. what is the ip address you are coming from 2. what was the UTC time stamp of the login attempt? 3. Does kinit also take so long? 4. Does logging in at https://accounts.fedoraproject.org/ take as long?
The reason to ask is to locate if it is a proxy or a network problem. Currently it takes 5 seconds at https://accounts.fedoraproject.org/ from 136.54.41.254 in the US at Wed 4 Jan 12:18:53 UTC 2023. However that is in the continental US and so may be an outlier.
Just tried to log in into Wiki with my stop watch ready and it took 30s and I have attempted the login just a seconds ago, i.e. 13:22 CET (12:22 UTC). The request could be coming from 89.176.165.208 or via Red Hat VPN, not sure. It seems that also kinit is slow. ~15s for password prompt and another ~15s to get the ticket. Login to https://accounts.fedoraproject.org/ took like 2s, so that is fine.
OK this is sounding like a networking issue. It is either going to be between your location and a proxy, or that proxy and the Fedora datacenter in the US. My fkinit takes 'no time' to get a password prompt and a token either on or off the VPN, and that is the US DC (host id.fedoraproject.org.) It is behind a firewall and it looks like currently the ip you last get a signal from will be 209.132.185.253
traceroute -n 209.132.185.253 ... 11 144.121.35.0 15.575 ms 15.482 ms 15.445 ms 12 104.207.214.80 15.400 ms 15.397 ms 15.859 ms 13 160.72.43.102 15.901 ms 15.857 ms 15.867 ms 14 209.132.185.253 21.182 ms 20.970 ms 21.091 ms
The EU Fedoraproject proxies are able to talk to this within 90ms on average. These proxies are:
in case that helps pin down a bad one you are getting which is slow.
any improvement or worsening?
Metadata Update from @smooge: - Issue priority set to: Waiting on Reporter (was: Needs Review) - Issue tagged with: authentication, medium-gain, medium-trouble, ops
Can https://github.com/fedora-copr/copr/issues/2418 be related?
I don't think so? @vondruch do you have a ipv6 address? Does accounts.fedoraproject.org resolve to ipv6 for you?
And does the problem still persist?
I have just tired login to accounts and it took like 10 seconds to get the password prompt and 10 seconds to redirect back. Kind of reasonable.
Login to Bodhi happened right away without any delay.
I don't have ipv6 address AFAIK
$ dig accounts.fedoraproject.org ; <<>> DiG 9.18.8 <<>> accounts.fedoraproject.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48578 ;; flags: qr rd ra; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 65494 ;; QUESTION SECTION: ;accounts.fedoraproject.org. IN A ;; ANSWER SECTION: accounts.fedoraproject.org. 300 IN CNAME wildcard.fedoraproject.org. wildcard.fedoraproject.org. 35 IN A 209.132.190.2 wildcard.fedoraproject.org. 35 IN A 18.159.254.57 wildcard.fedoraproject.org. 35 IN A 18.133.140.134 wildcard.fedoraproject.org. 35 IN A 18.192.40.85 wildcard.fedoraproject.org. 35 IN A 85.236.55.6 wildcard.fedoraproject.org. 35 IN A 185.141.165.254 wildcard.fedoraproject.org. 35 IN A 38.145.60.21 wildcard.fedoraproject.org. 35 IN A 38.145.60.20 wildcard.fedoraproject.org. 35 IN A 152.19.134.142 wildcard.fedoraproject.org. 35 IN A 152.19.134.198 ;; Query time: 31 msec ;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP) ;; WHEN: Tue Jan 31 13:27:12 CET 2023 ;; MSG SIZE rcvd: 238 $ nslookup accounts.fedoraproject.org Server: 127.0.0.53 Address: 127.0.0.53#53 Non-authoritative answer: accounts.fedoraproject.org canonical name = wildcard.fedoraproject.org. Name: wildcard.fedoraproject.org Address: 38.145.60.21 Name: wildcard.fedoraproject.org Address: 209.132.190.2 Name: wildcard.fedoraproject.org Address: 185.141.165.254 Name: wildcard.fedoraproject.org Address: 18.159.254.57 Name: wildcard.fedoraproject.org Address: 152.19.134.142 Name: wildcard.fedoraproject.org Address: 152.19.134.198 Name: wildcard.fedoraproject.org Address: 18.133.140.134 Name: wildcard.fedoraproject.org Address: 85.236.55.6 Name: wildcard.fedoraproject.org Address: 38.145.60.20 Name: wildcard.fedoraproject.org Address: 18.192.40.85 Name: wildcard.fedoraproject.org Address: 2600:2701:4000:5211:dead:beef:fe:fed3 Name: wildcard.fedoraproject.org Address: 2a05:d014:10:7803:f774:4d7c:e277:a457 Name: wildcard.fedoraproject.org Address: 2a05:d01c:c6a:cc01:269:da52:9ae1:43e6 Name: wildcard.fedoraproject.org Address: 2001:4178:2:1269::fed2 Name: wildcard.fedoraproject.org Address: 2605:bc80:3010:600:dead:beef:cafe:fed9 Name: wildcard.fedoraproject.org Address: 2604:1580:fe00:0:dead:beef:cafe:fed1
I am suspecting the long login times may have been sssd on one of our ipsilon hosts getting in a bad state. That's long since been corrected, so I think we can close this now?
Do reopen if you see long delays again and we can investigate more.
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.