I am trying to load the src.fp.o homepage/dashboard and I am getting a 504 Gateway Timeout error. This persists after reloading. According to Firefox's devtools, X-Fedora-ProxyServer: proxy01.iad2.fedoraproject.org and the request waited for 1 minute. I have been getting periodic 500 errors (@kevin helped troubleshoot and determined it was an upstream issue unrelated to the infra) for a little while, but this error message is new. I assume the increased load times and the resulting timeouts are occurring, because I have gained more packages.
X-Fedora-ProxyServer: proxy01.iad2.fedoraproject.org
N/A. This is a personal issue that probably falls low on your priority list ;).
Metadata Update from @mobrien: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: low-gain, medium-trouble, ops
Do you have timestamps for when this occurred? These events happen periodically and can be related to a lot of different factors (database, web proxy, other tools the backend is waiting on, etc). A timestamp is needed to know when this happened so any analysis of what set of things broke and when.
It occurred right around the time I initially reported this issue, and it's still happening now.
EDIT: It successfully loaded a couple times around 15:30 UTC, and now I'm getting the same error.
Ok I looked on pkgs01.iad2 logs and saw https://http.cat/504 in the logs several times around 15:30 and 15:50.. during times I was able to open the pages myself. @kevin looked at it also and saw that the difference between usages is the number of packages that get bundled in that transaction. He has a couple hundred and it works, you have 2000+ and it doesn't. This is probably quite a lot more than the system was designed to handle and it is timing out when doing so.
He is brainstorming if an increase in timeouts would help any
He has a couple hundred and it works, you have 2000+ and it doesn't. This is probably quite a lot more than the system was designed to handle and it is timing out when doing so.
Yeah, you have the go-sig to thank for that. We are discussing decreasing the amount of packages we maintain, but due to unbundling and the amount of dependencies go projects tend to have, we've ended up with a lot of packages. There's other people in the go-sig who probably have more directly owned packages than me and/or packages from other SIGs, so I'd be curious if/how they're affected by these type of issues.
Anyways, thanks for looking into this.
Can you check again now and see if it loads? Things are back to pretty normal on the db side...
This still takes just under a minute to load, but I'm not getting HTTP 5xx errors, which I'd consider an improvement.
Yeah, I am not sure if we can do much better without upstream pagure changes to be more efficient here. ;(
If it starts breaking again let us know.
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Agreed. Thank you for making it as best as possible for right now :).
Log in to comment on this ticket.