As far as I can tell, Pagure doesn't try to guess the character set of a file before trying to display it. Instead utf-8 is assumed. This could cause files to be rendered incorrectly. uchardet or python-chardet could be used to guess the charset and then a conversion can be done to utf-8. There could possibly be performance or security issues with doing this on user supplied files.
Well, the HTML rendered is clearly marked as 'utf-8':
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
So I do not think that this is or will be a problem.
I don't think you understood the point of the request. Some project may ship an iso-8859-1 documentation file. If that file is displayed as is on the pagure hosting service marked as a utf-8 file, then it will not be rendered correctly in browsers. There could be a requirement that all text files need to be in utf-8, but projects may have valid reasons for wanting to use other character sets, particularly for documentation. Now it may be that I missed something and Pagure does do character set conversion.
Hm I see your point, based on the spread of UTF-8 nowadays, I'm going to wait for the issue to arise for real but I would review a patch addressing this is someone wants to work on this.
Also, I must confess that I am not thrilled at the idea to add a new dependency to pagure.
I agree this is not a high priority with the prevelance of utf-8.
One other note, while looking for something else I found that pagure is already importing chardet so adding some other checks won't add any new external requirements.
Oh cool, that should make it easier indeed
Implemented in https://pagure.io/pagure/pull-request/1012