Wait a sec, https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html#tag-description describes supported markup, which is completely different from supported markdown. Maybe I should read closer before complaining. Let me ask around....

OK, the problem is bodhi translates markdown into markup, and it's using markup that is not supported by the clients. Solution is still the same: disable it.

CC: @mattia (in case you have any thoughts here).

I don't think we want to just do this without more discussion/input from consumers of this data.

Example bad update:

We really need this to never happen.

At a glance, Bodhi just stores in db what the user submitted as notes (in markdown format), then sanitize and markup the HTML output on the fly while rendering the HTML/RSS output.

I think it may be easier to sanitize the markup output to whatever generates the notes going into the updates metadata, but I'm not sure where it is done (I suppose it's in the composer and the metadata is generated by Pungi?)

I thought it was bodhi that inserted the updateinfo.xml at the end after pungi created the repo? That info is in updateinfo.xml right?

Yep, I've found it, the code is in bodhi-server/metadata.py under add_update().

AFAIK, Bodhi creates the updateinfo metadata with:

rec.description = update.notes.encode('utf-8')

and update.notes is just whatever the update submitter puts in the description of the update, which is markup text.
So it's the downstream package manager that translates that markup text in HTML inserting code unsupported by the appstream specification.

For example, supposing that the update reported in the screenshot is rpm-4.17.1-2.fc36, I can't find anymore the info in current updateinfo.xml because the update has been superseded, but I think Bodhi have put:

<description>Rebase to rpm 4.17.1 (http://rpm.org/wiki/Releases/4.17.1)

**Note:** During the first `DNF` transaction following the one in which this update is applied, a spurious warning will be shown if you run `DNF` in `--verbose` mode: `RPMDB altered outside of DNF`. You may ignore this warning as it&apos;s caused by a change in the algorithm used to generate the RPMDB hash.
</description>

and it was the downstream package manager that inserted <b><\b> and <tt><\tt>.

I don't think it's a Bodhi fault and I don't think we should limit Bodhi markup support as we're returning just plain markup that consumers can easily sanitize (or just ignore) when translating into markdown for their purposes. The above example shows that the returned plain text is perfectly readable if just rendered as is, while it shows a clear error in the downstream package manager that translated the emphasis with <tt> instead of <em>.

Agreed, thanks for investigating.

Hi, I'm reopening this because after fixing Software to not convert markdown into unsupported markup, now we are seeing Fedora updates released with raw markdown, and it just looks bad. Example here.

There is no point in using markdown in update notes if neither dnf nor Software support it. Just makes things look bad.

Hi, I'm reopening this because after fixing Software to not convert markdown into unsupported markup, now we are seeing Fedora updates released with raw markdown, and it just looks bad.

Really?
In the link you provided several comments above, ordered and unordered lists should be supported, but the provided screenshot clearly shows that the package manager is not fixed at all (the list under "upstream changes" is not rendered, while the list under "build changes" it is).
The markup in the updates notes is correct, you can check in the update webpage or in bodhi client output:

$ bodhi updates query --updateid FEDORA-2022-e538a8b8e7
================================================================================
     btrfs-progs-6.1-2.fc36
================================================================================
   Update ID: FEDORA-2022-e538a8b8e7
Content Type: rpm
     Release: Fedora 36
      Status: stable
        Type: enhancement
    Severity: unspecified
       Karma: 2
   Autokarma: True  [-3, 2]
    Autotime: True
   CI Status: All required tests passed
        Bugs: 2155906 - btrfs-progs-6.1 is available
            : 2156710 - btrfs lost subvolume UUID
       Notes: #### Upstream changes
            :    * filesystem df: add json output
            :    * qgroup show: add json output
            :    * new command: 'inspect-internal map-swapfile' to check
            : swapfile and its
            :      swapfile_offset value used for hibernation
            :    * corrupt-block: fix parsing of option --root argument
            :    * experimental (interfaces not finalized):
            :       * new command 'inspect-internal list-chunks'
            :       * new group reflink, command clone
            :    * other:
            :       * synchronize some files with kernel versions
            :       * docs updates
            :       * build: use gnu11
            : #### Build changes
            : * Use `libgcrypt` for crypto instead of built-in code
   Submitter: ngompa

Finally, as I wrote before, Bodhi doesn't inject any markup or markdown, the 'notes' field is just the text a user enters and we have no control on that.

In the link you provided several comments above, ordered and unordered lists should be supported, but the provided screenshot clearly shows that the package manager is not fixed at all (the list under "upstream changes" is not rendered, while the list under "build changes" it is).

I believe that link is no longer relevant because after your previous comment in this issue, GNOME Software stopped converting from markdown to markup. So now instead of markup that looks bad, we're left with markdown that looks bad.

Honestly I have no clue who designed this system or how it's really expected to work.

It might help to have @mcrha here since he is a GNOME Software developer and understands what's going on much better than I do.

I think the bug https://bugzilla.redhat.com/show_bug.cgi?id=2106872 references a related change for the RPM package mentioned above (I recall seeing that RPM-related screenshot too, but I cannot find the relevant change). It was a bug not showing transformed markdown to the markup as markup understood by Pango (through GtkLabel).

There are still some markdown forms the converted used in the gnome-software does not understand. For example the Evolution update uses headers, which are left as they are in the gnome-software (meaning they look like #### in the GUI).

The upstream maintainers are not much willing to extend the set of the supported tags, because they want to follow the appstream spec, which makes sense.

The only option I can see can be to extend the gnome-software's markdown to markup convertor to sanitize the known tags, which cannot be used as a markup.

ok, so sounds like there's nothing to do on the bodhi end? Or is there?

I believe that link is no longer relevant because after your previous comment in this issue, GNOME Software stopped converting from markdown to markup. So now instead of markup that looks bad, we're left with markdown that looks bad.

Uh, it seems I'm totally incorrect. Sorry about that.

ok, so sounds like there's nothing to do on the bodhi end? Or is there?

Depends. Milan has an idea:

The only option I can see can be to extend the gnome-software's markdown to markup convertor to sanitize the known tags, which cannot be used as a markup.

But I'm not sure if that will actually work.

Do GNOME Software and KDE Discover agree on what markdown is supported vs. what markdown is not supported? If so, can you give us a list of what markdown bodhi should stop supporting?

I believe there is nothing to be done on the bodhi side.

I do not think you should limit bodhi just because one of the apps consuming its plain text file cannot handle all the things other apps can use with no problem. It's not a thing for the bodhi, it's a thing for the apps/consumers of the data.

I do not know KDE Discover. You might find more similar apps, maybe like DnfDragora? The others might not be that strict with the markup/down as the gnome-software is, I've no idea.

In that case, @mcrha I will ask you to ensure GNOME Software supports all the markdown supported by bodhi. We need bodhi and GNOME Software to precisely agree on what is allowed and what is not; otherwise, maintainers cannot be expected to properly format non-broken update messages. The status quo is not OK.

Filled https://gitlab.gnome.org/GNOME/gnome-software/-/issues/2054 Feel free to close this one in favor of it.

Thanks and please let us know if we can help with the bodhi end...

please let us know if we can help with the bodhi end...

I'm not sure it belongs here, but let's treat is as an out of blue comment: I wanted to provide a link for the documentation on what markdown the bodhi supports in the upstream bug, but I realized I cannot do that due to the design of the bodhi. The help is part of the page, shown as a "modal window", which cannot be referenced from the outside by any basic way. When you look what does pagure here, it references Markdown Syntax, which is a link to a real page https://docs.pagure.org/pagure/usage/markdown.html , which I call a good standard. It has a significant advantage, I can have opened a tab with the help and just switch between the tabs when looking for help, instead of being distracted by a "modal window" every time I'm not sure about what to do. Do not even make me start on the waste of the bandwidth, because that "modal window" is not used that often, still it's always downloaded as part of the page, only hidden to the user (metered networks? anyone?). It would be interesting to know how many gigabytes are wasted this way every week... Anyway, I only struggle of the missing static page with the markdown syntax help on and for bodhi.