Download raw body.
gotwebd: render README.md as HTML, like GitHub does
On Mon, Sep 07, 2026 at 10:47:13AM +0200, akira.sato@keemail.me wrote:
> Hi,
>
> Right now gotwebd already finds README/README.md/README.txt <http://README.md/README.txt> in arepo's top-level tree (got_output_repo_tree() in got_operations.c)and shows the first match under the file listing on the tree page.The problem is that it's dumped through tp_write_htmlescape() into a<pre> block, so Markdown syntax (#, **, links, lists, etc.) shows upliterally instead of being rendered — unlike GitHub, GitLab, Gitea,or even cgit's "about" page.
>
> This patch adds got_render_readme_markdown(), a small wrapper aroundlowdown_buf(3) from textproc/lowdown (already a port, ISC/BSDlicensed, no external dependencies, and explicitly designed to workunder pledge(2) per lowdown(3) — it doesn't need any promise beyondwhat gotwebd's fcgi process already has). Only files ending in .mdgo through this path; plain README and README.txt keep being shownexactly as before, escaped inside <pre>.
>
> Since README content comes from repository data and not from thegotwebd operator, LOWDOWN_HTML_OWASP is enabled so any raw HTMLembedded in the Markdown gets sanitized rather than passed throughas-is.
>
> Also adds a small .readme block to gotweb.css so headings, codeblocks, tables and blockquotes inside the rendered README don't lookout of place next to the rest of the page.
>
> Thanks
Back when the README display feature was originally added, there were
some concerns about the complexity of lowdown relative to gotwebd:
See https://marc.gameoftrees.org/mail/1701959701.84492_0.html
where Omar said this:
"""
Initially i thought it was a good idea but after noticing that lowdown
is several times larger than the whole gotwebd codebase I've changed my
mind.
"""
But maybe we can revisit this decision today. An important change which
has since occurred is that gotwebd is much better separated nowadays.
As you point out, the fcgi.c code runs under pledge("stdio") today and
lowdown is happy with this. Back in 2023, when the READMe feature was
introduced, gotwebd would have been running lowdown under a much longer
list of pledge promises, namely:
pledge("stdio rpath inet recvfd proc exec sendfd unveil")
So lowdown might be large, but it is quite restricted in what it can do
to the system.
I won't mind depending on lowdown, but will add flags to the Makefiles to
make compilation with lowdown optional, probably enabled by default.
The very first implementation of gotweb had an external dependency on
the kcgi library (also written by Kristaps).
It would be nice to have a gotwebd.conf toggle for this feature as well.
Review comments inline below:
> diff --git a/gotwebd/files/htdocs/gotwebd/gotweb.css b/gotwebd/files/htdocs/gotwebd/gotweb.css
> index de0fc45..4b0ae1d 100755
> --- a/gotwebd/files/htdocs/gotwebd/gotweb.css
> +++ b/gotwebd/files/htdocs/gotwebd/gotweb.css
> @@ -382,6 +382,38 @@ header.subtitle h2 {
> margin-bottom: 20px;
> border-collapse: collapse;
> }
> +.readme {
> + max-width: 900px;
> + line-height: 1.5;
> + padding: 1em 0;
> +}
> +.readme h1, .readme h2, .readme h3 {
> + border-bottom: 1px solid LightSlateGray;
> + padding-bottom: 0.2em;
> +}
> +.readme pre {
> + background-color: #f6f6f6;
> + padding: 0.5em;
> + overflow-x: auto;
> +}
> +.readme code {
> + font-family: monospace;
> + background-color: #f6f6f6;
> + padding: 0.1em 0.3em;
> +}
> +.readme table {
> + border-collapse: collapse;
> +}
> +.readme th, .readme td {
> + border: 1px solid LightSlateGray;
> + padding: 0.3em 0.6em;
> +}
> +.readme blockquote {
> + border-left: 3px solid LightSlateGray;
> + margin-left: 0;
> + padding-left: 1em;
> + color: #555;
> +}
> .tree_wrapper:nth-child(odd) {
> background-color: #d8f3ef;
Did you check whether CSS specific to the light-theme or the dark-theme
would be needed here?
> }
> diff --git a/gotwebd/got_operations.c b/gotwebd/got_operations.c
> index 68e9ef0..5f0ac3a 100644
> --- a/gotwebd/got_operations.c
> +++ b/gotwebd/got_operations.c
> @@ -28,6 +28,8 @@
> #include <string.h>
> #include <unistd.h>
>
> +#include <lowdown.h>
> +
> #include "got_error.h"
> #include "got_object.h"
> #include "got_reference.h"
> @@ -760,6 +762,62 @@ got_output_repo_tree(struct request *c, char **readme,
> return 0;
> }
>
> +/*
> + * Render a Markdown README (buf/len) into a sanitized HTML5 fragment,
> + * the same way GitHub renders README.md on a repository's front page.
Why is it important to point out that unrelated software such as GitHub
has similar features? We are not documenting a specific workaround related
to another implementation, so I don't think it's worth mentioning GitHub.
> + * Returns 0 and a NUL-terminated *html on success, -1 on failure.
> + * The caller must free *html.
> + */
> +int
> +got_render_readme_markdown(char **html, const uint8_t *buf, size_t len)
> +{
> + struct lowdown_opts opts;
> + char *out = NULL;
> + size_t outlen = 0;
> +
> + *html = NULL;
> +
> + memset(&opts, 0, sizeof(opts));
> + opts.type = LOWDOWN_HTML;
> + opts.feat = LOWDOWN_AUTOLINK | LOWDOWN_TABLES | LOWDOWN_FENCED |
> + LOWDOWN_STRIKE | LOWDOWN_SUPER | LOWDOWN_COMMONMARK |
> + LOWDOWN_DEFLIST | LOWDOWN_ATTRS;
> + /*
> + * OWASP-sanitize any raw HTML embedded in the README: this content
> + * comes from repository data which gotwebd must not trust blindly.
> + */
> + opts.oflags = LOWDOWN_HTML_HEAD_IDS | LOWDOWN_HTML_NUM_ENT |
> + LOWDOWN_HTML_OWASP | LOWDOWN_SMARTY;
> +
> + if (!lowdown_buf(&opts, (const char *)buf, len, &out, &outlen, NULL))
> + return -1;
> +
> + /* lowdown_buf() does not guarantee NUL-termination. */
> + *html = malloc(outlen + 1);
> + if (*html == NULL) {
> + free(out);
> + return -1;
> + }
> + memcpy(*html, out, outlen);
Why copy the buffer instead of modifying the existing *out buffer?
Just cast away const, or remove use of const entirely by making *out
an actual output argument of this function.
> + (*html)[outlen] = '\0';
> + free(out);
> + return 0;
> +}
> +
> +/*
> + * Return non-zero if the given README file name should be rendered as
> + * Markdown (case-insensitive ".md" suffix), matching GitHub's convention.
> + */
The entire above comment states the obvious and could be removed.
> +int
> +got_readme_is_markdown(const char *name)
> +{
> + size_t len = strlen(name);
> +
> + if (len < 3)
> + return 0;
> + return strcasecmp(name + len - 3, ".md") == 0;
> +}
> +
> const struct got_error *
> got_open_blob_for_output(struct got_blob_object **blob, int *fd,
> int *binary, struct request *c, const char *directory, const char *file,
> diff --git a/gotwebd/gotwebd.h b/gotwebd/gotwebd.h
> index 09387b7..adcf4ba 100644
> --- a/gotwebd/gotwebd.h
> +++ b/gotwebd/gotwebd.h
> @@ -703,6 +703,8 @@ const struct got_error *got_get_repo_heads(struct request *);
> const struct got_error *got_open_diff_for_output(FILE **, struct request *);
> int got_output_repo_tree(struct request *, char **,
> int (*)(struct template *, struct got_tree_entry *));
> +int got_render_readme_markdown(char **, const uint8_t *, size_t);
> +int got_readme_is_markdown(const char *);
> const struct got_error *got_open_blob_for_output(struct got_blob_object **,
> int *, int *, struct request *, const char *, const char *, const char *);
> int got_output_blob_by_lines(struct template *, struct got_blob_object *,
> diff --git a/gotwebd/pages.tmpl b/gotwebd/pages.tmpl
> index a148101..26560ac 100644
> --- a/gotwebd/pages.tmpl
> +++ b/gotwebd/pages.tmpl
> @@ -737,7 +737,11 @@ nextsep(char *s, char **t)
> const struct querystring *qs = c->t->qs;
> struct gotweb_url url;
> char *readme = NULL;
> + char *readme_buf = NULL;
> + char *readme_html = NULL;
> + size_t readme_bufsz = 0;
> int binary;
> + int is_markdown = 0;
> const uint8_t *buf;
> size_t len;
> !}
> @@ -746,6 +750,8 @@ nextsep(char *s, char **t)
> </table>
> {{ if readme }}
> {!
> + is_markdown = got_readme_is_markdown(readme);
> +
> error = got_open_blob_for_output(&t->blob, &t->fd, &binary, c,
> qs->folder[0] ? qs->folder : NULL, readme,
> qs->commit[0] ? qs->commit : NULL);
> @@ -768,6 +774,47 @@ nextsep(char *s, char **t)
> {{ readme }}
> </a>
> </h2>
> + {{ if is_markdown }}
> + {!
> + /* Slurp the blob so lowdown can parse it as one buffer. */
> + for (;;) {
> + char *tmp;
If the blob is read from a loose object file, the first iteration of this
loop will see the blob object header, which needs to be skipped. There is
a function which returns the header's size. Look at install_blob() in
lib/worktree.c for an example.
Only trusted users with write access can add contents to README.md, but
we should enforce a maximum size limit beyond which we fall back on just
dumping the raw markdown file, rather than running out of memory when
someone makes a mistake.
Or maybe we should pass another tempfile handle to this process? Then we
could use got_object_blob_dump_to_file() and lowdown_file() instead of
managing blob contents in memory, and we won't have to worry much about
the size of the README file. Someone might still fill up a disk partition
with a very large README, but that will result in an easily recoverable
I/O error (truncate the temporary file and fall back on displaying the
raw markdown file) rather than an out-of-memory problem.
> +
> + error = got_object_blob_read_block(&len, t->blob);
> + if (error) {
> + free(readme);
> + free(readme_buf);
> + return (-1);
> + }
> + if (len == 0)
> + break;
> + buf = got_object_blob_get_read_buf(t->blob);
> + tmp = realloc(readme_buf, readme_bufsz + len);
> + if (tmp == NULL) {
> + free(readme);
> + free(readme_buf);
> + return (-1);
> + }
> + readme_buf = tmp;
> + memcpy(readme_buf + readme_bufsz, buf, len);
> + readme_bufsz += len;
> + }
> + if (got_render_readme_markdown(&readme_html,
> + (uint8_t *)readme_buf, readme_bufsz) == -1) {
> + free(readme);
> + free(readme_buf);
> + return (-1);
> + }
> + !}
> + <div class="readme markdown-body">
> + {! if (tp_write(tp, readme_html, strlen(readme_html)) == -1) {
> + free(readme);
> + free(readme_buf);
> + free(readme_html);
> + return (-1);
> + } !}
> + </div>
> + {{ else }}
> <pre>
> {!
> for (;;) {
> @@ -786,10 +833,15 @@ nextsep(char *s, char **t)
> }
> !}
> </pre>
> + {{ end }}
> {{ end }}
> {{ end }}
> {{ finally }}
> - {! free(readme); !}
> + {!
> + free(readme);
> + free(readme_buf);
> + free(readme_html);
> + !}
> {{ end }}
>
> {{ define gotweb_render_tree(struct template *tp) }}
>
> Index: gotwebd/Makefile
> ===================================================================
> --- gotwebd/Makefile
> +++ gotwebd/Makefile
> @@ -30,7 +30,7 @@
> MAN = ${PROG}.conf.5 ${PROG}.8
> CPPFLAGS += -I${.CURDIR}/../include -I${.CURDIR}/../lib -I${.CURDIR}
> CPPFLAGS += -I${.CURDIR}/../template
> -LDADD += -lz -levent -lutil -lm -lcrypto
> +LDADD += -lz -levent -lutil -lm -lcrypto -llowdown
> YFLAGS =
> DPADD = ${LIBEVENT} ${LIBUTIL} ${LIBM} ${LIBCRYPTO}
> #CFLAGS += -DGOT_NO_OBJ_CACHE
gotwebd: render README.md as HTML, like GitHub does