[RFC] template system for gotwebd

For some personal project(s) I ended up writing a small template """engine""" to generate HTML (but not only.) I did this because I found a bit of a pain to (safely) generate HTML by means of printf or the like. Since this has been working fine for me, I thought gotwebd could benefit from it too. Diff below is not intended to be committed as-is, neither to be reviewed or even tested, it's just for demonstration purpose: the added code is not used at runtime! If y'all like the direction, I can work on a better diff that moves the HTML generation to this template system. How it works? ============= At a higher level the templates are file written in a custom syntax which is mostly HTML except for some special constructs embedded within {{ ... }} or {! ... !}. Producing HTML (or anything else really) is achieved by defining "blocks". These blocks are turned at *compile time* into plain C functions that output the text via some provided functions. I'd like to stress the point that the templates are *compiled* into C and not executed at runtime, as happens with many other template systems. here's an example of how to define a block: {{ define gw_base(struct template *tp, const char *title) }} <!doctype html> <html> <head> <title>{{ title }}</title> </head> <body> {{ render tp->tp_body }} </body> </html> {{ end }} that gets turned into something like the following pseudo code: int gw_base(struct template *tp, const char *title) { /* print from <!doctype> to <title> */ /* ... */ print(htmlescape(title)); /* print until <body> */ /* ... */ if (tp->tp_body(tp) == -1) return (-1); /* print </body></html> */ return (0); } In short, it just turn some HTML fragments into glorified printfs, but providing an easy way to safely interpolate variables. It implicitly HTML-escape the variables and provides a mean to choose a different escape strategy case-by-case (e.g. to urlencode something or to *not* escape something else.) Everything is implicitly escaped, you have to be verbose to change/disable it. Different escaping "strategies" can be choosen as follows: {{ foo | unsafe }} aka don't escape {{ foo | urlescape }} There's also some small syntax for conditionals: {{ if something }} ... {{ else if x && !y }} ... {{ else }} ... {{ end }} and loops: {{ tailq-foreach rc &t->repo_commits entry }} ... {{ end }} Since these templates are turned into C code, there's a mean to inline some code that will be left as-is in the generated file using the {! /* code ... */ !} construct. Finally, there's a small syntax sugar for calling another block using the `render' keyword: {{ render another_block }} Pros and cons ============= The main advantage is that it avoids very ugly sections of if (fcgi_printf(c, "...") == -1) goto err; Writing HTML in a separate file allows to use a proper indentation and makes leaving dangling tags open harder. The template system also implicitly discards blanks at the start of the lines, so it also avoids sending blanks to the browser. I'd argue that it's easier/fun to hack on these templates files rather than writing HTML as bits of C strings. Turning this templates into C code that gets then compiled and linked as part of gotwebd also avoids interpreting stuff at runtime and allows for the compiler to do some optimizations here and there (not that I think it can optimize much tho.) It's possibly more efficient than what we currently have. The generated code uses a sequence of puts-like and putc-like functions to both output the HTML and do the escaping, so it avoids the overhead of the implicit vasprintf in fcgi_printf and all the alloc/free we're currently doing to escape the strings. The interpolated strings are automatically escaped by default, instead of requiring to manually escaping them (and free the allocated escaped string.) The "compiler" uses the #line directive, just like yacc does, so eventual C error or warnings are reported relatively to the template file and not the generated code. Even the debugger shows the template being processed. We can probably leverage the template system to write some regress tests for gotwebd. The template system is "output agnostic", it just uses the given functions to do the output. In gotwebd the output will be through fastcgi, but in a hypothetical regress suite we could override the templates and just print plain text to standard output. I have not yet investigated into this, it's pure speculation. The template 'language' is very Go html/template inspired, but is still decently different and this can be confusing. The "compiler" is not even remotely smart, it just bashes strings together, and the generated code assume there's a variable in scope called `tp' of type `struct template', so every block needs to have an argument with that name and of that type. The error reporting is not great, but as it's an internal tool I don't think it's a big deal. The fact that trims out the blanks leads to some possible edge cases where we want to include a space but the template "compiler" trims it out. It's possible to work around it by using {{ " " }}. I have not found a clean way to integrate the template generation into the bsd.prog.mk infrastructure. Diff belows assumes the template "compiler" is in ${.CURDIR}/template/obj/template, but one can also not `make obj'. -*-*- Diff below includes an (incomplete) example of how we could probably rewrite gotweb_render_header/footer and the commits fragment. For a more real-life example, you can take a look at `fragments.tmpl' in https://git.omarpolo.com/?action=summary&path=galileo.git where this template system originates from. What do you think? Would something like this be a good addition to gotwebd? diff /home/op/w/gwd-template commit - 611e5fc2074d428e17f920dc595496af4dd0dc77 path + /home/op/w/gwd-template blob - 01296896441d21b985a59ffc870ef3d3b03f4a31 file + gotwebd/Makefile --- gotwebd/Makefile +++ gotwebd/Makefile @@ -1,13 +1,14 @@ .PATH:${.CURDIR}/../lib +.PATH:${.CURDIR}/template -SUBDIR = libexec +SUBDIR = libexec template .include "../got-version.mk" .include "Makefile.inc" PROG = gotwebd SRCS = config.c sockets.c log.c gotwebd.c parse.y proc.c \ - fcgi.c gotweb.c got_operations.c + fcgi.c gotweb.c got_operations.c tmpl.o SRCS += blame.c commit_graph.c delta.c diff.c \ diffreg.c error.c fileindex.c object.c object_cache.c \ object_idset.c object_parse.c opentemp.c path.c pack.c \ @@ -19,9 +20,20 @@ SRCS += blame.c commit_graph.c delta.c diff.c \ diff_output_edscript.c diff_patience.c bloom.c murmurhash2.c \ worktree_open.c patch.c sigs.c date.c sockaddr.c +TEMPLATES = pages.tmpl + +.for t in ${TEMPLATES} +SRCS += ${t:.tmpl=.c} +CLEANFILES += ${t:.tmpl=.c} + +${t:.tmpl=.c}: ${t} + ${.CURDIR}/template/obj/template $? > $@ +.endfor + MAN = ${PROG}.conf.5 ${PROG}.8 CPPFLAGS += -I${.CURDIR}/../include -I${.CURDIR}/../lib -I${.CURDIR} +CPPFLAGS += -I${.CURDIR}/template/ LDADD += -lz -levent -lutil -lm YFLAGS = DPADD = ${LIBEVENT} ${LIBUTIL} blob - /dev/null file + gotwebd/pages.tmpl --- /dev/null +++ gotwebd/pages.tmpl @@ -0,0 +1,195 @@ +{! + +/* + * Copyright (c) 2022 Omar Polo <op@omarpolo.com> + * Copyright (c) 2016, 2019, 2020-2022 Tracey Emery <tracey@traceyemery.net> + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include <sys/types.h> +#include <sys/queue.h> +#include <sys/uio.h> + +#include <event.h> +#include <stdint.h> +#include <imsg.h> + +#include "proc.h" +#include "tmpl.h" + +#include "gotwebd.h" + +int gw_base(struct template *, const char *); +int gw_commits(struct template *); +int gw_navs(struct template *); + +static inline const char * +action_name(int action) +{ + switch (action) { + case BLAME: + return"blame"; + case BRIEFS: + return"briefs"; + case COMMITS: + return"commits"; + case DIFF: + return"diff"; + case SUMMARY: + return"summary"; + case TAG: + return"tag"; + case TAGS: + return"tags"; + case TREE: + return"tree"; + default: + return "unknown"; + } +} + +!} + +{{ define gw_base(struct template *tp, const char *title) }} +{! + struct request *c = tp->tp_arg; + struct server *srv = c->srv; + struct querystring *qs = c->t->qs; + const char *prfx = c->script_name; + const char *css = srv->custom_css; + char index[32]; + int r; + + r = snprintf(index, sizeof(index), "%d", qs->index_page); + if (r < 0 || (size_t)r >= sizeof(index)) + return -1; +!} +<!doctype html> +<html> + <head> + <meta charset="utf-8" /> + <title>{{ title }}</title> + <meta name="viewport" content="initial-scale=.75" /> + <meta name="msapplication-TileColor" content="#da532c" /> + <meta name="theme-color" content="#ffffff"/> + <link rel="apple-touch-icon" sizes="180x180" href="{{ prfx }}apple-touch-icon.png" /> + <link rel="icon" type="image/png" sizes="32x32" href="{{ prfx }}favicon-32x32.png" /> + <link rel="icon" type="image/png" sizes="16x16" href="{{ prfx }}favicon-16x16.png" /> + <link rel="manifest" href="{{ prfx }}site.webmanifest"/> + <link rel="mask-icon" href="{{ prfx }}safari-pinned-tab.svg" /> + <link rel="stylesheet" type="text/css" href="{{ prfx }}{{ css }}" /> + </head> + <body> + <div id="gw_body"> + <div id="header"> + <div id="got_link"> + <a href="{{ srv->logo_url }}" target="_blank"> + <imsg src="{{ prfx }}{{ srv->logo }}" + </a> + </div> + </div> + <div id="site_path"> + <div id="site_link"> + <a href="?index_page={{ index }}">{{ srv->site_link }}</a> + {{ if qs->path }} + {! /* ... */ !} + {{ end }} + {{ if qs->action != INDEX }} + {{ " / " }} {{ action_name(qs->action) }} + {{ end }} + </div> + </div> + <div id="content"> + {{ render tp->tp_body }} + </div> + </div> + </body> +</html> +{{ end }} + +{{ define gw_commits(struct template *tp) }} +{! + const struct got_error *error = NULL; + struct repo_commit *rc; + struct request *c = tp->tp_arg; + struct server *srv = c->srv; + struct transport *t = c->t; + + error = got_get_repo_commits(c, srv->max_commits_display); + if (error) + return -1; +!} +<div class="commits_title_wrapper" + <div class="commits_title">Commits</div> +</div> +<div class="commits_content"> + {{ tailq-foreach rc &t->repo_commits entry }} + <div class="commits_header_wrapper"> + <div class="commits_header"> + <div class="header_commit_title">Commit:</div> + <div class="header_commit">{{ rc->commit_id }}</div> + <div class="header_author_title">Author:</div> + <div class="header_author">{{ rc->author }}</div> + <div class="header_age_title">Date:</div> + <div class="header_age">...</div> + </div> + </div> + <div class="dotted_line"></div> + <div class="commit"> + {{ rc->commit_msg }} + </div> + <div class="navs_wrapper"> + <div class="navs"> + {! /* previous link */ !} + {! /* next link */ !} + </div> + </div> + <div class="dotted_line"></div> + {{ end }} + {{ if t->next_id || t->prev_id }} + {{ render gw_navs(tp) }} + {{ end }} +</div> +{{ end }} + +{{ define gw_navs(struct template *tp) }} +<div id="np_wrapper"> + <div id="nav_prev"> + {! /* ... */ !} + </div> + <div id="nav_next"> + {! /* ... */ !} + </div> +</div> +{{ end }} + +{! + +/* + * Example usage: + * + * Initialize the template: + * + * c->tp = template(c, puts_like_fns, putc_like_fns); + * if (c->tp == NULL) + * return got_error_from_errno("template"); + * + * The call a "block" defined with ``{{ define ... }}'' as a normal C + * function: + * + * if (gw_base(c->tp, "page title") == -1) + * // error + */ + +!} blob - /dev/null file + gotwebd/template/Makefile --- /dev/null +++ gotwebd/template/Makefile @@ -0,0 +1,21 @@ +PROG = template +SRCS = template.c parse.y + +# XXX +NOMAN = Yes + +CFLAGS += -I${.CURDIR} + +WARNINGS = yes + +CDIAGFLAGS = -Wall -Wextra -Wpointer-arith -Wuninitialized +CDIAGFLAGS+= -Wstrict-prototypes -Wmissing-prototypes -Wunused +CDIAGFLAGS+= -Wsign-compare -Wshadow -Wno-unused-parameter +CDIAGFLAGS+= -Wno-missing-field-initializers +CDIAGFLAGS+= -Werror + +.if make(regress) || make(clean) +SUBDIR = regress +.endif + +.include <bsd.prog.mk> blob - /dev/null file + gotwebd/template/parse.y --- /dev/null +++ gotwebd/template/parse.y @@ -0,0 +1,704 @@ +/* + * Copyright (c) 2022 Omar Polo <op@omarpolo.com> + * Copyright (c) 2007-2016 Reyk Floeter <reyk@openbsd.org> + * Copyright (c) 2004, 2005 Esben Norby <norby@openbsd.org> + * Copyright (c) 2004 Ryan McBride <mcbride@openbsd.org> + * Copyright (c) 2002, 2003, 2004 Henning Brauer <henning@openbsd.org> + * Copyright (c) 2001 Markus Friedl. All rights reserved. + * Copyright (c) 2001 Daniel Hartmeier. All rights reserved. + * Copyright (c) 2001 Theo de Raadt. All rights reserved. + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +%{ + +#include <sys/queue.h> + +#include <ctype.h> +#include <err.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdarg.h> +#include <stdint.h> +#include <string.h> +#include <unistd.h> + +#ifndef nitems +#define nitems(_a) (sizeof((_a)) / sizeof((_a)[0])) +#endif + +TAILQ_HEAD(files, file) files = TAILQ_HEAD_INITIALIZER(files); +static struct file { + TAILQ_ENTRY(file) entry; + FILE *stream; + char *name; + size_t ungetpos; + size_t ungetsize; + u_char *ungetbuf; + int eof_reached; + int lineno; + int errors; +} *file, *topfile; +int parse(const char *); +struct file *pushfile(const char *, int); +int popfile(void); +int yyparse(void); +int yylex(void); +int yyerror(const char *, ...) + __attribute__((__format__ (printf, 1, 2))) + __attribute__((__nonnull__ (1))); +int kw_cmp(const void *, const void *); +int lookup(char *); +int igetc(void); +int lgetc(int); +void lungetc(int); +int findeol(void); + +void dbg(void); +void printq(const char *); + +extern int nodebug; + +static int block; +static int in_define; +static int errors; +static char lerr[32]; + +typedef struct { + union { + char *string; + } v; + int lineno; +} YYSTYPE; + +%} + +%token DEFINE ELSE END ERROR ESCAPE FINALLY IF INCLUDE +%token RENDER TQFOREACH UNSAFE URLESCAPE +%token <v.string> STRING +%type <v.string> string +%type <v.string> stringy + +%% + +grammar : /* empty */ + | grammar include + | grammar verbatim + | grammar block + | grammar error { file->errors++; } + ; + +include : INCLUDE STRING { + struct file *nfile; + + if ((nfile = pushfile($2, 0)) == NULL) { + yyerror("failed to include file %s", $2); + free($2); + YYERROR; + } + free($2); + + file = nfile; + lungetc('\n'); + } + ; + +verbatim : '!' verbatim1 '!' { + if (in_define) { + /* TODO: check template status and exit in case */ + } + } + ; + +verbatim1 : /* empty */ + | verbatim1 STRING { + if (*$2 != '\0') { + dbg(); + puts($2); + } + free($2); + } + ; + +verbatims : /* empty */ + | verbatims verbatim + ; + +raw : STRING { + dbg(); + printf("if (tp->tp_puts(tp, "); + printq($1); + printf(") == -1) goto %s;\n", lerr); + + free($1); + } + ; + +block : define body end { + printf("%s:\n", lerr); + puts("return tp->tp_ret;"); + puts("}"); + in_define = 0; + } + | define body finally end { + puts("return tp->tp_ret;"); + puts("}"); + in_define = 0; + } + ; + +define : '{' DEFINE string '}' { + in_define = 1; + (void)snprintf(lerr, sizeof(lerr), "err%llu", + (unsigned long long)arc4random()); + + dbg(); + printf("int\n%s\n{\n", $3); + + free($3); + } + ; + +body : /* empty */ + | body verbatim + | body raw + | body special + ; + +special : '{' RENDER string '}' { + dbg(); + if (strrchr($3, ')') != NULL) + printf("if (%s == -1) goto %s;\n", + $3, lerr); + else + printf("if (%s != NULL && %s(tp) == -1) " + "goto %s;\n", $3, $3, lerr); + free($3); + } + | if body endif { puts("}"); } + | loop + | '{' string '|' ESCAPE '}' { + dbg(); + printf("if (tp->tp_escape(tp, %s) == -1) goto %s;\n", + $2, lerr); + free($2); + } + | '{' string '|' UNSAFE '}' { + dbg(); + printf("if (tp->tp_puts(tp, %s) == -1) goto %s;\n", + $2, lerr); + free($2); + } + | '{' string '|' URLESCAPE '}' { + dbg(); + printf("if (tp_urlescape(tp, %s) == -1) goto %s;\n", + $2, lerr); + free($2); + } + | '{' string '}' { + dbg(); + printf("if (tp->tp_escape(tp, %s) == -1) goto %s;\n", + $2, lerr); + free($2); + } + ; + +if : '{' IF stringy '}' { + dbg(); + printf("if (%s) {\n", $3); + free($3); + } + ; + +endif : end + | else body end + | elsif body endif + ; + +elsif : '{' ELSE IF string '}' { + dbg(); + printf("} else if (%s) {\n", $4); + free($4); + } + ; + +else : '{' ELSE '}' { + dbg(); + puts("} else {"); + } + ; + +loop : '{' TQFOREACH STRING STRING STRING '}' { + printf("TAILQ_FOREACH(%s, %s, %s) {\n", + $3, $4, $5); + free($3); + free($4); + free($5); + } body end { + puts("}"); + } + ; + +end : '{' END '}' + ; + +finally : '{' FINALLY '}' { + dbg(); + printf("%s:\n", lerr); + } verbatims + ; + +string : STRING string { + if (asprintf(&$$, "%s %s", $1, $2) == -1) + err(1, "asprintf"); + free($1); + free($2); + } + | STRING + ; + +stringy : STRING + | STRING stringy { + if (asprintf(&$$, "%s %s", $1, $2) == -1) + err(1, "asprintf"); + free($1); + free($2); + } + | '|' stringy { + if (asprintf(&$$, "|%s", $2) == -1) + err(1, "asprintf"); + free($2); + } + ; + +%% + +struct keywords { + const char *k_name; + int k_val; +}; + +int +yyerror(const char *fmt, ...) +{ + va_list ap; + char *msg; + + file->errors++; + va_start(ap, fmt); + if (vasprintf(&msg, fmt, ap) == -1) + err(1, "yyerror vasprintf"); + va_end(ap); + fprintf(stderr, "%s:%d: %s\n", file->name, yylval.lineno, msg); + free(msg); + return (0); +} + +int +kw_cmp(const void *k, const void *e) +{ + return (strcmp(k, ((const struct keywords *)e)->k_name)); +} + +int +lookup(char *s) +{ + /* this has to be sorted always */ + static const struct keywords keywords[] = { + { "define", DEFINE }, + { "else", ELSE }, + { "end", END }, + { "escape", ESCAPE }, + { "finally", FINALLY }, + { "if", IF }, + { "include", INCLUDE }, + { "render", RENDER }, + { "tailq-foreach", TQFOREACH }, + { "unsafe", UNSAFE }, + { "urlescape", URLESCAPE }, + }; + const struct keywords *p; + + p = bsearch(s, keywords, nitems(keywords), sizeof(keywords[0]), + kw_cmp); + + if (p) + return (p->k_val); + else + return (STRING); +} + +#define START_EXPAND 1 +#define DONE_EXPAND 2 + +static int expanding; + +int +igetc(void) +{ + int c; + + while (1) { + if (file->ungetpos > 0) + c = file->ungetbuf[--file->ungetpos]; + else + c = getc(file->stream); + + if (c == START_EXPAND) + expanding = 1; + else if (c == DONE_EXPAND) + expanding = 0; + else + break; + } + return (c); +} + +int +lgetc(int quotec) +{ + int c, next; + + if (quotec) { + if ((c = igetc()) == EOF) { + yyerror("reached end of filewhile parsing " + "quoted string"); + if (file == topfile || popfile() == EOF) + return (EOF); + return (quotec); + } + return (c); + } + + while ((c = igetc()) == '\\') { + next = igetc(); + if (next != '\n') { + c = next; + break; + } + yylval.lineno = file->lineno; + file->lineno++; + } + if (c == '\t' || c == ' ') { + /* Compress blanks to a sigle space. */ + do { + c = getc(file->stream); + } while (c == '\t' || c == ' '); + ungetc(c, file->stream); + c = ' '; + } + + if (c == EOF) { + /* + * Fake EOL when hit EOF for the first time. This gets line + * count rigchtif last line included file is syntactically + * invalid and has no newline. + */ + if (file->eof_reached == 0) { + file->eof_reached = 1; + return ('\n'); + } + while (c == EOF) { + if (file == topfile || popfile() == EOF) + return (EOF); + c = igetc(); + } + } + return (c); +} + +void +lungetc(int c) +{ + if (c == EOF) + return; + + if (file->ungetpos >= file->ungetsize) { + void *p = reallocarray(file->ungetbuf, file->ungetsize, 2); + if (p == NULL) + err(1, "reallocarray"); + file->ungetbuf = p; + file->ungetsize *= 2; + } + file->ungetbuf[file->ungetpos++] = c; +} + +int +findeol(void) +{ + int c; + + /* skip to either EOF or the first real EOL */ + while (1) { + c = lgetc(0); + if (c == '\n') { + file->lineno++; + break; + } + if (c == EOF) + break; + } + return (ERROR); +} + +int +yylex(void) +{ + char buf[8096]; + char *p = buf; + int c; + int token; + int starting = 0; + int ending = 0; + + if (!in_define && block == 0) { + while ((c = lgetc(0)) != '{' && c != EOF) { + if (c == '\n') + file->lineno++; + } + + if (c == EOF) + return (0); + +newblock: + c = lgetc(0); + if (c == '{' || c == '!') { + if (c == '{') + block = '}'; + else + block = c; + return (c); + } + if (c == '\n') + file->lineno++; + } + + while ((c = lgetc(0)) == ' ' || c == '\t' || c == '\n') { + if (c == '\n') + file->lineno++; + } + + if (c == EOF) { + yyerror("unterminated block"); + return (0); + } + + yylval.lineno = file->lineno; + + if (block != 0 && c == block) { + if ((c = lgetc(0)) == '}') { + if (block == '!') { + block = 0; + return ('!'); + } + block = 0; + return ('}'); + } + lungetc(c); + c = block; + } + + if (in_define && block == 0) { + if (c == '{') + goto newblock; + + do { + if (starting) { + if (c == '!' || c == '{') { + lungetc('{'); + lungetc(c); + break; + } + starting = 0; + lungetc(c); + c = '{'; + } else if (c == '{') { + starting = 1; + continue; + } + + *p++ = c; + if ((size_t)(p - buf) >= sizeof(buf)) { + yyerror("string too long"); + return (findeol()); + } + } while ((c = lgetc(0)) != EOF && c != '\n'); + *p = '\0'; + if (c == EOF) { + yyerror("unterminated block"); + return (0); + } + if (c == '\n') + file->lineno++; + if ((yylval.v.string = strdup(buf)) == NULL) + err(1, "strdup"); + return (STRING); + } + + if (block == '!') { + do { + if (ending) { + if (c == '}') { + lungetc(c); + lungetc(block); + break; + } + ending = 0; + lungetc(c); + c = block; + } else if (c == '!') { + ending = 1; + continue; + } + + *p++ = c; + if ((size_t)(p - buf) >= sizeof(buf)) { + yyerror("line too long"); + return (findeol()); + } + } while ((c = lgetc(0)) != EOF && c != '\n'); + *p = '\0'; + + if (c == EOF) { + yyerror("unterminated block"); + return (0); + } + if (c == '\n') + file->lineno++; + + if ((yylval.v.string = strdup(buf)) == NULL) + err(1, "strdup"); + return (STRING); + } + + if (c == '|') + return (c); + + do { + if (c == '|') { + lungetc(c); + break; + } + + if (ending) { + if (c == '}') { + lungetc(c); + lungetc('}'); + break; + } + ending = 0; + lungetc(c); + c = block; + } else if (c == '}') { + ending = 1; + continue; + } + + *p++ = c; + if ((size_t)(p - buf) >= sizeof(buf)) { + yyerror("string too long"); + return (findeol()); + } + } while ((c = lgetc(0)) != EOF && !isspace((unsigned char)c)); + *p = '\0'; + + if (c == EOF) { + yyerror("unterminated block"); + return (0); + } + if (c == '\n') + file->lineno++; + if ((token = lookup(buf)) == STRING) + if ((yylval.v.string = strdup(buf)) == NULL) + err(1, "strdup"); + return (token); +} + +struct file * +pushfile(const char *name, int secret) +{ + struct file *nfile; + + if ((nfile = calloc(1, sizeof(*nfile))) == NULL) + err(1, "calloc"); + if ((nfile->name = strdup(name)) == NULL) + err(1, "strdup"); + if ((nfile->stream = fopen(nfile->name, "r")) == NULL) { + warn("can't open %s", nfile->name); + free(nfile->name); + free(nfile); + return (NULL); + } + nfile->lineno = TAILQ_EMPTY(&files) ? 1 : 0; + nfile->ungetsize = 16; + nfile->ungetbuf = malloc(nfile->ungetsize); + if (nfile->ungetbuf == NULL) + err(1, "malloc"); + TAILQ_INSERT_TAIL(&files, nfile, entry); + return (nfile); +} + +int +popfile(void) +{ + struct file *prev; + + if ((prev = TAILQ_PREV(file, files, entry)) != NULL) + prev->errors += file->errors; + + TAILQ_REMOVE(&files, file, entry); + fclose(file->stream); + free(file->name); + free(file->ungetbuf); + free(file); + file = prev; + return (file ? 0 : EOF); +} + +int +parse(const char *filename) +{ + if ((file = pushfile(filename, 0)) == 0) + return (-1); + topfile = file; + + yyparse(); + errors = file->errors; + popfile(); + + return (errors ? -1 : 0); +} + +void +dbg(void) +{ + if (nodebug) + return; + + printf("#line %d ", yylval.lineno); + printq(file->name); + putchar('\n'); +} + +void +printq(const char *str) +{ + putchar('"'); + for (; *str; ++str) { + if (*str == '"') + putchar('\\'); + putchar(*str); + } + putchar('"'); +} blob - /dev/null file + gotwebd/template/regress/00-empty.tmpl blob - /dev/null file + gotwebd/template/regress/01-noise-only.tmpl --- /dev/null +++ gotwebd/template/regress/01-noise-only.tmpl @@ -0,0 +1,2 @@ +only +noise blob - /dev/null file + gotwebd/template/regress/02-only-verbatim.tmpl --- /dev/null +++ gotwebd/template/regress/02-only-verbatim.tmpl @@ -0,0 +1,13 @@ +{! #include <stdio.h> !} + +noise +here + +{! +int +main(void) +{ + puts("hello, world!"); + return 0; +} +!} blob - /dev/null file + gotwebd/template/regress/02.expected --- /dev/null +++ gotwebd/template/regress/02.expected @@ -0,0 +1 @@ +hello, world! blob - /dev/null file + gotwebd/template/regress/03-block.tmpl --- /dev/null +++ gotwebd/template/regress/03-block.tmpl @@ -0,0 +1,17 @@ +{! #include <stdlib.h> !} + +{{ define base(struct template *tp, const char *title) }} +{! char *foo = NULL; !} +<!doctype html> +<html> + <head> + <title>{{ title }}</title> + </head> + <body> + <h1>{{ title }}</h1> + {{ render tp->tp_body }} + </body> +</html> +{{ finally }} +{! free(foo); !} +{{ end }} blob - /dev/null file + gotwebd/template/regress/03.expected --- /dev/null +++ gotwebd/template/regress/03.expected @@ -0,0 +1,2 @@ +<!doctype html><html><head><title> *hello* </title></head><body><h1> *hello* </h1></body></html> +<!doctype html><html><head><title><hello></title></head><body><h1><hello></h1></body></html> blob - /dev/null file + gotwebd/template/regress/04-flow.tmpl --- /dev/null +++ gotwebd/template/regress/04-flow.tmpl @@ -0,0 +1,30 @@ +{! +#include <stdlib.h> +#include <string.h> +!} + +{{ define base(struct template *tp, const char *title) }} +{! char *foo = NULL; !} +<!doctype html> +<html> + <head> + <title>{{ title }}</title> + </head> + <body> + <h1>{{ title }}</h1> + {{ if strchr(title, '*') != NULL }} + <p>"{{ title }}" has a '*' in it</p> + {{ if 1 }} + <p>tautology!</p> + {{ end }} + {{ else if strchr(title, '=') != NULL }} + <p>"{{ title }}" has a '=' in it!</p> + {{ else }} + <p>"{{ title }}" doesn't have a '*' in it</p> + {{ end }} + {{ render tp->tp_body }} + </body> +</html> +{{ finally }} +{! free(foo); !} +{{ end }} blob - /dev/null file + gotwebd/template/regress/04.expected --- /dev/null +++ gotwebd/template/regress/04.expected @@ -0,0 +1,2 @@ +<!doctype html><html><head><title> *hello* </title></head><body><h1> *hello* </h1><p>" *hello* " has a '*' in it</p><p>tautology!</p></body></html> +<!doctype html><html><head><title><hello></title></head><body><h1><hello></h1><p>"<hello>" doesn't have a '*' in it</p></body></html> blob - /dev/null file + gotwebd/template/regress/05-loop.tmpl --- /dev/null +++ gotwebd/template/regress/05-loop.tmpl @@ -0,0 +1,31 @@ +{! +#include <sys/queue.h> +#include <string.h> +#include "lists.h" + +int list(struct template *, struct tailhead *); + +!} + +{{ define base(struct template *tp, struct tailhead *head) }} +<!doctype html> +<html> + <body> + {{ render list(tp, head) }} + </body> +</html> +{{ end }} + +{{ define list(struct template *tp, struct tailhead *head) }} + {! struct entry *np; !} + {{ if !TAILQ_EMPTY(head) }} + <p>items:</p> + <ul> + {{ tailq-foreach np head entries }} + <li>{{ np->text }}</li> + {{ end }} + </ul> + {{ else }} + <p>no items</p> + {{ end }} +{{ end }} blob - /dev/null file + gotwebd/template/regress/05.expected --- /dev/null +++ gotwebd/template/regress/05.expected @@ -0,0 +1,2 @@ +<!doctype html><html><body><p>items:</p><ul><li>1</li><li>2</li></ul></body></html> +<!doctype html><html><body><p>no items</p></body></html> blob - /dev/null file + gotwebd/template/regress/06-escape.tmpl --- /dev/null +++ gotwebd/template/regress/06-escape.tmpl @@ -0,0 +1,14 @@ +{! #include <stdlib.h> !} + +{{ define base(struct template *tp, const char *title) }} +<!doctype html> +<html> + <head> + <title>{{ title | urlescape }}</title> + </head> + <body> + <h1>{{ title | unsafe }}</h1> + {{ render tp->tp_body }} + </body> +</html> +{{ end }} blob - /dev/null file + gotwebd/template/regress/06.expected --- /dev/null +++ gotwebd/template/regress/06.expected @@ -0,0 +1,2 @@ +<!doctype html><html><head><title>%20*hello*%20</title></head><body><h1> *hello* </h1></body></html> +<!doctype html><html><head><title><hello></title></head><body><h1><hello></h1></body></html> blob - /dev/null file + gotwebd/template/regress/Makefile --- /dev/null +++ gotwebd/template/regress/Makefile @@ -0,0 +1,55 @@ +REGRESS_TARGETS = 00-empty \ + 01-noise-only \ + 02-only-verbatim \ + 03-block \ + 04-flow \ + 05-loop \ + 06-escape + +REGRESS_SETUP_ONCE = setup-comp +REGRESS_CLEANUP = clean-comp +NO_OBJ = Yes + +CFLAGS += -I${.CURDIR}/../ + +setup-comp: + cp ${.CURDIR}/../tmpl.c . + ln -f ${.CURDIR}/../template template || \ + ln -f ${.CURDIR}/../obj/template template + +clean-comp: + rm template + rm -f t got 0*.[cdo] runbase.[do] runlist.[do] tmpl.* + +.SUFFIXES: .tmpl .c .o + +.tmpl.c: + ./template $? > $@ + +00-empty: + ./template 00-empty.tmpl >/dev/null + +01-noise-only: + ./template 01-noise-only.tmpl >/dev/null + +02-only-verbatim: 02-only-verbatim.o tmpl.o + ${CC} 02-only-verbatim.o tmpl.o -o t && ./t > got + diff -u ${.CURDIR}/02.expected got + +03-block: 03-block.o runbase.o tmpl.o + ${CC} 03-block.o runbase.o tmpl.o -o t && ./t > got + diff -u ${.CURDIR}/03.expected got + +04-flow: 04-flow.o runbase.o tmpl.o + ${CC} 04-flow.o runbase.o tmpl.o -o t && ./t > got + diff -u ${.CURDIR}/04.expected got + +05-loop: 05-loop.o runlist.o tmpl.o + ${CC} 05-loop.o runlist.o tmpl.o -o t && ./t > got + diff -u ${.CURDIR}/05.expected got + +06-escape: 06-escape.o runbase.o tmpl.o + ${CC} 06-escape.o runbase.o tmpl.o -o t && ./t > got + diff -u ${.CURDIR}/06.expected got + +.include <bsd.regress.mk> blob - /dev/null file + gotwebd/template/regress/lists.h --- /dev/null +++ gotwebd/template/regress/lists.h @@ -0,0 +1,7 @@ +#include <sys/queue.h> + +TAILQ_HEAD(tailhead, entry); +struct entry { + char *text; + TAILQ_ENTRY(entry) entries; +}; blob - /dev/null file + gotwebd/template/regress/runbase.c --- /dev/null +++ gotwebd/template/regress/runbase.c @@ -0,0 +1,69 @@ +/* + * Copyright (c) 2022 Omar Polo <op@omarpolo.com> + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include <err.h> +#include <stdio.h> +#include <stdlib.h> + +#include "tmpl.h" + +int base(struct template *, const char *title); + +int +my_putc(struct template *tp, int c) +{ + FILE *fp = tp->tp_arg; + + if (putc(c, fp) < 0) { + tp->tp_ret = -1; + return (-1); + } + + return (0); +} + +int +my_puts(struct template *tp, const char *s) +{ + FILE *fp = tp->tp_arg; + + if (fputs(s, fp) < 0) { + tp->tp_ret = -1; + return (-1); + } + + return (0); +} + +int +main(int argc, char **argv) +{ + struct template *tp; + + if ((tp = template(stdout, my_puts, my_putc)) == NULL) + err(1, "template"); + + base(tp, " *hello* "); + puts(""); + + template_reset(tp); + + base(tp, "<hello>"); + puts(""); + + free(tp); + return (0); +} blob - /dev/null file + gotwebd/template/regress/runlist.c --- /dev/null +++ gotwebd/template/regress/runlist.c @@ -0,0 +1,89 @@ +/* + * Copyright (c) 2022 Omar Polo <op@omarpolo.com> + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include <err.h> +#include <stdio.h> +#include <stdlib.h> + +#include "tmpl.h" +#include "lists.h" + +int base(struct template *, struct tailhead *); + +int +my_putc(struct template *tp, int c) +{ + FILE *fp = tp->tp_arg; + + if (putc(c, fp) < 0) { + tp->tp_ret = -1; + return (-1); + } + + return (0); +} + +int +my_puts(struct template *tp, const char *s) +{ + FILE *fp = tp->tp_arg; + + if (fputs(s, fp) < 0) { + tp->tp_ret = -1; + return (-1); + } + + return (0); +} + +int +main(int argc, char **argv) +{ + struct template *tp; + struct tailhead head; + struct entry *np; + int i; + + if ((tp = template(stdout, my_puts, my_putc)) == NULL) + err(1, "template"); + + TAILQ_INIT(&head); + for (i = 0; i < 2; ++i) { + if ((np = calloc(1, sizeof(*np))) == NULL) + err(1, "calloc"); + if (asprintf(&np->text, "%d", i+1) == -1) + err(1, "asprintf"); + TAILQ_INSERT_TAIL(&head, np, entries); + } + + base(tp, &head); + puts(""); + + while ((np = TAILQ_FIRST(&head))) { + TAILQ_REMOVE(&head, np, entries); + free(np->text); + free(np); + } + + template_reset(tp); + + base(tp, &head); + puts(""); + + free(tp); + + return (0); +} blob - /dev/null file + gotwebd/template/template.c --- /dev/null +++ gotwebd/template/template.c @@ -0,0 +1,67 @@ +/* + * Copyright (c) 2022 Omar Polo <op@omarpolo.com> + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include <err.h> +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> + +int parse(const char *); + +int nodebug; + +static void __dead +usage(void) +{ + fprintf(stderr, "usage: %s [file...]\n", + getprogname()); + exit(1); +} + +int +main(int argc, char **argv) +{ + int ch, i; + + if (pledge("stdio rpath", NULL) == -1) + err(1, "pledge"); + + while ((ch = getopt(argc, argv, "G")) != -1) { + switch (ch) { + case 'G': + nodebug = 1; + break; + default: + usage(); + } + } + argc -= optind; + argv += optind; + + /* preamble */ + puts("#include \"tmpl.h\""); + + if (argc == 0) { + parse("/dev/stdin"); + exit(0); + } + + for (i = 0; i < argc; ++i) + if (parse(argv[i]) == -1) + return (1); + + return (0); +} blob - /dev/null file + gotwebd/template/tmpl.c --- /dev/null +++ gotwebd/template/tmpl.c @@ -0,0 +1,105 @@ +/* + * Copyright (c) 2022 Omar Polo <op@omarpolo.com> + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include <ctype.h> +#include <stdio.h> +#include <stdlib.h> + +#include "tmpl.h" + +int +tp_urlescape(struct template *tp, const char *str) +{ + int r; + char tmp[4]; + + for (; *str; ++str) { + if (iscntrl((unsigned char)*str) || + isspace((unsigned char)*str) || + *str == '\'' || *str == '"' || *str == '\\') { + r = snprintf(tmp, sizeof(tmp), "%%%2X", *str); + if (r < 0 || (size_t)r >= sizeof(tmp)) { + tp->tp_ret = -1; + return (-1); + } + if (tp->tp_puts(tp, tmp) == -1) + return (-1); + } else { + if (tp->tp_putc(tp, *str) == -1) + return (-1); + } + } + + return (0); +} + +int +tp_htmlescape(struct template *tp, const char *str) +{ + int r; + + for (; *str; ++str) { + switch (*str) { + case '<': + r = tp->tp_puts(tp, "<"); + break; + case '>': + r = tp->tp_puts(tp, ">"); + break; + case '&': + r = tp->tp_puts(tp, "&"); + break; + case '"': + r = tp->tp_puts(tp, """); + break; + case '\'': + r = tp->tp_puts(tp, "'"); + break; + default: + r = tp->tp_putc(tp, *str); + break; + } + + if (r == -1) { + tp->tp_ret = -1; + return (-1); + } + } + + return (0); +} + +struct template * +template(void *arg, tmpl_puts putsfn, tmpl_putc putcfn) +{ + struct template *tp; + + if ((tp = calloc(1, sizeof(*tp))) == NULL) + return (NULL); + + tp->tp_arg = arg; + tp->tp_escape = tp_htmlescape; + tp->tp_puts = putsfn; + tp->tp_putc = putcfn; + + return (tp); +} + +void +template_reset(struct template *tp) +{ + tp->tp_ret = 0; +} blob - /dev/null file + gotwebd/template/tmpl.h --- /dev/null +++ gotwebd/template/tmpl.h @@ -0,0 +1,42 @@ +/* + * Copyright (c) 2022 Omar Polo <op@omarpolo.com> + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#ifndef TMPL_H +#define TMPL_H + +struct template; + +typedef int (*tmpl_puts)(struct template *, const char *); +typedef int (*tmpl_putc)(struct template *, int); + +struct template { + void *tp_arg; + tmpl_puts tp_escape; + tmpl_puts tp_puts; + tmpl_putc tp_putc; + + int tp_ret; + + int (*tp_body)(struct template *); +}; + +int tp_urlescape(struct template *, const char *); +int tp_htmlescape(struct template *, const char *); + +struct template *template(void *, tmpl_puts, tmpl_putc); +void template_reset(struct template *); + +#endif

2022-09-22 09:44 Omar Polo:
[RFC] template system for gotwebd
- 2022-09-22 13:43 Tracey Emery:
  [RFC] template system for gotwebd