From: Omar Polo <op@omarpolo.com>
Subject: Re: optimise reading the file index
To: Christian Weisgerber <naddy@mips.inka.de>
Cc: Mark Jamsek <mark@jamsek.com>, gameoftrees@openbsd.org
Date: Thu, 27 Jul 2023 15:26:45 +0200

On 2023/07/27 15:11:04 +0200, Christian Weisgerber <naddy@mips.inka.de> wrote:
> Mark Jamsek:
> 
> > The main change is that we memory map the file index to avoid lots of
> > small reads that can be costly in large repos, for example, with many
> > files.
> 
> I'd think that "got status" in /usr/src is dominated by the 92,300
> fstatat() calls...
> 
> I tried this (cd /usr/src; got st) on an APU2, which qualifies as
> slow for amd64 purposes:
> 
> 0.92-current, three runs:
>     0m19.97s real     0m08.82s user     0m11.16s system
>     0m19.95s real     0m08.56s user     0m10.93s system
>     0m19.99s real     0m08.74s user     0m10.96s system
> 
> + patch, three runs:
>     0m19.28s real     0m08.21s user     0m10.96s system
>     0m19.14s real     0m08.02s user     0m10.76s system
>     0m19.33s real     0m07.72s user     0m11.42s system
> 
> I think we need more measurements to be certain that there is any
> effect at all. :->

A good 'benchmark' for the fileindex changes is 'got info <path>'.
Looking at it I can observe a small improvement with the proposed
patch in src.git

More than half of the running time is still dominated by the insertion
in the rbtree, specifically by comparing paths.