On Thu, Jul 27, 2023 at 09:07:13AM -0600, Todd C. Miller wrote:
> On Thu, 27 Jul 2023 15:19:47 +0200, Christian Weisgerber wrote:
> 
> > The current code uses fread(3), i.e., buffered stdio, which translates
> > to a long series of 16 kB read(2) system calls.
> 
> stdio uses the "optimal" buffer size reported for the device,
> st_blksize from struct stat.  The buffer size can be modified via
> a call to setvbuf().  The defaults were created when memory was
> more dear so you may be able to improve things by just bumping the
> buffer size.
> 
> > For a got status of /usr/src, 786 read(2) calls look rather negligible
> > compared to 92,300+ stat(2) calls.
> 
> Yikes, that is a lot.

It seems to be about one stat call per file which can't get much better?

$ find /usr/src | wc -l
   99998

In any case, this fileindex optimization isn't about making 'got status'
run faster. We noticed a visible delay when starting up tog, where the
newly added base-commit marker takes a short time to appear on some systems.
I determined that the time was spent reading the file index, which prompted
Mark to write this patch.