2024-09-05T21:59:43Z Dylan D'Silva:
> Hello all,
>
> Just to chime in here. I wrote that email on the aerc mailing list.
> For some context. This specific work around is in regards to using
> MAILDIR and notmuch with aerc. You can use notmuch and aerc without
> MAILDIR by removing maildir:// and copy-to from accounts.conf but you
> can only send from one account without some interesting solutions, If
> you have multiple accounts this isn't ideal. The aerc team has decided
> this approach is best practice when using aerc, MAILDIR and notmuch.
> This work around was to provide a working solution for people who want
> to follow the developments team best practice while using a BSD
> system.

Where and when exactly was this workaround deemed the best practice by
the 'aerc team'?

> Running without a matching openfile results in "Too many open file
> errors" and the application not functioning.
>
> Aerc uses fsnotify for watching files. Linux backend is inotify  which
> supports recurive watching. OpenBSD uses kqueue which doesn't.
>
> This was never meant to be an ideal solution. I also mentioned to
> Johannes, that a better approach would be to shrink your inbox and have
> a smaller openfile.
>
> I'm using aerc on a OpenBSD system as a desktop and I'm the sole user.
> So I'm not concerned with people opening multiple instances of aerc. I
> realise everyone might not be in that boat.
>
> On Thu Sep 5, 2024 at 11:35 PM AEST, Johannes Thyssen Tishman wrote:
> > 2024-09-05T12:55:06Z Stuart Henderson:
> > > On 2024/09/05 14:23, Johannes Thyssen Tishman wrote:
> > > > 2024-09-05T12:09:51Z Stuart Henderson:
> > > > > On 2024/09/05 12:52, Johannes Thyssen Tishman wrote:
> > > > > > The email below was sent to the aerc-discuss mailing list a couple of
> > > > > > days ago. Is it worth adding a pkg/README for this?
> > > > > > 
> > > > > > 2024-09-03T23:12:23Z "Dylan D'silva" <dylan@dsilva.email>:
> > > > > > Hello all,
> > > > > > 
> > > > > > After some digging I've come to a solution. For those on a BSD system
> > > > > > with large MAILDIR, you'll require something similar. Aerc uses
> > > > > > github.com/fsnotify/fsnotify to monitor changes to files. BSD system
> > > > > > rely on Kqueue which doesn't support recursive watching of files.
> > > > >                                                              ^^
> > > > > 							     directories
> > > > > > Therefore you require one file descriptor for each file.
> > > > >
> > > > > This does not seem to be a good approach for monitoring emails in
> > > > > Maildirs on kqueue systems.
> > > > >
> > > > > Most of the common changes to a Maildir result in either a new file,
> > > > > rename, or deletion and not actually changes to the contents of the
> > > > > files. AFAIK new files won't show up without rescanning the whole
> > > > > directory - for renamed files I don't think you can find the new
> > > > > name so again you need to rescan - it seems like really it's just
> > > > > going to need an efficient way to pick up changes in the dir.
> > > > >
> > > > > (I suppose this is not going to work well with NFS either which is
> > > > > not an uncommon place to store Maildirs..)
> > > > >
> > > > > > Assuming you are running aerc from the current account. You can
> > > > > > change the limit to match your maildir size.
> > > > > > 
> > > > > > For me, I store my maildir in ~/Mail 
> > > > > > find . -type f | wc -l #counts number of files
> > > > > > 71514
> > > > > > 
> > > > > > I run aerc from a staff account. Therefore I updated my /etc/login.conf
> > > > > > under staff to increase the openfile-cur size to 71600 and openfile-max
> > > > > > to 72000
> > > > > >  :openfiles-cur=71600:\
> > > > > >  :openfiles-max=72000:\
> > > > >
> > > > > If you're going to do this, I would recommend a "normal" openfiles-cur,
> > > > > just raise openfiles-max. Then start such software from a script or
> > > > > shell alias which uses ulimit -n to raise the limit just for that
> > > > > software.
>
> What do you consider normal? In your comment below, you made comment
> about not many people running computers from the 80's. 

I think Stuart meant that you could leave openfiles-cur with it's
default value (in case you hadn't changed it before for something else)
and only increase openfiles-max to your desired value. Then you can
start aerc from a script like the following:

#!/bin/sh

ulimit -n 72000
aerc

> Here is a commit for inotify in 2020 making the openfile limit dynamic
> between 8192 and 104576.
>
> From the commit:
> "The default value of inotify.max_user_watches sysctl parameter was set
> to 8192 since the introduction of the inotify feature in 2005 by
> commit 0eeca28 ("[PATCH] inotify"). Today this value is just too
> small for many modern usage. As a result, users have to explicitly set
> it to a larger value to make it work."
>
> https://github.com/torvalds/linux/commit/92890123749bafc317bbfacbe0a62ce08d78efb7
>
>
> It's worth mentioning but the syncthing pkg-readme suggest a openfile of
> 4096. 
>
> > > > >
> > > > > I think it would generally be better to run aerc pointed at an IMAP
> > > > > server though (obviously the IMAP server can just run on the machine
> > > > > running aerc if wanted).
>
> I agree this is a better solution. This was only meant as a work around
> to follow what the dev team are using. 
> > > > >
> > > > > > If you use a large login.conf you might want create a database version
> > > > > > with cap_mkdb /etc/login.conf see login.conf(5).
> > > > >
> > > > > We are mostly not running on computer systems from the 80s and the file
> > > > > is not really slow to parse - however having the db files around does
> > > > > cause trouble when someone forgets to rerun cap_mkdb after changing the
> > > > > file later. I would not recommend this.
>
> The man page for login(5).conf has that comment.
>
> > > > 
> > > > Thanks for the feedback Stuart. Should this advice be included in a
> > > > pkg/README as it is done for syncthing?
> > >
> > > It would be better if the situation was improved upstream, but we
> > > could do...
> >
> > I'll check with upstream.
> >
> > > Rather than documenting the workaround I think it's probably better if
> > > we go for this approach. Setting the OS to allow opening of tens of
> > > thousands of files is very much not ideal, and imagine what it would
> > > be like if you have a couple of users on the machine, all of whom want
> > > to run aerc...
>
> How about improving Kqueue to support recursive watching? 
>
> > I agree, this seems more sane. Thank you Stuart :)
> >
> > > Index: Makefile
> > > ===================================================================
> > > RCS file: /cvs/ports/mail/aerc/Makefile,v
> > > diff -u -p -r1.16 Makefile
> > > --- Makefile	31 Jul 2024 18:36:02 -0000	1.16
> > > +++ Makefile	5 Sep 2024 12:52:52 -0000
> > > @@ -7,6 +7,7 @@ MODGO_MODNAME =	git.sr.ht/~rjarry/aerc
> > >  MODGO_VERSION =	v0.0.0-20240729220005-e037c095a049
> > >  
> > >  V =		0.18.2
> > > +REVISION =	0
> > >  DISTNAME =	aerc-$V
> > >  
> > >  CATEGORIES =	mail
> > > Index: pkg/PLIST
> > > ===================================================================
> > > RCS file: /cvs/ports/mail/aerc/pkg/PLIST,v
> > > diff -u -p -r1.9 PLIST
> > > --- pkg/PLIST	31 Jul 2024 18:36:02 -0000	1.9
> > > +++ pkg/PLIST	5 Sep 2024 12:52:52 -0000
> > > @@ -43,4 +43,5 @@ share/aerc/templates/forward_as_body
> > >  share/aerc/templates/new_message
> > >  share/aerc/templates/quoted_reply
> > >  share/applications/aerc.desktop
> > > +share/doc/pkg-readmes/${PKGSTEM}
> > >  @tag update-desktop-database
> > > Index: pkg/README
> > > ===================================================================
> > > RCS file: pkg/README
> > > diff -N pkg/README
> > > --- /dev/null	1 Jan 1970 00:00:00 -0000
> > > +++ pkg/README	5 Sep 2024 12:52:52 -0000
> > > @@ -0,0 +1,12 @@
> > > ++-------------------------------------------------------------------------------
> > > +| Running ${PKGSTEM} on OpenBSD
> > > ++-------------------------------------------------------------------------------
> > > +
> > > +aerc's Maildir support on BSD uses kqueue(2), which is really intended for
> > > +watching individual files rather than large directories or trees. It requires
> > > +a file handle to be opened for each file in the monitored directories, which
> > > +will soon exceed default limits in the system.
> > > +
> > > +It is therefore recommended to run an IMAP daemon (for example Dovecot,
> > > +courier-imap, imap-uw) and connect over IMAP, even if this is just to the
> > > +local machine.