From: "Johannes Thyssen Tishman" Subject: Re: [revision] mail/aerc: include pkg/README? To: "Dylan D'Silva" , "Stuart Henderson" Cc: Date: Fri, 06 Sep 2024 00:32:24 +0200 2024-09-05T21:59:43Z Dylan D'Silva: > Hello all, > > Just to chime in here. I wrote that email on the aerc mailing list. > For some context. This specific work around is in regards to using > MAILDIR and notmuch with aerc. You can use notmuch and aerc without > MAILDIR by removing maildir:// and copy-to from accounts.conf but you > can only send from one account without some interesting solutions, If > you have multiple accounts this isn't ideal. The aerc team has decided > this approach is best practice when using aerc, MAILDIR and notmuch. > This work around was to provide a working solution for people who want > to follow the developments team best practice while using a BSD > system. Where and when exactly was this workaround deemed the best practice by the 'aerc team'? > Running without a matching openfile results in "Too many open file > errors" and the application not functioning. > > Aerc uses fsnotify for watching files. Linux backend is inotify which > supports recurive watching. OpenBSD uses kqueue which doesn't. > > This was never meant to be an ideal solution. I also mentioned to > Johannes, that a better approach would be to shrink your inbox and have > a smaller openfile. > > I'm using aerc on a OpenBSD system as a desktop and I'm the sole user. > So I'm not concerned with people opening multiple instances of aerc. I > realise everyone might not be in that boat. > > On Thu Sep 5, 2024 at 11:35 PM AEST, Johannes Thyssen Tishman wrote: > > 2024-09-05T12:55:06Z Stuart Henderson: > > > On 2024/09/05 14:23, Johannes Thyssen Tishman wrote: > > > > 2024-09-05T12:09:51Z Stuart Henderson: > > > > > On 2024/09/05 12:52, Johannes Thyssen Tishman wrote: > > > > > > The email below was sent to the aerc-discuss mailing list a couple of > > > > > > days ago. Is it worth adding a pkg/README for this? > > > > > > > > > > > > 2024-09-03T23:12:23Z "Dylan D'silva" : > > > > > > Hello all, > > > > > > > > > > > > After some digging I've come to a solution. For those on a BSD system > > > > > > with large MAILDIR, you'll require something similar. Aerc uses > > > > > > github.com/fsnotify/fsnotify to monitor changes to files. BSD system > > > > > > rely on Kqueue which doesn't support recursive watching of files. > > > > > ^^ > > > > > directories > > > > > > Therefore you require one file descriptor for each file. > > > > > > > > > > This does not seem to be a good approach for monitoring emails in > > > > > Maildirs on kqueue systems. > > > > > > > > > > Most of the common changes to a Maildir result in either a new file, > > > > > rename, or deletion and not actually changes to the contents of the > > > > > files. AFAIK new files won't show up without rescanning the whole > > > > > directory - for renamed files I don't think you can find the new > > > > > name so again you need to rescan - it seems like really it's just > > > > > going to need an efficient way to pick up changes in the dir. > > > > > > > > > > (I suppose this is not going to work well with NFS either which is > > > > > not an uncommon place to store Maildirs..) > > > > > > > > > > > Assuming you are running aerc from the current account. You can > > > > > > change the limit to match your maildir size. > > > > > > > > > > > > For me, I store my maildir in ~/Mail > > > > > > find . -type f | wc -l #counts number of files > > > > > > 71514 > > > > > > > > > > > > I run aerc from a staff account. Therefore I updated my /etc/login.conf > > > > > > under staff to increase the openfile-cur size to 71600 and openfile-max > > > > > > to 72000 > > > > > > :openfiles-cur=71600:\ > > > > > > :openfiles-max=72000:\ > > > > > > > > > > If you're going to do this, I would recommend a "normal" openfiles-cur, > > > > > just raise openfiles-max. Then start such software from a script or > > > > > shell alias which uses ulimit -n to raise the limit just for that > > > > > software. > > What do you consider normal? In your comment below, you made comment > about not many people running computers from the 80's. I think Stuart meant that you could leave openfiles-cur with it's default value (in case you hadn't changed it before for something else) and only increase openfiles-max to your desired value. Then you can start aerc from a script like the following: #!/bin/sh ulimit -n 72000 aerc > Here is a commit for inotify in 2020 making the openfile limit dynamic > between 8192 and 104576. > > From the commit: > "The default value of inotify.max_user_watches sysctl parameter was set > to 8192 since the introduction of the inotify feature in 2005 by > commit 0eeca28 ("[PATCH] inotify"). Today this value is just too > small for many modern usage. As a result, users have to explicitly set > it to a larger value to make it work." > > https://github.com/torvalds/linux/commit/92890123749bafc317bbfacbe0a62ce08d78efb7 > > > It's worth mentioning but the syncthing pkg-readme suggest a openfile of > 4096. > > > > > > > > > > > I think it would generally be better to run aerc pointed at an IMAP > > > > > server though (obviously the IMAP server can just run on the machine > > > > > running aerc if wanted). > > I agree this is a better solution. This was only meant as a work around > to follow what the dev team are using. > > > > > > > > > > > If you use a large login.conf you might want create a database version > > > > > > with cap_mkdb /etc/login.conf see login.conf(5). > > > > > > > > > > We are mostly not running on computer systems from the 80s and the file > > > > > is not really slow to parse - however having the db files around does > > > > > cause trouble when someone forgets to rerun cap_mkdb after changing the > > > > > file later. I would not recommend this. > > The man page for login(5).conf has that comment. > > > > > > > > > Thanks for the feedback Stuart. Should this advice be included in a > > > > pkg/README as it is done for syncthing? > > > > > > It would be better if the situation was improved upstream, but we > > > could do... > > > > I'll check with upstream. > > > > > Rather than documenting the workaround I think it's probably better if > > > we go for this approach. Setting the OS to allow opening of tens of > > > thousands of files is very much not ideal, and imagine what it would > > > be like if you have a couple of users on the machine, all of whom want > > > to run aerc... > > How about improving Kqueue to support recursive watching? > > > I agree, this seems more sane. Thank you Stuart :) > > > > > Index: Makefile > > > =================================================================== > > > RCS file: /cvs/ports/mail/aerc/Makefile,v > > > diff -u -p -r1.16 Makefile > > > --- Makefile 31 Jul 2024 18:36:02 -0000 1.16 > > > +++ Makefile 5 Sep 2024 12:52:52 -0000 > > > @@ -7,6 +7,7 @@ MODGO_MODNAME = git.sr.ht/~rjarry/aerc > > > MODGO_VERSION = v0.0.0-20240729220005-e037c095a049 > > > > > > V = 0.18.2 > > > +REVISION = 0 > > > DISTNAME = aerc-$V > > > > > > CATEGORIES = mail > > > Index: pkg/PLIST > > > =================================================================== > > > RCS file: /cvs/ports/mail/aerc/pkg/PLIST,v > > > diff -u -p -r1.9 PLIST > > > --- pkg/PLIST 31 Jul 2024 18:36:02 -0000 1.9 > > > +++ pkg/PLIST 5 Sep 2024 12:52:52 -0000 > > > @@ -43,4 +43,5 @@ share/aerc/templates/forward_as_body > > > share/aerc/templates/new_message > > > share/aerc/templates/quoted_reply > > > share/applications/aerc.desktop > > > +share/doc/pkg-readmes/${PKGSTEM} > > > @tag update-desktop-database > > > Index: pkg/README > > > =================================================================== > > > RCS file: pkg/README > > > diff -N pkg/README > > > --- /dev/null 1 Jan 1970 00:00:00 -0000 > > > +++ pkg/README 5 Sep 2024 12:52:52 -0000 > > > @@ -0,0 +1,12 @@ > > > ++------------------------------------------------------------------------------- > > > +| Running ${PKGSTEM} on OpenBSD > > > ++------------------------------------------------------------------------------- > > > + > > > +aerc's Maildir support on BSD uses kqueue(2), which is really intended for > > > +watching individual files rather than large directories or trees. It requires > > > +a file handle to be opened for each file in the monitored directories, which > > > +will soon exceed default limits in the system. > > > + > > > +It is therefore recommended to run an IMAP daemon (for example Dovecot, > > > +courier-imap, imap-uw) and connect over IMAP, even if this is just to the > > > +local machine.