Alexander Gromnitsky's Blog

How I read newsgroups in mutt

Latest update: 2023-05-01 12:12:00

It took me a long time, but I have finally removed inn+newsstar from my machine. I no longer use patches that add NNTP support to mutt. Yet, I still cannot let go of gmane because it is much more convenient to read mailing lists as newsgroups.

What if I just fetch last N posts from newsgroup foo.a & save them in an mbox file for viewing it in mutt later on? Then I can do the same for newsgroups foo.b, foo.c, & so on.

How do I fetch? Turns out, there is a nice CLI NNTP client already, called sinntp. The following command downloads fresh articles from comp.lang.c into a conspicuously named mbox file comp.lang.c:

$ sinntp pull --server news.eternal-september.org comp.lang.c

(2023/05 update: set PYTHONWARNINGS=ignore::DeprecationWarning env variable before running sinntp, until the maintainer of the app finds a replacement for the nntplib module.)

If you run it again, it won't re-download the same articles again, for it saves reported high water mark in ~/.local/share/sinntp/comp.lang.c@news.eternal-september.org file.

This only solves a problem for 1 news server & 1 newsgroup. I read multitudes of them; should I write a simple shell script then? If you follow this blog, you may have noticed I try not to employ shell scripts but write makefiles instead.

$ cat ~/.config/nntp2mbox/news.eternal-september.org.conf
comp.lang.c

This states that I want to grab articles from comp.lang.c newsgroup from news.eternal-september.org news server.

$ cat ~/.config/nntp2mbox/news.gmane.io.conf
gmane.comp.gnu.make.devel
gmane.comp.gnu.make.general
#gmane.comp.web.chromium.extensions
gmane.comp.window-managers.fvwm

In this example, the server name is news.gmane.io & 1 of the newsgroups is commented out.

There's no more configuration, everything else is done by nntp2mbox makefile:

#!/usr/bin/make -f

# a number of article to pull
limit := 500
g :=

conf := $(or $(XDG_CONFIG_HOME),~/.config)/nntp2mbox
servers := $(wildcard $(conf)/*.conf)
self := $(lastword $(MAKEFILE_LIST))

all: $(servers:%.conf=%.server)

# read a list of newsgroups & run Make for each newsgroup
%.server: %.conf
    awk '!/^#/ {print $$1 ".newsgroup"}' $< | grep $(call se,$(g)) | xargs -r $(make) -Bk server=$(notdir $(basename $<))

%.newsgroup:
    sinntp pull --server $(server) --limit $(limit) $*

make = $(MAKE) --no-print-directory -f $(self)
se = '$(subst ','\'',$1)'

The following command downloads fresh articles from all the newsgroups (from all the news servers above) to the current directory:

$ nntp2mbox
awk '!/^#/ {print $1 ".newsgroup"}' /home/alex/.config/nntp2mbox/localhost.conf | grep '' | xargs -r /usr/bin/make --no-print-directory -f /home/alex/bin/nntp2mbox -Bk server=localhost
awk '!/^#/ {print $1 ".newsgroup"}' /home/alex/.config/nntp2mbox/news.eternal-september.org.conf | grep '' | xargs -r /usr/bin/make --no-print-directory -f /home/alex/bin/nntp2mbox -Bk server=news.eternal-september.org
sinntp pull --server news.eternal-september.org --limit 500 comp.lang.c
awk '!/^#/ {print $1 ".newsgroup"}' /home/alex/.config/nntp2mbox/news.gmane.io.conf | grep '' | xargs -r /usr/bin/make --no-print-directory -f /home/alex/bin/nntp2mbox -Bk server=news.gmane.io
sinntp pull --server news.gmane.io --limit 500 gmane.comp.gnu.make.devel
sinntp pull --server news.gmane.io --limit 500 gmane.comp.gnu.make.general
sinntp pull --server news.gmane.io --limit 500 gmane.comp.window-managers.fvwm

(Yes, it invokes Make recursively, which is a big no-no in many Make circles.)

$ ls
comp.lang.c                gmane.comp.gnu.make.general
gmane.comp.gnu.make.devel  gmane.comp.window-managers.fvwm

It even supports filtering by a newsgroup name:

$ nntp2mbox g=fvwm

I don't actually read comp.lang.c. If there's anything sane left in comp.* hierarchy, please let me know.