Subject: newsbot - instantly grep and trigger actions on news articles
From: [email-masked] (Pierre Beyssac)
Date: 17 Sep 1996 23:19:40 +0200
Newsgroups: alt.sources

Archive-name: newsbot-1.2.1 
Submitted-by: [email-masked]

Newsbot reads article headers fed to it by INN, checks a series of
conditions on these headers or on article bodies, and can trigger
a series of actions if the conditions are met.

# This is a shell archive.  Save it in a file, remove anything before
# this line, and then unpack it by entering "sh file".  Note, it may
# create directories; files and directories will be owned by you and
# have default permissions.
#
# This archive contains:
#
#	ChangeLog
#	Makefile
#	README
#	newsbot.c
#	sample.config
#	sample.config/bin.cancel-global
#	sample.config/bin.cancel-local
#	sample.config/bin.mcc-sender-fr
#	sample.config/bin.mch-frbin
#	sample.config/bin.mh-newsmaster
#	sample.config/bin.mh-sender-fr
#	sample.config/bin.p-expl-frud
#	sample.config/binkill.conf
#	sample.config/dup.cancel-local
#	sample.config/dup.mh-sender
#	sample.config/from.cancel-local
#	sample.config/html.mh-sender-fr
#	sample.config/log.article
#	sample.config/log.headers
#	sample.config/newsbot.conf
#	sample.config/qp.mh-sender-fr
#	sample.config/spew.cancel-global
#	sample.config/spew.mh-newsmaster
#
echo x - ChangeLog
sed 's/^X//' >ChangeLog << 'END-of-ChangeLog'
X$Id: ChangeLog,v 1.8 1996/09/17 20:35:36 pb Exp $
X
XTue Sep 17 22:34:22 MET DST 1996 *** 1.2.1 released
X	- misc changes in README and config files
X	- ported to HP-UX 9.x by Pierre David <[email-masked]>
X
XMon Sep 16 23:59:02 MET DST 1996 *** 1.2 released
X	- improved README
X	- new minimal config file ready for binary detection
X	- new format for pattern declaration in config file: 'P'
X	- bug fix of hash function on 8 bit chars...
X
XSat Sep 14 17:00:50 MET DST 1996
X	- new keyword %newsbot-version% to include ident string.
X
XSat Sep 14 11:44:40 MET DST 1996 *** 1.1.1 released.
X	- fix to body pattern matching
X
XFri Sep 13 19:50:13 MET DST 1996 *** 1.1 released.
X
X	- new option "L" to match patterns in body.
X	- ported to SunOS 4.
X	- misc cleanups.
X
XThu Sep 12 01:20:33 MET DST 1996 *** 1.0.2 released.
END-of-ChangeLog
echo x - Makefile
sed 's/^X//' >Makefile << 'END-of-Makefile'
X# $Id: Makefile,v 1.8 1996/09/17 20:34:02 pb Exp $
X#
X# Makefile	Makefile for newsbot
X#		Targets:   all      compiles everything
X#		           install  installs the binaries
X#				    (not the example configuration files)
X#                          clean    cleans up
X#
X
X#
X# Edit the following paths to suit your configuration
X#
X
X# Your news spool
XPATHSPOOL	= /var/spool/news
X# Where you want to put newsbot config file
XPATHCONF	= /usr/local/news/newsbot.conf
X# Where you want to put newsbot pattern files
XPATHPATS	= /usr/local/news/newsbot
X# Facility for syslog.
XLOGFAC		= LOG_LOCAL6
X
X# The following are only needed for "make install"
X#
X# Where you want to install newsbot
XBIN_DIR		= /usr/local/news/bin
X# Where is ctlinnd
XCTLINND_DIR	= /usr/local/news/bin
X
X#
X# Comment-out the lines for your system
X#
X
X# SunOS 4.x
X# you need to install a Posix Regexp package (GNU rx for example)
X#
X#LDFLAGS	= -L/usr/local/lib -lregex
X#INCL	= -I/usr/local/include
X#CC	= gcc
X#CFLAGS	= -Wall -g
X
X# FreeBSD, NetBSD, Linux
X#
X#LDFLAGS	=
X#INCL	=
X#CC	= gcc
X#CFLAGS	= -Wall -g
X
X# HP-UX 9.x
X#
X#LDFLAGS	=
X#INCL	=
X#CC      = cc -Ae
X#CFLAGS  = -g
X
XPROGS	= newsbot
X
XBIN_OWNER	= news
XBIN_GROUP	= news
XBIN_COMBO	= $(BIN_OWNER).$(BIN_GROUP)
XINSTALL		= install -o $(BIN_OWNER) -g $(BIN_GROUP)
X
XDOPTS		=	-D_PATH_SPOOL=\"$(PATHSPOOL)\"	\
X			-D_PATH_CONF=\"$(PATHCONF)\"	\
X			-D_PATH_DIRPATS=\"$(PATHPATS)\"	\
X			-DLOG_FACILITY=$(LOGFAC)
X
Xall:		$(PROGS)
X
Xnewsbot:        newsbot.c
X		$(CC) $(INCL) $(DOPTS) $(CFLAGS) -o $@ newsbot.c $(LDFLAGS)
X
Xcleanobjs:
X		rm -f *.o *.bak
X
Xclean:		cleanobjs
X		@echo Type \"make clobber\" to really clean up.
X
Xclobber:	cleanobjs
X		rm -f $(PROGS)
X
Xinstall:	$(PROGS)
X		-mv -f $(BIN_DIR)/newsbot $(BIN_DIR)/newsbot.old
X		$(INSTALL) -m 755 newsbot $(BIN_DIR)
X		@echo "newsbot installed.....Don't forget to restart it, as user $(BIN_OWNER):"
X		@echo 
X		@echo "$(CTLINND_DIR)/ctlinnd begin 'NEWSBOT!'"
X		@echo
END-of-Makefile
echo x - README
sed 's/^X//' >README << 'END-of-README'
X$Id: README,v 1.9 1996/09/17 20:34:03 pb Exp $
X
X1. What is newsbot ?
X
XNewsbot reads article headers fed to it by INN, checks a series of
Xconditions on these headers or on article bodies, and can trigger
Xa series of actions if the conditions are met.
X
XThe latest newsbot release will generally be available on:
X
X	ftp://frmug.org/newsbot/
X	http://frmug.org/newsbot/
X
X2. Compiling and installing newsbot
X
XNewsbot has been running both on FreeBSD and Linux systems for months.
XIt compiles just fine on:
X
X	FreeBSD
X	NetBSD
X	SunOS 4.x
X	Linux
X	HP-UX 9.x
X
XYou might need to install a Posix regexp library (GNU rx for example)
Xif you don't have one on your system.
X
XTo compile newsbot, there is now a Makefile, thanks to Christian
XPerrier (I was too lazy to write one...). Just edit it to configure
Xyour system-dependent paths and options, then:
X
X	make
X
XTo install and configure newsbot, have a look at the sample config
Xfiles and the following explanations.
X
XUpdating the binary can be done with a:
X
X	make install
X
X3. Configuring newsbot
X
XYou can skip this and go directly to section 4 if you only want to
Xdo binary detection and cancelling.
X
XNewsbot reads its configuration at startup, then reads article
Xheaders (and possibly funnel feed indications) fed to it by INN,
Xchecks a series of conditions on these headers or on article bodies,
Xand can then trigger a series of actions if some conditions are
Xmet.
X
XAll of the configuration goes into the newsbot.conf file.
X
XAdditionnal templates can be placed either in the pattern directory
X(_PATH_DIRPATS) or in newsbot.conf itself.
X
XA condition can be a pattern matching on a header or on a body
Xline, a encoded binary article body, a size, an invalid From:
Xaddress, etc... or a conjunction or these:
X
X	>size	bigger than size
X		size is an integer optionally followed by multiplier
X		k (1024) of M (1024^2).
X
X	B	article is encoded binary
X
X	D	duplicate message
X		Same contents and From:/Newsgroups:/Subject:
X		as a previous article in the last 1000 received.
X
X	F	From: ok
X		At least one @ with at least one char before and
X		at least one char then one dot after
X
X	R	references ok
X		Subject: header does not begin in Re:, or does
X		begin in Re: and a References: header is present
X
X	Msize	estimated multipart size bigger than 'size'
X	M	multipart
X		This condition looks for two numbers in the subject,
X		and tries to determine if it is a multipart article,
X		and its global size.
X
X	~name	named expression matches
X		Pattern matching on headers.
X		"name" must be the name of a previously declared
X		header pattern, of the form:
X
X		~[ei]	name	header-name	regexp-or-substring
X
X		the [ei] part is optional.
X
X		Option e means that the last argument on the line
X		is a complete regular expression, else only ^ (beg.
X		of line) and $ (end of line) are recognized to
X		anchor the search.
X
X		Option i means that the match must be case-insensitive.
X
X		Headers on which pattern matching must be done have
X		to be previously declared with 'H', for example:
X
X		HContent-Type
X		HContent-Transfer-Encoding
X
X	Lname	named expression matches
X		Pattern matching on article body.
X		"name" must be the name of a previously declared
X		body pattern, of the form:
X
X		L[ei]	name	linerange	regexp-or-substring
X
X		The meaning is exactly the same as above, except
X		that the linerange indicates where to look for the
X		pattern in the article body, and can only be a
X		single integer (for a unique line) or two integers
X		separated by a '-' (for that range, including the
X		boundaries).
X
XAn action consists in reading a template, substituting % variables,
Xand piping the output to a program or another file. An action is
Xdeclared as follows:
X
XAname		template-name		pipe-to
X
Xtemplate-name can be the name of a previously declared template,
Xor a relative file name (it will only be found in the pattern
Xdirectory _PATH_DIRPATS).
X
XIn a template, the recognized expansions are:
X
X	%article-headers%		article headers
X	%article%			full article
X	%article-encoding%		binary encoding name.
X	%article-size%			article size in bytes
X	%article-sender%		sender (Reply-To: if found, else From:)
X	%moderated-sender%		as above, but use Approved: if present.
X	%article-sender-host%		as article-sender, cut after the @.
X	%header-XXX%			any header (builtin or added with 'H')
X	%date%				current date (GMT, RFC 822/1036 format)
X	%re-subject%			subject prepended with Re: if necessary
X	%cancel-message-id%		cancel message-id (Salz convention)
X	%truncated-message-id%		message-id without the leading '<'
X	%<filename%			include file (no expansions)
X	%newsbot-version%		newsbot version ident
X	%%				put a %
X
XPreprocessor conditionals (can be nested)
X	%if-header-XXX%			true if header XXX present and
X					not empty.
X	%else%, %endif%			self-documenting :-)
X
X
XTo link conditions and actions, use the 'I' declaration:
X
XIc1,c2,...cn	action
X
Xc1-cn are valid conditions; the action will be executed if the
Xconditions all match. To use the negation of a condition, prepend
Xit with '!'.
X
XTo take several actions on a given condition, use continuation lines
Xbeginning in ',':
X
X,		next-action
X
XIt is possible to test for additional conditions on continuation
Xlines, for example:
X
XIc1		action1
X,		action2
X,c2		action3
X
Xmeans that, if c1 is true, action1 and action2 will be taken.
XIf (c1 and c2) is true, action3 will be taken too.
X
XIn addition to the actions you declare, there are 3 builtin actions:
X'nop' (does nothing), 'stop' (stops evaluation for this article,
Xinstead of continuing on the next line) and 'remove' (directly
Xunlink the current article from the spool).
X
XNewsbot does lazy evaluation and computes "expensive" conditions
X(B for binary testing, for example, or L for grepping bodies) only
Xonce. In addition to that, it tries to compute expensive conditions
Xas late as possible when several conditions are in a conjunction
X(on the same line). Try to avoid constructs like:
X
XIB		action1
XI>20k		action2
X
Xwhich would test _every_ article for a binary encoding !
X
XFinally, the feed separator 'F' allows you to specify different
Xactions according to different feeds. To use it, you need to start
Xnewsbot with option -f and configure INN with a funnel feed to
Xnewsbot.
X
XFor example you can declare (in your newsfeeds file) feeds 'frcheck!'
Xand 'all!' both going to funnel 'NEWSBOT!', and in your newbot.conf:
X
XFfrcheck!
XI>100k		action1
X
XFall!
XI>200k		action2
X
X4. Simple config for binary detection
X
XYou just need to take file binkill.conf, edit it CAREFULLY to taste,
Xthen copy it as /usr/local/news/newsbot.conf (or any other place
Xyou specify in the Makefile).
X
X5. Starting newsbot
X
XInstall newsbot with a "make install".
X
XEdit your newsfeeds file to add something like:
X
X# newsbot
X#
Xfrcheck!:fr.*:Tm:NEWSBOT!
X#	 ^^^^ adapt to taste
X#
XNEWSBOT!:!*:Tc,W*H:\
X	/usr/local/news/bin/newsbot -f
X#	^^^^^^^^^^^^^^^^^^^ adapt to your setup
X
XNOTE: 'frcheck!' depends on the name you have used in your config.
XIt's the default name for the binary detection config provided in
Xbinkill.conf.
X
XThen start the feeds:
X	$ ctlinnd begin NEWSBOT\!
X	$ ctlinnd begin frcheck\!
X
XHave a look at syslog output for the facility you have configured
Xin (LOCAL6 by default) to check if newsbot is happy (it should
Xissue a start message).
X
XLet newsbot run for a moment. It will only log, not do, things.
XIf everything seems correct, add option '-a' before '-f' in the
XNEWSBOT! entry in the newsfeeds file, then restart newsbot for real
X(ctlinnd begin NEWSBOT\!).
X
XCheck (at least) twice before using option -a, or you might generate
Xa lot of mess and make a lot of people VERY, VERY angry. Then you
Xwould bitterly regret it (I'm not kidding).
X
X5. That's all folks
X
XGood luck, and please use newsbot wisely !
X
X		[email-masked] (Pierre Beyssac)
X
XThanks to:
X
X	Christian Perrier <[email-masked]>
X			for invaluable comments, testing, the Makefile,
X			a "netsend"-encoded sample...
X
X	Vincent Archer	<[email-masked]>
X			for the original canceler
X
X	Ollivier Robert	<[email-masked]>
X			for the RFC-checker
X
X	Thomas Quinot	<[email-masked]>
X			for the initial idea of body pattern matching
X			for spew detection
X
X	Pierre David	<[email-masked]>
X			HP-UX port and first major newsbot site
END-of-README
echo x - newsbot.c
sed 's/^X//' >newsbot.c << 'END-of-newsbot.c'
X/*
X * $Id: newsbot.c,v 1.15 1996/09/17 20:34:05 pb Exp $
X *
X * newsbot.c
X *
X * Author : Pierre Beyssac <[email-masked]>
X *
X * Tests a list of conditions on incoming articles.
X * Pipes a pattern file to defined programs on matches.
X * Can remove the article from local spool.
X *
X * The binary detector currently recognizes:
X *	uuencode, base64, binhex and netsend.
X *
X * Hack on rfc-checker by Ollivier Robert <[email-masked]>,
X * itself derivated from cyberspam by Vincent Archer <[email-masked]>
X *
X * ----
X * Use the following syntax in the newsfeeds(5) file.
X * DON'T USE THE -a OPTION UNLESS YOU ARE QUITE SURE YOUR CONFIG IS CORRECT !
X * Without -a, newsbot only logs the actions it would have done.
X *
X * - with a normal feed:
X *	NEWSBOT!:fr.*:Tc,WH: \
X *		/usr/local/news/bin/newsbot [-a]	\
X *		[-d] [-C config-file]			\
X *		[-D pattern-dir]
X *
X * - with a funnel feed (note use of option -f) :
X *	NEWSBOT!:!*:Tc,W*H: \
X *		/usr/local/news/bin/newsbot [-a] -f	\
X *		[-d] [-C config-file]			\
X *		[-D pattern-dir]
X *
X * ----
X * Example for a normal feed :
X *
X * NEWSBOT!:fr.*:Tc,WH: \
X *	/usr/local/news/bin/newsbot -a
X *
X * ----
X * Example for a funnel feed :
X *
X * rfc-check!:*,!comp.mail.sendmail:Tm:NEWSBOT!
X *
X * from-check!:*:Tm:NEWSBOT!
X *
X * bin-check!:fr.*:Tm:NEWSBOT!
X *
X * NEWSBOT!:!*:Tc,W*H: \
X *	/usr/local/news/bin/newsbot -a -f
X *
X * ----
X * In a pattern file, the recognized expansions are :
X *	%article-headers%		article headers
X *	%article%			full article
X *	%article-encoding%		binary encoding name.
X *	%article-size%			article size in bytes
X *	%article-sender%		sender (Reply-To: if found, else From:)
X *	%moderated-sender%		as above, but use Approved: if present.
X *	%article-sender-host%		as article-sender, cut after the @.
X *	%header-XXX%			any header (builtin or added with 'H')
X *	%date%				current date (GMT, RFC 822/1036 format)
X *	%re-subject%			subject prepended with Re: if necessary
X *	%cancel-message-id%		cancel message-id (Salz convention)
X *	%truncated-message-id%		message-id without the leading '<'
X *	%<filename%			include file (no expansions)
X *	%newsbot-version%		newsbot version ident
X *	%%				put a %
X *
X * Preprocessor conditionals (can be nested)
X *	%if-header-XXX%			true if header XXX present and
X *					not empty.
X *	%else%, %endif%			self-documenting :-)
X *
X * Syslog facility is by default LOG_LOCAL6
X */
X
X#ifndef lint
Xstatic char *const rcsid = "$Id: newsbot.c,v 1.15 1996/09/17 20:34:05 pb Exp $";
X#endif /* lint */
X
Xstatic char *const version = "newsbot 1.2";
X
X#ifndef _PATH_SPOOL
X# define _PATH_SPOOL "/var/spool/news"
X#endif /* _PATH_SPOOL */
X
X#ifndef _PATH_CONF
X# define _PATH_CONF "/usr/local/news/newsbot.conf"
X#endif /* _PATH_CONF */
X
X#ifndef _PATH_DIRPATS
X# define _PATH_DIRPATS "/usr/local/news/newsbot"
X#endif /* _PATH_DIRPATS */
X
X#ifndef LOG_FACILITY
X# define LOG_FACILITY   LOG_LOCAL6
X#endif /* LOG_FACILITY */
X
X#include <unistd.h>
X#include <stdlib.h>
X#include <sys/types.h>
X#include <sys/param.h>
X#include <stdio.h>
X#include <ctype.h>
X#include <time.h>
X#if !defined(__NetBSD__) && !defined(__FreeBSD__)
X# include <malloc.h>
X#endif /* 4.4BSD */
X#include <string.h>
X#include <syslog.h>
X#include <regex.h>
X
X#ifdef sun
Xint getopt();
Xextern char *optarg;
Xextern int optind, opterr;
X#endif /* SUNOS */
X
X#ifndef MAXPATHNAMELEN
X# define MAXPATHNAMELEN     1024
X#endif
X
X#define DEBUG
X
X#ifdef DEBUG
X#include <assert.h>
X# define MESSAGE(s)      if (opt_debug) syslog (LOG_NOTICE, (s))
X# define MESSAGE_1(s,a)  if (opt_debug) syslog (LOG_NOTICE, (s), (a))
X# define MESSAGE_2(s,a,b)  if (opt_debug) syslog (LOG_NOTICE,(s),(a),(b))
X# define MESSAGE_3(s,a,b,c)  if (opt_debug) syslog (LOG_NOTICE,(s),(a),(b),(c))
X# define MESSAGE_4(s,a,b,c,d)  if (opt_debug) syslog (LOG_NOTICE,(s),(a),(b),(c),(d))
X# define MESSAGE_5(s,a,b,c,d,e)  if (opt_debug) syslog (LOG_NOTICE,(s),(a),(b),(c),(d),(e))
X#else
X# define MESSAGE(s)
X# define MESSAGE_1(s,a)
X# define MESSAGE_2(s,a,b)
X# define MESSAGE_3(s,a,b,c)
X# define MESSAGE_4(s,a,b,c,d)
X# define MESSAGE_5(s,a,b,c,d,e)
X#endif /* DEBUG */
X
X#define MINLINE 80
X
X/* For article hash table */
X#define HASHMSGSIZE	1337
Xtypedef unsigned short hashmsg_t;
X
X/* For header hash table */
X#define HASHHDRSIZE	31
Xtypedef unsigned short hashhdr_t;
X
X/* Hash function for headers (case-insensitive) */
X#define HASHFUN(s) ((unsigned)(((s)[0]&&(s)[1]&&(s)[2])		\
X			? (((s)[0]^(s)[1]^(s)[2]) & ~0x20)	\
X			: ((s)[0] & ~0x20))			\
X			% HASHHDRSIZE)
X
X/* For article buckets */
X#define MAXARTS		1000
X
X/* For predefined header table */
X#define	MAXHEADERS	50
X
X/* For matching patterns (should fit 1/bit in a long) */
X#define	MAXMATCH	32
Xtypedef unsigned long matchflags_t;
X
X/* How many nested if/else/endif */
X#define	MAXIFNEST	32
X
X/*
X * The first in the list will get index 0, the next 1, and so on.
X * Both tables should be kept in sync.
X */
Xchar *header_init_table[] = {
X	"Bytes",
X	"Control",
X	"Xref",
X	"From",
X	"Subject",
X	"Newsgroups",
X	"Message-Id",
X	"Date",
X	"References",
X	"Sender",
X	"Reply-To",
X	"Approved",
X	NULL
X};
X
X#define	I_BYTES		0
X#define	I_CONTROL	1
X#define	I_XREF		2
X#define	I_FROM		3
X#define	I_SUBJECT	4
X#define	I_NEWSGROUPS	5
X#define	I_MESSAGE_ID	6
X#define	I_DATE		7
X#define	I_REFERENCES	8
X#define	I_SENDER	9
X#define	I_REPLY_TO	10
X#define	I_APPROVED	11
X
X/* Special flags for regex matching */
X#define M_CASE			1
X#define	M_BEGLINE		2
X#define	M_ENDLINE		4
X#define M_REGEX			8
X
Xstruct article_t
X{
X    long bodysize;	/* content size */
X    long headersize;	/* header size */
X
X    /* Checks */
X    char *coding;
X    char refok;
X
X    char fromok;
X    char multi;
X    char binary;
X    char duplicate;
X    matchflags_t hmatch_done, hmatch_val;
X    matchflags_t bmatch_done, bmatch_val;
X
X    long parts;
X
X    FILE *artfile;
X    char artfiletried;
X
X    hashmsg_t hash;
X    char *hashfields;
X
X    long firstbyte;
X
X    char *h[MAXHEADERS];
X    char control;
X
X    char hbuf[16384];
X};
X
X#define	T_TRUE		1
X#define	T_FALSE		0
X#define	T_DONTCARE	-1
X
Xstruct pattern_t {
X    struct pattern_t *next;
X    char *name;			/* pattern name */
X    char *val;			/* pattern value.
X				 * If NULL, use the file indicated
X				 * by the name above.
X				 */
X    int valsize;		/* Currently allocated size for val */
X    char *curpos;		/* If not a file, keep current position */
X    FILE *file;			/* If a file, use this */
X};
X
Xstruct action_t {
X    struct action_t *next;
X    char *name;			/* action name */
X    struct pattern_t *pattern;	/* associated pattern */
X    char *pipeto;		/* what program to pipe to */
X};
X
Xstruct cond_t {
X    struct cond_t *next;	/* next condition to check */
X
X    /* Comparison conditions */
X    long size;			/* min size */
X    long multisize;		/* multi min size */
X    char checksize;		/* check size */
X    char checkmultisize;	/* check multi and size */
X
X    /* Boolean conditions */
X    char binary;		/* check if binary */
X    char fromok;		/* check if correct from */
X    char refok;			/* check if correct refs */
X    char duplicate;		/* check if duplicate post */
X
X    /* Matches on headers */
X    matchflags_t hmatch_totest, hmatch_val;
X
X    /* Matches on body */
X    matchflags_t bmatch_totest, bmatch_val;
X
X    struct action_t *action;	/* action to take */
X    struct cond_t *and;		/* more conditions to check after action */
X};
X
X/* Description of a pattern matching */
Xstruct match_t {
X    char *name;			/* name */
X
X    char flags;			/* option flags */
X    char *expr;			/* expression to test */
X    regex_t re;			/* reg. expr (if option M_REGEX) */
X};
X
X/* Description of a pattern matching table */
Xstruct matchtable_t {
X    int free;			/* First free */
X    int max;			/* Max number */
X    struct match_t *array;	/* Array */
X};
X
X/* Linked list of actions to apply to a named feed */
Xstruct feed_t {
X    struct feed_t *next;
X    char *name;			/* feed name (from 'newsfeeds') */
X    struct cond_t *firstcond;	/* head of condition list */
X    struct cond_t *lastcond;	/* tail (used only when reading config file) */
X};
X
X/* Article buckets are used to save recent articles and detect duplicates */
Xstruct art_bucket_t {
X    struct art_bucket_t *next;	/* next article with same hash value */
X    hashmsg_t hash;		/* hash value */
X    long bodysize;		/* contents size */
X    char *fields;		/* file path and fields */
X};
X
X/* Entry in article hash table */
Xstruct art_list_t {
X    struct art_bucket_t *first, *last;
X};
X
X/* Header description */
Xstruct header_desc_t {
X    struct header_desc_t *next;
X    int index;			/* index to table */
X    const char *name;		/* name */
X};
X
X/* Range description */
Xstruct range_t {
X    struct range_t *next;
X    int first, last;		/* Number of first and last line */
X};
X
Xint firstart, freeart;
Xstruct art_bucket_t artb[MAXARTS];
X
Xint freehdr = 0;
Xstruct header_desc_t hdrb[MAXHEADERS];
X
X/* Header match table */
Xstruct match_t hmatch[MAXMATCH];
Xstruct matchtable_t hmt = { 0, MAXMATCH, hmatch };
Xint hmatch_index[MAXMATCH];	/* for each header match, index of header
X				   to test in header array */
X
X/* Body match table */
Xstruct match_t bmatch[MAXMATCH];
Xstruct matchtable_t bmt = { 0, MAXMATCH, bmatch };
Xint bmatch_beg[MAXMATCH];	/* for each body match, line range to test */
Xint bmatch_end[MAXMATCH];
X
Xstruct header_desc_t *hdrtable[HASHHDRSIZE];
Xstruct art_list_t msgtable[HASHMSGSIZE];
X
Xstatic char opt_act = 0;	/* -a : active mode enabled */
Xstatic char opt_debug = 0;
Xstatic char opt_feed = 0;	/* -f : enable multifeed */
X
Xstatic const char *path_conf = _PATH_CONF;
Xstatic const char *path_pats = _PATH_DIRPATS;
X
X/* Feed list */
Xstatic struct feed_t *firstfeed = NULL;
X
X/* Entry for default feed (not in feed list; gets all articles) */
Xstatic struct feed_t nullfeed = { NULL, "default", NULL };
X
X/* Current feed name during article processing */
Xstatic char *curfeedname = NULL;
X
X/* Builtin initial action list */
Xstruct action_t actnop =	{ NULL,		"nop" };
Xstruct action_t actremove =	{ &actnop,	"remove" };
Xstruct action_t actstop =	{ &actremove,	"stop" };
X
Xstatic struct action_t *firstact = &actstop;
X
X/* Pattern list */
Xstatic struct pattern_t *firstpat = NULL;
X
Xvoid *zmalloc(size_t size)
X{
X    void *p = malloc(size);
X    if (p == NULL) {
X	syslog(LOG_NOTICE, "could not malloc %d bytes", size);
X	exit(1);
X    }
X    return p;
X}
X
Xvoid *zrealloc(void *p, size_t size)
X{
X    p = realloc(p, size);
X    if (p == NULL) {
X	syslog(LOG_NOTICE, "could not realloc %d bytes", size);
X	exit(1);
X    }
X    return p;
X}
X
Xvoid
Xartb_init()
X{
X    firstart = 0;
X    freeart = MAXARTS;
X}
X
Xint
Xheader_insert(const char *h)
X{
X    unsigned short hv;
X    struct header_desc_t *hd, *ht;
X
X    hv = HASHFUN(h);
X
X    for (hd = hdrtable[hv]; hd; hd = hd->next)
X	if (!strcasecmp(hd->name, h))
X	    return hd->index;
X
X    if (freehdr == MAXHEADERS) {
X	syslog(LOG_NOTICE, "header table too small (%d entries)", MAXHEADERS);
X	exit(1);
X    }
X
X    ht = hdrb + freehdr;
X    ht->index = freehdr;
X    ht->name = h;
X
X    ht->next = hdrtable[hv];
X    hdrtable[hv] = ht;
X
X    return freehdr++;
X}
X
Xint
Xheader_to_index(const char *h)
X{
X    unsigned short hv;
X    struct header_desc_t *hd;
X
X    hv = HASHFUN(h);
X
X    for (hd = hdrtable[hv]; hd; hd = hd->next) {
X	if (!strcasecmp(hd->name, h))
X	    return hd->index;
X    }
X    return -1;
X}
X
Xvoid
Xheader_init()
X{
X    char **p = header_init_table;
X    while (*p) header_insert(*p++);
X}
X
X/*
X * Find a pattern by name
X * Return pointer to it, allocate a new if not found.
X */
X
Xstruct pattern_t *
Xpattern_find(char *name)
X{
X    char *pfname;
X    struct pattern_t *pf;
X    for (pf = firstpat; pf; pf = pf->next) {
X	if (!strcasecmp(name, pf->name)) {
X	    return pf;
X	}
X    }
X    pf = zmalloc(sizeof(struct pattern_t));
X
X    pfname = zmalloc(strlen(name)+1);
X    strcpy(pfname, name);
X
X    pf->name = pfname;
X    pf->val = NULL;
X    pf->file = NULL;
X    pf->next = firstpat;
X
X    firstpat = pf;
X    return pf;
X}
X
X/*
X * Open a pattern.
X * Return 0 if OK, else -1.
X */
X
Xint
Xpattern_open(struct pattern_t *p)
X{
X    if (p->val == NULL) {
X	/* It's a file... */
X	if (p->file == NULL) {
X	    char buff[8192];
X	    strcpy(buff, path_pats);
X	    strcat(buff, "/");
X	    strcat(buff, p->name);
X
X	    p->file = fopen(buff, "r");
X	    return p->file ? 0:-1;
X	} else {
X	    /* Already open: rewind */
X	    rewind(p->file);
X	}
X    } else {
X	/* It's in memory */
X	p->curpos = p->val;
X    }
X    return 0;
X}
X
Xvoid
Xpattern_close(struct pattern_t *p)
X{
X    if (p->val == NULL) {
X	fclose(p->file);
X	p->file = NULL;
X    }
X}
X
X/* Append a string to a in-memory pattern */
X
Xvoid
Xpattern_append(struct pattern_t *p, char *s)
X{
X    int size = strlen(s);
X    int usedsize;
X
X    if (p->val == NULL) {
X	p->valsize = size + 1;
X	p->val = zmalloc(size+1);
X	strcpy(p->val, s);
X	return;
X    }
X    usedsize = strlen(p->val)+1;
X    if (usedsize+size > p->valsize) {
X	p->valsize = usedsize + size;
X	p->val = zrealloc(p->val, p->valsize);
X    }
X    strcpy(p->val + usedsize - 1, s);
X}
X
X/*
X * Read up to buffsize bytes in pattern.
X *
X * Returns a NULL-ended string in buff and 0 if OK, or -1 if error.
X */
X
Xint
Xpattern_read(struct pattern_t *p, char *buff, int buffsize)
X{
X    if (p->val == NULL) {
X	/* It's a file */
X	return fgets(buff, buffsize, p->file) ? 0:-1;
X    } else {
X	/* It's there */
X	int rlen;
X	char *cp;
X
X	if (p->curpos == NULL)
X	    return -1;		/* Exhausted */
X
X	rlen = strlen(p->curpos) + 1;
X	if (rlen <= buffsize) {
X	    /* Enough room for everything */
X	    strcpy(buff, p->curpos);
X	    p->curpos = NULL;
X	} else {
X	    /* Try to copy a full line */
X	    buffsize--;		/* Reserve for trailing '\0' */
X	    buffsize--;		/* and a '\n' */
X	    for (cp = p->curpos; buffsize && *cp && *cp != '\n';) {
X		*buff++ = *cp++;
X		buffsize--;
X	    }
X	    if (*cp == '\n') {
X		*buff++ = *cp++;
X	    }
X	    *buff = '\0';
X	    /* Prepare for next call */
X	    if (*cp)
X		p->curpos = cp;		/* Next line */
X	    else
X		p->curpos = NULL;	/* End */
X	}
X	return 0;
X    }
X}
X
Xvoid
Xmatch_init(struct match_t *pm, char *matchname, char matchopt, char *expr)
X{
X    int len = strlen(matchname) + 1;
X    char *s;
X
X    if (matchopt & M_REGEX) {
X	char errbuf[80];
X	int rerr;
X	rerr = regcomp(&pm->re, expr, REG_EXTENDED|REG_NOSUB
X			|((matchopt & M_CASE) ? 0:REG_ICASE));
X	if (rerr < 0) {
X	    regerror(rerr, NULL, errbuf, sizeof errbuf);
X	    syslog(LOG_NOTICE, "%s: %s", expr, errbuf);
X	    exit(1);
X	}
X	s = (char *)zmalloc(len);
X	strcpy(s, matchname);
X	pm->name = s;
X	pm->expr = NULL;
X	pm->flags = M_REGEX;
X    } else {
X	len += strlen(expr) + 1;
X	s = (char *)zmalloc(len);
X	strcpy(s, expr);
X	strcpy(s + strlen(expr) + 1, matchname);
X
X	pm->name = s + strlen(expr) + 1;
X	pm->expr = s;
X	pm->flags = matchopt;
X    }
X}
X
Xint
Xmatch_binsert(char *matchname, char matchopt, char *expr, int beg, int end)
X{
X    struct match_t *pm = bmt.array + bmt.free;
X
X    MESSAGE_5("declaring match %s body %d-%d with opt %d expr %s",
X		matchname, beg, end, matchopt, expr);
X
X    if (bmt.free == bmt.max) {
X	syslog(LOG_NOTICE, "body match table too small (%d entries)",
X		bmt.max);
X	exit(1);
X    }
X
X    match_init(pm, matchname, matchopt, expr);
X    bmatch_beg[bmt.free] = beg;
X    bmatch_end[bmt.free] = end;
X    return bmt.free++;
X}
X
Xint
Xmatch_hinsert(char *matchname, char matchopt, char *expr, int hdrindex)
X{
X    struct match_t *pm = hmt.array + hmt.free;
X
X    MESSAGE_4("declaring match %s header %d with opt %d expr %s",
X		matchname, hdrindex, matchopt, expr);
X
X    if (hmt.free == hmt.max) {
X	syslog(LOG_NOTICE, "header match table too small (%d entries)",
X		hmt.max);
X	exit(1);
X    }
X
X    match_init(pm, matchname, matchopt, expr);
X    hmatch_index[hmt.free] = hdrindex;
X    return hmt.free++;
X}
X
X/*
X * Find a match by name in the indicated table.
X * Return index or -1 if not found.
X */
X
Xint
Xmatch_find(struct matchtable_t *mt, char *name)
X{
X    int i;
X    for (i = 0; i < mt->free; i++) {
X	if (!strcasecmp(name, mt->array[i].name)) {
X	    return i;
X	}
X    }
X    return -1;
X}
X
Xint
Xmatch_string(char *string, struct match_t *pm)
X{
X    char m;
X    char *expr = pm->expr;
X    int offset, le, la;
X
X    switch(pm->flags) {
X    case M_REGEX: {
X	char errbuf[80];
X	int rerr;
X	rerr = regexec(&pm->re, string, 0, NULL, 0);
X	if (rerr == 0)
X	    m = 1;
X	else if (rerr == REG_NOMATCH)
X	    m = 0;
X	else {
X	    regerror(rerr, NULL, errbuf, sizeof errbuf);
X	    syslog(LOG_NOTICE, "%s: %s", string, errbuf);
X	    exit(1);
X	}
X	break;
X    }
X    case 0:
X	le = strlen(expr);
X	la = strlen(string) - le;
X	m = 0;
X	while (la-- >= 0) {
X	    if (!strncasecmp(string, expr, le)) {
X		m = 1;
X		break;
X	    }
X	    string++;
X	}
X	break;
X    case M_CASE:
X	m = (strstr(string, expr) != NULL);
X	break;
X    case M_BEGLINE:
X	m = !strncasecmp(string, expr, strlen(expr));
X	break;
X    case M_CASE+M_BEGLINE:
X	m = !strncmp(string, expr, strlen(expr));
X	break;
X    case M_ENDLINE:
X	offset = strlen(string) - strlen(expr);
X	m = (offset < 0) ? 0 : !strcasecmp(string+offset, expr);
X	break;
X    case M_CASE+M_ENDLINE:
X	offset = strlen(string) - strlen(expr);
X	m = (offset < 0) ? 0 : !strcmp(string+offset, expr);
X	break;
X    case M_BEGLINE+M_ENDLINE:
X	m = !strcasecmp(string, expr);
X	break;
X    case M_CASE+M_BEGLINE+M_ENDLINE:
X	m = !strcmp(string, expr);
X	break;
X    default: /* ?! */
X	syslog(LOG_NOTICE, "%s: weird regexp flags %d", string, pm->flags);
X	exit(1);
X    }
X    return m;
X}
X
X/*
X * From group:artnum in 'groupnum', generate relative article path in 'path'.
X * Return pointer to char after if successful, NULL if error.
X */
X
Xconst char *
Xgenerate_path(const char *groupnum, char *path, int pathsize)
X{
X    const char *p, *q, *group, *art_num;
X
X    /* Keep room for the inserted '/' and the final '\0' */
X    if (pathsize < 2)
X	return NULL;
X    pathsize -= 2;
X
X    group = groupnum;
X
X    if ((p = strchr(group, ':')) == NULL)
X	return NULL;
X    art_num = p + 1;
X
X    /* Copy group to path, changing '.' to '/' */
X
X    for (q = group; q < p && pathsize; q++) {
X	*path++ = (*q == '.') ? '/' : *q;
X	pathsize--;
X    }
X    *path++ = '/';
X
X    /* Copy all digits of art_num */
X    for (q = art_num; isdigit(*q) && pathsize; q++) {
X	*path++ = *q;
X	pathsize--;
X    }
X    *path++ = '\0';
X
X    return pathsize ? q : NULL;
X}
X
XFILE *
Xget_artfile(struct article_t *current)
X{
X    char *p;
X    char path[MAXPATHNAMELEN];
X
X    if (current->artfiletried) {
X	if (current->artfile) rewind(current->artfile);
X	return current->artfile;
X    }
X
X    current->artfiletried = T_TRUE;	/* to try only once */
X    current->artfile = NULL;
X
X    /* skip the hostname */
X    p = strchr (current->h[I_XREF], ' ');
X    if (p == NULL)
X        return NULL;
X    p++;
X
X    MESSAGE_1 ("Generating path for %s", p);
X
X    if (generate_path(p, path, sizeof path) == NULL)
X	return NULL;
X
X    MESSAGE_1 ("path %s", path);
X
X    if ((current->artfile = fopen(path, "r")) == NULL) {
X	perror("cannot open");
X	return NULL;
X    }
X
X    return current->artfile;
X}
X
X/*
X * Compare current article contents with older article contents
X */
X
Xint
Xsame_contents(struct article_t *current, const char *oldpath)
X{
X    char line1[8192], line2[8192];
X    int linelen1, linelen2;
X
X    FILE *artfile, *oldfile;
X
X    MESSAGE_2("Comparing %s with %s", current->h[I_MESSAGE_ID], oldpath);
X
X    oldfile = fopen(oldpath, "r");
X    if (oldfile == NULL)
X	return 0;
X
X    artfile = get_artfile(current);
X    if (artfile == NULL) {
X	fclose(oldfile);
X	return 0;
X    }
X
X    /* Skip headers on new article */
X    for (;;) {
X        if (fgets (line1, sizeof (line1), artfile) == NULL) {
X	    fclose(oldfile);
X	    return 0;
X	}
X        line1 [strlen (line1) - 1] = '\0';
X        if (line1[0] == '\0')
X	    break;
X    }
X
X    /* Skip headers on old article */
X    for (;;) {
X        if (fgets (line2, sizeof (line2), oldfile) == NULL) {
X	    fclose(oldfile);
X	    return 0;
X	}
X        line2 [strlen (line2) - 1] = '\0';
X        if (line2[0] == '\0')
X	    break;
X    }
X
X    /* Examine body */
X
X    MESSAGE_2("Reading bodies for %s and %s", current->h[I_MESSAGE_ID], oldpath);
X
X    for (;;) {
X	/* Try to read a line from new article */
X	if (fgets (line1, sizeof (line1), artfile) == NULL) {
X	    if (fgets (line2, sizeof (line2), oldfile) == NULL) {
X		/* Simultaneous EOF, they are equal */
X		fclose(oldfile);
X		return 1;
X	    } else {
X		/* More lines on old article */
X		MESSAGE_1("Failure, %s shorter", current->h[I_MESSAGE_ID]);
X		fclose(oldfile);
X		return 0;
X	    }
X	}
X
X	/* Got a line from new article */
X	linelen1 = strlen(line1);
X	line1[linelen1 - 1] = '\0';
X
X	/* Try to read a line from old article */
X	if (fgets (line2, sizeof (line2), oldfile) == NULL) {
X	    MESSAGE_1("Failure, %s longer", current->h[I_MESSAGE_ID]);
X	    fclose(oldfile);
X	    return 0;
X	}
X
X	/* Got a line from old article */
X
X	linelen2 = strlen(line2);
X	if (linelen1 != linelen2) {
X	    MESSAGE_2("Failure, line lengths %d vs %d", linelen1, linelen2);
X	    fclose(oldfile);
X	    return 0;
X	}
X	line2[linelen2 - 1] = '\0';
X
X	if (strcmp(line1, line2)) {
X	    MESSAGE_2("Failure, line \"%s\" vs \"%s\"", line1, line2);
X	    fclose(oldfile);
X	    return 0;
X	}
X    }
X    return 1;
X}
X
X/*
X * Check articles (From:, Newsgroups: and contents)
X * and see if they are the same.
X */
X
Xint
Xsame_article(struct article_t *a, struct art_bucket_t *ab)
X{
X    char *p, *pb;
X    char *h, *hb;
X
X    /* Compare hashed values for From:/Newsgroups: */
X    if (a->hash != ab->hash)
X	return 0;
X
X    if (a->bodysize != ab->bodysize)
X	return 0;
X
X    MESSAGE_1 ("comparing hashed fields %s", a->h[I_MESSAGE_ID]);
X
X    /* Skip file paths and compare hashed headers */
X    p = a->hashfields; h = p;	while (*h) h++; h++;
X    pb = ab->fields; hb = pb;	while (*hb) hb++; hb++;
X    if (strcmp(h, hb))
X	return 0;
X
X    /* Hashed headers match; compare real body */
X
X    MESSAGE_2 ("comparing contents %s %s", a->h[I_MESSAGE_ID], pb);
X
X    return same_contents(a, pb);
X}
X
X/*
X * Store an article and drop the oldest.
X */
X
Xvoid
Xarticle_store(struct article_t *a)
X{
X    int newart;
X    struct art_bucket_t *ab;
X
X    newart = firstart++;
X    ab = artb + newart;
X
X    if (firstart == MAXARTS) firstart = 0;
X
X    if (freeart == 0) {
X	/* Drop the oldest article */
X	assert (msgtable[ab->hash].first == ab);
X	msgtable[ab->hash].first = ab->next;
X	free(ab->fields);
X    } else
X	freeart--;
X
X    ab->next = NULL;
X    ab->bodysize = a->bodysize;
X    ab->hash = a->hash;
X    ab->fields = a->hashfields;
X    a->hashfields = NULL;
X
X    if (msgtable[ab->hash].first == NULL) {
X	msgtable[ab->hash].last = msgtable[ab->hash].first = ab;
X    } else {
X	msgtable[ab->hash].last->next = ab;
X	msgtable[ab->hash].last = ab;
X    }
X}
X
Xvoid
Xeval_hmatch(struct article_t *current, matchflags_t matches)
X{
X    int i, hi;
X    char m;
X    struct match_t *pm;
X
X    for (i = 0; i < hmt.free; i++) {
X	if (matches & (1 << i)) {
X	    pm = hmt.array + i;
X	    hi = hmatch_index[i];
X
X	    m = match_string(current->h[hi], pm);
X
X	    current->hmatch_done |= (matchflags_t)(1 << i);
X	    current->hmatch_val |= (matchflags_t)(m << i);
X	    MESSAGE_4("matching %s: %s with %s is %d",
X		     hdrb[hi].name, current->h[hi],
X		     pm->expr ? pm->expr:"(precompiled)", m);
X	}
X    }
X}
X
X/* Eval hash function on article (selected headers, body size) */
X
Xvoid
Xeval_hash(struct article_t *current)
X{
X    char *p, *pc;
X    char path[MAXPATHNAMELEN];
X    hashmsg_t h;
X    int len;
X
X    /* skip the hostname */
X    p = strchr(current->h[I_XREF], ' ');
X    if (p == NULL) {
X	syslog(LOG_NOTICE, "bad Xref \"%s\"\n", current->h[I_XREF]);
X	exit(1);
X    }
X    do p++; while (isspace(*p));
X    if (generate_path(p, path, sizeof path) == NULL) {
X	syslog(LOG_NOTICE, "bad Xref \"%s\"\n", current->h[I_XREF]);
X	exit(1);
X    }
X
X    /* Allocate enough space for what we store */
X    len = strlen(path)
X	+ strlen(current->h[I_FROM])
X	+ strlen(current->h[I_NEWSGROUPS])
X	+ strlen(current->h[I_SUBJECT])
X	+ 5;
X    current->hashfields = pc = (char *)zmalloc(len);
X
X    /* Copy file path, including terminating 0 */
X    p = path;
X    while ((*pc++ = *p++));
X
X    /* Copy the fields we want to compare */
X    sprintf(pc, "%s\n%s\n%s\n",
X		current->h[I_FROM],
X		current->h[I_NEWSGROUPS],
X		current->h[I_SUBJECT]);
X
X    /* Evaluate hash value */
X    h = current->bodysize;
X    while (*pc) {
X	h *= 211;
X	h += *pc + 67;
X	pc++;
X    }
X    current->hash = h % HASHMSGSIZE;
X}
X
X/* Check for duplicate articles */
X
Xvoid
Xeval_duplicate(struct article_t *current)
X{
X    struct art_bucket_t *pab;
X
X    MESSAGE_1 ("checking duplicate %s", current->h[I_MESSAGE_ID]);
X
X    if (current->hashfields == NULL)
X	eval_hash(current);
X
X    for (pab = msgtable[current->hash].first;
X	 pab != NULL;
X	 pab = pab->next) {
X
X	MESSAGE_1 ("hash collision found %s", current->h[I_MESSAGE_ID]);
X
X	if (same_article(current, pab)) {
X	    MESSAGE_1 ("duplicate yields true on %s", current->h[I_MESSAGE_ID]);
X	    current->duplicate = T_TRUE;
X
X	    /*
X	     * Store anyway for future duplicate checking (the older article
X	     * might be removed from the FIFO soon, losing some potential
X	     * future duplicates detection).
X	     */
X	    article_store(current);
X	    return;
X	}
X    }
X
X    current->duplicate = T_FALSE;
X    article_store(current);
X}
X
Xvoid
Xeval_fromok(struct article_t *current)
X{
X    const char *p, *atpos;
X
X    /* check From: */
X    MESSAGE_1 ("checking From: %s", current->h[I_MESSAGE_ID]);
X
X    /* Check Reply-To: if found, else From: */
X    if (current->h[I_REPLY_TO][0] == '\0')
X	p = current->h[I_FROM];
X    else
X	p = current->h[I_REPLY_TO];
X
X    /*
X     * Check there's a '@' not on first char,
X     * followed by at least a '.' with at least one char between them.
X     */
X    atpos = strchr (p, '@');
X    current->fromok =
X	(atpos != NULL && atpos != p
X		&& atpos[1] != '.' && strchr (atpos, '.') != NULL);
X}
X
Xvoid
Xeval_refok(struct article_t *current)
X{
X    if (!strncmp(current->h[I_SUBJECT], "Re: ", 4))
X	current->refok = (current->h[I_REFERENCES][0] != '\0');
X    else
X	current->refok = T_TRUE;
X}
X
Xvoid
Xeval_multi(struct article_t *current)
X{
X    char *bracketpos = current->h[I_SUBJECT];
X
X    while ((bracketpos = strchr(bracketpos, '[')) != NULL) {
X
X	/*
X	 * Look for two numbers separated by anything
X	 * and ending with ']'. Whitespace is accepted except
X	 * in numbers.
X	 */
X
X	bracketpos++;
X	while (isspace(*bracketpos)) bracketpos++;
X	if (!*bracketpos) break;
X	if (!isdigit(*bracketpos)) continue;
X
X	/* Skip the first number */
X	do bracketpos++; while (isdigit(*bracketpos));
X	if (!*bracketpos) break;
X
X	/* Skip the separator */
X	do bracketpos++; while (!isdigit(*bracketpos));
X	if (!*bracketpos) break;
X
X	/* Skip the second number */
X
X	current->parts = atol(bracketpos + 1);
X	do bracketpos++; while (isdigit(*bracketpos));
X
X	if (!*bracketpos) break;
X	while (isspace(*bracketpos)) bracketpos++;
X	if (*bracketpos != ']') continue;
X
X	current->multi = T_TRUE;
X	return;
X    }
X    current->multi = T_FALSE;
X}
X
X/*
X * Pattern matching on article body.
X * Also recognizes binary formats: Base64, uuencode, Binhex, netsend.
X */
X
Xvoid
Xeval_body(struct article_t *current, char testbin, matchflags_t matches)
X{
X	char line[8192];
X	FILE *artfile;
X	int i;
X
X	/* Legal characters for Binhex. Go figure... */
X
X	char *bhchars =	
X	  "!\"#$%&'()*+,-012345689@ABCDEFGHIJKLMNPQRSTUVXYZ[`abcdefhijklmpqr";
X
X	int linecount = 0, linelen;
X	int uuline = 0, b64line = 0, bhline = 0, nsline = 0;
X
X	if ((artfile = get_artfile(current)) == NULL) {
X	    current->binary = T_FALSE;
X	    return;
X	}
X
X	/* Skip headers */
X	for (;;) {
X	    if (fgets (line, sizeof (line), artfile) == NULL)
X		return;
X	    line [strlen (line) - 1] = '\0';
X	    if (line[0] == '\0')
X		break;
X	}
X
X	/* Examine body */
X
X	for (;;) {
X	    if (fgets (line, sizeof (line), artfile) == NULL) {
X		break;
X	    }
X	    linelen = strlen(line);
X	    line[linelen - 1] = '\0';
X
X	    linecount++;
X
X	    /* pattern matching on body lines */
X
X	    for (i = 0; matches && i < bmt.free; i++) {
X		struct match_t *pm;
X		char m;
X
X		if (matches & (1 << i)) {
X		    if (linecount < bmatch_beg[i]) {
X			/* Line not in range for this pattern */
X			continue;
X		    }
X		    if (linecount > bmatch_end[i]) {
X			/* This pattern failed */
X			m = 0;
X		    } else {
X			pm = bmt.array + i;
X			m = match_string(line, pm);
X			if (m == 0 && linecount != bmatch_end[i])
X			    continue;	/* Not done, retry on next lines */
X		    }
X
X		    current->bmatch_done |= (matchflags_t)(1 << i);
X		    current->bmatch_val |= (matchflags_t)(m << i);
X
X		    /* Mark as evaluated */
X		    matches ^= (1 << i);
X		}
X	    }
X
X	    if (!matches && !testbin) {
X		/* Nothing more to evaluate, we're done */
X		return;
X	    }
X
X	    if (strncmp("begin ", line, 6) == 0) {
X		/* Bonus for typical uuencode line */
X		uuline += 30;
X	    } else if (strcmp("end", line) == 0) {
X		/* Bonus for typical uuencode line */
X		uuline += 30;
X	    } else if (((line[0] - ' ') & 077) * 4 == (linelen - 2) * 3) {
X		/* uuencode */
X		uuline++;
X	    } else if (linelen == 65
X		       || line[0] == ':'
X		       || line[linelen - 2] == ':') {
X		/* binhex or netsend */
X		char *pl, c;
X
X		/* binhex */
X		for (pl = line; *pl; pl++) {
X		    c = *pl;
X		    /* Accept a ':' as first or last char only */
X		    if (!strchr(bhchars, c)
X			&& (c != ':' || pl == line || pl[1] == '\0'))
X			break;
X		}
X		if (!*pl) {
X		    bhline++;
X		} else if (linelen == 65) {
X		    /* netsend */
X		    for (pl = line; *pl; pl++) {
X			c = *pl;
X			if (c == ' ' || c == 0x7f || c == '|' || c == '~'
X			    || (c & 0x80))
X			    break;
X		    }
X		    if (!*pl) nsline++;
X		}
X	    } else if (linelen > 50) {
X		/* base 64 */
X		char *pl, c;
X
X		for (pl = line; *pl; pl++) {
X		    c = *pl;
X		    if (!isalnum(c) && c != '/' && c != '+')
X			break;
X		}
X		if (!*pl) b64line++;
X	    }
X	}
X
X	/* Do more than 2 thirds look like some kind of binary encoding ? */
X	if (uuline * 3 > linecount * 2) {
X	    current->coding = "uuencode";
X	} else if (b64line * 3 > linecount * 2) {
X	    current->coding = "base64";
X	} else if (bhline * 3 > linecount * 2) {
X	    current->coding = "binhex";
X	} else if (nsline * 3 > linecount * 2) {
X	    current->coding = "netsend";
X	} else if ((bhline+uuline+b64line+nsline) * 3 > linecount * 2) {
X	    /* Too much binary is simply too much. */
X	    current->coding = "mixed";
X	}
X	if (current->coding[0]) {
X	    char *sender = current->h[I_FROM];
X	    if (current->h[I_REPLY_TO][0] != '\0')
X		sender = current->h[I_REPLY_TO];
X
X	    syslog (LOG_NOTICE, 
X		    "%s: %.100s %d (%s) Subject: %.100s groups=%.100s",
X		    curfeedname,
X		    current->h[I_MESSAGE_ID],
X		    current->bodysize,
X		    current->coding,
X		    current->h[I_SUBJECT], current->h[I_NEWSGROUPS]);
X
X	    current->binary = T_TRUE;
X	} else {
X	    current->binary = T_FALSE;
X	}
X	return;
X}
X
X/*
X * Generate article paths from Xref: field (for cross-posts)
X * and remove all the links.
X *
X * Xref syntax: group:art# [<SP>group:art#]
X */
X
Xint
Xremove_articles (struct article_t *current)
X{
X    const char *p;
X    char path[MAXPATHNAMELEN];
X
X    /* skip the hostname */
X    p = strchr(current->h[I_XREF], ' ');
X    if (p == NULL)
X        return 1;
X    do p++; while (isspace(*p));
X
X    while ((p = generate_path(p, path, sizeof path)) != NULL) {
X        unlink (path);
X	if (!isspace(*p))
X	    break;
X	p++;
X    }
X    return 0;
X}
X
X/*
X * Copy "in" to "out".
X * Stop at first empty line (not copied) if headers_only is not 0.
X */
X
Xvoid copy_file(FILE *out, FILE *in, char headers_only)
X{
X    char buff[8192];
X    while (fgets(buff, sizeof buff, in)) {
X	if (headers_only && buff[0] == '\n') break;
X	fputs(buff, out);
X    }
X}
X
X/*
X *  Process pattern, piping the output to pipeto
X *  If pipeto begins in ">", append to file instead.
X */
X
Xvoid
Xprocess_pattern(struct pattern_t *pattern,
X		const char *pipeto,
X		struct article_t *current)
X{
X    FILE *pp;
X    FILE *artfile;
X
X    char buff[8192];
X    char out[MAXIFNEST];
X    char *bp, *cp, *cp2;
X    int lb, ifnest;
X    char appending = 0;
X
X    ifnest = 0; out[0] = 1;
X
X    /* Open files */
X    if (pattern_open(pattern) < 0) {
Xabort();
X	return;
X	}
X
X    if (*pipeto == '>') {
X	pipeto++; appending++;
X	while (isspace(*pipeto)) pipeto++;
X	pp = fopen(pipeto, "a");
X    } else
X	pp = popen(pipeto, "w");
X
X    if (!pp) {
X	pattern_close(pattern);
X	return;
X    }
X
X    while (pattern_read(pattern, buff, sizeof buff) == 0) {
X	bp = buff; lb = strlen(buff);
X
X	while ((cp = strchr(bp, '%')) != NULL) {
X	    cp2 = strchr(cp + 1, '%');
X
X	    /* Found only one %, don't expand */
X	    if (!cp2) break;
X
X	    /* Found two % on same line */
X
X	    /* Write up to the first % */
X	    if (out[ifnest]) fwrite(bp, cp - bp, 1, pp);
X
X	    /* Expand the % part */
X	    *cp2 = '\0'; cp++;
X	    if (!strncasecmp(cp, "if-header-", 10)) {
X		int i;
X		i = header_to_index(cp + 10);
X		if (i < 0 || ifnest == MAXIFNEST-1) {
X		    /* Unknown header or nesting exceeded, don't expand */
X		    fprintf(pp, "%%%s%%", cp);
X		} else if (current->h[i][0] == '\0') {
X		    out[++ifnest] = 0;
X		} else {
X		    out[++ifnest] = 1;
X		}
X	    } else if (!strcasecmp(cp, "else")) {
X		if (ifnest > 0)
X		    out[ifnest] = !out[ifnest];
X		else
X		    fputs("%else%", pp);
X	    } else if (!strcasecmp(cp, "endif")) {
X		if (ifnest > 0) ifnest--; else fputs("%endif%", pp);
X	    } else if (!out[ifnest]) {
X		/* Do nothing */
X	    } else if (!strcasecmp(cp, "article-headers")
X		|| !strcasecmp(cp, "article")) {
X		if ((artfile = get_artfile(current)) == NULL)
X		    /* Can't open file, don't expand */
X		    fprintf(pp, "%%%s%%", cp);
X		else {
X		    copy_file(pp, artfile,
X			      strcasecmp(cp, "article"));
X		}
X	    } else if (!strcasecmp(cp, "article-sender")) {
X		if (current->h[I_REPLY_TO][0] != '\0')
X		    fputs(current->h[I_REPLY_TO], pp);
X		else
X		    fputs(current->h[I_FROM], pp);
X	    } else if (!strcasecmp(cp, "article-sender-host")) {
X		char *u = current->h[I_REPLY_TO];
X		char *atpos;
X		if (current->h[I_REPLY_TO][0] == '\0')
X		    u = current->h[I_FROM];
X		atpos = strchr(u, '@');
X		if (atpos)
X		    fputs(atpos + 1, pp);
X	    } else if (!strcasecmp(cp, "moderated-sender")) {
X		if (current->h[I_APPROVED][0] != '\0')
X		    fputs(current->h[I_APPROVED], pp);
X		else if (current->h[I_REPLY_TO][0] != '\0')
X		    fputs(current->h[I_REPLY_TO], pp);
X		else
X		    fputs(current->h[I_FROM], pp);
X	    } else if (!strcasecmp(cp, "article-size")) {
X		fprintf(pp, "%ld", current->bodysize);
X	    } else if (!strcasecmp(cp, "article-encoding")) {
X		fprintf(pp, "%s", current->coding[0] == '\0'
X			? "mixed encoding"
X			: current->coding);
X	    } else if (!strcasecmp(cp, "re-subject")) {
X		if (strncasecmp(current->h[I_SUBJECT], "Re: ", 4))
X		    fprintf(pp, "Re: %s", current->h[I_SUBJECT]);
X		else
X		    fputs(current->h[I_SUBJECT], pp);
X	    } else if (!strcasecmp(cp, "cancel-message-id")) {
X		fprintf(pp, "<cancel.%s", current->h[I_MESSAGE_ID] + 1);
X	    } else if (!strcasecmp(cp, "truncated-message-id")) {
X		fputs(current->h[I_MESSAGE_ID] + 1, pp);
X	    } else if (!strncasecmp(cp, "header-", 7)) {
X		int i;
X		i = header_to_index(cp + 7);
X		if (i < 0)
X		    /* Unknown header, don't expand */
X		    fprintf(pp, "%%%s%%", cp);
X		else
X		    fputs(current->h[i], pp);
X	    } else if (!strcasecmp(cp, "date")) {
X		char buffd[80];
X		time_t t;
X		time(&t);
X		strftime(buffd, sizeof buffd, "%d %b %Y %T GMT", gmtime(&t));
X		fputs(buffd, pp);
X	    } else if (!strcasecmp(cp, "newsbot-version")) {
X		fputs(version, pp);
X	    } else if (!strcasecmp(cp, "")) {
X		fputs("%", pp);
X	    } else if (*cp == '<') {
X		FILE *cpf = fopen(cp + 1, "r");
X		if (cpf) {
X		    copy_file(pp, cpf, 0);
X		    fclose(cpf);
X		} else {
X		    /* Unknown file, don't expand */
X		    fprintf(pp, "%%%s%%", cp);
X		}
X	    } else {
X		/* Unknown tag, don't expand */
X		fprintf(pp, "%%%s%%", cp);
X	    }
X	    /* Continue */
X	    lb -= cp2 + 1 - bp;
X	    bp = cp2 + 1;
X	}
X	if (out[ifnest]) fwrite(bp, lb, 1, pp);
X    }
X
X    pattern_close(pattern);
X    if (appending)
X	fclose(pp);
X    else
X	pclose(pp);
X}
X
X/*
X * Execute and log action
X *
X * Return 0 if this is to be the last action for this article.
X */
X
Xint
Xdo_action(struct action_t *action, struct article_t *current)
X{
X    syslog(LOG_NOTICE,
X	   "%s: %s %s on %s", opt_act ? "doing":"would do",
X	   curfeedname, action->name, current->h[I_MESSAGE_ID]);
X
X    if (action == &actremove && opt_act)
X	remove_articles(current);
X    else if (action == &actstop)
X	return 0;
X    else if (action == &actnop)
X	return 1;
X    else if (opt_act) {
X	MESSAGE_2("pipe %s to %s", action->pattern, action->pipeto);
X	process_pattern(action->pattern, action->pipeto, current);
X    }
X    return 1;
X}
X
X/*
X * Check condition against article.
X * Return 1 if ok, 0 if not.
X */
X
Xint
Xcheck_cond(struct cond_t *cond, struct article_t *current)
X{
X    /* Check article size */
X    if (cond->checksize != T_DONTCARE) {
X	MESSAGE_2("condition 0x%08x check size %ld", cond, cond->size);
X
X	if (cond->checksize == T_TRUE) {
X	    if (current->bodysize < cond->size)
X		return 0;
X	} else {
X	    if (current->bodysize >= cond->size)
X		return 0;
X	}
X    }
X
X    /* Check from */
X    if (cond->fromok != T_DONTCARE) {
X	MESSAGE_1("condition 0x%08x check from", cond);
X	if (current->fromok == T_DONTCARE)
X	    eval_fromok(current);
X	if (cond->fromok != current->fromok)
X	    return 0;
X    }
X
X    /* Check references */
X    if (cond->refok != T_DONTCARE) {
X	MESSAGE_1("condition 0x%08x check refs", cond);
X	if (current->refok == T_DONTCARE)
X	    eval_refok(current);
X	if (cond->refok != current->refok)
X	    return 0;
X    }
X
X    /* Check multi size */
X    if (cond->checkmultisize != T_DONTCARE) {
X	MESSAGE_1("condition 0x%08x check multi", cond);
X	if (current->multi == T_DONTCARE)
X	    eval_multi(current);
X	if (cond->checkmultisize == T_TRUE) {
X	    if (current->multi == T_FALSE
X		|| current->parts * current->bodysize < cond->size)
X		return 0;
X	} else {
X	    if (current->multi == T_TRUE
X		&& current->parts * current->bodysize >= cond->size)
X		return 0;
X	}
X    }
X
X    /* Check duplicate */
X    if (cond->duplicate != T_DONTCARE) {
X	MESSAGE_1("condition 0x%08x check duplicate", cond);
X	if (current->duplicate == T_DONTCARE)
X	    eval_duplicate(current);
X	if (cond->duplicate != current->duplicate)
X	    return 0;
X    }
X
X    /* Check header matches */
X    if (cond->hmatch_totest) {
X	MESSAGE_1("condition 0x%08x check header matches", cond);
X	if (~current->hmatch_done & cond->hmatch_totest)
X	    eval_hmatch(current, ~current->hmatch_done & cond->hmatch_totest);
X	if ((cond->hmatch_val ^ current->hmatch_val) & cond->hmatch_totest)
X	    return 0;
X    }
X
X    /* Check body (binary or pattern matching) */
X    if (cond->binary != T_DONTCARE || cond->bmatch_totest) {
X	MESSAGE_1("condition 0x%08x check body", cond);
X	if (current->binary == T_DONTCARE
X	    || (~current->bmatch_done & cond->bmatch_totest))
X	    eval_body(current,
X		      cond->binary != T_DONTCARE,
X		      ~current->bmatch_done & cond->bmatch_totest);
X	if ((cond->binary != T_DONTCARE && cond->binary != current->binary)
X	    || ((cond->bmatch_val ^ current->bmatch_val) & cond->bmatch_totest))
X	    return 0;
X    }
X    return 1;
X}
X
X/*
X * Handle condition
X */
X
Xint
Xhandle_cond (struct cond_t *condp, struct article_t *current)
X{
X    for (; condp && check_cond(condp, current); condp = condp->and) {
X	MESSAGE_1("condition 0x%08x TRUE", condp);
X	if (!do_action(condp->action, current))
X	    /* Stop processing for this article if we have to */
X	    return 0;
X    }
X    return 1;
X}
X
X/*
X * Handle article
X */
Xvoid
Xhandle_article (struct article_t *current, struct feed_t *f)
X{
X    struct cond_t *condp;
X
X    curfeedname = f->name;
X
X    for (condp = f->firstcond; condp; condp = condp->next)
X	if (!handle_cond(condp, current))
X		break;
X}
X
X/*
X * Convert size : integer optionaly followed by a multiplier (k or M)
X */
Xlong
Xconv_size(char *p, char **newp)
X{
X    long n;
X    if (isdigit(*p)) {
X	n = atol(p);
X	do p++; while (isdigit(*p));
X	if (*p == 'K' || *p == 'k') {
X	    n *= 1024;
X	    p++;
X	} else if (*p == 'M' || *p == 'm') {
X	    n *= 1024*1024;
X	    p++;
X	}
X    } else
X	n = 0;
X
X    *newp = p;
X    return n;
X}
X
X/*
X * Read configuration file
X */
X
Xvoid
Xread_conf(const char *conf)
X{
X    char line[2048];
X    char *mac, *p;
X    char conj;
X    struct action_t *pa;
X    struct cond_t *pc, *lastconj = NULL;
X    struct feed_t *pf;
X    struct feed_t *curfeed = &nullfeed;
X    struct pattern_t *inpattern = NULL;
X
X    FILE *cf = fopen(conf, "r");
X    if (cf == NULL)
X	exit(1);
X
X    while (fgets(line, sizeof line, cf) != NULL) {
X	if (inpattern) {
X	    if (line[0] == '=') {
X		pattern_append(inpattern, line+1);
X		continue;
X	    }
X	    if (line[0] != '#')
X		inpattern = NULL;
X	}
X
X	line[strlen(line) - 1] = '\0';
X
X	if (line[0] == '#' || line[0] == '\0')
X	    continue;
X
X	if (line[0] == 'P') {
X	    /* Search for: name */
X	    char *name;
X	    /* Keep pattern name */
X	    name = line + 1;
X	    /* Search and create */
X	    inpattern = pattern_find(name);
X	} else if (line[0] == 'A') {
X	    /* Search for: name pattern pipeto */
X
X	    char *name, *pattern, *pipeto;
X
X	    /* Keep action name */
X	    name = line + 1;
X
X	    /* Find pattern file name */
X	    for (p = name; !isspace(*p) && *p; p++);
X	    if (!*p) continue;
X	    *p++ = '\0';
X	    while (isspace(*p)) p++;
X	    if (!*p) continue;
X	    pattern = p;
X
X	    /* Find command name to pipe to */
X	    for (; !isspace(*p); p++);
X	    if (!*p) continue;
X	    *p++ = '\0';
X	    while (isspace(*p)) p++;
X	    if (!*p) continue;
X	    pipeto = p;
X
X
X	    /* Check for duplicate action */
X	    for (pa = firstact; pa; pa = pa->next)
X		if (strcasecmp(pa->name, name) == 0)
X		    break;
X
X	    if (pa)
X		syslog(LOG_NOTICE, "duplicate action name \"%s\" (ignored)",
X		       name);
X	    else {
X
X		pa = zmalloc(sizeof(struct action_t));
X		mac = zmalloc(strlen(name)+strlen(pipeto)+2);
X
X		MESSAGE_3("action %s: %s %s", name, pattern, pipeto);
X
X		pa->name = mac; strcpy(mac, name); mac += strlen(name)+1;
X		pa->pipeto = mac; strcpy(mac, pipeto); mac += strlen(pipeto)+1;
X		pa->pattern = pattern_find(pattern);
X
X		/* Insert in action list */
X		pa->next = firstact; firstact = pa;
X	    }
X	} else if (line[0] == 'I' || line[0] == ',') {
X	    char *condition, *action;
X
X	    /*
X	     * If the condition begins with ',', flag it to link it
X	     * as a conjunction with the previous condition.
X	     */
X
X	    if (line[0] == ',') {
X		conj = 1;
X		if (lastconj == NULL) {
X		    syslog(LOG_NOTICE, "conjunction with nothing, %s ignored",
X			   line);
X		    continue;
X		}
X	    } else {
X		conj = 0;
X	    }
X
X	    /* Search for: condition action */
X
X	    condition = line + 1;
X
X	    p = condition;
X
X	    /* Find action name */
X	    for (; !isspace(*p); p++);
X	    if (!*p) continue;
X	    *p++ = '\0';
X	    while (isspace(*p)) p++;
X	    if (!*p) continue;
X	    action = p;
X
X	    for (; !isspace(*p); p++);
X	    if (*p) *p = '\0';
X
X	    /* Find action in list */
X	    for (pa = firstact; pa; pa = pa->next)
X		if (!strcmp(pa->name, action))
X		    break;
X
X	    if (pa == NULL) {
X		syslog(LOG_NOTICE, "action %s not found, condition ignored",
X		       action);
X		continue;
X	    }
X
X	    pc = zmalloc(sizeof(struct cond_t));
X
X	    pc->fromok = T_DONTCARE;
X	    pc->refok = T_DONTCARE;
X	    pc->binary = T_DONTCARE;
X	    pc->duplicate = T_DONTCARE;
X	    pc->checksize = T_DONTCARE;
X	    pc->checkmultisize = T_DONTCARE;
X	    pc->hmatch_val = pc->hmatch_totest = 0;
X	    pc->bmatch_val = pc->bmatch_totest = 0;
X	    pc->action = pa;
X	    pc->and = NULL;
X
X	    /* Decode and store condition */
X
X	    MESSAGE_1("analyzing condition %s", condition);
X
X	    for (; *condition; condition++) {
X		char val; char curcond;
X		if (*condition == '!') {
X		    if (!*condition++)
X			break;
X		    val = T_FALSE;
X		} else
X		    val = T_TRUE;
X
X		MESSAGE_1("condition %c", *condition);
X		switch((curcond = *condition++)) {
X		case 'M':
X		    pc->multisize = conv_size(condition, &condition);
X		    pc->checkmultisize = val;
X		    break;
X		case '>':
X		    pc->size = conv_size(condition, &condition);
X		    pc->checksize = val;
X		    break;
X		case 'D':
X		    pc->duplicate = val;
X		    break;
X		case 'B':
X		    pc->binary = val;
X		    break;
X		case 'R':
X		    pc->refok = val;
X		    break;
X		case 'F':
X		    pc->fromok = val;
X		    break;
X		case 'L':
X		case '~': {
X		    char *mn;
X		    int i, c;
X		    mn = condition;
X		    while (isalnum(*condition)) condition++;
X
X		    c = *condition;
X		    *condition = '\0';
X		    i = match_find(curcond == 'L' ? &bmt:&hmt, mn);
X		    *condition = c;
X
X		    if (i < 0) {
X			syslog(LOG_NOTICE, "%s match %s unknown",
X				curcond == 'L' ? "body":"header", mn);
X		    } else {
X			if (curcond == 'L') {
X			    pc->bmatch_totest |= 1L << i;
X			    pc->bmatch_val |= val << i;
X			} else {
X			    pc->hmatch_totest |= 1L << i;
X			    pc->hmatch_val |= val << i;
X			}
X		    }
X		    break;
X		}
X		default:
X		    syslog(LOG_NOTICE, "invalid code '%c' in condition",
X			   *condition);
X		    break;
X		}
X		if (!*condition)
X		    break;
X		if (*condition != ',') {
X		    syslog(LOG_NOTICE, "invalid separator '%c' in condition",
X			   *condition);
X		    condition++;
X		}
X	    }
X
X	    /* Insert in condition list */
X
X	    if (conj && lastconj) {
X		lastconj->and = pc;
X		lastconj = pc;
X	    } else {
X		if (curfeed->lastcond)
X		    curfeed->lastcond->next = pc;
X		else
X		    curfeed->firstcond = pc;
X		lastconj = curfeed->lastcond = pc;
X		pc->next = NULL;
X	    }
X	} else if (line[0] == 'F') {
X	    char *feed;
X	    feed = p = line + 1;
X
X	    while (!isspace(*p)) p++;
X	    *p = '\0';
X
X	    /* Look if it's already known */
X	    for (pf = firstfeed; pf; pf = pf->next)
X		if (strcasecmp(pf->name, feed) == 0)
X		    break;
X
X	    if (pf)
X		syslog(LOG_NOTICE, "duplicate declaration for feed \"%s\"",
X		       feed);
X	    else {
X
X		/* Create a feed struct */
X		pf = zmalloc(sizeof(struct feed_t));
X
X		/* Copy the feed name */
X		pf->name = zmalloc(strlen(feed) + 1);
X		strcpy(pf->name, feed);
X
X		/* Initialize other fields */
X		pf->lastcond = NULL;
X		pf->firstcond = NULL;
X
X		/* Insert in feed list */
X		pf->next = firstfeed;
X		firstfeed = pf;
X	    }
X
X	    /* Make it the current feed */
X	    curfeed = pf;
X
X	    lastconj = NULL;
X	} else if (line[0] == 'H') {
X	    /* allocate enough space just for the header name + '\0' */
X	    char *new_header = (char *)zmalloc(strlen(line));
X	    MESSAGE_1("declaring header %s", line+1);
X
X	    strcpy(new_header, line+1);
X	    header_insert(new_header);
X	} else if (line[0] == '~' || line[0] == 'L') {
X	    int h = -1;
X	    int i;
X	    char *matchname, *arg1, *expr;
X	    char matchopt = M_CASE;
X	    char headermatch = line[0] == '~';
X	    char *p = line + 1;
X
X	    while (!isspace(*p)) {
X		switch (*p++) {
X		case 'c':	/* case sensitive (default) */
X		    matchopt |= M_CASE;
X		    break;
X		case 'i':	/* case insensitive */
X		    matchopt &= ~M_CASE;
X		    break;
X		case 'e':	/* regex */
X		    matchopt |= M_REGEX;
X		    break;
X		default:
X		    syslog(LOG_NOTICE, "unknown matching option %c", *p);
X		    break;
X		}
X	    }
X
X	    while (isspace(*p)) p++;
X	    if (!*p) continue;
X
X	    matchname = p;
X	    while (!isspace(*p)) p++;
X	    if (!*p) continue;
X	    *p++ = '\0';
X
X	    while (isspace(*p)) p++;
X	    if (!*p) continue;
X
X	    arg1 = p;
X	    while (!isspace(*p)) p++;
X	    if (!*p) continue;
X	    *p++ = '\0';
X
X	    while (isspace(*p)) p++;
X	    if (!*p) continue;
X
X	    expr = p;
X
X	    i = match_find(headermatch ? &hmt:&bmt, matchname);
X
X	    if (i >= 0) {
X		syslog(LOG_NOTICE, "duplicate match %s", matchname);
X	    } else {
X		int beg = 1, end = 1;
X		if (headermatch) {
X		    /* Header match */
X		    h = header_to_index(arg1);
X
X		    if (h < 0) {
X			syslog(LOG_NOTICE, "undeclared header %s", arg1);
X			break;
X		    }
X		} else {
X		    /* Body line match */
X		    /* XXX: range check currently very crude... */
X		    char *arg2;
X		    if ((arg2 = strchr(arg1, '-')) != NULL) {
X			/* Minus sign present */
X			*arg2++ = '\0';
X			sscanf(arg1, "%d", &beg);
X			sscanf(arg2, "%d", &end);
X		    } else {
X			sscanf(arg1, "%d", &beg);
X			end = beg;
X		    }
X		}
X
X		if (!(matchopt & M_REGEX)) {
X		    if (*expr == '^') {
X			expr++;
X			matchopt |= M_BEGLINE;
X		    }
X		    if (expr[strlen(expr)-1] == '$') {
X			expr[strlen(expr)-1] = '\0';
X			matchopt |= M_ENDLINE;
X		    }
X		}
X
X		if (headermatch) {
X		    match_hinsert(matchname, matchopt, expr, h);
X		} else {
X		    match_binsert(matchname, matchopt, expr, beg, end);
X		}
X	    }
X	} else
X	    syslog(LOG_NOTICE, "cannot parse line: %s", line);
X    }
X    fclose(cf);
X}
X
Xint
Xmain (int argc, char **argv)
X{
X    int c, i, lineno, len;
X    char feedline[256];
X
X    char *line; int linesize;
X    char *pc, *begline, *nextbegline;
X
X    struct article_t current;
X
X    if (chdir(_PATH_SPOOL) < 0) {
X	perror("cannot chdir to news spool");
X	exit(1);
X    }
X
X    while ((c = getopt(argc, argv, "aC:D:df")) != EOF) {
X	switch (c) {
X	    case 'C':
X		path_conf = optarg;
X		break;
X	    case 'D':
X		path_pats = optarg;
X		break;
X	    case 'd':
X		opt_debug++;
X		break;
X	    case 'f':
X		opt_feed++;
X		break;
X	    case 'a':
X		opt_act++;
X		break;
X	}
X    }
X    argc -= optind;
X    argv += optind;
X
X    header_init();
X    artb_init();
X    read_conf(path_conf);
X
X    if (opt_act) {
X	openlog ("newsbot+", LOG_NDELAY, LOG_FACILITY);
X	syslog (LOG_NOTICE, "Starting, active mode");
X    } else {
X	openlog ("newsbot", LOG_NDELAY, LOG_FACILITY);
X	syslog (LOG_NOTICE, "Starting, no actions");
X    }
X
X    for (;;) {
X
X	for (i = 0; i < freehdr; i++)
X	    current.h[i] = "";
X
X        current.bodysize = -1;
X	current.headersize = 0;
X	current.control = 0;
X
X	current.refok	= T_DONTCARE;
X	current.fromok	= T_DONTCARE;
X	current.multi	= T_DONTCARE;
X	current.binary	= T_DONTCARE;
X	current.duplicate = T_DONTCARE;
X	current.hmatch_done = 0;
X	current.hmatch_val = 0;
X	current.bmatch_done = 0;
X	current.bmatch_val = 0;
X	current.parts	= -1;
X        current.coding = "";
X
X	current.artfiletried	= T_FALSE;
X	current.artfile		= NULL;
X	current.hashfields	= NULL;
X
X	current.firstbyte	= 0;
X	line = current.hbuf;
X	linesize = sizeof(current.hbuf);
X
X	lineno = 0;
X
X	/* Process feed line if asked to */
X	if (opt_feed)
X	    if (fgets (feedline, sizeof (feedline), stdin) == NULL)
X		exit (0);
X
X	begline = NULL;
X
X        for (;;) {
X
X	    if (linesize < MINLINE) {
X		/* Space exhausted, skip to next article */
X		char nl[MINLINE];
X
X		for (;;) {
X		    if (fgets(nl, sizeof nl, stdin) == NULL)
X			exit(0);
X		    if (nl[strlen(nl)-1] != '\n')
X			continue;		/* line not complete */
X		    if (nl[0] == '\n')
X			break;
X		}
X		current.control = 1;		/* HACK ! */
X		break;
X	    }
X
X	    /* Get line if necessary */
X	    if (!begline) {
X		begline = line;
X		if (fgets (begline, linesize, stdin) == NULL)
X		    exit (0);
X	    }
X
X            if (begline[0] == '\n')
X                break;			/* End of headers, go process */
X
X	    len = strlen(begline);
X
X	    /*
X	     * Now begline points to the current line,
X	     * line points to linesize bytes of free space after.
X	     */
X
X	    nextbegline = NULL;
X	    current.headersize += len;
X	    lineno++;
X
X	    pc = strchr(begline, ':');
X	    if (pc && pc[1] == ' ' && *pc != ' ' && *pc != '\t') {
X		/*
X		 * There's a colon in the line, followed by a space,
X		 * and it's not a continuation line.
X		 */
X		MESSAGE_1 ("H=%s", begline);
X		*pc = '\0';
X		switch(i = header_to_index(begline)) {
X		case -1:
X		    break;
X		case I_CONTROL:
X		    current.control = 1;
X		    break;
X		case I_BYTES:
X		    if (lineno == 1) {
X			current.bodysize = atol(line + 7);
X			/* "Bytes:" is not stored in the file */
X			current.headersize -= len;
X		    }
X		    break;
X		case I_XREF:
X		    /*
X		     *"Xref:" is not stored in the file
X		     * if INN gives it in line 2 (no crosspost)
X		     */
X		    if (lineno == 2) {
X			current.headersize -= len;
X		    }
X		    /* FALLTHROUGH */
X		default:
X		    /*
X		     * This header is to be kept.
X		     * Find continuation lines, if any...
X		     */
X		    current.h[i] = pc + 2;
X		    line += len; linesize -= len;
X		    for (;;) {
X			if (linesize < MINLINE)
X			    break;
X			if (fgets(line, linesize, stdin) == NULL)
X			    exit(0);
X			if (*line != ' ' && *line != '\t') {
X			    /* Not a continuation, stop but keep it safe */
X			    nextbegline = line;
X			    break;
X			}
X			MESSAGE_1 ("HC=%s", line);
X			len = strlen(line);
X			line += len; linesize -= len;
X			current.headersize += len;
X		    }
X
X		    /* Hack to get at beginning of the line loop */
X		    if (linesize < MINLINE)
X			continue;
X
X		    /* Replace final '\n' */
X		    line[-1] = '\0';
X
X		    MESSAGE_1 ("Keeping %s", current.h[i]);
X		}
X	    } else {
X		MESSAGE_1 ("Hc=%s", begline);
X	    }
X	    begline = nextbegline;
X        }
X
X        if (!current.control) {
X            MESSAGE_1 ("got %s", current.h[I_MESSAGE_ID]);
X
X	    /* Compute article body size */
X	    current.bodysize -= current.headersize;
X
X	    /* Handle null feed */
X	    handle_article (&current, &nullfeed);
X
X	    /* Handle named feeds */
X	    if (opt_feed) {
X		struct feed_t *cf, *lf;
X		char *pf = feedline, *pf2;
X		char lastch;
X
X		cf = firstfeed;
X
X		do {
X		    /* Find the next feed name */
X		    for (pf2 = pf; *pf2 && !isspace(*pf2); pf2++);
X		    lastch = *pf2;
X		    *pf2 = '\0';
X
X		    /*
X		     * The feed name is now pointed to by pf,
X		     * try to find it in order.
X		     */
X		    for (lf = cf; cf; cf = cf->next)
X			if (strcasecmp(cf->name, pf) == 0)
X			    break;
X
X		    if (!cf) {
X			/*
X			 * Not found at partial pass.
X			 * Retry from the beginning.
X			 */
X			for (cf = firstfeed; cf; cf = cf->next)
X			    if (strcasecmp(cf->name, pf) == 0)
X				break;
X
X			if (cf) {
X			    /* Found at last. Reorder the list so that
X			       cf is just before lf. That will save us work
X			       next time...  */
X			     /* XXX: to be done... */
X			}
X		    }
X
X		    if (cf)
X			handle_article (&current, cf);
X		    else
X			syslog(LOG_NOTICE,
X			       "feed entry %s is not known", pf);
X
X		    pf = pf2 + 1;
X		} while (lastch != '\n');
X	    }
X
X	    /* Cleanup */
X	    if (current.artfile)
X		fclose(current.artfile);
X	    if (current.hashfields)
X		free(current.hashfields);
X        }
X    }
X    return 0;
X}
END-of-newsbot.c
echo c - sample.config
mkdir -p sample.config > /dev/null 2>&1
echo x - sample.config/bin.cancel-global
sed 's/^X//' >sample.config/bin.cancel-global << 'END-of-sample.config/bin.cancel-global'
XPath: cyberspam!bincancel!user
XNewsgroups: %header-newsgroups%
XFrom: %header-from%
X%if-header-sender%Sender: %header-sender%
X%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT
X%endif%Subject: cancel: %header-subject%
XMessage-Id: %cancel-message-id%
XControl: cancel %header-message-id%
XDate: %date%
XX-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT
XX-Bot: %newsbot-version%
X
X%article-size% bytes binary in discussion group (%article-encoding%)
END-of-sample.config/bin.cancel-global
echo x - sample.config/bin.cancel-local
sed 's/^X//' >sample.config/bin.cancel-local << 'END-of-sample.config/bin.cancel-local'
XPath: cyberspam!bincancel!user
XNewsgroups: %header-newsgroups%
XFrom: %header-from%
X%if-header-sender%Sender: %header-sender%
X%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT
X%endif%Subject: cancel: %header-subject%
XMessage-Id: %cancel-message-id%
XControl: cancel %header-message-id%
XDate: %date%
XX-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT
XX-Bot: %newsbot-version%
XDistribution: local
X
XLocal cancel, %article-size% bytes binary in discussion group (%article-encoding%)
END-of-sample.config/bin.cancel-local
echo x - sample.config/bin.mcc-sender-fr
sed 's/^X//' >sample.config/bin.mcc-sender-fr << 'END-of-sample.config/bin.mcc-sender-fr'
XFrom: user@I_NEED_TO_CONFIGURE_NEWSBOT (My Full Name)
XTo: %moderated-sender%
XSubject: %re-subject%
XPrecedence: junk
XOrganization: Usenet canal historique
XX-Bot: %newsbot-version%
XMime-Version: 1.0
XContent-Type: text/plain;charset=ISO-8859-1
XContent-Transfer-Encoding: 8bit
X
XIt seems you have posted a %article-size% bytes binary file as
Xarticle %header-message-id%.
X
XThis article has been autocancelled, as one or more of the groups you
Xposted to are general newsgroups, were binaries should not be posted.
X
XPlease use the appropriate hierarchy, for example alt.binaries.*, and
Xdon't crosspost.
X
XThis message has been automatically generated.
X
X---
XVous avez apparemment envoyi un fichier binaire de %article-size% octets
Xsous l'identification %header-message-id%.
X
XL'article a iti annuli, car un ou plusieurs des groupes destinataires ne
Xsont pas destinis aux binaires. Veuillez utiliser la hiirarchie appropriie,
Xpar exemple alt.binaries.*, et ne pas faire de crossposts.
X
XCe message a iti giniri automatiquement.
X
X---- copy of article follows ----
X%article%---- end of article ----
END-of-sample.config/bin.mcc-sender-fr
echo x - sample.config/bin.mch-frbin
sed 's/^X//' >sample.config/bin.mch-frbin << 'END-of-sample.config/bin.mch-frbin'
XFrom: newsmaster
XTo: [email-masked]
XSubject: BINAIRE dans fr.* : %header-subject%
XOrganization: Usenet canal historique
XX-Bot: %newsbot-version%
XMime-Version: 1.0
XContent-Type: text/plain;charset=ISO-8859-1
XContent-Transfer-Encoding: 8bit
X
XUn article de %article-size% octets vient d'jtre ditectj
X` news.univ-angers.fr dans un groupe de fr.*.
X
XL'article a iti annuli, et un article explicatif imis dans fr.usenet.divers.
X
X---- copie des entjtes ----
X%article-headers%---- fin des entjtes ----
END-of-sample.config/bin.mch-frbin
echo x - sample.config/bin.mh-newsmaster
sed 's/^X//' >sample.config/bin.mh-newsmaster << 'END-of-sample.config/bin.mh-newsmaster'
XFrom: newsmaster
XTo: newsmaster
XSubject: binary file: %header-subject%
XOrganization: Usenet canal historique
XX-Bot: %newsbot-version%
XMime-Version: 1.0
XContent-Type: text/plain;charset=ISO-8859-1
XContent-Transfer-Encoding: 8bit
X
X%article-size% bytes binary file detected as article %header-message-id%.
X
X---- copy of article headers follows ----
X%article-headers%---- end of article headers ----
END-of-sample.config/bin.mh-newsmaster
echo x - sample.config/bin.mh-sender-fr
sed 's/^X//' >sample.config/bin.mh-sender-fr << 'END-of-sample.config/bin.mh-sender-fr'
XFrom: user@I_NEED_TO_CONFIGURE_NEWSBOT (My Full Name)
XTo: %moderated-sender%
XSubject: %re-subject%
XPrecedence: junk
XOrganization: Usenet canal historique
XX-Bot: %newsbot-version%
XMime-Version: 1.0
XContent-Type: text/plain;charset=ISO-8859-1
XContent-Transfer-Encoding: 8bit
X
XIt seems you have posted a %article-size% bytes binary file as
Xarticle %header-message-id%.
X
XOne or more of the groups you posted to are general newsgroups, were
Xbinaries should not be posted.
X
XPlease use the appropriate hierarchy, for example alt.binaries.*, and
Xdon't crosspost.
X
XIt would be better to cancel this article using the appropriate option of
Xyour newsreader.
X
XThis message has been automatically generated.
X
X---
XVous avez apparemment envoyi un fichier binaire de %article-size% octets
Xsous l'identification %header-message-id%.
X
XUn ou plusieurs des groupes destinataires ne sont pas destinis aux
Xbinaires. Veuillez utiliser la hiirarchie appropriie, par exemple
Xalt.binaries.*, et ne pas faire de crossposts.
X
XIl serait trhs recommandi d'annuler cet article en utilisant
Xl'option adiquate de votre logiciel de lecture de news.
X
XCe message a iti automatiquement giniri.
X
X---- copy of article headers follows ----
X%article-headers%---- end of article headers ----
END-of-sample.config/bin.mh-sender-fr
echo x - sample.config/bin.p-expl-frud
sed 's/^X//' >sample.config/bin.p-expl-frud << 'END-of-sample.config/bin.p-expl-frud'
XNewsgroups: fr.usenet.divers
XSubject: BINAIRE DE %article-sender%
XOrganization: Usenet canal historique
XMessage-Id: <expl.%truncated-message-id%
XX-Bot: %newsbot-version%
XMime-Version: 1.0
XContent-Type: text/plain;charset=ISO-8859-1
XContent-Transfer-Encoding: 8bit
XReferences: %header-message-id%
X
XUn article binaire de %article-size% octets imis par
X	%article-sender%
Xvient d'jtre ditecti et annuli automatiquement.
X
XUne copie intigrale de l'article incrimini a iti envoyie ` son auteur.
X
XLe champ Path: de l'article d'annulation suit les conventions 'cyberspam' et
X'bincancel'. Si vous ne disirez pas honorer ces messages d'annulation il
Xvous est donc facile de les rejeter automatiquement.
X
XUne copie des entjtes a iti envoyie ` la liste <[email-masked]>.
XVous pouvez vous abonner ` cette liste en envoyant : subscribe fr-binaires
X						 ` : [email-masked]
X
X---- copie des entjtes de l'article ----
X%article-headers%---- fin des entjtes ----
END-of-sample.config/bin.p-expl-frud
echo x - sample.config/binkill.conf
sed 's/^X//' >sample.config/binkill.conf << 'END-of-sample.config/binkill.conf'
X# $Id: binkill.conf,v 1.3 1996/09/17 20:34:13 pb Exp $
X#
X# To configure this to your system, copy this file as newsbot.conf,
X# then look for and adapt the following strings:
X#
X#	USER@I_NEED_TO_CONFIGURE_NEWSBOT
X#			(for Approved:, X-Canceled-By: and possibly From:)
X#			-> a mail address to reach you
X#
X#	USER		(end of Path: for generated cancels)
X#			-> something nice. 'not-for-mail' if you like.
X#
X#	rnews		-> path to your rnews
X#	uuencode	-> path to your uuencode
X#	sendmail	-> path to your sendmail
X#	Mail		-> path of some mail user agent
X#
X#	frcheck!	-> feed name in your 'newsfeeds' file
X#
X# If you are issuing cancels in the fr.* hierarchy, uncomment the last line
X# (it's used to mail reports for later posting to fr.usenet.divers).
X#
X#####
X#
X# Pattern matching on article headers
X
X# test if an article is approved
X~ notappr	Approved			^$
X
X#####
X#
X# Patterns (as used in actions)
X# Patterns not declared here are read from files with the same name
X# in the pattern directory.
X#
X# First line begins with 'P' followed by the pattern name
X# Following lines begin with '='.
X#
X
X#####
X#
X# Pattern for global cancel
X#
XPbin.cancel-global
X=Path: cyberspam!bincancel!USER
X=Newsgroups: %header-newsgroups%
X=From: %header-from%
X=%if-header-sender%Sender: %header-sender%
X=%endif%%if-header-approved%Approved: USER@I_NEED_TO_CONFIGURE_NEWSBOT
X=%endif%Subject: cancel: %header-subject%
X=Message-Id: %cancel-message-id%
X=Control: cancel %header-message-id%
X=Date: %date%
X=X-Canceled-By: USER@I_NEED_TO_CONFIGURE_NEWSBOT
X=X-Bot: %newsbot-version%
X=
X=%article-size% bytes binary in discussion group (%article-encoding%)
X
X#####
X#
X# Pattern for cancel reports sent to [email-masked]
X#
XPlog.headers
X=%article-size%
X=%article-headers%
X
X#####
X#
X# Pattern for mail copy to sender (disabled)
X#
XPbin.mcc-sender-fr
X=From: USER@I_NEED_TO_CONFIGURE_NEWSBOT
X=To: %moderated-sender%
X=Subject: %re-subject%
X=Precedence: junk
X=Organization: Usenet canal historique
X=X-Bot: %newsbot-version%
X=Mime-Version: 1.0
X=Content-Type: text/plain;charset=ISO-8859-1
X=Content-Transfer-Encoding: 8bit
X=
X=It seems you have posted a %article-size% bytes binary file as
X=article %header-message-id%.
X=
X=This article has been autocancelled, as one or more of the groups you
X=posted to are general newsgroups, were binaries should not be posted.
X=
X=Please use the appropriate hierarchy, for example alt.binaries.*, and
X=don't crosspost.
X=
X=This message has been automatically generated.
X=
X=---
X=Vous avez apparemment envoyi un fichier binaire de %article-size% octets
X=sous l'identification %header-message-id%.
X=
X=L'article a iti annuli, car un ou plusieurs des groupes destinataires ne
X=sont pas destinis aux binaires. Veuillez utiliser la hiirarchie appropriie,
X=par exemple alt.binaries.*, et ne pas faire de crossposts.
X=
X=Ce message a iti giniri automatiquement.
X=
X=---- copy of article follows ----
X=%article%---- end of article ----
X
X#####
X#
X# Pattern for header copy to newsmaster (disabled)
X#
XPbin.mh-newsmaster
X=From: newsmaster
X=To: newsmaster
X=Subject: binary file: %header-subject%
X=Organization: Usenet canal historique
X=X-Bot: %newsbot-version%
X=Mime-Version: 1.0
X=Content-Type: text/plain;charset=ISO-8859-1
X=Content-Transfer-Encoding: 8bit
X=
X=%article-size% bytes binary file detected as article %header-message-id%.
X=
X
X#####
X#
X# Pattern for header copy to fr-binaires
X#
XPbin.mch-frbin
X=From: newsmaster
X=To: [email-masked]
X=Subject: BINAIRE dans fr.* : %header-subject%
X=Organization: Usenet canal historique
X=X-Bot: %newsbot-version%
X=Mime-Version: 1.0
X=Content-Type: text/plain;charset=ISO-8859-1
X=Content-Transfer-Encoding: 8bit
X=
X=Un article de %article-size% octets vient d'jtre ditecti
X=ici dans un groupe de fr.*.
X=
X=L'article a iti annuli, et un article explicatif imis dans fr.usenet.divers.
X=
X=---- copie des entjtes ----
X=%article-headers%---- fin des entjtes ----
X=
X=---- copy of article headers follows ----
X=%article-headers%---- end of article headers ----
X
X#####
X# Actions
X#
X# A ">" in front of "pipe to" means to append to the indicated file.
X# actions		pattern file		pipe to
X#
X
X# send full article to sender and warn for cancel regarding bin
X# (disabled)
XAbin.mcc-sender-fr	bin.mcc-sender-fr	/usr/sbin/sendmail -t -oi
X# send headers to local newsmaster regarding bin
X# (disabled)
XAbin.mh-newsmaster	bin.mh-newsmaster	/usr/sbin/sendmail -t -oi
X# send headers to [email-masked] regarding bin
X# (disabled)
XAbin.mch-frbin		bin.mch-frbin		/usr/sbin/sendmail -t -oi
X
X# send uuencoded copy of headers to [email-masked]
X# USE ONLY IF ISSUING CANCELS OVER fr.* !
XAmail.fr.bin	log.headers	/usr/bin/uuencode cancel|/usr/bin/Mail [email-masked]
X
X# Global bincancel
XAbin.cancel-global	bin.cancel-global	/bin/rnews
X
X###################
X#
X# Checks over fr.*
X#
XFfrcheck!
X
X# (This part is disabled)
X#
X# If over 3k, binary and valid-looking return address (condition F),
X# warn and send a copy to sender with explanations.
X#I>3k,B,F		bin.mcc-sender-fr
X# headers to local admin
X#I>3k,B			bin.mh-newsmaster
X# mail explanation in french to [email-masked]
X#I>3k,B			bin.mch-frbin
X
X# emit a global cancel if over 3k, binary, and not approved
XI>3k,B,~notappr		bin.cancel-global
X# then send a cancel report to [email-masked]
X# ENABLE ONLY IF ISSUING CANCELS OVER fr.* !
X#I>3k,B,~notappr		mail.fr.bin
END-of-sample.config/binkill.conf
echo x - sample.config/dup.cancel-local
sed 's/^X//' >sample.config/dup.cancel-local << 'END-of-sample.config/dup.cancel-local'
XPath: cyberspam!spewcancel!user
XNewsgroups: %header-newsgroups%
XFrom: %header-from%
X%if-header-sender%Sender: %header-sender%
X%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT
X%endif%Subject: cancel: %header-subject%
XMessage-Id: %cancel-message-id%
XControl: cancel %header-message-id%
XDate: %date%
XX-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT
XX-Bot: %newsbot-version%
XDistribution: local
X
XLocal cancel, duplicate article.
END-of-sample.config/dup.cancel-local
echo x - sample.config/dup.mh-sender
sed 's/^X//' >sample.config/dup.mh-sender << 'END-of-sample.config/dup.mh-sender'
XFrom: user@I_NEED_TO_CONFIGURE_NEWSBOT (My Full Name)
XTo: %moderated-sender%
XSubject: %re-subject%
XPrecedence: junk
XOrganization: Usenet canal historique
XX-Bot: %newsbot-version%
XMime-Version: 1.0
XContent-Type: text/plain;charset=ISO-8859-1
XContent-Transfer-Encoding: 8bit
X
XIt seems you have posted a duplicate news article as article
X%header-message-id%.
X
XThis article is exactly identical to another article you have just
Xposted in the same group(s). Posting duplicate articles is generally
Xa waste of network resources and makes newsgroups more difficult
Xto read.
X
XThis might be due to your using flawed software. Netscape, for
Xexample, is typically known to post duplicate messages to Usenet
Xvery easily.
X
XIf you can, please cleanup this article by using the "CANCEL" option
Xof your newsreader. Old versions of Netscape don't have this feature.
X
XWhen you post an article, if the sending results in an error message:
X	- cancel the article, just in case
X	  (it may have been posted anyway);
X	- resend it.
X
XArticles just sent are not always immediately visible in your
Xnewsreader, there can be a delay ranging from a few seconds to a
Xfew minutes.
X
XThanks.
X
XThis message has been automatically generated.
X
X---- copy of article headers follows ----
X%article-headers%---- end of article headers ----
END-of-sample.config/dup.mh-sender
echo x - sample.config/from.cancel-local
sed 's/^X//' >sample.config/from.cancel-local << 'END-of-sample.config/from.cancel-local'
XPath: cyberspam!user
XNewsgroups: %header-newsgroups%
XFrom: %header-from%
X%if-header-sender%Sender: %header-sender%
X%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT
X%endif%Subject: cancel: %header-subject%
XMessage-Id: %cancel-message-id%
XControl: cancel %header-message-id%
XDate: %date%
XX-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT
XX-Bot: %newsbot-version%
XDistribution: local
X
XLocal cancel, bad From:/Reply-To: field.
END-of-sample.config/from.cancel-local
echo x - sample.config/html.mh-sender-fr
sed 's/^X//' >sample.config/html.mh-sender-fr << 'END-of-sample.config/html.mh-sender-fr'
XFrom: user@I_NEED_TO_CONFIGURE_NEWSBOT (My Full Name)
XTo: %moderated-sender%
XSubject: %re-subject%
XPrecedence: junk
XOrganization: Usenet canal historique
XX-Bot: %newsbot-version%
XMime-Version: 1.0
XContent-Type: text/plain;charset=ISO-8859-1
XContent-Transfer-Encoding: 8bit
X
XVous avez apparemment envoyi sur Usenet un article au format HTML sous
Xl'identification %header-message-id%.
X
XLe HTML est un format congu pour les documents hypertexte et notamment
Xle World Wide Web. Il ne s'agit pas d'un format standard pour les news.
X
XLe HTML n'est pas lisible par la plupart des logiciels de news
Xet il est donc conseilli de l'iviter sur Usenet si vous voulez que
Xtous les utilisateurs puissent vous lire.
X
XSi vous le disirez, vous pouvez annuler cet article en utilisant l'option
Xadiquate de votre logiciel de lecture de news.
X
XCe message a iti giniri automatiquement.
X
X---- entjtes de l'article ----
X%article-headers%---- fin des entjtes ----
END-of-sample.config/html.mh-sender-fr
echo x - sample.config/log.article
sed 's/^X//' >sample.config/log.article << 'END-of-sample.config/log.article'
X%article%
END-of-sample.config/log.article
echo x - sample.config/log.headers
sed 's/^X//' >sample.config/log.headers << 'END-of-sample.config/log.headers'
X%article-size%
X%article-headers%
END-of-sample.config/log.headers
echo x - sample.config/newsbot.conf
sed 's/^X//' >sample.config/newsbot.conf << 'END-of-sample.config/newsbot.conf'
X# $Id: newsbot.conf,v 1.7 1996/09/17 20:34:15 pb Exp $
X#
X# This is intended purely as a SAMPLE config, DO NOT use "as is" !
X#
X#####
X# Customize header fields to keep track of (used in header matching)
X# Builtin (already known) headers are:
X#
X#	From
X#	Subject
X#	Newsgroups
X#	Message-Id
X#	Date
X#	References
X#	Sender
X#	Reply-To
X#	Approved
X#
XHContent-Type
XHContent-Transfer-Encoding
X
X#####
X# Pattern matching on article headers
X#
X# options:
X#	e	complete regular expression (else, simple substring match
X#		with special characters ^ and $ recognized)
X#	i	ignore case
X#
X# options name	header field			string to look for
X#						(empty == field not present)
X#
X~i qp		Content-Transfer-Encoding	^quoted-printable$
X~i html		Content-Type			html
X~ notappr	Approved			^$
X~ bbs		From				@brokenbbs.org
X
X# Pattern matching on article body
X# Same options as above except the header name is replaced by a line range.
X# Currently, only 'integer' or 'integer-integer' are recognized forms.
X
XL spew		1				^@FROM   :
XL mmf		1-60				CASH
X
X#####
X#
X# Patterns (as used in actions)
X# Patterns not declared here are read from files with the same name
X# in the pattern directory.
X#
X# First line begins with 'P' followed by the pattern name
X# Following lines begin with '='.
X#
X
XPbin.cancel-global
X=Path: cyberspam!bincancel!user
X=Newsgroups: %header-newsgroups%
X=From: %header-from%
X=%if-header-sender%Sender: %header-sender%
X=%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT
X=%endif%Subject: cancel: %header-subject%
X=Message-Id: %cancel-message-id%
X=Control: cancel %header-message-id%
X=Date: %date%
X=X-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT
X=X-Bot: %newsbot-version%
X=
X=%article-size% bytes binary in discussion group (%article-encoding%)
X
XPlog.article
X=%article%
X
X#####
X# Actions
X#
X# A ">" in front of "pipe to" means to append to the indicated file.
X# actions		pattern file		pipe to
X#
X
X# send headers to sender regarding bin
XAbin.mh-sender-fr	bin.mh-sender-fr	/usr/sbin/sendmail -t -oi
X# send full article to sender and warn for cancel regarding bin
XAbin.mcc-sender-fr	bin.mcc-sender-fr	/usr/sbin/sendmail -t -oi
X# send headers to local newsmaster regarding bin
XAbin.mh-newsmaster	bin.mh-newsmaster	/usr/sbin/sendmail -t -oi
X# send headers to [email-masked] regarding bin
XAbin.mch-frbin		bin.mch-frbin		/usr/sbin/sendmail -t -oi
X
X# Post explanation to fr.usenet.divers about bincancel
XAbin.p-expl-frud	bin.p-expl.frud		/usr/local/news/inews -h
X
X# Local bincancel
XAbin.cancel-local	bin.cancel-local	/bin/rnews
X# Global bincancel
XAbin.cancel-global	bin.cancel-global	/bin/rnews
X
X# send headers to local newsmaster regarding spew
XAspew.mh-newsmaster	spew.mh-newsmaster	/usr/sbin/sendmail -t -oi
X# Global spewcancel
XAspew.cancel-global	spew.cancel-global	/bin/rnews
X
X# send headers to sender and explanation regarding dups
XAdup.mh-sender		dup.mh-sender		/usr/sbin/sendmail -t -oi
X
X# send headers to sender and french explanation regarding use of QP
XAqp.mh-sender-fr	qp.mh-sender-fr		/usr/sbin/sendmail -t -oi
X# send headers to sender and french explanation regarding use of HTML
XAhtml.mh-sender-fr	html.mh-sender-fr	/usr/sbin/sendmail -t -oi
X
X# Local dupcancel
XAdup.cancel-local	dup.cancel-local	/bin/rnews
X
X# Logs
XAlog.fr.badfrom		log.headers	>/usr/local/news/newsbot/log/fr.badfrom
XAlog.fr.badref		log.headers	>/usr/local/news/newsbot/log/fr.badref
XAlog.fr.bin		log.headers	>/usr/local/news/newsbot/log/fr.bin
XAlog.fr.dup		log.headers	>/usr/local/news/newsbot/log/fr.dup
XAlog.fr.html		log.headers	>/usr/local/news/newsbot/log/fr.html
XAlog.fr.qp		log.headers	>/usr/local/news/newsbot/log/fr.qp
X
XAlog.badfrom		log.headers	>/usr/local/news/newsbot/log/badfrom
XAlog.badref		log.headers	>/usr/local/news/newsbot/log/badref
XAlog.bin		log.headers	>/usr/local/news/newsbot/log/bin
XAlog.dup		log.headers	>/usr/local/news/newsbot/log/dup
XAlog.html		log.headers	>/usr/local/news/newsbot/log/html
XAlog.qp			log.headers	>/usr/local/news/newsbot/log/qp
X
X###################
X#
X# Conditions:
X#
X# >size bigger than size
X# B	is a binary
X# D	duplicate message
X# F	from ok
X# R	references ok
X# Msize	estimated multipart size bigger than 'size'.
X# M	multipart
X# ~name	named expression matches (matches on headers)
X# Lname	named expression matches (matches on article body)
X#
X
X###################
X#
X# actions on "default" feed (all articles fed to us by INN) go there
X#
X# (none)
X
X###################
X#
X# Checks over fr.*
X#
XFfrcheck!
X
X###################
X# Warn and log on HTML and QP
X#
X
X# Warn by mail in french if Content-Transfer-Encoding is QP
XI~qp,F,~notappr		qp.mh-sender-fr
XI~qp			log.fr.qp
X
X# Warn by mail in french if Content-Type is html
XI~html,F,~notappr	html.mh-sender-fr
XI~html			log.fr.html
X
X# other logs for fr.* stats
XID			log.fr.dup
XI!F			log.fr.badfrom
XI!R			log.fr.badref
X
X###################
X# Binary over 15k
X
X# If return address looks valid, warn and send a copy to sender with
X# explanations in french about bincancel
XI>15k,B,F		bin.mcc-sender-fr
X# headers to local admin
XI>15k,B			bin.mh-newsmaster
X# post explanation in french to fr.usenet.divers
X,			bin.p-expl-frud
X# mail explanation in french to [email-masked]
X,			bin.mch-frbin
X# log headers
X,			log.fr.bin
X# global bincancel if not approved
X,~notappr		bin.cancel-global
X# don't continue
XI>15k,B			stop
X
X# Spew cancel
XI~bbs,Lspew		spew.cancel-global
X,			bin.mh-newsmaster
X
X###################
XFnotfrcheck!
X# Warn sender on duplicate posts and log for report
X#ID,F,~notappr		dup.mh-sender
XID			log.dup
X# Bad From:
XI!F			log.badfrom
X# Bad References:
XI!R			log.badref
X# QP
XI~qp			log.qp
X# HTML
XI~html			log.html
END-of-sample.config/newsbot.conf
echo x - sample.config/qp.mh-sender-fr
sed 's/^X//' >sample.config/qp.mh-sender-fr << 'END-of-sample.config/qp.mh-sender-fr'
XFrom: user@I_NEED_TO_CONFIGURE_NEWSBOT (My Full Name)
XTo: %moderated-sender%
XSubject: %re-subject%
XPrecedence: junk
XOrganization: Usenet canal historique
XX-Bot: %newsbot-version%
XMime-Version: 1.0
XContent-Type: text/plain;charset=ISO-8859-1
XContent-Transfer-Encoding: 8bit
X
XVous avez apparemment envoyi sur Usenet un article encodi en "quoted-printable"
Xsous l'identification %header-message-id%.
X
XLe "quoted-printable" est un format congu pour le courrier ilectronique.
XIl ne s'agit pas d'un format standard pour les news.
X
XLe "quoted-printable" n'est pas lisible par la plupart des logiciels
Xde news et il est donc conseilli de l'iviter sur Usenet si vous
Xvoulez que tous les utilisateurs puissent vous lire.
X
XIl serait prifirable de configurer votre lecteur de news pour utiliser,
X` la place du "quoted-printable", l'encodage "8bit" universellement
Xaccepti.
X
XSi vous le disirez, vous pouvez annuler cet article en utilisant l'option
Xadiquate de votre logiciel de lecture de news.
X
XCe message a iti giniri automatiquement.
X
X---- entjtes de l'article ----
X%article-headers%---- fin des entjtes ----
END-of-sample.config/qp.mh-sender-fr
echo x - sample.config/spew.cancel-global
sed 's/^X//' >sample.config/spew.cancel-global << 'END-of-sample.config/spew.cancel-global'
XPath: cyberspam!spewcancel!user
XNewsgroups: %header-newsgroups%
XFrom: %header-from%
X%if-header-sender%Sender: %header-sender%
X%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT
X%endif%Subject: cancel: %header-subject%
XMessage-Id: %cancel-message-id%
XControl: cancel %header-message-id%
XDate: %date%
XX-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT
XX-Bot: %newsbot-version%
X
XSpew cancelled.
END-of-sample.config/spew.cancel-global
echo x - sample.config/spew.mh-newsmaster
sed 's/^X//' >sample.config/spew.mh-newsmaster << 'END-of-sample.config/spew.mh-newsmaster'
XFrom: newsmaster
XTo: newsmaster
XSubject: spew: %header-subject%
XOrganization: Usenet canal historique
XX-Bot: %newsbot-version%
XMime-Version: 1.0
XContent-Type: text/plain;charset=ISO-8859-1
XContent-Transfer-Encoding: 8bit
X
XSpew detected as article %header-message-id%.
X
X---- copy of article headers follows ----
X%article-headers%---- end of article headers ----
END-of-sample.config/spew.mh-newsmaster
exit

-- 
Pierre Beyssac	    [email-masked] [email-masked]
{Free,Net,Open}BSD, Linux : il y a moins bien, mais c'est plus cher
    Free domains: http://www.eu.org/ or mail [email-masked]