Path: vixie!news1.digital.com!nntp-hub2.barrnet.net!cpk-news-hub1.bbnplanet.com!newsfeed.internetmci.com!in2.uu.net!brainstorm.eu.org!frmug.fr.net!fasterix.frmug.fr.net!not-for-mail From: pb@fasterix.frmug.fr.net (Pierre Beyssac) Newsgroups: alt.sources Subject: newsbot - instantly grep and trigger actions on news articles Followup-To: poster Date: 17 Sep 1996 23:19:40 +0200 Organization: considered harmful Lines: 3558 Distribution: inet Message-ID: <51n4lc$1gr@fasterix.frmug.fr.net> NNTP-Posting-Host: fasterix.frmug.fr.net Mime-Version: 1.0 Content-Type: text/plain;charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Keywords: Usenet news grep cancel binary Xref: vixie alt.sources:7717 Archive-name: newsbot-1.2.1 Submitted-by: pb@fasterix.freenix.fr Newsbot reads article headers fed to it by INN, checks a series of conditions on these headers or on article bodies, and can trigger a series of actions if the conditions are met. # This is a shell archive. Save it in a file, remove anything before # this line, and then unpack it by entering "sh file". Note, it may # create directories; files and directories will be owned by you and # have default permissions. # # This archive contains: # # ChangeLog # Makefile # README # newsbot.c # sample.config # sample.config/bin.cancel-global # sample.config/bin.cancel-local # sample.config/bin.mcc-sender-fr # sample.config/bin.mch-frbin # sample.config/bin.mh-newsmaster # sample.config/bin.mh-sender-fr # sample.config/bin.p-expl-frud # sample.config/binkill.conf # sample.config/dup.cancel-local # sample.config/dup.mh-sender # sample.config/from.cancel-local # sample.config/html.mh-sender-fr # sample.config/log.article # sample.config/log.headers # sample.config/newsbot.conf # sample.config/qp.mh-sender-fr # sample.config/spew.cancel-global # sample.config/spew.mh-newsmaster # echo x - ChangeLog sed 's/^X//' >ChangeLog << 'END-of-ChangeLog' X$Id: ChangeLog,v 1.8 1996/09/17 20:35:36 pb Exp $ X XTue Sep 17 22:34:22 MET DST 1996 *** 1.2.1 released X - misc changes in README and config files X - ported to HP-UX 9.x by Pierre David X XMon Sep 16 23:59:02 MET DST 1996 *** 1.2 released X - improved README X - new minimal config file ready for binary detection X - new format for pattern declaration in config file: 'P' X - bug fix of hash function on 8 bit chars... X XSat Sep 14 17:00:50 MET DST 1996 X - new keyword %newsbot-version% to include ident string. X XSat Sep 14 11:44:40 MET DST 1996 *** 1.1.1 released. X - fix to body pattern matching X XFri Sep 13 19:50:13 MET DST 1996 *** 1.1 released. X X - new option "L" to match patterns in body. X - ported to SunOS 4. X - misc cleanups. X XThu Sep 12 01:20:33 MET DST 1996 *** 1.0.2 released. END-of-ChangeLog echo x - Makefile sed 's/^X//' >Makefile << 'END-of-Makefile' X# $Id: Makefile,v 1.8 1996/09/17 20:34:02 pb Exp $ X# X# Makefile Makefile for newsbot X# Targets: all compiles everything X# install installs the binaries X# (not the example configuration files) X# clean cleans up X# X X# X# Edit the following paths to suit your configuration X# X X# Your news spool XPATHSPOOL = /var/spool/news X# Where you want to put newsbot config file XPATHCONF = /usr/local/news/newsbot.conf X# Where you want to put newsbot pattern files XPATHPATS = /usr/local/news/newsbot X# Facility for syslog. XLOGFAC = LOG_LOCAL6 X X# The following are only needed for "make install" X# X# Where you want to install newsbot XBIN_DIR = /usr/local/news/bin X# Where is ctlinnd XCTLINND_DIR = /usr/local/news/bin X X# X# Comment-out the lines for your system X# X X# SunOS 4.x X# you need to install a Posix Regexp package (GNU rx for example) X# X#LDFLAGS = -L/usr/local/lib -lregex X#INCL = -I/usr/local/include X#CC = gcc X#CFLAGS = -Wall -g X X# FreeBSD, NetBSD, Linux X# X#LDFLAGS = X#INCL = X#CC = gcc X#CFLAGS = -Wall -g X X# HP-UX 9.x X# X#LDFLAGS = X#INCL = X#CC = cc -Ae X#CFLAGS = -g X XPROGS = newsbot X XBIN_OWNER = news XBIN_GROUP = news XBIN_COMBO = $(BIN_OWNER).$(BIN_GROUP) XINSTALL = install -o $(BIN_OWNER) -g $(BIN_GROUP) X XDOPTS = -D_PATH_SPOOL=\"$(PATHSPOOL)\" \ X -D_PATH_CONF=\"$(PATHCONF)\" \ X -D_PATH_DIRPATS=\"$(PATHPATS)\" \ X -DLOG_FACILITY=$(LOGFAC) X Xall: $(PROGS) X Xnewsbot: newsbot.c X $(CC) $(INCL) $(DOPTS) $(CFLAGS) -o $@ newsbot.c $(LDFLAGS) X Xcleanobjs: X rm -f *.o *.bak X Xclean: cleanobjs X @echo Type \"make clobber\" to really clean up. X Xclobber: cleanobjs X rm -f $(PROGS) X Xinstall: $(PROGS) X -mv -f $(BIN_DIR)/newsbot $(BIN_DIR)/newsbot.old X $(INSTALL) -m 755 newsbot $(BIN_DIR) X @echo "newsbot installed.....Don't forget to restart it, as user $(BIN_OWNER):" X @echo X @echo "$(CTLINND_DIR)/ctlinnd begin 'NEWSBOT!'" X @echo END-of-Makefile echo x - README sed 's/^X//' >README << 'END-of-README' X$Id: README,v 1.9 1996/09/17 20:34:03 pb Exp $ X X1. What is newsbot ? X XNewsbot reads article headers fed to it by INN, checks a series of Xconditions on these headers or on article bodies, and can trigger Xa series of actions if the conditions are met. X XThe latest newsbot release will generally be available on: X X ftp://frmug.org/newsbot/ X http://frmug.org/newsbot/ X X2. Compiling and installing newsbot X XNewsbot has been running both on FreeBSD and Linux systems for months. XIt compiles just fine on: X X FreeBSD X NetBSD X SunOS 4.x X Linux X HP-UX 9.x X XYou might need to install a Posix regexp library (GNU rx for example) Xif you don't have one on your system. X XTo compile newsbot, there is now a Makefile, thanks to Christian XPerrier (I was too lazy to write one...). Just edit it to configure Xyour system-dependent paths and options, then: X X make X XTo install and configure newsbot, have a look at the sample config Xfiles and the following explanations. X XUpdating the binary can be done with a: X X make install X X3. Configuring newsbot X XYou can skip this and go directly to section 4 if you only want to Xdo binary detection and cancelling. X XNewsbot reads its configuration at startup, then reads article Xheaders (and possibly funnel feed indications) fed to it by INN, Xchecks a series of conditions on these headers or on article bodies, Xand can then trigger a series of actions if some conditions are Xmet. X XAll of the configuration goes into the newsbot.conf file. X XAdditionnal templates can be placed either in the pattern directory X(_PATH_DIRPATS) or in newsbot.conf itself. X XA condition can be a pattern matching on a header or on a body Xline, a encoded binary article body, a size, an invalid From: Xaddress, etc... or a conjunction or these: X X >size bigger than size X size is an integer optionally followed by multiplier X k (1024) of M (1024^2). X X B article is encoded binary X X D duplicate message X Same contents and From:/Newsgroups:/Subject: X as a previous article in the last 1000 received. X X F From: ok X At least one @ with at least one char before and X at least one char then one dot after X X R references ok X Subject: header does not begin in Re:, or does X begin in Re: and a References: header is present X X Msize estimated multipart size bigger than 'size' X M multipart X This condition looks for two numbers in the subject, X and tries to determine if it is a multipart article, X and its global size. X X ~name named expression matches X Pattern matching on headers. X "name" must be the name of a previously declared X header pattern, of the form: X X ~[ei] name header-name regexp-or-substring X X the [ei] part is optional. X X Option e means that the last argument on the line X is a complete regular expression, else only ^ (beg. X of line) and $ (end of line) are recognized to X anchor the search. X X Option i means that the match must be case-insensitive. X X Headers on which pattern matching must be done have X to be previously declared with 'H', for example: X X HContent-Type X HContent-Transfer-Encoding X X Lname named expression matches X Pattern matching on article body. X "name" must be the name of a previously declared X body pattern, of the form: X X L[ei] name linerange regexp-or-substring X X The meaning is exactly the same as above, except X that the linerange indicates where to look for the X pattern in the article body, and can only be a X single integer (for a unique line) or two integers X separated by a '-' (for that range, including the X boundaries). X XAn action consists in reading a template, substituting % variables, Xand piping the output to a program or another file. An action is Xdeclared as follows: X XAname template-name pipe-to X Xtemplate-name can be the name of a previously declared template, Xor a relative file name (it will only be found in the pattern Xdirectory _PATH_DIRPATS). X XIn a template, the recognized expansions are: X X %article-headers% article headers X %article% full article X %article-encoding% binary encoding name. X %article-size% article size in bytes X %article-sender% sender (Reply-To: if found, else From:) X %moderated-sender% as above, but use Approved: if present. X %article-sender-host% as article-sender, cut after the @. X %header-XXX% any header (builtin or added with 'H') X %date% current date (GMT, RFC 822/1036 format) X %re-subject% subject prepended with Re: if necessary X %cancel-message-id% cancel message-id (Salz convention) X %truncated-message-id% message-id without the leading '<' X %20k action2 X Xwhich would test _every_ article for a binary encoding ! X XFinally, the feed separator 'F' allows you to specify different Xactions according to different feeds. To use it, you need to start Xnewsbot with option -f and configure INN with a funnel feed to Xnewsbot. X XFor example you can declare (in your newsfeeds file) feeds 'frcheck!' Xand 'all!' both going to funnel 'NEWSBOT!', and in your newbot.conf: X XFfrcheck! XI>100k action1 X XFall! XI>200k action2 X X4. Simple config for binary detection X XYou just need to take file binkill.conf, edit it CAREFULLY to taste, Xthen copy it as /usr/local/news/newsbot.conf (or any other place Xyou specify in the Makefile). X X5. Starting newsbot X XInstall newsbot with a "make install". X XEdit your newsfeeds file to add something like: X X# newsbot X# Xfrcheck!:fr.*:Tm:NEWSBOT! X# ^^^^ adapt to taste X# XNEWSBOT!:!*:Tc,W*H:\ X /usr/local/news/bin/newsbot -f X# ^^^^^^^^^^^^^^^^^^^ adapt to your setup X XNOTE: 'frcheck!' depends on the name you have used in your config. XIt's the default name for the binary detection config provided in Xbinkill.conf. X XThen start the feeds: X $ ctlinnd begin NEWSBOT\! X $ ctlinnd begin frcheck\! X XHave a look at syslog output for the facility you have configured Xin (LOCAL6 by default) to check if newsbot is happy (it should Xissue a start message). X XLet newsbot run for a moment. It will only log, not do, things. XIf everything seems correct, add option '-a' before '-f' in the XNEWSBOT! entry in the newsfeeds file, then restart newsbot for real X(ctlinnd begin NEWSBOT\!). X XCheck (at least) twice before using option -a, or you might generate Xa lot of mess and make a lot of people VERY, VERY angry. Then you Xwould bitterly regret it (I'm not kidding). X X5. That's all folks X XGood luck, and please use newsbot wisely ! X X pb@fasterix.freenix.fr (Pierre Beyssac) X XThanks to: X X Christian Perrier X for invaluable comments, testing, the Makefile, X a "netsend"-encoded sample... X X Vincent Archer X for the original canceler X X Ollivier Robert X for the RFC-checker X X Thomas Quinot X for the initial idea of body pattern matching X for spew detection X X Pierre David X HP-UX port and first major newsbot site END-of-README echo x - newsbot.c sed 's/^X//' >newsbot.c << 'END-of-newsbot.c' X/* X * $Id: newsbot.c,v 1.15 1996/09/17 20:34:05 pb Exp $ X * X * newsbot.c X * X * Author : Pierre Beyssac X * X * Tests a list of conditions on incoming articles. X * Pipes a pattern file to defined programs on matches. X * Can remove the article from local spool. X * X * The binary detector currently recognizes: X * uuencode, base64, binhex and netsend. X * X * Hack on rfc-checker by Ollivier Robert , X * itself derivated from cyberspam by Vincent Archer X * X * ---- X * Use the following syntax in the newsfeeds(5) file. X * DON'T USE THE -a OPTION UNLESS YOU ARE QUITE SURE YOUR CONFIG IS CORRECT ! X * Without -a, newsbot only logs the actions it would have done. X * X * - with a normal feed: X * NEWSBOT!:fr.*:Tc,WH: \ X * /usr/local/news/bin/newsbot [-a] \ X * [-d] [-C config-file] \ X * [-D pattern-dir] X * X * - with a funnel feed (note use of option -f) : X * NEWSBOT!:!*:Tc,W*H: \ X * /usr/local/news/bin/newsbot [-a] -f \ X * [-d] [-C config-file] \ X * [-D pattern-dir] X * X * ---- X * Example for a normal feed : X * X * NEWSBOT!:fr.*:Tc,WH: \ X * /usr/local/news/bin/newsbot -a X * X * ---- X * Example for a funnel feed : X * X * rfc-check!:*,!comp.mail.sendmail:Tm:NEWSBOT! X * X * from-check!:*:Tm:NEWSBOT! X * X * bin-check!:fr.*:Tm:NEWSBOT! X * X * NEWSBOT!:!*:Tc,W*H: \ X * /usr/local/news/bin/newsbot -a -f X * X * ---- X * In a pattern file, the recognized expansions are : X * %article-headers% article headers X * %article% full article X * %article-encoding% binary encoding name. X * %article-size% article size in bytes X * %article-sender% sender (Reply-To: if found, else From:) X * %moderated-sender% as above, but use Approved: if present. X * %article-sender-host% as article-sender, cut after the @. X * %header-XXX% any header (builtin or added with 'H') X * %date% current date (GMT, RFC 822/1036 format) X * %re-subject% subject prepended with Re: if necessary X * %cancel-message-id% cancel message-id (Salz convention) X * %truncated-message-id% message-id without the leading '<' X * % X#include X#include X#include X#include X#include X#include X#if !defined(__NetBSD__) && !defined(__FreeBSD__) X# include X#endif /* 4.4BSD */ X#include X#include X#include X X#ifdef sun Xint getopt(); Xextern char *optarg; Xextern int optind, opterr; X#endif /* SUNOS */ X X#ifndef MAXPATHNAMELEN X# define MAXPATHNAMELEN 1024 X#endif X X#define DEBUG X X#ifdef DEBUG X#include X# define MESSAGE(s) if (opt_debug) syslog (LOG_NOTICE, (s)) X# define MESSAGE_1(s,a) if (opt_debug) syslog (LOG_NOTICE, (s), (a)) X# define MESSAGE_2(s,a,b) if (opt_debug) syslog (LOG_NOTICE,(s),(a),(b)) X# define MESSAGE_3(s,a,b,c) if (opt_debug) syslog (LOG_NOTICE,(s),(a),(b),(c)) X# define MESSAGE_4(s,a,b,c,d) if (opt_debug) syslog (LOG_NOTICE,(s),(a),(b),(c),(d)) X# define MESSAGE_5(s,a,b,c,d,e) if (opt_debug) syslog (LOG_NOTICE,(s),(a),(b),(c),(d),(e)) X#else X# define MESSAGE(s) X# define MESSAGE_1(s,a) X# define MESSAGE_2(s,a,b) X# define MESSAGE_3(s,a,b,c) X# define MESSAGE_4(s,a,b,c,d) X# define MESSAGE_5(s,a,b,c,d,e) X#endif /* DEBUG */ X X#define MINLINE 80 X X/* For article hash table */ X#define HASHMSGSIZE 1337 Xtypedef unsigned short hashmsg_t; X X/* For header hash table */ X#define HASHHDRSIZE 31 Xtypedef unsigned short hashhdr_t; X X/* Hash function for headers (case-insensitive) */ X#define HASHFUN(s) ((unsigned)(((s)[0]&&(s)[1]&&(s)[2]) \ X ? (((s)[0]^(s)[1]^(s)[2]) & ~0x20) \ X : ((s)[0] & ~0x20)) \ X % HASHHDRSIZE) X X/* For article buckets */ X#define MAXARTS 1000 X X/* For predefined header table */ X#define MAXHEADERS 50 X X/* For matching patterns (should fit 1/bit in a long) */ X#define MAXMATCH 32 Xtypedef unsigned long matchflags_t; X X/* How many nested if/else/endif */ X#define MAXIFNEST 32 X X/* X * The first in the list will get index 0, the next 1, and so on. X * Both tables should be kept in sync. X */ Xchar *header_init_table[] = { X "Bytes", X "Control", X "Xref", X "From", X "Subject", X "Newsgroups", X "Message-Id", X "Date", X "References", X "Sender", X "Reply-To", X "Approved", X NULL X}; X X#define I_BYTES 0 X#define I_CONTROL 1 X#define I_XREF 2 X#define I_FROM 3 X#define I_SUBJECT 4 X#define I_NEWSGROUPS 5 X#define I_MESSAGE_ID 6 X#define I_DATE 7 X#define I_REFERENCES 8 X#define I_SENDER 9 X#define I_REPLY_TO 10 X#define I_APPROVED 11 X X/* Special flags for regex matching */ X#define M_CASE 1 X#define M_BEGLINE 2 X#define M_ENDLINE 4 X#define M_REGEX 8 X Xstruct article_t X{ X long bodysize; /* content size */ X long headersize; /* header size */ X X /* Checks */ X char *coding; X char refok; X X char fromok; X char multi; X char binary; X char duplicate; X matchflags_t hmatch_done, hmatch_val; X matchflags_t bmatch_done, bmatch_val; X X long parts; X X FILE *artfile; X char artfiletried; X X hashmsg_t hash; X char *hashfields; X X long firstbyte; X X char *h[MAXHEADERS]; X char control; X X char hbuf[16384]; X}; X X#define T_TRUE 1 X#define T_FALSE 0 X#define T_DONTCARE -1 X Xstruct pattern_t { X struct pattern_t *next; X char *name; /* pattern name */ X char *val; /* pattern value. X * If NULL, use the file indicated X * by the name above. X */ X int valsize; /* Currently allocated size for val */ X char *curpos; /* If not a file, keep current position */ X FILE *file; /* If a file, use this */ X}; X Xstruct action_t { X struct action_t *next; X char *name; /* action name */ X struct pattern_t *pattern; /* associated pattern */ X char *pipeto; /* what program to pipe to */ X}; X Xstruct cond_t { X struct cond_t *next; /* next condition to check */ X X /* Comparison conditions */ X long size; /* min size */ X long multisize; /* multi min size */ X char checksize; /* check size */ X char checkmultisize; /* check multi and size */ X X /* Boolean conditions */ X char binary; /* check if binary */ X char fromok; /* check if correct from */ X char refok; /* check if correct refs */ X char duplicate; /* check if duplicate post */ X X /* Matches on headers */ X matchflags_t hmatch_totest, hmatch_val; X X /* Matches on body */ X matchflags_t bmatch_totest, bmatch_val; X X struct action_t *action; /* action to take */ X struct cond_t *and; /* more conditions to check after action */ X}; X X/* Description of a pattern matching */ Xstruct match_t { X char *name; /* name */ X X char flags; /* option flags */ X char *expr; /* expression to test */ X regex_t re; /* reg. expr (if option M_REGEX) */ X}; X X/* Description of a pattern matching table */ Xstruct matchtable_t { X int free; /* First free */ X int max; /* Max number */ X struct match_t *array; /* Array */ X}; X X/* Linked list of actions to apply to a named feed */ Xstruct feed_t { X struct feed_t *next; X char *name; /* feed name (from 'newsfeeds') */ X struct cond_t *firstcond; /* head of condition list */ X struct cond_t *lastcond; /* tail (used only when reading config file) */ X}; X X/* Article buckets are used to save recent articles and detect duplicates */ Xstruct art_bucket_t { X struct art_bucket_t *next; /* next article with same hash value */ X hashmsg_t hash; /* hash value */ X long bodysize; /* contents size */ X char *fields; /* file path and fields */ X}; X X/* Entry in article hash table */ Xstruct art_list_t { X struct art_bucket_t *first, *last; X}; X X/* Header description */ Xstruct header_desc_t { X struct header_desc_t *next; X int index; /* index to table */ X const char *name; /* name */ X}; X X/* Range description */ Xstruct range_t { X struct range_t *next; X int first, last; /* Number of first and last line */ X}; X Xint firstart, freeart; Xstruct art_bucket_t artb[MAXARTS]; X Xint freehdr = 0; Xstruct header_desc_t hdrb[MAXHEADERS]; X X/* Header match table */ Xstruct match_t hmatch[MAXMATCH]; Xstruct matchtable_t hmt = { 0, MAXMATCH, hmatch }; Xint hmatch_index[MAXMATCH]; /* for each header match, index of header X to test in header array */ X X/* Body match table */ Xstruct match_t bmatch[MAXMATCH]; Xstruct matchtable_t bmt = { 0, MAXMATCH, bmatch }; Xint bmatch_beg[MAXMATCH]; /* for each body match, line range to test */ Xint bmatch_end[MAXMATCH]; X Xstruct header_desc_t *hdrtable[HASHHDRSIZE]; Xstruct art_list_t msgtable[HASHMSGSIZE]; X Xstatic char opt_act = 0; /* -a : active mode enabled */ Xstatic char opt_debug = 0; Xstatic char opt_feed = 0; /* -f : enable multifeed */ X Xstatic const char *path_conf = _PATH_CONF; Xstatic const char *path_pats = _PATH_DIRPATS; X X/* Feed list */ Xstatic struct feed_t *firstfeed = NULL; X X/* Entry for default feed (not in feed list; gets all articles) */ Xstatic struct feed_t nullfeed = { NULL, "default", NULL }; X X/* Current feed name during article processing */ Xstatic char *curfeedname = NULL; X X/* Builtin initial action list */ Xstruct action_t actnop = { NULL, "nop" }; Xstruct action_t actremove = { &actnop, "remove" }; Xstruct action_t actstop = { &actremove, "stop" }; X Xstatic struct action_t *firstact = &actstop; X X/* Pattern list */ Xstatic struct pattern_t *firstpat = NULL; X Xvoid *zmalloc(size_t size) X{ X void *p = malloc(size); X if (p == NULL) { X syslog(LOG_NOTICE, "could not malloc %d bytes", size); X exit(1); X } X return p; X} X Xvoid *zrealloc(void *p, size_t size) X{ X p = realloc(p, size); X if (p == NULL) { X syslog(LOG_NOTICE, "could not realloc %d bytes", size); X exit(1); X } X return p; X} X Xvoid Xartb_init() X{ X firstart = 0; X freeart = MAXARTS; X} X Xint Xheader_insert(const char *h) X{ X unsigned short hv; X struct header_desc_t *hd, *ht; X X hv = HASHFUN(h); X X for (hd = hdrtable[hv]; hd; hd = hd->next) X if (!strcasecmp(hd->name, h)) X return hd->index; X X if (freehdr == MAXHEADERS) { X syslog(LOG_NOTICE, "header table too small (%d entries)", MAXHEADERS); X exit(1); X } X X ht = hdrb + freehdr; X ht->index = freehdr; X ht->name = h; X X ht->next = hdrtable[hv]; X hdrtable[hv] = ht; X X return freehdr++; X} X Xint Xheader_to_index(const char *h) X{ X unsigned short hv; X struct header_desc_t *hd; X X hv = HASHFUN(h); X X for (hd = hdrtable[hv]; hd; hd = hd->next) { X if (!strcasecmp(hd->name, h)) X return hd->index; X } X return -1; X} X Xvoid Xheader_init() X{ X char **p = header_init_table; X while (*p) header_insert(*p++); X} X X/* X * Find a pattern by name X * Return pointer to it, allocate a new if not found. X */ X Xstruct pattern_t * Xpattern_find(char *name) X{ X char *pfname; X struct pattern_t *pf; X for (pf = firstpat; pf; pf = pf->next) { X if (!strcasecmp(name, pf->name)) { X return pf; X } X } X pf = zmalloc(sizeof(struct pattern_t)); X X pfname = zmalloc(strlen(name)+1); X strcpy(pfname, name); X X pf->name = pfname; X pf->val = NULL; X pf->file = NULL; X pf->next = firstpat; X X firstpat = pf; X return pf; X} X X/* X * Open a pattern. X * Return 0 if OK, else -1. X */ X Xint Xpattern_open(struct pattern_t *p) X{ X if (p->val == NULL) { X /* It's a file... */ X if (p->file == NULL) { X char buff[8192]; X strcpy(buff, path_pats); X strcat(buff, "/"); X strcat(buff, p->name); X X p->file = fopen(buff, "r"); X return p->file ? 0:-1; X } else { X /* Already open: rewind */ X rewind(p->file); X } X } else { X /* It's in memory */ X p->curpos = p->val; X } X return 0; X} X Xvoid Xpattern_close(struct pattern_t *p) X{ X if (p->val == NULL) { X fclose(p->file); X p->file = NULL; X } X} X X/* Append a string to a in-memory pattern */ X Xvoid Xpattern_append(struct pattern_t *p, char *s) X{ X int size = strlen(s); X int usedsize; X X if (p->val == NULL) { X p->valsize = size + 1; X p->val = zmalloc(size+1); X strcpy(p->val, s); X return; X } X usedsize = strlen(p->val)+1; X if (usedsize+size > p->valsize) { X p->valsize = usedsize + size; X p->val = zrealloc(p->val, p->valsize); X } X strcpy(p->val + usedsize - 1, s); X} X X/* X * Read up to buffsize bytes in pattern. X * X * Returns a NULL-ended string in buff and 0 if OK, or -1 if error. X */ X Xint Xpattern_read(struct pattern_t *p, char *buff, int buffsize) X{ X if (p->val == NULL) { X /* It's a file */ X return fgets(buff, buffsize, p->file) ? 0:-1; X } else { X /* It's there */ X int rlen; X char *cp; X X if (p->curpos == NULL) X return -1; /* Exhausted */ X X rlen = strlen(p->curpos) + 1; X if (rlen <= buffsize) { X /* Enough room for everything */ X strcpy(buff, p->curpos); X p->curpos = NULL; X } else { X /* Try to copy a full line */ X buffsize--; /* Reserve for trailing '\0' */ X buffsize--; /* and a '\n' */ X for (cp = p->curpos; buffsize && *cp && *cp != '\n';) { X *buff++ = *cp++; X buffsize--; X } X if (*cp == '\n') { X *buff++ = *cp++; X } X *buff = '\0'; X /* Prepare for next call */ X if (*cp) X p->curpos = cp; /* Next line */ X else X p->curpos = NULL; /* End */ X } X return 0; X } X} X Xvoid Xmatch_init(struct match_t *pm, char *matchname, char matchopt, char *expr) X{ X int len = strlen(matchname) + 1; X char *s; X X if (matchopt & M_REGEX) { X char errbuf[80]; X int rerr; X rerr = regcomp(&pm->re, expr, REG_EXTENDED|REG_NOSUB X |((matchopt & M_CASE) ? 0:REG_ICASE)); X if (rerr < 0) { X regerror(rerr, NULL, errbuf, sizeof errbuf); X syslog(LOG_NOTICE, "%s: %s", expr, errbuf); X exit(1); X } X s = (char *)zmalloc(len); X strcpy(s, matchname); X pm->name = s; X pm->expr = NULL; X pm->flags = M_REGEX; X } else { X len += strlen(expr) + 1; X s = (char *)zmalloc(len); X strcpy(s, expr); X strcpy(s + strlen(expr) + 1, matchname); X X pm->name = s + strlen(expr) + 1; X pm->expr = s; X pm->flags = matchopt; X } X} X Xint Xmatch_binsert(char *matchname, char matchopt, char *expr, int beg, int end) X{ X struct match_t *pm = bmt.array + bmt.free; X X MESSAGE_5("declaring match %s body %d-%d with opt %d expr %s", X matchname, beg, end, matchopt, expr); X X if (bmt.free == bmt.max) { X syslog(LOG_NOTICE, "body match table too small (%d entries)", X bmt.max); X exit(1); X } X X match_init(pm, matchname, matchopt, expr); X bmatch_beg[bmt.free] = beg; X bmatch_end[bmt.free] = end; X return bmt.free++; X} X Xint Xmatch_hinsert(char *matchname, char matchopt, char *expr, int hdrindex) X{ X struct match_t *pm = hmt.array + hmt.free; X X MESSAGE_4("declaring match %s header %d with opt %d expr %s", X matchname, hdrindex, matchopt, expr); X X if (hmt.free == hmt.max) { X syslog(LOG_NOTICE, "header match table too small (%d entries)", X hmt.max); X exit(1); X } X X match_init(pm, matchname, matchopt, expr); X hmatch_index[hmt.free] = hdrindex; X return hmt.free++; X} X X/* X * Find a match by name in the indicated table. X * Return index or -1 if not found. X */ X Xint Xmatch_find(struct matchtable_t *mt, char *name) X{ X int i; X for (i = 0; i < mt->free; i++) { X if (!strcasecmp(name, mt->array[i].name)) { X return i; X } X } X return -1; X} X Xint Xmatch_string(char *string, struct match_t *pm) X{ X char m; X char *expr = pm->expr; X int offset, le, la; X X switch(pm->flags) { X case M_REGEX: { X char errbuf[80]; X int rerr; X rerr = regexec(&pm->re, string, 0, NULL, 0); X if (rerr == 0) X m = 1; X else if (rerr == REG_NOMATCH) X m = 0; X else { X regerror(rerr, NULL, errbuf, sizeof errbuf); X syslog(LOG_NOTICE, "%s: %s", string, errbuf); X exit(1); X } X break; X } X case 0: X le = strlen(expr); X la = strlen(string) - le; X m = 0; X while (la-- >= 0) { X if (!strncasecmp(string, expr, le)) { X m = 1; X break; X } X string++; X } X break; X case M_CASE: X m = (strstr(string, expr) != NULL); X break; X case M_BEGLINE: X m = !strncasecmp(string, expr, strlen(expr)); X break; X case M_CASE+M_BEGLINE: X m = !strncmp(string, expr, strlen(expr)); X break; X case M_ENDLINE: X offset = strlen(string) - strlen(expr); X m = (offset < 0) ? 0 : !strcasecmp(string+offset, expr); X break; X case M_CASE+M_ENDLINE: X offset = strlen(string) - strlen(expr); X m = (offset < 0) ? 0 : !strcmp(string+offset, expr); X break; X case M_BEGLINE+M_ENDLINE: X m = !strcasecmp(string, expr); X break; X case M_CASE+M_BEGLINE+M_ENDLINE: X m = !strcmp(string, expr); X break; X default: /* ?! */ X syslog(LOG_NOTICE, "%s: weird regexp flags %d", string, pm->flags); X exit(1); X } X return m; X} X X/* X * From group:artnum in 'groupnum', generate relative article path in 'path'. X * Return pointer to char after if successful, NULL if error. X */ X Xconst char * Xgenerate_path(const char *groupnum, char *path, int pathsize) X{ X const char *p, *q, *group, *art_num; X X /* Keep room for the inserted '/' and the final '\0' */ X if (pathsize < 2) X return NULL; X pathsize -= 2; X X group = groupnum; X X if ((p = strchr(group, ':')) == NULL) X return NULL; X art_num = p + 1; X X /* Copy group to path, changing '.' to '/' */ X X for (q = group; q < p && pathsize; q++) { X *path++ = (*q == '.') ? '/' : *q; X pathsize--; X } X *path++ = '/'; X X /* Copy all digits of art_num */ X for (q = art_num; isdigit(*q) && pathsize; q++) { X *path++ = *q; X pathsize--; X } X *path++ = '\0'; X X return pathsize ? q : NULL; X} X XFILE * Xget_artfile(struct article_t *current) X{ X char *p; X char path[MAXPATHNAMELEN]; X X if (current->artfiletried) { X if (current->artfile) rewind(current->artfile); X return current->artfile; X } X X current->artfiletried = T_TRUE; /* to try only once */ X current->artfile = NULL; X X /* skip the hostname */ X p = strchr (current->h[I_XREF], ' '); X if (p == NULL) X return NULL; X p++; X X MESSAGE_1 ("Generating path for %s", p); X X if (generate_path(p, path, sizeof path) == NULL) X return NULL; X X MESSAGE_1 ("path %s", path); X X if ((current->artfile = fopen(path, "r")) == NULL) { X perror("cannot open"); X return NULL; X } X X return current->artfile; X} X X/* X * Compare current article contents with older article contents X */ X Xint Xsame_contents(struct article_t *current, const char *oldpath) X{ X char line1[8192], line2[8192]; X int linelen1, linelen2; X X FILE *artfile, *oldfile; X X MESSAGE_2("Comparing %s with %s", current->h[I_MESSAGE_ID], oldpath); X X oldfile = fopen(oldpath, "r"); X if (oldfile == NULL) X return 0; X X artfile = get_artfile(current); X if (artfile == NULL) { X fclose(oldfile); X return 0; X } X X /* Skip headers on new article */ X for (;;) { X if (fgets (line1, sizeof (line1), artfile) == NULL) { X fclose(oldfile); X return 0; X } X line1 [strlen (line1) - 1] = '\0'; X if (line1[0] == '\0') X break; X } X X /* Skip headers on old article */ X for (;;) { X if (fgets (line2, sizeof (line2), oldfile) == NULL) { X fclose(oldfile); X return 0; X } X line2 [strlen (line2) - 1] = '\0'; X if (line2[0] == '\0') X break; X } X X /* Examine body */ X X MESSAGE_2("Reading bodies for %s and %s", current->h[I_MESSAGE_ID], oldpath); X X for (;;) { X /* Try to read a line from new article */ X if (fgets (line1, sizeof (line1), artfile) == NULL) { X if (fgets (line2, sizeof (line2), oldfile) == NULL) { X /* Simultaneous EOF, they are equal */ X fclose(oldfile); X return 1; X } else { X /* More lines on old article */ X MESSAGE_1("Failure, %s shorter", current->h[I_MESSAGE_ID]); X fclose(oldfile); X return 0; X } X } X X /* Got a line from new article */ X linelen1 = strlen(line1); X line1[linelen1 - 1] = '\0'; X X /* Try to read a line from old article */ X if (fgets (line2, sizeof (line2), oldfile) == NULL) { X MESSAGE_1("Failure, %s longer", current->h[I_MESSAGE_ID]); X fclose(oldfile); X return 0; X } X X /* Got a line from old article */ X X linelen2 = strlen(line2); X if (linelen1 != linelen2) { X MESSAGE_2("Failure, line lengths %d vs %d", linelen1, linelen2); X fclose(oldfile); X return 0; X } X line2[linelen2 - 1] = '\0'; X X if (strcmp(line1, line2)) { X MESSAGE_2("Failure, line \"%s\" vs \"%s\"", line1, line2); X fclose(oldfile); X return 0; X } X } X return 1; X} X X/* X * Check articles (From:, Newsgroups: and contents) X * and see if they are the same. X */ X Xint Xsame_article(struct article_t *a, struct art_bucket_t *ab) X{ X char *p, *pb; X char *h, *hb; X X /* Compare hashed values for From:/Newsgroups: */ X if (a->hash != ab->hash) X return 0; X X if (a->bodysize != ab->bodysize) X return 0; X X MESSAGE_1 ("comparing hashed fields %s", a->h[I_MESSAGE_ID]); X X /* Skip file paths and compare hashed headers */ X p = a->hashfields; h = p; while (*h) h++; h++; X pb = ab->fields; hb = pb; while (*hb) hb++; hb++; X if (strcmp(h, hb)) X return 0; X X /* Hashed headers match; compare real body */ X X MESSAGE_2 ("comparing contents %s %s", a->h[I_MESSAGE_ID], pb); X X return same_contents(a, pb); X} X X/* X * Store an article and drop the oldest. X */ X Xvoid Xarticle_store(struct article_t *a) X{ X int newart; X struct art_bucket_t *ab; X X newart = firstart++; X ab = artb + newart; X X if (firstart == MAXARTS) firstart = 0; X X if (freeart == 0) { X /* Drop the oldest article */ X assert (msgtable[ab->hash].first == ab); X msgtable[ab->hash].first = ab->next; X free(ab->fields); X } else X freeart--; X X ab->next = NULL; X ab->bodysize = a->bodysize; X ab->hash = a->hash; X ab->fields = a->hashfields; X a->hashfields = NULL; X X if (msgtable[ab->hash].first == NULL) { X msgtable[ab->hash].last = msgtable[ab->hash].first = ab; X } else { X msgtable[ab->hash].last->next = ab; X msgtable[ab->hash].last = ab; X } X} X Xvoid Xeval_hmatch(struct article_t *current, matchflags_t matches) X{ X int i, hi; X char m; X struct match_t *pm; X X for (i = 0; i < hmt.free; i++) { X if (matches & (1 << i)) { X pm = hmt.array + i; X hi = hmatch_index[i]; X X m = match_string(current->h[hi], pm); X X current->hmatch_done |= (matchflags_t)(1 << i); X current->hmatch_val |= (matchflags_t)(m << i); X MESSAGE_4("matching %s: %s with %s is %d", X hdrb[hi].name, current->h[hi], X pm->expr ? pm->expr:"(precompiled)", m); X } X } X} X X/* Eval hash function on article (selected headers, body size) */ X Xvoid Xeval_hash(struct article_t *current) X{ X char *p, *pc; X char path[MAXPATHNAMELEN]; X hashmsg_t h; X int len; X X /* skip the hostname */ X p = strchr(current->h[I_XREF], ' '); X if (p == NULL) { X syslog(LOG_NOTICE, "bad Xref \"%s\"\n", current->h[I_XREF]); X exit(1); X } X do p++; while (isspace(*p)); X if (generate_path(p, path, sizeof path) == NULL) { X syslog(LOG_NOTICE, "bad Xref \"%s\"\n", current->h[I_XREF]); X exit(1); X } X X /* Allocate enough space for what we store */ X len = strlen(path) X + strlen(current->h[I_FROM]) X + strlen(current->h[I_NEWSGROUPS]) X + strlen(current->h[I_SUBJECT]) X + 5; X current->hashfields = pc = (char *)zmalloc(len); X X /* Copy file path, including terminating 0 */ X p = path; X while ((*pc++ = *p++)); X X /* Copy the fields we want to compare */ X sprintf(pc, "%s\n%s\n%s\n", X current->h[I_FROM], X current->h[I_NEWSGROUPS], X current->h[I_SUBJECT]); X X /* Evaluate hash value */ X h = current->bodysize; X while (*pc) { X h *= 211; X h += *pc + 67; X pc++; X } X current->hash = h % HASHMSGSIZE; X} X X/* Check for duplicate articles */ X Xvoid Xeval_duplicate(struct article_t *current) X{ X struct art_bucket_t *pab; X X MESSAGE_1 ("checking duplicate %s", current->h[I_MESSAGE_ID]); X X if (current->hashfields == NULL) X eval_hash(current); X X for (pab = msgtable[current->hash].first; X pab != NULL; X pab = pab->next) { X X MESSAGE_1 ("hash collision found %s", current->h[I_MESSAGE_ID]); X X if (same_article(current, pab)) { X MESSAGE_1 ("duplicate yields true on %s", current->h[I_MESSAGE_ID]); X current->duplicate = T_TRUE; X X /* X * Store anyway for future duplicate checking (the older article X * might be removed from the FIFO soon, losing some potential X * future duplicates detection). X */ X article_store(current); X return; X } X } X X current->duplicate = T_FALSE; X article_store(current); X} X Xvoid Xeval_fromok(struct article_t *current) X{ X const char *p, *atpos; X X /* check From: */ X MESSAGE_1 ("checking From: %s", current->h[I_MESSAGE_ID]); X X /* Check Reply-To: if found, else From: */ X if (current->h[I_REPLY_TO][0] == '\0') X p = current->h[I_FROM]; X else X p = current->h[I_REPLY_TO]; X X /* X * Check there's a '@' not on first char, X * followed by at least a '.' with at least one char between them. X */ X atpos = strchr (p, '@'); X current->fromok = X (atpos != NULL && atpos != p X && atpos[1] != '.' && strchr (atpos, '.') != NULL); X} X Xvoid Xeval_refok(struct article_t *current) X{ X if (!strncmp(current->h[I_SUBJECT], "Re: ", 4)) X current->refok = (current->h[I_REFERENCES][0] != '\0'); X else X current->refok = T_TRUE; X} X Xvoid Xeval_multi(struct article_t *current) X{ X char *bracketpos = current->h[I_SUBJECT]; X X while ((bracketpos = strchr(bracketpos, '[')) != NULL) { X X /* X * Look for two numbers separated by anything X * and ending with ']'. Whitespace is accepted except X * in numbers. X */ X X bracketpos++; X while (isspace(*bracketpos)) bracketpos++; X if (!*bracketpos) break; X if (!isdigit(*bracketpos)) continue; X X /* Skip the first number */ X do bracketpos++; while (isdigit(*bracketpos)); X if (!*bracketpos) break; X X /* Skip the separator */ X do bracketpos++; while (!isdigit(*bracketpos)); X if (!*bracketpos) break; X X /* Skip the second number */ X X current->parts = atol(bracketpos + 1); X do bracketpos++; while (isdigit(*bracketpos)); X X if (!*bracketpos) break; X while (isspace(*bracketpos)) bracketpos++; X if (*bracketpos != ']') continue; X X current->multi = T_TRUE; X return; X } X current->multi = T_FALSE; X} X X/* X * Pattern matching on article body. X * Also recognizes binary formats: Base64, uuencode, Binhex, netsend. X */ X Xvoid Xeval_body(struct article_t *current, char testbin, matchflags_t matches) X{ X char line[8192]; X FILE *artfile; X int i; X X /* Legal characters for Binhex. Go figure... */ X X char *bhchars = X "!\"#$%&'()*+,-012345689@ABCDEFGHIJKLMNPQRSTUVXYZ[`abcdefhijklmpqr"; X X int linecount = 0, linelen; X int uuline = 0, b64line = 0, bhline = 0, nsline = 0; X X if ((artfile = get_artfile(current)) == NULL) { X current->binary = T_FALSE; X return; X } X X /* Skip headers */ X for (;;) { X if (fgets (line, sizeof (line), artfile) == NULL) X return; X line [strlen (line) - 1] = '\0'; X if (line[0] == '\0') X break; X } X X /* Examine body */ X X for (;;) { X if (fgets (line, sizeof (line), artfile) == NULL) { X break; X } X linelen = strlen(line); X line[linelen - 1] = '\0'; X X linecount++; X X /* pattern matching on body lines */ X X for (i = 0; matches && i < bmt.free; i++) { X struct match_t *pm; X char m; X X if (matches & (1 << i)) { X if (linecount < bmatch_beg[i]) { X /* Line not in range for this pattern */ X continue; X } X if (linecount > bmatch_end[i]) { X /* This pattern failed */ X m = 0; X } else { X pm = bmt.array + i; X m = match_string(line, pm); X if (m == 0 && linecount != bmatch_end[i]) X continue; /* Not done, retry on next lines */ X } X X current->bmatch_done |= (matchflags_t)(1 << i); X current->bmatch_val |= (matchflags_t)(m << i); X X /* Mark as evaluated */ X matches ^= (1 << i); X } X } X X if (!matches && !testbin) { X /* Nothing more to evaluate, we're done */ X return; X } X X if (strncmp("begin ", line, 6) == 0) { X /* Bonus for typical uuencode line */ X uuline += 30; X } else if (strcmp("end", line) == 0) { X /* Bonus for typical uuencode line */ X uuline += 30; X } else if (((line[0] - ' ') & 077) * 4 == (linelen - 2) * 3) { X /* uuencode */ X uuline++; X } else if (linelen == 65 X || line[0] == ':' X || line[linelen - 2] == ':') { X /* binhex or netsend */ X char *pl, c; X X /* binhex */ X for (pl = line; *pl; pl++) { X c = *pl; X /* Accept a ':' as first or last char only */ X if (!strchr(bhchars, c) X && (c != ':' || pl == line || pl[1] == '\0')) X break; X } X if (!*pl) { X bhline++; X } else if (linelen == 65) { X /* netsend */ X for (pl = line; *pl; pl++) { X c = *pl; X if (c == ' ' || c == 0x7f || c == '|' || c == '~' X || (c & 0x80)) X break; X } X if (!*pl) nsline++; X } X } else if (linelen > 50) { X /* base 64 */ X char *pl, c; X X for (pl = line; *pl; pl++) { X c = *pl; X if (!isalnum(c) && c != '/' && c != '+') X break; X } X if (!*pl) b64line++; X } X } X X /* Do more than 2 thirds look like some kind of binary encoding ? */ X if (uuline * 3 > linecount * 2) { X current->coding = "uuencode"; X } else if (b64line * 3 > linecount * 2) { X current->coding = "base64"; X } else if (bhline * 3 > linecount * 2) { X current->coding = "binhex"; X } else if (nsline * 3 > linecount * 2) { X current->coding = "netsend"; X } else if ((bhline+uuline+b64line+nsline) * 3 > linecount * 2) { X /* Too much binary is simply too much. */ X current->coding = "mixed"; X } X if (current->coding[0]) { X char *sender = current->h[I_FROM]; X if (current->h[I_REPLY_TO][0] != '\0') X sender = current->h[I_REPLY_TO]; X X syslog (LOG_NOTICE, X "%s: %.100s %d (%s) Subject: %.100s groups=%.100s", X curfeedname, X current->h[I_MESSAGE_ID], X current->bodysize, X current->coding, X current->h[I_SUBJECT], current->h[I_NEWSGROUPS]); X X current->binary = T_TRUE; X } else { X current->binary = T_FALSE; X } X return; X} X X/* X * Generate article paths from Xref: field (for cross-posts) X * and remove all the links. X * X * Xref syntax: group:art# [group:art#] X */ X Xint Xremove_articles (struct article_t *current) X{ X const char *p; X char path[MAXPATHNAMELEN]; X X /* skip the hostname */ X p = strchr(current->h[I_XREF], ' '); X if (p == NULL) X return 1; X do p++; while (isspace(*p)); X X while ((p = generate_path(p, path, sizeof path)) != NULL) { X unlink (path); X if (!isspace(*p)) X break; X p++; X } X return 0; X} X X/* X * Copy "in" to "out". X * Stop at first empty line (not copied) if headers_only is not 0. X */ X Xvoid copy_file(FILE *out, FILE *in, char headers_only) X{ X char buff[8192]; X while (fgets(buff, sizeof buff, in)) { X if (headers_only && buff[0] == '\n') break; X fputs(buff, out); X } X} X X/* X * Process pattern, piping the output to pipeto X * If pipeto begins in ">", append to file instead. X */ X Xvoid Xprocess_pattern(struct pattern_t *pattern, X const char *pipeto, X struct article_t *current) X{ X FILE *pp; X FILE *artfile; X X char buff[8192]; X char out[MAXIFNEST]; X char *bp, *cp, *cp2; X int lb, ifnest; X char appending = 0; X X ifnest = 0; out[0] = 1; X X /* Open files */ X if (pattern_open(pattern) < 0) { Xabort(); X return; X } X X if (*pipeto == '>') { X pipeto++; appending++; X while (isspace(*pipeto)) pipeto++; X pp = fopen(pipeto, "a"); X } else X pp = popen(pipeto, "w"); X X if (!pp) { X pattern_close(pattern); X return; X } X X while (pattern_read(pattern, buff, sizeof buff) == 0) { X bp = buff; lb = strlen(buff); X X while ((cp = strchr(bp, '%')) != NULL) { X cp2 = strchr(cp + 1, '%'); X X /* Found only one %, don't expand */ X if (!cp2) break; X X /* Found two % on same line */ X X /* Write up to the first % */ X if (out[ifnest]) fwrite(bp, cp - bp, 1, pp); X X /* Expand the % part */ X *cp2 = '\0'; cp++; X if (!strncasecmp(cp, "if-header-", 10)) { X int i; X i = header_to_index(cp + 10); X if (i < 0 || ifnest == MAXIFNEST-1) { X /* Unknown header or nesting exceeded, don't expand */ X fprintf(pp, "%%%s%%", cp); X } else if (current->h[i][0] == '\0') { X out[++ifnest] = 0; X } else { X out[++ifnest] = 1; X } X } else if (!strcasecmp(cp, "else")) { X if (ifnest > 0) X out[ifnest] = !out[ifnest]; X else X fputs("%else%", pp); X } else if (!strcasecmp(cp, "endif")) { X if (ifnest > 0) ifnest--; else fputs("%endif%", pp); X } else if (!out[ifnest]) { X /* Do nothing */ X } else if (!strcasecmp(cp, "article-headers") X || !strcasecmp(cp, "article")) { X if ((artfile = get_artfile(current)) == NULL) X /* Can't open file, don't expand */ X fprintf(pp, "%%%s%%", cp); X else { X copy_file(pp, artfile, X strcasecmp(cp, "article")); X } X } else if (!strcasecmp(cp, "article-sender")) { X if (current->h[I_REPLY_TO][0] != '\0') X fputs(current->h[I_REPLY_TO], pp); X else X fputs(current->h[I_FROM], pp); X } else if (!strcasecmp(cp, "article-sender-host")) { X char *u = current->h[I_REPLY_TO]; X char *atpos; X if (current->h[I_REPLY_TO][0] == '\0') X u = current->h[I_FROM]; X atpos = strchr(u, '@'); X if (atpos) X fputs(atpos + 1, pp); X } else if (!strcasecmp(cp, "moderated-sender")) { X if (current->h[I_APPROVED][0] != '\0') X fputs(current->h[I_APPROVED], pp); X else if (current->h[I_REPLY_TO][0] != '\0') X fputs(current->h[I_REPLY_TO], pp); X else X fputs(current->h[I_FROM], pp); X } else if (!strcasecmp(cp, "article-size")) { X fprintf(pp, "%ld", current->bodysize); X } else if (!strcasecmp(cp, "article-encoding")) { X fprintf(pp, "%s", current->coding[0] == '\0' X ? "mixed encoding" X : current->coding); X } else if (!strcasecmp(cp, "re-subject")) { X if (strncasecmp(current->h[I_SUBJECT], "Re: ", 4)) X fprintf(pp, "Re: %s", current->h[I_SUBJECT]); X else X fputs(current->h[I_SUBJECT], pp); X } else if (!strcasecmp(cp, "cancel-message-id")) { X fprintf(pp, "h[I_MESSAGE_ID] + 1); X } else if (!strcasecmp(cp, "truncated-message-id")) { X fputs(current->h[I_MESSAGE_ID] + 1, pp); X } else if (!strncasecmp(cp, "header-", 7)) { X int i; X i = header_to_index(cp + 7); X if (i < 0) X /* Unknown header, don't expand */ X fprintf(pp, "%%%s%%", cp); X else X fputs(current->h[i], pp); X } else if (!strcasecmp(cp, "date")) { X char buffd[80]; X time_t t; X time(&t); X strftime(buffd, sizeof buffd, "%d %b %Y %T GMT", gmtime(&t)); X fputs(buffd, pp); X } else if (!strcasecmp(cp, "newsbot-version")) { X fputs(version, pp); X } else if (!strcasecmp(cp, "")) { X fputs("%", pp); X } else if (*cp == '<') { X FILE *cpf = fopen(cp + 1, "r"); X if (cpf) { X copy_file(pp, cpf, 0); X fclose(cpf); X } else { X /* Unknown file, don't expand */ X fprintf(pp, "%%%s%%", cp); X } X } else { X /* Unknown tag, don't expand */ X fprintf(pp, "%%%s%%", cp); X } X /* Continue */ X lb -= cp2 + 1 - bp; X bp = cp2 + 1; X } X if (out[ifnest]) fwrite(bp, lb, 1, pp); X } X X pattern_close(pattern); X if (appending) X fclose(pp); X else X pclose(pp); X} X X/* X * Execute and log action X * X * Return 0 if this is to be the last action for this article. X */ X Xint Xdo_action(struct action_t *action, struct article_t *current) X{ X syslog(LOG_NOTICE, X "%s: %s %s on %s", opt_act ? "doing":"would do", X curfeedname, action->name, current->h[I_MESSAGE_ID]); X X if (action == &actremove && opt_act) X remove_articles(current); X else if (action == &actstop) X return 0; X else if (action == &actnop) X return 1; X else if (opt_act) { X MESSAGE_2("pipe %s to %s", action->pattern, action->pipeto); X process_pattern(action->pattern, action->pipeto, current); X } X return 1; X} X X/* X * Check condition against article. X * Return 1 if ok, 0 if not. X */ X Xint Xcheck_cond(struct cond_t *cond, struct article_t *current) X{ X /* Check article size */ X if (cond->checksize != T_DONTCARE) { X MESSAGE_2("condition 0x%08x check size %ld", cond, cond->size); X X if (cond->checksize == T_TRUE) { X if (current->bodysize < cond->size) X return 0; X } else { X if (current->bodysize >= cond->size) X return 0; X } X } X X /* Check from */ X if (cond->fromok != T_DONTCARE) { X MESSAGE_1("condition 0x%08x check from", cond); X if (current->fromok == T_DONTCARE) X eval_fromok(current); X if (cond->fromok != current->fromok) X return 0; X } X X /* Check references */ X if (cond->refok != T_DONTCARE) { X MESSAGE_1("condition 0x%08x check refs", cond); X if (current->refok == T_DONTCARE) X eval_refok(current); X if (cond->refok != current->refok) X return 0; X } X X /* Check multi size */ X if (cond->checkmultisize != T_DONTCARE) { X MESSAGE_1("condition 0x%08x check multi", cond); X if (current->multi == T_DONTCARE) X eval_multi(current); X if (cond->checkmultisize == T_TRUE) { X if (current->multi == T_FALSE X || current->parts * current->bodysize < cond->size) X return 0; X } else { X if (current->multi == T_TRUE X && current->parts * current->bodysize >= cond->size) X return 0; X } X } X X /* Check duplicate */ X if (cond->duplicate != T_DONTCARE) { X MESSAGE_1("condition 0x%08x check duplicate", cond); X if (current->duplicate == T_DONTCARE) X eval_duplicate(current); X if (cond->duplicate != current->duplicate) X return 0; X } X X /* Check header matches */ X if (cond->hmatch_totest) { X MESSAGE_1("condition 0x%08x check header matches", cond); X if (~current->hmatch_done & cond->hmatch_totest) X eval_hmatch(current, ~current->hmatch_done & cond->hmatch_totest); X if ((cond->hmatch_val ^ current->hmatch_val) & cond->hmatch_totest) X return 0; X } X X /* Check body (binary or pattern matching) */ X if (cond->binary != T_DONTCARE || cond->bmatch_totest) { X MESSAGE_1("condition 0x%08x check body", cond); X if (current->binary == T_DONTCARE X || (~current->bmatch_done & cond->bmatch_totest)) X eval_body(current, X cond->binary != T_DONTCARE, X ~current->bmatch_done & cond->bmatch_totest); X if ((cond->binary != T_DONTCARE && cond->binary != current->binary) X || ((cond->bmatch_val ^ current->bmatch_val) & cond->bmatch_totest)) X return 0; X } X return 1; X} X X/* X * Handle condition X */ X Xint Xhandle_cond (struct cond_t *condp, struct article_t *current) X{ X for (; condp && check_cond(condp, current); condp = condp->and) { X MESSAGE_1("condition 0x%08x TRUE", condp); X if (!do_action(condp->action, current)) X /* Stop processing for this article if we have to */ X return 0; X } X return 1; X} X X/* X * Handle article X */ Xvoid Xhandle_article (struct article_t *current, struct feed_t *f) X{ X struct cond_t *condp; X X curfeedname = f->name; X X for (condp = f->firstcond; condp; condp = condp->next) X if (!handle_cond(condp, current)) X break; X} X X/* X * Convert size : integer optionaly followed by a multiplier (k or M) X */ Xlong Xconv_size(char *p, char **newp) X{ X long n; X if (isdigit(*p)) { X n = atol(p); X do p++; while (isdigit(*p)); X if (*p == 'K' || *p == 'k') { X n *= 1024; X p++; X } else if (*p == 'M' || *p == 'm') { X n *= 1024*1024; X p++; X } X } else X n = 0; X X *newp = p; X return n; X} X X/* X * Read configuration file X */ X Xvoid Xread_conf(const char *conf) X{ X char line[2048]; X char *mac, *p; X char conj; X struct action_t *pa; X struct cond_t *pc, *lastconj = NULL; X struct feed_t *pf; X struct feed_t *curfeed = &nullfeed; X struct pattern_t *inpattern = NULL; X X FILE *cf = fopen(conf, "r"); X if (cf == NULL) X exit(1); X X while (fgets(line, sizeof line, cf) != NULL) { X if (inpattern) { X if (line[0] == '=') { X pattern_append(inpattern, line+1); X continue; X } X if (line[0] != '#') X inpattern = NULL; X } X X line[strlen(line) - 1] = '\0'; X X if (line[0] == '#' || line[0] == '\0') X continue; X X if (line[0] == 'P') { X /* Search for: name */ X char *name; X /* Keep pattern name */ X name = line + 1; X /* Search and create */ X inpattern = pattern_find(name); X } else if (line[0] == 'A') { X /* Search for: name pattern pipeto */ X X char *name, *pattern, *pipeto; X X /* Keep action name */ X name = line + 1; X X /* Find pattern file name */ X for (p = name; !isspace(*p) && *p; p++); X if (!*p) continue; X *p++ = '\0'; X while (isspace(*p)) p++; X if (!*p) continue; X pattern = p; X X /* Find command name to pipe to */ X for (; !isspace(*p); p++); X if (!*p) continue; X *p++ = '\0'; X while (isspace(*p)) p++; X if (!*p) continue; X pipeto = p; X X X /* Check for duplicate action */ X for (pa = firstact; pa; pa = pa->next) X if (strcasecmp(pa->name, name) == 0) X break; X X if (pa) X syslog(LOG_NOTICE, "duplicate action name \"%s\" (ignored)", X name); X else { X X pa = zmalloc(sizeof(struct action_t)); X mac = zmalloc(strlen(name)+strlen(pipeto)+2); X X MESSAGE_3("action %s: %s %s", name, pattern, pipeto); X X pa->name = mac; strcpy(mac, name); mac += strlen(name)+1; X pa->pipeto = mac; strcpy(mac, pipeto); mac += strlen(pipeto)+1; X pa->pattern = pattern_find(pattern); X X /* Insert in action list */ X pa->next = firstact; firstact = pa; X } X } else if (line[0] == 'I' || line[0] == ',') { X char *condition, *action; X X /* X * If the condition begins with ',', flag it to link it X * as a conjunction with the previous condition. X */ X X if (line[0] == ',') { X conj = 1; X if (lastconj == NULL) { X syslog(LOG_NOTICE, "conjunction with nothing, %s ignored", X line); X continue; X } X } else { X conj = 0; X } X X /* Search for: condition action */ X X condition = line + 1; X X p = condition; X X /* Find action name */ X for (; !isspace(*p); p++); X if (!*p) continue; X *p++ = '\0'; X while (isspace(*p)) p++; X if (!*p) continue; X action = p; X X for (; !isspace(*p); p++); X if (*p) *p = '\0'; X X /* Find action in list */ X for (pa = firstact; pa; pa = pa->next) X if (!strcmp(pa->name, action)) X break; X X if (pa == NULL) { X syslog(LOG_NOTICE, "action %s not found, condition ignored", X action); X continue; X } X X pc = zmalloc(sizeof(struct cond_t)); X X pc->fromok = T_DONTCARE; X pc->refok = T_DONTCARE; X pc->binary = T_DONTCARE; X pc->duplicate = T_DONTCARE; X pc->checksize = T_DONTCARE; X pc->checkmultisize = T_DONTCARE; X pc->hmatch_val = pc->hmatch_totest = 0; X pc->bmatch_val = pc->bmatch_totest = 0; X pc->action = pa; X pc->and = NULL; X X /* Decode and store condition */ X X MESSAGE_1("analyzing condition %s", condition); X X for (; *condition; condition++) { X char val; char curcond; X if (*condition == '!') { X if (!*condition++) X break; X val = T_FALSE; X } else X val = T_TRUE; X X MESSAGE_1("condition %c", *condition); X switch((curcond = *condition++)) { X case 'M': X pc->multisize = conv_size(condition, &condition); X pc->checkmultisize = val; X break; X case '>': X pc->size = conv_size(condition, &condition); X pc->checksize = val; X break; X case 'D': X pc->duplicate = val; X break; X case 'B': X pc->binary = val; X break; X case 'R': X pc->refok = val; X break; X case 'F': X pc->fromok = val; X break; X case 'L': X case '~': { X char *mn; X int i, c; X mn = condition; X while (isalnum(*condition)) condition++; X X c = *condition; X *condition = '\0'; X i = match_find(curcond == 'L' ? &bmt:&hmt, mn); X *condition = c; X X if (i < 0) { X syslog(LOG_NOTICE, "%s match %s unknown", X curcond == 'L' ? "body":"header", mn); X } else { X if (curcond == 'L') { X pc->bmatch_totest |= 1L << i; X pc->bmatch_val |= val << i; X } else { X pc->hmatch_totest |= 1L << i; X pc->hmatch_val |= val << i; X } X } X break; X } X default: X syslog(LOG_NOTICE, "invalid code '%c' in condition", X *condition); X break; X } X if (!*condition) X break; X if (*condition != ',') { X syslog(LOG_NOTICE, "invalid separator '%c' in condition", X *condition); X condition++; X } X } X X /* Insert in condition list */ X X if (conj && lastconj) { X lastconj->and = pc; X lastconj = pc; X } else { X if (curfeed->lastcond) X curfeed->lastcond->next = pc; X else X curfeed->firstcond = pc; X lastconj = curfeed->lastcond = pc; X pc->next = NULL; X } X } else if (line[0] == 'F') { X char *feed; X feed = p = line + 1; X X while (!isspace(*p)) p++; X *p = '\0'; X X /* Look if it's already known */ X for (pf = firstfeed; pf; pf = pf->next) X if (strcasecmp(pf->name, feed) == 0) X break; X X if (pf) X syslog(LOG_NOTICE, "duplicate declaration for feed \"%s\"", X feed); X else { X X /* Create a feed struct */ X pf = zmalloc(sizeof(struct feed_t)); X X /* Copy the feed name */ X pf->name = zmalloc(strlen(feed) + 1); X strcpy(pf->name, feed); X X /* Initialize other fields */ X pf->lastcond = NULL; X pf->firstcond = NULL; X X /* Insert in feed list */ X pf->next = firstfeed; X firstfeed = pf; X } X X /* Make it the current feed */ X curfeed = pf; X X lastconj = NULL; X } else if (line[0] == 'H') { X /* allocate enough space just for the header name + '\0' */ X char *new_header = (char *)zmalloc(strlen(line)); X MESSAGE_1("declaring header %s", line+1); X X strcpy(new_header, line+1); X header_insert(new_header); X } else if (line[0] == '~' || line[0] == 'L') { X int h = -1; X int i; X char *matchname, *arg1, *expr; X char matchopt = M_CASE; X char headermatch = line[0] == '~'; X char *p = line + 1; X X while (!isspace(*p)) { X switch (*p++) { X case 'c': /* case sensitive (default) */ X matchopt |= M_CASE; X break; X case 'i': /* case insensitive */ X matchopt &= ~M_CASE; X break; X case 'e': /* regex */ X matchopt |= M_REGEX; X break; X default: X syslog(LOG_NOTICE, "unknown matching option %c", *p); X break; X } X } X X while (isspace(*p)) p++; X if (!*p) continue; X X matchname = p; X while (!isspace(*p)) p++; X if (!*p) continue; X *p++ = '\0'; X X while (isspace(*p)) p++; X if (!*p) continue; X X arg1 = p; X while (!isspace(*p)) p++; X if (!*p) continue; X *p++ = '\0'; X X while (isspace(*p)) p++; X if (!*p) continue; X X expr = p; X X i = match_find(headermatch ? &hmt:&bmt, matchname); X X if (i >= 0) { X syslog(LOG_NOTICE, "duplicate match %s", matchname); X } else { X int beg = 1, end = 1; X if (headermatch) { X /* Header match */ X h = header_to_index(arg1); X X if (h < 0) { X syslog(LOG_NOTICE, "undeclared header %s", arg1); X break; X } X } else { X /* Body line match */ X /* XXX: range check currently very crude... */ X char *arg2; X if ((arg2 = strchr(arg1, '-')) != NULL) { X /* Minus sign present */ X *arg2++ = '\0'; X sscanf(arg1, "%d", &beg); X sscanf(arg2, "%d", &end); X } else { X sscanf(arg1, "%d", &beg); X end = beg; X } X } X X if (!(matchopt & M_REGEX)) { X if (*expr == '^') { X expr++; X matchopt |= M_BEGLINE; X } X if (expr[strlen(expr)-1] == '$') { X expr[strlen(expr)-1] = '\0'; X matchopt |= M_ENDLINE; X } X } X X if (headermatch) { X match_hinsert(matchname, matchopt, expr, h); X } else { X match_binsert(matchname, matchopt, expr, beg, end); X } X } X } else X syslog(LOG_NOTICE, "cannot parse line: %s", line); X } X fclose(cf); X} X Xint Xmain (int argc, char **argv) X{ X int c, i, lineno, len; X char feedline[256]; X X char *line; int linesize; X char *pc, *begline, *nextbegline; X X struct article_t current; X X if (chdir(_PATH_SPOOL) < 0) { X perror("cannot chdir to news spool"); X exit(1); X } X X while ((c = getopt(argc, argv, "aC:D:df")) != EOF) { X switch (c) { X case 'C': X path_conf = optarg; X break; X case 'D': X path_pats = optarg; X break; X case 'd': X opt_debug++; X break; X case 'f': X opt_feed++; X break; X case 'a': X opt_act++; X break; X } X } X argc -= optind; X argv += optind; X X header_init(); X artb_init(); X read_conf(path_conf); X X if (opt_act) { X openlog ("newsbot+", LOG_NDELAY, LOG_FACILITY); X syslog (LOG_NOTICE, "Starting, active mode"); X } else { X openlog ("newsbot", LOG_NDELAY, LOG_FACILITY); X syslog (LOG_NOTICE, "Starting, no actions"); X } X X for (;;) { X X for (i = 0; i < freehdr; i++) X current.h[i] = ""; X X current.bodysize = -1; X current.headersize = 0; X current.control = 0; X X current.refok = T_DONTCARE; X current.fromok = T_DONTCARE; X current.multi = T_DONTCARE; X current.binary = T_DONTCARE; X current.duplicate = T_DONTCARE; X current.hmatch_done = 0; X current.hmatch_val = 0; X current.bmatch_done = 0; X current.bmatch_val = 0; X current.parts = -1; X current.coding = ""; X X current.artfiletried = T_FALSE; X current.artfile = NULL; X current.hashfields = NULL; X X current.firstbyte = 0; X line = current.hbuf; X linesize = sizeof(current.hbuf); X X lineno = 0; X X /* Process feed line if asked to */ X if (opt_feed) X if (fgets (feedline, sizeof (feedline), stdin) == NULL) X exit (0); X X begline = NULL; X X for (;;) { X X if (linesize < MINLINE) { X /* Space exhausted, skip to next article */ X char nl[MINLINE]; X X for (;;) { X if (fgets(nl, sizeof nl, stdin) == NULL) X exit(0); X if (nl[strlen(nl)-1] != '\n') X continue; /* line not complete */ X if (nl[0] == '\n') X break; X } X current.control = 1; /* HACK ! */ X break; X } X X /* Get line if necessary */ X if (!begline) { X begline = line; X if (fgets (begline, linesize, stdin) == NULL) X exit (0); X } X X if (begline[0] == '\n') X break; /* End of headers, go process */ X X len = strlen(begline); X X /* X * Now begline points to the current line, X * line points to linesize bytes of free space after. X */ X X nextbegline = NULL; X current.headersize += len; X lineno++; X X pc = strchr(begline, ':'); X if (pc && pc[1] == ' ' && *pc != ' ' && *pc != '\t') { X /* X * There's a colon in the line, followed by a space, X * and it's not a continuation line. X */ X MESSAGE_1 ("H=%s", begline); X *pc = '\0'; X switch(i = header_to_index(begline)) { X case -1: X break; X case I_CONTROL: X current.control = 1; X break; X case I_BYTES: X if (lineno == 1) { X current.bodysize = atol(line + 7); X /* "Bytes:" is not stored in the file */ X current.headersize -= len; X } X break; X case I_XREF: X /* X *"Xref:" is not stored in the file X * if INN gives it in line 2 (no crosspost) X */ X if (lineno == 2) { X current.headersize -= len; X } X /* FALLTHROUGH */ X default: X /* X * This header is to be kept. X * Find continuation lines, if any... X */ X current.h[i] = pc + 2; X line += len; linesize -= len; X for (;;) { X if (linesize < MINLINE) X break; X if (fgets(line, linesize, stdin) == NULL) X exit(0); X if (*line != ' ' && *line != '\t') { X /* Not a continuation, stop but keep it safe */ X nextbegline = line; X break; X } X MESSAGE_1 ("HC=%s", line); X len = strlen(line); X line += len; linesize -= len; X current.headersize += len; X } X X /* Hack to get at beginning of the line loop */ X if (linesize < MINLINE) X continue; X X /* Replace final '\n' */ X line[-1] = '\0'; X X MESSAGE_1 ("Keeping %s", current.h[i]); X } X } else { X MESSAGE_1 ("Hc=%s", begline); X } X begline = nextbegline; X } X X if (!current.control) { X MESSAGE_1 ("got %s", current.h[I_MESSAGE_ID]); X X /* Compute article body size */ X current.bodysize -= current.headersize; X X /* Handle null feed */ X handle_article (¤t, &nullfeed); X X /* Handle named feeds */ X if (opt_feed) { X struct feed_t *cf, *lf; X char *pf = feedline, *pf2; X char lastch; X X cf = firstfeed; X X do { X /* Find the next feed name */ X for (pf2 = pf; *pf2 && !isspace(*pf2); pf2++); X lastch = *pf2; X *pf2 = '\0'; X X /* X * The feed name is now pointed to by pf, X * try to find it in order. X */ X for (lf = cf; cf; cf = cf->next) X if (strcasecmp(cf->name, pf) == 0) X break; X X if (!cf) { X /* X * Not found at partial pass. X * Retry from the beginning. X */ X for (cf = firstfeed; cf; cf = cf->next) X if (strcasecmp(cf->name, pf) == 0) X break; X X if (cf) { X /* Found at last. Reorder the list so that X cf is just before lf. That will save us work X next time... */ X /* XXX: to be done... */ X } X } X X if (cf) X handle_article (¤t, cf); X else X syslog(LOG_NOTICE, X "feed entry %s is not known", pf); X X pf = pf2 + 1; X } while (lastch != '\n'); X } X X /* Cleanup */ X if (current.artfile) X fclose(current.artfile); X if (current.hashfields) X free(current.hashfields); X } X } X return 0; X} END-of-newsbot.c echo c - sample.config mkdir -p sample.config > /dev/null 2>&1 echo x - sample.config/bin.cancel-global sed 's/^X//' >sample.config/bin.cancel-global << 'END-of-sample.config/bin.cancel-global' XPath: cyberspam!bincancel!user XNewsgroups: %header-newsgroups% XFrom: %header-from% X%if-header-sender%Sender: %header-sender% X%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT X%endif%Subject: cancel: %header-subject% XMessage-Id: %cancel-message-id% XControl: cancel %header-message-id% XDate: %date% XX-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT XX-Bot: %newsbot-version% X X%article-size% bytes binary in discussion group (%article-encoding%) END-of-sample.config/bin.cancel-global echo x - sample.config/bin.cancel-local sed 's/^X//' >sample.config/bin.cancel-local << 'END-of-sample.config/bin.cancel-local' XPath: cyberspam!bincancel!user XNewsgroups: %header-newsgroups% XFrom: %header-from% X%if-header-sender%Sender: %header-sender% X%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT X%endif%Subject: cancel: %header-subject% XMessage-Id: %cancel-message-id% XControl: cancel %header-message-id% XDate: %date% XX-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT XX-Bot: %newsbot-version% XDistribution: local X XLocal cancel, %article-size% bytes binary in discussion group (%article-encoding%) END-of-sample.config/bin.cancel-local echo x - sample.config/bin.mcc-sender-fr sed 's/^X//' >sample.config/bin.mcc-sender-fr << 'END-of-sample.config/bin.mcc-sender-fr' XFrom: user@I_NEED_TO_CONFIGURE_NEWSBOT (My Full Name) XTo: %moderated-sender% XSubject: %re-subject% XPrecedence: junk XOrganization: Usenet canal historique XX-Bot: %newsbot-version% XMime-Version: 1.0 XContent-Type: text/plain;charset=ISO-8859-1 XContent-Transfer-Encoding: 8bit X XIt seems you have posted a %article-size% bytes binary file as Xarticle %header-message-id%. X XThis article has been autocancelled, as one or more of the groups you Xposted to are general newsgroups, were binaries should not be posted. X XPlease use the appropriate hierarchy, for example alt.binaries.*, and Xdon't crosspost. X XThis message has been automatically generated. X X--- XVous avez apparemment envoyi un fichier binaire de %article-size% octets Xsous l'identification %header-message-id%. X XL'article a iti annuli, car un ou plusieurs des groupes destinataires ne Xsont pas destinis aux binaires. Veuillez utiliser la hiirarchie appropriie, Xpar exemple alt.binaries.*, et ne pas faire de crossposts. X XCe message a iti giniri automatiquement. X X---- copy of article follows ---- X%article%---- end of article ---- END-of-sample.config/bin.mcc-sender-fr echo x - sample.config/bin.mch-frbin sed 's/^X//' >sample.config/bin.mch-frbin << 'END-of-sample.config/bin.mch-frbin' XFrom: newsmaster XTo: fr-binaires@freenix.fr XSubject: BINAIRE dans fr.* : %header-subject% XOrganization: Usenet canal historique XX-Bot: %newsbot-version% XMime-Version: 1.0 XContent-Type: text/plain;charset=ISO-8859-1 XContent-Transfer-Encoding: 8bit X XUn article de %article-size% octets vient d'jtre ditectj X` news.univ-angers.fr dans un groupe de fr.*. X XL'article a iti annuli, et un article explicatif imis dans fr.usenet.divers. X X---- copie des entjtes ---- X%article-headers%---- fin des entjtes ---- END-of-sample.config/bin.mch-frbin echo x - sample.config/bin.mh-newsmaster sed 's/^X//' >sample.config/bin.mh-newsmaster << 'END-of-sample.config/bin.mh-newsmaster' XFrom: newsmaster XTo: newsmaster XSubject: binary file: %header-subject% XOrganization: Usenet canal historique XX-Bot: %newsbot-version% XMime-Version: 1.0 XContent-Type: text/plain;charset=ISO-8859-1 XContent-Transfer-Encoding: 8bit X X%article-size% bytes binary file detected as article %header-message-id%. X X---- copy of article headers follows ---- X%article-headers%---- end of article headers ---- END-of-sample.config/bin.mh-newsmaster echo x - sample.config/bin.mh-sender-fr sed 's/^X//' >sample.config/bin.mh-sender-fr << 'END-of-sample.config/bin.mh-sender-fr' XFrom: user@I_NEED_TO_CONFIGURE_NEWSBOT (My Full Name) XTo: %moderated-sender% XSubject: %re-subject% XPrecedence: junk XOrganization: Usenet canal historique XX-Bot: %newsbot-version% XMime-Version: 1.0 XContent-Type: text/plain;charset=ISO-8859-1 XContent-Transfer-Encoding: 8bit X XIt seems you have posted a %article-size% bytes binary file as Xarticle %header-message-id%. X XOne or more of the groups you posted to are general newsgroups, were Xbinaries should not be posted. X XPlease use the appropriate hierarchy, for example alt.binaries.*, and Xdon't crosspost. X XIt would be better to cancel this article using the appropriate option of Xyour newsreader. X XThis message has been automatically generated. X X--- XVous avez apparemment envoyi un fichier binaire de %article-size% octets Xsous l'identification %header-message-id%. X XUn ou plusieurs des groupes destinataires ne sont pas destinis aux Xbinaires. Veuillez utiliser la hiirarchie appropriie, par exemple Xalt.binaries.*, et ne pas faire de crossposts. X XIl serait trhs recommandi d'annuler cet article en utilisant Xl'option adiquate de votre logiciel de lecture de news. X XCe message a iti automatiquement giniri. X X---- copy of article headers follows ---- X%article-headers%---- end of article headers ---- END-of-sample.config/bin.mh-sender-fr echo x - sample.config/bin.p-expl-frud sed 's/^X//' >sample.config/bin.p-expl-frud << 'END-of-sample.config/bin.p-expl-frud' XNewsgroups: fr.usenet.divers XSubject: BINAIRE DE %article-sender% XOrganization: Usenet canal historique XMessage-Id: . XVous pouvez vous abonner ` cette liste en envoyant : subscribe fr-binaires X ` : majordomo@freenix.fr X X---- copie des entjtes de l'article ---- X%article-headers%---- fin des entjtes ---- END-of-sample.config/bin.p-expl-frud echo x - sample.config/binkill.conf sed 's/^X//' >sample.config/binkill.conf << 'END-of-sample.config/binkill.conf' X# $Id: binkill.conf,v 1.3 1996/09/17 20:34:13 pb Exp $ X# X# To configure this to your system, copy this file as newsbot.conf, X# then look for and adapt the following strings: X# X# USER@I_NEED_TO_CONFIGURE_NEWSBOT X# (for Approved:, X-Canceled-By: and possibly From:) X# -> a mail address to reach you X# X# USER (end of Path: for generated cancels) X# -> something nice. 'not-for-mail' if you like. X# X# rnews -> path to your rnews X# uuencode -> path to your uuencode X# sendmail -> path to your sendmail X# Mail -> path of some mail user agent X# X# frcheck! -> feed name in your 'newsfeeds' file X# X# If you are issuing cancels in the fr.* hierarchy, uncomment the last line X# (it's used to mail reports for later posting to fr.usenet.divers). X# X##### X# X# Pattern matching on article headers X X# test if an article is approved X~ notappr Approved ^$ X X##### X# X# Patterns (as used in actions) X# Patterns not declared here are read from files with the same name X# in the pattern directory. X# X# First line begins with 'P' followed by the pattern name X# Following lines begin with '='. X# X X##### X# X# Pattern for global cancel X# XPbin.cancel-global X=Path: cyberspam!bincancel!USER X=Newsgroups: %header-newsgroups% X=From: %header-from% X=%if-header-sender%Sender: %header-sender% X=%endif%%if-header-approved%Approved: USER@I_NEED_TO_CONFIGURE_NEWSBOT X=%endif%Subject: cancel: %header-subject% X=Message-Id: %cancel-message-id% X=Control: cancel %header-message-id% X=Date: %date% X=X-Canceled-By: USER@I_NEED_TO_CONFIGURE_NEWSBOT X=X-Bot: %newsbot-version% X= X=%article-size% bytes binary in discussion group (%article-encoding%) X X##### X# X# Pattern for cancel reports sent to fr-bincancel@bincancel.news.eu.org X# XPlog.headers X=%article-size% X=%article-headers% X X##### X# X# Pattern for mail copy to sender (disabled) X# XPbin.mcc-sender-fr X=From: USER@I_NEED_TO_CONFIGURE_NEWSBOT X=To: %moderated-sender% X=Subject: %re-subject% X=Precedence: junk X=Organization: Usenet canal historique X=X-Bot: %newsbot-version% X=Mime-Version: 1.0 X=Content-Type: text/plain;charset=ISO-8859-1 X=Content-Transfer-Encoding: 8bit X= X=It seems you have posted a %article-size% bytes binary file as X=article %header-message-id%. X= X=This article has been autocancelled, as one or more of the groups you X=posted to are general newsgroups, were binaries should not be posted. X= X=Please use the appropriate hierarchy, for example alt.binaries.*, and X=don't crosspost. X= X=This message has been automatically generated. X= X=--- X=Vous avez apparemment envoyi un fichier binaire de %article-size% octets X=sous l'identification %header-message-id%. X= X=L'article a iti annuli, car un ou plusieurs des groupes destinataires ne X=sont pas destinis aux binaires. Veuillez utiliser la hiirarchie appropriie, X=par exemple alt.binaries.*, et ne pas faire de crossposts. X= X=Ce message a iti giniri automatiquement. X= X=---- copy of article follows ---- X=%article%---- end of article ---- X X##### X# X# Pattern for header copy to newsmaster (disabled) X# XPbin.mh-newsmaster X=From: newsmaster X=To: newsmaster X=Subject: binary file: %header-subject% X=Organization: Usenet canal historique X=X-Bot: %newsbot-version% X=Mime-Version: 1.0 X=Content-Type: text/plain;charset=ISO-8859-1 X=Content-Transfer-Encoding: 8bit X= X=%article-size% bytes binary file detected as article %header-message-id%. X= X X##### X# X# Pattern for header copy to fr-binaires X# XPbin.mch-frbin X=From: newsmaster X=To: fr-binaires@freenix.fr X=Subject: BINAIRE dans fr.* : %header-subject% X=Organization: Usenet canal historique X=X-Bot: %newsbot-version% X=Mime-Version: 1.0 X=Content-Type: text/plain;charset=ISO-8859-1 X=Content-Transfer-Encoding: 8bit X= X=Un article de %article-size% octets vient d'jtre ditecti X=ici dans un groupe de fr.*. X= X=L'article a iti annuli, et un article explicatif imis dans fr.usenet.divers. X= X=---- copie des entjtes ---- X=%article-headers%---- fin des entjtes ---- X= X=---- copy of article headers follows ---- X=%article-headers%---- end of article headers ---- X X##### X# Actions X# X# A ">" in front of "pipe to" means to append to the indicated file. X# actions pattern file pipe to X# X X# send full article to sender and warn for cancel regarding bin X# (disabled) XAbin.mcc-sender-fr bin.mcc-sender-fr /usr/sbin/sendmail -t -oi X# send headers to local newsmaster regarding bin X# (disabled) XAbin.mh-newsmaster bin.mh-newsmaster /usr/sbin/sendmail -t -oi X# send headers to fr-binaires@freenix.fr regarding bin X# (disabled) XAbin.mch-frbin bin.mch-frbin /usr/sbin/sendmail -t -oi X X# send uuencoded copy of headers to fr-bincancel@bincancel.news.eu.org X# USE ONLY IF ISSUING CANCELS OVER fr.* ! XAmail.fr.bin log.headers /usr/bin/uuencode cancel|/usr/bin/Mail fr-bincancel@bincancel.news.eu.org X X# Global bincancel XAbin.cancel-global bin.cancel-global /bin/rnews X X################### X# X# Checks over fr.* X# XFfrcheck! X X# (This part is disabled) X# X# If over 3k, binary and valid-looking return address (condition F), X# warn and send a copy to sender with explanations. X#I>3k,B,F bin.mcc-sender-fr X# headers to local admin X#I>3k,B bin.mh-newsmaster X# mail explanation in french to fr-binaires@freenix.fr X#I>3k,B bin.mch-frbin X X# emit a global cancel if over 3k, binary, and not approved XI>3k,B,~notappr bin.cancel-global X# then send a cancel report to fr-bincancel@bincancel.news.eu.org X# ENABLE ONLY IF ISSUING CANCELS OVER fr.* ! X#I>3k,B,~notappr mail.fr.bin END-of-sample.config/binkill.conf echo x - sample.config/dup.cancel-local sed 's/^X//' >sample.config/dup.cancel-local << 'END-of-sample.config/dup.cancel-local' XPath: cyberspam!spewcancel!user XNewsgroups: %header-newsgroups% XFrom: %header-from% X%if-header-sender%Sender: %header-sender% X%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT X%endif%Subject: cancel: %header-subject% XMessage-Id: %cancel-message-id% XControl: cancel %header-message-id% XDate: %date% XX-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT XX-Bot: %newsbot-version% XDistribution: local X XLocal cancel, duplicate article. END-of-sample.config/dup.cancel-local echo x - sample.config/dup.mh-sender sed 's/^X//' >sample.config/dup.mh-sender << 'END-of-sample.config/dup.mh-sender' XFrom: user@I_NEED_TO_CONFIGURE_NEWSBOT (My Full Name) XTo: %moderated-sender% XSubject: %re-subject% XPrecedence: junk XOrganization: Usenet canal historique XX-Bot: %newsbot-version% XMime-Version: 1.0 XContent-Type: text/plain;charset=ISO-8859-1 XContent-Transfer-Encoding: 8bit X XIt seems you have posted a duplicate news article as article X%header-message-id%. X XThis article is exactly identical to another article you have just Xposted in the same group(s). Posting duplicate articles is generally Xa waste of network resources and makes newsgroups more difficult Xto read. X XThis might be due to your using flawed software. Netscape, for Xexample, is typically known to post duplicate messages to Usenet Xvery easily. X XIf you can, please cleanup this article by using the "CANCEL" option Xof your newsreader. Old versions of Netscape don't have this feature. X XWhen you post an article, if the sending results in an error message: X - cancel the article, just in case X (it may have been posted anyway); X - resend it. X XArticles just sent are not always immediately visible in your Xnewsreader, there can be a delay ranging from a few seconds to a Xfew minutes. X XThanks. X XThis message has been automatically generated. X X---- copy of article headers follows ---- X%article-headers%---- end of article headers ---- END-of-sample.config/dup.mh-sender echo x - sample.config/from.cancel-local sed 's/^X//' >sample.config/from.cancel-local << 'END-of-sample.config/from.cancel-local' XPath: cyberspam!user XNewsgroups: %header-newsgroups% XFrom: %header-from% X%if-header-sender%Sender: %header-sender% X%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT X%endif%Subject: cancel: %header-subject% XMessage-Id: %cancel-message-id% XControl: cancel %header-message-id% XDate: %date% XX-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT XX-Bot: %newsbot-version% XDistribution: local X XLocal cancel, bad From:/Reply-To: field. END-of-sample.config/from.cancel-local echo x - sample.config/html.mh-sender-fr sed 's/^X//' >sample.config/html.mh-sender-fr << 'END-of-sample.config/html.mh-sender-fr' XFrom: user@I_NEED_TO_CONFIGURE_NEWSBOT (My Full Name) XTo: %moderated-sender% XSubject: %re-subject% XPrecedence: junk XOrganization: Usenet canal historique XX-Bot: %newsbot-version% XMime-Version: 1.0 XContent-Type: text/plain;charset=ISO-8859-1 XContent-Transfer-Encoding: 8bit X XVous avez apparemment envoyi sur Usenet un article au format HTML sous Xl'identification %header-message-id%. X XLe HTML est un format congu pour les documents hypertexte et notamment Xle World Wide Web. Il ne s'agit pas d'un format standard pour les news. X XLe HTML n'est pas lisible par la plupart des logiciels de news Xet il est donc conseilli de l'iviter sur Usenet si vous voulez que Xtous les utilisateurs puissent vous lire. X XSi vous le disirez, vous pouvez annuler cet article en utilisant l'option Xadiquate de votre logiciel de lecture de news. X XCe message a iti giniri automatiquement. X X---- entjtes de l'article ---- X%article-headers%---- fin des entjtes ---- END-of-sample.config/html.mh-sender-fr echo x - sample.config/log.article sed 's/^X//' >sample.config/log.article << 'END-of-sample.config/log.article' X%article% END-of-sample.config/log.article echo x - sample.config/log.headers sed 's/^X//' >sample.config/log.headers << 'END-of-sample.config/log.headers' X%article-size% X%article-headers% END-of-sample.config/log.headers echo x - sample.config/newsbot.conf sed 's/^X//' >sample.config/newsbot.conf << 'END-of-sample.config/newsbot.conf' X# $Id: newsbot.conf,v 1.7 1996/09/17 20:34:15 pb Exp $ X# X# This is intended purely as a SAMPLE config, DO NOT use "as is" ! X# X##### X# Customize header fields to keep track of (used in header matching) X# Builtin (already known) headers are: X# X# From X# Subject X# Newsgroups X# Message-Id X# Date X# References X# Sender X# Reply-To X# Approved X# XHContent-Type XHContent-Transfer-Encoding X X##### X# Pattern matching on article headers X# X# options: X# e complete regular expression (else, simple substring match X# with special characters ^ and $ recognized) X# i ignore case X# X# options name header field string to look for X# (empty == field not present) X# X~i qp Content-Transfer-Encoding ^quoted-printable$ X~i html Content-Type html X~ notappr Approved ^$ X~ bbs From @brokenbbs.org X X# Pattern matching on article body X# Same options as above except the header name is replaced by a line range. X# Currently, only 'integer' or 'integer-integer' are recognized forms. X XL spew 1 ^@FROM : XL mmf 1-60 CASH X X##### X# X# Patterns (as used in actions) X# Patterns not declared here are read from files with the same name X# in the pattern directory. X# X# First line begins with 'P' followed by the pattern name X# Following lines begin with '='. X# X XPbin.cancel-global X=Path: cyberspam!bincancel!user X=Newsgroups: %header-newsgroups% X=From: %header-from% X=%if-header-sender%Sender: %header-sender% X=%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT X=%endif%Subject: cancel: %header-subject% X=Message-Id: %cancel-message-id% X=Control: cancel %header-message-id% X=Date: %date% X=X-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT X=X-Bot: %newsbot-version% X= X=%article-size% bytes binary in discussion group (%article-encoding%) X XPlog.article X=%article% X X##### X# Actions X# X# A ">" in front of "pipe to" means to append to the indicated file. X# actions pattern file pipe to X# X X# send headers to sender regarding bin XAbin.mh-sender-fr bin.mh-sender-fr /usr/sbin/sendmail -t -oi X# send full article to sender and warn for cancel regarding bin XAbin.mcc-sender-fr bin.mcc-sender-fr /usr/sbin/sendmail -t -oi X# send headers to local newsmaster regarding bin XAbin.mh-newsmaster bin.mh-newsmaster /usr/sbin/sendmail -t -oi X# send headers to fr-binaires@freenix.fr regarding bin XAbin.mch-frbin bin.mch-frbin /usr/sbin/sendmail -t -oi X X# Post explanation to fr.usenet.divers about bincancel XAbin.p-expl-frud bin.p-expl.frud /usr/local/news/inews -h X X# Local bincancel XAbin.cancel-local bin.cancel-local /bin/rnews X# Global bincancel XAbin.cancel-global bin.cancel-global /bin/rnews X X# send headers to local newsmaster regarding spew XAspew.mh-newsmaster spew.mh-newsmaster /usr/sbin/sendmail -t -oi X# Global spewcancel XAspew.cancel-global spew.cancel-global /bin/rnews X X# send headers to sender and explanation regarding dups XAdup.mh-sender dup.mh-sender /usr/sbin/sendmail -t -oi X X# send headers to sender and french explanation regarding use of QP XAqp.mh-sender-fr qp.mh-sender-fr /usr/sbin/sendmail -t -oi X# send headers to sender and french explanation regarding use of HTML XAhtml.mh-sender-fr html.mh-sender-fr /usr/sbin/sendmail -t -oi X X# Local dupcancel XAdup.cancel-local dup.cancel-local /bin/rnews X X# Logs XAlog.fr.badfrom log.headers >/usr/local/news/newsbot/log/fr.badfrom XAlog.fr.badref log.headers >/usr/local/news/newsbot/log/fr.badref XAlog.fr.bin log.headers >/usr/local/news/newsbot/log/fr.bin XAlog.fr.dup log.headers >/usr/local/news/newsbot/log/fr.dup XAlog.fr.html log.headers >/usr/local/news/newsbot/log/fr.html XAlog.fr.qp log.headers >/usr/local/news/newsbot/log/fr.qp X XAlog.badfrom log.headers >/usr/local/news/newsbot/log/badfrom XAlog.badref log.headers >/usr/local/news/newsbot/log/badref XAlog.bin log.headers >/usr/local/news/newsbot/log/bin XAlog.dup log.headers >/usr/local/news/newsbot/log/dup XAlog.html log.headers >/usr/local/news/newsbot/log/html XAlog.qp log.headers >/usr/local/news/newsbot/log/qp X X################### X# X# Conditions: X# X# >size bigger than size X# B is a binary X# D duplicate message X# F from ok X# R references ok X# Msize estimated multipart size bigger than 'size'. X# M multipart X# ~name named expression matches (matches on headers) X# Lname named expression matches (matches on article body) X# X X################### X# X# actions on "default" feed (all articles fed to us by INN) go there X# X# (none) X X################### X# X# Checks over fr.* X# XFfrcheck! X X################### X# Warn and log on HTML and QP X# X X# Warn by mail in french if Content-Transfer-Encoding is QP XI~qp,F,~notappr qp.mh-sender-fr XI~qp log.fr.qp X X# Warn by mail in french if Content-Type is html XI~html,F,~notappr html.mh-sender-fr XI~html log.fr.html X X# other logs for fr.* stats XID log.fr.dup XI!F log.fr.badfrom XI!R log.fr.badref X X################### X# Binary over 15k X X# If return address looks valid, warn and send a copy to sender with X# explanations in french about bincancel XI>15k,B,F bin.mcc-sender-fr X# headers to local admin XI>15k,B bin.mh-newsmaster X# post explanation in french to fr.usenet.divers X, bin.p-expl-frud X# mail explanation in french to fr-binaires@freenix.fr X, bin.mch-frbin X# log headers X, log.fr.bin X# global bincancel if not approved X,~notappr bin.cancel-global X# don't continue XI>15k,B stop X X# Spew cancel XI~bbs,Lspew spew.cancel-global X, bin.mh-newsmaster X X################### XFnotfrcheck! X# Warn sender on duplicate posts and log for report X#ID,F,~notappr dup.mh-sender XID log.dup X# Bad From: XI!F log.badfrom X# Bad References: XI!R log.badref X# QP XI~qp log.qp X# HTML XI~html log.html END-of-sample.config/newsbot.conf echo x - sample.config/qp.mh-sender-fr sed 's/^X//' >sample.config/qp.mh-sender-fr << 'END-of-sample.config/qp.mh-sender-fr' XFrom: user@I_NEED_TO_CONFIGURE_NEWSBOT (My Full Name) XTo: %moderated-sender% XSubject: %re-subject% XPrecedence: junk XOrganization: Usenet canal historique XX-Bot: %newsbot-version% XMime-Version: 1.0 XContent-Type: text/plain;charset=ISO-8859-1 XContent-Transfer-Encoding: 8bit X XVous avez apparemment envoyi sur Usenet un article encodi en "quoted-printable" Xsous l'identification %header-message-id%. X XLe "quoted-printable" est un format congu pour le courrier ilectronique. XIl ne s'agit pas d'un format standard pour les news. X XLe "quoted-printable" n'est pas lisible par la plupart des logiciels Xde news et il est donc conseilli de l'iviter sur Usenet si vous Xvoulez que tous les utilisateurs puissent vous lire. X XIl serait prifirable de configurer votre lecteur de news pour utiliser, X` la place du "quoted-printable", l'encodage "8bit" universellement Xaccepti. X XSi vous le disirez, vous pouvez annuler cet article en utilisant l'option Xadiquate de votre logiciel de lecture de news. X XCe message a iti giniri automatiquement. X X---- entjtes de l'article ---- X%article-headers%---- fin des entjtes ---- END-of-sample.config/qp.mh-sender-fr echo x - sample.config/spew.cancel-global sed 's/^X//' >sample.config/spew.cancel-global << 'END-of-sample.config/spew.cancel-global' XPath: cyberspam!spewcancel!user XNewsgroups: %header-newsgroups% XFrom: %header-from% X%if-header-sender%Sender: %header-sender% X%endif%%if-header-approved%Approved: user@I_NEED_TO_CONFIGURE_NEWSBOT X%endif%Subject: cancel: %header-subject% XMessage-Id: %cancel-message-id% XControl: cancel %header-message-id% XDate: %date% XX-Canceled-By: user@I_NEED_TO_CONFIGURE_NEWSBOT XX-Bot: %newsbot-version% X XSpew cancelled. END-of-sample.config/spew.cancel-global echo x - sample.config/spew.mh-newsmaster sed 's/^X//' >sample.config/spew.mh-newsmaster << 'END-of-sample.config/spew.mh-newsmaster' XFrom: newsmaster XTo: newsmaster XSubject: spew: %header-subject% XOrganization: Usenet canal historique XX-Bot: %newsbot-version% XMime-Version: 1.0 XContent-Type: text/plain;charset=ISO-8859-1 XContent-Transfer-Encoding: 8bit X XSpew detected as article %header-message-id%. X X---- copy of article headers follows ---- X%article-headers%---- end of article headers ---- END-of-sample.config/spew.mh-newsmaster exit -- Pierre Beyssac pb@fasterix.frmug.fr.net pb@fasterix.freenix.fr {Free,Net,Open}BSD, Linux : il y a moins bien, mais c'est plus cher Free domains: http://www.eu.org/ or mail dns-manager@EU.org