Subject: win95 networking, vm-pop, and smtpmail

From: [email-masked]
Date: Mon, 30 Dec 1996 15:03:11 -0500

I recently wrote that I sometimes get the Win95 "blue screen" when
mixing POP3 (vm-pop) and SMTP (smtpmail) in an NTemacs session.  (I
also wrote that having separate NTemacs invocations for sending and
receiving seemed to help, but that was a red herring and didn't
actually help.)  

I think I have found the source of the problem.  Win95 comes with a
UNIX-like "netstat" utility, so you can look at open connections and so
on.  I noticed that other (non-NTemacs) applications opened
connections which disappeared just about as soon as they closed.  For
vm-pop or smtpmail connections, the socket instead went into a
TIME_WAIT state for a while.  After a couple minutes, the socket would
go away.  I speculate that having too many of these sockets in this
condition (or using unlucky socket numbers or whatever) triggers some
bug in Win95 winsock.  smtpmail connections aggravate the problem
because each piece of email is sent with a separate connection and
SMTP session (whereas vm-pop retrieves all mail in a single POP3
session).

Ok, if that's the case, what to do about it?  I speculated that the
reason the sockets were going to TIME_WAIT was that they had some data
left to be read from them, but the NTemacs stream reader wouldn't
oblige (the process is already deleted).  In both the vm-pop and
smtpmail cases, the elisp sends a QUIT command, reads a (structured)
response, and then nukes the process.  My very inelegant fix is to put
a sleep of 2 seconds before the delete-process.  That allows the data
to drain from the socket even though it doesn't actually go to the
process filter (which doesn't care in this context).

For vm-pop.el, the end of the function vm-pop-move-mail now looks like
this in my copy:

      (if process
	  (save-excursion
	    (set-buffer (process-buffer process))
	    (vm-pop-send-command process "QUIT")
	    (vm-pop-read-response process)
		;; [email-masked]
		(sleep-for 2)  ;; to drain stream
	    (delete-process process))))))

For smtpmail.el, the end of smtpmail-via-smtp now looks like this:

      (if process
	  (save-excursion
	    (set-buffer (process-buffer process))
	    (smtpmail-send-command process "QUIT")
		;; [email-masked]
		(sleep-for 2)  ;; to drain stream
	    (smtpmail-read-response process)

;	    (if (or (null (car (setq response-code (smtpmail-read-response process))))
;		    (not (integerp (car response-code)))
;		    (>= (car response-code) 400))
;		(throw 'done nil)
;	      )
	    (delete-process process))))))

I just noticed now that I put the sleep in slightly different places
in the two files, but it doesn't seem to matter.  I've been using it
this way all day, and no more sockets stick around in TIME_WAIT.  And,
so far, no more "blue screen".

If anyone codes up a more elegant solution than (sleep-for 2), please
send it along.  I can pay the price of 2 seconds per connection, but
I'm not sure whether 2 is the right number for all situations.  And,
of course, it doesn't feel too good to do those sleeps.
-- 
  [email-masked]                    LZ 1C-320
  [email-masked]      (Voice/Fax) +1.908.576.2932
  AT&T Labs       /       AT&T WorldNet Service

---

From: [email-masked]
Date: Tue, 31 Dec 1996 11:51:33 -0500

wjc> Ok, if that's the case, what to do about it?  I speculated that
wjc> the reason the sockets were going to TIME_WAIT was that they had
wjc> some data left to be read from them, but the NTemacs stream
wjc> reader wouldn't oblige (the process is already deleted).  In both
wjc> the vm-pop and smtpmail cases, the elisp sends a QUIT command,
wjc> reads a (structured) response, and then nukes the process.  My
wjc> very inelegant fix is to put a sleep of 2 seconds before the
wjc> delete-process.  That allows the data to drain from the socket
wjc> even though it doesn't actually go to the process filter (which
wjc> doesn't care in this context).

A friend of mine, who has spent a lot more time in RFC-793 than I
have, clued me in to what was probably actually happening.  Since the
POP3 client and the SMTP client are both sending "QUIT" and then
reading the response from the server, there is a race condition about
shutting down the socket.  Both the client and the server shut down
the socket at nearly the same time.  If the client happens to see the
FIN packet from the server, it can just nuke the socket.  If it
doesn't see it, it must send a FIN and enter the TIME_WAIT state.
Since the server side is already gone, the TIME_WAIT state has to wait
to time out (a couple minutes).

My hack to sleep 2 seconds gives enough time that the server is
always seen as shutting down the socket, and the TIME_WAIT is
avoided.  This probably doesn't matter most places, but I imagine
there is a problem with Win95 coping with (too many?  unlucky?  bad
vibes?) sockets in that state.
-- 
  [email-masked]                    LZ 1C-320
  [email-masked]      (Voice/Fax) +1.908.576.2932
  AT&T Labs       /       AT&T WorldNet Service

---

From: [email-masked]
Date: Mon, 6 Jan 1997 10:48:40 -0500

gv> I apologize for answering your mail with more questions.  I'm
gv> still not sure I understand the scenario yet.  You have a POP3
gv> client and an SMTP client (to read and send mail at the same
gv> time?).  Are they both using the same socket to the server?  I
gv> would have thought that they would each have their own connection.

I have both an SMTP client (distribution smtpmail.el) and a POP3
client (vm-pop.el from standard VM).  I don't use them at the same
time or to the same socket or whatever.  I use them in the standard
ways (SMTP port 25, POP3 port 110).  The fact that I happen to be
using both within the same GNUemacs invocation probably doesn't cause
the problem; it just aggravates it.

So, I still speculate that the problem is caused by some bug in Win95
networking.  It happens to be stimulated by the race condition
inherent in the SMTP and POP3 protocols, where it's not clear who's
going to close the connection (both ends close it, but it's not
synchronized).

My hack is just a workaround that eliminates the race condition and
makes it look like the server side always shuts down the connection
first.
-- 
  [email-masked]                    LZ 1C-320
  [email-masked]      (Voice/Fax) +1.908.576.2932
  AT&T Labs       /       AT&T WorldNet Service