Subject: vm-pop and NTemacs heap

From: [email-masked]
Date: Mon, 30 Dec 1996 15:16:50 -0500

GNUemacs 19.34 on Win95; VM 5.96 (beta).

One of the lucky things I did during bringing up vm-pop.el to do POP3
from my laptop was to try to download a somewhat long message (around
1.5 MB).  NTemacs (on Win95) consistently croaked with a message
something like "process space exhausted".  Now, I have 32 MB of real
RAM and another 30 MB of swap space, so I figured something had to be
going on.  Also, I'd had no problem editing bigger messages on the
same machine (they'd been dragged over via something other than
vm-pop).

After a little investigation I found that vm-pop was doing the more or
less standard process filter stuff of just iterating and stuffing the
bits and pieces into a buffer as they came over the network.  On my
network, they were coming at 1-2k per iteration of the process
filter.  This results in about a 1000 inserts at the end of the
process buffer for that 1.5 MB message.

I wanted to slow vm-pop down so it would work on bigger chunks.  Here is
my inelegant solution.  The beginning of this function in vm-pop.el
now looks like this in my copy:

(defun vm-pop-retrieve-to-crashbox (process crash)
  (let ((start vm-pop-read-point) end)
    (goto-char start)
    (while (not (re-search-forward "^\\.\r\n" nil t))
      (accept-process-output process)
	  ;; [email-masked] ... to save wear and tear on the heap
	  (if (> (buffer-size)  20000) (sleep-for 1))
	  (if (> (buffer-size)  50000) (sleep-for 1))
	  (if (> (buffer-size) 100000) (sleep-for 1))
	  (if (> (buffer-size) 200000) (sleep-for 1))
	  (if (> (buffer-size) 500000) (sleep-for 1))
	  ;; [email-masked]
      (goto-char start))

This probably isn't needed on most other platforms, but it did the
trick on Win95.  The effect is to sleep more the bigger the received
message is.  The numbers are just completely guessed-at and are not
based on some keen analysis of anything.
-- 
  [email-masked]                    LZ 1C-320
  [email-masked]      (Voice/Fax) +1.908.576.2932
  AT&T Labs       /       AT&T WorldNet Service

---

From: [email-masked]
Date: Mon, 6 Jan 1997 10:36:33 -0500

gv> I'm not sure I understand why the default behavior works for other
gv> platforms but doesn't work for Win95; I suppose I also don't
gv> understand why 1000 inserts would use significantly more memory
gv> than one large insert.  Is there some state that is being kept
gv> around on each insert?  And is this state not garbage-collectable?

I don't know that it doesn't fail for other platforms since I have
only UNIX and Win95 to test on (and, you know how that UNIX takes a
lickin' and keeps on tickin').  I'm just guessing that it might be
failing under Win95 due to some bug in Win95 memory management or
whatever.  I figured if it were failing under NT, someone else
would've complained by now.  Maybe not; you have to download a pretty
big message before it fails.

As far as I can see, there is no per-iteration state being kept.  VM
is using a process filter that is just inserting the chunks at the end
of a buffer as they come in.  The only other thing it's doing is
looking for the end of the data.  VM's behavior looks proper and
ordinary to me.
-- 
  [email-masked]                    LZ 1C-320
  [email-masked]      (Voice/Fax) +1.908.576.2932
  AT&T Labs       /       AT&T WorldNet Service