Subject: Re: [[email-masked]: Emacs 20.0.92 on Windows NT 4.0: error converting cr-lf]

From: Marc Fleischeuers <[email-masked]>
Date: 01 Aug 1997 10:07:17 +0200

>     I'm not sure I understand what you are trying to do.  When a file is
>     inside of Emacs, line are always terminated by newlines.  The line
>     termination that exists when the file is in the filesystem is only
>     placed there when the file is written out.  There is no need to
>     explicitly place CR or LF characters in a file to change the
>     termination used.
> 
> You're right--but perhaps this can be a clue to finding a place
> where the documentation needs to be made clearer.  So it is worth figuring
> out why Marc got the wrong idea.

My intent was too straightforward, obviously. I noticed some problems
with msdos-type files (note: these files were not created with emacs,
but with other programs most notably netscape). I knew about the cr-lf
line ending convention so in an attempt to create a msdos file in
emacs, I ended lines with an explicit `C-q Cm C-q C-j'. Please note
that this works as expected in emacs 19.33 (i386-*-Win NT 4.0).

The variables Geoff mentioned (buffer-file-type and
coding-system-for-write) have sent me off on a chase though emacs'
help. Skip to the last paragraph if you are not interested in the dead
ends.

First, `C-h v buffer-file-type' mentions that this is a MS-DOG and
Windows NT-only variable, and that it's value is nil. I tried to set
the variable with M-x set-variable RET buffer-file-type but when I
press return all I get is [no match]. I don't think this is a great
loss though, surely with so many advanced encoding and decoding
facilities, there is no more need for MSDOG as a special case?

On to `coding-system-for-write'. The documentation mentions that this
is a variable of internal use only. Setting it would probably require
lisp. The appropriate values for this variable should be taken from
`coding-system-alist'. There is however no documentation for this
variable (`C-h v coding-system-alist' -> [no match]). Still, an
internal variable is not the first thing to use if I want to creat an
ms-dos file. 

Apropos'ing around I found another promising variable,
`buffer-file-format', valid values for which are found in
`format-alist'. In this alist there seems to be an appropriate format,
`ibm'. However, `M-x set-variable RET buffer-file-format' again gives
[no match]. 

What I should have used all along was `M-x
set-buffer-file-coding-system RET iso-latin-1-dos'. This function is
accessible from the menu ([menu-bar mule set-various-coding-systems
set-buffer-file-coding-system]) and from the C-x RET keymap. However,
it was only from the resulting file that I could see that it was what
I wanted (in fact there may still be a better way). The documentation
for `set-buffer-file-coding-system' does not mention to what values it
can be set, and the description in `M-x describe-coding-system' does
not mention what any of the listed coding systems do. In fact, after
I selected iso-latin-1-dos, it was described as

Current buffer file: buffer-file-coding-system
 - -- undecided-dos

The short answer is the documentation for describe-coding-system and
set-*-coding-system could be improved upon. For
describe-coding-system, why is it necessary to mention the priority of
coding systems? Instead, use the space to explain what the selected
coding systems do. For `set-*-coding-systems', it could be mentioned
to what values it can be set, and perhaps what they do.

Marc
-- 
Computer! End program!
Computer! Create _new_ program!

---

From: Richard Stallman <[email-masked]>
Date: Sat, 2 Aug 1997 06:18:47 -0400

I improved the doc of this command.
But that won't fully solve the problem.

What I really should do is to point you at this command from somewhere
else that you would naturally look.

Any suggestions for where that could be?

---

From: Richard Stallman <[email-masked]>
Date: Sun, 3 Aug 1997 00:23:09 -0400

I knew about the cr-lf

Is that really true?  What algorithm does 19.33 use for LF to CRLF
conversion?  Maybe we should change the Emacs 20 EOL conversion to do
the same thing.

---

From: Kenichi Handa <[email-masked]>
Date: Mon, 4 Aug 1997 10:33:49 +0900

Date: Sun, 3 Aug 1997 00:23:09 -0400
   From: Richard Stallman <[email-masked]>

       I knew about the cr-lf

Since the above is the first mail I get about this thread, this reply
may fail to catch the point...

I don't know why the above doesn't work for Emacs 20.

I've just tried the following.
1) At first, visit a new file.
2) type `a b c C-q C-m C-q C-j'
3) save it.
4) visit it again.

Then the file is read as `undecided-dos' and the buffer contents are
4-byte of:
	abc\C-j
This means that CR LF is decoded to single LF.  But, since
buffer-file-coding-system is undecided-dos, when I edit this file and
save it, all LFs are encoded back to CR LF.

---
Ken'ichi HANDA
[email-masked]

---

From: Marc Fleischeuers <[email-masked]>
Date: 04 Aug 1997 10:42:27 +0200

> Any suggestions for where that could be?

The command is in the menu and in the advertised C-x RET keymap; I
think anyone inversitgating emacs' new features should find these
functions easily (I did't go there straightaway because I first
followed the suggestions by Geoff Voelker). In the menu-bar there is
even a corresponding `describe' entry for input method and coding
systems. I think this is a good thing, this is where I would look if I
were a user. In fact I did look there when I first started emacs 20;
it's just that the descriptions are not very informative about what
the functions actually do for me (input methods do not work (yet?) so
I cannot comment on that).

If the documentation for set-buffer-file-coding-system, and `M-x
describe-coding-system' give information about the available,
resp. selected coding systems and what they do for me, I think this
should do it.

Marc

---

From: Marc Fleischeuers <[email-masked]>
Date: 04 Aug 1997 11:40:11 +0200

> save it, all LFs are encoded back to CR LF.

This is the way it should be, unfortunately it is not for me. I have
repeated the four steps above. 

When I first open a new file, the buffer-file-coding-system is nil and
the mode-line indicator is `:'.  If I insert `a b c C-q C-m C-q C-j'
in the buffer and then save the file, the buffer contains the five
bytes `abc\C-m\C-j', buffer-file-coding-system is still nil, and the
mode-line indicator is still `:'. With `c:\emacs\bin\hexl abc', the
contents of the file is `6162 630d 0d0a'.

If I then re-visit the file (`C-x C-v RET') it contains six bytes
`abc\C-j\C-j\C-j', buffer-file-coding-system is `- -- undecided-mac',
and the mode-line indicator is `/'.

I started emacs with `c:\emacs\bin\emacs.bat --no-site-file
--no-init-file' The batch file sets a number of environment
variables. It is not modified from the one generated by the install
process. I use emacs 20.0.92 on Windows NT 4.0, compiled with MS VC++
4.2. 

I have also used the following version, started the same way, to
perform exactly the same steps:
In GNU Emacs 19.33.1 (i386-*-nt4.0) of Wed Aug 14 1996 on BANANA-FISH
configured using `configure NT'

The file is written and read back in as the five bytes `abc\C-m\C-y'.
There is a mode-line indicator `(T:', both when I first open the file
and when I read it back in.

Marc

---

From: Kenichi Handa <[email-masked]>
Date: Mon, 4 Aug 1997 20:37:31 +0900

From: Marc Fleischeuers <[email-masked]>
   Date: 04 Aug 1997 11:40:11 +0200

Hmm, the sequence CR LF was written out as CR CR LF.  It seems that
the file is opened by O_TEXT instead of O_BINARY.  But, this should
have been fixed in 20.0.92 already.  Strange...

Could you please check the file src/fileio.c?  Is it applied the
following patch made by <[email-masked]>?

------------------------------------------------------------
RCS file: RCS/fileio.c,v
retrieving revision 1.250
retrieving revision 1.251
diff -u -r1.250 -r1.251
--- fileio.c    1997/07/12 06:43:08     1.250
+++ fileio.c    1997/07/13 20:37:01     1.251
@@ -3799,8 +3799,7 @@
   struct gcpro gcpro1, gcpro2, gcpro3, gcpro4, gcpro5;
   struct buffer *given_buffer;
 #ifdef DOS_NT
-  int buffer_file_type
-    = NILP (current_buffer->buffer_file_type) ? O_TEXT : O_BINARY;
+  int buffer_file_type = O_BINARY;
 #endif /* DOS_NT */
   struct coding_system coding;
 
------------------------------------------------------------

This is an expected behaviour when Emacs reads `a b c CR CR LF'.  When
Emacs encounters CR not followed by LF, it thinks the end-of-line
format for the file is CR (Mac's convention), and translate CR to LF.
LF is read as is.  So, the buffer contains three LFs.

So, the problem seems to be in writing a file.

Anyway, I don't have Windows NT.  I asked a person who is an expert of
Windows to check the code.

---
Ken'ichi HANDA
[email-masked]

---

From: Marc Fleischeuers <[email-masked]>
Date: 04 Aug 1997 14:18:51 +0200

> following patch made by <[email-masked]>?

This patch is applied (that is, it says ``int buffer_file_type =
O_BINARY'', that's what it should be I think)

> So, the problem seems to be in writing a file.

In a previous post today, I have described how both 19.33 and 20.0.92
both write the same bytes to disk. If the way in which this file is
read in is correct (as I understand from your and RMS' posts) then a)
the way the cr and lf sequences are interpreted differs between 19.34
and 20.0.92, and b) this difference in reading, is indeed not matched
by an appropriate difference in writing.

---

From: Kenichi Handa <[email-masked]>
Date: Mon, 4 Aug 1997 21:54:40 +0900

From: Marc Fleischeuers <[email-masked]>
   Date: 04 Aug 1997 14:18:51 +0200

I have just found that lisp/dos-w23.el is doing something about
deciding coding system.  Although I have not yet read the code in
detail, I suspect that the code decides that coding system for writing
a file on NT is undecided-dos.  If it is true, it explains everything,
because Emacs writes CR as is and converts LF to CR LF when it writes
a file by undecided-dos.

Perhaps, Mr. Voelker wrote this code so that NT users don't have to do
special thing to create a DOS file.  In your case, you don't have to
insert \C-m by hand to creat a DOS file.

Please just try the followings:
1) visit a new file
2) type `a b c RET'
3) save the file.
4) visit the file again.

You should be able to create a file of `a b c CR LF' by step 3, and
buffer-file-coding-system is set to undecided-dos by step 4.

Mr. Voelker?  Is this correct?

---
Ken'ichi HANDA
[email-masked]

---

From: Marc Fleischeuers <[email-masked]>
Date: 04 Aug 1997 14:58:42 +0200

> > So, the problem seems to be in writing a file.

I have examined the value of the lisp-variable `buffer-file-type' in
several stages after reading and writing files containing cr and lf
sequences. The value of this variable was always nil, indicating a
text (i.e., non-binary) file. In buffer-file-type-alist it is set that
files with extension '.tpu' are interpreted as binary, so I tried 

C-x C-f new.tpu a b c C-q C-m C-q C-j C-x C-s C-x C-v RET

This file is created containing the intended 5 bytes, and it is read
back in "correctly" (buffer contains `abc^M'). After reading the file
in, the mode line indicator is `=:', and buffer-file-coding-system is
`= -- no-conversion (alias: binary)'.

May I argue that the concept of `buffer-file-type' and its associated
variables and functions are removed from emacs 20?

Marc

---

From: Marc Fleischeuers <[email-masked]>
Date: 04 Aug 1997 15:36:27 +0200

> a file on NT is undecided-dos.  If it is true, it explains everything,

I was reading there too. It appears that emacs does a lot of thinking
for me. I have done some light testing, and it looks like everything
acts like I expect it to, when (untranslated-file-p filename) returns
t. This is what I'll be doing for a while, until something else
breaks.. 

Marc

---

From: Richard Stallman <[email-masked]>
Date: Mon, 4 Aug 1997 15:46:48 -0400

> the file is opened by O_TEXT instead of O_BINARY.

I would expect this is because of the usual DOS eol conversion.
and not because of O_TEXT.

I am not surprised.

The same code in Emacs that converts just LF to CR LF
will of course do so when the preceding character is a CR--
unless there is something special to stop it.

As far as I know, there is nothing special
to avoid encoding LF as CR LF based on the preceding character.
Handa, is there?

---

From: Richard Stallman <[email-masked]>
Date: Mon, 4 Aug 1997 15:55:46 -0400

If it is true, it explains everything,
    because Emacs writes CR as is and converts LF to CR LF when it writes
    a file by undecided-dos.

Yes, of course.  I've been telling both of you this over and over.
You've been trying to unravel a mystery which is not a mystery at all.

The real question is, should we put in a special feature to override
that behavior when the buffer contains a CR?  Should DOS-style EOL
conversion recognize when the buffer contains CR LF, and output it as
CR LF (rather than CR CR LF)?

---

From: Richard Stallman <[email-masked]>
Date: Mon, 4 Aug 1997 16:05:23 -0400

t.

That sentence is ambiguous; it could mean there is a no problem, or it
could mean there is a serious problem.

untranslated-file-p is supposed to return t only when the file resides
on a file system that is mounted on a Unix-like system.  That is an
unusual case for an MSDOS user; therefore, it is not the really
important case.  The really important case is when untranslated-file-p
returns nil.

So let's focus on the most important question first: what happens when
untranslated-file-p returns nil?  Do you get correct behavior in all
cases?  If not, can you tell us precisely which case is still
incorrect?

---

From: Kenichi Handa <[email-masked]>
Date: Tue, 5 Aug 1997 09:29:44 +0900

Date: Mon, 4 Aug 1997 15:46:48 -0400
   From: Richard Stallman <[email-masked]>

You are right.  I didn't wrote such a special code.

Please note that I joined this discussion from halfway.

I don't like it because it's too kluge (can I use this word as an
adjective?).  But, if DOS users want it, it's not that difficult to
implement it.

---
Ken'ichi HANDA
[email-masked]

---

From: Richard Stallman <[email-masked]>
Date: Tue, 5 Aug 1997 02:30:07 -0400

This is the right behavior, as Emacs is currently designed.
It may not be quite the best feature, but it is not a bug.

This is a bug.  The presence of CR CR LF in the file
should not cause mac EOL conversion to be used.

I think that Emacs is being too quick to use mac EOL conversion.
I suspect that right now any CR not followed by LF does this.

If the file contains a LF anywhere near the beginning,
then mac EOL conversion should not be used.

Handa, can you fix this?

---

From: Richard Stallman <[email-masked]>
Date: Tue, 5 Aug 1997 02:31:37 -0400

If you write the file out and then visit it again,
you are performing two experiments in series
and you are telling only the result of the two of them.

That isn't really useful.  You need to tell us the result
of each experiment.

In other words, what exactly is in the file when you write it with
Emacs 19 in this way?  Is it a b c CR CR LF or a b c CR LF or what?

---

From: Kenichi Handa <[email-masked]>
Date: Tue, 5 Aug 1997 17:10:48 +0900

> I suspect that right now any CR not followed by LF does this.

Right.

> then mac EOL conversion should not be used.

Yes.  But how about LF CR LF or CR LF LF?  Should they be recognized
as DOS format or Unix format?

Hmmm, how about accumulating how many times each possible end-of-line
format appears, and select the one which first occurs 3 times?  If
none occurs 3 times, perhaps we should select the one occurs last.
Then,
	CR CR LF -> DOS
	LF CR LF -> DOS
	CR LF LF -> Unix
	CR LF CR -> Mac
---
Ken'ichi HANDA
[email-masked]

---

From: Marc Fleischeuers <[email-masked]>
Date: 05 Aug 1997 10:11:00 +0200

> That isn't really useful.  You need to tell us the result

The emacsen are on different machines, hence I can safely use the same
pathnames.

GNU Emacs 19.33.1 (i386-*-nt4.0) 	GNU Emacs 20.0.92.1 (i386-*-nt4.0)
started with:				started with:
C:\> c:\emacs\bin\emacs.bat -nw 	C:\> c:\emacs\bin\emacs.bat -nw
     --no-site-file --no-init-file	     --no-site-file --no-init-file 
					     
Input:					Input:
C-x C-f t . t RET a b c C-q RET RET	C-x C-f t . t RET a b c C-q RET RET
d e f C-q RET RET C-x C-s		d e f C-q RET RET C-x C-s

Buffer looks like:			Buffer looks like:

abc^M					abc^M
def^M					def^M

File contents:				File contents:
6162 630d 0d0a 6465 660d 0d0a		6162 630d 0d0a 6465 660d 0d0a

Input:					Input:
C-x C-v RET				C-x C-v RET

Buffer looks like:			Buffer looks like:
abc^M					abc
def^M
-------
					def

                                        -------

Marc

---

From: Marc Fleischeuers <[email-masked]>
Date: 05 Aug 1997 10:23:02 +0200

> as DOS format or Unix format?

And what about CR CR LF LF? LF CR CR LF? 
Yes I'm joking. However, after spending two days chasing a bug
eventually finding myself outwitted by emacs' intelligence in
dos-w32.el, I tend to think that emacs should not be too smart. 

If the distribution of CR and LF throughout the file do not form a
clear pattern, would `no conversion' be an option?

Marc

---

From: Richard Stallman <[email-masked]>
Date: Tue, 5 Aug 1997 04:38:02 -0400

Then the file is read as `undecided-dos' and the buffer contents are
    4-byte of:
	    abc\C-j
    This means that CR LF is decoded to single LF.

Are you doing this on DOS, or on Unix?

If you are on Unix, this behavior is correct, because on Unix new
files are normally written with no EOL conversion.

But if this happened on DOS, it woudl be a bug.

I think Marc told us this is not what happens for him.
Rather, the file is written with DOS EOL conversion,
which is correct according to the current specs.
Then reading the file again mistakenly uses Mac EOL conversion.

---

From: Kenichi Handa <[email-masked]>
Date: Tue, 5 Aug 1997 21:00:37 +0900

> Are you doing this on DOS, or on Unix?

I'm using Unix.

Right.

> Then reading the file again mistakenly uses Mac EOL conversion.

Yes, I now know it.

> dos-w32.el, I tend to think that emacs should not be too smart. 

I agree.

> clear pattern, would `no conversion' be an option?

To decide the pattern is clear or not is very difficult.  In addition,
we had better not do exhaustive scanning throughout the file.

So, I suggest the following code.  This scans buffer until it
encounters 3 end-of-lines.  If it founds two different patterns while
scanning, it decides not to decode end-of-line (by returning
CODING_EOL_LF).  So, in any of the following cases, it doesn't decode
end-of-line.
	CR CR LF,  LF CR LF, CR LF LF, CR CR LF LF, LF CR CR LF.
I think it is clear enough, and users won't be surprised that much.

What do you all think?

---
Ken'ichi HANDA
[email-masked]

--in src/coding.c------------------------------------------------------------

/* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
   is encoded.  Return one of CODING_EOL_LF, CODING_EOL_CRLF,
   CODING_EOL_CR, and CODING_EOL_UNDECIDED.  */

#define MAX_EOL_CHECK_COUNT 3

int
detect_eol_type (src, src_bytes)
     unsigned char *src;
     int src_bytes;
{
  unsigned char *src_end = src + src_bytes;
  unsigned char c;
  int total = 0;		/* How many end-of-lines are found so far.  */
  int eol_type = CODING_EOL_UNDECIDED;
  int this_eol_type;

  while (src < src_end && total < MAX_EOL_CHECK_COUNT)
    {
      c = *src++;
      if (c == '\n' || c == '\r')
	{
	  total++;
	  if (c == '\n')
	    this_eol_type = CODING_EOL_LF;
	  else if (src >= src_end || *src != '\n')
	    this_eol_type = CODING_EOL_CR;
	  else
	    this_eol_type = CODING_EOL_CRLF, src++;

	  if (eol_type == CODING_EOL_UNDECIDED)
	    /* This is the first end-of-line.  */
	    eol_type = this_eol_type;
	  else if (eol_type != this_eol_type)
	    /* The found type is different from what found before.
	       We had better not decode end-of-line.  */
	    return CODING_EOL_LF;
	}
    }

  return (total ? eol_type : CODING_EOL_UNDECIDED);
}

---

From: Andrew Innes <[email-masked]>
Date: Tue, 5 Aug 1997 17:00:38 +0100 (BST)

(Replying to several messages.)

Aside: I haven't had a chance to study the new coding support yet, so my
comments may be based on misunderstanding or ignorance of it.

>CR LF (rather than CR CR LF)?

I agree with Handa that Emacs should not do this.  Nearly all text files
will use a single end-of-line convention throughout, and thus pose no
problem.  I can't think of cirumstances in which a user would encounter
text files containing extra CR characters like this.  If such
cirumstances really are rare, then having to edit in "binary" mode where
all CRs are explicit seems reasonable.

(BTW, does Emacs 20 distinguish between text files in CODING_EOF_LF, and
binary files?  I think such a distinction is useful - a binary file
might contain all sorts of odd combinations of CR and LF, but a text
file should normally use a single convention throughout.)

>On Tue, 5 Aug 1997 21:00:37 +0900, Kenichi Handa <[email-masked]> said:

We don't want to do exhaustive scanning, but we can easily detect if the
end-of-line convention we choose based on the initial scan is not used
uniformly.  I would want any text file which appears not to use a single
convention to be handled using an information preserving convention,
ie. CODING_EOL_LF (or preferrably marked as a binary file, not a text
file using Unix line-endings, if that distinction is made).

>I think it is clear enough, and users won't be surprised that much.

I like this proposal.  In addition, or perhaps instead, I would want
insert-file-contents to notice if the chosen convention is not used
uniformly, and either report an error or possibly reread the file using
CODING_EOL_LF.  (Indeed, the existing buffer contents could be patched
up without rereading since it would be known that all previous lines
used the original convention.)

If it is not appropriate to signal an error, or revert the coding in
situ, then at least insert-file-contents should indicate in some way
(eg. by setting a variable to the number of non-conforming end-of-lines)
so that other functions could inform the user about the discrepancy.

I'm not sure whether the same sort of argument applies to subprocess
output or not.

AndrewI

---

From: Richard Stallman <[email-masked]>
Date: Tue, 5 Aug 1997 13:30:33 -0400

This question is less important.  Neither choice is horrible.

So please fix the Mac-format problem right away
and don't let it be delayed by other questions like this one.

---

From: Richard Stallman <[email-masked]>
Date: Tue, 5 Aug 1997 13:34:45 -0400

This does not fit the practical needs.

The most important practical need is to avoid ever using mac format
for a file which really should be in dos format.

Therefore, the right solution is never use mac format if you can see
any linefeed at all.

So if all you can find is CR, never LF, then use mac format.

Otherwise, if you every LF has a CR before it, use dos format.

Otherwise, use Unix format.

---

From: Richard Stallman <[email-masked]>
Date: Tue, 5 Aug 1997 14:18:03 -0400

I think this is good enough.  I'll install it now.

---

From: Richard Stallman <[email-masked]>
Date: Tue, 5 Aug 1997 15:53:20 -0400

binary files?

There is a distinction which perhaps you could interpret in this way:
whether no-conversion is specified as the coding system.

		   I think such a distinction is useful - a binary file
    might contain all sorts of odd combinations of CR and LF, but a text
    file should normally use a single convention throughout.)

What, specifically, is it useful for?

---

From: Kenichi Handa <[email-masked]>
Date: Wed, 6 Aug 1997 10:10:27 +0900

> I think this is good enough.  I'll install it now.

I have just made a small change as below in FSF's code:

diff -c -r1.30 coding.c
*** coding.c    1997/08/05 18:19:33     1.30
--- coding.c    1997/08/06 01:06:38
***************
*** 2739,2745 ****
        }
      }
  
!   return (total ? eol_type : CODING_EOL_UNDECIDED);
  }
  
--- 2739,2745 ----
        }
      }
  
!   return eol_type;
  }

---
Ken'ichi HANDA
[email-masked]

---

From: Richard Stallman <[email-masked]>
Date: Thu, 31 Jul 1997 19:42:35 -0400

---

From: Kenichi Handa <[email-masked]>
Date: Wed, 6 Aug 1997 10:10:27 +0900

diff -c -r1.30 coding.c

  
!   return eol_type;
  }

---
Ken'ichi HANDA
[email-masked]

---

From: Marc Fleischeuers <[email-masked]>
Date: 06 Aug 1997 10:32:13 +0200

[email-masked] (Geoff Voelker) writes:

> But at some point, Marc and Handa expressed dismay at the code I added
> in dos-w32.el for determining the coding system for a file.  Do you
> still think this?

Yes, I still think this. But I think this little excersice has helped
in making the issue clearer.

Richard Stallman <[email-masked]> writes:

>     I would like this behaviour to be dependent on the
>     buffer-file-coding-system in effect for a buffer, *not* the operating
>     system emacs runs on.
> 
> It already is, I believe.  The default choice for buffer-file-coding-system
> is different on MSDOS.
> 
> Does this seem to be untrue in your experience?

My gripe comes down to this: I understand a different default for
buffer-file-coding-system; the problem is that as a simple user, I
don't *see* this. M-x describe-coding-system says
buffer-file-coding-system is nil. The mode line indicator is `:'. I
only found out what was going on when I looked at
`find-buffer-file-type-coding-system' in lisp/dos-w32.el.

At this point, I added the following definitions to my ~/.emacs:

(defun untranslated-file-p (filename)
  "Return t if FILENAME is on a filesystem that does not require 
CR/LF translation, and nil otherwise."
  t)

(setq-default buffer-file-coding-system 'undecided-dos)

These definitions are a gross hack and by no means do I recommend them
to use for emacs. However, my purpose was the following: first, I
wanted to disable the selection of a coding system based on file
system, file name and file existence in
`find-buffer-file-coding-system', second, I wanted to have an explicit,
user-visible, default. The above definitions serve this purpose
well.  This is how I like emacs to be.

Marc

---

From: Marc Fleischeuers <[email-masked]>
Date: 06 Aug 1997 11:01:38 +0200

> all CRs are explicit seems reasonable.

True, this is what you would like to see. If emacs is in
"no-conversion" or "unix" eol-mode (`buffer-file-coding-system'
matches "-unix", and the mode line indicator is `:'), then I think it
is not unreasonable that if you want to make a "dos" file, to enter
`C-q C-m C-q C-j'.

Incidentally, this actually works if `untranslated-file-p' returns `t'
indiscriminantly, and the default value of `buffer-file-coding-system
is `undecided-dos' (that is, my .emacs settings since
yesterday). Given the input 

C-x C-f M-backspace M-backspace n e w . f i l e return C-x return f u
n d e c i d e d - u n i x return a b c C-q RET C-q C-j a b c C-q RET
C-q C-j C-x C-s C-x C-v return

File new.file is written containing exactly the bytes I input (as I
wanted to, i.e., abc\C-m\C-jabc\C-m\C-j) and it's read in the way it
should be: as a dos-file. Heck, I can even create a Mac-file on an
MS-DOS machine like this!

> file should normally use a single convention throughout.)

No. There is a `CODING_EOF_UNDECIDED', but in src/coding.c this case
is treated the same as `CODING_EOF_LF'.

Marc

---

From: Marc Fleischeuers <[email-masked]>
Date: 06 Aug 1997 11:11:30 +0200

[email-masked] (Geoff Voelker) writes:

> I've been gone from Friday until this morning, and so I've been
> catching up on the mail that's been sent on this thread.  From what I
> can tell, Handa's latest patch fixes Marc's problem (have you had a
> chance to try this, Marc?).  

Yes I installed this new function. Emacs is a little more predictable
and I have seen no big surprises. The (admittedly incorrectly made)
msdos-file that contains \C-m\C-m\C-j line separator chars is still
not displayed "correctly", it is shown as

abc^M^M
abc^M^M

and buffer-file-coding-system is nil. I think however that this is
ok. For one, \C-m\C-m\C-j as a line separator does not agree with any
convention so the way that a file containing these characters is
displayed is arbitrary anyway. Instead, I think it is more fruitful to
avoid situations where files with these erroneous line-endings are
created, i.e. making it clearer for users what the eol-conventions at
any given time are.

Marc

---

From: Marc Fleischeuers <[email-masked]>
Date: 06 Aug 1997 11:58:54 +0200

[email-masked] (Geoff Voelker) writes:

> > press return all I get is [no match].
> >
> > ...
> >
> Marc,
> I couldn't quite tell if you had figured this out yet or not, but
> set-variable works on variables that have been defvar'd (which these
> have not).  For these, you would want to use setq.
> 
> -geoff

Er, yes,  was mimicking an average user. I think your advice was
addressed to an experienced emacs debugger so I was not really fair. I
do think however that end-of-line stuff should be at the control of a
user, without having to resort to lisp.

Marc

---

From: Andrew Innes <[email-masked]>
Date: Wed, 6 Aug 1997 13:30:20 +0100 (BST)

>whether no-conversion is specified as the coding system.

I'm not familiar with this; does no-conversion imply no character set
conversion, or no EOL conversion (or both)?  If the latter (no charset
or EOL conversion), then that nicely expresses the distinction I have
between binary and text.

>What, specifically, is it useful for?

It is probably only useful (in practical terms) in small ways, such as
if insert-file-contents were to check whether the chosen/specified EOL
coding is used consistently for all lines in text files; obviously that
would not be appropriate for binary files.

I guess the distinction is meaningful to me as a user, though of little
consequence to the way Emacs handles files.  That is, I think of certain
types of file (eg. .gz, .tar, .zip, .obj files etc) as binary, and I
therefore expect to do different things with such files in Emacs than I
would with text files.

So, while I would think it quite natural to change the EOL coding for a
buffer visiting a text file, I would want Emacs to query me if I tried
to do the same for a buffer visiting a binary file.  If I visited a text
file that had mixed EOL coding, I would want to be told about it, and
probably would want the option to have all lines converted to the same
coding.

I might like to have a find-file-hooks function that sets truncate-lines
to t for text files, and to nil for binary files.  That sort of thing.

It is not a big deal, but to my mind makes sense.

AndrewI

---

From: Richard Stallman <[email-masked]>
Date: Wed, 6 Aug 1997 13:50:52 -0400

second, I wanted to have an explicit,
    user-visible, default.

I agree it is better to make the decision about the default coding
system for a new file when the buffer is created--not delay it until
saving the file.

Can someone write this?

---

From: Richard Stallman <[email-masked]>
Date: Wed, 6 Aug 1997 14:06:15 -0400

no-conversion is useful for that.

---

From: Richard Stallman <[email-masked]>
Date: Wed, 6 Aug 1997 14:07:17 -0400

I think it would work, now, to do this by testing
whether buffer-file-coding-system is no-conversion.