Subject: Re: EOL conversion on MSDOS and MS-Windows

From: Eli Zaretskii <[email-masked]>
Date: Mon, 18 Aug 1997 18:31:36 +0300 (IDT)

On Sat, 16 Aug 1997, Geoff Voelker wrote:

> +   if (CODING_REQUIRE_EOL_CONVERSION (&coding))

This doesn't compile: there is no macro named 
CODING_REQUIRE_EOL_CONVERSION (or thereabouts).  I replaced this line 
with this:

	if (coding.eol_type == CODING_EOL_CRLF)

Geoff, is this what you meant, or did you miss something from the diffs?

After the above change, quick test indicates that this now works as 
Richard suggested.  But there is another problem: `write-region' always 
takes the coding system from `buffer-file-type' even if I write the 
region to another file.  To reproduce:

		emacs -q
		C-x C-f src/xfns.c
		C-SPC
		C-u 10 C-n
		M-x write-region RET xyzzy RET

Assuming src/xfns.c is in Unix format, this creates xyzzy also in Unix 
format.

I think this is wrong.  Since xyzzy did not exist, it should have been 
created in DOS format.

Do you agree?  Richard, how would this behave on Unix if src/xfns.c was 
in DOS format and xyzzy didn't exist?

What about the case where xyzzy already exists and Emacs is overwriting 
it?

---

From: Richard Stallman <[email-masked]>
Date: Tue, 19 Aug 1997 00:35:46 -0400

emacs -q
		    C-x C-f src/xfns.c
		    C-SPC
		    C-u 10 C-n

I am not sure.

Note that you can use C-x RET c to specify a different coding system
for this command.

      Richard, how would this behave on Unix if src/xfns.c was 
    in DOS format and xyzzy didn't exist?

As far as I know, it would do exactly the same thing as on DOS.

Emacs would not notice whether the file already exists.
Maybe it should, but I am not sure.

---

From: Kenichi Handa <[email-masked]>
Date: Tue, 19 Aug 1997 14:28:03 +0900

> I am not sure.

I think write-region should write in the format of
buffer-file-coding-system of the current buffer if no coding system is
specified explicitely by C-x RET c or in file-coding-system-alist.

> As far as I know, it would do exactly the same thing as on DOS.

In this case, "the same thing as on DOS" is "to write in the format of
buffer-file-coding-system".  So, xyzzy is written in DOS format.

> Maybe it should, but I am not sure.

I think write-region should not follow the format of already existing
file, but append-to-file had better follow the format.

---
Ken'ichi HANDA
[email-masked]

---

From: Eli Zaretskii <[email-masked]>
Date: Tue, 19 Aug 1997 15:56:43 +0300 (IDT)

>     I think this is wrong.  Since xyzzy did not exist, it should have been 

I'm not sure either, so let me explain why do I think it's wrong.

First, there's a general rule about non-existent files: they are
created with the default coding system.  This is documented behavior,
and users could think it applies to this case as well.

The other consideration that I think goes against inheriting the
coding system from the current buffer is that Emacs tailors many
aspects of its operation using the *name* of the file.  For example,
suppose Emacs were to do something special when writing a C source
file.  Would we expect that action to take place when I write a region
of an e-mail message to a .c file?  I think we would.  If you agree
with that, I think we should also expect the coding system (for
writing) be derived from the name of the file the region is being
written to.  This means that if the file's name to which the region is
written is not found in the various alists which define specific
coding systems, Emacs should fall back to the default coding system,
which is undecided-dos on DOS_NT platforms (unless Emacs is
customized).

Here's another aspect of this dilemma.  Suppose I visit a file that is
in Unix EOL format, and then use `C-x C-w' to write the entire buffer
to a file whose name *is* found in `file-name-buffer-type-alist'.
Would we expect the coding system to change to reflect the change in
the buffer's filename?  I think we would, but Emacs doesn't behave
this way now, because  Emacs consults `file-name-buffer-type-alist'
only when it reads files.

---

From: Richard Stallman <[email-masked]>
Date: Tue, 19 Aug 1997 11:25:18 -0400

Perhaps we should add a warning message to the user when a
    write-region will write a buffer in an eol format that is not in the
    default format of the file system being used.

I added text about this.  Thanks.

---

From: Richard Stallman <[email-masked]>
Date: Tue, 19 Aug 1997 12:10:08 -0400

I agree, append-to-file needs to do this.

Does it already, or is this a bug that needs fixing?

---

From: Eli Zaretskii <[email-masked]>
Date: Tue, 19 Aug 1997 20:05:05 +0300 (IDT)

> Does it already, or is this a bug that needs fixing?

No, it doesn't.  It just seeks to the end of file and writes the region 
with the coding system determined as usual.

I tried to append a portion of a Unix-style file to a DOS-style file and 
got the appended part in Unix format.

Unfortunately, I don't have time to fix this right now.  I will fix it
by tomorrow, if nobody will before that.

---

From: Richard Stallman <[email-masked]>
Date: Tue, 19 Aug 1997 14:42:28 -0400

It seems to me that this is one of those cases the concept of what is
"really right" is so complex, that it may be better to do something
simple and not try to do what is "really right".

---

From: Eli Zaretskii <[email-masked]>
Date: Wed, 20 Aug 1997 17:25:19 +0300 (IDT)

> simple and not try to do what is "really right".

I agree, but computing EOL conversion for writing from the filename
when it's not the default buffer file name doesn't seem too complex.

---

From: Richard Stallman <[email-masked]>
Date: Wed, 20 Aug 1997 12:40:57 -0400

I am not sure that is always right either.

---

From: Eli Zaretskii <[email-masked]>
Date: Wed, 20 Aug 1997 19:54:14 +0300 (IDT)

On Tue, 19 Aug 1997, Eli Zaretskii wrote:

> I agree, append-to-file needs to do this.

When you say that appending to a file should follow the format, do you 
mean only the EOL encoding, or the entire coding system?

It seems to me that if the EOLs are taken from the file to which the 
region is appended, the rest of the coding system should be also, no?

---

From: Richard Stallman <[email-masked]>
Date: Wed, 20 Aug 1997 19:35:08 -0400

I think so too.

---

From: Kenichi Handa <[email-masked]>
Date: Thu, 21 Aug 1997 09:39:02 +0900

>     region is appended, the rest of the coding system should be also, no?

I agree too.

---
Ken'ichi HANDA
[email-masked]

---

From: Eli Zaretskii <[email-masked]>
Date: Mon, 25 Aug 1997 08:30:10 +0300 (IDT)

>     region is appended, the rest of the coding system should be also, no?

To make append-to-file use the coding system of that file, I need to
decide where to put the test for this.

The relevant fragment from fileio.c is attached below for your
reference.

I think coding-system-for-write should take precedence over the file
to which we are appending, otherwise there would be no way for the
caller to force a specific coding system for this operation.  When
enable-multibyte-characters is nil, we shouldn't look at the coding
system of the file either, even if buffer-file-coding-system is local.
Do you agree?

If so, it seems to me that testing for the file's coding system before
the else clause and falling back to Ffind_operation_coding_system if
the file leaves the coding undecided, is the correct way.

----------- from fileio.c ------------------------------------------

  /* Decide the coding-system to be encoded to.  */
  {
    Lisp_Object val;

    if (auto_saving)
      val = Qnil;
    else if (!NILP (Vcoding_system_for_write))
      val = Vcoding_system_for_write;
    else if (NILP (current_buffer->enable_multibyte_characters))
      val = (NILP (Flocal_variable_p (Qbuffer_file_coding_system, Qnil))
	     ? Qnil
	     : Fsymbol_value (Qbuffer_file_coding_system));
    else
      {
	Lisp_Object args[7], coding_systems;

	args[0] = Qwrite_region, args[1] = start, args[2] = end,
	  args[3] = filename, args[4] = append, args[5] = visit,
	  args[6] = lockname;
	coding_systems = Ffind_operation_coding_system (7, args);
	val = (CONSP (coding_systems) && !NILP (XCONS (coding_systems)->cdr)
	       ? XCONS (coding_systems)->cdr
	       : current_buffer->buffer_file_coding_system);
      }
    setup_coding_system (Fcheck_coding_system (val), &coding); 
    if (!STRINGP (start) && !NILP (current_buffer->selective_display))
      coding.selective = 1;
  }

---

From: Kenichi Handa <[email-masked]>
Date: Mon, 25 Aug 1997 17:39:23 +0900

Eli Zaretskii <[email-masked]> writes:
> caller to force a specific coding system for this operation.

I agree.

> When enable-multibyte-characters is nil, we shouldn't look at the
> coding system of the file either, even if buffer-file-coding-system
> is local.  Do you agree?

I'm not sure.  Even if enable-multibyte-characters is nil, at least
EOL format is detected by insert-file-contents.  So, append-to-file
had better detect at least EOL format.  What do you think?

---
Ken'ichi HANDA
[email-masked]

---

From: Eli Zaretskii <[email-masked]>
Date: Mon, 25 Aug 1997 12:04:22 +0300 (IDT)

> had better detect at least EOL format.  What do you think?

Maybe I just don't understand well enough why that test for 
buffer-file-coding-system being a local variable is at all
required?  Can you explain it to me?

---

From: Kenichi Handa <[email-masked]>
Date: Mon, 25 Aug 1997 19:34:59 +0900

> required?  Can you explain it to me?

buffer-file-coding-system being set locally means that the file was
read with some kind of code conversion regardless of the current value
of enable-multibyte-characters.  There are two cases which cause this
situation.

1) The file was read before enable-multibyte-characters is set to nil.

2) EOL format of the file was not that of Unix files.

In both cases, I thought it was safer to encode the file with the same
coding system used for decoding.

Does this explanation help you?

---
Ken'ichi HANDA
[email-masked]

---

From: Eli Zaretskii <[email-masked]>
Date: Mon, 25 Aug 1997 13:57:54 +0300 (IDT)

> Does this explanation help you?

Yes, thanks.  I will make the patches when I build the next pretest and 
send them to you all for reviewing.

---

From: Richard Stallman <[email-masked]>
Date: Mon, 25 Aug 1997 16:57:47 -0400

That is right.

---

From: Richard Stallman <[email-masked]>
Date: Mon, 25 Aug 1997 17:12:25 -0400

This reasoning makes sense to me.