6. Basic operations

This chapter describes the commands for basic file manipulation: copying, moving (renaming), and deleting (removing).

6.1 cp: Copy files and directories		Copy files.
6.2 dd: Convert and copy a file		Convert and copy a file.
6.3 install: Copy files and set attributes		Copy files and set attributes.
6.4 mv: Move (rename) files		Move (rename) files.
6.5 rm: Remove files or directories		Remove files or directories.
6.6 shred: Remove files more securely		Remove files more securely.

6.1 cp: Copy files and directories

cp copies files (or, optionally, directories). The copy is completely independent of the original. You can either copy one file to another, or copy arbitrarily many files to a destination directory. Synopsis:

cp [option]... source dest cp [option]... source... directory

If the last argument names an existing directory, cp copies each source file into that directory (retaining the same name). Otherwise, if only two files are given, it copies the first onto the second. It is an error if the last argument is not a directory and more than two non-option arguments are given.

Generally, files are written just as they are read. For exceptions, see the `--sparse' option below.

By default, cp does not copy directories. However, the `-R', `-a', and `-r' options cause cp to copy recursively by descending into source directories and copying files to corresponding destination directories.

By default, cp follows symbolic links only when not copying recursively. This default can be overridden with the `--no-dereference' (`-d'), `--dereference' (`-L'), and `-H' options. If more than one of these options is specified, the last one silently overrides the others.

cp generally refuses to copy a file onto itself, with the following exception: if `--force --backup' is specified with source and dest identical, and referring to a regular file, cp will make a backup file, either regular or numbered, as specified in the usual ways (see section 2.1 Backup options). This is useful when you simply want to make a backup of an existing file before changing it.

The program accepts the following options. Also see 2. Common options.

`-a'

`--archive'

Preserve as much as possible of the structure and attributes of the original files in the copy (but do not attempt to preserve internal directory structure; i.e., `ls -U' may list the entries in a copied directory in a different order). Equivalent to `-dpR'.

`-b'

`--backup[=method]'

See section 2.1 Backup options. Make a backup of each file that would otherwise be overwritten or removed. As a special case, cp makes a backup of source when the force and backup options are given and source and dest are the same name for an existing, regular file. One useful application of this combination of options is this tiny Bourne shell script:

#!/bin/sh # Usage: backup FILE... # Create a GNU-style backup of each listed FILE. for i in "$@"; do cp --backup --force "$i" "$i" done

`-d'

`--no-dereference'

Copy symbolic links as symbolic links rather than copying the files that they point to, and preserve hard links between source files in the copies.

`-f'

`--force'

When copying without this option and an existing destination file cannot be opened for writing, the copy fails. However, with `--force'), when a destination file cannot be opened, cp then unlinks it and tries to open it again. Contrast this behavior with that enabled by `--link' and `--symbolic-link', whereby the destination file is never opened but rather is unlinked unconditionally. Also see the description of `--remove-destination'.

`-H'

If a command line argument specifies a symbolic link, then copy the file it points to rather than the symbolic link itself. However, copy (preserving its nature) any symbolic link that is encountered via recursive traversal.

`-i'

`--interactive'

Prompt whether to overwrite existing regular destination files.

`-l'

`--link'

Make hard links instead of copies of non-directories.

`-L'

`--dereference'

Always follow symbolic links.

`-p'

`--preserve'

Preserve the original files' owner, group, permissions, and timestamps. In the absence of this option, each destination file is created with the permissions of the corresponding source file, minus the bits set in the umask. See section 3. File permissions.

`-P'

`--parents'

Form the name of each destination file by appending to the target directory a slash and the specified name of the source file. The last argument given to cp must be the name of an existing directory. For example, the command:

cp --parents a/b/c existing_dir

copies the file `a/b/c' to `existing_dir/a/b/c', creating any missing intermediate directories.

Warning: the meaning of `-P' will change in the future to conform to POSIX. Use `--parents' for the old meaning, and `--no-dereference' for the new.

`-r'

Copy directories recursively, copying any non-directories and special files (e.g., symbolic links, FIFOs and device files) as if they were regular files. This means trying to read the data in each source file and writing it to the destination. It is usually a mistake to apply cp -r to special files like FIFOs and the ones typically found in the `/dev' directory. In most cases, cp -r will hang indefinitely trying to read from FIFOs and special files like `/dev/console', and it will fill up your destination disk if you use it to copy `/dev/zero'. Use the `--recursive' (`-R') option instead if you want to copy special files, preserving their special nature rather than reading from them to copy their contents.

`-R'

`--recursive'

Copy directories recursively, preserving non-directories (contrast with `-r' just above).

`--remove-destination'

Remove each existing destination file before attempting to open it (contrast with `-f' above).

`--sparse=when'

A sparse file contains holes---a sequence of zero bytes that does not occupy any physical disk blocks; the `read' system call reads these as zeroes. This can both save considerable disk space and increase speed, since many binary files contain lots of consecutive zero bytes. By default, cp detects holes in input source files via a crude heuristic and makes the corresponding output file sparse as well.

The when value can be one of the following:

`auto'

The default behavior: the output file is sparse if the input file is sparse.

`always'

Always make the output file sparse. This is useful when the input file resides on a filesystem that does not support sparse files (the most notable example is `efs' filesystems in SGI IRIX 5.3 and earlier), but the output file is on another type of filesystem.

`never'

Never make the output file sparse. This is useful in creating a file for use with the mkswap command, since such a file must not have any holes.

`--strip-trailing-slashes'

Remove any trailing slashes from each source argument. See section 2.4 Trailing slashes.

`-s'

`--symbolic-link'

Make symbolic links instead of copies of non-directories. All source file names must be absolute (starting with `/') unless the destination files are in the current directory. This option merely results in an error message on systems that do not support symbolic links.

`-S suffix'

`--suffix=suffix'

Append suffix to each backup file made with `-b'. See section 2.1 Backup options.

`--target-directory=directory'

Specify the destination directory. See section 2.3 Target directory.

`-v'

`--verbose'

Print the name of each file before copying it.

`-V method'

`--version-control=method'

Change the type of backups made with `-b'. The method argument can be `none' (or `off'), `numbered' (or `t'), `existing' (or `nil'), or `never' (or `simple'). See section 2.1 Backup options.

`-x'

`--one-file-system'

Skip subdirectories that are on different filesystems from the one that the copy started on. However, mount point directories are copied.

6.2 dd: Convert and copy a file

dd copies a file (from standard input to standard output, by default) with a changeable I/O block size, while optionally performing conversions on it. Synopsis:

dd [option]...

The program accepts the following options. Also see 2. Common options.

The numeric-valued options below (bytes and blocks) can be followed by a multiplier: `b'=512, `c'=1, `w'=2, `xm'=m, or any of the standard block size suffixes like `k'=1024 (see section 2.2 Block size).

Use different dd invocations to use different block sizes for skipping and I/O. For example, the following shell commands copy data in 512 kB blocks between a disk and a tape, but do not save or restore a 4 kB label at the start of the disk:

disk=/dev/rdsk/c0t1d0s2 tape=/dev/rmt/0 # Copy all but the label from disk to tape. (dd bs=4k skip=1 count=0 && dd bs=512k) <$disk >$tape # Copy from tape back to disk, but leave the disk label alone. (dd bs=4k seek=1 count=0 && dd bs=512k) <$tape >$disk

`if=file'

Read from file instead of standard input.

`of=file'

Write to file instead of standard output. Unless `conv=notrunc' is given, dd truncates file to zero bytes (or the size specified with `seek=').

`ibs=bytes'

Read bytes bytes at a time.

`obs=bytes'

Write bytes bytes at a time.

`bs=bytes'

Both read and write bytes bytes at a time. This overrides `ibs' and `obs'.

`cbs=bytes'

Convert bytes bytes at a time.

`skip=blocks'

Skip blocks `ibs'-byte blocks in the input file before copying.

`seek=blocks'

Skip blocks `obs'-byte blocks in the output file before copying.

`count=blocks'

Copy blocks `ibs'-byte blocks from the input file, instead of everything until the end of the file.

`conv=conversion[,conversion]...'

Convert the file as specified by the conversion argument(s). (No spaces around any comma(s).)

Conversions:

`ascii'

Convert EBCDIC to ASCII.

`ebcdic'

Convert ASCII to EBCDIC.

`ibm'

Convert ASCII to alternate EBCDIC.

`block'

For each line in the input, output `cbs' bytes, replacing the input newline with a space and padding with spaces as necessary.

`unblock'

Replace trailing spaces in each `cbs'-sized input block with a newline.

`lcase'

Change uppercase letters to lowercase.

`ucase'

Change lowercase letters to uppercase.

`swab'

Swap every pair of input bytes. GNU dd, unlike others, works when an odd number of bytes are read--the last byte is simply copied (since there is nothing to swap it with).

`noerror'

Continue after read errors.

`notrunc'

Do not truncate the output file.

`sync'

Pad every input block to size of `ibs' with trailing zero bytes. When use with `block' or `unblock', pad with spaces instead of zero bytes.

6.3 install: Copy files and set attributes

install copies files while setting their permission modes and, if possible, their owner and group. Synopses:

install [option]... source dest install [option]... source... directory install -d [option]... directory...

In the first of these, the source file is copied to the dest target file. In the second, each of the source files are copied to the destination directory. In the last, each directory (and any missing parent directories) is created.

install is similar to cp, but allows you to control the attributes of destination files. It is typically used in Makefiles to copy programs into their destination directories. It refuses to copy files onto themselves.

The program accepts the following options. Also see 2. Common options.

`-b'

`--backup[=method]'

See section 2.1 Backup options. Make a backup of each file that would otherwise be overwritten or removed.

`-c'

Ignored; for compatibility with old Unix versions of install.

`-d'

`--directory'

Create each given directory and any missing parent directories, setting the owner, group and mode as given on the command line or to the defaults. It also gives any parent directories it creates those attributes. (This is different from the SunOS 4.x install, which gives directories that it creates the default attributes.)

`-g group'

`--group=group'

Set the group ownership of installed files or directories to group. The default is the process' current group. group may be either a group name or a numeric group id.

`-m mode'

`--mode=mode'

Set the permissions for the installed file or directory to mode, which can be either an octal number, or a symbolic mode as in chmod, with 0 as the point of departure (see section 3. File permissions). The default mode is `u=rwx,go=rx'---read, write, and execute for the owner, and read and execute for group and other.

`-o owner'

`--owner=owner'

If install has appropriate privileges (is run as root), set the ownership of installed files or directories to owner. The default is root. owner may be either a user name or a numeric user ID.

`-p'

`--preserve-timestamps'

Set the time of last access and the time of last modification of each installed file to match those of each corresponding original file. When a file is installed without this option, its last access and last modification times are both set to the time of installation. This option is useful if you want to use the last modification times of installed files to keep track of when they were last built as opposed to when they were last installed.

`-s'

`--strip'

Strip the symbol tables from installed binary executables.

`-S suffix'

`--suffix=suffix'

Append suffix to each backup file made with `-b'. See section 2.1 Backup options.

`--target-directory=directory'

Specify the destination directory. See section 2.3 Target directory.

`-v'

`--verbose'

Print the name of each file before copying it.

`-V method'

`--version-control=method'

Change the type of backups made with `-b'. The method argument can be `none' (or `off'), `numbered' (or `t'), `existing' (or `nil'), or `never' (or `simple'). See section 2.1 Backup options.

6.4 mv: Move (rename) files

mv moves or renames files (or directories). Synopsis:

mv [option]... source dest mv [option]... source... directory

If the last argument names an existing directory, mv moves each other given file into a file with the same name in that directory. Otherwise, if only two files are given, it renames the first as the second. It is an error if the last argument is not a directory and more than two files are given.

mv can move any type of file from one filesystem to another. Prior to version 4.0 of the fileutils, mv could move only regular files between filesystems. For example, now mv can move an entire directory hierarchy including special device files from one partition to another. It first uses some of the same code that's used by cp -a to copy the requested directories and files, then (assuming the copy succeeded) it removes the originals. If the copy fails, then the part that was copied to the destination partition is removed. If you were to copy three directories from one partition to another and the copy of the first directory succeeded, but the second didn't, the first would be left on the destination partion and the second and third would be left on the original partition.

If a destination file exists but is normally unwritable, standard input is a terminal, and the `-f' or `--force' option is not given, mv prompts the user for whether to replace the file. (You might own the file, or have write permission on its directory.) If the response does not begin with `y' or `Y', the file is skipped.

Warning: If you try to move a symlink that points to a directory, and you specify the symlink with a trailing slash, then mv doesn't move the symlink but instead moves the directory referenced by the symlink. See section 2.4 Trailing slashes.

The program accepts the following options. Also see 2. Common options.

`-b'

`--backup[=method]'

See section 2.1 Backup options. Make a backup of each file that would otherwise be overwritten or removed.

`-f'

`--force'

Do not prompt the user before removing an unwritable destination file.

`-i'

`--interactive'

Prompt whether to overwrite each existing destination file, regardless of its permissions. If the response does not begin with `y' or `Y', the file is skipped.

`-u'

`--update'

Do not move a nondirectory that has an existing destination with the same or newer modification time.

`-v'

`--verbose'

Print the name of each file before moving it.

`--strip-trailing-slashes'

Remove any trailing slashes from each source argument. See section 2.4 Trailing slashes.

`-S suffix'

`--suffix=suffix'

Append suffix to each backup file made with `-b'. See section 2.1 Backup options.

`--target-directory=directory'

Specify the destination directory. See section 2.3 Target directory.

`-V method'

`--version-control=method'

Change the type of backups made with `-b'. The method argument can be `none' (or `off'), `numbered' (or `t'), `existing' (or `nil'), or `never' (or `simple'). See section 2.1 Backup options.

6.5 rm: Remove files or directories

rm removes each given file. By default, it does not remove directories. Synopsis:

rm [option]... [file]...

If a file is unwritable, standard input is a terminal, and the `-f' or `--force' option is not given, or the `-i' or `--interactive' option is given, rm prompts the user for whether to remove the file. If the response does not begin with `y' or `Y', the file is skipped.

The program accepts the following options. Also see 2. Common options.

`-d'

`--directory'

Attempt to remove directories with unlink instead of rmdir, and don't require a directory to be empty before trying to unlink it. This works only if you have appropriate privileges and if your operating system supports unlink for directories. Because unlinking a directory causes any files in the deleted directory to become unreferenced, it is wise to fsck the filesystem after doing this.

`-f'

`--force'

Ignore nonexistent files and never prompt the user. Ignore any previous `--interactive' (`-i') option.

`-i'

`--interactive'

Prompt whether to remove each file. If the response does not begin with `y' or `Y', the file is skipped. Ignore any previous `--force' (`-f') option.

`-r'

`-R'

`--recursive'

Remove the contents of directories recursively.

`-v'

`--verbose'

Print the name of each file before removing it.

One common question is how to remove files whose names begin with a `-'. GNU rm, like every program that uses the getopt function to parse its arguments, lets you use the `--' option to indicate that all following arguments are non-options. To remove a file called `-f' in the current directory, you could type either:

rm -- -f

or:

rm ./-f

The Unix rm program's use of a single `-' for this purpose predates the development of the getopt standard syntax.

6.6 shred: Remove files more securely

shred overwrites devices or files, to help prevent even very expensive hardware from recovering the data.

Ordinarily when you remove a file (see section 6.5 rm: Remove files or directories), the data is not actually destroyed. Only the index listing where the file is stored is destroyed, and the storage is made available for reuse. There are undelete utilities that will attempt to reconstruct the index and can bring the file back if the parts were not reused.

On a busy system with a nearly-full drive, space can get reused in a few seconds. But there is no way to know for sure. If you have sensitive data, you may want to be sure that recovery is not possible by actually overwriting the file with non-sensitive data.

However, even after doing that, it is possible to take the disk back to a laboratory and use a lot of sensitive (and expensive) equipment to look for the faint "echoes" of the original data underneath the overwritten data. If the data has only been overwritten once, it's not even that hard.

The best way to remove something irretrievably is to destroy the media it's on with acid, melt it down, or the like. For cheap removable media like floppy disks, this is the preferred method. However, hard drives are expensive and hard to melt, so the shred utility tries to achieve a similar effect non-destructively.

This uses many overwrite passes, with the data patterns chosen to maximize the damage they do to the old data. While this will work on floppies, the patterns are designed for best effect on hard drives. For more details, see the source code and Peter Gutmann's paper Secure Deletion of Data from Magnetic and Solid-State Memory, from the proceedings of the Sixth USENIX Security Symposium (San Jose, California, 22--25 July, 1996). The paper is also available online http://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html.

Please note that shred relies on a very important assumption: that the filesystem overwrites data in place. This is the traditional way to do things, but many modern filesystem designs do not satisfy this assumption. Exceptions include:

• Log-structured or journaled filesystems, such as those supplied with AIX and Solaris.

• Filesystems that write redundant data and carry on even if some writes fail, such as RAID-based filesystems.

• Filesystems that make snapshots, such as Network Appliance's NFS server.

• Filesystems that cache in temporary locations, such as NFS version 3 clients.

• Compressed filesystems.

If you are not sure how your filesystem operates, then you should assume that it does not overwrite data in place, which means that shred cannot reliably operate on regular files in your filesystem.

Generally speaking, it is more reliable to shred a device than a file, since this bypasses the problem of filesystem design mentioned above. However, even shredding devices is not always completely reliable. For example, most disks map out bad sectors invisibly to the application; if the bad sectors contain sensitive data, shred won't be able to destroy it.

shred makes no attempt to detect or report these problem, just as it makes no attempt to do anything about backups. However, since it is more reliable to shred devices than files, shred by default does not truncate or remove the output file. This default is more suitable for devices, which typically cannot be truncated and should not be removed.

shred [option]... file[...]

The program accepts the following options. Also see 2. Common options.

`-f'

`--force'

Override file permissions if necessary to allow overwriting.

`-NUMBER'

`-n NUMBER'

`--iterations=NUMBER'

By default, shred uses 25 passes of overwrite. This is enough for all of the useful overwrite patterns to be used at least once. You can reduce this to save time, or increase it if you have a lot of time to waste.

`-s BYTES'

`--size=BYTES'

Shred the first BYTES bytes of the file. The default is to shred the whole file. BYTES can be followed by a size specification like `k', `M', or `G' to specify a multiple. See section 2.2 Block size.

`-u'

`--remove'

After shredding a file, truncate it (if possible) and then remove it. If a file has multiple links, only the named links will be removed.

`-v'

`--verbose'

Display status updates as sterilization proceeds.

`-x'

`--exact'

Normally, shred rounds the file size up to the next multiple of the filesystem block size to fully erase the last block of the file. This option suppresses that behavior. Thus, by default if you shred a 10-byte file on a system with 512-byte blocks, the resulting file will be 512 bytes long. With this option, shred does not increase the size of the file.

`-z'

`--zero'

Normally, the last pass that shred writes is made up of random data. If this would be conspicuous on your hard drive (for example, because it looks like encrypted data), or you just think it's tidier, the `--zero' option adds an additional overwrite pass with all zero bits. This is in addition to the number of passes specified by the `--iterations' option.

`-'

Shred standard output.

This argument is considered an option. If the common `--' option has been used to indicate the end of options on the command line, then `-' will be interpreted as an ordinary file name.

The intended use of this is to shred a removed temporary file. For example

i=`tempfile -m 0600` exec 3<>"$i" rm -- "$i" echo "Hello, world" >&3 shred - >&3 exec 3>-

Note that the shell command `shred - >file' does not shred the contents of file, since it truncates file before invoking shred. Use the command `shred file' or (if using a Bourne-compatible shell) the command `shred - 1<>file' instead.

You might use the following command to erase all trace of the file system you'd created on the floppy disk in your first drive. That command takes about 20 minutes to erase a 1.44MB floppy.

shred --verbose /dev/fd0

Similarly, to erase all data on a selected partition of your hard disk, you could give a command like this:

shred --verbose /dev/sda5