This section discusses some of the commands that are useful for analyzing and monitoring the use of space on a Unix or linux system.

File types

As you know, the last part of the name of a file on some operating systems is used to indicate what the file contains. This filename extension appears as a period followed by a few characters. If a file is missing its extension, the system does not know what to do with it.

Although some programs on linux may use extensions, particularly GUI applications, these endings on filenames are usually ignored. Thus, most linux users do not bother with them. Suppose you come across a file named xxx. You want to examine it, but cautiously avoid using cat. This is because the file may not contain text. If you cat a non-text file to your terminal, it will make a mess, possibly setting your terminal to some unusable state. An obvious question is: what does xxx contain?

Unix and linux systems rely on a program to examine data and determine what it is. The name of this program is file. Thus, to examine xxx, you could use the command file xxx

$ file xxx
xxx: JPEG image data, JFIF standard 1.01

 file uses a set of heuristics to analyze the data given to it. The heuristics are encoded in a set of rules, the magic file, which is usually in /usr/share. Here is a simple rule from magic

This rule says that if the file contains the value 0x000001b3 at file offset 0 (the beginning of the file), the description to output is MPEG video stream data. The second field is a type. This type is belong, which means a four-byte integer in big-endian byte order. Let's look at another rule

This rule says that if the file contains the string  #! /bin/bash  at the beginning of the file (offset 0), the file is a Bourne-Again shell script text executable. The space between the exclamation point and the slash is optional.

Here is how the file program examines the data given to it:

• it checks to see if the data is a non-regular file. If so, it reports what it is: e.g., directory. Otherwise
• it runs the rules in the magic file one at a time. When one succeeds, the attached description is output and the file has been identified. If none succeed
• it 'guesses'. file simply examines the bytes in the file and indicates whether the file is text or not. If it's not text, file outputs data as the type. If it is text, file guesses the kind of text it is. Sample 'text guesses' are English text, and commands text.

The file program's success is only as good as the rules in the magic file. This is obvious if we use the file command on another Unix system to examine our file xxx, which was correctly identified on linux as a JPEG

If you ever come across a piece of data on Unix or linux and want to know what it is, file is the first program to use. Combining it with find can give you useful information, such as a list of the bash shell scripts beneath a directory

find . -type f -exec file {} \; | grep 'Bourne-Again' | cut -d: -f1

file is very useful in examining data reattached to lost+found

Disk space usage

As we now know, disk space is not allocated in bytes. Instead, it is allocated in blocks of some number of kB. Thus, calculating the size of a file or directory is better done by a program that counts blocks rather than bytes. That program is du

du stands for disk usage. It has two important options

• -s give a summary size. If du is given a directory, using -s will output the total usage for everything (recursively) in the directory. Without -s it will give usage of each item in the directory individually.
• -k use 1kB blocks. du uses a 'block size' which is system-dependent: either 512-byte or 1kB. To avoid confusion, you should always use -k.

If you want to find out what is taking up all the space in your home directory you might cd there and use 

du -sk *

Filesystem space statistics

We have used the display filesystems command df before. It is used to show the space allocated, used and free on the indicated filesystems. It also has two important options

• -k use 1kB blocks. Always use -k
• -i output statistics for inodes rather than for data blocks.