See this page as a slide show

The Filesystem

Original slides from Dr. James Walden at Northern Kentucky University.

Topics

1. Overview
2. Pathnames
3. Mounting
4. Structure
5. Organization
6. File types
7. Kernel Data Structures

Overview

• Basic purpose of the Linux filesystem: To represent and organize the system’s storage resources, i.e., directories and files.
• Because the filesystem was well organized and easy to access, system programmers uses it to store other resources:
  • Processes
  • Devices
  • Mount Points
  • Kernel Data
  • Communication Channels

Filesystem Components

• Namespace: A way of naming things and organizing them in a hierarchy
• Application Programmer Interface: A set of system calls for navigating and manipulating objects
• Security model: A scheme for protecting, hiding, and sharing things
• Implementation: Software that ties the logical model to the actual hardware

Filesystem Types

• ext2, ext3, ext4: Linux
• FAT16, FAT32, NTFS: Windows
• HFS, HFS+: Apple
• XFS: Silicon Graphics
• NFSv2, NFSv3, NFSv4: Sun (distributed)
• ISO 9660, UDF: Optical Discs

https://en.wikipedia.org/wiki/Comparison_of_file_systems

Hierarchy

Files are located by traversing a directory tree:

• Root directory: Represented by single forward slash (/)
• Home directory: Represented by the tilde symbol (~)
  • Not really a filesystem thing—a shell thing.
• Working directory: Processes have a current working directory (.)
• Parent directory: All directories except root have parent (..)
• Absolute path: Pathname all the way from root (/usr/local/bin)
• Relative path: Pathname from current location (../local/bin)

Linux filesystem is a single unified hierarchy, unlike Windows.

Filenames

• File, filename, path, pathname.
  • Files are represented by strings.
  • Pathname includes directory structure.
• File and directory names use the same namespace.
• Each component can be 255 characters.
• The entire path can be 4095 characters.
• Linux find command provides recursive listing.

Characters in Filenames

• Spaces and other special characters can be used.
  • Anything but slash or null is ok.
    • If you can sneak your bizarre character past the shell.
• Periods are followed by file extension, periods aren’t special.
  • Except that ls, by default, ignores files starting with one.
    • Bug or feature?

% touch '~`!@#$%^&*()-_=+[]\{}|;'\'':",.<>?½ →⻥☂'
% ls -l
total 0
-rw------- 1 applin fac 0 Sep  5 17:58 ~`!@#$%^&*()-_=+[]\{}|;':",.<>?½ →⻥☂

Mounted Filesystems

Global filesystem contains mounted filesystems:

• Textbook uses “file tree” for global filesystem
• Mounted filesystems are attached to file tree
• Mounted filesystems are usually disk partitions
• Also can be network file servers
• Each must obey the proper API
• Attached to the tree with the mount command

Mounting

• mount /dev/hd4 /MST3K
  • Installs filesystem stored on disk under the path /MST3K
  • Files that were originally at /MST3K no longer accessible!
  • Filesystem navigation occurs seamlessly
• /etc/fstab
  • List of filesystems customarily mounted on system
  • Filesystems can be checked using fsck command
  • Shows layout of system and allows mount -a
  • Show mounted filesystems and their types with df -T

Unmounting

• umount /MST3K
  • Detaches a file system
  • Cannot unmount a file system that is busy
    • No open files
    • No processes whose current working directory is in the file system
    • No executable programs from the filesystem
• umount -l /MST3K
  • Lazy unmount, makes it unavailable
  • Waits for none of the above to be true
  • Then physically unmounts

Identify Processes

fuser -mv /MST3K (Shows all open files, processes, and executables)

Code	Meaning
f	Process has a file open for reading
F	Process has a file open for writing
c	Process has a current working directory
e	Process is currently executing a file
r	Process has set a root directory (chroot)
m	Process has mapped a file or a shared library

Filesystem Structure

• Blocks that stores metadata with structural info
  • Superblock: filesystem type, size, status
• Blocks that store user data contained in files (really, in inodes)
  • Objects in filesystem represented by inodes
  • inode has file type, permissions, owner, group, file size, number of links, access control, file creation, access, modification times
  • Directory is a list of [filename,inode number] pairs

Examples

    $ ls -l /etc/passwd
    -rw-r--r--. 1 root root 2733 May 17 12:46 /etc/passwd
    $ ls -i /etc/passwd
    1468033 /etc/passwd
    $ stat /etc/passwd
    File: `/etc/passwd’
    Size: 2733 Blocks: 8 IO Block: 4096 regular file
    Device: 802h/2050d Inode: 1468033 Links: 1
    Access: (0644/-rw-r--r--) Uid: (0/ root) Gid: (0/ root)
    Access: 2012-08-29 18:50:01.213638229 -0600
    Modify: 2012-05-17 12:46:25.000000000 -0600
    Change: 2012-05-18 18:41:06.000000000 -0600

inode

An inode contains, at least:

• file type (regular file, symlink, directory, character, block, …)
• file size in bytes
• UID / GID
• protection bits (e.g., rwxr-x---)
• ctime: inode last changed (changes to any field, incl. times)
• mtime: modification time
• atime: access time
• link count (reference count)
• pointers to the disk blocks that store the file's contents

inode block pointers

A classic inode has thirteen pointers:

• ten direct blocks
• one indirect block
• one double-indirect block
• one triple-indirect block (not shown)

Some filesystems optimize the storage of tiny files by storing the data itself in the inode.

Filesystem Organization

• Organization has always been haphazard
• Several naming schemes used, some incompatible
• That’s what you get when things evolve.

Standard directories

Directory	Meaning	Directory	Meaning
/boot	Boot directory	/usr	Most standard programs
/dev	Device files	/var	Spool directories
/etc	Critical system files	/home	Mount point for users
/sbin	System utilities	/lib	Libs and parts of the C compiler
/bin	Important utilities	/media	Removable media
/tmp	Temp files	/opt	Optional applications

Standard directories

/usr/bin	Most commands and executables
/usr/include	Header files
/usr/lib	Libraries, support files for standard programs
/usr/local	Local software
/usr/local/bin	Local executables
/usr/local/…	Other local (etc, lib, sbin, src)

Standard directories

/usr/man	Man pages
/usr/sbin	Less essential sysadmin commands
/usr/share	Common to multiple systems
/usr/share/man	Shared man pages
/usr/src	Source code for nonlocal packages

Standard directories

/var/adm	Logs, system setup records
/var/log	System log files
/var/spool	Spooling directories (mail, printers)
/var/tmp	More temp space (preserved between boots)

File types

    $ ls -ld
    crw------- 1 root root 5, 1 Aug 24 15:24 /dev/console
    brw-rw---- 1 root disk 8, 0 Aug 24 15:23 /dev/sda
    srw-rw-rw- 1 root root    0 Aug 24 15:23 /dev/log

File type encoding

File type	Symbol	Created by	Removed by
Regular file	-	cp, mv, vi	rm
Directory	d	mkdir, cp -r	rmdir, rm -r
Character device file	c	mknod	rm
Block device file	b	mknod	rm
Local domain socket	s	socket(2)	rm
Named pipe	p	mknod	rm
Symbolic link	l	ln -s	rm

Regular file

• File of bytes
• Linux imposes no structure
• Sequential and random access allowed

Directories

• Contains named references to other files
• . Refers to current directory
• .. Refers to parent directory
• File’s name stored within the parent’s directory
  • Not in file itself.
    • Really!
  • More than one directory can refer to a file (links)
  • ln creates hard links, removed with rm
    • Indistinguishable from first entry. They’re all just links.

Character and Block Device files

• Device files allow programs to communicate with hardware and peripherals
• Modules that know how to talk to the devices are linked into the kernel when built
• Device file not the same as a device driver
  • File is just a rendezvous point
• Device files characterized by major, minor numbers
  • Major number tells kernel which driver
  • Minor number tells driver which device

Some Device Files

Name	Description
/dev/sda2	disk
/dev/ttyS0	RS-232 serial line
/dev/pts/1	Pseudo-tty (pty)
/dev/tty	An alias for your terminal
/dev/null	Read fails, write succeeds
/dev/full	Read succeeds, write fails
/dev/zero	Read yields zeroes
/dev/random	Read yields random data

Local domain sockets

• Sockets are communication mechanisms
  • Allow communication between processes
  • Or more importantly, sockets control network communication
• Local domain sockets
  • Only accessible from the local host
  • Filesystem object rather than a network port
  • Visible to other processes
  • Accessible only by connected processes
  • Examples include X, NFS

Named Pipes

Communication mechanisms

• Between processes running on the same host
• Pipes are also known as FIFOs.
• Similar to pipes on the command line, but the data is persistent.

Symbolic links

• Also commonly referred to as a “soft link” or “symlink”.
• A special kind of file that contains another name.
• For example, if /alpha/beta were a symlink to gamma/delta, then it’s the same as /alpha/gamma/delta
• However, if /alpha/beta were a symlink to /epsilon/iota, then it’s the same as /epsilon/iota
• Deleting the target can leave a dangling link.
• Permissions of the symlink just don’t matter.
• Where is the target string stored?

Hard Links

• A hard link is just another directory entry that refers to the same inode.
• An inode has a reference count.
  • When the reference count goes to zero, the inode (and associated data) is deallocated.
• Do you now see why you need directory write permission, and not file write permission, to delete a file?

Hard/Soft Link Differences

• Soft links can cross mount points, hard links cannot.
• Hard link with ln, soft link with ln -s
• Soft links work on directories, hard links do not.
• Soft links require additional filesystem reads.

Kernel Data Structures

• Filesystem acts as an interface to kernel data
• Separate filesystem mounted as /proc
• Can be used to obtain info about system

    % grep MHz /proc/cpuinfo
    cpu MHz         : 1199.000
    cpu MHz         : 1199.000
    cpu MHz         : 1199.000
    cpu MHz         : 1199.000
    cpu MHz         : 1199.000
    cpu MHz         : 1199.000
    cpu MHz         : 1199.000
    cpu MHz         : 1199.000

Modifying kernel parameters

#! /bin/bash

# Set display to half-bright:

for max in /sys/class/*/*/max_brightness
do
    echo $(($(<$max)/2)) >${max%/*}/brightness
done