Chapter 1: Computer Systems and Operating System Fundamentals

What Operating Systems Do

Introduction to the Role of Operating Systems

An operating system is a sophisticated software program that manages your computer's hardware and acts as the essential intermediary between you (the user) and the physical machine itself. Think of it like a manager at a company: it doesn't actually do the "work" itself, but it orchestrates and allocates resources so that meaningful work can happen.

Every computer—whether it's a smartphone, laptop, car, or server—contains four fundamental layers:

1. Hardware (CPU, memory, I/O devices) — the physical computing resources
2. Operating System — the manager coordinating hardware access
3. Application Programs (word processors, web browsers, etc.) — the tools you use
4. Users — the people interacting with the system

The operating system's central responsibility is to allocate hardware resources (CPU time, memory, storage, I/O access) to various programs and users in a way that is both efficient and fair.

The User's Perspective

From your perspective as a user, the operating system's design depends heavily on your computing device:

• Desktop/Laptop Users: You expect ease of use and maximum responsiveness for your work. The OS prioritizes maximizing your productivity while handling resource sharing invisibly in the background.

• Mobile Device Users: Smartphones and tablets present a touch-screen interface where you interact through finger gestures and voice commands rather than keyboards and mice. These systems must be highly responsive and power-efficient.

• Embedded System Users: Some computers have virtually no user interface at all. Devices like car engine controllers, home appliances, and industrial control systems run operating systems but operate mostly without human intervention. Users interact through simple controls like buttons or indicator lights.

The System's Perspective

From the computer's point of view, the operating system is a resource allocator and control program. When multiple programs demand access to limited hardware resources simultaneously, the operating system must decide how to distribute them fairly and efficiently. It also acts as a gatekeeper, preventing programs from interfering with one another or misusing the hardware.

Defining What an Operating System Is

Historically, computers were single-purpose machines. Today, computers exist everywhere—from aircraft navigation systems to smart refrigerators. This diversity created many different operating system designs.

There is no universally accepted definition of exactly which software components constitute "the operating system" versus separate programs. The most practical definition is that the operating system is the one program always running on the computer, known as the kernel. Around the kernel, there are also:

• System programs: Software associated with the OS but not part of the kernel (utilities, system tools)
• Application programs: All other software (your personal programs and third-party apps)
• Middleware: Frameworks that support application development with additional services like databases and graphics (especially common in mobile operating systems)

For our purposes in this course, we focus primarily on the kernel while discussing supporting components as needed to fully understand OS design.

Kernel: The one program running at all times on a computer, managing all hardware resources and providing core operating system services.

📝 Section Recap: An operating system manages hardware resources, coordinates access for multiple programs, and can be viewed from both the user's perspective (ease of use, responsiveness) and the system's perspective (resource allocation, control). The kernel—the always-running core program—is the fundamental element of any operating system.

Computer-System Organization

Overview of System Architecture

A modern general-purpose computer consists of one or more CPUs and multiple device controllers connected through a system bus—a shared communication pathway. Each device controller manages a specific type of device (disk drive, audio device, graphics display, USB hub, etc.) and maintains local buffer storage and special-purpose registers.

When a device finishes an operation, it doesn't wait passively. Instead, it signals the CPU through a mechanism called an interrupt, which alerts the CPU that attention is needed.

System Bus: The main communications pathway connecting the CPU, memory, device controllers, and other major components of the computer system.

The operating system provides a device driver for each device controller—specialized software that understands the controller's details and provides other OS components with a standard interface to access that device.

Interrupts: How the CPU Responds to Events

How Interrupts Work

Consider a typical I/O operation: a program needs to read data from a keyboard. The sequence unfolds like this:

1. The device driver loads instructions into the device controller's registers
2. The device controller interprets these registers and performs the requested action
3. Once the data transfer is complete, the controller signals the CPU via an interrupt
4. The CPU stops what it's doing and jumps to a special routine called the interrupt handler or interrupt service routine
5. The handler processes the event (collects the keyboard data), then returns execution to the interrupted program
6. The original program resumes as if nothing interrupted it

The interrupt mechanism is critical because I/O devices operate at vastly different speeds than the CPU. Without interrupts, the CPU would have to constantly "check" each device to see if it finished—an inefficient approach called polling. Interrupts allow the CPU to focus on productive work and only divert attention when something actually needs handling.

The Interrupt Vector and Interrupt Handling

To handle interrupts quickly, computers use an interrupt vector—a table of memory addresses that point to different interrupt handlers. When a device signals an interrupt, it provides an index number that the CPU uses to look up the correct handler in this table. This is far faster than having a single generic routine examine every possible source of interrupts.

For example, the CPU has a special memory location that always contains the starting address for handling interrupts. When interrupted, the CPU automatically jumps to this address and begins executing the appropriate handler.

The interrupt handler must:

1. Save the current CPU state (register values, program counter) so it can be restored later
2. Determine which device caused the interrupt
3. Service the device (move data, process the event)
4. Restore the saved CPU state
5. Execute a "return from interrupt" instruction to resume the interrupted program

Interrupt Vector: A table of memory addresses that point to the appropriate interrupt service routine for each type of device or event.

Advanced Interrupt Features

Real operating systems need more sophisticated interrupt handling than the basic mechanism. Modern systems include:

1. Interrupt Deferral: The ability to temporarily disable interrupts during critical sections of code that cannot be interrupted
2. Efficient Dispatch: A fast way to route each interrupt to the correct handler without searching through possibilities
3. Priority Levels: The ability to distinguish between high-priority and low-priority interrupts, allowing urgent work to preempt less urgent work

Modern CPUs typically have two interrupt request lines:

• Nonmaskable interrupt: Reserved for severe events like unrecoverable memory errors (cannot be turned off by the CPU)
• Maskable interrupt: Can be disabled by the CPU before executing critical code sequences; used by device controllers to request service

When there are more devices than available entries in the interrupt vector, computers use interrupt chaining, where each vector entry points to a list of handlers. The CPU calls handlers one by one until finding one that can service the interrupt. This balances the overhead of maintaining a huge table against the inefficiency of a single, overloaded handler.

Maskable Interrupt: An interrupt signal that the CPU can temporarily disable before executing critical code sections.

Nonmaskable Interrupt: An interrupt signal (typically for hardware errors) that the CPU cannot disable.

The interrupt mechanism also implements interrupt priority levels, which allow the CPU to defer low-priority interrupts while handling high-priority ones and prevent interrupts of all priorities from being blocked simultaneously.

📝 Section Recap: Interrupts are signals that alert the CPU when devices need service. The interrupt vector provides fast dispatch to the correct handler, and modern systems support priority levels, deferral, and interrupt chaining to efficiently manage the variety of devices and events that demand CPU attention.

Storage Structure and the Hierarchy of Memory

Primary Storage: Main Memory and Registers

The CPU can only execute instructions that reside in main memory (also called RAM, or random-access memory). Any program must be loaded into main memory before it can run. Main memory is typically implemented using DRAM (dynamic random-access memory), a semiconductor technology.

The CPU interacts with memory through two fundamental operations:

• Load: Copy data from a memory address into a CPU register
• Store: Copy data from a CPU register into a memory address

The CPU also automatically loads instructions from memory (using addresses stored in the program counter) for continuous execution. This is known as the fetch-decode-execute cycle.

However, main memory has critical limitations:

1. Volatile: It loses all data when power is removed
2. Limited Size: Not large enough to hold all programs and data permanently
3. Expensive: More costly per byte than secondary storage

The Problem: Volatile, Limited Memory

Because main memory is volatile and limited, computers need additional storage:

• Bootstrap Program: The first program to run when you power on a computer. It cannot reside in RAM (which is empty and volatile) but instead in EEPROM or firmware—nonvolatile memory that persists even when powered off.

• Secondary Storage: Hard-disk drives (HDDs) and nonvolatile memory devices (like SSDs) provide large, permanent storage for programs and data. Most application and system programs live here until loaded into main memory for execution.

Secondary storage is much slower than main memory but provides the permanence and capacity that volatile main memory cannot.

EEPROM: Electrically erasable programmable read-only memory; nonvolatile storage that can be rewritten but infrequently. Used for firmware and system configuration.

Secondary Storage: Long-term storage (hard drives, SSDs) that persists across power cycles and holds programs and data until they are loaded into main memory.

The Storage Hierarchy

Computer storage systems organize into a hierarchy based on speed and capacity:

The fundamental trade-off: storage that is faster tends to be smaller and more expensive, while storage that is larger tends to be slower and cheaper. This is why systems employ all these layers—to balance performance with cost and capacity.

Volatile storage (registers, cache, main memory) loses its contents when power is removed. Nonvolatile storage (secondary storage, tertiary storage) retains data indefinitely. For this reason, important data must be written to nonvolatile storage for safekeeping.

The top four levels of the hierarchy are constructed from semiconductor memory—electronic circuits built from semiconductor materials. Nonvolatile memory (NVM) devices, particularly flash memory, are faster than traditional hard disks and increasingly common in modern systems (SSDs, USB drives, smartphone storage).

Storage Hierarchy: The organization of memory systems from fastest/smallest (registers) to slowest/largest (magnetic tape), with each level serving different performance and capacity needs.

📝 Section Recap: Main memory is fast but volatile and limited; secondary storage is large and permanent but slow. Computers organize storage into a hierarchy from registers through tape, balancing speed, capacity, cost, and permanence. Data must be written to nonvolatile storage to survive power cycles.

Computer-System Architecture: Single vs. Multiple Processors

Single-Processor Systems

Historically, most general-purpose computers had a single processor—one CPU and a single main memory connected to controllers for devices like disks and printers. The operating system and user programs all competed for this one CPU's attention.

In these systems, context switching (the OS saving one program's state and loading another's) happened frequently but serially—the CPU could only execute one program's instructions at a time. However, through clever scheduling, the OS could create the illusion of parallelism, allowing multiple programs to seem to run simultaneously.

Multiprocessor Systems

Modern systems often contain multiple processors. This represents a major architectural shift with significant advantages and challenges.

Multiprocessor System: A computer with two or more processors, each capable of independent execution.

Advantages:

• Increased throughput: Multiple CPUs can truly execute instructions in parallel, doing more work in the same time
• Economy of scale: Sharing memory and peripherals is cheaper than multiple single-processor systems
• Increased reliability: If one processor fails, the system can continue operating (graceful degradation)

Two main multiprocessor designs:

1. Symmetric Multiprocessing (SMP)

In an SMP system, all processors are equal and can access shared main memory and I/O devices. Each processor has its own registers and cache, but they all see the same shared memory. An SMP system might have two or four processors that work cooperatively.

The challenge is synchronization: when multiple processors access the same memory locations, careful coordination prevents race conditions where inconsistent reads/writes corrupt data. Atomic operations and locks become essential.

2. Asymmetric Multiprocessing

In asymmetric systems, processors are not equal. One "master" processor controls the system, assigns tasks to other "slave" processors, and handles all scheduling decisions. This is simpler to program but less efficient, as the master becomes a bottleneck.

Modern systems strongly prefer SMP because of its efficiency and scalability.

Multicore Systems

Multicore processors place multiple processor cores on a single chip, each with its own registers and cache but sharing a system bus to memory. A computer might have a single multicore processor (e.g., a dual-core smartphone CPU) or multiple multicore processors (e.g., a server with four 16-core processors = 64 cores total).

Multicore Processor: A single physical processor containing multiple independent execution cores, each capable of executing instructions.

Multicore systems are very efficient because cores can share the L3 cache and avoid the cost of separate physical processors, while still achieving true parallelism.

Clustered Systems

A clustered system connects multiple complete computer systems (each with its own CPU, memory, and storage) together via high-speed networking, sharing storage and running the same operating system.

Clustering is used for:

• High availability: If one system fails, others continue operating
• High-performance computing: Parallel processing on a larger scale than a single multiprocessor
• Load balancing: Distributing work across multiple complete systems

Clustered systems are more complex and expensive than single multiprocessor systems but provide advantages when maximum reliability or performance is needed.

📝 Section Recap: Single-processor systems execute programs serially but can appear to run multiple programs through rapid context switching. Multiprocessor systems (SMP, asymmetric, multicore, or clustered) achieve true parallelism, improving throughput and reliability. SMP is the dominant design, balancing simplicity with efficiency.

Operating-System Operations and Modes

Dual-Mode Operation

Modern operating systems employ dual-mode operation to separate user programs from critical system functions, protecting the system from errors and malicious code.

The two modes are:

1. User Mode

• Where application programs and user code execute
• Restricted set of instructions available
• Cannot directly access hardware or modify system state
• Cannot execute privileged instructions (like halt or I/O operations)

2. Kernel Mode (Privileged Mode)

• Where the operating system kernel executes
• Full access to all hardware resources
• Can execute all available instructions, including privileged operations
• Can modify system state, enable/disable interrupts, modify memory protection registers

User Mode: Restricted execution mode for application programs; cannot directly access hardware or execute privileged instructions.

Kernel Mode: Privileged execution mode for the operating system; can execute all instructions and access all hardware resources.

Transitioning Between Modes

A program starts in user mode. When it needs a privileged operation (like performing disk I/O or halting), it issues a system call (also called a trap), which is a software interrupt that transfers execution to the operating system in kernel mode. The OS performs the requested operation, then uses a "return from interrupt" instruction to return to the program in user mode.

This design provides protection: user programs cannot directly damage the system because they cannot execute privileged operations. All hardware access must be mediated by the trusted kernel.

System Call: A request from a user program for the operating system to perform a privileged operation on its behalf.

Bootstrapping

When you power on a computer, the CPU starts in kernel mode and begins executing the bootstrap program stored in firmware (EEPROM). The bootstrap program:

1. Initializes all hardware components
2. Loads the operating system kernel from secondary storage into main memory
3. Transfers control to the kernel's starting address
4. The kernel initializes its data structures and device drivers
5. Creates the first user program (shell or GUI) in user mode
6. Normal operation begins

After bootstrapping, the system switches between kernel and user modes via system calls and interrupts.

📝 Section Recap: Dual-mode operation separates user programs (user mode, restricted) from the kernel (kernel mode, privileged), protecting system integrity. Programs request privileged operations via system calls, which transition to kernel mode. Bootstrapping initializes the system and loads the kernel when powered on.

Resource Management in Operating Systems

An operating system must manage multiple types of resources:

Process Management

The OS must create, schedule, and manage processes (running programs). It allocates CPU time fairly, switches between processes, handles synchronization, and manages communication between processes.

Memory Management

The OS allocates main memory to programs, manages virtual memory (extending effective memory beyond physical RAM), tracks which memory locations are in use, and protects one program's memory from access by others.

File-System Management

Programs and users interact with storage through a file system, which organizes data into files and directories. The OS manages file creation, reading, writing, deletion, and access controls.

Mass-Storage Management

The OS manages secondary storage (hard drives and SSDs), allocating space, scheduling disk accesses, and managing the movement of programs and data between main memory and storage.

Cache Management

Modern CPUs include caches—small, fast memory between the CPU and main memory that hold frequently accessed data and instructions. The OS and hardware work together to optimize what data resides in cache.

I/O System Management

The OS coordinates all input/output operations between programs, the CPU, and peripheral devices. Device drivers abstract hardware differences so that programs don't need to know device specifics.

📝 Section Recap: Operating systems manage processes, memory, file systems, storage, caches, and I/O systems. Effective resource management ensures fair allocation, protects programs from interfering with each other, and optimizes overall system performance.

Security and Protection

Users and Groups

Operating systems distinguish between different users through unique user identifiers (UIDs) and organize users into groups through group identifiers (GIDs). This enables fine-grained access control:

• The file owner can control who accesses their files
• Files are marked as owned by a specific user and group
• Permissions specify what each category (owner, group, others) can do

Privilege Escalation

Normally, programs run with the privileges of the user who launched them. However, some operations (like changing your password) require temporary elevated privileges. The setuid permission on a program allows it to execute with the privileges of the file's owner, not the user who ran it. This is carefully controlled to prevent security breaches.

User Identifier (UID): A unique number identifying a user in the operating system.

Group Identifier (GID): A unique number identifying a collection of users.

Setuid Permission: A special file permission that causes a program to execute with the privileges of its owner, not the user who launched it.

Virtualization and Emulation

Virtual Machines

A virtual machine is a software simulation of a complete computer. The hypervisor (virtualization software) runs on the host hardware and creates virtual computers that each think they own the entire system, complete with their own "CPU," "memory," and "storage."

Virtual machines enable:

• Running multiple operating systems on a single physical computer simultaneously
• Isolating applications for security (if one VM is compromised, others are unaffected)
• Efficient resource utilization (allocating more resources to VMs that need them)
• Easy migration of systems (a VM can be moved to different hardware)

Emulation

Emulation goes further, allowing software compiled for one processor architecture to run on a completely different architecture through interpretation. An emulator simulates the behavior of the original processor instruction-by-instruction. This is slower than virtualization but allows cross-platform execution.

Virtual Machine: A software simulation of a complete computer, allowing multiple operating systems to run isolated from each other on shared hardware.

Emulation: Software interpretation of one processor architecture's instructions to run on a different processor architecture.

📝 Section Recap: Virtual machines and emulation extend OS capabilities by creating abstraction layers. Virtual machines isolate operating systems and applications, while emulation enables cross-platform execution, though at reduced performance.

Computing Environments and Operating Systems

Operating systems vary dramatically across different computing environments:

Traditional Computing

General-purpose desktops and servers running operating systems like Windows, macOS, and Linux, with focus on user productivity and server performance.

Mobile Computing

Smartphones and tablets running specialized operating systems (iOS, Android) optimized for battery life, touch interfaces, and wireless connectivity. Mobile OSs often include middleware for common services.

Client-Server Computing

A network where client machines request services from more powerful server machines. The client OS handles the user interface; the server OS manages shared resources like files, printers, and databases.

Peer-to-Peer (P2P) Computing

Systems where computers are equal peers, each running both client and server functionality. Examples include file-sharing networks and distributed applications.

Cloud Computing

Virtualized computing resources delivered over networks. Users access services (infrastructure, platforms, or applications) without owning or maintaining physical hardware.

Real-Time Embedded Systems

Operating systems embedded in devices (car engines, medical equipment, industrial controllers) where predictable response times are critical. Hard real-time systems cannot miss deadlines; soft real-time systems try to meet deadlines but can tolerate occasional misses.

Data Structures in Operating Systems

Operating systems rely on standard data structures to organize information:

• Lists and Arrays: Storing sequences of items
• Stacks: Last-in, first-out data structure for managing function calls and interrupts
• Queues: First-in, first-out structure for managing waiting work
• Trees: Hierarchical structures (like file systems) and priority data structures
• Hash Functions: Rapid lookup of data by key (used in page tables, scheduling, etc.)
• Bitmaps: Compact representation of many binary states (which memory pages are free, which files are in use)

Free and Open-Source Operating Systems

Understanding operating systems is greatly enhanced by examining real, accessible code:

GNU/Linux

An open-source operating system kernel (Linux) combined with GNU utilities and tools. Widely used in servers, supercomputers, and increasingly in mobile devices (Android kernel is based on Linux). The source code is freely available and modifiable.

BSD UNIX

Berkeley Software Distribution UNIX, including FreeBSD, OpenBSD, and NetBSD. These are complete, stable operating systems with long histories and emphasis on security and reliability. BSD licenses are permissive, allowing commercial use without requiring open-source distribution of modifications.

Solaris

Originally developed by Sun Microsystems, now maintained by Oracle. A commercial UNIX variant with advanced features like dynamic kernel modules and advanced file systems.

Open-source and free operating systems have become critical infrastructure, powering the internet, scientific computing, and increasingly consumer devices. Learning how they work provides invaluable insight into operating system design.

📝 Section Recap: Computing environments range from traditional desktops to mobile devices, cloud systems, and embedded controllers, each with unique OS demands. Linux, BSD, and other free/open-source systems demonstrate OS principles with accessible source code, serving critical roles in modern infrastructure.

Key Takeaways

An operating system is the software that makes computer hardware usable by managing resources, providing interfaces to hardware, and protecting the system from errors and misuse. Modern operating systems must:

1. Provide abstractions (files, processes, memory) that hide hardware complexity
2. Allocate resources fairly and efficiently among competing programs
3. Protect programs from interfering with each other and with the system
4. Support diverse computing environments from smartphones to cloud servers
5. Handle interrupts and I/O efficiently
6. Maintain the distinction between user mode and privileged kernel mode
7. Manage storage hierarchies intelligently

Understanding these fundamentals, the hardware architecture supporting them, and how real systems implement these concepts provides the foundation for deeper exploration of operating system design.