How applications interact with kernel operations

When an application (running in user space) needs to access hardware or perform a privileged operation (e.g., reading a file, opening a network connection), it must "trigger" the kernel via system calls (syscalls). Here’s how this process works:

Step-by-Step Process

• Application Requests a Service The app executes code that requires a privileged action, such as:

Opening a file (open())

Writing to the screen (write())

Allocating memory (mmap())

Creating a process (fork()).

• Invoke a System Call The app triggers a syscall using a wrapper function (e.g., from the C library like glibc). For example:

   int fd = open("file.txt", O_RDONLY); // This triggers the open syscall.        

• CPU Mode Switch (User → Kernel)

The syscall instruction (e.g., syscall on x86-64, svc on ARM) triggers a software interrupt, forcing the CPU to switch from user mode (unprivileged) to kernel mode (privileged).

The CPU saves the app’s current state (registers, stack) to resume later.

• Kernel Handles the Request

The kernel’s syscall handler identifies the requested service (e.g., open) via a syscall number (e.g., 2 for open on Linux).

The kernel validates the request (e.g., checks permissions, ensures parameters are safe).

It performs the privileged operation (e.g., accesses the disk driver to open the file).

• Return to User Space

The kernel places results (e.g., a file descriptor) in registers/memory accessible to the app.

The CPU switches back to user mode and resumes the app’s execution.

Key Mechanisms

1. Syscall Table

The kernel maintains a table mapping syscall numbers to corresponding kernel functions. For example:

• Syscall 1 → sys_exit
• Syscall 2 → sys_open
• Syscall 3 → sys_close

2. Hardware Support

• CPUs enforce privilege levels (e.g., "Ring 0" for kernel mode, "Ring 3" for user mode).
• Attempting privileged operations (e.g., direct hardware access) in user mode triggers a fault (handled by the kernel).

3. Context Switching

Switching between user and kernel mode involves:

• Saving/restoring CPU registers.
• Updating memory management units (MMUs) to switch address spaces.

Example: Writing to a File

1. App calls write(fd, buffer, size) in user space.
2. glibc invokes the write syscall (e.g., syscall number 1 on x86 Linux).
3. CPU switches to kernel mode.
4. Kernel verifies fd is valid, checks buffer permissions, and copies data from user space to kernel buffers.
5. Kernel instructs the disk driver to write the data.
6. Result (bytes written or error code) is returned to the app.

Why This Matters

• Security: Apps can’t directly access hardware or kernel memory.
• Stability: A buggy app can’t crash the entire OS.
• Abstraction: Apps don’t need to know hardware details (e.g., disk type, network card model).

Real-World Analogy

Imagine a library visitor (app) needing a restricted book (hardware access). They ask the librarian (kernel) to fetch it. The librarian checks permissions, retrieves the book, and hands it over. The visitor never enters the restricted storage area.

Tools to Observe Syscalls

• strace (Linux): Trace syscalls made by a process.strace -e trace=open,write,read ./my_program
• dtruss (macOS): Similar to strace.
• Process Monitor (Windows): Monitor system activity.

By design, syscalls are the only safe gateway for apps to interact with the kernel and hardware. This architecture underpins modern operating systems’ security and reliability.