Preface: Most of the answers below are written by myself --- only instructors are given access to the exercise solutions. If you find anything wrong, please let me know so that I can correct them, thank you!

1. When a user process is interrupted or causes a processor exception, the x86 hardware switches the stack pointer to a kernel stack, before saving the current process state. Explain why.

This is because we should saving the current task state on (separate)kernel-level stack instead of user-level stack, for two reasons:

• Reliability. The process’s user-level stack pointer might not be a valid memory address (e.g., if the program has a bug, or the current stack is allocated on the user program‘s heap), but the kernel handler must continue to work properly and keep the integrity of user‘s data.
• Security. On a multiprocessor, other running threads may modify user memory during the system call. If the kernel handler stores the user task state on the user-level stack, the user program might be able to modify the kernel’s return address, potentially causing the kernel to jump to arbitrary code.

2. For the “Hello world” program, we mentioned that the kernel must copy the string from the user program to screen memory. Why must the screen’s buffer memory be protected? Explain what might happen if a malicious application could alter any pixel on the screen, not just those within its own window.

Let‘s say, a server is infected by cryptomining malware/virus. If the screen’s buffer memory can be modified arbitrarily, then the malicious process can modify the CPU occupancy rate displayed by the task manager, making it difficult for the system administrator to detect which process is occupying a large amount of computational resources.

3. For each of the three mechanisms that supports dual-mode operation — privileged instructions, memory protection, and timer interrupts — explain what might go wrong without that mechanism, assuming the system still had the other two.

• Without privileged instructions: Malware or bug application now can directly change their privilege level, thus executing any codes or changing the set of memory locations it can access or disabling processor interrupts, which makes process isolation, memory protection and interrupts all impossible.
• Without memory protection: Malware or bug application now can modify the operating system kernel’s code or data in memory to gain control over the system.
• Without timer interrupts: The operating system may never regain control of the processor and failed to witch to a new task.

4. Suppose you are tasked with designing the security system for a new web browser that supports rendering web pages with embedded web page scripts. What checks would you need to implement to ensure that executing buggy or malicious scripts could not corrupt or crash the browser?

Three checks will be implemented:

• Whether the instructions executed by the current page script conforms to its privilege level.
• Whether the memory that the current page script attempts to access is legal corresponding to its privilege level.
• Whether the resources (CPU time, memory space) occupied by the current page script meet its priority.

5. Define three types of user-mode to kernel-mode transfers.

• Interrupt: An asynchronous signal to the processor that some external event has occurred that may require its attention. As the processor executes instructions, it checks for whether an interrupt has arrived. If so, it completes or stalls any instructions that are in progress. Instead of fetching the next instruction, the processor hardware saves the current execution state and starts executing at a specially designated interrupt handler in the kernel.
• Processor exceptions: A hardware event caused by user program behavior that causes a transfer of control to a kernel handler. For example, a processor exception occurs whenever a process attempts to perform a privileged instruction or accesses memory outside of its own memory region.
• System calls: A procedure provided by the kernel(service) that can be called from user level.

6. Define four types of kernel-mode to user-mode transfers.

• New process. To start a new process, the kernel copies the program into memory, sets the program counter to the first instruction of the process, sets the stack pointer to the base of the user stack, and switches to user mode.
• Resume after an interrupt, processor exception, or system call. When the kernel finishes handling the request, it resumes execution of the interrupted process by restoring its program counter (in the case of a system call, the instruction after the trap), restoring its registers, and changing the mode back to user level.
• Switch to a different process. In some cases, such as on a timer interrupt, the kernel switches to a different process than the one that had been running before the interrupt. Since the kernel will eventually resume the old process, the kernel needs to save the process state — its program counter, registers, and so forth — in the process’s control block. The kernel can then resume a different process by loading its state — its program counter, registers, and so forth — from the process’s control block into the processor and then switching to user mode.
• User-level upcall. Many operating systems provide user programs with the ability to receive asynchronous notification of events. The mechanism is similar to kernel interrupt handling, except at user level.

7. Most hardware architectures provide an instruction to return from an interrupt, such as iret. This instruction switches the mode of operation from kernel-mode to user-mode.

a. Explain where in the operating system this instruction would be used.
b. Explain what happens if an application program executes this instruction.

a. As mentioned in question 6, in those four cases the iret command will be used.

b. The iret instruction will use the cs:eip, eflags and ss:esp from kernel‘s interrupt stack to restore the code segment, program counter, execution flags, stack segment, and stack pointer, thus switching the mode of operation from kernel-mode to user-mode.

8. A hardware designer argues that there is now enough on-chip transistors to provide 1024 integer registers and 512 floating point registers. As a result, the compiler should almost never need to store anything on the stack. As an operating system guru, give your opinion of this design.

a. What is the effect on the operating system of having a large number of registers?

b. What hardware features would you recommend adding to the design?

c. What happens if the hardware designer also wants to add a 16-stage pipeline into the CPU, with precise exceptions. How would that affect the user-kernel switching overhead?

a. The operating system will have the ability to execute user- and kernel-mode switches very efficiently, since it almost never need to save current task‘s state on stack with so many available registers.

b. Just like the SPARC architecture, we need to defined a set of register windows that operate like a hardware stack. Each register window includes a full set of the registers defined by the processor instruction set. When the processor performs a procedure call, it shifts to a new windows, so the compiler never needs to save and restore registers across procedure calls or mode switch, making them quite fast.

c. With the pipeline design, there is a chance when an interrupt arrives, the instructions will be in different stages on pipeline. So we have to let the hardware first completes all instructions that occur, in program
order, before the interrupted instruction. Then wait for the hardware to annul any instruction that occurs, in program order, after the interrupt or trap, even if the instruction is in progress when the processor detects the interrupt, which will increase the user-kernel switching overhead.

9. With virtual machines, the host kernel runs in privileged mode to create a virtual machine that runs in user mode. The virtual machine provides the illusion that the guest kernel runs on its own machine in privileged mode, even though it is actually running in user mode.

Early versions of the x86 architecture (pre-2006) were not completely virtualizable — these systems could not guarantee to run unmodified guest operating systems properly. One problem was the popf “pop flags” instruction that restores the processor status word. When popf was run in privileged mode, it changed both the ALU flags (e.g., the condition codes) and the systems flags (e.g., the interrupt mask). When popf was run in unprivileged mode, it changed just the ALU flags.

a. Why do instructions like popf prevent transparent virtualization of the (old) x86 architecture?

b. How would you change the (old) x86 hardware to fix this problem?

a. Let‘s say, a popf instruction is about to be executed in guest kernel mode, but from the respect of hardware, there is only one privileged mode - the host kernel mode, so the popf will only change the ALU flags, which is not what guest kernel expects to happen.

b. We should change the hardware so that the host operating system can trace whether a instruction like popf is executed in guest kernel mode or guest user mode(e.g., a flag bit), thus making transparent virtualization possible.

10. Which of the following components is responsible for loading the initial value in the program counter for an application program before it starts running: the compiler, the linker, the kernel, or the boot ROM?

Actually the definition of "application starts running" is quite vague. Is it means when __libc_start_main starts the main procedure in the program(the first line of code written by programmer), or when the kernel starts the process for the program?

Given that in most contexts, we are talking about the latter case, then it is the kernel(loader) who loads the initial value in the program counter. To start a new process, it copies the program into memory, sets the program counter to the first instruction of the process, sets the stack pointer to the base of the user stack, and finally switches to user mode.

11. We described how the operating system kernel mediates access to I/O devices for safety. Some newer I/O devices are virtualizable — they permit safe access from user-level programs, such as a guest operating system running in a virtual machine. Explain how you might design the hardware and software to get this to work. (Hint: The device needs much of the same hardware support as the operating system kernel.)

Hardware design for I/O virtualization: For I/O virtualization, underlying processor architecture can be designed such that a user level program like a guest operating system running in a virtual machine has the same hardware support as underlying host operating system and is able to directly handle its own system calls, interrupts and exceptions without delegating them to host operating system. Various I/O devices can also be designed in a way that they are able to have direct transfers with guest operating system without routing them through the host operating system.

Software design for I/O virtualization: On software front, new device drivers will need to be defined that facilitate direct transfers with guest operating system. Moreover, sandboxing feature can be incorporated in order to allow guest operating system to safely execute user-level/third-party applications.

12. System calls vs. procedure calls: How much more expensive is a system call than a procedure call? Write a simple test program to compare the cost of a simple procedure call to a simple system call (getpid() is a good candidate on UNIX; see the man page). To prevent the optimizing compiler from “optimizing out” your procedure calls, do not compile with optimization on. You should use a system call such as the UNIX gettimeofday() for time measurements. Design your code so the measurement overhead is negligible. Also, be aware that timer values in some systems have limited resolution (e.g., millisecond resolution).

Explain the difference (if any) between the time required by your simple procedure call and simple system call by discussing what work each call must do.

test.c:

#include <stdio.h>
#include <time.h>
#include <unistd.h>

#define CALLTIMES 10000

void simple_procedure(void)
{
    return;
}

int main(int argc, char const *argv[])
{
    clock_t start, end;
    double cpu_time_used_function, cpu_time_used_syscall;

    start = clock();
    {
        for (int i = 0; i < CALLTIMES; ++i)
        {
            simple_procedure();
        }
    }
    end = clock();
    cpu_time_used_function = ((double) (end - start)) / CLOCKS_PER_SEC;

    start = clock();
    {
        for (int i = 0; i < CALLTIMES; ++i)
        {
            getpid();
        }
    }
    end = clock();
    cpu_time_used_syscall = ((double) (end - start)) / CLOCKS_PER_SEC;

    printf("simple user function call for 10000 times: %f\n", cpu_time_used_function);
    printf("simple kernel system call for 10000 times: %f\n", cpu_time_used_syscall);

    return 0;
}

Output:

It seems like the overhead of this system call is nearly 100 times more than a user function call. That‘s because operating system have to switch between user mode and kernel mode when a system call happens, which is accompanied by necessary storage (such as saving user task status) and computing operations (such as determining whether the user task‘s status is legal at this time). But for the user mode call, it only requires parameter passing and call operations, which so takes much less time.

13. Suppose you have to implement an operating system on hardware that supports interrupts and exceptions but does not have a trap instruction. Can you devise a satisfactory substitute for traps using interrupts and/or exceptions? If so, explain how. If not, explain why.

Yes we can, if we treat trap as a kind of exceptions(actually I think this is what intel x86 implement it - the classes of exceptions are trap, fault and abort).

If the underlying hardware does not provide a trap instruction, an OS can use exceptions hardware support as a hack. For example, an OS can have the convention that executing an invalid instruction with a valid parameter(which was determined by OS) in specific place(e.g., register %rax) will make kernel act as executing a trap(or system call) instruction. This is because an invalid instruction will cause an exception that immediately transfers control to the kernel, and then the kernel can check the parameter user have passed and decide what to do next.

14. Suppose you have to implement an operating system on hardware that supports exceptions and traps but does not have interrupts. Can you devise a satisfactory substitute for interrupts using exceptions and/or traps? If so, explain how. If not, explain why.

This answer is written by [email protected]

In this situation, it would not be possible to use exceptions or traps to substitute for interrupts.Exceptions and traps are synchronous, and interrupts are inherently asynchronous; exceptions and traps happen as a result of process doing something, whereas interrupts are external events that happen at unpredictable times. Going back to the timer interrupt, for example, we would not be able to interrupt a process that has complete control over the CPU(e.g., an infinite loop) using exceptions or traps.

15. Explain the steps that an operating system goes through when the CPU receives an interrupt.

First, the OS switches to kernel mode, disable the interrupt, and jump to the handler routine at a pre-specified address according to the interrupt vector and interrupt code. The routine starts by saving all registers, possibly setting the allowed interrupts to a level that guarantees correct execution without being interrupted again, and then executes the code to serve the interrupt. When the handler is done, it restores the registers and returns to the kernel code called the handler before, which then restore the user task status and switches to user mode, executing the instruction following the one at which the interrupt occurred. If there is no runnable task, the operating system jumps to the idle loop and waits for another interrupt.

16. When an operating system receives a system call from a program, a switch to operating system code occurs with the help of the hardware. The hardware sets the mode of operation to kernel mode, calls the operating system trap handler at a location specified by the operating system, and lets the operating system return to user mode after it finishes its trap handling.

Consider the stack on which the operating system must run when it receives the system call. Should this stack be different from the one the application uses, or could it use the same stack as the application program? Assume that the application program is blocked while the system call runs.

The answer to this question is the same as first question‘s above.

17. Write a program to verify that the operating system on your computer correctly protects itself from rogue system calls. For a single system call — such as file system open — try all possible illegal calls: e.g., an invalid system call number, an invalid stack pointer, an invalid pointer stored on the stack, etc. What happens?

Actually we can just write a simple program and debug it with GDB, in which we can change the registers/syscall_number or memory/stack_pointer before the syscall instruction, then observer what will happen next.

test.c:

#include <stdio.h>
#include <fcntl.h>
#include <errno.h>

int main(int argc, char const *argv[])
{
    errno = 0;
    if( open("/home/liqiuhao/tmp/main.go", O_RDWR) == -1)
    {
        perror("Error");
    }
    return 0;
}

Compile and debug:

Set a breakpoint just before syscall in user sub __libc_open :

Registers:

By looking up the Linux Syscall Table for x86_64, we know that the compiler use openat instead of open :

First we can change the rax to an invalid system call number to see what will happen:

We find that the kernel correctly protects itself from this rogue system call and correctly set the error number:

Now we use an invalid stack pointer to see what will happen:

It seems like kernel correctly protects itself again, and it doesn‘t really care about our user ss:rsp registers --- just store and restore them. But the user program will have a highly chance to get a Segment fault later.

The invalid pointer experiment is the same as invalid stack pointer --- the kernel just keep the user stack during system call without any crash in itself.