Say we have a simple OS with a filesystem, and some programs on that filesystem. How then does a program get executed by the OS?

My understanding is that the executable file gets loaded into memory, but after that I'm a little fuzzy. I understand the OS creates a process, but I'm not sure exactly how it works. How does the OS tell the machine code to run, and how does the scheduler interact with the program to start, pause and resume execution per scheduling policy?

  • $\begingroup$ What research have you done? Have you tried reading standard OS textbooks? I expect you to do some research on your own before asking, and to tell us in the question what you tried and where you looked. $\endgroup$ – D.W. Sep 11 '14 at 23:56

Obviously, with the various forms of operating systems out there, this process can vary (and in some cases, be completely different) but this outlines a general overview.

Step 1: Load the program into memory.

This is pretty basic; grab the contents of the program and load it into memory. Depending on how memory is managed by the operating system and the platform, additional steps might be required. These could include configuring the memory page where the program is loaded to permit code execution as well as mapping page(s) to the application's initial stack.

Also, many operating systems won't load the entire program into memory immediately; they'll often allocate physical pages and load parts of the program into memory on-demand. This is generally an implementation detail that's transparent to the process, though.

Step 2: Begin executing the program.

Once the program is loaded into memory, the operating system will create whatever internal structures are required to manage the process (such as a struct task_struct in Linux) and do any other necessary bookkeeping. When that's through, the operating system will locate the program's entry point (often a main function or similar) and direct the processor to start executing the program at that location.

An operating system will usually start a new process in response to an interrupt (generally a system call of some sort) so in order to return to user mode, where the process will execute, a "return from interrupt" instruction (such as iret in x86 assembly) is typically used to restore the system state prior to the interrupt.

Step 3: Give control to another process.

This part depends heavily upon the multitasking architecture of the operating system, if it even supports multitasking. (If it doesn't, skip ahead to Step 5.)

Step 3a: (Preemptive Multitasking) Suspend the process after a short while.

In a preemptive system, a scheduler allocates a small timeslice for each running process. This timeslice is enforced by a system timer, which will fire an interrupt once a process's time is up. The interrupt will transfer control back to the kernel.

Step 3a: (Cooperative Multitasking) Wait for the process to give up control.

Cooperative systems depend on individual processes to manually give control back to the operating system on a regular basis. Of course, if a process misbehaves and doesn't return control, it would hang the operating system.

Step 3b: Save the relevant state of the process.

Once the operating system has control back from the previously executing process, it saves whatever state is necessary to return to the process at a later time. This includes the address of the next instruction the process would have executed, as well as some or all of the processor registers.

Step 4: Return control to the process.

When it's time to resume the process, the operating system will do the reverse process of the previous step; it will restore the processor registers and start executing code where the process previously left off. Like when a process is started, this is often done with a "return from interrupt" instruction. In a preemptive system, the operating system will also start the timeslice clock again.

Step 5: Handle a process's exit.

Once the process has completed its task, it will return control to the operating system for the last time by exiting. Often, the process will provide the operating system with some exit data; the operating system can do whatever it wants with this exit data. That could include notifying the process's parent of the exit, logging the completion of the process, or any number of other things. Once that's all said and done, the operating system will remove its record of the no-longer-running process and (generally) free the memory and resources the program had allocated during its execution.

Again, your mileage may vary depending on what operating system and platform you're working with. They come in all shapes and sizes, and they all work differently.

| cite | improve this answer | |
  • $\begingroup$ Thanks, that's a nice in-depth explanation. Just one thing, in step 2, you say "and direct the processor to start executing the program at that location". How exactly is this achieved (you can assume x86 or whatever you like)? I feel like I'm missing something. $\endgroup$ – Knyght Aug 30 '14 at 20:03
  • 1
    $\begingroup$ @Knyght in the general case, you can think of it like this: imagine that the entry point of your source code is marked by a label, and the operating system performs a goto to that label. In assembly, this would be done with a JMP or CALL instruction. $\endgroup$ – Adam Maras Aug 31 '14 at 5:41
  • 1
    $\begingroup$ @Knyght, also, depends what format the program is in, whether ELF (*NIX systems), PE (used by Windows), etc. The kernel will have to parse the appropriate headers and find the entry-point. $\endgroup$ – mmk Sep 1 '14 at 2:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy