Operating Systems: Week 6 - Semaphores

 This week brought us introduction of semaphores as an alternative or extension of locks to aid in controlling our programs that use multiple threads or resources.  As opposed to taking a lock and having conditional variables to signal processes waiting on the lock, semaphores are basically an integer value that is read and altered by two calls, sem_wait() and sem_post().

To start you initialize a semaphore you need to initialize it via sem_init(), typically initialized to the value of 1.

sem_wait() will decrease the integer value by one, and then wait only if the value is not greater than or equal to zero, otherwise it will return

sem_post() is called at the end of the critical section.  This will cause the semaphores value to increase by one.

As more threads are waiting on a semaphore it will cause the integer value to decrease (into negative values), and can be used to determine how many threads are waiting.

We also dove into common problems encountered when programming with currency ...bugs.  Things like deadlocks, but also issues like atomicity-violation issues .  We explored the different kinds of dead locks, like circular wait.  We also covered how to avoid those issues.

Read More

Operating Systems: Week 5 - Concurrency

This week there was a lot of spinning (get it? a threading joke).  We covered a lot of information this week about how an operating system can allow a programmer to safely and efficiently perform multiple tasks or requests.

The first part in enabling this ability is a way to allow a program to run multiple segments of code simultaneously, and this can be done with threading. Previously our programs all executed in a linear fashion.  We've been able to run multiple programs at once due to virtualization of the CPU, but within our program everything remains very linear.  Threads allow us to change that.  One can think of spawning a thread as sort of like spawning a new process, but a process that shares the same address space and so can access the same data.  

And that's for better OR for worse.  Given that multiple threads all have access to the same data, it's easy to imagine that the possibility for things to go wrong would be quite easy.  Thread A changes a variable, Thread B isn't aware of the change, still assuming the previous value, things can easily spiral out of sync and cause crashes or other indeterminate outcomes. 

To help control and manage multiple threads and give us a sense of concurrency, we learned about locks.  Locks allow programmers to have some control over how a thread access certain portion of code, this way a programmer can write their code where shared data is protected by locks and only one thread can access the region at at time.  Locks help synchronize and prevent race conditions that the operating scheduler might generate unknowingly.  We covered multiple different types of locks, spin locks, ticket locks, Futex locks.  Covered building locks with hardware primitives.  We also covered how if we want to use data structures across threads, they also need to be implemented in a thread safe manner, with locks.

Read More

Operating Systems: Week 4 - Memory Virtualization

 This week was a blur of mapping and translation.  We spent a lot of time learning the ins and outs of how memory is used by processes and how the operating system manages with those memory needs of a process in an efficient and safe manner. We learned about using a paging technique rather than segmentation to manage the usage of physical memory.  Then taking the next step we learned how specialized hardware inside the CPU (specifically inside the MMU) called the translation-lookaside buffer can help speed up memory access by not needing a page table for the most frequent memory requests.  We learned how to manually map a virtual memory address to a physical memory address.  A process that tripped me up a number of times initially until I finally sat down and walked through the translation while writing a small script to perform the translation, somehow the manual step of writing the script reinforces the knowledge and helps me absorb it.  We started to look at what happens when we've run out of physical memory yet processes are still asking for more.  Virtualized memory allows use to move pages out to slower storage (SSD, HDD, etc) but at a performance hit when we need that page again.  Strategies or algorithms around how to efficiently decide what pages to swap out, what pages to swap in, and when are another area we started to learn about, things like FIFO, LIFO, LRU, MFU, MRU.  These choices can effect how the end user's experience is perceived and thus it's important to understand the advantages and trade offs of each.

Read More

Operating Systems: Week 3 - Memory Virtualization Intro

This week was all about memory and how an operating system manages the physical memory to share (virtualize) between itself and all the applications a user might want to run.  Also how a programmer's can interact with and use memory, and how their choices can affect the operating system.  We learned about memory allocation interfaces in UNIX systems, calls like malloc to allocation memory of a certain size, free to heap memory no longer needed.  I learned about how the operation system uses address translation to help manage each individual process's memory space, ensuring each process only access their correctly memory space.  I learned about techniques to better help manage allocation and deallocation of memory in an efficient manger to help minimize things like fragmentation.  In covering segmentation, I learned how it's possible for programs to share certain memory segments, say if they both load the same support library, via the use of the hardware protection bit.  Basically a few bits are added per segment, informing whether or not a program can access said segment.

I also learned about tools like regex, grep, sed, and awk. These tools help build shell scripts, taking the output from one process and modifying it and allowing us to use it as input to another process.   

Read More

Operating Systems: Week 2 - Processes

This week's adventure in Operating Systems brought us the process.  We learned about the different parts of a process and how the operating system uses each to do things like save state, resume, kill.  This week also forced us to consider the different ways processes can get scheduled and executed on a CPU.  We explored both how different scheduling algorithms worked and their advantages and traded offs.

The homework's programming assignment this week was to reverse engineer a simplified scheduling system.  It was a fun exercise that definitely left me scratching my head more than a few times, but sticking with it was fun, experimenting and coming to understand how the scheduler was expected to work. 

Read More