Concurrent Programming

Introduction to concurrency

Definition

Concurrent programming is a paradigm in software development that allows multiple tasks or processes to be executed simultaneously, either through true parallel execution (as in multi-core or multiprocessor systems) or by interleaving tasks in a way that makes them appear to run at the same time.

Why Study Concurrency?

• Modeling real-world systems
• Usefulness in various computer architectures
• Necessity for certain applications

Examples of Concurrent Applications

• Web Servers: Handling multiple client requests at once.
• Simulations: Such as weather forecasting or flight simulations.
• Online Multiplayer Games: Managing actions from multiple players in real-time.
• Operating Systems: Running multiple applications or services concurrently for users.

Moore's Law

Introduction to Moore's Law

Moore's Law: The Prediction of Processing Power Growth

• Definition: Observes that the number of transistors on a microchip doubles approximately every two years, while the cost of computers is halved.
• Origin: Coined by Gordon Moore, co-founder of Intel, in 1965.
• Implication: Exponential growth in computing power and efficiency over time.
• Impact: Has been a guiding principle for the semiconductor industry, driving technological advancements.
• Innovation Acceleration: Drives advancements, leading to more powerful, compact, and energy-efficient devices.
• Economic Effects: Fuels the tech industry's economic model, emphasizing rapid development and the affordability of computing.

Logarithmic Graphic of number of transistors over time

The Limits in Processor Speed Improvements

Challenges in Continuously Increasing Processor Speed

The limit in improving the speed of computing power,
even as the number of transistors on a chip continues to grow,
began to become evident in the mid-2000s.

Reasons: Hitting the Heat and Power Wall

• Heat Generation: As transistor density increases, so does the heat generated, leading to reliability and performance issues.
• Power Consumption: Higher clock speeds significantly increase power consumption and heat, limiting the effectiveness of speed enhancements.
• Diminishing Returns: Beyond a certain point, increases in clock speed do not translate to proportional performance gains due to these thermal and power constraints.

Innovations Beyond Clock Speed

• Architectural Improvements: Advances in CPU architecture, like larger caches.
• Multi-core Processors:
  • Instead of increase clock speed
  • Add more processing cores, enabling parallel task execution
• Specialized Computing Units: For example GPU's

Parallelism and Hardware

• Multicore processors
• Multiple processors in a computer
• GPUs (Graphics Processing Units)
• Grid computing on networks

A Brief History of Concurrency in Computing

Early Days of Computing

• Computers initially ran a single program from start to end.
• Direct access to all machine resources.
• Inefficient resource utilization.

Introduction to Multiprogramming

• Shift from single to multiple program execution.
• Allows loading of several jobs into memory to be executed concurrently.
• Enhances CPU utilization by organizing jobs so CPU always has one to execute.

Uniprogramming with sequential Jobs

Multiprogramming with parallel Jobs

Operating Systems and Time-Sharing

• Time-sharing: a logical extension of multiprogramming.
• Processor's time is shared among multiple users simultaneously.
• Creates the illusion of a fast, dedicated machine for each user.

Why Concurrent Execution?

• Resource Utilization: Efficient use of idle time during I/O operations.
• Fairness: Equal resource sharing among multiple users/programs.
• Convenience: Easier to manage multiple small tasks than a monolithic program.

Process States

Scheduling

Cooperative

Tasks voluntarily yield control of the CPU, allowing other tasks to run.

Preemptive

The operating system controls the execution of tasks, forcibly interrupting and resuming them as needed to ensure fair and efficient resource allocation.

Cooperative Scheduling

Key Characteristics:

• Task Control: Tasks control their own relinquishment of the CPU.
• Yielding: A task yields the CPU either when it's idle or when it decides to allow other tasks to run.
• Advantages: Simplicity, low overhead, predictable resource utilization.
• Challenges:
  • Relies on tasks to be well-behaved.
  • A single misbehaving task can hog the CPU, affecting system responsiveness.

Ideal Use Cases:

• Environments where tasks can be trusted to yield regularly.
• Systems prioritizing simplicity over multitasking efficiency.

Preemptive Scheduling

Key Characteristics:

• Controlled by OS: The OS when a task should relinquish the CPU.
• Time Slicing: Tasks have CPU time slices and are preempted when they exceed it.
• Advantages: Improved responsiveness, fairness, better handling of real-time.
• Challenges:
  • Higher complexity in implementation.
  • Potential for resource contention and associated overhead.

Ideal Use Cases:

• General-purpose operating systems.
• Environments where tasks cannot be trusted to yield regularly.
• Real-time systems needing guaranteed response times.

Context Switch

• A Context Switch is the process of storing and restoring the state (context) of a CPU so that a process can be paused and another process can be resumed.

State saved/restored in a context switch

• Program Counter: The address of the next instruction to be executed for the process.
• CPU Registers
• Stack: Local variables, function parameters and return addresses.
• Others: Memory Management Information, Accounting Information, etc.

Concurrency vs Parallelism vs Interleaving

Concurrency:

• Concurrency refers to the ability of a system to manage multiple tasks or operations at the same time.
• It does not necessarily mean that these operations are actually executing simultaneously.
• In concurrent systems, different parts of a task or multiple tasks can be in progress at the same time, but they don't necessarily have to be running at the exact same moment.

Concurrency is about structure.

Parallelism

• Parallelism refers to the execution of multiple tasks or processes simultaneously.
• This requires hardware with multiple processing units, such as multicore processors.
• In parallel systems, tasks are literally executed at the same time, like parallel assembly lines in a factory working simultaneously.

Parallelism is about execution.

Interleaving

A technique to alternate between tasks rapidly, executing them in small slices.

Key Characteristics:

• Simulates Parallelism
• Time-sharing Approach: CPU time is divided among multiple processes in quick succession.
• Efficiency: Provides efficient CPU utilization by reducing idle time during tasks, like I/O operations.

Example: A chef preparing multiple dishes by switching between them, rather than cooking each one start to finish.

Interleaving and Overlapping

Processes vs Threads

Program

A collection of instructions and data that can be executed by a computer to perform a specific task or function.

Process

A running instance of a program. It is an independent unit that consists of its own address space, memory, data, and system resources.

Thread

A smallest unit of execution within a process.
It represents a single sequence of instructions that can be scheduled by the system's scheduler.
Multiple threads within a process share the same memory space but have their own registers, program counter, and stack.
This allows them to execute independently while still accessing the same resources and data as other threads in the same process.

Processes vs Threads

Type of threads

Kernel-Level vs User-Level

• User-Level: The work of thread management is done by the application and the kernel is not aware of the existence of threads.
• Kernel-Level: All of the work of thread management is done by the kernel. At the application level there is an API to the kernel thread facility.

Type of threads

Kernel-Level vs User-Level

Summary

• Concurrent programming history. Multiprogramming & Time-Sharing
• Scheduling. Type of scheduling.
• Processes and context switch
• Concurrency vs Parallelism & Interleaving
• Threads. Kernel vs user level threads.