Concurrency In Practice - PART II [Structuring Concurrent Applications]

Concurrency In Practice  - PART II  [Structuring Concurrent Applications]

Chapter 6 - Task Execution

Most concurrent applications are organized around the execution of tasks: abstract, discrete units of work.

Executing tasks in threads
Ideally, tasks are independent activities, which can be executed in parallel if there are adequate processing resources.

The Executor framework
Tasks are logical units of work, and threads are a mechanism by which tasks can run asynchronously.
It's a flexible thread pool implementation.
Executor may be a simple interface, but it forms the basis for a flexible and powerful framework for asynchronous task execution that supports a wide variety of task execution policies.
It provides a standard means of decoupling task submission from task execution, describing tasks with Runnable .
The Executor implementations also provide lifecycle support and hooks for adding statistics gathering, application management, and monitoring.

Executor is based on the producer-consumer pattern, where activities that submit tasks are the producers (producing units of work to be done) and the threads that execute tasks are the consumers (consuming those units of work).
Using an Executor is usually the easiest path to implementing a producer-consumer design in your application.

Whenever you see code of the form: new Thread(runnable).start()
and you think you might at some point want a more flexible execution policy, seriously consider replacing it with the use of an Executor as below.
private static final Executor exec = Executors.newFixedThreadPool(NTHREADS);
exec.execute(task);
Submitting a task with execute adds the task to the work queue, and the worker threads repeatedly dequeue tasks from the work queue and execute them.

A thread pool, as its name suggests, manages a homogeneous pool of worker threads.
A thread pool is tightly bound to a work queue holding tasks waiting to be executed.
Worker threads have a simple life: request the next task from the work queue, execute it, and go back to waiting for another task.
You can create a thread pool by calling one of the static factory methods in Executors :

newFixedThreadPool . A fixed-size thread pool creates threads as tasks are submitted, up to the maximum pool size, and then attempts to keep the pool size constant (adding new threads if a thread dies due to an unexpected Exception ).
newCachedThreadPool . A cached thread pool has more flexibility to reap idle threads when the current size of the pool exceeds the demand for processing, and to add new threads when demand increases, but places no bounds on the size of the pool.
newSingleThreadExecutor . A single-threaded executor creates a single worker thread to process tasks, replacing it if it dies unexpectedly. Tasks are guaranteed to be processed sequentially according to the order imposed by the task queue (FIFO, LIFO, priority order).
newScheduledThreadPool . A fixed-size thread pool that supports delayed and periodic task execution, similar to Timer .

Executor lifecycle
the ExecutorService interface extends Executor , adding a number of methods for lifecycle management.

public interface ExecutorService extends Executor {
void shutdown();
List shutdownNow();
boolean isShutdown();
boolean isTerminated();
boolean awaitTermination(long timeout, TimeUnit unit)
throws InterruptedException;
// ... additional convenience methods for task submission
}

The lifecycle implied by ExecutorService has three states—running, shutting down, and terminated. ExecutorService s are initially created in the running state.
The shutdown method initiates a graceful shutdown: no new tasks are accepted but previously submitted tasks are allowed to complete—including those that have not yet begun execution.
The shutdownNow method initiates an abrupt shutdown: it attempts to cancel outstanding tasks and does not start any tasks that are queued but not begun.
Once all tasks have completed, the ExecutorService transitions to the terminated state.

Finding exploitable parallelism
Result-bearing tasks: Callable and Future
The Executor framework uses Runnable as its basic task representation. Runnable is a fairly limiting abstraction; run cannot return a value or throw checked exceptions.

Runnable and Callable describe abstract computational tasks. Tasks are usually finite: they have a clear starting point and they eventually terminate. The lifecycle of a task executed by an Executor has four phases: created, submitted, started, and completed.
Since tasks can take a long time to run, we also want to be able to cancel a task.
In the Executor framework, tasks that have been submitted but not yet started can always be cancelled, and tasks that have started can sometimes be cancelled if they are responsive to interruption. Cancelling a task that has already completed has no effect.
Future represents the lifecycle of a task and provides methods to test whether the task has completed or been cancelled, retrieve its result, and cancel the task.
The behavior of get varies depending on the task state (not yet started, running, completed)
It returns immediately or throws an Exception if the task has already completed, but if not it blocks until the task completes.
The submit methods in ExecutorService all return a Future , so that you can submit a Runnable or a Callable to an executor and get back a Future that can be used to retrieve the result or cancel the task.

The real performance payoff of dividing a program’s workload into tasks comes when there are a large number of independent, homogeneous tasks that can be processed concurrently.

ExecutorCompletionService - uses Executor +BlockingQueue
Example :- https://dzone.com/articles/executorcompletionservice
https://github.com/prashanthmamidi/PraticeJava/blob/master/src/main/java/com/concurrent/api/ExecutorCompletionServiceDemo.java

Chapter 8 - Applying Thread Pools

ThreadLocal should not be used in pool threads to communicate values between tasks.
Thread starvation deadlock 
If tasks that depend on other tasks execute in a thread pool, they can deadlock.
In larger thread pools if all threads are executing tasks that are blocked waiting for other tasks still on the work queue. This is called thread starvation deadlock.

Sizing thread pools
The ideal size for a thread pool depends on the types of tasks that will be submitted and the characteristics of the deployment system.
Thread pool sizes should rarely be hard-coded; instead pool sizes should be provided by a configuration mechanism or computed dynamically by consulting Runtime.availableProcessors .

Configuring ThreadPoolExecutor
ThreadPoolExecutor is a flexible, robust pool implementation that allows a variety of customizations.
General constructor for ThreadPoolExecutor:

public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler) { ... }

The core pool size, maximum pool size, and keep-alive time govern thread creation and teardown.
The core size is the target size
The maximumpool size is the upper bound on how many pool threads can be active at once.
A thread that has been idle for longer than the keep-alive time becomes a candidate for reaping and can be terminated if the current pool size exceeds the core size.
ThreadPoolExecutor allows you to supply a BlockingQueue to hold tasks awaiting execution.

Saturation policies
When a bounded work queue fills up, the saturation policy comes into play. The saturation policy for a ThreadPoolExecutor can be modified by calling setRejectedExecutionHandler .
Several implementations of RejectedExecutionHandler are provided, each implementing a different saturation policy:
1. AbortPolicy - default policy, causes execute to throw the unchecked RejectedExecutionException ; the caller can catch this exception and implement its own overflow handling as it sees fit.
2. DiscardPolicy - silently discards the newly submitted task if it cannot be queued for execution
3.. DiscardOldestPolicy - discards the task that would otherwise be executed next and tries to resubmit the new task.
4. CallerRunsPolicy - implements a form of throttling that neither discards tasks nor throws an exception, but instead tries to slow down the flow of new tasks by pushing some of the work back to the caller.It executes the newly submitted task not in a pool thread, but in the thread that calls execute
Ex:-  Creating a fixed-sized thread pool with a bounded queue and the caller-runs saturation policy

ThreadPoolExecutor executor= new ThreadPoolExecutor(N_THREADS, N_THREADS,
0L, TimeUnit.MILLISECONDS,new LinkedBlockingQueue(CAPACITY));
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());

Thread factories
Whenever a thread pool needs to create a thread, it does so through a thread factory.
The default thread factory creates a new, nondaemon thread with no special configuration. Specifying a thread factory allows you to customize the configuration of pool threads. ThreadFactory has a single method, newThread , that is called whenever a thread pool needs to create a new thread.

Extending ThreadPoolExecutor
ThreadPoolExecutor was designed for extension, providing several “hooks” for subclasses to override— beforeExecute , afterExecute , and terminated —that can be used to extend the behavior of ThreadPoolExecutor .