Multi-threading in Java is not as smart as its GC process. Without fine tuning and full understanding of the service, you may encounter into the opposite consequence resulted by dead lock, lack of resources, unknown synchronization bottlenecks, etc. From my experience, it’s very hard to stay cool and keep clean when I have to fight with multi-threading issues. So I wrote some SIMPLE conclusions below, which I hope could help you avoid repeating my mistakes.
ParallelStream
ParallelStream is elegant. It hides the redundant manual parallelization process such as creating Callable/Runnables. It uses the default ForkJoinPool which comes with problems. You don’t have the capability of overriding the ForkJoinPool. There are a few important features I realized:
- Default # of threads (parallelism) in ForkJoinPool is the # of processors -1
- Threads in ForkJoinPool are designed for non-blocking lightweight tasks, meaning it’s not suggested to do blocking operations (e.g. database transaction), or require heavy context (e.g. deep stack of recursion or large size of data)
CompletableFuture
The problem of CompletableFuture also stems from its nature of using default ForkJoinPool. But one good thing is that you can provide your own implementation of ExectuorService for supplyAsync method, in which you have better control over the threadpool.
ExecutorService
Besides the asynchronous support, another thing I like CompletableFuture is its ability to chain the futures, and combine all into one(CompletableFuture::allOf) or wait for any (CompletableFuture::anyOf).
I was trying to combine all of my futures so that I can make one single get() call to wait for the completion of all tasks. The problem is that some of tasks may end up spinning forever. In that case, I still want the results from other successful futures. However, one thing they need to improve is to support separate task timeout and timeout exception handling even with a merged future by CompletableFuture::allOf.
What I did eventually is to use ExecutorService::invokeAll with time out paramters. This is a blocking method (waiting for the given timeout value at most)and will return a list of futures. If the task times out, you will get CancellationException when calling get() on that future.
Other than the above note which is specific to my case, you also need to be careful in initializing and configuring the ExectuorService such as the task priority, task queue type, thread pool size type, etc. In my case, I was using a blocking queue, normal priority and fixed thread pool size.
Exception handling
As we mentioned before, tasks can be spinning forever, can fail with unexpected exceptions, can even go into deadlock. Properly handling exception is a general good practice, and also very important in multi-threading.
In my case, the multi-threading tasks only generate partial result and we still collect results from other resources which may succeed, in which cases we still want to deliver those result to the customer. So:
- we fail silently under CancellationException and ExecutionException.
- we timeout our/cancel tasks before they reach the service SLA.
- we log those exceptions at WARN level instead of ERROR level since they don’t have critical business impact but we monitor error count to monitor the status
Other bottlenecks
Carefully consider the data structure you have to use under multithreading context. Sometimes, people just blindly go with ConcurrentHashMap, ConcurrentLinkedQueue, CopyOnWriteArrayList etc. without second thought. But did you consider returning the result other than mutating the data directly? Locks in those data structures are expensive, which is very likely to be more expensive than copying and merging the result from parallelization. If you feel that you “have to” mutate the data, you may want to split your parallelization into smaller independent parts.
Tasks may not be terminated as you expected. For example, even if you see the timeout exception, the thread for that task is still runing behind the scene. If you are able to cache the result within that task, do it. So that you can get the data from cache and won’t need to initiate a new task to do the same thing. On the other hand, since tasks are not terminated properly, it means the CPU could be overloaded by your historical tasks.
That’s it. These are just some simple findings during my work. I hope it can help you! If indeed, please applaude this article so that I know it helped!
