Parallel Looping Constructs in Java: Lambda expressions to the rescue?

Last November, Mark Reinhold announced that Closures will be added to the Java language and six months later we had an initial prototype based on the Straw-Man proposal and last month’s update was considered a significant progress for the language in years. As mentioned in Mark’s post, “Working with parallel arrays in Java, unfortunately, requires lots of boilerplate code to solve even simple problems. Closures can eliminate that boilerplate.” In my earlier post, I discussed how Lambda expressions make Aggregate operations a breeze to work with in Java. But, we still lack parallel looping constructs such as parallel for, foreach APIs that would take advantage of the multi-core machines we have today.

In this next Lambda adventure, I will be revisiting the Parallel ForEach example discussed by Howard in his “New Control Structures for Java” post, where he explains the use of anonymous inner classes to map a function that executes in parallel and retains the order of the original map, handles exceptions and implements continue operation. Here is the anonymous inner class version of the code:

https://gist.github.com/aruld/4949472.js

The program produces the below output:

{1=H, 2=e, 3=l, 4=l, 5=o}


Now, that we have a working Lambda prototype, this code could be modified to take advantage of Lambda expressions. Let us enable the Lambda finder option in the compiler that identifies potential candidates for converting an anonymous inner class instance to an Lambda expression. Here is the updated javac.bat used to run with the latest prototype compiler.

@echo off
C:jdk1.7.0binjava.exe -cp classes.jar com.sun.tools.javac.Main -XDidentifyLambdaCandidate=true %1


Compiling Parallel.java with this hidden compiler option identifies 2 candidates for Lambda expressions.

C:JDK7lambdasamples>javac.bat Parallel.java
Parallel.java:16: Note: This anonymous inner class creation can be turned into a lambda expression.
            return new Callable<O>() {
                                     ^
Parallel.java:70: Note: This anonymous inner class creation can be turned into a lambda expression.
        final ForEach<Character, Integer, Character> trim = new ForEach<Character, Integer, Character>() {
C:JDK7lambdasamples>

Let us quickly refactor Howard’s code using Lambda expressions and here is my take.

https://gist.github.com/aruld/4949474.js

This version does not significantly change from its anonymous counterpart, it reduces at most 10 lines of code. But, it introduces two important concepts in addition to use of Lambda expressions. Lines 6 and 13 uses the new exception transparency mechanism and Line 62 shows how lexical scoping makes Lambda expressions different from anonymous inner classes.

Exception Transparency

Exception transparency is a new feature proposed as part of the umbrella Lambda proposal. While this feature is not required to support Lambda expressions, it is useful in library implementations that require abstraction over generic exceptions. Java Generics does a decent job at abstracting over method return types and argument types, leaving abstraction over types of checked exceptions in limbo. This makes it difficult for libraries to support generic exception types in APIs. The two extreme cases where libraries usually model their exception types are :

In case of Runnable, the run() method throws nothing.

public interface Runnable {
    public abstract void run();
}

In case of Callable, the call() method throws everything.

public interface Callable {
    V call() throws Exception;
}

As more and more libraries would take advantage of executing Lambda expressions as a block of code, the inability to generify thrown exception types makes this proposal a compelling use case.

A throws type parameter is a generic type parameter that is introduced with the keyword throws; throws type parameters implicitly have an upper bound of Exception (though any upper bound that is a subtype of Throwable may be explicitly specified) and can correspond to zero or more actual types that are subtypes of the upper bound.

With this support in the language, we should be able to abstract generic exception types:

interface Block {
    public void execute() throws E;
}

Lexical scoping

Unlike anonymous inner classes, lambda expressions are lexically scoped. This means that code inside the Lambda body sees only what is available immediately outside the lambda expression. For instance, access to names from inherited classes/interfaces is not allowed as seen in Line #62 of our example above.

I wanted to briefly touch upon these concepts that are relevant to our discussion. Please refer to the proposal for more details.

.NET world

Let us look at how modern languages support Parallelism in their libraries. Here is an closest example using C# Task Parallel Library (TPL) (from Patterns of Parallel Programming Guide).

https://gist.github.com/aruld/4949484.js

But this is just an attempt if someone wanted to write their own parallel foreach construct using existing C# APIs. But, this is not required. In C#, parallel constructs are available for a while now. Using the Parallel.ForEach construct in TPL, iterating all the Test records of a Student and calculate the GPA would look something like:

https://gist.github.com/aruld/4949503.js

Fair enough, this construct is concise and abstracts all the machinery from the Client code. Are we close enough to get a similar construct in Java? I think so, we do have parallel collections library that supports parallelism using Fork-Join framework today. It looks like Doug Lea has recently revealed his plans for developing new library components for parallel operations on aggregates (collections and arrays) for JDK8 (based on proposed language support for lambdas). I believe this effort would produce these constructs in Java.

For the sake of fun, I compiled the extry166y sources with the compiler option “-XDidentifyLambdaCandidate=true”, it identified 168 candidates that can be turned into a lambda expression. Less code means more fun and you see that for real.

Update (6/11/2011) : Updated the Parallel class (Lambda version) to compile with the latest lambda prototype compiler. Abstract classes with single abstract method are no longer considered valid SAM types as per the revised rules.