Java 8 Streams and JPA

Since the definition of the JPA Standard predates the release of Java 8 it is not surprising that the JPA API is only based on collections. There is no way to obtain a Java 8 Stream out of a query object.

For those out there using Hibernate as their JPA provider there is an interesting way to create a stream for your JPQL queries.

Hibernate Scrollable Results

Hibernate supports many more features than those offered by the JPA Standard only. Hibernate’s Query class allows us to obtain an iterator-like object for a JQPL query. This iterator-like class is named ScrollableResults.

You can see a great example of this feature in this blog post about Reading Large Result Sets with Hibernate and MySQL. I have taken the liberty of copying one of their examples:

Query query = session.createQuery(query);
query.setReadOnly(true);
query.setFetchSize(100);
ScrollableResults results = query.scroll(ScrollMode.FORWARD_ONLY);
while (results.next()) {
   Object row = results.get();
   // process the entity here
}
results.close();

Creating an Iterator Wrapper around Scrollable Results

Let’s take this a bit further now by implementing a Java Iterator wrapper around the Hibernate’s scrollable results object as shown below:

class ScrollableResultIterator<T> implements Iterator<T> {
   private final ScrollableResults results;
   private final Class<T> type;

   ScrollableResultIterator(ScrollableResults results, Class<T> type) {
      this.results = results;
      this.type = type;
   }

   @Override
   public boolean hasNext() {
      return results.next();
   }

   @Override
   public T next() {
      return type.cast(results.get(0));
   }
}

Gaining Access to Hibernate Session through JPA Entity Manager

As you can see the examples above use a Hibernate session object to obtain an instance of a Hibernate query object. Supposing we are using JPA, we typically have access to an EntityManager, but not to any particular implementation objects of the underlaying provider.

To gain access to the Hibernate’s Session object we can use a special method in the entity manger.

Session session = entityManager.unwrap(Session.class);

Be warned that at this point we are escaping from the safety of vendor agnostic code. In other words, if we ever wanted to use another provider like OpenJPA or EclipseLink, we would be forced to find a different alternative to implement our code here since those other providers won’t offer anything like Hibernate’s Session and ScrollableResults.

From Iterators to Spliterators to Streams

Our next step consists in obtaining a Java 8 stream out of this iterator. Lukas Eder recently wrote an interesting article about the difficulties to obtain a stream out of an iterable object which will be pretty handy here.

To achieve our goal we now need to use two utility classes in Java 8 named StreamSupport and Spliterators.

For instance, we could take an iterator, like the one we defined above and transform it into a Spliterator by doing:

private Spliterator<T> toSplitIterator(ScrollableResults scroll, Class<T> type){
   return Spliterators.spliteratorUnknownSize(
      new ScrollableResultIterator<>(scroll, type),
         Spliterator.DISTINCT | Spliterator.NONNULL | 
         Spliterator.CONCURRENT | Spliterator.IMMUTABLE
      );
}

This Splititerator is an intermediate product that we can now use to create a Java 8 stream out of it:

StreamSupport.stream(spliterator, false);

At this point we have built enough to create our first stream out of a JPQL query. We can do it as follows:

public Stream<T> getResultStream(
      String sql, 
      Integer fetchSize, 
      Map<String,Object> parameters) 
{
   Query query = session.createQuery(sql);
   if (fetchSize != null) {
      query.setFetchSize(fetchSize);
   }
   for (Map.Entry<String, Object> parameter : parameters.entrySet()) {
      query.setParameter(parameter.getKey(), parameter.getValue());
   }
   query.setReadOnly(true);
   ScrollableResults scroll = query.scroll(ScrollMode.FORWARD_ONLY);
   return StreamSupport.stream(toSplitIterator(scroll, type), false)
            .onClose(scroll::close);
}

A JPA Stream API

We can do a little bit better by defining the basics of the type of query object we are currently missing in JPA Standard API:

public interface StreamQuery<T> {
   Stream<T> getResultStream();
   StreamQuery<T> setParameter(String name, Object value);
   StreamQuery<T> setFetchSize(int fetchSize);
}

And putting together all we have learned so far we could create this implementation of our new StreamQuery interface:

public class HibernateQueryStream<T> implements StreamQuery<T> {

   private final Session session;
   private final String sql;
   private final Class<T> type;
   private final Map<String, Object> parameters = new HashMap<>();
   private Integer fetchSize;

   public HibernateQueryStream(
      EntityManager entityManager, 
      String sql, 
      Class<T> type) 
   {
     this.session = entityManager.unwrap(Session.class);
      this.sql = sql;
      this.type = type;
   }

   @Override
   public StreamQuery<T> setParameter(String name, Object value) {
      parameters.put(name, value);
      return this;
   }
   
   @Override
   public StreamQuery<T> setFetchSize(int fetchSize) {
      this.fetchSize = fetchSize;
      return this;
   }

   @Override
   public Stream<T> getResultStream() {
      Query query = session.createQuery(sql);
      if (fetchSize != null) {
         query.setFetchSize(fetchSize);
      }
      query.setReadOnly(true);
      for (Map.Entry<String, Object> parameter : parameters.entrySet()) {
         query.setParameter(parameter.getKey(), parameter.getValue());
      }
      ScrollableResults scroll = query.scroll(ScrollMode.FORWARD_ONLY);
      return StreamSupport.stream(toSplitIterator(scroll, type), false)
               .onClose(scroll::close);
   }
   
   private Spliterator<T> toSplitIterator(ScrollableResults scroll, Class<T> type){
      return Spliterators.spliteratorUnknownSize(
         new ScrollableResultIterator<>(scroll, type),
            Spliterator.DISTINCT | Spliterator.NONNULL | 
            Spliterator.CONCURRENT | Spliterator.IMMUTABLE
      );
   }

   private static class ScrollableResultIterator<T> implements Iterator<T> {

      private final ScrollableResults results;
      private final Class<T> type;
      
      ScrollableResultIterator(ScrollableResults results, Class<T> type) {
         this.results = results;
         this.type = type;
      }
      
      @Override
      public boolean hasNext() {
         return results.next();
      }
      
      @Override
      public T next() {
         return type.cast(results.get(0));
      }
   }
}

The Repository Layer

On top of this component we can now build our repository layer.

@Repository
public class StreamRepository {

   @PersistenceContext(unitName="demo")
   private EntityManager entityManager;

   public Stream<Order> getOrderHistory(String email) {
      String jpql = "SELECT o FROM Order o WHERE o.customer.email=:email";
      StreamQuery<Order> query = new HibernateQueryStream<>(
          entityManager, 
          jpql, 
          Order.class
      );
      return query.getResultStream();
   }
}

And from here the rest is a piece of cake:

orderRepository.findOrderHistory(email)
   .filter(order -> order.total() > 100)
   .map(Order::getOrderLines)
   .flatMap(List::stream)
   .map(OrderLine::getTotal)
   .reduce(0, (x,y) -> x + y);

It is important to highlight that the Hibernate session must be alive by the time we actually consume the stream, because it is at that point that the entities will be mapped. So, make sure that wherever you consume the stream the Hibernate session or you entity manager context is still alive.

Further Reading

Functional Programming with Java 8 Functions

Edwin Dalorzo I have finally had the opportunity to work in a couple of projects being entirely developed with Java 8 and during the past months I’ve experienced, first hand, many of the new functional programming features in the language. So I decided to write a series of articles about some of these things I’ve being learning and how some of the well known functional programming constructs can be translated into Java 8.

Preliminaries

Ok, let’s start with something simple. The following is a lambda expression (i.e. an anonymous function) that takes an argument x and increments it by one. In other words a function that receives, apparently an integer, and returns a new incremented integer:

x -> x + 1

And what is the type of this function in Java?

Well, the answer is that it depends. In Java the same lambda expression could be bound to variables of different types. For instance, the following two are valid declarations in Java:

Function<Integer,Integer> add1 = x -> x + 1;
Function<String,String> concat1 = x -> x + 1;

The first one increments an integer x by one, whereas the second one concatenates the integer 1 to any string x.

And how can we invoke these functions?

Well, now that they are bound to a reference we can treat them pretty much like we treat any object:

Integer two = add1.apply(1); //yields 2
String answer = concat1.apply("0 + 1 = "); //yields "0 + 1 = 1"

So, as you can see every function has a method apply that we use to invoke it and pass it an argument.

And what if I already have a method that does that, can I use it as a function?

Yes, since an alternative way to create functions is by using methods we had already defined and that are compatible with our function definition.

Suppose that we have the following class definition with methods as defined below:

public class Utils {
   public static Integer add1(Integer x) { return x + 1; }
   public static String concat1(String x) { return x + 1; }
}

As you can see the methods in this class are compatible with our original function definitions, so we could use them as “method references” to create the same functions we did before with lambda expressions.

Function<Integer,Integer> add1 = Utils::add1;
Function<String,String> concat1 = Utils::concat1;

These two are just the same thing as the ones we did before.

High Order Programming

The cool thing about functions is that they encapsulate behavior, and now we can take a piece of code, put it into a function and pass it around to other functions or methods for them to use it. This type of functions that operate on (or produce new) functions are typically called high order functions and the programming style based on exploiting this powerful feature is called, unsurprisingly, high order programming.

About Functions that Create Functions

Let’s see a couple of examples of how we can do this using function objects. Let’s consider the following example:

Function<Integer, Function<Integer,Integer>> makeAdder = x -> y -> x + y;

Above we have a function called makeAdder that takes an integer x and creates a new function that takes an integer y and when invoked adds x to y. We can tell this is a high order function because it produces a new function.

Now, we use this to create our original add1 function:

Function<Integer,Integer> add1 = makeAdder.apply(1);
Function<Integer,Integer> add2 = makeAdder.apply(2);
Function<Integer,Integer> add3 = makeAdder.apply(3);

With our new high order function, however, we could also create add2, add3, ..., addn, right?

Can’t we define this in a simpler way as we did before with the Utils class methods?

Yes, we can. Consider the following addition to the Utils class:

public class Utils {
    public static Function<Intger, Integer> adder(Integer x) {
       return y -> x + y;
    }
}

This signature is a bit simpler to read than that in our lambda expression, but as you can see it is pretty much the same thing. The function continues to receive the same number and type of arguments and continues to return the same type of result.

We can now use this simpler function factory to create our makeAdder and add1 functions again:

Function<Integer, Function<Integer,Integer>> makeAdder = Utils::adder;
Function<Integer,Integer> add1 = makeAdder.apply(1);

And there you have it, this is exactly the same thing as before.

About Functions that Receive Functions as Arguments

Let’s suppose we had the following two functions defined:

Function<Integer,Integer> add1 = x -> x + 1;
Function<Integer,Integer> mul3 = x -> x * 3;

Now, naively, we could invoke this two functions together to increment and multiply a number by 3, right?. Like this:

Integer x = 10;
Integer res = mul3.apply(add1.apply(x)); //yields 33

But what if we created a function that did both things instead?

Consider the following pseudocode:

(f,g) -> x -> g( f(x) )

This would be a function that takes two other unary functions and creates yet another function that applies the original two in certain order. This is a classical example of what is called function composition.

In some languages there is even an operator to compose two functions in this way:

h = f o g

Where o would be a binary operator that would compose functions f and g pretty much as we did in pseudocode above and produce a new function h.

How can we do function composition in Java?

I can think of two ways to do this in Java, one more difficult than the other. Let’s start with the more difficult first, because that will let us appreciate the value of the simpler solution later on.

Function Composition Strategy 1

First, we must start by realizing that the function in pseudocode above is a binary function (i.e. a function that receives two arguments). But all our examples so far have dealt only with unary functions.

It would seem this is not important, but in Java it is, since functions of different arities have different target functional interfaces. In Java, a function that receives two arguments is called BinaryOperator.

For instance, using a BinaryOperator we could implement a sum operator:

BinaryOperator<Integer> sum = (a,b) -> a + b;
Integer res = sum.apply(1,2); // yields 3

Well, just as easily we could implement the compose operator, right? Only that in this case, instead of two simple integers, we receive two unary functions:

BinaryOperator<Function<Integer,Integer>> compose;
compose = (f,g) -> x -> g.apply(f.apply(x));

Now we can easily use this to fabricate a compound function that adds 1 and multiplies by 3, one after the other.

Function<Integer,Integer> h = compose.apply(add1,mul3);
Integer res = h.apply(10); //yields 33	

And now we can beautifully, and in really simple way, combine two unary integer functions.

Function Composition Strategy 2

Now, function composition is something so common that it would have been a mistake if the Java Expert Group would have not considered it in their API design, and so, to make our lives simpler, all Function objects have a method called compose that allows us to very easily compose two functions together.

The following code produces the exact same result as above:

Function<Integer,Integer> h = mul3.compose(add1);
Integer res = h.apply(10);	

Partial Function Application or Currying

In most functional programming languages it is possible to create partially applied functions. That is, if a function is receiving multiple arguments, we can partially invoke the function providing just a few arguments and receive a partially applied function out of it. This is typically called currying.

Although you have not noticed it, we have already covered that in this article, but now we are going to make it much more evident :-)

So, consider the following pseudo code

sum = x -> y -> x + y

Then we say that sum is a function that accepts one parameter x and fabricates another anonymous function that in turn accepts one parameter y that, when invoked, sums x and y.

In many functional programming languages a construct like this can be invoked as if this was just one simple function by doing:

sum 10 5 //yields 15

But the truth is that this is just syntactic sugar to do:

sum(10)(5) //yields 15

Since sum is a function that returns a function.

The beauty of this idiom is that now we could partially apply sum:

plus10 = sum 10

And now plus10 is a partially applied function of sum. Can you see now where we had already talked about this?

plus10(5) //yields 15

Can we do this with Java?

The truth is that we have already done it above, we just probably did not notice. Unfortunately, in Java we do not have the syntactic sugar that some other language have, and therefore this is a bit more verbose:

Function<Integer,Function<Integer,Integer>> sum = x -> y -> x + y;

Well, you can see sum is declared in a “currified” way. And now we can partially apply it:

Function<Integer, Integer> plus10 = sum.apply(10);
Integer res = plus10.apply(5); //yields 15

Unary, Binary, Ternary and n-ary Functions

So, as mentioned above, Java uses different functional interfaces for different function arities. And so Function<T,R> is a functional interface for any unary function where the domain and the codomain of the function may be of different types.

For instance, we could define a function that receives a string value and parses it as an integer:

Function<String,Integer> atoi = s -> Integer.valueOf(s);

But most of our examples above are for integer functions whose argument and return value are of this same type. For those cases we could alternatively use the UnaryOperator<T> instead. This is just a Function<T,T>.

Thus, some our declarations above could be slightly simplified with this functional interface:

UnaryOperator<Integer> add1 = n -> n + 1;
UnaryOperator<String> concat1 = s -> s + 1;
Function<Integer, UnaryOperator<Integer>> sum = x -> y -> x + y;
UnaryOperator<Integer> sum10 = sum.apply(10);

I have already written another article that explains Why Java 8 has Interface Pollution like this in case you are interested in an explanation.

Value Types and Primitive Type Functions?

Evidently using a type like Integer incurs into the costs of boxing and unboxing when our functions have to deal with a primitive type like int.

As you know, Java does not support value types as type arguments in generic declarations, so to deal with this problem we can use alternative functional interfaces like ToIntFunction, IntFunction or IntUnaryOperator.

Or we can define our own primitive function.

interface IntFx {
	public int apply(int value);
}

Then we can do:

IntFx add1 = n -> n + 1;
		
IntFunction<IntFx> sum = x -> y -> x + y;
IntFx sum10 = sum.apply(10);
sum10.apply(4); //yields 14

Similar functional interfaces can be found in the Java API for types double and long as well. This topic is also covered in the alternative article mentioned above about interface pollution.

And that’s it for the time being. I hope that in a future article I can take this a bit further and show some practical applications derived from my experience in actual projects.

Further Reading

How to Configure SML REPL with rlwrap

The SML REPL in smlnj does not have support for readline. So, it’s pretty annoying that the REPL does not have command history, line editing and syntax completion capabilities. However all this problems can be alleviated if we use rlwrap. This program intercepts user input in order to provide readline’s line editing, persistent history and completion.

Supposing you already have smlnj installed. Start by installing rlwrap. There are different ways to install it depending on your operating system. I tried it in my Ubuntu box and my OSX Mavericks machine, in this latter using Homebrew.

So, Ubuntu/Debian users can do:
apt-get install rlwrap

And MAC users can do:
brew install rlwrap

Now, all we have to do is to configure our sml command to use rlwrap. In my case I wrote an alias in my .bashrc file

alias sml="rlwrap -f /usr/local/share/sml/keywords -t sml /usr/bin/sml"

The keywords is a text file that I created and which I put all SML keywords and that will provide syntax autocompletion when you’re in the REPL.

These are the contents of the my keywords file. I hope I have not missed any keyword.

abstype and andalso as case do datatype else
eqtype end exception fn fun functor handle
if in include infix infixr let local nonfix
of op open orelse raise rec sharing sig
signature struct structure then type val
with withtype while

And voila! We’re read to go. Now simply invoke the sml command, and this time you should have line editing capabilities, command history and syntax autocompletion.

How to Declare Modules in Node.js

Edwin DalorzoOne of those aspects of Node.js that took me a while to fully understand initially is how to properly declare modules. At first it looks kind of obvious and intuitively simple, but later, you realize you can expose different types of objects like functions, constructors, properties or entire object instances. So I have decided to write this article with different examples and techniques I learned while writing different type of modules in Node.js.

On Modules, Import and Export

Let’s start by the most obvious and simple thing. Something probably everyone learns since the first day of work with Node: every code file is considered a module. The variables, properties, functions, constructors that we declared in it are private to the module and other modules cannot gain access to them or use them unless the programmer of the module explicitly expose them to the public; namely everything we declare inside a module is encapsulated and hidden from the outside world by default unless explicitly stated otherwise. To expose something the programmer has access to a special object called module, which has a special property called exports. Everything that you publish in the module.exports object is made publicly available to other modules. For instance, in the code below, the variable pi is inaccessible to any other modules but foo.js, whereas the property named bar is made publicly available to any other modules importing the module foo.js. Note that this is a fundamental difference from JavaScript in Node.js when compared with JavaScript as executed in a browser where objects are publicly exposed in a global object (i.e. window).

//module foo.js
var pi = 3.14;
module.exports.bar = 'Hello World';

Now a second module baz.js can “import” the module foo.js and gain access to the property bar. In Node, we achieve this effect by means of using a global function named require. Somewhat as follows:

//module baz.js
var foo = require('./foo');
console.log(foo.bar); //yields Hello World

Technique 1 – Extending exports Object with Additional Functionality

So, one technique to expose the functionality in a module consists in adding functions and properties to the module.exports object. When this is the case, Node provides a direct access to the exports object to make things simpler for us. For instance:

//module foo.js
exports.serviceOne = function(){ };
exports.serviceTwo = function(){ };
exports.serviceThree = function(){ };

And as you might expect, the users of this module, at importing it, would obtain a reference to the exports object and by this they would gain access to all the functionality exposed in it.

//module bar.js
var foo = require('./foo');
foo.serviceOne();
foo.serviceTwo();
foo.serviceThree();

Technique 2 – Substitute Default exports Object with Another Object

By this point you probably suspect that given the fact that module.exports is just an object that exposes the public part of a module then we could probably define our own object and then replace the default module.exports object with our own. For instance:

//module foo.js
var service = {
   serviceOne: function(){ },
   serviceTwo: function(){ },
   serviceThree = function(){ }
};

module.exports = service;

The code in this last example would behave exactly as the code in the previous example, it’s just that this time we have explicitly created our exported object instead of using the one provided by default by Node.

Technique 3 – Substitute Default exports Object with a Constructor Function

In the examples so far we have always used an instance of an object as our exposed target. However there are occasions in which it may seem more convenient to allow the user to create as many instances of a given type as she wants. Nothing prevents us from replacing the module.exports object with other types of objects like a constructor function. In the example below we expose a constructor which the user can use to create many instances of the Foo type.

//module Foo.js
function Foo(name){
   this.name = name;
}

Foo.prototype.serviceOne = function(){ };
Foo.prototype.serviceTwo = function(){ };
Foo.prototype.serviceThree = function(){ };

module.exports = Foo;

And the user of this module can simply do something like this:

//module bar.js
var Foo = require('./Foo');
var foo = new Foo('Obi-wan');
foo.serviceOne();
foo.serviceTwo();
foo.serviceThree();

Technique 4 – Substitute Default exports Object with Plain Old Function

It is easy to imagine now that if we can use a constructor function then we might just as well be able to use any other plain old JavaScript function as the target exposed in module.exports.
As in the following example in which our exported function allows the user of this module to gain access to one of several other encapsulated service objects.

//foo.js
var serviceA = {};
serviceA.serviceOne = function(){ };
serviceA.serviceTwo = function(){ };
serviceA.serviceThree = function(){ };

var serviceB = {};
serviceB.serviceOne = function(){ };
serviceB.serviceTwo = function(){ };
serviceB.serviceThree = function(){ };

module.exports = function(name){
   switch(name){
      case 'A': return serviceA;
      case 'B': return serviceB;
      default: throw new Error('Unknown service name: ' + name);
   }
};

Now the user that imports this module receives a reference to our anonymous function declared above and then she can simply invoke the function to gain access to one of our encapsulated objects. For instance:

//module bar.js
var foo = require('./foo');
var obj = foo('A');
obj.serviceOne();
obj.serviceTwo();
obj.serviceThree();

Many programmers ordinarily invoke the function immediately returned by require instead of assigning it to a reference first. For instance:

//module bar.js
var foo = require('./foo')('A');
foo.serviceOne();
foo.serviceTwo();
foo.serviceThree();

So, in summary, it is as simple as follows: everything that we expose in module.exports is what we get when we invoke require. And using different techniques we could expose objects, constructors functions, properties, etc.

About Modules and the use Global State

An interesting aspect of modules is the way they are evaluated. The module is evaluated the first time it is required and then it is cached. This means that after it has been evaluated no matter how many times we require it, we will always get the same exported object back.

This means that, although Node provides a global object, it is probably better to use modules to store shared stated instead of putting it directly into the global object. For instance, the following module exposes the configuration of a Mongo database.

//module config.js

dbConfig = {
  url:'mongodb://foo',
  user: 'anakin',
  password: '*******'
}

module.exports = dbConfig;

We can easily share this module with as many other modules as we want, and everyone of them will get the exact same instance of the configuration object since the module is evaluated only once and the exported object is cached from there on.

//foo.js
var dbConfig1 = require('./config');
var dbConfig2 = require('./config');
var assert = require('assert');
assert(dbConfig1==dbConfi2);

10 Useful Apple Store Apps for Programmers and Alpha Geeks

Edwin Dalorzo

Recently, Lukas Eder wrote a post about The Top 10 Productivity Booster Techs for Programmers and in one of the comments he challenged me to write my own list of cool tools. I decided to give it a try and in my attempt I discovered it is not so easy to define your 10 most favorite tools. Actually, right now, my board is full with names of cool tools that I, as a programmer, use very often. I realize that I could probably make a 10-list on many different categories. So, to enrich what Lukas originally wrote, I have decided to start with a list of the kind of cool apps that any alpha-geek programmers would probably love to install in their Apple devices.

Lisping

If you are a fan of functional programming and particularly of the Lisp family of programming languages, then you are going to love Lisping. It is available for iPhone and iPad and you can create and run programs for the Scheme and Clojure programming languages. To avoid confusion with all the parenthesis the applications offers a very interesting and intuitive way of edition that allows you to focus on a specific context at once. Honestly I found it difficult to edit programs with it in my iPad, but still it is a tool that any alpha-geek can appreciate. Now you can test your small Scheme programs from the SICP book while you watch the extended DVD edition of the Lord of the Rings for the fifth time.

 

Raskell

So, not a fan of dynamically-typed functional programming languages? Well, if you prefer the safety of static typing instead then you’ll surely love Raskell. There you can edit and run programs for the Haskell programming language and it is also available for iPhone and iPad. Now, you can test those code snippets from Learn you a Haskell for Great Good while you’re comfortably reading in bed and trying to see if this time you can actually figure out what the heck is a monad!!!.

 

Pythonista

If multiparadigm programming languages is your thing, and particularly if you are fan of the Python programming language then Pythonista is the application you are looking for. The app is available for both iPhone and iPad and as you might expect lets you edit and run programs for the Python programming language, but it goes beyond that, it provides support for multi-touch, animations and sound, so that your programs can really take advantage of your device. Now you can let your Pythonic spirit run free while you wait the end of the five-minute TV commercials broadcast during your latest episode of The Big Bang Theory.

 

Textastic

Well, not every one of our favorite programming languages has a cool app editor. So, for those of us fan of other programming languages, we have this great application editor called Textastic which provides syntax highlighting support for more then 80 different programming languages and with really cool edition features that makes it really easy to use in mobile devices. When used in the iPad or the iPhone the keyboard is extended with a new set of keys typically used by programmers and that makes writing code much simpler. The app is also available for MAC OSx. So, now you can edit your cool open source programs while your girlfriend tries to tell you all about her day at dinner in that fancy restaurant.

Penultimate

What about software design? Well, I have not been able to find a decent app to do things like flowcharts or UML diagrams. However, I found that if you have an stylus, you can create your own handwritten designs in Penultimate. I have actually defined a good collection of flowcharts and UML diagrams in this way. It is a great tool to discuss design ideas, algorithms or simply to keep your thoughts and great software ideas properly recorded.

 

Dash (Docs & Snippets)

Are you tired of searching the Web for the Javadocs of a given class? Did you forget the methods of one of those angular services? What were the parameters of that git command? Heck, every time I want to read the Javadocs I have to first search the URL in Google. Well, with Dash those days are over. It is like the Google search of programmers. It offers offline instant access to dozens of different API documentations. Just type a term and it’ll find the answer you were looking for.

 

Code Runner

Does it happen to you too that sometimes you would like to test just small code snippet of Java, or JavaScript and you are forced to setup an entire project in you IDE just to test a small idea? Well, with Code Runner you avoid all these complications. It is a simple way to run small code snippets or simply a tool to run any program in any programming language with just a single click. It comes with a set of predefined programming languages, but you can extend it with more.

Instapaper

It is hard to keep up with the latest trends in technology. Every week I find several interesting articles I would like to read and often I try to remember articles I had read in the past. In order to keep up with all this information, for me, there is no tool like Instapaper. I save there articles I want to read and then read them offline later on, when I have time, directly from my iPad or iPhone. So this is the perfect tool for those days when you girlfriend says she’s got a headache. Keep it near bed, it will help you concentrate on something else and believe me you’re going to be the most well informed geek in your community.

Evernote

Finally I have a tool to keep all my ideas organized and available at all times. I have been in dozens of projects and always lose important data from every one. I can barely remember details of requirements, or the IP address for a server, or the location of all repositories, or what about the FQDN of the development server which I need to access via SSH? And those other cool ideas for projects or investigations in progress. Not to mention those great ideas for articles posts, etc. Well, with Evernote all this gets very simple. Never ever again lose any of those cool ideas or investigations in progress, or important things to remember. And if you combine it with Evernote Web Clipper it will serve the same purpose as Instapaper, mentioned above.

SSH Term Pro

So you’re watching again the remasterized  version of Star Wars and then you feel curiosity to know if that script you left running on your server on Friday before going home has finished successfully. Well, you no longer need to get up from your couch. With SSH Term Pro you can access your server via SSH and have access to a terminal and go crazy. Your boss is going to love you for getting this tool.

Well, that’s all I’ve got for now. If you guys know of any other cool apps that you consider are a must-have for alpha geeks do not hesitate a second to leave a comment and maybe we can enrich this list with even more cool apps like these.

 

Why There Is Interface Pollution in Java 8

Edwin DalorzoI was reading this interesting post about The Dark Side of Java 8. In it, Lukas Eder, the author, mentions how bad it is that in the JDK 8 the types are not simply called functions. For instance, in a language like C#, there is a set of predefined function types accepting any number of arguments with an optional return type (Func and Action each one going up to 16 parameters of different types T1, T2, T3, …, T16), but in the JDK 8 what we have is a set of different functional interfaces, with different names and different method names, and whose abstract methods represent a subset of well know function signatures/function arities (i.e. nullary, unary, binary, ternary, etc).

The Type Erasure Issue

So, in a way, both languages suffer from some form of interface pollution (or delegate pollution in C#). The only difference is that in C# they all have the same name. In Java, unfortunately, due to type erasure, there is no difference between Function<T1,T2> and Function<T1,T2,T3> or Function<T1,T2,T3,...Tn>, so evidently, we couldn’t simply name them all the same way and we had to come up with creative names for all possible types of function combinations.

Don’t think the expert group did not struggle with this problem. In the words of Brian Goetz in the lambda mailing list:

[…] As a single example, let’s take function types. The lambda strawman offered at devoxx had function types. I insisted we remove them, and this made me unpopular. But my objection to function types was not that I don’t like function types — I love function types — but that function types fought badly with an existing aspect of the Java type system, erasure. Erased function types are the worst of both worlds. So we removed this from the design.

But I am unwilling to say “Java never will have function types” (though I recognize that Java may never have function types.) I believe that in order to get to function types, we have to first deal with erasure. That may, or may not be possible. But in a world of reified structural types, function types start to make a lot more sense […]

So, how does this affect us as developers? The following is a categorization of some of the most important new functional interfaces (and some old ones) in the JDK 8 organized by function return type and the number of expected arguments in the interface method.

Functions with Void Return Type

In the realm of functions with a void return type, we have the following:

Type of Function Lambda Expression Known Functional Interfaces
Nullary
				() -> doSomething()
			
Runnable
Unary
				foo  -> System.out.println(foo)
			
Consumer
IntConsumer
LongConsumer
DoubleConsumer
Binary
				(console,text) -> console.print(text)
			
BiConsumer
ObjIntConsumer
ObjLongConsumer
ObjDoubleConsumer
n-ary
				(sender,host,text) -> sender.send(host, text)
			
Define your own

Functions with Some Return Type T

In the realm of functions with a return type T, we have the following:

Type of Function Lambda Expression Known Functional Interfaces
Nullary
				() -> "Hello World"
			
Callable
Supplier
BooleanSupplier
IntSupplier
LongSupplier
DoubleSupplier
Unary
				n -> n + 1
				n -> n >= 0
			
Function
IntFunction
LongFunction
DoubleFunction
IntToLongFunction
IntToDoubleFunction
LongToIntFunction
LongToDoubleFunction
DoubleToIntFunction
DoubleToLongFunction
UnaryOperator
IntUnaryOperator
LongUnaryOperator
DoubleUnaryOperator
Predicate
IntPredicate
LongPredicate
DoublePredicate
Binary
				(a,b) -> a > b ? 1 : 0
				(x,y) -> x + y
				(x,y) -> x % y == 0
			
Comparator
BiFunction
ToIntBiFunction
ToLongBiFunction
ToDoubleBiFunction
BinaryOperator
IntBinaryOperator
LongBinaryOperator
DoubleBinaryOperator
BiPredicate
n-ary
				(x,y,z) -> 2 * x + Math.sqrt(y) - z
			
Define your own

An advantage of this approach is that we can define our own interface types with methods accepting as many arguments as we would like, and we could use them to create lambda expressions and method references as we see fit. In other words, we have the power to pollute the world with yet even more new functional interfaces. Also we can create lambda expressions even for interfaces in earlier versions of the JDK or for earlier versions of our own APIs that defined SAM types like these. And so now we have the power to use Runnable and Callable as functional interfaces.

However, these interfaces become more difficult to memorize since they all have different names and methods.

Still, I am one of those wondering why they didn’t solve the problem as in Scala, defining interfaces like Function0, Function1, Function2, …, FunctionN. Perhaps, the only argument I can come up with against that is that they wanted to maximize the possibilities of defining lambda expressions for interfaces in earlier versions of the APIs as mentioned before.

Lack of Value Types

So, evidently type erasure is one driving force here. But if you are one of those wondering why we also need all these additional functional interfaces with similar names and method signatures and whose only difference is the use of a primitive type, then let me remind you that in Java we also lack of value types like those in a language like C#. This means that the generic types used in our generic classes can only be reference types, and not primitive types.

In other words, we can’t do this:

List<int> numbers = asList(1,2,3,4,5);

But we can indeed do this:

List<Integer> numbers = asList(1,2,3,4,5);

The second example, though, incurs in the cost of boxing and unboxing of the wrapped objects back and forth from/to primitive types. This can become really expensive in operations dealing with collections of primitive values. So, the expert group decided to create this explosion of interfaces to deal with the different scenarios. To make things “less worse” they decided to only deal with three basic types: int, long and double.

Quoting the words of Brian Goetz in the lambda mailing list:

More generally: the philosophy behind having specialized primitive streams (e.g., IntStream) is fraught with nasty tradeoffs. On the one hand, it’s lots of ugly code duplication, interface pollution, etc. On the other hand, any kind of arithmetic on boxed ops sucks, and having no story for reducing over ints would be terrible. So we’re in a tough corner, and we’re trying to not make it worse.

Trick #1 for not making it worse is: we’re not doing all eight primitive types. We’re doing int, long, and double; all the others could be simulated by these. Arguably we could get rid of int too, but we don’t think most Java developers are ready for that. Yes, there will be calls for Character, and the answer is “stick it in an int.” (Each specialization is projected to ~100K to the JRE footprint.)

Trick #2 is: we’re using primitive streams to expose things that are best done in the primitive domain (sorting, reduction) but not trying to duplicate everything you can do in the boxed domain. For example, there’s no IntStream.into(), as Aleksey points out. (If there were, the next question(s) would be “Where is IntCollection? IntArrayList? IntConcurrentSkipListMap?) The intention is many streams may start as reference streams and end up as primitive streams, but not vice versa. That’s OK, and that reduces the number of conversions needed (e.g., no overload of map for int -> T, no specialization of Function for int -> T, etc.)

We can see that this was a difficult decision for the expert group. I think few would agree that this is cool, and most of us would most likely agree it was necessary.

The Checked Exceptions Issue

There was a third driving force that could have made things even worse, and it is the fact that Java supports two type of exceptions: checked and unchecked. The compiler requires that we handle or explicitly declare checked exceptions, but it requires nothing for unchecked ones. So, this creates an interesting problem, because the method signatures of most of the functional interfaces do not declare to throw any exceptions. So, for instance, this is not possible:

Writer out = new StringWriter();
Consumer<String> printer = s -> out.write(s); //oops! compiler error

It cannot be done because the write operation throws a checked exception (i.e. IOException) but the signature of the Consumer method does not declare it throws any exception at all. So, the only solution to this problem would have been to create even more interfaces, some declaring exceptions and some not (or come up with yet another mechanism at the language level for exception transparency). Again, to make things “less worse” the expert group decided to do nothing in this case.

In the words of Brian Goetz in the lambda mailing list:

Yes, you’d have to provide your own exceptional SAMs. But then lambda conversion would work fine with them.

The EG discussed additional language and library support for this problem, and in the end felt that this was a bad cost/benefit tradeoff.

Library-based solutions cause a 2x explosion in SAM types (exceptional vs not), which interact badly with existing combinatorial explosions for primitive specialization.

The available language-based solutions were losers from a complexity/value tradeoff. Though there are some alternative solutions we are going to continue to explore — though clearly not for 8 and probably not for 9 either.

In the meantime, you have the tools to do what you want. I get that you prefer we provide that last mile for you (and, secondarily, your request is really a thinly-veiled request for “why don’t you just give up on checked exceptions already”), but I think the current state lets you get your job done.

So, it’s up to us, the developers, to craft yet even more interface explosions to deal with these in a case-by-case basis:

interface IOConsumer<T> {
   void accept(T t) throws IOException;
}

static<T> Consumer<T> exceptionWrappingBlock(IOConsumer<T> b) {
   return e -> {
	try { b.accept(e); }
	catch (Exception ex) { throw new RuntimeException(ex); }
   };
}

In order to do:

Writer out = new StringWriter();
Consumer<String> printer = exceptionWrappingBlock(s -> out.write(s));

Probably, in the future (maybe JDK 9) when we get Support for Value Types in Java and Reification, we will be able to get rid of (or at least no longer need to use anymore) these multiple interfaces.

In summary, we can see that the expert group struggled with several design issues. The need, requirement or constraint to keep backwards compatibility made things difficult, then we have other important conditions like the lack of value types, type erasure and checked exceptions. If Java had the first and lacked of the other two the design of JDK 8 would probably have been different. So, we all must understand that these were difficult problems with lots of tradeoffs and the EG had to draw a line somewhere and make a decisions.

So, when we find ourselves in the dark side of Java 8, probably we need to remind ourselves that there is a reason why things are dark in that side of the JDK :-)

Further Reading

  • Related Answer in Stackoverflow
  • Lambda Straw-Man Proposal
  • Neal Gafter on The Future of Java
  • Memoized Fibonacci Numbers with Java 8

    Edwin DalorzoSince today is Fibonacci Day, I decided that it would be interesting to publish something related to it.

    I believe one of the first algorithms we all see when learning non-linear recursion is that of calculating a Fibonacci number. I found a great explanation on the subject in the book Structure and Interpretation of Computer Programs [SIC] and I dedicated some time to playing with the Fibonacci algorithm just for fun. While doing so I found an interesting way to improve the classical recursive algorithm by using one of the new methods (added in Java 8) in the Map interface and which I used here to implement a form of memoization.

    Classical Recursive Fibonacci

    In the classical definition of Fibonacci we learn that:

    fib(n) = \left\{ \begin{array}{ll} 0 & \mbox{if n=0}\\1 & \mbox{if n=1}\\fibn(n-1)+fib(n-2) & \mbox{otherwise} \end{array} \right.

    We program this very easily in Java:

    public static long fibonacci(int x) {
       if(x==0 || x==1)
          return x;
       return fibonacci(x-1) + fibonacci(x-2);
    }
    

    Now the problem with this algorithm is that, with the exception of the base case, we recursively invoke our function twice and interestingly one of the branches recalculates part of other branch every time we invoke the function. Consider the following image (taken from SIC) that represents an invocation to fibonacci(5).

    Clearly the branch to the right is redoing all the work already done during the recursive process carried out by the left branch. Can you see how many times fibonacci(2) was calculated? The problem gets worse as the function argument gets bigger. In fact this problem is so serious that the calculation of a small argument like fibonacci(50) might take quite a long time.

    Memoized Recursive Fibonacci

    However, there is a way to improve the performance of the original recursive algorithm (I mean without having to resort to a constant-time algorithm using, for instance, Binet’s formula).

    The serious problem we have in the original algorithm is that we do too much rework. So, we could alleviate the problem by using memoization, in other words by providing a mechanism to avoid repeated calculations by caching results in a lookup table that can later be used to retrieve the values of already processed arguments.

    In Java we could try to store the Fibonacci numbers in a hast table or map. In the case of the left branch we’ll have to run the entire recursive process to obtain the corresponding Fibonacci sequence values, but as we find them, we update the hash table with the results obtained. This way, the right branches will only perform a table lookup and the corresponding value will be retrieved  from the hash table and not through a recursive calculation again.

    Some of the new methods in the class Map , in Java 8, simplify a lot the writing of such algorithm, particularly the method computeIfAbsent(key, function). Where the key would be the number for which we would like to look up the corresponding Fibonacci number and the function would be a lambda expression capable of triggering the recursive calculation if the corresponding value is not already present in the map.

    So, we can start by defining a map and putting the values in it for the base cases, namely, fibonnaci(0) and fibonacci(1):

    private static Map<Integer,Long> memo = new HashMap<>();
    static {
       memo.put(0,0L); //fibonacci(0)
       memo.put(1,1L); //fibonacci(1)
    }
    

    And for the inductive step all we have to do is redefine our Fibonacci function as follows:

    public static long fibonacci(int x) {
       return memo.computeIfAbsent(x, n -> fibonacci(n-1) + fibonacci(n-2));
    }
    

    As you can see, the method computeIfAbsent will use the provided lambda expression to calculate the Fibonacci number when the number is not present in the map, this recursive process will be triggered entirely for the left branch, but the right branch will use the momoized values. This represents a significant improvement.

    Based on my subjective observation, this improved recursive version was outstandingly faster for a discrete number like fibonacci(70). With this algorithm we can safely calculate up to fibonacci(92) without running into long overflow. Even better, to be sure that our algorithm would never cause overflows without letting the user know we could also use one of the new methods in Java 8 added to the Math class and which throws an ArithmeticException when overflow occurs. So we could change our code as follows:

    public static long fibonacci(int x) {
       return memo.computeIfAbsent(x, n -> Math.addExact(fibonacci(n-1),
                                                         fibonacci(n-2)));
    }
    

    This method would start failing for fibonacci(93). If we need to go over 92 we would have to use BigInteger in our algorithm, instead of just long.

    Notice that the memozied example uses mutations, therefore, in order to use this code in a multithreaded environment we would first need to add some form of synchronization to the proposed code, or use a different map implementation, perhaps a ConcurrentHashMap, which evidently, may impact performance as well. Arguably, this would still be better than the original recursive algorithm.