February 5th, 2015

by Ivan St. Ivanov


Last week in the Bulgarian JUG we had an event dedicated on trying out the OpenJDK Valhalla project’s achievements in the area of using primitives parameters of generics. Our colleague and dedicated blogger Mihail Stoynov already wrote about our workshop. You can find very useful links in his post about the slides that I showed, the VM that he prepared (which you can use to try Valhalla yourself) and even the recording of the meeting (which unfortunately for some of you is in Bulgarian).

In this three-part series of blogs I would like to go in some more details about the current implementation proposal and the reasoning behind the decisions. No, I am not an expert in this matter. I just try to follow the project Valhalla mailing list as well as I read Brian Goetz’s State of Specialization document. So look at this series more as explaining the generics proposals in layman’s terms.


Java generics is one of its most widely commented topics. While the discussion whether they should be reified, i.e. the generic parameter information is not erased by javac, is arguably the hottest topic for years now, the lack of support for primitives as parameter types is something that at least causes some confusion. It leads to applying unnecessary boxing when for example you want to put an int into a List (read on to find out about the performance penalty). It also leads to adding “companion” classes in most of the generic APIs, like IntStream and LongStream for example.

One of the goals of OpenJDK’s project Valhalla is among others to research the possibility to generify over primitives in the language, the Virtual Machine and the standard libraries. Yes, it’s just research. Which means that the current proposal may or may not appear in the future versions of Java. What I am pretty sure is that it won’t make it to Java 9, which is about to be shipped next year.

The State of Generics

As already mentioned, it is not possible at the moment to define generic type or method parameterized with a primitive type. So if you want to create, let’s say,  a List of integers, you will have to consider using the wrapper class:

List<Integer> intList = new ArrayList<>();

Besides looking kind of artificial, this brings also huge performance penalty. It comes from the way reference types are layed out in memory. If we take the array list above, internally it is represented as an array (of Integer’s). Which means that we will get an array with references to Integer objects scattered in the heap. Looking at a single Integer object we need memory not only for the int itself but for all other things needed by the virtual machine: pointer to the class object, some space for the garbage collector flags, others for the object monitor used by the Java synchronization infrastructure, etc. So instead of beautifully ordered ints, we potentially get something like this:



The memory overhead is not the only issue here. The modern processor architectures rely on several layers of cache. This makes going to the main memory for fetching the value of a certain variable a very expensive operation in terms of CPU cycles. That is why most of the optimizations done by the JVM tend to put as much data in the registers as possible. But the problem is that when talking about arrays (remember, ArrayList is represented as an array), the CPU instructions can only cache contiguous memory addresses. Thus if our Integers are scattered around the heap, most probably our VM will not be able to put them in the registers and we’ll have to pay the performance penalty of the cache misses.

Why not generics over primitives?

Normal question here would be why doesn’t Java support generifying over primitive types. The short answer is: because of generic type erasure by javac. Slightly longer answer follows in the next few paragraphs.

Let’s suppose that we have the following class definition:

public class Box<T> {
    private T value;
    public Box(T value) {
        this.value = value;

    public T get() {
        return value;


The generic type T is only used by javac to ensure that correct types are boxed and then retrieved. If you decompile the product of the above class’s compilation, you will notice that the type of the value variable, the constructor parameter and the return type of the get() method are all java.lang.Object. Simply the compiler “erases” the information that you coded above and replaces it with the type that is the parent of all reference types. In Java there is no such thing as a common type of all types (both reference and primitive). Something like Any in Scala for example. That is why you cannot apply erasure to all types: with the current implementation javac doesn’t know to what they should be converted.

Why erasure at all?

Astute readers will ask the question: “But why we need this erasure anyway?” Before trying to answer it, let me first elaborate a bit on the compatibility topic.

Suppose that we have type A. And then we have a class C that uses or extends A. And class A is changed in some manner. We say that this change is source incompatible if the class C does not compile any more after this change (this is rather simplistic explanation, there are also a couple of other subtle causes of source incompatibilities, but let’s keep it simple). Some of you might remember when the enum keyword was added to the language in Java 5 – all the code that used that as identifier did not compile anymore. The same will be the fortune of all of you that use sun.misc.Unsafe BTW ;).

Next, suppose that we change somehow A and let’s say that both classes live in different jars. Class C may still compile after A’s change, however if you drop the hypothetical A.jar in the class path of our program, class C may refuse to link. This is considered as binary incompatibility. You may refer to the Java Language Specification for more information on that matter.

Going back to the generics story. If it was decided upon their introduction in Java 5 that the generic type is not going to be erased, then most likely it would break at least the binary compatibility of your classes. As generics were applied to the most widely used part of the API: the collection library, it would mean that virtually any meaningful Java program in this world would have to be at least recompiled on the day its users decided to upgrade to Java 5. The situation becomes even more complicated, because most of the libraries that we use in our program are developed and maintained by someone else. Which means that if one wanted to upgrade to Java 5, they would need to wait for all the external libraries to be recompiled with Java 5.

The bottom line is that non-erased generic type parameters would have brought a “flag day” when everybody should have recompiled and delivered a new version of their libraries. Which might be fine for smaller or more obedient language communities, but is not the case for Java.


So there are really compelling reasons why we are not able to generify our types and methods over primitive types. In the next installment of this series we’ll look at the current proposal in OpenJDK’s project Valhalla on how it can be implemented without breaking compatibility with older releases of Java.

October 6th, 2014

by Ivan St. Ivanov


The last day at JavaOne started as usual with the community keynote. I didn’t go to it, because I wanted to have a rest after the Aerosmith and Macklemore & Ryan Lewis concert last night and also wanted to catch up with my blogs. However, the people that I follow on twitter were kind enough to update me with the most interesting bits of the session. Additionally, there’s already a blog from Ben Evans about it.

Basically Oracle understood their mistake from the opening keynote on Sunday and let Mark Reinhold and Brian Goetz on the stage to do their technical talk that was cut so abruptly a few days ago. Apparently Java 9 was the central topic. Modularity, value types, primitive types in generics, native data and code access, array rework, etc. are the most prominent things that will hopefully come with the next Java release.

The keynote continued with a panel moderated by Mark Reinhold, having Brian Goetz as well as James Gosling (the father of Java), Charles Nutter (the JRuby guy, from RedHat), John Rose (JVM Architect, known for things like invokedynamic and the value types in Java) and Brian Oliver (Oracle’s in memory data grid architect). They answered various questions marked hashed with #j1qa on Twitter (BTW, it was trending in top 10 on the Thursday morning). The most tweeted statement from the conversion was that James Gosling was not sorry for introducing null in the language.

The final part of the keynote was dedicated to the community (at the end, this was the community keynote). The most important news was that like last year all the JavaOne 2014 sessions will be on Parleys.

JBoss Forge

I don’t know whether there is a reader of this blog that does not know what JBoss Forge is (check out its new cool website). But still: it is a rapid application development enabler tool, built on extensible platform and as of its second version with abstraction over its user input. Yes, as of Forge 2 you can run the tool both on the command line as well as with wizards in your favorite IDE. Actually a Forge generated or managed project doesn’t have any dependency to the tool. It builds all the knowledge from the pom.xml for Maven projects and build.gradle for Gradle projects. The extensibility of Forge is because of its modularity. No, it’s not OSGi, Forge is based on JBoss modules and has quite some machinery based on Maven on top.

Where does Forge help, you would ask? In any areas, I would answer:

  • Figure out for you the right dependency that you have to add to a Java (EE) project
  • Put the right entry for your JPA provider in the persistence.xml
  • Help you declare a bi-directional one-to-many dependency
  • Plumb for you the boring CRUD classes
  • Prepare a mock UI (a.k.a. a scaffold) out of your domain model

This is done by already existing Forge addons, but you can very easily create one of your own to help you in your daily job activities. If you have a tedious task, where you constantly copy and paste artifacts from one project to another, then you can think of implementing your own addon. It’s built with Maven and you can use dependency injection for the dependencies from the Forge core that your addon might require (CDI or the service loader). Once you have developed your addon Forge gives you a way to test it in the Arquillian way, i.e. in-container without any need to mock anything, and then deploy it on the Forge runtime. You can also develop listeners to observe events before and after Forge command executions.

Go and try Forge and then join us and contribute! And you might get one of these.

Be reactive with Java 8

The next logical step after applying functional programming practices to your APIs is to make your modules containing those APIs reactive. Well, this might sound as the next buzzword, but in fact it is not. The reactive programming style was something that was invented long ago, we forgot it and now we found how cool it is.

How does your program become reactive? Well, if it serves requests from external clients and if these requests take longer than expected, the call on the client side should not block on one side and on the other side the server should respond in timely manner that it is processing the request. It is very important to not spawn a separate thread for every request – if the client retries the request again and again, the threads may quickly expire and the memory could be filled up. Using futures is not a solution to that: yes, they seem asynchronous, but at the end you have to wait on their get() method. Callbacks seem a neat solution, but the problem with them is that they cannot be nested.

So, how do you achieve the efficiency of the reactive programming style in Java 8? There are quite a few libraries, but the last session that I happened to visit at JavaOne (by Venkat Subramaniam) focused on one of them: RxJava library from Netflix.

Its API is built around the Observable class. You create an instance of that class on the server side. It receives kind of recipe as a parameter, telling the RxJava library what to do upon an event happening on the server side. This recipe is an implementation of the Observer interface and you can call any of its methods inside the create() method body. Then the client can call one of the subscribe() methods of the received Observable to pass lambdas that will react on a given event happening on the server side. The types of events that the client can react upon are success (the onNext event), error (onError) and completion (onComplete). For each one of them there is a special subscribe overloading method, as well as a method receiving a full blown extension of the Subscribed abstract class.

You can call the unsubscribe method of the Observable on the client side to stop “receiving” updates from the server part. Besides those, Observable class has lots of other methods that make it look like a real [functional] lazy abstraction: skip() and skipWhile() for skipping the first events happening on the server, take() to take some of the events and then stop observing, combine() to combine this with other Observlables, etc.

The RxJava library provides APIs for other languages on the JVM besides Java: Groovy, Scala, Clojure to name a few. I guess that this is a really great new topic as of Java 8, so it deserves much more exploration than these few paragraphs.

Java2Days preparation instead of map, flatmap and reduce

I was really interested to visit another functional talk as well. It would go at another level comparing others that I’ve seen in the recent couple of years. It promised to not only go through the usual stream collectors and reduce methods, not stop at forEach and filter or at the simplest map methods. It would go into other functional terms like flatmap or more complex reduction (foldleft probably?). All in the Java world.

However, this was the only slot when I could meet Reza Rahman from Oracle to discuss what we are going to prepare for the Java2Days conference that we are running in Sofia the week after Devoxx. So I decided to watch that talk on Parleys later and had a beer with Reza during the closing ours at the Duke’s caf?. I can promise you that if everything goes fine, we’ll have a really cool agenda for our Bulgarian conference: hands on labs covering the latest Java SE 8 and Java EE 7 technologies plus tools like Forge and Arquillian, Java SE 9 hacking, talks on Java EE 8, etc. Can’t wait it to happen!

October 3rd, 2014

by Ivan St. Ivanov


This post will be covering days two and three of JavaOne. The reason for that is that yesterday morning I had to prepare for my first ever talk at JavaOne.

The batch talk

In March this year I had great time at the JavaLand conference. Along with other great people, I met there the freelancer and blog author Roberto Cortez. He told me that he is going to send a few session proposals to JavaOne and asked me whether I wanted to join him for the Java EE Batch talk. I hadn’t heard much about that topic at that time, but I agreed. Then the proposal got accepted and here I am at JavaOne now. What do you know…

Our intention for this talk was to introduce the audience to the Batch applications as a whole with topics like what is batch, why do we need it, is there batch processing in real life, etc. Then we wanted to continue with a review of the JSR 352 (Batch Applications for the Java platform) APIs. And last but not least, we wanted to live code a real world application. The topic that we chose was aggregating World of Warcraft auctions data per realms, processing and presenting some statistical data like minimum, maximum and average bid for the items at various auction houses. We (well, mostly Roberto, to be honest) even prepared that application beforehand.

The risk here was that the topic is so broad, that we were afraid that we may not have time to deliver everything as planned. We also made the things even more complex by letting the audience ask questions anytime they have it. And of course the time was not enough and we could not do the live coding session. On top of that I was surprisingly to myself extremely nervous during the session and delivered one of the poorest performances in my talk-giving history. I will not go into details on what is batch and what’s the patch processing in Java API, Roberto already uploaded the slides.

The coolest thing about this is that we (and especially myself) have our lessons learned. And the next time we give this talk we’ll be much better prepared. And this next time will be the Java2Days conference in Sofia mid of November. I would love that the order would be the other way round, but…

Java concurrency internals

A while ago I read a brilliant article by Gleb Smirnov about how setting and reading volatile variables in Java works under the hood. This year at JavaOne I had the opportunity to see the author of this post live on stage explaining all these things for the mere mortals as me.

In a nutshell, javac and then the JIT compiler make optimizations of the executed instructions. Sometimes these optimizations are more aggressive (in the server VMs for example), which means that the compiler would make more assumptions on the program flow. One of these optimizations is reordering instructions to avoid cache misses when a variable value is read or written. To make things even more complicated, in a multicore environment we have multiple level of caches on every core.

So if we want to make sure that the most recent variable value is read by all cores at any time, we declare that variable volatile. This would make the compiler emit memory barrier instructions. What a memory barrier ensures is that all the read operations above that instruction will stay there and will not move below it. The same if the read operations are below the barrier and the same with the write instructions. The other thing that the volatile keyword implies is that long and double variables that have it are written in atomic fashion.

This might sound very simplistic, but if you want to get into more details go and read the article. The slides are also available online.

After the talk I asked the speaker whether the synchronized blocks have the same effect. He told me that in principle yes. But they do also many other sorts of things so for something as simple as telling the compiler not to reorder read and write instructions, volatile is a better option.

Java EE. Again and again

I visited a lot of Java EE talks these days most of them given by distinguished luminaries from the industry.

First, Dave Blevins of Tomitribe gave one of those talks that only he is capable of on Java EE game changers. Let alone the fact that his “slides” were actually in the IDE, he again managed to present the topic without just boring us with endless rumbles on the other cool feature in the platform. Amongst the biggest game changers he mentioned the introduction of annotations (heavily inspired by XDocklet) which allowed the revolutionary change in Java EE 5 that returned the people back to the platform. Being able to package almost everything in a single (war) archive was another important turning point in the Java EE history. It removed the impression of java web applications looking like Russian dolls. The embedded EJB container functionality introduced in Java EE 6 makes it possible to very easily create a microservice out of a Java EE application. Even now you can take the TomEE server and run a Servlet, CDI, JAX-RS and what not in your public static void main class as described in this great blog post from Alex Soto and can be showcased in this simple Java class. Dave shared also his regret that Spring is not on the JCP. The talk finished with an outlook on the future: how the Java EE APIs can be adapted with respect to the big thing in Java 8: the lambdas and especially the method references.

This year I went to only one talk by Adam Bien. In that he challenged a lot of practices considered as a norm in most of the Java EE projects these days:

  • Too many layers. He made a joke that in a project he deleted most of the layers and the app was still working
  • Thinking about wrong thing first: create all the exceptions (and putting them in an exception package) or deciding on what framework to use prior to even thinking about the domain model and requirements
  • Including all sorts of external dependencies without even checking whether the JDK or Java EE has the same functionality

Among the good practices he mentioned having a simple design and implementation (there are even tools that measure that), continuous quality, short development cycles, bottom up approach on architecture, boundary-control-entity layout of the application, etc. In his opinion Arquillian is a great tool, but is overly used these days. For simple tests he proposed having a simple unit test with some mocks. It wouldn’t be Adam Bien if the talk didn’t contain live coding part.

Manfred Riem (very well known in the JSF world) gave a talk on one of the newly proposed JSRs for Java EE 8 – JSR 371, the Model View Controller specification. It provoked a lot of comments in the java world these days: why do we need another MVC solution in Java EE as we already have JSF, do we want to have Struts again, etc. So Manfred went on to explain the difference between the Component-based MVC (JSF being of this type) and action based MVC (e.g. Spring MVC). The main difference is that in the first type the request processing is responsibility of the framework, while in the other case the application developer should take care of it. At the end, it was the community that wanted this in the survey done by Oracle a few months ago. Manfred showed what his view of the future API will be. The second part of the presentation was on some extensions in Jersey (the JAX-RS reference implementation), which allow to have real action based MVC in Java EE even now. I think it is a very good opportunity for JUGs to adopt this JSR, as it is on one side a cool thing to work on and on the other side it is something very important for the industry. That is why I will propose it to our JUG once I get into Bulgaria, hoping that we have some experts in the area.

Finally I went to a talk describing various recipes for concurrency in Java EE. As of Java EE 7 application modules can get the good old java.util.concurrent.Executor service (managed by the containers of course) and do things like executing jobs in the background (also showcased in our Java EE batch presentation), scheduling tasks for executing later or repeatedly at a certain interval, etc. You can even obtain and execute plain old Java threads now inside a Java EE application using the ManagedThreadFactory.

Hacking @ JavaOne

This year I spent quite some time at the Hackergarten area in the exhibition hall. It becomes a tradition for me to make contributions to the Arquillian group of projects at JavaOne. This year I worked on something new: a Groovy DSL for running the tests. After some help from the project lead (happy birthday, Aslak!) I was managed to come with my pull request: added support for modelling deployments to different runtimes (just modelling, not real deployment).

I took part in the Adopt OpenJDK  hands on lab as well. Not too many showed up, but I helped Martijn from LCJ to introduce how you download the sources, configure and build them. I wanted to show the people how they can change a class (StrgingBuilder for example) and then run their program against JDK built with that change in. But things were running a bit slow, so there was no time for that final step.

At the end we sit together with George Gastaldi from the Forge team. He helped me with FORGE-1610. Unfortunately we did not complete it. We must work on making Furnace test run faster on Windows (Really? OK)!

So much good things to come…