Adam Bien's Weblog

Friday Feb 26, 2010

Jigsaw / JDK 1.7 will be the solution for 80% of the modularization challenges

Jigsaw will come with JDK 1.7 and is now part of the openjdk project and so opensource. Other JDK implementations could simply reuse it.It will become interesting, because:

  1. It will be shipped with every Oracle / Sun JDK 1.7 (at least it was the plan)
  2. Jigsaw will partition JDK 1.7 and will be loaded before most of the rt.jar code. So is already there - no reason to introduce another framework
  3. Its pragmatic: you can split packages across modules. Package splitting isn't a best practice, but makes the incremental modularization of legacy projects a lot easier
  4. Modules are loaded statically - no reloading. This is good enough for the most projects. Its funny - but in most NetBeans RCP / Eclipse RCP projects we had to deactivate the ability to load modules dynamically in production :-). Server operators don't like dynamic behavior either.
  5. It seems like there will be only one classloader, which will load all modules. This will eliminate majority of NoClassDefFoundErrors.
  6. javac will be jigsaw aware - this is a major advantage. Module dependencies should be checked as early as possible - it fits well with the nature of Java.
  7. Jigsaw should increase the classloading performance and significantly reduce the size of the jars and so the deployment


Special Event: Java 8 with Java EE 7: "More Power with Less Code", 13th October, 2014

A book about rethinking Java EE Patterns

Comments:

Hi,

It has been a while since I saw the name "Oracle" on one of your postings ;)

I am looking forward to JDK7, too. How do you position/rate the new modularity features against OSGi?

Thanks,
Markus

Posted by Markus on February 26, 2010 at 12:25 PM CET #

@Markus,

1. there was a lot of oracle mentions on my blog: http://www.google.de/search?client=safari&rls=en&q=site:adam-bien.com+oracle&ie=UTF-8&oe=UTF-8&redir_esc=&ei=bpeHS9HmHuOmsQblu-GwDw

2. Although useful, OSGi is a technology from 1945 (not that bad, say from 2000), without annotations and lots of plumbing. Its just impossible to use OSGI without dedicated tools. Very comparable with EJB 2 :-). I'm just looking forward to something new.

3. Even more interesting - the first OSGi book was written by ...Sun :-): http://www.adam-bien.com/roller/abien/entry/osgi_was_supported_by_sun

thanks for the comment,

adam

Posted by Adam Bien on February 26, 2010 at 12:41 PM CET #

Why settle for an 80% solution in about a year (maybe), when we have a ~95% solution now in OSGi (and working hard on that remaining 5%)?

One classloader will indeed eliminate NoClassDefFoundErrors... by allowing modules to cheat. You will be able to load classes that you never imported, just because they happen to be on the classpath. This will hardly encourage developers to spend time getting their dependencies correct. In OSGi, you only get NoClassDefFoundErrors when you try to load something that you didn't import... the best solution is to fix your imports, not fudge the module system.

You're right that package splitting is not best practice but sometimes necessary for modularisation of legacy code. That's why OSGi supports split packages, but strongly discourages them.

Dynamic vs static... well, OSGi supports static loading too. Nobody forces you to reload at runtime.

Regarding your comments in response to Markus... OSGi started in 1998 (JSR8), so it's a little younger than Java itself. "Without annotations" is incorrect, take a look at iPOJO or Bnd's annotations for Declarative Services. "Without lots of plumbing"... err, define plumbing??

Impossible to use OSGi without dedicated tools? True, an IDE or a tool like Bnd (600k JAR file including sources) is very useful. You think you will be able to manage Jigsaw modules without any tools besides javac? Is that even how you develop Java now, with vi and javac? Come off it :-)

Posted by Neil Bartlett on February 26, 2010 at 02:14 PM CET #

@Neil,

"You think you will be able to manage Jigsaw modules without any tools besides javac? Is that even how you develop Java now, with vi and javac? Come off it :-)"

Because #jigsaw will come with JDK, I would expect support of any IDE (community edition). Eclipse could be a bit late again - as it was the case with the support of JDK 1.5 :-). I wouldn't be surprise if some tools would be even shipped with the JDK.

I started with OSGi around 2000 - but at that time it wasn't even possible to download the documentation without extensive registration - no idea what was the situation in 1998 :-).

"Why settle for an 80%" - if #jigsaw will be good enough for my purposes and will be directly delivered with JDK - I would prefer it over any other solution. External dependencies and framework just make your project bloated and over-complicated. This fact has nothing to do with OSGi.

"One classloader will indeed eliminate NoClassDefFoundErrors... by allowing modules to cheat"

Cheating - yes. It is still required. Thats the reason, why the majority of Eclipse RCP projects needs the "Buddy Classloading" policy: http://wiki.eclipse.org/index.php/Context_Class_Loader_Enhancements

Thanks - it was one of the fastest responses to my posts :-),

adam

Posted by Adam Bien on February 26, 2010 at 02:39 PM CET #

I don't know where you got the idea that "the majority of Eclipse RCP projects" need buddy loading. It is only used in a few edge-cases. And in fact there are better standards-based solutions for those edge cases than using Eclipse-specific extensions.

There's a saying in the legal profession: "hard cases make bad law". It's true that in OSGi we sometimes have difficulties with awkward libraries that want to use reflection to load any class from any module. These are our hard cases, and we do have ways of allowing them to "cheat", but still in a very controlled way. The alternative seems to be to give up on modularity completely, and simply allow any module to load non-imported classes, just because a very small number of libraries need to do it. The hard cases shouldn't make the law for everybody.

Incidentally I'm sure Eclipse will support Jigsaw if users actually ask for it (and even more likely, if somebody steps up and builds support for it). The same with NetBeans and IDEA... I don't think any of those IDEs will waste time building stuff that nobody uses.

Posted by Neil Bartlett on February 26, 2010 at 03:26 PM CET #

@Neil,

you are right again. The issues were mainly caused by incompatible opensource, but very popular, frameworks. We had trouble with Hibernate, or even simple things like Log4J in the past.

thanks!,

adam

Posted by Adam Bien on February 26, 2010 at 04:40 PM CET #

Any hint about the _installed_ module format and loading optimizations? Right now Sun JDK has the CDS, which allows limited optimization (ahead-of-time symbol resolution, linking, quickening, verification) and sharing (~50% of core bytecode; no sharing for the other 50%, or for non-core packages, or for JIT code). I'd expect Jigsaw to come together with a much better replacement, perhaps allowing full AOT compilation / JIT caching, at least for the Client VM (where the benefits of loading time and memory saving are clearly more important than some dynamic optimizations that can't fit in a stable/shared native image).

Posted by Osvaldo Pinali Doederlein on February 26, 2010 at 07:04 PM CET #

you can find the proposed module file format here:
http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/

discussion is on this list:
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2010-January/thread.html#470

Posted by michael bien on February 26, 2010 at 08:18 PM CET #

I'm watching the list; the proposed module format is a distribution format only. Nobody expects that those files (with Pack200 and all) will be just copied as-is to the filesystem. The module installation process will certainly extract the classes and reorganize them into some other "installed format" that's more efficient for runtime usage. This installed format can be something as simple as current JAR files without compression. But this would be a sever missed opportunity. The .NET platform has significantly superior loading-time and resource usage (code sharing) only because it uses good module formats, not only for distribution (Assemblies, EXEs) but also for installation. (The installed format is just PE/COFF.) The IBM JDK already supports a JIT caching facility, more similar to CDS (one huge file / not modularized) and it still allows dynamic optimizations (will just store several versions of compiled methods, and the dependency info so each process can use the correct variant).

Posted by Osvaldo Pinali Doederlein on February 26, 2010 at 09:54 PM CET #

all this is nice and all but ... if I need to individually load or unload a class, how does jigsaw compare against OSGi?

Posted by 79.9.156.182 on February 26, 2010 at 11:29 PM CET #

All these arguments remind me of the discussion around the introduction of Java Logging vs using Log4J.
In the end, Log4J is still better, but now the world is split and we get to marvel at glue libraries like Commons Logging and SLF4J.

Posted by Dimitar Dimitrov on February 27, 2010 at 04:43 AM CET #

@Neil: "In OSGi, you only get NoClassDefFoundErrors when you try to load something that you didn't import... the best solution is to fix your imports, not fudge the module system."

There are so many assumptions in this statement that it would take an hour's talk just to unravel them. Package v. module v. bundle dependencies; bundle/module:classloader mapping; the role of accessibility in properly hiding classes.

However, the main point is clear. You think it is acceptable to make potentially significant changes in the codebase ("fix imports" falsely implies simplicity) so as to satisfy the module system. We do not.

Also, I never said that OSGi does not support split packages. It supports them at runtime. This is quite undesirable. The JDK is interested in /safe/ split packages, which was the term used in the talk and refers to the ability to statically split packages in deployment units then "join" them at runtime. I don't recall if the implementation does this now, but Jigsaw could happily disallow split packages at runtime.

Posted by 76.191.146.64 on February 27, 2010 at 05:32 AM CET #

The comment above is mine. Didn't get a warning about missing name because this blog, unusually, anonymous comments.

Posted by Alex Buckley on February 27, 2010 at 05:33 AM CET #

@Dimitar,

although Log4j is more powerful, than java.util.logging, if it is sufficient, I just use java.util.logging and get rid off an additional dependency. Everything is a trade-off between more power and an additional dependency which has to be deployed and maintained,

thanks!,

adam

Posted by Adam Bien on February 27, 2010 at 10:52 AM CET #

@Alex,

on my blog even anonymous comments are welcome :-) -> thanks for the resolution of your ip-address :-).

adam

Posted by Adam Bien on February 27, 2010 at 10:59 AM CET #

@Osvaldo JRockIt supports code caches since 1.4.2 (http://download.oracle.com/docs/cd/E13188_01/jrockit/docs142/userguide/codecach.html) IMO its rather an optimization as something which should be in a spec of a file format.

E.g you could probably even use the distribution module format for deploying your app with an initial code cache. It looks flexible enough to allow such things. But you should ask on the mailinglist to have a clear answer.

Posted by michael bien on February 27, 2010 at 03:28 PM CET #

Thank you for interesting information. I was searching this information for a long time.

Posted by Elena on February 28, 2010 at 09:04 PM CET #

"Modules are loaded statically - no reloading"

Why does Java hate zero-turnaround-productivity? Why oh why cant we have instant evaluation in a "-dev"-cli-enabled modus?

Posted by Sakuraba on March 01, 2010 at 11:39 AM CET #

I agree that the code caching is just an optimization, and implementation detail that needs no standards/specs. I guess if Sun is working on this as part of the Jigsaw effort, they won't likely discuss it publicly, or at least they won't have a JSR for it. There's no need for that; in fact, I'm happy enough if this optimization is implemented (and for bonus points, if it is contributed to OpenJDK when ready).

I disagree though that the distributable module spec is any good as a deployed format. After you remove compression, deal with the module dependencies etc., it boils down to the old and tired JAR files. Even the current Sun CDS is massively superior to that.

So, let's just wait and hope for the best.

Posted by Osvaldo Pinali Doederlein on March 01, 2010 at 09:00 PM CET #

A casual observer's lament:

Does Sun have some kind of self-loathing that they feel that the Java community must always be riven by endless incompatibility and tribal wars against its own community?

It seems that time after time, Sun looks at the community, sees it has a need for something that was missing from Java, and then sees that it has even developed, deployed, and unified around a community solution. And then Sun says "We want to do something very similar but in an incompatible way, so people will have headaches working with different modules from different module systems, so that CTOs will look at Java and see confusion, strife, and enmity, and they'll respond 'Steer clear of that nightmare; use .NET -- it just works'."

I mean, come off it guys. We're all trying to do more with Java, but all this fractious fighting has left the community bleeding, and the community and sustainability of a technology is usually more important than its internal technical merits. Keep going this way and Java will die the death of a thousand squabbles.

And it's not just about javac; it's about how the technology works with everything else that's out there (Maven, iPOJO, even Eclipse and NetBeans...)

Posted by 203.143.164.226 on March 02, 2010 at 05:05 AM CET #

I haven't taken a look at the implementation details yet, but some of the points brought up by this post ring alarm bells.

Please put a lot of pressure on the community to evaluate Jigsaw before freezing its specification. I love the idea behind it, but I am afraid it will become another Generics debacle.

Please elaborate on the problems with dynamic module loading (point #4) and how Jigsaw will "significantly reduce the size of the jars" (point #7)

Posted by Gili on March 02, 2010 at 07:02 AM CET #

Adam Bien: "although Log4j is more powerful, than java.util.logging, if it is sufficient, I just use java.util.logging and get rid off an additional dependency. Everything is a trade-off between more power and an additional dependency which has to be deployed and maintained,"

The "additional dependency" argument is pretty weird in this context for two reasons:

1. First of all, the point of Dimitar was: when java.util.logging was designed and added, Log4J already existed. For some reason, java.util.logging was designed as a lesser solution. If, at the time, instead of going for an "80% solution", you had gone for at least feature parity with log4j, or a robust way to extend in a way to achieve this afterwards, people would actually have embraced this.

Similarly, The "80%" strategy is bound to spell doom for Jigsaw:

* if I am on Java 6 and lower, I cannot use it (OSGi will work on any existing Java version).
* If I am in a scenario why dynamic reloading _at some level_ in the infrastructure is required, this will not help me (OSGi will).
* If I want to support other Java vendors other than Oracle, I cannot use it. (OSGi is a broadly accepted specification)

From an end-user perspective, this can feel a bit limiting. From a framework builder perspective, if you want a module system, you'd always pick OSGi for at least the next half decade. And most end-users these days mostly use some framework technology, not just vanilla JDK.

2. Adding modularity as a core concept to Java will actually make Log4j have quite a bit more level playing field with Java.util.logging. The main downside of "have to add an additonal jar" is because adding jars is hard. A module system will make this easier. I recon java.util.logging will be yet another isolated module in this system as well.

So the "use the stuff already in the JDK to avoid jar hell" argument is no longer valid when there is no more jar hell, and the JDK is evolving in fragmented modules as well.

Posted by Peter Hendriks on March 02, 2010 at 10:12 PM CET #

I know it's not this simple, but in theory, (with a different VM module) could you use the modularity to create a JavaME profile?

Posted by Eric on March 03, 2010 at 03:26 PM CET #

"Keep It Simple Stupid" is the most important principle for software development for me. Do I really need dynamic loading of modules on my mobile? Or on the other end, do I really want dynamic loading on my mission critical 24x7 server? What are the issues you solve with it? What does it cost you? Can you easily train an apprentice to understand it? I prefer simple things even if they lack the last 20%.

Posted by Haug Bürger on March 06, 2010 at 07:04 PM CET #

Good post, that lot of useful information.

You can find here too:
http://qnatech.wordpress.com

Posted by qnaguru on March 14, 2010 at 09:15 PM CET #

@Haug,

You comment is exactly my opinion - thanks!,

adam

Posted by Adam Bien on March 14, 2010 at 11:48 PM CET #

Thanks for this post Adam. I think OSGi can be useful in some projects, but, most of the time, it's an overkill solution for simple problems. Jigsaw project will help us to resolve most of our common modularity issues. KISS!

Posted by Anthony MÜLLER on June 18, 2010 at 03:23 PM CEST #

Incidentally I'm sure Eclipse will support Jigsaw if users actually ask for it (and even more likely, if somebody steps up and builds support for it). The same with NetBeans and IDEA... I don't think any of those IDEs will waste time building stuff that nobody uses.Good Luck

Posted by isis solar on September 06, 2011 at 08:34 PM CEST #

@Isis,

Jigsaw will be a part of JDK 8. Are you saying, that Eclipse will not support JDK 8, because no one will request that?

thanks,

adam

Posted by Adam Bien on September 07, 2011 at 04:07 PM CEST #

Post a Comment:
  • HTML Syntax: NOT allowed
realworldpatterns.com
...the last 150 posts
...the last 10 comments
Links
License