Replace JAI - STEMLab/geotools GitHub Wiki

Replace Java Advanced Imaging

GeoTools has long been hosted on top of closed source software - up until OpenJDK Java was only available under a dual Sun Binary License / Sun Research License. The image processing library we use, Java Advanced Imaging (JAI), was originally released as a Java extension using these two licenses.

There are two problems with JAI:

  • The Binary License / Research License is not open source
  • The project has been abandoned by Oracle

In 2015 we took the first step on this road - replacing the Vecmath library used for matrix calculations. This allows GeoTools to be used, on a pure open source platform, for vector processing tasks. The next step is Replacing JAI allowing GeoTools to provide a similar freedom to those working with raster data.

Scope

Create a Tile Processing Engine:

  • New modern Java API
  • Pure Java implementation
  • Ability to stage tiles in memory and process tiles in parallel
  • Clear image processing operations, allowing installations to use native libs to accelerate processing if available

Backwards compatibility is not a requirement - it is explicitly to be avoided. This is a new project providing a functional replacement free of any Oracle encumbrance.

Approach

We would like to carefully create a replacement, with a minimal disruption, keeping in mind that applications (and organizations) world wide are dependent on this functionality.

GeoTools operations currently wrap JAI operations. We are not in position to reuse JAI Interfaces (due to license restrictions). Our approach is to create a new API, and update the GeoTools wrappers, carefully shielding downstream projects from the change of image proccecssing library.

Clarification

JAI provides both tile staging (loading data into an in-memory tile cache) and image processing (working on the individual tiles once in memory). This is a key point of difference allowing GeoTools to work with images larger than memory. Indeed image processing libraries can be used to implement "native" JAI Operations at this stage.

To keep this goal in mind we will focus on "Tile Processing" when discussing scope.

ImageIO and ImageIO-EXT

We should double check the relationship with ImageIO and ImageIO-Ext.

The ImageIO project was also abandoned by Oracle. It has been picked up in the form of https://java.net/projects/jai-imageio-core:

  • jai-imageio-core version 1.1 is available under a BSD style license

By excluding the dependencies (and CODECLib library) ImageIO can be used as a BSD project. Some glue code may be required to connect this subset up to the tile processing engine.

Planning

Interested Parties

  • Boundless
  • GeoSolutions
  • H2Gis
  • Eclipse Foundation (LocationTech and Science working groups)
  • OSGeo (GeoTools, GeoServer, ...)

Projects:

  • GeoTools, GeoServer, H2Gis, JAI-EXT, ImageIO-EXT, JAI-Tools, uDig
  • Image-IO - need to sort out approach here

Fund Raising

  • Define scope / planning

First Phase

First phase is the creation of a "TileProcessingEngine" and porting the JAI-Ext operators over to new API.

  • Define API
    • Review with interested parties
  • Initial Implementation
    • Tile Processing Engine, TileCache
  • Port JAI-EXT Operators
    • crop, sample, ...

Second Phase

Second stage is migrating GeoTools over to TileProcessingEngine and testing in affected applications.

  • Tile Processing Engine API Documentation and Tutorials
  • Update GeoTools Operation wrappers and pass test cases
  • Call out to downstream projects for testing
    • Update GeoServer and pass tests
    • Update uDig dependencies and pass tests

Background and Discussion

From GeoTools 11-17-2015 meeting:

JAI Replacement Update
----------------------

- Technical debt page: https://github.com/geotools/geotools/wiki/Replace-JAI (please help update this)
- LocationTech Steeting Committee: will worth with LocationTech to fundraise for development.
- OSGeo Board has responded positively: Will work with OSGeo to update downstream GeoTools / GeoServer and related project
- EclipseCon Europe - had a good discussion the "science working group", and the GeoTrellis lead Rob Emanuele
- The replacement should be a joint LocationTech/OSGeo project.
- Still need a good name for this project (TileProcessingEngine, OpenJI, OpenTI (for tile imaging, OpenTIP)?

From OSGeo Board 11-12-2015 meeting:

14:50 Quick GeoTools update from Jody
* We have been working to make GeoTools depend only on open source
this year we have a much harder task: https://github.com/geotools/geotools/wiki/Replace-JAI
* I wanted to bring this effort, established at the last geotools/geoserver meeting to the attention of the board.
* jeff: OSGeo is interested as it affects several projects
* helena: we need to check this kind of support with the conditions of our non profit status
* mike: as part of our 501c4, organizations could support this but they wouldn't receive a tax deduction for doing so

Andrea from GeoTools Developer mailing list:

Two pretty important aspects to take into account:
* Ability to process tiles in parallel
* The basic implementation must all be pure java, we cannot have core depend on native libs (it makes deployment
  significantly harder, in some enviroment native deps are simply banned)

Retaining the ability for native libs accelerate processing if installed and available is a definite plus though

From GeoTools 11-03-2015 meeting:

ImageProcessing
---------------
To review - Java Advanced Imaging is "end of life" and we are looking at
stratagies to replace.

JAI
- key feature is the tile based processing engine, you set up your
processing chain (kind of like functional programming) and the engine
will bring in raster data a chunk at a time allowing processing of files
larger than memory.
- no_data and region of interest support has been added as additional
operations via JAI-EXT for the processing engine.

Alternatives:

1. OpenCV (see above)
- http://opencv.org/
- BSD license, C/C++ codebase with bindings for Java
- Use OpenCL to share work with CPU/GPU
- Does it have the ability to work data larger then memory? Need to
check ...

2. ImageJ
- had a look, seems to be comparable to JAI (ie it is an image
processing engine)
- Eclipse Science Working Group is starting in on this one (and may have
room for collaboration)
- Found the engine here: http://imagej.net/ImgLib2
- Could not see any support for working with raster data larger than
memory. Has region of interest (ROI) support, not sure about no_data.
- At least one nD dimensional data format.
- has high level operators (feaure extractions etc...)

3. JAI Replacement
- Need a project name for JAI replacement, then start fund raising
- Interest from Boundless, GeoSolutions, LocationTech, H2Gis too
- would like work though OSGeo and LocationTech
- mostly focused on the technical requirements
- name? JAI-NG, GTAI, GeoAI, JSCIP (Java Scalable Image Processing)
JAI-15, OpenJImaging, OpenJI? OpenJAI? JIOpen? GeoNee
- No use of JAI interfaces ... because Oracle
- "Java" is a trademark
- This is a *lot* of work, both in the project, and in migrating
GeoTools and GeoServer.
- Q: How well does GeoTools protect downstream projects? Pretty good
(except for direct rendering path and some WPS). Remaining code is
hidden behind "image worker".
- Note: RenderedImage is a Java class, not PlanarImage which is the JAI
subclass.

4. GeoTrellis
- requires data to be broken into tiles before use, making use of sparc
to handle concurrent processing.
- Not sure about no_data and ROI support.

OpenCV Discussion
-----------------

Jeff Bower has a proposal sent out, Andrea has had a quick glance at the
proposal.

Discussion:
- possible replacement for native JAI replacment? Ie implement JAI
Operators we care about in terms of OpenCV (Scale, Translate, General
Affine, Warp, Crop, Lookup) and then give the a higher priority so they
are used if OpenCV plugin they are used.
- discussion focused on WMS optimization above, Jeff's customers may be
using WPS for "image analyitics" - so they may be using WPS?!
- standard operations may be worse because of latency in calling JAI
operations via an accelerator.
- concern about moving data on and off GPU? OpenCV can be configured to
stay on CPU
- note: GIS procesing is not normal image processing - requires region
of interest (similar to a mask), and 'no data' (some values should not
be processed).
- OpenCV has some support for the above :)