The [ResourceStore Design] outlines an API for stream based access to files in the data directory. Transitioning the codebase to use this API will require a bit of planning.

{toc}

GeoServerResourceLoader

• Update internals of GeoServerResourceLoader
• Add Resource access methods to GeoServerResourceLoader
• Remove use of GeoServerResourceLoader search locations
  • WEB_INF/web.xml file access

GeoServerDataDirectory and GeoServerSecurityManager

• Update internals of GeoServerDataDirectory
  • Already uses GeoServerResourceLoader
  • Update methods to use Resource internally
• Add Resource access methods to GeoServerDataDirectory
• Update internals of GeoServerSecurityManager to use Resource
• Add Resource access methods to GeoServerSecurityManager

File interaction:

• Research

  • Review use org.geoserver.platform.FileWatcher
  • Review use org.geoserver.security.file.FileWatcher
  • Review GWC LockProvider

• Port LockProvider and integrate with ResourceStore

  • Bridge from GeoServer LockProvider to GWC LockProvider

• Port FileWatcher implementations to use events

  • Update internals of platform FileWatcher so subclasses continue to function unchanged
  • Update internals of security FileWatcher so subclasses continue to function unchanged

• Update security to use Resource and Resource events directly rather than FileWatcher

Update GeoServer codebase module by module (see also technical debt)

• Direct use of ResourceStore/Resource and Resource.out () for “stream access”
• Replace GeoServerDataDirectory.findOrCreateDir() by ResourceStore.get()
• Use of Resource.file () for “file access” (i.e. to pass file or URL reference to external library)
• Replace GeoServerResourceLoader “directory access” with use of Resource
• Modules:
  • main
  • geowebcache
  • wms
  • wfs
  • web
  • kml
  • rest & restconfig
  • security
  • extensions
    • importer
    • wps
    • csw
    • xslt
    • monitor
    • css
    • control-flow
    • printing
    • ogr
  • community
    • geofence, geofence-server
    • script
    • netcdf
    • validation
    • rest-upload
    • wms-eo
    • wps-download
    • hz-cluster
    • jms-cluster
    • ftp

• Update internals of GeoServerResourceLoader
  • This “migrates” the codebase to use ResourceStore for lookup
• Introduce methods for Resource access
• Remove use of GeoServerResourceLoader search locations
  • GeoServerExtensions.file ( path ) provided for access to servletContext
  • Fix GeoServerJ2eeRoleService access of WEB*INF/web.xml
• Transition checklist
  • org.geoserver
  • org.geoserver.catalog
  • org.geoserver.catalog.impl
  • org.geoserver.catalog.rest
  • org.geoserver.catalog.util
  • org.geoserver.config
  • org.geoserver.config.util
  • org.geoserver.csw
  • org.geoserver.csw.store.internal
  • org.geoserver.data.test
  • org.geoserver.ftp
  • org.geoserver.gwc.config
  • org.geoserver.gwc.layer
  • org.geoserver.h2
  • org.geoserver.inspire
  • org.geoserver.jdbcconfig
  • org.geoserver.jdbcconfig.internal
  • org.geoserver.logging
  • org.geoserver.monitor
  • org.geoserver.monitor.auditlog
  • org.geoserver.ows
  • org.geoserver.security
  • org.geoserver.security.impl
  • org.geoserver.security.ldap
  • org.geoserver.spatialite
  • org.geoserver.template
  • org.geoserver.test
  • org.geoserver.wcs
  • org.geoserver.web
  • org.geoserver.web.admin
  • org.geoserver.web.util
  • org.geoserver.web.wicket
  • org.geoserver.wfs
  • org.geoserver.wfs.response
  • org.geoserver.wfs.web
  • org.geoserver.wfs.xml
  • org.geoserver.wfs.xml.v1*1_0
  • org.geoserver.wms
  • org.geoserver.wms.eo
  • org.geoserver.wps
  • org.vfny.geoserver.global
  • org.vfny.geoserver.wms.responses.map.htmlimagemap

GeoServerJ2eeRoleService makes use of the following:

File webXML = loader.find( "web.xml" );

This is the only example of using search locations to find content outside of the data directory.

Functionality moved to GeoServerExtensions in order to use servlet context:

File webXML = GeoServerExtensions.file( "WEB_INF/web.xml" );

• Update internals of GeoServerDataDirectory
• Add Resource access methods to GeoServerDataDirectory
• Transition to Resource use where possible
• Checklist (106 references)
  • org.geoserver.catalog
  • org.geoserver.community.css.web
  • org.geoserver.config
  • org.geoserver.csw.store.simple
  • org.geoserver.data
  • org.geoserver.ftp
  • org.geoserver.gwc
  • org.geoserver.kml.icons
  • org.geoserver.monitor.hib
  • org.geoserver.script
  • org.geoserver.script
  • org.geoserver.security
  • org.geoserver.security.filter
  • org.geoserver.security.impl
  • org.geoserver.security.ldap
  • org.geoserver.template
  • org.geoserver.test
  • org.geoserver.web.admin
  • org.geoserver.wfs.xslt.config
  • org.geoserver.wms
  • org.geoserver.wps
  • org.vfny.geoserver.global

The following regressions were found, often due to code taking advantageous of gaps in the implementation of GeoServerDataDirectory.

GWCIntegrationTest:

File h2DefaultStore = dd.findDataFile("gwc/diskquota_page_store_h2");

GetCoverageTest.onSetUp has the following:

File data = getDataDirectory().findDataDir("watertemp");

In these cases I am fixing the test case, but will be more concerned if production code is effected by the dataRoot (false) implementation issues as shown below.

{warning} The code is asking to “data/watertemp” but manages to find “watertemp” due to a strange turn of events:

• findDataDir (“watertemp”) is implemented as:

  return dataDir( false, "watertemp" );

• Where dataDir ends up with:

  return resourceLoader.find( dataRoot(false), "watertemp");

• Which is where things go wrong, dataRoot (false) fails to find the data directory, resulting in null

  return resourceLoader.find( null, "watertemp");

• Allowing resourceLoader.find to locate “watertemp” (rather than “data/watertemp”). {warning}

These methods determine a file location based on a StyleInfo and a filename. A few test cases such as DynamicDimensionsTest made use of a relative path when unpacking class path resources:

testData.addStyle(styleName, "../temperature.sld", getClass(), getCatalog());

This resulted in temperature.sld being unpacked outside the styles directory:

styles/
temperature.sld

And a relative path coming out of StyleInfo:

String path = Paths.path("styles",style.getFilename());

• Changed testData add style to ignore relative paths when determining the target filename.
• Added Paths.convert ( path, filename ) to allow for relative filenames (as long as they stay inside the data directory)

Action: refactor this one out of existence!

It is mostly used to access the data directory:

File dataDir = GeoserverDataDirectory.getGeoserverDataDirectory();
File storeDir = new File(dataDir, "data/wcs/mosaicfordelete");        

It is also used to resolve file references:

if (path.startsWith("file:")) {
   File fixedPath = GeoserverDataDirectory.findDataFile(path);
   entry.setValue(DataUtilities.fileToURL(fixedPath).toExternalForm());
}

• Replace each reference to GeoserverDataDirectory with GeoServerResoruceLoader

• Replace GeoserverDataDirectory.getResourceLoader () with GeoServerExtensions.bean (GeoServerResource)

• Feedback: Make GeoServerExtensions easier to use

  • Introduced GeoServerExtensionsHelper

• Update internals of GeoServerSecurityManager to use Resource

• Add Resource access methods to GeoServerSecurityManager

• Transition GeoServerSecurityManager direct file access with Resource use where possible

• Checklist (106 references)

  • SecurityManager.getSecurityRoot () - 18 items
  • getRoleRoot () - 14 items
  • getPasswordPolicyRoot - 3 items
  • getUserGroupRoot - 13 items
  • getAuthRoot - 3 items
  • getFilterRoot - 5 items
  • getRoot - 3 items
  • getMasterPasswordProviderRoot - 5 items

• Platform FileWatcher
• Security FileWatcher
• ⚠️ GeoServerTemplateLoader - looks to be complicated