Version 2.9.0 of ml-gradle includes support for migrating a Roxy project into an ml-gradle project. More support will be provided in future release, but what's supported in 2.9.0 is described below.

Overview

This guide assumes all pre-requisites for this plugin have been installed. Please check the Quick Start for these requirements.

Migrating a Roxy project into an ml-gradle project involves three tasks:

1. Migrating the source files, properties, and build steps from the Roxy project files
2. Exporting your Roxy project resources from MarkLogic, which requires that the Roxy project already be deployed to MarkLogic.
3. Review and clean up.

These tasks are described in more detail below.

1. Migrating the source files

On a new directory, we create a build.gradle file that contains atleast the following lines:

plugins {
  id "net.saliman.properties" version "1.4.6"
  id "com.marklogic.ml-gradle" version "3.1.0" 
}

Now we run the code migration task

gradle mlRoxyMigrateProject -ProxyProjectPath=/path/to/roxy/project

Alternatively, we can run smaller tasks to allow us to more carefully and examine the output of each task before invoking another one

Task	Description
mlRoxyMigrateBuildSteps	Migrate build steps from deploy/app_specific.rb into custom Gradle tasks. Use -ProxyProjectPath to define the location of your Roxy project, and -PappSpecificPath to define a path other than deploy/app_specific.rb
mlRoxyMigrateFiles	Migrate Roxy source files into this Gradle project. Use -ProxyProjectPath to define the location of your Roxy project.
mlRoxyMigrateProperties	Migrate Roxy properties into the gradle.properties file in this project. Use -ProxyProjectPath to define the location of your Roxy project.

gradle mlRoxyMigrateFiles -ProxyProjectPath=/path/to/roxy/project
gradle mlRoxyMigrateBuildSteps -ProxyProjectPath=/path/to/roxy/project
gradle mlRoxyMigrateProperties -ProxyProjectPath=/path/to/roxy/project

There are no dependencies between these tasks, so you can run them in any order.

The sections below have more information on each task.

Migrated source files

The "mlRoxyMigrateFiles" task copies files from the Roxy "src" and "rest-api" directories into the correct locations in the ml-gradle project. All MarkLogic modules in an ml-gradle project are under "src/main/ml-modules" - the mapping of Roxy files to directories under that directory is:

• src -> /root
• rest-api/config -> /options
• rest-api/ext -> /services
• rest-api/transforms -> transforms

Version 2.9.0 of ml-gradle also includes support for deleting test modules. See the property reference for more information - specifically, the mlDeleteTestModules and mlDeleteTestModulesPattern properties. Assuming you're using the Gradle properties plugin, you can simply e.g. set mlDeleteTestModules=true in gradle-(environmentName).properties.

Migrated properties

The "mlRoxyMigrateProperties" task copies Roxy properties from deploy/default.properties and deploy/build.properties into the standard Gradle properties file, which is gradle.properties.

This task will map known Roxy properties into their associated Gradle properties. Any Roxy property that doesn't map to a Gradle property is copied over as-is. As of version 2.9.0, the following Roxy properties are mapped:

• app-name -> mlAppName
• app-port -> mlRestPort
• user -> mlUsername
• password -> mlPassword
• group -> mlGroupName
• content-forests-per-host -> mlContentForestsPerHost

Please see II.B gradle.properties reference for more information on Gradle properties.

NOTE - ml-gradle will create a REST API server by default unless the property "mlNoRestServer" is set to "true". Version 3.0.0 will automatically add this property to gradle.properties. But with 2.9.0, you'll most likely want to manually add this property, as your exported application (see below) will likely have all the app servers it wants already.

It is also important to note that this task will rely on build.properties alone and ignore any environment specific properties file, e.g. local.properties.

Migrated build steps

The "mlRoxyMigrateBuildSteps" task attempts to migrate custom Roxy build steps in the ./deploy/app_specific.rb file into custom Gradle tasks. This support is limited - it relies on parsing text, and it only works for build steps that executes queries. For each such build step, this task will add to the build.gradle file a new task that makes a call to /v1/eval with the same XQuery as the associated Roxy build step.

2. Exporting the current Roxy application

Instead of trying to convert the contents of the Roxy ml-config.xml into Manage API files, the recommended approach is to first deploy your Roxy application, and then use the ml-gradle export task for exporting the resources in your Roxy application as Manage API files.

The Wiki page for exporting resources provides instructions on how to select resources to export.

Resources are by default exported to the location that ml-gradle expects to find them - src/main/ml-config. A few caveats about this:

1. You can use a prefix or regex to specify the resources to export, but you may need to use the properties file approach to get everything. It's common for applications to have resources like privileges that don't start with a common prefix, and thus you may need to specify them explicitly in a properties file.
2. The export feature in ml-gradle doesn't yet support exporting every resource type. You can always manually export tasks via the Manage API on port 8002 of your MarkLogic server.
3. The export feature prints out helpful messages about the resources it has exported. Pay attention to these, as you may need to make some manual edits. For example, MarkLogic does not allow for a user password to be exported for obvious reasons, so a default password is used that you'll most likely want to change.

3. Review and clean up

The first step is bound to create too many properties within the gradle.properties file. It is also possible to have duplicate entries for the same property.

The second step may include extra files depending on the regex or filter that has been used. These generated files, e.g. database, server, etc, also do not make use of tokens and may end up conflicting with the generated gradle.properties file, e.g. ports. This typically happens when your ml-roxy project has values in local.properties that override values set in build.properties. It is recommended that such settings/properties be modified to use tokens instead. See II.C gradle.properties, tokens, files... for more info.

---

Test deployment of the migrated project

To fully test the viability of the migrated project, we recommend targetting a clean install of MarkLogic. Or at least one that does not have the ml-roxy project pre-installed. Once this is available, we try deploying the application (the "-i" flag tells Gradle to log at the info level, which is normally helpful when using ml-gradle, as you'll see every call that's made to the Manage API):

gradle -i mlDeploy mlLoadModules

Loading data

ml-gradle doesn't have a pre-defined task for loading data from a specific directory like Roxy does - e.g. the "data" directory and "ml local deploy content". Instead, you can configure as many data-loading tasks as you want using MLCP (Content Pump). Please see this guide for more information.

The reason why ml-gradle doesn't have a pre-defined task for loading data is so that ml-gradle doesn't depend on a specific version of MLCP. Instead, you can use whatever version of MLCP you need via Gradle's normal dependency management.

Troubleshooting

Here are some of the problems you may run into that require manual changes or exporting additional resources:

1. Something wasn't exported from your Roxy application that's required for deployment - i.e. a role requires a privilege, and the role was exported but not the privilege.
2. Two or more roles have circular dependencies. This is difficult to resolve with the Manage API, as each role is created in a separate transaction. You'll need to define two files for one of the roles - one that doesn't depend on any roles, and one that includes the role dependencies.
3. Some build steps weren't migrated that are needed. You'll need to manually recreate these as Gradle tasks - file an issue if you need help with this.

Current limitations

SQL views and schemas are currently not included in the exported resources. These would to be manually ported to the new project. Refer to this wiki for more info on how these schemas should be structured in an ml-gradle project.