loading data - marklogic/ml-gradle GitHub Wiki
ml-gradle includes a simple mechanism for loading data as part of a deployment. For a typical project, you may have the need for the far more powerful capabilities in the MarkLogic Flux tool for loading data. But some deployment use cases may simply require loading a set of version-controlled files from a directory with specific collections and permissions.
This feature has the following default behavior:
-
src/main/ml-datais the default path for finding files to load. - Any file in any data path will be loaded with a URI relative to the data path that it belongs to; e.g. the file
src/main/ml-data/my/data/test.jsonwill be loaded with a URI of/my/data/test.json. - Collections and permissions can be specified via files in each directory.
- The files will be loaded via the connection associated with the value of
mlRestPort. If a database name is provided via the below properties, the value ofmlAppServicesPortwill be used for the connection instead.
The following properties are available for configuring this feature (all, unless otherwise noted, were introduced in 3.13.0):
| Property | Description |
|---|---|
| mlDataBatchSize | The number of documents to include in each call to MarkLogic. Defaults to 100. |
| mlDataCollections | Comma-delimited list of collection names assigned to each document. No default value. |
| mlDataDatabaseName | Database to load documents into; if set, then ml-app-deployer will connect via the App-Services app server to load the documents. No default value. |
| mlDataPaths | Comma-delimited list of data paths. Defaults to src/main/ml-data. |
| mlDataLoadingEnabled | Whether this feature is enabled. Defaults to true. |
| mlDataLogUris | Whether the URI of every document inserted should be logged. Defaults to true. |
| mlDataPermissions | Comma-delimited list of permissions (role1,capability1,role2,capability2,etc) assigned to each document. No default value. |
| mlDataReplaceTokens | Whether tokens should be replaced in each document, where tokens are obtained from the custom tokens map on the mlAppConfig object. Defaults to true. |
As of 4.6.0, the properties mlCascadeCollections and mlCascadePermissions can be set to true so that the settings in collections.properties and permissions.properties will be applied to child directories, unless a child directory has its own files.