Setting up the template - experianplc/experian-data-studio-connector-mulesoft GitHub Wiki

Initial Setup of Template

The Collibra <-> Pandora /Aperture connector works by using an Anypoint Studio template. The template must be configured properly in order to function.

Configuration Options

Configuration Option Template Property Required Example
Username collibraUserName Yes Guest
Password collibraPassword Yes SamplePassword
Application URL collibraUrl Yes https://your-instance.collibra.com
Test Rule Categories testRuleCategories No Test,Test2
Community Name communityName Yes MyCommunity
External System ID externalSystemId Yes Pandora 5.9.4
Pandora Import Directory pandoraAlertTriggerDirectory Yes C:\pandora\import
Pandora Alert Trigger File Name `pandoraAlertTriggerFileName Yes TEST_VOYA_trigger.xml
Rulebook Domain ID rulebookDomainId Yes xxxxxxxxx-6949-42bb-9869-xxxxxxxxxxxx
Test Rule Categories testRuleCategories Yes null
Connection String External Database Regex connectionStringExternalDatabaseRegex No .*\\(EXTERNAL_DATABASE_NAME=([^\\)]+).* Used if External Database Name is not populated in Pandora
Connection String External Schema Regex connectionStringExternalSchemaRegex No .*\\(EXTERNAL_SCHEMA_NAME=([^\\)]+).* Used if External Schema Name is not populated in Pandora
Parameters Regex parametersRegex .*EXTERNAL_COLUMN_NAME=([^;]*).* Used if External Column Name is not populated in Pandora
SFTP Toggle useSFTP No (unless SFTP is to be used ) false
SFTP Port sftpPort No (unless SFTP Toggle is enabled) 22
SFTP Host sftpHost No (unless SFTP Toggle is enabled) localhost
SFTP Username sftpUsername No (unless SFTP Toggle is enabled test
SFTP Password sftpPassword No (unless SFTP Toggle is enabled test
Gateway Host gatewayHost Yes localhost
Gateway Port gatewayPort Yes 8083
Gateway Username gatewayUsername Yes connect_user
Gateway Password gatewayPassword Yes connect_password
Database JDBC Url databaseJdbcUrl Yes jdbc:postgresql://localhost:5432/postgres?user=postgres&password=postgres
Database Driver Class Name databaseDriverClassName Yes org.postgresql.Driver
Rules SQL Query updateRulesSqlQuery Yes SELECT * from pandora_rules WHERE last_validated > #
Rules Default Value updateRulesDefaultExpression Yes 1970-01-01 00:00:00
Rules Update Frequency updateRulesUpdateFrequencyInSeconds Yes 15
Send New Rules Update Frequency sendNewRulesUpdateFrequencyInSeconds Yes 15
Send New Rules Default Value sendNewRulesDefaultExpression Yes 0
Send Alerts SQL Query sendAlertsSqlQuery Yes SELECT * from "pandora_alerts" where "last_modified_on" > #[flowVars.lastModifiedDate] ORDER BY "last_modified_on" DESC;
Send Alerts Default Value sendAlertsDefaultExpression Yes 0
Send Alerts Update Frequency sendAlertsUpdateFrequencyInSeconds Yes 15
Check Alerts SQL Query checkAlertsSqlQuery Yes SELECT "id" from "pandora_alerts" WHERE "id" = #[flowVars.collibraRule.id]
Update Alerts SQL Query updateAlertsSqlQuery Yes UPDATE pandora_alerts SET last_modified_by = #[flowVars.lastModifiedBy], last_modified_on = #[flowVars.collibraRule.lastModificationDate], name = #[flowVars.collibraRule.name] WHERE id = #[flowVars.collibraRule.id];
Inserts To Alerts SQL Query insertToAlertsSqlQuery Yes INSERT INTO pandora_alerts (id, created_on, created_by, last_modified_on, last_modified_by, name, domain_id, domain_name) VALUES (#[flowVars.collibraRule.id], #[flowVars.collibraRule.creationDate], #[flowVars.createdBy], #[flowVars.collibraRule.lastModificationDate], #[flowVars.lastModifiedBy], #[flowVars.collibraRule.name], #[flowVars.collibraRule.domain.id], #[flowVars.collibraRule.domain.name];

image

The above options are what you need to set. Configuration options are located in data-quality-sync/src/main/app/mule-app.properties. You will need to replace the default properties with values that are from your environment.


Additional configuration options are present for the connector under experian-data-quality-connector/src/main/app/mule-app.properties.

Configuration Option Template Property Required Example
Username dgc.config.user Yes Guest
Password dgc.config.password Yes SamplePassword
Application URL dgc.config.baseApplicationUrl Yes https://your-instance.collibra.com
Data Quality Dimension to Data Quality Metric Relation` dataQualityDimensionToDataQualityMetricId Yes xxxxxxxxx-6949-42bb-9869-xxxxxxxxxxxx
Rulebook Domain Name ruleBookAssetsDomainName Yes Data Quality Rules