Setting up the template - experianplc/experian-data-studio-connector-mulesoft GitHub Wiki
Initial Setup of Template
The Collibra <-> Pandora /Aperture connector works by using an Anypoint Studio template. The template must be configured properly in order to function.
Configuration Options
Configuration Option | Template Property | Required | Example |
---|---|---|---|
Username | collibraUserName |
Yes | Guest |
Password | collibraPassword |
Yes | SamplePassword |
Application URL | collibraUrl |
Yes | https://your-instance.collibra.com |
Test Rule Categories | testRuleCategories |
No | Test,Test2 |
Community Name | communityName |
Yes | MyCommunity |
External System ID | externalSystemId |
Yes | Pandora 5.9.4 |
Pandora Import Directory | pandoraAlertTriggerDirectory |
Yes | C:\pandora\import |
Pandora Alert Trigger File Name | `pandoraAlertTriggerFileName | Yes | TEST_VOYA_trigger.xml |
Rulebook Domain ID | rulebookDomainId |
Yes | xxxxxxxxx-6949-42bb-9869-xxxxxxxxxxxx |
Test Rule Categories | testRuleCategories |
Yes | null |
Connection String External Database Regex | connectionStringExternalDatabaseRegex |
No | .*\\(EXTERNAL_DATABASE_NAME=([^\\)]+).* Used if External Database Name is not populated in Pandora |
Connection String External Schema Regex | connectionStringExternalSchemaRegex |
No | .*\\(EXTERNAL_SCHEMA_NAME=([^\\)]+).* Used if External Schema Name is not populated in Pandora |
Parameters Regex | parametersRegex |
.*EXTERNAL_COLUMN_NAME=([^;]*).* Used if External Column Name is not populated in Pandora |
|
SFTP Toggle | useSFTP |
No (unless SFTP is to be used ) | false |
SFTP Port | sftpPort |
No (unless SFTP Toggle is enabled) | 22 |
SFTP Host | sftpHost |
No (unless SFTP Toggle is enabled) | localhost |
SFTP Username | sftpUsername |
No (unless SFTP Toggle is enabled | test |
SFTP Password | sftpPassword |
No (unless SFTP Toggle is enabled | test |
Gateway Host | gatewayHost |
Yes | localhost |
Gateway Port | gatewayPort |
Yes | 8083 |
Gateway Username | gatewayUsername |
Yes | connect_user |
Gateway Password | gatewayPassword |
Yes | connect_password |
Database JDBC Url | databaseJdbcUrl |
Yes | jdbc:postgresql://localhost:5432/postgres?user=postgres&password=postgres |
Database Driver Class Name | databaseDriverClassName |
Yes | org.postgresql.Driver |
Rules SQL Query | updateRulesSqlQuery |
Yes | SELECT * from pandora_rules WHERE last_validated > # |
Rules Default Value | updateRulesDefaultExpression |
Yes | 1970-01-01 00:00:00 |
Rules Update Frequency | updateRulesUpdateFrequencyInSeconds |
Yes | 15 |
Send New Rules Update Frequency | sendNewRulesUpdateFrequencyInSeconds |
Yes | 15 |
Send New Rules Default Value | sendNewRulesDefaultExpression |
Yes | 0 |
Send Alerts SQL Query | sendAlertsSqlQuery |
Yes | SELECT * from "pandora_alerts" where "last_modified_on" > #[flowVars.lastModifiedDate] ORDER BY "last_modified_on" DESC; |
Send Alerts Default Value | sendAlertsDefaultExpression |
Yes | 0 |
Send Alerts Update Frequency | sendAlertsUpdateFrequencyInSeconds |
Yes | 15 |
Check Alerts SQL Query | checkAlertsSqlQuery |
Yes | SELECT "id" from "pandora_alerts" WHERE "id" = #[flowVars.collibraRule.id] |
Update Alerts SQL Query | updateAlertsSqlQuery |
Yes | UPDATE pandora_alerts SET last_modified_by = #[flowVars.lastModifiedBy], last_modified_on = #[flowVars.collibraRule.lastModificationDate], name = #[flowVars.collibraRule.name] WHERE id = #[flowVars.collibraRule.id]; |
Inserts To Alerts SQL Query | insertToAlertsSqlQuery |
Yes | INSERT INTO pandora_alerts (id, created_on, created_by, last_modified_on, last_modified_by, name, domain_id, domain_name) VALUES (#[flowVars.collibraRule.id], #[flowVars.collibraRule.creationDate], #[flowVars.createdBy], #[flowVars.collibraRule.lastModificationDate], #[flowVars.lastModifiedBy], #[flowVars.collibraRule.name], #[flowVars.collibraRule.domain.id], #[flowVars.collibraRule.domain.name]; |
The above options are what you need to set. Configuration options are located in data-quality-sync/src/main/app/mule-app.properties
. You will need to replace the default properties with values that are from your environment.
Additional configuration options are present for the connector under experian-data-quality-connector/src/main/app/mule-app.properties
.
Configuration Option | Template Property | Required | Example |
---|---|---|---|
Username | dgc.config.user |
Yes | Guest |
Password | dgc.config.password |
Yes | SamplePassword |
Application URL | dgc.config.baseApplicationUrl |
Yes | https://your-instance.collibra.com |
Data Quality Dimension to Data Quality Metric Relation` | dataQualityDimensionToDataQualityMetricId |
Yes | xxxxxxxxx-6949-42bb-9869-xxxxxxxxxxxx |
Rulebook Domain Name | ruleBookAssetsDomainName |
Yes | Data Quality Rules |