Orchestrate workflows with Jenkins - MaastrichtU-IDS/data2services-pipeline GitHub Wiki

Execute data2services modules by deploying Jenkins pipelines.

Start Jenkins

docker run -it --name jenkins-container \
  -p 3333:8080  \
  -v /data/jenkins:/data \
  -v /var/run/docker.sock:/var/run/docker.sock \
  deepakunni3/ncats-jenkins 

• Access at http://localhost:3333
• Check GitHub plugin to install it
• Jenkins Home directory in /var/jenkins_home

Design a pipeline

You can find a simple pipeline executing Docker containers to validate a knowledge graph.

Here an example of a simple pipeline to build a Docker image and execute a SPARQL query using this image.

pipeline {
  agent any
  parameters {
    string(name: 'GraphUri', defaultValue: 'https://w3id.org/data2services/graph/biolink/date', description: 'URI of the Graph to validate')
    string(name: 'SparqlRepositoryUri', defaultValue: 'http://graphdb.dumontierlab.com/repositories/public/statements', description: 'URI of the repository used to insert the computed statistics')
    string(name: 'TriplestoreUsername', defaultValue: 'import_user', description: 'Username for the triplestore')
    string(name: 'TriplestorePassword', defaultValue: 'changeme', description: 'Password for the triplestore')
  }
  stages {
    stage('Build and install') {
      steps {
        sh "git clone --recursive https://github.com/vemonet/data2services-insert.git"
        sh 'docker build --rm -t rdf4j-sparql-operations $WORKSPACE/data2services-insert/rdf4j-sparql-operations'
      }
    }
    stage('Compute and insert statistics') {
      steps {
        sh "docker run -t --rm --volumes-from jenkins-container rdf4j-sparql-operations -rq '$WORKSPACE/data2services-insert/compute-statistics' -url '${params.SparqlRepositoryUri}' -un ${params.TriplestoreUsername} -pw ${params.TriplestorePassword} -var inputGraph:${params.GraphUri}"
      }
    }
  }
  post {
    always {
      //archiveArtifacts artifacts: 'results/*', onlyIfSuccessful: true // archive contents in results folder
      deleteDir()
      cleanWs()
    }
  }
}