Operations: Deploy 22acacia Owned Projects - 22Acacia/sossity GitHub Wiki

These projects are core to the open source efforts of 22acacia. These applications and SDKs are available in both source and binary form under the Apache license. We recommend that you use 22acacia provided binary artifacts but suggest that you also have backups of production critical artifacts on a separate site. If you plan to use 22acaia provided resources for these projects the provided pipeline-controller project is configured to use these and it is not required that the user go through these steps. If you do chose to build these on your own they should be configured similarly to the Domain Specific Projects.

##Projects

Sossity: the namesake of the project is the pipeline orchestrator. It converts a DAG of sources, sinks and pipelines into a terraform configuration that describes the entire data processing flow including built in health checks, error processing and all of the business logic. The current version of this artifact can be downloaded at https://storage.googleapis.com/build-artifacts-public-eu/sossity/sossity-0.1.0-SNAPSHOT-standalone.jar. If you chose to build this yourself, the repository is at github.com/22Acacia/sossity and has a circle.yml file to direct circleci. It has one required and three suggested environment variables be set in circleci. Required:
GOOGLE_CREDENTIALS, which is the json file saved from the creation of the service account
GSTORAGE_DEST_BUCKET, google storage bucket to save artifacts in Suggested:
CIRCLE_TOKEN, which you can generate for your account at: circleci.com/account/api
GITHUB_ORG, name of your github organization, probably holidayextras
NEXT_PROJECT, the name of the repository in github that is next in the CI chain
AngledDream: a Google Cloud Dataflow SDK made to interoperate with Sossity. This project provides a simple interface to write stream processing functions in Java. The current version of this artifact can be added to your project using maven with the following pom file entries:

<repositories>
  <repository>
    <id>jitpack.io</id>
    <url>https://jitpack.io</url>
  </repository>
</repositories>
<dependencies>   
  <dependency>
    <groupId>com.github.22Acacia</groupId>
    <artifactId>angled-dream</artifactId>
    <version>-SNAPSHOT</version>
  </dependency>
</dependencies>

If you chose to build this yourself, the repository is at github.com/22Acacia/angled-dream and has a circle.yml file to direct circleci. It has one required and three suggested environment variables be set in circleci. Required:

GOOGLE_CREDENTIALS, which is the json file saved from the creation of the service account
GSTORAGE_DEST_BUCKET, google storage bucket to save artifacts in Suggested:
CIRCLE_TOKEN, which you can generate for your account at: circleci.com/account/api
GITHUB_ORG, name of your github organization, probably holidayextras
NEXT_PROJECT, the name of the repository in github that is next in the CI chain
terraform components: terraform is an open source infrastructure as a service management application. Sossity uses this to manage all cloud resources. We require at least version 0.6.9. There are three custom plugins that are needed:
terraform-provider-googlecli: you can use the 22acacia provided artifact at gs://build-artifacts-public-eu/terraform-provider-googlecli. This is an unversioned resource and will change as the underlying google cli changes. This CLI tends to not have backwards compatibility so there will only be one version. If you want to build this yourself, the code is available at: github.com/22Acacia/terraform-provider-googlecli and it contains a circle.yml to direct circleci to operate correctly. This project requires two env variables:
GOOGLE_CREDENTIALS, which is the json file saved from creation the service account
GSTORAGE_DEST_BUCKET, google storage bucket to save artifacts in
terraform-provider-googlebigquery: you can use the 22acacia provided artifact at gs://build-artifacts-public-eu/terraform-provider-googlebigquery. This is an unversioned resource because we intend to roll this into upstream terraform as time permits. If you want to build this yourself, the code is available at: github.com/22Acacia/terraform-provider-googlebigquery and it contains a circle.yml to direct circleci to operate correctly. This project requires two env variables:
GOOGLE_CREDENTIALS, which is the json file saved from creation the service account
GSTORAGE_DEST_BUCKET, google storage bucket to save artifacts in
terraform-provider-googleappengine: you can use the 22acacia provided artifact at gs://build-artifacts-public-eu/terraform-provider-googleappengine. This is an unversioned resource because we intend to roll this into upstream terraform as time permits. If you want to build this yourself, the code is available at: github.com/22Acacia/terraform-provider-googleappengine and it contains a circle.yml to direct circleci to operate correctly. This project requires two env variables:
GOOGLE_CREDENTIALS, which is the json file saved from creation the service account
GSTORAGE_DEST_BUCKET, google storage bucket to save artifacts in
cowbox: this is a java project built to run in appengine, google’s PaaS, and is a generic API that accepts a POST with a json body and writes it into a pubsub topic. This is the first step for getting data into Sossity. The current version of this artifact can be found in appengine ready format at:

gs://build-artifacts-public-eu/source-1.0-ALPHA/

If you chose to build this yourself, the repository is at github.com/22Acacia/cowbox and has a circle.yml file to direct circleci to operate correclty. It requires that two environment variables be set in circleci:
GOOGLE_CREDENTIALS, which is the json file saved from creation the service account
GSTORAGE_DEST_BUCKET, google storage bucket to save artifacts in
dreamboat: this is a docker image that contains a heartbeat application for any listed input API, usually a headspring image. This heart beat should be used to verify overall health of the system and can be used for performance analysis. This image is provided by 22acacia and can be pulled from the google cloud image repository (GCR) at gcr.io/hx-test/dreamboat or you can find the source for this at github.com/22Acacia/dreamboat-multi. This project contains a circle.yml for building in circleci and it requires two environment variables:
GOOGLE_CREDENTIALS, which is the json file saved from creation the service account
GOOGLE_PROJECT, which is a namespace to use in gcr, google cloud’s container registry
yellow-submarine: this is a docker image that allows for dropping in an arbitrary function that will accept messages from the PubSub and then the input function, or “lambda function”, is responsible for forming the output message in the desired format for the downstream sink. The writing of the message to its final storage location is handled by this image. This image is provided by 22acacia and can be pulled from the google cloud image repository (GCR) at gcr.io/hx-test/yellow-submarine or you can find the source for this at https://github.com/22Acacia/yellow-submarine. This project contains a circle.yml for building in circleci and it requires two environment variables:
GOOGLE_CREDENTIALS, which is the json file saved from creation the service account
GOOGLE_PROJECT, which is a namespace to use in gcr, google cloud’s container registry

Operations: Deploy 22acacia Owned Projects - 22Acacia/sossity GitHub Wiki

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️