Technical Data Questionnaire Template - novoda/novoda GitHub Wiki

We have Gradle Build Properties Plugin as an example

Overview of the project. What are its technical capabilities compared to the industry standard?

Development of a plugin system for open source applications that require private keys and configs to be ran but these should not be shared in public locations. for example an analytics library that needs a key to login to the system but not all people should access to this key. The project focuses on making the possibility of adding private information as seamless as possible, so that the projects can be used simply but not expose information. This offers features not available currently on the market and although other variations of this technique are possible they do not make the solution quite as easy.

What platforms and tools did you use to make it? (encouraged to namedrop components/languages/databases etc)

Groovy, Gradle for development, Guava, Truth and Findbugs for testing

Describe, in detail, the technical challenges faced (specifically what development required more than routine techniques) and what was engineered to overcome them?

It is really important to discuss the iterative journey to explain at each stage of production (each release) what was engineered.

Write this as if you are explaining the challenges to a technical colleague.

Attempt to write at least four paragraphs that cover these points:

  • Technical advances
  • How uncertainties were overcome
  • Any iterative approaches
  • Final solution

Currently to solve the problem of not sharing private keys, developers would cut and paste when they were needed for production and a lot of human interaction was needed to keep these keys private, but still useful. Many repositories would require five or more keys, therefore it required coordination with the build system to make sure each was put in the right place and used at the right time. The gradle build properties plugin automates all of this and allows you to drop in one file, that is read by the gradle system and pulls out the properties or keys and puts them in the right place. This is a technical advancement on any of the current solutions as before it was not centralised or automated meaning keys where in many different places and the chance for security issues was high. With this centralised solution it can allow us to keep all the keys in a secure place and use security credential to access them at the time needed.

A big uncertainty was our knowledge around connecting a remote secure location for the keys to the build system that would be using them. We did not know if we could access or communicate with a secure system in a timely fashion that would allow for the normal build process. This uncertainty was overcome through investigating distribution systems based on maven repositories. With this investigation it was found two way secure communication could happen and therefore we could separate the private keys from the public repository.

When injecting the private keys into a gradle build it has to be done at the correct step. With the creation of this plugin we were unsure in our knowledge of gradle plugins and where we would hook into the build system. This problem was overcome with the development approach of investigating other gradle plugins, this lead us to the Android plugin that also hooks in for other reasons, and we adapted and expanded on this knowledge to allow our key injection to work.

The first approach was to load from a local properties file, take the keys from a separate file, this split apart building and key loading. The second was being able to override at runtime any errors, therefore giving fallback strategies if keys were not available. For example if the properties file was missing a private key, the key could also be set as a system property and we could now fallback from one to the other in a recursive fashion. We then added other fallbacks for secondary files, command line and hard coded values. We then iterated again to allow for the fallback file to be a remote file and take security credentials to open or load it.

The final solution now allows for developers to create open source repositories that use private keys without them needing to be public. The integration is as simple as defining a file, that is not checked into version control but can be in one of multiple places, and then providing fallback strategies if this secure file is not there, for example providing fallback debug keys that can be public and checked into the repository. This simplified any developers use of private keys in a public or private repositories and will likely lead to a decrease in their time spent in this area by a factor of 10x.