Moving Kieker to a pipeline-supported development process: Discussing recent achievements and lessons learned

The Kieker Monitoring Framework [1] is an open-source project mainly developed by researchers in Kiel and Stuttgart. It provides tools for application performance monitoring and the analysis of the collected performance data.

Recently we decided to turn the rather people-centric development process into a pipeline-centric one. Before the changes were put in place the infrastructure was a self-hosted Trac [2], Gitlab [3] and Jenkins [4] setup. To give an example, well-known developers were allowed to commit their code changes directly to the main VCS branch without having a quality check prior to their commit. The continouous integration was triggered after commits were pushed to the repository. This led to cases where the main branch was erroneous for some time until the issue was fixed.
We migrated from a self-hosted Trac, Gitlab, Jenkins to publicly available SaaS-Services (GitHub [5], Atlassian Cloud (Confluence, JIRA) [6], Snap CI [7]. Only some small tasks are still done by a self-hosted Jenkins instance. Today, all commits must pass the --- now explicitly implemented --- continouous delivery pipeline before the changes get into the stable Kieker branch.


The drivers for said changes are manifold:

  • Better public visibility: As most non-commercialized open-source projects are driven by contributions from the public, the visibility plays a major part in the lifecycle of an open-source project. Therefore, a renowned public and transparent repository, wiki, and issue tracking are desirable. (-> GitHub, Atlassian Cloud)
  • Minimizing operations effort: Every effort that goes into maintaining and managing the infrastructure of an open-source project reduces the time the developers can use for the actual software project. In order to reduce this effort to a minimum, using resources that are free to the public are preferable. (-> GitHub, Atlassian Cloud, Snap-CI)
  • Independence of individual system configurations: The development environments of developers are very diverse, which may lead to different behavior in terms of tests and code analysis. To reduce such impact there should be a default environment that one can easily test against. (-> Docker)
  • Comparable and concise feedback for developers: Not knowing whether the local results are valid for the build environment is unsatisfying and annoying when everything passes locally but fails in the continuous integration pipeline and renders the current repository state unstable. (-> GitHub and SnapCI using webhooks to trigger the continouous delivery pipelines)
  • Keeping the configuration infrastructure under revision control: When many people work on a common project, it is desirable that everybody can see the configuration that is made even for the infrastructure and the enacted processes. To achieve this, it is important to also have such configuration under revision control. This increases the transparency and traceability. (-> Use GitHub to store configurations and scripts used in Snap-CI and Gradle)
  • Enforce development processes: Grown projects often have no strict policies regarding the development process from day one, which leads to different approaches for developing and publishing contributions to the project. In order to unify and simplify this, a process that ensures code quality and transparency is needed. (-> GitHub, JIRA)

In the talk we will go through the different aspects and present the original state, what we wanted to change and why; and which considerations led us to the newly established state and what lessons we learned. Even though we went a long way already to achieve the current state, we are not finished yet, as there are still some issues to be tackled.

As all our infrastructure is public, you can see the current state here:

The idea of this talk is to share our experiences and lessons learned, which may be helpful to other open-source projects in the community to improve their processes. Furthermore, at the symposium this would be an interesting topic to discuss in a wider audience.

[1] Kieker: kieker-monitoring.net
[2] Trac: trac.edgewall.org
[3] Gitlab: about.gitlab.com
[4] Jenkins: jenkins.io
[5] Github: www.github.com
[6] Atlassian Cloud: www.atlassian.com
[7] Snap CI: snap-ci.com