A Reference Platform for Software Performance Engineering in DevOps

Ensuring adequate performance of software systems is the ultimate goal of software performance engineering (SPE). Many approaches, e.g., performance monitoring, performance modeling, performance prediction and resource provisioning, have been proposed to improve software performance in various stages of software development and operation. With the transition into the DevOps practice, a faster way to deliver new features from development to production is possible by utilizing a continuous deployment pipeline that combines different stages into one automated process.

Although the new practice provides many benefits, most of the existing performance engineering approaches do not fit very well. For instance, DevOps aims for fast and frequent releases, while performance engineering aims for mature and long running load tests. Therefore, establishing a reference platform that can be used by all researchers is an important step to provide a common ground for further research and advancements in SPE.

This talk presents a reference platform based on RedHat's open source DevOps platform, called Fabric8 (https://fabric8.io/). Fabric8 provides a platform for continuous deployment pipeline from the development, e.g., code repository and software testing, to the infrastructure and the deployment of the applications, e.g., cloud platform for container cluster, rolling upgrade, and scaling of applications. The platform employs software container technology, i.e., Docker, Kubernetes and OpenShift, as the underlying infrastructure which is suitable for microservice architectures and allows flexibility in both infrastructure and applications.

To provide a suitable continuous deployment pipeline for performance engineering activities, we are currently extending the standard pipeline to include activities such as feedback loop for performance testing and runtime performance monitoring. The existing application performance monitoring tools, such as Kieker or inspectIT, can be deployed to collect performance metrics and present the results to the developers and operators. Resource provisioning approaches, such as Descartes, can also be used to model the infrastructure and application such that the resource and demand can be estimated and efficiently allocated. Performance modeling and prediction, e.g., Palladio Component Model (PCM), can be applied to predict when the scaling of the application should be performed. Furthermore, other approaches such as anomaly detection, failure prediction, reliability prediction, or data visualization, can also benefit from this platform since the pipeline and the user interface can be extended.

The platform can be easily provisioned thanks to the Infrastructure as Code (IaC) approach. We utilize Ansible and provide a playbook that can automatically and remotely setup the platform on any physical hardware or virtualized environments. The playbook is publicly available and can be downloaded at github.com/spec-rgdevops/blueprint-fabric8. We also welcome and encourage all contributions to extend the platform to support other research in software performance engineering and related fields.

The presented work is an activity of the DevOps Performance Working Group of the Standard Performance Evaluation Corporation’s Research Group (SPEC RG, research.spec.org/devopswg) as part of a larger project on building an open best practice reference infrastructures for performance-aware DevOps.