For the purpose of executing the core APM activities [1] — data collection, processing and storing, presentation, and interpretation — in this talk, we present a monitoring pipeline based on Kieker [2] and Elastic APM [3]. Elastic APM consists of i) APM agent, which collects the data, ii) APM server, which gathers the data and creates documents which are stored into iii) Elasticsearch, and visualized using iv) Kibana APM UI. This pipeline handles the aforementioned activities by using different frameworks and open source tools.
In our pipeline, the extensible Kieker Framework is used for data collection, i.e., Kieker’s monitoring probes and the Kieker.Monitoring component act as APM Agents. We developed a new Kieker writer to be used to correlate the collected measurements and to create a trace representation. This writer receives each Kieker monitoring record from the probes and builds a span-tree by using the execution order index and the execution stack size. All the records are buffered until the span-tree has been built entirely. This approach is necessary because the Elastic APM Server uses spans to represent a trace. After that, the span tree representation will be serialized to JSON and sent to the Elastic APM Server endpoint through the JSON API. This server stores the data in an Elasticsearch instance. Kibana is usually used to pull the data from Elasticsearch, to visualize the traces and the underlying spans.
Our main goal was to harvest the collected data and to analyze it using our diagnoseIT tool [4]. This way we would be able to use diagnoseIT for problem diagnosis in systems based on the Elastic Stack. Because diagnoseIT uses the OPEN.xtrace format [5], we also developed an OPEN.xtrace adapter for Elasticsearch. This adapter is used to periodically pull the data from the Elasticsearch instance and convert it into the OPEN.xtrace format.
In our example, all of the components of the pipeline are deployed using Docker containers. Kieker was used to instrument the TeaStore sample application [6]. Pushing the data from Kieker to Elastic APM Server was done by implementing the server’s Intake API. On the other side, the OPEN.xtrace adapter pulls the data from Elasticsearch using HTTP requests.
In this talk, we will present our experience in developing the pipeline, the challenges we were facing, as well as how we plan to extend the pipeline further to connect more open source APM tools.
[1] Christoph Heger, André van Hoorn, Mario Mann, Dušan Okanović. Application Performance Management: State of the Art and Challenges for the Future. In Proceedings of the 8th joint ACM/SPEC International Conference on Performance Engineering (ICPE 2017). pp. 429-432.
[2] André van Hoorn, Jan Waller, Wilhelm Hasselbring. Kieker: A framework for application performance monitoring and dynamic software analysis. In Proceedings of the 3rd joint ACM/SPEC International Conference on Performance Engineering (ICPE 2012). pp 247–248.
[3] Getting Started with APM - Overview, www.elastic.co/guide/en/apm/get-started/current/overview.html
[4] Christoph Heger, André van Hoorn, Dušan Okanović, Stefan Siegl, Alexander Wert. Expert-Guided Automatic Diagnosis of Performance Problems in Enterprise Applications. In Proceedings of 12th European Dependable Computing Conference (EDCC 2016). pp 185-188.
[5] Dušan Okanović, André van Hoorn, Christoph Heger, Alexander Wert, Stefan Siegl: Towards Performance Tooling Interoperability: An Open Format for Representing Execution Traces. In Proceedings of 13th European Workshop on Performance Engineering (EPEW 2016). pp 94-108.
[6] Jóakim von Kistowski, Simon Eismann, Norbert Schmitt, André Bauer, Johannes Grohmann, and Samuel Kounev. TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling, and Resource Management Research. In Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS '18).