Example 1: Spring Petclinic + Apexbeat + ELK

This demo connects the Petclinic example to Elasticsearch using the apex collector agent and apexbeat and will retrieve a set of metrics from the application and the underlying spring boot stack.

Apex will react on

all HTTP requests to petclinic measuring performance and recording HTTP response states.
creation of new pet owners (recording their names)
creation of new pets (dumping complete instance contexts)
the triggered exception of the petclinic

This example can be run locally using docker. You can modify and experiment with the APEX configs and scripts. Just clone and fire up the complete stack with docker-compose.

# clone the sample from github
git clone https://github.com/verticle-io/apex-petclinic-demo.git

# head to the repo root
cd apex-petclinic-demo

# boot the stack
docker-compose up

Visit the Github site for more details. All images are provided via Docker Hub.

Github Docker Hub

Example 2: Apache Tomcat + APEX Collector + APEX Service

In order to get familiar with the mechanisms, we demonstrate how easy APEX can be applied to a standard Apache Tomcat Webserver, tracing the deployments done on the container.

Step1: Configurations and scripts for instrumentation

We define our target classes and methods to hook into and apply the script handler. It will run a groovy script on each execution. The script will report the currently deployed webapp contexts.

$ cat ./localrepo/tomcat/io.verticle.apex.tomcat/8.5.6/deployedApps.json
{
  "meta" : {
    "purpose" : "show deployed webapps"
  },
  "qualifier" : "deployedapps",
  "targetClass" : "org.apache.catalina.startup.HostConfig",
  "instrumentationInstructions" : [
  {
    "@class" : "io.verticle.oss.apex.agent.instrument.repository.model.instrumentation.InstrumentationInstructionModel",
    "qualifier" : "tomcat.deployedapps",
    "methodName" : "deployWARs",
    "signatureClasses" : ["java.io.File","[Ljava.lang.String;"],
    "handlers" : [ {
      "@class" : "io.verticle.oss.apex.agent.instrument.repository.model.instrumentation.HandlerModel",
      "handlerClass" : "io.verticle.oss.apex.agent.sdk.handler.BasicScriptedHandler",
      "options" : {
        "groovy" : "DeployedApps.groovy"
      }
    } ],
    "enabled" : true,
    "constructor" : false
  } ]
}
$ cat ./localrepo/tomcat/io.verticle.apex.tomcat/8.5.6/DeployedApps.groovy
// A simple groovy script that gets executed on each instrumented method call
def execute(){
    println("method executed")
    def instance = context.getInstance()
    instance.serviced.each{
      println("serviced: ${it.key}")
    }
    instance.deployed.each{
      println("deployed: ${it.key}")
    }
    println("file:" + arg1.toString())
    println("contexts:" + arg2.toString())
    message.addField("deployments", arg2.toString())
}

Step2: Starting Tomcat with the Apex Collector Agent

The APEX collector agent is attached to Tomcat via the -javaagent directive. It will connect to the configured git repository to checkout/update the instructions from Step1. Once target classes are loaded by the Classloader it will try to weave in the hooks. Any call to the instrumented method will trigger the handlers on method entry and exit.

$ ./catalina.sh run

Using CATALINA_BASE: /Users/x/github/apache-tomcat-8.5.6
Using CATALINA_HOME: /Users/x/github/apache-tomcat-8.5.6
# starting apex agent ...
...
INFO i.v.o.a.a.bootstrap.BootstrapAgent - verticle.io APM Agent starting
INFO i.v.o.a.a.bootstrap.BootstrapAgent - premain method invoked with args:
/Users/x/github/apexAgent/apmAgentConfig.properties and inst: sun.instrument.InstrumentationImpl@4e515669
INFO i.v.o.a.a.c.config.AgentConfigurer - Configuring MetricsServer Remote
INFO i.v.o.a.a.c.c.AgentConfiguration - reading agent configuration from
/Users/x/github/apexAgent/apmAgentConfig.properties
INFO i.v.o.a.a.c.c.AgentConfiguration - APM RabbitMQ instance configured ... OK
INFO i.v.o.a.a.c.c.AgentConfiguration - APM Instrumentation Configuration configured ... OK
INFO i.v.o.a.a.c.c.AgentConfiguration - done reading agent configuration
# fetching instrumenation configuration from git
INFO i.v.o.a.a.i.InstrumentationRegistry - not updating repository (disabled)
INFO i.v.o.a.a.i.InstrumentationRegistry -   ***********************************************************************
INFO i.v.o.a.a.i.InstrumentationRegistry -   *** APEX COLLECTOR Configuration ***
INFO i.v.o.a.a.i.InstrumentationRegistry -   ***********************************************************************
INFO i.v.o.a.a.i.InstrumentationRegistry -   * targets:
INFO i.v.o.a.a.i.InstrumentationRegistry -        asset: tomcat
INFO i.v.o.a.a.i.InstrumentationRegistry -        package: io.verticle.apex.tomcat / 8.5.6
INFO i.v.o.a.a.i.InstrumentationRegistry -        fingerprint: -
INFO i.v.o.a.a.i.InstrumentationRegistry -        target: org.apache.catalina.startup.HostConfig
INFO i.v.o.a.a.i.InstrumentationRegistry -        qualifier: deployedapps
INFO i.v.o.a.a.i.InstrumentationRegistry -        - method: deployWARs :: [class java.io.File, class [Ljava.lang.String;]
INFO i.v.o.a.a.i.InstrumentationRegistry -        - qualifier: tomcat.deployedapps
INFO i.v.o.a.a.i.InstrumentationRegistry -          - handler: class io.verticle.oss.apex.agent.sdk.handler.BasicScriptedHandler
INFO i.v.o.a.a.i.InstrumentationRegistry -   ***********************************************************************
# attaching to apex service
INFO i.v.o.a.a.c.t.h.ApexServiceHTTPAdapter - initializing ApexServiceHTTPAdapter
INFO i.v.o.a.a.bootstrap.BootstrapAgent - Start sending heartbeats to service ...
DEBUG i.v.o.a.a.bootstrap.BootstrapAgent - sendHeartbeat
[ApexService#sendHeartbeat] ---> POST http://localhost:9005/collector/heartbeat HTTP/1.1
# weaving marked target classes on class loading
INFO i.v.o.a.a.i.AgentClassFileTransformer - trying to weave org.apache.catalina.startup.HostConfig
INFO i.v.o.a.a.i.AgentClassFileTransformer - weaving method org.apache.catalina.startup.HostConfig : deployWARs : [class java.io.File, class [Ljava.lang.String;]
INFO i.v.o.a.a.i.AgentClassFileTransformer - successfully weaved method org.apache.catalina.startup.HostConfig : deployWARs : [class java.io.File, class [Ljava.lang.String;]
...
INFORMATION [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
INFORMATION [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
INFORMATION [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["ajp-nio-8009"]
INFORMATION [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
INFORMATION [main] org.apache.catalina.startup.Catalina.load Initialization processed in 1444 ms
INFORMATION [main] org.apache.catalina.core.StandardService.startInternal Starting service Catalina
INFORMATION [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.6
...
# hook triggered, executing handler
DEBUG i.v.o.a.a.instrument.AdvisorDelegate - instance: org.apache.catalina.startup.HostConfig@3244da18
INFO i.v.o.a.a.s.h.BasicScriptedHandler - Loading groovy script /Users/x/dev/apexrepo/tomcat/io.verticle.apex.tomcat/8.5.6/DeployedApps.groovy
INFO i.v.o.a.a.s.h.BasicScriptedHandler - Parsing handlerscript
INFO i.v.o.a.a.s.h.BasicScriptedHandler - Invoking handlerscript
# handler output
method executed
file:/Users/x/github/apache-tomcat-8.5.6/webapps
contexts:[host-manager, manager]

# sending gathered data to apex service
INFO i.v.o.apex.agent.collector.Collector - VERTICLE.IO COLLECTOR starts harvesting ...
INFO i.v.o.apex.agent.collector.Collector - VERTICLE.IO COLLECTOR ID COLLECTOR-CHnrN9gQcZ
INFO i.v.o.a.a.c.APMCollectorFactory - instantiated class io.verticle.oss.apex.agent.collector.Collector
[ApexService#sendMetric] ---> POST http://localhost:9005/collector/metrics HTTP/1.1
[ApexService#sendMetric] {
  "meta" : {
    "src_ip" : "192.168.188.21",
    "src_mac_hash" : "9f4ada0e81563c88dd66000e200a1dbb782c0438",
    "src_collect" : "COLLECTOR-CHnrN9gQcZ",
    "@timestamp" : 1477574835792,
    "src_class" : "io.verticle.oss.apex.agent.sdk.handler.BasicScriptedHandler",
    "domain" : "application",
    "qualifier" : "org.apache.catalina.startup.HostConfig"
  },
  "metrics" : {
    "deployments" : "[host-manager, manager]"
  }
}

Step3: The listening apex service

The gathered information is routed to the apex service. The basic implementation template will just output incoming data to the console. Fork the template project and extend it according to your needs.

$ mvn spring-boot:run
# bootstrap omitted
...
heartbeat 192.168.188.21 9f4ada0e81563c88dd66000e200a1dbb782c0438 COLLECTOR-CHnrN9gQcZ 1477574834051

==========
application.org.apache.catalina.startup.HostConfig 2016-10-27T15:27:15.792+02:00
----------
deployments: [host-manager, manager]
@timestamp: 2016-10-27T15:27:15.792+02:00
==========

heartbeat 192.168.188.21 9f4ada0e81563c88dd66000e200a1dbb782c0438 COLLECTOR-CHnrN9gQcZ 1477574839051
heartbeat 192.168.188.21 9f4ada0e81563c88dd66000e200a1dbb782c0438 COLLECTOR-CHnrN9gQcZ 1477574844055
...