Monitoring and Managing Tomcat

Table of Contents

Introduction

Monitoring is a key aspect of system administration. Looking inside a running server, obtaining some statistics or reconfiguring some aspects of an application are all daily administration tasks.

Enabling JMX Remote

Note: This configuration is needed only if you are going to monitor Tomcat remotely. It is not needed if you are going to monitor it locally, using the same user that Tomcat runs with.

The Oracle website includes the list of options and how to configure JMX Remote on Java 8: https://docs.oracle.com/javase/6/docs/technotes/guides/management/agent.html.

The following is a quick configuration guide for Java 8:

Add the following parameters to setenv.bat script of your Tomcat (see RUNNING.txt for details).
Note: This syntax is for Microsoft Windows. The command has to be on the same line. It is wrapped to be more readable. If Tomcat is running as a Windows service, use its configuration dialog to set java options for the service. For Linux, MacOS, etc, remove "set " from beginning of the line. 

set CATALINA_OPTS=-Dcom.sun.management.jmxremote.port=%my.jmx.port%
  -Dcom.sun.management.jmxremote.rmi.port=%my.rmi.port%
  -Dcom.sun.management.jmxremote.ssl=false
  -Dcom.sun.management.jmxremote.authenticate=false

If you don't set com.sun.management.jmxremote.rmi.port then the JSR 160 JMX-Adaptor will select a port at random which will may it difficult to configure a firewall to allow access.

If you require TLS:

  1. change and add this: 
      -Dcom.sun.management.jmxremote.ssl=true
  -Dcom.sun.management.jmxremote.registry.ssl=true
  2. to configure the protocols and/or cipher suites use: 
      -Dcom.sun.management.jmxremote.ssl.enabled.protocols=%my.jmx.ssl.protocols%
  -Dcom.sun.management.jmxremote.ssl.enabled.cipher.suites=%my.jmx.cipher.suites%
  3. to client certificate authentication use: 
      -Dcom.sun.management.jmxremote.ssl.need.client.auth=%my.jmx.ssl.clientauth%

If you require authorization (it is strongly recommended that TLS is always used with authentication):

  1. change and add this: 
      -Dcom.sun.management.jmxremote.authenticate=true
  -Dcom.sun.management.jmxremote.password.file=../conf/jmxremote.password
  -Dcom.sun.management.jmxremote.access.file=../conf/jmxremote.access
  2. edit the access authorization file $CATALINA_BASE/conf/jmxremote.access: 
    monitorRole readonly
controlRole readwrite
  3. edit the password file $CATALINA_BASE/conf/jmxremote.password: 
    monitorRole tomcat
controlRole tomcat
    Tip: The password file should be read-only and only accessible by the operating system user Tomcat is running as.
  4. Alternatively, you can configure a JAAS login module with: 
      -Dcom.sun.management.jmxremote.login.config=%login.module.name%

If you need to specify a host name to be used in the RMI stubs sent to the client (e.g. because the public host name that must be used to connect is not the same as the local host name) then you can set:

set CATALINA_OPTS=-Djava.rmi.server.hostname

If you need to specify a specific interface for the JMX service to bind to then you can set:

set CATALINA_OPTS=-Dcom.sun.management.jmxremote.host

Manage Tomcat with JMX remote Ant Tasks

To simplify JMX usage with Ant, a set of tasks is provided that may be used with antlib.

antlib: Copy your catalina-ant.jar from $CATALINA_HOME/lib to $ANT_HOME/lib.

The following example shows the JMX Accessor usage:
Note: The name attribute value was wrapped here to be more readable. It has to be all on the same line, without spaces.

<project name="Catalina Ant JMX"
      xmlns:jmx="antlib:org.apache.catalina.ant.jmx"
      default="state"
      basedir=".">
  <property name="jmx.server.name" value="localhost" />
  <property name="jmx.server.port" value="9012" />
  <property name="cluster.server.address" value="192.168.1.75" />
  <property name="cluster.server.port" value="9025" />

  <target name="state" description="Show JMX Cluster state">
    <jmx:open
      host="${jmx.server.name}"
      port="${jmx.server.port}"
      username="controlRole"
      password="tomcat"/>
    <jmx:get
      name=
"Catalina:type=IDataSender,host=localhost,
senderAddress=${cluster.server.address},senderPort=${cluster.server.port}"
      attribute="connected"
      resultproperty="IDataSender.backup.connected"
      echo="false"
    />
    <jmx:get
      name="Catalina:type=ClusterSender,host=localhost"
      attribute="senderObjectNames"
      resultproperty="senderObjectNames"
      echo="false"
    />
    <!-- get current maxActiveSession from ClusterTest application
       echo it to Ant output and store at
       property <em>clustertest.maxActiveSessions.orginal</em>
    -->
    <jmx:get
      name="Catalina:type=Manager,context=/ClusterTest,host=localhost"
      attribute="maxActiveSessions"
      resultproperty="clustertest.maxActiveSessions.orginal"
      echo="true"
    />
    <!-- set maxActiveSession to 100
    -->
    <jmx:set
      name="Catalina:type=Manager,context=/ClusterTest,host=localhost"
      attribute="maxActiveSessions"
      value="100"
      type="int"
    />
    <!-- get all sessions and split result as delimiter <em>SPACE</em> for easy
       access all session ids directly with Ant property sessions.[0..n].
    -->
    <jmx:invoke
      name="Catalina:type=Manager,context=/ClusterTest,host=localhost"
      operation="listSessionIds"
      resultproperty="sessions"
      echo="false"
      delimiter=" "
    />
    <!-- Access session attribute <em>Hello</em> from first session.
    -->
    <jmx:invoke
      name="Catalina:type=Manager,context=/ClusterTest,host=localhost"
      operation="getSessionAttribute"
      resultproperty="Hello"
      echo="false"
    >
      <arg value="${sessions.0}"/>
      <arg value="Hello"/>
    </jmx:invoke>
    <!-- Query for all application manager.of the server from all hosts
       and bind all attributes from all found manager MBeans.
    -->
    <jmx:query
      name="Catalina:type=Manager,*"
      resultproperty="manager"
      echo="true"
      attributebinding="true"
    />
    <!-- echo the create properties -->
<echo>
senderObjectNames: ${senderObjectNames.0}
IDataSender.backup.connected: ${IDataSender.backup.connected}
session: ${sessions.0}
manager.length: ${manager.length}
manager.0.name: ${manager.0.name}
manager.1.name: ${manager.1.name}
hello: ${Hello}
manager.ClusterTest.0.name: ${manager.ClusterTest.0.name}
manager.ClusterTest.0.activeSessions: ${manager.ClusterTest.0.activeSessions}
manager.ClusterTest.0.counterSend_EVT_SESSION_EXPIRED:
 ${manager.ClusterTest.0.counterSend_EVT_SESSION_EXPIRED}
manager.ClusterTest.0.counterSend_EVT_GET_ALL_SESSIONS:
 ${manager.ClusterTest.0.counterSend_EVT_GET_ALL_SESSIONS}
</echo>

  </target>

</project>

import: Import the JMX Accessor Project with <import file="${CATALINA.HOME}/bin/catalina-tasks.xml" /> and reference the tasks with jmxOpen, jmxSet, jmxGet, jmxQuery, jmxInvoke, jmxEquals and jmxCondition.

JMXAccessorOpenTask - JMX open connection task

List of Attributes

Attribute Description Default value
url Set JMX connection URL - service:jmx:rmi:///jndi/rmi://localhost:8050/jmxrmi
host Set the host, shortcut the very long URL syntax. localhost
port Set the remote connection port 8050
username remote JMX connection user name.
password remote JMX connection password.
ref Name of the internal connection reference. With this attribute you can configure more the one connection inside the same Ant project. jmx.server
echo Echo the command usage (for access analysis or debugging) false
if Only execute if a property of the given name exists in the current project.
unless Only execute if a property of the given name not exists in the current project.

Example to open a new JMX connection 

  <jmx:open
    host="${jmx.server.name}"
    port="${jmx.server.port}"
  />

Example to open a JMX connection from URL, with authorization and store at other reference 

  <jmx:open
    url="service:jmx:rmi:///jndi/rmi://localhost:9024/jmxrmi"
    ref="jmx.server.9024"
    username="controlRole"
    password="tomcat"
  />

Example to open a JMX connection from URL, with authorization and store at other reference, but only when property jmx.if exists and jmx.unless not exists 

  <jmx:open
    url="service:jmx:rmi:///jndi/rmi://localhost:9024/jmxrmi"
    ref="jmx.server.9024"
    username="controlRole"
    password="tomcat"
    if="jmx.if"
    unless="jmx.unless"
  />

Note: All properties from jmxOpen task also exists at all other tasks and conditions.

JMXAccessorGetTask: get attribute value Ant task

List of Attributes

Attribute Description Default value
name Full qualified JMX ObjectName -- Catalina:type=Server
attribute Existing MBean attribute (see Tomcat MBean description above)
ref JMX Connection reference jmx.server
echo Echo command usage (access and result) false
resultproperty Save result at this project property
delimiter Split result with delimiter (java.util.StringTokenizer) and use resultproperty as prefix to store tokens.
separatearrayresults When return value is an array, save result as property list ($resultproperty.[0..N] and $resultproperty.length) true

Example to get remote MBean attribute from default JMX connection 

  <jmx:get
    name="Catalina:type=Manager,context=/servlets-examples,host=localhost"
    attribute="maxActiveSessions"
    resultproperty="servlets-examples.maxActiveSessions"
  />

Example to get and result array and split it at separate properties 

  <jmx:get
      name="Catalina:type=ClusterSender,host=localhost"
      attribute="senderObjectNames"
      resultproperty="senderObjectNames"
  />

Access the senderObjectNames properties with: 

  ${senderObjectNames.length} give the number of returned sender list.
  ${senderObjectNames.[0..N]} found all sender object names

Example to get IDataSender attribute connected only when cluster is configured.
Note: The name attribute value was wrapped here to be more readable. It has to be all on the same line, without spaces. 


  <jmx:query
    failonerror="false"
    name="Catalina:type=Cluster,host=${tomcat.application.host}"
    resultproperty="cluster"
  />
  <jmx:get
    name=
"Catalina:type=IDataSender,host=${tomcat.application.host},
senderAddress=${cluster.backup.address},senderPort=${cluster.backup.port}"
    attribute="connected"
    resultproperty="datasender.connected"
    if="cluster.0.name" />

JMXAccessorSetTask: set attribute value Ant task

List of Attributes

Attribute Description Default value
name Full qualified JMX ObjectName -- Catalina:type=Server
attribute Existing MBean attribute (see Tomcat MBean description above)
value value that set to attribute
type type of the attribute. java.lang.String
ref JMX Connection reference jmx.server
echo Echo command usage (access and result) false

Example to set remote MBean attribute value 

  <jmx:set
    name="Catalina:type=Manager,context=/servlets-examples,host=localhost"
    attribute="maxActiveSessions"
    value="500"
    type="int"
  />

JMXAccessorInvokeTask: invoke MBean operation Ant task

List of Attributes

Attribute Description Default value
name Full qualified JMX ObjectName -- Catalina:type=Server
operation Existing MBean operation
ref JMX Connection reference jmx.server
echo Echo command usage (access and result) false
resultproperty Save result at this project property
delimiter Split result with delimiter (java.util.StringTokenizer) and use resultproperty as prefix to store tokens.
separatearrayresults When return value is an array, save result as property list ($resultproperty.[0..N] and $resultproperty.length) true

stop an application 

  <jmx:invoke
    name="Catalina:type=Manager,context=/servlets-examples,host=localhost"
    operation="stop"/>

Now you can find the sessionid at ${sessions.[0..N} properties and access the count with ${sessions.length} property.

Example to get all sessionids 

  <jmx:invoke
    name="Catalina:type=Manager,context=/servlets-examples,host=localhost"
    operation="listSessionIds"
    resultproperty="sessions"
    delimiter=" "
  />

Now you can find the sessionid at ${sessions.[0..N} properties and access the count with ${sessions.length} property.

Example to get remote MBean session attribute from session ${sessionid.0} 

  <jmx:invoke
    name="Catalina:type=Manager,context=/ClusterTest,host=localhost"
    operation="getSessionAttribute"
    resultproperty="hello">
     <arg value="${sessionid.0}"/>
     <arg value="Hello" />
  </jmx:invoke>

Example to create a new access logger valve at vhost localhost 

 <jmx:invoke
         name="Catalina:type=MBeanFactory"
         operation="createAccessLoggerValve"
         resultproperty="accessLoggerObjectName"
 >
     <arg value="Catalina:type=Host,host=localhost"/>
 </jmx:invoke>

Now you can find new MBean with name stored at ${accessLoggerObjectName} property.

JMXAccessorQueryTask: query MBean Ant task

List of Attributes

Attribute Description Default value
name JMX ObjectName query string -- Catalina:type=Manager,*
ref JMX Connection reference jmx.server
echo Echo command usage (access and result) false
resultproperty Prefix project property name to all founded MBeans (mbeans.[0..N].objectname)
attributebinding bind ALL MBean attributes in addition to name false
delimiter Split result with delimiter (java.util.StringTokenizer) and use resultproperty as prefix to store tokens.
separatearrayresults When return value is an array, save result as property list ($resultproperty.[0..N] and $resultproperty.length) true

Get all Manager ObjectNames from all services and Hosts 

  <jmx:query
    name="Catalina:type=Manager,*
    resultproperty="manager" />

Now you can find the Session Manager at ${manager.[0..N].name} properties and access the result object counter with ${manager.length} property.

Example to get the Manager from servlet-examples application an bind all MBean properties 

  <jmx:query
    name="Catalina:type=Manager,context=/servlet-examples,host=localhost*"
    attributebinding="true"
    resultproperty="manager.servletExamples" />

Now you can find the manager at ${manager.servletExamples.0.name} property and can access all properties from this manager with ${manager.servletExamples.0.[manager attribute names]}. The result object counter from MBeans is stored ad ${manager.length} property.

Example to get all MBeans from a server and store inside an external XML property file 

<project name="jmx.query"
            xmlns:jmx="antlib:org.apache.catalina.ant.jmx"
            default="query-all" basedir=".">
<property name="jmx.host" value="localhost"/>
<property name="jmx.port" value="8050"/>
<property name="jmx.username" value="controlRole"/>
<property name="jmx.password" value="tomcat"/>

<target name="query-all" description="Query all MBeans of a server">
  <!-- Configure connection -->
  <jmx:open
    host="${jmx.host}"
    port="${jmx.port}"
    ref="jmx.server"
    username="${jmx.username}"
    password="${jmx.password}"/>

  <!-- Query MBean list -->
  <jmx:query
    name="*:*"
    resultproperty="mbeans"
    attributebinding="false"/>

  <echoproperties
    destfile="mbeans.properties"
    prefix="mbeans."
    format="xml"/>

  <!-- Print results -->
  <echo message=
    "Number of MBeans in server ${jmx.host}:${jmx.port} is ${mbeans.length}"/>
</target>
</project>

Now you can find all MBeans inside the file mbeans.properties.

JMXAccessorCreateTask: remote create MBean Ant task

List of Attributes

Attribute Description Default value
name Full qualified JMX ObjectName -- Catalina:type=MBeanFactory
className Existing MBean full qualified class name (see Tomcat MBean description above)
classLoader ObjectName of server or web application classloader
( Catalina:type=ServerClassLoader,name=[server,common,shared] or
Catalina:type=WebappClassLoader,context=/myapps,host=localhost)
ref JMX Connection reference jmx.server
echo Echo command usage (access and result) false

Example to create remote MBean 

  <jmx:create
    ref="${jmx.reference}"
    name="Catalina:type=MBeanFactory"
    className="org.apache.commons.modeler.BaseModelMBean"
    classLoader="Catalina:type=ServerClassLoader,name=server">
    <arg value="org.apache.catalina.mbeans.MBeanFactory" />
  </jmx:create>

Warning: Many Tomcat MBeans can't be linked to their parent once
created. The Valve, Cluster and Realm MBeans are not automatically
connected with their parent. Use the MBeanFactory create
operation instead.

JMXAccessorUnregisterTask: remote unregister MBean Ant task

List of Attributes

Attribute Description Default value
name Full qualified JMX ObjectName -- Catalina:type=MBeanFactory
ref JMX Connection reference jmx.server
echo Echo command usage (access and result) false

Example to unregister remote MBean 

  <jmx:unregister
    name="Catalina:type=MBeanFactory"
  />

Warning: A lot of Tomcat MBeans can't be unregister.
The MBeans are not unlinked from their parent. Use MBeanFactory
remove operation instead.

JMXAccessorCondition: express condition

List of Attributes

Attribute Description Default value
url Set JMX connection URL - service:jmx:rmi:///jndi/rmi://localhost:8050/jmxrmi
host Set the host, shortcut the very long URL syntax. localhost
port Set the remote connection port 8050
username remote JMX connection user name.
password remote JMX connection password.
ref Name of the internal connection reference. With this attribute you can configure more the one connection inside the same Ant project. jmx.server
name Full qualified JMX ObjectName -- Catalina:type=Server
echo Echo condition usage (access and result) false
if Only execute if a property of the given name exists in the current project.
unless Only execute if a property of the given name not exists in the current project.
value (required) Second arg for operation
type Value type to express operation (support long and double) long
operation express one
  • == equals
  • != not equals
  • > greater than (&gt;)
  • >= greater than or equals (&gt;=)
  • < lesser than (&lt;)
  • <= lesser than or equals (&lt;=)
 ==

Wait for server connection and that cluster backup node is accessible 

<target name="wait">
  <waitfor maxwait="${maxwait}" maxwaitunit="second" timeoutproperty="server.timeout" >
    <and>
      <socket server="${server.name}" port="${server.port}"/>
      <http url="${url}"/>
      <jmx:condition
        operation="=="
        host="localhost"
        port="9014"
        username="controlRole"
        password="tomcat"
        name=
"Catalina:type=IDataSender,host=localhost,senderAddress=192.168.111.1,senderPort=9025"
        attribute="connected"
        value="true"
      />
    </and>
  </waitfor>
  <fail if="server.timeout" message="Server ${url} don't answer inside ${maxwait} sec" />
  <echo message="Server ${url} alive" />
</target>

JMXAccessorEqualsCondition: equals MBean Ant condition

List of Attributes

Attribute Description Default value
url Set JMX connection URL - service:jmx:rmi:///jndi/rmi://localhost:8050/jmxrmi
host Set the host, shortcut the very long URL syntax. localhost
port Set the remote connection port 8050
username remote JMX connection user name.
password remote JMX connection password.
ref Name of the internal connection reference. With this attribute you can configure more the one connection inside the same Ant project. jmx.server
name Full qualified JMX ObjectName -- Catalina:type=Server
echo Echo condition usage (access and result) false

Wait for server connection and that cluster backup node is accessible 

<target name="wait">
  <waitfor maxwait="${maxwait}" maxwaitunit="second" timeoutproperty="server.timeout" >
    <and>
      <socket server="${server.name}" port="${server.port}"/>
      <http url="${url}"/>
      <jmx:equals
        host="localhost"
        port="9014"
        username="controlRole"
        password="tomcat"
        name=
"Catalina:type=IDataSender,host=localhost,senderAddress=192.168.111.1,senderPort=9025"
        attribute="connected"
        value="true"
      />
    </and>
  </waitfor>
  <fail if="server.timeout" message="Server ${url} don't answer inside ${maxwait} sec" />
  <echo message="Server ${url} alive" />
</target>

Using the JMXProxyServlet

Tomcat offers an alternative to using remote (or even local) JMX connections while still giving you access to everything JMX has to offer: Tomcat's JMXProxyServlet.

The JMXProxyServlet allows a client to issue JMX queries via an HTTP interface. This technique offers the following advantages over using JMX directly from a client program:

  • You don't have to launch a full JVM and make a remote JMX connection just to ask for one small piece of data from a running server
  • You don't have to know how to work with JMX connections
  • You don't need any of the complex configuration covered in the rest of this page
  • Your client program does not have to be written in Java

A perfect example of JMX overkill can be seen in the case of popular server-monitoring software such as Nagios or Icinga: if you want to monitor 10 items via JMX, you will have to launch 10 JVMs, make 10 JMX connections, and then shut them all down every few minutes. With the JMXProxyServlet, you can make 10 HTTP connections and be done with it.

You can find out more information about the JMXProxyServlet in the documentation for the Tomcat manager.