JMX Plug-In - Technical Reference

Introduction

The JMX plug-in can connect to Java applications which expose their state in accordance with the JMX specification. The application exposes state in the form of managed beans (mbeans) each of which has attributes whose values can be retrieved. The plug-in can be configured to connect to the monitored application, look-up specific mbeans and retrieve the values of their attributes. These applications can include off-the-shelf applications such as WebSphere and WebLogic as well as bespoke-in-house Java applications.

Views

View

The plug-in produces a single view, a representative example of which is given below. Since the user is responsible for producing the contents of the view there is intentionally no reference to headline or table legends.

Plug-In Configuration

Caution: When you update the configuration of certain Java-based plug-ins, including this one, the Netprobe running it has to restart. Be aware of other potentially unrelated plug-ins that can be affected when you configure this particular plug-in.

The plug-in parameters below describe how to configure a JMX plugin to connect to a WebLogic, WebSphere or Java application (e.g. Tomcat 6.0)

The following parameters can be configured for this plug-in:

connection

Describes the type of application to which the plug-in will attempt to connect and monitor.

Possible values:

Mandatory: Yes

connection > generic

Specifies connection settings to a Java application.

connection > generic > serviceURL

Specifies the location where the monitored application resides, e.g.

service:jmx:rmi:///jndi/rmi://192.168.10.161:25000/jmxrmi

For more granularity, the connectionDetails can be used instead. See section Connecting using the JMXMP protocol.

connection > generic > connectionDetails

This allows more granularity and can be used instead of serviceURL.

connection > generic > connectionDetails > host

The ip address (e.g. 192.168.10.161) or resolved host name (e.g. itrspc27) of the machine on which the monitored application runs.

connection > generic > connectionDetails > port

The port which the monitored application accepts connections on (e.g. 25000).

connection > generic > connectionDetails > urlPath

The part of the serviceUrl which appears after the port number, excluding the leading front-slash. For example, if the serviceUrl is:

service:jmx:rmi:///jndi/rmi://192.168.10.161:25000/jmxrmi

Then, the urlpath is specified as:

jmxrmi
connection > generic > jmxId

connection > generic > jmxId

This is the local Java application's process ID.

connection > generic > username

Specifies the username with which to connect to a secured Java application.

connection > generic > password

Specifies the password with which to connect to a secured Java application. To set an encrypted password, click on the "Set Password" button, then enter and confirm the password to be used.

connection > generic > timeout

Specifies the number of milliseconds to wait for connections to be established or JMX commands to return. This setting should not be used unless it is found that the sampler seems to be waiting for inordinate amounts of time to connect to an application which is clearly down. This allows a timeout which the user decides is appropriate, for which to wait for a connection, before deciding that the application should be considered down.

Default: 10000

Unit: milliseconds (ms)

connection > websphere > host

Specifies the location where the server resides (e.g. 192.168.10.200).

connection > websphere > port

Specifies the port on which to connect to the server (e.g. 8881).

connection > websphere > username

Specifies the username to use when connecting to the server.

connection > websphere > password

Specifies the password to use when connecting to the server. To set an encrypted password, click on the "Set Password" button, then enter and confirm the password to be used.

connection > websphere > type

The protocol for the connection to use when sending and receiving information (e.g. SOAP).

t3://192.168.10.200:7011

t3s://192.168.10.200:7002

$1=java.lang:type=GarbageCollector,name=Eden Space

$2=java.lang:type=GarbageCollector,*

$1

java.lang:type=GarbageCollector,*

LastGcInfo.memoryUsageAfterGc[match(key,'Perm Gen.*')]

LastGcInfo.memoryUsageAfterGc

aliases > alias > summary > enable

Enable summary mode.

Mandatory: Yes

aliases > alias > summary > grouping

Optional parameter for grouping the summary MBeans. MBeans aliases can be grouped into multiple logical instances matching on the grouping parameter. If no grouping parameter is specified then the alias will always map to a single instance combining all MBeans, if a grouping parameter is used then the alias can map to multiple instances each combining a subset of the MBeans.

For example, if thealias value was "java.lang:type=MemoryPool,*" and the grouping parameter was "Type" we would expect the alias to map to two grouped instances, one of Type "HEAP" and the other of Type "NON_HEAP" - each mapping to more than one MBean.

Grouped Summary MBeans can only be displayed using column mode. If they are referenced in row mode> or headline variables, only the first group will be returned.

Grouping parameters can either be attributes within the MBean or key properties of the MBean. To specify that the parameter is an attribute it can prefixed with a full stop, to specify that the parameter is a key property it can be prefixed with a colon. If no prefix is used, key properties will be searched first, followed by attributes.

Mandatory: No

aliases > alias > showCommands

Note: Currently, commands are not supported for WebLogic and WebSphere JMX.

headlines > headline > name

The label to display for a particular headline

Mandatory: No

headlines > headline > value

The expression to evaluate and display as the value for a particular headline. The Alias (for example: $1) value should not have a wildcard (*) character and should return a single MBean. An error message is displayed when a wildcard alias is used.

Mandatory: No

headlines > headline > mask

The mask with which to format the value of a particular headline

Mandatory: No

rows

Identifies the values to display for each row in the dataview.

Mandatory: Yes

rows > columnLabels

Describes the labels to use for each of the column headers in the table.

Mandatory: Yes

rows > columnMasks

Describes the masks to use to format each row value with

Mandatory: Yes

rows > row

Describes the expressions or string literals to display for each cell in a particular row.

See section Attribute Paths for more details.

Mandatory: Yes

columns

Describes the columns that should comprise the dataview. Consists of one or more column instances, each of which describe both a column header and an expression to evaluate for a cell in that column.

Mandatory: Yes

columns > column

Describes what to label a table column header and provides an expression to evaluate as the row value for that column.

Mandatory: Yes

columns > column > label

The name to display as a column header for a particular column

Mandatory: Yes

columns > column > rowTemplate

The expression to evaluate as the value of a cell in a particular column with. This can be an mbean expression or an objectname expression.

An mbean expression makes use of one or more dot-delimited attributes on an mbean with an optional quoted literal e.g.

($1.Uptime + $1.Downtime) 'secs'

An objectname expression makes use of colon-delimited fields on an objectname using quoted literals to separate each field e.g.

$1:type '/' $1:index '/'

Dot-delimited attribute paths can contain filter expressions. E.g.

$1.LastGcInfo.memoryUsageAfterGc[key='Perm Gen']

- returns the child object within LastGcInfo.memoryUsageAfterGc where the key value equals "Perm Gen".

If a summary MBean alias is being references then summary functions can be used.

max($1.value)

Mandatory: Yes

columns > column > mask

The mask with which to format the value of a cell in a particular column with

Mandatory: Yes

jproperties

Holds a list of properties that can be used to configure how the JVM is created.

Mandatory: No

jproperties > jproperty

Allows user to specify a comma separated list of supported JVM arguments. This comprises those flags which begin with -D or the -verbose flag. Flags starting with -X are JVM implementation dependent and may or may not work.

For example,

JProperties=-Xms2M,-Xmx4M,-verbose

Mandatory: No

idAttributes

Allows the user to specify a way of uniquely identifying several mbeans, each of which were located using a common mbean expression.

idAttributes > idAttribute

Used to to create an id to identify each of several mbeans which have no uniquely identifying attribute. For example, suppose you have an MBean alias.

$1 = java.lang:type=Geneos, *

which evaluates to 2 Mbeans with these objectnames

java.lang:type=Geneos,host=WinBox, port=3344, os=Vista
java.lang:type=Geneos,host=SunBox, port=4455, os=Solaris5.8

Now suppose that the mbeans have no attribute which uniquely identifies them. How would the user differentiate between each? Ideally we would like to have a leading column in the dataview called Id, that hold a unique id for each mbean. Since an objectname uniquely identifies each mbean, we would like to compose this id using values from that objectname. We don't want to automatically use the objectname as the id, since some objectnames are very, very long !

If you now specify 3 idAttribute instances with the names os, host and port, an Id column will now be displayed in the dataview. Each entry in that column will contain an id which uniquely identifies each row. This id will be created by conjoining the values matching os, host and port in that mbean's objectname.

So in our example, we would end up with an Id column which looks like this

Id
Vista/WinBox/3344
Solaris5.8/SunBox/4455

We recommend using the IdAttribute parameter if you use an MBean alias which evaluates to multiple Mbeans and those MBean do not have a unique attribute suitable for distinguishing one from the other.

exclusionList

A list of comma-delimited key-value pairs which are matched against the key-value pairs of the objectnames of each of the returned mbeans. If all the key-value pairs are matched, that mbean's attributes are not published.

For example, suppose you have an MBean alias

$1 = earth.insect:type=Ant, *

which evaluates to 3 Mbeans with these objectnames:

earth.insect:type=Ant, name=Pharaoh
earth.insect:type=Ant, name=Fire
earth.insect:type=Ant, name=Thief*

and you specify the following exclusion list

ExlusionList
name=Thief
name=Fire*

Then only the following mbeans are eligible to be published:

earth.insect:type=Ant, name=Pharaoh*

Mandatory: No

private logJavaMemoryInfo

Whether Java memory consumption statistics should be written to the netprobe log file or not. This is useful if you suspect that the plug-in is consuming large amounts of memory and you need to ascertain whether this is happening in the JVM or in the netprobe.

Mandatory: No

Attribute Paths

Attribute Paths can be used within the MBean alias > alias parameter and within the rows > row > value or columns > column > rowTemplate parameters. An attribute path is a drill down into attributes and sub-attributes of a bean using a dot-delimited path.

E.g. a.b.c

Collection Filters

Where an attribute is a collection type (e.g. TabularData or CompositeData) its children can be reduced to a sub-set using a collection filter. These objects can be reference by either index or search filters. An example of these types is represented using the java.lang.Runtime object's SystemProperties attribute. Using Java's JConsole, the following represents that object.

SystemProperties attribute of java.lang.Runtime is an instance of TabularDataSupport, which is an instance based on java.util.Map. This property contains a key-value pair.

The above object and attribute will be used in the example as we describe the options on how to access TabularDataSupport using the attributePath.

The following Collection Filters are supported:

• [*] — returns all sub-objects.
• [match(attribute, 're')] — returns all sub-objects where the specified attribute matches a regular expression.

For example, getting the value with key of java.class.version, first define the Alias with attributePath that filters the SystemProperties using the match format:

Regular expression must be enclosed with either single or double quote. You can now reference the value or key attribute of the item to be returned:

Having these setup the output in the metrics view should display the '50' as the value.

You can search not only by key, but using either of all attributes. For example, rather than key search the TabularDataSupport by value:

Having the example above as configured for the attibutePath we can access the item and display the key instead.

• [attribute = 'value']: Lookup child object where the specified attribute exactly matches a regular expression.

Example use of this method is as follows:

This should output the value that matches the whole string of 'java.runtime.version':

Look up or filter by value or any other attribute can also be used in this option.

• [#num]: Lookup an element in a container by index number. Index starts with zero (0).

Typically the collection filter should map to only one sub-object within a row > value or rowTemplate expression. In the case that it maps to multiple objects only the first will be used. Within an MBean Alias where a Column Style setup is used, a filter may well map to multiple objects.

function($1.attributePath)

 <space> . / ~ ! @ # $ % ^ & * ( ) { } | \ _ + ` - = ; < > ? '

$1.number of partitions

$1."number of partitions"

<space> . / ~ ! @ # $ % ^ & * ( ) { } | \ _ + ` - = ; < > ? '

weblogic.jar
cryptoj.jar
EccpressoCore.jar

weblogic.jar
cryptoj.jar

C:\> java -jar geneos-plugins.jar jmx WebLogicJmxApi t3://192.168.10.200:7011 weblogic weblogic

C:\> java -jar geneos-plugins.jar jmx WebLogic t3://192.168.10.200:7011 weblogic weblogic

<sampler name="2-3-4">
    <plugin>
        <jmx-server>
            <connection>
                <weblogic>
                    <java.naming.provider.url>
                        <data>t3://192.168.10.200:7011</data>
                    </java.naming.provider.url>
                    <java.naming.security.principal>
                        <data>weblogic</data>
                    </java.naming.security.principal>
                    <java.naming.security.credentials>
                        <data>weblogic</data>
                    </java.naming.security.credentials>
                    <sslDetails>
                    </sslDetails>
                </weblogic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>com.bea: Name=MedRecServer,Type=ServerRuntime</data>
                    </value>
                </alias>
            </aliases>
            <rows>
                <columnLabels>
                    <label>
                        <data>Attribute</data>
                    </label>
                    <label>
                        <data>Value</data>
                    </label>
                </columnLabels>
                <row>
                    <value>
                        <data>HealthState,$1.HealthState</data>
                    </value>
                </row>
            </rows>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

<sampler name="2-3-5">
    <plugin>
        <jmx-server>
            <connection>
                <weblogic>
                    <java.naming.provider.url>
                        <data>t3://192.168.10.200:7011</data>
                    </java.naming.provider.url>
                    <java.naming.security.principal>
                        <data>weblogic</data>
                    </java.naming.security.principal>
                    <java.naming.security.credentials>
                        <data>weblogic</data>
                    </java.naming.security.credentials>
                    <sslDetails>
                    </sslDetails>
                </weblogic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>com.bea:Type=ExecuteQueueRuntime,*</data>
                    </value>
                </alias>
            </aliases>
            <columns>
                <column>
                    <label>Queue Name</label>
                    <rowTemplate>
                        <data>$1.Name</data>
                    </rowTemplate>
                </column>
                <column>
                    <label>Total Threads</label>
                    <rowTemplate>
                        <data>$1.ExecuteThreadTotalCount</data>
                    </rowTemplate>
                </column>
            </columns>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

Connect to a WebLogic server using an SSL-secured connection

C:\> java -jar geneos-plugins.jar jmx WebLogicJmxApi t3s://192.168.10.200:7002 weblogic weblogic clientkey.pem clientcert.pem keypass

C:\> java -jar geneos-plugins.jar jmx WebLogic t3s://192.168.10.200:7002 weblogic weblogic clientkey.pem clientcert.pem keypass

<connection>
        <weblogic>
                <java.naming.provider.url>
                        <data>t3s://192.168.10.200:7002</data>
                </java.naming.provider.url>
                <java.naming.security.principal>
                        <data>weblogic</data>
                </java.naming.security.principal>
                <java.naming.security.credentials>
                        <data>weblogic</data>
                </java.naming.security.credentials>
                <sslDetails>
                        <SSLClientCertificateFile>
                                <data>clientkey.pem</data>
                        </SSLClientCertificateFile>
                        <SSLClientCertificateKeyFile>
                                <data>clientcert.pem</data>
                        </SSLClientCertificateKeyFile>
                        <password>
                                <stdAES>+encs+F9F29CF67AFA80F117FA3961A3ED370D</stdAES>
                        </password>
                </sslDetails>
        </weblogic>
</connection>

Configuring Advanced JProperties settings

To connect to an SSL-secured JMX application, the plug-in needs to be configured with few jproperties, so that it can successfully connect to the application.

//Jproperties
-Dweblogic.home=\path\to\bea\wlserver_x.x\server
-Dweblogic.security.TrustKeyStore=DemoTrust

The weblogic home location is required because WebLogic searches for TrustKeyStore under weblogic_homelib folder. So, one can specify -Dweblogic.home=. and copy the TrustKeyStore file in the Netprobe folder. In above example, DemoTrust is the default trust keystore configured with Weblogic Server. It is not advisable to use this default Trust KeyStore. Instead, one should use the CustomTrust KeyStore and specify it in JProperties as:

//Jproperties
-Dweblogic.home=\path\to\bea\wlserver_x.x\server
-Dweblogic.security.TrustKeyStore=CustomTrust
-Dweblogic.security.CustomTrustKeyStoreFileName=<TrustKeyStoreFile>-Dweblogic.security.CustomTrustKeyStorePassPhrase=<password>

There are some optional settings that may be required depending upon the configuration:

//Optional Jproperties:
-Dweblogic.security.SSL.ignoreHostnameVerification=true

Above setting to be used to set off strict host name verification in case when one gets a Host name Verification failure error.

//Optional SSL debug Jproperties:
-Dssl.debug=true
-Dweblogic.StdoutDebugEnabled=true

These settings turn on extra SSL debug option and produce more output in the Netprobe log in case of any problem.

Connect to a Weblogic server using one-way SSL

To connect JMX plug-in to Weblogic using one-way SSL, follow these steps:

1. Create the wlfullclient.jar for JDK 1.6 client application.
2. Go to bea\wlserver\server\lib
3. Use this command to create wlfullclient.jar in the server/lib directory.

java -jar wljarbuilder.jar

4. Rename the created wlfullclient.jar to weblogic.jar
5. Copy the following files from bea\wlserver\server\lib to your Netprobe folder.

For Weblogic 10.3.6:

weblogic.jar
cryptoj.jar
EccpressoCore.jar

For Weblogic 12.1.1:

weblogic.jar
cryptoj.jar

6. Create a lib folder in your Netprobe directory.
7. Copy DemoTrust.jks from bea\wlserver\server\lib to netprobe\lib folder.
8. Define the following environment variables in your Netprobe startup script.

setenv JAVA_HOME /usr/java/jdk1.6.0_45/jre
setenv LD_LIBRARY_PATH $JAVA_HOME/lib/i386:$JAVA_HOME/lib/i386/client:/home/dev/netprobe:.
setenv PATH $JAVA_HOME/bin:$PATH

9. In Gateway Setup Editor, kindly add the following java arguments in JMX plug-in Advanced > Jproperties.

-Dweblogic.home=<netprobe_dir>

This argument tells where the lib folder can be located which contains the DemoTrust.jks file.

-Dweblogic.security.SSL.ignoreHostnameVerification=true

This argument tells that the server's host name will not be verified since we're using demo or self-signed certificates.

-Dweblogic.security.TrustKeyStore=DemoTrust

This argument is only used if you are using the demo certificates.

10. Run the Netprobe using start_netprobe script.

Connect to a WebSphere 8.5.5.x server

Note: Beginning Geneos 4.x.x, this plugin no longer supports connection on WebSphere 6.x. To view the versions that the JMX plugin supports, see Application and plug-in specific information in 5.x Compatibility Matrix.

setenv JAVA_HOME /usr/java/ibm-java-i386-60/jre
setenv LD_LIBRARY_PATH $JAVA_HOME/lib/i386/j9vm
setenv PATH $JAVA_HOME/bin:$PATH

C:\> java -jar geneos-plugins.jar jmx WebSphere 192.168.10.200 8880 websphere websphere SOAP

C:\> java -jar geneos-plugins.jar jmx WebSphere 192.168.10.200 8880 websphere websphere SOAP <clientTrustFile.jks> <clientTrustPassword> <clientKeyFile.jks> <clientKeyPassword>

<sampler name="WAS855_template">
    <plugin>
        <jmx-server>
            <connection>
                <websphere>
                    <host>
                        <var ref="WAS855_HOST"></var>
                    </host>
                    <port>
                        <var ref="WAS855_PORT"></var>
                    </port>
                    <type>
                        <var ref="WAS855_TYPE"></var>
                    </type>
                    <sslKeyStoreDetails></sslKeyStoreDetails>
                    <sslTrustStoreDetails></sslTrustStoreDetails>
                </websphere>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>WebSphere:name=JVM,process=nodeagent,platform=proxy,node=node01,j2eeType=JVM,J2EEServer=nodeagent,version=8.5.5.0,type=JVM,mbeanIdentifier=JVM,cell=cell01,spec=1.0</data>
                    </value>
                </alias>
            </aliases>
            <columns>
                <column>
                    <label>Java Vendor</label>
                    <rowTemplate>
                        <data>$1.javaVendor</data>
                    </rowTemplate>
                </column>
                <column>
                    <label>Java Version</label>
                    <rowTemplate>
                        <data>$1.javaVersion</data>
                    </rowTemplate>
                </column>
            </columns>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>pet>

C:\> java -jar geneos-plugins.jar jmx Generic rmi:///jndi/rmi 192.168.10.200 5003 4000 jmxrmi *:*

java    -Dcom.sun.management.jmxremote
        -Dcom.sun.management.jmxremote.port=5003
        -Dcom.sun.management.jmxremote.ssl=false
        -Dcom.sun.management.jmxremote.authenticate=false
        -jar MyJavaApplication.jar

C:\> java -jar geneos-plugins.jar jmx Generic rmi:///jndi/rmi 192.168.10.200 5003 4000 jmxrmi *:* kyoto origami

java -Dcom.sun.management.jmxremote -jar MyJavaApplication.jar XXX 5000

"C:\Program Files (x86)\Java\jre1.6.0_12\bin"\java
-Djava.library.path="C:\Program Files (x86)\Java\jre1.6.0_12\bin"
-Djava.home="C:\Program Files (x86)\Java\jre1.6.0_12"
-jar geneos-plugins.jar jmx Local XXX 5000

/usr/java/jre1.6.0_06/bin/java
-Djava.library.path=/usr/java/jre1.6.0_06/lib/i386
-Djava.home=/usr/java/jre1.6.0_06
-jar geneos-plugins.jar jmx Local XXX 5000

/usr/jre1.6.0_06/bin/java
-Djava.library.path=/usr/jre1.6.0_06/lib/sparc
-Djava.home=/usr/jre1.6.0_06
-jar geneos-plugins.jar jmx Local XXX 5000

<sampler name="2-5-4">
    <plugin>
        <jmx-server>
            <connection>
                <generic>
                    <serviceURL>
                        <data>service:jmx:rmi:///jndi/rmi://192.168.10.200:5003/jmxrmi</data>
                    </serviceURL>
                </generic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>java.lang:type=Runtime</data>
                    </value>
                </alias>
            </aliases>
            <rows>
                <columnLabels>
                    <label>
                        <data>ClassPath</data>
                    </label>
                    <label>
                        <data>LibraryPath</data>
                    </label>
                </columnLabels>
                <row>
                    <value>
                        <data>$1.ClassPath,$1.LibraryPath </data>
                    </value>
                </row>
            </rows>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

<sampler name="8096-win">
    <plugin>
        <jmx-server>
            <connection>
                <generic>
                    <jmxId>
                        <data>MyUniqueID</data>
                    </jmxId>
                </generic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>java.lang:type=Runtime</data>
                    </value>
                </alias>
            </aliases>
            <columns>
                <column>
                    <label>ClassPath</label>
                    <rowTemplate>
                        <data>$1.ClassPath</data>
                    </rowTemplate>
                </column>
                <column>
                    <label>LibraryPath</label>
                    <rowTemplate>
                        <data>$1.LibraryPath </data>
                    </rowTemplate>
                </column>
            </columns>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

<sampler name="2-5-5">
    <plugin>
        <jmx-server>
            <connection>
                <generic>
                    <serviceURL>
                        <data>service:jmx:rmi:///jndi/rmi://192.168.10.200:5003/jmxrmi</data>
                    </serviceURL>
                </generic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>java.lang:type=GarbageCollector,*</data>
                    </value>
                </alias>
            </aliases>
            <columns>
                <column>
                    <label>CollectionTime</label>
                    <rowTemplate>
                        <data>$1.CollectionTime</data>
                    </rowTemplate>
                </column>
                <column>
                    <label>CollectionCount</label>
                    <rowTemplate>
                        <data>$1.CollectionCount</data>
                    </rowTemplate>
                </column>
            </columns>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

Connecting to applications using password and access files

Java supports the use of password files and access files to restrict access to a jmx-enabled application. In this case the plugin also needs to provide a username and password before being allowed to connect to and retrieve information from the application.

We explain how to configure a sample Java application for authenticated access and then show how to configure the plugin to connect to that application. The instructions given are suitable for UNIX and LINUX.

Please see http://java.sun.com/j2se/1.5.0/docs/guide/management/security-windows.html for instructions on how to secure an application for windows.

Securing the application

The first thing to ensure is that we have a valid rolename-permission pair in the access file that resides inside the JRE installation directory i.e. $JRE/lib/management/jmxremote.access:

By default the following rolename-permission pairs exist:

monitorRole     readonly
controlRole     readwrite

So we add our own entry choosing kyoto as our rolename and granting it readwrite access. Our file should now look like this:

monitorRole     readonly
controlRole     readwrite
kyoto           readwrite

We then create a local password file called jmxremote.password in our current working directory for the java application. We need to chmod 600 the file otherwise Java will complain and not use the file. We then add our rolename along with origami as the password of our choice.

The jmxremote.password file should now look like this:

kyoto           origami

java    -Dcom.sun.management.jmxremote
        -Dcom.sun.management.jmxremote.port=5003
        -Dcom.sun.management.jmxremote.ssl=false
        -Dcom.sun.management.jmxremote.authenticate=true
        -Dcom.sun.management.jmxremote.password.file=jmxremote.password
        JmxServer

Finally we configure a jmx-generic plugin to connect to our secured application and retrieve information about the garbage collectors currently running in its JVM.

<sampler name="2-7-1">
    <plugin>
        <jmx-server>
            <connection>
                <generic>
                    <serviceURL>
                        <data>service:jmx:rmi:///jndi/rmi://192.168.10.200:5003/jmxrmi</data>
                    </serviceURL>
                    <username>
                        <data>kyoto</data>
                    </username>
                    <password>
                        <stdAES>+encs+DEB9AA8E492C4CFDD274DE66BDF08BFC</stdAES>
                    </password>
                </generic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>java.lang:type=GarbageCollector,*</data>
                    </value>
                </alias>
            </aliases>
            <columns>
                <column>
                    <label>CollectionTime</label>
                    <rowTemplate>
                        <data>$1.CollectionTime</data>
                    </rowTemplate>
                </column>
                <column>
                    <label>CollectionCount</label>
                    <rowTemplate>
                        <data>$1.CollectionCount</data>
                    </rowTemplate>
                </column>
            </columns>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

Connecting using an SSL-secured connection

To connect to an SSL-secured JMX application, the plug-in needs to be configured with specific jproperties, so that it can successfully connect to the application. These jproperties are used to configure the type of connection that the plug-in needs to create. A detailed description of these properties and what they mean can be found on Sun's web site which defines how to create SSL connections e.g.

http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html

// Assumes the following variables have been defined (e.g. in a Type)
// groupProtocol = protocol
// serviceUrlSsl = service:jmx:rmi:///jndi/rmi://localhost:19763/jmxrmi

<sampler name="withssl">
    <var-group>
        <var ref="groupProtocol"></var>
    </var-group>
    <sampleInterval>
        <data>5</data>
    </sampleInterval>
    <plugin>
        <jmx-server>
            <connection>
                <generic>
                    <serviceURL>
                        <var ref="serviceUrlSsl"></var>
                    </serviceURL>
                </generic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>java.lang:type=Runtime</data>
                    </value>
                    <showCommands>
                        <headlines>true</headlines>
                        <tableColumns>true</tableColumns>
                        <dataviews>true</dataviews>
                        <samplers>true</samplers>
                    </showCommands>
                </alias>
                <alias>
                    <name>$2</name>
                    <value>
                        <data>java.lang:type=Threading,*</data>
                    </value>
                    <showCommands>
                        <headlines>true</headlines>
                        <tableColumns>true</tableColumns>
                        <dataviews>true</dataviews>
                        <samplers>true</samplers>
                    </showCommands>
                </alias>
            </aliases>
            <rows>
                <columnLabels>
                    <label>
                        <data>ClassPath</data>
                    </label>
                    <label>
                        <data>LibraryPath</data>
                    </label>
                </columnLabels>
                <row>
                    <value>
                        <data>$1.ClassPath, $1.LibraryPath</data>
                    </value>
                </row>
            </rows>
            <jproperties>
                <jproperty>
                    <data>-Dcom.sun.management.jmxremote.ssl=true</data>
                </jproperty>
                <jproperty>
                    <data>-Dcom.sun.management.jmxremote.ssl.need.client.auth=true</data>
                </jproperty>
                <jproperty>
                    <data>-Djavax.net.ssl.keyStore=</data>
                    <var ref="sslDirectory"></var>
                    <data>\sslcertificates\jmxplugin\jmx.goodkeystore</data>
                </jproperty>
                <jproperty>
                    <data>-Djavax.net.ssl.keyStorePassword=plugpassword</data>
                </jproperty>
                <jproperty>
                    <data>-Djavax.net.ssl.trustStore=</data>
                    <var ref="sslDirectory"></var>
                    <data>\sslcertificates\jmxplugin\jmx.goodtruststore</data>
                </jproperty>
                <jproperty>
                    <data>-Djavax.net.ssl.trustStorePassword=plugtrustword</data>
                </jproperty>
            </jproperties>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

Connecting using the JMXMP protocol

Up until now we have been using the JMXRMI protocol with which to connect to an application to monitor it. However, we are also allowed to connect using other protocols. For example, another common protocol is JMXMP. To connect to an application which exposes its state via a connector that uses the JMXMP protocol we need to make two changes in the manner in which we connect to it.

Firstly, we need to specify the new protocol in the serviceURL sampler parameter e.g.

<connection>
        <generic>
                <serviceURL>
                        <var ref="serviceUrlSsl"></var>
                </serviceURL>
        </generic>
</connection>

Secondly, as additional classes are needed to establish this type of connection, we need to get some additional files. These can be downloaded from Sun's website. You need to download the latest JMX Remote API Reference Implementation and extract rmissl.jar, jmxremote.jar and jmxremote_optional.jar. These files should then be stored in the same directory as the netprobe.

Supported Connection Settings

The following values can be configured in the Settings section for Generic samplers:

• host - the ip address of resolved hostname of the machine on which the JMX application runs
• port - the port on which to connect to the JMX application
• urlpath - the urlpath to use when connecting to the JMX application
• timeout - milliseconds to wait for connections to be established or JMX commands to return. This setting should not be used unless it is found that the sampler seems to be waiting for inordinate amounts of time to connect to an application which is clearly down. This allows a timeout which the user decides is appropriate, for which to wait for a connection, before deciding that the application should be considered down.
• serviceUrl - combines the host, port and urlpath into a single URL. If this is used, then host, port and urlpath should not be used.
• JAVA_HOME - allows the user to specify the path upto where the JRE resides. This only needs to be specified, if you have started your application as a local application. Must be used in conjunction with JMXID. See section How to configure plug-in to connect to a Java application for an example set-up for Windows.
• JMXID - allows the user a way of identifying the local application to connect to. See section How to configure plug-in to connect to a Java application for an example set-up for Windows.

The official Sun Java documentation has the following to say about ServiceUrls:

A serviceUrl must conform to the following format:

The address is an Abstract Service URL for SLP, as defined in RFC 2609 and amended by RFC 3111.
It must look like this:

service:jmx:protocol:sap

Here, protocol is the transport protocol to be used to connect to the connector server. It is a
string of one or more ASCII characters, each of which is a letter, a digit, or one of the characters
+ or -. The first character must be a letter. Uppercase letters are converted into lowercase ones.

sap is the address at which the connector server is found. This address uses a subset of the syntax
defined by RFC 2609 for IP-based protocols. It is a subset because the user@host syntax is not
supported.

The other syntaxes defined by RFC 2609 are not currently supported by this class.

The supported syntax is:

//[host[:port]][url-path]

Square brackets [] indicate optional parts of the address. Not all protocols will recognize all
optional parts.

The host is a host name, an IPv4 numeric host address, or an IPv6 numeric address enclosed
in square brackets.

The port is a decimal port number. 0 means a default or anonymous port, depending on the protocol.

The host and port can be omitted. The port cannot be supplied without a host.

The url-path, if any, begins with a slash (/) or a semicolon (;) and continues to the end of the address.
It can contain attributes using the semicolon syntax specified in RFC 2609. Those attributes are not parsed
by this class and incorrect attribute syntax is not detected.

Although it is legal according to RFC 2609 to have a url-path that begins with a semicolon, not
all implementations of SLP allow it, so it is recommended to avoid that syntax.

Case is not significant in the initial service:jmx:protocol string or in the host part of the address.
Depending on the protocol, case can be significant in the url-path.

Arithmetic Expression Support

The user can compose symbolic expressions using the managed bean attributes which are numbers. These symbolic expressions are converted into arithmetic expressions and then evaluated to yield a final value.

There is also support for appending a string to the end of the final calculated result to aid understanding of what is being viewed.

Displaying non-atomic attributes

The plugin can only display sensible values for attributes which are primitives or reference type java.lang.Long or which are non-primitive but have sensible display representations provided for them by the implementer of the managed bean to which they belong.

However, the plugin also has built-in support for displaying the value of attributes which are not part of the set described above but which occur so commonly that we have made special provisions for them.

The examples below assume that the user has configured an instance of the plugin to connect to a Java application listening for jmx requests on port 5003 and demonstrates how the plugin treats some attributes in a special way.

<sampler name="2-9-1">
    <plugin>
        <jmx-server>
            <connection>
                <generic>
                    <serviceURL>
                        <data>service:jmx:rmi:///jndi/rmi://192.168.10.200:5003/jmxrmi</data>
                    </serviceURL>
                </generic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>java.lang:type=Threading</data>
                    </value>
                </alias>
                <alias>
                    <name>$2</name>
                    <value>
                        <data>java.lang:type=GarbageCollector,name=MarkSweepCompact</data>
                    </value>
                </alias>
            </aliases>
            <rows>
                <columnLabels>
                    <label>
                        <data>AllThreadIds</data>
                    </label>
                    <label>
                        <data>MemoryPoolNames</data>
                    </label>
                </columnLabels>
                <row>
                    <value>
                        <data>$1.AllThreadIds,$2.MemoryPoolNames</data>
                    </value>
                </row>
            </rows>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

<sampler name="2-9-2">
    <plugin>
        <jmx-server>
            <connection>
                <generic>
                    <serviceURL>
                        <data>service:jmx:rmi:///jndi/rmi://192.168.10.200:5003/jmxrmi</data>
                    </serviceURL>
                </generic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>java.lang:type=MemoryPool,*</data>
                    </value>
                </alias>
            </aliases>
            <columns>
                <column>
                    <label>PeakUsage</label>
                    <rowTemplate>
                        <data>$1.PeakUsage.committed</data>
                    </rowTemplate>
                </column>
                <column>
                    <label>MaxUsage</label>
                    <rowTemplate>
                        <data>$1.PeakUsage.max</data>
                    </rowTemplate>
                </column>
            </columns>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

Displaying attributes that are Maps

Another non-atomic data type would be Maps, or data that are structured using a unique key returning a value. Just like in regular attribute paths, Map members can be accessed two ways: first, by using the [key='keyName'] expression, or by using the [match(key,'keyNameExp')] expression, the latter which can accept filters. So for example, to access the managed bean java.lang:type=GarbageCollector,name=MarkSweepCompact and the attribute committed under the key Code Cache which is stored in the Map memoryUsageAfterGc which in turn is under LastGcInfo, the setup below can be used.

<sampler name="2-9-2">
    <plugin>
        <jmx-server>
            <connection>
                <generic>
                    <serviceURL>
                        <data>service:jmx:rmi:///jndi/rmi://192.168.10.200:5003/jmxrmi</data>
                    </serviceURL>
                </generic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>java.lang:type=GarbageCollector,name=MarkSweepCompact</data>
                    </value>
                </alias>
            </aliases>
            <columns>
                <column>
                    <label>Code Cache Committed</label>
                    <rowTemplate>
                        <data>$1.LastGcInfo.memoryUsageAfterGc[key=&apos;Code Cache&apos;].value.committed</data>
                    </rowTemplate>
                </column>
                <column>
                    <label>MaxUsage</label>
                    <rowTemplate>
                        <data>$1.PeakUsage.max</data>
                    </rowTemplate>
                </column>
            </columns>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

Using Format masks

Sometimes the retrieved value of an attribute is returned as a long but would really be more readable if it were formatted as a date. For example, in the managed bean with the name java.lang:type=Runtime there is an attribute called StartTime of type long. It represents the time when the Java virtual machine was started up as the number of milliseconds elapsed between the Epoch (00:00:00 GMT on January 1, 1970) and the point in time at which it was started. This is obviously not as intuitive as displaying the actual date at the virtual machine was started at.

The example below shows how to format this specific value as a date and ignore any other attributes being retrieved. It demonstrates how to do this for both headline values and table values.

<sampler name="2-10">
    <plugin>
        <jmx-server>
            <connection>
                <generic>
                    <serviceURL>
                        <data>service:jmx:rmi:///jndi/rmi://192.168.10.200:5003/jmxrmi</data>
                    </serviceURL>
                </generic>
            </connection>
            <aliases>
                <alias>
                    <name>$1</name>
                    <value>
                        <data>java.lang:type=Runtime</data>
                    </value>
                </alias>
            </aliases>
            <headlines>
                <headline>
                    <name>StartTime</name>
                    <value>
                        <data>$1.StartTime</data>
                    </value>
                    <mask>
                        <data>DATE</data>
                    </mask>
                </headline>
                <headline>
                    <name>Uptime</name>
                    <value>
                        <data>$1.Uptime</data>
                    </value>
                    <mask>
                        <data>IGNORE</data>
                    </mask>
                </headline>
            </headlines>
            <rows>
                <columnLabels>
                    <label>
                        <data>StartTime</data>
                    </label>
                    <label>
                        <data>Uptime</data>
                    </label>
                </columnLabels>
                <columnMasks>
                    <mask>
                        <data>DATE</data>
                    </mask>
                    <mask>
                        <data>IGNORE</data>
                    </mask>
                </columnMasks>
                <row>
                    <value>
                        <data>$1.StartTime,$1.Uptime</data>
                    </value>
                </row>
            </rows>
            <idAttributes></idAttributes>
        </jmx-server>
    </plugin>
</sampler>

This produces a view that may look like this

Built-in Format Masks

There are two types of Format Masks: Built-in Masks and Custom Masks. There are 5 built-in Masks included by default and can be readily be used. These are:

Custom Format Masks

Extracted data can sometimes have a different format compared to data handled by built-in Member and Stats masks. It can be data from JMX known objects or data outputted by custom beans. In order to have greater flexibility in handling different kinds of data, the capability to create custom Format Masks is available. Client developers can create their own masking by implementing the ITRS JMX Java Mask interface.

Below is the Mask interface:

/**
 * Copyright 2012 ITRS Group Ltd
 */
package com.itrsgroup.jmx.masks;

public interface Mask {

        /**
         * Returns the masked, formatted, or extracted data based on
         * {@link java.lang.Object} data and array of {@link String} arguments.
         * Arguments will be passed as found in the Mask field.
         *
         * @param data - Raw data from JMX
         * @param args - Array of {@link String} based arguments
         * @return String
         * @throws MaskException
         */
        String get(Object data, String[] args) throws MaskException;

}

Implementing Mask

When format masking is required and custom masking is defined, the plugin will call the get(Object, String[]) method of the Mask implementation.

The method will receive an Object which can be in the form of String object. This object is the representation of the value based on the configured Row or Column value.

The String[] of arguments will have the data inside the Mask parenthesis. If there are multiple arguments, it must be separated by using comma. Each comma delimited data will be passed as array item. For the example above, the args have a single item containing "Role" string on the first element.

The method get() must return String which will be the value of the dataview cell being masked.

As example, let us assume that the value will have this format:

"35,John,Engineer,Male"

We will be creating a mask for it, which requires to reference an index of the data from the comma delimited string literal. We expect that the first argument is the index of the data to be extracted.

package com.itrsgroup.jmx.masks;

public class CSV implements Mask
{
        public String get(Object data, String[] args) throws MaskException {

                int index = Integer.parseInt(args[0]);
                String csvData = (String) data;
                String[] splitData = csvData.split(",");
                return splitData[index];
        }
}

First, we need to implement the Mask interface, where we will be required to implement the get() method. In the implementation, we assume that the first index of the arguments is the required index of data to be extracted. On the succeeding lines we split the string literal into multiple strings using comma as separator. To make it easier we used the split function of the String object. Lastly, we return the string reference by the passed index argument.

We directly used the CSV mask in the Mask field. The argument passed is "1". In the example above the CSV class name was used as the Mask name, and anything else that is in between the parenthesis will be considered as arguments. It is the responsibility of the developer of the Mask to identify the arguments and validate them.

String[] splitData = csvData.split(",");
if(splitData.length < index)
{
        throw new MaskException("Invalid Index");
}
return splitData[index];

For example, if the index is beyond the actual size of the comma separated data, then it should throw a MaskException.

By default, the package name of the Mask should be com.itrsgroup.jmx.masks, if in any case another package name is given, the complete name should be configured in the Mask field, as shown below

Deploying Masks

All masks can be packaged as a separate Java Archive (jar), and must be deployed and copied on the same folder as the netprobe executable or the geneos-plugins.jar.

The archive will be automatically loaded on restarting the Netprobe and will be ready for use.