Collection Agent setup

Overview Copied

This guide outlines the step-by-step procedures you need to start running the Collection Agent in a standard environment.

Caution

Make sure to read Upgrading Collection Agent, which outlines the breaking changes that may have an impact on your upgrade, before you upgrade to the latest version of Geneos and Collection Agent.

For information on running the Collection Agent in an orchestrated environment, see Data collection in an orchestrated environment.

Note

Beginning Geneos 5.1, the Collection Agent is included in the Netprobe binaries for Windows and generic Linux platforms.

Prerequisites Copied

System requirements Copied

The following requirements must be met on the machine where the Netprobe and Collection Agent are installed:

Java requirements Copied

Geneos components requirements Copied

Setup and configuration Copied

To set up and configure your Collection Agent, do the following.

  1. Configure your Collection Agent depending on how you want to run it. You can run the Collection Agent in two ways:

    Regardless of how you run your Collection Agent, it will always run as a Java process.

  2. Create Dynamic Entities

  3. Add Dynamic Entities in your Netprobe

Run Collection Agent via Netprobe Copied

Caution

Collection Agent and its plugins is no longer packaged with the Netprobe in Geneos 5.14.7 and the subsequent 5.x versions. If you want to run Collection Agent via Netprobe, please upgrade to the current 6.x version of Geneos.

You can run the Collection Agent with Netprobes running in the normal, floating, or self-announcing modes on the same host.

To run the Netprobe and the Collection Agent on different hosts, you must Run Collection Agent as a stand-alone Java process.

To run the Collection Agent via Netprobe, follow these steps:

  1. In Gateway Setup Editor, click Dynamic entities > Collection agent parameters, and then click New Collection agent parameters.

  2. Select Managed in the Collection Agent dropdown list.

  3. Choose your preferred YAML type in the Yaml dropdown list and configure your Collection Agent parameters.

    Collection Agent parameters

    YAML type Configure your Collection Agent in
    Bundled YAML

    the collection_agent/collection-agent.yml packaged with the Netprobe binary.
    Custom YAML the specified YAML file that can have any name. The file can be in any folder but if you are using a relative path, then it should be relative to the working directory.
    Managed YAML the Gateway Setup Editor. For more information, see Managed Collection Agent in Dynamic Entities.

    Note

    If you are starting the Collection Agent using the Netprobe, all the ports configured in the Gateway Setup Editor will automatically apply to the running Collection Agent.

  4. Configure your plugins.

    YAML type Configure your plugins by
    Bundled YAML

    adding the parameters of the desired plugins in the collection_agent/collection-agent.yml packaged with the Netprobe binary.
    Custom YAML

    adding the parameters of the desired plugins in the specified YAML file in Step 3.
    Managed YAML adding the parameters in the Dynamic Entities > Collectors section in the Gateway Setup Editor. For more configuration information, see Collectors in Dynamic Entities.

    Note

    If you need to set up a new plugin that is not yet included in the package, you can download it from the ITRS Downlaods and add it to the collection_agent directory.

Self-Announcing Netprobe Copied

For Self-Announcing Netprobes, you can start the Collection Agent by adding the following configuration in the selfAnnounce settings in the Netprobe setup file:


<dynamicEntities>
 <collectionAgentParameters>CollectionAgentParams</collectionAgentParameters>
 <mappingType>DynamicEntitiesMapping</mappingType>
</dynamicEntities>

For more information, see Self-announce settings in Netprobe setup.

Note

 You still have to Create Dynamic Entities and make sure of the following:

  • The value inside the collectionAgentParameters tag in the XML above is the name of your Collection agent parameters in Gateway Setup Editor.
  • The value inside the mappingType tag in the XML above is the name of your Mapping type in Gateway Setup Editor.

Secure Collection Agent Copied

To use a secure connection between a Netprobe (normal, floating, or self-announcing modes) and Collection Agent, enable TLS using the secure command line option. Additionally, set the relevant SSL files on the command line. For more information, see Netprobe Command-line Options.

Note

Secure Collection Agent does not support Bundled YAML. Use Managed YAML or Custom YAML instead.

Run Collection Agent as a stand-alone Java process Copied

Running the Collection Agent as a stand-alone Java process allows you to run the Netprobe and the Collection Agent on different hosts.

If you wish to run the Netprobe and the Collection Agent on the same host, you must Run Collection Agent via Netprobe.

To run Collection Agent as a stand-alone Java process, follow these steps:

  1. Download the Collection Agent jar file, collection-agent-<version>-exec.jar, from the ITRS Downloads.

  2. Create a configuration file in YAML format, for example, collection-agent.yml. Ensure the following:

    • The TCP reporter fields are configured:
      • hostname: - host where the Netprobe is running.
      • port: - a unique port number.
    • The environment variables in the collection-agent.yml are defined on your host.
    pluginDirectory: ${env:CA_PLUGIN_DIR}
monitoring:
  healthCheck:
    listenPort: ${env:HEALTH_CHECK_PORT}
  metrics:
    dimensions:
      app_name: collection-agent
reporters:
  - type: tcp
    name: tcp
    hostname: localhost
    port: ${env:TCP_REPORTER_PORT}
collectors:
  - name: statsd
    type: plugin
    className: StatsdServer
workflow:
  storeDirectory: .
  metrics:
    reporter: tcp
  events:
    reporter: tcp

    Note

    To run a secure Collection Agent, add the tlsConfig under reporters configuration of collection-agent.yml. Ensure that Netprobe is also running securely. For more information on configuring the Collection Agent, see Collection Agent configuration reference.

  3. In Gateway Setup Editor, click Dynamic entities > Collection agent parameters, and then click New Collection agent parameters.

  4. Select Unmanaged in the Collection Agent dropdown list.

  5. Input the port value from your YAML file in the Reporter port field.

Collection Agent parameters

  1. If you need to set up a new plugin, download it from the ITRS Downloads and put it in the same directory as collection-agent-<version>-exec.jar.

  2. Run the Collection Agent using the following command:

    java -jar collection-agent-<version>-exec.jar collection-agent.yml

  3. To stop the Collection Agent, stop the Java process you started in Step 7.

    Note

    The only way to start or to stop the Collection Agent as a stand-alone Java process is to run the Java command in Step 8 and to stop the Java process in the terminal, respectively.

Create Dynamic Entities Copied

Netprobes receive datapoints from Collection Agents. Each datapoint can have a different set of dimensions, attributes, and properties. You should create mappings to construct a Geneos hierarchy from datapoints. This Geneos hierarchy determines how data is displayed in your Active Console. You should also create a mapping type, which allows you to easily deploy the same configuration on multiple probes.

  1. Open Gateway Setup Editor.
  2. Click Dynamic entities > Mapping, and then click New Mapping.
  3. Enter a mapping name in the Name field.
  4. In Options, choose custom or builtin. For more information, see Dynamic entities > Mapping > Options in Dynamic Entities.

    Note

    A built-in Collection Agent V3 mapping is packaged with the Netprobe, which automatically sets up the default mappings needed to display the Collection Agent self-monitoring metrics.

  1. Click Dynamic entities > Mapping types, and then click New Mapping type.
  2. If you configured a managed Collection Agent with a managed YAML type in the previous steps, then in the Collectors field, click Add new to add the name of the collectors you created in Run Collection Agent via Netprobe.
  3. In the Mappings field, click Add new to add the name of the mapping created in Step 4.

Mapping type

  1. To monitor the logs collected by the Collection Agent plugin, you can add an FKM plugin sampler in the Samplers field. The FKM plugin consumes these logs as streams. To configure your FKM plugin to monitor these streams, see files > file > source > stream in configuration.

Add Dynamic Entities in your Netprobe Copied

  1. Open Gateway Setup Editor.
  2. In Probes, select your Netprobe and go to the Dynamic Entities tab.
  3. Choose a mapping type in the Mapping type dropdown list.
  4. Choose a collection agent parameter in the Collection agent parameters dropdown list.

Collection agent parameters

Note

If mappings are defined in both the Mapping type and in the Collector, then the Netprobe will apply the mappings defined in the Mapping type first then the mappings defined in the Collectors.

Collection Agent self-monitoring Metrics Copied

If you started the Collection Agent with the built-in Collection Agent V3 mapping, then the Collection Agent self-monitoring metrics are displayed in your Active Console.

Dynamic Entities Health Copied

The plugin monitors the performance of and communication. It tracks the state of all metrics received by a from the , and all associated with a .

For guidance on how to set up the Dynamic Entities Health sampler, see Dynamic Entities Health.

Collection Agent log file Copied

The Collection Agent uses a separate log file from the Netprobe. The log settings for Collection Agent use Logback and are stored in logback.xml.

For example, you want to set the Logback with the following limitations:

You can use the following configuration:


<appender name="FILE" class="ch.qos.logback.core.FileAppender">
  <file>${COLLECTION_AGENT_DIR:-.}/collection-agent.log</file>
  <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
    <!-- rollover daily -->
    <fileNamePattern>mylog-%d{yyyy-MM-dd}.%i.txt</fileNamePattern>
     <!-- each file should be at most 100MB, keep 60 days worth of history, but at most 20GB -->
     <maxFileSize>100MB</maxFileSize>
     <maxHistory>60</maxHistory>
     <totalSizeCap>20GB</totalSizeCap>
  </rollingPolicy>

For more information on the other available options for modifying the logger settings, see Logback.

By default, the Collection Agent log file is stored in the collection-agent.log of the working directory. If you are running a managed Collection Agent and you configured the Log directory parameter in GSE, then the collection-agent.log will be in that specified location.

Connection and lifecycle Copied

When run with the Netprobe, Collection Agent runs as a Java process alongside the Netprobe process.

Common failure modes Copied

The Collection Agent will not start in the following scenarios:

Netprobe connection Copied

The following describes the behaviour of the Netprobe connection depending on the state of the Collection Agent:

Note

A Collection Agent started independently of the Netprobe is not managed by the Netprobe.

Health checks Copied

The Netprobe periodically pings the managed Collection Agent through the defined health check port.

The health checks are governed by the following variables:

Name Description
CA_HEALTHCHECK_INTERVAL Maximum number of attempts that the Netprobe makes to contact a managed Collection Agent.

Default value: 3

CA_MAX_HEALTHCHECK_ATTEMPTS Maximum number of attempts that the Netprobe makes to contact a managed Collection Agent.

Default value: 3

CA_HEALTHCHECK_TIMEOUT Time in seconds that the Netprobe waits for a response from the Collection Agent during a health check.

Default value: 5

CA_MAX_RESTARTS Maximum number of attempts that the makes to restart an unresponsive managed .

If the Netprobe fails to communicate with the managed Collection Agent after the maximum number of attempts, then the Netprobe stops monitoring the managed Collection Agent.

Default value: 3

["Geneos"] ["Geneos > Netprobe"] ["User Guide"]

Was this topic helpful?