Google Stackdriver
EOL
This documentation is archived and will no longer be updated.
Since Geneos 6.5.x, the Google Cloud Platform Collection Agent plugin has replaced the Google Stackdriver integration.
Overview Copied
Google Stackdriver integration enables you to retrieve all metrics and metadata available in your Stackdriver setup and display them as dataviews in Geneos.
The integration provides a Gateway configuration file that enables monitoring of Stackdriver through a set of pre-configured Toolkit samplers. You can query and list the available metrics for your Google Cloud environment and running instances.
This guide walks you through the integration process to get the Toolkit sampler running with default Stackdriver views.
Audience Copied
This user guide is intended for experienced Geneos users who want to set up and configure an integration between Geneos and Stackdriver. This is also intended for Geneos users who are monitoring data from Stackdriver.
As a user, you should be familiar with Stackdriver as well as the Google Cloud Platform (GCP) services.
Prerequisites Copied
- Python 3.6 or higher.
- Python package manager (pip).
- Google API Client for Python.
- Google Authentication Library.
- Google Cloud Platform service account key in a form of a JSON file.
- This requires an additional licence to use. Please contact your ITRS Account Manager for more information.
Installation procedure Copied
The Stackdriver integration is largely pre-configured and involves three primary steps:
- Download the integration files from the resources site.
- Load the integration file in the Gateway Setup Editor.
- Configure the environment.
- Associate the samplers with a managed entity.
Note
The include files are intended to be read-only and should not be edited (unless the integration specifically requires for them to be modified). This is because include files can be shared between a few instances of a Gateway and best practice is to share them in their default state. It is also to make the upgrades and maintenance process of the integration files easier. All configuration should be carried out in your local Gateway files instead.
Download the integration files Copied
- Download the integration package
geneos-integration-gcp-<version>.zip
from the Downloads site. - As a best practice, save the files on the same machine where your Gateway is running.
The integration package includes a number of files. These are the files that determine the integration setup:
include/gcpMonitoring.xml
— Gateway configuration file (also called an include file) containing Toolkit samplers configured for Stackdriver integration.scripts/list_instances.py
— Python script that calls Stackdriver API and return a JSON format. The JSON values are then processed in a CSV format and displayed in Active Console. This displays available metrics for your services.scripts/requirements.txt
— list of the required Google libraries. They can be installed by running the pip command against this requirements file:
pip install -r requirements.txt
Load the integration files Copied
To load the integration files into the Gateway Setup Editor, follow these steps:
- Open Gateway Setup Editor.
- In the Navigation panel, click Includes to create a new file.
- Enter the location of the file to include in the Location field. In this example, it is the
include/gcpMonitoring.xml
. - Update the Priority field. This can be any value except
1
. If you input a priority of1
, the Gateway Setup Editor will return an error. - Expand the file location in the Include section.
- Select Click to load.
- Click Yes to load the new include file.
Success
Gateway configuration now appears in the Includes section.
Environment variables Copied
The gpsMonitoring.xml
script provides the variables that are set in the Environments section. They are configured to ensure that the integration monitors your Stackdriver environment.
Variable | Type | Description |
---|---|---|
GOOGLE_APPLICATION_CREDENTIALS | String |
Path of the JSON file with your Google Cloud Platform service key. |
GOOGLE_MANUAL | Integer |
Frequency of sampling. If checks are executed manually, the value is shown as 0. Note: The default configuration of this integration is to perform manual sampling. This is because Google charges for each request that you make,. |
GOOGLE_PROJECT_ID | String |
ID of the Google project you are connecting to. |
GOOGLE_DRILL_METRIC | String | Key path of a specific metric used for creating dataviews of a single Stackdriver metric. |
GOOGLE_DRILL_VIEW | String | A sub-string used to match a set of metric key paths. Used for creating dataviews of multiple metrics. |
PYTHON_PATH | String | Path of the Python library located on the machine where the Netprobeis running. |
Configure the environment Copied
The default environment configuration is created by the gcpMonitoring.xml
script. You should create a new environment in your main Gateway setup based on the default one.
- In the Gateway Setup Editor, locate your Gateway under the Includes section.
- Copy the default
GCP
environment. - In the Gateway Setup Editor, create a new environment by right-clicking the Environment folder. Paste the copied environment.
- Rename the new environment to distinguish it from the default one. Example:
GCP_env
- Modify the environment variables according to your system.
Note
Standard variable scoping rules apply when variable reference is used in include files. Please take note of that when creating your own variables. For more information, see Variable Scoping and Resolution.
Associate the samplers with a managed entity Copied
After you have loaded the configuration file, the Stackdriver monitoring samplers become available for use. These samplers are associated with a pre-defined Type. To associate the samplers with a managed entity, follow these steps:
- In the Gateway Setup Editor, create a new managed entity by right-clicking the Managed entities folder.
- Under the Basic tab for the new managed entity, add a pre-configured
GCP
type. - Specify the environment associated with the samplers. This is the new environment that you created in your main Gateway setup. Example:
GCP_env
. - Apply the managed entity to the running netprobe.
- Under the Basic tab for the new managed entity, add a pre-configured
GCP
type. - Apply the managed entity to the running Netprobe.
- Click Validate current document to check your configuration.
- Click Save current document to apply the changes.
Success
After you save your changes to the managed entity, the Google Stackdriver dataviews appear under the managed entity on the state tree.
For more information about include files, see the Include files in Gateway Setup Editor.
Dataviews configuration Copied
This integration provides a number of default dataviews, but you can add more based on which metrics and services your Stackdriver is monitoring. The default StackDriver-Monitoring dataview lists all the available metrics and their respective key paths. These key paths are used to configure the aggregated mode and detail views.
Configure a sampler with a single metric Copied
To configure a sampler with a particular Stackdriver metric, use a GOOGLE_DRILL_METRIC
environment variable. The value of that variable is a key path of the metric you want to see the details of. Key paths are displayed by the StackDriver-Monitoring dataview.
Configure a sampler with multiple metrics Copied
To configure a sampler with a set of Stackdriver metrics, use a GOOGLE_DRILL_VIEW
environment variable. The value of that variable is a sub-string of the target key paths. Key paths are displayed by the StackDriver-Monitoring dataview.
For example, to display a set of metrics referring to CPU, you can use the compute.googleapis.com/instance/cpu
sub-string and it will cross reference the following metrics:
compute.googleapis.com/instance/cpu/reserved_cores
compute.googleapis.com/instance/cpu/usage_time
compute.googleapis.com/instance/cpu/utilization
Metrics and dataviews Copied
Stackdriver integration enables you to display all the metrics and running instances that are available in your Google Cloud Platform environment. The dataviews are divided into three categories:
- Configuration mode — dataview that displays all the available services and metrics along with their respective key paths. These key paths are used to configure the aggregated mode and detail views. This integration provides a default configuration mode dataview named StackDriver-Monitoring.
- Aggregated mode — dataviews that display service level views. In this mode, instances are always displayed as rows and all available metrics are displayed as columns. Example configuration to show all compute engine instances and available metrics:
key path = compute.googleapis.com
- Detail view — dataviews that display individual metrics such as CPU utilization or disk read bytes. The configuration uses key paths specific to the individual metric. Example configuration to only show a detailed view of CPU utilization for each instance of the compute engine.
key path = compute.googleapis.com/instance/cpu/utilization
The sections below list metrics and example dataviews available with the Stackdriver integration.
StackDriver-Monitoring Copied
This is a configuration mode sampler that displays all available metrics with their key paths and descriptions. These key paths are then used to create dataviews that display details of the selected metrics.
Headline field | Description |
---|---|
execDuration |
Elapsed time for the Toolkit plug-in to run the script. Unit: seconds |
Column Name | Description |
---|---|
Display Name | Name of the available metric. |
Description | Description of the metric. |
Key Path |
Key path of the specific metric. The key path is used to drill down to more details of the selected metric. |
StackDriver-GoogleComputeEngine Copied
This sampler displays an aggregated view of all metrics and their values for each instance. To see details of the metrics you can create more aggregated views or detail views. Sections below present examples of such views.
StackDriver-CPU Copied
This sampler displays an aggregated view of all CPU instances. To see the detailed metrics of each instance, use the detail views.
StackDriver-CPU-UsageTime Copied
This sampler displays a detail view of the CPU instances providing metrics of the usage time.
StackDriver-CPU-Utilization Copied
This sampler displays a detail view of the CPU instances providing metrics of the CPU utilization.
StackDriver-Disk Copied
This sampler displays an aggregated view of all metrics available to monitor disks. To see the detailed values for each metric, use the detail views.
StackDriver-Disk-Read-Bytes Copied
This sampler displays a detail view of the read_bytes_count
metric for each instance and disk.
StackDriver-Firewall Copied
This sampler displays an aggregated view of all metrics available for firewall. To see the detailed values for each metric, use the detail views.
StackDriver-Firewall-Dropped-Bytes Copied
This sampler displays a detail view of the dropped_bytes_count
metric for each instance.
StackDriver-Firewall-Dropped-Packets Copied
This sampler displays a detail view of the dropped_packets_count
metric for each instance.