Gateway Hub 1.x Release Notes
Gateway Hub 1.8.0 Copied
Released: March 2020
This release delivers the updated binaries for in the ITRS Downloads page.
Highlights Copied
These are the highlights of this release:
- Application keys.
- Grafana Datasource plugin.
- Event browser.
Application keys Copied
You can now generate Application keys that allow you to connect third party applications to Gateway Hub securely.
Application keys are designed to be used by a single application and can be centrally revoked if required.
Grafana Data source Copied
The Grafana data source has been released as a full feature. The Geneos data source for Grafana lets you seamlessly integrate data from Gateway Hub into your Grafana dashboards.
Event browser Copied
You can now identify alerting irregularities using the new Events page in the Web Console. The Events page allows you to visualise events generated over time, enabling noisy entities to be identified and remedied.
New features and enhancements Copied
These are the new features and enhancements of this release:
Issue key | Release description |
---|---|
STARK-12613 | The Apache Commons Compress library has been upgraded to prevent a security vulnerability. |
STARK-12594 | The Jackson Databind library has been upgraded to prevent a security vulnerability. |
STARK-12384 | Netty has been upgraded to prevent a security vulnerability. |
STARK-12346 | Webhooks are now fired correctly when Kafka attempts to commit a batch. |
STARK-12305 | The Gateway Hub sizing tool is now available as a separate download. |
STARK-12300 | You can now filter the list of entity attributes and possible values by the deletedEntities parameter. |
STARK-12289 | Gateway Hub can now ingest much larger dataviews without having to increase Kafka’s maximum message size. |
STARK-12194 | The UI controls for historical charts are now significantly more compact. |
STARK-12187 | The hubcheck script now checks that custom ports are free. |
STARK-12129 | Gateway Hub will now retain a maximum of 14 archived logs OR 1GB of log files, whichever is least. Logs are rolled (archived) each day or when the file size reaches 250MB. |
STARK-11937 | You can now specify the retention policy for Apache Spark logs. This can help conserve disk space. |
STARK-11835 | The Webconsole now shows a status bar when Webhooks are active. |
STARK-11806 | The historical metric chart now shows data from a 24 hour time window by default. |
STARK-11778 | You only need to specify the SSL password once during installation. |
STARK-11183 | When creating chained entity filters Gateway Hub now constrains the possible values of each new filter based on the preceding filters. |
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-12460 | Gateway Hub now includes additional units of measurement. |
STARK-12401 | Gateway Hub now correctly handles errors when an expired oauth token is passed with an API request. |
STARK-12388 | Resources for Spark are now calculated correctly. |
STARK-12168 | Metric snapshot performance has been improved by running clean-up tasks more frequently. |
STARK-12148 | Significantly improved the performance of real-time updates to metrics, particularly when new metrics are added to an entity. |
STARK-11971 | You can now perform upgrades on installations that are using custom ports. |
STARK-11879 | The hubcheck script now ensures that disks are correctly shared. |
STARK-11841 | Upgrades using a non-default user now work correctly. |
STARK-11805 | The weekly view chart now calculates the time range correctly. |
STARK-11692 | The historical metric chart now correctly performs Y-axis rounding. |
STARK-11689 | Historical metric data crossing the Y-axis is now displayed correctly after panning. |
STARK-11663 | Ingestion Errors link is now styled correctly. |
STARK-11609 | The Web Console Home page no longer displays dual scrollbars in Firefox. |
STARK-11069 | The entity filter drop-down list is no longer obscured. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-11411 | The Archiver log might show an error the very first time the Archiver runs. The Archiver should heal and run successfully the next time it runs. |
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name, for example, “test;”. To resolve this issue update the dataview column name removing the semicolon. |
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.7.3 Copied
Released: 13 February 2020
Note
Gateway Hub 1.7.3 is a patch release containing a number of fixes for security and deployment issues.
This release delivers the updated binaries for in the ITRS Downloads page.
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-12458 | Gateway Hub now correctly calculates the required memory allocation for YARN during installation. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-11411 | The Archiver log might show an error the very first time the Archiver runs. The Archiver should heal and run successfully the next time it runs. |
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name, for example, “test;”. To resolve this issue update the dataview column nameremoving the semicolon. |
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.7.2 Copied
Released: 1 February 2020
Note
Gateway Hub 1.7.2 is a patch release containing a number of fixes for security and deployment issues.
This release delivers the updated binaries for in the ITRS Downloads page.
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-12289 | Gateway Hub can now ingest much larger dataviews without having to increase Kafka’s maximum message size. |
STARK-12283 | You can now configure Kafka’s maximum message size for persistent entity snapshots. Setting the maximum message size to a higher value than the default of 1 MB may be necessary when working with very large dataviews (in excess of 20,000 cells). |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-11411 | The Archiver log might show an error the very first time the Archiver runs. The Archiver should heal and run successfully the next time it runs. |
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name, for example, “test;”. To resolve this issue update the dataview column name removing the semicolon. |
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.7.1 Copied
Released: 23 January 2020
Note
Gateway Hub 1.7.1 is a patch release containing a number of fixes for security and deployment issues.
This release delivers the updated binaries for in the ITRS Downloads page.
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-12168 | Significantly improved metric snapshot performance by running clean-up tasks more frequently. |
STARK-12148 | Significantly improved the performance of real-time updates to metrics, particularly when new metrics are added to an entity. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-11411 | The Archiver log might show an error the very first time the Archiver runs. The Archiver should heal and run successfully the next time it runs. |
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name, for example, “test;”. To resolve this issue update the dataview column name removing the semicolon. |
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.7.0 Copied
Released: 17 January 2020
This release delivers the updated binaries for in the ITRS Downloads page.
Highlights Copied
These are the highlights of this release:
- You can now use Webhooks to publish Geneos events to external services like ticketing systems.
New features and enhancements Copied
These are the new features and enhancements of this release:
Issue key | Release description |
---|---|
STARK-11947 | You can now configure Gateway Hub TLS/SSL using a PEM file. If a PEM file is not provided, Gateway Hub will default to using self-signed certificates. |
STARK-11739 | Improved error handling in Gateway Hub dataset generation. |
STARK-11647 | You can now use Webhooks to publish Geneos events to external services like ticketing systems. |
STARK-11638 | The hubcheck tool no longer asks for a password twice when creating ssh connections. |
STARK-11203 | Simplified the controls for the metric history chart. |
STARK-11202 | Improved date selection controls for the metric history chart. |
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-12073 | Gateway Hub now correctly updates the snapshot daemon and Kafka client when upgrading from a configuration using a non-default password for SSL. |
STARK-11702 | Certificates are no longer stored in the /tmp folder after configuration is completed. |
STARK-11635 | You can now re-run hubctl setup ssl on a failed ssl setup. |
STARK-11447 | The hubcheck logs are now generated in the correct location. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-11973 | Kafka depends upon Apache Log4j 1.2.17 which has known vulnerabilities. |
STARK-11723 | BlackDuck scans show a critical risk in Apache Log4j if used for remote logging. Our investigation shows that Gateway Hub and MapR Control System do not use Apache Log4j remote logging and so are not affected by this vulnerability. |
STARK-11411 | The Archiver log might show an error the very first time the Archiver runs. The Archiver should heal and run successfully the next time it runs. |
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name, for example, “test;”. To resolve this issue update the dataview column name removing the semicolon. |
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.6.0 Copied
Released: 12 December 2019
This release delivers the updated binaries for in the ITRS Downloads page.
Highlights Copied
These are the highlights of this release:
- Web Console monitoring UI.
- Event query API.
- Grafana data source (Technology Preview).
- Centralised configuration validation.
Web Console monitoring UI Copied
The Web Console monitoring UI brings a brand new look for real-time monitoring, to supplement Active Console. The following is available in the Web Console:
- Entities, their statuses and related metrics.
- The ability to see the historical data for additional context and incident investigation when metric and event history is enabled in Gateway Hub.
The Web Console is available free of charge for all Geneos users.
Event query API Copied
Events such as severity, snooze and user assignment data can now be queried using an open API, providing access to the raw data for further analysis of trends and problem entities.
Centralised configuration validation Copied
Updates to centralised configuration files can now be validated against all Gateways dependent on that configuration, in order to avoid updates causing issues remotely. By default, changes that would cause errors in any dependent Gateways are refused.
The API for centralised configuration has been finalised since the early access version, and as a result all Gateways using GA centralised configuration must be version 5.0.
Breaking changes Copied
These are the breaking changes in this release:
Issue key | Release description |
---|---|
STARK-9578 | The API for centralised configuration has changed since Early Access. Upgrade to Gateway 5.0 to use centralised configuration with Gateway Hub 1.6 and above. |
New features and enhancements Copied
These are the new features and enhancements of this release:
Issue key | Release description |
---|---|
STARK-10951 | Metric and event endpoints now accept the Content type application/x-yaml. |
STARK-10830 | Basic authentication is refreshed more frequently, reducing the likelihood that the user will need to re-enter their credentials. |
STARK-10676 | Error messages now contain more detailed information. |
STARK-10674 | When security is disabled, Gateway Hub responds to GET requests to either of /security and /security/server/roles/mappings passing a Basic Authorization header with a 200 OK response. |
STARK-10556 | The Web Console now contains a User Profile page, which shows user details such as roles. |
STARK-10440 | The Web Console now remembers the open or closed state of the sidebar menu between sessions. |
STARK-10358 | New publishing filters can now be added using drop-down lists. |
STARK-9578 | Updated the centralised configuration API. |
STARK-8703, STARK-8391 | Changes to centralised configuration resources are now validated centrally. |
STARK-8390 | Added KPIs to the home dashboard showing:
|
STARK-7798 | Improved the scrollbar style. |
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-11157 | The description for CDF metric aggregation is now correct. |
STARK-10834 | The Web Console no longer supplies an empty diagnostic file. |
STARK-10817 | Improved the readability of 401 error screens when in dark mode. |
STARK-10744 | Application buttons are no longer shown on error pages when the user is not authenticated. |
STARK-10706 | Gateway Hub now correctly handles errors produced by incorrectly configuring the Kerberos settings. |
STARK-10701 | Gateway Hub now correctly handles an error when attempting to change roles with an expired token. |
STARK-10678 | Gateway Hub now returns a 400 error when an authorisation token expires. |
STARK-10616 | The Web Console will now show an error when you provide an invalid security configuration. |
STARK-10593 | Improved the error message when a user changes a role mapping that would remove their own admin access. |
STARK-10539 | The security endpoint no longer returns a 400 error when security is disabled. |
STARK-9515 | Data with a unit of measure assigned (for example, Kilobytes ) is no longer published in the canonical unit (for example, Bytes ). The values are published in the form originally set by the Gateway. |
STARK-9665 | Long error messages are now correctly formatted in the system and ingestion error page sidebars. |
STARK-6928 | The hubctl setup licence command no longer fails silently when a licence file is omitted. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-11411 | The Archiver log might show an error the very first time the Archiver runs. The Archiver should heal and run successfully the next time it runs. |
STARK-8239 | BlackDuck scan shows a Critical risk in log4j if used for remote logging. Our investigation shows that Gateway Hub and MapR do not use log4j remote logging and so are not affected by this vulnerability. |
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name e.g. “test;”. To resolve this issue update the dataview column name removing the semicolon. |
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.5.4 Copied
Released: 13 November, 2019
Note
Gateway Hub 1.5.4 is a patch release containing a number of fixes for security and deployment issues. Versions 1.5.1 — 1.5.3 were only released internally.
This release delivers the updated binaries for in the ITRS Downloads page.
New features and enhancements Copied
These are the new features and enhancements of this release:
Issue key | Release description |
---|---|
STARK-11306 | Upgraded to Kafka 2.3.1. |
STARK-11226 | It is now possible to allow the installer to do a self-installation without needing to use sudo and using a root user. |
STARK-11032 | Upgraded to jackson-databind library 2.10. |
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-11327 | The Node Manager did not start if /tmp had noexec set. This is was fixed by the addition of a new setting in the hubctl JSON configuration: hub.install.java_io_tmpdiz . It should point to a directory that is mounted on a disk that does not have noexec set. |
STARK-10046 | Fixed an issue caused by a recent security update to systemd, that resulted in multiple copies of Warden and Gateway Hub services to be run and deployment to fail. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-10849 | Web Dashboard is not compatible with SSO Agent in Gateway Hub. This will be addressed in the next release. |
STARK-10831 | Webslinger is not compatible with SSO Agent in Gateway Hub. This will be addressed in the next release. |
STARK-10046 | Gateway Hub is not compatible with systemd version 219-65 due to an issue with the underlyingMapR platform. A previous version of systemd is required. |
STARK-9515 | Data with a unit of measure assigned (for example, Kilobytes ) is currently published in the canonical unit (for example, Bytes ). This means that certain metrics may have a misleading name (for example, KilobytesInQueue ). |
STARK-7596 | The SSL certificate install incorrectly uses the default ‘hub’ user when applying an SSL certificate. To specify a different install user please add the following to the hubctl installation file:
|
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name e.g. “test;”. To resolve this issue update the dataview column name removing the semicolon. |
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.5.0 Copied
Released: 8 October, 2019
This release delivers the updated binaries for in the ITRS Downloads page.
Highlights Copied
These are the highlights of this release:
- Anomaly detection rules.
- Kafka publishing filter.
- Role-based security.
Anomaly detection rules Copied
One of the major sources of monitoring noise is due to simple thresholds set on metrics that are inherently variable over time. Many business processes follow a fairly predictable pattern, peaking around market open and close, and dormant over the weekends.
Anomaly detection rules provide a better way of monitoring these kinds of metrics. By analysing the history of a metric stored in Gateway Hub, an anomaly detection rule can determine what is normal, alerting only when this normality is deviated from.
For example, an anomaly detection rule can be set to trigger an alert if orders fall outside a certain tolerance for a given time and day of the week, for that specific server, or for all similar servers.
Switching to anomaly detection rules can result in significantly more accurate alerting and substantially reduced events over time for metrics that follow a pattern or cycle. Some metrics are more suited to anomaly detection rules than others.
For more information, see Anomaly Detection.
Kafka publishing filter Copied
In addition to the improved data format, normalisation and schemas published to Kafka from Gateway Hub, filtering options are now available to restrict the data that is published to an external Kafka cluster.
The publishing of metrics and events can now be individually controlled. In addition, publishing can be restricted to certain entities such as where Division = Securities
and certain metrics, such as Bloomberg only.
For more information, see Publish to Kafka.
Role based security Copied
This release brings easy-to-configure single sign on (SSO) and role based security to Gateway Hub. With this version, predefined Administrator and Operator roles are available to provide write and read access to Gateway Hub. Gateway Access control is still configured at the Gateway, although this will be centralised in a future version.
Role based security requires integration with Active Directory or LDAP.
Breaking changes Copied
These are the breaking changes in this release:
Issue key | Release description |
---|---|
STARK-9970 | The UI and SSO endpoints were previously accessed on unique ports (defaulting to 8888 and 9999 , respectively). These are now accessed using the same port as the rest of Gateway Hub API calls, defaulting to 8080 . |
STARK-9969 | Before you upgrade to 1.5, save your SSO configuration separately. The configuration will be lost during the upgrade. Once 1.5 is successfully installed, SSO will need to be configured using the Security section of the Web UI. |
New features and enhancements Copied
These are the new features and enhancements of this release:
Issue key | Release description |
---|---|
STARK-10538 | Apache Commons BeanUtils library was updated. |
STARK-10452 | The user’s display name is now shown in the application when SSO is enabled. |
STARK-10302 | Improved YARN scheduler to allow important jobs to be prioritised correctly. |
STARK-10186 | YARN logs are now included in diagnostics files. |
STARK-10106 | The licence is now checked before enabling metric and event storage. |
STARK-9970 | The SSO Agent and UI are now served by the Gateway Hub API, and the standalone agents have been removed. Note that the UI and SSO API calls are now done using the same port as the rest of the Gateway Hub API. |
STARK-9969 | The previous standalone SSO and UI agents are removed on upgrade from version 1.4. |
STARK-9908 | Resources are now more fairly allocated between the various components within Gateway Hub, providing better stability and performance particularly for large queries. |
STARK-9560 | Dashboard panels are now always displayed while the data is still loading. |
STARK-9444 | Publishing endpoint was updated to allow message filtering. |
STARK-9398 | Improved robustness of installer when attempting to request a licence key on a slow internet connection. |
STARK-9236 | The Web Console is now served on the same port as the REST API. |
STARK-9179 | Entity change events are now published as a new message type. They indicate if a new entity has been detected, removed, or the attributes of an entity have changed. |
STARK-9086 | Various improvements were introduced to the Dark theme of the application. |
STARK-9039 | Published events can now be filtered by event type, such as severity. |
STARK-8859 | The /admin/retention endpoints now require ISO 8601 duration formats in line with other endpoints. |
STARK-8821 | The Storage page in the Web Console now allows metric and event storage to be explicitly enabled and disabled. |
STARK-8547 | React was upgraded to version 16.8.x. |
STARK-8540 | Various changes were made to Spark and YARN configuration, resulting in better performance and robustness, particularly with smaller clusters. |
STARK-8402 | The Web Console now defaults to using the browser time zone in the absence of a configured user preference. |
STARK-8401 | The layout of the errors tables has been improved when viewed on larger monitors. |
STARK-8388 | The number of system errors in the system is now displayed as a KPI on the home dashboard. |
STARK-8258 | GET /versions is now available to provide the API versions supported by Gateway Hub. |
STARK-8136 | Now that the SSO endpoints are hosted on the same port as the rest of the Gateway Hub endpoints, the Open API definition has been unified. |
STARK-8135 | The Web Console now routes SSO calls via the same port as other Gateway Hub API calls. |
STARK-8134 | The SSO REST API endpoints are now hosted on the same port as the rest of the Gateway Hub REST API endpoints. |
STARK-7742 | Material UI was updated to version 3.9.3. |
STARK-7512 | Kafka publishing can now be filtered by sampler name. |
STARK-7511 | Kafka publishing can now be filtered by entity query, such as all entities where Division = Securities . |
STARK-7510 | Kafka publishing can now be filtered by message type, such as metrics or events. |
STARK-7233 | retainMetricsAndEvents: true/false has been added to the retention API, allowing metric and event history to be explicitly enabled and disabled. |
STARK-6828 | Administrator and Operator roles are available for assignment to LDAP users and groups. |
STARK-6705 | Centralised configuration resources can now be locked and unlocked via API. |
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-10330 | Metric queries now return data in the same unit of measure as the one displayed in Active Console, rather than the canonical unit of measure. |
STARK-10087 | Fixed an issue where retention could not be updated after previous invalid input. |
STARK-9925 | An error is now displayed when a metric query contains one or more invalid metric names. |
STARK-9681 | Fixed an issue where archiving was getting stuck in m5.2xlarge AWS instances. |
STARK-9634 | Errors without a message body, for example, a URL with more than 2048 characters, are now handled correctly. |
STARK-9612 | Configuring SSL using hubctl setup ssl no longer misallocates YARN resources. |
STARK-9337 | An issue with the archiving process occasionally getting stuck due to a lack of resources is now resolved through better allocation of resources on install. |
STARK-9231 | Kafka publishing configuration is no longer validated when publishing is disabled. |
STARK-9161 | Handling of invalid metric query API parameters was improved. |
STARK-9138 | Uploading a licence is now supported in the Web Console when the system is unlicensed. |
STARK-9128 | The ingestion errors loading indicator is now correctly positioned inside the table control. |
STARK-9063 | Fixed colour of Show related errors control in the Errors pages. |
STARK-9014 | Various fixes were introduced to the Web Console when used in Firefox. |
STARK-8664 | Repeatedly misconfiguring Kafka publishing in the Web Console no longer gives the impression that the invalid config has been saved successfully. |
STARK-8535 | Fixed an issue where the error search control only showed tooltip intermittently. |
STARK-8510 | Invalid publishing configuration no longer causes the Web Console to hang. |
STARK-8506 | Publishing status bar is now refreshed when configuration is updated. |
STARK-8448 | Attributes with special characters are now encoded correctly in the URL. |
STARK-8432 | The ingestion and system errors pages are now fully responsive down to mobile screens. |
STARK-8387 | Refreshing the Web Console while a page is loading no longer displays an incorrect error screen. |
STARK-7672 | Archiving is now done more frequently, instead of inside the maintenance window. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-10849 | Web Dashboard is not compatible with SSO Agent in Gateway Hub. This will be addressed in the next release. |
STARK-10831 | Webslinger is not compatible with SSO Agent in Gateway Hub. This will be addressed in the next release. |
STARK-10046 | Gateway Hub is not compatible with systemd version 219-65 due to an issue with the underlyingMapR platform. A previous version of systemd is required. |
STARK-9730 | The operating system language must be set to English, otherwise the build fails. |
STARK-9515 | Data with a unit of measure assigned (for example, Kilobytes ) is currently published in the canonical unit (for example, Bytes ). This means that certain metrics may have a misleading name (for example, KilobytesInQueue ). |
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name e.g. “test;”. To resolve this issue update the dataview column name removing the semicolon. |
STARK-7596 | The SSL certificate install incorrectly uses the default ‘hub’ user when applying an SSL certificate. To specify a different install user please add the following to the hubctl installation file:
|
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.4.1 Copied
Released: 14 August, 2019
This release delivers the updated binaries for in the ITRS Downloads page.
New features and enhancements Copied
These are the new features and enhancements of this release:
Issue key | Release description |
---|---|
STARK-9778 | Gateway Hub can now be installed to custom directories. However, please note that MapR must still be installed in /opt/mapr . |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-9730 | The operating system language must be set to English, otherwise the build will fail. |
STARK-9095 | Occasionally a container synchronisation issue occurs during an upgrade, resulting in the following error message:
If this occurs, resync the containers manually using the following command:
|
STARK-9515 | Data with a unit of measure assigned (for example, Kilobytes ) is currently published in the canonical unit (for example, Bytes ). This means that certain metrics may have a misleading name (for example, KilobytesInQueue ). |
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name e.g. “test;”. To resolve this issue update the dataview column name removing the semicolon. |
STARK-7596 | The SSL certificate install incorrectly uses the default ‘hub’ user when applying an SSL certificate. To specify a different install user please add the following to the hubctl installation file:
|
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.4.0 Copied
Released: 31 July, 2019
This release delivers the updated binaries for in the ITRS Downloads page.
Highlights Copied
These are the highlights of this release:
- Improved Gateway integration.
- Centralised configuration locking.
- Improved API documentation.
- Custom install directory for Gateway Hub. This release is also focused on improving stability and robustness of metric queries. A number of stability issues have been resolved.
Improved Gateway integration Copied
Integration between Gateway and Gateway Hub has been optimised, resulting in almost 25% less disk required for a standard Gateway Hub with metric storage compared to the previous version. Data will also be available in the Gateway Hub significantly faster than before due to the improved processing. These improvements require Gateway version 4.12.x.
Centralised Gateway configuration locking Copied
Centralised configuration files can now be centrally locked, preventing concurrent updates from causing issues. This feature requires Gateway 4.12.x.
For more information, see Centralised Gateways User Guide.
Improved API documentation Copied
API documentation is now published using redoc allowing better navigation and examples.
For more information, see Gateway Hub API documentation.
Custom install directory Copied
You can now perform new installations of Gateway Hub in custom locations, allowing Gateway Hub, the Web Console SSO, and Kafka to be installed in any location.
Note that the install location for MapR cannot be changed from /opt/mapr
.
Note
Changing install directories as part of an upgrade is not supported.
New features and enhancements Copied
These are the new features and enhancements of this release:
Issue key | Release description |
---|---|
STARK-9518 | The maintenance window section has been removed from the Storage screen. Archiving is now done several times per day rather than during a maintenance period. |
STARK-9469 | Apache Commons Collections library was updated. |
STARK-9355 | The allocation of resources is now more sensibly calculated, providing more resources to the components that need it, such as Spark and YARN. This will prevent a number of OOM issues seen previously. |
STARK-9247 | The raw and enriched internal Kafka topics have been replaced with a single merged topic for 4.12 Gateways and above. |
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-9463 | Fixed instances where a metric query would fail with the following exception:
|
STARK-9337 | An issue with the archiving process occasionally getting stuck due to a lack of resources is now resolved through better allocation of resources on install. |
STARK-9316 | Querying a metric that had previously changed schema no longer returns an empty result set. |
STARK-9269 | Added retry and wait steps to the install, providing more robustness when installing Gateway Hub over an unreliable connection. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-9730 | The operating system language must be set to English, otherwise the build will fail. |
STARK-9515 | Data with a unit of measure assigned (for example, Kilobytes ) is currently published in the canonical unit (for example, Bytes ). This means that certain metrics may have a misleading name (for example, KilobytesInQueue ). |
STARK-9095 | Occasionally a container synchronisation issue occurs during an upgrade, resulting in the following error message:
If this occurs, resync the containers manually using the following command:
|
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name e.g. “test;”. To resolve this issue update the dataview column name removing the semicolon. |
STARK-7596 | The SSL certificate install incorrectly uses the default ‘hub’ user when applying an SSL certificate. To specify a different install user please add the following to the hubctl installation file:
|
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.3.0 Copied
Released: 14 June, 2019
This release delivers the updated binaries for in the ITRS Downloads page.
Highlights Copied
These are the highlights of this release:
- An open query API for retrieving metric data.
- A new directory for storing Entity information.
- Centralised Gateway configuration.
Metric Query API Copied
Metric data can now be retrieved using an open REST API call:
POST /metrics/query
This endpoint can be used to retrieve metric data, grouped by time bucket with a number of built in aggregations such as median, mean, and percentiles, supporting simpler display of large amounts of monitored data. As of version 1.3 this endpoint is tuned for reporting and results can take several minutes, which is a substantial improvement over querying a relational database. Performance will be improved further in a future release.
A list of the aggregation functions is available using the following REST API call:
GET /metrics/aggregations
Entity directory Copied
This release marks the first version of the Entity directory, providing the basis for the next generation of the state tree. The entity directory brings a number of significant advantages over the existing Geneos state tree, including:
- Support for significantly larger monitored estates
- Better analysis and retrieval of data across groups of similar servers
The entity store is an important part of our long term goal to simplify the data model, in particular removing Gateways and Netprobes from the XPath hierarchy and moving towards dealing with Entities and Metrics only. This will have a number of benefits, including easier configuration of Self-announcing Netprobes that rebalance across different Gateways frequently, as well as ultimately simplifying the way monitoring is configured in Geneos. Although the existing state tree will remain available for existing setups, over time we anticipate you will choose to switch to the simpler model.
In 1.4, the Entity Directory will also serve as the backbone for the new web-based monitoring UI.
Centralised Gateway configuration Copied
Gateway Hub can now store and serve Gateway configuration centrally. This version allows include files to be uploaded to Gateway Hub and used by connected Gateways. This is the first step in the journey to significantly simplify administration of Gateways at scale. More features will be added in the coming release, including centralised validation and locking, as well as administration UI screens in the Web Console.
For more information about this feature, see Centralised Gateways User Guide.
New features and enhancements Copied
These are the new features and enhancements of this release:
Issue key | Release description |
---|---|
STARK-8477 | JMX instrumentation by various components has now been secured. |
STARK-8315 | Processing of schema messages from the Gateway has been substantially improved. |
STARK-8248 | The style for snackbar error and information messages in the Web Console has been updated. |
STARK-8020 | Web Console has improved handling of concurrency when changing configuration. |
STARK-7913 | Added a confirmation modal when leaving a Web Console page with errors. |
STARK-7832 | Improved design for system and ingestion error pages on mobile devices for the Web Console. |
STARK-7713 | In the Web Console, save buttons are now disabled in the presence of validation errors. |
STARK-7628 | Improved memory management for various Gateway Hub components. |
STARK-7625 | The Archiving jobs have been combined into a single Spark job for increased performance. |
STARK-7623 | A ITRS-Doc-Version HTTP header is required on all PUT endpoints, preventing concurrency issues. GET endpoints now return the current document version. |
STARK-7302 | Password field types in the Web Console have been improved to allow better password input. |
STARK-7250 | The documentation link in the About page is now correctly linked to the current Web Console version. |
STARK-7228 | The Web Console will now preserve the collapsed/expanded status of the sidebar between sessions. |
STARK-7182 | The status of Kafka publishing is now displayed in the publishing screen in the Web Console. |
STARK-7132 | Web Console now handles an unlicensed system better. |
STARK-7074 | Ports are now configurable for Kafka, REST API, Web Console, SSO API, and MCS monitoring UI. |
STARK-6877 | DD MMM YYYY has been added to the list of supported date formats in the Web Console. |
STARK-6688 | MapR Enterprise Components (MEP) have been upgraded to 6.1. |
STARK-6686 | The MapR instance underpinning Gateway Hub has been upgraded to 6.1. |
STARK-6526 | System errors are no longer displayed alongside ingestion errors and have their own endpoint and Web Console screen instead. |
STARK-6525 | The Kafka offsets have been removed from ingestion errors and replaced with the raw message, so that older errors did not become outdated by Kafka rebalances and retention settings. |
STARK-6520 | The ingestion errors pages in the Web Console have been reworked for significantly better performance and robustness. Outdated links to old errors that have been resolved are now handled correctly. |
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-8991 | Dataview columns and headline cells with a full stop/period in the name no longer cause an error when queried for metrics. |
STARK-8931 | Dataview columns with backticks (`) in the name no longer cause the archiver to crash. |
STARK-8668 | Updating a metric’s type no longer causes the metric to be marked as deleted. |
STARK-7617 | Various improvements to the way Zookeeper is configured to improve stability. |
STARK-7585 | The /dataview endpoint has been updated to include a path parameter, allowing filtering by partial dataview path. |
STARK-7227 | The Web Console sidebar will no longer get stuck when collapsed under certain circumstances. |
STARK-7174 | Special characters (for example, ñ) are now handled correctly in Licence page of the Web Console. |
STARK-7116 | When publishing is disabled, schema and snapshot requests are no longer attempted. |
STARK-6726 | Column name escaping has been improved, preventing crashes when querying data from certain plug-ins. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-9095 | Occasionally a container synchronisation issue occurs during an upgrade, resulting in the following error message:
If this occurs, resync the containers manually using the following command:
|
STARK-7691 | Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name e.g. “test;”. To resolve this issue update the dataview column name removing the semicolon. |
STARK-7596 | The SSL certificate install incorrectly uses the default ‘hub’ user when applying an SSL certificate. To specify a different install user please add the following to the hubctl installation file:
|
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named “sampleTime”, “netprobeTime” or “dataViewId”. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
STARK-4350 | Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine. |
Gateway Hub 1.2.0 Copied
Released: 11 March, 2019
This release delivers the updated binaries for in the ITRS Downloads page.
Highlights Copied
These are the highlights of this release:
- Publishing of normalised data with schema to an external Kafka interface.
- Additional administration screens for the Web Console.
Kafka publishing Copied
The Kafka Publishing available with Gateway Hub builds upon the feature previously available in the Gateway. With Gateway Hub, schemas can now be published along with the metric and event data, in order to provide easier integration with other applications, and the published data is now normalised, enabling simpler usage downstream.
Web Console Copied
This release brings the new administration screens for the Web Console, allowing licence management and configuration of Publishing.
Breaking changes Copied
These are the breaking changes in this release:
Issue key | Breaking change description |
---|---|
STARK-6346 | Corrects the spelling of the “licence” keyword in the installation json file to UK English. Update the previous installation files to reflect this change. |
New features and enhancements Copied
These are the new features and enhancements of this release:
Issue key | Release description |
---|---|
STARK-6860 | When the Gateway Hub is not licensed, a placeholder appears in the Web Console. |
STARK-6733 | Dataview schemas can be requested to be published to Kafka using a REST command to Gateway Hub. |
STARK-6732 | Dataview snapshots can be requested to be published to Kafka using a REST command to Gateway Hub. |
STARK-6619 | Updates Web Console favicon. |
STARK-6361 | Gateway Hub can be used without saving metric and event data to disk. |
STARK-6346 | Standardises the use of the noun “licence” to UK English. |
STARK-5983 | Gateway Hub licences can be uploaded directly in the Web Console. |
STARK-5980 | Publishing to Kafka can be configured using a new Web Console screen. |
STARK-5913 | Sets the cluster name to hub.itrsgroup.com. |
STARK-5613 | Nodes can be added to the cluster. |
STARK-4749 | Dates in the Web Console, such as 20 September 2018 , are now localised correctly. |
STARK-4403 | Allows you to navigate Web Console UI entirely by the keyboard. |
STARK-4308 | Upgrades Kafka to 2.0. |
STARK-3270 | A screen showing the current installed version of Gateway Hub is available, along with a control to download the diagnostic info if required for a support ticket. |
STARK-3269 | A new Web Console page for viewing and uploading Gateway Hub licences. |
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-6431 | The installation script validates that the runtime user is assigned to the correct group before the installation starts. |
STARK-6423 | The installation script validates that the install user is configured correctly before the installation starts. |
STARK-5922 | Changing the units of measures no longer prevents ingestion errors from being returned. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-7368 | While performing an upgrade from 1.1 to 1.2, there may be some errors in the upgrade log. This is due to a change in the message format and will not affect metric ingestion. The error messages can be ignored. |
STARK-7118 | If a snapshot is requested from Gateway with an unrecognised snapshot type, then the request is accepted by Gateway Hub but ignored by the Gateway. The error appears in the Gateway log. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
Gateway Hub 1.1.0 Copied
Released: 21 January, 2019
This release delivers the updated binaries for in the ITRS Downloads page.
Highlights Copied
These are the highlights of this release:
- Ability to store historical data for all Geneos metrics and events.
- A brand new web-based UI for Gateway Hub Administration.
Historical metric and event storage Copied
Historical metric and event data can now be stored in Gateway Hub and retrieved using Active Console. This is significantly more reliable than database logging and is designed to handle all metrics being logged, rather than a subset.
Metric and event storage in Gateway Hub also provides the foundation for improved monitoring over the next few releases, including adaptive rules (anomaly detection), automatic baselining, predictive and causal analysis through machine learning.
Web Console Copied
his release brings the first release of the new web-based UI for Geneos, the Web Console, focusing on administration of Gateway Hub. The Web Console will be expanded upon in future releases to initially complement, and eventually replace, Active Console.
Breaking changes Copied
Note
These changes are applicable to Early Access customers only.
These are the breaking changes in this release:
Issue key | Breaking change description |
---|---|
STARK-5233 | The unit of measure endpoints have been simplified significantly. |
STARK-3441 | Dataview was previously written using incorrect case (e.g. “dataView”) in the REST endpoints. These have been corrected. |
STARK-1566 | Publishing to Gateway Hub is done using a secured connection. All publishing Gateway must be reconfigured to use SSL encryption as unsecured connections are no longer supported. |
New features and enhancements Copied
These are the new features and enhancements of this release:
Issue key | Release description |
---|---|
STARK-6565 | Bundled SSO Agent upgrades to version 1.3.0. |
STARK-6538 | Increases message throughput due to improved MapR-DB batching strategy. |
STARK-6344 | Gateway Hub configuration can be updated through the command line. |
STARK-6326 | Kafka disk location can be specified as part of the installation parameters. |
STARK-6316 | Better validation when adding units of measure without a locale or text. |
STARK-6191 | Perfmon errors are correctly converted to null. |
STARK-6167 | Keystore password is configurable. |
STARK-6007 | Error codes are updated to make them unique. |
STARK-5896 | The kernel settings vm.swappiness , vm.overcommit.memory , and net.ipv4.tcp_retries2 are no longer modified as part of the install and must be managed separately as a prerequisite. |
STARK-5768 | Removes the need to store runtime user credentials for normaliserd. |
STARK-5707 | Dataview errors can be filtered by error code. |
STARK-5591 | Ingestion errors can be filtered by partial or full dataview path. |
STARK-5582 | Adds filter and skip parameters to ingestion errors endpoint to allow paging. |
STARK-5541 | Truncation of decimal data when attempting to save as an int generates ingestion errors. |
STARK-4834 | Passwordless SSH is no longer required. During installation, you can interactively enter the password. |
STARK-4547 | The installation is done using an offline package. Gateway Hub nodes do not longer require internet access. |
STARK-4545 | After installation, you can enter a custom SSL certificate. |
STARK-3637 | Web Console is packaged with Gateway Hub. |
STARK-3020 | Retention policies are now specified for metric and events. |
Issues fixed Copied
These are the issues we have fixed in this release:
Issue key | Release description |
---|---|
STARK-6736 | Error messages with missing parameters can cause the error endpoints to fail. |
STARK-6486 | Zookeeper status is not reset correctly during factory reset. |
STARK-6480 | Logs entries are stored in UTC. |
STARK-6290 | Validation errors updated with unique titles. |
STARK-6287 | Better validation when uploading invalid licences. |
STARK-6246 | Duplicates unrecognisedField error message suppressed. |
STARK-6237 | Adds command line flag for generating faster diagnostic files. |
STARK-6042 | Resolves race condition causing “Unhandled Error: Failed to create table” log entries. |
STARK-6039 | Processing error messages no longer significantly slows down data ingestion. |
STARK-6005 | Systems with umask set to 077 no longer cause issues during installation. |
STARK-5886 | Simplifies user model to avoid permission issues in persistence. |
STARK-5839 | Fixes to skip and limit parameters for errors endpoint. |
STARK-5783 | Changes to Kafka ISR settings significantly improve resilience. |
STARK-5751 | Ingestion errors are no longer aggregated by dataview row name. |
STARK-5283 | Dataview queries return the most recent results first. If a limit is specified, the least recent results may be truncated. |
STARK-5203 | If a dataview had a computed row that was later removed, then the Gateway will now correctly report hasEnrichedRows as false. |
STARK-5008 | Changes to the casing of certain error parameters means that old error results will be cleared. These are repopulated as new errors are received and will not be an issue. |
STARK-4965 | Fixes an issue causing the error endpoint to return no data. |
STARK-4957 | Log files roll over correctly. |
STARK-4517 | Restarting Gateway Hub no longer causes Kafka to become corrupted under certain circumstances. |
STARK-3962 | Moves Kafka logs to the correct log location. |
Known issues Copied
These are the known issues affecting this release:
Issue key | Known issue description |
---|---|
STARK-7239 | 4.10 Gateways are not able to publish to Gateway Hub 1.1. This will be resolved as a matter of priority. |
STARK-6975 | Immediately after an upgrade from 1.0.3 to 1.1, the dataview query API returns empty results. This issue is resolved automatically after a few minutes and normal results will be returned. |
STARK-6505 | Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named sampleTime, netprobeTime, or dataViewId. |
STARK-4463 | Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns. |
Other Gateway Hub releases Copied
Release Notes | Release Date |
---|---|
2.x Release Notes | Released: 8 April 2020 |
1.x Release Notes | Released: 21 January 2019 |
Disclaimer
The information contained in this document is for general information and guidance on our products, services, and other matters. It is only for information purposes and is not intended as advice which should be relied upon. We try to ensure that the content of this document is accurate and up-to-date, but this cannot be guaranteed. Changes may be made to our products, services, and other matters which are not noted or recorded herein. All liability for loss and damage arising from reliance on this document is excluded (except where death or personal injury arises from our negligence or loss or damage arises from any fraud on our part).