Performance Issues in Database Logging
This problem is related to DatabaseLogging maxRequestQueueSize error and .dat cache dump files.
Problem Copied
Users may find Database Logging not writing expected records in the database or there are delays. If the delay persists, the gateway will write temporary cache .dat files under the ./database directory.
In the Gateway log, you may see the following error:
ERROR: DatabaseLogging maxRequestQueueSize [4001/4000] has breached, dumping data to file 'file://./database/database.1453595297.1014075536.dat'.
If the issue is severe, you may see the following icon (crossed out database) besides the Gateway in the Gateways dockable.
Possible causes Copied
The gateway database logging function makes use of standard client libraries of database vendors. Performance issues that are specific to database logging are quite likely environment related, which may involve the network and database setup.
If this is a new installation, please note the general recommendation that the Gateway and database server should be located together within the same data centre. This will reduce possible network latency that can greatly affect the performance of database transactions.
There could be database level issues like:
- The database server and client versions should be compatible and be up-to-date.
- Indexes of database tables are somehow corrupted or lost.
- Certain users or customized scripts that may lock database tables unexpectedly.
- General database performance issues, which may need to run tests to query and insert database records as benchmark.
Possible solutions Copied
The Gateway-Database Logging plugin enables users to monitor whether Database Logging is working normally and any potential performance issues.
The first thing to check is the Summary view which should show connectionStatus as Connected.
Then there are figures reported in the Cache and Items view, namely cacheFiles, cacheSize, and queueSize. If these numbers increase over time, it means data are not written fast enough to the database. Users can configure gateway rules to alert potential issues.
The Geneos administrator may need to involve the DBA in their company to diagnose the database level issues mentioned in Possible Causes section above.
For more information, see Gateway database logging, performance and .dat dump files and Handling Database Logging errors pointing to maxRequestQueueSize being breached.