Internal documentation only

This page has been marked as draft.

Changing Collector Cluster Hostname or IP

While it’s not a routine operation, there may be times when you need to reconfigure the Hostname and/or IP address of nodes within an Opsview Collector Cluster.

This process cannot be completed solely via opsview-deploy; several manual steps are required both on the collector(s) and the orchestrator.

• NOTE: If changing the Hostname, historical graphing data for the host will no longer be visible due to the name no longer matching the timeseries data.

Before you make any changes, place the affected host into Scheduled Downtime in the Opsview UI.

• As per the upcoming step 5 within “Update the Collector”, the whole collector cluster is going to be affected and updated, so ensure to place the entire cluster into Scheduled Downtime

Make your Hostname or IP change Copied

• Within the UI, update the host information you are changing: Either Hostname/IP to its new one
• Next, these steps are customised to your organisation
  • In short, this would be using the hostnamectl set-hostname command
    • For IP changes, there are many different ways to update the networking
    • Finally, upon post-reboot of the server, the opsview-monit summary may display errors due to the change, but you may progress to “Update the Collector” steps below to assist with stopping the opsview-components

Update the Collector Copied

1. Stop Opsview services on the collector:

/opt/opsview/watchdog/bin/opsview-monit stop all

2. Change the hostname

hostnamectl set-hostname <new_hostname>

3. Ensure all opsview processes have actually stopped

• Background processes such as “epmd -daemon” may still be there, so kill these off

ps -fu opsview

4a. Remove opsview-messagequeue (and opsview-datastore if the cluster contains more than one collector):

• Ensure you are not in the /opt/opsview/messagequeue directory before these package removal and directory steps are run

For RHEL/CentOS:

yum remove opsview-messagequeue
yum erase opsview-messagequeue

For Debian/Ubuntu:

apt remove opsview-messagequeue
yum purge opsview-messagequeue

4b. Remove the messagequeue directory

rm -rf /opt/opsview/messagequeue

IFthe collector cluster is larger than itself i.e. more than one collector in the cluster, the opsview-datastore is required to be removed also (repeating steps 4a and 4b, but for the opsview-datastore package and /opt/opsview/datastore directory)

5. Repeat on all other collectors in the cluster to ensure there are no future issues with connectivity due to the clustered nature of the opsview-messagequeue and opsview-datastore

Update the Orchestrator Copied

Note: High Availability (HA) and Disaster Recovery (DR) steps are also pointed out below

1. Update configuration files:

- /opt/opsview/deploy/etc/opsview_deploy.yml
- /opt/opsview/deploy/etc/user_vars.yml
- /etc/hosts

• The Hostname will show in the user_vars.yml if being used for a Forward/Reverse SSH Tunnel
  • /etc/hosts will only need to be updated if the ansible playbooks (deployment process) are not managing it (it may be that the host isn’t displayed in there anyway)
  • HA/DR: Reminder to update these configuration files on the secondary orchestrator

2. Run the deployment playbooks:

• Use the collector_cluster_name ansible variable so all collectors in the cluster are updated, shown as your collector cluster name within opsvie_deploy.yml
• It is prefixed with opsview_cluster_
  • e.g. “London Collectors” would become opsview_cluster_london_collectors
  • Any spaces are replaced with underscores and any capitalisation is should be lowercase
  • To limit the deployment playbooks to run against only specific collector cluster, us the -l limit flag/option
• Step 1 clears the orchestrator “…/cache/facts/” files, which are built and used by the deployment playbooks for information about servers (collectors and so forth) within it’s configuration

Commands:

1. # rm -rf /opt/opsview/deploy/var/cache/facts/ /opt/opsview/deploy/var/fact_cache/
2. # /opt/opsview/deploy/bin/opsview-deploy /opt/opsview/deploy/lib/playbooks/gather-machine-refs.yml
3. # /opt/opsview/deploy/bin/opsview-deploy -l opsview_cluster_<collector_cluster_name> /opt/opsview/deploy/lib/playbooks/setup-hosts.yml
4. # /opt/opsview/deploy/bin/opsview-deploy -l opsview_cluster_<collector_cluster_name> /opt/opsview/deploy/lib/playbooks/setup-infrastructure.yml
5. # /opt/opsview/deploy/bin/opsview-deploy -l <orchestrator_hostname> /opt/opsview/deploy/lib/playbooks/messagequeue-install.yml
6. # /opt/opsview/deploy/bin/opsview-deploy -l opsview_cluster_<collector_cluster_name> /opt/opsview/deploy/lib/playbooks/setup-opsview.yml
7. # /opt/opsview/deploy/bin/opsview-deploy -l <orchestrator_host> /opt/opsview/deploy/lib/playbooks/ssh-tunnels-install.yml
8. # /opt/opsview/deploy/bin/opsview-deploy -l '< orchestrator_hostname>,opsview_cluster_<collector_cluster_name>' /opt/opsview/deploy/lib/playbooks/setup-monitoring.yml

• HA/DR: Reminder to perform the “-l <orchestrator_host>” commands on the secondary orchestrator. The steps involving the <collector_cluster_name> do not need to be repeated, apart from “setup-monitoring.yml”

3. Apply Changes

4. Post-Change Checks

• Confirm: the Collector Monitoring page reflects the new Hostname/IP
• Ensuring the Apply Changes has finished, verify that the /opt/opsview/scheduler/var/collection_plan.json is updated with a new timestamp.
  • This confirms opsview-messagequeue connectivity between orchestrator and collector is functioning.