|v2||Included some open questions under Xapi point 2|
|v3||Added new error, task, and assumptions|
When hosts use an aggregated local storage SR, then disks are going to be mirrored to several different hosts in the pool (RAID). This ensures that if a host goes down (e.g. due to a reboot after installing a hotfix or upgrade, or when “fenced” by the HA feature), all disk contents in the SR are still accessible. This also means that if all disks are mirrored to just two hosts (worst-case scenario), just one host may be down at any point in time to keep the SR fully available.
When a node comes back up after a reboot, it will resynchronise all its disks with the related mirrors on the other hosts in the pool. This syncing takes some time, and only after this is done, we may consider the host “up” again, and allow another host to be shut down.
Therefore, when installing a hotfix to a pool that uses aggregated local storage, or doing a rolling pool upgrade, we need to make sure that we do hosts one-by-one, and we wait for the storage syncing to finish before doing the next.
This design aims to provide guidance and protection around this by blocking hosts to be shut down or rebooted from the XenAPI except when safe, and setting the
host.allowed_operations field accordingly.
If an aggregated local storage SR is in use, and one of the hosts is rebooting or down (for whatever reason), or resynchronising its storage, the operations
shutdown will be removed from the
host.allowed_operations field of all hosts in the pool that have a PBD for the SR.
This is a conservative approach in that assumes that this kind of SR tolerates only one node “failure”, and assumes no knowledge about how the SR distributes its mirrors. We may refine this in future, in order to allow some hosts to be down simultaneously.
The presence of the
reboot operation in
host.allowed_operations indicates whether the
host.reboot XenAPI call is allowed or not (similarly for
host.shutdown). It will not, of course, prevent anyone from rebooting a host from the dom0 console or power switch.
Clients, such as XenCenter, can use
host.allowed_operations, when applying an update to a pool, to guide them when it is safe to update and reboot the next host in the sequence.
host.shutdown is called while the storage is busy resyncing mirrors, the call will fail with a new error
Xapi needs to be able to:
Taskfor it (in the DB). This task can be used to track progress, if available.
host.allowed_operationsfor all hosts in the pool according to the rules described above. This comes down to updating the function
xapi_host_helpers.ml, and will need to use a combination of the functionality from the two points above, plus and indication of host liveness from
host_metrics.liveis updated to detect pool slaves going up and down (probably at least in
host.shutdowncall is executed:
All of the above runs on the pool master (= SR master) only.
The above will be safe if the storage cluster is equal to the XenServer pool. In general, however, it may be desirable to have a storage cluster that is larger than the pool, have multiple XS pools on a single cluster, or even share the cluster with other kinds of nodes.
To ensure that the storage is “safe” in these scenarios, xapi needs to be able to ask the storage backend:
If the cluster is equal to the pool, then xapi can do point 2 without asking the storage backend, which will simplify things. For the moment, we assume that the storage cluster is equal to the XS pool, to avoid making things too complicated (while still need to keep in mind that we may change this in future).