mgr/cephadm: don't remove and deploy new daemon if ports change during upgrade

mgr/cephadm: don't remove and deploy new daemon if ports change during upgrade

If we're not mid-upgrade, a port/ip change is most likely a user
changing the configuration for the daemon and starting a new
daemon with the new ports can be justified. During upgrade, their
are a number of different upgrade paths (espeically considering
our N+2 upgrade support) where something internal to cephadm
has made the ip we're binding to or the ports being used
change. In these cases, the process of upgrading the daemon
will resolve the issue. Howeverm by having the scheduler
unilaterally remove and deploy fresh versions of daemons
when it sees port changes, we may effectively "upgrade" some
daemons out of the intended order just to make the ports
match up. This was seen with nvmeof which needs to be upgraded
after the mon daemons, but was being removed and redeployed
after the mgr upgrade once cephadm saw the set of expected
ports had changed.

This patch adds a new "upgrade_in_progress" attribute to the
HostAssignment class in the scheduler to make it aware of
an ongoing upgrade. It also changes the behavior specifically
around whether a daemon matches another if the ports match
up when "upgrade_in_progress" was set to True.

Signed-off-by: Adam King <adking@redhat.com>