mon, msgr: rework the public addrs support
A monitor does support two kinds of addresses when binding msgr:
1) `bind_addrs` which are basically passed down the stack to
e.g. the `bind()` syscall -- they are physical, low-level ones;
2) `public_addrs` which are exposed to peers over Ceph's on-wire
protocols.
This differentation is crucial for Rook and K8S use scenarios
where, due to ephemeral nature of node's physical IPs, virtual
_cluster IPs_ must be used instead for mon accesses.
Although monitors still bind to physical ones, virtuals are exposed.
Then, thankfully to NAT, the latter are converted to the former ones.
Unfortunately, the current implmentation of this feature is based
on the idea to 1) bind-as-usually and then 2) change-the-addrs
via powerful `set_addrs()` setter. This setter is too powerful
but the main issue is the `rebind()` sceario. It doesn't happen
in monitors but OSDs may rebind when they get marked down by
a monitor.
Fixes: https://tracker.ceph.com/issues/58779
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit
1b24588dc8e03adaff472a94f594f2851b4acec4)