msg/Policy: make stateless_server default to anon (again)
Midway through the octopus cycle, we made stateless server more stateless
in the sense that it would not register incoming client connections. And,
in so doing, it would not enforce that client connections came from
unique addresses, by closing an existing connection from the same addr
when a new connection was accepted.
This turned out to cause out of order OSD ops because the OSD needed that
behavior. See https://tracker.ceph.com/issues/42328. We fixed that by
reverting to the old behavior for all but monitor connections, where we
needed it, in
507d213cc453ed86ab38619590f710f33245c652.
This, in turn, breaks most OSD <-> OSD communication (and probably lots
of other things) with cephadm, because we make entity_addr_t unique with
a nonce that is populated by getpid()... and the containerized daemons
all have pid 1. When we finally merged the follow-on fixes for the change
above cephadm OSDs can't ping each other.
In my view, the 'anon' connection handling is a good idea in the general
case. So, let's adjust our fix for #42328 so that it is only the OSD
client-side interface that registers client connections and makes them
unique.
Fixes: https://tracker.ceph.com/issues/44358
Signed-off-by: Sage Weil <sage@redhat.com>