mgr/mgr_util: switch using unshared cephfs connections whenever possible

mgr/mgr_util: switch using unshared cephfs connections whenever possible

The connection pool used by mgr/volumes (mgr/snap_schedule, mgr/mirroring)
is basically a cached libcephfs connection handle. Since its a singleton
lock contention may occur in the client (contention on `client_lock`),
especially when an MDS operation taken too long to complete.

Having a pool of connections to a cephfs file system, ensures that concurrent
access to ceph file system via the connection pool use unshared libcephfs
handles whenever possible thereby not contending for `client_lock`. However,
there is a max cap of unshared connections per cephfs file system, post which
connections start to get shared between users.

Note that to totally benefit from this, its desirable to have multiple active
MDSs for a ceph file system to rule out the possibility of the contention in
`mds_lock`.

Fixes: http://tracker.ceph.com/issues/51256
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 2e7adc48d141dacf7fc90d4c4f8d61c287e77984)