tools/cephfs_mirror: Do remote fs sync once instead of fsync on each fd
Do remote fs sync once just before taking snapshot
as it's faster than doing fsync on each fd after
file copy.
Moreover, all the datasync threads use the same sinlge libceph
onnection and doing ceph_fsync concurrently on different fds on
a single libcephfs connection could cause hang as observed in
testing as below. This issue is tracked at
https://tracker.ceph.com/issues/75070
-----
Thread 2 (Thread 0xffff644cc400 (LWP 74020) "d_replayer-0"):
0 0x0000ffff8e82656c in __futex_abstimed_wait_cancelable64 () from /lib64/libc.so.6
1 0x0000ffff8e828ff0 [PAC] in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libc.so.6
2 0x0000ffff8fc90fd4 [PAC] in ceph::condition_variable_debug::wait ...
3 0x0000ffff9080fc9c in ceph::condition_variable_debug::wait<Client::wait_on_context_list ...
4 Client::wait_on_context_list ... at /lsandbox/upstream/ceph/src/client/Client.cc:4540
5 0x0000ffff9083fae8 in Client::_fsync ... at /lsandbox/upstream/ceph/src/client/Client.cc:13299
6 0x0000ffff90840278 in Client::_fsync ...
7 0x0000ffff90840514 in Client::fsync ... at /lsandbox/upstream/ceph/src/client/Client.cc:13042
8 0x0000ffff907f06e0 in ceph_fsync ... at /lsandbox/upstream/ceph/src/libcephfs.cc:316
9 0x0000aaaaad5b2f88 in cephfs::mirror::PeerReplayer::copy_to_remote ...
----