rbd mirror pool enable [--site-name {local-site-name}] {pool-name} {mode}
-The mirroring mode can either be ``image`` or ``pool``:
-
-* **image**: When configured in ``image`` mode, mirroring must
- `explicitly enabled`_ on each image.
-* **pool** (default): When configured in ``pool`` mode, all images in the pool
- with the journaling feature enabled are mirrored.
+The mirroring mode can be ``image``, ``pool`` or ``init-only``:
+
+* **image**: When configured in ``image`` mode, mirroring must be
+ `explicitly enabled`_ for each intended image in the default namespace
+ of the pool. Other namespaces aren't affected and must be
+ `configured separately`_.
+* **pool**: When configured in ``pool`` mode, all images in the default
+ namespace of the pool with the journaling feature enabled are mirrored.
+ Other namespaces aren't affected and must be `configured separately`_.
+* **init-only**: When configured in ``init-only`` mode, no images in the
+ default namespace of the pool will be mirrored but other namespaces can
+ still be configured. This is needed to allow some other namespace to be
+ mirrored to the default namespace of the remote pool but can be useful
+ on its own as well.
For example::
Namespace Configuration
=======================
-Mirroring can be configured on a namespace in a pool. The pool must already
-have been configured for mirroring. The namespace can be mirrored to a namespace
-with the same or a different name in the remote pool.
+Mirroring can be enabled on non-default namespaces of a pool independent of
+the default namespace. The pool must be configured for mirroring in advance.
+A given namespace can be mirrored to a namespace with the same or a different
+name in the remote pool, including to the default namespace (referred to as
+``''`` or ``""``).
Enable Mirroring
----------------
The mirroring mode can either be ``image`` or ``pool``:
-* **image**: When configured in ``image`` mode, mirroring must
- `explicitly enabled`_ on each image.
-* **pool** (default): When configured in ``pool`` mode, all images in the namespace
+* **image**: When configured in ``image`` mode, mirroring must be
+ `explicitly enabled`_ for each intended image in the namespace.
+* **pool**: When configured in ``pool`` mode, all images in the namespace
with the journaling feature enabled are mirrored.
For example::
$ rbd --cluster site-a mirror pool enable image-pool/namespace-a image --remote-namespace namespace-b
$ rbd --cluster site-b mirror pool enable image-pool/namespace-b image --remote-namespace namespace-a
-This will set up image mode mirroring between image-pool/namespace-a on cluster
-site-a and image-pool/namespace-b on cluster site-b.
-The namespace and remote-namespace pair configured on a cluster must
+This will set up image mode mirroring between ``image-pool/namespace-a`` on
+cluster ``site-a`` and ``image-pool/namespace-b`` on cluster ``site-b``.
+The namespace and remote-namespace pair configured on a local cluster must
match the remote-namespace and namespace respectively on the remote cluster.
If the ``--remote-namespace`` option is not provided, the namespace will be
mirrored to a namespace with the same name in the remote pool.
+To set up pool mode mirroring between ``image-pool`` (default namespace) on
+cluster ``site-a`` and ``image-pool/namespace-c`` on cluster ``site-b``::
+
+ $ rbd --cluster site-a mirror pool enable image-pool pool --remote-namespace namespace-c
+ $ rbd --cluster site-b mirror pool enable image-pool init-only
+ $ rbd --cluster site-b mirror pool enable image-pool/namespace-c pool --remote-namespace ""
+
+To set up pool mode mirroring between ``image-pool`` (default namespace) on
+cluster ``site-a`` and ``image-pool/namespace-d`` on cluster ``site-b`` and
+at the same time image mode mirroring between ``image-pool`` (default namespace)
+on cluster ``site-b`` and ``image-pool/namespace-e`` on cluster ``site-a``::
+
+ $ rbd --cluster site-a mirror pool enable image-pool pool --remote-namespace namespace-d
+ $ rbd --cluster site-a mirror pool enable image-pool/namespace-e image --remote-namespace ""
+ $ rbd --cluster site-b mirror pool enable image-pool image --remote-namespace namespace-e
+ $ rbd --cluster site-b mirror pool enable image-pool/namespace-d pool --remote-namespace ""
+
Disable Mirroring
-----------------
.. _rbd: ../../man/8/rbd
.. _ceph-conf: ../../rados/configuration/ceph-conf/#running-multiple-clusters
.. _explicitly enabled: #enable-image-mirroring
+.. _configured separately: #namespace-configuration
.. _bootstrap token: #bootstrap-peers
.. _force resync command: #force-image-resync
.. _demote the image: #image-promotion-and-demotion
. $(dirname $0)/rbd_mirror_helpers.sh
setup
-testlog "TEST: add image and test replay"
+testlog "TEST: mirror from default namespace to non-default namespace"
start_mirrors ${CLUSTER1}
image=test
+rbd --cluster ${CLUSTER1} mirror pool enable ${POOL} init-only
+rbd --cluster ${CLUSTER2} mirror pool enable ${POOL} init-only
+rbd --cluster ${CLUSTER1} mirror pool disable ${POOL}/${NS1}
+rbd --cluster ${CLUSTER2} mirror pool disable ${POOL}/${NS1}
+rbd --cluster ${CLUSTER2} mirror pool enable ${POOL} ${MIRROR_POOL_MODE} --remote-namespace ${NS1}
+rbd --cluster ${CLUSTER1} mirror pool enable ${POOL}/${NS1} ${MIRROR_POOL_MODE} --remote-namespace ""
+create_image_and_enable_mirror ${CLUSTER2} ${POOL} ${image} ${RBD_MIRROR_MODE}
+wait_for_image_replay_started ${CLUSTER1} ${POOL}/${NS1} ${image}
+write_image ${CLUSTER2} ${POOL} ${image} 100
+wait_for_replay_complete ${CLUSTER1} ${CLUSTER2} ${POOL}/${NS1} ${POOL} ${image}
+wait_for_replaying_status_in_pool_dir ${CLUSTER1} ${POOL}/${NS1} ${image}
+compare_images ${CLUSTER1} ${CLUSTER2} ${POOL}/${NS1} ${POOL} ${image}
+remove_image_retry ${CLUSTER2} ${POOL} ${image}
+wait_for_image_present ${CLUSTER1} ${POOL}/${NS1} ${image} 'deleted'
+rbd --cluster ${CLUSTER1} mirror pool disable ${POOL}/${NS1}
+rbd --cluster ${CLUSTER2} mirror pool enable ${POOL} init-only
+
+testlog "TEST: mirror from non-default namespace to default namespace"
+rbd --cluster ${CLUSTER2} mirror pool enable ${POOL}/${NS1} ${MIRROR_POOL_MODE} --remote-namespace ""
+rbd --cluster ${CLUSTER1} mirror pool enable ${POOL} ${MIRROR_POOL_MODE} --remote-namespace ${NS1}
+create_image_and_enable_mirror ${CLUSTER2} ${POOL}/${NS1} ${image} ${RBD_MIRROR_MODE}
+wait_for_image_replay_started ${CLUSTER1} ${POOL} ${image}
+write_image ${CLUSTER2} ${POOL}/${NS1} ${image} 100
+wait_for_replay_complete ${CLUSTER1} ${CLUSTER2} ${POOL} ${POOL}/${NS1} ${image}
+wait_for_replaying_status_in_pool_dir ${CLUSTER1} ${POOL} ${image}
+compare_images ${CLUSTER1} ${CLUSTER2} ${POOL} ${POOL}/${NS1} ${image}
+remove_image_retry ${CLUSTER2} ${POOL}/${NS1} ${image}
+wait_for_image_present ${CLUSTER1} ${POOL} ${image} 'deleted'
+rbd --cluster ${CLUSTER1} mirror pool enable ${POOL} init-only
+rbd --cluster ${CLUSTER2} mirror pool disable ${POOL}/${NS1}
+rbd --cluster ${CLUSTER1} mirror pool enable ${POOL} ${MIRROR_POOL_MODE}
+rbd --cluster ${CLUSTER2} mirror pool enable ${POOL} ${MIRROR_POOL_MODE}
+rbd --cluster ${CLUSTER1} mirror pool enable ${POOL}/${NS1} ${MIRROR_POOL_MODE}
+rbd --cluster ${CLUSTER2} mirror pool enable ${POOL}/${NS1} ${MIRROR_POOL_MODE}
+
+testlog "TEST: add image and test replay"
create_image_and_enable_mirror ${CLUSTER2} ${POOL} ${image} ${RBD_MIRROR_MODE}
set_image_meta ${CLUSTER2} ${POOL} ${image} "key1" "value1"
set_image_meta ${CLUSTER2} ${POOL} ${image} "key2" "value2"
break;
case cls::rbd::MIRROR_MODE_IMAGE:
case cls::rbd::MIRROR_MODE_POOL:
+ case cls::rbd::MIRROR_MODE_INIT_ONLY:
enabled = true;
break;
default:
int r = read_key(hctx, mirror::MODE, &mirror_mode);
if (r < 0 && r != -ENOENT) {
return r;
- } else if (r == 0 && mirror_mode != cls::rbd::MIRROR_MODE_DISABLED) {
+ } else if (r == 0 && (mirror_mode != cls::rbd::MIRROR_MODE_DISABLED &&
+ mirror_mode != cls::rbd::MIRROR_MODE_INIT_ONLY)) {
CLS_ERR("cannot set mirror remote namespace while mirroring enabled");
return -EINVAL;
}
case MIRROR_MODE_POOL:
os << "pool";
break;
+ case MIRROR_MODE_INIT_ONLY:
+ os << "init-only";
+ break;
default:
os << "unknown (" << static_cast<uint32_t>(mirror_mode) << ")";
break;
}
enum MirrorMode {
- MIRROR_MODE_DISABLED = 0,
- MIRROR_MODE_IMAGE = 1,
- MIRROR_MODE_POOL = 2
+ MIRROR_MODE_DISABLED = 0,
+ MIRROR_MODE_IMAGE = 1,
+ MIRROR_MODE_POOL = 2,
+ MIRROR_MODE_INIT_ONLY = 3
};
enum GroupImageLinkState {
typedef enum {
RBD_MIRROR_MODE_DISABLED, /* mirroring is disabled */
RBD_MIRROR_MODE_IMAGE, /* mirroring enabled on a per-image basis */
- RBD_MIRROR_MODE_POOL /* mirroring enabled on all journaled images */
+ RBD_MIRROR_MODE_POOL, /* mirroring enabled on all journaled images */
+ RBD_MIRROR_MODE_INIT_ONLY /* mirroring is initialized but not enabled
+ (default namespace only) */
} rbd_mirror_mode_t;
typedef enum {
return r;
}
- if (mirror_mode == cls::rbd::MIRROR_MODE_DISABLED) {
+ if (mirror_mode == cls::rbd::MIRROR_MODE_DISABLED ||
+ mirror_mode == cls::rbd::MIRROR_MODE_INIT_ONLY) {
ldout(m_cct, 10) << "mirroring is not enabled for destination pool"
<< dendl;
return 0;
return r;
}
- if (mirror_mode == cls::rbd::MIRROR_MODE_DISABLED) {
+ if (mirror_mode == cls::rbd::MIRROR_MODE_DISABLED ||
+ mirror_mode == cls::rbd::MIRROR_MODE_INIT_ONLY) {
lderr(cct) << "cannot enable mirroring: mirroring is not enabled on a "
<< pool_or_namespace(ictx) << dendl;
return -EINVAL;
case cls::rbd::MIRROR_MODE_DISABLED:
case cls::rbd::MIRROR_MODE_IMAGE:
case cls::rbd::MIRROR_MODE_POOL:
+ case cls::rbd::MIRROR_MODE_INIT_ONLY:
*mirror_mode = static_cast<rbd_mirror_mode_t>(mirror_mode_internal);
break;
default:
case RBD_MIRROR_MODE_DISABLED:
case RBD_MIRROR_MODE_IMAGE:
case RBD_MIRROR_MODE_POOL:
+ case RBD_MIRROR_MODE_INIT_ONLY:
next_mirror_mode = static_cast<cls::rbd::MirrorMode>(mirror_mode);
break;
default:
return -EINVAL;
}
+ if (next_mirror_mode == cls::rbd::MIRROR_MODE_INIT_ONLY &&
+ !io_ctx.get_namespace().empty()) {
+ lderr(cct) << "init-only mode cannot be set on a namespace" << dendl;
+ return -EINVAL;
+ }
+
int r;
if (next_mirror_mode == cls::rbd::MIRROR_MODE_DISABLED) {
// fail early if pool still has peers registered and attempting to disable
}
}
}
- } else if (next_mirror_mode == cls::rbd::MIRROR_MODE_DISABLED) {
+ } else if (next_mirror_mode == cls::rbd::MIRROR_MODE_DISABLED ||
+ next_mirror_mode == cls::rbd::MIRROR_MODE_INIT_ONLY) {
while (true) {
bool retry_busy = false;
bool pending_busy = false;
const std::string& remote_namespace) {
CephContext *cct = reinterpret_cast<CephContext *>(io_ctx.cct());
ldout(cct, 20) << dendl;
-
- std::string local_namespace = io_ctx.get_namespace();
-
- if (local_namespace.empty() && !remote_namespace.empty()) {
- lderr(cct) << "cannot mirror the default namespace to a "
- << "non-default namespace." << dendl;
- return -EINVAL;
- }
-
- if (!local_namespace.empty() && remote_namespace.empty()) {
- lderr(cct) << "cannot mirror a non-default namespace to the default "
- << "namespace." << dendl;
- return -EINVAL;
- }
int r = cls_client::mirror_remote_namespace_set(&io_ctx, remote_namespace);
if (r < 0) {
mirror_mode = "image"
elif mirror_mode == rbd.RBD_MIRROR_MODE_POOL:
mirror_mode = "pool"
+ elif mirror_mode == rbd.RBD_MIRROR_MODE_INIT_ONLY:
+ mirror_mode = "init-only"
else:
mirror_mode = "unknown"
MIRROR_MODES = {
rbd.RBD_MIRROR_MODE_DISABLED: 'disabled',
rbd.RBD_MIRROR_MODE_IMAGE: 'image',
- rbd.RBD_MIRROR_MODE_POOL: 'pool'
+ rbd.RBD_MIRROR_MODE_POOL: 'pool',
+ rbd.RBD_MIRROR_MODE_INIT_ONLY: 'init-only'
}
@handle_rbd_mirror_error()
_RBD_MIRROR_MODE_DISABLED "RBD_MIRROR_MODE_DISABLED"
_RBD_MIRROR_MODE_IMAGE "RBD_MIRROR_MODE_IMAGE"
_RBD_MIRROR_MODE_POOL "RBD_MIRROR_MODE_POOL"
+ _RBD_MIRROR_MODE_INIT_ONLY "RBD_MIRROR_MODE_INIT_ONLY"
ctypedef enum rbd_mirror_peer_direction_t:
_RBD_MIRROR_PEER_DIRECTION_RX "RBD_MIRROR_PEER_DIRECTION_RX"
_RBD_MIRROR_MODE_DISABLED "RBD_MIRROR_MODE_DISABLED"
_RBD_MIRROR_MODE_IMAGE "RBD_MIRROR_MODE_IMAGE"
_RBD_MIRROR_MODE_POOL "RBD_MIRROR_MODE_POOL"
+ _RBD_MIRROR_MODE_INIT_ONLY "RBD_MIRROR_MODE_INIT_ONLY"
ctypedef enum rbd_mirror_peer_direction_t:
_RBD_MIRROR_PEER_DIRECTION_RX "RBD_MIRROR_PEER_DIRECTION_RX"
RBD_MIRROR_MODE_DISABLED = _RBD_MIRROR_MODE_DISABLED
RBD_MIRROR_MODE_IMAGE = _RBD_MIRROR_MODE_IMAGE
RBD_MIRROR_MODE_POOL = _RBD_MIRROR_MODE_POOL
+RBD_MIRROR_MODE_INIT_ONLY = _RBD_MIRROR_MODE_INIT_ONLY
RBD_MIRROR_PEER_DIRECTION_RX = _RBD_MIRROR_PEER_DIRECTION_RX
RBD_MIRROR_PEER_DIRECTION_TX = _RBD_MIRROR_PEER_DIRECTION_TX
Positional arguments
<pool-spec> pool specification
(example: <pool-name>[/<namespace>]
- <mode> mirror mode [image or pool]
+ <mode> mirror mode [image, pool or init-only]
Optional arguments
-p [ --pool ] arg pool name
rbd: image name was not specified
[22]
$ rbd mirror pool enable rbd
- rbd: must specify 'image' or 'pool' mode.
+ rbd: mirror mode was not specified
[22]
$ rbd mirror pool peer add rbd
rbd: remote cluster was not specified
RBD_FEATURE_DEEP_FLATTEN, RBD_FEATURE_FAST_DIFF,
RBD_FEATURE_OBJECT_MAP,
RBD_MIRROR_MODE_DISABLED, RBD_MIRROR_MODE_IMAGE,
- RBD_MIRROR_MODE_POOL, RBD_MIRROR_IMAGE_ENABLED,
- RBD_MIRROR_IMAGE_DISABLED, MIRROR_IMAGE_STATUS_STATE_UNKNOWN,
+ RBD_MIRROR_MODE_POOL, RBD_MIRROR_MODE_INIT_ONLY,
+ RBD_MIRROR_IMAGE_ENABLED, RBD_MIRROR_IMAGE_DISABLED,
+ MIRROR_IMAGE_STATUS_STATE_UNKNOWN,
RBD_MIRROR_IMAGE_MODE_JOURNAL, RBD_MIRROR_IMAGE_MODE_SNAPSHOT,
RBD_LOCK_MODE_EXCLUSIVE, RBD_OPERATION_FEATURE_GROUP,
RBD_OPERATION_FEATURE_CLONE_CHILD,
eq(rados.get_fsid(), self.rbd.mirror_site_name_get(rados))
def test_mirror_remote_namespace(self):
- remote_namespace = "remote-ns"
- # cannot set remote namespace for the default namespace
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
+ # cannot set remote namespace while mirroring enabled
+ assert_raises(InvalidArgument, self.rbd.mirror_remote_namespace_set,
+ ioctx, "remote-ns1")
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
assert_raises(InvalidArgument, self.rbd.mirror_remote_namespace_set,
- ioctx, remote_namespace)
+ ioctx, "")
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
+ self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_INIT_ONLY)
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
+ # set remote namespace for the default namespace while in init-only
+ self.rbd.mirror_remote_namespace_set(ioctx, "remote-ns1")
+ eq("remote-ns1", self.rbd.mirror_remote_namespace_get(ioctx))
+ # reset remote namespace for the default namespace while in init-only
+ self.rbd.mirror_remote_namespace_set(ioctx, "")
eq("", self.rbd.mirror_remote_namespace_get(ioctx))
+ # change remote namespace for the default namespace while in init-only
+ self.rbd.mirror_remote_namespace_set(ioctx, "remote-ns2")
+ eq("remote-ns2", self.rbd.mirror_remote_namespace_get(ioctx))
+ # disabling mirroring also resets remote namespace
+ self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_DISABLED)
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
+ # set remote namespace for the default namespace while mirroring disabled
+ self.rbd.mirror_remote_namespace_set(ioctx, "remote-ns3")
+ eq("remote-ns3", self.rbd.mirror_remote_namespace_get(ioctx))
+ # reset remote namespace for the default namespace while mirroring disabled
+ self.rbd.mirror_remote_namespace_set(ioctx, "")
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
+ # change remote namespace for the default namespace while mirroring disabled
+ self.rbd.mirror_remote_namespace_set(ioctx, "remote-ns4")
+ eq("remote-ns4", self.rbd.mirror_remote_namespace_get(ioctx))
+ # moving to init-only or an enabled state preserves remote namespace
+ self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_INIT_ONLY)
+ eq("remote-ns4", self.rbd.mirror_remote_namespace_get(ioctx))
+ self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_POOL)
+ eq("remote-ns4", self.rbd.mirror_remote_namespace_get(ioctx))
+ self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_IMAGE)
+ eq("remote-ns4", self.rbd.mirror_remote_namespace_get(ioctx))
+
self.rbd.namespace_create(ioctx, "ns1")
ioctx.set_namespace("ns1")
+ eq("ns1", self.rbd.mirror_remote_namespace_get(ioctx))
+ # set remote namespace on a non-default namespace while mirroring disabled
+ self.rbd.mirror_remote_namespace_set(ioctx, "remote-ns1")
+ eq("remote-ns1", self.rbd.mirror_remote_namespace_get(ioctx))
+ # reset remote namespace on a non-default namespace while mirroring disabled
+ self.rbd.mirror_remote_namespace_set(ioctx, "ns1")
+ eq("ns1", self.rbd.mirror_remote_namespace_get(ioctx))
+ # change remote namespace on a non-default namespace while mirroring disabled
+ self.rbd.mirror_remote_namespace_set(ioctx, "remote-ns2")
+ eq("remote-ns2", self.rbd.mirror_remote_namespace_get(ioctx))
+ # cannot move to init-only on a non-default namespace
+ assert_raises(InvalidArgument, self.rbd.mirror_mode_set,
+ ioctx, RBD_MIRROR_MODE_INIT_ONLY)
+ eq(RBD_MIRROR_MODE_DISABLED, self.rbd.mirror_mode_get(ioctx))
+ # moving to an enabled state preserves remote namespace
self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_IMAGE)
+ eq("remote-ns2", self.rbd.mirror_remote_namespace_get(ioctx))
+ self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_POOL)
+ eq("remote-ns2", self.rbd.mirror_remote_namespace_get(ioctx))
+ # disabling mirroring also resets remote namespace
+ self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_DISABLED)
+ eq("ns1", self.rbd.mirror_remote_namespace_get(ioctx))
+ # set remote namespace on a non-default namespace to the default namespace
+ self.rbd.mirror_remote_namespace_set(ioctx, "")
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
+ # moving to an enabled state preserves remote namespace
+ self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_POOL)
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
+ self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_IMAGE)
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
# cannot set remote namespace while mirroring enabled
assert_raises(InvalidArgument, self.rbd.mirror_remote_namespace_set,
- ioctx, remote_namespace)
- self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_DISABLED)
- # cannot set remote namespace to the default namespace
+ ioctx, "remote-ns1")
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
+ assert_raises(InvalidArgument, self.rbd.mirror_remote_namespace_set,
+ ioctx, "ns1")
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
assert_raises(InvalidArgument, self.rbd.mirror_remote_namespace_set,
ioctx, "")
- self.rbd.mirror_remote_namespace_set(ioctx, remote_namespace)
- eq(remote_namespace, self.rbd.mirror_remote_namespace_get(ioctx))
+ eq("", self.rbd.mirror_remote_namespace_get(ioctx))
+ # disabling mirroring clears remote namespace
+ self.rbd.mirror_mode_set(ioctx, RBD_MIRROR_MODE_DISABLED)
+ eq("ns1", self.rbd.mirror_remote_namespace_get(ioctx))
ioctx.set_namespace("")
self.rbd.namespace_remove(ioctx, "ns1")
#include "tools/rbd_mirror/Threads.h"
#include "common/Formatter.h"
+#include <atomic>
+
namespace librbd {
namespace {
Return(r)));
}
- void expect_mirror_mode_get(librados::MockTestMemIoCtxImpl *io_ctx_impl,
- cls::rbd::MirrorMode mirror_mode, int r) {
- bufferlist out_bl;
- encode(mirror_mode, out_bl);
-
- EXPECT_CALL(*io_ctx_impl,
- exec(RBD_MIRRORING, _, StrEq("rbd"), StrEq("mirror_mode_get"),
- _, _, _, _))
- .WillOnce(DoAll(WithArg<5>(Invoke([out_bl](bufferlist *bl) {
- *bl = out_bl;
- })),
- Return(r)));
- }
-
- void expect_mirror_mode_get(librados::MockTestMemIoCtxImpl *io_ctx_impl) {
+ void expect_mirror_mode_get(librados::MockTestMemIoCtxImpl* io_ctx_impl,
+ std::atomic<bool>* default_namespace_enabled,
+ int r) {
EXPECT_CALL(*io_ctx_impl,
exec(RBD_MIRRORING, _, StrEq("rbd"), StrEq("mirror_mode_get"),
_, _, _, _))
- .WillRepeatedly(DoAll(WithArg<5>(Invoke([](bufferlist *bl) {
- encode(cls::rbd::MIRROR_MODE_POOL, *bl);
+ .WillRepeatedly(DoAll(WithArg<5>(Invoke(
+ [io_ctx_impl, default_namespace_enabled](bufferlist* bl) {
+ if (io_ctx_impl->get_namespace().empty() &&
+ *default_namespace_enabled == false) {
+ encode(cls::rbd::MIRROR_MODE_DISABLED, *bl);
+ } else {
+ encode(cls::rbd::MIRROR_MODE_POOL, *bl);
+ }
})),
- Return(0)));
+ Return(r)));
}
void expect_mirror_remote_namespace_get(
peer_spec.mon_host = "123";
peer_spec.key = "234";
- auto mock_default_namespace_replayer = new MockNamespaceReplayer();
- expect_namespace_replayer_is_blocklisted(*mock_default_namespace_replayer,
- false);
-
MockThreads mock_threads(m_threads);
expect_work_queue(mock_threads);
auto mock_remote_pool_poller = new MockRemotePoolPoller();
expect_remote_pool_poller_init(*mock_remote_pool_poller,
{"remote mirror uuid", ""}, 0);
- expect_namespace_replayer_init(*mock_default_namespace_replayer, 0);
expect_leader_watcher_init(*mock_leader_watcher, 0);
MockServiceDaemon mock_service_daemon;
remote_cct->put();
expect_leader_watcher_shut_down(*mock_leader_watcher);
- expect_namespace_replayer_shut_down(*mock_default_namespace_replayer);
expect_remote_pool_poller_shut_down(*mock_remote_pool_poller, 0);
pool_replayer.shut_down();
TEST_F(TestMockPoolReplayer, AcquireReleaseLeader) {
PeerSpec peer_spec{"uuid", "cluster name", "client.name"};
- peer_spec.mon_host = "123";
- peer_spec.key = "234";
-
- auto mock_default_namespace_replayer = new MockNamespaceReplayer();
- expect_namespace_replayer_is_blocklisted(*mock_default_namespace_replayer,
- false);
MockThreads mock_threads(m_threads);
expect_work_queue(mock_threads);
auto mock_remote_pool_poller = new MockRemotePoolPoller();
expect_remote_pool_poller_init(*mock_remote_pool_poller,
{"remote mirror uuid", ""}, 0);
- expect_namespace_replayer_init(*mock_default_namespace_replayer, 0);
expect_leader_watcher_init(*mock_leader_watcher, 0);
MockServiceDaemon mock_service_daemon;
m_local_io_ctx.get_id(), peer_spec, {});
pool_replayer.init("siteA");
+ expect_service_daemon_add_or_update_attribute(
+ mock_service_daemon, SERVICE_DAEMON_LEADER_KEY, true);
+
+ C_SaferCond on_acquire;
+ mock_leader_watcher->listener->post_acquire_handler(&on_acquire);
+ ASSERT_EQ(0, on_acquire.wait());
+
+ expect_service_daemon_remove_attribute(mock_service_daemon,
+ SERVICE_DAEMON_LEADER_KEY);
+
+ C_SaferCond on_release;
+ mock_leader_watcher->listener->pre_release_handler(&on_release);
+ ASSERT_EQ(0, on_release.wait());
+
+ expect_leader_watcher_shut_down(*mock_leader_watcher);
+ expect_remote_pool_poller_shut_down(*mock_remote_pool_poller, 0);
+
+ pool_replayer.shut_down();
+}
+
+TEST_F(TestMockPoolReplayer, DefaultNamespaceEnabled) {
+ PeerSpec peer_spec{"uuid", "cluster name", "client.name"};
+
+ std::atomic<bool> default_namespace_enabled = true;
+ auto mock_default_namespace_replayer = new MockNamespaceReplayer();
+ expect_namespace_replayer_is_blocklisted(*mock_default_namespace_replayer,
+ false);
+ expect_namespace_replayer_get_remote_namespace(*mock_default_namespace_replayer,
+ "");
+
+ MockThreads mock_threads(m_threads);
+ expect_work_queue(mock_threads);
+
+ auto mock_leader_watcher = new MockLeaderWatcher();
+ expect_leader_watcher_get_leader_instance_id(*mock_leader_watcher);
+ expect_leader_watcher_list_instances(*mock_leader_watcher);
+ expect_leader_watcher_is_blocklisted(*mock_leader_watcher, false);
+
+ auto& mock_cluster = get_mock_cluster();
+ auto mock_local_rados_client = mock_cluster.do_create_rados_client(
+ g_ceph_context);
+ auto mock_local_io_ctx = mock_local_rados_client->do_create_ioctx(
+ m_local_io_ctx.get_id(), m_local_io_ctx.get_pool_name());
+ auto mock_remote_rados_client = mock_cluster.do_create_rados_client(
+ g_ceph_context);
+
+ expect_clone(mock_local_io_ctx);
+ expect_mirror_mode_get(mock_local_io_ctx, &default_namespace_enabled, 0);
+ expect_mirror_remote_namespace_get(mock_local_io_ctx, "", -ENOENT);
+
+ InSequence seq;
+
+ expect_connect(mock_cluster, mock_local_rados_client, "ceph", nullptr);
+ expect_connect(mock_cluster, mock_remote_rados_client, "cluster name",
+ nullptr);
+ expect_create_ioctx(mock_local_rados_client, mock_local_io_ctx);
+ expect_mirror_uuid_get(mock_local_io_ctx, "uuid", 0);
+ auto mock_remote_pool_poller = new MockRemotePoolPoller();
+ expect_remote_pool_poller_init(*mock_remote_pool_poller,
+ {"remote mirror uuid", ""}, 0);
+ expect_leader_watcher_init(*mock_leader_watcher, 0);
+
+ MockServiceDaemon mock_service_daemon;
+ std::string instance_id = stringify(mock_local_io_ctx->get_instance_id());
+ expect_service_daemon_add_or_update_instance_id_attribute(
+ mock_service_daemon, instance_id);
+
+ C_SaferCond on_default_namespace_init;
+ expect_namespace_replayer_init(*mock_default_namespace_replayer, 0);
+ expect_service_daemon_add_namespace(mock_service_daemon, "");
+ expect_namespace_replayer_handle_update_leader(*mock_default_namespace_replayer,
+ "", &on_default_namespace_init);
+
+ MockPoolReplayer pool_replayer(&mock_threads, &mock_service_daemon, nullptr,
+ &m_pool_meta_cache,
+ m_local_io_ctx.get_id(), peer_spec, {});
+ pool_replayer.init("siteA");
+
+ ASSERT_EQ(0, on_default_namespace_init.wait());
+
expect_service_daemon_add_or_update_attribute(
mock_service_daemon, SERVICE_DAEMON_LEADER_KEY, true);
expect_namespace_replayer_handle_acquire_leader(
mock_leader_watcher->listener->pre_release_handler(&on_release);
ASSERT_EQ(0, on_release.wait());
- expect_leader_watcher_shut_down(*mock_leader_watcher);
+ expect_service_daemon_remove_namespace(mock_service_daemon, "");
expect_namespace_replayer_shut_down(*mock_default_namespace_replayer);
+ expect_leader_watcher_shut_down(*mock_leader_watcher);
+ expect_remote_pool_poller_shut_down(*mock_remote_pool_poller, 0);
+
+ pool_replayer.shut_down();
+}
+
+TEST_F(TestMockPoolReplayer, DefaultNamespaceEnableDisable) {
+ PeerSpec peer_spec{"uuid", "cluster name", "client.name"};
+
+ g_ceph_context->_conf.set_val(
+ "rbd_mirror_pool_replayers_refresh_interval", "1");
+
+ std::atomic<bool> default_namespace_enabled = false;
+ auto mock_default_namespace_replayer = new MockNamespaceReplayer();
+ expect_namespace_replayer_is_blocklisted(*mock_default_namespace_replayer,
+ false);
+ expect_namespace_replayer_get_remote_namespace(*mock_default_namespace_replayer,
+ "");
+
+ MockThreads mock_threads(m_threads);
+ expect_work_queue(mock_threads);
+
+ auto mock_leader_watcher = new MockLeaderWatcher();
+ expect_leader_watcher_get_leader_instance_id(*mock_leader_watcher);
+ expect_leader_watcher_list_instances(*mock_leader_watcher);
+ expect_leader_watcher_is_blocklisted(*mock_leader_watcher, false);
+
+ auto& mock_cluster = get_mock_cluster();
+ auto mock_local_rados_client = mock_cluster.do_create_rados_client(
+ g_ceph_context);
+ auto mock_local_io_ctx = mock_local_rados_client->do_create_ioctx(
+ m_local_io_ctx.get_id(), m_local_io_ctx.get_pool_name());
+ auto mock_remote_rados_client = mock_cluster.do_create_rados_client(
+ g_ceph_context);
+
+ expect_clone(mock_local_io_ctx);
+ expect_mirror_mode_get(mock_local_io_ctx, &default_namespace_enabled, 0);
+ expect_mirror_remote_namespace_get(mock_local_io_ctx, "", -ENOENT);
+
+ InSequence seq;
+
+ expect_connect(mock_cluster, mock_local_rados_client, "ceph", nullptr);
+ expect_connect(mock_cluster, mock_remote_rados_client, "cluster name",
+ nullptr);
+ expect_create_ioctx(mock_local_rados_client, mock_local_io_ctx);
+ expect_mirror_uuid_get(mock_local_io_ctx, "uuid", 0);
+ auto mock_remote_pool_poller = new MockRemotePoolPoller();
+ expect_remote_pool_poller_init(*mock_remote_pool_poller,
+ {"remote mirror uuid", ""}, 0);
+ expect_leader_watcher_init(*mock_leader_watcher, 0);
+
+ MockServiceDaemon mock_service_daemon;
+ std::string instance_id = stringify(mock_local_io_ctx->get_instance_id());
+ expect_service_daemon_add_or_update_instance_id_attribute(
+ mock_service_daemon, instance_id);
+
+ MockPoolReplayer pool_replayer(&mock_threads, &mock_service_daemon, nullptr,
+ &m_pool_meta_cache,
+ m_local_io_ctx.get_id(), peer_spec, {});
+ pool_replayer.init("siteA");
+
+ expect_service_daemon_add_or_update_attribute(
+ mock_service_daemon, SERVICE_DAEMON_LEADER_KEY, true);
+
+ C_SaferCond on_acquire;
+ mock_leader_watcher->listener->post_acquire_handler(&on_acquire);
+ ASSERT_EQ(0, on_acquire.wait());
+
+ C_SaferCond on_default_namespace_acquire;
+ expect_namespace_replayer_init(*mock_default_namespace_replayer, 0);
+ expect_service_daemon_add_namespace(mock_service_daemon, "");
+ expect_namespace_replayer_handle_acquire_leader(
+ *mock_default_namespace_replayer, 0, &on_default_namespace_acquire);
+ expect_namespace_replayer_handle_instances_added(
+ *mock_default_namespace_replayer);
+
+ default_namespace_enabled = true;
+ ASSERT_EQ(0, on_default_namespace_acquire.wait());
+
+ C_SaferCond on_default_namespace_shut_down;
+ expect_service_daemon_remove_namespace(mock_service_daemon, "");
+ expect_namespace_replayer_shut_down(*mock_default_namespace_replayer,
+ &on_default_namespace_shut_down);
+
+ default_namespace_enabled = false;
+ ASSERT_EQ(0, on_default_namespace_shut_down.wait());
+
+ expect_service_daemon_remove_attribute(mock_service_daemon,
+ SERVICE_DAEMON_LEADER_KEY);
+
+ C_SaferCond on_release;
+ mock_leader_watcher->listener->pre_release_handler(&on_release);
+ ASSERT_EQ(0, on_release.wait());
+
+ expect_leader_watcher_shut_down(*mock_leader_watcher);
expect_remote_pool_poller_shut_down(*mock_remote_pool_poller, 0);
pool_replayer.shut_down();
TEST_F(TestMockPoolReplayer, Namespaces) {
PeerSpec peer_spec{"uuid", "cluster name", "client.name"};
- peer_spec.mon_host = "123";
- peer_spec.key = "234";
g_ceph_context->_conf.set_val(
"rbd_mirror_pool_replayers_refresh_interval", "1");
MockNamespace mock_namespace;
+ std::atomic<bool> default_namespace_enabled = true;
auto mock_default_namespace_replayer = new MockNamespaceReplayer();
expect_namespace_replayer_is_blocklisted(*mock_default_namespace_replayer,
false);
+ expect_namespace_replayer_get_remote_namespace(*mock_default_namespace_replayer,
+ "");
auto mock_ns1_namespace_replayer = new MockNamespaceReplayer("ns1");
expect_namespace_replayer_is_blocklisted(*mock_ns1_namespace_replayer,
g_ceph_context);
expect_clone(mock_local_io_ctx);
- expect_mirror_mode_get(mock_local_io_ctx);
+ expect_mirror_mode_get(mock_local_io_ctx, &default_namespace_enabled, 0);
expect_mirror_remote_namespace_get(mock_local_io_ctx, "", -ENOENT);
InSequence seq;
auto mock_remote_pool_poller = new MockRemotePoolPoller();
expect_remote_pool_poller_init(*mock_remote_pool_poller,
{"remote mirror uuid", ""}, 0);
- expect_namespace_replayer_init(*mock_default_namespace_replayer, 0);
expect_leader_watcher_init(*mock_leader_watcher, 0);
MockServiceDaemon mock_service_daemon;
expect_service_daemon_add_or_update_instance_id_attribute(
mock_service_daemon, instance_id);
+ C_SaferCond on_default_namespace_init;
+ expect_namespace_replayer_init(*mock_default_namespace_replayer, 0);
+ expect_service_daemon_add_namespace(mock_service_daemon, "");
+ expect_namespace_replayer_handle_update_leader(*mock_default_namespace_replayer,
+ "", &on_default_namespace_init);
+
MockPoolReplayer pool_replayer(&mock_threads, &mock_service_daemon, nullptr,
&m_pool_meta_cache,
m_local_io_ctx.get_id(), peer_spec, {});
pool_replayer.init("siteA");
+ ASSERT_EQ(0, on_default_namespace_init.wait());
+
C_SaferCond on_ns1_init;
expect_namespace_replayer_init(*mock_ns1_namespace_replayer, 0);
expect_service_daemon_add_namespace(mock_service_daemon, "ns1");
mock_leader_watcher->listener->post_acquire_handler(&on_acquire);
ASSERT_EQ(0, on_acquire.wait());
+ C_SaferCond on_ns2_acquire;
expect_namespace_replayer_init(*mock_ns2_namespace_replayer, 0);
expect_service_daemon_add_namespace(mock_service_daemon, "ns2");
- C_SaferCond on_ns2_acquire;
expect_namespace_replayer_handle_acquire_leader(
*mock_ns2_namespace_replayer, 0, &on_ns2_acquire);
expect_namespace_replayer_handle_instances_added(
expect_service_daemon_remove_namespace(mock_service_daemon, "ns2");
expect_namespace_replayer_shut_down(*mock_ns2_namespace_replayer,
&on_ns2_shut_down);
+
mock_namespace.remove("ns2");
ASSERT_EQ(0, on_ns2_shut_down.wait());
mock_leader_watcher->listener->pre_release_handler(&on_release);
ASSERT_EQ(0, on_release.wait());
+ expect_service_daemon_remove_namespace(mock_service_daemon, "");
+ expect_namespace_replayer_shut_down(*mock_default_namespace_replayer);
expect_service_daemon_remove_namespace(mock_service_daemon, "ns1");
expect_namespace_replayer_shut_down(*mock_ns1_namespace_replayer);
expect_leader_watcher_shut_down(*mock_leader_watcher);
- expect_namespace_replayer_shut_down(*mock_default_namespace_replayer);
expect_remote_pool_poller_shut_down(*mock_remote_pool_poller, 0);
pool_replayer.shut_down();
TEST_F(TestMockPoolReplayer, NamespacesError) {
PeerSpec peer_spec{"uuid", "cluster name", "client.name"};
- peer_spec.mon_host = "123";
- peer_spec.key = "234";
g_ceph_context->_conf.set_val(
"rbd_mirror_pool_replayers_refresh_interval", "1");
MockNamespace mock_namespace;
+ std::atomic<bool> default_namespace_enabled = true;
auto mock_default_namespace_replayer = new MockNamespaceReplayer();
expect_namespace_replayer_is_blocklisted(*mock_default_namespace_replayer,
false);
+ expect_namespace_replayer_get_remote_namespace(*mock_default_namespace_replayer,
+ "");
+
auto mock_ns1_namespace_replayer = new MockNamespaceReplayer("ns1");
+
auto mock_ns2_namespace_replayer = new MockNamespaceReplayer("ns2");
expect_namespace_replayer_is_blocklisted(*mock_ns2_namespace_replayer,
false);
+ expect_namespace_replayer_get_remote_namespace(*mock_ns2_namespace_replayer,
+ "ns2");
+
auto mock_ns3_namespace_replayer = new MockNamespaceReplayer("ns3");
MockThreads mock_threads(m_threads);
g_ceph_context);
expect_clone(mock_local_io_ctx);
- expect_mirror_mode_get(mock_local_io_ctx);
+ expect_mirror_mode_get(mock_local_io_ctx, &default_namespace_enabled, 0);
expect_mirror_remote_namespace_get(mock_local_io_ctx, "", -ENOENT);
InSequence seq;
auto mock_remote_pool_poller = new MockRemotePoolPoller();
expect_remote_pool_poller_init(*mock_remote_pool_poller,
{"remote mirror uuid", ""}, 0);
- expect_namespace_replayer_init(*mock_default_namespace_replayer, 0);
expect_leader_watcher_init(*mock_leader_watcher, 0);
MockServiceDaemon mock_service_daemon;
expect_service_daemon_add_or_update_instance_id_attribute(
mock_service_daemon, instance_id);
+ C_SaferCond on_default_namespace_init;
+ expect_namespace_replayer_init(*mock_default_namespace_replayer, 0);
+ expect_service_daemon_add_namespace(mock_service_daemon, "");
+ expect_namespace_replayer_handle_update_leader(*mock_default_namespace_replayer,
+ "", &on_default_namespace_init);
+
MockPoolReplayer pool_replayer(&mock_threads, &mock_service_daemon, nullptr,
&m_pool_meta_cache,
m_local_io_ctx.get_id(), peer_spec, {});
pool_replayer.init("siteA");
+ ASSERT_EQ(0, on_default_namespace_init.wait());
+
// test namespace replayer init fails for non leader
C_SaferCond on_ns1_init;
on_ns1_init.complete(r);
});
expect_namespace_replayer_init(*mock_ns1_namespace_replayer, -EINVAL, ctx);
+
mock_namespace.add("ns1");
ASSERT_EQ(-EINVAL, on_ns1_init.wait());
- // test acquire leader fails when default namespace replayer fails
-
- expect_service_daemon_add_or_update_attribute(
- mock_service_daemon, SERVICE_DAEMON_LEADER_KEY, true);
- expect_namespace_replayer_handle_acquire_leader(
- *mock_default_namespace_replayer, -EINVAL);
-
- C_SaferCond on_acquire1;
- mock_leader_watcher->listener->post_acquire_handler(&on_acquire1);
- ASSERT_EQ(-EINVAL, on_acquire1.wait());
-
- // test acquire leader succeeds when non-default namespace replayer fails
+ // test acquire leader succeeds even when all namespace replayers fail
C_SaferCond on_ns2_init;
expect_namespace_replayer_init(*mock_ns2_namespace_replayer, 0);
expect_service_daemon_add_namespace(mock_service_daemon, "ns2");
expect_namespace_replayer_handle_update_leader(*mock_ns2_namespace_replayer,
"", &on_ns2_init);
+
mock_namespace.add("ns2");
ASSERT_EQ(0, on_ns2_init.wait());
expect_service_daemon_add_or_update_attribute(
mock_service_daemon, SERVICE_DAEMON_LEADER_KEY, true);
expect_namespace_replayer_handle_acquire_leader(
- *mock_default_namespace_replayer, 0);
-
+ *mock_default_namespace_replayer, -EPERM);
expect_namespace_replayer_handle_acquire_leader(*mock_ns2_namespace_replayer,
-EINVAL);
ctx = new LambdaContext(
- [&mock_namespace](int) {
+ [&mock_namespace, &default_namespace_enabled](int) {
+ default_namespace_enabled = false;
mock_namespace.remove("ns2");
});
+ expect_service_daemon_remove_namespace(mock_service_daemon, "");
+ expect_namespace_replayer_shut_down(*mock_default_namespace_replayer, nullptr);
expect_service_daemon_remove_namespace(mock_service_daemon, "ns2");
expect_namespace_replayer_shut_down(*mock_ns2_namespace_replayer, ctx);
- mock_namespace.add("ns2");
- C_SaferCond on_acquire2;
- mock_leader_watcher->listener->post_acquire_handler(&on_acquire2);
- ASSERT_EQ(0, on_acquire2.wait());
+ C_SaferCond on_acquire;
+ mock_leader_watcher->listener->post_acquire_handler(&on_acquire);
+ ASSERT_EQ(0, on_acquire.wait());
// test namespace replayer init fails on acquire leader
-EINVAL);
expect_service_daemon_remove_namespace(mock_service_daemon, "ns3");
expect_namespace_replayer_shut_down(*mock_ns3_namespace_replayer, ctx);
+
mock_namespace.add("ns3");
ASSERT_EQ(0, on_ns3_shut_down.wait());
expect_leader_watcher_shut_down(*mock_leader_watcher);
- expect_namespace_replayer_shut_down(*mock_default_namespace_replayer);
expect_remote_pool_poller_shut_down(*mock_remote_pool_poller, 0);
pool_replayer.shut_down();
TEST_F(TestMockPoolReplayer, RemoveCalloutOnInit) {
PeerSpec peer_spec{"uuid", "cluster name", "client.name"};
- auto mock_default_namespace_replayer = new MockNamespaceReplayer();
- expect_namespace_replayer_is_blocklisted(*mock_default_namespace_replayer,
- false);
-
auto mock_leader_watcher = new MockLeaderWatcher();
expect_leader_watcher_get_leader_instance_id(*mock_leader_watcher);
expect_leader_watcher_is_blocklisted(*mock_leader_watcher, false);
expect_remote_pool_poller_init(*mock_remote_pool_poller,
{"remote mirror uuid", ""}, 0);
- expect_namespace_replayer_init(*mock_default_namespace_replayer, 0);
expect_leader_watcher_init(*mock_leader_watcher, 0);
expect_service_daemon_remove_callout(mock_service_daemon, 123);
pool_replayer.init("siteA");
expect_leader_watcher_shut_down(*mock_leader_watcher);
- expect_namespace_replayer_shut_down(*mock_default_namespace_replayer);
expect_remote_pool_poller_shut_down(*mock_remote_pool_poller, 0);
pool_replayer.shut_down();
po::options_description *options) {
at::add_pool_options(positional, options, true);
positional->add_options()
- ("mode", "mirror mode [image or pool]");
+ ("mode", "mirror mode [image, pool or init-only]");
add_site_name_optional(options);
options->add_options()
mirror_mode = RBD_MIRROR_MODE_IMAGE;
} else if (mode == "pool") {
mirror_mode = RBD_MIRROR_MODE_POOL;
+ } else if (mode == "init-only") {
+ mirror_mode = RBD_MIRROR_MODE_INIT_ONLY;
} else {
- std::cerr << "rbd: must specify 'image' or 'pool' mode." << std::endl;
+ std::cerr << "rbd: mirror mode was not specified" << std::endl;
return -EINVAL;
}
}
if (vm.count(REMOTE_NAMESPACE_NAME)) {
+ if (mirror_mode == RBD_MIRROR_MODE_INIT_ONLY) {
+ std::cerr << "rbd: cannot specify remote namespace for init-only mode"
+ << std::endl;
+ return -EINVAL;
+ }
remote_namespace = vm[REMOTE_NAMESPACE_NAME].as<std::string>();
} else {
remote_namespace = namespace_name;
}
- std::string original_remote_namespace;
librbd::RBD rbd;
- r = rbd.mirror_remote_namespace_get(io_ctx, &original_remote_namespace);
- if (r < 0) {
- std::cerr << "rbd: failed to get the current remote namespace."
- << std::endl;
- return r;
- }
- if (original_remote_namespace != remote_namespace) {
- r = rbd.mirror_remote_namespace_set(io_ctx, remote_namespace);
+ if (mirror_mode != RBD_MIRROR_MODE_INIT_ONLY) {
+ std::string original_remote_namespace;
+ r = rbd.mirror_remote_namespace_get(io_ctx, &original_remote_namespace);
if (r < 0) {
- std::cerr << "rbd: failed to set the remote namespace."
- << std::endl;
+ std::cerr << "rbd: failed to get the current remote namespace."
+ << std::endl;
return r;
}
+
+ if (original_remote_namespace != remote_namespace) {
+ r = rbd.mirror_remote_namespace_set(io_ctx, remote_namespace);
+ if (r < 0) {
+ std::cerr << "rbd: failed to set the remote namespace."
+ << std::endl;
+ return r;
+ }
+ }
}
bool updated = false;
case RBD_MIRROR_MODE_POOL:
mirror_mode_desc = "pool";
break;
+ case RBD_MIRROR_MODE_INIT_ONLY:
+ mirror_mode_desc = "init-only";
+ break;
default:
mirror_mode_desc = "unknown";
break;
std::lock_guard locker{m_lock};
+ f->open_object_section("namespace_replayer_status");
+ f->dump_string("local_namespace", m_local_namespace_name);
+ f->dump_string("remote_namespace", m_remote_namespace_name);
+
m_instance_replayer->print_status(f);
if (m_image_deleter) {
m_image_deleter->print_status(f);
f->close_section();
}
+
+ f->close_section(); // namespace_replayer_status
}
template <typename I>
m_pool_meta_cache->set_local_pool_meta(
m_local_io_ctx.get_id(), {m_local_mirror_uuid});
- m_default_namespace_replayer.reset(NamespaceReplayer<I>::create(
- "", "", m_local_io_ctx, m_remote_io_ctx, m_local_mirror_uuid, m_peer.uuid,
- m_remote_pool_meta, m_threads, m_image_sync_throttler.get(),
- m_image_deletion_throttler.get(), m_service_daemon,
- m_cache_manager_handler, m_pool_meta_cache));
-
- C_SaferCond on_init;
- m_default_namespace_replayer->init(&on_init);
- r = on_init.wait();
- if (r < 0) {
- derr << "error initializing default namespace replayer: " << cpp_strerror(r)
- << dendl;
- m_callout_id = m_service_daemon->add_or_update_callout(
- m_local_pool_id, m_callout_id, service_daemon::CALLOUT_LEVEL_ERROR,
- "unable to initialize default namespace replayer");
- m_default_namespace_replayer.reset();
- return;
- }
-
m_leader_watcher.reset(LeaderWatcher<I>::create(m_threads, m_local_io_ctx,
&m_leader_listener));
r = m_leader_watcher->init();
}
m_leader_watcher.reset();
- if (m_default_namespace_replayer) {
- C_SaferCond on_shut_down;
- m_default_namespace_replayer->shut_down(&on_shut_down);
- on_shut_down.wait();
- }
- m_default_namespace_replayer.reset();
-
if (m_remote_pool_poller) {
C_SaferCond ctx;
m_remote_pool_poller->shut_down(&ctx);
std::unique_lock locker{m_lock};
- if (m_leader_watcher->is_blocklisted() ||
- m_default_namespace_replayer->is_blocklisted()) {
+ if (m_leader_watcher->is_blocklisted()) {
m_blocklisted = true;
m_stopping = true;
+ break;
}
for (auto &it : m_namespace_replayers) {
ctx=gather_ctx->new_sub()](int r) {
std::lock_guard locker{m_lock};
if (r < 0) {
- derr << "failed to initialize namespace replayer for namespace "
- << names.first << ": " << cpp_strerror(r) << dendl;
+ derr << "failed to initialize namespace replayer for namespace '"
+ << names.first << "': " << cpp_strerror(r) << dendl;
delete namespace_replayer;
mirroring_namespaces.erase(names.first);
} else {
ceph_assert(ceph_mutex_is_locked(m_lock));
std::vector<std::string> names;
-
int r = librbd::api::Namespace<I>::list(m_local_io_ctx, &names);
if (r < 0) {
derr << "failed to list namespaces: " << cpp_strerror(r) << dendl;
return r;
}
+ // handle the default namespace the same way
+ names.push_back("");
+
for (auto &name : names) {
librados::IoCtx ns_ioctx;
ns_ioctx.dup(m_local_io_ctx);
ns_ioctx.set_namespace(name);
- std::string remote_namespace;
cls::rbd::MirrorMode mirror_mode = cls::rbd::MIRROR_MODE_DISABLED;
int r = librbd::cls_client::mirror_mode_get(&ns_ioctx, &mirror_mode);
if (r < 0 && r != -ENOENT) {
- derr << "failed to get namespace mirror mode: " << cpp_strerror(r)
- << dendl;
+ derr << "failed to get mirror mode for namespace '" << name << "': "
+ << cpp_strerror(r) << dendl;
if (m_namespace_replayers.count(name) == 0) {
continue;
}
- } else if (mirror_mode == cls::rbd::MIRROR_MODE_DISABLED) {
+ } else if (mirror_mode == cls::rbd::MIRROR_MODE_DISABLED ||
+ mirror_mode == cls::rbd::MIRROR_MODE_INIT_ONLY) {
dout(10) << "mirroring is disabled for namespace " << name << dendl;
continue;
}
+
+ std::string remote_namespace;
r = librbd::cls_client::mirror_remote_namespace_get(&ns_ioctx,
&remote_namespace);
- if (r < 0) {
+ if (r < 0) {
if (r != -ENOENT && r != -EOPNOTSUPP) {
- derr << "failed to get remote mirror namespace: " << cpp_strerror(r)
- << dendl;
+ derr << "failed to get remote namespace for namespace '" << name
+ << "': " << cpp_strerror(r) << dendl;
continue;
} else {
remote_namespace = name;
}
}
-
+
dout(10) << " local namespace=" << name << ", remote namespace="
<< remote_namespace << dendl;
namespaces->insert(std::make_pair(name, remote_namespace));
on_finish = new LambdaContext(
[this, name, on_finish](int r) {
if (r < 0) {
- derr << "failed to handle acquire leader for namespace: "
- << name << ": " << cpp_strerror(r) << dendl;
+ derr << "failed to handle acquire leader for namespace '"
+ << name << "': " << cpp_strerror(r) << dendl;
// remove the namespace replayer -- update_namespace_replayers will
// retry to create it and acquire leader.
f->close_section(); // deletion_throttler
}
- if (m_default_namespace_replayer) {
- m_default_namespace_replayer->print_status(f);
- }
-
- f->open_array_section("namespaces");
+ f->open_array_section("namespace_replayers");
for (auto &it : m_namespace_replayers) {
- f->open_object_section("namespace");
- f->dump_string("name", it.first);
it.second->print_status(f);
- f->close_section(); // namespace
}
- f->close_section(); // namespaces
+ f->close_section(); // namespace_replayers
f->close_section(); // pool_replayer_status
}
m_manual_stop = false;
- if (m_default_namespace_replayer) {
- m_default_namespace_replayer->start();
- }
for (auto &it : m_namespace_replayers) {
it.second->start();
}
m_manual_stop = true;
- if (m_default_namespace_replayer) {
- m_default_namespace_replayer->stop();
- }
for (auto &it : m_namespace_replayers) {
it.second->stop();
}
return;
}
- if (m_default_namespace_replayer) {
- m_default_namespace_replayer->restart();
- }
for (auto &it : m_namespace_replayers) {
it.second->restart();
}
return;
}
- if (m_default_namespace_replayer) {
- m_default_namespace_replayer->flush();
- }
for (auto &it : m_namespace_replayers) {
it.second->flush();
}
m_service_daemon->add_or_update_attribute(m_local_pool_id,
SERVICE_DAEMON_LEADER_KEY,
true);
- auto ctx = new LambdaContext(
+ auto ctx = librbd::util::create_async_context_callback(
+ m_threads->work_queue, new LambdaContext(
[this, on_finish](int r) {
if (r == 0) {
std::lock_guard locker{m_lock};
m_leader = true;
}
on_finish->complete(r);
- });
+ }));
auto cct = reinterpret_cast<CephContext *>(m_local_io_ctx.cct());
auto gather_ctx = new C_Gather(cct, ctx);
- m_default_namespace_replayer->handle_acquire_leader(
- gather_ctx->new_sub());
-
for (auto &it : m_namespace_replayers) {
namespace_replayer_acquire_leader(it.first, gather_ctx->new_sub());
}
m_leader = false;
m_service_daemon->remove_attribute(m_local_pool_id,
SERVICE_DAEMON_LEADER_KEY);
+ auto ctx = librbd::util::create_async_context_callback(
+ m_threads->work_queue, on_finish);
auto cct = reinterpret_cast<CephContext *>(m_local_io_ctx.cct());
- auto gather_ctx = new C_Gather(cct, on_finish);
-
- m_default_namespace_replayer->handle_release_leader(
- gather_ctx->new_sub());
+ auto gather_ctx = new C_Gather(cct, ctx);
for (auto &it : m_namespace_replayers) {
it.second->handle_release_leader(gather_ctx->new_sub());
std::lock_guard locker{m_lock};
- m_default_namespace_replayer->handle_update_leader(leader_instance_id);
-
for (auto &it : m_namespace_replayers) {
it.second->handle_update_leader(leader_instance_id);
}
return;
}
- m_default_namespace_replayer->handle_instances_added(instance_ids);
-
for (auto &it : m_namespace_replayers) {
it.second->handle_instances_added(instance_ids);
}
return;
}
- m_default_namespace_replayer->handle_instances_removed(instance_ids);
-
for (auto &it : m_namespace_replayers) {
it.second->handle_instances_removed(instance_ids);
}
const RemotePoolMeta& remote_pool_meta) {
dout(5) << "remote_pool_meta=" << remote_pool_meta << dendl;
- if (!m_default_namespace_replayer) {
+ if (!m_leader_watcher) {
m_remote_pool_meta = remote_pool_meta;
return;
}
std::unique_ptr<remote_pool_poller::Listener> m_remote_pool_poller_listener;
std::unique_ptr<RemotePoolPoller<ImageCtxT>> m_remote_pool_poller;
- std::unique_ptr<NamespaceReplayer<ImageCtxT>> m_default_namespace_replayer;
std::map<std::string, NamespaceReplayer<ImageCtxT> *> m_namespace_replayers;
std::string m_asok_hook_name;
void ServiceDaemon<I>::add_or_update_namespace_attribute(
int64_t pool_id, const std::string& namespace_name, const std::string& key,
const AttributeValue& value) {
- if (namespace_name.empty()) {
- add_or_update_attribute(pool_id, key, value);
- return;
- }
-
dout(20) << "pool_id=" << pool_id << ", "
<< "namespace=" << namespace_name << ", "
<< "key=" << key << ", "
boost::apply_visitor(attribute_dump_visitor, attribute.second);
}
- if (!pool_pair.second.ns_attributes.empty()) {
- f.open_object_section("namespaces");
- for (auto& [ns, attributes] : pool_pair.second.ns_attributes) {
- f.open_object_section(ns.c_str());
- for (auto& [key, value] : attributes) {
- AttributeDumpVisitor attribute_dump_visitor(&f, key);
- boost::apply_visitor(attribute_dump_visitor, value);
- }
- f.close_section(); // namespace
+ f.open_object_section("namespaces");
+ for (auto& [ns, attributes] : pool_pair.second.ns_attributes) {
+ f.open_object_section(ns.c_str());
+ for (auto& [key, value] : attributes) {
+ AttributeDumpVisitor attribute_dump_visitor(&f, key);
+ boost::apply_visitor(attribute_dump_visitor, value);
}
- f.close_section(); // namespaces
+ f.close_section(); // namespace
}
+ f.close_section(); // namespaces
+
f.close_section(); // pool
}
f.close_section(); // pools
projects / ceph.git / commitdiff