]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
David Zafman [Fri, 18 Aug 2017 17:00:10 +0000 (10:00 -0700)]
tests: run-standalone.sh skip core_pattern if already set
If sudo requires a password it is nice to be able to avoid
sudo sysctl by setting core_pattern ahead of time.
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
599d7622dd0bd7f73145cc229b576d5de7f04fda )
David Zafman [Fri, 18 Aug 2017 10:49:42 +0000 (03:49 -0700)]
tests: run-standalone.sh improve error message
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
175b10795d12922e833b3a33591918d8033fb407 )
David Zafman [Tue, 15 Aug 2017 01:17:09 +0000 (18:17 -0700)]
tests: qa/standalone: Add trap for signals to resetore the kernel core pattern
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
982aef784c7f8157a4fc162b38413f7516b3f76d )
David Zafman [Thu, 10 Aug 2017 01:58:21 +0000 (18:58 -0700)]
qa: Fix broken test_activate_osd() due to missing space
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
e24ac51a825e06dbd23819084a532ae14aeffac2 )
David Zafman [Wed, 9 Aug 2017 15:43:57 +0000 (08:43 -0700)]
qa: Fix races with waiting for scrubs
The trigger_scrub sets the last_scrub_stamp backwards to
force a scheduled scrub. In a small window this stamp could get propagated
to the mgr. A test failure occurred because wait_for_scrub() was confused
by seeing a backward moving date.
The most critical change is having wait_for_scrub() make sure that the
date advances past the previous in value.
A test failed because the random backoff kept delayed triggered scrub, so
set osd_scrub_backoff throughout.
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
ae2c5331fb5ec3b2c81724d19259302903aabdf5 )
David Zafman [Wed, 9 Aug 2017 14:19:36 +0000 (07:19 -0700)]
qa: Testing of ceph-helpers.sh, teardown on fail to dump logs, save cores
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
dddda523d1103847dc8b294d4998962c4bb6ddb4 )
David Zafman [Tue, 8 Aug 2017 14:22:33 +0000 (07:22 -0700)]
osd: Avoid confusion over legacy snaps when head_exists corrupt
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
1fe6cb0f0243eac384803d1a6e4a67c51090e94c )
David Zafman [Mon, 7 Aug 2017 19:48:27 +0000 (12:48 -0700)]
osd: Fix Paxos shutdown handling for commit_finish race
Fixes: http://tracker.ceph.com/issues/20921
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
c1404574895bc5a091f36b8119c7500fe1e287f1 )
David Zafman [Fri, 4 Aug 2017 19:34:47 +0000 (12:34 -0700)]
qa: Add support for core dumps
Save core dumps when running tests locally
Dump logs to output whenever cores seen
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
229de6b71d0b90de1ce95c9c692a132b3cf056b1 )
David Zafman [Thu, 3 Aug 2017 15:16:04 +0000 (08:16 -0700)]
qa: For FreeBSD skip osd-dup.sh because there is no bluestore
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
4db5124e1aa2b1071ecea1ee2cf62fb4eca7a779 )
David Zafman [Thu, 3 Aug 2017 13:06:03 +0000 (06:06 -0700)]
qa: Raise mon-data-avail-warn to pass tests with less space
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
61bfd236ad3d8c276ad7577b3ed6be613f47aec6 )
David Zafman [Wed, 2 Aug 2017 04:35:34 +0000 (21:35 -0700)]
qa: Add common generalized inject_eio() to ceph-helpers.sh
Retry for a while to allow pool to appear
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
574b3cd3d416a3ba3611775f871646ab137266f5 )
David Zafman [Wed, 2 Aug 2017 20:26:19 +0000 (13:26 -0700)]
qa: osd-scrub-repair.sh handle older versions of jq
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
3988ebab43c575ab3532052d22940248b8988b46 )
David Zafman [Tue, 1 Aug 2017 23:04:10 +0000 (16:04 -0700)]
qa: Add support for specifying sub-tests with run-standalone.sh
Fix test-ceph-helpers.sh to pass additional arguments on
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
2a679a36deac7bc9bfdeda8b6472ebaf113bbe19 )
David Zafman [Tue, 1 Aug 2017 22:19:01 +0000 (15:19 -0700)]
qa: ceph-helpers.sh fixes
Add missing teardown to cleanup test directory
Fix pgid due to elimination of initial default pool
Testing could never fail because run_tests return ignored
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
69413618a0f47edcb96111cfa012302f4a1df3d2 )
David Zafman [Tue, 1 Aug 2017 20:42:29 +0000 (13:42 -0700)]
qa: Skip ceph-helpers.sh which is tested through test-ceph-helpers.sh
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
d99f2a77e97c0aa7e6f0ae83e3c8731751730669 )
David Zafman [Mon, 31 Jul 2017 23:17:16 +0000 (16:17 -0700)]
qa: Add create_pool() which sleeps 1 second like python variant
wait_for_clean() can miss the new pool if it races with pool create.
Fixes: http://tracker.ceph.com/issues/20465
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
99ad4bbd91da6a8407c1ac85e0d6fa1e7711f546 )
David Zafman [Mon, 31 Jul 2017 16:28:12 +0000 (09:28 -0700)]
qa: Add special test_failure.sh script (not run by default)
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
b20dfc2864751d0ae75b63f997c5aa3c57242585 )
David Zafman [Mon, 31 Jul 2017 16:27:15 +0000 (09:27 -0700)]
qa: run-standalone.sh improvements
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
8c768050a59d61866398ca4c549f68663a0e09e9 )
David Zafman [Fri, 28 Jul 2017 22:06:13 +0000 (15:06 -0700)]
qa: Dump logs after daemons are killed to make sure everything is flushed
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
4314cdd66648fa81a8eda18f4b81621fd0bde5e3 )
Greg Farnum [Thu, 10 Aug 2017 17:28:09 +0000 (10:28 -0700)]
qa: test that "fs new" correctly set the application_metadata
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit
c85af7b14681be916cea1119f3f9cdf556e2707a )
Greg Farnum [Wed, 9 Aug 2017 21:34:44 +0000 (14:34 -0700)]
mdsmon: treat the osdmon correctly when doing plugged updates
Make sure it's writeable before invoking changes, and propose_pending()
on it when we're done.
Make the PaxosService::C_RetryMessage public so we can do this from FSCommands.
Maybe-
Fixes: http://tracker.ceph.com/issues/20959
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit
435717791ec499f71c9d1485b1e4e63239a343e2 )
Greg Farnum [Wed, 9 Aug 2017 20:46:30 +0000 (13:46 -0700)]
mdsmon: don't add pool application metadata until running fully-luminous
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit
bcd3554bc5a3965077af655498fd434910f13040 )
Sage Weil [Thu, 10 Aug 2017 18:41:40 +0000 (13:41 -0500)]
Merge pull request #16969 from linuxbox2/wip-disable-dynreshard-L
rgw: disable dynamic reshading for 1st L point release
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Matt Benjamin [Thu, 10 Aug 2017 14:40:25 +0000 (10:40 -0400)]
rgw: disable dynamic reshading for 1st L point release
Temporarily default RGW dynamic bucket resharding to OFF (usability
standup).
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Mark Nelson [Mon, 7 Aug 2017 14:23:12 +0000 (09:23 -0500)]
os/bluestore: clearer comments, not slower code.
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit
da92220d9d6936ef1aa19333038b7a1d982485a4 )
Sage Weil [Tue, 8 Aug 2017 22:43:22 +0000 (18:43 -0400)]
mon/Elector: force election epoch bump on start
We are generally careful when bumping the epoch so that we can join
existing rounds. However, if we restart in the middle of an election,
and change versions, we need to be certain that our previous ACK (as
$version - 1) isn't accepted as truth for the restarted daemon (running
$version) keeping the same epoch.
The conservatism with bumping is to avoid spurious election cycles, but
mon restarts are more rare, and we need them here.
Fixes: http://tracker.ceph.com/issues/20949
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
ef425374250014393c1d432a3eda95179bb70537 )
Sage Weil [Wed, 9 Aug 2017 13:11:05 +0000 (09:11 -0400)]
qa/suites/upgrade/kraken-x/stress-split: more whitelisting
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
b61be07d45e7cc1b1f4b606408bf4ed04853fb9b )
Alfredo Deza [Thu, 10 Aug 2017 15:58:15 +0000 (11:58 -0400)]
Merge pull request #16970 from ceph/backport-16919
Backport: "ceph-volume: adds functional CI testing #16919"
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Alfredo Deza [Thu, 10 Aug 2017 13:11:58 +0000 (09:11 -0400)]
ceph-volume: is_mounted should use a bytes->string util to compare strings
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
4bb4c432dc17139b4f891d0fda229007b4913c0d )
Alfredo Deza [Thu, 10 Aug 2017 13:11:27 +0000 (09:11 -0400)]
ceph-volume: create a utf-8 string decoder for py3 compat
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
b50f1fb5527447c323fa51fb5c0ba260655dfb0d )
Alfredo Deza [Wed, 9 Aug 2017 19:56:53 +0000 (15:56 -0400)]
ceph-volume: tests add tests for the is_mounted utility
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
dd4db2f5671c3ce0aded73013b64a0cc559e5502 )
Alfredo Deza [Wed, 9 Aug 2017 19:24:15 +0000 (15:24 -0400)]
ceph-volume: lvm activate should check if the device is mounted to prevent errors from mount
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
c61aea41f1d07b824e169bf12328b7eb0055e23f )
Alfredo Deza [Wed, 9 Aug 2017 19:10:18 +0000 (15:10 -0400)]
ceph-volume util add a helper to check if a device is mounted
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
d77d86aae11fba01834bb8d60633f3f49126c783 )
Alfredo Deza [Wed, 9 Aug 2017 12:20:33 +0000 (08:20 -0400)]
ceph-volume: lvm activate should not ignore exit status codes
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
c866123017a1defac249bebe76cc7bbaddf3cf67 )
Andrew Schoen [Tue, 8 Aug 2017 17:43:53 +0000 (12:43 -0500)]
ceph-volume: remove unused config from vagrant_variables.yml files
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
855ce630695ed9ca53c314b7e261ec3cc499787d )
Andrew Schoen [Tue, 8 Aug 2017 17:24:41 +0000 (12:24 -0500)]
ceph-volume: adds CEPH_VOLUME_DEBUG=1 to functional tests
This will show us tracebacks if ceph-volume fails
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
5a90f4c577bf371a36bf602dc8ea01663aaffe00 )
Andrew Schoen [Tue, 8 Aug 2017 17:05:40 +0000 (12:05 -0500)]
ceph-volume: add placeholders for prepare_activate testing in tox.ini
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
852a94734f69bfb5544e2d3af34b9e71057df851 )
Andrew Schoen [Tue, 8 Aug 2017 16:44:22 +0000 (11:44 -0500)]
ceph-volume: adds the xenial distro factor
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
1b4275417d457e56eb0c3ac07597447cad7737ae )
Andrew Schoen [Tue, 8 Aug 2017 16:41:15 +0000 (11:41 -0500)]
ceph-volume: create a centos7 factor for functional testing
We want to run these tests on multiple distros so this change sets the
foundation for that.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
c8e3be6faed1c062826117bc1355b7b752c01bd4 )
Alfredo Deza [Tue, 8 Aug 2017 15:07:02 +0000 (11:07 -0400)]
ceph-volume: vagrantfile runs storagectl once
It assumes that if there is a disk left it has already run. This avoids
issues when reloading/restarting machines with vagrant.
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
476d1f50b82f81addd0de218c57b773f81883b0f )
Andrew Schoen [Tue, 8 Aug 2017 15:16:12 +0000 (10:16 -0500)]
ceph-volume: setup nodes for testinfra testing
This uses the playbook that exists in ceph-ansible to prepare the nodes for
testing by installing net-tools.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
97b216fdd1657b34db93b264ce814a8f72434d7b )
Alfredo Deza [Mon, 7 Aug 2017 20:34:22 +0000 (16:34 -0400)]
ceph-volume: tox define vagrant cwd
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
89ccbd8ab4f10832c6bb7e3660e00cce62af4a6b )
Andrew Schoen [Fri, 4 Aug 2017 15:40:39 +0000 (10:40 -0500)]
ceph-volume: adds a functional testing scenario for lvm create
This setups up the basic test harness and adds a test for the create
subcommand. The test uses ceph-ansible to deploy a cluster using
``ceph-volume lvm create``, tests the cluster state using the
ceph-ansible test suite, reboots the nodes and then tests again.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
750d9f4125783b29a7af44acb2c3caa43bee707a )
Sage Weil [Thu, 10 Aug 2017 13:50:54 +0000 (09:50 -0400)]
Revert "qa/suites/upgrade/jewel-x/parallel: thrash layout"
This reverts commit
cabd44af3503c368160fef7e56b637dfbf0e9921 . This test
combinatoin is not yet stable.
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Tue, 8 Aug 2017 16:08:31 +0000 (12:08 -0400)]
qa/tasks/ceph.py: tolerate flush pg stats exception
If the OSD doesn't see IO, it won't flush more pg/osd stats when the
luminous flag is not yet set (legacy pgmonitor mode).
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
9da7e63c641d343853ed83bb22490cb6af1f3d6d )
Sage Weil [Wed, 9 Aug 2017 20:40:43 +0000 (16:40 -0400)]
qa/suites/upgrade/jewel-x/parallel: thrash layout
We can't kill and restart osds because that will interfere with
the upgrade process. We can, however, thrash the layout by
tweaking osd weights and so on. This will exercise osd recovery
paths during the upgrade that aren't normally exercised (outside
of stress-split..which doesn't upgrade individual osds while they
are non-clean).
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
435777dbffc77c93d06476caf83be141359a5778 )
Sage Weil [Wed, 9 Aug 2017 16:50:57 +0000 (12:50 -0400)]
osd/PG: force rebuild of missing set on jewel upgrade
Previously we were detecting the need to rebuild missing based on
whether the "divergent_priors" omap key was present. Unfortunately,
jewel does not always set this, so it is not a reliable indicator.
(It only gets set if you actually have a divergent prior at some
point in the PG's life time on that OSD.)
Fix by using the info_struct_v on the PG to detect whether we need
to do the conversion. We didn't bump the value when we adding
the missing persistence, but the fastinfo was also added during
the same period between jewel and kraken, so it will work just as
well.
Fixes: http://tracker.ceph.com/issues/20958
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
dd1a25218c5a1def02146edb2dce0d97a71e4436 )
Gregory Farnum [Wed, 9 Aug 2017 18:29:00 +0000 (11:29 -0700)]
Merge pull request #16922 from ivancich/luminous-16755
Merge pull request #16755 from ivancich/wip-pull-new-dmclock
Sage Weil [Tue, 8 Aug 2017 19:56:18 +0000 (15:56 -0400)]
mon/OSDMonitor: implement 'osd crush ls <node>'
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
aeed87945b7e3af3c76b8e39739725ce6b09ad56 )
Sage Weil [Wed, 9 Aug 2017 13:18:54 +0000 (09:18 -0400)]
qa/suites/upgarde/jewel-x/point-to-point-x: disable app warnings
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
bbd5fe354c507bce7aad0c7c37036a47dbf624a3 )
Jason Dillaman [Wed, 9 Aug 2017 16:27:13 +0000 (12:27 -0400)]
Merge pull request #16948 from dillaman/wip-32bit-compat-fixes-luminous
luminous: rbd-mirror: align use of uint64_t in service_daemon::AttributeType
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
James Page [Wed, 9 Aug 2017 09:04:37 +0000 (10:04 +0100)]
Align use of uint64_t in service_daemon::AttributeType
size_t on a 32-bit architecture is a 32 bit unsigned int which
created ambiguity when casting to bool, uint64_t or std::string
(which are boost::variants for service_daemon::AttributeType).
Align to use of uint64_t to resolve compilation failures in
all 32-bit architectures.
Signed-off-by: James Page <james.page@ubuntu.com>
(cherry picked from commit
87fe8e81bc8c9b55c6bef4144714a33e042dc2f7 )
Jason Dillaman [Wed, 9 Aug 2017 14:32:51 +0000 (10:32 -0400)]
Merge pull request #16946 from trociny/wip-20954-luminous
luminous: qa/workunits/rbd: use command line option to specify watcher asok
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Mykola Golub [Tue, 8 Aug 2017 18:50:47 +0000 (20:50 +0200)]
qa/workunits/rbd: use command line option to specify watcher asok
The previous method to get the watcher admin socket was fragile
and had started to fail after the recent changes to vstart ceph.conf.
Fixes: http://tracker.ceph.com/issues/20954
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit
6a575136a76b9e291c0948e1179f33a2f73853fb )
Abhishek L [Wed, 9 Aug 2017 14:03:57 +0000 (16:03 +0200)]
Merge pull request #16943 from theanalyst/wip-luminous-16889
luminous: rgw: Use namespace for lc_pool and roles_pool
Reviewed-By: Orit Wasserman <owasserm@redhat.com>
Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>
Orit Wasserman [Tue, 8 Aug 2017 08:24:06 +0000 (11:24 +0300)]
rgw: use namespace for roles pool
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit
4c378ffbbd984d4c7985415ea067661b7b3a2e98 )
Orit Wasserman [Tue, 8 Aug 2017 08:22:42 +0000 (11:22 +0300)]
rgw: initialize lc pool as namespace
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Fixes: http://tracker.ceph.com/issues/20177
(cherry picked from commit
39d76cad38272b8f6db79bdb51a054fa41189b41 )
Sage Weil [Tue, 8 Aug 2017 13:23:31 +0000 (09:23 -0400)]
Revert "os/bluestore: allow multiple DeferredBatches in flight at once"
This reverts commit
ca32d575eb2673737198a63643d5d1923151eba3 .
If we have multiple batches in flight then we have to worry about writes
to the same blocks reordering.
Also,
3c6a6c46d5808d6c42ed4dcfb441bad64366686b is sufficient to avoid the
stall/deadlock in http://tracker.ceph.com/issues/20295.
# Conflicts:
# src/os/bluestore/BlueStore.cc
Fixes: http://tracker.ceph.com/issues/20925
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
917858516a904c82fabe1bd65d2fa88436319713 )
Sage Weil [Tue, 8 Aug 2017 14:28:10 +0000 (10:28 -0400)]
mon: add mon_health_preluminous_compat_warning
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
7c22b46e907ad239fb9efaee96f3ef424beb220a )
John Spray [Tue, 8 Aug 2017 18:53:11 +0000 (19:53 +0100)]
osd: downgrade (ok) PG scrub messages to debug
Otherwise someone watching the log at INFO level gets
pelted with potentially millions of log messages
while the system is scrubbing.
Fixes: http://tracker.ceph.com/issues/20947
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit
0443fdb0221bbcd33c8d1278fdd3959e710fcacb )
John Spray [Tue, 8 Aug 2017 18:36:03 +0000 (19:36 +0100)]
mon: downgrade "scrub ok" message to debug.
This hides lines like:
[INF] scrub ok on 0,1,2: ScrubResult(keys {pgmap_pg=13} crc {pgmap_pg=
2458062599 })
from the normal cluster log views.
Fixes: http://tracker.ceph.com/issues/20947
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit
f394ca0bad248e05665a912558382ca2ea560a91 )
Jason Dillaman [Wed, 9 Aug 2017 01:19:25 +0000 (21:19 -0400)]
Merge pull request #16921 from dillaman/wip-rbd-ls-luminous
luminous: rbd: parallelize rbd ls -l
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Gregory Farnum [Tue, 8 Aug 2017 21:27:28 +0000 (14:27 -0700)]
Merge pull request #16755 from ivancich/wip-pull-new-dmclock
osd: bring in dmclock library changes
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit
25f1edefbf21f17f5501d9894f0c4979c04b3f08 )
Piotr Dałek [Wed, 7 Jun 2017 14:01:37 +0000 (16:01 +0200)]
rbd: parallelize rbd ls -l
When a cluster contains a large number of images, "rbd ls -l" takes a
long time to finish. In my particular case, it took about 58s to
process 3000 images.
"rbd ls -l" opens each of image and that takes majority of time, so
improve this by using aio_open() and aio_close() to do it
asynchronously. This reduced total processing time down to around 15
seconds when using default 10 concurrently opened images.
Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
(cherry picked from commit
8f76fc861b0a628fa2269b04f77b7e31d4a7a006 )
Abhishek L [Tue, 8 Aug 2017 18:53:40 +0000 (20:53 +0200)]
Merge pull request #16914 from theanalyst/wip-16734
luminous: rgw_lc: support for AWSv4 authentication
Reviewed-By: Daniel Gryniewicz <dang@redhat.com>
Reviewed-By: Radoslaw Zarzynski <rzarzynski@redhat.com>
Reviewed-By: Matt Benjamin <mbenjami@redhat.com>
Abhishek Lekshmanan [Tue, 1 Aug 2017 15:42:31 +0000 (17:42 +0200)]
rgw_lc: support for AWSv4 authentication
adding support for AWSv4 authentication for Put Object LC, also adding
types to all of the LC ops in the process
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit
cc51f32c22f294f59b369350e54b86892015cbab )
Jason Dillaman [Tue, 8 Aug 2017 17:52:46 +0000 (13:52 -0400)]
Merge pull request #16912 from dillaman/wip-20701-luminous
luminous: doc: update rbd-mirroring documentation
Reviewed-by: Sage Weil <sage@redhat.com>
Jason Dillaman [Tue, 8 Aug 2017 16:46:56 +0000 (12:46 -0400)]
doc/release-notes: indicate that rbd-mirror should use unique IDs
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
3ecc35350afa8777139a76c9e63b121775c6096f )
Jason Dillaman [Tue, 8 Aug 2017 16:43:32 +0000 (12:43 -0400)]
doc: updated rbd-mirror daemon instructions
Fixes: http://tracker.ceph.com/issues/20701
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
b7ba3f68c3baf5ad8b0b307afcf2bfbfa18d597c )
Jason Dillaman [Tue, 8 Aug 2017 15:53:42 +0000 (11:53 -0400)]
doc: re-ordered rbd table of contents
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
34ff1ddca1d228bb785ec04f3aef6ccfdccdc5de )
Jason Dillaman [Tue, 8 Aug 2017 16:57:24 +0000 (12:57 -0400)]
Merge pull request #16875 from dillaman/wip-lirbd-group-luminous
luminous: librbd: remove consistency group rbd cli and API support
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Patrick Donnelly [Tue, 8 Aug 2017 16:40:27 +0000 (09:40 -0700)]
Merge PR #16378 into HEAD
* refs/remotes/upstream/pull/16378/head:
doc: remove accidental additions to release notes
qa/cephfs: Fix race in test_volume_client
qa/cephfs: Test filtered df
PendingReleaseNotes: add note about df filtering
client: Support new, filtered MStatfs
objecter: Support new, filtered MStatfs
mon/PGMap stats: Support new, filtered MStatfs
messages: Add optional data pool to MStatfs
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Patrick Donnelly [Tue, 8 Aug 2017 16:28:57 +0000 (09:28 -0700)]
doc: remove accidental additions to release notes
Presumably this was caused by a bad rebase.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Jason Dillaman [Tue, 8 Aug 2017 15:23:49 +0000 (11:23 -0400)]
Merge pull request #16903 from dillaman/wip-16877-luminous
luminous: test/librbd: fix race condition with OSD map refresh
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Jason Dillaman [Mon, 7 Aug 2017 18:29:07 +0000 (14:29 -0400)]
test/librbd: fix race condition with OSD map refresh
Fixes: http://tracker.ceph.com/issues/20918
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
5c29664434c6e9e2f72aa4b02b369613250694e8 )
Jason Dillaman [Tue, 8 Aug 2017 13:58:28 +0000 (09:58 -0400)]
Merge pull request #16899 from dillaman/wip-20941-luminous
luminous: librbd: default localize parent reads to false
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Jason Dillaman [Tue, 8 Aug 2017 13:28:32 +0000 (09:28 -0400)]
Merge pull request #16895 from dillaman/wip-15339-luminous
luminous: rbd-ggate: tool to map images on FreeBSD via GEOM Gate
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Jason Dillaman [Mon, 7 Aug 2017 21:44:30 +0000 (17:44 -0400)]
librbd: default localize parent reads to false
Fixes: http://tracker.ceph.com/issues/20941
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
cfc3d4603668e23232c2f0b9af6fd838040f47ec )
Mykola Golub [Sun, 6 Aug 2017 14:27:22 +0000 (16:27 +0200)]
test: add wrapper to run rbd-ggate test on FreeBSD
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
Mykola Golub [Sun, 14 May 2017 09:00:24 +0000 (09:00 +0000)]
rbd-ggate: tool to map images on FreeBSD via GEOM Gate
rbd-ggate spawns a process responsible for the creation of ggate
device and forwarding I/O requests between the GEOM Gate kernel
subsystem and RADOS.
On FreeBSD it provides functionality similar to rbd-nbd on Linux.
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
Adam C. Emerson [Mon, 7 Aug 2017 21:46:38 +0000 (17:46 -0400)]
rgw: Fix the last policy use-after-free
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit
5353d952683a5a13a681c594e119b570bfdc3c39 )
Adam C. Emerson [Mon, 7 Aug 2017 21:27:53 +0000 (17:27 -0400)]
rgw: Fix another use after free
This one was caused by iterator invalidation in set operations. In
this case just replace the set entirely with a bitfield.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit
97d026dde679cabf1aaf026be3f08bfef63c140f )
Adam C. Emerson [Mon, 24 Jul 2017 20:10:11 +0000 (16:10 -0400)]
rgw: Fix use after free in IAM policy parser
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit
8377ba6525de5ebfe33a7dda14f17d96e8ac4ef4 )
Sage Weil [Mon, 7 Aug 2017 20:02:33 +0000 (16:02 -0400)]
qa/suites/upgrade/kraken-x/stress-split*: whitelist
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
bf29142b0828d953da09aab83a7fe44a5ce4fe78 )
Sage Weil [Mon, 7 Aug 2017 19:57:55 +0000 (15:57 -0400)]
qa/suites/upgrade/kraken-x/parallel: whitelist
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
2234a0ed11ba1b3688e2ae506a1128840507883d )
Sage Weil [Mon, 7 Aug 2017 13:49:55 +0000 (09:49 -0400)]
qa/suites/upgrade/jewel-x/parallel: fix POOL_APP_NOT_ENABLED disable
This code runs on the mgr.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
3e7d157871880fc5c7acc436dcb4eafa078df128 )
Sage Weil [Sat, 5 Aug 2017 19:33:37 +0000 (15:33 -0400)]
mon/MonCommands: mark 'pg force_create_pg' deprecated
It's deprecated.
Also, this avoids a dup when we have an upgrading mon cluster
and it's also in PGMonitorCommands.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
7c37c86bb20ebc16c1d34a3acb7ad3f183d6e0e0 )
Sage Weil [Sat, 5 Aug 2017 19:30:15 +0000 (15:30 -0400)]
mon: include PGMonitor commands with mixed-version mons
While we have a mixed version cluster, we have to advertise our
PGMonitor commands to our peons or else commands like 'pg dump'
won't work.
Once the mon feature flag is set, we can drop that because each
mon will include the mgr commands (either those stored in paxos
or the statically compiled ones until that point).
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
2d9045a9a57a3b08d525f4f1ecb3b255add26b1f )
Sage Weil [Sat, 5 Aug 2017 19:08:26 +0000 (15:08 -0400)]
mon: use vector<MonCommand> throughput for commands
The old code was pretty messy. This is standardizes on std::vector
throughout. We also drop the win_election command args because
when we win an election we always set the leader commands to our
commands, and we can do that inside win_command() without passing
them in from here.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
349b3c9bef12728c336cf584ea5c70b8184b66e9 )
Jason Dillaman [Mon, 7 Aug 2017 19:57:11 +0000 (15:57 -0400)]
Merge pull request #16878 from dillaman/wip-rbd-mirror-image-ids-luminous
luminous: rbd-mirror: simplify notifications for image assignment
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Jason Dillaman [Mon, 31 Jul 2017 12:37:38 +0000 (08:37 -0400)]
rbd-mirror: clean up logging during test shut down
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 27 Jul 2017 19:35:15 +0000 (15:35 -0400)]
rbd-mirror: avoid silently failing to register image replayer admin sockets
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Fri, 21 Jul 2017 15:18:46 +0000 (11:18 -0400)]
rbd-mirror: restore deletion propagation and image replayer cleanup
The previous intermediate commits removed handling for deletion
propagation and image replayer cleanup since this logic has been
moved from instance to image replayer. Note that eventually the
policy's release notification will be responsible for the cleanup
of image replayers.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Fri, 21 Jul 2017 19:40:03 +0000 (15:40 -0400)]
rbd-mirror: ensure threads are mockable for testing
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Fri, 21 Jul 2017 20:12:13 +0000 (16:12 -0400)]
rbd-mirror: moved wait for deletion logic within image replayer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 20 Jul 2017 20:30:42 +0000 (16:30 -0400)]
rbd-mirror: simplified acquire/release notification messages
The other instances should already know about the possible peers, so
we can co-opt these messages for use by the policy mapper.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 20 Jul 2017 20:39:51 +0000 (16:39 -0400)]
rbd-mirror: co-opt image acquire/release notifications
This notifications will now be used by the policy when a (re)map is
required.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 20 Jul 2017 20:10:39 +0000 (16:10 -0400)]
rbd-mirror: new 'peer image removed' notification to instance watchers
Note: this commit intentionally breaks backwards compatibility with
Luminous RC instance watcher notifications.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Wed, 19 Jul 2017 20:18:23 +0000 (16:18 -0400)]
rbd-mirror: simplify resync handling within image replayer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>