]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Guillaume Abrioux [Fri, 20 Sep 2019 12:57:58 +0000 (14:57 +0200)]
ceph-volume: do not fail when trying to remove crypt mapper
In a containerized context, at some point, need to run `simple scan` on a device
from a separate container (not the existing and running corresponding container
to that device), but this can't work because when it tries to remove the
mapper which is still in use by the corresponding running osd container,
it fails.
This can be a bit more permissive and simply throw a warning.
Closes: https://tracker.ceph.com/issues/41392
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit
80d3e4ca054b4d1334c2fb99fede46ab54ca4305 )
Alfredo Deza [Thu, 12 Sep 2019 21:52:54 +0000 (17:52 -0400)]
Merge pull request #29038 from dcasier/wip-40100-luminous
luminous: ceph-volume: lvm.activate: Return an error if WAL/DB devices absent
Reviewed-by: Alfredo Deza <adeza@redhat.com>
David Casier [Wed, 3 Jul 2019 21:24:34 +0000 (23:24 +0200)]
ceph-volume: lvm.activate: Return an error if WAL/DB devices absent
Fixes: https://tracker.ceph.com/issues/40100
Signed-off-by: David Casier <david.casier@aevoo.fr>
(cherry picked from commit
de439a0a236e4752e64a0cd63b7ce1ad3278eff3 )
Jan Fajerski [Thu, 12 Sep 2019 13:00:16 +0000 (15:00 +0200)]
Merge pull request #29771 from alfredodeza/luminous-bz-
1738379
luminous ceph-volume: use the OSD identifier when reporting success
Alfredo Deza [Thu, 12 Sep 2019 10:55:35 +0000 (06:55 -0400)]
Merge pull request #30296 from alfredodeza/luminous-rm41378-2
luminous ceph-volume: pre-install python-apt and its variants before test runs
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Jan Fajerski [Thu, 12 Sep 2019 10:05:25 +0000 (12:05 +0200)]
Merge pull request #30305 from alfredodeza/luminous-rm40664
luminous ceph-volume zap always skips block.db, leaves them around
Alfredo Deza [Wed, 11 Sep 2019 19:30:36 +0000 (15:30 -0400)]
Merge pull request #30299 from alfredodeza/luminous-rm41660
luminous ceph-volume: fix stderr failure to decode/encode when redirected
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Alfredo Deza [Mon, 9 Sep 2019 22:55:13 +0000 (18:55 -0400)]
ceph-volume tests create a test file for checking unicode output
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
79196354aca3b3d30c7de1e2ac8b5a3c06ee8330 )
Alfredo Deza [Mon, 9 Sep 2019 22:54:51 +0000 (18:54 -0400)]
ceph-volume tests create a shell test for functional unicode
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
595e492783283359b613d199a75f3e7495bfb9ed )
Alfredo Deza [Mon, 9 Sep 2019 19:20:13 +0000 (15:20 -0400)]
ceph-volume tests verify new logging fallback and encodings in terminal
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
ee18ebc078b2d380b88f3726737184f34345e2f3 )
Alfredo Deza [Mon, 9 Sep 2019 17:45:08 +0000 (13:45 -0400)]
ceph-volume terminal remove unicode stream handler
This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
fe66e01f6a4b19bb15695537713a629a951b28ce )
Alfredo Deza [Mon, 9 Sep 2019 17:44:36 +0000 (13:44 -0400)]
ceph-volume: instantiate the new terminal logger in main()
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
bfb8422e262dcce49c50610a3977bfe1edc2a435 )
Alfredo Deza [Mon, 9 Sep 2019 17:44:17 +0000 (13:44 -0400)]
ceph-volume create a logger for the terminal
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
95b16b516c0736839fe50f12ecbeb7cd62c67335 )
Alfredo Deza [Mon, 9 Sep 2019 17:40:06 +0000 (13:40 -0400)]
ceph-volume create a new tox.ini for shell-based tests
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
4612ab3ae62a3e8e1cc9e51489feab0c938296d3 )
Alfredo Deza [Wed, 11 Sep 2019 13:31:13 +0000 (09:31 -0400)]
Merge pull request #30302 from alfredodeza/luminous-ceph-volume-zap-fix
luminous ceph-volume lvm.zap fix cleanup for db partitions
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Dominik Csapak [Tue, 28 May 2019 14:29:21 +0000 (16:29 +0200)]
ceph-volume lvm.zap fix cleanup for db partitions
this uses the correct type 'db' for db type partitions, else
a block.db parition does not get cleaned up by ceph-volume zap
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
(cherry picked from commit
140af8a7eee8d230ac9d94ba100320a20c137a12 )
Alfredo Deza [Wed, 11 Sep 2019 12:54:34 +0000 (08:54 -0400)]
Merge pull request #30331 from jan--f/wip-
9e2175c -luminous
luminous: ceph-volume: set a lvm_size property on the fakedevice fixture
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Andrew Schoen [Thu, 6 Dec 2018 18:37:56 +0000 (12:37 -0600)]
ceph-volume: set a lvm_size property on the fakedevice fixture
This is so tests can continue to set sys_api['size'] and the code
can retrieve that as Device.lvm_size
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
9e2175c4001434f7b0ed8ce6f723e063cf001440 )
Alfredo Deza [Thu, 11 Jul 2019 15:56:07 +0000 (11:56 -0400)]
ceph-volume tests: verify that wipefs tries several times
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
845e62cb4ba2636325304db2c2d65a185ace25cf )
Alfredo Deza [Thu, 11 Jul 2019 13:34:10 +0000 (09:34 -0400)]
ceph-volume lvm.zap: retry wipefs several times to prevent race condition failures
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
b7b8b106fa485d3b8345b583fbc7f74e30cbc03e )
Alfredo Deza [Tue, 3 Sep 2019 14:00:14 +0000 (10:00 -0400)]
ceph-volume tests pre-instrall python-apt to prevent auto-install failing later
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
40cac7b9510f7904c2f9dfb1294d809e04bdcf7e )
Nathan Cutler [Mon, 9 Sep 2019 22:29:13 +0000 (00:29 +0200)]
Merge pull request #30129 from smithfarm/wip-41644-luminous
luminous: install-deps.sh: Remove CR repo
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Neha Ojha [Mon, 9 Sep 2019 19:12:52 +0000 (12:12 -0700)]
Merge pull request #29910 from vumrao/wip-vumrao-bluefs-log-level
luminous: os/bluestore: apply shared_alloc_size to shared device with log level change
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Neha Ojha [Mon, 9 Sep 2019 17:36:12 +0000 (10:36 -0700)]
Merge pull request #29190 from badone/wip-luminous-lazy-omap-stats-backport-tracker-38551
luminous: core: lazy omap stat collection
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Neha Ojha [Sat, 7 Sep 2019 19:17:30 +0000 (12:17 -0700)]
Merge pull request #29175 from neha-ojha/wip-40583-luminous
luminous: common/options.cc: Lower the default value of osd_deep_scrub_large_omap_object_key_threshold
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Neha Ojha [Fri, 6 Sep 2019 17:10:42 +0000 (10:10 -0700)]
Merge pull request #30149 from neha-ojha/wip-40769-luminous
luminous: common/options: Set concurrent bluestore rocksdb compactions to 2
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Mark Nelson [Fri, 12 Jul 2019 16:45:17 +0000 (12:45 -0400)]
common/options: Set concurrent bluestore rocksdb compactions to 2
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit
26dc5189e73a1f5a22f89ee67eef5430ef660b5d )
Brad Hubbard [Thu, 22 Nov 2018 00:07:22 +0000 (10:07 +1000)]
install-deps.sh: Remove CR repo
Remove the continuous release repos for CentOS and Virtuozzo 7 as they
should no longer be needed since http://tracker.ceph.com/issues/13997 is
no longer relevant and the newer versions of selinux packages pulled in
by the build system are causing problems for systems without CR repos
enabled.
Fixes: https://tracker.ceph.com/issues/37335
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
f39d9b7d1f4f2b63233802211a9ed0e641446ca1 )
Conflicts:
install-deps.sh
Alfredo Deza [Tue, 3 Sep 2019 17:51:42 +0000 (13:51 -0400)]
Merge pull request #30094 from jan--f/wip-41613-luminous
luminous: ceph-volume: lvm list is O(n^2)
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Rishabh Dave [Mon, 19 Aug 2019 13:33:21 +0000 (19:03 +0530)]
ceph-volume: test number of times LVs list was created
Add a test that cofirm that the LVs list was created only once when
calling devices.lvm.listing.list()
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit
20d6264538dab444c0ef13f40c6c21ec3d2057f8 )
Rishabh Dave [Wed, 14 Aug 2019 15:57:38 +0000 (21:27 +0530)]
ceph-volume: reuse list of LVs
Avoid creating of list of LVs in different methods of the same module
and resue them.
Fixes: https://tracker.ceph.com/issues/37490
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit
ab3a58e78124a984d162b2f63992d01c5e59e2a7 )
Alfredo Deza [Tue, 20 Aug 2019 12:04:56 +0000 (08:04 -0400)]
ceph-volume devices.lvm zap use the identifier to report success
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
d0d5750c1ba508152c1d5111f2003a039a27145f )
Alfredo Deza [Tue, 20 Aug 2019 12:04:26 +0000 (08:04 -0400)]
ceph-volume tests check success message when zapping
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
88c7429a60fdab6222d14132bea93b9bf773c720 )
Alfredo Deza [Tue, 27 Aug 2019 11:49:26 +0000 (07:49 -0400)]
Merge pull request #29901 from alfredodeza/luminous-rm41378
luminous ceph-volume tests set the noninteractive flag for Debian
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Vikhyat Umrao [Thu, 22 Aug 2019 16:45:40 +0000 (09:45 -0700)]
os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1
Fixes: https://tracker.ceph.com/issues/41399
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
(cherry picked from commit
96d48af3252f0ec2c1864383ad5eee993ecfeb4f )
Alfredo Deza [Mon, 26 Aug 2019 15:53:49 +0000 (11:53 -0400)]
Merge pull request #29791 from jan--f/wip-41373-luminous
luminous: batch functional idempotency test fails since message is now on stderr
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Alfredo Deza [Wed, 21 Aug 2019 18:15:32 +0000 (14:15 -0400)]
ceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
89231c9a60a46ba0691f787efbfe5ab35f05c679 )
Josh Durgin [Sat, 24 Aug 2019 02:52:52 +0000 (22:52 -0400)]
os/bluestore: record gifts < expected size in superblock
Returning 0 from _balance_bluefs_freespace() skips recording
allocations in the superblock, so we fail the consistency check on
startup. The elseif branch handles this case already, so just remove
it from the first branch. This is luminous/mimic specific, since
bluefs extents are not recorded in the superblock in later releases.
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Gregory Farnum [Wed, 21 Aug 2019 21:08:13 +0000 (14:08 -0700)]
Merge pull request #27551 from smithfarm/wip-osd-markdown-luminous
luminous: qa/standalone/osd/osd-markdown: fix dup command disabling
Gregory Farnum [Wed, 21 Aug 2019 21:00:12 +0000 (14:00 -0700)]
Merge pull request #27471 from dzafman/wip-38567
luminous: doc: osd_recovery_priority is not documented (but osd_recovery_op_priority is)
Jan Fajerski [Thu, 15 Aug 2019 11:28:42 +0000 (13:28 +0200)]
ceph-volume: fix batch functional tests, idempotent test must check stderr
Fixes: https://tracker.ceph.com/issues/41295
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
88807110f3014e7a2a39c37ae5c64bb3ab107d80 )
Conflicts:
src/ceph-volume/ceph_volume/tests/functional/batch/playbooks/test_explicit.yml
removed
Sage Weil [Mon, 19 Aug 2019 15:09:36 +0000 (10:09 -0500)]
os/bluestore: try to gift large extents, then fall back to small extents
First try to gift using the larger alloc_size (normally bluefs_alloc_size,
but here max(bluefs_alloc_size,bluefs_shared_alloc_size) just in case the
settings are weird. If that fails, then try the shared_alloc_size. If
that fails, fail and complain as before, with an more accurate error
message.
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Mon, 19 Aug 2019 13:47:40 +0000 (08:47 -0500)]
os/bluestore: gift freespace to bluefs based on shared_alloc_size
This does not appear in master because this code was removed with the
allocator fallback behavior, which was not backported.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
98c04c48c318ea15fdcbb331f879d9a962243559 )
Sage Weil [Mon, 19 Aug 2019 13:46:09 +0000 (08:46 -0500)]
os/bluestore/BlueFS: apply shared_alloc_size to shared device
Keep an alloc_size vector so that we have this value handy at all times.
Allow bluestore to fetch this value directly instead of looking at the
bluefs_* config options since this encapsulates things a bit better, and
also isn't vulnerable to the config setting changing at runtime.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
e8b5a458c306dfe166b09239fc9f57c04fa47ead )
# Conflicts:
# src/os/bluestore/BlueFS.cc
- missing assert(got != 0) in luminous
- ROUND_UP_TO vs round_up_to
- missing fallback allocations
- no BDEV_NEW*
# src/os/bluestore/BlueFS.h
- adjacent declarations
# src/os/bluestore/BlueStore.cc
- P2ROUNDUP vs p2roundup
Sage Weil [Wed, 7 Aug 2019 15:44:38 +0000 (10:44 -0500)]
os/bluestore/BlueFS: add bluefs_shared_alloc_size
Add a separate config option that controls the alloc_size for the shared
device (BDEV_SLOW).
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
c69c9539f628c61a1cf4cae6aadd39cb48a8b022 )
- no Option::TYPE_SIZE
Sage Weil [Fri, 16 Aug 2019 21:44:57 +0000 (16:44 -0500)]
os/bluestore: cleanup around allocator calls
Both stupid and bitmap allocator returs -ENOSPC if they're
unable to allocate any space. Existing callers aren't always
respect this - hence doing some cleanup.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
a1246da5705e2a699656c2561cebc5ed29628c90 )
- allocator fallback not there
- allocator fallback not there
Alfredo Deza [Tue, 13 Aug 2019 12:05:02 +0000 (08:05 -0400)]
Merge pull request #29603 from jan--f/wip-41202-luminous
luminous: ceph-volume prints log messages to stdout
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Jan Fajerski [Thu, 8 Aug 2019 08:42:37 +0000 (10:42 +0200)]
ceph-volume: never log to stdout, use stderr instead
We should never print log messages to stdout, as this should be reserved
for output of ceph-volume.
Fixes: https://tracker.ceph.com/issues/41158
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
b8d6dcbe9f803c96c0af68da54f1262e9b6a9e77 )
Kefu Chai [Sun, 24 Mar 2019 09:11:31 +0000 (17:11 +0800)]
ceph-volume: terminal: encode unicode when writing to stdout
python determins the encoding of stdout and stderr based on the LC_CTYPE
and PYTHONIOENCODING env variable, by default, python3's sys.stdout uses
'utf-8' as its encoding, so it will be able to write unicode string even
the stdout is not attached to a tty device. but when it comes to
python2, it will default to ascii if neither of these variabls is set.
so, if we are writing unicode using `_Write` in an environment where
LC_CTYPE and/or PYTHONIOENCODING are using non UTF-8 encoding, it chokes
by raising `UnicodeEncodeError` exception.
in this change, we add a wrapper around `_Write._writer` so it is able
to write unicode string in such a non-unicode-friendly environment.
for more info related the encoding of stdout and stderr, see
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONIOENCODING .
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
77912c0c71874a23785d3bdd109ad61e4f4c0e28 )
Brad Hubbard [Tue, 23 Jul 2019 07:42:54 +0000 (17:42 +1000)]
luminous: tests: Convert to boost::regex and add blocking scrub
Convert lazy_omap_stats_test to boost::regex since std::regex is not
fully implemented in the C++ libs used to compile luminous. Also add the
ability to block until scrub completes as that is not available in
luminous.
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
Xie Xingguo [Mon, 12 Aug 2019 00:32:46 +0000 (08:32 +0800)]
Merge pull request #27637 from xiexingguo/wip-luminous-pr-21646
luminous: qa/suites/rados/verify/tasks/rados_api_tests: whitelist OBJECT_MISPLACED
Reviewed-by: Sage Weil <sage@redhat.com>
Neha Ojha [Thu, 8 Aug 2019 17:55:37 +0000 (10:55 -0700)]
Merge pull request #29539 from neha-ojha/wip-40704-luminous
luminous: bluestore: Inspect allocations
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Neha Ojha [Wed, 7 Aug 2019 20:33:56 +0000 (16:33 -0400)]
os/bluestore/BlueStore.cc: _open_fm() expects a bool
bug fix for
974eae57d55a0ea847b25b486b58540287292254
Signed-off-by: Neha Ojha <nojha@redhat.com>
Adam Kupczyk [Wed, 7 Aug 2019 09:52:29 +0000 (11:52 +0200)]
doc/rados/operations/health-checks: document BlueStore fragmentation and BlueFS space available features
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
713f9b4d093759d3541a62c8aa43ce0a655b2d64 )
Conflicts:
doc/rados/operations/health-checks.rst - trivial resolution
Adam Kupczyk [Tue, 6 Aug 2019 10:01:52 +0000 (12:01 +0200)]
test/objectstore: Allocator_test. Add test for dumping free regions and fragmentation_score.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
355dc85372a15713ab1cdd78f44ea02db8e08a62 )
Adam Kupczyk [Mon, 5 Aug 2019 13:15:07 +0000 (15:15 +0200)]
BlueStore/allocator: Add command to inspect how much BlueStore's block can go to BlueFS.
Moved reduced BlueFSDeviceExpander interface.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
16a9dac1678613b45fab9576b8bc4368a58c7434 )
Conflicts:
src/os/bluestore/BlueFS.cc - trivial resolution
src/os/bluestore/BlueFS.h
src/os/bluestore/BlueStore.cc
src/os/bluestore/BlueStore.h
Adam Kupczyk [Wed, 7 Aug 2019 14:21:29 +0000 (10:21 -0400)]
os/bluestore: Import necessary functions to fix rebase.
This commit is specific to luminous to make the backport of
https://github.com/ceph/ceph/pull/29425 simpler.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adam Kupczyk [Wed, 31 Jul 2019 11:31:04 +0000 (13:31 +0200)]
tools/ceph-bluestore-tool: add commands free-dump and free-score
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
e7f5e53cde3b586701ff8e759a941986c17af1a8 )
Adam Kupczyk [Tue, 30 Jul 2019 11:54:58 +0000 (13:54 +0200)]
common/admin_socket: Add 'execute_command' that allows for self-reflection.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
838320a9b22c1749f609433843542690fccdfaee )
Adam Kupczyk [Tue, 30 Jul 2019 10:26:33 +0000 (12:26 +0200)]
common/admin_socket: Adapted old protocol to use new protocol, simplifies handle path.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
2b5da1f671a76f77107b6eaefc67e0e9c9179ca7 )
Adam Kupczyk [Fri, 2 Aug 2019 11:54:13 +0000 (13:54 +0200)]
BlueStore/allocator: Add ability to dump free allocations via admin socket interface.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
278055d35ae3ca2eb06f59f4712ad9ce5d843d86 )
Adam Kupczyk [Fri, 2 Aug 2019 11:53:37 +0000 (13:53 +0200)]
BlueStore/allocator: Give allocator names, so they can be distinguished.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
0d64893c3ebc8a56ceac61596efc6f75ed06fbaf )
Adam Kupczyk [Thu, 21 Jun 2018 11:58:45 +0000 (13:58 +0200)]
BlueStore/allocator: Improved (but slower) method of calculating fragmentation.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
c1ac6a1207cdb71a471321e42ceca517383b36c8 )
Jan Fajerski [Wed, 7 Aug 2019 06:10:20 +0000 (08:10 +0200)]
Merge pull request #29508 from jan--f/wip-41139-luminous
luminous: ceph-volume prints errors to stdout with --format json
Brad Hubbard [Tue, 9 Jul 2019 05:03:24 +0000 (15:03 +1000)]
luminous: osd: Modify lazy omap stats to only show one copy
Only show stats from the primary to get more in line with other pg stats
representations.
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
490829c8eaf14f41de978ac0f93ecda3df8bcdef )
Brad Hubbard [Mon, 4 Mar 2019 08:12:27 +0000 (18:12 +1000)]
luminous: tests: Add test for lazy omap stat collection
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
88e9ca58a06ba572ff4c6f6125a8ba7f8e0a7795 )
Brad Hubbard [Tue, 11 Dec 2018 04:00:35 +0000 (14:00 +1000)]
luminous: osd: Implement lazy omap usage statistics
Opportunistic gathering of omap statistics during deep scrub.
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
9e21ef0e40ff89b10dfdf857c703336496b3171f )
Conflicts:
doc/dev/placement-group.rst: Trivial
src/osd/PGBackend.cc: Small difference in ScrubMap::object init
src/osd/osd_types.cc: Differences in structure versions and
members
src/osd/osd_types.h: Differences in structure versions and
members
Jan Fajerski [Mon, 4 Mar 2019 10:28:13 +0000 (11:28 +0100)]
ceph-volume: print most logging messages to stderr
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Fixes: http://tracker.ceph.com/issues/38548
(cherry picked from commit
1f8f1e95dee88b4bab2d6931e720ed83584d2887 )
Conflicts:
src/pybind/mgr/test_orchestrator/module.py
removed
Jan Fajerski [Tue, 6 Aug 2019 11:35:34 +0000 (13:35 +0200)]
Merge pull request #29462 from jan--f/wip-41057-luminous
luminous: ceph-volume does not recognize wal/db partitions created by ceph-disk
Alfredo Deza [Mon, 5 Aug 2019 14:52:33 +0000 (10:52 -0400)]
Merge pull request #29415 from jan--f/wip-41020-luminous
luminous: simple: when 'type' file is not present activate fails
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Jan Fajerski [Tue, 30 Jul 2019 15:32:06 +0000 (17:32 +0200)]
ceph-volume: fall back to PARTTYPE if PARTLABEL is empty
In some cases ceph-disk does not populate PARTLABEL for wal and db
partitions. This commit adds the assumption that the empty string is a
valid label and falls back to identifying those disks by PARTTYPE.
Fixes: https://tracker.ceph.com/issues/40917
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
82d2ae7ffdf62ca93c7848fb62e506db4cda893c )
Jan Fajerski [Tue, 30 Jul 2019 15:29:09 +0000 (17:29 +0200)]
ceph-volume: adjust tests for empty PARTLABEL fields
It seems like ceph-disk sometimes creates OSDs where the wal and db
partitions have no PARTLABEL set. This commit alters tests so that the
empty string is considered a valid label.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
bb29110c3c6e081568231131c70522dee6d12bb5 )
Jan Fajerski [Tue, 30 Jul 2019 11:29:22 +0000 (13:29 +0200)]
ceph-volume: refactor ceph-disk_member unittests
This adds a parameterized fixture as input to the tests. Should make
writing new tests easier since only fixture parameters need to be added.
In the future we should be able to reuse the fixtures to build other
fixtures and reuse the parametrization.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
f42287c3416ad09c112e64756f5051c5c13aa344 )
Josh Durgin [Wed, 31 Jul 2019 15:57:47 +0000 (08:57 -0700)]
Merge pull request #28647 from dvanders/zstd
luminous: compressor/zstd: improvements
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Wed, 31 Jul 2019 15:35:09 +0000 (08:35 -0700)]
Merge pull request #26498 from k0ste/luminous_backports
luminous: mgr/balancer: blame if upmap won't actually work
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Yuri Weinstein [Wed, 31 Jul 2019 15:32:03 +0000 (08:32 -0700)]
Merge pull request #28666 from varshar16/wip-fs-authorize-err
luminous: mon/AuthMonitor: don't validate fs caps on authorize
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Yuri Weinstein [Wed, 31 Jul 2019 15:31:08 +0000 (08:31 -0700)]
Merge pull request #28972 from neha-ojha/wip-40720-luminous
[AFTER: #28644] luminous: os/bluestore: default to bitmap allocator for bluestore/bluefs
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Yuri Weinstein [Wed, 31 Jul 2019 15:30:17 +0000 (08:30 -0700)]
Merge pull request #29213 from smithfarm/wip-38962-luminous
luminous: mgr: DaemonServer::handle_conf_change - broken locking
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Alfredo Deza [Mon, 29 Jul 2019 19:50:14 +0000 (15:50 -0400)]
ceph-volume tests: ensure that better heuristics exist for objectstore detection
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
e48ad3ab26fdab72755bc933781db3bfc6001a30 )
Alfredo Deza [Mon, 29 Jul 2019 19:46:48 +0000 (15:46 -0400)]
ceph-volume simple.activate better detect bluestore/filestore when type is not found
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
b6a449e2e3241aaa2c0eb8fa9f74b5d939c97852 )
Alfredo Deza [Fri, 26 Jul 2019 13:35:28 +0000 (09:35 -0400)]
Merge pull request #29351 from jan--f/wip-40978-luminous
luminous: ceph-volume: missing string substitution when reporting mounts
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Shyukri Shyukriev [Fri, 7 Jun 2019 12:09:19 +0000 (12:09 +0000)]
ceph-volume:util: Use proper param substition
Fixes: https://tracker.ceph.com/issues/25030
Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
(cherry picked from commit
654660cd89fe9d93087d390805651c860bf4e862 )
Yuri Weinstein [Thu, 25 Jul 2019 21:50:52 +0000 (14:50 -0700)]
Merge pull request #27056 from ifed01/wip-ifed-fix-storetest-luminous
luminous: test/store_test: fix/workaround for BlobReuseOnOverwriteUT and garbageCollection
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 21:49:55 +0000 (14:49 -0700)]
Merge pull request #27618 from smithfarm/wip-39332-luminous
luminous: build/ops: rpm: Build with lttng on openSUSE
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 21:48:52 +0000 (14:48 -0700)]
Merge pull request #27808 from xiexingguo/wip-39424
luminous: mgr/BaseMgrModule: drop GIL in set_config
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 21:47:11 +0000 (14:47 -0700)]
Merge pull request #28644 from ifed01/wip-ifed-fix-40080-luminous
luminous: os/bluestore: fix duplicate allocations in bmap allocator
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 21:46:28 +0000 (14:46 -0700)]
Merge pull request #28742 from smithfarm/wip-40548-luminous
luminous: common: Keyrings created by ceph auth get are not suitable for ceph auth import
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 21:45:31 +0000 (14:45 -0700)]
Merge pull request #28851 from jdurgin/wip-18096-luminous
luminous: osd: report omap/data/metadata usage
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 21:44:55 +0000 (14:44 -0700)]
Merge pull request #28895 from ifed01/wip-ifed-fix-no-compress-lum
luminous: os/bluestore: load OSD all compression settings unconditionally.
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 21:43:56 +0000 (14:43 -0700)]
Merge pull request #28933 from dzafman/wip-backport-24178
luminous: qa/workunits/rados/test_envlibrados_for_rocksdb: install g++ not g++-4.7
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:43:03 +0000 (16:43 -0700)]
Merge pull request #28594 from xiexingguo/wip-40229
luminous: mon, osd: parallel clean_pg_upmaps
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 22:43:31 +0000 (15:43 -0700)]
Merge pull request #28110 from smithfarm/wip-38880-luminous
luminous: ENOENT in collection_move_rename on EC backfill target
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 22:42:52 +0000 (15:42 -0700)]
Merge pull request #28112 from smithfarm/wip-39373-luminous
luminous: core: ceph tell osd.xx bench help : gives wrong help
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 22:41:26 +0000 (15:41 -0700)]
Merge pull request #28811 from trociny/wip-40463-luminous
luminous: journal: properly advance read offset after skipping invalid range
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 15:20:27 +0000 (08:20 -0700)]
Merge pull request #27905 from smithfarm/wip-38954-luminous
luminous: tests: krbd discard qa fixes
Yuri Weinstein [Wed, 24 Jul 2019 15:19:52 +0000 (08:19 -0700)]
Merge pull request #28149 from trociny/wip-37692-luminous
luminous: librbd: disable image mirroring when moving to trash
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 15:19:14 +0000 (08:19 -0700)]
Merge pull request #28158 from dillaman/wip-38508-luminous
luminous: librbd: add missing shutdown states to managed lock helper
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Wed, 24 Jul 2019 15:18:42 +0000 (08:18 -0700)]
Merge pull request #28773 from dillaman/wip-40574-luminous
luminous: librbd: properly track in-flight flush requests
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Nathan Cutler [Wed, 24 Jul 2019 10:36:11 +0000 (12:36 +0200)]
Merge pull request #28329 from smithfarm/wip-40092-luminous
luminous: doc: Missing Documentation for radosgw-admin reshard commands (man pages)
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Nathan Cutler [Wed, 24 Jul 2019 10:29:40 +0000 (12:29 +0200)]
Merge pull request #27185 from smithfarm/wip-38902-luminous
luminous: doc: Minor rados related documentation fixes
Reviewed-by: David Zafman <dzafman@redhat.com>