Jan Fajerski [Wed, 31 Oct 2018 13:59:05 +0000 (14:59 +0100)]
ceph-volume: add inventory command
The inventory command provides information about a nodes disk inventory.
Existing logical volumes on a disk or one of its partitions are scanned
and reported.
The output can be formatted as plain text or json.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
57adfc6bb8abd9fc84b1378d5f69a16994d19e11)
Alfredo Deza [Wed, 7 Nov 2018 11:46:32 +0000 (06:46 -0500)]
Merge pull request #24960 from alfredodeza/luminous-rm36672
luminous ceph-volume tests install ceph-ansible's requirements.txt dependencies
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Andrew Schoen [Tue, 6 Nov 2018 21:46:05 +0000 (15:46 -0600)]
Merge pull request #24958 from alfredodeza/luminous-rm36704
luminous: ceph-volume tests.systemd update imports for systemd module
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Alfredo Deza [Thu, 1 Nov 2018 12:48:28 +0000 (08:48 -0400)]
ceph-volume tests install ceph-ansible's requirements.txt dependencies
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
52ddb69813c5849506db5f850a3f2532ff1124b5)
Alfredo Deza [Tue, 6 Nov 2018 14:45:08 +0000 (09:45 -0500)]
ceph-volume tests.systemd update imports for systemd module
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
a63a6c86a03738bf50fa71a490951343a83afff6)
Alfredo Deza [Tue, 6 Nov 2018 14:44:39 +0000 (09:44 -0500)]
ceph-volume systemd import main so console_scripts work for executable
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
280e2baab27cc061d2d5e456f098cd3405f36873)
Andrew Schoen [Tue, 6 Nov 2018 21:05:48 +0000 (15:05 -0600)]
Merge pull request #24827 from alfredodeza/luminous-rm36246
luminous ceph-volume util.encryption don't push stderr to terminal
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Andrew Schoen [Tue, 6 Nov 2018 21:04:30 +0000 (15:04 -0600)]
Merge pull request #24837 from alfredodeza/luminous-no-scripts
luminous ceph-volume: use console_scripts
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Andrew Schoen [Tue, 6 Nov 2018 20:38:21 +0000 (14:38 -0600)]
Merge pull request #24853 from ceph/luminous-rm36648
luminous ceph-volume systemd import main so console_scripts work for executable
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Yuri Weinstein [Mon, 5 Nov 2018 22:14:30 +0000 (14:14 -0800)]
Merge pull request #24902 from neha-ojha/wip-revert-23493-luminous
luminous: Revert "PG: add custom_reaction Backfilled and release reservations after backfill
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Mon, 5 Nov 2018 22:12:42 +0000 (14:12 -0800)]
Merge pull request #24903 from neha-ojha/wip-revert-pg-log-limit
luminous: Revert pg log limit changes
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Yuri Weinstein [Mon, 5 Nov 2018 22:11:08 +0000 (14:11 -0800)]
Merge pull request #24432 from smithfarm/wip-35935-luminous
luminous: OSDMapMapping does not handle active.size() > pool size
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Mon, 5 Nov 2018 22:10:12 +0000 (14:10 -0800)]
Merge pull request #24706 from smithfarm/wip-36552-luminous
luminous: gperftools-libs-2.6.1-1 or newer required for binaries linked against corresponding version at build time
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Yuri Weinstein [Mon, 5 Nov 2018 22:08:23 +0000 (14:08 -0800)]
Merge pull request #24746 from smithfarm/wip-36591-luminous
luminous: segv in BlueStore::OldExtent::create
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Mon, 5 Nov 2018 22:07:56 +0000 (14:07 -0800)]
Merge pull request #24802 from tchaikov/wip-luminous-35067
luminous: core: by pass cache if performing deep scrub
Reviewed-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Mon, 5 Nov 2018 19:47:26 +0000 (14:47 -0500)]
doc: make note of pg log hard limit upgrade bug in PendingReleaseNotes
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:51:03 +0000 (19:51 -0400)]
Revert "osd: make calc_trim_to() independent of min_last_complete_ondisk"
This reverts commit
3ecbf2a41fef3591ff247a212338b9a2d02a3468.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:50:50 +0000 (19:50 -0400)]
Revert "osd: print pg log length and trim_to"
This reverts commit
21a72d2a52f8241ebe0a5455f030d2095e413025.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:50:38 +0000 (19:50 -0400)]
Revert "osd: handle trim() during backfill"
This reverts commit
bdfc2a3b4fa8038ade31ccab5f5479508e9afff3.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:50:22 +0000 (19:50 -0400)]
Revert "osd: allow trim() to proceed when there are missing items"
This reverts commit
d08f0aec86ecd9b23310f1c41f9cb9060f10bc6b.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:50:03 +0000 (19:50 -0400)]
Revert "osd: reset complete_to when trimming the log past it"
This reverts commit
715d1fdbfc407de81686784d99fe28b51c6d97ef.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:49:41 +0000 (19:49 -0400)]
Revert "osd/PGLog: allow pg log trim when complete_to is less than trim_to"
This reverts commit
be8b406d9162a1f46b3eac2fca07b0cf20697b90.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:49:21 +0000 (19:49 -0400)]
Revert "osd/PGLog.cc: use lgeneric_subdout instead of generic_dout"
This reverts commit
f2ea4f8b67ac972cd428e8ea8f2ff062b1624677.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:47:54 +0000 (19:47 -0400)]
Revert "src/osd/PG.cc: remove redundant call to trim_log()"
This reverts commit
31616627dcfb2b6ab7db7da025e191dc34094bb8.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:47:37 +0000 (19:47 -0400)]
Revert "osd/PGLog.cc: check if complete_to points to log.end()"
This reverts commit
ac772c90dfc3749527246242a3a84e5e48ff52e1.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:46:22 +0000 (19:46 -0400)]
Revert "osd/PrimaryLogPG.cc: limit trimming at can_rollback_to"
This reverts commit
0e57c101c9dd51ee3a84123c96e626799ab6d21e.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:45:30 +0000 (19:45 -0400)]
Revert "osd/PrimaryLogPG: avoid dereferencing invalid complete_to"
This reverts commit
eb3bdf0714b9c78c5b3634d4efe3b603824fea09.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:34:18 +0000 (19:34 -0400)]
Revert "osd/PrimaryLogPG: fix potential pg-log overtrimming"
This reverts commit
85a029a7242a1c6eef4e90ae246541e08207302a.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 2 Nov 2018 23:37:17 +0000 (19:37 -0400)]
Revert "PG: add custom_reaction Backfilled and release reservations after backfill"
This reverts commit
f512f905b140bfa9cc3241793e441de2c377e60b.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Thu, 1 Nov 2018 16:10:36 +0000 (09:10 -0700)]
Merge pull request #24854 from cbodley/wip-36536
luminous: rgw: user stats account for resharded buckets
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 1 Nov 2018 16:10:00 +0000 (09:10 -0700)]
Merge pull request #24855 from cbodley/wip-36534
luminous: cls/user: cls_user_remove_bucket writes modified header
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 1 Nov 2018 16:09:27 +0000 (09:09 -0700)]
Merge pull request #24862 from jonsger/luminous-backport-pr#23242
luminous: rgw: set default objecter_inflight_ops = 24576
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 1 Nov 2018 16:06:06 +0000 (09:06 -0700)]
Merge pull request #24454 from croit/luminous-36041
luminous: rgw/beast: drop privileges after binding ports
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 1 Nov 2018 16:05:36 +0000 (09:05 -0700)]
Merge pull request #24507 from pdvian/wip-36212-luminous
luminous: rgw: RGWAsyncGetBucketInstanceInfo does not access coroutine memory
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 1 Nov 2018 16:05:10 +0000 (09:05 -0700)]
Merge pull request #24519 from pdvian/wip-36214-luminous
luminous: rgw: fix leak of curl handle on shutdown
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 1 Nov 2018 16:04:35 +0000 (09:04 -0700)]
Merge pull request #24570 from joke-lee/luminous-list-RGWPostObj-when-enable-versioning
luminous: rgw: list bucket can not show the object uploaded by RGWPostObj when enable bucket versioning
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 1 Nov 2018 16:03:53 +0000 (09:03 -0700)]
Merge pull request #24857 from jonsger/luminous-backport-pr#23534
luminous: rgw: multisite: enforce spawn_window for data full sync
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Matt Benjamin [Wed, 25 Jul 2018 20:06:33 +0000 (16:06 -0400)]
rgw: set default objecter_inflight_ops = 24576
Fixes: http://tracker.ceph.com/issues/25109
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit
102c5bd)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
Conflicts:
src/rgw/rgw_main.cc: use the existing vector, there is no map
Alfredo Deza [Tue, 30 Oct 2018 19:50:28 +0000 (15:50 -0400)]
ceph-volume systemd import main so console_scripts work for executable
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
0bcda52566279404b88184b2ce7b2200d581b0b7)
Casey Bodley [Wed, 17 Oct 2018 18:50:18 +0000 (14:50 -0400)]
cls/user: cls_user_remove_bucket writes modified header
if cls_user_remove_bucket() modifies the header with dec_header_stats(),
it should write the updated header back
Fixes: http://tracker.ceph.com/issues/36496
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
dac26de1fa7b56ce0ec2fd43d26ab9a9b6a32e3d)
Casey Bodley [Mon, 15 Oct 2018 20:38:31 +0000 (16:38 -0400)]
rgw: cls_user_set_buckets_info can update bucket id
if cls_user_set_bucket_info() finds an existing bucket entry, it does
not update its bucket id
Fixes: https://tracker.ceph.com/issues/24505
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
e1b97fc2e804d040e8aa6fc8a94e336222d1175e)
Casey Bodley [Mon, 15 Oct 2018 20:25:20 +0000 (16:25 -0400)]
rgw: rgw_user_sync_all_stats uses bucket entrypoint
when bucket reshard completes, rgw_link_bucket() passes the new bucket
instance id down to cls_user, but cls_user_set_buckets_info() does not
change the instance id when it's updating an existing bucket. so when
rgw_user_sync_all_stats() looks up each of the user's buckets, it uses
the original bucket instance id instead of the resharded one and
calculates user stats that may not match the current bucket stats
as a workaround, rgw_user_sync_all_stats() no longer relies on the
bucket instance id it gets from rgw_read_user_buckets(), and instead
calls get_bucket_info() to look up the current instance in the bucket
entrypoint
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
b33328ff9bacd11d8449cb3b5c52f94d9dffec6e)
Casey Bodley [Mon, 13 Aug 2018 19:47:29 +0000 (15:47 -0400)]
rgw: data sync checks whether the lease failed to renew
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry pick from commit
3754871)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
Conflicts:
src/rgw/rgw_data_sync.cc: trivial
Casey Bodley [Sat, 11 Aug 2018 15:39:35 +0000 (11:39 -0400)]
rgw: data sync holds lease over transition from full to incremental
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked commit from
3e9ac0f)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
Conflicts:
src/rgw/rgw_data_sync.cc: use ldout instead of tn->log, reflect
state of luminous in multiple places
Gregory Farnum [Tue, 30 Oct 2018 22:26:41 +0000 (15:26 -0700)]
Merge pull request #24808 from ceph/wip-luminous-mclock-test
test/common: unittest_mclock_priority_queue builds with "make" command
Mehdi Abaakouk [Fri, 26 Oct 2018 09:59:55 +0000 (11:59 +0200)]
ceph-volume: use console_scripts
Using "#!/usr/bin/env python" is not recommended as it's not portable.
setuptools provides an console_scripts entry_point that generates
scripts that always have the good sheban whatever the target operating
system and python version/distribution.
http://tracker.ceph.com/issues/36601
Signed-off-by: Mehdi Abaakouk <sileht@sileht.net>
(cherry picked from commit
6a2ed6c285b74f463f223d3501def8ef4e6283b1)
Casey Bodley [Fri, 10 Aug 2018 20:34:42 +0000 (16:34 -0400)]
rgw: enforce spawn_window for data full sync
limit the number of concurrent bucket sync crs spawned by data full sync
Fixes: http://tracker.ceph.com/issues/26897
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
895c28a)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
Conflicts:
src/rgw/rgw_data_sync.cc: parameters in yield spawn
Alfredo Deza [Wed, 3 Oct 2018 12:31:20 +0000 (08:31 -0400)]
ceph-volume util.encryption don't push stderr to terminal when cryptsetup fails to find device
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
40f919850fddd14659243c47860224981439906a)
Andrew Schoen [Mon, 29 Oct 2018 15:43:58 +0000 (10:43 -0500)]
Merge pull request #24783 from alfredodeza/luminous-evelu-ceph-volume-choose_disk
luminous Additional work on ceph-volume to add some choose_disk capabilities
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Xiaoguang Wang [Thu, 30 Aug 2018 02:26:41 +0000 (10:26 +0800)]
os/bluestore: fix deep-scrub operation againest disk silent errors
Say a object who has data caches, but in a while later, caches' underlying
physical device has silent disk erros accidentally, then caches and physical
data are not same. In such case, deep-scrub operation still tries to read
caches firstly and won't do crc checksum, then deep-scrub won't find such
data corruptions timely.
Here introduce a new flag 'CEPH_OSD_OP_FLAG_BYPASS_CLEAN_CACHE' which tells
deep-scrub to bypass object caches. Note that we only bypass cache who is in
STATE_CLEAN state. For STATE_WRITING caches, currently they are not written
to physical device, so deep-scrub operation can not read physical device and
can read these dirty caches safely. Once they are in STATE_CLEAN state(or not
added to bluestore cache), next round deep-scurb can check them correctly.
As to above discussions, I refactor BlueStore::BufferSpace::read sightly,
adding a new 'flags' argument, whose value will be 0 or:
enum {
BYPASS_CLEAN_CACHE = 0x1, // bypass clean cache
};
flags 0: normal read, do not bypass clean or dirty cache
flags BYPASS_CLEAN_CACHE: bypass clean cache, currently only for deep-scrube
operation
Test:
I deliberately corrupt a object with cache, with this patch, deep-scrub
can find data error very timely.
Signed-off-by: Xiaoguang Wang <xiaoguang.wang@easystack.cn>
(cherry picked from commit
a7f1af25dd2ba88a322ed21828f073a277b09d02)
Conflicts:
src/include/rados.h
src/os/bluestore/BlueStore.cc: trivial resolution
J. Eric Ivancich [Thu, 7 Sep 2017 21:47:00 +0000 (17:47 -0400)]
test/common: unittest_mclock_priority_queue builds with "make" command
Although it would be built with "make tests",
unittest_mclock_priority_queue was not built with a simple "make".
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit
494b2f8f5ae11a689bb2032e4527968ff09bcdeb)
Erwan Velu [Wed, 10 Oct 2018 18:26:01 +0000 (20:26 +0200)]
ceph_volume: Checking device validity at init time
When initializing the Device structure, it have to run is_valid() to
ensure the data structures (_is_valid & rejected_reasons) to be
populated accordingly to the device state.
Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit
ef0ceef7c717ef615dc214299cb4a6b32c1edbe6)
Erwan Velu [Tue, 9 Oct 2018 20:51:39 +0000 (22:51 +0200)]
ceph_volume: Rejecting locked devices
If we cannot open a block device in O_RDWR in exclusive mode, it means
someone is actually using it like a raw database or similar.
In that case, the device should be considered as unusable as OSDs will
not be in a position to use it.
Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit
d5de9583ee6c4d9ccaab2b1146606a0ee316cc27)
Erwan Velu [Tue, 9 Oct 2018 20:28:19 +0000 (22:28 +0200)]
ceph_volume: Reporting nr_requests
We are already reporting the rotational & scheduler of a disk device.
Reporting the nr_requests could be useful to get how many concurrent IOs
the device supports/reports.
That could help detecting badly detected/configured devices.
Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit
e0ea3d475a85844afdcdf92cf638b8128c8e2d9e)
Erwan Velu [Tue, 9 Oct 2018 20:26:28 +0000 (22:26 +0200)]
ceph_volume: Reporting firmware revision
We are already reporting model & vendor of a given disk, let's also
report the revision of the firmware. That is useful to filter-out some
known broken revisions.
Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit
59720794079bef0ba7f8bc2d9dfc8aaddd210005)
Erwan Velu [Tue, 9 Oct 2018 20:02:31 +0000 (22:02 +0200)]
ceph_volume: Rejecting Read-only devices
If a devices is said to be read-only, there is no chance we can actually
use it. So let's report it as unusable.
Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit
f1a94350064263f27f67d2c715c5ca5922481a01)
Erwan Velu [Tue, 9 Oct 2018 19:04:02 +0000 (21:04 +0200)]
ceph_volume: Adding Device.is_valid()
A block device can be filtered-out/ignored because it have features that
doesn't match Ceph's expectations.
As of today, the current code was rejected removable devices but it was
pretty hidden from the user, and implicit in the get_devices() function.
This patch is creating a new is_valid() function to perform all the
rejection tests and returns if this device can be used in the Ceph
context or not.
If is_valid() is returning False, the 'rejected_reasons' list reports all
the reasons why that devices got rejected.
Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit
d8fdf0b7532cdffcae56511c377679e51841caec)
Alfredo Deza [Fri, 26 Oct 2018 13:46:27 +0000 (09:46 -0400)]
Merge pull request #24759 from ceph/backport-luminous-24587
luminous: ceph-volume: adds a --prepare flag to `lvm batch`
Yuri Weinstein [Thu, 25 Oct 2018 22:22:14 +0000 (18:22 -0400)]
Merge pull request #24482 from pdvian/wip-36149-luminous
luminous: crush/CrushWrapper: fix crush tree json dumper
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Thu, 25 Oct 2018 22:21:46 +0000 (18:21 -0400)]
Merge pull request #24532 from dzafman/wip-test-luminous
luminous: backport and other test fixes for osd-scrub-repair.sh
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Yuri Weinstein [Thu, 25 Oct 2018 22:21:09 +0000 (18:21 -0400)]
Merge pull request #24538 from pdvian/wip-36229-luminous
luminous: qa: add test that builds example librados programs
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Thu, 25 Oct 2018 22:20:47 +0000 (18:20 -0400)]
Merge pull request #24574 from pdvian/wip-36295-luminous
luminous: osdc/Objecter: possible race condition with connection reset
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Thu, 25 Oct 2018 22:20:10 +0000 (18:20 -0400)]
Merge pull request #24582 from smithfarm/wip-36437-luminous
luminous: ec: src/common/interval_map.h: 161: FAILED assert(len > 0)
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Thu, 25 Oct 2018 22:19:46 +0000 (18:19 -0400)]
Merge pull request #24593 from dzafman/wip-36419
luminous: osd: get loadavg per cpu for scrub load threshold check
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Thu, 25 Oct 2018 22:19:20 +0000 (18:19 -0400)]
Merge pull request #24602 from pdvian/wip-36297-luminous
luminous: osd: add creating to pg_string_state
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Thu, 25 Oct 2018 22:18:10 +0000 (18:18 -0400)]
Merge pull request #24659 from tchaikov/luminous-26890
luminous: osd: cast 'whoami' to unsigned so it can be used as the seed for RNG
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
David Galloway [Thu, 25 Oct 2018 21:47:04 +0000 (17:47 -0400)]
Andrew Schoen [Thu, 25 Oct 2018 12:51:41 +0000 (07:51 -0500)]
ceph-volume: update man page for batch --prepare
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
faa3aed4ad4df2dbdd853235e364d6e3b9b176b6)
Andrew Schoen [Mon, 15 Oct 2018 15:38:43 +0000 (10:38 -0500)]
ceph-volume: docs for --prepare flag of lvm batch
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
b4825e1bd7b75a0a53ee5678230b262286293d75)
Andrew Schoen [Mon, 15 Oct 2018 15:35:12 +0000 (10:35 -0500)]
ceph-volume: when --prepare is added to batch the OSDs are only prepared
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
abd238dc1f33252a43adc8b86efb3d3b4a076964)
Andrew Schoen [Mon, 15 Oct 2018 14:33:30 +0000 (09:33 -0500)]
ceph-volume: adds a --prepare flag to ceph-volume lvm batch
This flag will only prepare the OSDs, not activate them. This is useful
in our containerized ceph solution.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
4368ca7d48f4ce35f67854e8b0ca2be9babc5371)
Andrew Schoen [Thu, 25 Oct 2018 16:27:42 +0000 (11:27 -0500)]
Merge pull request #24754 from alfredodeza/luminous-rm36386
luminous ceph-volume remove version reporting from help menu
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Yuri Weinstein [Thu, 25 Oct 2018 16:20:23 +0000 (12:20 -0400)]
Merge pull request #23493 from VictorDenisov/backport_24333
luminous: PG: add custom_reaction Backfilled and release reservations after bac…
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Alfredo Deza [Wed, 10 Oct 2018 19:35:31 +0000 (15:35 -0400)]
ceph-volume remove version reporting from help menu
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
20db7bf585f6433c6705396264005a4227cbf2e3)
Jenkins Build Slave User [Wed, 24 Oct 2018 21:04:19 +0000 (21:04 +0000)]
12.2.9
Andrew Schoen [Wed, 24 Oct 2018 20:06:35 +0000 (15:06 -0500)]
Merge pull request #24741 from alfredodeza/luminous-rm36492
luminous ceph-volume: do not send (lvm) stderr/stdout to the terminal, use the logfile
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Sage Weil [Mon, 22 Oct 2018 19:38:48 +0000 (14:38 -0500)]
os/bluestore: fix race between SharedBlobSet::lookup and SharedBlob::put
A B
SharedBlobSet::lookup()
takes lock
nref is not 0
SharedBlob::put()
--nref
returns SharedBlobRef,
++nref
takes cache lock
SharedBlobSet::remove
takes lock
removes
deletes SharedBlob
-> A ends up with a ref to deleted SharedBlob
Fix by verifying that nref is still zero in SharedBlobSet::remove(),
while we are holding the SharedBlobSet::lock. The lock ensures that we
have increased the ref for the lookup before entering remove, so we can
verify that nref is still zero before removing it. If not, we have
raced, and put() bails out and does nothing.
Fixes: http://tracker.ceph.com/issues/36526
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
020bd7b5f38a82d9eef5e25e6f4a4dd12b066915)
Conflicts:
src/os/bluestore/BlueStore.h
Alfredo Deza [Wed, 24 Oct 2018 14:55:06 +0000 (10:55 -0400)]
ceph-volume tests.api update monkeypatching to use **kw
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
6d6bcda75f846bc7c44c7b16b3687a65fed7cafc)
Alfredo Deza [Wed, 24 Oct 2018 14:54:48 +0000 (10:54 -0400)]
ceph-volume tests allow **kw when monkeypatching
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
04e51c956eadcbd38f39ec3dd4b78009d2cd4a70)
Alfredo Deza [Wed, 24 Oct 2018 14:54:18 +0000 (10:54 -0400)]
ceph-volume api.lvm: do not spit out errors on the terminal from LVM
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
581eed36ea0e7b5c5e41f8b83350403bcf73a31a)
Brad Hubbard [Tue, 16 Oct 2018 01:57:05 +0000 (11:57 +1000)]
rpm: Use updated gperftools-libs at runtime
Due to ABI breakage in libtcmalloc.so.4 we need to specify the minimum
version to be used at runtime to be greater than or equal to the version
used at build time.
Fixes: http://tracker.ceph.com/issues/36508
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
24636267f957ff9518396ac1b8212f2402286d82)
Nathan Cutler [Wed, 11 Apr 2018 17:46:01 +0000 (19:46 +0200)]
build/ops: move parted runtime dependency to ceph-base
Since ceph-disk has been moved from "ceph-osd" to "ceph-base", all of
ceph-disk's dependencies should also be moved, provided they are not
needed by anything else.
Fixes: http://tracker.ceph.com/issues/23657
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit
acf6423cf6c6a47dbf59a99052a227c229d47004)
Conflicts:
ceph.spec.in
- trivial resolution
Nathan Cutler [Wed, 11 Apr 2018 17:27:14 +0000 (19:27 +0200)]
build/ops: rpm: move sgdisk-providing dep to ceph-base
In Debian "gdisk" is already listed under ceph-base, so this commit is
RPM-only.
sgdisk is only used by ceph-disk (part of "ceph-base"), not by ceph-volume
(part of "ceph-osd")
Fixes: http://tracker.ceph.com/issues/23657
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit
e23a12c2cb90ebe93f7c02cf8e98ff99fc34fd3f)
Conflicts:
ceph.spec.in
- trivial resolution
Yuri Weinstein [Sun, 21 Oct 2018 16:58:21 +0000 (12:58 -0400)]
Merge pull request #24620 from dillaman/wip-36431-luminous
luminous: qa/workunits: replace 'realpath' with 'readlink -f' in fsstress.sh
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Thu, 18 Oct 2018 19:35:16 +0000 (15:35 -0400)]
Merge pull request #24650 from yuriw/wip-yuriw-fix-jewel-x-luminous
Excluded 'python34-cephfs' from the install tasks
Reviewed-by: David Galloway <dgallowa@redhat.com>
Yuri Weinstein [Wed, 17 Oct 2018 23:27:54 +0000 (16:27 -0700)]
Excluded 'python34-cephfs','python34-rados','python34-rbd','python34-rgw','python34-ceph-argparse','python3-cephfs','python3-rados' from the install tasks
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Andrew Schoen [Thu, 18 Oct 2018 15:21:10 +0000 (10:21 -0500)]
Merge pull request #24663 from alfredodeza/luminous-rm23650
luminous: specs: require of e2fsprogs
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Alfredo Deza [Thu, 18 Oct 2018 14:39:55 +0000 (11:39 -0300)]
rpm: require 2fsprogs needed for ceph-disk when using dmcrypt
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Guillaume Abrioux [Wed, 11 Apr 2018 10:05:33 +0000 (12:05 +0200)]
specs: require of e2fsprogs
in ceph/ceph-container we've realized that `e2fsprogs` isn't installed in
centos container image because ceph hasn't a dependency for it.
It has for consequence to fail when deploying a containerized cluster
with dmcrypt when using centos image.
Typical error encountered:
typical error:
```
......
get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sda5 uuid path is /sys/dev/block/8:5/dm/uuid
populate: Creating lockbox fs on %s: mkfs -t ext4 /dev/sda5
command_check_call: Running command: /usr/sbin/mkfs -t ext4 /dev/sda5
mkfs.ext4: No such file or directory
Traceback (most recent call last):
File "/usr/sbin/ceph-disk", line 9, in <module>
load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')()
......
```
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit
a99177834120e7a2c4592054f6a8b8736e0ffb92)
Kefu Chai [Thu, 18 Oct 2018 10:29:49 +0000 (18:29 +0800)]
osd: cast `whoami` to unsigned so it can be used as the seed for RNG
default_random_engine's result_type is `unsigned int`, so we need to
pass an `unsigned int` as its seed.
Fixes: http://tracker.ceph.com/issues/26890
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
src/osd/OSD.cc: this breaks the build with clang. and in master
we are not using std::default_random_engine for setting the scrub
interval. so this change is not cherry-picked from master.
Andrew Schoen [Tue, 16 Oct 2018 19:40:41 +0000 (14:40 -0500)]
Merge pull request #24589 from ceph/backport-luminous-24404
luminous: ceph-volume: make `lvm batch` idempotent
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Andrew Schoen [Tue, 16 Oct 2018 18:13:59 +0000 (13:13 -0500)]
Merge pull request #24451 from alfredodeza/luminous-wip-rm24795
luminous: ceph-volume lvm.prepare update help to indicate partitions are needed, not devices
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Jason Dillaman [Fri, 12 Oct 2018 14:02:35 +0000 (10:02 -0400)]
qa/tasks/workunit: use suite branch/SHA1 when cloning workunits
Right now it's using the Ceph branch/SHA1 but it's using the suite
Git URL.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
1e3dc02604cf7c0d3991dfd9fe2596ea34e80bad)
Jason Dillaman [Thu, 11 Oct 2018 20:21:35 +0000 (16:21 -0400)]
qa/tasks: qemu task now uses a relative path in suite repo for test
This makes it easier to re-run tests against a suite branch without
requiring a full ceph-ci build and repo.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
c1f950236b2f1ba3b588fc638ebf8cb785c36e08)
Jason Dillaman [Thu, 11 Oct 2018 19:17:25 +0000 (15:17 -0400)]
qa/workunits: replace 'realpath' with 'readlink -f' in fsstress.sh
Fixes: http://tracker.ceph.com/issues/36409
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
ddb7e5eb80ebb78beff6dfb25042f0016a474491)
Ilya Dryomov [Thu, 6 Sep 2018 14:07:08 +0000 (16:07 +0200)]
qa/tasks/cram: tasks now must live in the repository
Commit
0d8887652d53 ("qa/tasks/cram: use suite_repo repository for all
cram jobs") removed hardcoded git.ceph.com links, but as it turned out
it is still used for nightlies. There is no good way to accommodate
the different URL schemes, so let's get rid of URLs altogether.
Fixes: https://tracker.ceph.com/issues/27211
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
592f566b4e270d3833ba896dabb193d5241dd5de)
Conflicts:
qa/suites/krbd/basic/tasks/krbd_blkroset.yaml
qa/suites/krbd/basic/tasks/krbd_huge_image.yaml
qa/suites/krbd/basic/tasks/krbd_msgr_segments.yaml
qa/suites/krbd/basic/tasks/krbd_parent_overlap.yaml
qa/suites/krbd/basic/tasks/krbd_whole_object_discard.yaml
- in master, the cram task is referred to in these additional yaml
files, but in luminous it's only referred to in
qa/suites/krbd/unmap/tasks/unmap.yaml
Ilya Dryomov [Thu, 6 Sep 2018 14:53:25 +0000 (16:53 +0200)]
qa/tasks/workunit: factor out overrides and refspec logic
Allow for reuse in the cram task.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
e1c89b51c80407fadbfee82d5d396cfcbd791aae)
Ilya Dryomov [Mon, 3 Sep 2018 15:40:08 +0000 (17:40 +0200)]
qa/tasks/cram: use suite_repo repository for all cram jobs
Currently git.ceph.com is hardcoded for all cram jobs. Testing
modifications is a pain: one needs to push to either ceph/ceph.git or
ceph/ceph-ci.git (depending on where the ceph branch is at, triggering
unnecessary builds in the latter case) and wait for the mirror to sync.
Runs scheduled against branches in developer's forks fail.
Move away from git.ceph.com to allow mixing branches and repositories,
similar to workunits.
Fixes: https://tracker.ceph.com/issues/27211
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
0d8887652d5312f7059ab2bdb52c948aa01680b0)
Conflicts:
qa/suites/krbd/basic/tasks/krbd_blkroset.yaml
qa/suites/krbd/basic/tasks/krbd_huge_image.yaml
qa/suites/krbd/basic/tasks/krbd_msgr_segments.yaml
qa/suites/krbd/basic/tasks/krbd_parent_overlap.yaml
qa/suites/krbd/basic/tasks/krbd_whole_object_discard.yaml
- in master, the cram task is referred to in these additional yaml
files, but in luminous it's only referred to in
qa/suites/krbd/unmap/tasks/unmap.yaml
Dan van der Ster [Tue, 25 Sep 2018 08:39:37 +0000 (10:39 +0200)]
osd: add creating to pg_string_state
Fixes: http://tracker.ceph.com/issues/36174
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit
d38f6a11701ec788e4d384aa5b0ae65b8e57da64)
Conflicts:
src/osd/osd_types.cc : Resolved in pg_string_state
Andrew Schoen [Wed, 10 Oct 2018 19:28:29 +0000 (15:28 -0400)]
ceph-volume: extracts batch.filter_devices from Batch._get_strategy
This allows us to easily provide tests for that method.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
df7ef5383bb476ee020b898cd2e8fbce044fc07e)