Matt Benjamin [Wed, 25 Jul 2018 20:06:33 +0000 (16:06 -0400)]
rgw: set default objecter_inflight_ops = 24576
Fixes: http://tracker.ceph.com/issues/25109 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 102c5bd) Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
Conflicts:
src/rgw/rgw_main.cc: use the existing vector, there is no map
when bucket reshard completes, rgw_link_bucket() passes the new bucket
instance id down to cls_user, but cls_user_set_buckets_info() does not
change the instance id when it's updating an existing bucket. so when
rgw_user_sync_all_stats() looks up each of the user's buckets, it uses
the original bucket instance id instead of the resharded one and
calculates user stats that may not match the current bucket stats
as a workaround, rgw_user_sync_all_stats() no longer relies on the
bucket instance id it gets from rgw_read_user_buckets(), and instead
calls get_bucket_info() to look up the current instance in the bucket
entrypoint
Casey Bodley [Sat, 11 Aug 2018 15:39:35 +0000 (11:39 -0400)]
rgw: data sync holds lease over transition from full to incremental
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked commit from 3e9ac0f) Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
Conflicts:
src/rgw/rgw_data_sync.cc: use ldout instead of tn->log, reflect
state of luminous in multiple places
Mehdi Abaakouk [Fri, 26 Oct 2018 09:59:55 +0000 (11:59 +0200)]
ceph-volume: use console_scripts
Using "#!/usr/bin/env python" is not recommended as it's not portable.
setuptools provides an console_scripts entry_point that generates
scripts that always have the good sheban whatever the target operating
system and python version/distribution.
Xiaoguang Wang [Thu, 30 Aug 2018 02:26:41 +0000 (10:26 +0800)]
os/bluestore: fix deep-scrub operation againest disk silent errors
Say a object who has data caches, but in a while later, caches' underlying
physical device has silent disk erros accidentally, then caches and physical
data are not same. In such case, deep-scrub operation still tries to read
caches firstly and won't do crc checksum, then deep-scrub won't find such
data corruptions timely.
Here introduce a new flag 'CEPH_OSD_OP_FLAG_BYPASS_CLEAN_CACHE' which tells
deep-scrub to bypass object caches. Note that we only bypass cache who is in
STATE_CLEAN state. For STATE_WRITING caches, currently they are not written
to physical device, so deep-scrub operation can not read physical device and
can read these dirty caches safely. Once they are in STATE_CLEAN state(or not
added to bluestore cache), next round deep-scurb can check them correctly.
As to above discussions, I refactor BlueStore::BufferSpace::read sightly,
adding a new 'flags' argument, whose value will be 0 or:
enum {
BYPASS_CLEAN_CACHE = 0x1, // bypass clean cache
};
flags 0: normal read, do not bypass clean or dirty cache
flags BYPASS_CLEAN_CACHE: bypass clean cache, currently only for deep-scrube
operation
Test:
I deliberately corrupt a object with cache, with this patch, deep-scrub
can find data error very timely.
Erwan Velu [Wed, 10 Oct 2018 18:26:01 +0000 (20:26 +0200)]
ceph_volume: Checking device validity at init time
When initializing the Device structure, it have to run is_valid() to
ensure the data structures (_is_valid & rejected_reasons) to be
populated accordingly to the device state.
Erwan Velu [Tue, 9 Oct 2018 20:28:19 +0000 (22:28 +0200)]
ceph_volume: Reporting nr_requests
We are already reporting the rotational & scheduler of a disk device.
Reporting the nr_requests could be useful to get how many concurrent IOs
the device supports/reports.
That could help detecting badly detected/configured devices.
Erwan Velu [Tue, 9 Oct 2018 20:26:28 +0000 (22:26 +0200)]
ceph_volume: Reporting firmware revision
We are already reporting model & vendor of a given disk, let's also
report the revision of the firmware. That is useful to filter-out some
known broken revisions.
Sage Weil [Mon, 22 Oct 2018 19:38:48 +0000 (14:38 -0500)]
os/bluestore: fix race between SharedBlobSet::lookup and SharedBlob::put
A B
SharedBlobSet::lookup()
takes lock
nref is not 0
SharedBlob::put()
--nref
returns SharedBlobRef,
++nref
takes cache lock
SharedBlobSet::remove
takes lock
removes
deletes SharedBlob
-> A ends up with a ref to deleted SharedBlob
Fix by verifying that nref is still zero in SharedBlobSet::remove(),
while we are holding the SharedBlobSet::lock. The lock ensures that we
have increased the ref for the lookup before entering remove, so we can
verify that nref is still zero before removing it. If not, we have
raced, and put() bails out and does nothing.
Brad Hubbard [Tue, 16 Oct 2018 01:57:05 +0000 (11:57 +1000)]
rpm: Use updated gperftools-libs at runtime
Due to ABI breakage in libtcmalloc.so.4 we need to specify the minimum
version to be used at runtime to be greater than or equal to the version
used at build time.
build/ops: move parted runtime dependency to ceph-base
Since ceph-disk has been moved from "ceph-osd" to "ceph-base", all of
ceph-disk's dependencies should also be moved, provided they are not
needed by anything else.
Yuri Weinstein [Wed, 17 Oct 2018 23:27:54 +0000 (16:27 -0700)]
Excluded 'python34-cephfs','python34-rados','python34-rbd','python34-rgw','python34-ceph-argparse','python3-cephfs','python3-rados' from the install tasks
in ceph/ceph-container we've realized that `e2fsprogs` isn't installed in
centos container image because ceph hasn't a dependency for it.
It has for consequence to fail when deploying a containerized cluster
with dmcrypt when using centos image.
Typical error encountered:
typical error:
```
......
get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sda5 uuid path is /sys/dev/block/8:5/dm/uuid
populate: Creating lockbox fs on %s: mkfs -t ext4 /dev/sda5
command_check_call: Running command: /usr/sbin/mkfs -t ext4 /dev/sda5
mkfs.ext4: No such file or directory
Traceback (most recent call last):
File "/usr/sbin/ceph-disk", line 9, in <module>
load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')()
......
```
Kefu Chai [Thu, 18 Oct 2018 10:29:49 +0000 (18:29 +0800)]
osd: cast `whoami` to unsigned so it can be used as the seed for RNG
default_random_engine's result_type is `unsigned int`, so we need to
pass an `unsigned int` as its seed.
Fixes: http://tracker.ceph.com/issues/26890 Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
src/osd/OSD.cc: this breaks the build with clang. and in master
we are not using std::default_random_engine for setting the scrub
interval. so this change is not cherry-picked from master.
qa/tasks/cram: tasks now must live in the repository
Commit 0d8887652d53 ("qa/tasks/cram: use suite_repo repository for all
cram jobs") removed hardcoded git.ceph.com links, but as it turned out
it is still used for nightlies. There is no good way to accommodate
the different URL schemes, so let's get rid of URLs altogether.
Conflicts:
qa/suites/krbd/basic/tasks/krbd_blkroset.yaml
qa/suites/krbd/basic/tasks/krbd_huge_image.yaml
qa/suites/krbd/basic/tasks/krbd_msgr_segments.yaml
qa/suites/krbd/basic/tasks/krbd_parent_overlap.yaml
qa/suites/krbd/basic/tasks/krbd_whole_object_discard.yaml
- in master, the cram task is referred to in these additional yaml
files, but in luminous it's only referred to in
qa/suites/krbd/unmap/tasks/unmap.yaml
qa/tasks/cram: use suite_repo repository for all cram jobs
Currently git.ceph.com is hardcoded for all cram jobs. Testing
modifications is a pain: one needs to push to either ceph/ceph.git or
ceph/ceph-ci.git (depending on where the ceph branch is at, triggering
unnecessary builds in the latter case) and wait for the mirror to sync.
Runs scheduled against branches in developer's forks fail.
Move away from git.ceph.com to allow mixing branches and repositories,
similar to workunits.
Conflicts:
qa/suites/krbd/basic/tasks/krbd_blkroset.yaml
qa/suites/krbd/basic/tasks/krbd_huge_image.yaml
qa/suites/krbd/basic/tasks/krbd_msgr_segments.yaml
qa/suites/krbd/basic/tasks/krbd_parent_overlap.yaml
qa/suites/krbd/basic/tasks/krbd_whole_object_discard.yaml
- in master, the cram task is referred to in these additional yaml
files, but in luminous it's only referred to in
qa/suites/krbd/unmap/tasks/unmap.yaml
Dan van der Ster [Tue, 25 Sep 2018 08:39:37 +0000 (10:39 +0200)]
osd: add creating to pg_string_state
Fixes: http://tracker.ceph.com/issues/36174 Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit d38f6a11701ec788e4d384aa5b0ae65b8e57da64)
Conflicts:
src/osd/osd_types.cc : Resolved in pg_string_state
projects / ceph.git / log