]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Yuri Weinstein [Tue, 30 Jul 2019 16:10:43 +0000 (09:10 -0700)]
Merge pull request #29197 from smithfarm/wip-38963-mimic
mimic: mgr: DaemonServer::handle_conf_change - broken locking
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Tue, 30 Jul 2019 16:09:58 +0000 (09:09 -0700)]
Merge pull request #29201 from smithfarm/wip-40268-mimic
mimic: common: data race in OutputDataSocket
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 30 Jul 2019 16:09:25 +0000 (09:09 -0700)]
Merge pull request #29217 from smithfarm/wip-39692-mimic
mimic: bluestore: _txc_add_transaction error (39) Directory not empty not handled on operation 21 (op 1, counting from 0)
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Jan Fajerski [Mon, 29 Jul 2019 15:57:41 +0000 (17:57 +0200)]
Merge pull request #29243 from jan--f/mimic-c-v-extend-batch
mimic: c-v extend batch
Andrew Schoen [Tue, 27 Nov 2018 23:34:10 +0000 (17:34 -0600)]
ceph-volume: create two NVME drives for testing
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: bz#
1650306
(cherry picked from commit
893b61b256bcff142dbd90b777b2c9df9e032e9d )
Andrew Schoen [Wed, 28 Nov 2018 16:10:24 +0000 (10:10 -0600)]
ceph-volume: set block_db_size to 9GB in centos7 mixed type test
This is to enable testing of the --block-db-size flag
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
eab962c6ef68c588674bdac6fc4a86c84714f300 )
Andrew Schoen [Wed, 28 Nov 2018 16:05:45 +0000 (10:05 -0600)]
ceph-volume: use two NVME drives in centos7 mixed_type tests
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
934c32acb318aac9aff3c800cdbf99bbdee04d06 )
Alfredo Deza [Fri, 26 Jul 2019 18:33:21 +0000 (14:33 -0400)]
Merge pull request #29350 from jan--f/wip-40977-mimic
mimic: ceph-volume: missing string substitution when reporting mounts
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Shyukri Shyukriev [Fri, 7 Jun 2019 12:09:19 +0000 (12:09 +0000)]
ceph-volume:util: Use proper param substition
Fixes: https://tracker.ceph.com/issues/25030
Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
(cherry picked from commit
654660cd89fe9d93087d390805651c860bf4e862 )
Alfredo Deza [Thu, 25 Jul 2019 15:54:37 +0000 (11:54 -0400)]
Merge pull request #26991 from ceph/backport-mimic-26957
mimic: ceph-volume: look for rotational data in lsblk
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Andrew Schoen [Thu, 14 Mar 2019 20:14:40 +0000 (15:14 -0500)]
ceph-volume: look for rotational data in lsblk
Also ask lsblk if a device is rotational if no information
is found in /sys/block, default to True if nothing is found.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=
1666822
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
de748f9aef894b247622ee5dc49c1fbc31097ad0 )
Yuri Weinstein [Thu, 25 Jul 2019 14:37:25 +0000 (07:37 -0700)]
Merge pull request #28918 from smithfarm/wip-40222-mimic
mimic: mds: reset heartbeat during long-running loops in recovery
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 14:36:55 +0000 (07:36 -0700)]
Merge pull request #29187 from xiaoxichen/wip-40875-mimic
mimic: cephfs: client: bump ll_ref from int32 to uint64_t
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 14:36:25 +0000 (07:36 -0700)]
Merge pull request #29200 from smithfarm/wip-39685-mimic
mimic: cephfs: ceph-fuse: client hang because its bad session PipeConnection to mds
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 14:35:41 +0000 (07:35 -0700)]
Merge pull request #29208 from smithfarm/wip-40317-mimic
mimic: tests: test_volume_client: print python version correctly
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 14:35:08 +0000 (07:35 -0700)]
Merge pull request #29210 from smithfarm/wip-38099-mimic
mimic: mds: remove cache drop admin socket command
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 14:34:17 +0000 (07:34 -0700)]
Merge pull request #29211 from smithfarm/wip-38687-mimic
mimic: kcephfs TestClientLimits.test_client_pin fails with client caps fell below min
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:06:01 +0000 (16:06 -0700)]
Merge pull request #27028 from mrandybu/mimic-alt
mimic: ceph_detect_init: Add support for ALT Linux
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:04:23 +0000 (16:04 -0700)]
Merge pull request #28210 from votdev/fix_rgw_issue
mimic: mgr/dashboard: Fix various RGW issues
Reviewed-by: Tiago Melo <tmelo@suse.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:03:24 +0000 (16:03 -0700)]
Merge pull request #28911 from bk201/wip-40660-mimic
mimic: mgr/dashboard: cephfs multimds graphs stack together
Reviewed-by: Ricardo Dias <rdias@suse.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:02:19 +0000 (16:02 -0700)]
Merge pull request #29024 from ifed01/wip-ifed-fix-len0-mimic
mimic: os/bluestore: avoid length overflow in extents returned by Stupid Allocator
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:00:55 +0000 (16:00 -0700)]
Merge pull request #29174 from neha-ojha/wip-40583-mimic
mimic: common/options.cc: Lower the default value of osd_deep_scrub_large_omap_object_key_threshold
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 22:59:50 +0000 (15:59 -0700)]
Merge pull request #29189 from badone/wip-mimic-lazy-omap-stats-backport-tracker-38552
mimic: core: lazy omap stat collection
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 22:58:23 +0000 (15:58 -0700)]
Merge pull request #29196 from smithfarm/wip-37749-mimic
mimic: tools: Add clear-data-digest command to objectstore tool
Reviewed-by: David Zafman <dzafman@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 15:26:33 +0000 (08:26 -0700)]
Merge pull request #29016 from ivancich/mimic-abort-multipart-fix
mimic: rgw: abort multipart fix
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 15:26:07 +0000 (08:26 -0700)]
Merge pull request #29021 from smithfarm/wip-39074-mimic
mimic: rgw: multisite: data sync loops back to the start of the datalog after reaching the end
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 15:25:38 +0000 (08:25 -0700)]
Merge pull request #29139 from ivancich/mimic-fix-resharding-race
mimic: rgw: fix race b/w bucket reshard and ops waiting on reshard completion
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 15:22:59 +0000 (08:22 -0700)]
Merge pull request #29017 from smithfarm/wip-38080-mimic
mimic: multisite: overwrites in versioning-suspended buckets fail to sync
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Jan Fajerski [Wed, 16 Jan 2019 12:48:25 +0000 (13:48 +0100)]
ceph-volume: add explicit mixed-type functional tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
bea35c63c83fc2ab87fc983774ed8181cbe74e5b )
Andrew Schoen [Thu, 6 Dec 2018 18:45:10 +0000 (12:45 -0600)]
ceph-volume: add tests for Device.lvm_size
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
01185fb32479f271beeb14dcf94e32b1401ad2d9 )
Andrew Schoen [Wed, 5 Dec 2018 15:43:28 +0000 (09:43 -0600)]
ceph-volume: adds a lvm_size property to util.Device
This will give us the size lvm should report because it takes into
account the 1GB physical extent size we set.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
5197c16b44fcf344862201549777dc2935fc19a3 )
Jan Fajerski [Thu, 20 Dec 2018 12:53:41 +0000 (13:53 +0100)]
ceph-volume: functional tests, add libvirt customization
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
c330a4758755e3815b30e4f00a4a00e8319db399 )
Sage Weil [Thu, 2 May 2019 16:39:31 +0000 (11:39 -0500)]
os/bluestore: be verbose about objects that existing on rmcoll
This is always a bug (OSD doesn't try to remove a collection unless it
thinks it is empty), and not seeing it at default debug levels makes it
hard to track down.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
e387cbc45037db78db5dfbb01483b8d76e1c276f )
Conflicts:
src/os/bluestore/BlueStore.cc
- mimic does not have
87e8231d3fa61bc95ea0cda4d731e30f3d4de9d5
Sage Weil [Thu, 2 May 2019 16:30:09 +0000 (11:30 -0500)]
osd/PrimaryLogPG: disallow ops on objects with an empty name
These may conflict with pgmeta objects. And are going to cause other
problems later (e.g., https://tracker.ceph.com/issues/38724).
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
bf6e8a4a4db8b7d7101ca80b844ee953aa4580b2 )
Sage Weil [Thu, 18 Jul 2019 16:09:47 +0000 (11:09 -0500)]
osd/PG: fix cleanup of pgmeta-like objects on PG deletion
If an object has an empty 'name' field, it "looks" like a pgmeta object,
and the PG cleanup code was skipping it. However, we were letting these
objects get created.
Fix by only skipping *our* pgmeta object. If there are other pgmeta-like
objects in the PG collection, clean them up.
Fixes: https://tracker.ceph.com/issues/38724
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
a0f48db5cbfc48761eeac0daa259fecbaa573969 )
# Conflicts:
# src/osd/PG.cc
- location of this code is do_delete_work in master, _delete_some in n
Casey Bodley [Thu, 6 Jun 2019 13:48:42 +0000 (09:48 -0400)]
common: OutputDataSocket retakes mutex on error path
the list of buffers is protected by mutex m_lock. when dump_data() fails
and puts unwritten buffers back into the list, it needs to reaquire the
lock
Fixes: http://tracker.ceph.com/issues/40188
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
8645e24dd4fb3ecc9f02b34a1be916620b5f71b1 )
Conflicts:
src/common/OutputDataSocket.cc
- use Mutex::Locker instead of std::scoped_lock
xie xingguo [Tue, 26 Mar 2019 08:08:43 +0000 (16:08 +0800)]
mgr/DaemonServer: handle_conf_change - fix broken locking
The is_locked_by_me() is only meant to be safe if you know
you hold the lock. It's meant to be used in an assertion.
Fixes: http://tracker.ceph.com/issues/38899
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit
a929a5a01508e661fc7f3cea52822320d39983d2 )
Conflicts:
src/mgr/DaemonServer.cc
- use Mutex::Locker instead of std::lock_guard
liyichao [Wed, 5 Dec 2018 04:22:20 +0000 (12:22 +0800)]
tools: Add clear-data-digest command to objectstore tool.
There may be a situation where data digest in object info is
inconsistent with that computed from object data, then deep-scrub
will fail even though all three repicas have the same object data.
Fixes: https://tracker.ceph.com/issues/37935
Signed-off-by: Li Yichao <liyichao.good@gmail.com>
(cherry picked from commit
da5832b2b49be4ab23f7ffc5ba1ba630168b886f )
Conflicts:
src/tools/ceph_objectstore_tool.cc
- change cbegin() to begin() to get it to build
Yan, Zheng [Thu, 7 Mar 2019 13:12:28 +0000 (21:12 +0800)]
qa/cephfs: relax min_caps_per_client check
new kernel client proactively release caps. caps count can go below
mds_min_caps_per_client
Fixes: http://tracker.ceph.com/issues/38270
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
8e81bd74c59259a06f587bb425d58bf2b2176341 )
Conflicts:
qa/tasks/cephfs/test_client_limits.py
- mimic TestClientLimits does not have expected_caps method at all
Patrick Donnelly [Wed, 23 Jan 2019 02:26:38 +0000 (18:26 -0800)]
mds: remove cache drop asok command
`cache drop` is a long running command that will block the asok interface
(while the tell version does not). Attempting to abort the command with ^C or
equivalents will simply cause the `ceph` command to exit but won't stop the
asok command handler from waiting for the cache drop operation to complete.
Instead, just allow the tell version.
Fixes: http://tracker.ceph.com/issues/38020
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
7fa1e3c37f8c7fb709ae9e070a3154a7084c0584 )
Conflicts:
PendingReleaseNotes
qa/tasks/cephfs/test_misc.py
src/mds/MDSDaemon.cc
Lianne [Tue, 11 Jun 2019 10:27:06 +0000 (18:27 +0800)]
qa/tasks/cephfs/test_volume_client: print py2 or py3 which the test case runs
Fixes: http://tracker.ceph.com/issues/40184
Signed-off-by: Lianne <liyan.wang@xtaotech.com>
(cherry picked from commit
7c7c7870d38902a0df83a0fdecaa56baad556d82 )
IvanGuan [Wed, 17 Apr 2019 02:21:54 +0000 (10:21 +0800)]
client: fix fuse client hang because its bad session PipeConnection
to mds.
There still has a risk client will hang all the time and which
happened in my enviroment a few days ago. Well konw, the start
of mds will go through four stages at least(replay, reconnect,
rejoin, active). According to current logic, we recived
replay mdsmap and marked down pipe will return directly and if
we miss the next reconnect mdsmap the client may hang all the
time becuase we lost the opportunity due to old_inc equal to
new_inc. So we shuold judge if we have lost MDSMap::reconnect
by comparing newstate and oldstate.
Fixes: http://tracker.ceph.com/issues/39305
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
(cherry picked from commit
51c9f5cf9af61fe317259ac79a503ef8a7139622 )
Conflicts:
src/client/Client.cc
- mimic has "kick_maxsize_requests(session); wake_inode_waiters(session);"
where master has just "wake_up_session_caps(session, true)
Brad Hubbard [Tue, 9 Jul 2019 05:03:24 +0000 (15:03 +1000)]
mimic: osd: Modify lazy omap stats to only show one copy
Only show stats from the primary to get more in line with other pg stats
representations.
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
490829c8eaf14f41de978ac0f93ecda3df8bcdef )
Brad Hubbard [Mon, 4 Mar 2019 08:12:27 +0000 (18:12 +1000)]
mimic: tests: Add test for lazy omap stat collection
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
88e9ca58a06ba572ff4c6f6125a8ba7f8e0a7795 )
Brad Hubbard [Tue, 11 Dec 2018 04:00:35 +0000 (14:00 +1000)]
mimic: osd: Implement lazy omap usage statistics
Opportunistic gathering of omap statistics during deep scrub.
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
9e21ef0e40ff89b10dfdf857c703336496b3171f )
Conflicts:
doc/dev/placement-group.rst: Trivial
src/osd/osd_types.h: Trivial
Xiaoxi CHEN [Fri, 19 Jul 2019 15:31:24 +0000 (08:31 -0700)]
Client: bump ll_ref from int32 to uint64_t
ll_ref can surge up if application repeatly lookup same dentry
and finally cause overflow.
https://tracker.ceph.com/issues/40775 is an example.
in kernel fuse driver and libfuse, uint64_t is used, update
ceph-fuse side to match.
Fixes: https://tracker.ceph.com/issues/40775
Signed-off-by: Xiaoxi CHEN <xiaoxchen@ebay.com>
(cherry picked from commit
bb46de885336a8cd2adc9c0507e543c6832346e7 )
Conflicts:
src/client/Client.h
use new code, skip all non-toched changes.
src/client/Inode.h
use new code.
Neha Ojha [Fri, 28 Jun 2019 16:31:35 +0000 (09:31 -0700)]
common/options.cc: Lower the default value of osd_deep_scrub_large_omap_object_key_threshold
Warning when there are already 2M omap keys does not help, reduce this
threshold so that preemptive measures can be taken based on this.
Fixes: https://tracker.ceph.com/issues/40583
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit
1ec948409ab547effbd5c656af8f0c9565f1717a )
Casey Bodley [Fri, 25 Jan 2019 21:44:03 +0000 (16:44 -0500)]
rgw: add RGWRados::repair_olh() to reconstruct olh attrs
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
1309e2d50ee8118afe1138cd6677d84438edb55a )
Conflicts:
src/rgw/rgw_rados.cc
- no rgw_bl_str() in mimic
- mimic does not have
a4d3977cfacc5eb0de16618f2c957bb417505597
Casey Bodley [Fri, 25 Jan 2019 21:43:36 +0000 (16:43 -0500)]
rgw: add RGWRados::bi_get_olh() to read/decode olh entry
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
b8d7f0ad1f882b999e0fd7f3e45f8579599714cf )
Conflicts:
src/rgw/rgw_rados.cc
- bufferlist::cbegin()
Casey Bodley [Fri, 25 Jan 2019 21:42:50 +0000 (16:42 -0500)]
rgw: bi_get() takes bucket_info
and calls the bucket_info overload of BucketShard::init() to avoid
reading the bucket info again
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
a15e8736340c916120fef1f3a5ff1b982cf7c837 )
Casey Bodley [Fri, 25 Jan 2019 21:17:19 +0000 (16:17 -0500)]
rgw: add BucketShard::init overload for bucket_info,obj
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
c6c445f6074df113a28adcae89248b909d031f8e )
Casey Bodley [Wed, 2 Jan 2019 16:24:45 +0000 (11:24 -0500)]
test/rgw: add multisite test_version_suspended_incremental_sync
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
81fa354c1dadf4f957592f20fe9e02acd70671e0 )
Conflicts:
src/test/rgw/rgw_multi/tests.py
Casey Bodley [Fri, 4 Jan 2019 20:53:34 +0000 (15:53 -0500)]
rgw: fetch_remote_obj filters out olh attrs
each zone manages its own olh log, so we don't want any of the source
zone's olh attributes. in one case (sync of an object overwrite in a
versioning-suspended bucket), these olh attributes were being written
directly to the head object and leading to link_olh errors due to the
olh_tag mismatch
Fixes: http://tracker.ceph.com/issues/37792
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
a4195fc55fb76f0a2576d5ff0b2c99ad12fd7a49 )
Nathan Cutler [Mon, 22 Jul 2019 09:19:00 +0000 (11:19 +0200)]
Merge pull request #29020 from smithfarm/wip-39047-mimic
mimic: rgw: update resharding documentation
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Nathan Cutler [Mon, 22 Jul 2019 08:53:54 +0000 (10:53 +0200)]
Merge pull request #29014 from smithfarm/wip-37891-mimic
mimic: doc: wrong value of usage log default in logging section
Reviewed-by: Casey Bodley <cbodley@redhat.com>
J. Eric Ivancich [Mon, 1 Apr 2019 14:59:51 +0000 (10:59 -0400)]
rgw: fix race b/w bucket reshard and ops waiting on reshard completion
A previous commit (
f84f70d4 ) added functionality to clean up old
bucket instances and bucket shards once sharding completed
successfully. However the existing code that allowed a bucket
operation to wait until resharding completed was relying on a field in
the old bucket instance to know the bucket instance id of the new
bucket instance. This created a race condition as to whether the
clean-up or read of the bucket instance id would occur first.
This solution rereads the bucket entry point object when resharding
completes to determine the bucket instance id of the new bucket
thereby avoiding the race.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit
26b793680805721cac86f64cdd234b273bf33be4 )
Conflicts:
src/rgw/rgw_rados.cc: equivalent changes made to rgw_reshard.cc
J. Eric Ivancich [Wed, 27 Mar 2019 20:32:17 +0000 (16:32 -0400)]
rgw: correct comment describing function try_refresh_bucket_info
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit
d0894e6f6c062907796d10b524088c2abf7e4252 )
J. Eric Ivancich [Tue, 12 Mar 2019 19:38:54 +0000 (15:38 -0400)]
rgw: fix log messages using "proceeding" to "processing"
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit
1c2aa902cb2bec690387ace8eb6b8afb5678373f )
Volker Theile [Wed, 22 May 2019 09:30:50 +0000 (11:30 +0200)]
mgr/dashboard: Fix various RGW issues
* Fix handling of tenanted users
* Better exception handling if RGW backend is not available
* RGW user/bucket "max. size" should be hidden when "user/bucket quota" is not enabled
* Fix bug in loading user quota.
Fixes: https://tracker.ceph.com/issues/38800
partial manual backport of
61995970916cb11f66c3800bcf869755f8dae32c
partial manual backport of
e6f130d47023c24f8a9c742f145abd34d7320cd8
partial manual backport of
945e790cab57785e1bc2dbf708d5ddafec618aba
partial manual backport of
a98bca6a2f161140f78f34cfa5f5ebec07af2f82
partial manual backport of
975736aba546adb09ce4f38a1ae061621f7240cd
This fix is not cherry-picked because the code was completely re-worked for Nautilus (codebase is too far apart).
Signed-off-by: Volker Theile <vtheile@suse.com>
Sébastien Han [Fri, 1 Feb 2019 17:45:45 +0000 (18:45 +0100)]
ceph-volume: add space between words
The current help of batch was showing:
rotational drivesare passed in DEVICES
This commit fixes it and now we have:
rotational drives are passed in DEVICES
Tremendous fix, I know.
Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit
a151e111a2710e864bd66a44db38685459cf55c8 )
Andrew Schoen [Fri, 10 May 2019 14:02:08 +0000 (09:02 -0500)]
ceph-volume: use the Device.rotational property instead of sys_api
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=
1666822
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
8ebff475dfa136c88101ef7364820fe3c5cee547 )
(cherry picked from commit
aa6da416cb14ba521796c950fba9667e7b581b27 )
Andrew Schoen [Thu, 6 Dec 2018 18:37:56 +0000 (12:37 -0600)]
ceph-volume: set a lvm_size property on the fakedevice fixture
This is so tests can continue to set sys_api['size'] and the code
can retrieve that as Device.lvm_size
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
9e2175c4001434f7b0ed8ce6f723e063cf001440 )
Jan Fajerski [Sat, 26 Jan 2019 10:08:01 +0000 (11:08 +0100)]
ceph-volume: don't crash on unusable data devices
Also add test case for a MixedStrategy with unusable data devices.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Conflicts:
src/ceph-volume/ceph_volume/tests/devices/lvm/strategies/test_filestore.py
Pick test addition and error-value change
Jan Fajerski [Thu, 24 Jan 2019 17:10:45 +0000 (18:10 +0100)]
ceph-volume: add a wal device specific unit test and size validator
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Conflicts:
src/ceph-volume/ceph_volume/tests/devices/lvm/strategies/test_bluestore.py
Pick test addition and error.value change
Jan Fajerski [Wed, 23 Jan 2019 14:04:29 +0000 (15:04 +0100)]
ceph-volume: add osd_ids argument
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Jan Fajerski [Thu, 17 Jan 2019 14:13:05 +0000 (15:13 +0100)]
ceph-volume: strategies allow for external wal lv's
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Jan Fajerski [Mon, 7 Jan 2019 13:43:12 +0000 (14:43 +0100)]
ceph-volume: add default argument values to strategy
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
5465712e4d71315ff8daac9345be6ba96d1f44c4 )
Conflicts:
src/ceph-volume/ceph_volume/tests/devices/lvm/strategies/test_bluestore.py
src/ceph-volume/ceph_volume/tests/devices/lvm/strategies/test_filestore.py
pick with_auto_devices and error.value changes
Volker Theile [Thu, 29 Nov 2018 12:04:14 +0000 (13:04 +0100)]
Add a unit test that tests the ceph_volume.devices.lvm.batch.Batch::get_devices method.
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit
19a38189fd15f57907449631a0f6e9faf908dc66 )
Jan Fajerski [Fri, 4 Jan 2019 10:30:30 +0000 (11:30 +0100)]
ceph-volume strategies: rename db_devs to db_or_journal_devs
This aims at reducing confusion when reading the code that is shared
between the bluestore and filestore strategies.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
029414e89b773896fddebe0a5869b57ec13f527e )
Jan Fajerski [Thu, 13 Dec 2018 14:42:49 +0000 (15:42 +0100)]
ceph-volume batch: rename block_devs to data_devs for consistency
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
6dbf43e53ccd3ff06e3cdfc8e6de97475cc0c510 )
Jan Fajerski [Wed, 28 Nov 2018 11:20:20 +0000 (12:20 +0100)]
ceph-volume batch: refactor filtered_devices and strategy retrieval
Keep filtered_devices as a member variable of the batch object and pass
this to the report functionality instead of having the report functions
retrieve filtered_devices. Also keep selected strategy as a batch member
variable to avoid re-retrival in e.g. the report methods.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
dc2c4c597249d61ea6f48d37dbe02d8b97e3d55d )
Jan Fajerski [Tue, 27 Nov 2018 14:49:01 +0000 (15:49 +0100)]
ceph-volume: refactor strategies for explicit device arguments
This commit refactors lvm strategies to:
a) change the constructor to receive usage specific device lists
b) add classmethod with_auto_devices of automatic splitting by the
rotational flag (keep backwards compatible behavior)
c) rename ssds and hdds member variables to wal_devs and block_devs
d) add db_devs member variable
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
60dec8b1dfdd056662ba6a030de291f93b2f3650 )
Jan Fajerski [Fri, 21 Dec 2018 10:01:27 +0000 (11:01 +0100)]
ceph-volume: move arg parsing to __init__ and add explicit device lists
Moving the arg parsing to __init__ allows to get the help message with
running as root.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
a7717818ab3790b226067da3dee8f8c216ab8f4b )
Andrew Schoen [Thu, 6 Dec 2018 16:49:03 +0000 (10:49 -0600)]
ceph-volume: use Device.lvm_size in batch strategy validators
This will validate the devices on the size that lvm will allow to be
used, not the raw physical size of the device.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
aaa915cfe6678d5326b2c15730febfff79e40463 )
Andrew Schoen [Tue, 4 Dec 2018 20:23:47 +0000 (14:23 -0600)]
ceph-volume: batch mixed type scenarios have no need to calulate data size
We know with a mixed type scenario the device used for data will be used
at 100% capacity. This means we do not need to be explict when asking
for the size of the data lvs, which avoids rounding errors with very
small device sizes.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
0023961397d19659200eaf420f2789db17a71b34 )
Andrew Schoen [Fri, 30 Nov 2018 17:55:27 +0000 (11:55 -0600)]
ceph-volume: set use_large_block_db in validate, not compute
The self.use_large_block_db property was never getting set because
the block in compute was never called as block_db_size was reset in
validate if it was 0. We needed to set self.use_large_block_db in
validate instead of compute.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
fdfb79b91c52f82877d08190cb626ed00b45f3e8 )
Andrew Schoen [Thu, 6 Dec 2018 15:56:58 +0000 (09:56 -0600)]
ceph-volume: use Device.lvm_size in batch strategies
We should show the user what the size of the device will be after lvm
creates a pv out of it. This way there isn't a discrepency between the
sizes that are reported to the user and what is actually created.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
071e7ce41a58702c0eb84c35bd9388e2ec24ac13 )
Andrew Schoen [Tue, 27 Nov 2018 23:39:38 +0000 (17:39 -0600)]
ceph-volume: use extents instead of size when creating block.db lvs
In the batch command use extents instead of size when creating lvs. This
gives a more precise size and avoids rounding errors.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
add9f88dc800913adf387c90cbf4352e392b0a75 )
Conflicts:
src/ceph-volume/ceph_volume/devices/lvm/strategies/bluestore.py
remove json import; pick ours and add self.use_large_block_db = False
(cherry picked from commit
83e71842d8cd77b7d45efdd344bd8fac31078745 )
Yuri Weinstein [Thu, 18 Jul 2019 19:52:11 +0000 (12:52 -0700)]
Merge pull request #28206 from pdvian/wip-39475-mimic
mimic: common/util: handle long lines in /proc/cpuinfo
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Thu, 18 Jul 2019 19:51:39 +0000 (12:51 -0700)]
Merge pull request #28217 from smithfarm/wip-40005-mimic
mimic: build/ops: do_cmake.sh: source not found
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Yuri Weinstein [Thu, 18 Jul 2019 19:50:33 +0000 (12:50 -0700)]
Merge pull request #28741 from smithfarm/wip-40547-mimic
mimic: common: Keyrings created by ceph auth get are not suitable for ceph auth import
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Thu, 18 Jul 2019 19:49:29 +0000 (12:49 -0700)]
Merge pull request #28894 from ifed01/wip-ifed-fix-no-compress-mimic
mimic: os/bluestore: load OSD all compression settings unconditionally.
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Thu, 18 Jul 2019 19:49:00 +0000 (12:49 -0700)]
Merge pull request #28967 from liewegas/wip-set-aio-write-max-mimic
mimic: os/bluestore: fix >2GB bluefs writes
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Thu, 18 Jul 2019 19:47:58 +0000 (12:47 -0700)]
Merge pull request #28970 from neha-ojha/wip-40720-mimic
mimic: os/bluestore: default to bitmap allocator for bluestore/bluefs
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Thu, 18 Jul 2019 19:47:14 +0000 (12:47 -0700)]
Merge pull request #28995 from smithfarm/wip-38751-mimic
mimic: core: should report EINVAL in ErasureCode::parse() if m<=0
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 23:01:04 +0000 (16:01 -0700)]
Merge pull request #28220 from pdvian/wip-39513-mimic
mimic: osd: take heartbeat_lock when calling heartbeat()
Reviewed-by: David Zafman <dzafman@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 22:19:26 +0000 (15:19 -0700)]
Merge pull request #26499 from k0ste/mimic_backports2
mimic: mgr/balancer: restrict automatic balancing to specific weekdays
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Yuri Weinstein [Wed, 17 Jul 2019 22:18:24 +0000 (15:18 -0700)]
Merge pull request #27986 from pdvian/wip-39311-mimic
mimic: crushtool crash on Fedora 28 and newer
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 22:17:30 +0000 (15:17 -0700)]
Merge pull request #28089 from dzafman/wip-39720
mimic: osd: Include dups in copy_after() and copy_up_to()
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 22:17:00 +0000 (15:17 -0700)]
Merge pull request #28097 from pdvian/wip-39374-mimic
mimic: osd/OSD.cc: make osd bench description consistent with parameters
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 22:14:53 +0000 (15:14 -0700)]
Merge pull request #28142 from pdvian/wip-39422-mimic
mimic: osd/OSDMap: Replace get_out_osds with get_out_existing_osds
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 22:14:06 +0000 (15:14 -0700)]
Merge pull request #28165 from pdvian/wip-39458-mimic
mimic: mgr/prometheus: replace whitespaces in metrics' names
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Yuri Weinstein [Wed, 17 Jul 2019 22:13:21 +0000 (15:13 -0700)]
Merge pull request #28201 from dzafman/wip-38341
mimic: osd: pg stuck in backfill_wait with plenty of disk space
Reviewed-by: Neha Ojha <nojha@redhat.com>
Alfredo Deza [Wed, 17 Jul 2019 20:44:53 +0000 (16:44 -0400)]
Merge pull request #29074 from ceph/backport-mimic-26397
mimic: ceph-volume: update testing playbook 'deploy.yml'
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 19:46:02 +0000 (12:46 -0700)]
Merge pull request #27949 from smithfarm/wip-37613-mimic
mimic: rpm: missing dependency on python34-ceph-argparse from python34-cephfs (and others?)
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 19:45:14 +0000 (12:45 -0700)]
Merge pull request #28090 from pdvian/wip-39233-mimic
mimic: cephfs: mount.ceph.c: do not pass nofail to the kernel
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 19:44:45 +0000 (12:44 -0700)]
Merge pull request #28186 from pdvian/wip-39472-mimic
mimic: mds: fix SnapRealm::resolve_snapname for long name
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 19:44:23 +0000 (12:44 -0700)]
Merge pull request #28274 from pdvian/wip-39669-mimic
mimic: mds: output lock state in format dump
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 19:43:59 +0000 (12:43 -0700)]
Merge pull request #28337 from pdvian/wip-39679-mimic
mimic: cephfs: pybind: added lseek()
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Wed, 17 Jul 2019 19:43:35 +0000 (12:43 -0700)]
Merge pull request #28381 from pdvian/wip-39689-mimic
mimic: mds: check dir fragment to split dir if mkdir makes it oversized.
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>