]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Jianpeng Ma [Fri, 10 Jan 2020 06:33:55 +0000 (14:33 +0800)]
kv/RocksDBStore: make options compaction_threads/disableWAL/flusher_threads/compact_on_mount work.
This bug introduce by commit:
5f72c376deb64562e .
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit
ce3a1ca56df917bc3fa186d0e8c7b4256d2a37e9 )
Jenkins Build Slave User [Mon, 10 Aug 2020 20:15:22 +0000 (20:15 +0000)]
14.2.11
Yuri Weinstein [Fri, 7 Aug 2020 23:54:32 +0000 (16:54 -0700)]
Merge pull request #36526 from batrick/i46860-followup
qa: skip cache_size check
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Patrick Donnelly [Fri, 7 Aug 2020 21:51:09 +0000 (14:51 -0700)]
qa: skip cache_size check
Now that we're using the memory limit, this is check is not necessary.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Fri, 7 Aug 2020 23:25:04 +0000 (16:25 -0700)]
Merge pull request #36513 from batrick/i46860
nautilus: mds: do not raise "client failing to respond to cap release" when client working set is reasonable
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Yuri Weinstein [Fri, 7 Aug 2020 23:22:56 +0000 (16:22 -0700)]
Merge pull request #36512 from batrick/i46858
nautilus: qa: add debugging for volumes plugin use of libcephfs
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Fri, 7 Aug 2020 23:21:50 +0000 (16:21 -0700)]
Merge pull request #36511 from batrick/i46856
nautilus: client: static dirent for readdir is not thread-safe
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Yuri Weinstein [Fri, 7 Aug 2020 23:20:22 +0000 (16:20 -0700)]
Merge pull request #36462 from ukernel/nautilus-46831
nautilus: mds: initialize MDSlaveUpdate::waiter
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Fri, 7 Aug 2020 23:18:58 +0000 (16:18 -0700)]
Merge pull request #35272 from jan--f/wip-doc-allow_standby_replay-nautilus
nautilus: doc: obsolete entries for allow_standby_replay
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 7 Aug 2020 20:26:47 +0000 (13:26 -0700)]
qa: use real entity for config_get
So this changed in Octopus/Pacific to allow
ceph config get mds foo
but it doesn't work in Nautilus which expects a real entity:
https://pulpito.ceph.com/yuriw-2020-08-07_15:03:29-kcephfs-wip-yuri4-testing-2020-08-07-1350-nautilus-distro-basic-smithi/
5308647 /
2020-08-07T18:56:49.830 INFO:teuthology.orchestra.run.smithi106.stderr:2020-08-07 18:56:49.833
7fbe0d7fa700 1 -- 172.21.15.106:0/
369883181 <== mon.1 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=-22 unrecognized entity 'mds' v1) v1 ==== 131+0+0 (secure 0 0 0) 0x7fbdf8000940 con 0x7fbdf40064c0
2020-08-07T18:56:49.830 INFO:teuthology.orchestra.run.smithi106.stderr:Error EINVAL: unrecognized entity 'mds'
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Thu, 6 Aug 2020 17:57:20 +0000 (10:57 -0700)]
qa: add tests for mds_min_caps_working_set
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
f4fc138849cc821d1d327ac1c82206c14dac8028 )
Conflicts:
qa/tasks/cephfs/test_client_limits.py
Patrick Donnelly [Thu, 6 Aug 2020 16:04:23 +0000 (09:04 -0700)]
mds: add working set minimum for caps
A client may hold many inodes pinned in its cache for open files. That
client may be unable to release those caps to respond to cache pressure
from the MDS (or quiescent client cap recall). We should not complain if
that number of capabilities is reasonable (< 10k by default).
Fixes: https://tracker.ceph.com/issues/46830
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
63392e1b65fbead6ef8c7acd6a70e6ef5b322390 )
Patrick Donnelly [Thu, 6 Aug 2020 17:56:57 +0000 (10:56 -0700)]
qa: use config_set/config_get
It's simpler and does not require MDS restarts.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
edc5c14d1cb194760f91dcdb0385007df32aef45 )
Conflicts:
qa/tasks/cephfs/test_client_limits.py
Patrick Donnelly [Thu, 6 Aug 2020 17:55:21 +0000 (10:55 -0700)]
qa: do not append file names to dirname
Otherwise the files generated are not actually under the sub-directory!
This is correcting a confusing aspect of the test infrastructure but
doesn't actually require any changes to the tests.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
ac6c150eb07ddd61c33fc4fad034fee6f5fbad35 )
Patrick Donnelly [Thu, 6 Aug 2020 17:53:58 +0000 (10:53 -0700)]
qa: add exception for test timeouts
To make this easier to catch. It is still a RuntimeError so it should
not affect current tests by default.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
dbc44e5e342605c16e1213120bec0bfec1c0091d )
Patrick Donnelly [Wed, 5 Aug 2020 17:50:03 +0000 (10:50 -0700)]
qa: log entrance/exit into the volumes plugin
Fixes: https://tracker.ceph.com/issues/46851
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
a89b1633e1940d8bd35baaa46ce894fb46670cd5 )
Conflicts:
src/pybind/mgr/volumes/module.py
Patrick Donnelly [Wed, 5 Aug 2020 16:03:54 +0000 (09:03 -0700)]
qa: add debug client for volumes plugin qa
Fixes: https://tracker.ceph.com/issues/46851
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
9bcda7ca42d5ad0cc60a41812e046e1ce7057474 )
Patrick Donnelly [Thu, 6 Aug 2020 21:04:57 +0000 (14:04 -0700)]
client: use non-static dirent for thread-safety
Fixes: https://tracker.ceph.com/issues/46832
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
9b00083489d1d4777519da38f3003cf164dae480 )
Yan, Zheng [Wed, 5 Aug 2020 06:44:35 +0000 (14:44 +0800)]
mds: initialize MDSlaveUpdate::waiter
introduced by backport commit
4273de4cfaf
"mds: preserve ESlaveUpdate::OP_PREPARE logevent before doing commit"
Fixes: https://tracker.ceph.com/issues/46831
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Yuri Weinstein [Mon, 3 Aug 2020 18:42:19 +0000 (11:42 -0700)]
Merge pull request #36439 from batrick/i46533
nautilus: mds: fix nullptr dereference in MDCache::finish_rollback
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yan, Zheng [Tue, 14 Jul 2020 12:40:11 +0000 (20:40 +0800)]
mds: fix nullptr dereference in MDCache::finish_rollback
introduced by commit
4940ab62e0d
"mds: preserve ESlaveUpdate::OP_PREPARE logevent before doing commit"
Fixes: https://tracker.ceph.com/issues/46533
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
6715514aa577842942a1221a0f25924c53e03834 )
Yuri Weinstein [Fri, 31 Jul 2020 23:29:09 +0000 (16:29 -0700)]
Merge pull request #35733 from idryomov/wip-msgr21-nautilus
nautilus: New msgr2 crc and secure modes (msgr2.1)
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Yuri Weinstein [Fri, 31 Jul 2020 00:02:09 +0000 (17:02 -0700)]
Merge pull request #36377 from batrick/i46763
nautilus: qa/tasks: call super class's setUp()
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Thu, 30 Jul 2020 23:59:34 +0000 (16:59 -0700)]
Merge pull request #36339 from smithfarm/wip-46741-nautilus
nautilus: osd: fix crash in _committed_osd_maps if incremental osdmap crc fails
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Kefu Chai [Fri, 14 Feb 2020 12:31:25 +0000 (20:31 +0800)]
qa/tasks: call super class's setUp()
to address the regression introduced by
87292811215f6ded9a784d3216a910faaef648e2
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
7d262db1145a14a73da481acba66e51b5f30e4af )
Yuri Weinstein [Wed, 29 Jul 2020 16:23:13 +0000 (09:23 -0700)]
Merge pull request #36353 from neha-ojha/wip-46731-nautilus
nautilus: qa/*/nautilus-p2p-stress-split: rework objectstore backends
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Ramana Raja [Wed, 29 Jul 2020 14:23:46 +0000 (19:53 +0530)]
Merge pull request #36314 from kotreshhr/wip-46641-nautilus
nautilus: qa: Fix name collisions
Reviewed-by: Ramana Raja <rraja@redhat.com>
Neha Ojha [Tue, 28 Jul 2020 17:36:09 +0000 (10:36 -0700)]
qa/suites/rados/thrash/crc-failures: randomly inject bad incremental osdmap crc
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit
f9399b508c6584e539e323da858372fa7115be3b )
Dan van der Ster [Mon, 27 Jul 2020 15:40:27 +0000 (17:40 +0200)]
osd: don't write transaction when inc crc failed
80da5f9a987c6a48b93f25228fdac85890013520 exposed a flaw in how
handle_osd_map falls back to a full osdmap if the crc of an incremental
failed.
If the first message in a map message had a crc error, then the
loop would exit with last < start, which would then cause a null
dereference in _committed_osd_maps.
Fixes: https://tracker.ceph.com/issues/46443
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit
99463cb523949289384a6210ea250114fbe852f4 )
Dan van der Ster [Mon, 27 Jul 2020 12:23:54 +0000 (14:23 +0200)]
qa/standalone/osd: add bad-inc-map.sh
Test that the osd doesn't crash when it gets a bad incremental osdmap.
Related-to: https://tracker.ceph.com/issues/46443
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit
b550112dba23528655ec150cf9d6d093318abb8c )
Neha Ojha [Tue, 28 Jul 2020 21:08:27 +0000 (14:08 -0700)]
qa/*/nautilus-p2p-stress-split: rework objectstore backends
Earlier we were symlinking to qa/objectstore, which causes older versions of
nautilus to fail when trying to use hybrid allocator.
Fixes: https://tracker.ceph.com/issues/46731
Signed-off-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Tue, 28 Jul 2020 18:17:09 +0000 (11:17 -0700)]
Merge pull request #36250 from dillaman/wip-46673
nautilus: librbd: new 'write_zeroes' API methods to suppliment the `discard` APIs
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Tue, 28 Jul 2020 17:35:26 +0000 (10:35 -0700)]
Merge pull request #36292 from dzafman/wip-46706-nautilus
nautilus: osd: Cancel in-progress scrubs (not user requested)
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Tue, 28 Jul 2020 17:34:42 +0000 (10:34 -0700)]
Merge pull request #36190 from smithfarm/wip-46458-nautilus
nautilus: build/ops: selinux: allow ceph_t amqp_port_t:tcp_socket
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Tue, 28 Jul 2020 17:34:07 +0000 (10:34 -0700)]
Merge pull request #36161 from smithfarm/wip-46090-nautilus
nautilus: osd/PG: fix history.same_interval_since of merge target again
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Kotresh HR [Mon, 6 Jul 2020 16:54:03 +0000 (22:24 +0530)]
qa: Fix name collisions
Addresses the name collisions of volumes, subvolumes,
clones, subvolume groups and snapshots within the tests.
Fixes: https://tracker.ceph.com/issues/43517
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
3b6f6f1abf1beba54c7b4eb610ae05985cdf42b1 )
Conflicts:
qa/tasks/cephfs/test_volumes.py: Few of the test cases were not
present in nautilus which were present in master, removed them.
Ramana Raja [Mon, 27 Jul 2020 18:38:28 +0000 (00:08 +0530)]
Merge pull request #36167 from ajarr/nautilus-mgr-fs-volumes-misc
nautilus: mgr/fs/volumes misc fixes
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:35:08 +0000 (08:35 -0700)]
Merge pull request #36294 from ajarr/wip-46470
nautilus: client: release the client_lock before copying data in read
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:34:32 +0000 (08:34 -0700)]
Merge pull request #36215 from vshankar/wip-46388
nautilus: client: introduce timeout for client shutdown
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:33:57 +0000 (08:33 -0700)]
Merge pull request #36180 from ajarr/wip-46464-nautilus
nautilus: pybind/cephfs: fix custom exception raised by cephfs.pyx
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:33:08 +0000 (08:33 -0700)]
Merge pull request #36179 from smithfarm/wip-46523-nautilus
nautilus: mds: fix hang issue when accessing a file under a lost parent directory
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:32:43 +0000 (08:32 -0700)]
Merge pull request #36178 from smithfarm/wip-46521-nautilus
nautilus: mds: do not submit omap_rm_keys if the dir is the basedir of merge.
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:31:44 +0000 (08:31 -0700)]
Merge pull request #36177 from smithfarm/wip-46517-nautilus
nautilus: cephfs: client: fix directory inode can not call release callback
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:30:13 +0000 (08:30 -0700)]
Merge pull request #36175 from smithfarm/wip-46474-nautilus
nautilus: mds: make threshold for MDS_TRIM configurable
Yuri Weinstein [Mon, 27 Jul 2020 15:29:40 +0000 (08:29 -0700)]
Merge pull request #36173 from smithfarm/wip-46409-nautilus
nautilus: cephfs: client: fix setxattr for 0 size value (NULL value)
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:29:10 +0000 (08:29 -0700)]
Merge pull request #36172 from smithfarm/wip-46310-nautilus
nautilus: tests: qa/tasks/cephfs/cephfs_test_case.py: skip cleaning the core dumps when in program case
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:28:36 +0000 (08:28 -0700)]
Merge pull request #36171 from smithfarm/wip-46200-nautilus
nautilus: tests: Revert "Revert "qa/suites/rados/mgr/tasks/module_selftest: whitelist …
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:27:57 +0000 (08:27 -0700)]
Merge pull request #36170 from smithfarm/wip-46189-nautilus
nautilus: mds: reset heartbeat in EMetaBlob replay
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:27:16 +0000 (08:27 -0700)]
Merge pull request #36169 from smithfarm/wip-46187-nautilus
nautilus: cephfs: client: fix snap directory atime
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Mon, 27 Jul 2020 15:15:19 +0000 (08:15 -0700)]
Merge pull request #36270 from yuriw/wip-yuriw-nautilus-p2p-nautilus
nautilus: qa/tests: changed rbd-python.yaml workunit to v14.2.10
Reviewed-by: Neha Ojha <nojha@redhat.com>
Kotresh HR [Mon, 27 Jul 2020 10:57:18 +0000 (16:27 +0530)]
nautilus: qa/test_volumes: Modify tests not to use json for 'ceph fs status'
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Ramana Raja [Wed, 22 Jul 2020 10:26:36 +0000 (15:56 +0530)]
nautilus: mgr/volumes: use MgrModule's mon_command()
... instead of 'check_mon_command' which is not in nautilus, and not
compatible with python2 and early versions of py3.
Signed-off-by: Ramana Raja <rraja@redhat.com>
Ramana Raja [Mon, 20 Jul 2020 09:08:29 +0000 (14:38 +0530)]
nautilus: mgr/volumes: fix import typing error
... for python versions earlier than 3.5.
Signed-off-by: Ramana Raja <rraja@redhat.com>
Patrick Donnelly [Tue, 30 Jun 2020 15:33:59 +0000 (08:33 -0700)]
pybind/mgr/volumes: fix volume search for pools
Loop logic would bail out if it first sees any file system that does not
match the volume it's looking for.
Fixes: https://tracker.ceph.com/issues/46277
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
be74a81e19ef3b5e8204d52e6a3f7f0c57d49854 )
Kotresh HR [Tue, 23 Jun 2020 18:19:22 +0000 (23:49 +0530)]
mgr/volumes: Validate mon_allow_pool_delete before volume deletion
Volume deletion wasn't validating mon_allow_pool_delete config
before destroying volume metadata. Hence when mon_allow_pool_delete
is set to false, it was deleting metadata but failed to delete pool
resulting in inconsistent state. This patch validates the config
before going ahead with deletion.
Fixes: https://tracker.ceph.com/issues/45662
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
e770bb9075d931913847a572d121e02a2e349ca8 )
Patrick Donnelly [Mon, 10 Feb 2020 18:46:09 +0000 (10:46 -0800)]
qa: manage config changes through mons
This provides a generic framework for modifying Ceph configuration
changes in tests through the monitors rather than the asok interface or
local ceph.conf changes. Any changes are reverted during test teardown.
A future patch will convert existing tests manipulating the local
ceph.conf or admin socket.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
87292811215f6ded9a784d3216a910faaef648e2 )
Kotresh HR [Fri, 5 Jun 2020 17:58:36 +0000 (23:28 +0530)]
mgr/volumes: Fix pool removal on volume deletion
While volume deletion, the associated pools are not always
removed. The pools are removed only if the volume is created
using mgr plugin and not if created with custom osd pools.
This is because mgr plugin generates pool names with specific
pattern. Both create and delete volume relies on it. This
patch fixes the issue by identifying the pools of the volume
without relying on the pattern.
Fixes: https://tracker.ceph.com/issues/45910
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
d07ea8db471b1a1d9082756e1cb775d5543b1307 )
Conflicts:
src/pybind/mgr/volumes/fs/operations/volume.py:
- In nautilus, fs volume create doesn't have placement arg
src/pybind/mgr/volumes/fs/volume.py:
- In nautilus, VolumeClient code not moved to mgr_util.py
00111048 [Fri, 5 Jun 2020 06:46:06 +0000 (14:46 +0800)]
client: release the client_lock before copying data in read
Fixes: https://tracker.ceph.com/issues/46025
Signed-off-by: Chencan <chen.can2@zte.com.cn>
(cherry picked from commit
db292da7d5571ed3addedfb4c5ab8c61fe9bb5e8 )
Conflicts:
src/client/Client.cc
- in nautilus, not switched bufferlist::copy() to iterators
Igor Fedotov [Sat, 25 Jul 2020 19:46:21 +0000 (22:46 +0300)]
Merge pull request #35500 from ifed01/wip-ifed-hybrid-alloc-nau
nautilus: os/bluestore: implement Hybrid allocator
Reviewed-by: akupczyk@redhat.com
Igor Fedotov [Mon, 20 Apr 2020 11:31:50 +0000 (14:31 +0300)]
qa/objectstore: test hybrid allocator instead of avl one.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
0f639740a3b99d0600f6508d90d426ca12501a03 )
Igor Fedotov [Thu, 16 Apr 2020 15:06:04 +0000 (18:06 +0300)]
os/bluestore: set hybrid allocator as a default for bluefs/bluestore
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
cacc7391dca779c7e6e3b57e6d1753a53b8c49b9 )
Conflicts:
src/test/objectstore/store_test.cc
trivial
Igor Fedotov [Wed, 25 Mar 2020 14:19:12 +0000 (17:19 +0300)]
os/bluestore: improve adjustent extents merge in hybrid allocatoc
Bitmap counterpart is inspected for adjustent free extents when
releasing extent and trying to add it to AVL tree.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
c27490e3d641f2b8ea65759fd79640e5f6dd398c )
Igor Fedotov [Mon, 23 Mar 2020 12:43:09 +0000 (15:43 +0300)]
os/bluestore: remove HybridAllocator::init_add_free method.
It's a 100% replica of AvlAllocator one which is inherited.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
cccbc62528e03f4e49882137342adfea20b9a0f2 )
Igor Fedotov [Fri, 14 Feb 2020 14:26:17 +0000 (17:26 +0300)]
os/bluestore: configure hybrid allocator threshold to fallback.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
b682d9a8e8d3f41d55ad10d2fb3deca8edf8540a )
Conflicts:
src/common/options.cc
trivial
Igor Fedotov [Thu, 13 Feb 2020 22:04:39 +0000 (01:04 +0300)]
os/bluestore: add hybrid (avl+bitmap) allocator
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
c90eee34955734a5d4d2d4f0d1ff04b275374fb1 )
Conflicts:
src/os/CMakeLists.txt
src/test/objectstore/Allocator_bench.cc
src/test/objectstore/Allocator_test.cc
src/test/objectstore/CMakeLists.txt
GTEST_SKIP not available, +trivial
Igor Fedotov [Thu, 13 Feb 2020 22:06:46 +0000 (01:06 +0300)]
os/bluestore: do not require min_length to be power of two for bmap
allocator.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
21532045efeb8d357518686df84a529ee3d32a60 )
Conflicts:
src/os/bluestore/fastbmap_allocator_impl.h
variable rename
Igor Fedotov [Thu, 13 Feb 2020 21:57:29 +0000 (00:57 +0300)]
os/bluestore: extend AvlAllocator interface.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
cccf94da463aa15370fe674bda24d168d331ceca )
Igor Fedotov [Thu, 13 Feb 2020 21:56:07 +0000 (00:56 +0300)]
tests/test_bluestore_types: show AvlAllocator's range_seg_t size.
Extend corresponding test case to be aware of the struct's sizeof.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
719cd7d9423d1d16e3005cc0b807699ed24533de )
Igor Fedotov [Wed, 12 Feb 2020 13:17:40 +0000 (16:17 +0300)]
os/bluestore: make AvlAllocator overridable.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
59c0aa5eb40957364ddb0e9d62dffa3449991249 )
Igor Fedotov [Wed, 12 Feb 2020 12:55:03 +0000 (15:55 +0300)]
os/bluestore: intdoduce spillover mechanics to AvlAllocator.
This to be utilized in AvlAllocator's descendant class(es) to be able
to limit amount of entries allocator tracks. Hence bringing a cap for
RAM used by the allocator.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
efedcc077c7ed4e1e5ea26ae989fa3d6cbd287d4 )
Igor Fedotov [Mon, 19 Aug 2019 13:01:21 +0000 (16:01 +0300)]
os/bluestore: avoid constant name replica in bitmap allocator.
This was introduced by Adam's https://github.com/ceph/ceph/pull/29425
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
683f7bab5a571cc3a6c7106fdb1d4017cc843c5d )
Igor Fedotov [Mon, 19 Aug 2019 12:51:51 +0000 (15:51 +0300)]
os/bluestore: implement original allocator fragmentation inspection.
Using admin socket interface: bluestore allocator fragmentation
[block|bluefs-db|bluefs-wal|bluefs-slow]
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
11b5feb01d273365c32ec6419faead629fb8c20b )
Igor Fedotov [Mon, 19 Aug 2019 12:19:25 +0000 (15:19 +0300)]
os/bluestore: simplify Allocator::get_fragmentation() signature
Remove alloc_unit parameter.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
3a96db924968b65afed5fc1355acbf185455c5c4 )
Igor Fedotov [Tue, 9 Jul 2019 18:19:37 +0000 (21:19 +0300)]
os/bluestore: StupidAllocator stucks on 4GB allocations
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
1a91a67bf2fab64d27a000b3c62d3d7beac1aa75 )
Conflicts:
src/test/objectstore/Allocator_test.cc
Previous gtest version
Igor Fedotov [Mon, 19 Aug 2019 11:39:31 +0000 (14:39 +0300)]
test/allocator_test: additional coverage for get_fragmentation_score
Tests the function results agains the same cases as we do for
get_fragmentation.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
6ae213181433dd2ae392d8eed85c2606b9c4bb54 )
Adam Kupczyk [Thu, 12 Mar 2020 12:29:03 +0000 (13:29 +0100)]
os/bluestore: Adds shutdown in destructor of AvlAllocator
Adds shutdown. Fixes incorrect mempool accounting.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
ea95309f478428b6e4b3af93b19459f6b5cc162d )
Conflicts:
src/test/objectstore/Allocator_bench.cc
trivial
xie xingguo [Mon, 9 Oct 2017 12:31:57 +0000 (20:31 +0800)]
qa: add AvlAllocator test script
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit
29512a70d45628ff4efcd4fd45b5d689b21b01d8 )
xie xingguo [Sun, 8 Oct 2017 07:52:34 +0000 (15:52 +0800)]
src/test/objectstore/Allocator_{test,bench}.cc: test AvlAllocator too
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit
97dfbc9c4816027d766731f0facf1ffa03455491 )
Conflicts:
src/test/objectstore/Allocator_bench.cc
src/test/objectstore/Allocator_test.cc
GTEST_SKIP not present, +trivial
xie xingguo [Sat, 7 Oct 2017 05:18:31 +0000 (13:18 +0800)]
os/bluestore: implement avl-extent-based allocator
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
adbc79a8ac28115c692a9fe6d4f6c098a7c7a5ea )
Conflicts:
src/common/options.cc
non-backported options
Yuri Weinstein [Fri, 24 Jul 2020 22:04:26 +0000 (15:04 -0700)]
Merge pull request #36290 from neha-ojha/wip-fio-nautilus
nautilus: qa/suites/rados/perf: run on ubuntu only
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Neha Ojha [Fri, 24 Jul 2020 21:53:32 +0000 (21:53 +0000)]
qa/suites/rados/perf: run on ubuntu only
This change is specific to nautilus since fio tests have been failing
on centos and rhel 7.8 due to dependency issues.
Signed-off-by: Neha Ojha <nojha@redhat.com>
David Zafman [Tue, 7 Jul 2020 01:02:08 +0000 (18:02 -0700)]
test: Check for interuption of scrubs with nosrub/nodeep_scrub
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
365e48d6ec71cec7c85acbf918927a20856d55da )
Conflicts:
qa/standalone/scrub/osd-scrub-test.sh (The scrub/deep_scrub trigger
doesn't use ceph tell)
Yuri Weinstein [Fri, 24 Jul 2020 19:12:09 +0000 (12:12 -0700)]
Merge pull request #36074 from neha-ojha/wip-46184-nautilus
nautilus: mgr/DaemonServer.cc: make 'config show' on fsid work
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Fri, 24 Jul 2020 19:11:40 +0000 (12:11 -0700)]
Merge pull request #35713 from dzafman/wip-46116
nautilus: tools: Add statfs operation to ceph-objecstore-tool
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Fri, 24 Jul 2020 19:09:04 +0000 (12:09 -0700)]
Merge pull request #35464 from bk201/wip-45837-nautilus
nautilus: monitoring: fixing some issues in RBD detail dashboard
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Yuri Weinstein [Fri, 24 Jul 2020 19:08:21 +0000 (12:08 -0700)]
Merge pull request #35237 from smithfarm/wip-45677-nautilus
nautilus: qa/workunits: Remove redundant Xenial cmake3 requirements
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Yuri Weinstein [Fri, 24 Jul 2020 19:06:20 +0000 (12:06 -0700)]
Merge pull request #35841 from Vicente-Cheng/wip-46191-nautilus
nautilus: mds: fix filelock state when Fc is issued
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
David Zafman [Thu, 2 Jul 2020 17:05:57 +0000 (10:05 -0700)]
osd: Cancel in-progress scrubs (not user requested)
This change adds new scrubber.req_scrub to track user
requested scrubs, deep_scrub or repair.
Fixes: https://tracker.ceph.com/issues/46275
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
33749cc3c39131d7abed9b8c14064dbfaa87f3a2 )
Conflicts:
PendingReleaseNotes (trivial)
src/osd/PG.cc (Due to code re-arrangement changes add manually)
David Zafman [Thu, 23 Jul 2020 16:40:54 +0000 (09:40 -0700)]
osd: Arrange code so that it is clearer should not cause any change
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
ef9ab850d87a84ce42c2c8df1c9b502b2fb04989 )
David Zafman [Tue, 21 Jul 2020 20:58:42 +0000 (13:58 -0700)]
test: mon-last-epoch-clean.sh fixed to avoid shell globbing
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
f272768802648f2cb4d519c4c790ab219ad43f9f )
Yuri Weinstein [Thu, 23 Jul 2020 17:16:51 +0000 (10:16 -0700)]
qa/tests: changed rbd-python.yaml workunit to v14.2.10
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Thomas Serlin [Fri, 10 Jul 2020 05:36:28 +0000 (01:36 -0400)]
selinux: add amqp and soundd types to ceph.te
Signed-off-by: Thomas Serlin <tserlin@redhat.com>
(cherry picked from commit
c3c5129748d2580ec95ef11609b124eef4dc29ff )
Conflicts:
selinux/ceph.te
kalebskeithley [Wed, 8 Jul 2020 19:20:30 +0000 (15:20 -0400)]
selinux: allow ceph_t amqp_port_t:tcp_socket
allow ceph_t amqp_port_t:tcp_socket name_connect;
allow ceph_t soundd_port_t:tcp_socket name_connect;
Required for running RabbitMQ
(soundd_port_t) for running RabbitMQ on port 8000
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1854083
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
(cherry picked from commit
05c523185b2d5ddd9e10f425c7e1f1ee1e409ba2 )
Yuri Weinstein [Wed, 22 Jul 2020 20:07:20 +0000 (13:07 -0700)]
Merge pull request #36186 from smithfarm/wip-45930-nautilus
nautilus: rgw: Add support wildcard subuser for bucket policy
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Yuri Weinstein [Wed, 22 Jul 2020 20:06:50 +0000 (13:06 -0700)]
Merge pull request #36163 from ivancich/nautilus-bucket-listing-clear-results
nautilus: rgw: cls_bucket_list_(un)ordered should clear results collection
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Jason Dillaman [Tue, 30 Jun 2020 12:47:48 +0000 (08:47 -0400)]
rbd: use new 'write_zeroes' API to ensure zeroed extents will be zeroed
Fixes: https://tracker.ceph.com/issues/45986
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
f319ecfbec3486917824eab84ab76347319b8bde )
(cherry picked from commit
f53e43c6b1c520733ffd2c506a958fb7c4a45f9c )
Jason Dillaman [Tue, 30 Jun 2020 15:20:31 +0000 (11:20 -0400)]
pybind/rbd: add new write_zeroes/aio_write_zeroes API methods
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
c27d744a339d3644b756ee496445909f63c8903e )
(cherry picked from commit
6a884184b680b89e2b46ad001651e07e58514eb0 )
Jason Dillaman [Tue, 30 Jun 2020 12:44:38 +0000 (08:44 -0400)]
librbd: add 'write_zeroes' public C/C++ API methods
Unlike the existing 'discard' option which is more of a hint to
attempt to release space, the new 'write_zeroes' APIs will ensure
that the entire provided extent is fully zeroed.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
ae6dd86b22e928dc23a385faf41cf76b0d293576 )
Conflicts:
src/librbd/api/Io.h/cc: logic exists in ImageRequestWQ files
src/librbd/librbd.cc: trivial resolution due missing api::Io
src/test/librbd/test_librbd.cc: trivial resolution
(cherry picked from commit
0e3add97ff1f919d81a1c644b150672cbd691c84 )
Conflicts:
src/librbd/io/ImageRequestWQ.cc: different locks and no overlap guard
src/test/librbd/test_librbd.cc: trivial resolution
Yuri Weinstein [Wed, 22 Jul 2020 17:37:02 +0000 (10:37 -0700)]
Merge pull request #35601 from smithfarm/wip-46012-nautilus
nautilus: qa/vstart_runner: update vstart_runner.LocalRemote.sh
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Yuri Weinstein [Wed, 22 Jul 2020 17:34:45 +0000 (10:34 -0700)]
Merge pull request #35401 from smithfarm/wip-45854-nautilus
nautilus: tools/cephfs: don't bind to public_addr
Reviewed-by: Ramana Raja <rraja@redhat.com>