]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Paul Cuzner [Thu, 8 Oct 2020 03:30:56 +0000 (16:30 +1300)]
mgr/prometheus: Add healthcheck metric for SLOW_OPS
SLOW_OPS is triggered by op tracker, and generates a health
alert but healthchecks do not create metrics for prometheus to
use as alert triggers. This change adds SLOW_OPS metric, and
provides a simple means to extend to other relevant health
checks in the future
If the extract of the value from the health check message fails
we log an error and remove the metric from the metric set. In
addition the metric description has changed to better reflect
the scenarios where SLOW_OPS can be triggered.
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit
2010432b5045c8f1dd0c052def497e590a1bf2fe )
Conflicts:
src/pybind/mgr/prometheus/module.py
- Nautilus doesn't have the profile_method decorator, so needed to be removed
Yuri Weinstein [Fri, 26 Feb 2021 17:28:26 +0000 (09:28 -0800)]
Merge pull request #39441 from rhcs-dashboard/49278-py2-unicode-password
nautilus: mgr/dashboard: python 2: error when setting non-ASCII password
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Yuri Weinstein [Fri, 26 Feb 2021 01:11:01 +0000 (17:11 -0800)]
Merge pull request #39129 from smithfarm/wip-48376-nautilus
nautilus: cephfs: client: check rdonly file handle on truncate
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Fri, 26 Feb 2021 01:09:56 +0000 (17:09 -0800)]
Merge pull request #37906 from ideepika/backport-
a7f18e4
nautilus: qa/tasks/{ceph,ceph_manager}: drop py2 support
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Yuri Weinstein [Thu, 25 Feb 2021 19:40:27 +0000 (11:40 -0800)]
Merge pull request #39001 from Vicente-Cheng/wip-48520-nautilus
nautilus: client: add ceph.{cluster_fsid/client_id} vxattrs suppport
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Thu, 25 Feb 2021 18:07:54 +0000 (10:07 -0800)]
Merge pull request #39544 from aaSharma14/wip-48733-nautilus
nautilus: mgr/dashboard: minimize console log traces of Ceph backend API tests
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Yuri Weinstein [Thu, 25 Feb 2021 18:07:25 +0000 (10:07 -0800)]
Merge pull request #39518 from aaSharma14/wip-49323-nautilus
nautilus: mgr/dashboard: fix MTU Mismatch alert
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Yuri Weinstein [Thu, 25 Feb 2021 18:06:21 +0000 (10:06 -0800)]
Merge pull request #38885 from yanghonggang/wip-48857-nautilus
nautilus: rgw: multisite: Verify if the synced object is identical to source
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Alfonso Martínez [Thu, 25 Feb 2021 14:55:06 +0000 (15:55 +0100)]
nautilus: mgr/dashboard: python 2: error when setting user's non-ASCII password
Fixes: https://tracker.ceph.com/issues/49278
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Ramana Raja [Thu, 25 Feb 2021 14:26:17 +0000 (09:26 -0500)]
Merge pull request #39658 from kotreshhr/ceph-vol-client-update-fix-volumes-key-nautilus
nautilus: pybind/ceph_volume_client: Update the 'volumes' key to 'subvolumes' in auth-metadata file
Reviewed-by: Ramana Raja <rraja@redhat.com>
Kotresh HR [Fri, 19 Feb 2021 11:27:23 +0000 (16:57 +0530)]
mgr/volumes: Bump up AuthMetadataManager's version
With ceph_volume_client and mgr-volumes co-existing
for sometime, the version of both needs to be same.
The ceph_volume_client version <=5 can't decode
'subvolumes' key in auth-metadata file. Hence to
handle version in-compatibility, the version of
ceph_volume_client is bumped up to 6 and the same
needs to be done in mgr-volume's AuthMetadataManager
Fixes: https://tracker.ceph.com/issues/49374
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
818c7781ff0467c6521bf2b3002b094fb8a71257 )
Kotresh HR [Fri, 19 Feb 2021 11:12:33 +0000 (16:42 +0530)]
pybind/ceph_volume_client: Bump up the version and compat_version to 6
With 'volumes' key updated to 'subvolumes', the version of
ceph_volume_client <= 5 can't decode auth-metadata file. Hence
bumping up ceph_volume_client version and compat_version to 6.
Fixes: https://tracker.ceph.com/issues/49294
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
ce55a0bcdc5db139fc8f39665c372f1b7cc274a7 )
Kotresh HR [Fri, 19 Feb 2021 11:08:31 +0000 (16:38 +0530)]
pybind/ceph_volume_client: Fix auth-metadata file recovery
Fixes: https://tracker.ceph.com/issues/49294
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
99cdfdad59175a19bc06859e69301ca0478a2db4 )
Kotresh HR [Mon, 15 Feb 2021 16:26:51 +0000 (21:56 +0530)]
pybind/ceph_volume_client: Update the 'volumes' key to 'subvolumes' in auth metadata file
The older auth metadata files before nautilus release stores
the authorized subvolumes using the 'volumes' key. As the
notion of 'subvolumes' brought in by mgr/volumes, it makes
sense to use 'subvolumes' key. This patch would be tranparently
update 'volumes' key to 'subvolumes' and newer auth metadata
files would store them with 'subvolumes' key.
Also fails the deauthorize if the auth-id doesn't exist.
Fixes: https://tracker.ceph.com/issues/49294
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
dee03c8d5c0b86cf51865090bec203419a3008a9 )
Michael Fritch [Wed, 13 May 2020 23:14:35 +0000 (17:14 -0600)]
pybind/ceph_volume_client: logger.warn() -> logger.warning()
logger.warn() is the same as logger.warning(), but was removed in py3.3
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit
fe4c7fd7471bf724c4a1f027580d9bae3ddc24eb )
Alfonso Martínez [Tue, 23 Feb 2021 13:01:14 +0000 (14:01 +0100)]
Merge pull request #39626 from rhcs-dashboard/wip-49420-nautilus
nautilus: mgr/dashboard: set security headers
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Deepika Upadhyay [Tue, 8 Dec 2020 14:34:13 +0000 (20:04 +0530)]
qa/tasks/ceph_manager: use s/ByteIO/StringIO in stdout for ceph-objectstore-tool
wrt master, we have moved to using run_ceph_objectstore_tool which uses
StringIO for stdout and stderr, to make the changes compatible with
nautilus, replacing use of ByteIO with StringIO.
see also:
d8d44ed
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
Sage Weil [Tue, 23 Feb 2021 03:26:30 +0000 (21:26 -0600)]
Merge pull request #39584 from liewegas/pr-39561-nautilus
nautilus: qa: use normal build for valgrind
Sage Weil [Tue, 23 Feb 2021 00:48:27 +0000 (18:48 -0600)]
qa/valgrind.supp: ignore valgrind leak
<error>
<unique>0x12ee80e</unique>
<tid>1</tid>
<threadname>ceph-mon</threadname>
<kind>Leak_StillReachable</kind>
<xwhat>
<text>8 bytes in 1 blocks are still reachable in loss record 2 of 31</text>
<leakedbytes>8</leakedbytes>
<leakedblocks>1</leakedblocks>
</xwhat>
<stack>
<frame>
<ip>0x4C2AC38</ip>
<obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
<fn>operator new[](unsigned long)</fn>
<dir>/builddir/build/BUILD/valgrind-3.15.0/coregrind/m_replacemalloc</dir>
<file>vg_replace_malloc.c</file>
<line>433</line>
</frame>
<frame>
<ip>0xEB02D40</ip>
<obj>/usr/lib64/libtcmalloc.so.4.4.5</obj>
</frame>
<frame>
<ip>0x400F9B2</ip>
<obj>/usr/lib64/ld-2.17.so</obj>
<fn>_dl_init</fn>
</frame>
<frame>
<ip>0x4001179</ip>
<obj>/usr/lib64/ld-2.17.so</obj>
</frame>
<frame>
<ip>0x5</ip>
</frame>
<frame>
<ip>0x1FFF000BBE</ip>
</frame>
<frame>
<ip>0x1FFF000BC7</ip>
</frame>
<frame>
<ip>0x1FFF000BCA</ip>
</frame>
<frame>
<ip>0x1FFF000BD4</ip>
</frame>
<frame>
<ip>0x1FFF000BD9</ip>
</frame>
<frame>
<ip>0x1FFF000BDC</ip>
</frame>
</stack>
</error>
Signed-off-by: Sage Weil <sage@newdream.net>
Avan Thakkar [Wed, 10 Feb 2021 22:12:49 +0000 (03:42 +0530)]
mgr/dashboard: set security headers
Fixes: https://tracker.ceph.com/issues/49243
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit
87bb3777877580f70018f4880488e264decc16e4 )
Conflicts:
qa/tasks/mgr/dashboard/test_requests.py
src/pybind/mgr/dashboard/services/auth.py
- Adopting the master branch changes.
Yang Honggang [Fri, 19 Feb 2021 05:56:13 +0000 (05:56 +0000)]
rgw multisite: fix single-part-MPU object etag misidentify problem
The single-part-MPU object is treated as a Atomic object.
Because its part_size is 0.
As MPU object's start_part_num is not 0 which can be used to
distinguish MPU object and Atomic object.
Fixes: https://tracker.ceph.com/issues/49357
Signed-off-by: Yang Honggang <yanghonggang@kuaishou.com>
(cherry picked from commit
63f989ca2bdc43142713b5da1eee2668ba238716 )
Sage Weil [Thu, 18 Feb 2021 18:04:16 +0000 (12:04 -0600)]
qa/valgrind.supp: suppress free[] warnings
This is apparently not going to get fixed any time soon.
https://github.com/gperftools/gperftools/issues/792
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
28fe149ea276693b47cc4d600780971f723672e4 )
Sage Weil [Fri, 19 Feb 2021 19:48:32 +0000 (13:48 -0600)]
qa/suites: do not use notcmalloc flavor
teuthology now knows how to run valgrind against a tcmalloc binary
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
dc64ccf063f1ddf843fae4d2621ec25187f56135 )
# Conflicts:
# qa/suites/rados/valgrind-leaks/1-start.yaml
# qa/suites/rgw/multisite/valgrind.yaml
# qa/suites/rgw/sts/tasks/0-install.yaml
# qa/suites/rgw/verify/validater/valgrind.yaml
# qa/suites/rgw/website/overrides.yaml
Aashish Sharma [Tue, 14 Jul 2020 04:58:57 +0000 (10:28 +0530)]
mgr/dashboard:minimize console log traces of Dashboard backend API tests
Changed log levels in some files to reduce console log traces
Fixes: https://tracker.ceph.com/issues/45822
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit
f5dd0195cd45b6e00e43ef97e209e565e6452777 )
Yuri Weinstein [Wed, 17 Feb 2021 16:30:40 +0000 (08:30 -0800)]
Merge pull request #39438 from rhcs-dashboard/wip-49271-nautilus
nautilus: mgr/dashboard: delete EOF when reading passwords from file
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Yuri Weinstein [Wed, 17 Feb 2021 16:29:26 +0000 (08:29 -0800)]
Merge pull request #39197 from ybwang0211/CrushWrapper
nautilus: crush/CrushWrapper: rebuild reverse maps after rebuilding crush map
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Wed, 17 Feb 2021 16:28:50 +0000 (08:28 -0800)]
Merge pull request #39160 from smithfarm/wip-48495-nautilus
nautilus: mon: paxos: Delete logger in destructor
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Aashish Sharma [Mon, 15 Feb 2021 04:43:39 +0000 (10:13 +0530)]
mgr/dashboard:fix MTU Mismatch alert
This PR intends to fix the expression used for MTU Mismatch alert in prometheus
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit
8527489b9148a8845d6fccca6461b23ed7989172 )
Patrick Donnelly [Wed, 11 Nov 2020 19:37:09 +0000 (11:37 -0800)]
client: check rdonly file handle on truncate
Fixes: https://tracker.ceph.com/issues/48202
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
3bb209772eada4ba5010be521f7ecaae567598d0 )
Patrick Donnelly [Wed, 11 Nov 2020 19:29:12 +0000 (11:29 -0800)]
test/libcephfs: test truncate on rdonly fd
Fixes: https://tracker.ceph.com/issues/48202
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
2b41ddf343e8e314dc60fae0d5f4e007ce3857e4 )
Conflicts:
src/test/libcephfs/test.cc
- rewrite a trivial string + int concatenation to eliminate fmt::format
Xiubo Li [Tue, 24 Nov 2020 08:49:06 +0000 (16:49 +0800)]
client: add ceph.{cluster_fsid/client_id} vxattrs suppport
These two vxattrs will only exist in local client side, with which
we can easily know which mountpoint the file belongs to and also
they can help locate the debugfs path quickly.
Fixes: https://tracker.ceph.com/issues/48337
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit
5635f25dd4a134f2fb32d2cb9dd73c0e947295e6 )
Conflicts:
src/client/Client.cc
- add .hidden member because we still need it
in nautilus
src/client/Client.h
- drop the mirror.info xattr related code
because nautilus does not introduce it
Yuri Weinstein [Wed, 17 Feb 2021 00:06:44 +0000 (16:06 -0800)]
Merge pull request #39134 from smithfarm/wip-48879-nautilus
nautilus: mds: update defaults for recall configs
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Wed, 17 Feb 2021 00:06:18 +0000 (16:06 -0800)]
Merge pull request #39133 from smithfarm/wip-48837-nautilus
nautilus: cephfs: mount.ceph: collect v2 addresses for non-legacy ms_mode options
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Wed, 17 Feb 2021 00:04:47 +0000 (16:04 -0800)]
Merge pull request #39130 from smithfarm/wip-48814-nautilus
nautilus: mds: avoid spurious sleeps
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Wed, 17 Feb 2021 00:04:06 +0000 (16:04 -0800)]
Merge pull request #39128 from smithfarm/wip-48371-nautilus
nautilus: mds: dir->mark_new() should together with dir->mark_dirty()
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Wed, 17 Feb 2021 00:03:34 +0000 (16:03 -0800)]
Merge pull request #39127 from smithfarm/wip-48130-nautilus
nautilus: cephfs: release client dentry_lease before send caps release to mds
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Wed, 17 Feb 2021 00:03:12 +0000 (16:03 -0800)]
Merge pull request #38690 from batrick/
4ca8aaafa32bb7397eec80a8eb366d2dd781a72f -nautilus
nautilus: qa: unmount volumes before removal
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Yuri Weinstein [Wed, 17 Feb 2021 00:02:41 +0000 (16:02 -0800)]
Merge pull request #38552 from batrick/i48374
nautilus: cephfs: client: dump which fs is used by client for multiple-fs
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Ernesto Puerta [Tue, 16 Feb 2021 19:40:53 +0000 (20:40 +0100)]
Merge pull request #39117 from rhcs-dashboard/wip-48961-nautilus
nautilus: mgr/dashboard: Fix for incorrect validation in rgw user form
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Ernesto Puerta [Tue, 16 Feb 2021 19:40:20 +0000 (20:40 +0100)]
Merge pull request #39104 from k0ste/wip-49023-nautilus
nautilus: mgr/dashboard: trigger alert if some nodes have a MTU different than the median value
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Yuri Weinstein [Mon, 15 Feb 2021 16:39:44 +0000 (08:39 -0800)]
Merge pull request #39184 from smithfarm/wip-49067-nautilus
nautilus: build/ops: install-deps.sh,deb,rpm: move python-saml deps into debian/control and ceph.spec.in
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Yuri Weinstein [Mon, 15 Feb 2021 16:39:03 +0000 (08:39 -0800)]
Merge pull request #39136 from p-se/wip-48608-nautilus
nautilus: mgr/dashboard: enable different URL for users of browser to Grafana
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Yuri Weinstein [Mon, 15 Feb 2021 16:38:29 +0000 (08:38 -0800)]
Merge pull request #38839 from aaSharma14/wip-48738-nautilus
nautilus: mgr/dashboard: Use secure cookies to store JWT Token
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Jan Fajerski [Mon, 15 Feb 2021 10:18:21 +0000 (11:18 +0100)]
Merge PR #39270 into nautilus
* refs/pull/39270/head:
ceph-volume: mock VGs with string properties in tests
ceph-volume: add some flexibility to bytes_to_extents
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Jan Fajerski [Mon, 15 Feb 2021 09:10:49 +0000 (10:10 +0100)]
Merge PR #39221 into nautilus
* refs/pull/39221/head:
ceph-volume: Fix usage of is_lv
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Ramana Raja [Fri, 12 Feb 2021 22:40:21 +0000 (17:40 -0500)]
Merge pull request #39292 from kotreshhr/wip-mgr-backports-nautilus
nautilus: mgr/volume: subvolume auth_id management and few bug fixes
Reviewed-by: Ramana Raja <rraja@redhat.com>
Alfonso Martínez [Tue, 9 Feb 2021 10:17:52 +0000 (11:17 +0100)]
mgr/dashboard: delete EOF when reading passwords from file
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit
caeadf1397db00c6b7ba218b1910508099802e39 )
Ernesto Puerta [Fri, 12 Feb 2021 10:31:58 +0000 (11:31 +0100)]
Merge pull request #39419 from rhcs-dashboard/wip-49248-nautilus
nautilus: mgr/dashboard: customize CherryPy Server Header
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Patrick Donnelly [Mon, 2 Nov 2020 18:01:59 +0000 (10:01 -0800)]
qa: leave one standby available to avoid warning
Four file systems will use all MDS and generate this warning:
2020-11-02T03:48:33.407 INFO:teuthology.orchestra.run.smithi003.stdout:2020-11-02T03:24:21.817337+0000 mon.a (mon.0) 481 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)
Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
59451923d31f5e4f707aa6d22ececc8edd395ca9 )
Patrick Donnelly [Fri, 30 Oct 2020 21:40:25 +0000 (14:40 -0700)]
qa: unmount volumes before removal
To avoid potential failures/hangs in umount.
Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
4ca8aaafa32bb7397eec80a8eb366d2dd781a72f )
Kotresh HR [Fri, 5 Feb 2021 18:05:22 +0000 (23:35 +0530)]
qa: Fix a few mgr/volume test cases
Recovering dirty auth metadata file might not retain the order,
fixed the comparison in 'test_recover_auth_metadata_during_authorize'
and 'test_recover_auth_metadata_during_deauthorize'.
Fixes: https://tracker.ceph.com/issues/49192
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
4952d2132ffd92594e749a652970d5d9415c721d )
anurag [Mon, 20 Apr 2020 23:13:07 +0000 (04:43 +0530)]
This commit is intended to hide CherryPy name and version, from HTTP header 'Server' response.
Fixes: <https://tracker.ceph.com/issues/44935>
Signed-off-by: anurag <abandhu@redhat.com>
(cherry picked from commit
6fea0f2ce861360643b86774fd86391b781c3789 )
Casey Bodley [Fri, 18 Sep 2020 14:10:50 +0000 (10:10 -0400)]
rgw: use static_ptr for etag verifiers
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
45a060612ec376110b84c2a2b7783c8a6aee191c )
Casey Bodley [Fri, 18 Sep 2020 00:58:36 +0000 (20:58 -0400)]
rgw: add factory function create_etag_verifier()
move all of the etag verifier initialization into a helper function.
none of the errors there should be fatal and fail the download, they
should just turn etag verification off
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
8fa8974bbd63fbc8be9cdf929a875910e2147d65 )
Casey Bodley [Fri, 18 Sep 2020 00:56:52 +0000 (20:56 -0400)]
rgw: move etag verifiers to namespace rgw::putobj
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
6ad2e3eef3f9bbf41de471ef5bad9502023e113c )
Casey Bodley [Fri, 18 Sep 2020 00:15:11 +0000 (20:15 -0400)]
rgw: simplify out SourceObjType
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
21cc9034410bb6e675b1b04888e1b85e3eb5d71f )
Casey Bodley [Thu, 17 Sep 2020 20:23:17 +0000 (16:23 -0400)]
rgw: rgw_sync_obj_etag_verify accounts for compressed multipart uploads
the etag verifier for multipart uploads uses the manifest to get the
logical offsets for each part. but when compression is enabled, those
are offsets into the compressed data. use the source object's compression
info to translate those compressed part offsets back to their original
offsets
Fixes: https://tracker.ceph.com/issues/45992
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
51f79fad8398d35e07f01fa45704124e16fadeec )
Casey Bodley [Thu, 17 Sep 2020 23:26:51 +0000 (19:26 -0400)]
rgw: ETagVerifier_MPU takes existing offset vector
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
2c1934bd3746697249b95b3f79f4c05425d7b40e )
Casey Bodley [Thu, 17 Sep 2020 17:26:32 +0000 (13:26 -0400)]
rgw: add helper to decode compression info from single attr
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
a5520be135156c867a6502845603e2afdbb0a44a )
Conflicts:
src/rgw/rgw_compression.cc
src/rgw/rgw_compression.h
Prasad Krishnan [Tue, 24 Mar 2020 18:19:59 +0000 (18:19 +0000)]
RGW:Multisite: Check rgw_sync_obj_etag_verify option only once
Signed-off-by: Prasad Krishnan <prasad.krishnan@flipkart.com>
(cherry picked from commit
f92cfaf74f1eb8809653867b8c67a3ace37619f2 )
Prasad Krishnan [Fri, 20 Mar 2020 20:01:35 +0000 (20:01 +0000)]
RGW:Multisite: Convert is_mpu_obj into an enum SourceObjType
Signed-off-by: Prasad Krishnan <prasad.krishnan@flipkart.com>
(cherry picked from commit
fa5422597837032d97f9afceff8b5a22fad0cda7 )
Prasad Krishnan [Thu, 19 Mar 2020 01:25:13 +0000 (01:25 +0000)]
RGW:Multisite: Rename rgw_sync_obj_integrity to rgw_sync_obj_etag_verify
Signed-off-by: Prasad Krishnan <prasad.krishnan@flipkart.com>
(cherry picked from commit
6c4262bfa7d54563ea8bf616154a6b3491d59347 )
Prasad Krishnan [Wed, 18 Mar 2020 18:03:15 +0000 (18:03 +0000)]
RGW:Multisite: Rename rgw_copy_verify_object to rgw_sync_obj_integrity
This patch renames the option rgw_copy_verify_object to
rgw_sync_obj_integrity and incorporates more changes suggested through
code-review comments.
Signed-off-by: Prasad Krishnan <prasad.krishnan@flipkart.com>
(cherry picked from commit
31e944fced60e47139973361cbb753aeaeb3c863 )
Prasad Krishnan [Mon, 16 Mar 2020 18:45:36 +0000 (18:45 +0000)]
RGW:Multisite: Create a new filter for ETag Verifier
This patch re-writes the ETag verifier into a filter that peeks into the
incoming stream of data and calculates MD5 checksum.
Signed-off-by: Prasad Krishnan <prasad.krishnan@flipkart.com>
(cherry picked from commit
2677c4b88806d4af6d525157e7006c1b0ca1b964 )
Conflicts:
src/rgw/rgw_obj_manifest.h
src/rgw/rgw_rados.cc
Prasad Krishnan [Fri, 6 Mar 2020 05:08:05 +0000 (05:08 +0000)]
[RGW][Multisite] Add multisite verifier support for MPU objects
The Etag for MPU objects is calculated using a method different from how
it is done for atomic objects. This patch makes use of the RGWObjManifest
to determine the parts in the source cluster and re-computes the ETag in
a similar fashion at the destination cluster during multisite sync for
verification.
Signed-off-by: Prasad Krishnan <prasad.krishnan@flipkart.com>
(cherry picked from commit
6f085bb9cb6cffddcf2eed18a7295a3dd59f755f )
Conflicts:
src/rgw/rgw_obj_manifest.h
Prasad Krishnan [Sun, 23 Feb 2020 06:09:49 +0000 (11:39 +0530)]
RGW:Multisite: Verify if the synced object is identical to source
Introduce an option 'rgw_copy_verify_object' which allows the object
copied from remote cluster through multisite sync is identical to the
source object. This is done by generating the MD5 checksum of the data
being copied and compared to the ETAG stored as part of the object's
attribute.
Signed-off-by: Prasad Krishnan <prasad.krishnan@flipkart.com>
(cherry picked from commit
23fb5d7b786f1034a02c991c6a648e455b5b9c30 )
Conflicts:
src/rgw/rgw_rados.cc
Rishabh Dave [Wed, 1 Apr 2020 11:21:06 +0000 (16:51 +0530)]
qa/cephfs: add methods to read/write on CephFS mounts
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit
3f0284f272231c3b62b0f3f201cbaaecfa405bcd )
Conflicts:
qa/tasks/cephfs/mount.py: get_file and IP module is not present in
nautilus
Kotresh HR [Sat, 23 Jan 2021 17:03:32 +0000 (22:33 +0530)]
ceph_volume_client: Fix failure of test_idempotency
With the test environment, 'args must be encodeable
as a bytearray' error is seen for 'ceph_mds_command'.
Hence removed tuple and passed the JSON formatted string.
Fixes: https://tracker.ceph.com/issues/48830
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
1c6c172a9b665d6b769d67e301061dbd7b044472 )
Kotresh HR [Fri, 15 Jan 2021 20:07:14 +0000 (01:37 +0530)]
mgr/volumes: Evict clients based on auth-IDs and subvolume mounted
Add subvolume evict command which evicts the subvolume mounts
which are mounted using particular auth-ID.
Fixes: https://tracker.ceph.com/issues/44928
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
269adcc8b8ab0742ba741ed7c2b59ccfb17a63f9 )
Conflicts:
qa/tasks/cephfs/test_volumes.py: Few of the tests are re-organized,
hence the conflicts. Resolved the same.
Kotresh HR [Tue, 5 Jan 2021 13:25:32 +0000 (18:55 +0530)]
doc/mgr/volumes: Document 'fs subvolume authorized_list' cli
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
9f9f8adc47486432c746565ea5a1f204736632c1 )
Kotresh HR [Wed, 16 Dec 2020 12:49:42 +0000 (18:19 +0530)]
qa: Add tests for list auth-ids of a subvolume
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
1b98e63e75236ff1cd1c9cb1ead8eb965698d5b6 )
Conflicts:
qa/tasks/cephfs/test_volumes.py: Few of the tests are re-organized,
hence the conflicts. Resolved the same.
Kotresh HR [Tue, 5 Jan 2021 12:55:54 +0000 (18:25 +0530)]
mgr/volumes: Update the 'volumes' key to 'subvolumes' in auth metadata file
The older auth metadata files created by CephVolumeClient stores the
authorized subvolumes using the 'volumes' key as the notion of
'subvolumes' brought in by mgr/volumes. Hence, this would be tranparently
updated to 'subvolumes' and newer auth metadata files would store them
with 'subvolumes' key.
Also fails the deauthorize if the auth-id doesn't exist.
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
5f32eb15918657bcf1f73025fbb3adf0cc06317e )
Kotresh HR [Tue, 15 Dec 2020 12:12:25 +0000 (17:42 +0530)]
mgr/volumes: Optionally authorize existing auth-ids
Optionally allow authorizing auth-ids not created by mgr plugin
via the option 'allow_existing_id'. This can help existing deployers
of manila to disallow/allow authorization of pre-created auth IDs
via a manila driver config that sets 'allow_existing_id' to False/True.
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
713270d1869e2370b674b1a4bc6f6a37023a5917 )
Kotresh HR [Tue, 15 Dec 2020 12:01:54 +0000 (17:31 +0530)]
mgr/volumes: Preserve existing caps while authorize/deauthorize auth-id
Authorize/Deauthorize used to overwrite the caps of auth-id which would
end up deleting existing caps. This patch fixes the same by retaining
the existing caps by appending or deleting the new caps as needed.
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
2dece3be081fe572455d6b634e38a663d1643dc8 )
Kotresh HR [Mon, 4 Jan 2021 13:04:54 +0000 (18:34 +0530)]
mgr/volumes: Disallow authorize existing auth_id
This patch disallow the mgr plugin to authorize the auth_id
which is not created via mgr plugin. Those auth_ids could be
created by other means for other use cases which should not be modified
via mgr plugin.
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
c02890404e47a5a95c5cc16b699306045d586c7f )
Kotresh HR [Mon, 23 Nov 2020 12:19:04 +0000 (17:49 +0530)]
mgr/volumes: Add subvolume authorized_list command
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
1abec3d0ca8c4fa405cdbf56c55f44f37aca9ca8 )
Kotresh HR [Mon, 23 Nov 2020 06:08:27 +0000 (11:38 +0530)]
mgr/volumes: Add tenant_id option to subvolume authorize
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
39acfcc91c1b6a85f6fa96a5e894d81a7225f9dc )
Kotresh HR [Wed, 18 Nov 2020 10:13:25 +0000 (15:43 +0530)]
mgr/volumes: Persist auth and subvolume metadata
1. Subvolume create and delete operations create and delete subvolume
metadata file respectively.
2. Subvolume authorize creates the auth meta file and persists the
required metadata on subvolume metadata file and auth metdata file
on disk. Subvolume deauthorize clears the required metadata on
both metadata files.
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
04d876ced756ca86580bdff4ac116333dbb102e5 )
Kotresh HR [Fri, 18 Dec 2020 11:33:14 +0000 (17:03 +0530)]
mgr/volumes: Filter inherited snapshots while listing snapshots
Filter inherited snapshots resulted as part of a snapshot
at ancestor level while listing snapshots of a subvolume
and subvolumegroup
Also, fail the snapshot info on inherited snapshot.
Fixes: https://tracker.ceph.com/issues/48501
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
bd49b6409be79dba4a119c809983a05687242732 )
Conflicts:
qa/tasks/cephfs/test_volumes.py: Few of the tests are re-organized,
hence the conflicts. Resolved the same.
Kotresh HR [Fri, 11 Sep 2020 04:26:22 +0000 (09:56 +0530)]
doc/mgr/volumes: Document authorize/deauthorize cli commands
Fixes: https://tracker.ceph.com/issues/40401
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
1426c23ab50a4d64e1c4f1f61185117a270c8ec7 )
Ramana Raja [Thu, 11 Jul 2019 11:14:35 +0000 (16:44 +0530)]
qa/tasks/cephfs: test `fs subvolume authorize/deauthorize`
Fixes: https://tracker.ceph.com/issues/40401
Signed-off-by: Ramana Raja <rraja@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
7c98dc1ad35a2244cbf949be9782a3412886b305 )
Conflicts:
qa/tasks/cephfs/test_volumes.py: Few of the tests are re-organized
and CLIENTS_REQUIRED should be 2 for these tests which was 1
for existing tests. Resolved the same.
Ramana Raja [Fri, 5 Jul 2019 06:41:52 +0000 (12:11 +0530)]
mgr/volumes: Allow/deny auth IDs access to FS subvolumes
... via the `ceph fs subvolume authorize/deauthorize` command.
Fixes: https://tracker.ceph.com/issues/40401
Signed-off-by: Ramana Raja <rraja@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
6c3b7547fbf3d987e715e9502359acd873374831 )
Conflicts:
src/pybind/mgr/volumes/fs/volume.py: subvolume pin is not available
in nautilus.
Ramana Raja [Wed, 10 Jul 2019 19:03:20 +0000 (00:33 +0530)]
mon/MonCap: allow 'profile mgr' to create/update/del auth IDs
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit
d3aea5579778a73dc2418e7b137dc21717944c3e )
Ilya Dryomov [Thu, 4 Feb 2021 17:09:31 +0000 (18:09 +0100)]
Merge pull request #39202 from idryomov/wip-krbd-msgr2-nautilus
nautilus: krbd: add support for msgr2 (kernel 5.11)
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Wed, 3 Feb 2021 18:42:24 +0000 (10:42 -0800)]
Merge pull request #38894 from smithfarm/wip-48518-nautilus
nautilus: pybind/cephfs: fix missing terminating NULL char in readlink()'s C string
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Wed, 3 Feb 2021 18:41:49 +0000 (10:41 -0800)]
Merge pull request #38627 from batrick/i48634
nautilus: qa: fix tox failures
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Wed, 3 Feb 2021 18:41:16 +0000 (10:41 -0800)]
Merge pull request #38101 from kotreshhr/wip-48192-nautilus
nautilus: cephfs: mds: throttle cap acquisition via readdir
Reviewed-by: Ramana Raja <rraja@redhat.com>
Jan Fajerski [Tue, 22 Dec 2020 14:44:10 +0000 (15:44 +0100)]
ceph-volume: mock VGs with string properties in tests
The real things uses string properties as well.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
c4bbb1b999b08bd84d793537f2602fc8e4c93bae )
Jan Fajerski [Tue, 22 Dec 2020 13:29:02 +0000 (14:29 +0100)]
ceph-volume: add some flexibility to bytes_to_extents
For batch we want to report the projected sizes before creating any LVM
structures, we use the device size for that. This means we pass this
projected size to lvm/api.py::create_lv. This could result in situations
where we request more extents for the new LV than are actually
available, as LVM takes up some available space on the block device
(plus alignment).
We fix this by adjusting the extent count downwards if its less than 1% off the
requested value.
Fixes: https://tracker.ceph.com/issues/47758
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
ece6e7eb9a92cb944b2f201a05a5acfb877c2dba )
Michał Nasiadka [Tue, 12 Jan 2021 14:21:51 +0000 (15:21 +0100)]
ceph-volume: Fix usage of is_lv
In commit
e09d3cd1a5bcfb42972b6958fb7e6c42a645549d is_lv method was removed from
api/lvm.py. This change fixes util/disk.py to not use this missing method.
Fixes: https://tracker.ceph.com/issues/48784
Signed-off-by: Michał Nasiadka <mnasiadka@gmail.com>
(cherry picked from commit
5050a88e7a85bb1aa6cd5473c9083a186e2264d5 )
Yuri Weinstein [Mon, 1 Feb 2021 17:31:48 +0000 (09:31 -0800)]
Merge pull request #39125 from smithfarm/wip-48595-nautilus
nautilus: tests: cancelling both noscrub *and* nodeep-scrub
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Yuri Weinstein [Mon, 1 Feb 2021 17:29:07 +0000 (09:29 -0800)]
Merge pull request #39124 from smithfarm/wip-48379-nautilus
nautilus: mon: validate crush-failure-domain
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Mon, 1 Feb 2021 17:28:39 +0000 (09:28 -0800)]
Merge pull request #39076 from k0ste/wip-48957-nautilus
nautilus: mgr/prometheus: Fix 'pool filling up' with >50% usage
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Yuri Weinstein [Mon, 1 Feb 2021 17:27:53 +0000 (09:27 -0800)]
Merge pull request #39075 from k0ste/wip-49005-nautilus
nautilus: mgr: update mon metadata when monmap is updated
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Mon, 1 Feb 2021 17:26:23 +0000 (09:26 -0800)]
Merge pull request #37948 from aclamk/fix-46194-bluefs-log-growth-nau
nautilus: bluestore: Add protection against bluefs log file growth
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Yuri Weinstein [Mon, 1 Feb 2021 16:40:32 +0000 (08:40 -0800)]
Merge pull request #39123 from smithfarm/wip-49026-nautilus
nautilus: rbd/bench: include used headers
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Mon, 1 Feb 2021 16:39:19 +0000 (08:39 -0800)]
Merge pull request #39121 from smithfarm/wip-48691-nautilus
nautilus: librbd: clear implicitly enabled feature bits when creating images
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Mon, 1 Feb 2021 16:38:10 +0000 (08:38 -0800)]
Merge pull request #39047 from smithfarm/wip-48969-nautilus
nautilus: ocf: add support for mapping images within an RBD namespace
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Mon, 1 Feb 2021 16:35:36 +0000 (08:35 -0800)]
Merge pull request #38830 from smithfarm/wip-48083-nautilus
nautilus: rbd-nbd: reexpand the conf meta in child process
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Patrick Seidensal [Tue, 3 Nov 2020 12:47:23 +0000 (13:47 +0100)]
mgr/dashboard: enable different URL for users of browser to Grafana
Fixes: https://tracker.ceph.com/issues/47386
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit
676f82923d875cfe9528a401963c09f29a6de7f4 )
Conflicts:
doc/mgr/dashboard.rst
src/pybind/mgr/dashboard/controllers/grafana.py
Resolved some trivial conflicts and replaced the heading of the newly
added section in the documentation to another level, as the
documentation between master and octopus differs quite much.
(cherry picked from commit
56f304d31b154d87362620f08046720a3e0f3203 )
Conflicts:
Removed cephadm documentation from cherry-picked commit as cephadm is
not part of Ceph Nautilus.
Removed paragraph that mentioned cephadm in dashboard documentation.