]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Kefu Chai [Fri, 24 Sep 2021 15:33:03 +0000 (23:33 +0800)]
os/bluestore: list obj which equals to pend
otherwise we could have failures like
scrub : stat mismatch, got 3/4 objects, 1/2 clones, 3/4 dirty, 3/4 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 49/56 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes."
where the numbers of scrubbed object, clones, dirty and omap are always
less than the total number of corresponding numbers, if the PG contains
object(s) whose hash happens to be 0xffffffff.
in this change, if the calculated hash of the upper bound is greater
than the maximum possible number represented by uint32_t, in addition to
setting the hash of the upper bound hobj to 0xffffffff, we also set the
nspace of hobj of the upper bound to "\xff", so that the upper bound
is greater than an hobj whose hash happens to be 0xfffffff. please note,
the nspace of "\xff" is not an ascii string, so it's not likely to be
less than a real-world nspace of an hobj.
with this new *greater* upper bound, we are able to include the previous
missing hobj when listing the objects in a PG. so the scrub won't be
annoyed when the number of objects does not match.
Fixes: https://tracker.ceph.com/issues/52705
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit
ffab13bcd9006c1f961a24b8016df9d1fe06ba1d )
Kefu Chai [Wed, 22 Sep 2021 16:42:33 +0000 (00:42 +0800)]
os/bluestore: use scope_guard to log latency
simpler this way, and avoid using `goto`.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit
715a83822ebc1a3d102d1ec13323b69db0600719 )
Yuri Weinstein [Tue, 12 Oct 2021 12:44:53 +0000 (05:44 -0700)]
Merge pull request #43368 from tchaikov/pacific-pr-39602
pacific: mgr/influx: use "N/A" for unknown hostname
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Yuri Weinstein [Tue, 12 Oct 2021 12:44:18 +0000 (05:44 -0700)]
Merge pull request #43351 from rhcs-dashboard/wip-52772-pacific
pacific: qa/mgr/dashboard: add extra wait to test
Reviewed-by: Nizamudeen A <nia@redhat.com>
Yuri Weinstein [Tue, 12 Oct 2021 12:43:31 +0000 (05:43 -0700)]
Merge pull request #43347 from rhcs-dashboard/wip-52763-pacific
pacific: mgr/dashboard: Move force maintenance test to the workflow test suite
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Yuri Weinstein [Tue, 12 Oct 2021 12:41:57 +0000 (05:41 -0700)]
Merge pull request #43167 from ktdreyer/pacific-52610-cmake-thread-libs-init
pacific: cmake: link Threads::Threads instead of CMAKE_THREAD_LIBS_INIT
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Yuri Weinstein [Fri, 8 Oct 2021 13:34:17 +0000 (06:34 -0700)]
Merge pull request #43199 from vshankar/wip-52627
pacific: mgr/mirroring: remove unnecessary fs_name arg from daemon status command
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Fri, 8 Oct 2021 13:33:42 +0000 (06:33 -0700)]
Merge pull request #43198 from vshankar/wip-52444
pacific: cephfs-mirror: shutdown ClusterWatcher on termination
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Yuri Weinstein [Fri, 8 Oct 2021 13:32:12 +0000 (06:32 -0700)]
Merge pull request #43148 from lxbsz/fair_mutex
pacific: mds: switch mds_lock to fair mutex to fix the slow performance issue
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Yuri Weinstein [Tue, 5 Oct 2021 15:00:24 +0000 (08:00 -0700)]
Merge pull request #43348 from cfsnyder/wip-52350-pacific
pacific: rgw: fix sts memory leak
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 5 Oct 2021 14:59:37 +0000 (07:59 -0700)]
Merge pull request #42643 from cfsnyder/wip-51803-pacific
pacific: rgw/notifications: send correct size in case of delete marker creation
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 4 Oct 2021 15:18:03 +0000 (08:18 -0700)]
Merge pull request #43235 from MrFreezeex/wip-51839-pacific
pacific: ceph.spec: selinux scripts respect CEPH_AUTO_RESTART_ON_UPGRADE
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>
Yuri Weinstein [Fri, 1 Oct 2021 15:17:47 +0000 (08:17 -0700)]
Merge pull request #43264 from cfsnyder/wip-52332-pacific
pacific: cmake: s/Python_EXECUTABLE/Python3_EXECUTABLE/
Reviewed-by: Michael Fritch <mfritch@suse.com>
Yuri Weinstein [Thu, 30 Sep 2021 22:53:16 +0000 (15:53 -0700)]
Merge pull request #43099 from cfsnyder/wip-51952-pacific
pacific: osd: fix to recover adjacent clone when set_chunk is called
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Thu, 30 Sep 2021 22:52:46 +0000 (15:52 -0700)]
Merge pull request #43306 from myoungwon/pacific-backport-52322
pacific: osd: fix to allow inc manifest leaked
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Kefu Chai [Mon, 22 Feb 2021 05:53:42 +0000 (13:53 +0800)]
mgr/influx: use "N/A" for unknown hostname
in theory, there is chance that get_metadata() returns None, so let use
"N/A" in this case.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
e457ca50011f70cf01a62323998af233a484f338 )
Ernesto Puerta [Wed, 29 Sep 2021 19:34:41 +0000 (21:34 +0200)]
Merge pull request #43238 from rhcs-dashboard/wip-52685-pacific
pacific: mgr/dashboard: Fix failing config dashboard e2e check
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Ernesto Puerta [Wed, 22 Sep 2021 12:10:28 +0000 (14:10 +0200)]
qa/mgr/dashboard: add extra wait to test
Fixes: https://tracker.ceph.com/issues/49344
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit
9ff778cdaa1ef40fcfa04f221a1da786a0e19655 )
yuliyang_yewu [Tue, 17 Aug 2021 03:04:02 +0000 (11:04 +0800)]
rgw: fix sts memory leak
fix https://tracker.ceph.com/issues/52290
Signed-off-by: yuliyang_yewu <yuliyang_yewu@cmss.chinamobile.com>
(cherry picked from commit
ef921bcdaa78d33ed0611a60ec58826d8e6ccb45 )
Yuval Lifshitz [Thu, 15 Jul 2021 13:40:06 +0000 (16:40 +0300)]
rgw/notifications: send correct size in case of delete marker creation
Fixes: https://tracker.ceph.com/issues/51681
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit
d81e27faa1033c5290cfd0b4cf27cdaf98d34bc4 )
Conflicts:
src/rgw/rgw_op.cc
src/test/rgw/bucket_notification/test_bn.py
Cherry-pick notes:
- src/test/rgw/bucket_notification/test_bn.py changes manually applied to src/test/rgw/rgw_multi/tests_ps.py for Pacific
- conflicts in rgw_op.cc due to rename of RGWObject to Object after Pacific
Nizamudeen A [Tue, 17 Aug 2021 16:56:52 +0000 (22:26 +0530)]
mgr/dashboard: Move force maintenance test to the workflow test suite
Fixes: https://tracker.ceph.com/issues/52276
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit
e33d9e15b92f922d3c5c40475cf493c238e91d75 )
Ernesto Puerta [Mon, 27 Sep 2021 11:13:32 +0000 (13:13 +0200)]
Merge pull request #43185 from rhcs-dashboard/wip-52617-pacific
pacific: mgr/dashboard: Incorrect MTU mismatch warning
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Ilya Dryomov [Mon, 27 Sep 2021 09:07:40 +0000 (11:07 +0200)]
Merge pull request #43113 from idryomov/wip-rbd-validate-pool-async-pacific
pacific: librbd: fix pool validation lockup
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
myoungwon oh [Tue, 17 Aug 2021 12:51:29 +0000 (21:51 +0900)]
test: allowing >= the real refcount to avoid false alarm
Current dedup allow to contain multiple same sources using
multiset, which results in inconsistent situation as follow
(during set_chunk, but not confined in set_chunk).
1. User issues set_chunk
2. OSD receives the set_chunk, and sends increment message
to an object in the low tier (INPROGRESS).
3. OSD map is changed (841 → 843)
3.5. on_change() is called
4. the set_chunk op is reenqueued by requeue_op()
5. OSD handles the duplicated set_chunk, but it is not able to
know the set_chunk is duplicated because it does not log on the disk yet.
6. OSD issues increment message again to the object
in the low tier. (increment operation is executed twice)
To fix this, this commit allows >= the real refcount in test cases
fixes: https://tracker.ceph.com/issues/51000
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
126df96 )
Yuri Weinstein [Fri, 24 Sep 2021 15:21:29 +0000 (08:21 -0700)]
Merge pull request #41731 from sseshasa/wip-51117-pacific
pacific: osd: Run osd bench test to override default max osd capacity for mclock
Reviewed-by: Neha Ojha <nojha@redhat.com>
Michael Fritch [Tue, 17 Aug 2021 21:37:13 +0000 (15:37 -0600)]
tools/setup-virtualenv: verify the `--python` param
when given an invalid or empty (PYTHON_BINARY)
fixup for
494fea50ced7
Fixes: https://tracker.ceph.com/issues/52304
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit
c8d4c131adfe41739fd557091dc2931c1b7f15c5 )
Michael Fritch [Tue, 17 Aug 2021 21:36:50 +0000 (15:36 -0600)]
cmake: s/Python_EXECUTABLE/Python3_EXECUTABLE/
pass the python3 exec when creating the ceph-volume build venv
fixup for
5fc657b40dc7
Fixes: https://tracker.ceph.com/issues/52304
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit
7db830598507d90d1c9e1f4468f818bebce58037 )
Yuri Weinstein [Wed, 22 Sep 2021 14:24:00 +0000 (07:24 -0700)]
Merge pull request #43242 from ceph/wip-yuriw-p2p-pacific
qa/tests: advanced version to reflect the latest 16.2.6 release
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Tue, 21 Sep 2021 13:16:48 +0000 (06:16 -0700)]
qa/tests: advanced version to reflect the latest 16.2.6 release
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Yuri Weinstein [Tue, 21 Sep 2021 19:12:31 +0000 (12:12 -0700)]
Merge pull request #42949 from yuvalif/wip-51597-pacific
pacific: rgw/notifications: cache object size to avoid accessing invalid memory
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 21 Sep 2021 19:11:55 +0000 (12:11 -0700)]
Merge pull request #42947 from yuvalif/wip-51511-pacific
pacific: rgw/notifications: support v4 auth for topics and notifications
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
Yuri Weinstein [Tue, 21 Sep 2021 19:11:12 +0000 (12:11 -0700)]
Merge pull request #42946 from yuvalif/wip-51350-pacific
pacific: rgw/notification: make notifications agnostic of bucket reshard
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 21 Sep 2021 19:10:21 +0000 (12:10 -0700)]
Merge pull request #42945 from yuvalif/wip-51045-pacific
pacific: rgw: fix spelling of eTag in S3 message structure
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Sebastian Wagner [Tue, 21 Sep 2021 11:58:02 +0000 (13:58 +0200)]
Merge pull request #43075 from sebastian-philipp/pacific-backport-41510-41509-41964-41574-42278-42073-42406-42463-42627-42680-
pacific: mgr/nfs: nfs-rgw batch backport
Reviewed-by: Sage Weil <sage@newdream.net>
Sebastian Wagner [Tue, 21 Sep 2021 10:30:18 +0000 (12:30 +0200)]
doc/cephfs/nfs: Remove missing target
Fixes: `Unknown target name: "nfs-ganesha nfs server".`
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
Ernesto Puerta [Tue, 21 Sep 2021 10:01:07 +0000 (12:01 +0200)]
Merge pull request #43188 from rhcs-dashboard/wip-51274-pacific
pacific: mgr/dashboard: deprecated variable usage in Grafana dashboards
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: p-se <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Nizamudeen A [Fri, 17 Sep 2021 16:07:09 +0000 (21:37 +0530)]
mgr/dashboard: Fix failing config dashboard e2e check
Recently a new osd config has been added in
6ca32bde2e1d0dd58df168126582a570ac09aad6 and this is getting Modified.
So on our dashboard e2e config check which checks for the Modified
filter, this is also coming on the entry. So we need to increase the
count.
Fixes: https://tracker.ceph.com/issues/52649
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit
53f78d330c98f803c09a2314397d48f253db0866 )
Ernesto Puerta [Tue, 21 Sep 2021 09:57:58 +0000 (11:57 +0200)]
Merge pull request #43190 from rhcs-dashboard/wip-52251-pacific
pacific: mgr/dashboard: cephfs MDS Workload to use rate for counter type metric
Ernesto Puerta [Tue, 21 Sep 2021 09:54:46 +0000 (11:54 +0200)]
Merge pull request #42678 from callithea/wip-52021-pacific
pacific: mgr/dashboard: Visual regression tests for ceph dashboard
Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Dan van der Ster [Mon, 12 Jul 2021 13:35:39 +0000 (15:35 +0200)]
ceph.spec: selinux scripts respect CEPH_AUTO_RESTART_ON_UPGRADE
In /etc/sysconfig/ceph we allow operators to define if ceph daemons
should be restarted on upgrade: CEPH_AUTO_RESTART_ON_UPGRADE.
But the post selinux scripts will stop ceph.target regardless if this
is set to `no`, leading to operators adding various hacks to prevent
these unexpected or inconvenient daemon restarts. By now, if users
are using rpms directly, they are likely orchestrating their own
daemon restarts so should not rely on the rpm itself to do this.
Fixes: https://tracker.ceph.com/issues/21672
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit
092a6e3e83e9ef8e37cb6f1033c345dcb5224cfc )
Yuri Weinstein [Fri, 17 Sep 2021 17:15:02 +0000 (10:15 -0700)]
Merge pull request #43002 from ktdreyer/pacific-52472-rm-virtualenv
pacific: *: s/virtualenv/python -m venv/
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Willem Jan Withagen [Sat, 28 Aug 2021 15:40:11 +0000 (17:40 +0200)]
tools/cephfs_mirror: fix lock declaratie/locking
Clang complains:
```
/home/jenkins/workspace/ceph-master-compile/src/tools/cephfs_mirror/ClusterWatcher.cc:175:19: error: cannot use parentheses when declaring variable with deduced class template specialization type
std::scoped_lock(m_lock);
^
```
fixes: https://github.com/ceph/ceph/pull/42751
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit
3f952b9c9b69a361441952b279a39b164881ce12 )
Venky Shankar [Mon, 2 Aug 2021 04:26:28 +0000 (00:26 -0400)]
doc: fix `daemon status` interface (exclude file system name)
Fixes: http://tracker.ceph.com/issues/51989
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit
ed253597c9376ac76171ca27bece09be8a72e269 )
Venky Shankar [Mon, 2 Aug 2021 04:24:05 +0000 (00:24 -0400)]
test: adjust mirroring tests for `daemon status` change
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit
11b61b4fb91f731ee1d04cc545bfa4937e844bb5 )
Venky Shankar [Mon, 2 Aug 2021 04:23:23 +0000 (00:23 -0400)]
mgr/mirroring: `daemon status` command does not require file system name
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit
746278727e10c9a2b28f0f9a43401ed88d23de70 )
Venky Shankar [Thu, 5 Aug 2021 08:35:10 +0000 (04:35 -0400)]
cephfs-mirror: shutdown ClusterWatcher on termination
Fixes: https://tracker.ceph.com/issues/52062
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit
9b8d82009278cc275a71102914bcb34850ce03b8 )
Jenkins Build Slave User [Thu, 16 Sep 2021 14:27:19 +0000 (14:27 +0000)]
16.2.6
Jan Horáček [Thu, 27 May 2021 00:03:01 +0000 (02:03 +0200)]
[mgr/dashboard] cephfs metrics in MDS Workload panels to use rate because of counter type metric
Fixes: https://tracker.ceph.com/issues/51954
Signed-off-by: Jan Horacek <jan.horacek@livesport.eu>
(cherry picked from commit
5bf516dcc7f59a970c6369ab075cbca7d81ac7d5 )
Patrick Seidensal [Tue, 30 Mar 2021 18:20:49 +0000 (20:20 +0200)]
mgr/dashboard: deprecated variable usage in Grafana dashboards
Fixes: https://tracker.ceph.com/issues/50059
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit
a709abf8bf5a6b25c21db100e87af3a6c2cf382d )
Aashish Sharma [Thu, 2 Sep 2021 06:27:57 +0000 (11:57 +0530)]
mgr/dashboard: Incorrect MTU mismatch warning
The MTU mismatch warning was being fired for those NIC's as well that are in down state. This PR intends to fix this issue
Fixes:https://tracker.ceph.com/issues/52028
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit
58d635455d1f59921d5ad821168f31b6f937588a )
Yuri Weinstein [Wed, 15 Sep 2021 14:30:22 +0000 (07:30 -0700)]
Merge pull request #43168 from linuxbox2/pacific-perl-semicolon
workunits/rgw: semicolon terminates perl statements
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Matt Benjamin [Tue, 14 Sep 2021 17:16:23 +0000 (13:16 -0400)]
workunits/rgw: semicolon terminates perl statements
Fixes a lexical error in one line of code added in
90e9307ab0a52da260bc1ebb50329cd8ff942eb9 , removing the dependency
on lsb_release, on 8/16/2021.
Fixes: https://tracker.ceph.com/issues/52613
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit
221fdb858be083a981697cdb59c19f0659be3f1d )
Ken Dreyer [Fri, 20 Aug 2021 14:44:39 +0000 (10:44 -0400)]
cmake: link Threads::Threads instead of CMAKE_THREAD_LIBS_INIT
CMAKE_THREAD_LIBS_INIT can be empty. On newer platforms, this leads to
an error:
CMake Error at cmake/modules/Buildpmem.cmake:47 (set_target_properties):
set_target_properties called with incorrect number of arguments.
This is because glibc 2.34 merged libpthread.so into libc.so.
Fedora 35 and RHEL 9 have glibc 2.34.
Fixes: https://tracker.ceph.com/issues/52353
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
Co-authored-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
6c4f31eccfd1233e4413132b11e6424e6f4e5990 )
Yuri Weinstein [Tue, 14 Sep 2021 15:43:58 +0000 (08:43 -0700)]
Merge pull request #43151 from ceph/pacific-systemd
pacific: debian/control: dh-systemd is part of debhelper now
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Tue, 14 Sep 2021 15:43:13 +0000 (08:43 -0700)]
Merge pull request #43131 from ceph/remove-cython
pacific: debian/control: remove cython from Build-Depends
Reviewed-by: Kefu Chai <kchai@redhat.com>
David Galloway [Fri, 10 Sep 2021 13:55:56 +0000 (09:55 -0400)]
pacific: debian/control: dh-systemd is part of debhelper now
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=822670
Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit
4f21d41b73e30b71c9222841751dcebf898613cc )
Sridhar Seshasayee [Wed, 25 Aug 2021 07:20:23 +0000 (12:50 +0530)]
qa/tasks: Set default caps for 'osd' type in generate_caps()
Assign the default caps for osds to be the same as what the AuthMonitor
sets for a new osd. See AuthMonitor::validate_osd_new() which sets the
following caps for a new osd:
mon='allow profile osd'
mgr='allow profile osd'
osd=''allow *'
When an actual real world cluster is deployed, the above caps are applied.
Unless the user modifies the defaults, a cluster will operate with the
above caps. Therefore, it makes sense to use the defaults when testing
Ceph so that issues if any due to the default settings may be caught and
fixed.
Therefore, the caps for the 'osd' type is reset to the default in
generate_caps(). The caps for 'mgr' already reflects the system defaults.
The caps for 'mds' type is not changed in this commit and will be
investigated and changed if necessary later.
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit
4b0dba28b62bcc89c46e1f5ff0dab8583af4397c )
Sridhar Seshasayee [Thu, 19 Aug 2021 13:52:02 +0000 (19:22 +0530)]
mon/MonCap: Update osd profile to allow cmd to set iops capacity on mon db
The default mon caps for osds is set to "allow profile osd", which allows
only "rw" capability. Osds with mclock scheduler enabled store their max
iops capacity on the mon config store. This can be achieved by executing
the "config set" command. However, since the osd(s) by default do not have
the execute permission, the command fails with "Permission denied" error.
Therefore, modify the default osd profile to allow running the "config set"
command with restriction to only set keys with name matching either (regex)
"osd_mclock_max_capacity_iops_hdd" or "osd_mclock_max_capacity_iops_ssd"
so that the osd has the permission to update the mon config store with the
desired information.
Fixes: https://tracker.ceph.com/issues/52329
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit
2cdbe81d7dd34b65e5c3c51005df5214a1e3a577 )
Sridhar Seshasayee [Tue, 3 Aug 2021 11:26:55 +0000 (16:56 +0530)]
doc: Update mclock-config-ref doc steps to override osd max iops capacity.
Update the steps in the mclock config reference document to manually
override an OSDs max IOPS capacity. Provide information on the alternative
ways to override the osd_mclock_max_capacity_iops_[hdd,ssd] options for
an OSD.
Fixes: https://tracker.ceph.com/issues/52025
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit
7c25511a41e452bcc5806f192e90aac6cd113f9e )
Sridhar Seshasayee [Tue, 3 Aug 2021 08:33:26 +0000 (14:03 +0530)]
osd: Add config option to skip running the OSD benchmark on start-up.
Introduce a new dev config option "osd_mclock_skip_benchmark" that
when set skips running the OSD benchmark on start-up. By default
this option is disabled. This is useful in the following scenarios:
- Dev/CI testing,
- Configurations that don't need QoS.
If the option is enabled, the default OSD iops capacity is read from
osd_mclock_max_capacity_iops_[hdd,ssd].
Fixes: https://tracker.ceph.com/issues/52025
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit
6ca32bde2e1d0dd58df168126582a570ac09aad6 )
Conflicts:
src/common/options/osd.yaml.in
- Removed non-existent file: src/common/options/osd.yaml.in since the
switch to yaml for config options is not available in pacific yet.
Xiubo Li [Mon, 2 Aug 2021 03:53:37 +0000 (11:53 +0800)]
mds: switch mds_lock to fair mutex
The implementations of the Mutex (e.g. std::mutex in C++) do not
guarantee fairness, they do not guarantee that the lock will be
acquired by threads in the order that they called the lock().
In most case this works well, but in corner case in the Finisher
thread in mds daemon, which may call more than one complete()s
once the mdlog flushing succeeds, after the mdlog flushing is done
it will call the queued complete callbacks and the Finisher thread
could always successfully acquire the mds_lock in successive
complete callbakcs even there may have other threads already being
stuck waiting the mds_lock. This will make the other threads starve
and if they are client's requests, it will cause several or even
tens of seconds long delay for user's operations.
This will switch the mds_lock to fair mutex and it could make sure
that the all the mds_lock waiters are in FIFO order and the Finisher
thread won't hold the mds_lock that long.
At the same time, if the finisher thread has many completes needed
to run the fair mutex could guarantee that the finisher won't be
scheduled out due to fair mutex unlock() if no any other mds_lock
waiter queued.
Fixes: https://tracker.ceph.com/issues/51722
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit
5ade254c15d0b9f96cdd5cd2d5f7861ef2232812 )
Conflicts:
src/mds/MDSRank.cc
src/mds/MDSRank.h
Xiubo Li [Mon, 2 Aug 2021 06:20:52 +0000 (14:20 +0800)]
common/Timer: make SafeTimer a template
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit
215b12ae0abc781430f3e5d30991b7f5055aeb95 )
Conflicts:
src/common/Timer.cc
src/librbd/ImageCtx.h
Xiubo Li [Mon, 2 Aug 2021 03:52:57 +0000 (11:52 +0800)]
common/fair_mutex: add is_locked_by_me support
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
98ace572a1d047157f26f8ed40d4d848502bb973 )
Kefu Chai [Wed, 18 Aug 2021 06:10:03 +0000 (14:10 +0800)]
test/common/test_fair_mutex: increase NR_ROUNDS from 256 to 512
so the test is more tolerant.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
ab3dc0811baa17ff03c777d71f4ebe096a650cec )
Kefu Chai [Wed, 18 Aug 2021 06:06:55 +0000 (14:06 +0800)]
test/common/test_fair_mutex: s/std::thread/std::async()/
for better readability, as we don't need to care about the details of
the executor. and the returned futures are blocked in their dtor as
> these actions will not block for the shared state to become ready,
> except that it may block if all of the following are true: the
> shared state was created by a call to std::async, the shared state
> is not yet ready, and this was the last reference to the shared state.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
4e35ac670bc75e3507777d4375bbbb0fea8533ba )
Kefu Chai [Fri, 30 Jul 2021 04:44:52 +0000 (12:44 +0800)]
common: add ceph::fair_mutex
a mutex which enqueues and wakes up the waiters in FIFO order, to
ensure the fairness of the mutex.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
8c07345d331495e0b76cbe02e95593f66a5fcc43 )
Kefu Chai [Sun, 21 Mar 2021 14:55:37 +0000 (22:55 +0800)]
debian/control: remove cython from Build-Depends
as cython3 is enough. and we've dropped the python2 support.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
bcd7f3ac0f299855516df871a56ef26c1ee21bc9 )
Yuri Weinstein [Thu, 9 Sep 2021 17:26:33 +0000 (10:26 -0700)]
Merge pull request #43100 from ifed01/wip-ifed-fix-migrate-pac
pacific: os/bluestore: fix bluefs migrate command
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Thu, 9 Sep 2021 17:25:24 +0000 (10:25 -0700)]
Merge pull request #42976 from idryomov/wip-51419-pacific
pacific: common/buffer: fix SIGABRT in rebuild_aligned_size_and_memory
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Guillaume Abrioux [Thu, 9 Sep 2021 15:38:33 +0000 (17:38 +0200)]
Merge pull request #43116 from guits/wip-52560-pacific
pacific: ceph-volume: fix lvm activate arguments
Guillaume Abrioux [Thu, 9 Sep 2021 14:53:14 +0000 (16:53 +0200)]
Merge pull request #43110 from guits/wip-52554-pacific
pacific: ceph-volume: fix lvm migrate without args
Sage Weil [Wed, 11 Aug 2021 16:31:21 +0000 (11:31 -0500)]
doc/mgr/nfs: add section on updating an nfs cluster
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
7cc4c91dce2a1f1bcb6a02efb231bdbda0929500 )
Sage Weil [Thu, 5 Aug 2021 14:17:40 +0000 (10:17 -0400)]
mgr/cephadm: drop daemon_id arg to CephadmService.config()
Unused (and nonsensical since this is *service* config).
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
e9dabdab6540d949bd3401fef66ec583368b33de )
Sage Weil [Thu, 5 Aug 2021 21:57:58 +0000 (17:57 -0400)]
mgr/nfs: add --port to 'nfs cluster create' and port to 'nfs cluster info'
Fixes: https://tracker.ceph.com/issues/51787
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
8ebe341198d95da1a0fbfe4d5ff5bbbec7aa155e )
Sage Weil [Thu, 5 Aug 2021 18:41:20 +0000 (14:41 -0400)]
qa/suites/orch/cephadm/smoke-roleless: test taking ganeshas offline
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
a5e8227a83c229698af462a18f9548c03c559408 )
Sage Weil [Thu, 5 Aug 2021 20:17:42 +0000 (16:17 -0400)]
qa/tasks/vip: exec with bash -ex
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
3c1e086be013bf875bfe32b5ba9a297fc27410a3 )
Sage Weil [Thu, 5 Aug 2021 18:00:07 +0000 (14:00 -0400)]
qa/suites/orch/cephadm: separate test_nfs from test_orch_cli
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
0ae020979d964b50f550cab6366ca019133e1dd1 )
Varsha Rao [Thu, 5 Aug 2021 11:41:00 +0000 (17:11 +0530)]
src/pybind/mgr/nfs/tests: pass cluster_id to from_export_block()
As from_export_block() requires cluster id.
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
13c0525619ab928d664e6896489029aa7548e159 )
Varsha Rao [Thu, 5 Aug 2021 11:24:45 +0000 (16:54 +0530)]
src/pybind/mgr/nfs: remove `tag` option
'tag' is required for NFSv3 mounts. We don't support v3, so let's remove it.
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
ab80ab4675d70c3c9fbf5f5f0c3ec6680f055b17 )
Varsha Rao [Thu, 5 Aug 2021 10:11:27 +0000 (15:41 +0530)]
src/pybind/mgr/nfs: remove per daemon config test
This is not required as config object is per cluster.
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
399edcd321e95338ff96fa1f795485b5ed97bbcb )
Varsha Rao [Wed, 4 Aug 2021 20:15:14 +0000 (01:45 +0530)]
src/pybind/mgr/nfs: directly use cluster_id and remove daemon related stuff
Fixes: https://tracker.ceph.com/issues/51614
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
3c51638d4ee5e116e55238a4836d0824ca6d9bf3 )
Varsha Rao [Wed, 4 Aug 2021 11:03:43 +0000 (16:33 +0530)]
.github/labeler: add nfs dev doc
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
482a13077a079b88a1dcf7ed2582e345f568f38c )
Varsha Rao [Wed, 4 Aug 2021 10:46:29 +0000 (16:16 +0530)]
doc/dev/vstart-ganesha: update about RGW export
Fixes: https://tracker.ceph.com/issues/51683
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
265561bef6b31e446ec2fd00d508c0eed1796c7a )
Varsha Rao [Wed, 4 Aug 2021 10:39:54 +0000 (16:09 +0530)]
src/vstart: update ganesha pid dir location
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
accac3beb9f9f06e8ec7776af4b455421585ed34 )
Varsha Rao [Wed, 4 Aug 2021 10:35:47 +0000 (16:05 +0530)]
src/vstart: create rgw export for nfs
Fixes: https://tracker.ceph.com/issues/51800
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
8a77e017fd2110ec9d7fbc0a33de0fb85f14cbca )
Conflicts:
src/vstart.sh
Varsha Rao [Fri, 23 Jul 2021 09:45:26 +0000 (15:15 +0530)]
src/pybind/mgr/test_orchestrator: remove nfs pool and namespace
'orch apply nfs' no longer requires pool and namespace.
Fixes: https://tracker.ceph.com/issues/51794
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
da5aa88844531c7b9ef0420cf274a72d5b4e308e )
Varsha Rao [Fri, 23 Jul 2021 09:40:52 +0000 (15:10 +0530)]
src/vstart: update nfs-ganesha pool name
Recently, the default pool name for nfs was changed from 'nfs-ganesha' to
'.nfs'. Change the pool name in vstart too.
Fixes: https://tracker.ceph.com/issues/51795
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
ff33b4f23b9263ae7e189cffef18ec2128e52a35 )
Sage Weil [Tue, 27 Jul 2021 14:46:25 +0000 (10:46 -0400)]
mgr/nfs: remove unused 'realm' arg for 'nfs export create rgw'
This argument is not used. The rgw exports currently only work on the
default realm.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
2db3cfabc3d5154036ccf3168252f26a75ef0935 )
Sage Weil [Mon, 26 Jul 2021 20:14:44 +0000 (16:14 -0400)]
doc/mgr/rook: update title
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
9aeefbc666e2d373f95aca20deb15a9e9a0b6f86 )
Sage Weil [Mon, 26 Jul 2021 20:12:45 +0000 (16:12 -0400)]
doc/mgr/nfs: reference customizing ingress
Link to the cephadm docs on modifying the service directly.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
7d3443412ce8feadac4258f852c44d6947dd1389 )
Sage Weil [Mon, 26 Jul 2021 20:04:15 +0000 (16:04 -0400)]
doc/mgr/nfs: add section for manual ganesha config; reframe
This documentation is incomplete because this mode of operation is not
tested/validated.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
b5a32d632d2acb77664df9b25d98fb6489b673fc )
Sage Weil [Mon, 26 Jul 2021 19:58:08 +0000 (15:58 -0400)]
doc/mgr/nfs: document ingress in more detail
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
8d9db910f797557d615b5e5e3e612bed7dd00f22 )
Sage Weil [Mon, 26 Jul 2021 19:27:59 +0000 (15:27 -0400)]
doc/mgr/nfs: typo
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
0448a8d39de71e5135061873a40f9302ad2c5dce )
Sage Weil [Mon, 26 Jul 2021 14:51:02 +0000 (10:51 -0400)]
doc/mgr/nfs: add note about incomplete ingress
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
684c6a2dbaf5c6c2d8d2abe24243f6b973d14c58 )
Sage Weil [Thu, 15 Jul 2021 20:49:12 +0000 (16:49 -0400)]
qa/suites/orch/cephadm: add rgw nfs export test
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
cd089ee74eb01bb9d2208400296e61e9636df5ab )
Sage Weil [Mon, 19 Jul 2021 22:12:24 +0000 (18:12 -0400)]
mgr/cephadm: ingress: tolerate no daemons
This doesn't normally happen, but did before the daemon inventory breakage
(see previous patches) was fixed.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
bae406f74607b16c0da1382537d9bab49d45061e )
Sage Weil [Mon, 19 Jul 2021 17:47:12 +0000 (13:47 -0400)]
mgr/nfs: add --squash option to 'nfs export create rgw ...'
The cephfs variant already has it.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
ad5fe4452d8ae5226f0d9bda918b048078b548e0 )
Sage Weil [Mon, 19 Jul 2021 17:46:38 +0000 (13:46 -0400)]
mgr/nfs: use bucket owner creds for rgw bucket export
The bucket owner can always read/write to the bucket, so use those creds
for the export. This is less complicated than setting up a dedicated
user anyway.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
7d2f7efe3fd36e88e5ba9469cc918c4e915b7e82 )
Sage Weil [Fri, 16 Jul 2021 19:29:15 +0000 (15:29 -0400)]
mgr/cephadm: use new CEPH_IMAGE_TYPES for all daemons using ceph container image
We were using CEPH_TYPES + GATEWAY_TYPES, but that isn't really accurate.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
deaff0c42cb362b21a03dc76cff551741c761561 )
Sage Weil [Thu, 15 Jul 2021 20:33:48 +0000 (15:33 -0500)]
qa/tasks/python: simple task to run python code
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
45737fe95ac3a2b53dda466179e426c2ce574fb6 )