]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Kefu Chai [Tue, 24 Nov 2020 06:12:54 +0000 (14:12 +0800)]
cmake: define BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT for Boost.Asio users
see also
https://www.boost.org/doc/libs/1_74_0/doc/html/boost_asio/std_executors.html#boost_asio.std_executors.polymorphic_i_o_executor
we could use `asio::any_io_executor` later on though for better
performance.
also, define CMP0093, so FindBoost reports Boost_VERSION in x.y.z
format. it is simpler to use `VERSION_GREATER_EQUAL` to compare its
version with 1.74 instead of its C macro version ("107000").
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
3d708219092d0e89a1434c30ffc8a4999f062cc0 )
Conflicts:
CMakeLists.txt
src/librbd/CMakeLists.txt: trivial resolutions
Yuri Weinstein [Tue, 22 Dec 2020 17:44:57 +0000 (09:44 -0800)]
Merge pull request #37362 from smithfarm/wip-47085-octopus
octopus: mds: dump granular cap info in mds_sessions
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Tue, 22 Dec 2020 16:47:40 +0000 (08:47 -0800)]
Merge pull request #38553 from batrick/i47095
octopus: mds: provide altrenatives to increase the total cephfs subvolume snapshot counts to greater than the current 400 across a Cephfs volume
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Tue, 22 Dec 2020 16:46:39 +0000 (08:46 -0800)]
Merge pull request #38551 from batrick/i48372
octopus: client: dump which fs is used by client for multiple-fs
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Yuri Weinstein [Tue, 22 Dec 2020 16:45:38 +0000 (08:45 -0800)]
Merge pull request #38550 from batrick/i48524
octopus: qa: accept timeout argument in run_shell
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Tue, 22 Dec 2020 16:44:49 +0000 (08:44 -0800)]
Merge pull request #38202 from Vicente-Cheng/wip-48111-octopus
octopus: doc: document MDS cache configuration
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Tue, 22 Dec 2020 16:44:10 +0000 (08:44 -0800)]
Merge pull request #38095 from kotreshhr/wip-48191-octopus
octopus: mds: throttle cap acquisition via readdir
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Tue, 22 Dec 2020 16:43:37 +0000 (08:43 -0800)]
Merge pull request #37989 from Vicente-Cheng/wip-48109-octopus
octopus: cephfs: client: increment file position on _read_sync near eof
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Tue, 22 Dec 2020 16:43:08 +0000 (08:43 -0800)]
Merge pull request #37987 from Vicente-Cheng/wip-48098-octopus
octopus: cephfs: osdc: restart read on truncate/discard
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Tue, 22 Dec 2020 16:42:33 +0000 (08:42 -0800)]
Merge pull request #37985 from Vicente-Cheng/wip-48096-octopus
octopus: cephfs: mds: move start_files_to_recover() to recovery_done
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Tue, 22 Dec 2020 16:41:49 +0000 (08:41 -0800)]
Merge pull request #37690 from smithfarm/wip-47659-octopus
octopus: qa: ignore logrotate state rename error
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Thu, 17 Dec 2020 23:32:28 +0000 (15:32 -0800)]
Merge pull request #37794 from ifed01/wip-ifed-fix-alloc-multi-reg-octopus
octopus: bluestore: provide a different name for fallback allocator
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Yuri Weinstein [Thu, 17 Dec 2020 20:53:07 +0000 (12:53 -0800)]
Merge pull request #37860 from smithfarm/wip-47708-octopus
octopus: bluestore: Support flock retry
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Yuri Weinstein [Thu, 17 Dec 2020 18:18:55 +0000 (10:18 -0800)]
Merge pull request #38338 from smithfarm/wip-48127-octopus
octopus: rgw: keep syncstopped flag when copying bucket shard headers
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 17 Dec 2020 18:14:44 +0000 (10:14 -0800)]
Merge pull request #37766 from smithfarm/wip-47963-octopus
octopus: cephadm: Revert "spec: Podman (temporarily) requires apparmor-abstractions on suse"
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Yuri Weinstein [Thu, 17 Dec 2020 17:27:47 +0000 (09:27 -0800)]
Merge pull request #38344 from smithfarm/wip-48341-octopus
octopus: pybind/mgr/rbd_support: delay creation of progress module events
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Thu, 17 Dec 2020 17:25:40 +0000 (09:25 -0800)]
Merge pull request #38343 from smithfarm/wip-48225-octopus
octopus: librbd: update hidden global config when removing pool config override
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Thu, 17 Dec 2020 17:25:04 +0000 (09:25 -0800)]
Merge pull request #38342 from smithfarm/wip-48086-octopus
octopus: mgr/rbd_support: store global schedule without localized prefix
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Thu, 17 Dec 2020 17:24:29 +0000 (09:24 -0800)]
Merge pull request #38341 from smithfarm/wip-48085-octopus
octopus: qa/workunits/rbd: fix permission issue when removing mirror peer
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Thu, 17 Dec 2020 17:23:51 +0000 (09:23 -0800)]
Merge pull request #38340 from smithfarm/wip-48084-octopus
octopus: global: reexpand conf meta in child process
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Thu, 17 Dec 2020 17:22:23 +0000 (09:22 -0800)]
Merge pull request #37861 from smithfarm/wip-47892-octopus
octopus: bluestore: attach csum for compressed blobs
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Yuri Weinstein [Thu, 17 Dec 2020 17:21:17 +0000 (09:21 -0800)]
Merge pull request #37859 from smithfarm/wip-47749-octopus
octopus: build/ops: doc: pass --use-feature=2020-resolver to pip
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Thu, 17 Dec 2020 16:58:46 +0000 (08:58 -0800)]
Merge branch 'octopus' into wip-47963-octopus
Jenkins Build Slave User [Wed, 16 Dec 2020 17:29:50 +0000 (17:29 +0000)]
15.2.8
Patrick Donnelly [Wed, 16 Dec 2020 16:21:26 +0000 (08:21 -0800)]
Merge branch 'CVE-2020-27781-octopus' into octopus
* CVE-2020-27781-octopus:
tasks/cephfs/test_volume_client: Add tests for authorize/deauthorize
pybind/ceph_volume_client: Optionally authorize existing auth-ids
pybind/ceph_volume_client: Preserve existing caps while authorize/deauthorize auth-id
pybind/ceph_volume_client: Disallow authorize auth_id
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Kotresh HR [Tue, 1 Dec 2020 10:44:17 +0000 (16:14 +0530)]
tasks/cephfs/test_volume_client: Add tests for authorize/deauthorize
1. Add testcase for authorizing auth_id which is not added by
ceph_volume_client
2. Add testcase to test 'allow_existing_id' option
3. Add testcase for deauthorizing auth_id which has got it's caps
updated out of band
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
aa4beb3d993649a696af95cf27150cc460baaf70 )
Conflicts:
qa/tasks/cephfs/test_volume_client.py
Kotresh HR [Sun, 6 Dec 2020 07:10:20 +0000 (12:40 +0530)]
pybind/ceph_volume_client: Optionally authorize existing auth-ids
Optionally allow authorizing auth-ids not created by ceph_volume_client
via the option 'allow_existing_id'. This can help existing deployers
of manila to disallow/allow authorization of pre-created auth IDs
via a manila driver config that sets 'allow_existing_id' to False/True.
Fixes: https://tracker.ceph.com/issues/48555
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
77b42496e25cbd4af2e80a064ddf26221b53733f )
Kotresh HR [Thu, 26 Nov 2020 09:18:16 +0000 (14:48 +0530)]
pybind/ceph_volume_client: Preserve existing caps while authorize/deauthorize auth-id
Authorize/Deauthorize used to overwrite the caps of auth-id which would
end up deleting existing caps. This patch fixes the same by retaining
the existing caps by appending or deleting the new caps as needed.
Fixes: https://tracker.ceph.com/issues/48555
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
47100e528ef77e7e82dc9877424243dc6a7e7533 )
Ramana Raja [Wed, 25 Nov 2020 11:14:35 +0000 (16:44 +0530)]
pybind/ceph_volume_client: Disallow authorize auth_id
This patch disallow the ceph_volume_client to authorize the auth_id
which is not created by ceph_volume_client. Those auth_ids could be
created by other means for other use cases which should not be modified
by ceph_volume_client.
Fixes: https://tracker.ceph.com/issues/48555
Signed-off-by: Ramana Raja <rraja@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
3a85d2d04028a323952a31d18cdbefb710be2e2b )
Ernesto Puerta [Mon, 14 Dec 2020 13:37:20 +0000 (14:37 +0100)]
Merge pull request #37973 from callithea/wip-47607-octopus
octopus: mgr/dashboard/api: move/create OSD histogram in separate endpoint
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Ernesto Puerta [Mon, 14 Dec 2020 13:36:45 +0000 (14:36 +0100)]
Merge pull request #38405 from callithea/wip-48343-octopus
octopus: mgr/dashboard: Disable sso without python3-saml
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Ernesto Puerta [Mon, 14 Dec 2020 13:35:43 +0000 (14:35 +0100)]
Merge pull request #38331 from votdev/wip-48396-octopus
octopus: mgr/dashboard: Disable TLS 1.0 and 1.1
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Ernesto Puerta [Mon, 14 Dec 2020 13:34:56 +0000 (14:34 +0100)]
Merge pull request #38094 from bk201/wip-48226-octopus
octopus: mgr/dashboard: Use pipe instead of calling function within template
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Yan, Zheng [Wed, 5 Aug 2020 15:31:03 +0000 (23:31 +0800)]
qa/workunits/fs: add test for subvolume
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
df7e9aa42b97f3203e92387d2ceea5636f5230ad )
Yan, Zheng [Wed, 5 Aug 2020 09:31:57 +0000 (17:31 +0800)]
mds: don't move inode with nlink > 1 to global snaprealm if it's in subvolume
Fixes: https://tracker.ceph.com/issues/46074
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
63da28d6a8be20183a294744892a41f47e8ba24b )
Conflicts: resolution similar to nautilus backport
src/mds/Server.cc
Yan, Zheng [Wed, 5 Aug 2020 08:04:30 +0000 (16:04 +0800)]
mds: disallow hardlink across subvolume
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
55cc3f7d7d27837bfe64ee46444358c0e45d16f6 )
Yan, Zheng [Wed, 5 Aug 2020 07:37:20 +0000 (15:37 +0800)]
mds: disallow across subvolume rename
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
53348ab67f0f3e691cfeb6b02d80b868fdb98d89 )
Yan, Zheng [Wed, 5 Aug 2020 07:33:10 +0000 (15:33 +0800)]
mds: disallow creating snapshot on descendent directory of subvolume
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
34c6b4437a2e8a49bb76e86d07cd86b740e3f581 )
Yan, Zheng [Tue, 4 Aug 2020 13:26:26 +0000 (21:26 +0800)]
mds: add vxattr that marks/clears subvolume flag
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
8d531680cdf48ba43d93becd5a19072844e92c19 )
Conflicts: resolutions similar to nautilus backport
src/mds/Server.cc
src/mds/SnapRealm.h
src/mds/snap.h
Yuri Weinstein [Sat, 12 Dec 2020 21:50:22 +0000 (13:50 -0800)]
Merge pull request #37728 from ideepika/wip-octopus-tracker-47915
octopus: qa: drop hammer branch qa tests
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Yuri Weinstein [Sat, 12 Dec 2020 21:47:18 +0000 (13:47 -0800)]
Merge pull request #38345 from smithfarm/wip-48228-octopus
octopus: mon: Log "ceph health detail" periodically in cluster log
Reviewed-by: Neha Ojha <nojha@redhat.com>
Zhi Zhang [Mon, 16 Nov 2020 10:01:40 +0000 (18:01 +0800)]
client: dump which fs is used by client for multiple-fs
Fixes: https://tracker.ceph.com/issues/48246
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
(cherry picked from commit
8daff24c2be2a7c54e569d0abf964986d9a472d4 )
Conflicts:
src/client/Client.cc: surrounding changes unrelated
Patrick Donnelly [Fri, 11 Dec 2020 23:49:40 +0000 (15:49 -0800)]
qa: accept timeout argument in run_shell
This was fixed differently in master in a way that's not easy to
backport.
Fixes: https://tracker.ceph.com/issues/48524
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Deepika Upadhyay [Thu, 15 Oct 2020 12:02:06 +0000 (17:32 +0530)]
qa: drop hammer branch qa tests
fixes: https://tracker.ceph.com/issues/47731
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
(cherry picked from commit
7ef18559cbf68c7a09d73c528db600a3c959ac2c )
Neha Ojha [Fri, 24 Apr 2020 18:45:19 +0000 (18:45 +0000)]
qa/suites/rados/thrash-old-clients/1-install: remove hammer
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit
c258ff2c15aa295221d9c2c91af573ed9fbaaae7 )
Jan Fajerski [Wed, 9 Dec 2020 16:01:39 +0000 (17:01 +0100)]
Merge PR #38205 into octopus
* refs/pull/38205/head:
ceph-volume: pass *-slots arguments to LV creation
use extent count for slots conversion instead of free count
ceph-volume: available_lvm: vg space takes precedence
Jan Fajerski [Wed, 9 Dec 2020 15:57:32 +0000 (16:57 +0100)]
Merge PR #38014 into octopus
* refs/pull/38014/head:
ceph-volume: consume mount opt in simple activate
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Jan Fajerski [Wed, 9 Dec 2020 15:56:45 +0000 (16:56 +0100)]
Merge PR #38047 into octopus
* refs/pull/38047/head:
ceph-volume: fix lvm help test
ceph-volume: remove mention of dmcache from docs and help text
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Jan Fajerski [Wed, 9 Dec 2020 15:55:47 +0000 (16:55 +0100)]
Merge PR #38199 into octopus
* refs/pull/38199/head:
ceph-volume: fix test_setup_device_device_name_is_none
ceph-volume: cover devices.lvm.prepare.setup_device
ceph-volume: fix filestore/dmcrypt activate
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Jan Fajerski [Wed, 9 Dec 2020 15:54:56 +0000 (16:54 +0100)]
Merge PR #38280 into octopus
* refs/pull/38280/head:
ceph-volume batch: reject partitions in argparser
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Jan Fajerski [Wed, 9 Dec 2020 15:54:12 +0000 (16:54 +0100)]
Merge PR #38426 into octopus
* refs/pull/38426/head:
ceph-volume: implement the --log-level flag
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Dimitri Savineau [Mon, 9 Nov 2020 15:50:20 +0000 (10:50 -0500)]
ceph-volume: fix lvm help test
ed5ceb0 changed the LVM help code but not the associated test.
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit
879ed30984de2b94879959de1c3611083c85bd99 )
Andrew Schoen [Wed, 21 Oct 2020 15:55:46 +0000 (10:55 -0500)]
ceph-volume: remove mention of dmcache from docs and help text
With the introduction of bluestore dmcache is no longer needed and
is no longer supported with `ceph-volume lvm`.
Resolves: rhbz#
1876827
Fixes: https://tracker.ceph.com/issues/48039
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
ed5ceb04fc8ff57c5f7e2b5fa5e859c2cdbf2ffd )
Sebastian Wagner [Mon, 7 Dec 2020 09:03:18 +0000 (10:03 +0100)]
Merge pull request #38155 from sebastian-philipp/octopus-backport-37432-37397-37112-36622-37411-37521-37500-37104-37542-37650-37621-3764
octopus: cephadm batch backport November
Reviewed-by: Michael Fritch <mfritch@suse.com>
Patrick Donnelly [Mon, 26 Oct 2020 22:38:25 +0000 (15:38 -0700)]
doc: document MDS cache configuration
Fixes: https://tracker.ceph.com/issues/48010
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
cb9b2e5815529f2be28a136462c57d86f004657f )
Patrick Donnelly [Mon, 26 Oct 2020 22:37:24 +0000 (15:37 -0700)]
test: add DecayCounter test
In particular, look at the steady state decay. (i.e. a counter that is
continuously "refilled".)
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
ce151e3e472da75d707fb12ab8c7c32cb65c9248 )
Patrick Donnelly [Mon, 2 Nov 2020 20:01:29 +0000 (12:01 -0800)]
client: increment file position on _read_sync near eof
Fixes: https://tracker.ceph.com/issues/48076
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
83f84f601aada043fdece5c733a85230786545cb )
Patrick Donnelly [Tue, 20 Oct 2020 19:38:15 +0000 (12:38 -0700)]
osdc: restart read on truncate/discard
This can be reliably reproduced by reading a large file (~500MB) on a
ceph-fuse mount and then evicting the client.
Now we see in the logs:
2020-10-21T01:46:29.679+0000
7f00beffd700 10 objectcacher.object(
10000000000 .
00000030 /head) truncate object[
10000000000 .
00000030 /head oset 0x7f00a0006b08 wr 0/0] to 0
2020-10-21T01:46:29.679+0000
7f00beffd700 10 objectcacher.object(
10000000000 .
00000030 /head) restarting reads post-truncate
...
2020-10-21T01:46:29.762+0000
7f00bffff700 20 objectcacher.object(
10000000000 .
00000030 /head) map_read error bh[ 0x7f0070004e30
1048576 ~
3145728 0x7f008404a720 (0) v 0 error=-108] waiters = {}
2020-10-21T01:46:29.762+0000
7f00bffff700 10 objectcacher readx hit bh bh[ 0x7f0070004e30
1048576 ~
3145728 0x7f008404a720 (0) v 0 error=-108] waiters = {}
2020-10-21T01:46:29.762+0000
7f00bffff700 10 objectcacher readx has all buffers
2020-10-21T01:46:29.762+0000
7f00bffff700 20 objectcacher readx done 0x7f007c00a9b0 -108
...
2020-10-21T01:46:29.763+0000
7f0077fff700 3 client.4497 ll_read 0x7f0080023720
202637312 ~131072 = -108
Fixes: https://tracker.ceph.com/issues/46434
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
b53390031378a06980fa8e7dae074a7095307c12 )
Conflicts:
src/osdc/ObjectCacher.cc
- keep the specific data type instead using auto
Simon Gao [Sun, 9 Aug 2020 07:38:30 +0000 (15:38 +0800)]
mds : move start_files_to_recover() to recovery_done
the requests in the queue, named waiting_for_replay, may modify the state of the filelock,
resulting in the wrong lock state when repairing file (start_files_to_recover)
Fixes : https://tracker.ceph.com/issues/46906
Signed-off-by: Simon Gao <simon29rock@gmail.com>
(cherry picked from commit
fafb5b4f84e12ba00a68550ffb73fc9bcde867a0 )
Yuri Weinstein [Sat, 5 Dec 2020 21:06:40 +0000 (13:06 -0800)]
Merge pull request #38287 from yuriw/wip-yuriw-cleanup-octopus
octopus: qa/tests: added rhel 8.2
Reviewed-by: Rakesh Manohar <rgowdege@redhat.com>
Xiubo Li [Mon, 23 Nov 2020 12:55:01 +0000 (20:55 +0800)]
common: do not dup the options when reexpanding
The old code will store all the options, which has `$pid` in them,
in may_reexpand_meta map. And when reexpanding later, the reexpand
code will dup them with a higher priority(CONF_OVERRIDE).
This will be a problem, if the default value has `$pid` and be
stored in the may_reexpand_meta map, and then the code set a new
different value, which may have no `$pid`, from CLI or config file.
The reexpand will override it with the default value always.
This will do not duplicate the options with CONF_OVERRIDE priority
when reexpanding, just refresh them and call the observers. And the
finalize_reexpand_meta() will always be called after the fork() is
done in child processes.
Fixes: https://tracker.ceph.com/issues/48240
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit
56ca87ae3b3c341a78a8df8f95d3bf31828c9108 )
Xiubo Li [Mon, 23 Nov 2020 08:27:51 +0000 (16:27 +0800)]
common/config_proxy: clear the values.changed in _gather_changes()
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit
7496be719c3419224ac7fc29d6db4e282b193554 )
Xiubo Li [Fri, 13 Nov 2020 08:08:31 +0000 (16:08 +0800)]
global: reexpand the conf meta in all the child processes
Especially for the tools or the daemons whose config options need
to expand the '$pid', they will be always expanded with the parent
processes. We need to reexpand them in child processes just after
the fork is done.
Fixes: https://tracker.ceph.com/issues/48240
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit
73a48dfe8ed165cb6c8295a88f133acb30e7065e )
Conflicts:
PendingReleaseNotes
Dimitri Savineau [Mon, 26 Oct 2020 19:12:59 +0000 (15:12 -0400)]
ceph-volume: consume mount opt in simple activate
When running ceph-volume simple activate command on a Filestore OSD
then the data device is mounted without any specific options so the
one from the ceph configuration file are ignored.
When deploying Filestore with the lvm subcommand then everything is
fine because the filestore_activate method uses mount_osd which relies
on the mount options defined in the ceph configuration file (if any).
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1891557
Fixes: https://tracker.ceph.com/issues/48018
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit
1f4301a15df82bf31468d76fbcccc1c5fa192e38 )
Patrick Donnelly [Mon, 28 Sep 2020 18:03:21 +0000 (11:03 -0700)]
qa: fix proc exit status check
Fixes: f30a84b6a7a21670d8b5c5a7fe19df70e4371260
Fixes: https://tracker.ceph.com/issues/47677
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
eb95dabc3f5cac18c4efd33f595bbf9799901f33 )
Patrick Donnelly [Sun, 20 Sep 2020 21:05:13 +0000 (14:05 -0700)]
qa: ignore logrotate state rename error
This is hopefully a transient issue that can be ignored.
Fixes: https://tracker.ceph.com/issues/42433
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
f30a84b6a7a21670d8b5c5a7fe19df70e4371260 )
Aashish Sharma [Wed, 16 Sep 2020 13:43:04 +0000 (19:13 +0530)]
mgr/dashboard/api: move/create OSD histogram in separate endpoint
Added a separate endpoint for osd/histogram - api/osd/{svc_id}/histogram
Fixes:https://tracker.ceph.com/issues/46898
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit
1c812e0d61ec8320cb5d3864199af9c31b2aa238 )
Andrew Schoen [Thu, 29 Oct 2020 19:11:14 +0000 (14:11 -0500)]
ceph-volume: implement the --log-level flag
The --log-level flag was being ignored and
the file log level was always set to DEBUG.
Fixes: https://tracker.ceph.com/issues/48045
Resolves: rhbz#
1867717
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
ecbd6c13f116b390c782c9ae14b5becd0bdecc8e )
David Zafman [Thu, 3 Dec 2020 01:38:11 +0000 (17:38 -0800)]
Merge pull request #38359 from dzafman/wip-47767-octopus
octopus: osd: Check for nosrub/nodeep-scrub in between chunks, to avoid races
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Brad Hubbard [Wed, 2 Dec 2020 22:05:55 +0000 (08:05 +1000)]
Merge pull request #38361 from badone/wip-octopus-mon-scrub-testing
octopus: mon scrub testing
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Brad Hubbard [Wed, 2 Dec 2020 20:28:26 +0000 (06:28 +1000)]
Merge pull request #38396 from badone/wip-octopus-run-tox-mgr-insights-six-missing
octopus: mgr/insights: Test environment requires 'six'
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Kevin Meijer [Sat, 14 Nov 2020 18:44:07 +0000 (19:44 +0100)]
mgr/dashboard: Disable sso without python3-saml
Removed the requirement for the python3-saml package when wanting to disable SSO for the dashboard, this is currently relevant since the official container that runs Ceph mgr does not have this package installed.
So when upgrading from an older, non-containerized version, you would be stuck using a non-functional dashboard.
This pull requests changes that and allows the ceph dashboard sso disable command without the requirement of the library so that we SSO can always be disabled again.
Fixes: https://tracker.ceph.com/issues/48237 Signed-off-by: Kevin Meijer <admin@kevinmeijer.nl>
(cherry picked from commit
0c18437d2c786ef1ade8b89e42dbf4b0e163aafe )
Brad Hubbard [Wed, 2 Dec 2020 05:24:16 +0000 (15:24 +1000)]
mgr/insights: Test environment requires 'six'
Not a backport because python2 support was dropped in master and only
nautilus and octopus seem to be affected at this time.
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
Volker Theile [Wed, 2 Dec 2020 08:54:24 +0000 (09:54 +0100)]
mgr/dashboard: Disable TLS 1.0 and 1.1
Disable these TLS versions because of security issues.
Fixes: https://tracker.ceph.com/issues/48360
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit
5cf222b6d2a5cdc7ff23357b12878d34a4e68801 )
wanghongxu [Wed, 15 Apr 2020 11:52:40 +0000 (19:52 +0800)]
bluestore: Support flock retry
When the block changes, systemd-udevd will open the block,
read some information and close it. Then a failure occurs here.
So we need to try again here.
Fixes: https://tracker.ceph.com/issues/46124
Signed-off-by: wanghongxu <x.ken@outlook.com>
(cherry picked from commit
23f8c323d2ba9133c3ef5834ae93115425c862d1 )
Igor Fedotov [Tue, 15 Sep 2020 20:06:20 +0000 (23:06 +0300)]
os/bluestore: attach csum for compressed blobs
Fixes: https://tracker.ceph.com/issues/47475
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
53346d06beb4de971703e63627444bca40a1fa44 )
Josh Durgin [Tue, 1 Dec 2020 17:36:51 +0000 (09:36 -0800)]
Merge remote-tracking branch 'origin/octopus-saved' into octopus
Brad Hubbard [Wed, 11 Nov 2020 01:58:01 +0000 (11:58 +1000)]
qa/suites/rados/monthrash: Exercise mon scrub error injectors
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
a5bcca7f415790521a76213620ae079318e7bee1 )
Conflicts:
qa/suites/rados/monthrash/ceph.yaml - whitelist vs. ignorelist
Brad Hubbard [Wed, 11 Nov 2020 01:49:41 +0000 (11:49 +1000)]
qa/config/rados.yaml: Test mon scrub
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
f85001e5d5fb11718ab2fd8b708402cd2db951d4 )
David Zafman [Tue, 24 Nov 2020 23:23:20 +0000 (15:23 -0800)]
osd: Check for nosrub/nodeep-scrub in between chunks, to avoid races
Fixes: https://tracker.ceph.com/issues/47767
Signed-off-by: David Zafman <dzafman@redhat.com>
Igor Fedotov [Mon, 14 Sep 2020 20:28:42 +0000 (23:28 +0300)]
os/bluestore: provide a different name for fallback allocator
Originally primary Hybrid allocator provided its own name when creating a
secondary fallback allocator. This resulted in duplicate admin socket
command registrations for both allocator. Registration return code was
ignored and henoe nobody was aware of the issue. Nautilus might suffer
from the issue though since it asserts on command deregistration failure.
And duplicate name causes such a failure for the secode
unregister_command() call.
Fixes: https://tracker.ceph.com/issues/47443
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
b0866b60461b06e6563cad47d0ad3ce9302114f5 )
Jenkins Build Slave User [Mon, 30 Nov 2020 19:58:30 +0000 (19:58 +0000)]
15.2.7
Casey Bodley [Mon, 23 Nov 2020 23:06:26 +0000 (18:06 -0500)]
rgw: temporarily disable calls to defer_gc() in RGWGetObj
cls_rgw_gc_queue_update_entry() is known to cause data loss when called
on objects that have not actually been scheduled for garbage collection
RGWGetObj is the only caller, and uses defer_gc() when reads are taking
a long time compared to rgw_gc_obj_min_wait. if an object has since been
deleted and submitted for garbage collection, this allows RGWGetObj to
defer that gc until the entire read completes
by disabling these calls to defer_gc(), very long reads (longer than 1hr,
with default configuration) may fail if the object gets deleted, and a
retry will result in a 404 Not Found error as expected
Fixes: https://tracker.ceph.com/issues/47866
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
94df9cd37a1ca457130f90803281b166a5fa7eef )
J. Eric Ivancich [Sat, 21 Nov 2020 16:10:35 +0000 (11:10 -0500)]
rgw: during GC defer, prevent new GC enqueue
With the new queue-based GC code, when a GC defer operation is
performed, it adds an "urgent" record to prevent GC from removing
objects that are still being read. It does not check whether the
objects are on the GC queue or not and that's OK for the urgent
record.
The code *also* adds a new GC entry to the queue to cause GC to occur
at a later time. This would be incorrect if there was no GC entry to
begin with, however. In such a case this would cause GC to delete tail
objects when no user-initiated remove has happend. In other words a
READ could cause a DELETE of tail objects and therefore data loss.
This fix prevents such a new GC entry from being enqueued, thus
preventing the data loss in this rare case. There is a new risk that
tail object orphans to be created, but as an immediate fix to prevent
data loss, this is appropriate and it is a rare event. A follow-on PR
that will handle these cases is likely.
This PR adds a level 0 log entry as a way to potentially confirm this
case is being triggered in real-world cases. In time, this log entry
should be deleted.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit
2603485bcb4402260e0f7aadd2f2c8ab05b07399 )
Prashant D [Fri, 30 Oct 2020 10:40:43 +0000 (06:40 -0400)]
mon: Log "ceph health detail" periodically in cluster log
change mon_health_to_clog_interval from 1_hr -> 10_min to
log health summary or detail more frequently.
Fixes: https://tracker.ceph.com/issues/48042
Signed-off-by: Prashant Dhange <pdhange@redhat.com>
(cherry picked from commit
f45712c19077c5cf5a9938fc3fd17b64ffe3a4ec )
Conflicts:
PendingReleaseNotes
- next Octopus release is 15.2.8 now
Jason Dillaman [Thu, 19 Nov 2020 20:59:34 +0000 (15:59 -0500)]
pybind/mgr/rbd_support: delay creation of progress event
Create the progress module event upon receipt of the first
progress callback from the librbd API. This will help to ensure
that all prereqs have been validated for retryable errors like
scheduling an image to be removed while it still has attached
cloned children.
Fixes: https://tracker.ceph.com/issues/48296
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
c0069b00e4748974d4bf5cfa1bdab68d6f043abb )
Jason Dillaman [Thu, 19 Nov 2020 21:08:22 +0000 (16:08 -0500)]
pybind/mgr/rbd_support: include retryable error message in task
If a retryable error occurs, include the reason message in the
task JSON structure that is exposed via the "rbd task list"
MGR command.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
66ddfee8590769935f260becee5e8fc070bce66d )
Jason Dillaman [Thu, 19 Nov 2020 20:25:52 +0000 (15:25 -0500)]
pybind/mgr/rbd_support: add linear backoff to retried tasks
When a task fails but will be retried, linearly increase the
delay for the next retry up to a maximum of 5 minutes.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
b8a4685cf9bf82a1a072e0de635a8b2a1a6973ba )
Jason Dillaman [Sat, 7 Nov 2020 15:09:32 +0000 (10:09 -0500)]
librbd: update hidden global config when removing pool config override
The remove notification was missed and therefore in-use images will not
properly remove pool config overrides.
Fixes: https://tracker.ceph.com/issues/48145
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
4ca1c4912b52d6a7945decd443ddcaf3533dcdf9 )
Mykola Golub [Wed, 28 Oct 2020 09:20:27 +0000 (09:20 +0000)]
mgr/rbd_support: backward compatibility when loading global schedule
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit
8e9e2a73f5b8abd743933e59a3b986598765a681 )
Mykola Golub [Tue, 27 Oct 2020 18:00:52 +0000 (18:00 +0000)]
mgr/rbd_support: store global schedule without localized prefix
so it is still used after mgr failover
Fixes: https://tracker.ceph.com/issues/48020
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit
e46e80474f07af2b9324f9ec0e893ba6b5e72033 )
Jason Dillaman [Wed, 28 Oct 2020 18:06:57 +0000 (14:06 -0400)]
qa/workunits/rbd: fix permission issue when removing mirror peer
Fixes: https://tracker.ceph.com/issues/48032
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
caaba8eabecd3911ba8bc4233891160134b7f2eb )
Xiubo Li [Fri, 30 Oct 2020 02:02:51 +0000 (22:02 -0400)]
rbd-nbd: reexpand the conf meta in child process
Fixed: https://tracker.ceph.com/issues/48046
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit
f482b8ca7973c998cfa02ef84aba92709aa1f49b )
Ilsoo Byun [Thu, 29 Oct 2020 08:10:45 +0000 (17:10 +0900)]
rgw: keep syncstopped flag when copying bucket shard headers
Fixes: https://tracker.ceph.com/issues/48037
Signed-off-by: Ilsoo Byun <ilsoobyun@linecorp.com>
(cherry picked from commit
5a94be030b67085ef08c7b3a4450d3d19420a9c4 )
Brad Hubbard [Fri, 27 Nov 2020 03:17:33 +0000 (13:17 +1000)]
Merge pull request #38294 from badone/wip-octopus-dont-run-tests-if-build-fails
octopus: run-make-check.sh: Don't run tests if build fails
Reviewed-by: Kefu Chai <kchai@redhat.com>
Brad Hubbard [Thu, 26 Nov 2020 05:30:55 +0000 (15:30 +1000)]
run-make-check.sh: Make sure a build failure will exit
We 'set -e' but that is ignored because 'build tests' is executed in a
'&&' list (see 'man set') so move the echo to the following line.
Follow-up to
03ff2146f95
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
e70483133db87a3f04bc1fff31d8472465c305b3 )
Conflicts:
run-make-check.sh - Trivial change in logging output
Brad Hubbard [Thu, 26 Nov 2020 01:18:02 +0000 (11:18 +1000)]
run-make-check.sh: Don't run tests if build fails
When run-make was taken out we lost the 'set -e' call and therefore
continue after an error.
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
03ff2146f95c7e03a84df1f8c3b38bbbb315b708 )
Ernesto Puerta [Thu, 26 Nov 2020 12:52:49 +0000 (13:52 +0100)]
Merge pull request #38189 from votdev/issue_48295_widget
octopus: mgr/dashboard: The performance 'Client Read/Write' widget shows incorrect write values
Reviewed-by: avanthakkar <athakkar@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: nizamial09 <nia@redhat.com>
Ernesto Puerta [Thu, 26 Nov 2020 12:50:59 +0000 (13:50 +0100)]
Merge pull request #37763 from bk201/wip-47943-octopus
octopus: mgr/dashboard: Merge disable and disableDesc
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Jan Fajerski [Thu, 26 Nov 2020 12:48:23 +0000 (13:48 +0100)]
Merge PR #38299 into octopus
* refs/pull/38299/head:
ceph-volume inventory: make libstoragemgmt data retrieval optional
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>