]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Sebastian Wagner [Mon, 18 Jan 2021 15:02:38 +0000 (16:02 +0100)]
mgr/orchestrator: disallow_untyped_defs = True
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
c95ba878c66aae44816b1451049449685d444f0a )
Sebastian Wagner [Tue, 5 Jan 2021 11:58:08 +0000 (12:58 +0100)]
pybind/mgr: disallow_untyped_defs=True for mgr_util
(increases test coverage a tiny bit)
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
e308bf8989556617985716aea0c3bfb5da95799f )
Sebastian Wagner [Mon, 18 Jan 2021 15:02:38 +0000 (16:02 +0100)]
mgr/orchestrator: disallow_untyped_defs = True
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
c95ba878c66aae44816b1451049449685d444f0a )
Sage Weil [Wed, 20 Jan 2021 18:34:55 +0000 (12:34 -0600)]
mgr/devicehealth: fix 'device monitoring on|off' return value
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
0ee2b2b807706c3e53d4edc7f2ca0211b2a454da )
Patrick Donnelly [Thu, 14 Jan 2021 17:52:22 +0000 (09:52 -0800)]
cephadm: fix rgw osd cap tag
The syntax is "allow rwx tag rgw *=*'.
Sorry, I thought this would have gotten caught in testing :(
Fixes: 373cc847cf0f8b4ec7aefbfe64c01c3f18a4e021
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
de1efbd62b9258630c2a2d55bfd12034cc8b603f )
Sebastian Wagner [Mon, 18 Jan 2021 15:02:38 +0000 (16:02 +0100)]
mgr/orchestrator: disallow_untyped_defs = True
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
c95ba878c66aae44816b1451049449685d444f0a )
Sebastian Wagner [Thu, 14 Jan 2021 11:04:03 +0000 (12:04 +0100)]
cephadm: Don't make sysctl spam the log file
`sysctl -a` really spams the log file in Teuthology
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
70722a2efb5207f7d6fc7e8ecf4aab71e2a1a39d )
Adam King [Fri, 15 Jan 2021 01:04:27 +0000 (20:04 -0500)]
mgr/cephadm: fix ha-rgw removal
removing the ha-rgw service from a host was failing due to some
issues with daemon type <-> service type conversions
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit
657027088e4d8be4789496020c034aa2837a07b7 )
Sage Weil [Sat, 23 Jan 2021 17:20:52 +0000 (11:20 -0600)]
qa/distros: add ubuntu 20.04 targets
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
24c2d2b1244d95c232f927dad1417717a2768ece )
Sage Weil [Sat, 23 Jan 2021 17:21:58 +0000 (11:21 -0600)]
qa/distros: change ubuntu_latest to 20.04
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
a07ca4c5c0f04c43c71231a898c2a89e99a80606 )
Sage Weil [Sun, 24 Jan 2021 00:56:42 +0000 (18:56 -0600)]
qa/workunits/cephadm/test_repos: adjust target version
15.2.7 is available for 20.04; 15.1.1 is not
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
7fe3853af97120fc97406f5b8123bbaa4adc9c6e )
Sage Weil [Sat, 23 Jan 2021 17:26:06 +0000 (11:26 -0600)]
qa/suites/rados/cephadm: add 20.04 targets
Add 20.04 targets.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
8304bf6510c4477cbde438a39a9eb9f60dfd5772 )
Sage Weil [Sat, 23 Jan 2021 17:23:13 +0000 (11:23 -0600)]
qa/suites/rados/cephadm: add 20.04 target
Link directly to the distro version... no need to use _latest here since
it obscures the podman vs docker difference.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
170dd14af4f72bdac2a43742d11fa194c84819f5 )
Sage Weil [Thu, 21 Jan 2021 15:10:51 +0000 (09:10 -0600)]
cephadm: fix 2> syntax in unit.run
We need a space between the command (which ends with a container name)
and the 2> or else the 2 is considered part of the command. E.g.,
! /usr/bin/podman rm -f ceph-
a9a8c7ee -5b72-11eb-8f93-
001a4aab830c -mon.a2> /dev/null
Fixes: 1bed46e4b0094863a119df59c6ae5f254c2e211d
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
ce4743f72e6c7eea9514b8c9b6c20309fea5d455 )
Sebastian Wagner [Fri, 22 Jan 2021 10:32:40 +0000 (11:32 +0100)]
mgr/cephadm: Minor pep8 fix
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
2df47b88abd00aeee9153e8f9404b33f0c3a1d13 )
Sebastian Wagner [Fri, 22 Jan 2021 10:32:31 +0000 (11:32 +0100)]
mgr/cephadm: mypy says: dd.status can be None
`DaemonDescription.status` is annotated
as `Optional[int]` and thus can be `None`
This is a conflict between
c95ba878c66aae44816b1451049449685d444f0a
and
01f60cf4e0a751c314120c02956d4ff941eb71b4
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
fb1976950372502c5d3dcfee86288724771cffbf )
Sage Weil [Tue, 19 Jan 2021 19:33:25 +0000 (13:33 -0600)]
qa/suites/rados/cephadm/upgrade: avoid allow_ptrace with old cephadm
Older cephadm is not smart enough to not combine --cap-add=SYS_PTRACE
and --privileged, which some version of podman cannot handle.
For upgrades, leave off the allow_ptrace behavior since we may be starting
on one of those old versions.
See also https://tracker.ceph.com/issues/46429
Fixes: https://tracker.ceph.com/issues/48142
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
9eef1bcced3e04fada1b64779c549239f4eb7670 )
Joao Eduardo Luis [Wed, 20 Jan 2021 12:46:52 +0000 (11:46 -0100)]
cephadm: fix stuff
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
9e81a5dfc5abdd8caeecf42faa17111fc309b8d3 )
Joao Eduardo Luis [Sat, 16 Jan 2021 15:33:43 +0000 (14:33 -0100)]
cephadm: annotate variable type for mypy
mypy was complaining about the type of the variable being None, later on
on assignment with an int.
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
4fa175e96783f698f5b1e78515fef0016e4e026e )
Joao Eduardo Luis [Thu, 14 Jan 2021 15:35:33 +0000 (14:35 -0100)]
cephadm/test: support context class
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
5178403833dc446e198bd19cbde3ce218bdb4547 )
Joao Eduardo Luis [Fri, 15 Jan 2021 10:55:04 +0000 (09:55 -0100)]
cephadm: have context handle config options
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
c785dc56796068b5e430ad199168b44efd941ccb )
Joao Eduardo Luis [Thu, 14 Jan 2021 15:34:55 +0000 (14:34 -0100)]
cephadm: drop ctx class from where we don't need it
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
b56eb958dfef75d7e181fdbaae9a7817887a0b6a )
Joao Eduardo Luis [Thu, 31 Dec 2020 01:55:25 +0000 (01:55 +0000)]
cephadm/bootstrap: move mon net discovery to function
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
bc256983c119d5d91576e659d56af267af37b760 )
Joao Eduardo Luis [Thu, 31 Dec 2020 01:48:42 +0000 (01:48 +0000)]
cephadm: don't forget ipv6 behind
Fixes a bug introduced earlier in the patchset. It's just too hard to go
back and fix it up with the right commit.
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
13addf6f54ff256206679283e5bf0c9f0b0b8e04 )
Joao Eduardo Luis [Thu, 31 Dec 2020 01:44:44 +0000 (01:44 +0000)]
cephadm: split-off config work on bootstrap
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
b5e9c81321f9195801a794de03d02ec4f9499ba6 )
Joao Eduardo Luis [Thu, 31 Dec 2020 01:28:05 +0000 (01:28 +0000)]
cephadm: split-off dashboard setup on bootstrap
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
4d30ee7e5dafc1736375b0538e379ff4b25ec52a )
Joao Eduardo Luis [Thu, 31 Dec 2020 01:21:05 +0000 (01:21 +0000)]
cephadm: split-off ssh preparations on bootstrap
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
9ef0a6792633d246c77077685a4f3f2d9001b501 )
Joao Eduardo Luis [Thu, 31 Dec 2020 01:07:55 +0000 (01:07 +0000)]
cephadm: split-off mgr creation
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
60329c6be3bc2cefe32108068597ecb6dff39220 )
Joao Eduardo Luis [Thu, 31 Dec 2020 01:05:48 +0000 (01:05 +0000)]
cephadm: split mon prepare and create
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
1dd2dcc8ed98a6ffb7c23d059dc09ebc7229814c )
Joao Eduardo Luis [Thu, 31 Dec 2020 00:41:48 +0000 (00:41 +0000)]
cephadm: move configuration out of the way
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
adc58849feac24d61c110f8252a30cdb7fa57af3 )
Joao Eduardo Luis [Wed, 30 Dec 2020 19:29:19 +0000 (19:29 +0000)]
cephadm/bootstrap: spin-off waiting for mon
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
6cab3f02aadf8e20bb3064a1a52fa5892ff38f85 )
Joao Eduardo Luis [Wed, 30 Dec 2020 19:19:03 +0000 (19:19 +0000)]
cephadm: split some of bootstrap to other functions
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
6dec4d500817e14618682b336faba25f2df23fe6 )
Joao Eduardo Luis [Wed, 30 Dec 2020 13:23:32 +0000 (13:23 +0000)]
cephadm: fix linting/types issues
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
b3b6639b7f91c3d9463c8be2d970f09785b94630 )
Joao Eduardo Luis [Wed, 30 Dec 2020 10:22:30 +0000 (10:22 +0000)]
cephadm: split main into an init function
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
3afec2ab30c1ba0162d210d3e36773b93667a350 )
Joao Eduardo Luis [Tue, 29 Dec 2020 19:23:01 +0000 (19:23 +0000)]
cephadm: introduce context, drop global variables
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit
da309f21da880572653b39b7ea1e891cc00e9581 )
Sage Weil [Tue, 19 Jan 2021 22:49:08 +0000 (16:49 -0600)]
mgr/cephadm: raise HEALTH_WARN when cephadm daemon in 'error' state
If cephadm daemons are not happy we should raise a warning. Aside from
being an important part of the user experience, this will also help us
catch teuthology test errors.
Fixes: https://tracker.ceph.com/issues/45628
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
01f60cf4e0a751c314120c02956d4ff941eb71b4 )
Sage Weil [Tue, 19 Jan 2021 22:31:25 +0000 (16:31 -0600)]
mgr/cephadm: fix host maintenance health check update
We need to update the self.health_checks dict and use that or else we
will clobber any other exising checks.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
315f58bee20154a7e83cb65f301dce4e22f9c9f6 )
Sage Weil [Tue, 19 Jan 2021 16:12:58 +0000 (10:12 -0600)]
cephadm: latest stable release is now pacific
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
da803ea8a7aee1bfce0089a4cff722cff3230104 )
Sebastian Wagner [Mon, 18 Jan 2021 15:27:18 +0000 (16:27 +0100)]
mgr/orchestrator: ignore Liskov substitution principle violation
Right now, we're violating the Liskov substitution principle by deriving from `Orchestrator` but `process` takes a sub class of `Completion`:
See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
The idea is to make Orchestrator a type constructor with `CompletionT` as argument, but this is not supported by mypy: https://github.com/python/typing/issues/548
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
9923d2a1193c374b9c2e00b5ee69023cb72a9f00 )
Sebastian Wagner [Mon, 18 Jan 2021 15:02:38 +0000 (16:02 +0100)]
mgr/orchestrator: disallow_untyped_defs = True
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
c95ba878c66aae44816b1451049449685d444f0a )
Sebastian Wagner [Mon, 18 Jan 2021 11:33:36 +0000 (12:33 +0100)]
cephadm: Fix node-exporter deployment.
Fixes: 2ce828d5f3682d3eee61e4a4a07a9eedb6a3d04e
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
6f7e01ee89b582b02cab141b9f5c08215c4c8d47 )
Sebastian Wagner [Mon, 18 Jan 2021 11:27:14 +0000 (12:27 +0100)]
qa/cephadm: Add yaml output to smoke test
this will provide a more detailed output, like
```yaml
...snip...
service_type: node-exporter
service_name: node-exporter
placement:
host_pattern: '*'
status:
created: '2021-01-18T11:21:56.024810Z'
last_refresh: '2021-01-18T11:23:24.477672Z'
running: 0
size: 1
events:
- "2021-01-18T11:23:09.602644Z service:node-exporter [ERROR] \"Failed while placing\
\ node-exporter.ubuntuon ubuntu: cephadm exited with an error code: 1, stderr:Deploy\
\ daemon node-exporter.ubuntu ...\nVerifying port 9100 ...\nTraceback (most recent\
\ call last):\n File \"<stdin>\", line 7274, in <module>\n File \"<stdin>\", line\
\ 1563, in _default_image\n File \"<stdin>\", line 3698, in command_deploy\n File\
\ \"<stdin>\", line 2338, in deploy_daemon\n File \"<stdin>\", line 1961, in create_daemon_dirs\n\
AssertionError\""
...snip...
```
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
88c6c34e2ba32e38c7fca93722737c3b4d31fe6c )
Mykola Golub [Sat, 23 Jan 2021 08:50:58 +0000 (10:50 +0200)]
Merge pull request #39023 from dillaman/wip-48882-pacific
pacific: librbd: bump minor version to match pacific
Reviewed-by: Mykola Golub <mgolub@suse.com>
Mykola Golub [Sat, 23 Jan 2021 08:50:33 +0000 (10:50 +0200)]
Merge pull request #39033 from dillaman/wip-ssd-doc-pacific
pacific: doc: update rwl doc to include ssd config
Reviewed-by: Mykola Golub <mgolub@suse.com>
Ramana Raja [Fri, 22 Jan 2021 21:26:18 +0000 (02:56 +0530)]
Merge pull request #39014 from kotreshhr/revert-removal-ceph-volume-client-pacific
pacific: Revert "*: remove legacy ceph_volume_client.py library"
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Josh Durgin [Fri, 22 Jan 2021 21:06:00 +0000 (13:06 -0800)]
Merge pull request #38994 from liewegas/fix-interactive-error-pacific
pacific: qa/tasks/ceph: do not update info.yaml if ctx.archive is not set
Reviewed-by: Neha Ojha <nojha@redhat.com>
Mahati Chamarthy [Fri, 22 Jan 2021 05:25:29 +0000 (10:55 +0530)]
doc: update rwl doc to include ssd config
... options.
Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
(cherry picked from commit
23327ce374706514b784d1842cdec3779e244c81 )
Jason Dillaman [Fri, 22 Jan 2021 13:31:08 +0000 (08:31 -0500)]
librbd: bump minor version to match pacific
This will assist layered projects like QEMU and libvirt
determine which features are supported.
Fixes: https://tracker.ceph.com/issues/48882
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Mykola Golub [Fri, 22 Jan 2021 08:20:30 +0000 (10:20 +0200)]
Merge pull request #39011 from dillaman/wip-librbd-crypto-fixes-pacific
pacific: librbd: miscellaneous fixes for crypto layer
Reviewed-by: Mykola Golub <mgolub@suse.com>
Kotresh HR [Fri, 22 Jan 2021 07:48:30 +0000 (13:18 +0530)]
Revert "*: remove legacy ceph_volume_client.py library"
This reverts commit
a3db265ad5b3ff899d8b71164a7f9ea5a426b19e .
The ceph_volume_client is being used by manila and it's
not fully equipped to use mgr/volumes and expects to use
ceph_volume_client for one more release. Hence reverting
this commit for pacific release.
Note that this is a pacific only patch and hence no corresponding
master patch is available.
Fixes: https://tracker.ceph.com/issues/48923
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Jason Dillaman [Thu, 21 Jan 2021 20:09:26 +0000 (15:09 -0500)]
Merge pull request #38986 from dillaman/wip-rbd-error-pacific
pacific: rbd: deprecation warning should be printed to stderr
Reviewed-by: Mykola Golub <mgolub@suse.com>
Or Ozeri [Wed, 20 Jan 2021 16:43:47 +0000 (18:43 +0200)]
librbd: don't restart empty copyups in crypto layer
This commit fixes a bug where an empty parent copyup is restarted indefinitely.
Signed-off-by: Or Ozeri <oro@il.ibm.com>
(cherry picked from commit
cd6f3ccd3363f40c4fbe1c52bba9cc77e4162950 )
Or Ozeri [Wed, 20 Jan 2021 13:36:10 +0000 (15:36 +0200)]
tools/rbd-nbd: change block size to 4K for LUKS2 images
This commit sets the NBD device block size to 4K for images encrypted using LUKS2 (whose sector size is by default 4K(.
Signed-off-by: Or Ozeri <oro@il.ibm.com>
(cherry picked from commit
988039f6ec9210d13f7bdaee9d73ac009c0ceb91 )
Or Ozeri [Wed, 20 Jan 2021 10:59:06 +0000 (12:59 +0200)]
qa/tasks/rbd: increase image size of encrypted disks
This commit increases the rbd image sizes in used by qemu to support the storage requirements of the luks encryption header.
Signed-off-by: Or Ozeri <oro@il.ibm.com>
(cherry picked from commit
114a7cfb2ea3cd86509588e0d528beb1494b36a0 )
Neha Ojha [Wed, 20 Jan 2021 23:39:10 +0000 (15:39 -0800)]
Merge pull request #38993 from neha-ojha/wip-48906-pacific
pacific: osd: Remove override for osd_async_recovery_min_cost for mclock profiles
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Deepika Upadhyay [Fri, 8 Jan 2021 15:02:05 +0000 (20:32 +0530)]
qa/tasks/ceph: do not update info.yaml if ctx.archive is not set
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
(cherry picked from commit
c5b1d0ac46527dd77d143d538395eff2be0fef44 )
Sridhar Seshasayee [Mon, 18 Jan 2021 08:34:46 +0000 (14:04 +0530)]
osd: Remove override for osd_async_recovery_min_cost for mclock profiles
Overriding osd_async_recovery_min_cost as part of enabling a built-in
mclock profile has the undesirable side effect of peers not choosing
the correct async recovery targets if osds are using mixed schedulers
(this could happen during upgrades or if "debug_random" is set for
osd_op_queue config option). Due to the above, osds get into a
"choose_acting" loop during peering.
The solution is to remove the override of osd_async_recovery_min_cost.
Fixes: https://tracker.ceph.com/issues/48906
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit
922d93f4491207411b4ee9427e49ccf08b0ef648 )
Jason Dillaman [Tue, 19 Jan 2021 16:46:38 +0000 (11:46 -0500)]
rbd: deprecation warning should be printed to stderr
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
8af6f8bb5fdd9d01f862cdea399b6e489d147259 )
Brad Hubbard [Wed, 20 Jan 2021 00:42:01 +0000 (10:42 +1000)]
Merge pull request #38979 from badone/wip-pacific-valgrind-tracker-48918
pacific: osd: initialise m_interval_start
Reviewed-by: Neha Ojha <nojha@redhat.com>
Brad Hubbard [Tue, 19 Jan 2021 02:11:45 +0000 (12:11 +1000)]
osd: initialise m_interval_start
Fixes: https://tracker.ceph.com/issues/48918
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
45f36bbfc6c39e4a94529d13fe0c2f74fd1fecd4 )
Or Ozeri [Thu, 14 Jan 2021 15:03:42 +0000 (17:03 +0200)]
:qa/tasks/rbd: test qemu on top of rbd encryption
This commit adds new qemu xfstests workloads that run on top of librbd luks1/luks2 encryption.
This is currently done via nbd, instead of the qemu rbd driver.
Signed-off-by: Or Ozeri <oro@il.ibm.com>
(cherry picked from commit
3754c665a11edba1f0a978527e35fd4de31d8212 )
Sage Weil [Mon, 18 Jan 2021 15:59:14 +0000 (09:59 -0600)]
ceph_release: dev -> rc
Signed-off-by: Sage Weil <sage@newdream.net>
Neha Ojha [Sun, 17 Jan 2021 02:24:13 +0000 (18:24 -0800)]
Merge pull request #38920 from sunnyku/wip-ceph-mclock-op
osd: handle ceph specific config changes for the mclock scheduler
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Neha Ojha [Sun, 17 Jan 2021 02:23:25 +0000 (18:23 -0800)]
Merge pull request #38675 from sseshasa/wip-dmclock-config-sets
osd: Add mclock config sets that implement built-in and custom profiles
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Jason Dillaman [Sat, 16 Jan 2021 22:04:46 +0000 (17:04 -0500)]
Merge pull request #38933 from wjwithagen/wjw-fix-QCOW
librbd: remove inclusion of endian.h
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Sridhar Seshasayee [Thu, 14 Jan 2021 13:06:04 +0000 (18:36 +0530)]
osd: Set recovery specific Ceph options for mclock profiles to work.
Set and disable relevant recovery specific Ceph options for mclock
profiles to work as expected. Broadly,
- Set low value for recovery min cost
- High values for max active recoveries and max backfills
- Disable recovery sleep{_hdd, _ssd, _hybrid}
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Sridhar Seshasayee [Tue, 12 Jan 2021 11:32:37 +0000 (17:02 +0530)]
osd: Handle configuration changes to mclock config options
Handle configuration changes to the mclock cost per io, the max
capacity options and the mclock profile. Handle the case where the
profile is changed from a built-in profile to the custom profile.
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Sridhar Seshasayee [Mon, 7 Dec 2020 11:09:57 +0000 (16:39 +0530)]
osd: Add mclock profile infrastructure and implement mclock profiles
Define config options to specify the cost per io for an osd (hdd & ssd).
- osd_mclock_cost_per_io_msec
- osd_mclock_cost_per_io_msec_hdd
- osd_mclock_cost_per_io_msec_ssd
Define config options to set max osd capacity (hdd & ssd) to be allocated
between clients of dmclock namely,
- osd_mclock_max_capacity_iops
- osd_mclock_max_capacity_iops_hdd
- osd_mclock_max_capacity_iops_ssd
Define config option "osd_mclock_profile" to specify the built-in profile
to enable.
Also, Set the number of op shards being used in the osd within the mclock
scheduler as well. This is necessary to calculate the per shard limits
within the mclock scheduler.
With the above information, enable the specified mclock profile by
calling the appropriate method to set the profile specific mclock
parameters and Ceph options.
Prior to enqueuing an op, the scheduler performs a calculation to scale
up or down the cost associated for the OpSchedulerItem. This calculation
is done based on the existing item cost, the max osd capacity provided
and an additional cost factor based on underlying device type(hdd/ssd).
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Sunny Kumar [Fri, 15 Jan 2021 02:53:05 +0000 (02:53 +0000)]
osd: handle ceph specific config changes for the mclock scheduler
The below ceph parameters are set automatically
while enabling the mclock scheduler with a built-in profile.
The user in this case will not be able to modify these
ceph specific config options during runtime.
a. osd_async_recovery_min_cost
b. osd_recovery_max_active{_hdd,_ssd}
c. osd_max_backfills
d. osd_recovery_sleep{_hdd,_ssd,_hybrid}
If the custom profile is enabled for the mclock scheduler,
the user can modify these parameters.
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
Patrick Donnelly [Sat, 16 Jan 2021 18:40:51 +0000 (10:40 -0800)]
Merge PR #37297 into master
* refs/pull/37297/head:
qa: add new client tests
test: add client tests
client: wire up alternate_name
mds: fix alternate_name durability
mds: add alternate_name feature support for dentries
mds: add static encode/decode helpers for remote in CDentry
mds: add "fscrypt" flag support for inode_t
mds: add new CDentry tags support
include: cleanup filepath cons
mds: add comment on linkage durability
client: constify some RWRef methods
common: add strescape helper
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Willem Jan Withagen [Sat, 16 Jan 2021 13:33:16 +0000 (14:33 +0100)]
librbd: remove inclusion of endian.h
The file is not available on FreeBSD.
Fixes: https://github.com/ceph/ceph/pull/38862
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
Kefu Chai [Sat, 16 Jan 2021 06:57:47 +0000 (14:57 +0800)]
Merge pull request #38880 from xxhdx1985126/wip-crimson-bug-fix
crimson/osd: make PGRecovery::start_primary_recovery_ops take objs recovered by UrgentRecovery into account
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Sat, 16 Jan 2021 04:20:15 +0000 (12:20 +0800)]
Merge pull request #38245 from batrick/ceph-status-pg-state-sort
osd: use more efficient CachedStackStringStream
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Sat, 16 Jan 2021 02:36:47 +0000 (18:36 -0800)]
Merge pull request #37391 from aclamk/wip-kv-onode-cache
Add ability to control rocksdb block cache
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Sat, 16 Jan 2021 01:40:55 +0000 (17:40 -0800)]
Merge pull request #38925 from thomasgoirand/master
common/ipaddr: Allow binding on lo
Reviewed-by: Kefu Chai <kchai@redhat.com>
Patrick Donnelly [Fri, 15 Jan 2021 04:17:42 +0000 (20:17 -0800)]
qa: add new client tests
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Mon, 11 Jan 2021 21:39:42 +0000 (13:39 -0800)]
test: add client tests
For alternate_name, so far.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Mon, 11 Jan 2021 21:39:27 +0000 (13:39 -0800)]
client: wire up alternate_name
Here we're exposing a public Client::walk (aka path_walk) so that the
user can inspect dentries (not something normally possible in POSIX).
We're going to skip exposing such an interface in libcephfs since
there's no reason to do that (who would use it?) except for testing.
Instead, a follow-up PR will add Client tests (for the first time, yay!)
that will exercise this code.
Also, ideally we'd also expose alternate_name via readdir results but
that is a bit more complicated since dirents do not normally refer to
external memory. So, just rely on Client::walk for testing for now.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 15 Jan 2021 03:55:43 +0000 (19:55 -0800)]
mds: fix alternate_name durability
This is a collection of fixes to Xiubo's prior work. Namely:
- Add new mds_alternate_name_max option to limit the size of
alternate_name. Otherwise a Client could trick the MDS into creating
an alternate_name of any size!
- Clean up how alternate_name is assigned to CDentry. In the general
case, this should be assigned as part of creating the dentry. We want
this value to be immutable for the life of the dentry. Even for the
very special case of rename(2) where the destination dentry already
exists. We explicitly check (after discussion with Jeff) that the
target dentry alternate_name already matches what the rename RPC is
giving.
- The MDS is now properly journaling the alternate_name.
- The MDS rejoin phase is properly transmitting each dentry's
alternate_name. I've discovered that this MMDSCacheRejoin message
actually wasn't versioned which I've raised in a tracker [1]. In the
mean time, we'll just bump CEPH_MDS_PROTOCOL as usual.
[1] https://tracker.ceph.com/issues/48886
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Xiubo Li [Mon, 21 Sep 2020 04:55:31 +0000 (12:55 +0800)]
mds: add alternate_name feature support for dentries
This will support the "alternate_name" filename support, and will save
an alternate name for each dentry. This alternate name is not used in
path lookup or for any other usual file system purpose. The name is
simply an added blob of metadata on the dentry that is distributed to
clients so that "long" file names may be supported for clients which
require them. In the case of an fscrypt enhanced kernel mount driver,
the long name may be the cyphertext (exceeding FILENAME_MAX) of a long
file name.
Because this affects only files with long file names, the use of this
feature should be rare but could be common for some unusual
applications.
The client mount should check the CEPHFS_FEATURE_ALTERNATE_NAME feature
bit first to check whether the MDS has support for this feature or not.
The alternate_name is transmitted as part of the message payload in
MClientRequest when setting the alternate_name. The LeaseStat structure
in MClientReply contains the alternate_name.
When executing a metadata mutation RPC, the client will set the
alternate_name (if it exists) as part of the operation. The MDS will
pick that up and set it on the new or mutated dentry.
Fixes: https://tracker.ceph.com/issues/47162
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Xiubo Li [Fri, 20 Nov 2020 02:02:42 +0000 (10:02 +0800)]
mds: add static encode/decode helpers for remote in CDentry
This will unify all the same work in different places.
Fixes: https://tracker.ceph.com/issues/47162
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Xiubo Li [Sun, 27 Sep 2020 03:53:55 +0000 (11:53 +0800)]
mds: add "fscrypt" flag support for inode_t
If the client has set the "encryption.ctx" attribute, the MDS side
will set the "fscrypt" flag to truth, false as default.
The clients could use this flag to get to know whether the current
inodes are under enscrypted or not.
Fixes: https://tracker.ceph.com/issues/47162
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Xiubo Li [Wed, 23 Sep 2020 02:54:34 +0000 (10:54 +0800)]
mds: add new CDentry tags support
This will add new tag 'i' for inode and new tag 'l' for remote link
for the CDentry. And at the same time will add one proper dentry
version, which will be helpful to add new features/members in future
for the CDentry.
Fixes: https://tracker.ceph.com/issues/47162
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Patrick Donnelly [Tue, 12 Jan 2021 03:15:06 +0000 (19:15 -0800)]
include: cleanup filepath cons
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 15 Jan 2021 03:58:29 +0000 (19:58 -0800)]
mds: add comment on linkage durability
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 15 Jan 2021 03:54:20 +0000 (19:54 -0800)]
client: constify some RWRef methods
And make some Client state checks const.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 15 Jan 2021 03:54:46 +0000 (19:54 -0800)]
common: add strescape helper
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Neha Ojha [Sat, 16 Jan 2021 01:02:19 +0000 (17:02 -0800)]
Merge pull request #38730 from ronen-fr/wip-ronenf-scrub-interval
osd: improved handling of new-interval events by the scrubber
Reviewed-by: David Zafman <dzafman@redhat.com>
Neha Ojha [Sat, 16 Jan 2021 00:27:59 +0000 (16:27 -0800)]
Merge pull request #38856 from dzafman/wip-48789
test: Fix osd-scrub-scaps.sh to handle DB format change
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 15 Jan 2021 21:46:30 +0000 (13:46 -0800)]
Merge pull request #38726 from ronen-fr/wip-ronenf-48720
qa/standalone/scrub/osd-recovery-scrub: handle primary change when waiting for scrub
Reviewed-by: David Zafman <dzafman@redhat.com>
Patrick Donnelly [Fri, 15 Jan 2021 19:53:02 +0000 (11:53 -0800)]
Merge PR #38160 into master
* refs/pull/38160/head:
doc/mgr/volumes: Document 'fs subvolume authorized_list' cli
qa: Add tests for list auth-ids of a subvolume
mgr/volumes: Update the 'volumes' key to 'subvolumes' in auth metadata file
mgr/volumes: Optionally authorize existing auth-ids
mgr/volumes: Preserve existing caps while authorize/deauthorize auth-id
mgr/volumes: Disallow authorize existing auth_id
mgr/volumes: Add subvolume authorized_list command
mgr/volumes: Add tenant_id option to subvolume authorize
mgr/volumes: Persist auth and subvolume metadata
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Mykola Golub [Fri, 15 Jan 2021 18:58:30 +0000 (20:58 +0200)]
Merge pull request #38862 from dillaman/wip-librbd-migration-qcow
librbd/migration: basic QCOW format handler
Reviewed-by: Mykola Golub <mgolub@suse.com>
David Zafman [Tue, 12 Jan 2021 00:00:05 +0000 (16:00 -0800)]
test: Fix osd-scrub-scaps.sh to handle DB format change
Caused by:
f9c95fa7fc7b0ee992c0249ff090fa7f751e9719
Fixes: https://tracker.ceph.com/issues/48789
Signed-off-by: David Zafman <dzafman@redhat.com>
Ernesto Puerta [Fri, 15 Jan 2021 18:07:16 +0000 (19:07 +0100)]
Merge pull request #38900 from Exotelis/update-i18n
mgr/dashboard: Update locale en to en-US to match transifex
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Kefu Chai [Fri, 15 Jan 2021 17:51:27 +0000 (01:51 +0800)]
Merge pull request #38722 from tchaikov/wip-mgr-args-ann
mgr: use annotations for argdesc
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Kefu Chai [Fri, 15 Jan 2021 17:49:46 +0000 (01:49 +0800)]
Merge pull request #38899 from ronen-fr/wip-ronenf-blue-dangling
os/bluestore: fixing a dangling pointer
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 15 Jan 2021 17:48:30 +0000 (01:48 +0800)]
Merge pull request #38902 from ifed01/wip-ifed-fix-deleting
osd/PG: fix no exit() call for PeeringState::Deleting state
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 15 Jan 2021 17:46:43 +0000 (01:46 +0800)]
Merge pull request #38857 from dzafman/wip-48842
test: osd-recovery-prio.sh replace sleep with wait for both PGs recovering
Reviewed-by: Neha Ojha <nojha@redhat.com>
Kefu Chai [Fri, 15 Jan 2021 17:44:38 +0000 (01:44 +0800)]
Merge pull request #38762 from peng-jiaqi/master
mgr: fix deadlock in ActivePyModules::get_osdmap()
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 15 Jan 2021 17:41:04 +0000 (01:41 +0800)]
Merge pull request #38878 from tchaikov/wip-ceph-time-print
common/ceph_time: print {milli,nano}seconds as float
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>