]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Sage Weil [Thu, 17 Jun 2021 21:12:15 +0000 (17:12 -0400)]
mgr/nfs: binding -> pseudo_path
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
5a2382a5a85eeba80bdd7f6192d4bc7a4af094e5 )
Sage Weil [Thu, 17 Jun 2021 17:55:48 +0000 (12:55 -0500)]
mgr/nfs: more revisions based on review
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
b276adbd4780702fc98303f05b78d946cb677182 )
Sage Weil [Thu, 17 Jun 2021 20:17:24 +0000 (16:17 -0400)]
mgr/nfs: adjust NFSExceptoin errno arg
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
9cf2ae5e8d84d60d63d67643596eca7b32082cf7 )
Sage Weil [Thu, 17 Jun 2021 20:15:03 +0000 (16:15 -0400)]
doc/cephfs: update 'nfs export {get,apply}' docs
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
f0798cf0681325bb780d2737db3bf86c199c0aa1 )
Sage Weil [Thu, 17 Jun 2021 20:01:50 +0000 (16:01 -0400)]
mgr/nfs: merge FSExport back into ExportMgr
This class captures no data, and the current methods cover both cephfs
and rgw exports. There is little value to making the class separate from
ExportMgr.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
1529f78ecf43ddc878ae028e6a56baa87daeed3b )
Sage Weil [Thu, 17 Jun 2021 19:45:17 +0000 (15:45 -0400)]
doc/radosgw/nfs: document mgr/nfs way to add/remove rgw exports
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
0907658f21b643dd767abdafcbbbf173213a6558 )
Sage Weil [Tue, 15 Jun 2021 17:34:47 +0000 (12:34 -0500)]
mgr/nfs: merge 'nfs export {update,import}' -> 'nfs export apply'
The only thing we lose is a strict 'update' that errors out if the
export didn't already exist, and we don't have any known need for that.
Fixes: https://tracker.ceph.com/issues/51265
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
2cb2abb385f97714cbe59371aa788394799db925 )
Sage Weil [Fri, 11 Jun 2021 23:22:03 +0000 (19:22 -0400)]
mgr/nfs: test export creation and list
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
69681f04ba78ad377219d113593759218e5bf4e4 )
Sage Weil [Fri, 11 Jun 2021 20:55:25 +0000 (16:55 -0400)]
mgr/nfs: test export_update (+ fixes)
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
cfee205826d6a3ce1a9cb0fc5a66088f67e49186 )
Sage Weil [Fri, 11 Jun 2021 19:55:27 +0000 (15:55 -0400)]
mgr/nfs: test Export.validate(); several fixes
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
01a8d0ba3d5aaf09b26de3acee33772c43ed9cac )
Sage Weil [Fri, 11 Jun 2021 19:34:27 +0000 (15:34 -0400)]
mgr/nfs: test that export <-> block+dict conversions go both ways
Fixes: https://tracker.ceph.com/issues/50449
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
16c5cb62f797b0154362678aeed0bf7aebe12f40 )
Sage Weil [Fri, 11 Jun 2021 19:34:14 +0000 (15:34 -0400)]
mgr/nfs: clean up test a bit
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
59587fd0b4f27e6322935cc1d3773b057f6cb184 )
Sage Weil [Wed, 9 Jun 2021 13:40:23 +0000 (09:40 -0400)]
mgr/nfs/export: fix export validation
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
fdd47c476e4b6232461a62f956837a049faf6ca0 )
Sage Weil [Tue, 8 Jun 2021 17:38:37 +0000 (13:38 -0400)]
mgr/nfs/export: fix tests
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
473097a36f92d4e6694d0be7b5cac123d723a8e6 )
Sage Weil [Mon, 7 Jun 2021 20:52:30 +0000 (16:52 -0400)]
mgr/nfs: handle option addr/client block in create_export()
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
d6d9e59ef76f33f0f9611acfb6406d849b9e29a8 )
Sage Weil [Mon, 7 Jun 2021 20:45:29 +0000 (16:45 -0400)]
mgr/nfs: allow multiple addrs for new exports
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
aaa57afd2930b6dafcf4dde4e2d939c428d290f4 )
Sage Weil [Mon, 7 Jun 2021 20:38:23 +0000 (16:38 -0400)]
mgr/nfs: fix/finish rgw export
- create the rgw keys for the export
- cleanup the rgw user on export deletion
Fixes: https://tracker.ceph.com/issues/47172
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
119555b84d4ae460b59ec74c665a6b215e2ba628 )
Sage Weil [Mon, 7 Jun 2021 18:51:41 +0000 (14:51 -0400)]
mgr/nfs/module: clusterid -> cluster_id
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
b54998106330cc8e4754a35b872e470a95568ba9 )
Sage Weil [Wed, 2 Jun 2021 20:16:26 +0000 (16:16 -0400)]
mgr/nfs/export: fix export_update_1 to type check
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
6771de220435d5a8f6ebc7601f0aa451d8e3c68b )
Sage Weil [Wed, 2 Jun 2021 18:43:52 +0000 (14:43 -0400)]
mgr/nfs/cluster: fix type error
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
f9d291b5755520feca393c7fca964de999a6e90a )
Sage Weil [Wed, 2 Jun 2021 18:41:15 +0000 (14:41 -0400)]
mgr/nfs/export: wrap long lines
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
1cb98a04974da044af81b9ab6c998c6f746c6473 )
Sebastian Wagner [Wed, 2 Jun 2021 15:54:23 +0000 (17:54 +0200)]
mgr/nfs: ExportMgr._delete_export only works for cephfs for now
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit
8ba3b5bf2e0492c73c0982fa1389cb46edbf2475 )
Sebastian Wagner [Wed, 2 Jun 2021 15:46:52 +0000 (17:46 +0200)]
mgr/nfs: Remove pool_ns from NFSCluster
Cause this is a singleton instance. Better to explictly pass it along.
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit
e6db0c810187fccbd3d5b77576727c63e3fc1cc3 )
Sebastian Wagner [Wed, 2 Jun 2021 14:20:54 +0000 (16:20 +0200)]
mgr/nfs: Remove ExportMgr.rados_namespace
Cause this is a singleton instance. Better to explictly pass it along.
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit
e0a9d9dccf4474253aec43290d54da8f9cd3742d )
Sebastian Wagner [Wed, 2 Jun 2021 14:04:21 +0000 (16:04 +0200)]
mgr/nfs: flake8
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit
a24e912099851a7c77d453afa84154858233e7d8 )
Conflicts:
src/pybind/mgr/tox.ini
Sebastian Wagner [Wed, 2 Jun 2021 13:55:02 +0000 (15:55 +0200)]
mgr/nfs: Add type checking
Also introduce RawBlock class
Fixes: https://tracker.ceph.com/issues/50816
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit
42d01c5ed3da749d2af6d3debffd3e117d18856f )
Conflicts:
src/mypy.ini
Sebastian Wagner [Wed, 2 Jun 2021 10:42:54 +0000 (12:42 +0200)]
mgr/nfs: Add __eq__ method to Export
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit
b3ddff838461be1805612184a6cd8ccbedb03ae5 )
Sebastian Wagner [Wed, 2 Jun 2021 10:42:28 +0000 (12:42 +0200)]
mgr/nfs: Add some compatibility to mgr/dashboard
provide default values for `squash` and `security_label`
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit
42a8d383e6b6da46cc4ec011f491f9221b48ec4d )
Sebastian Wagner [Wed, 2 Jun 2021 10:41:36 +0000 (12:41 +0200)]
mgr/nfs: Fix whitespace handling
keep whitespace in lines starting with %url
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit
e1928d57060ca3e180d6b8fc7fe5b8ee8c4aefad )
Sebastian Wagner [Wed, 2 Jun 2021 10:40:24 +0000 (12:40 +0200)]
mgr/nfs: Copy unit tests from mgr/dashboard
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit
a30328c49677d887c35697aa89619ca1bf6159f4 )
Sage Weil [Tue, 1 Jun 2021 16:23:37 +0000 (12:23 -0400)]
mgr/nfs: partially implement rgw export support
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
377756c9dff1bf690a5be700e99c22d63d0ebad2 )
Sage Weil [Tue, 1 Jun 2021 16:23:24 +0000 (12:23 -0400)]
mgr/nfs: abstract FSAL; add RGWFSAL
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
5c33431ce132ef7fc5db72d0c06cb89c5524c3a1 )
Sage Weil [Thu, 27 May 2021 23:43:22 +0000 (19:43 -0400)]
mgr/nfs: refactor to merge 'update' and 'import' code
- update must be an existing export
- import can be a new or existing export, and we will either update or
create as appropriate.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
097e4f9aa3b9ed663116381a13eadf279f77b8a2 )
Sage Weil [Thu, 27 May 2021 23:25:40 +0000 (19:25 -0400)]
mgr/nfs: add 'nfs export import' command
Allow import of JSON to create a new export. Similar to 'update', which
allows an import of JSON to update an existing export.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
778674a9c728b2b04f1b6922005798303475d352 )
Sage Weil [Thu, 27 May 2021 23:25:06 +0000 (19:25 -0400)]
mgr/nfs: refactor 'nfs export update' and export validation
Move validation to the Export class, so it can check any object in
place.
Refactor update code to extract allowed changes checks from the old
validation class.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
2b93356fac49a5bd50a9e4b97314db87ade51666 )
Sage Weil [Wed, 26 May 2021 19:55:22 +0000 (15:55 -0400)]
mgr/nfs: fix _fetch_export to distinguish between clusters
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
0eea3d772667a6d9ba4d42427404e8e6730e5df5 )
Sage Weil [Wed, 26 May 2021 20:27:34 +0000 (16:27 -0400)]
mgr/nfs: move export ganesha conf translation into caller
This belongs in the (one) caller, not the helper.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
e4b3673262be61ed21178d46cd53ecf7e1969598 )
Sage Weil [Wed, 26 May 2021 20:18:44 +0000 (16:18 -0400)]
mgr/nfs: name nfs cephfs client key 'nfs.{cluster_id}.{export_id}'
Better than '{cluster_id}{export_id}', which is confusing and ambiguous.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
9b1372dc388e5c783208ecc9251598f0985383a8 )
Sage Weil [Wed, 26 May 2021 18:37:34 +0000 (14:37 -0400)]
mgr/nfs: add --addr to 'nfs export create'
Ganesha exports can have multiple client blocks with addresses and
access modes/squash behavior. This cannot easily be exposed via a CLI.
However, providing no client address restriction is pretty useful to users
since it allows full access to the mount point with no restrictions.
Add an --addr argument that creates a single client block. Set the top
export-level access_type to 'none' and squash to 'none' in this case.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
137090dac07f95c3af0b09afc19da210b085019b )
Sage Weil [Wed, 26 May 2021 17:59:18 +0000 (13:59 -0400)]
mgr/nfs: add --squash to 'nfs export create'
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
2450447b45110ea98237618eda777e4e9285c6bd )
Sage Weil [Wed, 26 May 2021 17:45:53 +0000 (13:45 -0400)]
mgr/nfs/export_utils: include false but non-None items in config
For example, attr_expiration_time = 0 is the default. Include this since
it is not None.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
fe1155f0151e4619e13eb3df49dc3dd54eb80511 )
Sage Weil [Wed, 26 May 2021 17:44:04 +0000 (13:44 -0400)]
vstart.sh: enable nfs module
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
9b9c5bc7a4bebec2b03b7ff4367b8964f02614ae )
Conflicts:
src/vstart.sh
Sage Weil [Wed, 26 May 2021 17:43:57 +0000 (13:43 -0400)]
mgr/cephadm: nfs: drop attr_expiration_time from top-level config
This is added on a per-export basis with the same default of 0.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
283a8c184e748a0af5a5deedc2d80b855d747924 )
Sage Weil [Wed, 26 May 2021 15:50:59 +0000 (11:50 -0400)]
mgr/cephadm: remove Dir_Chunk = 0
This breaks RGW exports, and is likely to be beneficial for libcephfs.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
9d114e910e953c0adf4ba5d31ab1d17bdfa1f638 )
Ramana Raja [Tue, 22 Jun 2021 02:47:55 +0000 (22:47 -0400)]
doc/cephfs/nfs: update recommendation for versions
... of Ceph and NFS-Ganesha packages.
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit
e031a5969120c43adc5b20ca004e6540dffd1579 )
Conflicts:
doc/cephfs/nfs.rst
Sage Weil [Fri, 4 Jun 2021 17:35:46 +0000 (13:35 -0400)]
mgr/nfs: fix 'nfs export create' argument order
Put path before --readonly so that it can still be positional.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
ffce51af8e7be9835ff01c8d9a5f378f19d4d746 )
Sage Weil [Tue, 1 Jun 2021 17:39:03 +0000 (13:39 -0400)]
qa/tasks/cephfs/test_nfs: fix export create test
Everything after --readonly is non-positional.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
8683cccd06077bfbfd39e9e5ca3750078937dc0e )
Sage Weil [Sun, 23 May 2021 14:34:26 +0000 (10:34 -0400)]
mgr/k8sevents: fix help strings
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
fd77020e73bb43f254ec8a4ee7093dccd1b63ff9 )
Sage Weil [Sat, 22 May 2021 17:33:40 +0000 (13:33 -0400)]
pybind/mgr/mgr_module: fix help desc formatting
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
4bc37ba3b53930589f2fcd3f137b45ea9872f9e6 )
Sage Weil [Sat, 22 May 2021 17:28:37 +0000 (13:28 -0400)]
mgr/orchestrator: clean up 'orch {daemon add,apply} rgw' args
Make placement the only optional positional. This means that the "usual"
'orch orch apply <whatever> 3' will do placement=3.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
5d8cfb149b34d068b4dd6942e9a38e40aced299f )
Conflicts:
src/pybind/mgr/orchestrator/module.py
Sage Weil [Sat, 22 May 2021 17:21:04 +0000 (13:21 -0400)]
mgr/orchestrator: reformat a few methods
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
27c2b83a8e4c7efd319fcd6c1b80698f76dea6c4 )
Sage Weil [Mon, 24 May 2021 21:03:33 +0000 (16:03 -0500)]
doc/cephfs/nfs: remove documented limitation
At the time NFS support was added, this limitation applied.
However, in
https://github.com/nfs-ganesha/nfs-ganesha/commit/
b3d97f8157a131f55d848ff57e46af91b746b944
and
https://github.com/nfs-ganesha/nfs-ganesha/commit/
1cfe7e2df96f9785367ba94d41559140f584a875
we added support for multiple filesystems and started mixing
the fscid into the filehandle.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
a534f7be3601a18cd7e26af67031f2dfe3fead7b )
Conflicts:
doc/cephfs/nfs.rst
Yuri Weinstein [Wed, 1 Sep 2021 21:16:59 +0000 (14:16 -0700)]
Merge pull request #42939 from batrick/i51832
pacific: mds: META_POP_READDIR, META_POP_FETCH, META_POP_STORE, and cache_hit_rate are not updated
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Wed, 1 Sep 2021 21:16:17 +0000 (14:16 -0700)]
Merge pull request #42938 from batrick/i51932
pacific: mds: MDCache.cc:5319 FAILED ceph_assert(rejoin_ack_gather.count(mds->get_nodeid()))
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Wed, 1 Sep 2021 21:15:38 +0000 (14:15 -0700)]
Merge pull request #42936 from batrick/i51198
pacific: msg: active_connections regression
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Wed, 1 Sep 2021 21:14:45 +0000 (14:14 -0700)]
Merge pull request #42935 from batrick/i51935
pacific: mds: add debugging when rejecting mksnap with EPERM
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Wed, 1 Sep 2021 18:59:40 +0000 (11:59 -0700)]
Merge pull request #42956 from ifed01/wip-ifed-fix-huge-omap-rename-pac
pacific: os/bluestore: cap omap naming scheme upgrade transaction.
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Sebastian Wagner [Wed, 1 Sep 2021 10:54:21 +0000 (12:54 +0200)]
Merge pull request #42534 from liewegas/use-quay-pacific
pacific: cephadm: use quay, not docker
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Ernesto Puerta [Wed, 1 Sep 2021 10:25:24 +0000 (12:25 +0200)]
Merge pull request #42889 from rhcs-dashboard/wip-52378-pacific
pacific: mgr/dashboard: stats=false not working when listing buckets
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Ernesto Puerta [Wed, 1 Sep 2021 09:29:40 +0000 (11:29 +0200)]
Merge pull request #42974 from s0nea/wip-52308-pacific
pacific: mgr/dashboard: don't notify for suppressed alerts
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Yuri Weinstein [Tue, 31 Aug 2021 19:34:04 +0000 (12:34 -0700)]
Merge pull request #42898 from smithfarm/wip-52380-pacific
pacific: cmake: exclude "grafonnet-lib" target from "all"
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Yuri Weinstein [Tue, 31 Aug 2021 19:31:07 +0000 (12:31 -0700)]
Merge pull request #42897 from ifed01/wip-ifed-fix-spurious-read-errors-warn-pac
pacific: os/bluestore: respect bluestore_warn_on_spurious_read_errors setting
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Tue, 31 Aug 2021 19:30:32 +0000 (12:30 -0700)]
Merge pull request #42888 from callithea/wip-51951-pacific
pacific: Don't persist report data
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Igor Fedotov [Tue, 9 Feb 2021 15:29:01 +0000 (18:29 +0300)]
os/bluestore: cap omap naming scheme upgrade transactoin.
We shouldn't use single per-onode transaction for such an upgrade when onode's omap list is huge. This results in similarly sized WAL/SST files which are inefficient, might cause high memory usage and sometimes error-prone.
Fixes: https://tracker.ceph.com/issues/49170
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
e897fa243c1dd38329733b452872616023f14ac8 )
Ilya Dryomov [Tue, 31 Aug 2021 14:01:29 +0000 (16:01 +0200)]
Merge pull request #42969 from idryomov/wip-krbd-escape-match-sysattr-pacific
pacific: krbd: escape udev_enumerate_add_match_sysattr values
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Tatjana Dehler [Thu, 12 Aug 2021 14:06:44 +0000 (16:06 +0200)]
mgr/dashboard: don't notify for suppressed alerts
Fixes: https://tracker.ceph.com/issues/51987
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit
e9f316d678dd24e25e82474da95dbfbb72d763b5 )
Sage Weil [Wed, 11 Aug 2021 16:21:32 +0000 (12:21 -0400)]
cephadm: quay.io for non-ceph images too
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Wed, 11 Aug 2021 16:20:24 +0000 (12:20 -0400)]
cephadm,mgr/cephadm: use quay for ceph image
Signed-off-by: Sage Weil <sage@newdream.net>
Ilya Dryomov [Sat, 28 Aug 2021 09:05:28 +0000 (11:05 +0200)]
krbd: escape udev_enumerate_add_match_sysattr values
libudev uses fnmatch(3) for matching attributes, meaning that shell
glob pattern matching is employed instead of literal string matching.
Escape glob metacharacters to suppress pattern matching.
Fixes: https://tracker.ceph.com/issues/52425
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
8841029b0a4705825ba394541240b3cb4eb2cf5c )
Kamoltat Sirivadhna [Fri, 27 Aug 2021 18:56:00 +0000 (14:56 -0400)]
Merge pull request #42428 from kamoltat/wip-ksirivad-backport-pacific-42036
pacific: mgr/pg_autoscaler: Introduce autoscaler scale-down feature
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Kamoltat [Thu, 26 Aug 2021 19:50:14 +0000 (19:50 +0000)]
PendingReleaseNotes: add autoscaler profile
Added autoscaler profile release notes for
>=16.2.6
Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit
ea094f769f554ff7d5e4a17e76130d00044ef4e4 )
Kamoltat [Fri, 6 Aug 2021 04:23:29 +0000 (04:23 +0000)]
doc/rados/operations/placement-groups: added bias + profile
Added documentations on the autoscale profile and bias
Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit
65fec053e003967aa3ffa5008730661baa46c868 )
Kamoltat [Fri, 30 Jul 2021 19:29:21 +0000 (19:29 +0000)]
pybind/mgr/pg_autoscaler: Added PROFILE to autoscale-status
autoscaler now shows the PROFILE status of each pool
in `ceph osd pool autoscale-status`
Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit
4a2912bbad72b00ae9fe0d892f07ca760e5cefb4 )
Kamoltat [Fri, 25 Jun 2021 22:40:43 +0000 (22:40 +0000)]
pybind/mgr/autoscaler: don't scale pools with overlapping roots
In the previous version of get_subtree_resource_status() in
src/pybind/mgr/pg_autoscaler/module.py we ignore overlapping
pools which in some cases if combined with the new `scale-down`
algorithm in https://github.com/ceph/ceph/pull/38805 can cause
some pools to scale up/down to inapproriate amount of pgs.
Therefore, the PR identifies the overlapping roots and prevent the pools
with such roots from scaling. This only happens with `scale-down` profile
as we see no problem with the default `scale-up` profile.
Removed the variable `pool_root` since it is not used anywhere in
the code, it only gets assigned and reassigned
Also included a unit test test_overlapping_roots.py that tests the function
identify_subtrees_and_overlaps() as well as edited test_cal_final_pg_target.py
to account for pools that contain overlapping roots, therefore, those pools
are expected not to scale.
Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit
16eb42a1d8cef5cf008b04b27d51e13dbd6ec495 )
Kefu Chai [Mon, 28 Jun 2021 03:32:41 +0000 (11:32 +0800)]
pybind/mgr/pg_autoscaler: add typing annotations
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
1b0b32e44cddacbd7acf7024181247f4251c92f9 )
Conflicts:
src/mypy.ini - trivial fix
Kefu Chai [Mon, 28 Jun 2021 05:24:41 +0000 (13:24 +0800)]
pybind/mgr/pg_autoscaler: do not index pool_root with pool_name
pool_root is indexed by pool_id, and we never index it with pool_name.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
83f841295d86d10aadf3b4dff4e3eaa80fe139c3 )
Kefu Chai [Mon, 28 Jun 2021 04:39:12 +0000 (12:39 +0800)]
pybind/mgr/pg_autoscaler: do not create a tuple long_desc
long_desc is supposed to be a str, not a tuple of str.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
f725e09ae63702097f0eef1395391933083a9d96 )
Kefu Chai [Mon, 28 Jun 2021 04:28:17 +0000 (12:28 +0800)]
pybind/mgr/pg_autoscaler: extract CrushSubtreeResourceStatus out
as it also serves as part of interface of get_subtree_resource_status(),
not only its internals. to ease adding the type annotations, this class
is promoted out of the class.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
9bb55e84809e17e0aa1cb1aa4ae712a3ea4fffed )
Kefu Chai [Mon, 28 Jun 2021 03:44:05 +0000 (11:44 +0800)]
pybind/mgr/pg_autoscaler: define commands using CLICommand
simpler this way
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
4efd7400804ff42b5965ad2691223471b4ae38a7 )
Kefu Chai [Mon, 28 Jun 2021 03:34:09 +0000 (11:34 +0800)]
pybind/mgr/pg_autoscaler: define options using Option
more consistent and less error-prune this way.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
b40de19c23b6b49f8d63bb3911bdebe9a2e19255 )
Kamoltat [Tue, 13 Apr 2021 21:12:40 +0000 (21:12 +0000)]
pybind/mgr/pg_autoscaler: Added autoscale-profile feature
The autoscaler by default will start out each pool with minimal
pgs and `scale-up` the pgs when there is more usage in each pool.
Users can now use the commands:
`osd pool set autoscale-profile scale-down` to make the pools
start out with a full complement of pgs and only `scale-down`
when usage ratio across the pools are not even.
`osd pool set autoscale-profile scale-up` (by default) to make the pools
start out with minimal pgs and `scale-up` the pgs when there
is more usage in each pool.
Edited KVMonitor.cc file to make the `autoscale_profile` variable
persistent.
Edited tests/test_cal_final_pg_target.py so that it takes into account
the new `profile` argument when calling cal_final_pg_target(). Also,
added some new test cases for when profile is `scale-up`
Renamed tests/test_autoscaler.py to a more appropriate name:
tests/test_cal_ratio.py
Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit
0e49683b538026e4c51a040950da3dbb9c7e92d5 )
Kamoltat [Thu, 7 Jan 2021 15:39:19 +0000 (15:39 +0000)]
mgr/pg_autoscaler: avoid scale-down until there is pressure
The autoscaler will start out with scaling each
pools to have a full complements of pgs from the start
and will only decrease it when pools need more due to
increased usage.
Introduced a unit test that tests only the
function get_final_pg_target_and_ratio() which
deals with the distrubtion of pgs amongst the
pools
Edited workunit script to reflect the change
of how pgs are calculated and distrubted.
Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit
daeb6f6ac0c8f77ae07147f9d1e2ed18d6d8e4cc )
Yuri Weinstein [Thu, 26 Aug 2021 18:38:48 +0000 (11:38 -0700)]
Merge pull request #42914 from vshankar/wip-51544
pacific: mgr/volumes: use dedicated libcephfs handles for subvolume calls and …
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Thu, 26 Aug 2021 18:38:02 +0000 (11:38 -0700)]
Merge pull request #42900 from ajarr/wip-51834-pacific
pacific: mds: create file system with specific ID
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yongseok Oh [Thu, 8 Jul 2021 06:36:19 +0000 (15:36 +0900)]
mds: update META_POP_READDIR/FETCH/STORE and cache_hit_rate for dump loads
Signed-off-by: Yongseok Oh <yongseok.oh@linecorp.com>
(cherry picked from commit
24e131a58ed255cb8f6845c020ab956450a5fc10 )
chencan [Wed, 2 Dec 2020 07:55:32 +0000 (15:55 +0800)]
mds: check rejoin_ack_gather before enter rejoin_gather_finish
Fixes: https://tracker.ceph.com/issues/48422
Signed-off-by: chencan <chen.can2@zte.com.cn>
(cherry picked from commit
88548775330f66226b4e58a34256f1cc5a72221f )
Sage Weil [Wed, 19 May 2021 19:27:56 +0000 (15:27 -0400)]
qa/tasks/cephfs/test_sessionmap: reap connections immediately
We have to reap connections promptly for this test to work.
This test was broken indirectly by
d51d80b3234e17690061f65dc7e1515f4244a5a3 ,
which moved the counter decrement to reap time instead of mark_down/stop
time.
The reaping is asynchronous, so allow for a delay in the count change.
Fixes: https://tracker.ceph.com/issues/50622
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
c8c5071dcd4b0b788f5e924a678095ce5dc1d7f8 )
Sage Weil [Wed, 19 May 2021 19:23:26 +0000 (15:23 -0400)]
msg/async: configurable threshold for reaping dead connections
It is helpful to set this to 1 for tests.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
8129d6bb953015cc05db458afa6aa9b8f5f62614 )
Conflicts:
src/common/options/global.yaml.in
Patrick Donnelly [Tue, 6 Jul 2021 18:03:19 +0000 (11:03 -0700)]
mds: add debugging when rejecting mksnap with EPERM
Fixes: https://tracker.ceph.com/issues/51543
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
92ec79fd57b841c54bd9a5490031f62166ed6ab1 )
Yuri Weinstein [Wed, 25 Aug 2021 19:04:56 +0000 (12:04 -0700)]
Merge pull request #42083 from kotreshhr/wip-51411-pacific
pacific: mgr/mgr_util: switch using unshared cephfs connections whenever possible
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Wed, 25 Aug 2021 14:43:53 +0000 (07:43 -0700)]
Merge pull request #42908 from cfsnyder/wip-52330-pacific
pacific: cls/cmpomap: empty values are 0 in U64 comparisons
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Venky Shankar [Fri, 18 Jun 2021 07:13:01 +0000 (03:13 -0400)]
mgr/volumes: use dedicated libcephfs handles for subvolume calls and async jobs
Fixes: http://tracker.ceph.com/issues/51271
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit
cb2883feac1a5c141a3d72120c2320f7a8ffdea8 )
Yuri Weinstein [Tue, 24 Aug 2021 21:54:48 +0000 (14:54 -0700)]
Merge pull request #42860 from ideepika/wip-52336-pacific
pacific: mon/PGMap: remove DIRTY field in ceph df detail when cache tiering is not in use
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Tue, 24 Aug 2021 21:54:12 +0000 (14:54 -0700)]
Merge pull request #42844 from aclamk/resharding-column-options-pacific
pacific: kv/RocksDBStore: Add handling of block_cache option for resharding
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Tue, 24 Aug 2021 21:53:25 +0000 (14:53 -0700)]
Merge pull request #42615 from neha-ojha/wip-51830-pacific
pacific: common/options: Set osd_client_message_cap to 256.
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Tue, 24 Aug 2021 21:52:25 +0000 (14:52 -0700)]
Merge pull request #42300 from callithea/wip-51241-pacific
pacific: debian/control: ceph-mgr-modules-core does not Recommend ceph-mgr-roo…
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Tue, 24 Aug 2021 21:51:21 +0000 (14:51 -0700)]
Merge pull request #42578 from batrick/i51983
pacific: mon/MDSMonitor: do not pointlessly kill standbys that are incompatible with current CompatSet
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Tue, 24 Aug 2021 21:50:20 +0000 (14:50 -0700)]
Merge pull request #42096 from varshar16/wip-nfs-backport-june
pacific: nfs backport June
Reviewed-by: Ramana Raja <rraja@redhat.com>
Gregory Farnum [Tue, 24 Aug 2021 20:12:23 +0000 (13:12 -0700)]
Merge pull request #42909 from gregsfortytwo/wip-51290-pacific
pacific: mon: Sanely set the default CRUSH rule when creating pools in stretch…
Reviewed-by: Samuel Just <sjust@redhat.com>
Greg Farnum [Thu, 17 Jun 2021 19:56:20 +0000 (19:56 +0000)]
mon: Sanely set the default CRUSH rule when creating pools in stretch mode
If we get a pool create request while in stretch mode that does not explicitly
specify a crush rule, look at the stretch-mode pools and their rules, and
select the most common one.
Also update set_up_stretch_mode.sh to add a few more rules that let me test
this locally.
Fixes: https://tracker.ceph.com/issues/51270
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit
4f9fdca136bde10479fc173c9bcf4288ea960ace )