]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agomgr/nfs: binding -> pseudo_path
Sage Weil [Thu, 17 Jun 2021 21:12:15 +0000 (17:12 -0400)]
mgr/nfs: binding -> pseudo_path

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5a2382a5a85eeba80bdd7f6192d4bc7a4af094e5)

3 years agomgr/nfs: more revisions based on review
Sage Weil [Thu, 17 Jun 2021 17:55:48 +0000 (12:55 -0500)]
mgr/nfs: more revisions based on review

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b276adbd4780702fc98303f05b78d946cb677182)

3 years agomgr/nfs: adjust NFSExceptoin errno arg
Sage Weil [Thu, 17 Jun 2021 20:17:24 +0000 (16:17 -0400)]
mgr/nfs: adjust NFSExceptoin errno arg

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9cf2ae5e8d84d60d63d67643596eca7b32082cf7)

3 years agodoc/cephfs: update 'nfs export {get,apply}' docs
Sage Weil [Thu, 17 Jun 2021 20:15:03 +0000 (16:15 -0400)]
doc/cephfs: update 'nfs export {get,apply}' docs

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f0798cf0681325bb780d2737db3bf86c199c0aa1)

3 years agomgr/nfs: merge FSExport back into ExportMgr
Sage Weil [Thu, 17 Jun 2021 20:01:50 +0000 (16:01 -0400)]
mgr/nfs: merge FSExport back into ExportMgr

This class captures no data, and the current methods cover both cephfs
and rgw exports.  There is little value to making the class separate from
ExportMgr.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1529f78ecf43ddc878ae028e6a56baa87daeed3b)

3 years agodoc/radosgw/nfs: document mgr/nfs way to add/remove rgw exports
Sage Weil [Thu, 17 Jun 2021 19:45:17 +0000 (15:45 -0400)]
doc/radosgw/nfs: document mgr/nfs way to add/remove rgw exports

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0907658f21b643dd767abdafcbbbf173213a6558)

3 years agomgr/nfs: merge 'nfs export {update,import}' -> 'nfs export apply'
Sage Weil [Tue, 15 Jun 2021 17:34:47 +0000 (12:34 -0500)]
mgr/nfs: merge 'nfs export {update,import}' -> 'nfs export apply'

The only thing we lose is a strict 'update' that errors out if the
export didn't already exist, and we don't have any known need for that.

Fixes: https://tracker.ceph.com/issues/51265
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2cb2abb385f97714cbe59371aa788394799db925)

3 years agomgr/nfs: test export creation and list
Sage Weil [Fri, 11 Jun 2021 23:22:03 +0000 (19:22 -0400)]
mgr/nfs: test export creation and list

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 69681f04ba78ad377219d113593759218e5bf4e4)

3 years agomgr/nfs: test export_update (+ fixes)
Sage Weil [Fri, 11 Jun 2021 20:55:25 +0000 (16:55 -0400)]
mgr/nfs: test export_update (+ fixes)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit cfee205826d6a3ce1a9cb0fc5a66088f67e49186)

3 years agomgr/nfs: test Export.validate(); several fixes
Sage Weil [Fri, 11 Jun 2021 19:55:27 +0000 (15:55 -0400)]
mgr/nfs: test Export.validate(); several fixes

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 01a8d0ba3d5aaf09b26de3acee33772c43ed9cac)

3 years agomgr/nfs: test that export <-> block+dict conversions go both ways
Sage Weil [Fri, 11 Jun 2021 19:34:27 +0000 (15:34 -0400)]
mgr/nfs: test that export <-> block+dict conversions go both ways

Fixes: https://tracker.ceph.com/issues/50449
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 16c5cb62f797b0154362678aeed0bf7aebe12f40)

3 years agomgr/nfs: clean up test a bit
Sage Weil [Fri, 11 Jun 2021 19:34:14 +0000 (15:34 -0400)]
mgr/nfs: clean up test a bit

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 59587fd0b4f27e6322935cc1d3773b057f6cb184)

3 years agomgr/nfs/export: fix export validation
Sage Weil [Wed, 9 Jun 2021 13:40:23 +0000 (09:40 -0400)]
mgr/nfs/export: fix export validation

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit fdd47c476e4b6232461a62f956837a049faf6ca0)

3 years agomgr/nfs/export: fix tests
Sage Weil [Tue, 8 Jun 2021 17:38:37 +0000 (13:38 -0400)]
mgr/nfs/export: fix tests

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 473097a36f92d4e6694d0be7b5cac123d723a8e6)

3 years agomgr/nfs: handle option addr/client block in create_export()
Sage Weil [Mon, 7 Jun 2021 20:52:30 +0000 (16:52 -0400)]
mgr/nfs: handle option addr/client block in create_export()

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d6d9e59ef76f33f0f9611acfb6406d849b9e29a8)

3 years agomgr/nfs: allow multiple addrs for new exports
Sage Weil [Mon, 7 Jun 2021 20:45:29 +0000 (16:45 -0400)]
mgr/nfs: allow multiple addrs for new exports

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit aaa57afd2930b6dafcf4dde4e2d939c428d290f4)

3 years agomgr/nfs: fix/finish rgw export
Sage Weil [Mon, 7 Jun 2021 20:38:23 +0000 (16:38 -0400)]
mgr/nfs: fix/finish rgw export

- create the rgw keys for the export
- cleanup the rgw user on export deletion

Fixes: https://tracker.ceph.com/issues/47172
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 119555b84d4ae460b59ec74c665a6b215e2ba628)

3 years agomgr/nfs/module: clusterid -> cluster_id
Sage Weil [Mon, 7 Jun 2021 18:51:41 +0000 (14:51 -0400)]
mgr/nfs/module: clusterid -> cluster_id

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b54998106330cc8e4754a35b872e470a95568ba9)

3 years agomgr/nfs/export: fix export_update_1 to type check
Sage Weil [Wed, 2 Jun 2021 20:16:26 +0000 (16:16 -0400)]
mgr/nfs/export: fix export_update_1 to type check

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6771de220435d5a8f6ebc7601f0aa451d8e3c68b)

3 years agomgr/nfs/cluster: fix type error
Sage Weil [Wed, 2 Jun 2021 18:43:52 +0000 (14:43 -0400)]
mgr/nfs/cluster: fix type error

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f9d291b5755520feca393c7fca964de999a6e90a)

3 years agomgr/nfs/export: wrap long lines
Sage Weil [Wed, 2 Jun 2021 18:41:15 +0000 (14:41 -0400)]
mgr/nfs/export: wrap long lines

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1cb98a04974da044af81b9ab6c998c6f746c6473)

3 years agomgr/nfs: ExportMgr._delete_export only works for cephfs for now
Sebastian Wagner [Wed, 2 Jun 2021 15:54:23 +0000 (17:54 +0200)]
mgr/nfs: ExportMgr._delete_export only works for cephfs for now

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8ba3b5bf2e0492c73c0982fa1389cb46edbf2475)

3 years agomgr/nfs: Remove pool_ns from NFSCluster
Sebastian Wagner [Wed, 2 Jun 2021 15:46:52 +0000 (17:46 +0200)]
mgr/nfs: Remove pool_ns from NFSCluster

Cause this is a singleton instance. Better to explictly pass it along.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e6db0c810187fccbd3d5b77576727c63e3fc1cc3)

3 years agomgr/nfs: Remove ExportMgr.rados_namespace
Sebastian Wagner [Wed, 2 Jun 2021 14:20:54 +0000 (16:20 +0200)]
mgr/nfs: Remove ExportMgr.rados_namespace

Cause this is a singleton instance. Better to explictly pass it along.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e0a9d9dccf4474253aec43290d54da8f9cd3742d)

3 years agomgr/nfs: flake8
Sebastian Wagner [Wed, 2 Jun 2021 14:04:21 +0000 (16:04 +0200)]
mgr/nfs: flake8

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit a24e912099851a7c77d453afa84154858233e7d8)

Conflicts:
src/pybind/mgr/tox.ini

3 years agomgr/nfs: Add type checking
Sebastian Wagner [Wed, 2 Jun 2021 13:55:02 +0000 (15:55 +0200)]
mgr/nfs: Add type checking

Also introduce RawBlock class

Fixes: https://tracker.ceph.com/issues/50816
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 42d01c5ed3da749d2af6d3debffd3e117d18856f)

Conflicts:
src/mypy.ini

3 years agomgr/nfs: Add __eq__ method to Export
Sebastian Wagner [Wed, 2 Jun 2021 10:42:54 +0000 (12:42 +0200)]
mgr/nfs: Add __eq__ method to Export

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit b3ddff838461be1805612184a6cd8ccbedb03ae5)

3 years agomgr/nfs: Add some compatibility to mgr/dashboard
Sebastian Wagner [Wed, 2 Jun 2021 10:42:28 +0000 (12:42 +0200)]
mgr/nfs: Add some compatibility to mgr/dashboard

provide default values for `squash` and `security_label`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 42a8d383e6b6da46cc4ec011f491f9221b48ec4d)

3 years agomgr/nfs: Fix whitespace handling
Sebastian Wagner [Wed, 2 Jun 2021 10:41:36 +0000 (12:41 +0200)]
mgr/nfs: Fix whitespace handling

keep whitespace in lines starting with %url

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e1928d57060ca3e180d6b8fc7fe5b8ee8c4aefad)

3 years agomgr/nfs: Copy unit tests from mgr/dashboard
Sebastian Wagner [Wed, 2 Jun 2021 10:40:24 +0000 (12:40 +0200)]
mgr/nfs: Copy unit tests from mgr/dashboard

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit a30328c49677d887c35697aa89619ca1bf6159f4)

3 years agomgr/nfs: partially implement rgw export support
Sage Weil [Tue, 1 Jun 2021 16:23:37 +0000 (12:23 -0400)]
mgr/nfs: partially implement rgw export support

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 377756c9dff1bf690a5be700e99c22d63d0ebad2)

3 years agomgr/nfs: abstract FSAL; add RGWFSAL
Sage Weil [Tue, 1 Jun 2021 16:23:24 +0000 (12:23 -0400)]
mgr/nfs: abstract FSAL; add RGWFSAL

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5c33431ce132ef7fc5db72d0c06cb89c5524c3a1)

3 years agomgr/nfs: refactor to merge 'update' and 'import' code
Sage Weil [Thu, 27 May 2021 23:43:22 +0000 (19:43 -0400)]
mgr/nfs: refactor to merge 'update' and 'import' code

- update must be an existing export
- import can be a new or existing export, and we will either update or
  create as appropriate.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 097e4f9aa3b9ed663116381a13eadf279f77b8a2)

3 years agomgr/nfs: add 'nfs export import' command
Sage Weil [Thu, 27 May 2021 23:25:40 +0000 (19:25 -0400)]
mgr/nfs: add 'nfs export import' command

Allow import of JSON to create a new export.  Similar to 'update', which
allows an import of JSON to update an existing export.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 778674a9c728b2b04f1b6922005798303475d352)

3 years agomgr/nfs: refactor 'nfs export update' and export validation
Sage Weil [Thu, 27 May 2021 23:25:06 +0000 (19:25 -0400)]
mgr/nfs: refactor 'nfs export update' and export validation

Move validation to the Export class, so it can check any object in
place.

Refactor update code to extract allowed changes checks from the old
validation class.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2b93356fac49a5bd50a9e4b97314db87ade51666)

3 years agomgr/nfs: fix _fetch_export to distinguish between clusters
Sage Weil [Wed, 26 May 2021 19:55:22 +0000 (15:55 -0400)]
mgr/nfs: fix _fetch_export to distinguish between clusters

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0eea3d772667a6d9ba4d42427404e8e6730e5df5)

3 years agomgr/nfs: move export ganesha conf translation into caller
Sage Weil [Wed, 26 May 2021 20:27:34 +0000 (16:27 -0400)]
mgr/nfs: move export ganesha conf translation into caller

This belongs in the (one) caller, not the helper.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e4b3673262be61ed21178d46cd53ecf7e1969598)

3 years agomgr/nfs: name nfs cephfs client key 'nfs.{cluster_id}.{export_id}'
Sage Weil [Wed, 26 May 2021 20:18:44 +0000 (16:18 -0400)]
mgr/nfs: name nfs cephfs client key 'nfs.{cluster_id}.{export_id}'

Better than '{cluster_id}{export_id}', which is confusing and ambiguous.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9b1372dc388e5c783208ecc9251598f0985383a8)

3 years agomgr/nfs: add --addr to 'nfs export create'
Sage Weil [Wed, 26 May 2021 18:37:34 +0000 (14:37 -0400)]
mgr/nfs: add --addr to 'nfs export create'

Ganesha exports can have multiple client blocks with addresses and
access modes/squash behavior.  This cannot easily be exposed via a CLI.
However, providing no client address restriction is pretty useful to users
since it allows full access to the mount point with no restrictions.

Add an --addr argument that creates a single client block.  Set the top
export-level access_type to 'none' and squash to 'none' in this case.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 137090dac07f95c3af0b09afc19da210b085019b)

3 years agomgr/nfs: add --squash to 'nfs export create'
Sage Weil [Wed, 26 May 2021 17:59:18 +0000 (13:59 -0400)]
mgr/nfs: add --squash to 'nfs export create'

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2450447b45110ea98237618eda777e4e9285c6bd)

3 years agomgr/nfs/export_utils: include false but non-None items in config
Sage Weil [Wed, 26 May 2021 17:45:53 +0000 (13:45 -0400)]
mgr/nfs/export_utils: include false but non-None items in config

For example, attr_expiration_time = 0 is the default. Include this since
it is not None.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit fe1155f0151e4619e13eb3df49dc3dd54eb80511)

3 years agovstart.sh: enable nfs module
Sage Weil [Wed, 26 May 2021 17:44:04 +0000 (13:44 -0400)]
vstart.sh: enable nfs module

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9b9c5bc7a4bebec2b03b7ff4367b8964f02614ae)

Conflicts:
src/vstart.sh

3 years agomgr/cephadm: nfs: drop attr_expiration_time from top-level config
Sage Weil [Wed, 26 May 2021 17:43:57 +0000 (13:43 -0400)]
mgr/cephadm: nfs: drop attr_expiration_time from top-level config

This is added on a per-export basis with the same default of 0.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 283a8c184e748a0af5a5deedc2d80b855d747924)

3 years agomgr/cephadm: remove Dir_Chunk = 0
Sage Weil [Wed, 26 May 2021 15:50:59 +0000 (11:50 -0400)]
mgr/cephadm: remove Dir_Chunk = 0

This breaks RGW exports, and is likely to be beneficial for libcephfs.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9d114e910e953c0adf4ba5d31ab1d17bdfa1f638)

3 years agodoc/cephfs/nfs: update recommendation for versions
Ramana Raja [Tue, 22 Jun 2021 02:47:55 +0000 (22:47 -0400)]
doc/cephfs/nfs: update recommendation for versions

... of Ceph and NFS-Ganesha packages.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit e031a5969120c43adc5b20ca004e6540dffd1579)

Conflicts:
doc/cephfs/nfs.rst

3 years agomgr/nfs: fix 'nfs export create' argument order
Sage Weil [Fri, 4 Jun 2021 17:35:46 +0000 (13:35 -0400)]
mgr/nfs: fix 'nfs export create' argument order

Put path before --readonly so that it can still be positional.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ffce51af8e7be9835ff01c8d9a5f378f19d4d746)

3 years agoqa/tasks/cephfs/test_nfs: fix export create test
Sage Weil [Tue, 1 Jun 2021 17:39:03 +0000 (13:39 -0400)]
qa/tasks/cephfs/test_nfs: fix export create test

Everything after --readonly is non-positional.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8683cccd06077bfbfd39e9e5ca3750078937dc0e)

3 years agomgr/k8sevents: fix help strings
Sage Weil [Sun, 23 May 2021 14:34:26 +0000 (10:34 -0400)]
mgr/k8sevents: fix help strings

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit fd77020e73bb43f254ec8a4ee7093dccd1b63ff9)

3 years agopybind/mgr/mgr_module: fix help desc formatting
Sage Weil [Sat, 22 May 2021 17:33:40 +0000 (13:33 -0400)]
pybind/mgr/mgr_module: fix help desc formatting

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4bc37ba3b53930589f2fcd3f137b45ea9872f9e6)

3 years agomgr/orchestrator: clean up 'orch {daemon add,apply} rgw' args
Sage Weil [Sat, 22 May 2021 17:28:37 +0000 (13:28 -0400)]
mgr/orchestrator: clean up 'orch {daemon add,apply} rgw' args

Make placement the only optional positional.  This means that the "usual"
'orch orch apply <whatever> 3' will do placement=3.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5d8cfb149b34d068b4dd6942e9a38e40aced299f)

Conflicts:
  src/pybind/mgr/orchestrator/module.py

3 years agomgr/orchestrator: reformat a few methods
Sage Weil [Sat, 22 May 2021 17:21:04 +0000 (13:21 -0400)]
mgr/orchestrator: reformat a few methods

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 27c2b83a8e4c7efd319fcd6c1b80698f76dea6c4)

4 years agodoc/cephfs/nfs: remove documented limitation
Sage Weil [Mon, 24 May 2021 21:03:33 +0000 (16:03 -0500)]
doc/cephfs/nfs: remove documented limitation

At the time NFS support was added, this limitation applied.
However, in
 https://github.com/nfs-ganesha/nfs-ganesha/commit/b3d97f8157a131f55d848ff57e46af91b746b944
and
 https://github.com/nfs-ganesha/nfs-ganesha/commit/1cfe7e2df96f9785367ba94d41559140f584a875
we added support for multiple filesystems and started mixing
the fscid into the filehandle.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a534f7be3601a18cd7e26af67031f2dfe3fead7b)

Conflicts:
doc/cephfs/nfs.rst

4 years agoMerge pull request #42939 from batrick/i51832
Yuri Weinstein [Wed, 1 Sep 2021 21:16:59 +0000 (14:16 -0700)]
Merge pull request #42939 from batrick/i51832

pacific: mds: META_POP_READDIR, META_POP_FETCH, META_POP_STORE, and cache_hit_rate are not updated

Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agoMerge pull request #42938 from batrick/i51932
Yuri Weinstein [Wed, 1 Sep 2021 21:16:17 +0000 (14:16 -0700)]
Merge pull request #42938 from batrick/i51932

pacific: mds: MDCache.cc:5319 FAILED ceph_assert(rejoin_ack_gather.count(mds->get_nodeid()))

Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agoMerge pull request #42936 from batrick/i51198
Yuri Weinstein [Wed, 1 Sep 2021 21:15:38 +0000 (14:15 -0700)]
Merge pull request #42936 from batrick/i51198

pacific: msg: active_connections regression

Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agoMerge pull request #42935 from batrick/i51935
Yuri Weinstein [Wed, 1 Sep 2021 21:14:45 +0000 (14:14 -0700)]
Merge pull request #42935 from batrick/i51935

pacific: mds: add debugging when rejecting mksnap with EPERM

Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agoMerge pull request #42956 from ifed01/wip-ifed-fix-huge-omap-rename-pac
Yuri Weinstein [Wed, 1 Sep 2021 18:59:40 +0000 (11:59 -0700)]
Merge pull request #42956 from ifed01/wip-ifed-fix-huge-omap-rename-pac

pacific: os/bluestore: cap omap naming scheme upgrade transaction.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #42534 from liewegas/use-quay-pacific
Sebastian Wagner [Wed, 1 Sep 2021 10:54:21 +0000 (12:54 +0200)]
Merge pull request #42534 from liewegas/use-quay-pacific

pacific: cephadm: use quay, not docker

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
4 years agoMerge pull request #42889 from rhcs-dashboard/wip-52378-pacific
Ernesto Puerta [Wed, 1 Sep 2021 10:25:24 +0000 (12:25 +0200)]
Merge pull request #42889 from rhcs-dashboard/wip-52378-pacific

pacific: mgr/dashboard: stats=false not working when listing buckets

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
4 years agoMerge pull request #42974 from s0nea/wip-52308-pacific
Ernesto Puerta [Wed, 1 Sep 2021 09:29:40 +0000 (11:29 +0200)]
Merge pull request #42974 from s0nea/wip-52308-pacific

pacific: mgr/dashboard: don't notify for suppressed alerts

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
4 years agoMerge pull request #42898 from smithfarm/wip-52380-pacific
Yuri Weinstein [Tue, 31 Aug 2021 19:34:04 +0000 (12:34 -0700)]
Merge pull request #42898 from smithfarm/wip-52380-pacific

pacific: cmake: exclude "grafonnet-lib" target from "all"

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
4 years agoMerge pull request #42897 from ifed01/wip-ifed-fix-spurious-read-errors-warn-pac
Yuri Weinstein [Tue, 31 Aug 2021 19:31:07 +0000 (12:31 -0700)]
Merge pull request #42897 from ifed01/wip-ifed-fix-spurious-read-errors-warn-pac

pacific: os/bluestore: respect bluestore_warn_on_spurious_read_errors setting

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #42888 from callithea/wip-51951-pacific
Yuri Weinstein [Tue, 31 Aug 2021 19:30:32 +0000 (12:30 -0700)]
Merge pull request #42888 from callithea/wip-51951-pacific

pacific: Don't persist report data

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoos/bluestore: cap omap naming scheme upgrade transactoin. 42956/head
Igor Fedotov [Tue, 9 Feb 2021 15:29:01 +0000 (18:29 +0300)]
os/bluestore: cap omap naming scheme upgrade transactoin.

We shouldn't use single per-onode transaction for such an upgrade when onode's omap list is huge. This results in similarly sized WAL/SST files which are inefficient, might cause high memory usage and sometimes error-prone.

Fixes: https://tracker.ceph.com/issues/49170
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit e897fa243c1dd38329733b452872616023f14ac8)

4 years agoMerge pull request #42969 from idryomov/wip-krbd-escape-match-sysattr-pacific
Ilya Dryomov [Tue, 31 Aug 2021 14:01:29 +0000 (16:01 +0200)]
Merge pull request #42969 from idryomov/wip-krbd-escape-match-sysattr-pacific

pacific: krbd: escape udev_enumerate_add_match_sysattr values

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
4 years agomgr/dashboard: don't notify for suppressed alerts 42974/head
Tatjana Dehler [Thu, 12 Aug 2021 14:06:44 +0000 (16:06 +0200)]
mgr/dashboard: don't notify for suppressed alerts

Fixes: https://tracker.ceph.com/issues/51987
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit e9f316d678dd24e25e82474da95dbfbb72d763b5)

4 years agocephadm: quay.io for non-ceph images too 42534/head
Sage Weil [Wed, 11 Aug 2021 16:21:32 +0000 (12:21 -0400)]
cephadm: quay.io for non-ceph images too

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agocephadm,mgr/cephadm: use quay for ceph image
Sage Weil [Wed, 11 Aug 2021 16:20:24 +0000 (12:20 -0400)]
cephadm,mgr/cephadm: use quay for ceph image

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agokrbd: escape udev_enumerate_add_match_sysattr values 42969/head
Ilya Dryomov [Sat, 28 Aug 2021 09:05:28 +0000 (11:05 +0200)]
krbd: escape udev_enumerate_add_match_sysattr values

libudev uses fnmatch(3) for matching attributes, meaning that shell
glob pattern matching is employed instead of literal string matching.
Escape glob metacharacters to suppress pattern matching.

Fixes: https://tracker.ceph.com/issues/52425
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 8841029b0a4705825ba394541240b3cb4eb2cf5c)

4 years agoMerge pull request #42428 from kamoltat/wip-ksirivad-backport-pacific-42036
Kamoltat Sirivadhna [Fri, 27 Aug 2021 18:56:00 +0000 (14:56 -0400)]
Merge pull request #42428 from kamoltat/wip-ksirivad-backport-pacific-42036

pacific: mgr/pg_autoscaler: Introduce autoscaler scale-down feature
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoPendingReleaseNotes: add autoscaler profile 42428/head
Kamoltat [Thu, 26 Aug 2021 19:50:14 +0000 (19:50 +0000)]
PendingReleaseNotes: add autoscaler profile

Added autoscaler profile release notes for
>=16.2.6

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit ea094f769f554ff7d5e4a17e76130d00044ef4e4)

4 years agodoc/rados/operations/placement-groups: added bias + profile
Kamoltat [Fri, 6 Aug 2021 04:23:29 +0000 (04:23 +0000)]
doc/rados/operations/placement-groups: added bias + profile

Added documentations on the autoscale profile and bias

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 65fec053e003967aa3ffa5008730661baa46c868)

4 years agopybind/mgr/pg_autoscaler: Added PROFILE to autoscale-status
Kamoltat [Fri, 30 Jul 2021 19:29:21 +0000 (19:29 +0000)]
pybind/mgr/pg_autoscaler: Added PROFILE to autoscale-status

autoscaler now shows the PROFILE status of each pool
in `ceph osd pool autoscale-status`

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 4a2912bbad72b00ae9fe0d892f07ca760e5cefb4)

4 years agopybind/mgr/autoscaler: don't scale pools with overlapping roots
Kamoltat [Fri, 25 Jun 2021 22:40:43 +0000 (22:40 +0000)]
pybind/mgr/autoscaler: don't scale pools with overlapping roots

In the previous version of get_subtree_resource_status() in
src/pybind/mgr/pg_autoscaler/module.py we ignore overlapping
pools which in some cases if combined with the new `scale-down`
algorithm in https://github.com/ceph/ceph/pull/38805 can cause
some pools to scale up/down to inapproriate amount of pgs.

Therefore, the PR identifies the overlapping roots and prevent the pools
with such roots from scaling. This only happens with `scale-down` profile
as we see no problem with the default `scale-up` profile.

Removed the variable `pool_root` since it is not used anywhere in
the code, it only gets assigned and reassigned

Also included a unit test test_overlapping_roots.py that tests the function
identify_subtrees_and_overlaps() as well as edited test_cal_final_pg_target.py
to account for pools that contain overlapping roots, therefore, those pools
are expected not to scale.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 16eb42a1d8cef5cf008b04b27d51e13dbd6ec495)

4 years agopybind/mgr/pg_autoscaler: add typing annotations
Kefu Chai [Mon, 28 Jun 2021 03:32:41 +0000 (11:32 +0800)]
pybind/mgr/pg_autoscaler: add typing annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1b0b32e44cddacbd7acf7024181247f4251c92f9)

Conflicts:
src/mypy.ini - trivial fix

4 years agopybind/mgr/pg_autoscaler: do not index pool_root with pool_name
Kefu Chai [Mon, 28 Jun 2021 05:24:41 +0000 (13:24 +0800)]
pybind/mgr/pg_autoscaler: do not index pool_root with pool_name

pool_root is indexed by pool_id, and we never index it with pool_name.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 83f841295d86d10aadf3b4dff4e3eaa80fe139c3)

4 years agopybind/mgr/pg_autoscaler: do not create a tuple long_desc
Kefu Chai [Mon, 28 Jun 2021 04:39:12 +0000 (12:39 +0800)]
pybind/mgr/pg_autoscaler: do not create a tuple long_desc

long_desc is supposed to be a str, not a tuple of str.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit f725e09ae63702097f0eef1395391933083a9d96)

4 years agopybind/mgr/pg_autoscaler: extract CrushSubtreeResourceStatus out
Kefu Chai [Mon, 28 Jun 2021 04:28:17 +0000 (12:28 +0800)]
pybind/mgr/pg_autoscaler: extract CrushSubtreeResourceStatus out

as it also serves as part of interface of get_subtree_resource_status(),
not only its internals. to ease adding the type annotations, this class
is promoted out of the class.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 9bb55e84809e17e0aa1cb1aa4ae712a3ea4fffed)

4 years agopybind/mgr/pg_autoscaler: define commands using CLICommand
Kefu Chai [Mon, 28 Jun 2021 03:44:05 +0000 (11:44 +0800)]
pybind/mgr/pg_autoscaler: define commands using CLICommand

simpler this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4efd7400804ff42b5965ad2691223471b4ae38a7)

4 years agopybind/mgr/pg_autoscaler: define options using Option
Kefu Chai [Mon, 28 Jun 2021 03:34:09 +0000 (11:34 +0800)]
pybind/mgr/pg_autoscaler: define options using Option

more consistent and less error-prune this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b40de19c23b6b49f8d63bb3911bdebe9a2e19255)

4 years agopybind/mgr/pg_autoscaler: Added autoscale-profile feature
Kamoltat [Tue, 13 Apr 2021 21:12:40 +0000 (21:12 +0000)]
pybind/mgr/pg_autoscaler: Added autoscale-profile feature

The autoscaler by default will start out each pool with minimal
pgs and `scale-up` the pgs when there is more usage in each pool.

Users can now use the commands:

`osd pool set autoscale-profile scale-down` to make the pools
start out with a full complement of pgs and only `scale-down`
when usage ratio across the pools are not even.

`osd pool set autoscale-profile scale-up` (by default) to make the pools
start out with minimal pgs and `scale-up` the pgs when there
is more usage in each pool.

Edited KVMonitor.cc file to make the `autoscale_profile` variable
persistent.

Edited tests/test_cal_final_pg_target.py so that it takes into account
the new `profile` argument when calling cal_final_pg_target(). Also,
added some new test cases for when profile is `scale-up`

Renamed tests/test_autoscaler.py to a more appropriate name:
tests/test_cal_ratio.py

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 0e49683b538026e4c51a040950da3dbb9c7e92d5)

4 years agomgr/pg_autoscaler: avoid scale-down until there is pressure
Kamoltat [Thu, 7 Jan 2021 15:39:19 +0000 (15:39 +0000)]
mgr/pg_autoscaler: avoid scale-down until there is pressure

The autoscaler will start out with scaling each
pools to have a full complements of pgs from the start
and will only decrease it when pools need more due to
increased usage.

Introduced a unit test that tests only the
function get_final_pg_target_and_ratio() which
deals with the distrubtion of pgs amongst the
pools

Edited workunit script to reflect the change
of how pgs are calculated and distrubted.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit daeb6f6ac0c8f77ae07147f9d1e2ed18d6d8e4cc)

4 years agoMerge pull request #42914 from vshankar/wip-51544
Yuri Weinstein [Thu, 26 Aug 2021 18:38:48 +0000 (11:38 -0700)]
Merge pull request #42914 from vshankar/wip-51544

pacific: mgr/volumes: use dedicated libcephfs handles for subvolume calls and …

Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agoMerge pull request #42900 from ajarr/wip-51834-pacific
Yuri Weinstein [Thu, 26 Aug 2021 18:38:02 +0000 (11:38 -0700)]
Merge pull request #42900 from ajarr/wip-51834-pacific

pacific: mds: create file system with specific ID

Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agomds: update META_POP_READDIR/FETCH/STORE and cache_hit_rate for dump loads 42939/head
Yongseok Oh [Thu, 8 Jul 2021 06:36:19 +0000 (15:36 +0900)]
mds: update META_POP_READDIR/FETCH/STORE and cache_hit_rate for dump loads

Signed-off-by: Yongseok Oh <yongseok.oh@linecorp.com>
(cherry picked from commit 24e131a58ed255cb8f6845c020ab956450a5fc10)

4 years agomds: check rejoin_ack_gather before enter rejoin_gather_finish 42938/head
chencan [Wed, 2 Dec 2020 07:55:32 +0000 (15:55 +0800)]
mds: check rejoin_ack_gather before enter rejoin_gather_finish

Fixes: https://tracker.ceph.com/issues/48422
Signed-off-by: chencan <chen.can2@zte.com.cn>
(cherry picked from commit 88548775330f66226b4e58a34256f1cc5a72221f)

4 years agoqa/tasks/cephfs/test_sessionmap: reap connections immediately 42936/head
Sage Weil [Wed, 19 May 2021 19:27:56 +0000 (15:27 -0400)]
qa/tasks/cephfs/test_sessionmap: reap connections immediately

We have to reap connections promptly for this test to work.

This test was broken indirectly by d51d80b3234e17690061f65dc7e1515f4244a5a3,
which moved the counter decrement to reap time instead of mark_down/stop
time.

The reaping is asynchronous, so allow for a delay in the count change.

Fixes: https://tracker.ceph.com/issues/50622
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c8c5071dcd4b0b788f5e924a678095ce5dc1d7f8)

4 years agomsg/async: configurable threshold for reaping dead connections
Sage Weil [Wed, 19 May 2021 19:23:26 +0000 (15:23 -0400)]
msg/async: configurable threshold for reaping dead connections

It is helpful to set this to 1 for tests.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8129d6bb953015cc05db458afa6aa9b8f5f62614)

Conflicts:
src/common/options/global.yaml.in

4 years agomds: add debugging when rejecting mksnap with EPERM 42935/head
Patrick Donnelly [Tue, 6 Jul 2021 18:03:19 +0000 (11:03 -0700)]
mds: add debugging when rejecting mksnap with EPERM

Fixes: https://tracker.ceph.com/issues/51543
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 92ec79fd57b841c54bd9a5490031f62166ed6ab1)

4 years agoMerge pull request #42083 from kotreshhr/wip-51411-pacific
Yuri Weinstein [Wed, 25 Aug 2021 19:04:56 +0000 (12:04 -0700)]
Merge pull request #42083 from kotreshhr/wip-51411-pacific

pacific: mgr/mgr_util: switch using unshared cephfs connections whenever possible

Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agoMerge pull request #42908 from cfsnyder/wip-52330-pacific
Yuri Weinstein [Wed, 25 Aug 2021 14:43:53 +0000 (07:43 -0700)]
Merge pull request #42908 from cfsnyder/wip-52330-pacific

pacific: cls/cmpomap: empty values are 0 in U64 comparisons

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agomgr/volumes: use dedicated libcephfs handles for subvolume calls and async jobs 42914/head
Venky Shankar [Fri, 18 Jun 2021 07:13:01 +0000 (03:13 -0400)]
mgr/volumes: use dedicated libcephfs handles for subvolume calls and async jobs

Fixes: http://tracker.ceph.com/issues/51271
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit cb2883feac1a5c141a3d72120c2320f7a8ffdea8)

4 years agoMerge pull request #42860 from ideepika/wip-52336-pacific
Yuri Weinstein [Tue, 24 Aug 2021 21:54:48 +0000 (14:54 -0700)]
Merge pull request #42860 from ideepika/wip-52336-pacific

pacific: mon/PGMap: remove DIRTY field in ceph df detail when cache tiering is not in use

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #42844 from aclamk/resharding-column-options-pacific
Yuri Weinstein [Tue, 24 Aug 2021 21:54:12 +0000 (14:54 -0700)]
Merge pull request #42844 from aclamk/resharding-column-options-pacific

pacific: kv/RocksDBStore: Add handling of block_cache option for resharding

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #42615 from neha-ojha/wip-51830-pacific
Yuri Weinstein [Tue, 24 Aug 2021 21:53:25 +0000 (14:53 -0700)]
Merge pull request #42615 from neha-ojha/wip-51830-pacific

pacific: common/options: Set osd_client_message_cap to 256.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #42300 from callithea/wip-51241-pacific
Yuri Weinstein [Tue, 24 Aug 2021 21:52:25 +0000 (14:52 -0700)]
Merge pull request #42300 from callithea/wip-51241-pacific

pacific: debian/control: ceph-mgr-modules-core does not Recommend ceph-mgr-roo…

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #42578 from batrick/i51983
Yuri Weinstein [Tue, 24 Aug 2021 21:51:21 +0000 (14:51 -0700)]
Merge pull request #42578 from batrick/i51983

pacific: mon/MDSMonitor: do not pointlessly kill standbys that are incompatible with current CompatSet

Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agoMerge pull request #42096 from varshar16/wip-nfs-backport-june
Yuri Weinstein [Tue, 24 Aug 2021 21:50:20 +0000 (14:50 -0700)]
Merge pull request #42096 from varshar16/wip-nfs-backport-june

pacific: nfs backport June

Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agoMerge pull request #42909 from gregsfortytwo/wip-51290-pacific
Gregory Farnum [Tue, 24 Aug 2021 20:12:23 +0000 (13:12 -0700)]
Merge pull request #42909 from gregsfortytwo/wip-51290-pacific

pacific: mon: Sanely set the default CRUSH rule when creating pools in stretch…

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agomon: Sanely set the default CRUSH rule when creating pools in stretch mode 42909/head
Greg Farnum [Thu, 17 Jun 2021 19:56:20 +0000 (19:56 +0000)]
mon: Sanely set the default CRUSH rule when creating pools in stretch mode

If we get a pool create request while in stretch mode that does not explicitly
specify a crush rule, look at the stretch-mode pools and their rules, and
select the most common one.

Also update set_up_stretch_mode.sh to add a few more rules that let me test
this locally.

Fixes: https://tracker.ceph.com/issues/51270
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 4f9fdca136bde10479fc173c9bcf4288ea960ace)