]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agoMerge pull request #44087 from guits/guits-fix-cv-rootfs
Sebastian Wagner [Wed, 1 Dec 2021 09:33:01 +0000 (10:33 +0100)]
Merge pull request #44087 from guits/guits-fix-cv-rootfs

ceph-volume: remove --root param from nsenter cmd

Reviewed-by: Sébastien Han <seb@redhat.com>
3 years agoMerge pull request #44143 from devlikai/master
Sebastian Wagner [Wed, 1 Dec 2021 09:24:00 +0000 (10:24 +0100)]
Merge pull request #44143 from devlikai/master

doc/mgr/diskprediction: fix a typo.

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #44116 from lxbsz/caps_doc1
Venky Shankar [Wed, 1 Dec 2021 09:01:18 +0000 (14:31 +0530)]
Merge pull request #44116 from lxbsz/caps_doc1

doc: fix the style of the cephfs capability doc

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44129 from spdfnet/patch-1
Sebastian Wagner [Wed, 1 Dec 2021 08:48:42 +0000 (09:48 +0100)]
Merge pull request #44129 from spdfnet/patch-1

doc: fix typo in cephadm host management

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #44038 from lxbsz/wip-53082
Venky Shankar [Wed, 1 Dec 2021 04:08:46 +0000 (09:38 +0530)]
Merge pull request #44038 from lxbsz/wip-53082

client: fix crash when iterating and deleting sessions

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #43878 from jtlayton/wip-53214
Venky Shankar [Wed, 1 Dec 2021 04:08:05 +0000 (09:38 +0530)]
Merge pull request #43878 from jtlayton/wip-53214

qa: account for split of the kclient "metrics" debugfs file

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #43850 from batrick/i53194
Venky Shankar [Wed, 1 Dec 2021 04:07:00 +0000 (09:37 +0530)]
Merge pull request #43850 from batrick/i53194

mds: defer messages to bootstrapping ranks

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #43297 from yongseokoh/test-dir-max-entries
Venky Shankar [Wed, 1 Dec 2021 04:04:54 +0000 (09:34 +0530)]
Merge pull request #43297 from yongseokoh/test-dir-max-entries

qa: add mds_dir_max_entries workunit test case

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #41334 from vshankar/wip-kcephfs-new-mount-syntax
Venky Shankar [Wed, 1 Dec 2021 04:03:29 +0000 (09:33 +0530)]
Merge pull request #41334 from vshankar/wip-kcephfs-new-mount-syntax

mount: introduce new device mount syntax

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #42710 from yehudasa/wip-rgw-mgr-module
Yehuda Sadeh [Tue, 30 Nov 2021 22:32:39 +0000 (14:32 -0800)]
Merge pull request #42710 from yehudasa/wip-rgw-mgr-module

mgr/rgw: new rgw manager module

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge PR #44140 into master
Sage Weil [Tue, 30 Nov 2021 20:19:55 +0000 (15:19 -0500)]
Merge PR #44140 into master

* refs/pull/44140/head:
python-common/ceph/deployment/drive_group: fix 'orch ls --format yaml'

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agopython-common/rgw: fix style issues 42710/head
Yehuda Sadeh [Tue, 30 Nov 2021 16:45:06 +0000 (08:45 -0800)]
python-common/rgw: fix style issues

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
3 years agodoc/rgw: fix docs build
Yehuda Sadeh [Mon, 29 Nov 2021 23:31:42 +0000 (15:31 -0800)]
doc/rgw: fix docs build

Workaround rgw modules conflict, as there are two separate modules named
rgw: src/pybind/rgw, src/pybind/mgr/rgw

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
3 years agoMerge pull request #44072 from ronen-fr/wip-rf-latescrub-count
Ronen Friedman [Tue, 30 Nov 2021 17:43:13 +0000 (19:43 +0200)]
Merge pull request #44072 from ronen-fr/wip-rf-latescrub-count

qa/standalone: osd-scrub-repair.sh: fix expected "not scrubbed since"…

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
3 years agoMerge pull request #44118 from sebastian-philipp/cephadm-inventory-changed-while...
Sebastian Wagner [Tue, 30 Nov 2021 16:50:15 +0000 (17:50 +0100)]
Merge pull request #44118 from sebastian-philipp/cephadm-inventory-changed-while-iterated

mgr/cephadm: Inventory: Fix `dictionary changed size during iteration `

Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #44115 from rhcs-dashboard/fix-tooltip-fast-diff
Ernesto Puerta [Tue, 30 Nov 2021 13:54:20 +0000 (14:54 +0100)]
Merge pull request #44115 from rhcs-dashboard/fix-tooltip-fast-diff

mgr/dashboard: avoid tooltip if disk_usage=null and fast-diff enabled

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44083 from wangbo-yw/wangbo-yw
Ernesto Puerta [Tue, 30 Nov 2021 13:53:59 +0000 (14:53 +0100)]
Merge pull request #44083 from wangbo-yw/wangbo-yw

mgr/dashboard: add some test for controllers/pool.py

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #43855 from zhangmengqianyw/zmq-unittest
Ernesto Puerta [Tue, 30 Nov 2021 13:52:44 +0000 (14:52 +0100)]
Merge pull request #43855 from zhangmengqianyw/zmq-unittest

mgr/dashboard:add unittest in test_osd.py

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #44082 from pcuzner/fix-prometheus-timings
Sebastian Wagner [Tue, 30 Nov 2021 09:48:48 +0000 (10:48 +0100)]
Merge pull request #44082 from pcuzner/fix-prometheus-timings

mgr/prometheus: Fix the per method stats exported

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agodoc/mgr/diskprediction: fix a typo. 44143/head
Kyle [Tue, 30 Nov 2021 07:27:26 +0000 (15:27 +0800)]
doc/mgr/diskprediction: fix a typo.

doc: remove extra comma.

This commit remove extra comma of "To disable prediction,:".

Fixes: https://tracker.ceph.com/issues/53433
Signed-off-by: devlikai <likai_lc@inspur.com>
3 years agoMerge pull request #43529 from curtbruns/rgw-lua-storageclass
Yuval Lifshitz [Tue, 30 Nov 2021 06:57:23 +0000 (08:57 +0200)]
Merge pull request #43529 from curtbruns/rgw-lua-storageclass

rgw/lua: allow read/write of StorageClass field

3 years agoMerge pull request #42504 from arjune123/rgw-bug-fixes
Yuval Lifshitz [Tue, 30 Nov 2021 06:56:26 +0000 (08:56 +0200)]
Merge pull request #42504 from arjune123/rgw-bug-fixes

rgw/notification: assigning the value of zonegroup to awsRegion

3 years agodoc, man: document new device syntax for kclient mount 41334/head
Venky Shankar [Fri, 14 May 2021 04:38:19 +0000 (00:38 -0400)]
doc, man: document new device syntax for kclient mount

Fxies: http://tracker.ceph.com/issues/47277
Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agoqa: use v1 mount-syntax for upgrade tests
Venky Shankar [Tue, 30 Nov 2021 06:06:31 +0000 (01:06 -0500)]
qa: use v1 mount-syntax for upgrade tests

Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agotest: mount kclient using new-style (v2) syntax
Venky Shankar [Thu, 11 Nov 2021 04:15:41 +0000 (23:15 -0500)]
test: mount kclient using new-style (v2) syntax

But, do not throw away the old style mount syntax since we would
want to continue testing it since users (scripts) might still be
using it.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agomount: use mds_mode=prefer-crc with new-style mount syntax if possible
Venky Shankar [Wed, 10 Nov 2021 10:02:51 +0000 (05:02 -0500)]
mount: use mds_mode=prefer-crc with new-style mount syntax if possible

Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agomount: optionally do not fall-back to old-style mount syntax
Venky Shankar [Mon, 8 Nov 2021 09:12:22 +0000 (04:12 -0500)]
mount: optionally do not fall-back to old-style mount syntax

Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agomount: introduce new mount syntax
Venky Shankar [Thu, 6 May 2021 04:59:27 +0000 (00:59 -0400)]
mount: introduce new mount syntax

Old mount device (source) have the following problems:

- mounts to the same cluster but with different fsnames
  and/or creds have identical device string which can
  confuse xfstests.

- device shown in /proc/mounts is different that what
  is used to mount.

New proposed syntax is as follows:

  # mount -t ceph cephuser@<cluster-fsid>.mycephfs2=/path

The cluster-id is optional but is always passed to the kernel
via mount() syscall (by fetching it from cluster configuration
file if available).

The mount helper tries the mounting the file system with
the new device syntax, falling back to using the old
syntax if required.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agomount: record cluster fsid when reading ceph config file
Venky Shankar [Thu, 20 May 2021 12:11:05 +0000 (08:11 -0400)]
mount: record cluster fsid when reading ceph config file

This will be required when switching to new mount device
syntax when cluster fsid will be passed to kernel as mount
option.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agomount: accept monitor host (`mon_host`) mount option
Venky Shankar [Mon, 3 May 2021 10:30:34 +0000 (06:30 -0400)]
mount: accept monitor host (`mon_host`) mount option

With new mount device syntax monitor host(s) can be passed
during mount as option parameter. This option is not mandatory
as it can be fetched from the cluster configuration file if
available.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agopython-common/ceph/deployment/drive_group: fix 'orch ls --format yaml' 44140/head
Sage Weil [Tue, 30 Nov 2021 04:25:43 +0000 (23:25 -0500)]
python-common/ceph/deployment/drive_group: fix 'orch ls --format yaml'

In f76c02a658ed1de54a380381932bb58b6ab91d41 we may leave out the placement
key when it is empty.  If we take that path, we need to tolerate specs
that are missing that key.

Fixes: f76c02a658ed1de54a380381932bb58b6ab91d41
Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoMerge pull request #43760 from yaarith/release-checklists
Josh Durgin [Tue, 30 Nov 2021 02:18:43 +0000 (18:18 -0800)]
Merge pull request #43760 from yaarith/release-checklists

doc/dev/release-checklist: add manual review of telemetry reports

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge PR #44018 into master pacific-16.2.7_RC1
Sage Weil [Mon, 29 Nov 2021 19:18:33 +0000 (14:18 -0500)]
Merge PR #44018 into master

* refs/pull/44018/head:
mon: fix quorum_age() regression

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge PR #44030 into master
Sage Weil [Mon, 29 Nov 2021 18:56:43 +0000 (13:56 -0500)]
Merge PR #44030 into master

* refs/pull/44030/head:
mgr/cephadm: add some debug output for serve loop
ceph-volume: adjust arguments for 'ceph-volume raw activate'
ceph-volume: add raw support for db/wal for list and activate

Reviewed-by: Sébastien Han <seb@redhat.com>
3 years agoMerge PR #44107 into master
Sage Weil [Mon, 29 Nov 2021 18:56:28 +0000 (13:56 -0500)]
Merge PR #44107 into master

* refs/pull/44107/head:
qa/tasks/cephadm_cases/test_cli: fix test_daemon_restart

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agomgr/cephadm: add some debug output for serve loop 44030/head
Sage Weil [Wed, 24 Nov 2021 14:17:03 +0000 (09:17 -0500)]
mgr/cephadm: add some debug output for serve loop

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoceph-volume: adjust arguments for 'ceph-volume raw activate'
Sage Weil [Sat, 20 Nov 2021 15:19:36 +0000 (10:19 -0500)]
ceph-volume: adjust arguments for 'ceph-volume raw activate'

Take a list of devices, so that we can selectively activate a raw osd
with db/wal.

Remove the argument type kludge introduced in 2c228a9a409176c0f1679f176443fd3ead219c7a
since it is no longer needed.

Note that we're making this change because (1) it allows db/wal and (2)
because there are no known users of 'raw activate'.  The only known user
is via 'ceph-volume activate' and we've fixed that caller in this commit.

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoceph-volume: add raw support for db/wal for list and activate
Sage Weil [Fri, 19 Nov 2021 20:15:18 +0000 (15:15 -0500)]
ceph-volume: add raw support for db/wal for list and activate

Currently 'prepare' doesn't support db/wal, but we want it in list and
activate because 'ceph-volume activate ...' tries raw before lvm.

Note that I'm not sure we really want to accept --block.db and --block.wal
here at all.

Fixes: 3d7ceec684b0ac5b83fae4c397b134236fac485e
Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoqa/tasks/cephadm_cases/test_cli: fix test_daemon_restart 44107/head
Sage Weil [Thu, 25 Nov 2021 14:10:28 +0000 (08:10 -0600)]
qa/tasks/cephadm_cases/test_cli: fix test_daemon_restart

We cannot schedule a daemon start if there is another daemon action
with a higher priority (including stop) scheduled.  However,
that state isn't cleared until *after* the osd goes down, the
systemctl command returns, and mgr/cephadm gets around to updating
the inventory scheduled_daemon_action state.

Semi-fix: (1) wait for the orch status to change, and then (2)
wait a few more seconds after that.

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoMerge pull request #44100 from adk3798/infer-config-fix
Sebastian Wagner [Mon, 29 Nov 2021 16:03:22 +0000 (17:03 +0100)]
Merge pull request #44100 from adk3798/infer-config-fix

cephadm: only infer conf from mon if fsid matches

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #44101 from adk3798/agent-down-multiplier
Sebastian Wagner [Mon, 29 Nov 2021 14:35:32 +0000 (15:35 +0100)]
Merge pull request #44101 from adk3798/agent-down-multiplier

mgr/cephadm: agent: allow agent down multiplier to be configured

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agodoc: fix typo in cephadm host management 44129/head
Brian_P [Mon, 29 Nov 2021 14:13:17 +0000 (14:13 +0000)]
doc: fix typo in cephadm host management

3 years agoMerge pull request #42378 from sebastian-philipp/no-grafana-admin
Sebastian Wagner [Mon, 29 Nov 2021 13:08:22 +0000 (14:08 +0100)]
Merge pull request #42378 from sebastian-philipp/no-grafana-admin

mgr/cephadm: Add GrafanaSpec.initial_admin_password

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
3 years agoMerge pull request #44011 from adk3798/repr-device
Sebastian Wagner [Mon, 29 Nov 2021 08:50:28 +0000 (09:50 +0100)]
Merge pull request #44011 from adk3798/repr-device

python-common: add string representation for Device and DeviceSelection classes

Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #44114 from orozery/librbd-memory-leaks
Mykola Golub [Mon, 29 Nov 2021 07:36:08 +0000 (09:36 +0200)]
Merge pull request #44114 from orozery/librbd-memory-leaks

librbd: fix various memory leaks

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #43530 from myoungwon/wip-seastore-nvme-device
Samuel Just [Mon, 29 Nov 2021 04:21:39 +0000 (20:21 -0800)]
Merge pull request #43530 from myoungwon/wip-seastore-nvme-device

seastore: add nvme commands to nvme device class

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44068 from rzarzynski/wip-crimson-weakref-in-sharedlru
Samuel Just [Mon, 29 Nov 2021 01:59:54 +0000 (17:59 -0800)]
Merge pull request #44068 from rzarzynski/wip-crimson-weakref-in-sharedlru

crimson/common: don't assume pointer-from-SharedLRU can't outlive it.

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44110 from rzarzynski/wip-crimson-alienstore-syncumountread
Samuel Just [Mon, 29 Nov 2021 00:36:47 +0000 (16:36 -0800)]
Merge pull request #44110 from rzarzynski/wip-crimson-alienstore-syncumountread

crimson/os: fix a shutdown-related race condition in AlienStore.

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agodoc: fix the style of the cephfs capability doc 44116/head
Xiubo Li [Fri, 26 Nov 2021 08:27:17 +0000 (16:27 +0800)]
doc: fix the style of the cephfs capability doc

Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agoMerge pull request #43481 from myoungwon/wip-dedup-tool-repair
Samuel Just [Mon, 29 Nov 2021 00:10:46 +0000 (16:10 -0800)]
Merge pull request #43481 from myoungwon/wip-dedup-tool-repair

tool: add repair command to ceph-dedup-tool

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agolibrbd/crypto: remove unused member from ShutDownCryptoRequest 44114/head
Or Ozeri [Thu, 25 Nov 2021 18:17:26 +0000 (20:17 +0200)]
librbd/crypto: remove unused member from ShutDownCryptoRequest

m_crypto is not used - remove it.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agotest/librbd: fix memory leak in TestMockShutDownCryptoRequest
Or Ozeri [Thu, 25 Nov 2021 17:53:52 +0000 (19:53 +0200)]
test/librbd: fix memory leak in TestMockShutDownCryptoRequest

fix memory leak in TestMockShutDownCryptoRequest.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agotest/librbd: fix memory leak in TestMockCryptoLoadRequest
Or Ozeri [Thu, 25 Nov 2021 17:52:48 +0000 (19:52 +0200)]
test/librbd: fix memory leak in TestMockCryptoLoadRequest

fix memory leak in TestMockCryptoLoadRequest.CryptoAlreadyLoaded

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agotest/librbd: fix memory leak in TestMockCryptoCryptoObjectDispatch
Or Ozeri [Thu, 25 Nov 2021 17:51:45 +0000 (19:51 +0200)]
test/librbd: fix memory leak in TestMockCryptoCryptoObjectDispatch

fix memory leak in TestMockCryptoCryptoObjectDispatch.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agolibrbd/crypto: fix memory leak in openssl/DataCryptor
Or Ozeri [Thu, 25 Nov 2021 13:49:33 +0000 (15:49 +0200)]
librbd/crypto: fix memory leak in openssl/DataCryptor

Re-initializing the same datacryptor, causes a memory leak of the old encryption key.
This commit fixes this issue.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agolibrbd/crypto: fix memory leak in ShutDownCryptoRequest
Or Ozeri [Thu, 25 Nov 2021 13:47:54 +0000 (15:47 +0200)]
librbd/crypto: fix memory leak in ShutDownCryptoRequest

If crypto object dispatch does not exist, a context pointer is leaked.
This commit fixes this issue.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agotest/librbd: fix memory leak in TestMockParentCacheObjectDispatch
Or Ozeri [Thu, 25 Nov 2021 13:45:00 +0000 (15:45 +0200)]
test/librbd: fix memory leak in TestMockParentCacheObjectDispatch

fix memory leak in TestMockParentCacheObjectDispatch.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agotest/librbd: fix memory leak in TestMockCryptoLuksFormatRequest
Or Ozeri [Thu, 25 Nov 2021 13:44:09 +0000 (15:44 +0200)]
test/librbd: fix memory leak in TestMockCryptoLuksFormatRequest

fix memory leak in TestMockCryptoLuksFormatRequest.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agotest/librbd: fix memory leak in TestMockCryptoLuksLoadRequest
Or Ozeri [Thu, 25 Nov 2021 13:42:22 +0000 (15:42 +0200)]
test/librbd: fix memory leak in TestMockCryptoLuksLoadRequest

fix memory leak in TestMockCryptoLuksLoadRequest.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agotest/librbd: fix bad TearDown in TestCryptoOpensslDataCryptor
Or Ozeri [Thu, 25 Nov 2021 13:11:36 +0000 (15:11 +0200)]
test/librbd: fix bad TearDown in TestCryptoOpensslDataCryptor

Fix the TearDown function in TestCryptoOpensslDataCryptor
to call the right class parent function.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agotest/librbd: fix memory leak in TestCryptoOpensslDataCryptor
Or Ozeri [Thu, 25 Nov 2021 13:10:14 +0000 (15:10 +0200)]
test/librbd: fix memory leak in TestCryptoOpensslDataCryptor

One of the tests leaks an encryption context.
This commit fixes this issue.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agolibrbd/crypto: fix memory leak in when DataCryptor fails
Or Ozeri [Thu, 25 Nov 2021 13:08:47 +0000 (15:08 +0200)]
librbd/crypto: fix memory leak in when DataCryptor fails

If DataCryptor fails, either in init_context or update_context,
the encryption context is not returned, which causes a memory leak.
This commit fixes this issue.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agotest/librbd: fix memory leak in TestMockCryptoBlockCrypto
Or Ozeri [Thu, 25 Nov 2021 13:05:55 +0000 (15:05 +0200)]
test/librbd: fix memory leak in TestMockCryptoBlockCrypto

fix memory leak in TestMockCryptoBlockCrypto.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
3 years agoMerge PR #43997 into master
Sage Weil [Fri, 26 Nov 2021 20:15:51 +0000 (15:15 -0500)]
Merge PR #43997 into master

* refs/pull/43997/head:
mgr/cephadm: make logging about agent less verbose

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge PR #44079 into master
Sage Weil [Fri, 26 Nov 2021 20:15:42 +0000 (15:15 -0500)]
Merge PR #44079 into master

* refs/pull/44079/head:
mgr/cephadm: skip osd_stats check if osd removal queue is empty

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge PR #44075 into master
Sage Weil [Fri, 26 Nov 2021 20:15:27 +0000 (15:15 -0500)]
Merge PR #44075 into master

* refs/pull/44075/head:
mgr/cephadm: drop osdspec_affinity tracking

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge PR #44073 into master
Sage Weil [Fri, 26 Nov 2021 20:15:12 +0000 (15:15 -0500)]
Merge PR #44073 into master

* refs/pull/44073/head:
pybind/mgr/mgr_module: cache mgr_ip

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agomgr/cephadm: simplify HostCache.get_daemon_types 44118/head
Sebastian Wagner [Fri, 26 Nov 2021 17:00:10 +0000 (18:00 +0100)]
mgr/cephadm: simplify HostCache.get_daemon_types

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agomgr/cephadm: Inventory: Fix `dictionary changed size during iteration`
Sebastian Wagner [Fri, 26 Nov 2021 16:53:14 +0000 (17:53 +0100)]
mgr/cephadm: Inventory: Fix `dictionary changed size during iteration`

Use `.copy()` for that.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge PR #43936 into master
Sage Weil [Fri, 26 Nov 2021 15:38:58 +0000 (10:38 -0500)]
Merge PR #43936 into master

* refs/pull/43936/head:
qa/tasks/cephadm: pull image to all hosts in parallel
qa/tasks/cephadm: add hosts via mon remote
qa/tasks/cephadm: use shortname for remote directory
qa/tasks/cephadm: deploy no more than 5 mons in roleless mode
qa/tasks/radosbench: default clients to all clients (not client.0)
qa/tasks/ceph_manager: parallelize flush_pg_stats()
qa/suites/big: remove thrasher
qa/suites/big: update for cephadm

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge PR #44080 into master
Sage Weil [Fri, 26 Nov 2021 15:37:27 +0000 (10:37 -0500)]
Merge PR #44080 into master

* refs/pull/44080/head:
mgr/cephadm: record when finished with scheduled daemon action

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agomgr/cephadm: grafana.ini: Set `cookie_secure = true` 42378/head
Sebastian Wagner [Fri, 26 Nov 2021 10:15:51 +0000 (11:15 +0100)]
mgr/cephadm: grafana.ini: Set `cookie_secure = true`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agomgr/cephadm: Add GrafanaSpec.initial_admin_password
Sebastian Wagner [Fri, 16 Jul 2021 14:20:32 +0000 (16:20 +0200)]
mgr/cephadm: Add GrafanaSpec.initial_admin_password

By default, we're not creating any admin accout for Grafana now,
but we're adding an option to set the grafana password manually using:

```yaml
service_type: grafana
spec:
  initial_admin_password: mypassword
```

Users can then easily log into Grafana with the given password.

Fixes: https://tracker.ceph.com/issues/48291
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agopython-common: Reparent AlertManagerSpec to MonitoringSpec
Sebastian Wagner [Fri, 23 Jul 2021 01:20:43 +0000 (03:20 +0200)]
python-common: Reparent AlertManagerSpec to MonitoringSpec

And remove duplicated members

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agopython-common: Move AlertManagerSpec below MonitoringSpec
Sebastian Wagner [Fri, 23 Jul 2021 01:15:53 +0000 (03:15 +0200)]
python-common: Move AlertManagerSpec below MonitoringSpec

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agopython-common: test_yaml(): add a few tests
Sebastian Wagner [Fri, 23 Jul 2021 07:05:59 +0000 (09:05 +0200)]
python-common: test_yaml(): add a few tests

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agopython-common: prettify `yaml.dump(MonitoringSpec())`
Sebastian Wagner [Fri, 23 Jul 2021 00:59:59 +0000 (02:59 +0200)]
python-common: prettify `yaml.dump(MonitoringSpec())`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agopyhton-common: move some tests from cephadm/test_spec.py
Sebastian Wagner [Fri, 23 Jul 2021 00:36:27 +0000 (02:36 +0200)]
pyhton-common: move some tests from cephadm/test_spec.py

Cause they don't have any dependencies to cephadm

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agomgr/dashboard: avoid tooltip if disk_usage=null and fast-diff enabled 44115/head
Avan Thakkar [Thu, 25 Nov 2021 19:59:17 +0000 (01:29 +0530)]
mgr/dashboard: avoid tooltip if disk_usage=null and fast-diff enabled

Fixes: https://tracker.ceph.com/issues/53404
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
3 years agomgr/dashboard: add some test for controllers/pool.py 44083/head
wangbo-yw [Wed, 24 Nov 2021 02:30:05 +0000 (21:30 -0500)]
mgr/dashboard: add some test for controllers/pool.py

Signed-off-by: wangbo-yw <wangbo2_yewu@cmss.chinamobile.com>
3 years agoMerge pull request #44106 from sebastian-philipp/mgr-tox-37
Sebastian Wagner [Thu, 25 Nov 2021 16:54:26 +0000 (17:54 +0100)]
Merge pull request #44106 from sebastian-philipp/mgr-tox-37

mgr/tox.ini: Add python 3.7 environment

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #43943 from sebastian-philipp/osd-memeory-hyperconverged
Sebastian Wagner [Thu, 25 Nov 2021 16:27:26 +0000 (17:27 +0100)]
Merge pull request #43943 from sebastian-philipp/osd-memeory-hyperconverged

doc/cephadm: OSD memory autotuning for hyperconverged

Reviewed-by: Adam King <adking@redhat.com>
3 years agocrimson/os: fix a shutdown-related race condition in AlienStore. 44110/head
Radoslaw Zarzynski [Wed, 24 Nov 2021 15:41:22 +0000 (15:41 +0000)]
crimson/os: fix a shutdown-related race condition in AlienStore.

This is supposed to tackle crashes like the following one:

```
INFO  2021-11-17 16:33:12,048 [shard 0] alienstore - stat
...
DEBUG 2021-11-17 16:33:12,789 [shard 0] ms - [osd.2(hb_front) v2:0.0.0.0:6813/34383 >> osd.0 v2:127.0.0.1:6809/34293@56992] closed!
DEBUG 2021-11-17 16:33:12,791 [shard 0] ms - [osd.2(hb_front) v2:0.0.0.0:6813/34383@53359 >> osd.7 v2:0.0.0.0:6815/34448] closed!
INFO  2021-11-17 16:33:12,795 [shard 0] alienstore - umount
INFO  2021-11-17 16:33:12,804 [shard 0] osd - osd.2: committed_osd_maps(23, 62)
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-8896-gf35358f1/rpm/el8/BUILD/ceph-17.0.0-8896-gf35358f1/src/rocksdb/db/db_impl/db_impl.cc:1615: rocksdb::Status rocksdb::DBImpl::GetImpl(const rocksdb::ReadOptions&, const rocksdb::Slice&, rocksdb::DBImpl::GetImplOptions&): Assertion `get_impl_options.column_family' failed.
Aborting.
Backtrace:
INFO  2021-11-17 16:33:13,542 [shard 0] ms - [osd.2(cluster) v2:172.21.15.17:6804/34383 >> osd.3 v2:172.21.15.17:6806/34387@50001] execute_ready(): fault at READY with nothing to send, going to STANDBY -- std::system_error (error crimson::net:4, read eof)
DEBUG 2021-11-17 16:33:13,542 [shard 0] ms - [osd.2(cluster) v2:172.21.15.17:6804/34383 >> osd.3 v2:172.21.15.17:6806/34387@50001] TRIGGER STANDBY, was READY
 0# gsignal in /lib64/libc.so.6
 1# abort in /lib64/libc.so.6
 2# 0x00007F12FA13FC89 in /lib64/libc.so.6
 3# 0x00007F12FA14DA76 in /lib64/libc.so.6
 4# rocksdb::DBImpl::GetImpl(rocksdb::ReadOptions const&, rocksdb::Slice const&, rocksdb::DBImpl::GetImplOptions&) in ceph-osd
 5# rocksdb::DBImpl::Get(rocksdb::ReadOptions const&, rocksdb::ColumnFamilyHandle*, rocksdb::Slice const&, rocksdb::PinnableSlice*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) in ceph-osd
 6# rocksdb::DBImpl::Get(rocksdb::ReadOptions const&, rocksdb::ColumnFamilyHandle*, rocksdb::Slice const&, rocksdb::PinnableSlice*) in ceph-osd
 7# RocksDBStore::get(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*, unsigned long, ceph::buffer::v15_2_0::list*) in ceph-osd
 8# BlueStore::Collection::get_onode(ghobject_t const&, bool, bool) in ceph-osd
 9# BlueStore::read(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ghobject_t const&, unsigned long, unsigned long, ceph::buffer::v15_2_0::list&, unsigned int) in ceph-osd
10# 0x00005584E516577F in ceph-osd
11# crimson::os::ThreadPool::loop(std::chrono::duration<long, std::ratio<1l, 1000l> >, unsigned long) in ceph-osd
12# 0x00005584E54E71E9 in ceph-osd
13# 0x00007F12FB861BA3 in /lib64/libstdc++.so.6
14# 0x00007F12FBB3C14A in /lib64/libpthread.so.0
15# clone in /lib64/libc.so.6
Content of /proc/self/maps:
7fff7000-8fff7000 rw-p 00000000 00:00 0
```

The problem happened in RocksDB:

```cpp
Status DBImpl::GetImpl(const ReadOptions& read_options, const Slice& key,
                       GetImplOptions& get_impl_options) {
  assert(get_impl_options.value != nullptr ||
         get_impl_options.merge_operands != nullptr);

  assert(get_impl_options.column_family);
  // ...
```

```cpp
tatus DBImpl::Get(const ReadOptions& read_options,
                   ColumnFamilyHandle* column_family, const Slice& key,
                   PinnableSlice* value, std::string* timestamp) {
  GetImplOptions get_impl_options;
  get_impl_options.column_family = column_family;
  get_impl_options.value = value;
  get_impl_options.timestamp = timestamp;
  Status s = GetImpl(read_options, key, get_impl_options);
  return s;
}
```

```cpp
int RocksDBStore::get(
  const string& prefix,
  const char *key,
  size_t keylen,
  bufferlist *out)
{
  ceph_assert(out && (out->length() == 0));
  utime_t start = ceph_clock_now();
  int r = 0;
  rocksdb::PinnableSlice value;
  rocksdb::Status s;
  auto cf = get_cf_handle(prefix, key, keylen);
  if (cf) {
    s = db->Get(rocksdb::ReadOptions(),
                cf,
                rocksdb::Slice(key, keylen),
                &value);
  } else {
    string k;
    combine_strings(prefix, key, keylen, &k);
    s = db->Get(rocksdb::ReadOptions(),
                default_cf,
                rocksdb::Slice(k),
                &value);
  }
  // ...
```

It may be explained by a race condition between `AlienStore::stat()`
and `AlienStore::umount()`. Umounting a BlueStore means nullifying
`default_cf`:

```cpp
void RocksDBStore::close()
{
  // ...
  default_cf = nullptr;
  delete db;
  db = nullptr;
}
```

```
INFO  2021-11-17 16:33:12,048 [shard 0] alienstore - stat
...
INFO  2021-11-17 16:33:12,795 [shard 0] alienstore - umount
INFO  2021-11-17 16:33:12,804 [shard 0] osd - osd.2: committed_osd_maps(23, 62)
```

Although `AlienStore` synchronizes `umount()` and `do_transaction()`
with a `seastar::gate`, it lacks similar mechanism for read-like operations.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoqa/tasks/cephadm: pull image to all hosts in parallel 43936/head
Sage Weil [Mon, 15 Nov 2021 18:00:52 +0000 (12:00 -0600)]
qa/tasks/cephadm: pull image to all hosts in parallel

This doesn't affect bootstrap, but it does mean we avoid any delay
the first time we cephadm.shell on some non-boostrap host.

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoqa/tasks/cephadm: add hosts via mon remote
Sage Weil [Mon, 15 Nov 2021 17:55:52 +0000 (11:55 -0600)]
qa/tasks/cephadm: add hosts via mon remote

If we use a new remote for each shell command, we end up waiting
for the image to pull on every host in sequence.

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoqa/tasks/cephadm: use shortname for remote directory
Sage Weil [Fri, 12 Nov 2021 20:52:46 +0000 (14:52 -0600)]
qa/tasks/cephadm: use shortname for remote directory

This aligns with what the ceph and syslog tasks do.

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoqa/tasks/cephadm: deploy no more than 5 mons in roleless mode
Sage Weil [Wed, 10 Nov 2021 20:48:13 +0000 (14:48 -0600)]
qa/tasks/cephadm: deploy no more than 5 mons in roleless mode

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoqa/tasks/radosbench: default clients to all clients (not client.0)
Sage Weil [Wed, 10 Nov 2021 17:27:53 +0000 (11:27 -0600)]
qa/tasks/radosbench: default clients to all clients (not client.0)

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoqa/tasks/ceph_manager: parallelize flush_pg_stats()
Sage Weil [Wed, 10 Nov 2021 17:23:51 +0000 (11:23 -0600)]
qa/tasks/ceph_manager: parallelize flush_pg_stats()

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoqa/suites/big: remove thrasher
Sage Weil [Wed, 10 Nov 2021 16:35:39 +0000 (10:35 -0600)]
qa/suites/big: remove thrasher

This doesn't work with roleless (yet)

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agoqa/suites/big: update for cephadm
Sage Weil [Mon, 8 Nov 2021 15:29:41 +0000 (09:29 -0600)]
qa/suites/big: update for cephadm

Signed-off-by: Sage Weil <sage@newdream.net>
3 years agomgr/cephadm/tests: remove `_deploy_cephadm_binary` 44106/head
Sebastian Wagner [Thu, 25 Nov 2021 12:29:01 +0000 (13:29 +0100)]
mgr/cephadm/tests: remove `_deploy_cephadm_binary`

(not needed)

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agomgr/tox.ini: Add python 3.7 environment
Sebastian Wagner [Thu, 25 Nov 2021 12:22:06 +0000 (13:22 +0100)]
mgr/tox.ini: Add python 3.7 environment

Plus fixes.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agomgr/cephadm: agent: allow agent down multiplier to be configured 44101/head
Adam King [Wed, 24 Nov 2021 23:52:10 +0000 (18:52 -0500)]
mgr/cephadm: agent: allow agent down multiplier to be configured

Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: only infer conf from mon if fsid matches 44100/head
Adam King [Wed, 24 Nov 2021 22:23:01 +0000 (17:23 -0500)]
cephadm: only infer conf from mon if fsid matches

fixes: https://tracker.ceph.com/issues/53394

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/rgw: ignore mypy errors
Yehuda Sadeh [Wed, 24 Nov 2021 20:51:12 +0000 (12:51 -0800)]
mgr/rgw: ignore mypy errors

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
3 years agomgr/rgw: use tool_exec instead of directly spawning commands
Yehuda Sadeh [Mon, 22 Nov 2021 20:02:16 +0000 (12:02 -0800)]
mgr/rgw: use tool_exec instead of directly spawning commands

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
3 years agodocs: document mgr/rgw module
Yehuda Sadeh [Tue, 2 Nov 2021 14:13:55 +0000 (07:13 -0700)]
docs: document mgr/rgw module

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
3 years agomgr/rgw: change zone-creds cli commands
Yehuda Sadeh [Mon, 1 Nov 2021 16:33:29 +0000 (09:33 -0700)]
mgr/rgw: change zone-creds cli commands

 - ceph rgw zone-creds create
 - ceph rgw zone-creds remove

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
3 years agorgwam: reorganize code
Yehuda Sadeh [Mon, 1 Nov 2021 15:45:39 +0000 (08:45 -0700)]
rgwam: reorganize code

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>