]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agocrimson/osd: write the 'ready' meta on OSD::mkfs(). 43355/head
Radoslaw Zarzynski [Wed, 29 Sep 2021 20:12:18 +0000 (20:12 +0000)]
crimson/osd: write the 'ready' meta on OSD::mkfs().

This is for the compliance with the classical OSD. No problem
with lack of this file has been identified yet.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/osd: write the 'osd_key' meta on OSD::mkfs().
Radoslaw Zarzynski [Wed, 29 Sep 2021 20:00:20 +0000 (20:00 +0000)]
crimson/osd: write the 'osd_key' meta on OSD::mkfs().

This commit fixes an issue identified during the Rook-crimson effort.

Missing the `write_meta()` on `osd_key` made the CephX inoperational
because of imposibility to load the keyring. Disabling CephX in turn
caused the auth method negotation to fail when reaching out to a monitor.

```
ERROR 2021-09-28 21:19:46,598 [shard 0] none - auth: unable to find a keyring on /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory
ERROR 2021-09-28 21:19:46,598 [shard 0] none - AuthRegistry(0x7fa38c322b68) no keyring found at /var/lib/ceph/osd/ceph-0/keyring, disabling cephx
...
INFO  2021-09-28 21:19:46,601 [shard 0] monc - get_auth_request(con=[client.?(temp_mon_client) 172.17.0.1:0/2910147961@63138 >> mon.? v2:10.108.187.31:3300/0], auth_method=0)
INFO  2021-09-28 21:19:46,601 [shard 0] monc - get_auth_request no methods is supported
...
WARN  2021-09-28 21:20:06,612 [shard 0] monc - cannot establish the active_con with any mon
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #43150 from sebastian-philipp/doc-cephadm-serves-sub-tree
Sebastian Wagner [Tue, 28 Sep 2021 09:20:47 +0000 (11:20 +0200)]
Merge pull request #43150 from sebastian-philipp/doc-cephadm-serves-sub-tree

doc/cephadm: move services under doc/cephadm/services ?

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #43221 from badone/wip-install-deps-new-pip-resolver
Deepika Upadhyay [Tue, 28 Sep 2021 08:45:41 +0000 (14:15 +0530)]
Merge pull request #43221 from badone/wip-install-deps-new-pip-resolver

install-deps.sh: Use legacy pip resolver

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #43285 from rhcs-dashboard/api-endpoint-version-compatibility
Ernesto Puerta [Mon, 27 Sep 2021 12:13:07 +0000 (14:13 +0200)]
Merge pull request #43285 from rhcs-dashboard/api-endpoint-version-compatibility

mgr/dashboard: make modified API endpoints backward compatible

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
3 years agodoc/cephadm: move services into services/ 43150/head
Sebastian Wagner [Mon, 13 Sep 2021 15:15:33 +0000 (17:15 +0200)]
doc/cephadm: move services into services/

This is going to clean up the toctree a bit.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agodoc/cephadm: move service-management to services/index.rst
Sebastian Wagner [Mon, 13 Sep 2021 15:12:01 +0000 (17:12 +0200)]
doc/cephadm: move service-management to services/index.rst

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #43160 from sebastian-philipp/doc-spec-networks
Sebastian Wagner [Mon, 27 Sep 2021 10:19:17 +0000 (12:19 +0200)]
Merge pull request #43160 from sebastian-philipp/doc-spec-networks

doc/cephadm: Add networks to spec documentation

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #43276 from lxbsz/fix_typo
Venky Shankar [Mon, 27 Sep 2021 06:30:11 +0000 (12:00 +0530)]
Merge pull request #43276 from lxbsz/fix_typo

mds: fix typo in the comments and log when setting new loner

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #43301 from athanatos/sjust/wip-seastore-fixes-2021-09-24
Samuel Just [Mon, 27 Sep 2021 05:04:18 +0000 (22:04 -0700)]
Merge pull request #43301 from athanatos/sjust/wip-seastore-fixes-2021-09-24

crimson/os/seastore/.../btree_range_pin: fix is_parent_of

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #43291 from badone/wip-missing-hostname-dependency
Brad Hubbard [Mon, 27 Sep 2021 04:21:14 +0000 (14:21 +1000)]
Merge pull request #43291 from badone/wip-missing-hostname-dependency

rpm, debian: We depend on 'hostname'

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agorpm, debian: Add hostname as a "make check" dependency 43291/head
Brad Hubbard [Fri, 24 Sep 2021 02:15:53 +0000 (12:15 +1000)]
rpm, debian: Add hostname as a "make check" dependency

test_hostname.cc depends on the hostname binary.

Fixes: https://tracker.ceph.com/issues/52695
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
3 years agoMerge pull request #43191 from ronen-fr/wip-rf-scrub-locations
Ronen Friedman [Sun, 26 Sep 2021 17:33:49 +0000 (20:33 +0300)]
Merge pull request #43191 from ronen-fr/wip-rf-scrub-locations

osd/scrub: collecting scrub-related files into a separate directory

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43304 from inspur-wyq/wip-doc-3
Kefu Chai [Sun, 26 Sep 2021 13:55:57 +0000 (21:55 +0800)]
Merge pull request #43304 from inspur-wyq/wip-doc-3

doc/rbd/rbd-windows.rst: fix typos

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoosd/scrub: collecting scrub-related files into a separate directory 43191/head
Ronen Friedman [Sun, 11 Apr 2021 18:17:41 +0000 (21:17 +0300)]
osd/scrub: collecting scrub-related files into a separate directory

Cleaning src/osd from scrub implementation files. Triggered by:
- the matching Crimson scrub structure;
- the proliferation of scrub related code files (inc. in coming PRs);

scrubber_common.h, which defines the scrubber's interface, remains
in src/osd.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
3 years agoMerge pull request #40984 from ronen-fr/wip-ronenf-scrub-sched
Ronen Friedman [Sun, 26 Sep 2021 11:59:49 +0000 (14:59 +0300)]
Merge pull request #40984 from ronen-fr/wip-ronenf-scrub-sched

osd/scrub: extracting scrub scheduling code from OSD.cc

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43220 from idryomov/wip-keyring-resolve-error
Ilya Dryomov [Sun, 26 Sep 2021 10:09:43 +0000 (12:09 +0200)]
Merge pull request #43220 from idryomov/wip-keyring-resolve-error

auth,mon: don't log "unable to find a keyring" error when key is given

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #43207 from idryomov/wip-full-try-regressions
Ilya Dryomov [Sun, 26 Sep 2021 10:07:06 +0000 (12:07 +0200)]
Merge pull request #43207 from idryomov/wip-full-try-regressions

librados,neorados,librbd: fix FULL_TRY handling regressions

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
3 years agomgr/dashboard: replace string version with class 43285/head
Ernesto Puerta [Fri, 24 Sep 2021 15:46:42 +0000 (17:46 +0200)]
mgr/dashboard: replace string version with class

* APIVersion:
  * Moved to a separate file
  * Added doctests
  * Added sentinel values:
    * DEFAULT = 1.0
    * EXPERIMENTAL = 0.1
    * NONE = 0.0
  * Added to_mime_type() helper method
* Controllers.__init__:
  * Added type hints
  * Replaced string versions with APIVersions
* Feedback controller:
  * Replaced with EXPERIMENTAL (probably it should be NONE)

Fixes: https://tracker.ceph.com/issues/52480
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
3 years agoMerge pull request #43228 from idryomov/wip-rbd-mirror-snapshot-rx-only
Ilya Dryomov [Sat, 25 Sep 2021 08:58:17 +0000 (10:58 +0200)]
Merge pull request #43228 from idryomov/wip-rbd-mirror-snapshot-rx-only

rbd-mirror: unbreak one-way snapshot-based mirroring

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agomds: fix the set loner log 43276/head
Xiubo Li [Sat, 25 Sep 2021 08:56:39 +0000 (16:56 +0800)]
mds: fix the set loner log

Once here the current loner has already been dropped if there has,
and the get_wanted_loner() is the new loner it trying to set to.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agomds: fix typo in the comments
Xiubo Li [Tue, 10 Aug 2021 07:06:54 +0000 (15:06 +0800)]
mds: fix typo in the comments

Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agocrimson/os/seastore/.../btree_range_pin: fix is_parent_of 43301/head
Samuel Just [Sat, 25 Sep 2021 00:10:43 +0000 (17:10 -0700)]
crimson/os/seastore/.../btree_range_pin: fix is_parent_of

The LBA tree implementation only requires that the start addr of
a logical extent be contained within the leaf range.  It's entirely
possible for the end of a logical extent to extend past the end addr
of the containing leaf node.

Fixes: https://tracker.ceph.com/issues/52709
Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/transaction_manager: whitespace cleanup submit_transaction_direct
Samuel Just [Thu, 23 Sep 2021 09:07:35 +0000 (02:07 -0700)]
crimson/os/seastore/transaction_manager: whitespace cleanup submit_transaction_direct

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43282 from Madhu-1/csi-ceph-conf
Ilya Dryomov [Fri, 24 Sep 2021 16:04:20 +0000 (18:04 +0200)]
Merge pull request #43282 from Madhu-1/csi-ceph-conf

doc/rbd/rbd-kubernetes: add ceph.conf configmap

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #43294 from badone/wip-run-tox-python-common-timedelta-fix
Sebastian Wagner [Fri, 24 Sep 2021 15:36:57 +0000 (17:36 +0200)]
Merge pull request #43294 from badone/wip-run-tox-python-common-timedelta-fix

mgr/dashboard: More robust handling of timedelta() representation

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #41931 from pritha-srivastava/wip-rgw-sts-51219
Casey Bodley [Fri, 24 Sep 2021 15:04:36 +0000 (11:04 -0400)]
Merge pull request #41931 from pritha-srivastava/wip-rgw-sts-51219

rgw/sts: code for returning an error when an IAM policy

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
3 years agodoc/rbd/rbd-kubernetes: add ceph.conf configmap 43282/head
Madhu Rajanna [Thu, 23 Sep 2021 09:42:40 +0000 (15:12 +0530)]
doc/rbd/rbd-kubernetes: add ceph.conf configmap

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
3 years agoMerge PR #42384 into master
Sage Weil [Fri, 24 Sep 2021 14:36:41 +0000 (10:36 -0400)]
Merge PR #42384 into master

* refs/pull/42384/head:
mgr/cephadm: kick serve loop if new metadata makes all hosts metadata up to date
mgr/cephadm: fix upgrade to version with agent
mgr/cephadm: delay actions on agent daemons if root cert not created
mgr/cephadm: handle making certs when we have hostname but no address
mgr/cephadm: make agent certs compatible with OpenSSL hostname and ip checks
mgr/cephadm: offline host handling improvements for agent
mgr/cephadm: handle use_agent being turned on and off
mgr/cephadm: better handling of offline hosts with agent
mgr/cephadm: convert networks from set to list + don't reset con on down agent hosts
mgr/cephadm: add ceph volume to metadata agent reports
mgr/cephadm: implement 2-way ssl in mgr -> MgrListener comm line
cephadm: allow mgr listener to handle variable length json strings
mgr/cephadm: cephadm agent 2.0

Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #43298 from sebastian-philipp/cephadm-check_time_sync-mock
Sebastian Wagner [Fri, 24 Sep 2021 13:48:40 +0000 (15:48 +0200)]
Merge pull request #43298 from sebastian-philipp/cephadm-check_time_sync-mock

cephadm: TestCheckHost: also mock check_time_sync

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
3 years agomgr/dashboard: make modified API endpoints backward compatible
Avan Thakkar [Thu, 23 Sep 2021 11:15:16 +0000 (16:45 +0530)]
mgr/dashboard: make modified API endpoints backward compatible

Fixes: https://tracker.ceph.com/issues/52480
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Introducing APIVersion class to handle versioning for API-endpints and making
them backward compatible.

3 years agomgr/cephadm: kick serve loop if new metadata makes all hosts metadata up to date 42384/head
Adam King [Thu, 23 Sep 2021 16:41:52 +0000 (12:41 -0400)]
mgr/cephadm: kick serve loop if new metadata makes all hosts metadata up to date

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: fix upgrade to version with agent
Adam King [Mon, 13 Sep 2021 17:20:30 +0000 (13:20 -0400)]
mgr/cephadm: fix upgrade to version with agent

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: delay actions on agent daemons if root cert not created
Adam King [Fri, 10 Sep 2021 17:09:50 +0000 (13:09 -0400)]
mgr/cephadm: delay actions on agent daemons if root cert not created

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: handle making certs when we have hostname but no address
Adam King [Fri, 10 Sep 2021 15:46:27 +0000 (11:46 -0400)]
mgr/cephadm: handle making certs when we have hostname but no address

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: make agent certs compatible with OpenSSL hostname and ip checks
Adam King [Thu, 9 Sep 2021 17:36:40 +0000 (13:36 -0400)]
mgr/cephadm: make agent certs compatible with OpenSSL hostname and ip checks

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: offline host handling improvements for agent
Adam King [Mon, 6 Sep 2021 23:11:34 +0000 (19:11 -0400)]
mgr/cephadm: offline host handling improvements for agent

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: handle use_agent being turned on and off
Adam King [Thu, 2 Sep 2021 17:29:32 +0000 (13:29 -0400)]
mgr/cephadm: handle use_agent being turned on and off

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: better handling of offline hosts with agent
Adam King [Wed, 1 Sep 2021 15:03:16 +0000 (11:03 -0400)]
mgr/cephadm: better handling of offline hosts with agent

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: convert networks from set to list + don't reset con on down agent hosts
Adam King [Mon, 30 Aug 2021 17:39:56 +0000 (13:39 -0400)]
mgr/cephadm: convert networks from set to list + don't reset con on down agent hosts

networks needs to be list so they can be encoded in  a json string
resetting con on the hosts where agent isn't reporting (possibly offline hosts)

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: add ceph volume to metadata agent reports
Adam King [Mon, 30 Aug 2021 12:45:58 +0000 (08:45 -0400)]
mgr/cephadm: add ceph volume to metadata agent reports

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: implement 2-way ssl in mgr -> MgrListener comm line
Adam King [Thu, 26 Aug 2021 19:20:23 +0000 (15:20 -0400)]
mgr/cephadm: implement 2-way ssl in mgr -> MgrListener comm line

Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: allow mgr listener to handle variable length json strings
Adam King [Wed, 25 Aug 2021 13:25:11 +0000 (09:25 -0400)]
cephadm: allow mgr listener to handle variable length json strings

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: cephadm agent 2.0
Adam King [Wed, 16 Jun 2021 12:02:15 +0000 (08:02 -0400)]
mgr/cephadm: cephadm agent 2.0

Creates an http endpoint in mgr/cephadm to receive
http requests and an agent that can be deployed on
each host that will gather metadata on the host and
send it to the mgr/cephadm http endpoint. Should save the
cephadm mgr module a lot of time it would have to spend
repeatedly ssh-ing into each host to gather the metadata
and help performance on larger clusters.

Fixes: https://tracker.ceph.com/issues/51004
Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: Fix TestMaintenance trying to list /var/lib/ceph 43298/head
Sebastian Wagner [Fri, 24 Sep 2021 10:55:03 +0000 (12:55 +0200)]
cephadm: Fix TestMaintenance trying to list /var/lib/ceph

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: TestCheckHost: also mock `check_time_sync`
Sebastian Wagner [Fri, 24 Sep 2021 10:46:54 +0000 (12:46 +0200)]
cephadm: TestCheckHost: also mock `check_time_sync`

Fixes: https://tracker.ceph.com/issues/52722
```
TestCheckHost.test_container_engine fails at cephadm:5834: Error cephadm.Error: No time synchronization is active
```

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoqa/suites/rbd: test case for one-way snapshot-based mirroring 43228/head
Ilya Dryomov [Fri, 24 Sep 2021 10:29:34 +0000 (12:29 +0200)]
qa/suites/rbd: test case for one-way snapshot-based mirroring

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #43037 from ljflores/perf-histogram-formatting
Ernesto Puerta [Fri, 24 Sep 2021 09:55:59 +0000 (11:55 +0200)]
Merge pull request #43037 from ljflores/perf-histogram-formatting

mgr/dashboard: improve formatting of histograms in Telemetry preview form

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
3 years agoMerge pull request #43295 from liu-chunmei/fix_store_nbd
Samuel Just [Fri, 24 Sep 2021 06:29:12 +0000 (23:29 -0700)]
Merge pull request #43295 from liu-chunmei/fix_store_nbd

crimson/store-nbd: fix store_nbd build error for futurized store mkfs

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/store-nbd: fix store_nbd build error for futurized store mkfs 43295/head
chunmei-liu [Fri, 24 Sep 2021 04:20:39 +0000 (21:20 -0700)]
crimson/store-nbd: fix store_nbd build error for futurized store mkfs

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
3 years agoMerge pull request #43261 from rzarzynski/wip-crimson-mkfs_ertr
Samuel Just [Fri, 24 Sep 2021 02:49:08 +0000 (19:49 -0700)]
Merge pull request #43261 from rzarzynski/wip-crimson-mkfs_ertr

crimson/{common,os,osd}: errorate the FuturizedStore::mkfs() paths

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #43288 from rzarzynski/wip-crimson-skip-our-frames-in-bt
Samuel Just [Fri, 24 Sep 2021 02:48:04 +0000 (19:48 -0700)]
Merge pull request #43288 from rzarzynski/wip-crimson-skip-our-frames-in-bt

crimson/common: skip first 4 frames when dumping a backtrace.

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
3 years agomgr/dashboard: More robust handling of timedelta() representation 43294/head
Brad Hubbard [Fri, 24 Sep 2021 02:22:42 +0000 (12:22 +1000)]
mgr/dashboard: More robust handling of timedelta() representation

The representation of timedelta() appears to have changed recently so
we should try and be more accommodating in that regard.

Fixes: https://tracker.ceph.com/issues/52696
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
3 years agoinstall-deps.sh: Use legacy pip resolver 43221/head
Brad Hubbard [Mon, 20 Sep 2021 04:42:41 +0000 (14:42 +1000)]
install-deps.sh: Use legacy pip resolver

Due to the issue described in https://github.com/pypa/pip/issues/9818 I
am seeing lots of downloads and prolonged runtime when installing
dependencies.

Fixes: https://tracker.ceph.com/issues/52660
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
3 years agoMerge pull request #43275 from adk3798/maint-target
Sebastian Wagner [Thu, 23 Sep 2021 20:15:28 +0000 (22:15 +0200)]
Merge pull request #43275 from adk3798/maint-target

mgr/cephadm: base maintenance mode enter/exit success/failure on returned message

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #43121 from mgfritch/cephadm-pull-error
Sebastian Wagner [Thu, 23 Sep 2021 20:12:01 +0000 (22:12 +0200)]
Merge pull request #43121 from mgfritch/cephadm-pull-error

cephadm: raise error during `pull` failure

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
3 years agoMerge pull request #43062 from strenuous-life/wip-cephadm-zap-device
Sebastian Wagner [Thu, 23 Sep 2021 20:11:30 +0000 (22:11 +0200)]
Merge pull request #43062 from strenuous-life/wip-cephadm-zap-device

mgr/cephadm: osd should not be zap when it is running

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #37544 from kamoltat/wip-mgr-progress-global-efficiency
Kamoltat Sirivadhna [Thu, 23 Sep 2021 18:12:16 +0000 (14:12 -0400)]
Merge pull request #37544 from kamoltat/wip-mgr-progress-global-efficiency

mgr/progress: optimize global recovery module

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agomgr/dashboard: improve unittest 43037/head
Laura Flores [Thu, 23 Sep 2021 17:34:05 +0000 (17:34 +0000)]
mgr/dashboard: improve unittest

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agotests/crimson: make the virtual methods of SeaStoreTestState final. 43261/head
Radoslaw Zarzynski [Wed, 22 Sep 2021 17:43:25 +0000 (17:43 +0000)]
tests/crimson: make the virtual methods of SeaStoreTestState final.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/os: workaround the segfaulting GCC 11 issue.
Radoslaw Zarzynski [Wed, 22 Sep 2021 14:39:30 +0000 (14:39 +0000)]
crimson/os: workaround the segfaulting GCC 11 issue.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/osd, crimson/os: errorate the FuturizedStore::mkfs() paths.
Radoslaw Zarzynski [Tue, 21 Sep 2021 23:01:06 +0000 (23:01 +0000)]
crimson/osd, crimson/os: errorate the FuturizedStore::mkfs() paths.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/common: skip first 4 frames when dumping a backtrace. 43288/head
Radoslaw Zarzynski [Thu, 23 Sep 2021 14:25:10 +0000 (14:25 +0000)]
crimson/common: skip first 4 frames when dumping a backtrace.

It's all about these items:

```
 0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
 1# FatalSignal::signaled(int, siginfo_t const&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/ostream:570
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:
62
 3# 0x00007F16BBA13B30 in /lib64/libpthread.so.0
```

They are part of our backtrace handling and typically developers
are not interested in them. Let's be consistent with the classical
OSD and hide them.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agodoc/cephadm: Add `networks` to spec documentation 43160/head
Sebastian Wagner [Tue, 14 Sep 2021 14:07:14 +0000 (16:07 +0200)]
doc/cephadm: Add `networks` to spec documentation

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agodoc/cephadm: add mgr.rst
Sebastian Wagner [Tue, 14 Sep 2021 14:15:29 +0000 (16:15 +0200)]
doc/cephadm: add mgr.rst

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agodoc/cephadm: Add `Specifying Networks` to rgw.rst
Sebastian Wagner [Tue, 14 Sep 2021 14:08:11 +0000 (16:08 +0200)]
doc/cephadm: Add `Specifying Networks` to rgw.rst

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #43262 from rzarzynski/wip-crimson-alienstore-fix-oncommit wip-rgw-zipper-standalone 43173/head
Samuel Just [Thu, 23 Sep 2021 05:59:57 +0000 (22:59 -0700)]
Merge pull request #43262 from rzarzynski/wip-crimson-alienstore-fix-oncommit

crimson/os/alienstore: fix nullptr deref in OnCommit::finish().

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agorgw/sts: code to check IAM policy and return an 41931/head
Pritha Srivastava [Mon, 19 Jul 2021 10:27:52 +0000 (15:57 +0530)]
rgw/sts: code to check IAM policy and return an
appropriate error incase Resource specified in the
IAM policy is incorrect and is discarded. The IAM
policy can be a resource policy or an identity policy.
This is for policies that have already been set.

Fixes: https://tracker.ceph.com/issues/51219
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
3 years agorgw/sts: code for returning an error when an IAM policy
Pritha Srivastava [Fri, 18 Jun 2021 10:06:10 +0000 (15:36 +0530)]
rgw/sts: code for returning an error when an IAM policy
resource belongs to someone else's tenant.

While parsing the policy it discards the resource element,
but then when an operation is evaluated, since the resource element
is empty, it doesnt evaluate the resource at all and the policy
ends up erroneously allowing actions on resources in other tenants.

Fixes: https://tracker.ceph.com/issues/51219
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
3 years agomgr/cephadm: base maintenance enter/exit success off of return message 43275/head
Adam King [Wed, 22 Sep 2021 21:43:43 +0000 (17:43 -0400)]
mgr/cephadm: base maintenance enter/exit success off of return message

rather than on whether there was any stdout from the command

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: unit tests for maintenance enter/exit properly handling success/failure...
Adam King [Wed, 22 Sep 2021 21:42:41 +0000 (17:42 -0400)]
mgr/cephadm: unit tests for maintenance enter/exit properly handling success/failure messages

Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: unit tests for maitenance mode return values
Adam King [Wed, 22 Sep 2021 18:34:12 +0000 (14:34 -0400)]
cephadm: unit tests for maitenance mode return values

Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: fix mypy complaints for ThreadedChildWatcher class
Adam King [Tue, 21 Sep 2021 18:00:11 +0000 (14:00 -0400)]
cephadm: fix mypy complaints for ThreadedChildWatcher class

Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: fix exiting maintenance when systemd target doesn't exist
Adam King [Mon, 20 Sep 2021 12:02:44 +0000 (08:02 -0400)]
cephadm: fix exiting maintenance when systemd target doesn't exist

If the systemd target doesn't exist we need to just bypass enabling
it and return success or the host will just be stuck in maitnenance
mode.

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/dashboard: fix linting in unittest
Laura Flores [Wed, 22 Sep 2021 21:36:33 +0000 (21:36 +0000)]
mgr/dashboard: fix linting in unittest

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agomgr/dashboard: clarify comment and variable name
Laura Flores [Wed, 22 Sep 2021 19:29:11 +0000 (19:29 +0000)]
mgr/dashboard: clarify comment and variable name

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agomgr/progress: optimize global recovery module 37544/head
Kamoltat [Mon, 5 Oct 2020 09:38:35 +0000 (09:38 +0000)]
mgr/progress: optimize global recovery module

Instead of fetching `pg_stats` from the python
part of manager module, we filter out the pgs
that are in active + clean state in ActivePyModules.cc
then parse these pgs along with `reported_epoch` and
the `total_num_pgs` of the clusters to global recovery
module.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
3 years agomgr/dashboard: add unit test for telemetry replacer method
Laura Flores [Wed, 22 Sep 2021 17:55:40 +0000 (17:55 +0000)]
mgr/dashboard: add unit test for telemetry replacer method

This unit test checks that the "replacer" method in telemetry.component.ts works as it should. The replacer method takes the telemetry report and changes the ranges and values of the 'osd_perf_histograms' field from arrays to strings, thereby making the report more readable in the Dashboard Telemetry Preview.

This unit test needs improvement since it currently uses a test report rather than the real one.

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agocrimson/os/alienstore: fix nullptr deref in OnCommit::finish(). 43262/head
Radoslaw Zarzynski [Wed, 22 Sep 2021 16:42:19 +0000 (16:42 +0000)]
crimson/os/alienstore: fix nullptr deref in OnCommit::finish().

`seastar::engine()` is available only for Seastar's threads;
it shouldn't be called outside of a reactor thread.
Unfortunately, this assumption is violated in `AlienStore`
where `OnCommit::finish()`, executed from a finisher thread
of `BlueStore`, calls `alien()` on `seastar::engine()`.
The net effect are crashes like the following one:

```
INFO  2021-09-22 14:26:33,214 [shard 0] osd - operator() writing superblock cluster_fsid 1d8f7908-2ebf-4a91-ae70-f445668c126b osd_fsid 4da9fe9a-1da5-4ea9-aa79-a1178165ede5         [381/1839]
Segmentation fault.
Backtrace:
 0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
 1# FatalSignal::signaled(int, siginfo_t const&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/ostream:570
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:
62
 3# 0x00007F16BBA13B30 in /lib64/libpthread.so.0
 4# (anonymous namespace)::OnCommit::finish(int) at /home/rzarzynski/ceph1/build/../src/crimson/os/alienstore/alien_store.cc:53
 5# Context::complete(int) at /home/rzarzynski/ceph1/build/../src/include/Context.h:100
 6# Finisher::finisher_thread_entry() at /home/rzarzynski/ceph1/build/../src/common/Finisher.cc:65
 7# 0x00007F16BBA0915A in /lib64/libpthread.so.0
 8# clone in /lib64/libc.so.6
Dump of siginfo:
  ...
  si_addr: 0x10
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #40728 from guits/doc_fix_pattern_rgw_nfs
Sébastien Han [Wed, 22 Sep 2021 15:08:31 +0000 (17:08 +0200)]
Merge pull request #40728 from guits/doc_fix_pattern_rgw_nfs

doc/rgw-nfs: use same pattern for keyring name

3 years agoMerge pull request #43254 from cyx1231st/wip-seastore-fix-onode-order
Samuel Just [Wed, 22 Sep 2021 14:05:01 +0000 (07:05 -0700)]
Merge pull request #43254 from cyx1231st/wip-seastore-fix-onode-order

crimson/onode-staged-tree: convert hash to the reversed version

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43249 from cyx1231st/wip-seastore-fix-omap-hint
Kefu Chai [Wed, 22 Sep 2021 13:47:37 +0000 (21:47 +0800)]
Merge pull request #43249 from cyx1231st/wip-seastore-fix-omap-hint

crimson/os/seastore: add missing hints in omap tree

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43055 from soumyakoduri/wip-skoduri-lua
Daniel Gryniewicz [Wed, 22 Sep 2021 12:12:23 +0000 (08:12 -0400)]
Merge pull request #43055 from soumyakoduri/wip-skoduri-lua

rgw/lua: Install the packages only for RadosStore

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Daniel Gryniewicz <dang@redhat.com>

3 years agoMerge pull request #43054 from soumyakoduri/wip-skoduri-dbstore-vstart
Daniel Gryniewicz [Wed, 22 Sep 2021 12:11:30 +0000 (08:11 -0400)]
Merge pull request #43054 from soumyakoduri/wip-skoduri-dbstore-vstart

rgw: Add option to configure backend store

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #42911 from soumyakoduri/wip-skoduri-dbstore-object
Daniel Gryniewicz [Wed, 22 Sep 2021 12:10:52 +0000 (08:10 -0400)]
Merge pull request #42911 from soumyakoduri/wip-skoduri-dbstore-object

rgw/dbstore object APIs

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #43241 from sebastian-philipp/suites-orch-labelere
Sebastian Wagner [Wed, 22 Sep 2021 09:34:35 +0000 (11:34 +0200)]
Merge pull request #43241 from sebastian-philipp/suites-orch-labelere

.github: fix path to cephadm suite

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agocrimson/onode-staged-tree: consolidate laddr hint calculation 43254/head
Yingxin Cheng [Wed, 22 Sep 2021 07:43:23 +0000 (15:43 +0800)]
crimson/onode-staged-tree: consolidate laddr hint calculation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/onode-staged-tree: convert hash to the reversed version
Yingxin Cheng [Wed, 22 Sep 2021 07:36:22 +0000 (15:36 +0800)]
crimson/onode-staged-tree: convert hash to the reversed version

Store the reversed version of object hash to make sure that onodes in
the same PG are sorted together.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/onode-staged-tree: print cursh-hash in hex mode
Yingxin Cheng [Wed, 22 Sep 2021 06:52:18 +0000 (14:52 +0800)]
crimson/onode-staged-tree: print cursh-hash in hex mode

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore: add missing hints in omap tree 43249/head
Yingxin Cheng [Sat, 18 Sep 2021 07:10:38 +0000 (15:10 +0800)]
crimson/os/seastore: add missing hints in omap tree

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agodoc/rbd/rbd-windows.rst: fix typos 43304/head
wangyunqing [Wed, 22 Sep 2021 03:05:40 +0000 (11:05 +0800)]
doc/rbd/rbd-windows.rst: fix typos

Signed-off-by: wangyunqing <wangyunqing@inspur.com>
3 years agoMerge pull request #43247 from rzarzynski/wip-crimson-ertr-safe_then_unpack
Samuel Just [Wed, 22 Sep 2021 00:12:58 +0000 (17:12 -0700)]
Merge pull request #43247 from rzarzynski/wip-crimson-ertr-safe_then_unpack

crimson/common: add safe_then_unpack() to errorated futures

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agocrimson/common: implement singleton_ec.
Radoslaw Zarzynski [Tue, 21 Sep 2021 22:58:56 +0000 (22:58 +0000)]
crimson/common: implement singleton_ec.

Unfortunately, GCC explodes when it sees that.

```
[rzarzynski@o06 build]$ ninja crimson-osd vstart
[1/3] Building CXX object src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o
FAILED: src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o
/usr/bin/ccache /opt/rh/gcc-toolset-9/root/usr/bin/c++ -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DHAVE_CONFIG_H -DSEASTAR_API_LEVEL=6 -DWITH_SEASTAR=1 -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -Isrc/include -I../src -I../src/seastar/include -Isrc/seastar/gen/include -isystem boost/include -isystem include -isystem ../src/xxHash -isystem ../src/rapidjson/include -O2 -g -DNDEBUG -fPIC   -U_FORTIFY_SOURCE -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wstrict-null-sentinel -Woverloaded-virtual -fno-new-ttp-matching -fstack-protector-strong -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -ftemplate-backtrace-limit=0 -Wno-non-virtual-dtor -std=gnu++17 -U_FORTIFY_SOURCE -Wno-maybe-uninitialized -DSEASTAR_SSTRING -Wno-error=unused-result -std=c++17 -MD -MT src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o -MF src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o.d -o src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o -c ../src/crimson/os/cyanstore/cyan_store.cc
during IPA pass: inline
../src/crimson/os/cyanstore/cyan_store.cc: In member function ‘std::string crimson::os::singleton_ec<MsgV>::this_error_category::message(int) const [with const char* MsgV = (& msg)]’:
../src/crimson/os/cyanstore/cyan_store.cc:97:17: internal compiler error: Segmentation fault
   97 |     std::string message([[maybe_unused]] const int ev) const final {
      |                 ^~~~~~~
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
Preprocessed source stored into /tmp/ccFi4GGw.out file, please attach this to your bugreport.
ninja: build stopped: subcommand failed.
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/common: add safe_then_unpack() to errorated futures. 43247/head
Radoslaw Zarzynski [Tue, 21 Sep 2021 21:28:45 +0000 (21:28 +0000)]
crimson/common: add safe_then_unpack() to errorated futures.

It was a prerequisite for another commit I finally thrown
away. However, this little bit can be still be useful even
for the sake of compliance with the interruptible variant.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/common: assert_moveable() doesn't depend on 3rd party's always_false<>.
Radoslaw Zarzynski [Tue, 21 Sep 2021 21:28:02 +0000 (21:28 +0000)]
crimson/common: assert_moveable() doesn't depend on 3rd party's always_false<>.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #43243 from rzarzynski/wip-crimson-net-dangling-bindvec2
Samuel Just [Tue, 21 Sep 2021 18:56:12 +0000 (11:56 -0700)]
Merge pull request #43243 from rzarzynski/wip-crimson-net-dangling-bindvec2

crimson/net: fix dangling addrvec in bind(), the repeat_until_value() part

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43209 from rzarzynski/wip-crimson-silent-check-bot
Samuel Just [Tue, 21 Sep 2021 18:55:30 +0000 (11:55 -0700)]
Merge pull request #43209 from rzarzynski/wip-crimson-silent-check-bot

tests/crimson: don't be so verbose when run by the 'make check' bot.

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge PR #43219 into master
Patrick Donnelly [Tue, 21 Sep 2021 18:03:08 +0000 (14:03 -0400)]
Merge PR #43219 into master

* refs/pull/43219/head:
doc: clarify MDS failed state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge PR #43156 into master
Sage Weil [Tue, 21 Sep 2021 15:37:50 +0000 (11:37 -0400)]
Merge PR #43156 into master

* refs/pull/43156/head:
qa/suites/orch/rook/smoke: add flannel back in

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocrimson/net: fix dangling addrvec in bind(), the repeat_until_value() part. 43243/head
Radoslaw Zarzynski [Mon, 20 Sep 2021 13:59:30 +0000 (13:59 +0000)]
crimson/net: fix dangling addrvec in bind(), the repeat_until_value() part.

This is a follow-up to commit 480273082718230d64d2c068db1b22f44997bdc8.

`seastar::do_with()` doesn't extend the life-time of the callable's

```cpp
auto
do_with_impl(T1&& rv1, T2&& rv2, More&&... more) {
    // ...
    auto&& just_values = cherry_pick_tuple(idx(), std::move(all));
    auto&& just_func = std::move(std::get<nr>(std::move(all)));
    // ...
    auto task = std::apply(
        [](auto&&... x) {
            return std::make_unique<internal::do_with_state<value_tuple, ret_type>>(std::forward<decltype(x)>(x)...);
        },
        std::move(just_values));
    auto fut = std::apply(just_func, task->data());
    // ...
}
```

closure. This means the `addrs` needs to be captured by value also
by the lambda passed to `repeat_until_value()`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>