]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agoosd/scrub: collecting scrub-related files into a separate directory 43191/head
Ronen Friedman [Sun, 11 Apr 2021 18:17:41 +0000 (21:17 +0300)]
osd/scrub: collecting scrub-related files into a separate directory

Cleaning src/osd from scrub implementation files. Triggered by:
- the matching Crimson scrub structure;
- the proliferation of scrub related code files (inc. in coming PRs);

scrubber_common.h, which defines the scrubber's interface, remains
in src/osd.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
3 years agoMerge pull request #40984 from ronen-fr/wip-ronenf-scrub-sched
Ronen Friedman [Sun, 26 Sep 2021 11:59:49 +0000 (14:59 +0300)]
Merge pull request #40984 from ronen-fr/wip-ronenf-scrub-sched

osd/scrub: extracting scrub scheduling code from OSD.cc

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43220 from idryomov/wip-keyring-resolve-error
Ilya Dryomov [Sun, 26 Sep 2021 10:09:43 +0000 (12:09 +0200)]
Merge pull request #43220 from idryomov/wip-keyring-resolve-error

auth,mon: don't log "unable to find a keyring" error when key is given

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #43207 from idryomov/wip-full-try-regressions
Ilya Dryomov [Sun, 26 Sep 2021 10:07:06 +0000 (12:07 +0200)]
Merge pull request #43207 from idryomov/wip-full-try-regressions

librados,neorados,librbd: fix FULL_TRY handling regressions

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
3 years agoMerge pull request #43228 from idryomov/wip-rbd-mirror-snapshot-rx-only
Ilya Dryomov [Sat, 25 Sep 2021 08:58:17 +0000 (10:58 +0200)]
Merge pull request #43228 from idryomov/wip-rbd-mirror-snapshot-rx-only

rbd-mirror: unbreak one-way snapshot-based mirroring

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #43282 from Madhu-1/csi-ceph-conf
Ilya Dryomov [Fri, 24 Sep 2021 16:04:20 +0000 (18:04 +0200)]
Merge pull request #43282 from Madhu-1/csi-ceph-conf

doc/rbd/rbd-kubernetes: add ceph.conf configmap

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #43294 from badone/wip-run-tox-python-common-timedelta-fix
Sebastian Wagner [Fri, 24 Sep 2021 15:36:57 +0000 (17:36 +0200)]
Merge pull request #43294 from badone/wip-run-tox-python-common-timedelta-fix

mgr/dashboard: More robust handling of timedelta() representation

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #41931 from pritha-srivastava/wip-rgw-sts-51219
Casey Bodley [Fri, 24 Sep 2021 15:04:36 +0000 (11:04 -0400)]
Merge pull request #41931 from pritha-srivastava/wip-rgw-sts-51219

rgw/sts: code for returning an error when an IAM policy

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
3 years agodoc/rbd/rbd-kubernetes: add ceph.conf configmap 43282/head
Madhu Rajanna [Thu, 23 Sep 2021 09:42:40 +0000 (15:12 +0530)]
doc/rbd/rbd-kubernetes: add ceph.conf configmap

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
3 years agoMerge PR #42384 into master
Sage Weil [Fri, 24 Sep 2021 14:36:41 +0000 (10:36 -0400)]
Merge PR #42384 into master

* refs/pull/42384/head:
mgr/cephadm: kick serve loop if new metadata makes all hosts metadata up to date
mgr/cephadm: fix upgrade to version with agent
mgr/cephadm: delay actions on agent daemons if root cert not created
mgr/cephadm: handle making certs when we have hostname but no address
mgr/cephadm: make agent certs compatible with OpenSSL hostname and ip checks
mgr/cephadm: offline host handling improvements for agent
mgr/cephadm: handle use_agent being turned on and off
mgr/cephadm: better handling of offline hosts with agent
mgr/cephadm: convert networks from set to list + don't reset con on down agent hosts
mgr/cephadm: add ceph volume to metadata agent reports
mgr/cephadm: implement 2-way ssl in mgr -> MgrListener comm line
cephadm: allow mgr listener to handle variable length json strings
mgr/cephadm: cephadm agent 2.0

Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #43298 from sebastian-philipp/cephadm-check_time_sync-mock
Sebastian Wagner [Fri, 24 Sep 2021 13:48:40 +0000 (15:48 +0200)]
Merge pull request #43298 from sebastian-philipp/cephadm-check_time_sync-mock

cephadm: TestCheckHost: also mock check_time_sync

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
3 years agomgr/cephadm: kick serve loop if new metadata makes all hosts metadata up to date 42384/head
Adam King [Thu, 23 Sep 2021 16:41:52 +0000 (12:41 -0400)]
mgr/cephadm: kick serve loop if new metadata makes all hosts metadata up to date

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: fix upgrade to version with agent
Adam King [Mon, 13 Sep 2021 17:20:30 +0000 (13:20 -0400)]
mgr/cephadm: fix upgrade to version with agent

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: delay actions on agent daemons if root cert not created
Adam King [Fri, 10 Sep 2021 17:09:50 +0000 (13:09 -0400)]
mgr/cephadm: delay actions on agent daemons if root cert not created

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: handle making certs when we have hostname but no address
Adam King [Fri, 10 Sep 2021 15:46:27 +0000 (11:46 -0400)]
mgr/cephadm: handle making certs when we have hostname but no address

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: make agent certs compatible with OpenSSL hostname and ip checks
Adam King [Thu, 9 Sep 2021 17:36:40 +0000 (13:36 -0400)]
mgr/cephadm: make agent certs compatible with OpenSSL hostname and ip checks

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: offline host handling improvements for agent
Adam King [Mon, 6 Sep 2021 23:11:34 +0000 (19:11 -0400)]
mgr/cephadm: offline host handling improvements for agent

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: handle use_agent being turned on and off
Adam King [Thu, 2 Sep 2021 17:29:32 +0000 (13:29 -0400)]
mgr/cephadm: handle use_agent being turned on and off

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: better handling of offline hosts with agent
Adam King [Wed, 1 Sep 2021 15:03:16 +0000 (11:03 -0400)]
mgr/cephadm: better handling of offline hosts with agent

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: convert networks from set to list + don't reset con on down agent hosts
Adam King [Mon, 30 Aug 2021 17:39:56 +0000 (13:39 -0400)]
mgr/cephadm: convert networks from set to list + don't reset con on down agent hosts

networks needs to be list so they can be encoded in  a json string
resetting con on the hosts where agent isn't reporting (possibly offline hosts)

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: add ceph volume to metadata agent reports
Adam King [Mon, 30 Aug 2021 12:45:58 +0000 (08:45 -0400)]
mgr/cephadm: add ceph volume to metadata agent reports

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: implement 2-way ssl in mgr -> MgrListener comm line
Adam King [Thu, 26 Aug 2021 19:20:23 +0000 (15:20 -0400)]
mgr/cephadm: implement 2-way ssl in mgr -> MgrListener comm line

Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: allow mgr listener to handle variable length json strings
Adam King [Wed, 25 Aug 2021 13:25:11 +0000 (09:25 -0400)]
cephadm: allow mgr listener to handle variable length json strings

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: cephadm agent 2.0
Adam King [Wed, 16 Jun 2021 12:02:15 +0000 (08:02 -0400)]
mgr/cephadm: cephadm agent 2.0

Creates an http endpoint in mgr/cephadm to receive
http requests and an agent that can be deployed on
each host that will gather metadata on the host and
send it to the mgr/cephadm http endpoint. Should save the
cephadm mgr module a lot of time it would have to spend
repeatedly ssh-ing into each host to gather the metadata
and help performance on larger clusters.

Fixes: https://tracker.ceph.com/issues/51004
Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: Fix TestMaintenance trying to list /var/lib/ceph 43298/head
Sebastian Wagner [Fri, 24 Sep 2021 10:55:03 +0000 (12:55 +0200)]
cephadm: Fix TestMaintenance trying to list /var/lib/ceph

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: TestCheckHost: also mock `check_time_sync`
Sebastian Wagner [Fri, 24 Sep 2021 10:46:54 +0000 (12:46 +0200)]
cephadm: TestCheckHost: also mock `check_time_sync`

Fixes: https://tracker.ceph.com/issues/52722
```
TestCheckHost.test_container_engine fails at cephadm:5834: Error cephadm.Error: No time synchronization is active
```

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoqa/suites/rbd: test case for one-way snapshot-based mirroring 43228/head
Ilya Dryomov [Fri, 24 Sep 2021 10:29:34 +0000 (12:29 +0200)]
qa/suites/rbd: test case for one-way snapshot-based mirroring

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #43037 from ljflores/perf-histogram-formatting
Ernesto Puerta [Fri, 24 Sep 2021 09:55:59 +0000 (11:55 +0200)]
Merge pull request #43037 from ljflores/perf-histogram-formatting

mgr/dashboard: improve formatting of histograms in Telemetry preview form

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
3 years agoMerge pull request #43295 from liu-chunmei/fix_store_nbd
Samuel Just [Fri, 24 Sep 2021 06:29:12 +0000 (23:29 -0700)]
Merge pull request #43295 from liu-chunmei/fix_store_nbd

crimson/store-nbd: fix store_nbd build error for futurized store mkfs

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/store-nbd: fix store_nbd build error for futurized store mkfs 43295/head
chunmei-liu [Fri, 24 Sep 2021 04:20:39 +0000 (21:20 -0700)]
crimson/store-nbd: fix store_nbd build error for futurized store mkfs

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
3 years agoMerge pull request #43261 from rzarzynski/wip-crimson-mkfs_ertr
Samuel Just [Fri, 24 Sep 2021 02:49:08 +0000 (19:49 -0700)]
Merge pull request #43261 from rzarzynski/wip-crimson-mkfs_ertr

crimson/{common,os,osd}: errorate the FuturizedStore::mkfs() paths

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #43288 from rzarzynski/wip-crimson-skip-our-frames-in-bt
Samuel Just [Fri, 24 Sep 2021 02:48:04 +0000 (19:48 -0700)]
Merge pull request #43288 from rzarzynski/wip-crimson-skip-our-frames-in-bt

crimson/common: skip first 4 frames when dumping a backtrace.

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
3 years agomgr/dashboard: More robust handling of timedelta() representation 43294/head
Brad Hubbard [Fri, 24 Sep 2021 02:22:42 +0000 (12:22 +1000)]
mgr/dashboard: More robust handling of timedelta() representation

The representation of timedelta() appears to have changed recently so
we should try and be more accommodating in that regard.

Fixes: https://tracker.ceph.com/issues/52696
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
3 years agoMerge pull request #43275 from adk3798/maint-target
Sebastian Wagner [Thu, 23 Sep 2021 20:15:28 +0000 (22:15 +0200)]
Merge pull request #43275 from adk3798/maint-target

mgr/cephadm: base maintenance mode enter/exit success/failure on returned message

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #43121 from mgfritch/cephadm-pull-error
Sebastian Wagner [Thu, 23 Sep 2021 20:12:01 +0000 (22:12 +0200)]
Merge pull request #43121 from mgfritch/cephadm-pull-error

cephadm: raise error during `pull` failure

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
3 years agoMerge pull request #43062 from strenuous-life/wip-cephadm-zap-device
Sebastian Wagner [Thu, 23 Sep 2021 20:11:30 +0000 (22:11 +0200)]
Merge pull request #43062 from strenuous-life/wip-cephadm-zap-device

mgr/cephadm: osd should not be zap when it is running

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #37544 from kamoltat/wip-mgr-progress-global-efficiency
Kamoltat Sirivadhna [Thu, 23 Sep 2021 18:12:16 +0000 (14:12 -0400)]
Merge pull request #37544 from kamoltat/wip-mgr-progress-global-efficiency

mgr/progress: optimize global recovery module

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agomgr/dashboard: improve unittest 43037/head
Laura Flores [Thu, 23 Sep 2021 17:34:05 +0000 (17:34 +0000)]
mgr/dashboard: improve unittest

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agotests/crimson: make the virtual methods of SeaStoreTestState final. 43261/head
Radoslaw Zarzynski [Wed, 22 Sep 2021 17:43:25 +0000 (17:43 +0000)]
tests/crimson: make the virtual methods of SeaStoreTestState final.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/os: workaround the segfaulting GCC 11 issue.
Radoslaw Zarzynski [Wed, 22 Sep 2021 14:39:30 +0000 (14:39 +0000)]
crimson/os: workaround the segfaulting GCC 11 issue.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/osd, crimson/os: errorate the FuturizedStore::mkfs() paths.
Radoslaw Zarzynski [Tue, 21 Sep 2021 23:01:06 +0000 (23:01 +0000)]
crimson/osd, crimson/os: errorate the FuturizedStore::mkfs() paths.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/common: skip first 4 frames when dumping a backtrace. 43288/head
Radoslaw Zarzynski [Thu, 23 Sep 2021 14:25:10 +0000 (14:25 +0000)]
crimson/common: skip first 4 frames when dumping a backtrace.

It's all about these items:

```
 0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
 1# FatalSignal::signaled(int, siginfo_t const&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/ostream:570
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:
62
 3# 0x00007F16BBA13B30 in /lib64/libpthread.so.0
```

They are part of our backtrace handling and typically developers
are not interested in them. Let's be consistent with the classical
OSD and hide them.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #43262 from rzarzynski/wip-crimson-alienstore-fix-oncommit wip-rgw-zipper-standalone 43173/head
Samuel Just [Thu, 23 Sep 2021 05:59:57 +0000 (22:59 -0700)]
Merge pull request #43262 from rzarzynski/wip-crimson-alienstore-fix-oncommit

crimson/os/alienstore: fix nullptr deref in OnCommit::finish().

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agorgw/sts: code to check IAM policy and return an 41931/head
Pritha Srivastava [Mon, 19 Jul 2021 10:27:52 +0000 (15:57 +0530)]
rgw/sts: code to check IAM policy and return an
appropriate error incase Resource specified in the
IAM policy is incorrect and is discarded. The IAM
policy can be a resource policy or an identity policy.
This is for policies that have already been set.

Fixes: https://tracker.ceph.com/issues/51219
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
3 years agorgw/sts: code for returning an error when an IAM policy
Pritha Srivastava [Fri, 18 Jun 2021 10:06:10 +0000 (15:36 +0530)]
rgw/sts: code for returning an error when an IAM policy
resource belongs to someone else's tenant.

While parsing the policy it discards the resource element,
but then when an operation is evaluated, since the resource element
is empty, it doesnt evaluate the resource at all and the policy
ends up erroneously allowing actions on resources in other tenants.

Fixes: https://tracker.ceph.com/issues/51219
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
3 years agomgr/cephadm: base maintenance enter/exit success off of return message 43275/head
Adam King [Wed, 22 Sep 2021 21:43:43 +0000 (17:43 -0400)]
mgr/cephadm: base maintenance enter/exit success off of return message

rather than on whether there was any stdout from the command

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/cephadm: unit tests for maintenance enter/exit properly handling success/failure...
Adam King [Wed, 22 Sep 2021 21:42:41 +0000 (17:42 -0400)]
mgr/cephadm: unit tests for maintenance enter/exit properly handling success/failure messages

Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: unit tests for maitenance mode return values
Adam King [Wed, 22 Sep 2021 18:34:12 +0000 (14:34 -0400)]
cephadm: unit tests for maitenance mode return values

Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: fix mypy complaints for ThreadedChildWatcher class
Adam King [Tue, 21 Sep 2021 18:00:11 +0000 (14:00 -0400)]
cephadm: fix mypy complaints for ThreadedChildWatcher class

Signed-off-by: Adam King <adking@redhat.com>
3 years agocephadm: fix exiting maintenance when systemd target doesn't exist
Adam King [Mon, 20 Sep 2021 12:02:44 +0000 (08:02 -0400)]
cephadm: fix exiting maintenance when systemd target doesn't exist

If the systemd target doesn't exist we need to just bypass enabling
it and return success or the host will just be stuck in maitnenance
mode.

Signed-off-by: Adam King <adking@redhat.com>
3 years agomgr/dashboard: fix linting in unittest
Laura Flores [Wed, 22 Sep 2021 21:36:33 +0000 (21:36 +0000)]
mgr/dashboard: fix linting in unittest

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agomgr/dashboard: clarify comment and variable name
Laura Flores [Wed, 22 Sep 2021 19:29:11 +0000 (19:29 +0000)]
mgr/dashboard: clarify comment and variable name

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agomgr/progress: optimize global recovery module 37544/head
Kamoltat [Mon, 5 Oct 2020 09:38:35 +0000 (09:38 +0000)]
mgr/progress: optimize global recovery module

Instead of fetching `pg_stats` from the python
part of manager module, we filter out the pgs
that are in active + clean state in ActivePyModules.cc
then parse these pgs along with `reported_epoch` and
the `total_num_pgs` of the clusters to global recovery
module.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
3 years agomgr/dashboard: add unit test for telemetry replacer method
Laura Flores [Wed, 22 Sep 2021 17:55:40 +0000 (17:55 +0000)]
mgr/dashboard: add unit test for telemetry replacer method

This unit test checks that the "replacer" method in telemetry.component.ts works as it should. The replacer method takes the telemetry report and changes the ranges and values of the 'osd_perf_histograms' field from arrays to strings, thereby making the report more readable in the Dashboard Telemetry Preview.

This unit test needs improvement since it currently uses a test report rather than the real one.

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agocrimson/os/alienstore: fix nullptr deref in OnCommit::finish(). 43262/head
Radoslaw Zarzynski [Wed, 22 Sep 2021 16:42:19 +0000 (16:42 +0000)]
crimson/os/alienstore: fix nullptr deref in OnCommit::finish().

`seastar::engine()` is available only for Seastar's threads;
it shouldn't be called outside of a reactor thread.
Unfortunately, this assumption is violated in `AlienStore`
where `OnCommit::finish()`, executed from a finisher thread
of `BlueStore`, calls `alien()` on `seastar::engine()`.
The net effect are crashes like the following one:

```
INFO  2021-09-22 14:26:33,214 [shard 0] osd - operator() writing superblock cluster_fsid 1d8f7908-2ebf-4a91-ae70-f445668c126b osd_fsid 4da9fe9a-1da5-4ea9-aa79-a1178165ede5         [381/1839]
Segmentation fault.
Backtrace:
 0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
 1# FatalSignal::signaled(int, siginfo_t const&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/ostream:570
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:
62
 3# 0x00007F16BBA13B30 in /lib64/libpthread.so.0
 4# (anonymous namespace)::OnCommit::finish(int) at /home/rzarzynski/ceph1/build/../src/crimson/os/alienstore/alien_store.cc:53
 5# Context::complete(int) at /home/rzarzynski/ceph1/build/../src/include/Context.h:100
 6# Finisher::finisher_thread_entry() at /home/rzarzynski/ceph1/build/../src/common/Finisher.cc:65
 7# 0x00007F16BBA0915A in /lib64/libpthread.so.0
 8# clone in /lib64/libc.so.6
Dump of siginfo:
  ...
  si_addr: 0x10
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #40728 from guits/doc_fix_pattern_rgw_nfs
Sébastien Han [Wed, 22 Sep 2021 15:08:31 +0000 (17:08 +0200)]
Merge pull request #40728 from guits/doc_fix_pattern_rgw_nfs

doc/rgw-nfs: use same pattern for keyring name

3 years agoMerge pull request #43254 from cyx1231st/wip-seastore-fix-onode-order
Samuel Just [Wed, 22 Sep 2021 14:05:01 +0000 (07:05 -0700)]
Merge pull request #43254 from cyx1231st/wip-seastore-fix-onode-order

crimson/onode-staged-tree: convert hash to the reversed version

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43249 from cyx1231st/wip-seastore-fix-omap-hint
Kefu Chai [Wed, 22 Sep 2021 13:47:37 +0000 (21:47 +0800)]
Merge pull request #43249 from cyx1231st/wip-seastore-fix-omap-hint

crimson/os/seastore: add missing hints in omap tree

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43055 from soumyakoduri/wip-skoduri-lua
Daniel Gryniewicz [Wed, 22 Sep 2021 12:12:23 +0000 (08:12 -0400)]
Merge pull request #43055 from soumyakoduri/wip-skoduri-lua

rgw/lua: Install the packages only for RadosStore

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Daniel Gryniewicz <dang@redhat.com>

3 years agoMerge pull request #43054 from soumyakoduri/wip-skoduri-dbstore-vstart
Daniel Gryniewicz [Wed, 22 Sep 2021 12:11:30 +0000 (08:11 -0400)]
Merge pull request #43054 from soumyakoduri/wip-skoduri-dbstore-vstart

rgw: Add option to configure backend store

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #42911 from soumyakoduri/wip-skoduri-dbstore-object
Daniel Gryniewicz [Wed, 22 Sep 2021 12:10:52 +0000 (08:10 -0400)]
Merge pull request #42911 from soumyakoduri/wip-skoduri-dbstore-object

rgw/dbstore object APIs

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #43241 from sebastian-philipp/suites-orch-labelere
Sebastian Wagner [Wed, 22 Sep 2021 09:34:35 +0000 (11:34 +0200)]
Merge pull request #43241 from sebastian-philipp/suites-orch-labelere

.github: fix path to cephadm suite

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agocrimson/onode-staged-tree: consolidate laddr hint calculation 43254/head
Yingxin Cheng [Wed, 22 Sep 2021 07:43:23 +0000 (15:43 +0800)]
crimson/onode-staged-tree: consolidate laddr hint calculation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/onode-staged-tree: convert hash to the reversed version
Yingxin Cheng [Wed, 22 Sep 2021 07:36:22 +0000 (15:36 +0800)]
crimson/onode-staged-tree: convert hash to the reversed version

Store the reversed version of object hash to make sure that onodes in
the same PG are sorted together.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/onode-staged-tree: print cursh-hash in hex mode
Yingxin Cheng [Wed, 22 Sep 2021 06:52:18 +0000 (14:52 +0800)]
crimson/onode-staged-tree: print cursh-hash in hex mode

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore: add missing hints in omap tree 43249/head
Yingxin Cheng [Sat, 18 Sep 2021 07:10:38 +0000 (15:10 +0800)]
crimson/os/seastore: add missing hints in omap tree

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #43247 from rzarzynski/wip-crimson-ertr-safe_then_unpack
Samuel Just [Wed, 22 Sep 2021 00:12:58 +0000 (17:12 -0700)]
Merge pull request #43247 from rzarzynski/wip-crimson-ertr-safe_then_unpack

crimson/common: add safe_then_unpack() to errorated futures

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agocrimson/common: implement singleton_ec.
Radoslaw Zarzynski [Tue, 21 Sep 2021 22:58:56 +0000 (22:58 +0000)]
crimson/common: implement singleton_ec.

Unfortunately, GCC explodes when it sees that.

```
[rzarzynski@o06 build]$ ninja crimson-osd vstart
[1/3] Building CXX object src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o
FAILED: src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o
/usr/bin/ccache /opt/rh/gcc-toolset-9/root/usr/bin/c++ -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DHAVE_CONFIG_H -DSEASTAR_API_LEVEL=6 -DWITH_SEASTAR=1 -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -Isrc/include -I../src -I../src/seastar/include -Isrc/seastar/gen/include -isystem boost/include -isystem include -isystem ../src/xxHash -isystem ../src/rapidjson/include -O2 -g -DNDEBUG -fPIC   -U_FORTIFY_SOURCE -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wstrict-null-sentinel -Woverloaded-virtual -fno-new-ttp-matching -fstack-protector-strong -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -ftemplate-backtrace-limit=0 -Wno-non-virtual-dtor -std=gnu++17 -U_FORTIFY_SOURCE -Wno-maybe-uninitialized -DSEASTAR_SSTRING -Wno-error=unused-result -std=c++17 -MD -MT src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o -MF src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o.d -o src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o -c ../src/crimson/os/cyanstore/cyan_store.cc
during IPA pass: inline
../src/crimson/os/cyanstore/cyan_store.cc: In member function ‘std::string crimson::os::singleton_ec<MsgV>::this_error_category::message(int) const [with const char* MsgV = (& msg)]’:
../src/crimson/os/cyanstore/cyan_store.cc:97:17: internal compiler error: Segmentation fault
   97 |     std::string message([[maybe_unused]] const int ev) const final {
      |                 ^~~~~~~
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
Preprocessed source stored into /tmp/ccFi4GGw.out file, please attach this to your bugreport.
ninja: build stopped: subcommand failed.
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/common: add safe_then_unpack() to errorated futures. 43247/head
Radoslaw Zarzynski [Tue, 21 Sep 2021 21:28:45 +0000 (21:28 +0000)]
crimson/common: add safe_then_unpack() to errorated futures.

It was a prerequisite for another commit I finally thrown
away. However, this little bit can be still be useful even
for the sake of compliance with the interruptible variant.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/common: assert_moveable() doesn't depend on 3rd party's always_false<>.
Radoslaw Zarzynski [Tue, 21 Sep 2021 21:28:02 +0000 (21:28 +0000)]
crimson/common: assert_moveable() doesn't depend on 3rd party's always_false<>.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #43243 from rzarzynski/wip-crimson-net-dangling-bindvec2
Samuel Just [Tue, 21 Sep 2021 18:56:12 +0000 (11:56 -0700)]
Merge pull request #43243 from rzarzynski/wip-crimson-net-dangling-bindvec2

crimson/net: fix dangling addrvec in bind(), the repeat_until_value() part

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43209 from rzarzynski/wip-crimson-silent-check-bot
Samuel Just [Tue, 21 Sep 2021 18:55:30 +0000 (11:55 -0700)]
Merge pull request #43209 from rzarzynski/wip-crimson-silent-check-bot

tests/crimson: don't be so verbose when run by the 'make check' bot.

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge PR #43219 into master
Patrick Donnelly [Tue, 21 Sep 2021 18:03:08 +0000 (14:03 -0400)]
Merge PR #43219 into master

* refs/pull/43219/head:
doc: clarify MDS failed state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge PR #43156 into master
Sage Weil [Tue, 21 Sep 2021 15:37:50 +0000 (11:37 -0400)]
Merge PR #43156 into master

* refs/pull/43156/head:
qa/suites/orch/rook/smoke: add flannel back in

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocrimson/net: fix dangling addrvec in bind(), the repeat_until_value() part. 43243/head
Radoslaw Zarzynski [Mon, 20 Sep 2021 13:59:30 +0000 (13:59 +0000)]
crimson/net: fix dangling addrvec in bind(), the repeat_until_value() part.

This is a follow-up to commit 480273082718230d64d2c068db1b22f44997bdc8.

`seastar::do_with()` doesn't extend the life-time of the callable's

```cpp
auto
do_with_impl(T1&& rv1, T2&& rv2, More&&... more) {
    // ...
    auto&& just_values = cherry_pick_tuple(idx(), std::move(all));
    auto&& just_func = std::move(std::get<nr>(std::move(all)));
    // ...
    auto task = std::apply(
        [](auto&&... x) {
            return std::make_unique<internal::do_with_state<value_tuple, ret_type>>(std::forward<decltype(x)>(x)...);
        },
        std::move(just_values));
    auto fut = std::apply(just_func, task->data());
    // ...
}
```

closure. This means the `addrs` needs to be captured by value also
by the lambda passed to `repeat_until_value()`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #43225 from guits/fix-iscsi-doc
Ilya Dryomov [Tue, 21 Sep 2021 12:35:59 +0000 (14:35 +0200)]
Merge pull request #43225 from guits/fix-iscsi-doc

doc/iscsi: fix some typos

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years ago.github: fix path to cepham suite 43241/head
Sebastian Wagner [Tue, 21 Sep 2021 11:35:03 +0000 (13:35 +0200)]
.github: fix path to cepham suite

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #43237 from javicacheiro/fix_upgrade_now_uses_quay.io
Sebastian Wagner [Tue, 21 Sep 2021 11:29:09 +0000 (13:29 +0200)]
Merge pull request #43237 from javicacheiro/fix_upgrade_now_uses_quay.io

doc:  from v16.2.6 you have to point to the quay.io image

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #43142 from sebastian-philipp/c-v-cephadm-osd-activate
Sebastian Wagner [Tue, 21 Sep 2021 11:12:07 +0000 (13:12 +0200)]
Merge pull request #43142 from sebastian-philipp/c-v-cephadm-osd-activate

doc/ceph-volume/activate: Users should call cephadm instead

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoMerge pull request #43162 from sebastian-philipp/cephadm-service-mgmt-reoder
Sebastian Wagner [Tue, 21 Sep 2021 11:03:08 +0000 (13:03 +0200)]
Merge pull request #43162 from sebastian-philipp/cephadm-service-mgmt-reoder

doc/cephadm/service-management: Move "Updating Service Specifications"

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #43210 from rhcs-dashboard/config-e2e-failure
Ernesto Puerta [Tue, 21 Sep 2021 09:53:41 +0000 (11:53 +0200)]
Merge pull request #43210 from rhcs-dashboard/config-e2e-failure

mgr/dashboard: Fix failing config dashboard e2e check

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agodoc: from v16.2.6 you have to point to the quay.io image if using docker 43237/head
Javier Cacheiro [Tue, 21 Sep 2021 09:37:42 +0000 (11:37 +0200)]
doc: from v16.2.6 you have to point to the quay.io image if using docker

New images are no longer uploaded to the Docker Hub registry, so Docker users should point the upgrade
command to the quay.io image.

Fixes: https://tracker.ceph.com/issues/52683
Signed-off-by: Javier Cacheiro <javier.cacheiro.lopez@cesga.es>
3 years agoMerge pull request #43233 from badone/wip-install-xmlstarlet
Brad Hubbard [Tue, 21 Sep 2021 05:29:28 +0000 (15:29 +1000)]
Merge pull request #43233 from badone/wip-install-xmlstarlet

Revert "rpm, debian: drop xmlstarlet from build deps"

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoRevert "rpm, debian: drop xmlstarlet from build deps" 43233/head
Brad Hubbard [Tue, 21 Sep 2021 03:14:17 +0000 (13:14 +1000)]
Revert "rpm, debian: drop xmlstarlet from build deps"

This reverts commit 707edc0b2d5283104aeb472131ee94dd59544290.

This commit was somewhat premature in removing the xmlstarlet
dependency.

Fixes: https://tracker.ceph.com/issues/52681
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
3 years agoMerge PR #43179 into master
Patrick Donnelly [Tue, 21 Sep 2021 00:48:00 +0000 (20:48 -0400)]
Merge PR #43179 into master

* refs/pull/43179/head:
qa: lengthen grace for fs map showing dead MDS

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
3 years agoMerge PR #43164 into master
Patrick Donnelly [Tue, 21 Sep 2021 00:47:19 +0000 (20:47 -0400)]
Merge PR #43164 into master

* refs/pull/43164/head:
test/libcephfs: put inodes after lookup

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge PR #42763 into master
Patrick Donnelly [Tue, 21 Sep 2021 00:46:25 +0000 (20:46 -0400)]
Merge PR #42763 into master

* refs/pull/42763/head:
mon/FSCommands: add 'recover' flag in `fs new` command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agorbd-mirror: fix a couple of brainos in log messages
Ilya Dryomov [Mon, 20 Sep 2021 20:36:10 +0000 (22:36 +0200)]
rbd-mirror: fix a couple of brainos in log messages

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agorbd-mirror: unbreak one-way snapshot-based mirroring
Ilya Dryomov [Mon, 20 Sep 2021 19:52:57 +0000 (21:52 +0200)]
rbd-mirror: unbreak one-way snapshot-based mirroring

Snapshot replayer needs the remote's mirror peer uuid to find its
snapshots in the remote image.  It is obtained by listing remote's
mirror peers but RemotePoolPoller::handle_mirror_peer_list() skips
tx-only (MIRROR_PEER_DIRECTION_TX) peers.  In effect only rx-tx
(MIRROR_PEER_DIRECTION_RX_TX) peers are considered for matching
and snapshot replayer always fails with "failed to retrieve mirror
peer uuid from remote pool" error.

Instead, skip rx-only (MIRROR_PEER_DIRECTION_RX) peers as we are
definitely not interested in anything having to do with mirroring
_to_ the remote cluster.

Fixes: https://tracker.ceph.com/issues/52675
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agorgw/dbstore: Update README 42911/head
Soumya Koduri [Wed, 15 Sep 2021 19:29:35 +0000 (00:59 +0530)]
rgw/dbstore: Update README

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agoMerge pull request #43222 from sebastian-philipp/rook-ceph-client-before-3.8
Kefu Chai [Mon, 20 Sep 2021 16:16:35 +0000 (00:16 +0800)]
Merge pull request #43222 from sebastian-philipp/rook-ceph-client-before-3.8

mgr/rook: update rook-client-python to 2737c7467b7

Reviewed-by: Joseph Sawaya <jsawaya@redhat.com>
3 years agoMerge PR #43213 into master
Sage Weil [Mon, 20 Sep 2021 15:00:23 +0000 (11:00 -0400)]
Merge PR #43213 into master

* refs/pull/43213/head:
mon/OSDMonitor: make weight set create safer

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge PR #43211 into master
Sage Weil [Mon, 20 Sep 2021 14:59:56 +0000 (10:59 -0400)]
Merge PR #43211 into master

* refs/pull/43211/head:
qa/suites/upgrade/pacific-x: skip c-v install on pacific start
qa/packages: install ceph-volume

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agodoc/iscsi: fix some typos 43225/head
Guillaume Abrioux [Mon, 20 Sep 2021 14:02:33 +0000 (16:02 +0200)]
doc/iscsi: fix some typos

This fixes some typos in the iscsi documentation.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agorgw/dbstore: Object APIs
Soumya Koduri [Wed, 1 Sep 2021 12:10:59 +0000 (17:40 +0530)]
rgw/dbstore: Object APIs

Support for simple Put, Get, Delete, List Ops of Regular Objects on dbstore.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agorgw/dbstore: Insert testid user on startup 43054/head
Soumya Koduri [Wed, 8 Sep 2021 19:10:10 +0000 (00:40 +0530)]
rgw/dbstore: Insert testid user on startup

Its temporary change to insert testid user while
starting up rgw server on dbstore.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agorgw: Add option to configure backend store
Soumya Koduri [Sun, 5 Sep 2021 07:21:18 +0000 (12:51 +0530)]
rgw: Add option to configure backend store

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agorgw: Install the lua packages only for RadosStore 43055/head
Soumya Koduri [Thu, 9 Sep 2021 08:26:16 +0000 (13:56 +0530)]
rgw: Install the lua packages only for RadosStore

Installation, remove and listing of lua packages is supported
only on RadosStore. Verify the same.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agoMerge pull request #43193 from dang/wip-dang-zipper-lifecycle
Daniel Gryniewicz [Mon, 20 Sep 2021 13:12:52 +0000 (09:12 -0400)]
Merge pull request #43193 from dang/wip-dang-zipper-lifecycle

RGW Zipper - clean up and clarify lifecycle of s->object, s->bucket, and friends

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Soumya Koduri <skoduri@redhat.com>
3 years agoqa/suites/upgrade/pacific-x: skip c-v install on pacific start 43211/head
Sage Weil [Mon, 20 Sep 2021 12:47:45 +0000 (08:47 -0400)]
qa/suites/upgrade/pacific-x: skip c-v install on pacific start

Signed-off-by: Sage Weil <sage@newdream.net>