doc/rgw: warn about "trust forwarded https" security

Warn users about the implications of enabling this option when there is
no trusted proxy in front of radosgw.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #35139 from smithfarm/wip-proxy-is-removed

qa/cephtool: remove/deprecate proxy cache-tier mode

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35177 from wjwithagen/wjw-fix-ceph-backport.sh

script: fix bash invocation on systems where bash live elsewhere

Reviewed-by: Nathan Cutler <ncutler@suse.com>

script: fix bash invocation on systems where bash live elsewhere

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

qa/cephtool: clean up after removal of cache-tier modes

The "proxy" and "forward" cache-tier modes have been completely removed,
so it's sufficient to test once that they cannot be set.

Fixes: a0a3ed324a130a9934735927600f01d1fa65bc0d
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35160 from rzarzynski/wip-crimson-objclass-subop_num

crimson/osd: implement cls_current_subop_num of the objclass API.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35154 from tchaikov/wip-crimson-admin-cleanup

crimson/admin: remove the unix domain socket for asock when stopping

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: implement cls_current_subop_num of the objclass API.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35114 from mgfritch/cephadm-reconfig-port-in-use

cephadm: skip port in use check during reconfig/redeploy

Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #35087 from sebastian-philipp/cephadm-fix-osd-service-create

mgr/cephadm: Fix OSDs not being created

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35112 from athanatos/sjust/wip-manifest-doc

doc: update manifest.rst with plans for manifest work

Reviewed-by: Myoungwon Oh <omwmw@sk.com>

crimson/admin: remove the unix domain socket file for asock when stopping

should remove the leftovers when stopping

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #33286 into master

* refs/pull/33286/head:
cephfs-shell: support cephfs-shell.conf

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

cephfs-shell: support cephfs-shell.conf

cephfs-shell options should reside in cephfs-shell.conf and not in
ceph.conf. Please note, unlike before, -c option of cephfs-shell must
take path to cephfs-shell.conf instead of path to ceph.conf.

This commit also updates the docs and the tests for cephfs-shell accordingly
and renames the variable config_path in cephfs-shell to shell_conf_path so
that it's easy to distinguish.

Fixes: https://tracker.ceph.com/issues/44127
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge PR #33264 into master

* refs/pull/33264/head:
test_cephfs_shell: refactor test_put_and_get_without_target_directory

Reviewed-by: Varsha Rao <varao@redhat.com>

mon/OSDMonitor: do not tell user 'proxy' cache-mode allowed

The 'proxy' cache-mode is deprecated and can no longer be used.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35088 from sebastian-philipp/cephadm-admin-socket

doc/cephadm: accessing the admin socket

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35125 from tchaikov/wip-cephadm-test

qa/tasks/cephadm: ignore tarfile.ReadError

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/dev/osd_internals:manifest.rst: clarification for manifest design and future steps

I've added a few significant changes:
* TIER_PROMOTE purely ensures that data is resident in the base pool
* Add EVICT_CHUNK to permit a tiering agent to selectively cold space
  from HEAD/snapshots independently of SET_CHUNK.
* Avoid using DIRTY entirely for dedup targets.
* Instead of modifying clone_range when updating clone manifests, simply
  update ReplicatedBackend::calc_*_subsets to also consider the clone
  obc manifest.

I've also added sections on how tiering agents are rbd/rgw are meant
to interact as well as on testing.

Signed-off-by: Samuel Just <sjust@redhat.com>

doc: more information on future work for manifest tiering

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

doc: more information on status and future work for manifest tiering

Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

doc: add more information for manifest tiering

Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

Merge pull request #35119 from ShraddhaAg/update-docs

doc: add machine type in teuthology commands

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35110 from neha-ojha/wip-revert-34894

Revert "qa/tests: replaced ubuntu_latest.yaml with ubuntu 20.04"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Yuri Weinstein <yuriw@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35113 from athanatos/sjust/wip-crimson-clang-cleanup

crimson clang cleanups

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: remove unnecessary captures

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os: remove some unnecessary lambda captures

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore/journal: avoid capturing destructured labels

clang at least doesn't allow those labels to be captured.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson: remove a few unnecessary lambda captures

Signed-off-by: Samuel Just <sjust@redhat.com>

errorator: don't rely on non-evaluation of static_assert in constexpr branch

At least in clang, we appear to fail that static assert even with another
branch taken.  Instead, assert that one of the other conditions is true.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #35127 from ronen-fr/scrub4

test/librbd, Crimson: remove some unused captures

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: add machine type in teuthology commands

This commit adds mandatory --machine-type flag in teuthology
command examples in docs.

Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>

Merge pull request #32927 from linuxbox2/wip-rgw-lc-threads

rgwlc: enable thread-parallelism in RGWLC

Crimson: cleanup Clang issues in Crimson code

Mostly: remove redundant captures

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

doc/cephadm: accessing the admin socket

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

test/librbd: remove some unused captures

...silencing Clang warnings

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #34980 from sebastian-philipp/cephadm-workunits-rm-ubuntu_2020

qa/cephadm: rm ubuntu_latest for now

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephadm: ignore tarfile.ReadError

there is chance that /var/log/ceph (and other directories) is not
created when cephadm fails, but we always try to collect /var/log/ceph
by tar the directory on the test node and transfer it back for archiving
it. when we fail to do so, an exception is thrown, and this exception is
in turn logged as the cause of the failure instead of the one which
fails the test.

in this change, `tarfile.TarError`s is caught and ignored if it is
thrown by `misc.pull_directory()`, as normally, we don't consider it a fatal
failure when `misc.pull_directory()` fails because of `tar`. the
failures for the test are.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/cephadm: Fix OSDs not being created

Fixes: https://tracker.ceph.com/issues/45560
This was caused by completions no longer getting
executed synchonously.

Also

* We don't want to have a busy loop when applying OSDs
* Added call to `_apply_all_services` to OSD test

This is a follow-up on f7bf4d0c597080f45bb8466bd6bab3b9cecb03c4

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35101 from sebastian-philipp/cephadm-ls-systemd-unit

cephadm: `ls`: also return systemd_unit

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35074 from dillaman/wip-45544

librbd: avoid completing mirror:DisableRequest while holding its lock

Reviewed-by: Mykola Golub <mgolub@suse.com>

cephadm: fixup command_deploy log line

fixup log line to reflect the deploy action
(e.g. deploy, reconfig, redeploy)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: skip `port_in_use` during redeploy

skip the `port_in_use` check during reconfig/redeploy
of the monitoring components and nfs-ganesha

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: remove unused `reconfig` param from create_daemon_dirs

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: don't check port in use during nfs reconfig

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #35062 into master

* refs/pull/35062/head:
qa/tasks/cephfs/fuse_mount.py: retry when the admin socket is not ready

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #34571 into master

* refs/pull/34571/head:
mds: don't shallow copy when decoding xattr map

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #34144 into master

* refs/pull/34144/head:
Filer: Add error handler in _do_purge_range()
mds: Handle blacklisted error in purge queue

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #34596 into master

* refs/pull/34596/head:
test: add a new program for testing ino_release_cb
client: add a new inode release request callback
client: expose ceph_ll_register_callbacks via libcephfs
client: move callback typedefs and arg struct into ceph_ll_client.h
client: rename ceph_statx.h to ceph_ll_client.h
client: make client_dentry_callback_t more friendly for C
client: only override umask_cb with non-NULL values

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #34890 into master

* refs/pull/34890/head:
cephfs-shell: Remove get_error_code() call on OSError type exception objects

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #34951 into master

* refs/pull/34951/head:
qa/cephfs: run() cleanup whether FS was mounted or not

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Revert "qa/tests: replaced ubuntu_latest.yaml with ubuntu 20.04"

This reverts commit 835f2f5f511e7363f4056e5137382982aac1bfbf.

Reverting this to narrow down recent failures seen in rados and upgrade suites.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #35058 into master

* refs/pull/35058/head:
qa/tasks: logger.warn() -> logger.warning()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35108 from trociny/wip-45589

mgr/rbd_support: rename "rbd_trash_trash_purge_schedule" oid

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34900 from gekios/wip-cephadm-bootstrapped

qa/tasks/cephadm.py: Support already bootstrapped clusters

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35071 from sebastian-philipp/cephadm-upgrade-check-mon-command

mgr/cephadm: upgrade: use check_mon_command

Merge pull request #35068 from bk201/wip-45163

mgr/cephadm: config iSCSI gateways in Dashboard

Merge pull request #35084 from jschmid1/caching_osdspec_previews

mgr/cephadm: implement cached osdspec previews

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/rbd_support: rename "rbd_trash_trash_purge_schedule" oid

to "rbd_trash_purge_schedule"

Fixes: https://tracker.ceph.com/issues/45589
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge branch 'wip-joshd-fix'

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/orchestrator: adapting orch interface to preview changes

Signed-off-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #35015 from smithfarm/wip-45514

doc: drop obsolete cache tier options

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35083 from p-se/wip-pse-cephadm-prom-scrape-interval

mgr/cephadm: increase global scrape interval

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35092 from varshar16/wip-cephadm-mdcache-ganesha

 mgr/cephadm/nfs: Instead of CACHEINODE use MDCACHE

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

librbd: avoid completing mirror:DisableRequest while holding its lock

Ensure that the lock is released before another thread has the chance
to complete the state machine and attempt to destruct the in-use lock.

Fixes: https://tracker.ceph.com/issues/45544
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/cephadm: upgrade: use check_mon_command

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34964 from dillaman/wip-45137

librbd: replace ImageRequestWQ with pluggable ImageDispatcher

Reviewed-by: Mykola Golub <mgolub@suse.com>

doc/cephadm: Synopsis: add `cephadm unit`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

cephadm: `ls`: also return systemd_unit

Let users directly know the systemd unit name.

Enable things like

```
cephadm ls | jq -r '.[].systemd_unit' | xargs -n 1 systemctl status | cat
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: increase global scrape interval

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: support setting iSCSI gateways from orchestrator

- Dashboard doesn't actively query iSCSI gateways from orchestrator anymore.
  Instead, Orchestrator backend invokes iscsi-gateway-add CLI after any
  iSCSI daemons are deployed.
- Add an optional argument `name` to `iscsi-gateway-add` command. If
  `name` is provided, Dashboard won't issue API request to iSCSI gateway
  to get the hostname. This is required because when orchestrator sets
  the gateway via CLI, the gateway is just spawned and might be not
  ready yet.

Fixes: https://tracker.ceph.com/issues/45163
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

mgr/cephadm: config iSCSI gateways in Dashboard

- Add `daemon_check_post` fuction to allow services implement actions
  needed to be done after daemons are checked.
- After iSCSI daemons are spawned, call dashboard CLI to
  set the API URL of the daemons.

Fixes: https://tracker.ceph.com/issues/45163
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #32682 from lorenzen-b1/patch-1

doc/architecture.rst: clarify scrubbing

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35091 from tchaikov/wip-crimson-cleanup

crimson/osd: avoid using variadic future

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #35081 into master

* refs/pull/35081/head:
qa: do not save/restore ip_forward sysctl

Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #34764 from ianwatsonrh/master

Docs: vault kv and transit engine updates

Reviewed-by: Casey Bodley <cbodley@redhat.com>

 mgr/cephadm/nfs: Instead of CACHEINODE use MDCACHE

As CACHEINODE will be deprecated soon.

Signed-off-by: Varsha Rao <varao@redhat.com>

crimson/osd: avoid using variadic future

it is deprecated by seastar. let's use future<tuple<...>> instead

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35025 from dillaman/wip-45518

librbd: copy API should not inherit v1 image format by default

Merge pull request #34789 from trociny/wip-rbd-nbd-quiesce

rbd-nbd: add quiesce/unquiesce hooks

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35023 from yuvalif/fix_issue_45513

cls/2pc_queue: fix timing issue in test

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34959 from yison/rwl-image-writeback-cache-seq10

rbd/cache: Replicated Write Log core codes - load existing cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34930 from trociny/wip-45409

rbd-mirror: wait for events to replay before shut down journal replay

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35042 from tchaikov/wip-cmake-remove-iwyu-cppcheck

cmake: remove cppcheck and iwyu targets

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #35070 from Devp00l/wip-45545-2

mgr/dashboard: Updates jest to fix cache failure

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34913 from athanatos/sjust/wip-seastore-journal

crimson/os/seastore: initial PR

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/cephadm: move iSCSI service class to its own file

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

mgr/cephadm: adapt cephadm to osdspec preview caching

Signed-off-by: Joshua Schmid <jschmid@suse.de>

mgr/cephadm: add preview caching to inventory.py

Signed-off-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #35052 from tspmelo/wip-unittest-warnings

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #35013 from matthewoliver/cephadm-iscsi-log-to-stderr

cephadm: enable logging to stderr in ceph-iscsi

Reviewed-by: Michael Fritch <mfritch@suse.com>

dec/dev/seastore.rst: add more detailed design information and status

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore: add initial cache support with tests

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore: introduce initial journal implementation and tests

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/gtest_seastar: add helpers for building seastar gtest unit tests

Also adds unsafe_get methods for errorator for use within unit tests.

Signed-off-by: Samuel Just <sjust@redhat.com>

qa: do not save/restore ip_forward sysctl

We don't really care if this is left on during the course of the mount.
These settings don't persist across tests anyway.

Fixes: https://tracker.ceph.com/issues/45525
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa/tasks/cephfs/fuse_mount.py: retry when the admin socket is not ready

Fixes: https://tracker.ceph.com/issues/45552
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #35077 from yuriw/wip-yuriw-crontab-master

qa/tests: reduced powercycle jobs amount

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: reduced powercycle jobs amount

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #32829 from Devp00l/wip-43765

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>