ceph.in: catch KeyboardInterrupt from outer code

This avoids getting random tracebacks when you control-C early in the
process.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph.in: fix second exception on ^C

ret isn't defined yet, and we know we were interrupted by the user, so use
an error code appropriate for that.

Fixes: https://tracker.ceph.com/issues/44157
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33289 into master

* refs/pull/33289/head:
qa/tasks/cephadm: deploy rgw daemons too

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #32775 into master

* refs/pull/32775/head:
ceph.spec.in: fix python3 dependencies in centos7

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33129 into master

* refs/pull/33129/head:
osd/PeeringState: do not start renewing leases until PG is activated

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #33163 into master

* refs/pull/33163/head:
msg/Policy: limit unregistered anon connections to mon

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #33350 from rzarzynski/wip-crimson-clang-watchnotify

crimson/osd: fix the Clang build in create_watch_info().

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33347 from tchaikov/wip-orch-addr

mgr/orchestrator: "addr" is optional for constructing InventoryNode

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #33349 from ronen-fr/clang-5

crimson/osd: remove unneeded captures - pg.cc

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: fix the Clang build in create_watch_info().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #33325 from tchaikov/wip-super-setup

qa/tasks: call super class's setUp()

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #33348 from tchaikov/wip-rbd-mirror-test

mgr/dashboard: s/fsid/mirror_uuid/

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/orchestrator: "addr" is optional for constructing InventoryNode

this addresses a regression introduced by 5276871e15

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/mgr/orch: s/service ls/ps/

to fix the broken test of "test_load_data". it's a regression introduced
by aacc9a650f052fd5be543e9265ec94833b8e8bb3

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: remove unneeded captures - pg.cc

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #33243 from tchaikov/wip-43795

ceph_argparse: put args from env before existing ones

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/tasks: call super class's setUp()

to address the regression introduced by
87292811215f6ded9a784d3216a910faaef648e2

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: s/fsid/mirror_uuid/

to fix the broken rbd-mirror test. it's a regression introduced by
7b07e3c9dcf1eda325fc4fe7960765c019243076

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33345 from athanatos/sjust/wip-fix-crimson-build

Crimson build fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33343 into master

* refs/pull/33343/head:
qa/suites/rados/cephadm/upgrade: add simple upgrade test
qa/tasks/cephadm: improve shell subcommand

Reviewed-by: Michael Fritch <mfritch@suse.com>

qa/tasks/cephadm: deploy rgw daemons too

Signed-off-by: Sage Weil <sage@redhat.com>

test_alien_echo: convert Condition to use readable_eventfd

Should have been included in 5f05a50bae8bb4889dba0d249ed5fc3a2fcdcfa5.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/CMakeLists.txt: link pthread to libcrimson for setting thread affinity

Signed-off-by: Samuel Just <sjust@redhat.com>

qa/suites/rados/cephadm/upgrade: add simple upgrade test

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/cephadm: improve shell subcommand

- pass teuth job params through environment
- run commands via bash -c

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33073 into master

* refs/pull/33073/head:
qa/suites/rados/cephadm: deploy prometheus.a
mgr/cephadm: implement prometheus add/update
mgr/cephadm: teach _create_daemon how to provision prometheus
mgr/orch: add prom hooks

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>

Merge pull request #33332 from jan--f/c-v-filestore-zap-fix

ceph-volume: don't remove vg twice when zapping filestore

Merge pull request #33320 from jan--f/c-v-fix-filestore-journal-size

ceph-volume: pass journal_size as Size not string

ceph-volume: don't remove vg twice when zapping filestore

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Fixes: https://tracker.ceph.com/issues/44149

Merge pull request #32679 from rzarzynski/wip-crimson-watchnotify_part1

crimson: add support for watch / notify, part 1

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33296 from tchaikov/wip-crimson-cmake

cmake: compile crimson-auth with crimson::cflags

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #33097 from dillaman/wip-43933

librbd: tweak deep-copy to avoid creating last snapshot until sync is complete

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #33298 from sebastian-philipp/debian-rook-jsonpatch

debian: add python3-jsonpatch as dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-volume: pass journal_size as Size not string

Fixes: https://tracker.ceph.com/issues/44148
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #32174 from ronen-fr/admin_commands_3

common,crimson: supporting admin-socket commands

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

debian: add python3-jsonpatch as dependency

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #33283 from jan--f/c-v-zap-on-vg-without-lv

ceph-volume: avoid calling zap_lv with a LV-less VG

Merge pull request #32690 from rosinL/wip-timeevent

msg/async/dpdk: destroy fd in do_request

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: compile crimson-auth with crimson::cflags

* move auth related stuff into crimson/CMakeLists.txt, so we can
  link them against crimson::cflags, which populates the necessary
  definitions and other cxx flags when building these source files.
* expose crimson::cflags as a public library of crimson, as
  crimson-osd links against crimson. and the cflags should be populated
  to crimson-osd, otherwise they are compiled with different compiler
  options.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33084 from tchaikov/wip-crimson-pg-coll

crimson/os: init PG with pg coll not meta coll

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #33241 from nagpavan-chilakam/patch-1

doc/architecture.rst: fix a typo in EC section

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33194 into master

* refs/pull/33194/head:
qa: add tests for mds_join_fs cluster affinity
qa: update cluster warning message for removed MDS
doc: add section on new mds_join_fs behavior
mon/MDSMonitor: enforce mds_join_fs cluster affinity
mon/MDSMonitor: use type of info.rank or mds_rank_t
qa: accept operation on current fs status
qa: add method to enable multifs
qa: fix nested generator use
qa: manage config changes through mons

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: fix the lifetime management of OpsExecuter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: dispatch MWatchNotify on watch removal.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: improve debugs in OpsExecuter::do_op_notify_ack.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: implement the complete phase of Notify propagation.

Q: How to test?
A:  * `${CEPHBIN:-bin}/rados -p rbd touch moj_watch`
    * `${CEPHBIN:-bin}/rados -p rbd watch moj_watch`
    * `${CEPHBIN:-bin}/rados -p rbd notify moj_watch test2`

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: dispatch MWatchNotify on CEPH_OSD_OP_NOTIFY.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

messages: MWatchNotify takes notifier_gid during construction.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: create_n_propagate

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: pass real connection in OpsExecuter::do_op_watch_subop_watch()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osd: minor fix formatting in Watch.cc.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osd: clean-up raw connection checks.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osd: unify connected() and is_connected() of Watch.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: drop the legacy path for undecodable notify acks.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: implement CEPH_OSD_OP_NOTIFY_ACK.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: implement CEPH_OSD_OP_NOTIFY.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: implement CEPH_OSD_WATCH_OP_PING.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: implement CEPH_OSD_WATCH_OP_RECONNECT.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: add execution units for WATCH and UNWATCH.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: introduce watchers to ObjectContext.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #33244 into master

* refs/pull/33244/head:
doc/mgr/orchestrator_modules: update docs
mgr/orch: break out apply_$type() in orchestrator.py interface
qa/tasks/cephadm: adapt to new cli
mgr/orch: add --force arg to 'daemon rm'
mgr/orch: fix host set-addr prefix
mgr/orch: service action cleanup
mgr/orch: prevent rm mon|mgr
mgr/orch: service rm -> rm
mgr/cephadm: add mon
mgr/orch: add mon hook
mgr/cephadm: add mgr
mgr/orch: add 'daemon add mgr'
mgr/orch: orch * add -> orch daemon add *
mgr/volumes: use apply_mds
mgr/orch: move to service_apply; remove update_foo()
mgr/rook: move to remove_service()
mgr/orch: remove old remove_{osds,rgw,mds,nfs,...} methods
mgr/orch: test remove_daemons and remove_service

Reviewed-by: Sebastian Wagner <swagner@suse.com>

qa/suites/rados/cephadm: deploy prometheus.a

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: implement prometheus add/update

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: teach _create_daemon how to provision prometheus

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: add prom hooks

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PeeringState: do not start renewing leases until PG is activated

The activate() work renews the lease so that we can send lease info out
to the peers immediately.  However, these messages may get delayed.  Since
we immediately start scheduling renewals, it's possible for the renewal
to go out before the PG is active, crashing the replicas.

Fix by not scheduling renewals until the PG is really active.

Also, renew aggressively at that point in time, since it may have been a
while since we first started the activation.

Fixes: https://tracker.ceph.com/issues/44041
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #30862 from zhangsw/fix-rgw-lc-repeated

rgw: set bucket attr twice when delete lifecycle config.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

Merge PR #33248 into master

* refs/pull/33248/head:
cephadm: fix deploy crash when no `args.fsid`

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #31088 from joke-lee/rgw-storage-class-apend-obj

rgw: set correct storage class for append

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #31661 from joke-lee/sts-args

rgw: sts: add all http args to req_info

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #33053 from tpsilva/add_max_connections

rgw: make max_connections configurable in beast

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33165 from clwluvw/policy-subuser

rgw: Add support bucket policy for subuser

Reviewed-By: Adam Emersen <aemerson@redhat.com>

Merge pull request #32636 from ivancich/wip-avoid-0-length-ordered-list-result

rgw: address 0-length listing results when non-vis entries dominate

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #32805 from smithfarm/wip-cmake-logging

rpm: add cmake_verbose_logging switch

Reviewed-by: David Disseldorp <ddiss@suse.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge PR #29427 into master

* refs/pull/29427/head:
mgr/rook: Make use of rook-client-python when talking to Rook
cmake: Integrate Rook client generation
mgr/rook: Automatically generate Rook client interface
Add submodule to rook-client-python.git

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>

ceph-volume: avoid calling zap_lv with a LV-less VG

Fixes: https://tracker.ceph.com/issues/44125
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

crimson/common: add show_config(Formatter) to ConfigProxy API

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

common/cmdparse: pass cmdmap by const reference

Signed-off-by: Kefu Chai <kchai@redhat.com>

common,crimson: supporting admin-socket commands

- basic infrastructure:
  - the seastar::thread that handles commands arriving over the UNIX_domain socket
  - command dispatching
- the implementation of some commands.
- simplified locking scheme: access to the API blocks (the registration elements) is
  now under one course-grained RW-lock.
- OSD shutdown support;

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

qa: add tests for mds_join_fs cluster affinity

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: update cluster warning message for removed MDS

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc: add section on new mds_join_fs behavior

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mon/MDSMonitor: enforce mds_join_fs cluster affinity

Before this, "mds_join_fs" config enforced a preference for the standby
the monitors would select. Now the monitors actively enforce this
by purposefully removing an MDS wither lower "affinity". An MDS standby
has highest affinity if its mds_join_fs is the file system in question
or a vanilla standby (no mds_join_fs).

Fixes: https://tracker.ceph.com/issues/43392
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mon/MDSMonitor: use type of info.rank or mds_rank_t

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: accept operation on current fs status

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add method to enable multifs

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: fix nested generator use

Apparently Python doesn't allow nested generator calls. Hurray.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: manage config changes through mons

This provides a generic framework for modifying Ceph configuration
changes in tests through the monitors rather than the asok interface or
local ceph.conf changes. Any changes are reverted during test teardown.

A future patch will convert existing tests manipulating the local
ceph.conf or admin socket.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc/mgr/orchestrator_modules: update docs

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: break out apply_$type() in orchestrator.py interface

Also clean up the method descriptions.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/cephadm: adapt to new cli

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: add --force arg to 'daemon rm'

This is needed when removing mons.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: fix host set-addr prefix

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: service action cleanup

- no 'service' prefix
- single name arg (type[.id...])

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: prevent rm mon|mgr

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: service rm -> rm

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: add mon

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: add mon hook

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: add mgr

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: add 'daemon add mgr'

Signed-off-by: Sage Weil <sage@redhat.com>