Merge PR #36097 into master

* refs/pull/36097/head:
mds: fix nullptr dereference in MDCache::finish_rollback

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35969 into master

* refs/pull/35969/head:
mds: optimize random threshold lookup for dentry load

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #35953 into master

* refs/pull/35953/head:
qa: Fix name collisions

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #35902 into master

* refs/pull/35902/head:
ceph-dedup-tool: fix warning
mds: slave -> peer
mds: master -> leader

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36139 into master

* refs/pull/36139/head:
mon: refactor ServiceMap::Service::get_task_summary()
mon: only dump non-empty "task status"
mgr/ServiceMap: move ServiceMap::Daemon implementation into .cc file

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #36140 into master

* refs/pull/36140/head:
mgr/volumes: fix "ceph nfs export" help messages

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35851 from dang/wip-dang-zipper-6

Wip dang zipper 6

Clean up some leaks in rgw_admin

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

NULL out g_ceph_context when deleting it

This avoids a use-after-free when there is a crash on shutdown (or when
something like ASAN triggers an ABORT on leak).

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Zipper Object

This pass modifies RGW to use RGWObject and RGWRadosObject.
Also improves RGWBucket usage.  This converts many of the APIs to
unique_ptr.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #35553 from zdover23/wip-doc-iscsi-target-cli-daemon

doc/rbd: add rbd-target-gw enable and start

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35846 from sebastian-philipp/podman-pull-retry

cephadm: Retry pull on transient error

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #34696 from tspmelo/wip-locl

mgr/dashboard: Use $localize and locl for TS translations

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>

Merge pull request #36000 from tspmelo/wip-favicon

mgr/dashboard: Display cluster status in favicon

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #35456 from sebastian-philipp/cephadm-events

mgr/cephadm: Add k8s-style event system

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #35985 from bk201/wip-46413

mgr/dashboard: fix regression in delete OSD modal

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge PR #36124 into master

* refs/pull/36124/head:
script/ptl-tool: add flag to turn off credits

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #36077 from dillaman/wip-librbd-parent-cache

librbd: fix parent cache initialization failures

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: Use $localize and locl for TS translations

Fixes: https://tracker.ceph.com/issues/45950
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #36128 from xxhdx1985126/wip-crimson-fix-osd-shutdown

crimson/osd: fix osd shutdown problem

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36142 from tchaikov/wip-crimson-cleanup

crimson: do not capture unused variables

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35987 from tspmelo/wip-update-lodash

mgr/dashboard: Fix some npm audit warnings

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #36138 into master

* refs/pull/36138/head:
doc/cephfs/nfs: Update about deploying nfs-ganesha with cephadm in vstart

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35960 from votdev/issue_46395_required_field_message

mgr/dashboard: Use same required field message accross the UI

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35777 from ofriedma/rgw-enoent-cache

rgw: add negative cache to the system object

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36070 from tspmelo/wip-fix-async-unittest

mgr/dashboard: Block the import of async from @angular/core/testing

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #36113 from trociny/wip-snap-create-error-msg

librbd: fix snapshot create error message

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35974 from tspmelo/wip-clean-headers

mgr/dashboard: Clean custom header styles

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

crimson: do not capture unused variables

this change silences warnings like:

mson/osd/backfill_state.cc:131:11: warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
      [&, this](const pg_log_entry_t& e) {
        ~~^~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36099 from dillaman/wip-rbd-nbd-hook

rbd-nbd: quisce hook should be installed to /usr/libexec/...

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/volumes: fix "ceph nfs export" help messages

They say "<attach>" but they mean "<clusterid>".

Fixes: https://tracker.ceph.com/issues/46572
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35605 from ceph/osd-compact-on-start

osd: Add config option to compact OSD's object store on start

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mon: refactor ServiceMap::Service::get_task_summary()

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon: only dump non-empty "task status"

* mgr/ServiceMap: add ServiceMap::Service::has_running_tasks() to tell
  if a service has any running tasks
* mon/Monitor: do not print an empty "task status" section when printing
  cluster status. it's just confusing.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/ServiceMap: move ServiceMap::Daemon implementation into .cc file

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/cephfs/nfs: Update about deploying nfs-ganesha with cephadm in vstart

Fixes: https://tracker.ceph.com/issues/46571
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #36122 from zdover23/wip-doc-rados-redundant-a-typo

doc/rados: remove a redundant "a" from a sentence

This sentence removes an "a" from a sentence, which makes the sentence grammatically correct.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: Block the import of async from @angular/core/testing

We no longer need this and it was causing unexpected results in some tests.

Fixes: https://tracker.ceph.com/issues/46500
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Display cluster status in favicon

Fixes: https://tracker.ceph.com/issues/44542
Signed-off-by: Tiago Melo <tmelo@suse.com>

osd: Add config option to compact OSD's object store on start

By setting this configuration option an OSD will compact it's
store's OMAP on start.

This way admin's can trigger an offline compaction by setting this
configuration value to 'true' and then restarting the OSD.

This is easier than using tools like 'ceph-kvstore-tool' with requires
more manual work on the CLI and might be more difficult for users.

Signed-off-by: Wido den Hollander <wido@42on.com>

mgr/cephadm: Set exception context to populate orch events.

Like when if daemon deployment fails

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

crimson/osd: fix osd shutdown problem

when doing full-system shutdown, monitors may go down before OSDs in which case
the osd shutdown hangs waiting for monc to successfully send the markmedown msg
to monitors

Fixes: https://tracker.ceph.com/issues/46564
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

Merge pull request #36103 from dillaman/wip-44311

crush/CrushWrapper: rebuild reverse maps after rebuilding crush map

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #35768 from rhcs-dashboard/fix-45185-master

mgr/dashboard: fix pool usage calculation

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #36095 from orozery/striper_get_file_offset

osdc/Striper: add get_file_offset function

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36091 from tchaikov/wip-46495

mgr/DaemonServer: keep daemon in service map when it has no tasks

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #36051 from Songweibin/wip-rados-setomapval

tools/rados: add an option "--input-file file" to `setomapval` command

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/cephadm: Add event when deploying a daemon

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/rados: remove a redundant "a" from a sentence

This commit changes the (not quite) sentence "Once
you have a deployed a Ceph Storage Cluster, you may
begin operating your cluster." to "Once you have
deployed a Ceph Storage Cluster, you may begin
operating your cluster."

Fixes: https://tracker.ceph.com/issues/46554
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #35130 from rzarzynski/wip-crimson-backfill

crimson: add support for backfill, part 0

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/cephadm: Add an event when creating a service

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add inventory.EventStore

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/orch: Add events to tests/test_orchestrator

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/orch: Add OrchestratorEvent class

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

cephadm: also retry on digest mismatch

Fixes: https://tracker.ceph.com/issues/46534
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/rbd: add rbd-target-gw enable and start

This commit adds the following commands to  the "Configuring the iSCSI Target Using the Command Line" page: "systemctl enable rbd-target-gw" and "systemctl start rbd-target-gw"

Fixes: https://tracker.ceph.com/issues/45987
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #33770 from majianpeng/osd-simple-logger

osd/OSD: simple perfcounter usage in OSDService

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35989 from bigwhite37/fix-dedup-tool-unused-var-warn

tools/ceph_dedup_tool: fix unused variable warning

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

script/ptl-tool: add flag to turn off credits

To speed up integration branch creation.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30780 from zhangsw/fix-rgw-multipart-abortdate

rgw: add abort multipart date and rule-id header to init multipart upload response

Merge pull request #35950 from zdover23/wip-doc-dev-guide-add-zdover-to-lead-list

doc/devel: add zdover to leads list

Merge pull request #34537 from ideepika/wip-bluestore-mem-tracking

os/bluestore: mempool split for finer granularity

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

librbd: fix parent cache races and error handling

If the plugin fails to connect to the daemon at start-up it will
crash the process due to a resource deadlock exception being
thrown as the client is destroyed. Additionally, librbd will support
concurrent IO thread processing in the future so the client needs
to be protected by a lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35901 from sebastian-philipp/juanmi-firewall-refactor

cephadm: refactor update_firewalld

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35862 from adk3798/cephadm_45724

mgr/cephadm: check-host should not fail as hard using fqdn

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

librbd: fix snapshot create error message

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #35471 from sebastian-philipp/cephadm-daemon-spec

mgr/cephadm: Add CephadmDaemonSpec class

Merge pull request #35838 from mgfritch/cephadm-rgw-spec-validate

python-common: RGW service_id might not contain a zone

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

librbd: move ContextWQ::queue definition to header

The parent cache plugin uses the ContextWQ::queue method and therefore
requires its definition to properly dynamically link into the librbd
process. If future plugins require additional functions this can be
reconsidered by using interfaces, static libraries, or moving generic
functions to libcommon.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc/devel: add zdover to leads list

This commit adds "zdover" to the list of
leads as the Docs Lead.

Fixes: https://tracker.ceph.com/issues/46378
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #36096 from BenoitKnecht/doc-osd-scrub-during-recovery

doc/rados: Fix osd_scrub_during_recovery default value

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #36052 from smithfarm/wip-focce

bluestore: fix typo in _flush function declaration

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35991 from votdev/adapt_cephadm_vagrantbox_to_el8

mgr/cephadm: Adapt Vagrantbox to el8

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35315 from weixinwei/master

blk:BlockDevice.cc: use pending_aios instead of iovec size as ios num

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #35844 from wuhongsong/fuse-assert-Dprocess

common:  ignore SIGHUP prior to fork

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36054 from tchaikov/wip-blk-no-unused-label

blk/BlockDevice: use ceph_assertf() for assert

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #35744 from jmolmo/document_unmanaged_use

doc: Add information about unmanaged parameter

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>

mgr/cephadm: Add test for _check_daemons()

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: refactor `_create_daemon()`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add CephadmDaemonSpec class

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35709 from zdover23/wip-doc-preferring-yaml-to-cli-in-cephadm

doc/cephadm: add yaml info for orch apply mon

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/DaemonServer: keep daemon in service map when it has no tasks

we should not remove a daemon from service map even if it has no task
running, as it is not an evidence that the service is removed from the
cluster.

this change partially reverts 5a7d0632d989536e09b2d916b5ea9713dc8dbea5

Fixes: https://tracker.ceph.com/issues/46495
Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/DaemonServer: pass task_status instead of a message

be explicit that the caller is only interested in the task_status

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29283 from myoungwon/wip-refcount-snap

osd: refcounting chunks for snapshotted manifest object

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com> (portions Sam wrote)

crush/CrushWrapper: rebuild reverse maps after rebuilding crush map

The Objecter will crash when localized reads are enabled and two threads
attempt to rebuild the (invalidated) reverse maps concurrently. This
should address the issue for the Objecter use-case without the need to
add additional locking.

Fixes: https://tracker.ceph.com/issues/44311
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-nbd: quisce hook should be installed to /usr/libexec/...

The rbd-nbd daemon expects the rbd-nbd_quiesce script to be installed
in the libexec directory on all distros.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35955 from jschmid1/deprecate_filstore

mgr/cephadm: do not accept filestore osds in osdspecs

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

doc/cephadm: add yaml infor for orch apply mon

This commit breaks one of the long sentences into
three shorter sentences, and adds a parenthetical
comment walking the reader through a series of commands,
explaining what each command does and why only the last
of the commands ends up having a lasting effect on the Ceph
environment.

Fixes: https://tracker.ceph.com/issues/46133
Signed-off-by: Zac Dover <zac.dover@gmail.com>

ceph-dedup-tool: fix warning

Signed-off-by: Sage Weil <sage@newdream.net>

mds: slave -> peer

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: check-host should not fail as hard using fqdn

Print error message instead of traceback when check-host
fails in due to host address not being found

Fixes: https://tracker.ceph.com/issues/45724
Signed-off-by: Adam King <adking@redhat.com>

mds: master -> leader

Signed-off-by: Sage Weil <sage@newdream.net>

cephadm: refactor update_firewalld

* split into three different methods
* untangle service and port lists

This helps in making our firewall handling more dynamic.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Co-authored-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

mds: fix nullptr dereference in MDCache::finish_rollback

introduced by commit 4940ab62e0d
"mds: preserve ESlaveUpdate::OP_PREPARE logevent before doing commit"

Fixes: https://tracker.ceph.com/issues/46533
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

doc/rados: Fix osd_scrub_during_recovery default value

Since 8dca17c, `osd_scrub_during_recovery` defaults to `false`, but the
documentation was still stating that its default value is `true`.

Fixes: https://tracker.ceph.com/issues/46531
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

osdc/Striper: add get_file_offset function

This commit adds a get_file_offset translating (object_no, object_off) -> file_offset.
This is useful for encryption object dispatch layer in librbd
to comply with disk-encryption standards that require the file offset as input.

Signed-off-by: Or Ozeri <oro@il.ibm.com>

qa: Fix name collisions

Addresses the name collisions of volumes, subvolumes,
clones, subvolume groups and snapshots within the tests.

Fixes: https://tracker.ceph.com/issues/43517
Signed-off-by: Kotresh HR <khiremat@redhat.com>

mgr/dashboard: Fix some npm audit warnings

Version of lodash that was use by all packages was causing some warnings.

Fixes: https://tracker.ceph.com/issues/46431
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #36013 from Devp00l/wip-46376

cephadm: Make Vagrantfile more flexible

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #36072 from zdover23/wip-doc-dev-word-choice-guidance

doc/dev: remove extraneous word "guidance".