crimson/osd: can't ‘operator<’ ‘ceph_release_t’ and ‘int’

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>

Merge PR #27717 into master

* refs/pull/27717/head:
mds: output lock state in format dump

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28009 from ricardoasmarques/rbd-feature-migrating

Add 'RBD_FEATURE_MIGRATING' to rbd.pyx

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27680 from theanalyst/aws-rest-fixes

rgw: rest client fixes for cloud sync XML outputs

Reviewed-By: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Remove messages.xlf (#28020)

mgr/dashboard: Remove messages.xlf

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #27976 from mykaul/patch-2

doc: Update link to Red Hat documentation

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Remove messages.xlf

Removes the messages.xlf file from the git repository, due to a lot of merge conflicts.
It won't cause any problems, because the file is not important for building or the translation during runtime.

Fixes: https://tracker.ceph.com/issues/38003
Signed-off-by: Sebastian Krah <skrah@suse.com>

mds: output lock state in format dump

dump cache in plain text will print lock state. But in json format dump,
it won't. It is not convenient to debug some MDS lock issues without
such information.

Fixes: http://tracker.ceph.com/issues/39645
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>

Merge pull request #26654 from sebastian-philipp/orchestrator-progress

mgr/orchestrator: add progress events to all orchestrators

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

doc: Update link to Red Hat documentation

The previous link was to version 2, which is a bit older. Fix to version 3.

Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

Merge pull request #27855 from tchaikov/wip-ceph-release

src/: define ceph_release_t and use it

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27689 from Jeegn-Chen/wip-39390-fix

os/filestore: assure sufficient leaves in pre-split

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #27925 from tchaikov/wip-38628

mgr/restful: do not use filter() for list

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #27962 from xiexingguo/wip-jira-3756

mgr/DaemonServer: safe-to-destroy - do not consider irrelevant pgs

Reviewed-by: Yan Jun <yan.jun8@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28036 from neha-ojha/wip-fix-ar-doc

doc/dev/osd_internals/async_recovery: update cost calculation

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #28034 from letterwuyu/forward_incompat_pg_missing

ceph-object-corpus: forward_incompat pg_missing_item and pg_missing_t

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph-object-corpus: forward_incompat pg_missing_item and pg_missing_t

Signed-off-by: lishuhao <lishuhao@unitedstack.com>

doc/dev/osd_internals/async_recovery: update cost calculation

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #27688 into master

* refs/pull/27688/head:
pybind: added lseek()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27482 into master

* refs/pull/27482/head:
client: fix fuse client hang because its bad session PipeConnection

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27761 into master

* refs/pull/27761/head:
qa: use skipTest method instead of exception

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #27480 into master

* refs/pull/27480/head:
mds: check dir fragment to split dir if mkdir makes it oversized.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #27765 from linuxbox2/wip-rgw-crytpexcept

rgw: crypto: throw DigestException from Digest and HMAC

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #27843 into master

* refs/pull/27843/head:
os/bluestore: fix missing discard in BlueStore::_kv_sync_thread

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #27716 into master

* refs/pull/27716/head:
cephfs-shell: fix string decoding for ls command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28013 from cbodley/wip-39611

rgw: use explicit to_string() overload for boost::string_ref

Reviewed-by: Nathan Cutler <nculter@suse.com>

mgr/dashboard: fix help message for minimum blob size in pool form (#28017)

mgr/dashboard: fix help message for minimum blob size in pool form

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #27813 from ricardoasmarques/do-not-log-iscsi-get-requests

mgr/dashboard: iSCSI GET requests should not be logged

Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #27946 from ricardoasmarques/iscsi-form-ipv6

mgr/dashboard: Fix iSCSI form when using IPv6

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

mgr/dashboard: iSCSI GET requests should not be logged

Fixes: https://tracker.ceph.com/issues/39492
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Add 'RBD_FEATURE_MIGRATING' to rbd.pyx

Fixes: http://tracker.ceph.com/issues/39609
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/orchestrator: add progress events to all orchestrators

Now, progress events are part of `WriteCompletion` istead of part of the orchestrator module.

It does not yet provide a way to just show orchestrator events.

Also fixes issue in the SSH orchestrator

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #27873 from rzarzynski/wip-crimson-writes

crimson: add support for basic write path

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/thread: include reactor.hh to avoid linking error.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: add support for basic write path, part 2.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: make ObjectState mutable. Evict mutated state on error.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os: implement bits needed for basic write path.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #27413 from liu-chunmei/ceph_seastar_io_read

crimson/osd: support write pid_file when osd start

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: fix help message for minimum blob size in pool form

A value less than maximum blob size is required.

Fixes: http://tracker.ceph.com/issues/39624
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #27665 from badone/wip-improve-dump_pgstate_history-json

osd: Improve dump_pgstate_history json output

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #28015 from tchaikov/wip-install-deps.sh

install-deps.sh: install libboost-test for seastar

Reviewed-by: Samuel Just <sjust@redhat.com>

os/bluestore: fix missing discard in BlueStore::_kv_sync_thread

Discard comand should be issued before space released.

Signed-off-by: Junhui Tang <tangjunhui@sangfor.com.cn>
Fixes: https://tracker.ceph.com/issues/39621

install-deps.sh: install libboost-test for seastar

see
https://github.com/scylladb/seastar/blob/5858fe30c16a1a059200c129d20180ceb0199d99/cmake/SeastarDependencies.cmake#L58-L68

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: support write pid_file when osd start

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>

Merge pull request #27979 from yuriw/wip-yuriw-crontab-master

qa/test: reduce over all number of runs

Merge pull request #27941 from dzafman/wip-39011

doc: Describe recovery and backfill prioritizations

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

rgw: use explicit to_string() overload for boost::string_ref

Fixes: http://tracker.ceph.com/issues/39611
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27981 from dillaman/wip-39598

qa/workunits/rbd: wait for rbd-nbd unmap to complete

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #27911 into master

* refs/pull/27911/head:
os/bluestore: dump onode meta before "no spanning blob" assertion.
os/bluestore: move _dump_xxx methods out of BlueStore class

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27929 into master

* refs/pull/27929/head:
os/bluestore: be verbose about objects that existing on rmcoll
osd/PrimaryLogPG: disallow ops on objects with an empty name
osd/PG: fix cleanup of pgmeta-like objects on PG deletion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26488 from mogeb/per-pool-scrub-status

mon: show no[deep-]scrub flags per pool in the status

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

qa/test: reduce over all number of runs

We kill thousands of queued jobs every week, so why do we even schedule them ?
Another point was that we run numerous of tests as part of PRs testing on released versions anyway, so it's duplicating effort

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

mgr/dashboard: All RBD features are supported by 'tcmu:runner' (#28000)

mgr/dashboard: All RBD features are supported by 'tcmu:runner'

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #27830 from dzafman/wip-38124

osd: Don't include user changeable flag in snaptrim related assert

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27841 from richael02/multiNVMESSD

vstart.sh: enable creating multiple OSDs backed by spdk backend

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28008 from tchaikov/wip-seastar

seastar: pick up changes for better performance

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: do not build unit test of seastar

Signed-off-by: Kefu Chai <kchai@redhat.com>

seastar: pick up changes for better performance

to be specific, a78fb44c96e2912c6f39b2151f94a0bb2b5796a6 helps to
improve the performance of future implementation -- with this change
future can always reference its local state without checking its `_promise`
and dereferencing it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard/qa: Fix a few linting issues in dashboard backend tests (#27865)

mgr/dashboard/qa: Fix a few linting issues in dashboard backend tests

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #27947 from smithfarm/wip-39579

pybind/tox: pass additional command line arguments through to tox

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: All RBD features are supported by 'tcmu:runner'

Fixes: https://tracker.ceph.com/issues/39607
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #27969 from yuvalif/wip-yuval-add-tenant-to-user

rgw: add tenant as parameter to User in multisite tests

Merge pull request #27987 from tchaikov/wip-crimson-osd-shutdown

crimson/osd: shutdown services in the right order

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: object state can be manually evicted from OSD's cache.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: retrieve object state once per MOSDOp.

Justification behind the change is behaviour of classical OSD.
It calls PrimaryLogPG::find_object_context() far before going
through OSDOps in ::do_osd_ops().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: add support for basic write path, part 1.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os: add inclusion guard in Transaction.h

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: PG caches ObjectState instead of object_info_t.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: PG::do_osd_op() takes OSDOp by reference, not pointer.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: PG::do_osd_op throws about unsupported op.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: handle hot CaynStore exceptions without throwing.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #27931 from yuvalif/pubsub_info_update_fix

rgw/pubsub: fix doc on updates. fix multi-notifications

rgw: add tenant as parameter to User in multisite tests

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>

mgr/dashboard: Fix iSCSI form when using IPv6

Fixes: https://tracker.ceph.com/issues/39578
Signed-off-by: Ricardo Marques <rimarques@suse.com>

crimson/osd: shutdown services in the right order

we should stop config service *after* osd is stopped, as osd depends on
a working and alive config subsystem when stopping itself. for instance,
the destructor of AuthRegistry unregisters itself from the ObserverMgr,
which is in turn a member variable of ConfigProxy, so if ConfigProxy is
destroyed before we destroy mon::Client, we will have a segfault with
following backtrace

ObserverMgr<ceph::md_config_obs_impl<ceph::common::ConfigProxy>
>::remove_observer(ceph::md_config_obs_impl<ceph::common::ConfigProxy>*)
at /var/ssd/ceph/build/../src/common/config_obs_mgr.h:78
AuthRegistry::~AuthRegistry() at
/var/ssd/ceph/build/../src/crimson/common/config_proxy.h:101
 (inlined by) AuthRegistry::~AuthRegistry() at
/var/ssd/ceph/build/../src/auth/AuthRegistry.cc:28
ceph::mon::Client::~Client() at
/var/ssd/ceph/build/../src/crimson/mon/MonClient.h:44
ceph::mon::Client::~Client() at
/var/ssd/ceph/build/../src/crimson/mon/MonClient.h:44
OSD::~OSD() at /usr/include/c++/9/bits/unique_ptr.h:81

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: Improve dump_pgstate_history json output

Fixes: http://tracker.ceph.com/issues/38846
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #27975 from tchaikov/wip-pidfile_write

global/pidfile: pass string_view instead of ConfigProxy to pidfile_wr…

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #27932 into master

* refs/pull/27932/head:
osd: clean up osdmap sharing
osd: minor cleanup in dequeue_op

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

vstart.sh: enable creating multiple OSDs backed by spdk backend

Currently vstart.sh only support deploying one OSD based on NVMe SSD.
The following two cases will cause errors:
1.There are 2 more NVMe SSDs from the same vendor on the machine
2.Trying to deploy 2 more OSDs if we only get 1 pci_id available

Add the support for allowing deploying multiple OSDs on a machine with
multiple NVME SSDs.

Change-Id: I6016435c1438bb4d16aff31f4575e03ccd3c9b3d

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>

Merge pull request #26152 from noonedeadpunk/mgr/zabbix_discovery

mgr/zabbix Added pools discovery and per-pool statistics

Reviewed-by: Wido den Hollander <wido@42on.com>

rgw/pubsub: revert the RGWSysObjectCtx change

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>

qa/workunits/rbd: wait for rbd-nbd unmap to complete

The "unmap" request is asynchronous, so wait for a short amount
of time for the "rbd-nbd" daemon process to exit.

Fixes: http://tracker.ceph.com/issues/39598
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Apply suggestions from code review

Signed-off-by: Dmitriy Rabotjagov <noonedeadpunk@ya.ru>

Merge pull request #27967 from liewegas/wip-fix-rgw-bias

rgw/rgw_tools: fix 'osd pool set' json syntax

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Allow the decrease of pg's of an existing pool (#27785)

mgr/dashboard: Allow the decrease of pg's of an existing pool

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #27977 from cbodley/wip-39597

doc/rgw: document CreateBucketConfiguration for s3 PUT Bucket api

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #27968 from iotcg/master

vstart.sh: correct ceph-run path

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/rgw: document CreateBucketConfiguration for s3 PUT Bucket api

Fixes: http://tracker.ceph.com/issues/39597
Signed-off-by: Casey Bodley <cbodley@redhat.com>

global/pidfile: pass string_view instead of ConfigProxy to pidfile_write()

there is no need to pass ConfigProxy to this function. and passing a
string_view also make it easier to reuse this function.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27970 from Liuchang0812/duplicate-tags-bl

rgw: eliminates duplicated tags_bl var

Reviewed-by: Casey Bodley <cbodley@redhat.com>

pybind/tox: pass additional command line arguments through to tox

Fixes: 9426f1f2045d0ae0f319530c3dc3a9240d838d07
Fixes: https://tracker.ceph.com/issues/39579
Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: fix small typo in dashboard documentation (#27850)

doc: fix small typo in dashboard documentation

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

vstart.sh: correct ceph-run path

ceph-run is in the same directory as vstart.sh. It's often that
vstart.sh is run under build directory. Without giving the right
directory, ceph-run file can't be found.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>

rgw: eliminates duplicated tags_bl var

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

Merge pull request #27538 from iotcg/radosgw

init-radosgw: use ceph-conf to get cluster configuration value

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: be verbose about objects that existing on rmcoll

This is always a bug (OSD doesn't try to remove a collection unless it
thinks it is empty), and not seeing it at default debug levels makes it
hard to track down.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: disallow ops on objects with an empty name

These may conflict with pgmeta objects.  And are going to cause other
problems later (e.g., https://tracker.ceph.com/issues/38724).

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: fix cleanup of pgmeta-like objects on PG deletion

If an object has an empty 'name' field, it "looks" like a pgmeta object,
and the PG cleanup code was skipping it.  However, we were letting these
objects get created.

Fix by only skipping *our* pgmeta object.  If there are other pgmeta-like
objects in the PG collection, clean them up.

Fixes: https://tracker.ceph.com/issues/38724
Signed-off-by: Sage Weil <sage@redhat.com>

rgw/rgw_tools: fix 'osd pool set' json syntax

Fixes af30e3a512dd1e5c51639d1f823b99ed9b43fb50

Signed-off-by: Sage Weil <sage@redhat.com>

rgw/pubsub: cleanup tests for multiple notifications

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>