crimson/net: minor logging cleanup

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: throw bad_peer_address when reconnect address doesn't match

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: REPLACING state to resolve racing and retain session

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: handle fault for READY, CONNECTING and ACCEPTING

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: WAIT state and backoff for client

Client goes to WAIT state when it is delayed to reconnect, or wants to
be replaced by a newly established socket.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: SERVER_WAIT state for accepting server

Server wait for peer client close the socket at SERVER_WAIT.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: STANDBY state for lossless server or peer

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: allow REPLACING state wait for protocol exit

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: send AckFrame for lossless policy

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: maintain the sent queue for lossless policy

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: reset write state with reset_write()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: allow connecting state reentrant

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: reset handshake status when connecting/accepting

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: pending_q to store the pending(sending) messages

We cannot left the pending messages in the out_q, because with lossless
policy, they can be partially sent and even acknowledged.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: wait_write_exit() to wait for writer stopped

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: is_queued() to check if there's any pending writes

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: fix variables for stateful connection

server_cookie, client_cookie, connect_seq and global_seq are identifiers
of a stateful connection.

We already have some related implementations, but they are stub code
when implement lossy policy and cannot work properly.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: clean up, exsiting_conn and existing_proto

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: next_step_t for explicit decision of next state

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: introduce 3 ways to abort the active protocol state

* abort_in_fault(): a fault is happening and needs to be handled.
* abort_protocol(): abort the current protocol state due to preemptive
                    state change.
* abort_in_close(): close this connection and abort the current protocol
                    state due to some fatal error.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: improve get_global_seq()

Implement single global_seq and non-racing get_global_seq() interface.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #29341 from zhangbingyi1/master

test/librados: avoid residual crush rule after test case execution

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/librados: avoid residual crush rule after test case execution
Fixes: http://tracker.ceph.com/issues/40970
Signed-off-by: Bingyi Zhang <zhangbingyi@umcloud.com>

Merge pull request #29348 from myoungwon/fix-wrong-class

osd: fix wrong arguments when dropping refcount

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29345 from majianpeng/bluestore-flush-waiter

os/bluestore: avoid unnecessary notify.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #29126 from xiexingguo/wip-clamp-ratio-to-0

mgr/progress: clamp pg recovery ratio to 0

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28012 from rouming/bootstrap-after-setuid

global/global_init: do first transport connection after setuid()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27985 from ifed01/wip-ifed-slow-omaps

tools/ceph-objectstore-tool: introduce list-slow-omap command.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29238 into master

* refs/pull/29238/head:
os/bluestore: it's better to erase spanning blob only once

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #29239 into master

* refs/pull/29239/head:
qa/tasks/ceph_manager.py: always use self.logger

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #29305 into master

* refs/pull/29305/head:
mgr: Reduce logging noise when handling commands

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29307 into master

* refs/pull/29307/head:
monmaptool: added --addv option to usage description

Reviewed-by: João Eduardo Luís <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29316 into master

* refs/pull/29316/head:
common,tools: make sure the destination buffer can handle the size of the string
src/tools: initialize variables before the goto statement
src/mount: check before dereference buf
src/crush: check before dereference out2
src/test: s/strcpy/strncpy

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29358 into master

* refs/pull/29358/head:
mgr/telemetry: salt osd ids too

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #29054 from dillaman/wip-40621

mgr/rbd_support: support scheduling long-running background operations

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #29361 from tchaikov/wip-qa-osd-pause

qa: fixes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #28685 into master

* refs/pull/28685/head:
client: more precise CEPH_CLIENT_CAPS_PENDING_CAPSNAP
client: unify kicking cap flushes and kicking snapcap flushes
client: define helper function that sends flushsnap message
client: cleanup tracking of early kicked flushing caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28793 into master

* refs/pull/28793/head:
qa/cephfs: make run_shell() accept args as str too
qa/vstart_runner.py: ignores when source and destination are same
test_cephfs_shell: add a test for cd with arguments
test_cephfs_shell: add a test for cd with no arguments
test_cephfs_shell: group test methods
test_cephfs_shell: add a new method to return script output
test_cephfs_shell: allow running CephFS shell script
cephfs-shell: cd with no args shouldn't print an error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29064 into master

* refs/pull/29064/head:
mds: execute PurgeQueue on_error handler in finisher

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29302 into master

* refs/pull/29302/head:
cephfs: fix a memory leak

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29066 into master

* refs/pull/29066/head:
mds: Move inline functions with loops or switch statements to CInode.cc
mds: Reorganize class members in CInode header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29336 into master

* refs/pull/29336/head:
qa: wait for MDS to come back after removing it

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #29359 from jmunhoz/wip-msite-az-tests

rgw: az: add archive zone tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard/qa: add more fields to report

to be in sync with #28847

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/rest/mgr-restful: whitelist OSDMAP_FLAGS

`rest/test-restful.sh` calls `test_mgr_rest_api.py`, which in turn
calls

```
    ('patch',  '/config/osd', {'pause': True}),
```

and rest module translates it to `ceph osd set key=pause`

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: az: add archive zone tests

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

mgr/telemetry: salt osd ids too

Better to fully obfuscate here.

This has a nice side-effect of assigning entity names that are *globally*
unique across the full telemetry data set, since the salts are unique and
sha1 is (sufficiently) collision-free.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #29330 into master

* refs/pull/29330/head:
mgr/telemetry: obscure entity_name with a salt

Reviewed-by: Dan Mick <dmick@redhat.com>

mgr: Reduce logging noise when handling commands

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #29237 from adk3798/usertest

mgr/dashboard: RGW user E2E Tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #29192 from ceph/dependabot/npm_and_yarn/src/pybind/mgr/dashboard/frontend/lodash-4.17.13

build(deps): bump lodash from 4.17.11 to 4.17.13 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Tiago Melo <tmelo@suse.com>

pybind/mgr: don't log exception when cannot find RBD task by id

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: fix wrong arguments when dropping refcount

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

Merge pull request #29198 from rhcs-dashboard/e2e-ci

mgr/dashboard: run e2e tests against prod build (jenkins job)

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29280 from rhcs-dashboard/bucket-tests

mgr/dashboard: RGW bucket creation when no placement target received

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

os/bluestore: avoid unnecessary notify.

Add parameter count waiting list for Onode::flush to avoid unnecessary
notify.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common,tools: make sure the destination buffer can handle the size of the string

fix klocwork issues:
Width is not specified for 's' conversion specifier.
This can result in an overflow of the buffer
provided in argument 3 of a call to 'sscanf'

Signed-off-by: songweibin <song.weibin@zte.com.cn>

src/tools: initialize variables before the goto statement

CID 172143 (#15 of 15): Branch past initialization (PW.BRANCH_PAST_INITIALIZATION)
1. branch_past_initialization:
2. name_at_decl_position: variable "oid" (declared at line 220)
3. name_at_decl_position: variable "exkeys" (declared at line 241)
4. name_at_decl_position: variable "exvals" (declared at line 242)
5. name_at_decl_position: variable "safe" (declared at line 278)

Signed-off-by: songweibin <song.weibin@zte.com.cn>

src/mount: check before dereference buf

CID 180671 (#1 of 1): Dereference null return value (NULL_RETURNS)

Signed-off-by: songweibin <song.weibin@zte.com.cn>

src/crush: check before dereference out2

CID 174874 (#2 of 2): Dereference after null check (FORWARD_NULL)
30. var_deref_op: Dereference null pointer out2.

Signed-off-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #29264 from rjfd/wip-dashboard-fix-qa-rbd

mgr/dashboard: qa: fix RBD test when matching error strings

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28597 from markhpc/wip-bs-ind-cache

os/bluestore: Refactor Bluestore Caches

mds: execute PurgeQueue on_error handler in finisher

Fix: https://tracker.ceph.com/issues/40695
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge PR #28847 into master

* refs/pull/28847/head:
doc/mgr/telemetry: update
mgr/telemetry: move contact info to an 'ident' channel
mgr/telemetry: accept channel list to 'telemetry show'
mgr/telemetry: always generate new report for 'telemetry show'
mgr/telemetry: add 'device' channel and call out to devicehealth module
mgr/telemetry: add telemetry channel 'device'
mgr/telemetry: add separate channels

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge PR #29130 into master

* refs/pull/29130/head:
os/bluestore: Reduce unnecessary wakeups.
os/bluestore: Reduce unnecessary wakeups.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #29221 into master

* refs/pull/29221/head:
mon/OSDMonitor: do not clobber pending xinfo during boot
osd/OSDMap: add is_dead() helper
mon/OSDMonitor: 'osd down ... --definitely-dead'
osd: send MOSDMarkMeDead when we are marked down
mon/OSDMonitor: implement MOSDMarkMeDead handling
mon/OSDMonitor: set dead_epoch for immediate failure (ECONNREFUSED)
osd/OSDMap: add dead_epoch to xinfo

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #29323 into master

* refs/pull/29323/head:
rocksdb: enable rocksdb_rmrange=true by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #29300 into master

* refs/pull/29300/head:
doc: fix youtube channel url
doc: fix urls in posix.rst

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28874 into master

* refs/pull/28874/head:
ceph-mds: dump all info of ceph_file_layout, InodeStoreBase, frag_info_t, inode_t

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29103 into master

* refs/pull/29103/head:
mgr/volumes: add `ceph fs subvolumegroup getpath` command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29166 into master

* refs/pull/29166/head:
mds: Reorganize class members in Capability header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #29149 into master

* refs/pull/29149/head:
cephfs-shell: Fix flake8 E302 blank line error
cephfs-shell: Fix flake8 indentation error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28860 into master

* refs/pull/28860/head:
mds: Move scalar type initializations of CDir data members to its header
mds: Reorganize class members in CDir header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

qa: wait for MDS to come back after removing it

Fixes: http://tracker.ceph.com/issues/40967
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mon/OSDMonitor: do not clobber pending xinfo during boot

If we have a pending xinfo change and also process a boot message, we
should not throw out the pending xinfo change.

This triggers when you mark an osd down and it sends both a MOSDMarkMeDead
and a new MOSDBoot message in quick succession: the first message sets
dead_epoch but the boot message processing clobbers it.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: add is_dead() helper

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: 'osd down ... --definitely-dead'

Like 'osd down', but we also set dead_epoch.  Should only be used if we
know the process/host is already completely dead and unresponsive.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #29036 from mikechristie/rbd-nbd-add-nl-resize

rbd-nbd: add nl resize

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #28955 into master

* refs/pull/28955/head:
doc: Update mount CephFS index

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29111 into master

* refs/pull/29111/head:
qa/mgr/progress: Update the test suite for progress module

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: send MOSDMarkMeDead when we are marked down

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: implement MOSDMarkMeDead handling

Allow updates to the dead_epoch in osd_xinfo, so that the OSD can inform
us that they know they are down (as of a particular epoch).

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/telemetry: obscure entity_name with a salt

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr: handle duplicate rbd task commands

The 'ceph' CLI will duplicate commands within teuthology to test
the MONs idempotency. This shouldn't be required for the MGR module,
but we can keep a fixed set of completed tests to handle this
possible command replay.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mds: Move inline functions with loops or switch statements to CInode.cc

Signed-off-by: Varsha Rao <varao@redhat.com>

mds: Reorganize class members in CInode header

Fixes: https://tracker.ceph.com/issues/40787
Signed-off-by: Varsha Rao <varao@redhat.com>

doc/mgr/telemetry: update

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/telemetry: move contact info to an 'ident' channel

Signed-off-by: Sage Weil <sage@redhat.com>

rocksdb: enable rocksdb_rmrange=true by default

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #29117 from tchaikov/wip-mgr-set_health_checks-unicode

mgr: check for unicode passed to "set_health_checks()"

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #29320 into master

* refs/pull/29320/head:
os/bluestore: remove unused arg to _get_deferred_op()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #29282 from yuriw/wip-yuriw-upgrade-rwg-splits-master

qa/tests: added rgw into upgrade sequence to improve coverage - splits

os/bluestore: remove unused arg to _get_deferred_op()

Fixes: http://tracker.ceph.com/issues/40918
Signed-off-by: Sage Weil <sage@redhat.com>

qa: test case for new rbd background tasks

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr: rbd tasks now provide a dict to the progress events

This dict will include an origin key fixed to 'rbd_support' as well
as pool and image references.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr: mark progress events as failed if an error occurs

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr: support marking progress events as failed

The failed events can also include a failure message to indicate
the reason for the failure.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr: new 'rbd task' background task management

Fixes: http://tracker.ceph.com/issues/40621
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/rbd: new OperationCanceled exception

This allows the error to be directly caught instead of attempting
to parse the OSError.errno

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

monmaptool: added --addv option to usage description

Signed-off-by: Ricardo Dias <rdias@suse.com>