client: Fix setxattr with 0 size specified

When xattrs are set with 0 size, the value should be set to an empty string.
Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>

Merge pull request #35668 from bk201/wip-46105

mgr/dashboard: fix browserTarget for zh-Hans and zh-Hant

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35653 from ideepika/fix-cpatch

src/cstart: fix typo in cstart script

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #35676 from ishanrai05/autocomplete-background

mgr/dashboard: fix autocomplete input backgrounds in chrome and firefox

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge PR #35627 into master

* refs/pull/35627/head:
qa: fix type error in stderr processing

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #35635 from zdover23/wip-doc-man-deploy-mon-bootstrapping-2020-jun

doc/install: add sudo to command

Merge pull request #35595 from jan--f/c-v-fix-lvm-functests

ceph-volume: fix lvm functional tests

Merge pull request #35027 from yuvalif/add_doxygen_target

cmake: add doxygen target for c++ code under src

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: fix autocomplete input backgrounds in chrome and firefox

Fixes: https://tracker.ceph.com/issues/46109
Signed-off-by: Ishan Rai <ishanrai05@gmail.com>

Merge pull request #35688 from tchaikov/wip-doc-sphinx

admin/doc-requirements: pin sphinx-autodoc-typehints to 1.10.3

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35698 from tchaikov/wip-man-print-file-which-is-non-ascii

man/conf.py: print out path to file which contains undecodable chars

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35701 from tchaikov/wip-doc-sans-utf8

doc/man/8/ceph-objectstore-tool.rst: drop UTF-8 chars

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/man/8/ceph-objectstore-tool.rst: drop UTF-8 chars

it's a regression introduced by 5a09d7a1ca2e93811505345c31367627bedbea0e

Signed-off-by: Kefu Chai <kchai@redhat.com>

man/conf.py: print out path to file which contains undecodable chars

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35640 from sebastian-philipp/doc-cephadm-adopt-placment

doc/cephadm: adopt: fix misleading `apply` examples

Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35587 from sebastian-philipp/doc-orch-stateless-cli

doc/mgr/orch: Update documentation to the new CLI

Reviewed-by: Zac Dover <zac.dover@gmail.com>

ceph-volume: fix lvm functional tests

This is related to https://github.com/ceph/ceph-ansible/pull/5413 and
adjusts the tests to work with the related ansible fix.

Fixes: https://tracker.ceph.com/issues/46131
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #35675 from smithfarm/wip-doc-backport-labelling

doc/SubmittingPatches-backports.rst: clarify backport PR labels

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/install: add sudo to command

This commit adds "sudo" to a ceph-authtool command
that is missing the "sudo".

Fixes: https://tracker.ceph.com/issues/46065
Signed-off-by: Zac Dover <zac.dover@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35691 from bk201/wip-46130

doc/man: fix ceph-objectstore-tool description

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35682 from tchaikov/wip-cmake-cxx

cmake: only apply C++ compile options to C++ language

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/man: fix ceph-objectstore-tool description

There should be a `--` between the name and the description.

Fixes: https://tracker.ceph.com/issues/46130
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

doc/conf.py: s/add_javascript/add_js_file/

to address following warning:

jenkins-build/build/workspace/ceph-pr-docs/doc/conf.py:102: RemovedInSphinx40Warning: The app.add_javascript() is deprecated. Please use app.add_js_file() instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>

admin/doc-requirements: pin sphinx-autodoc-typehints to 1.10.3

to silence following error:

ERROR: sphinx-autodoc-typehints 1.11.0 has requirement Sphinx>=3.0, but you'll have sphinx 2.4.3 which is incompatible.

* update breathe to the latest release, 4.14.2, which supports sphinx <
  3.0
* pin sphinx to 2.4.4 the last sphinx before 3.0.0. as if sphinx >= 3.0
is used, we have errors like:

Invalid C declaration: Expected identifier in nested name, got keyword: int [error at 18]
  CEPH_RADOS_API int rados_aio_append (rados_ioctx_t io, const char *oid, rados_completion_t completion, const char *buf, size_t len)
  ------------------^

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35658 from dillaman/wip-46083

librbd: race when disabling object map with overlapping in-flight writes

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #35588 from jschmid1/osdspec_affinity_to_metadata

osd: expose osdspec_affinity to osd_metadata

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35569 from sungjunyoung/master

install_deps.sh: Add debian based elementary os

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35294 from agayev/punch-hole

os/bluestore: Add missing punch_hole call so that object offsets are …

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #34189 from changchengx/bufferlist

common/buffer: refine bufferlist implementation

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35592 from tchaikov/wip-ceph-in-de2

ceph.in: fallback to install path of "ceph" when "ceph-conf" is not found

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35078 from idryomov/wip-msgr21

msg/async: New msgr2 crc and secure modes (msgr2.1)

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35607 from wuxingyi/schedulerlogger

osd/scheduler: add white space for better readability

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35584 from aclamk/wip-mempool-bufferlist-c_str

common/mempool: copies bufferlist assignment to mempool through rebuild

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #32271 from toliu/disable-restful-basic-auth

mgr/restful: allow users to disable http authentication

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: race when disabling object map with overlapping in-flight writes

The block guard that protects against overlapping updates to the object
map needs to be flushed prior to closing the object map instance.

Fixes: https://tracker.ceph.com/issues/46083
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #33823 from zdover23/wip-doc-ceph-objectstore-tool-2020-mar

doc/man: add manpage for ceph-objectstore-tool

cmake: only apply C++ compile options to C++ language

we have sources written in C language, and they are compiled along with
C++ source files which uses Seastar library. so when the C compiler sees
the options exposed by seastar, it complains like

1: warning: command line option '-fconcepts' is valid for C++/ObjC++ but
not for C

with this change, only C++ options is added for C++ language.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35681 from yuriw/wip-yuriw-crontab-master

qa/tests: removed mimic runs as EOL

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tests: removed mimic runs as EOL

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #35673 from tspmelo/wip-chromium

mgr/dashboard: Fix e2e chromium binary validation

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: Fix e2e chrome and chromium binaries verification

Fixes: https://tracker.ceph.com/issues/46110
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35223 from smithfarm/wip-releases-ordering

doc/releases: list archived releases in correct order

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

doc/SubmittingPatches-backports.rst: clarify backport PR labels

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35586 from tspmelo/wip-service-root

mgr/dashboard: Provide all services in root

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #30670 from trociny/wip-41736

mgr: fix race between module load and notify

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #35141 from matthewoliver/cephadm_iscsi_lib_module_mount

cephadm: Give iscsci a RO /lib/modules bind mounted

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35190 from cybozu/ceph-volume-fix-rejected-reason-in-device-type-mismatch

ceph-volume: report correct rejected reason in inventory if device type is invalid

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

mgr/dashboard: fix browserTarget for zh-Hans and zh-Hant

Fixes: https://tracker.ceph.com/issues/46105
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

ceph-volume: show correct rejected reason in inventory if device type is not acceptable

If device type is not acceptable in `c-v inventory`, its rejected reason
becomes "Insufficient space (<5GB)" by mistake. It's because sys_api is
empty due to skipping devices that are neither `disk` nor `device`. We
should report the target device is not acceptable in this case.

Fixes: https://tracker.ceph.com/issues/46102
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>

ceph-volume: cleanup code

Simplify the logic and fix a typo.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>

incorporating dzafman's suggestions

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #35652 from rzarzynski/wip-crimson-thread-cleanup

crimson: clean-up the crimson/thread directory

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35632 from dzafman/wip-46064

tools: Add statfs operation to ceph-objecstore-tool

Reviewed-by: Neha Ojha <nojha@redhat.com>

tools: Add statfs operation to ceph-objecstore-tool

Fixes: https://tracker.ceph.com/issues/46064
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #35329 from trociny/wip-quiesce_complete

librbd: quiesce/unquiesce API improvements

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #35649 into master

* refs/pull/35649/head:
vstart: allow specifying a different ganesha binary
vstart: handle multiple ganesha's more sanely

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

vstart: allow specifying a different ganesha binary

Signed-off-by: Jeff Layton <jlayton@redhat.com>

vstart: handle multiple ganesha's more sanely

Currently, if you have more than one ganesha daemon, it clobbers the
log, config and pidfiles from the previous daemons. Give them each
their own files.

While we're in here, move the log and pidfile to the out/ directory
with all of the other logfiles/pidfiles.

Fixes: https://tracker.ceph.com/issues/46079
Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #35650 from trociny/wip-46076

test/rbd_mirror: fix race on test shut down

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35631 from tchaikov/wip-cmake-cleanup

cmake: use add_compile_options() and add_definitions()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

src/cstart: fix typo in cstart script

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #35630 from tchaikov/wip-systemd-subvars

cmake: move include(GNUInstallDirs) up

Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson: move Throttle to src/crimson/common.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: make ThreadPool specific to AlienStore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

qa: fix type error in stderr processing

Fixes: 422d736568bd4d92fdf380b07da0ccac46cf3efd
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

test/rbd_mirror: fix race on test shut down

in TestMockImageReplayerSnapshotReplayer.UnlinkRemoteSnapshot

Fixes: https://tracker.ceph.com/issues/46076
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #35637 from bk201/wip-46067

stop.sh: do not block script when there is no running cluster

Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #35603 from dillaman/wip-46034

mgr/prometheus: automatically discover RBD pools for stats gathering

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: Provide all services in root

Fixes: https://tracker.ceph.com/issues/46030
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35574 from tspmelo/wip-tslint

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

doc/cephadm: adopt: fix misleading `apply` examples

Fixes: https://tracker.ceph.com/issues/46052
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35634 from mgfritch/cephadm-manpage-shell-mount

doc/man: add missing cephadm shell `--mount` option

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/mgr/orch: Update documentation to the new CLI

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

cmake: add definitions using add_definitions()

also, add definitions as a list instead of a single string for better
readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: use add_compile_options()

instead of appending compile flags to CMAKE_C_FLAGS, use
add_compile_options(), as COMPILE_OPTIONS is a list, it'd simpler to
append options to it and to access it in a structured way.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: use string(APPEND...) for appending cflags

* use string(APPEND...) for less repeatings
* `-rdynamic` is a linker option, so it should be added to
  `CMAKE_EXE_LINKER_FLAGS` for adding symbols to dynamic
  symbol tables so they can be consumed by plugins.

Signed-off-by: Kefu Chai <kchai@redhat.com>

stop.sh: do not block script when there is no running cluster

A query for current fsid is called inside `do_killcephadm`. This blocks
the script when there is no running cluster. The fix avoids entering the
function if cephadm command fails or returns no daemons.

The change also hides the following output for non-cephadm environments:

```
Unable to locate any of ['podman', 'docker']
```

Fixes: https://tracker.ceph.com/issues/46067
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #35469 from xxhdx1985126/wip-crimson-block-io-for-recovery

crimson: block io for recovery ---- recovery part 2

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #35626 into master

* refs/pull/35626/head:
mds: fix whitespace

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc/man: add missing cephadm shell `--mount` option

also clean-up `--mount` help usage

Signed-off-by: Michael Fritch <mfritch@suse.com>

crimson/osd: share map with clients when osdmap epoch doesn't match

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson: do not assert in start_backfill_ops

osd_recovery_max_single_start is not necessarily used up in start_recovery_ops, and
start_backfill_ops could be invoked even there's no need to backfill, so do not assert
here, just return.

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson/osd: fix client op version issue

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson/monc: take care of connection lost when sending messages

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson/osd: send "mark me down" messages to monitor when shutting down

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson: restart ongoing client_request after peering

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson: block io for object recovery

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

Merge pull request #35507 from aclamk/wip-bluestore-avl-description

BlueStore, added descriptions for AVL allocator parameters

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

cmake: move include(GNUInstallDirs) up

so systemd/CMakeLists.txt can have access to the variables defined by
it. quote from https://cmake.org/cmake/help/latest/command/include.html.

> Variable reads and writes access the scope of the caller (dynamic
> scoping).

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35623 from liewegas/wip-bloomberg

doc/foundation: add bloomberg

doc/dev/msgr2: fix inconsistencies and update for msgr2.1

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

msg/async/ProtocolV2: add msgr2.1 feature bit

Use msgr2.1 if the peer supports it and fall back to msgr2.0
otherwise.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

msg/async/ProtocolV2: store supported features instead of required

We aren't interested in peer_required_features anywhere outside
_handle_peer_banner_payload() -- once we know there is no mismatch,
it's all about peer_supported_features.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

msg/async/ProtocolV2: short circuit empty segments and epilogue

In both msgr2.0 and msgr2.1, segments can be empty.  In msgr2.1,
epilogue can be empty as well.

Handle both by calling the respective handler function directly
instead of allocating a buffer::ptr_node for an empty buffer and
passing that through READ[_RXBUF].

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

msg/async/frames_v2: add initial unit tests

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

msg/async/crypto_onwire: implement msgr2.1 nonce format

Move to a 64-bit counter to avoid wrapping and having to reset
the session before the counter repeats.  This is in line with NIST
Recommendation for GCM [1]:

  "... this Recommendation suggests, but does not require, that
  the leading (i.e., leftmost) 32 bits of the IV hold the fixed
  field; and that the trailing (i.e., rightmost) 64 bits hold the
  invocation field."

See commit bb61e6a5adc3 ("msg/async/ProtocolV2: avoid AES-GCM nonce
reuse vulnerabilities").

[1] https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-38d.pdf

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

msg/async/frames_v2: implement msgr2.1 wire format

Implement msgr2.1-crc and msgr2.1-secure modes.

Issues with existing msgr2.0-crc and msgr2.0-secure modes and
their resolution will be described in doc/dev/msgr2.rst.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mds: fix whitespace

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc/foundation: add bloomberg

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/dashboard: Fix tslint warnings

Signed-off-by: Tiago Melo <tmelo@suse.com>