rgw: Add missing extension to file in sources list

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #38737 from IlsooByun/objmeta-key-underscore

rgw: keep underscore in metatdata key

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #39749 from linuxbox2/wip-olock-fixes

rgw: objectlock: improve client error messages

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #40263 from ivancich/wip-update-listing-comments

rgw: update bucket listing log output

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #40570 from zdover23/wip-doc-mclock-rewrite-first-half-2021-Apr-2

doc/rados: rewrite mclock-config-ref

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #40639 from yuvalif/wip-yuval-fix-50201

rgw/multisite: return correct error code when op fails

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #40581 from weixinwei/master

os/bluestore/BlueFS: do not _flush_range deleted files

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #40601 from cbodley/wip-50147

qa/rgw: move ignore-pg-availability.yaml out of suites/rgw

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #39907 from guits/fix_get_first_lv

ceph-volume: `get_first_lv()` refactor

Merge pull request #40624 from rhcs-dashboard/read-only-config-access-disable

mgr/dashboard: Revoke read-only user's access to Manager modules

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

rgw/multisite: return correct error code when op fails

when trying to disable/enbale sync on non-master zone

Fixes: https://tracker.ceph.com/issues/50201
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

ceph-volume: `get_first_*()` refactor

As indicated by commit 17957d9beb42a04b8f180ccb7ba07d43179a41d3 those
fuctions were meant to avoid writing something like following:

```
lvs = get_lvs()
if len(lvs) >= 1:
lvs = lv[0]
```

Those functions should return `None` if 0 or more than 1 item is returned.
The current name of these functions are confusing and can lead to thinking that
we just want the first item returned, even though it returns more than 1
item, let's rename them to `get_single_pv()`, `get_single_vg()` and
`get_single_lv()`

Closes: https://tracker.ceph.com/issues/49643
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #40637 from ronen-fr/wip-ronenf-rgw-logbacking

rgw: fix lambda capture of a non-variable
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #40604 from liewegas/podman-registries-conf

qa/distros/podman: preserve registries.conf

Reviewed-by: David Galloway <dgallowa@redhat.com>

Merge pull request #40487 from tchaikov/wip-boost-1.75

install-deps.sh: install boost 1.75

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40583 from a16bitsysop/include

tools/cephfs_mirror/PeerReplayer.cc: add missing include

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40607 from tchaikov/wip-journald-if-msghdr

common: disable journald logging backend if struct msghdr is not found

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #40609 from tchaikov/wip-common-buffer-cleanup

common/buffer.cc: use shift_round_up() when appropriate

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: fix lambda capture of a non-variable

One cannot just capture a structured binding "non-variable".

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

os/bluestore/BlueFS: do not _flush_range deleted files

Fixes: https://tracker.ceph.com/issues/49861
Signed-off-by: weixinwei <weixw3@lenovo.com>

Merge PR #40603 into master

* refs/pull/40603/head:
qa/tasks/ceph.conf: shorten cephx TTL for testing

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40190 from cbodley/wip-qa-rgw-sigv4-warnings

rgw: silence some unused variable warnings

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #40602 into master

* refs/pull/40602/head:
qa/tasks/cephadm: add apply() method/task

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #40597 into master

* refs/pull/40597/head:
cephadm: pass '-i' to docker|podman run for shell|enter

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #40553 from dang/wip-dang-zipper-list

RGW Zipper - Make sure bucket list progresses

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge PR #39939 into master

* refs/pull/39939/head:
cephfs: ceph-dokan - properly log the mounted root
cephfs: Update ceph-dokan "--removable" flag
cephfs: document using multiple fs on Windows
cephfs: provide additional volume details on Windows
cephfs: add ceph-dokan unmap command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40418 into master

* refs/pull/40418/head:
test: unmount when finished ino_release_cb
test: wait a time for inode release
qa: move ino_release_cb to libcephfs sub-suite
qa: simplify recall triggers for bug
qa: fix name for qa task referencing tracker issue

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #40460 into master

* refs/pull/40460/head:
client: only check pool permissions for regular files

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40465 into master

* refs/pull/40465/head:
test: bump up retries for `test_mirroring_init_failure_with_recovery` test
test: fix typo
test: disable mgr/mirroring for `test_mirroring_init_failure_with_recovery` test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40468 into master

* refs/pull/40468/head:
mds/metrics: add one whitespace between metric type the metainfo

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #40469 into master

* refs/pull/40469/head:
qa: check mounts attribute in ctx

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40501 into master

* refs/pull/40501/head:
client: fix the opened inodes counter increasing

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/rados: rewrite mclock-config-ref

This PR rewrites the material provided in
PR#40531, improving its elegance and
readability.

This PR rewrites the first 48 percent of the
material provided in PR#40531.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #40113 from yuvalif/wip-yuval-fix-49799

rgw/notification: set correct type to "post" and "copy" notifications

mgr/dashboard: Revoke read-only user's access to Manager modules

This will disable read only user to read/open Manager Modules page in
Ceph Dashboard where some of the security related informations are
shown.

Fixes: https://tracker.ceph.com/issues/50174
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge PR #40620 into master

* refs/pull/40620/head:
cephfs-top: fix typo in help

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40613 into master

* refs/pull/40613/head:
doc/cephfs/nfs: add user id, fs name and key to FSAL block

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40616 from rhcs-dashboard/fix-50044-master

mgr/dashboard: debug nodeenv hangs

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40615 from tchaikov/wip-doc-header-css

doc/_themes: fix the styling of section header

Reviewed-by: Sebastian Wagner <swagner@suse.com>

doc/_themes: fix the styling of section header and list items

in the latest document generated from RtD, the section headers are now
in <section> tags instead of <div class="section">, so update the css
accordingly.

also tweak the style of the list items in unordered list to be the same
as it was.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40587 from tchaikov/wip-addr-parse-cleanup

src: use entity_addr_t::parse(string_view) when possible

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: debug nodeenv hangs

Increase verbosity in nodeenv command for debugging purposes.

Fixes: https://tracker.ceph.com/issues/50044
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

doc/cephfs/nfs: add user id, fs name and key to FSAL block

Fixes: https://tracker.ceph.com/issues/50161
Signed-off-by: Varsha Rao <varao@redhat.com>

common/buffer.cc: use shift_round_up() when appropriate

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw/notification: set correct type to "post" and "copy" notifications

Fixes: https://tracker.ceph.com/issues/49799
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

common: disable journald logging backend if struct msghdr is not found

* cmake/modules/CephChecks.cmake: detect the existence of struct msghdr,
  define HAVE_MSGHDR if it is found
* src/common/CMakeLists.txt: do not compile journald.cc if HAVE_MSGHDR
  is FALSE. as in that case, we cannot use sendmsg() to write to
  journald unix domain socket
* src/test/CMakeLists.txt, src/test/common/CMakeLists.txt: disable test
  exercising journald logging backend if HAVE_MSGHDR is not defined
* src/common/Journald.h: define a dummy JournaldLogger and a dummy
  JournaldClusterLogger when HAVE_MSGHDR is not defined, in order to
  minimize the change in Log.h and Log.cc, otherwise the source code of
  Log.h and Log.cc would be segmented into smaller chunks by
  `ifdef HAVE_MSGHDR` macros.
* src/include/config-h.in.cmake: define a new macro named
  HAVE_MSGHDR.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cephfs-top: fix typo in help

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #40582 from a16bitsysop/32bit

src/common/buffer.cc: change cast to static_cast

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40578 from tchaikov/wip-cmake-pmem

cmake: require libpmem 1.7 and cleanups

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #40572 from athanatos/sjust/wip-48613

osd/PeeringState: fix acting_set_writeable min_size check

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40475 from tchaikov/wip-qa-focal

qa/tests: replaced ubuntu_latest.yaml with ubuntu 20.04

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #40591 from tchaikov/wip-dencoder

tools/ceph-dencoder: link against libtcmalloc

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Sage Weil <sage@redhat.com>

osd/PeeringState: fix acting_set_writeable min_size check

acting.size() >= pool.info.min_size is meant to check min_size against
acting set participants, but acting is a vector with placeholders.
actingset is the representation with placeholders removed.

The upshot of this bug is that the activation process will basically
ignore min_size for an ec pool allowing writes in cases where it
shouldn't.  PastIntervals::check_new_interval, however, performs
the check correctly, and will therefore discount intervals in which
we really did serve writes as not writeable.  This can trigger many
different problem conditions including but not limited to:
  - Unfound objects due to accepting a last_update with insufficient
    osds
  - Lost writes
  - Crashes due to peering rules being violated

This bug was originally introduced with recovery below min_size in
e5a96fd, and then preserved through refactors in 749a13d and 95bec9.

7cb818a exposed it with with expansion of recovery below min_size
to include ec pools (acting.size() is sufficient for replicated
pools).

Fixes: https://tracker.ceph.com/issues/48613
Fixes: https://tracker.ceph.com/issues/48417
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge PR #40599 into master

* refs/pull/40599/head:
rpm: add missing % in %dir directive

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/distros/podman: preserve registries.conf

Otherwise we'll lose the mirror configuration.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #40598 from TRYTOBE8TME/wip-rgw-kafka-tests-fix

src/rgw: Issue #50138 fix

qa/tasks/ceph.conf: shorten cephx TTL for testing

Rotate tickets frequently to exercise those code paths during testing.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge PR #40025 into master

* refs/pull/40025/head:
cephadm: use split cgroup strategy for podman
cephadm: use class to represent container engine

Reviewed-by: Adam King <adking@redhat.com>

Merge PR #40459 into master

* refs/pull/40459/head:
mgr/cephadm: don't cleanup the daemon keyring on failed redeploy

Reviewed-by: Adam King <adking@redhat.com>

Merge PR #40508 into master

* refs/pull/40508/head:
mgr/cephadm: fix orch host add with multiple labels and no addr

Reviewed-by: Sage Weil <sage@redhat.com>

qa/rgw: move ignore-pg-availability.yaml out of suites/rgw

this was added in https://github.com/ceph/ceph/pull/39820 to
avoid repeating the override in rgw subsuites, but teuthology
was picking up suites/rgw/ignore-pg-availability.yaml as a job
of its own, and led to 'list index out of range' failures

Fixes: https://tracker.ceph.com/issues/50147
Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/tasks/cephadm: add apply() method/task

Signed-off-by: Sage Weil <sage@newdream.net>

rgw/test: stop kafka consumer if tests fails

This PR fixes https://tracker.ceph.com/issues/50138 by stopping the kafka consumer if the test fails.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>

rpm: add missing % in %dir directive

de6c8250a6d91403e6d334aeb901bf9720ba40eb added an explicit %dir directive for
a new directory added to the ceph-common package, but -- due to a typo --
neglected to include the "%". As a result, RPM builds started to fail with:

Processing files: ceph-common-17.0.0-2787.gde6c8250.el8.x86_64
error: File must begin with "/": {_libdir}/ceph/denc/

RPM build errors:
    File must begin with "/": {_libdir}/ceph/denc/

Fixes: de6c8250a6d91403e6d334aeb901bf9720ba40eb
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #40584 from smithfarm/wip-denc-fix

rpm: make ceph-common own the new denc directory

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40585 from yuvalif/wip-yuval-fix-50135

rgw/multisite: handle case when empty marker is provided

Merge pull request #39738 from SMIL-Infra/journald

common: enable log to journald

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

cephadm: pass '-i' to docker|podman run for shell|enter

This allows us to pipe things to stdin.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #40579 from tchaikov/wip-mgr-typing

pybind/mgr/mgr_util: fix typing annotation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #40500 into master

* refs/pull/40500/head:
doc/cephadm: remove keepalived_user from haproxy docs

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>

cmake: dedup components when finding pmem

otherwise we always find libpmem twice if libpmem is explicitly
specified when find_package()

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: find pmem once instead twice

less noise in cmake output

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/ceph-dencoder: print human-readable error message

if $CEPH_LIB is not specified.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/ceph-dencoder: link against libtcmalloc

to address the failure like

src/tcmalloc.cc:332] Attempt to free invalid pointer 0x5627da496130
Aborted

we need to link and load libtcmalloc before it is loaded by a shared
library, otherwise the memory chunk allocated by libc will be freed
by tcmalloc after it is loaded.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake,install-deps,make-dist: use boost 1.75

so we can use the latest boost

Signed-off-by: Kefu Chai <kchai@redhat.com>

src: use entity_addr_t::parse(string_view) when possible

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/msg_types: pass TYPE_DEFAULT as default_type to entity_addr_t::parse()

TYPE_NONE equals to 0, and we use it for the default value of entity's
messenger protocol version, if it is not specified. but we always use
TYPE_DEFAULT for the protocol version if it's not specified in the
parsed string or the parameter.

so we can just simplify the logic by assigning it to `newtype`, and
assign the parse value to `new_type` for better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/msg_types: add a default_type param to entity_addr_t::parse()

so it can be used in more places.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw/multisite: handle case when empty marker is provided

marker is potional, however, it may also be provided empty

Fixes: https://tracker.ceph.com/issues/50135
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

qa/tests: replaced ubuntu_latest.yaml with ubuntu 20.04

resurrect 835f2f5f511e7363f4056e5137382982aac1bfbf, which was reverted by
1f885b6c2fdde1f3ab249a56f7b4b763556f4373

so all tests for `pacific` will be tested on the latest ubuntu version

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/cephfs_mirror/PeerReplayer.cc: add missing include

alpine linux needs this include for compilation

Fixes: https://tracker.ceph.com/issues/50134
Signed-off-by: Duncan Bellamy <dunk@denkimushi.com>

rpm: make ceph-common own the new denc directory

2d3c6561b4ac1473a728e81c232d7dfe6fc0188c introduced a new library directory
"%{_libdir}/ceph/denc/" in ceph-common but did not explicitly state that it
should be owned by the package. This caused OBS builds to fail as follows:

[ 5515s] ceph-common-17.0.0-2786.1.x86_64.rpm: directories not owned by a package:
[ 5515s]  - /usr/lib64/ceph/denc

Fixes: 2d3c6561b4ac1473a728e81c232d7dfe6fc0188c
Signed-off-by: Nathan Cutler <ncutler@suse.com>

src/common/buffer.cc: change cast to static_cast in ceph::buffer::list::page_aligned_appender::_refill

part of fixing compilation on 32bit platforms

Signed-off-by: Duncan Bellamy <dunk@denkimushi.com>

pybind/mgr/mgr_util: fix typing annotation

and refactor lock_timeout_log() a little bit to drop `locked`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: require libpmem 1.7

detect the libpmem libraries' version using pkg-config

please note, the version is not identical to the ones defined in
libpmem.h or libpmemobj/base.h

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: consolidate the find logic in Findpmem.cmake

less repeating this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40124 from adk3798/format-help

pybind/ceph_argparse: print --format flag name in help descs

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #40316 from batrick/i49605

pybind/mgr/volumes: avoid deadlock in ceph-mgr Finisher thread

Reviewed-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #40555 from jecluis/wip-cephadm-fix-apparmor.master

cephadm: ignore apparmor if profiles file is empty

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #38500 from pdvian/wip-print-weight

crush/CrushCompiler: print weight with uniform precision

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38258 from badone/wip-tracker-48336-monmaptool-enforce-ip-better

monmaptool: Don't call set_port on an invalid address

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40518 from Rethan/fix-client-message-throttle-max

osd: allow osd_client_message_size throttle set to zero

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40444 from TheJJ/ensure-fsck-transactions

os/bluestore: ensure fsck repair transactions succeed

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #40457 from lihuibng/crush-bug

crush/CrushLocation: do not print logging message in constructor

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40488 from lxbsz/typo

mgr: fix the desc typo for mds PyCFunction methods

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40168 from amathuria/wip-amathuria-fix-mon-trimming

mon: Modifying trim logic to change paxos_service_trim_max dynamically

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40498 from guits/guits-quick_fix

cephadm: fix a typo

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #40538 from tchaikov/wip-msg-type-cleanup

msg/msg_types: let ceph_entity_name::parse() accept string_view

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #40556 from liewegas/mon-no-config-indent

mon/ConfigMonitor: do not indent 'config dump' output

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40574 from badone/wip-submanager-contatiner-failure-tracker-50118

install-deps: Use dnf config-manager

Reviewed-by: Kefu Chai <kchai@redhat.com>