common/buffer: add hint to compiler

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/buffer: do not deref _raw ptr when checking for hypercombind buf

_raw could be null when the buffer::ptr is std::moved away. and ASan
complains when we try to dereference a nullptr. so use a little awkward
way to calculate the address instead

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37000 from xxhdx1985126/wip-47226

crimson/osd: construct log_keys_debug when reading pglog entries

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: construct log_keys_debug when reading pglog entries

Fixes: https://tracker.ceph.com/issues/47226
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #36990 from tchaikov/wip-doc-prompt

doc/dev: use prompt directive when appropriate

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/dev: drop unused label

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev/developer_guide/tests-unit-tests: use literal "make check"

the ``make check`` is better than a hyper link here.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev: fix broken "make check" link

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev: link to label not file

it's less error prone, and can help us avoid broken links if we move a
file around

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #36983 into master

* refs/pull/36983/head:
client: make the osdmap wait under the mount_state reader check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36580 into master

* refs/pull/36580/head:
Client: defer puting the inode
client: defer cancelling the timer event
client: remove _sync_write_commit which makes no sense any more

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36850 into master

* refs/pull/36850/head:
mds: add performance counter for cap messages

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #36464 into master

* refs/pull/36464/head:
client: get rstat from auth mds to reduce forwarding requests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/dev: use prompt directive when appropriate

for 2 reasons:

* sphinx renders codeblock using python syntax highlighting by default,
  so it's not surprising that it highlight keywords like "export" in
  command line samples. so to render command line code blocks, we'd
  better specify the syntax explicitly for better rendering result.
* with the help of "prompt" directive, user is able to copy and paste
  the command without the prompt. for instance, with the default
  "::" directive, user will copy "$ ceph df", which is not very
  convenient, but with "prompt" directive, user only copies
  "ceph df".

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36824 from SchoolGuy/feature/replace-png-with-svg

mgr/dashboard: Exchange PNGs for SVGs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34831 from rhcs-dashboard/41060-API-wrong-error-code

mgr/dashboard: REST API returns 500 when no Content-Type is specified

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36059 from liu-chunmei/seastore-extentmap-tree

crimson/seastore: add extent map tree

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #36865 from ishanrai05/global-styles-fix

mgr/dashboard: fix style being applied globally

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

client: make the osdmap wait under the mount_state reader check

It will make no sense to check this if the unmouting is already
going on.

https://tracker.ceph.com/issues/47293
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #36489 from zdover23/wip-doc-dev-merge-request-2020-aug

doc/dev: add proper merge commit format

Merge pull request #36977 from tchaikov/wip-47290

mon/OSDMonitor: only take in osd into consideration when trimming osd…

Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/dev: add proper Merge PR format

This commit adds an example of proper Merge PR format
to the Developer Guide. Though this might not be of
great use to most people, Zac Dover will find it most
helpful.

Fixes: https://tracker.ceph.com/issues/46843
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #36954 from HeinleinSupport/wip-doc-remove-ceph-deploy

doc: remove references to ceph-deploy

Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge PR #36902 into master

* refs/pull/36902/head:
mds: fix incorrect check for if dirfrag is being fragmented

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36933 into master

* refs/pull/36933/head:
cephfs_mirror: Fix Clang compile errors

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35575 from cbodley/wip-45626

test/rgw: test_datalog_autotrim filters out new entries

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #36758 from cbodley/wip-rgw-sal-user-sync-stats

rgw: rgw_user_sync_all_stats() uses is_truncated()

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #36913 from cbodley/wip-47217

radosgw-admin: period pull command is not always a raw_storage_op

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #28489 from croit/fix-40255

rgw: fix bucket limit check fill_status warnings

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30584 from zhangsw/fix-rgw-expiration-header

rgw: filter prefix should be used when generating expiration header

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #31058 from cbodley/wip-rgw-skip-bilog

cls/rgw: avoid read_bucket_header() when bi logging is disabled

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #31354 from xxcs/xxcs05

rgw: The rgw_bucket overload <operator, need to consider bucket_id

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #31663 from cfanz/wip-rgw-fix-partial-copy-bug

rgw: do initialization in init_processing() for range copy

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #33942 from pritha-srivastava/rgw-sts-new-keycloak-examples

rgw: Updating STS document with references to new keycloak

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge PR #36546 into master

* refs/pull/36546/head:
vstart_runner: log commands in a more usable form

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #35122 from BryceCao/wip-swift-buckets-jammed-metadata-sync

rgw: URL encode bucket name in metadata sync

Reviwed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #35123 from BryceCao/wip-bug-fix-swift-bucket-create

rgw: urlencode bucket name when forwarding request

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35551 from joke-lee/v4-auth

rgw: replace '+' with "%20" in canonical query string for s3 v4 auth.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35984 from matthewoliver/rgw_swift_api_401_for_v1_and_keystone

rgw: Swift API anonymous access should 401

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36502 from cbodley/wip-rgw-objv-tracker-inc

rgw: RGWObjVersionTracker tracks version over increments

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #36971 into master

* refs/pull/36971/head:
doc: fix typo and correct the affiliation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/OSDMonitor: only take in osd into consideration when trimming osdmaps

we should not take down osd into consideration when trimming osdmap. in
e62269c892, we decrease the upper bound of range of osdmaps to be trimmed
if the given osd is out. but we should have to decrease it only if the
osd in question is still *in*.

so, in this change, the min_lec is decreased only if the osd in question
is *in*.

Fixes: https://tracker.ceph.com/issues/47290
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36218 from mkogan1/wip-cache-put-post

rgw: add PUT and POST req support to data cache

Reviewed-by: Or Friedmann <ofriedma@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #36542 from mkogan1/wip-rgw-fix-sync-stats

rgw: fix user stats iterative increment

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #36932 from theMarix/patch-1

rpm: Fix typo in summary of prometheus-alerts package

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36951 from cbodley/wip-47240

cls/rgw: unlink_instance writes updated olh log on early return

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36908 from rosinL/wip-valid-chunk

msg/async/rdma: use wr_id address to check valid chunk

Reviewed-by: Changcheng Liu <changcheng.liu@aliyun.com>

Merge pull request #36894 from pcuzner/device-ls-with-lsm

orchestrator: intergrate LSM information with device ls

Merge pull request #36911 from sebastian-philipp/cephadm-lsmcli-jinja

mgr/cephadm: customize blink_device_light cmd via j2

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36943 from sebastian-philipp/cephadm-add-comments-to-secondary-containers

mgr/cephadm: Add comments to secondary contaieners

Reviewed-by: Matthew Oliver <moliver@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #36949 from sebastian-philipp/mgr-re-add_osd_support

Revert "mgr/osd_support: remove module and all traces"

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #32779 from petrutlucian94/windows.6

Windows support [part 6]

mds: add performance counter for cap messages

Signed-off-by: Kotresh HR <khiremat@redhat.com>
Fixes: https://tracker.ceph.com/issues/47102

doc: fix typo and correct the affiliation

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

changes the link to internal

Signed-off-by: Robert Sander <gurubert@gurubert.de>

Merge pull request #36919 from tchaikov/wip-crimson-check

crimson/osd: check if req should be server with obc lock

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #36960 from tchaikov/wip-crimson-admin

crimson/admin: use tell_result_t(unique_ptr<Formatter>)

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: REST API returns 500 when no Content-Type is specified

Fixes: https://tracker.ceph.com/issues/41060
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit ea031de0908a249b416617a1d9cc806c356520e4)
(cherry picked from commit a5bf74e6a6c7678bbb3689a4b52fe817418da58b)

Merge pull request #36959 from ypdai/patch-3

doc: Update multisite-sync-policy.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/admin: use tell_result_t(unique_ptr<Formatter>)

to address the comment of

> Taking the `Formatter` by pointer was a bit confusing in the matter of
> lifetime control.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36775 from tchaikov/wip-spdk

spdk: pick up spdk v20.07

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/orchestrator: add test command testcase for "device ls"

Quick test to ensure changes that break wide mode are picked
up

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

doc: Update multisite-sync-policy.rst

Signed-off-by: ypdai <self19900924@gmail.com>

Merge pull request #36956 from yuriw/wip-yuriw-crontab-master_2

qa/tests: fixed subset for runs on master

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tests: fixed subset for runs on master

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #36837 from yuriw/wip-yuriw-crontab-master_2

qa/tests: run 'rados' tests every day

Reviewed-by: Neha Ojha <nojha@redhat.com>

doc: remove references to ceph-deploy

The documentation still has many traces of ceph-deploy. This tool has
been deprecated with the Octopus release. This commit tries to remove
most of ceph-deploy occurences.

Signed-off-by: Robert Sander <r.sander@heinlein-support.de>

vstart_runner: log commands in a more usable form

Currently commands are printed as ['arg1', 'arg2', 'arg3']. Instead, log
them as '> arg1 arg2 arg3' so that it's simpler to copy and run them
manually. The reason behind prepending '> ' to these logs entries is
just to follow the practice followed by teuthology logs.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

rgw: RGWDeleteObj::execute initializes olh epoch

the uninitialized value was causing a comparison to fail in cls/rgw and
the deletion of an object version wasn't applied correctly

Fixes: https://tracker.ceph.com/issues/47240
Signed-off-by: Casey Bodley <cbodley@redhat.com>

cls/rgw: unlink_instance writes updated olh log on early return

Fixes: https://tracker.ceph.com/issues/47240
Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/tests: run 'rados' tests every day

Also changed the "subset" to 100000 to reduce daily runs on master branch

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #36883 from mhackett/wip-mhackett-47190

doc: document tuning of radosgw lifecycle

Merge PR #36939 into master

* refs/pull/36939/head:
ceph.spec, debian: changes for building cephfs-mirror daemon

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #36940 from rhcs-dashboard/notification_cleanup

mgr/dashboard: Improving notification sidebar card spacing

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Revert "mgr/osd_support: remove module and all traces"

This reverts commit a55c1dd0fac826978edf22e448c3456e7634cbc3.

Kept to keep upgrades from older point releases working.
This module can be removed as soon as we no longer
support upgrades from old octopus point releases.

Revert "build/debian: remove osd_support"

This reverts commit 8ff2824beb78dfd03710e94302f038ea70fb4561.

Fixes: https://tracker.ceph.com/issues/47109
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #36594 from matthewoliver/ipv6_prometheus_module

prometheus: Properly split the port off IPv6 addresses

Merge pull request #34293 from yuvalif/notifications_persistent_delivery2

rgw/notifications: add persistent delivery to notifications

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36847 from X0Ken/fix-journal-size

ceph-volume: fix journal size argument not work

Merge pull request #36545 from AmnonHanuhov/wip-seastar_metrics

crimson/admin: Add a hook for Seastar metrics

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #36743 from cbodley/wip-rgw-sal-bucket-ctime

rgw: RGWBucket uses creation_time from RGWBucketInfo

ceph-volume: fix journal size argument not work

Journal size argument not work in batch prepare
filestore with SSD journal device.

Fixes: https://tracker.ceph.com/issues/41374
Signed-off-by: wanghongxu <wanghongxu@t2cloud.net>

ceph.spec, debian: changes for building cephfs-mirror daemon

Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #35802 from rhcs-dashboard/wip-46160-fix_cephfs_api_inconsistency

mgr/dashboard: Address API inconsistencies in Cephfs endpoint

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/cephadm: Add comments to secondary contaieners

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Co-authored-by: Michael Fritch <mfritch@suse.com>

Merge pull request #36873 from sebastian-philipp/cephadm-osd-activate-rm-storage

cephadm: properly remove `osd-activate` container

Reviewed-by: Michael Fritch <mfritch@suse.com>

crimson/osd: Register Seastar metrics hook

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>

crimson/admin: Add Seastar metrics hook

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>

Merge pull request #36923 from rhcs-dashboard/fix-47232-master

mgr/dashboard/api: print logs when vstart fails

Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: Improving notification sidebar card spacing

There were some unnecessary spacings in the notification sidebar cards. This commit is a cleanup for taking care of that spaces.

Fixes: https://tracker.ceph.com/issues/47262
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #36222 from ishanrai05/docs

doc: Create new chapter about the Dashboard's landing page

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34644 from rhcs-dashboard/26954-histogram

mgr/dashboard: Remove Histogram Component

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: fix style being applied globally

fixes: https://tracker.ceph.com/issues/45018

Signed-off-by: Ishan Rai <ishanrai05@gmail.com>

Merge pull request #36869 from tchaikov/wip-mon-no-MMonMetadata

mon/Monitor: do not handle MMonMetadata

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cephfs_mirror: Fix Clang compile errors

Don't use `class CephContext;` in this way.
Including `./src/include/common_fwd.h` does it in a crimson compatible
way.

```
/home/jenkins/workspace/ceph-master-compile/src/tools/cephfs_mirror/ClusterWatcher.h:15:1: error: declaration conflicts with target of using declaration already in scope
class CephContext;
^
/home/jenkins/workspace/ceph-master-compile/src/include/common_fwd.h:10:9: note: target of using declaration
  class CephContext;
        ^
/home/jenkins/workspace/ceph-master-compile/src/include/common_fwd.h:22:24: note: using declaration
using TOPNSPC::common::CephContext;
```

Also fix a reference in a lambda-block,
```
/home/jenkins/workspace/ceph-master-compile/src/tools/cephfs_mirror/Mirror.cc:266:51: error: 'fs_name' in capture list does not name a variable
    Context *on_finish = new LambdaContext([this, fs_name](int r) {
                                                  ^
/home/jenkins/workspace/ceph-master-compile/src/tools/cephfs_mirror/Mirror.cc:267:62: error: reference to local binding 'fs_name' declared in enclosing function 'cephfs::mirror::Mirror::run'
                                             handle_shutdown(fs_name, r);
                                                             ^
/home/jenkins/workspace/ceph-master-compile/src/tools/cephfs_mirror/Mirror.cc:259:15: note: 'fs_name' declared here
  for (auto &[fs_name, fs_mirror] : m_fs_mirrors) {
              ^
```

fixes: https://github.com/ceph/ceph/pull/35697
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge PR #35697 into master

* refs/pull/35697/head:
doc: document cephfs mirroring developement status
cephfs-mirror: cephfs-mirror daemon
cephfs-mirror: FSMirror class to synchronize directory snaps
common: introduce configuration options for cephfs-mirror daemon
cephfs-mirror: InstanceWatcher class to register mirror instance
cephfs-mirror: MirrorWatcher class to watch cephfs-mirror object
cephfs-mirror: Watcher class to receive/ack watch notifications
cephfs-mirror: ClusterWatcher class for watching peer changes
librados: add API for fetching client addrs
common: add cephfs_mirror subsystem
cephfs-mirror: filesystem specification class

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35899 from myoungwon/wip-fix-manifest-tier

osd, test: refactoring manfiest-tier

Reviewed-by: Samuel Just <sjust@redhat.com>

rpm: Fix typo in summary of prometheus-alerts package

Signed-off-by: Matthias Bach <marix@marix.org>

Merge pull request #36915 from yuriw/wip-yuriw-46529-master

qa/tests: use bionic only for old clients in rados/thrash-old-clients

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

crimson/osd: check if req should be server with obc lock

there is chance that osdmap is updated after the obc lock is acquired
for a certain object, and before the lock is acquired, the OSD is able
to serve the request. and the updated osdmap changes the OSD from
primary to a stray OSD of the PG in question. so we need to move the
check closer to where we actually handle the op.

Fixes: https://tracker.ceph.com/issues/47212
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36700 from leseb/retry-on-lock

ceph-volume: retry when acquiring lock fails