crush: improve straw2 algorithm's readability

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>

Merge PR #19760 into master

* refs/pull/19760/head:
mds: add dump flag for CInode
mds: add command "openfiles ls"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19825 into master

* refs/pull/19825/head:
client: properly choose target mds for rmsnap/renamesnap
mds: properly setup need_snapflush for snapped inode
mds: ignore MClientCaps::FLAG_NO_CAPSNAP

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19833 into master

* refs/pull/19833/head:
common: add C_SaferCond ctor with name argument
Client: use C_SaferCond for "_flush_range"
client: use C_SaferCond for _fallocate and ll_write_block
client: use C_SaferCond for _sync_fs and _fsync
client: use C_SaferCond in Client::_write
client: use C_SaferCond for _read_sync
client: use C_SaferCond in _read_async
client: use C_SaferCond for "Client::_read"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19836 into master

* refs/pull/19836/head:
Avoid the assert failure when the inode for the cap_export from other MDS happened not in MDCache

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19842 into master

* refs/pull/19842/head:
client: unify function name in debug log for Client.cc

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19991 into master

* refs/pull/19991/head:
cephfs-journal-tool: move shutdown to the deconstructor of MDSUtility

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20001 into master

* refs/pull/20001/head:
mds: use the steady clock on tick

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20012 into master

* refs/pull/20012/head:
mds: fix scrub crash

Reviewed-by: Douglas Fuller <dfuller@redhat.com>

Merge PR #20079 into master

* refs/pull/20079/head:
mds: set higher priority for some perf counters

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20113 into master

* refs/pull/20113/head:
osdc/ObjectCacher: don't merge TX buffer heads
osdc/ObjectCacher: fix off-by-one error in bh_write_commit()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20000 from zhouyuan/rgw_beast_listen

rgw: allow beast frontend to listen on specific IP address

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20058 from cbodley/wip-doc-rgw-frontends

doc/rgw: add page for http frontend configuration

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge PR #20165 into master

* refs/pull/20165/head:
doc/client: uniform naming

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20164 from luomuyao/typo

rgw: fix a typo in comment

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #20159 from shun-s/fix-rbd-log

librbd: clean up ManagedLock log prefix

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

client: properly choose target mds for rmsnap/renamesnap

Old code does not set inode or dentry for rmsnap/renamesnap requests.
So rmsnap/renamesnap requests are sent to random mds.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: properly setup need_snapflush for snapped inode

MDCache::cow_inode() checks "cap->issued() & CEPH_CAP_ANY_WR" to decide
if it needs to setup need_snapflush for the new snapped inode.
Locker::handle_client_caps() should call Locker::_do_cap_update() first,
then update issued caps. Otherwise MDCache::cow_inode() will fail to
setup need_snapflush if cap messages flushes dirty caps and releases the
same caps.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #19770 from shinobu-x/mempool_lock

common: s/unique_lock/lock_guard/, if manual lock/unlock are not necessary

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rbd: clean up ManagedLock log prefix

  clean up ManagedLock log prefix

Signed-off-by: shun-s <song.shun3@zte.com.cn>

Merge pull request #20052 from ceph/revert-19253-wip-rgw-22124

Revert "rgw: reshard should not update stats when linking new bucket instance"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20171 from linuxbox2/rgw_rvrt_disable_exact

Revert "rgw_file: disable FLAG_EXACT_MATCH enforcement"

Revert "rgw_file: disable FLAG_EXACT_MATCH enforcement"

This reverts commit 5e0bea71b52f652a5cb6323cf56391ada215c80b.
Fixes: http://tracker.ceph.com/issues/22827
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #20167 from cernceph/dvanders_withhold

osd: change log level when withholding pg creation

Reviewed-by: Sage Weil <sage@redhat.com>

osd: change log level when withholding pg creation

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

rgw: fix a typo in comment

Signed-off-by: luomuyao <luo.muyao@zte.com.cn>

Merge pull request #19758 from nrdmn/fix-dashboard-wip

pybind/mgr/dashboard: fix reverse proxy support

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20142 from shangzhong/wip-java-native

java/native: s/jni: lstat/jni: stat in native_ceph_stat

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20130 from k0ste/newfeature

mgr: balancer: fixed mistype "AttributeError: 'Logger' object has no attribute 'err'"

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20127 from majianpeng/issue_op_cleanup

osd/ReplicatedBackend: clean up code.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20119 from Liuchang0812/remove-health-service

mon: remove health service

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20092 from tangwenjun3/wip-fix-write-label

os/bluestore: no need to fsync when failed to write label

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20078 from renhwztetecs/renhw-wip-mon-paxos-cleanup

mon: paxos cleanup

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

doc/client: uniform naming

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #20126 from tchaikov/wip-doc-mgr-auth

doc: update mgr related auth settings

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19990 from dillaman/wip-group-cleanup

librbd: group and snapshot cleanup

Reviewed-by: Mykola Golub <mgolub@suse.com>

rbd: normalize group command aliases

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: dump op features in image info

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/rbd: normalize image group API

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

java/native: s/jni: lstat/jni: stat in native_ceph_stat

Signed-off-by: Shangzhong Zhu <zhu.shangzhong@zte.com.cn>

Merge pull request #19828 from xxhdx1985126/wip_xxh_trackedop_tracking_start

common: mark events of TrackedOp outside its constructor

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20022 from badone/wip-OSDMonitor-error-message-outputs

mon/OSDMonitor: Fix OSDMonitor error message outputs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20115 from liewegas/wip-features

mon: fix structure of 'features' command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20096 from myoungwon/wip-manifest-two-pool-test

src/test/osd: add two pool test for manifest objects

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20018 from smithfarm/wip-python3-only

build/ops: rpm: conditionalize Python 2 availability to enable Ceph build on Python 3-only system

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #20134 from tchaikov/wip-spdk-aarch64

spdk: update SPDK to fix the build failure on aarch64

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ReplicatedBackend: clean up code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

librbd: normalize image group C/C++ API

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: shorten the generated group image snapshot name

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: properly handle group snap add failure case

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: simplified retrieving snapshots from image header

Added a new cls 'snapshot_get' method that retrieves all snapshot
specific details to avoid the need to add a new cls method to
retrieve each individual snapshot data field.

The cls 'get_snapshot_namespace' method has been dropped since Luminous
clients will treat that as a user-generated snapshot (and that's all
that is supported by Luminous clients). The cls 'get_snapshot_timestamp'
method has been deprecated and can be removed once <=Luminous clients
are no longer supported.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls/rbd: automatically manage group op feature on images

When an image is added to a group, set the group op feature
bit and when removed, clear the bit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls/rbd: renamed image group add/remove/get operations

The methods are used by the luminous release of librbd but are essentially
no-ops since group support was removed from the API. However, we need to
prevent odd interactions once a new client creates a group and uses an
older client to manipulate the image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls/rbd: add new op feature bits for groups and snap trash

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls/rbd: removed unused snapshot feature bitmask

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls/rbd: cleanup helper method visibility

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: group image add/remove ops should notify update

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: remove 'consistency' verbiage from group feature

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: treat empty features environment variable as old format

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

src/test/osd/TestRados: add a comment

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

pybind/mgr: fix typo

Signed-off-by: Nick Erdmann <n@nirf.de>

pybind/mgr/dashboard: fix reverse proxy support

This fixes http redirection for reverse http proxies

Fixes: http://tracker.ceph.com/issues/22557
Signed-off-by: Nick Erdmann <n@nirf.de>

Merge pull request #20064 from tchaikov/wip-cmake-py3

cmake: enabled py3 only build

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

spdk: enable SPDK on AArch64 by default

Enable SPDK in Ceph on AArch64 by default.

Change-Id: I81ad8e0b860968d62fd2740ae04923f477dab75a
Signed-off-by: Tone Zhang <tone.zhang@arm.com>

spdk: update SPDK to fix the build failure on aarch64

to include 29be88fab6bb026df27c93ee89c61965b3c43133 which addresses a
build failure on aarch64.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr: be compatible with py3

so ceph-mgr can be compiled using py3

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: build ceph-mgr with specified version of python

* add an option named "MGR_PYTHON_VERSION", so we can build ceph-mgr
  which use py3 for running plugins
* also drop the line to specify the "Python_ADDITIONAL_VERSIONS", because
  2.7 is listed by all the the FindPythonInterp and FindPythonLibs in
  cmake 2.8.12 and up.
* use ${MGR_PYTHON_EXECUTABLE} for holding the path to the python
  interpreter used by mgr. because this variable might be overwritten by
  "find_package(PythonInterp 2 REQUIRED)" when checking for building env
  of pybinding for python2.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add WITH_PYTHON2 option

* add WITH_PYTHON2 option, so we can build python3 bindings only.
* change the default value of WITH_PYTHON3 option to "OFF", as the
option() command in cmake only allow the initial value to be "ON" or
"OFF". we could use a cached string for this option, but i think it
would be more explicit to continue using the option() command.
* fix the installation dir of "ceph_rest_api.py". please note, we still
have a *default* python version, which is specified by the last element
of ${py_vers}. for instance, ${PYTHON_VERSION} will be 3 if ${py_vers}
is 2;3. in this change, 2 is still the default python version, if both
WITH_PYTHON2 and WITH_PYTHON3 are enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr: balancer: fixed mistype "AttributeError: 'Logger' object has no attribute 'err'"

Signed-off-by: Konstantin Shalygin <k0ste@k0ste.ru>

Merge pull request #19922 from Rubab-Syed/performance_counter_browser

mgr/dashboard: performance counter browsers

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20009 from Liuchang0812/fix-22727

mon: do not use per_pool_sum_delta to show recovery summary

Reviewed-by: Varada Kari <varada.kari@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20048 from jcsp/wip-22096

mon: fix mgr using auth_client_required policy

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: update mgr related auth settings

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/thrash/workloads: add paramters to support two pools

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

build/ops: rpm: correct Group for python libraries

Signed-off-by: Nathan Cutler <ncutler@suse.com>

make-dist: add OBS-specific release suffix on SUSE

This is needed to prevent OBS from overriding the nice release number generated
by make-dist.

For more information, see
https://en.opensuse.org/openSUSE:Build_Service_Tips_and_Tricks#How_to_control_a_Release_number_of_resulted_packages

Signed-off-by: Nathan Cutler <ncutler@suse.com>

cmake: empty INSTALL_RPATH for libceph_zstd.so.2.0.0

See 235448879e5 for explanation.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: rpm: introduce _python_buildid macro

This eliminates several ugly conditional blocks. This commit also does some
cleanup like dropping the ceph-base runtime dependency on "python", which is
implicit in "python-requests".

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: rpm: move distro-conditional make check deps

These were in the wrong section.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: PendingReleaseNotes: python-ceph-compat deprecate/drop

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: rpm: python3-ceph-argparse only if Python 2 available

The python3-ceph-argparse package is only necessary in environments where
Python 3 might not be available. For distros where Python 3 has become
standard with Python 2 an optional extension (instead of vice versa as
in the current status quo), the files in this package can (and should)
be included in ceph-common directly.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: rpm: package byte-compiled ceph-rest-api

This is only necessary on Python 3-only systems.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: rpm: conditionalize Python 2 availability

Not all build targets have Python 2 available. Allow the build
to succeed on pure Python 3 systems.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19994 from runsisi/wip-objecter-cleanup

objecter: minor cleanups

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19651 from yanghonggang/master

mon/OSDMonitor.cc: fix expected_num_objects interpret error

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20124 from dillaman/wip-devstack

qa/workunits/rbd: switch devstack to pike release

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20105 from dillaman/wip-22791

librbd: force removal of a snapshot cannot ignore dependent children

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #19320 from vshankar/rbd-mirror-image-map-policy-test

test/rbd-mirror: image map policy test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19974 from shun-s/wip-accelerate-rbd-snap-create-rm-speed

librbd: cache last index position to accelerate snap create/rm

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19996 from Songweibin/wip-rbd-deep-cp

rbd: add deep cp CLI method

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: performance counter browsers

Signed-off-by: Rubab-Syed <rubab.syed21@gmail.com>

qa/workunits/rbd: switch devstack to pike release

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #20060 into master

* refs/pull/20060/head:
ceph-debug-docker.sh: allow specifying branch sha1

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

librbd: force removal of a snapshot cannot ignore dependent children

Fixes: http://tracker.ceph.com/issues/22791
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Revert "rgw: reshard should not update stats when linking new bucket instance"

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #20047 from jcsp/wip-prometheus-qa

qa: add new prometheus test to rados/mgr suite

Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19596 from qrGitHub/wip-rgw-lc-scheduleNextStartTime

rgw: optimize next start time for lifecycle

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #20077 from smithfarm/wip-spdk-fast-forward

spdk: update submodule to more recent upstream

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon: remove health service

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

cmake: build Boost.python with specified version of python

Signed-off-by: Kefu Chai <kchai@redhat.com>