rgw: create sync-module instance when execute radosgw-admin data sync run

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

Merge PR #19421 into master

* refs/pull/19421/head:
cephfs-journal-tool: tool would miss to report some invalid range

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19277 into master

* refs/pull/19277/head:
cephfs: potential adjust failure in lru_expire

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #19550 from dillaman/wip-22461

rbd-mirror: cluster watcher should ensure it has latest OSD map

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #19458 from wido/ceph-volume-eol

ceph-volume: Add linesep/newline at end of JSON file when writing

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #19518 from jcsp/wip-mgr-access-denied

mgr: fix "access denied" message

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19515 from tchaikov/wip-cmake-lttng-leak

 cmake: link libcommon with libstdc++ statically if WITH_STATIC_LIBSTDCXX

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19490 from adamemerson/wip-hypomodern

Update C++ standard to 14 and clean up

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19486 from liewegas/wip-22419

osd: pass pool options to ObjectStore on pg create

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #19385 from liewegas/wip-stdout-logs

mon: allow cluter and debug logs to go to stderr, with appropriate prefix

Reviewed-by: Kefu Chai <kchai@redhat.com>

cephfs-journal-tool: tool would miss to report some invalid range

Fixes: http://tracker.ceph.com/issues/22459
Signed-off-by: dongdong tao <tdd21151186@gmail.com>

cephfs: potential adjust failure in lru_expire

Fix: the first adjust is no needed,it will never take real effect.
     the second 'adjust' may never get the chance to be executed
     suppose we can reach the second 'adjust', it will crash because the bottom list is empty now.

Fixes: http://tracker.ceph.com/issues/22458
Signed-off-by: dongdong tao <tdd21151186@gmail.com>

Merge pull request #19464 from myoungwon/wip-bug-fix-22369

osd: fix unordered read bug (for chunked object)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #19535 into master

* refs/pull/19535/head:
doc: clarify type of client in config ref

Reviewed-by: Jos Collin <jcollin@redhat.com>

rbd-mirror: cluster watcher should ensure it has latest OSD map

Fixes: http://tracker.ceph.com/issues/22461
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #18512 into master

* refs/pull/18512/head:
mon,mgr,doc,test: standardize argument as role

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18899 into master

* refs/pull/18899/head:
mds: add asok command that dumps metadata popularity

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19170 into master

* refs/pull/19170/head:
mds: Improve the log info for read pointers

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #19271 into master

* refs/pull/19271/head:
client: update Dir::num_null_dentries in Dentry::{link,unlink}
client: drop null child dentries before try pruning inode's alias

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19414 into master

* refs/pull/19414/head:
mds: properly eval locks after importing inode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19440 into master

* refs/pull/19440/head:
mds: update mds option descriptions
mds: obsolete MDSMap option configs
mds: organize Filesystem class def

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19540 from dillaman/wip-rbd-trash-list-enoent

rbd: 'trash list --long' will return a failure on non-cloned images

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

rbd: 'trash list --long' will return a failure on non-cloned images

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19493 from cernceph/dvanders_balancer

mgr/balancer: cast config vals to int or float

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19482 from jcsp/wip-22361

mgr/balancer: don't use 'foo' tags on commands

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19465 from shinobu-x/no_copy_ctor_assignment

common,mds,osd: Explicitly delete copy ctor if noncopyable

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19211 from ZVampirEM77/wip-em-pg-cleanup

osd: fix typos and some cleanups

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19013 from yuyuyu101/wip-fix-racing

msg/async/AsyncConnection: unregister connection when racing happened

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19361 from Songweibin/wip-init-image-id

pybind/rbd: support open the image by image_id

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19317 from shinobu-x/journaler_trivial_cleanup

journal: trivial cleanup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19406 from trociny/wip-deep-copy-pybind

pybind/rbd: add deep_copy method

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19436 from trociny/wip-22333

rbd-nbd: support optionally setting device timeout

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19477 from wido/mgr-int-json

ceph-volume: Try to cast OSD metadata to int while scanning directory

Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: clarify type of client in config ref

The kernel configuration is different.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

common: Switch from boost::shared_mutex to std::shared_mutex

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common: Get rid of artifact filename

Since we're on C++14, we're just backporting from future standards
generally.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common: Remove backported max and make_unique

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #19516 from tchaikov/wip-install-deps

install-deps.sh: use tee for writing a file

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #19517 from qrGitHub/wip-rgw-typo-signle

rgw: fix typo signle to single

Reviewed-by: Jos Collin <jcollin@redhat.com>

mgr: fix "access denied" message

As pointed out on https://github.com/ceph/ceph/pull/18716

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #18716 from shinobu-x/sk-mgr_daemonserver

mgr/DaemonServer: [Cleanup] Remove redundant code

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

rgw: fix typo signle to single

Signed-off-by: Bingyin Zhang <zhangbingyin@cloudin.cn>

install-deps.sh: use tee for writing a file

because the stdout redirect is performed by current shell not `$SUDO
cat`, so $SUDO does not help here. use `$SUDO tee` instead, which is
able to read the stdin and write to the given file just as expected.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19492 from dillaman/wip-rbd-mirror-dashboard

rbd-mirror: ensure unique service daemon name is utilized

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

cmake: link global-static against common

* as global-static references symbols offered by common.
* and remove DPDK_LIBRARIES from global, as it is libcommon which
  is using DPDK.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: link libcommon with libstdc++ statically if WITH_STATIC_LIBSTDCXX

this matches the linkage of libceph-common, because valgrind reports
Leak_StillReachable if daemons are linked against libceph-common, will
link daemons against common instead, and let common link libstdc++
statically, if WITH_STATIC_LIBSTDCXX=ON.

this change also reverts e6695bb6

Fixes: http://tracker.ceph.com/issues/22438
Signed-off-by: Kefu Chai <kchai@redhat.com>

global: remove unused header

the asock functionality was moved into ceph_context.cc in 12f6e8f1.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/mds/osd: Explicitly delete if no copy, and remove incorrect comment

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

common: Swap C++14 template type aliases for variables

C++14 provides the ability to define template variables, which the C++17
library puts to good use. We define those same aliases.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

build: Uplevel to C++14

Fix a couple problems.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #19470 from tchaikov/wip-async-dpdk

msg/async: update to work with dpdk shipped with spdk v17.10

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #19466 from shinobu-x/super_redundant_public_label

osdc: Remove a bit too redundant public label...

Merge pull request #19494 from smithfarm/wip-22362

build/ops: rpm: set permissions 0755 on rbd resource agent

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19495 from ZVampirEM77/wip-em-pg-cleanup2

osd/PrimaryLogPG: cleanup do_sub_op && do_sub_op_reply and define soid in prepare_transaction more appropriate

Reviewed-by: Sage Weil <sage@redhat.com>

mds: update mds option descriptions

Fixes: http://tracker.ceph.com/issues/21764
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: obsolete MDSMap option configs

These configs were used for initialization but it is more appropriate to
require setting these file system attributes via `ceph fs set`. This is similar
to what was already done with max_mds. There are new variables added for `fs
set` where missing.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19425 from kungf/calc_replicated_acting

osd: reduce all_info map find to get primary

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

Merge pull request #17691 from amitkumar50/cov-osd-3

osd: Initialization of data members

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #19033 from david-z/wip-fix-fusestore-df

os/FuseStore: fix incorrect used space statistics for fuse's statfs interface

Reviewed-by: Sage Weil <sage@redhat.com>

build/ops: rpm: set permissions 0755 on rbd resource agent

Fixes: http://tracker.ceph.com/issues/22362
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19468 from tchaikov/wip-install-deps.sh

install-deps.sh: avoid re-installing g++-7

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

src/test: remove unnecessary stat

This commit prevents unnecessary stat that
invokes promote_object()
promote_object() makes object clean state

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

qa/suites/rados/thrash: remove write op

current chunked object and ChunkReadOp are
only for the read case.
write op and promote_object() still be tested without ChunkReadOp
by another ceph_test_rados in the same test suite (with --set_chunk)

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

mgr/balancer: cast config vals to int or float

upmap_max_iterations and other config vals need to be numeric.
Cast them appropriately.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Fixes: http://tracker.ceph.com/issues/22429

mgr/dashboard: use unique instance id as daemon identifier

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: utilize application metadata to detect RBD pools

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: ensure each daemon instance uses a unique service name

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

PrimaryLogPG: cleanup do_sub_op and do_sub_op_reply

The related code has been cleaned in https://github.com/ceph/ceph/pull/17247,
but it seems something was missed.

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

PrimaryLogPG: define soid just before using it

Define the soid just before using it to avoid a redundant
definition if an error happens and function prepare_transaction
returns directly.

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

Merge pull request #19453 from liewegas/wip-pg-primary

osd/PG: include primary in PG operator<< for ec pools

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PG: pass pool options to ObjectStore on pg create

We were handling the load case and the pool change case, but not newly
created PG.

Fixes: http://tracker.ceph.com/issues/22419
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #19478 from tchaikov/wip-silence-warnings

qa/standalone/ceph-helpers.sh: silence ceph-disk DEPRECATION_WARNING

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/balancer: don't use 'foo' tags on commands

This looks weird in logs when code elsewhere logs
that it didn't handle a command with tag 'foo'

Fixes: http://tracker.ceph.com/issues/22361
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #19404 from jan--f/mon-fix-digest-timer-inactive-paxos

mon: reenable timer to send digest when paxos is temporarily inactive

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19366 from liewegas/wip-21557

osd: more debugging for snapmapper bug

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19367 from wjwithagen/wip-wjw-clang-xlist-string

include: xlist: Fix Clang error for missing string

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19365 from liewegas/wip-tell

qa/tasks/ceph_manager: tolerate tell osd.* error

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19353 from shinobu-x/common_clock

common: Remove ceph_clock_gettime, extern keyword

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19352 from chardan/jfw-wip-issue-20811

osd: migrate PGLOG_* macros to constexpr

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19329 from ZVampirEM77/cleanup-revise-have

osd/PGLog: cleanup unused function revise_have

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19056 from adamemerson/wip-with-val

common/config: use with_val() for better performance

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19114 from tchaikov/wip-rm-request-slow

mgr/PGMap: drop REQUEST_{SLOW,STUCK} HEALTH_WARNs

Reviewed-by: John Spray <john.spray@redhat.com>

qa/standalone/ceph-helpers.sh: silence ceph-disk DEPRECATION_WARNING

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19229 from wido/mgr-influx

mgr/influx: PEP-8 and other fixes to Influx module

Reviewed-by: John Spray <john.spray@redhat.com>

ceph-volume: Try to cast OSD metadata to int while scanning directory

By doing so values like 'whoami' and 'bluefs' will be stored as a
integer in the resulting JSON rather then a String.

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #19472 from qrGitHub/wip-rgw-typo-woild

rgw: fix typo woild to would

Reviewed-by: Jos Collin <jcollin@redhat.com>

rgw: correct typo woild to would

Signed-off-by: Bingyin Zhang <zhangbingyin@cloudin.cn>

msg/async: update to work with dpdk shipped with spdk v17.10

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: DPDK works w/o CRYPTOPP

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: fix unordered read bug (for chunked object)

The current implementation for chunked object only supports
proxy_read(if offset is within range) and write(local write)
In this case, a read request can be handled before a write request.
This commit prevents unordered read processing because
proxy_read() will be executed if the chunk is missing state.
If chunked object has been overwritten, its state will not be missing.

Fixes: http://tracker.ceph.com/issues/22369
Signed-off-by: Myoungwon Oh <omwmw@sk.com>

install-deps.sh: avoid re-installing g++-7

* add ubuntu-toolchain-r mirrors in case the ppa.launchpad.net is not
  accessible
* add ppa repo manually, it's faster than installing
  `software-properties-common` and then launch `add-apt-repository`
* hardwire $old to the gcc version shipped by the distro, simpler this
  way.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osdc: Remove a bit too redundant public label...

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

msg/async/AsyncConnection: unregister connection when racing happened

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #17651 from ceph/wip-ceph-deploy-upgrade

qa/tests: Wip ceph deploy upgrade

ceph-volume: Add linesep/newline at end of JSON file when writing

Instead of:

    "systemd": "",
    "type": "bluestore",
    "whoami": "0"
  }root@alpha:~#

A newline is added to the JSON file when writing:

    "type": "bluestore",
    "whoami": "0"
  }
  root@alpha:~#

Makes it a bit easier to read the JSON files on a terminal

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #19144 from zhangsw/cleanup-rgwputobj-md5

rgw: no need to deal with md5 header in get_data.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19167 from joke-lee/Bucket_Policy_SourceIp_Condition

rgw: implement ipv4 aws:SourceIp condition for bucket policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19210 from Liuchang0812/rgw-camelcase

 rgw: use camelcase format in request headers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #19236 from dplyakin/get-website-error-code

rgw: fix GET website response error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19250 from qrGitHub/wip-rgw-unused-param2

rgw: remove unused param in AWSGeneralAbstractor::get_auth_data_v4

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>