Merge pull request #19486 from liewegas/wip-22419

osd: pass pool options to ObjectStore on pg create

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #19385 from liewegas/wip-stdout-logs

mon: allow cluter and debug logs to go to stderr, with appropriate prefix

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19464 from myoungwon/wip-bug-fix-22369

osd: fix unordered read bug (for chunked object)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #19535 into master

* refs/pull/19535/head:
doc: clarify type of client in config ref

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #18512 into master

* refs/pull/18512/head:
mon,mgr,doc,test: standardize argument as role

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18899 into master

* refs/pull/18899/head:
mds: add asok command that dumps metadata popularity

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19170 into master

* refs/pull/19170/head:
mds: Improve the log info for read pointers

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #19271 into master

* refs/pull/19271/head:
client: update Dir::num_null_dentries in Dentry::{link,unlink}
client: drop null child dentries before try pruning inode's alias

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19414 into master

* refs/pull/19414/head:
mds: properly eval locks after importing inode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19440 into master

* refs/pull/19440/head:
mds: update mds option descriptions
mds: obsolete MDSMap option configs
mds: organize Filesystem class def

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19540 from dillaman/wip-rbd-trash-list-enoent

rbd: 'trash list --long' will return a failure on non-cloned images

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

rbd: 'trash list --long' will return a failure on non-cloned images

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19493 from cernceph/dvanders_balancer

mgr/balancer: cast config vals to int or float

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19482 from jcsp/wip-22361

mgr/balancer: don't use 'foo' tags on commands

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19465 from shinobu-x/no_copy_ctor_assignment

common,mds,osd: Explicitly delete copy ctor if noncopyable

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19211 from ZVampirEM77/wip-em-pg-cleanup

osd: fix typos and some cleanups

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19013 from yuyuyu101/wip-fix-racing

msg/async/AsyncConnection: unregister connection when racing happened

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19361 from Songweibin/wip-init-image-id

pybind/rbd: support open the image by image_id

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19317 from shinobu-x/journaler_trivial_cleanup

journal: trivial cleanup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19406 from trociny/wip-deep-copy-pybind

pybind/rbd: add deep_copy method

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19436 from trociny/wip-22333

rbd-nbd: support optionally setting device timeout

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19477 from wido/mgr-int-json

ceph-volume: Try to cast OSD metadata to int while scanning directory

Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: clarify type of client in config ref

The kernel configuration is different.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19516 from tchaikov/wip-install-deps

install-deps.sh: use tee for writing a file

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #19517 from qrGitHub/wip-rgw-typo-signle

rgw: fix typo signle to single

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18716 from shinobu-x/sk-mgr_daemonserver

mgr/DaemonServer: [Cleanup] Remove redundant code

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

rgw: fix typo signle to single

Signed-off-by: Bingyin Zhang <zhangbingyin@cloudin.cn>

install-deps.sh: use tee for writing a file

because the stdout redirect is performed by current shell not `$SUDO
cat`, so $SUDO does not help here. use `$SUDO tee` instead, which is
able to read the stdin and write to the given file just as expected.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19492 from dillaman/wip-rbd-mirror-dashboard

rbd-mirror: ensure unique service daemon name is utilized

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

common/mds/osd: Explicitly delete if no copy, and remove incorrect comment

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

Merge pull request #19470 from tchaikov/wip-async-dpdk

msg/async: update to work with dpdk shipped with spdk v17.10

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #19466 from shinobu-x/super_redundant_public_label

osdc: Remove a bit too redundant public label...

Merge pull request #19494 from smithfarm/wip-22362

build/ops: rpm: set permissions 0755 on rbd resource agent

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19495 from ZVampirEM77/wip-em-pg-cleanup2

osd/PrimaryLogPG: cleanup do_sub_op && do_sub_op_reply and define soid in prepare_transaction more appropriate

Reviewed-by: Sage Weil <sage@redhat.com>

mds: update mds option descriptions

Fixes: http://tracker.ceph.com/issues/21764
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: obsolete MDSMap option configs

These configs were used for initialization but it is more appropriate to
require setting these file system attributes via `ceph fs set`. This is similar
to what was already done with max_mds. There are new variables added for `fs
set` where missing.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19425 from kungf/calc_replicated_acting

osd: reduce all_info map find to get primary

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

Merge pull request #17691 from amitkumar50/cov-osd-3

osd: Initialization of data members

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #19033 from david-z/wip-fix-fusestore-df

os/FuseStore: fix incorrect used space statistics for fuse's statfs interface

Reviewed-by: Sage Weil <sage@redhat.com>

build/ops: rpm: set permissions 0755 on rbd resource agent

Fixes: http://tracker.ceph.com/issues/22362
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19468 from tchaikov/wip-install-deps.sh

install-deps.sh: avoid re-installing g++-7

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

src/test: remove unnecessary stat

This commit prevents unnecessary stat that
invokes promote_object()
promote_object() makes object clean state

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

qa/suites/rados/thrash: remove write op

current chunked object and ChunkReadOp are
only for the read case.
write op and promote_object() still be tested without ChunkReadOp
by another ceph_test_rados in the same test suite (with --set_chunk)

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

mgr/balancer: cast config vals to int or float

upmap_max_iterations and other config vals need to be numeric.
Cast them appropriately.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Fixes: http://tracker.ceph.com/issues/22429

mgr/dashboard: use unique instance id as daemon identifier

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: utilize application metadata to detect RBD pools

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: ensure each daemon instance uses a unique service name

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

PrimaryLogPG: cleanup do_sub_op and do_sub_op_reply

The related code has been cleaned in https://github.com/ceph/ceph/pull/17247,
but it seems something was missed.

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

PrimaryLogPG: define soid just before using it

Define the soid just before using it to avoid a redundant
definition if an error happens and function prepare_transaction
returns directly.

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

Merge pull request #19453 from liewegas/wip-pg-primary

osd/PG: include primary in PG operator<< for ec pools

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PG: pass pool options to ObjectStore on pg create

We were handling the load case and the pool change case, but not newly
created PG.

Fixes: http://tracker.ceph.com/issues/22419
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #19478 from tchaikov/wip-silence-warnings

qa/standalone/ceph-helpers.sh: silence ceph-disk DEPRECATION_WARNING

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/balancer: don't use 'foo' tags on commands

This looks weird in logs when code elsewhere logs
that it didn't handle a command with tag 'foo'

Fixes: http://tracker.ceph.com/issues/22361
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #19404 from jan--f/mon-fix-digest-timer-inactive-paxos

mon: reenable timer to send digest when paxos is temporarily inactive

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19366 from liewegas/wip-21557

osd: more debugging for snapmapper bug

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19367 from wjwithagen/wip-wjw-clang-xlist-string

include: xlist: Fix Clang error for missing string

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19365 from liewegas/wip-tell

qa/tasks/ceph_manager: tolerate tell osd.* error

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19353 from shinobu-x/common_clock

common: Remove ceph_clock_gettime, extern keyword

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19352 from chardan/jfw-wip-issue-20811

osd: migrate PGLOG_* macros to constexpr

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19329 from ZVampirEM77/cleanup-revise-have

osd/PGLog: cleanup unused function revise_have

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19056 from adamemerson/wip-with-val

common/config: use with_val() for better performance

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19114 from tchaikov/wip-rm-request-slow

mgr/PGMap: drop REQUEST_{SLOW,STUCK} HEALTH_WARNs

Reviewed-by: John Spray <john.spray@redhat.com>

qa/standalone/ceph-helpers.sh: silence ceph-disk DEPRECATION_WARNING

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19229 from wido/mgr-influx

mgr/influx: PEP-8 and other fixes to Influx module

Reviewed-by: John Spray <john.spray@redhat.com>

ceph-volume: Try to cast OSD metadata to int while scanning directory

By doing so values like 'whoami' and 'bluefs' will be stored as a
integer in the resulting JSON rather then a String.

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #19472 from qrGitHub/wip-rgw-typo-woild

rgw: fix typo woild to would

Reviewed-by: Jos Collin <jcollin@redhat.com>

rgw: correct typo woild to would

Signed-off-by: Bingyin Zhang <zhangbingyin@cloudin.cn>

msg/async: update to work with dpdk shipped with spdk v17.10

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: DPDK works w/o CRYPTOPP

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: fix unordered read bug (for chunked object)

The current implementation for chunked object only supports
proxy_read(if offset is within range) and write(local write)
In this case, a read request can be handled before a write request.
This commit prevents unordered read processing because
proxy_read() will be executed if the chunk is missing state.
If chunked object has been overwritten, its state will not be missing.

Fixes: http://tracker.ceph.com/issues/22369
Signed-off-by: Myoungwon Oh <omwmw@sk.com>

install-deps.sh: avoid re-installing g++-7

* add ubuntu-toolchain-r mirrors in case the ppa.launchpad.net is not
  accessible
* add ppa repo manually, it's faster than installing
  `software-properties-common` and then launch `add-apt-repository`
* hardwire $old to the gcc version shipped by the distro, simpler this
  way.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osdc: Remove a bit too redundant public label...

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

msg/async/AsyncConnection: unregister connection when racing happened

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #17651 from ceph/wip-ceph-deploy-upgrade

qa/tests: Wip ceph deploy upgrade

Merge pull request #19144 from zhangsw/cleanup-rgwputobj-md5

rgw: no need to deal with md5 header in get_data.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19167 from joke-lee/Bucket_Policy_SourceIp_Condition

rgw: implement ipv4 aws:SourceIp condition for bucket policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19210 from Liuchang0812/rgw-camelcase

 rgw: use camelcase format in request headers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #19236 from dplyakin/get-website-error-code

rgw: fix GET website response error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19250 from qrGitHub/wip-rgw-unused-param2

rgw: remove unused param in AWSGeneralAbstractor::get_auth_data_v4

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #19390 from yaozongyou/remove-duplicated-include

rgw,common: remove already included header files

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #19391 from qrGitHub/wip-rgw-memory-optimize

rgw: optimize memory usage in function rgw_bucket::get_key

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19430 from tianshan/fix_recursive_lock

rgw: fix recursive lock

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd/PG: include primary in PG operator<< for ec pools

Otherwise it is confusing!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #19418 from tchaikov/wip-gcc-7-on-xenial

install-deps.sh, cmake: use GCC-7 on xenial also

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19402 from b-ranto/wip-fix-arm-build

test/librbd: Do not instantiate TrimRequest template class

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mon/LogMonitor: add mon_cluster_log_to_stderr

Optionally send cluster log messages to stderr (prefixed by the
channel).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #19437 from trociny/wip-mock-valgrind

test/rbd_mirror: "use of uninitialised value" valgrind warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

log: add log_stderr_prefix option

Allows you to set a prefix for debug log messages send to stderr (e.g.,
"debug ").

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #19253 from oritwas/wip-rgw-22124

rgw: reshard should not update stats when linking new bucket instance

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rbd-nbd: support optionally setting device timeout

Fixes: http://tracker.ceph.com/issues/22333
Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

Merge remote-tracking branch 'gh/mimic-dev1'

Merge pull request #19435 from smithfarm/wip-20559

doc: globally change CRUSH ruleset to CRUSH rule

Reviewed-by: Sage Weil <sage@redhat.com>

librbd: Do not instantiate TrimRequest template class

We include TrimRequest.cc in librbd tests at two places:
 - operation/test_mock_TrimRequest.cc
 - operation/test_mock_ResizeRequest.cc

That causes linking errors when doing the builds because some of the
instantiated classes are defined twice.

We can fix this by not instantiating the template class in the
TrimReqeust.cc file when including it in the tests.

Signed-off-by: Boris Ranto <branto@redhat.com>

test/rbd_mirror: "use of uninitialised value" valgrind warning

The `on_call` context serves as a barrier and should be completed
after the `on_start_ctx` context is assigned.

The warning was observed sporadically e.g. by repeating
WaitingOnNonLeaderAcquireLeader test under valgrind.

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

mds: organize Filesystem class def

This follows the coding style guidelines:

https://google.github.io/styleguide/cppguide.html#Declaration_Order

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19191 from tchaikov/wip-mon-cleanup

mon: remove_is_write_ready()

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #17438 from mikulely/fix-mstop

vstart.sh: fix mstop.sh can not stop rgw

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: fix GET website response error code

Change NoSuchKey error code to NoSuchWebsiteConfiguration, when bucket doesn't have website configuration.

Fixes: http://tracker.ceph.com/issues/22272
Signed-off-by: Dmitry Plyakin <dplyakin@gmail.com>

cmake: link executables against ceph-common

so they can have access to libstdc++ if it is linked statically.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add WITH_STATIC_LIBSTDCXX option

to link libceph-common with libstdc++ if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>