Merge pull request #19317 from shinobu-x/journaler_trivial_cleanup

journal: trivial cleanup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19406 from trociny/wip-deep-copy-pybind

pybind/rbd: add deep_copy method

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19436 from trociny/wip-22333

rbd-nbd: support optionally setting device timeout

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19477 from wido/mgr-int-json

ceph-volume: Try to cast OSD metadata to int while scanning directory

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #19516 from tchaikov/wip-install-deps

install-deps.sh: use tee for writing a file

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #19517 from qrGitHub/wip-rgw-typo-signle

rgw: fix typo signle to single

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18716 from shinobu-x/sk-mgr_daemonserver

mgr/DaemonServer: [Cleanup] Remove redundant code

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

rgw: fix typo signle to single

Signed-off-by: Bingyin Zhang <zhangbingyin@cloudin.cn>

install-deps.sh: use tee for writing a file

because the stdout redirect is performed by current shell not `$SUDO
cat`, so $SUDO does not help here. use `$SUDO tee` instead, which is
able to read the stdin and write to the given file just as expected.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19492 from dillaman/wip-rbd-mirror-dashboard

rbd-mirror: ensure unique service daemon name is utilized

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #19470 from tchaikov/wip-async-dpdk

msg/async: update to work with dpdk shipped with spdk v17.10

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #19466 from shinobu-x/super_redundant_public_label

osdc: Remove a bit too redundant public label...

Merge pull request #19494 from smithfarm/wip-22362

build/ops: rpm: set permissions 0755 on rbd resource agent

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19495 from ZVampirEM77/wip-em-pg-cleanup2

osd/PrimaryLogPG: cleanup do_sub_op && do_sub_op_reply and define soid in prepare_transaction more appropriate

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19425 from kungf/calc_replicated_acting

osd: reduce all_info map find to get primary

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

Merge pull request #17691 from amitkumar50/cov-osd-3

osd: Initialization of data members

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #19033 from david-z/wip-fix-fusestore-df

os/FuseStore: fix incorrect used space statistics for fuse's statfs interface

Reviewed-by: Sage Weil <sage@redhat.com>

build/ops: rpm: set permissions 0755 on rbd resource agent

Fixes: http://tracker.ceph.com/issues/22362
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19468 from tchaikov/wip-install-deps.sh

install-deps.sh: avoid re-installing g++-7

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

mgr/dashboard: use unique instance id as daemon identifier

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: utilize application metadata to detect RBD pools

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: ensure each daemon instance uses a unique service name

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

PrimaryLogPG: cleanup do_sub_op and do_sub_op_reply

The related code has been cleaned in https://github.com/ceph/ceph/pull/17247,
but it seems something was missed.

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

PrimaryLogPG: define soid just before using it

Define the soid just before using it to avoid a redundant
definition if an error happens and function prepare_transaction
returns directly.

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

Merge pull request #19453 from liewegas/wip-pg-primary

osd/PG: include primary in PG operator<< for ec pools

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19478 from tchaikov/wip-silence-warnings

qa/standalone/ceph-helpers.sh: silence ceph-disk DEPRECATION_WARNING

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19404 from jan--f/mon-fix-digest-timer-inactive-paxos

mon: reenable timer to send digest when paxos is temporarily inactive

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19366 from liewegas/wip-21557

osd: more debugging for snapmapper bug

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19367 from wjwithagen/wip-wjw-clang-xlist-string

include: xlist: Fix Clang error for missing string

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19365 from liewegas/wip-tell

qa/tasks/ceph_manager: tolerate tell osd.* error

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19353 from shinobu-x/common_clock

common: Remove ceph_clock_gettime, extern keyword

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19352 from chardan/jfw-wip-issue-20811

osd: migrate PGLOG_* macros to constexpr

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19329 from ZVampirEM77/cleanup-revise-have

osd/PGLog: cleanup unused function revise_have

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19056 from adamemerson/wip-with-val

common/config: use with_val() for better performance

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19114 from tchaikov/wip-rm-request-slow

mgr/PGMap: drop REQUEST_{SLOW,STUCK} HEALTH_WARNs

Reviewed-by: John Spray <john.spray@redhat.com>

qa/standalone/ceph-helpers.sh: silence ceph-disk DEPRECATION_WARNING

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19229 from wido/mgr-influx

mgr/influx: PEP-8 and other fixes to Influx module

Reviewed-by: John Spray <john.spray@redhat.com>

ceph-volume: Try to cast OSD metadata to int while scanning directory

By doing so values like 'whoami' and 'bluefs' will be stored as a
integer in the resulting JSON rather then a String.

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #19472 from qrGitHub/wip-rgw-typo-woild

rgw: fix typo woild to would

Reviewed-by: Jos Collin <jcollin@redhat.com>

rgw: correct typo woild to would

Signed-off-by: Bingyin Zhang <zhangbingyin@cloudin.cn>

msg/async: update to work with dpdk shipped with spdk v17.10

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: DPDK works w/o CRYPTOPP

Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: avoid re-installing g++-7

* add ubuntu-toolchain-r mirrors in case the ppa.launchpad.net is not
  accessible
* add ppa repo manually, it's faster than installing
  `software-properties-common` and then launch `add-apt-repository`
* hardwire $old to the gcc version shipped by the distro, simpler this
  way.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osdc: Remove a bit too redundant public label...

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

Merge pull request #17651 from ceph/wip-ceph-deploy-upgrade

qa/tests: Wip ceph deploy upgrade

Merge pull request #19144 from zhangsw/cleanup-rgwputobj-md5

rgw: no need to deal with md5 header in get_data.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19167 from joke-lee/Bucket_Policy_SourceIp_Condition

rgw: implement ipv4 aws:SourceIp condition for bucket policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19210 from Liuchang0812/rgw-camelcase

 rgw: use camelcase format in request headers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #19236 from dplyakin/get-website-error-code

rgw: fix GET website response error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19250 from qrGitHub/wip-rgw-unused-param2

rgw: remove unused param in AWSGeneralAbstractor::get_auth_data_v4

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #19390 from yaozongyou/remove-duplicated-include

rgw,common: remove already included header files

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #19391 from qrGitHub/wip-rgw-memory-optimize

rgw: optimize memory usage in function rgw_bucket::get_key

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19430 from tianshan/fix_recursive_lock

rgw: fix recursive lock

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd/PG: include primary in PG operator<< for ec pools

Otherwise it is confusing!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #19418 from tchaikov/wip-gcc-7-on-xenial

install-deps.sh, cmake: use GCC-7 on xenial also

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19402 from b-ranto/wip-fix-arm-build

test/librbd: Do not instantiate TrimRequest template class

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19437 from trociny/wip-mock-valgrind

test/rbd_mirror: "use of uninitialised value" valgrind warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19253 from oritwas/wip-rgw-22124

rgw: reshard should not update stats when linking new bucket instance

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rbd-nbd: support optionally setting device timeout

Fixes: http://tracker.ceph.com/issues/22333
Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

Merge remote-tracking branch 'gh/mimic-dev1'

Merge pull request #19435 from smithfarm/wip-20559

doc: globally change CRUSH ruleset to CRUSH rule

Reviewed-by: Sage Weil <sage@redhat.com>

librbd: Do not instantiate TrimRequest template class

We include TrimRequest.cc in librbd tests at two places:
 - operation/test_mock_TrimRequest.cc
 - operation/test_mock_ResizeRequest.cc

That causes linking errors when doing the builds because some of the
instantiated classes are defined twice.

We can fix this by not instantiating the template class in the
TrimReqeust.cc file when including it in the tests.

Signed-off-by: Boris Ranto <branto@redhat.com>

test/rbd_mirror: "use of uninitialised value" valgrind warning

The `on_call` context serves as a barrier and should be completed
after the `on_start_ctx` context is assigned.

The warning was observed sporadically e.g. by repeating
WaitingOnNonLeaderAcquireLeader test under valgrind.

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #19191 from tchaikov/wip-mon-cleanup

mon: remove_is_write_ready()

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #17438 from mikulely/fix-mstop

vstart.sh: fix mstop.sh can not stop rgw

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: fix GET website response error code

Change NoSuchKey error code to NoSuchWebsiteConfiguration, when bucket doesn't have website configuration.

Fixes: http://tracker.ceph.com/issues/22272
Signed-off-by: Dmitry Plyakin <dplyakin@gmail.com>

cmake: link executables against ceph-common

so they can have access to libstdc++ if it is linked statically.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add WITH_STATIC_LIBSTDCXX option

to link libceph-common with libstdc++ if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #19432 into master

* refs/pull/19432/head:
doc: correct the new fs allow opts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: globally change CRUSH ruleset to CRUSH rule

Since kraken, Ceph enforces a 1:1 correspondence between CRUSH ruleset and
CRUSH rule, so effectively ruleset and rule are the same thing, although
the term "ruleset" still survives - notably in the CRUSH rule itself, where it
effectively denotes the number of the rule.

This commit updates the documentation to more faithfully reflect the current
state of the code.

Fixes: http://tracker.ceph.com/issues/20559
Signed-off-by: Nathan Cutler <ncutler@suse.com>

pybind/rbd: add deep_copy method

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

install-deps.sh: use GCC-7 on xenial also

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19416 from trociny/wip-flatten-cleanup

librbd: remove unused member in FlattenRequest

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/influx: Print (in debug) time it takes to send data to Influx

On large clusters it might take a very long time to send data to Influx
due to the gathering and parsing of statistics.

By keeping a counter and printing it admins can adjust the interval if it
becomes to heavy for their cluster.

Signed-off-by: Wido den Hollander <wido@42on.com>

mgr/influx: Revise configuration of module

It's a lot like the Zabbix module and allows for setting configuration
options on run-time and also fetch them from the module.

A few additional commands have been registered to make sure it is easy
to interact with the module.

Signed-off-by: Wido den Hollander <wido@42on.com>

mgr/influx: Send fsid as a tag to Influx to identify cluster

This allows for multiple Ceph clusters to send their data to the
same Influx database.

Using the fsid values for different clusters can be queried from
Influx

Signed-off-by: Wido den Hollander <wido@42on.com>

mgr/influx: PEP-8 fixes to InfluxDB module

The code now scores:

  Your code has been rated at 8.94/10

Various indentation fixes, whitespaces and other PEP-8 related changes

Signed-off-by: Wido den Hollander <wido@42on.com>

doc: correct the new fs allow opts

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

rgw: fix recursive lock

Signed-off-by: Tianshan Qu <tianshan@xsky.com>

Merge pull request #19426 from tchaikov/wip-cmake

cmake: disable FAIL_ON_WARNINGS for rocksdb

cmake: disable FAIL_ON_WARNINGS for rocksdb

otherwise -Werror=implicit-fallthrough will fail the build with GCC-7

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: reduce all_info map find to get primary

Signed-off-by: kungf <yang.wang@easystack.cn>

Merge PR #19373 into master

* refs/pull/19373/head:
client: drop duplicate friend

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19417 from tchaikov/wip-install-deps-gcc

install-deps.sh: install new gcc as the default the right way

Reviewed-by: Sage Weil <sage@redhat.com>

qa/suites/rados: add missing openstack volumes

Signed-off-by: Sage Weil <sage@redhat.com>

install-deps.sh: install new gcc as the default the right way

* should install software-properties-common beforehand, otherwise
  the `add-apt-repository` command will not be available.
* the update-alternative commandline were copied from ceph-build,
  should remove the escape characters.

Signed-off-by: Kefu Chai <kchai@redhat.com>

librbd: remove unused member in FlattenRequest

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #19398 from tchaikov/wip-install-deps-use-dts

install-deps.sh: use DTS on centos if GCC is too old

Reviewed-by: Sage Weil <sage@redhat.com>

journal: Trivial cleanup

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

Merge pull request #19368 from dzafman/wip-22086

ceph-objectstore-tool: Add option "dump-import" to examine an export

Reviewed-by: Sage Weil <sage@redhat.com>

ceph.spec: use dts-7 on rhel

devtoolset-7 is available for RHEL, but not for centos. only dts-6 is
available for centos.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: ceph_objectstore_tool.py: Perform dump-import

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #19382 from rishabh-d-dave/docfix-ceph-osd-ls

doc/man: add "ls" to "ceph osd" command's subcommands list

osd: migrate PGLOG_* macros to constexpr

Fixes: http://tracker.ceph.com/issues/20811
Signed-off-by: Jesse Williamson <jwilliamson@suse.de>

mon: reenable timer to send digest when paxos is temporarily inactive

Fixes: http://tracker.ceph.com/issues/22142
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #19005 from shinobu-x/src_krbd

rbd: drop unnecessary using declaration, etc

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

install-deps.sh: enable testing repo on centos on aarch64

the DTS-6 is only available in the testing repo

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19338 from vshankar/rbd-mirror-image-map-timestamp

rbd-mirror: persist image map timestamp

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19383 from trociny/wip-snapshot-copy

librbd: deep_copy: don't create snapshots above snap_id_end

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

install-deps.sh: use DTS on centos if GCC is too old

please note, run-make-check.sh sources install-deps.sh here to import
the $PATH and other environmental variables, which could be changed by
the the DTS "enable" script.

Signed-off-by: Kefu Chai <kchai@redhat.com>