common: get_str_vec and friends use for_each_substr

Signed-off-by: Casey Bodley <cbodley@redhat.com>

common: add for_each_substr() for cheap string split

using boost::string_view avoids copies (and potential allocation) of
the substrings

using a callback model avoids baking in the container type, along with
the copies and allocations required for insertion

this interface is ideal for cases where you just want to iterate over
the substrings, and don't actually need to store them in a container

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/common: extend str_list tests to include set

Signed-off-by: Casey Bodley <cbodley@redhat.com>

common: add simplified interfaces to get_str_*

the simpler interfaces rely on return value optimization to avoid
copying the result. removing the container from the argument list makes
it easier to provide a default argument for 'delims'

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18823 from ceph/wip-bz1498200

ceph-volume: support GPT and other deployed OSDs

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #18817 from ZVampirEM77/wip-em-zone-cleanup

rgw cleanup: some unnecessary function called and repeated assignment

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18683 from tchaikov/wip-erasure-cleanup

erasure-code: refactor the interfaces to hide internals from public

Reviewed-by: Sage Weil <sage@redhat.com>

doc/ceph-volume include the top level systemd.rst in the toctree

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.activate add --file in help docstring as an example

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume activate has a --file option for json files

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.activate enable the osd when activating

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume systemd.systemctl always force symlink when masking ceph-disk

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.activate use the contents of the JSON file, fallback to CLI args

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.activate only check for missing id and uuid if file is not passed in

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.activate allow to pass a file to activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.util add tests for OSDPath validator

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume systemd.systemctl masking ceph-disk needs to be done by directly symlinking to dev/null

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume systemd.systemctl masking needs to use systemctl mask, not disable

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.scan if mounted device cannot be detected, raise an error

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.activate remove str formatting from logging

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.arg_validators always convert the path to an absolute path

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #18781 from iliul/correct-error-messages

rgw/rgw_op.cc: Fix error message in rgw_user_get_all_buckets_stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18796 from ivancich/wip-fix-fmt-warnings

rgw: address warnings due to incorrect format code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #17659 from majianpeng/osd-wait-ratherthan-waitinterval

osd/OSD: Using Wait rather than WaitInterval to wait queue.is_empty().

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18759 from tchaikov/wip-mon-log-last

mon/LogMonitor: "log last" should return up to n entries

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18779 from tchaikov/wip-osd-remove-unused-objects_read_async

osd: remove unused ReplicatedBackend::objects_read_async()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Xinze Chi <xinze@xsky.com>

Merge pull request #18809 from tchaikov/wip-21904

mgr/zabbix: ignore osd with 0 kb capacity

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Wido den Hollander <wido@42on.com>

rgw/rgw_op.cc: Fix error message in rgw_user_get_all_buckets_stats

Signed-off-by: iliul <liul.stone@gmail.com>

ceph-volume util.arg_validators require superuser privs to check paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.scan skip directories to scan

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.devices.simple add activate CLI unit tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.devices add help menu unit test

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume systemd describe the interaction with 'simple' activation

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume index initial description of simple

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume lvm add back a system.rst with just the lvm portion of it

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume make systemd.rst generic enough to be related to other sub-commands

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume move lvm/systemd.rst to top level, making it generic

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume simple.activate initial take on documenting activation

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume lvm update systemd anchor to reflect lvm parent

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume lvm.activate update systemd anchor to reflect lvm parent

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume lvm.systemd update anchor to reflect lvm parent

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume document the scan functionality

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #18793 from idryomov/wip-qa-cram-path

qa/tasks/cram: include /usr/sbin in the PATH for all commands

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18556 from joke-lee/validate_cors_rule_header_in_cors_option_request

rgw: add cors header rule check in cors option request

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18795 from idryomov/wip-krbd-enable-generic-050-448

qa/suites/krbd: enable generic/050 and generic/448

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18706 from cbodley/wip-rgw-policy-emplace

rgw: simplify use of map::emplace in iam

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #18770 from tchaikov/wip-doc-log-file

doc: fix desc of option "mon cluster log file"

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: cleanup repeated assignment to need_zone_update in zone modify command

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

doc: fix desc of option "mon cluster log file"

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/zabbix: ignore osd with 0 kb capacity

Fixes: http://tracker.ceph.com/issues/21904
Signed-off-by: Ilja Slepnev <islepnev@gmail.com>

Merge PR #18807 into master

* refs/pull/18807/head:
ptl-tool: put common use-cases up front in README
ptl-tool: only create tag if new branch
ptl-tool: hint how to skip contributor

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ptl-tool: put common use-cases up front in README

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

ptl-tool: only create tag if new branch

It doesn't make sense to create tags for branches that already exist.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

ptl-tool: hint how to skip contributor

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

rgw: address warnings due to incorrect format code

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #18709 from ivancich/wip-bktidx-race-bugfix

rgw: fix BZ 1500904, stale bucket index entry remains after obj delete

qa/suites/krbd: enable generic/050 and generic/448

generic/448 has been failing on ext4 and is now fixed upstream.
The fix for generic/050 is in testing, scheduled for 4.15-rc1.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #18476 from majianpeng/objecter-misc-cleanup

osdc/Objecter: misc cleanups

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18768 from wumingqiao/wip-WaitActingChange

PG: fix name of WaitActingChange

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18789 from jcsp/wip-doc-config-set

doc: replace injectargs usage with "config set"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18769 from yaozongyou/wip-dashboard-format-label

mgr/dashboard: format tooltip's label as user friendly string

Reviewed-by: John Spray <john.spray@redhat.com>

doc: replace injectargs usage with "config set"

Cleaner and easier.  Also implicitly documents
the config set command, which hadn't been explicitly
called out in the docs before.

Signed-off-by: John Spray <john.spray@redhat.com>

osd: use is_erasure() when appropriate

sometimes, we are using pool.info.require_rollback() even when what we
mean is "pool.info.is_erasure()", so fix it to improve the readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: remove unused ReplicatedBackend::objects_read_async()

async read is not used by replica pool at all.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #18689 into master

* refs/pull/18689/head:
cephfs: Do not check auth gid when not specified

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18386 into master

* refs/pull/18386/head:
ptl-tool: allow setting project/repo for GH req

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18658 from linuxbox2/wip-s3-post-20201

RGW: update S3 POST policy handling of Content-Type

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc/ceph-volume update the index to include the simple sub command and files

Signed-off-by: Alfredo Deza <adeza@redhat.com>

qa/tasks/cram: include /usr/sbin in the PATH for all commands

/usr/sbin is not in non-login PATH on centos.  We already do this for
workunits, see commit 0e53f5f38b0c ("workunit: include /usr/sbin in the
PATH for all commands").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

ceph-volume tests.simple add tests for trigger parsing from systemd

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.simple create etc_path scan tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.simple add checks for get_contents in scan

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.util add binary verification tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests create a fixture for temporary files

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume systemd.systemctl helpers to mask ceph-disk

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume devices include simple at the module import level

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume main add simple to the sub-commands

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util add an OSDPath argument validator

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.system add a helper to check if a file is a binary or not

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.system add a context manager for temporary mounting/unmounting

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.scan initial take on directory/device scanning

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.trigger enable systemd triggering for simple setups

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.activate initial take on activation

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple add __init__ for the module

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple create the initial, main module

Signed-off-by: Alfredo Deza <adeza@redhat.com>

mgr/dashboard: format chart's tooltip's label as friendly string

The default tooltip's label is a long number string, so format
the label as a user friendly string.

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>

common/WorkQueue: fix a race avoiding drain endless.

In function ShardedThreadPool::shardedthreadpool_worker, when call
_proces, it don't get shardedpool_lock. So there is a race between
return_waiting_threads and _proces. If return_waiting_threads first run
and _process will lost this signal and wait until.
This may cause ShardedThreadPool::drain can't complete.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #16238 from trociny/wip-copy-deep

librbd: generalized deep copy function

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

PG: fix name of WaitActingChange

the parent state of WaitActingChange is Primary, not Peering.

Signed-off-by: wumingqiao <wumingqiao@inspur.com>

osd/OSD: Using Wait rather than WaitInterval to wait queue.is_empty().

Why use WaitInterval, there is a comment:"optimistically sleep a moment; maybe another work item will come along."
But in fact,we don't see any benefit for this optimization.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

rgw: cleanup some unnecessary function called in RGWZoneGroup::add_zone

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

qa/suites/rbd: enable fsx deep_copy test case

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

test/librbd: tweak fsx to optionally use deep-copy instead of clone

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

librbd: add deep_copy function to API

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

rbd-mirror: use generalized deep copy for image sync

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: generalized deep copy function

(based on rbd-mirror image sync)

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: export ImageOptions print operator

(so it could be used for logging image options)

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

test/librbd: set rand seed for all tests

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

mon/LogMonitor: "log last" should return up to n entries

limit the # of returned entries to "num", and backoff the start iterator
by one if it reaches the rend().

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18635 from majianpeng/bluestore-omap-flush

os/bluestore: remove unused parameters.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>