ceph-volume lvm.zap loop over all vgs from a pv to destroy them

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.util ensure dmcrypt_close will skip non-existent paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.encryption if crypt path does not exist, skip it

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.zap unmount lvs and close dmcrypt on partitions

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional batch add filestore sceanrios to tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.zap iterate over multiple lvs on a pv

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add xenial batch tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add bluestore,filestore playbooks

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add centos7 filestore batch tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add centos7 bluestore batch tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.util assert new as_int,as_float helper methods

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests verify new utility for journal sizes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.prepare add a helper to get journal sizes from ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.disk add as_int and as_float helpers for Size objects

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.strategies fix a bug where incorrect template was used

The template wouldn't render the total OSDs and the removed section was
repeated

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.strategies use get_journal_size helper for filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.strategies update filestore use of Size objects

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.strategies update bluestore use of Size

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.strategies create single type filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.util verify that Size object don't change state

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.util update to reflect Size object changes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.disk do not modify self instance of Size objects

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.strategies use new template location in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util create a templates module

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados update journal size defaults to 5GB

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22967 from linuxbox2/wip-slo-nosize

rgw: Swift SLO size_bytes member is optional

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23056 from stiopaa1/fixRgwAdminUser

rgw: fix dependencies/target_link_libraries

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #23282 from zhangsw/fix-rgw-server-header

rgw: add Http header 'Server' in response headers.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23405 from joke-lee/WIP-RGW-BUCKET-NOTIFY

rgw: add s3 notification sub resources

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23434 from joke-lee/wip-cors-rule-num-limit

rgw: cors rules num limit

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #23210 from alfredodeza/wip-bluestore-sizing

doc: document sizing for `block.db`

Reviewed-by: Ben England <bengland@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23511 from excellentkf/WIP-test

common/options: correct the description of rgw_enable_lc_threads option

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23496 from bstillwell/crush_doc_fixes

doc: Fix a couple typos and improve diagram formatting

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23477 from liewegas/wip-kv-merge-name

kv/KeyValueDB: return const char* from MergeOperator::name()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23474 from tchaikov/wip-config-expand-meta

common: expand meta in parse_argv()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23481 from Yan-waller/wip-walle-180808-smooth-intervals

mgr,mon: fix to apply changed mon_stat_smooth_intervals

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #23482 from Yan-waller/wip-walle-180808-mgr-deadlock

mgr: fix beacon interruption caused by deadlock

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23487 from xiexingguo/wip-hb-peers-v2

osd/OSD: choose heartbeat peers more carefully

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yan Jun <yan.jun8@zte.com.cn>

Merge pull request #23514 from bstillwell/operations_doc_fixes

doc: Multiple spelling fixes

Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: Multiple spelling fixes

I ran a lot of the docs through aspell and found a number of spelling problems.

Signed-off-by: Bryan Stillwell <bstillwell@godaddy.com>

Merge PR #23471 into master

* refs/pull/23471/head:
mon/PGMap: fix spacing around pretty-printed SI units
include/types: render SI units adjacent to number

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge PR #23473 into master

* refs/pull/23473/head:
OSD: translate the error result

Reviewed-by: Sage Weil <sage@redhat.com>

doc: Fix a couple typos and improve diagram formatting

I found a couple misspelled words in the crush-map documentation, and also
tweaked the formatting of the CRUSH hierarchy diagram to center some of the
entries.

Signed-off-by: Bryan Stillwell <bstillwell@godaddy.com>

Merge PR #23422 into master

* refs/pull/23422/head:
log/Log: fix buffer overflow

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/options: correct the description of rgw_enable_lc_threads option

Signed-off-by: excellentkf <likefei@cmss.chinamobile.com>

Merge pull request #23466 from tspmelo/wip-update-packages

mgr/dashboard: Update frontend packages

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #23431 from falcon78921/wip-doc-rados-pool

doc: fixed links in Pools section

Reviewed-by: John Wilkins <jowilkin@redhat.com>

osd/OSD.cc: force updating heartbeat peers periodically

Because the cluster topology may change (e.g., because we add some
new racks, hosts and disks) and we want the existing osds are then
aware of new incoming osds, guaranteeing osds are always trying to
do heartbeat as wide as possible(e.g., across all racks, hosts etc.).

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush/CrushWrapper: check against dead crush rule

We currently remove a rule without adjusting the **rules** array
because we don't recycle the rule_no and hence there can be
holes in the "rules" array.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSD.cc: choose heartbeat peers by failure domain

By default, monitor requires at least two valid failure votes/reports from
different hosts to mark an OSD down, which turns out to be impossible sometimes
for a replicated-pool of size of 2 in those clusters made up of hosts
with contiguous labeled OSDs.

This patch instead does a breadth-first search based on the highest level of
failure domain at cluster-wide, to try to make heartbeat peers can cover all failure domains
whenever possible, which can hopefully help accelerating osd failure detection
in the above case..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23441 from tchaikov/wip-cmake-rocksdb-cleanup

cmake: rocksdb related cleanup

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #23462 into master

* refs/pull/23462/head:
doc: cleanup in cephfs/capabilities

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mon/PGMap: fix spacing around pretty-printed SI units

"1k foo" not "1kfoo"

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #23489 from alfredodeza/wip-cv-mgr

ceph-volume tests/functional add mgrs daemons to lvm tests

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #23475 into master

* refs/pull/23475/head:
qa/standalone/crush/crush-choose-args: run mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #23478 into master

* refs/pull/23478/head:
mgr/localpool: pg_num is an int arg to 'osd pool create'

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23267 from ifed01/wip-ifed-rm-fio-dir

test/fio:  pass config params to object store in a different manner

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-volume tests/functional add mgrs daemons to lvm tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

crush/CrushWrapper: get_children_of_type - switch outcome to vector

Because we don't really care about the order.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mgr: fix beacon interruption caused by deadlock

There is potential deadlock between this code and DaemonServer::send_report()

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

mgr: fix to apply changed mon_stat_smooth_intervals

While using config set or injectargs command to decrease
mon_stat_smooth_intervals, it didn't apply the new value.
this change fix it.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

mgr/localpool: pg_num is an int arg to 'osd pool create'

In bd565bca3f42d7946c392b2833bb3d9982a8ac5d we start returning EINVAL if
the pg_num argument is not valid, whereas before it was silently ignored.

Signed-off-by: Sage Weil <sage@redhat.com>

kv/KeyValueDB: return const char* from MergeOperator::name()

This avoids a c_str() on a temporary string later down the line, which
avoids a use-after-free.

Fixes: http://tracker.ceph.com/issues/26875
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #23472 from jcsp/wip-orchestrator-rook-doc

doc: Rook/orchestrator doc fixes

Reviewed-by: Alfredo Deza <adeza@redhat.com>

qa/standalone/crush/crush-choose-args: run mgr

The osd purge command needs a running mgr.

Fixes: d2b41d4095074447df247907934626f4287cd91b
Signed-off-by: Sage Weil <sage@redhat.com>

common: expand meta in parse_argv()

it fixes the issue where the conf->foo fails to expand if it is set
as "$bar" in a .conf file, and "bar" is another config var, and bar
is set using config_proxy.parse_argv().

because before 773ef117, apply_changes() calls update_legacy_vals(),
which expand meta vars for legacy config vals, but after 773ef117,
we cannot change config_values in apply_changes() anymore. so
update_legacy_vals() calls is moved to apply_changes()'s callers
accordingly. but the apply_changes() is also called by global_init()
directly instead of by the routines in config.cc. so in 773ef117,
i missed it.

in this change, it's put back into parse_argv() instead of into
global_init(), as update_legacy_vals() is a low level function, and
parse_argv() could change the config_values, it's better to call
update_legacy_vals() out there, just like set_mon_vals() and
parse_config_files().

Signed-off-by: Kefu Chai <kchai@redhat.com>

OSD: translate the error result

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

test/fio: pass config params to object store in a different manner

This avoids an existing bug with config params substituion.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

doc: Rook/orchestrator doc fixes

Signed-off-by: John Spray <john.spray@redhat.com>

Merge PR #23432 into master

* refs/pull/23432/head:
Revert "rocksdb: pickup change to link against libsnappy.a"
cmake,make-dist: revert "build gperftools if WITH_STATIC_LIBSTDCXX"
cmake: cleanup snappy related script
debian: strip "-Wl,-Bsymbolic-functions" from LDFLAGS

Merge PR #23243 into master

* refs/pull/23243/head:
mon/OSDMonitor: remove ambiguous type case for 'osd pool set' value
mon: use throwing variant of cmd_getval
common/cmdparse: add cmd_getval_throws variants that throw bad_cmd_get
mon: catch bad_cmd_exception and reply EINVAL
common/cmdparse: add bad_cmd_get exception

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23338 into master

* refs/pull/23338/head:
msg/Messenger: send_message -> send_to
msg/msg_types: add entity_addr_t::legacy_or_front_addr()
mon/MonMap: kill get_inst()
mon: kill MonMap::get_inst() users
mon/MonClient: replace get_mon_{addr,inst} with get_mon_addrs
mon/MonClient: get_connection -> connect_to_mon
src/test: get_connection -> connect_to_*
mon/MonMap: add get_addrs() (temp wrappers)
msg/Messenger: make connect_to() key method, get_connection() becomes wrapper

Merge PR #23412 into master

* refs/pull/23412/head:
mgr: add Orchestrator.available method + CLI
doc: add general documentation on orchestrator
doc: add page for Rook module
doc: add development guide for Ceph+Rook in k8s
script: add kubejacker script
mgr/rook: Rook orchestrator module
mgr/orchestrator_cli: module for calling into orchestators
mgr: introduce `orchestrator` python interface

Reviewed-by: Sage Weil <sage@redhat.com>

include/types: render SI units adjacent to number

Make

 out << si_u_t(foo) << " objects"

result in strings like

 "0 objects"
 "123k objects"
 "3M objects"

instead of

 "0  objects"
 "0 k objects"
 "3 M objects"

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #23430 from tchaikov/wip-silence-clang-warnings

tools, mgr: silence clang warnings

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #23420 from tspmelo/fix-24902

mgr/dashboard: Escape regex pattern in DeletionModalComponent

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #23414 from tspmelo/wip-breadcrumbs

mgr/dashboard: Add breadcrumbs component

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #23465 from ukernel/wip-revert-22850

Revert "cephfs-journal-tool: enable purge_queue journal's event comma…

mgr/dashboard: Update prettier package

Prettier updated the following rule, that implied we update our code:
"Never inline decorators unless they’re lone parameter decorators"

Signed-off-by: Tiago Melo <tmelo@suse.com>

Revert "cephfs-journal-tool: enable purge_queue journal's event commands"

This reverts commit 8ebfa93982a95d6ccc66faa2a828b9386af45238. That
commit changes PurgeItem's encode/decode function, breaks backward
compatibility

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mgr/dashboard: Update Codelyzer package

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update all @angular packages

no-unused-variable is deprecated because typescript now covers most of its
functionality.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update frontend packages

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #22562 from wjwithagen/wjw-fix-dashboard-build

mgr/dashboard: Fix building under FreeBSD

Reviewed-by: Tiago Melo <tmelo@suse.com>

doc: cleanup in cephfs/capabilities

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #23450 from neha-ojha/wip-fix-complete-to

osd/PGLog.cc: check if complete_to points to log.end()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

FreeBSD: update build script to current state

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #23456 from yuriw/wip-yuriw-crontab-master

qa/tests: changed ceph qa email address to bypass dreamhost's spam filter

qa/tests: changed ceph qa email address to bypass dreamhost's spam filter

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #23448 from alfredodeza/wip-rm26862

ceph-volume: `lvm batch` allow extra flags (like dmcrypt) for bluestore

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #23433 into master

* refs/pull/23433/head:
client: fix compile warning

Reviewed-by: Zheng Yan <zyan@redhat.com>

doc/bluestore detail the different sizing strategies for block.db based on cluster workload

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/bluestore expand on configuration with fast and slow devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge PR #22850 into master

* refs/pull/22850/head:
cephfs-journal-tool: enable purge_queue journal's event commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22999 into master

* refs/pull/22999/head:
mds: consider max age of dispatch queue when finding stale client
mds: reset heartbeat map at potential time-consuming places
mds: change MDSRank::finished_queue to deque

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23341 into master

* refs/pull/23341/head:
mds: calculate load by checking self CPU usage

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23370 into master

* refs/pull/23370/head:
client: check for unmounted condition before printing debug output

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #23387 from jcsp/wip-mgr-rw

mgr: fix permissions on `balancer execute`

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23205 from jcsp/wip-devicehealth-cleanup

mgr/devicehealth: clean up error handling

Reviewed-by: Sage Weil <sage@redhat.com>

osd/PGLog.cc: check if complete_to points to log.end()

Signed-off-by: Neha Ojha <nojha@redhat.com>