qa/workunits/rados/test.sh: ensure tee output is valid filename

In the future we'll add --gtest_filter bits to the items, which confuse
tee.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21449 from rjfd/wip-dashboard-fix-frontend-build

cmake: fix frontend cmake build

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21360 from rjfd/wip-dashboard-rbd

mgr/dashboard: RBD management (backend)

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21445 from rjfd/wip-dashboard-fix-run-tox

mgr/dashboard: run-tox: pass CEPH_BUILD_DIR value into tox script

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: fix frontend cmake build

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20929 from JianyuLi/joa-bluestore-spdk

os/bluestore: free the spdk qpair resource correctly in destructor of SharedDriverQueueData

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21419 from ceph/wip-da-SCA-20180329

common,osd,rgw: Fixes for issues found during SCA

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21429 from wjwithagen/wjw-kStore-statfs

KStore: statfs needs extra includes on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21431 from Songweibin/wip-ec-cleanups

osd: miscellaneous cleanups

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: run-tox: pass CEPH_BUILD_DIR value into tox script

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21170 from dragonylffly/wip-revise-organizationmap

.organizationmap: add Xin Yuan and Yichao Li's affiliation

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21433 from tchaikov/wip-qa-env_librados_test

qa: disable -Werror when compiling env_librados_test

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21402 from tspmelo/wip-module-cleanup

mgr/dashboard: Angular modules cleanup

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile vtheile@suse.com

Merge pull request #21398 from ivancich/wip-revert-dmclock-feature-bit-plus

osd: Revert use of dmclock message feature bit since not yet finalized

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21403 from dillaman/wip-rbd-mirror-dead-instances

test/rbd-mirror: improve coverage for dead instance handling

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #21322 into master

* refs/pull/21322/head:
osdc: cleanup code.
osdc: Remove useless condition.

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge PR #21341 into master

* refs/pull/21341/head:
client: Fix the gid_count check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21370 into master

* refs/pull/21370/head:
client: Take reference for operator= function parameter

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #21391 into master

* refs/pull/21391/head:
qa: fix log typo

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #21396 into master

* refs/pull/21396/head:
ceph-fuse: exit with failure on failed mount
common: ignore errors during preforker exit
client: do not overload system errnos

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #20398 from wjwithagen/wip-posix_fallocate

common: posix_fallocate on ZFS returns EINVAL

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20752 from jcsp/wip-dashboard-build-tweak

mgr/dashboard: build tweaks

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21108 from xiexingguo/wip-kill-subset-of

interval_set: kill subset_of()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21369 from majianpeng/test-objectstore_bench-usage

test/objectstore_bench: Don't forget judging whether call usage.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20100 from mogeb/iostat-plugin

mgr/iostat: implement 'ceph iostat' as a mgr plugin

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21373 from ovh/bp-hb-divbyzero

osd: fix crash caused by divide by zero in heartbeat code

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21395 from liewegas/wip-log-channels

mon/LogMonitor: separate out summary by channel

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19935 from myoungwon/wip-manifest-ref-count

osd: refcount for manifest object (redirect, chunked)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19117 from jan--f/jan-object-counts-decimal

include/types: format decimal numbers with decimal factor

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21427 from liewegas/wip-balancer-roots

mgr/balancer: improve error message

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #21401 from liewegas/wip-mgr-help

ceph-mgr: exit after usage

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #21430 from tchaikov/wip-osdmap-prune

mon/OSDMonitor: txsize should be greater or eq to prune_interval - 1

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

qa: disable -Werror when compiling env_librados_test

to silence warnings like

utilities/env_librados.cc:175:33: warning: unused parameter ‘offset’ [-Wunused-parameter]
   Status InvalidateCache(size_t offset, size_t length) {
                                 ^~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/OSDMonitor: txsize should be greater or eq to prune_interval - 1

so we can prune 1 or more intervals in a single iteration. and in each
interval, we prune (prune_interval - 1) versions of osdmap.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/OSD: drop unused header

Signed-off-by: songweibin <song.weibin@zte.com.cn>

osd/ECBackend: kill redundant assert

Signed-off-by: songweibin <song.weibin@zte.com.cn>

osd/ECBackend: s/union_of/union_insert/

Signed-off-by: songweibin <song.weibin@zte.com.cn>

osd/ECTransaction: drop duplicated **#include <vector>**

Signed-off-by: songweibin <song.weibin@zte.com.cn>

osd/ECTransaction: kill redudant asserts

Signed-off-by: songweibin <song.weibin@zte.com.cn>

osd/ECBackend: kill extra member **want**

Signed-off-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #21404 from tchaikov/wip-cmake-boost

cmake: changes of BuildBoost.cmake to ready seastar

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21410 from badone/wip-test_large_omap_detection-scrub-pgs-instead-of-osds

qa/workunits/rados/test_large_omap_detection: Scrub pgs instead of OSDs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

KStore: statfs needs extra includes on FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

mgr/balancer: identify roots that osds overlap in

This error message isn't very informative; tell the user which osds
appear in mulitple roots, and which roots those are!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20853 from theanalyst/rgw/recalc-stats

rgw: add an option to recalculate user stats

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

test_rgw_admin_log.cc: lenght() returns unsigned

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/rgw: fix some format qualifiers

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/OSDMonitor.cc: fix format qualifiers

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/TrackedOp.cc: fix format qualifiers for uint

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/str_map.cc: do boundry checks before access

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

CephBroker.cc: fix format qualifier for unsigned int

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

krbd.cc: fix parameter to variadic function

Fix for:

[src/krbd.cc:549]: (portability) Passing NULL after the last typed
 argument to a variadic function leads to undefined behaviour.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

lz4/LZ4Compressor.h: remove always true check

Fix for:

[src/compressor/lz4/LZ4Compressor.h:105]: (style) Condition
 'r!=(int)compressed_pairs[i].first' is always true

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSD.h: fix format qualifier to use PRIu32

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/fio/fio_ceph_objectstore.cc: fix uninitialized variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_website.h: fix uninitialized variable

Co-authored-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

msg/async/EventKqueue.cc: fix memory leak due to realloc error

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #21421 from liewegas/wip-doc-balancer

doc/mgr/balancer: document

Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>

Merge pull request #21425 from cbodley/wip-rgw-fix-set_threaded

rgw: fix sync status conflict with cloud sync

rgw: fix conflict with cloud sync

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20027 from Leeshine/wip-rgw-sync-bucket-shards

rgw: improve sync status: display behind bucket shards

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21123 from theanalyst/wip-civetweb-1.1

rgw: civetweb fixes for v1.1 upgrade

doc/mgr/balancer: document

Not sure how we missed this for luminous!

Signed-off-by: Sage Weil <sage@redhat.com>

cls/refcount: add chunk_refcount_client()

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

src/test: add a unit test for manifest refcount

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

osd: decrease refcount when object is deleted (has manifest) if the object is referenced

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

osd: increase refcount if redirect or chunked object is enabled

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

osd: add refcount_manifest() for the object that has manifest

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

osd/osd_types.h: add HAS_REFERENCE flag

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

librados: add with_reference flag when redirect or chunk is enabled

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

cls/refcount: add chunk_refcount

refcount() for chunked object is added (based on source id,
pool id and offset)

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

rgw: update civetweb submodule sha1 to track ceph-master

Updating the sha1 to track ceph-master which currently updates civetweb to 1.1

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

cmake: set OPENSSL_API_1_1 in civetweb conditionally

While not ideal, we set the civetweb openssl 1.1 conditional compile based on
the openssl version that cmake reports. In future we should make civetweb itself
do this based on OPENSSL_VERSION_COMPAT

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

qa: drop rfc2616 checking on civetweb

Since newer versions of civetweb are also strict on rfc2616 checks let's enforce
strict rfc2616 checks in s3tests

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: update civetweb to 1.1

Introduces the following additions in rgw:
- allow_unicode_in_urls introduced with a corresponding downstream commit in
civetweb, as the newer version of civetweb validates that urls are url encoded
which swifttests do not follow, so introduce this as a configurable which we set
as true
- mg header struct changes in civetweb update, use auto here
- drop info->uri and use local_uri instead as the former is deprecated

wip: rgw: civetweb fixes for v1.1 upgrade

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

common/options: change mon_data_size_warn type to TYPE_SIZE

As the option represents a byte count, TYPE_SIZE is appropriate and the
correct IEC unit prefixes will be parsed.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

qa/workunits/cephtool/test.sh: fix SI unit test, add IEC unit test

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

qa/tasks/mgr/dashboard: rbd: use JSON schema validator

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: changed REST API prefix to /block/image

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: viewcache: debug log messages

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: fix disk-usage and free ioctx and image resources

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: image copy test

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: image copy implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: snapshot clone test

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: snapshot clone implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: snapshot rollback test

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: taskmanager: support exception handler

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: snapshot rollback implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: major refactoring for using the Task decorator

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: restcontroller: support for detail methods

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Task decorator for controller endpoints

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: snapshot edit tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: snapshot edit implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: use dashboard pool API for creating pools

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: image edit tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: image edit implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: snapshot create/delete tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: snapshot create/delete implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: image deletion test

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: image delete implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>