cmake: set OPENSSL_API_1_1 in civetweb conditionally

While not ideal, we set the civetweb openssl 1.1 conditional compile based on
the openssl version that cmake reports. In future we should make civetweb itself
do this based on OPENSSL_VERSION_COMPAT

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

qa: drop rfc2616 checking on civetweb

Since newer versions of civetweb are also strict on rfc2616 checks let's enforce
strict rfc2616 checks in s3tests

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: update civetweb to 1.1

Introduces the following additions in rgw:
- allow_unicode_in_urls introduced with a corresponding downstream commit in
civetweb, as the newer version of civetweb validates that urls are url encoded
which swifttests do not follow, so introduce this as a configurable which we set
as true
- mg header struct changes in civetweb update, use auto here
- drop info->uri and use local_uri instead as the former is deprecated

wip: rgw: civetweb fixes for v1.1 upgrade

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #20962 from ricardoasmarques/wip-task-manager-frontend

mgr/dashboard: Asynchronous tasks (frontend)

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #21099 from ricardoasmarques/wip-table-selection-refresh

mgr/dashboard: Update selected items on table refresh

Reviewed-by: Volker Theile vtheile@suse.com
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21268 from smithfarm/wip-doc-config-ceph

doc: some improvements to ceph-conf.rst

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>

mgr/dashboard: Asynchronous tasks front-end

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #21320 from tchaikov/wip-rocksdb

rocksdb: sync with upstream

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21329 from liewegas/wip-23622-b

mon/ConfigKeyService: dump: print placeholder value for binary blobs

Reviewed-by: John Spray <jspray@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #21357 from gregsfortytwo/wip-doc-remove-naked-repo

doc: remove references to unversioned repository addresses

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21325 from xiexingguo/wip-fix-up-seg-error

 crush/CrushWrapper: fix potential invalid use of iterator

Reviewed-by: Sage Weil <sage@redhat.com>

doc: remove references to unversioned repository addresses

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

mon/ConfigKeyService: dump: print placeholder value for binary blobs

JSON cannot express arbitrary binary blobs.  Instead of outputting invalid
and unparseable JSON, represent the value of blobs as something like
'<<< binary blob of length 12 >>>'.

Fixes: http://tracker.ceph.com/issues/23622
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21336 from ceph/wip-yuriw-crontab

qa/tests: Changed rhel7.4 to rhel7.5

Merge pull request #21311 from liewegas/wip-validate-caps

mon: expand cap validity check for mgr, osd, mds

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #21265 into master

* refs/pull/21265/head:
doc: outline steps to take CephFS fs down

Reviewed-by: Douglas Fuller <dfuller@redhat.com>

Merge pull request #21236 from jecluis/wip-mon-auth-bootstrap

mon/AuthMonitor: create bootstrap keys on create_initial()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21102 from trociny/wip-23487

mon: add  'ceph osd pool get erasure allow_ec_overwrites' command

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #21300 from dillaman/wip-rbd-mirror-active-active

rbd-mirror: integrate image map policy as incremental step to active-active

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21314 from badone/wip-ms_inject_socket_failures-SIGFPE

msg/async/AsyncConnection: Fix FPE in process_connection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21308 from dillaman/wip-23597

librbd: disable ENOENT tracking within the object cacher

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21318 from badone/wip-qa-mon-misc-add-osdmap-prune-tests

qa/standalone/mon/misc.sh: Add osdmap-prune tests

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21306 from rzarzynski/wip-bs-bug23333-allow-eio

os/bluestore: alter the allow_eio policy regarding kernel's error list.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21288 from jcsp/wip-23584

mgr/prometheus: fix PG state names

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #21281 from zealoussnow/wip-leozhang-0407

common: retry_sys_call no need take address of a function pointer

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21297 from tchaikov/wip-ceph-w-utf-8

ceph: print output of "status" as string not as bytes

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #21275 from smithfarm/wip-23296

osdc/Objecter: add ignore overlay flag if got redirect reply

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #21259 from rzarzynski/wip-common-hint-dout

common: hint the main branch of dout() accordingly to default verbosity.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20186 from jecluis/wip-codingstyle

CodingStyle: add python and javascript/typescript

CodingStyle: add python and javascript/typescript

As we are rapidly expanding our code base beyond C++ and C, it is a good
idea to enforce coding styles for other languages.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #21334 from batrick/i23643

qa: fix typo in test_full

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21349 from wjwithagen/wjw-freebsd-deps

FreeBSD: add new required packages to be installed

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21258 from votdev/rgw_proxy

mgr/dashboard: Implement a RGW proxy

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #21347 from SUSE/wip-doc-mgr-plugin-limitations

doc/mgr: Add limitations section to plugin guide

Reviewed-by: Lenz Grimmer <lenz@grimmer.com>
Reviewed-by: John Spray <john.spray@redhat.com>

FreeBSD: add new required packages to be installed

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

mgr/dashboard: Update selected items on table refresh

Signed-off-by: Ricardo Marques <rimarques@suse.com>

doc/mgr: Add limitations section to plugin guide

Signed-off-by: Tim Serong <tserong@suse.com>

mgr/dashboard: Takeover PR #20799 and adapt comments.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Implement RGW proxy

This implementation is basically a Rados Gateway reverse proxy.  It
additionally takes care of the authentication to the Rados Gateway, but to use
it you will have to be authenticated against the dashboards RESTful API.

The corresponding credentials can be configured using the following commands:

    dashboard set-rgw-api-secret-key <secret-key>

    dashboard set-rgw-api-access-key <access-key>

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: RGW proxy: Include `python-requests-aws`

This commit will include the `python-requests-aws` [1] library.  This library
extends the `requests` library by an authentication mechanism to S3 gateways.

[1] https://github.com/tax/python-requests-aws

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #21022 from sebastian-philipp/dashboard_get_range

mgr/dashboard: Refactor multiple duplicates of `get_rate()`

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #20920 from sebastian-philipp/dashboard_v2-erasure-code-profile

mgr/dashboard: Add ErasureCodeProfile controller

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #21086 from ricardoasmarques/fix-table-without-fetch-data

mgr/dashboard: Fix table without fetchData

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile vtheile@suse.com

Merge pull request #21041 from votdev/custom_validators

mgr/dashboard: Add custom validators

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21078 from tspmelo/wip-notifications

mgr/dashboard: Add notification service/component

Reviewed-by: Volker Theile vtheile@suse.com
Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Loading indicator only if fechData provided

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Hide refresh button if no fetchData provided

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add custom validators.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge PR #19220 into master

* refs/pull/19220/head:
mds: limit run time of load balancer
mds: add list to track recently used sub-directories
mds: calculate other mds' last_epoch_under locally
mds: cleanup mds_load map access/update
mds: check export pin when choosing dirfrags for exporting
mds: optimize MDBalancer::find_exports()
mds: avoid creating unnecessary subtrees during load balance
mds: mds: optimize MDBalancer::try_rebalance()
mds: don't try exporting subdir if dirfrag is already being exported
mds: don't try exporting dirfrags under mds's own mdsdir
mds: cleanup MDBalancer::try_rebalance()
mds: always pass current time to MDBalancer::{hit_inode,hit_dir}
mds: remove unused MDBalancer::export_empties()
mds: don't decay load counters when printing load
mds: adjust pop_nested when importing subtree
mds: adjust subtree popularity when import fails
mds: adjust subtree popularity after rename
mds: fix request rate calculation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21310 from tchaikov/wip-23610

osd: resend osd_pgtemp if it's not acked

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20947 from dzafman/wip-23364

Special scrub handling of hinfo_key errors

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

common: hint the main branch of dout() accordingly to default verbosity.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

qa/tests: Changed rhel7.4 to rhel7.5

Note: This is a temp fix till we get $ implementaion

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

doc: Add release notes for the output format changes

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Be smarter about when jsonschema can be used

Signed-off-by: David Zafman <dzafman@redhat.com>

osd rados command: Show snapset in list-inconsistent-snapset

Add SnapSet bufferlist to inconsistent_snapset_t

Partial fix for http://tracker.ceph.com/issues/23428

Signed-off-by: David Zafman <dzafman@redhat.com>

rbd-mirror: image map shut down should reset task context

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: wire image map between pool watcher and instance watcher

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: wire instance notifications through to pool replayer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: integrate image map init/shut down into pool replayer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: initial policy instance update should detect dead instances

At initial promotion to the leader position, it can be nearly always
assumed that at least one instance has died and needs to have its images
shuffled to another instance (i.e. the old leader).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: simplify the interface between image mapper and policy

The context callbacks between the image map and its policy required
too many hooks between the two classes. Additionally, the original
implementation didn't support re-sending of acquire messages after
initializing (to close a potential race condition).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: image map listener should provide a callback context

The context wraps the necessary data and provides a pass-through chain
to the instance notification helpers.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: moved policy inner-types to common types header

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: convert PoolReplayer to templated class

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: block instance add/remove notifications until post-locked

Avoid a possible race condition with instances being detected before the
pool replayer leader handling is fully initialized.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: new add/remove instances listener

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: batch peer instances ack and timeout handlers

This throttles the on-disk updates and also will eventually help to
throttle the shuffling of images between alive peer instances.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: removed declared but undefined leader watcher methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: leader watcher should always initialize status watcher

In the case of active/active, both the leader and the followers will be
sending status updates for their images. Therefore, all rbd-mirror daemons
should be registered.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: moved leader watcher listener interface out of templated class

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: moved pool watcher listener interface out of templated class

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd_mirror: document the basic state flow for pool watcher

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: SnapSet::dump() handle snapid_t not in clone_overlap or clone_size

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Add getjson variable to save output

Signed-off-by: David Zafman <dzafman@redhat.com>

osd rados command: Change error name snapset_mismatch to snapset_error

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Clean-up test and use local values for number of objects and osds

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Without snapdir we don't have a head mismatch error anymore

Signed-off-by: David Zafman <dzafman@redhat.com>

osd rados command: list-inconsistent-obj attribute improvements

System attributes shown as "object_info", "snapset" and "hashinfo"
Only output user attributes as "attrs"
Drop leading undescore "_" for user attribute keys
Improve logic as to when to show user attributes or specific system attributes

Signed-off-by: David Zafman <dzafman@redhat.com>

osd rados command: Change "oi" to "info" in scrub handling errors

data_digest_mismatch_oi -> data_digest_mismatch_info
omap_digest_mismatch_oi -> omap_digest_mismatch_info
size_mismatch_oi -> size_mismatch_info
obj_size_oi_mismatch -> obj_size_info_mismatch

Signed-off-by: David Zafman <dzafman@redhat.com>

osd rados command: Change "oi_attr" to "info" in scrub handling errors

oi_attr_missing -> info_missing
oi_attr_corrupted -> info_corrupted

Signed-off-by: David Zafman <dzafman@redhat.com>

osd rados command: Rename ss_attr_missing/ss_attr_corrupted to snapset_missing/snapset_corrupted

Signed-off-by: David Zafman <dzafman@redhat.com>

osd rados command: Improve scrub handling of HashInfo (hinfo_key xattr)

Fixes: http://tracker.ceph.com/issues/23364
Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Don't crash in get_hash_info() with a corrupt HashInfo

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Remove check that masks differences (let diff fail)

Signed-off-by: David Zafman <dzafman@redhat.com>

qa: fix typo in test_full

Cause: 577737d007c05bc7a3972158be8c520ab73a1517

Fixes: http://tracker.ceph.com/issues/23643
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21321 into master

* refs/pull/21321/head:
tools/cephfs: instantiate MDSUtility after global_init()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21130 from ceph/wip-rm23067

ceph-volume  Implement an 'activate all' to help with dense servers or migrating OSDs

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #21304 from rjfd/wip-rbd-pybind-fixes

rbd: python bindings fixes and improvements

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21272 from cbodley/wip-rgw-beast-experiment

rgw: beast frontend no longer experimental

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #21120 from oritwas/wip-rgw-21619

rgw: reshard cancel command should clear bucket resharding flag

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: beast frontend no longer experimental

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #21302 into wip-sage-testing-20180410.153118

* refs/pull/21302/head:
qa/suites/rados/singleton/all/random-eio: whitelist eio error message

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #21315 into wip-sage-testing-20180410.153007

* refs/pull/21315/head:
qa/tasks/mgr/test_failover: fix for id -> name rename in mgr metadata

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

githubmap: update contributors

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21324 from tchaikov/wip-23495

qa/suites: whitelist SLOW_OPS

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mgr/dashboard: Replace toastr calls with notification service

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add notification component

This component is used on the navigation bar and will show the user the 10 most
recent notifications.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add notification service

This service should be used to display any notification/toast to the user.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #19283 from yehudasa/wip-rgw-mfa

rgw: mfa support

Reviewed-by: Casey Bodley <cbodley@redhat.com>