Merge pull request #21931 from yuriw/wip-yuriw-fix-mimic-subset

tests/qa - fix mimic subset for nightlies

Reviewed-by: Kefu Chai <kchai@redhat.com>

tests/qa - fix mimic subset for nightlies

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #21680 from cbodley/wip-rm-replica-log

rgw: remove all traces of cls replica_log

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21831 from tchaikov/wip-23966

osdc: invoke notify finish context on linger commit failure

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21898 from adamemerson/wip-fedora28

src: Eliminate new warnings in Fedora 28

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21861 from adamemerson/wip-clangtasm

osd,mon,pybind: Make able to compile with Clang

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21912 from dillaman/wip-24064

rbd: disk-usage can now optionally compute exact on-disk usage

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21910 from dillaman/wip-rbd-help

rbd: don't load config overrides from monitor initially

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21823 from dillaman/wip-24009

rbd-mirror: ensure remote demotion is replayed locally

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21817 from dillaman/wip-24008

rbd-mirror: fixed potential crashes during shut down

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21853 from rishabh-d-dave/volclient-keyargs-for-all-exceptions

rados.pyx: make all exceptions accept keyword arguments

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21886 from jcsp/wip-no-install-hello

cmake: do not install `hello` demo module

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21782 from wido/mgr-telegraf

mgr/telegraf: Telegraf module for Ceph Mgr

Reveiwed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21891 from cbodley/wip-cmake-crypto-plugins

cmake: move crypto_plugins target

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Revert "stop.sh: use --no-mon-config when trying to unmap rbd devices"

This reverts commit 6c722cd5f3f829ccf14280e7e36387bafd819c7a.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21770 from zy751713126/remove-redundant-code

rbd: add the judgment of resizing the image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21914 from cbodley/wip-common-async

common: add a generic async Completion for use with boost::asio

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #21909 from neha-ojha/wip-fix-powercycle

qa/suites: set osd_pg_log_dups_tracked in cfuse_workunit_suites_fsync.yaml

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: Silence sign comparison warnings

Since we're iterating with a size_t, we may as well compare against one.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common: add an async SharedMutex

Signed-off-by: Casey Bodley <cbodley@redhat.com>

common: forward_handler() for handlers with move-only args

executors will always call the handler's lvalue overload of operator(),
so forward_handler() is required to wrap calls to bind_handler() with
move-only argument types. Completion uses this to forward all of its
CompletionHandlers

Signed-off-by: Casey Bodley <cbodley@redhat.com>

common: add a generic async Completion interface

a generic Completion for storing asio completion handlers for deferred
execution

Signed-off-by: Casey Bodley <cbodley@redhat.com>

build: Remove -Wno-invalid-offsetof

Since C++17 relaxed the rules for using offsetof, this is no longer needed.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

tools: Do not memset uuid_d

It is perfectly able to default initialize itself.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test/rbd/fsx: Disable overflow-format warning

No return type or other way to react to it.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test/libcephfs: Expand buffer slightly

So the snprintf output isn't truncated.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: There is no point in const return values

Pointer-to-const or reference-to-const yes. Const value, no.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Catch exceptions at const references

/Never/ by value.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Remove extra 'const' from cast

Casting to pointer-to-const makes sense. Casting to a const
value (even a const pointer) doesn't.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rdma: Silence qualifier warnings

No point in reinterpret_casting something const anyway, you can always
assign non-const to const.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

msg/rdma/infiniband: Silence memset warning

Though it may be better in the long-run to replace the malloc-like
interface with an Allocator type interface.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Exceptions should be caught by const reference

/Not/ by value.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

FuseStore: Expand buffers to prevent overflow

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

perfglue: Silence truncation warning

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mount: Enlarge buffer to fix overflow warning

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mon: Remove unused variable

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

blkdev: Check return of snprintf and error on truncation

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

blkdev: Get rid of get_device_by_uuid

This function's interface is bad, in that it takes char*s with no
sizes. Also nobody calls it.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test: Enlarge a few buffers

To get rid of overflow warnings.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mds: Do not memset fnode_t

fnode_t can initialize itself.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

auto: Catch exceptions by const reference

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

client: Do not memset frag_info_t and inode_t

They do a perfectly good job of initializing themselves.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test: Increase buffer to prevent possible truncation

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Fix uninitialized value warning

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #21906 from vshankar/iscsi-target-cli-typo

doc: fix iscsi target name when configuring target

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: disk-usage can now optionally compute exact on-disk usage

Fixes: http://tracker.ceph.com/issues/24064
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: don't load config overrides from monitor initially

The overrides will be loaded when we connect to the cluster via librados
and the current approach prevents running 'rbd help' without a running
cluster.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20974 from XiangxiangMeng/rgw-policy-keyword-modify

rgw: policy: modify some operation permission keyword

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

doc: fix iscsi target name when configuring target

Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #21788 from cbodley/wip-22914

rgw: bucket sync status improvements, part 1

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #21851 from votdev/improve_error_panel

mgr/dashboard: Improve error panel

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/telegraf: Telegraf module for Ceph Mgr

Telegraf is a agent for collecting and reporting metrics.

It has multiple inputs and can send data to various outputs like
for example InfluxDB or ElasticSearch.

This module works by using the socket_listener of Telegraf and can
send data over UDP, TCP and a local Unix Socket.

Signed-off-by: Wido den Hollander <wido@42on.com>

rbd-mirror: ensure remote demotion is replayed locally

The bootstrap process cannot immediately quit if it notices the remote
image is not primary. Instead, it needs to continue if the local image is
still chained to the remote.

Fixes: http://tracker.ceph.com/issues/24009
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cmake: move crypto_plugins target

the crypto_plugins target was defined in
src/crypto/isa-l/CMakeLists.txt, but this is only included
if(HAVE_INTEL AND HAVE_BETTER_YASM_ELF64 AND (NOT APPLE))

moving it out of the if() block allows the os target to depend on it
even if no plugins are built

Signed-off-by: Casey Bodley <cbodley@redhat.com>

osdc: guard op->on_notify_finish with lock

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c16371965cc785f2e2b87d8c6684854fa2de65e1)

librados: fix potential race condition if notify immediately fails

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit cc06fbab08ef5b7f1b18f2af333c019ad86de03d)

osdc: invoke notify finish context on linger commit failure

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fcd414f6324136ce0695fcf30a60eec6ff338a98)

Merge pull request #21609 from dragonylffly/wip-fix-create-ioctx

librados: return ENOENT if pool_id invalid

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21882 from sebastian-philipp/dashboard-run-backend-api-request-insecure

mgr/dashboard: Allow insecure HTTPS in run-backend-api-request

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21820 from tspmelo/wip-ui-error-handling

mgr/dashboard: Add error handling on the frontend

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #21819 from tspmelo/wip-prettier

mgr/dashboard: Add Prettier formatter to the frontend

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add Prettier formatter to the frontend

This will force the use of prettier on frontend code,
bringing a more consistent formatting to all the code.

The current configuration will apply the prettier formatting,
on all staged files, during the precommit hook.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21643 from s0nea/wip-dashboard-configurations-table

mgr/dashboard: replace configuration html table with cd-table

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21777 from votdev/proxy_conf_ssl

mgr/dashboard: Use HTTPS in dev proxy configuration and HACKING.rst

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add error handling on the frontend

Now all error notifications are processed on the api-interceptor-service,
removing the need to handle them on every api request.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21066 from sebastian-philipp/dashboard_error_handling

mgr/dashboard: Improve exception handling

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge PR #21808 into master

* refs/pull/21808/head:
qa: ignore version in auth metadata comp
ceph_volume_client: allow volumes without namespace isolation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/suites: set osd_pg_log_dups_tracked in cfuse_workunit_suites_fsync.yaml

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #21890 from alfredodeza/wip-rm24036

ceph-volume describe better the options for migrating away from ceph-disk

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

doc/rados add an anchor to reference replacing osds

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume describe the options for migrating away from ceph-disk

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume high-level description of simple activate at index.rst

Signed-off-by: Alfredo Deza <adeza@redhat.com>

cleanup: Remove unused variables/captures

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

build: Allow use of Clang to build Python extensions under Fedora 28

Fedora builds Python with hardening flags that are unsupported by
Clang. Python insists on building extensions with all the flags it was
given, not just ones relevant to finding Python libraries and
includes.

Ended up monkey patching customize_compiler to pull out the offending
flags when Clang is in use.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge PR #21810 into master

* refs/pull/21810/head:
MDSMonitor: promote standby after fs creation
MDSMonitor: always prints standbys even if no fs

Reviewed-by: John Spray <john.spray@redhat.com>

mgr/dashboard: fix table in order to view boolean values

The table didn't handle boolean values correctly. If a table contained
booleans it resulted in a TypeError because the value can't be converted to
lower case. We need to convert the boolean value to string at first.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Fix duplicate params error

* Remove params, if they use the `{name:regex}` syntax.
* Fixes http://tracker.ceph.com/issues/23823

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Exception handling: browsable API

* Added display of Exceptions
* Fixed missing sub-path
* Added delete-form
* Fixed default arguments

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Exceptions: Added Documentation

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Applied Exception handling to RBDs

* DashboardTestCase: Removed waiter-thread

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

qa: ignore version in auth metadata comp

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

ceph_volume_client: allow volumes without namespace isolation

Fixes: https://tracker.ceph.com/issues/23695
Signed-off-by: Ramana Raja <rraja@redhat.com>

cmake: do not install `hello` demo module

Signed-off-by: John Spray <john.spray@redhat.com>

mgr/dashboard: Change to Status Code 400 by default.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Add task info to exception result

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Improved Exception handling in Tasks

* Set default status code to 400
* Added tests
* Fixed globbing in `test_task.py`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Applied Exception Handling

* Minor changes to CephFS, OSD, Pool, RbdMirroring, Summary and RGW

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Improve exception handling

* Added `dashboard_exception_handler()` to catch our exceptions.
* Changed the behaviour of `ViewCache` to raise exceptions
* Added `RadosReturnError` raised in `send_command()`
* Added unit tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Improve error panel

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #21803 from alfredodeza/cv-auto

ceph-volume initial take on auto sub-command

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

mgr/dashboard: run-backend-api-request insecure HTTPS

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: replace configuration html table with cd-table

The configuration documentation page still uses a normal html table.
This commit replaces the table with the cd-table to make use of its
advanced features here.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #21845 from tchaikov/wip-gcc-warnings

mds,messages: silence -Wclass-memaccess warnings

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #21856 from tchaikov/wip-drop-dpdk-submodule

dpdk: drop dpdk submodule

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge PR #21712 into master

* refs/pull/21712/head:
qa/tasks/cephfs: add test for renewing stale session
client: invalidate caps and leases when session becomes stale
client: fix race in concurrent readdir

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21688 into master

* refs/pull/21688/head:
client: change vxattr flags field to unsigned int

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21668 into master

* refs/pull/21668/head:
mds: include nfiles/nsubdirs of directory inode in MClientCaps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21858 from yuriw/wip-yuriw-crontab

qa/tests: fixed typo

Merge pull request #21707 from cbodley/wip-qa-upgrade-ragweed

qa/rgw: move ragweed upgrade test into upgrade/luminous-x

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21794 from rjfd/wip-dashboard-fix-awsauth

mgr/dashboard: awsauth: fix python3 string decode problem

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Lenz Grimmer <LGrimmer@suse.com>