mgr/dashboard: Fix "merge" audit warning

Fixes: https://tracker.ceph.com/issues/36702
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove unit tests warnings

Recent Angular update is causing a warning to be shown when you call
"router.navigate" outside an angular zone.

This is not affecting the code itself, just the unit tests.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update remaining packages

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update Angular packages

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update rxjs and ngx-bootstrap packages

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update prettier package

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24766 from familyuu/crush

mgr/dashboard: CRUSH map viewer

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add CRUSH map viewer

Fixes: http://tracker.ceph.com/issues/35684
Signed-off-by: familyuu <guodan1@lenovo.com>

Merge pull request #24920 from wjwithagen/wjw-fix-build-symbolic

cmake: do not pass -B{symbolic,symbolic-functions} to linker on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24814 into master

* refs/pull/24814/head:
Merge PR #24804 into nautilus
include/types: fixed compile warning for signed/unsigned comparison
osd/PrimaryLogPG: uncommitted dup ops should respond with logged return code
osd/PrimaryLogPG: propagate error return codes on object copy_get ops
osd/PGLog: optionally record error return codes for extra_reqids
osd/osd_types: include PG log return codes in object copy data

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #24891 from iridescent-rsy/fix-return-value

rbd: make sure the return-value 'r' will be returned

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24893 from iridescent-rsy/need-to-return

rbd: return error code when the source and distination namespace are different

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24823 from Devp00l/wip-issue-36615

[RFC] mgr/dashboard: Cleanup of OSD list methods

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24475 from votdev/feature_36193

mgr/dashboard: Audit REST API calls

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #24856 from leseb/spec-rbd-mirror

spec: add missing rbd mirror bootstrap directory

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24892 from p-na/pna-cleanup-pool-form-simpl

mgr/dashboard: Simplification of PoolForm method

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

cmake: do not pass -B{symbolic,symbolic-functions} to linker on FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

spec: add missing rbd mirror bootstrap directory

Currently /var/lib/ceph/bootstrap-rbd-mirror is absent, which means we
need to create it manually in order to pool the
client.bootstrap-rbd-mirror key.

Signed-off-by: Sébastien Han <seb@redhat.com>

mgr/dashboard: Audit REST API calls

Fixes: https://tracker.ceph.com/issues/36193
Enable API auditing with 'ceph dashboard set-audit-api-enabled true' (default is false). If you do not want to log the request payload, then disable it via 'set-audit-api-log-payload false' (default is true).

Example output:
2018-10-08 10:25:21.850994 mgr.x [INF] [DASHBOARD] from='https://[::1]:44410' path='/api/auth' method='POST' user='None' params='{"username": "admin", "password": "***", "stay_signed_in": false}'

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #23190 from maxcuttins/master

doc: rewrote the iscsi-target-cli installation

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #24870 into master

* refs/pull/24870/head:
client: request next osdmap for blacklisted client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: rewrote the iscsi-target-cli installation

Rewrote the documentation in a step-by-step style, reformatted using RST's
titles instead of bolds, added installation of GIT if needed, added
installation of compilers needed for tcmu-runner

Signed-off-by: Massimiliano Cuttini <massimiliano.cuttini@phoenixweb.it>

Merge pull request #24895 from votdev/feature_36681

rgw: Return tenant field in bucket_stats function

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24864 from dillaman/wip-36659

rbd-mirror: update mirror status when stopping

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24836 from dillaman/wip-26951

rbd: support namespaces for image migration

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24812 from gregsfortytwo/wip-readable-fails-on-empty

test: make readable.sh fail if it doesn't run anything

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22923 from ceph/wip-fix-latest-distro

qa/tests: update links for centos latest to point to 7.5

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #24897 from trociny/wip-36683

test/rbd_mirror: race in TestMockImageMap.AddInstancePingPongImageTest

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test/rbd_mirror: race in TestMockImageMap.AddInstancePingPongImageTest

Remove a context from peer_ack_ctxs before completing to avoid a race
with a listener inserting a new one.

Fixes: http://tracker.ceph.com/issues/36683
Signed-off-by: Mykola Golub <mgolub@suse.com>

rgw: Return tenant field in bucket_stats function

Fixes: https://tracker.ceph.com/issues/36681
Signed-off-by: Volker Theile <vtheile@suse.com>

rbd: return error code when the source and distination namespace are different.

Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>

mgr/dashboard: Simplification of PoolForm method

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

rbd: make sure the return-value 'r' will be returned.

Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>

Merge pull request #24616 from tchaikov/wip-wip-librados-3

librados: split C++ and C APIs into different source files

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge PR #24555 into master

* refs/pull/24555/head:
doc: document optional timeout argument for `cache drop`
mds: make timeout parameter optional for "cache drop"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24747 into master

* refs/pull/24747/head:
client: add new routine to get fscid from a ceph_mount

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24868 from liewegas/wip-doc-fix

doc/rados/operations/crush-map-edits: fix 'take' syntax

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: update links for centos latest to point to 7.5

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

rpm,deb: package libceph-common.so.* not libceph-common.so*

be more explicit on what we are packaging. because only
libceph-common.so.${soversion} will be packaged, since libceph-common.so
won't be installed by cmake anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: do not create symlink for libceph-common.so.*

and bump up its soversion, so it won't conflict with libceph-common.so.0
offered by librados2.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: add librados3 to exclude list of pre-nautilus install tasks

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: s/librados2/librados3/

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: s/librados2/librados3/

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian: s/librados2/librados3/

Signed-off-by: Kefu Chai <kchai@redhat.com>

rpm: s/librados2/librados3/

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: bump librados so version

Signed-off-by: Kefu Chai <kchai@redhat.com>

librados: bump version number

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: update test_envlibrados_for_rocksdb.sh for libradospp split

include a patch so rocksdb can use libradospp instead of librados. will
upstream the patch and make it work for both pre-nautilus librados and
nautilus libradospp

Signed-off-by: Kefu Chai <kchai@redhat.com>

rpm: add libradospp* packages

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian: add libradospp* packages

Signed-off-by: Kefu Chai <kchai@redhat.com>

librados: move C++ APIs into libradospp

the goal is to decouple C++ API from C API, and to version them
differently, as they are targeting different consumers.

this allows us to change the C++ API and bumping up its soversion
without requiring consumer to recompile the librados client for
using the new librados. in this way, C++ API can move faster than
C API. for example, if bufferlist interface is changed for better
performance, and this breaks existing API/ABI, we can bump up
the C++ library's soversion, and and the C library's version unchanged
but ship the new librados's C binding. so the librados client linked
against librados's C library will be able to take advantage of
the improvement in C++ library. while the librados client
linked against C++ library won't break at runtime due to unresolved
symbol or changed structure layout.

this is massive change, the genereal idea is to

* split librados.cc into two source files: librados_c.cc and
  librados_cxx.cc, the former for implementing C APIs, the later
  for C++ APIs.
* extract the C++ API in librados into librados-cxx, the library
  name will be libradospp. but we can change it before nautilus
  is released.
* link these librados libraries with static libraries which it
  depends on, so "-Wl,--exclude-libs,ALL" link flags can help
  hide the non-public symbols.
* extract the tests exercising librados' C++ API into a different
  source file named *_cxx.cc. for instance, to move the C++ tests
  in aio.cc into aio_cxx.cc
* extract the shared helper functions which do not use any librados
  or librados-cxx APIs into test_shared{.cc,h}. the "shared" here
  means, *shared* by C++ and C tests.
* extract the test fixtures, i.e., the subclasses of testing::Test,
  for testing C++ APIs into testcase_cxx.cc.
* update qa/workunits/rados/test.sh accordingly to add the splitted
  tests
* update the consumers of librados to link against librados-cxx
  instead, if they are using the C++ API.

Signed-off-by: Kefu Chai <kchai@redhat.com>

librados: move get_inconsistent_pgs() into RadosClient

so we don't need parse the pg string and render it again for librados C
API.

the downside of this change is that, get_inconsistent_pgs() could be
implemented using RadosClient, instead be *in* it. but before we have
a place for the helper functions for these higher-level functions,
RadosClient is a good place for hosting them.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24406 from ivancich/wip-rgw-renew-reshard-lock

rgw: renew resharding locks to prevent expiration

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Cleanup of OSD list methods

Some test changes were required, too.

Fixes: https://tracker.ceph.com/issues/36615
Signed-off-by: Stephan Müller <smueller@suse.com>

rbd: support namespaces for image migration

Fixes: http://tracker.ceph.com/issues/26951
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20598 from rishabh-d-dave/fix-rbdfuse-look-for-ceph-conf

rbd-fuse: look for ceph.conf in standard locations

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24829 from p-na/pna-fix-docstring

mgr/dashboard: Fix links to external documentation

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

client: request next osdmap for blacklisted client

When client became blacklisted, it did't subscribe next osdmap. So when
it was removed from blacklist by expiration or commandline, it didn't
get updated osdmap which is the only way for the client to reset its
blacklisted flag.

Fixes: http://tracker.ceph.com/issues/36668
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>

doc/rados/operations/crush-map-edits: fix 'take' syntax

Fixes: https://tracker.ceph.com/issues/21496
Signed-off-by: Remy Zandwijk <remy@luckyhands.nl>
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21767 from cfanz/wip-fix-rgw-race-condition

rgw: avoid race condition in RGWHTTPClient::wait()

rbd-mirror: update mirror status when stopping

Fixes: http://tracker.ceph.com/issues/36659
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24810 from rhcs-dashboard/35691-landing-page-chart-improvements

mgr/dashboard: Landing Page: chart improvements

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20794 from ghost/wip-nfs-doc-err

doc: replace rgw_namespace_expire_secs with rgw_nfs_namespace_expire_secs

Merge pull request #21599 from cfanz/wip-fix-rgw-build-error

rgw: fix radosgw-admin build error

Merge pull request #19983 from gohighsec/validate_hitset

mon:validate hit_set values before set

Merge pull request #24848 from joke-lee/wip-doc-radosgw-admin

rgw: add --op-mask in radosgw-admin help info

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24840 from ceph/wip-rm36648

ceph-volume systemd import main so console_scripts work for executable

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

rgw: add --op-mask in radosgw-admin help info

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>

Merge PR #19411 into master

* refs/pull/19411/head:
osd: Step down RecoveryCtx from class to struct

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #19288 into master

* refs/pull/19288/head:
bluestore/BlueRocksEnv: Object initialization after deletion

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #19142 into master

* refs/pull/19142/head:
doc/dev/kernel-client-troubleshooting: Add kernel dynamic debuggin

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #19146 into master

* refs/pull/19146/head:
doc/dev/logs: Removes non-existence file path

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #19726 into master

* refs/pull/19726/head:
osd: Sanity check, if too full or not

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #24809 into master

* refs/pull/24809/head:
os/bluestore: omit redundant '/' in OSD path for ceph-bluestore-tool if
os/bluestore: improve error handling for migrate ops in
qa/standtalone/osd-bluefs-volume-ops: remove redundant code.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #21009 into master

* refs/pull/21009/head:
rados: fix error message for cache(-try)-flush-evict-all commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #24534 into master

* refs/pull/24534/head:
librados: Copy buffer on AIO read

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24545 into master

* refs/pull/24545/head:
os/bluestore: debug_omit_block_device_write isn't always respected.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24548 into master

* refs/pull/24548/head:
Objecter: add ignore cache flag if got redirect reply

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge PR #24599 into master

* refs/pull/24599/head:
os/bluestore/KernelDevice: Honor discard_granularity

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24742 into master

* refs/pull/24742/head:
osd: slightly refactor PrimaryLogPG::do_op.
osd: avoid unnecessary map lookups in PrimaryLogPG::do_op.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

ceph-volume systemd import main so console_scripts work for executable

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22132 from yehudasa/wip-menv

ceph-menv: mrun shell environment

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #24755 into master

* refs/pull/24755/head:
mgr/diskprediction: use global device_failure_prediction_mode setting

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #24764 from tspmelo/wip-jest-ci

mgr/dashboard: Reduce Jest logs in CI

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #24633 from Devp00l/wip-issue-36467

mgr/dashboard: Add a unit test form helper class

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Cleanup imports

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Fix links to documentations

- Added anchor to Grafana in URL of OSD, Overall Performance
- Fixed URL in Object Gateway, Buckets

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge PR #24668 into master

* refs/pull/24668/head:
mgr/DaemonState: fix get_config_defaults locking
mgr: adjust DaemonState device index when daemon metadata is updated
mgr/DaemonState: clear devices in set_metadata

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #24773 from sileht/no-scripts

ceph-volume: use console_scripts

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>

rbd-fuse: look for ceph.conf in standard locations

Fixes: http://tracker.ceph.com/issues/12219
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Renamed CdFormGroups inital test description

Fixes: https://tracker.ceph.com/issues/36467
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Use form helper in dashboard tests

Using form helper in pool, RGW user, user and role form tests and
although in the cd-validator tests.

Fixes: https://tracker.ceph.com/issues/36467
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #24715 from ricardoasmarques/fix-36564

mgr/dashboard: Fix ts error on iSCSI page

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add a unit test form helper class

This class helps testing forms, most methods were outsourced from the
pool form test suite.

Fixes: https://tracker.ceph.com/issues/36467
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #24735 from b-ranto/wip-rpm-dashboards

build/ops: rpm: Package grafana dashboards

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23569 from jcsp/wip-kubejacker

script/kubejacker: include cls libs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22833 from rjfd/wip-dashboard-jwt

mgr/dashboard: JWT authentication

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Landing Page: chart improvements
- Updated chart colors.
- Added border for chart slices (visual impairment help).
- Adjusted chart vertical centering.

Fixes: https://tracker.ceph.com/issues/35691
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #21243 from majianpeng/osd-misc-cleanup

osd/ReplicatedBackend: remove useless assert

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ReplicatedBackend: remove useless assert.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>