rbd: show info about mirror instance in image mirror status output

It is particularly useful when running multiple rbd-mirror instances
in Active-Passive or Active-Active mode.

Signed-off-by: Mykola Golub <mgolub@suse.com>

rbd-mirror: provide instance_id for `ceph service status`

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: API to get info which rbd-mirror daemon is mirroring image

Signed-off-by: Mykola Golub <mgolub@suse.com>

cls/rbd: new methods to get image mirror instance info

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge PR #24870 into master

* refs/pull/24870/head:
client: request next osdmap for blacklisted client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24895 from votdev/feature_36681

rgw: Return tenant field in bucket_stats function

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24864 from dillaman/wip-36659

rbd-mirror: update mirror status when stopping

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24836 from dillaman/wip-26951

rbd: support namespaces for image migration

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24812 from gregsfortytwo/wip-readable-fails-on-empty

test: make readable.sh fail if it doesn't run anything

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22923 from ceph/wip-fix-latest-distro

qa/tests: update links for centos latest to point to 7.5

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #24897 from trociny/wip-36683

test/rbd_mirror: race in TestMockImageMap.AddInstancePingPongImageTest

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test/rbd_mirror: race in TestMockImageMap.AddInstancePingPongImageTest

Remove a context from peer_ack_ctxs before completing to avoid a race
with a listener inserting a new one.

Fixes: http://tracker.ceph.com/issues/36683
Signed-off-by: Mykola Golub <mgolub@suse.com>

rgw: Return tenant field in bucket_stats function

Fixes: https://tracker.ceph.com/issues/36681
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #24616 from tchaikov/wip-wip-librados-3

librados: split C++ and C APIs into different source files

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge PR #24555 into master

* refs/pull/24555/head:
doc: document optional timeout argument for `cache drop`
mds: make timeout parameter optional for "cache drop"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24747 into master

* refs/pull/24747/head:
client: add new routine to get fscid from a ceph_mount

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24868 from liewegas/wip-doc-fix

doc/rados/operations/crush-map-edits: fix 'take' syntax

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: update links for centos latest to point to 7.5

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

rpm,deb: package libceph-common.so.* not libceph-common.so*

be more explicit on what we are packaging. because only
libceph-common.so.${soversion} will be packaged, since libceph-common.so
won't be installed by cmake anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: do not create symlink for libceph-common.so.*

and bump up its soversion, so it won't conflict with libceph-common.so.0
offered by librados2.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: add librados3 to exclude list of pre-nautilus install tasks

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: s/librados2/librados3/

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: s/librados2/librados3/

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian: s/librados2/librados3/

Signed-off-by: Kefu Chai <kchai@redhat.com>

rpm: s/librados2/librados3/

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: bump librados so version

Signed-off-by: Kefu Chai <kchai@redhat.com>

librados: bump version number

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: update test_envlibrados_for_rocksdb.sh for libradospp split

include a patch so rocksdb can use libradospp instead of librados. will
upstream the patch and make it work for both pre-nautilus librados and
nautilus libradospp

Signed-off-by: Kefu Chai <kchai@redhat.com>

rpm: add libradospp* packages

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian: add libradospp* packages

Signed-off-by: Kefu Chai <kchai@redhat.com>

librados: move C++ APIs into libradospp

the goal is to decouple C++ API from C API, and to version them
differently, as they are targeting different consumers.

this allows us to change the C++ API and bumping up its soversion
without requiring consumer to recompile the librados client for
using the new librados. in this way, C++ API can move faster than
C API. for example, if bufferlist interface is changed for better
performance, and this breaks existing API/ABI, we can bump up
the C++ library's soversion, and and the C library's version unchanged
but ship the new librados's C binding. so the librados client linked
against librados's C library will be able to take advantage of
the improvement in C++ library. while the librados client
linked against C++ library won't break at runtime due to unresolved
symbol or changed structure layout.

this is massive change, the genereal idea is to

* split librados.cc into two source files: librados_c.cc and
  librados_cxx.cc, the former for implementing C APIs, the later
  for C++ APIs.
* extract the C++ API in librados into librados-cxx, the library
  name will be libradospp. but we can change it before nautilus
  is released.
* link these librados libraries with static libraries which it
  depends on, so "-Wl,--exclude-libs,ALL" link flags can help
  hide the non-public symbols.
* extract the tests exercising librados' C++ API into a different
  source file named *_cxx.cc. for instance, to move the C++ tests
  in aio.cc into aio_cxx.cc
* extract the shared helper functions which do not use any librados
  or librados-cxx APIs into test_shared{.cc,h}. the "shared" here
  means, *shared* by C++ and C tests.
* extract the test fixtures, i.e., the subclasses of testing::Test,
  for testing C++ APIs into testcase_cxx.cc.
* update qa/workunits/rados/test.sh accordingly to add the splitted
  tests
* update the consumers of librados to link against librados-cxx
  instead, if they are using the C++ API.

Signed-off-by: Kefu Chai <kchai@redhat.com>

librados: move get_inconsistent_pgs() into RadosClient

so we don't need parse the pg string and render it again for librados C
API.

the downside of this change is that, get_inconsistent_pgs() could be
implemented using RadosClient, instead be *in* it. but before we have
a place for the helper functions for these higher-level functions,
RadosClient is a good place for hosting them.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24406 from ivancich/wip-rgw-renew-reshard-lock

rgw: renew resharding locks to prevent expiration

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rbd: support namespaces for image migration

Fixes: http://tracker.ceph.com/issues/26951
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20598 from rishabh-d-dave/fix-rbdfuse-look-for-ceph-conf

rbd-fuse: look for ceph.conf in standard locations

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24829 from p-na/pna-fix-docstring

mgr/dashboard: Fix links to external documentation

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

client: request next osdmap for blacklisted client

When client became blacklisted, it did't subscribe next osdmap. So when
it was removed from blacklist by expiration or commandline, it didn't
get updated osdmap which is the only way for the client to reset its
blacklisted flag.

Fixes: http://tracker.ceph.com/issues/36668
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>

doc/rados/operations/crush-map-edits: fix 'take' syntax

Fixes: https://tracker.ceph.com/issues/21496
Signed-off-by: Remy Zandwijk <remy@luckyhands.nl>
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21767 from cfanz/wip-fix-rgw-race-condition

rgw: avoid race condition in RGWHTTPClient::wait()

rbd-mirror: update mirror status when stopping

Fixes: http://tracker.ceph.com/issues/36659
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24810 from rhcs-dashboard/35691-landing-page-chart-improvements

mgr/dashboard: Landing Page: chart improvements

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20794 from ghost/wip-nfs-doc-err

doc: replace rgw_namespace_expire_secs with rgw_nfs_namespace_expire_secs

Merge pull request #21599 from cfanz/wip-fix-rgw-build-error

rgw: fix radosgw-admin build error

Merge pull request #19983 from gohighsec/validate_hitset

mon:validate hit_set values before set

Merge pull request #24848 from joke-lee/wip-doc-radosgw-admin

rgw: add --op-mask in radosgw-admin help info

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24840 from ceph/wip-rm36648

ceph-volume systemd import main so console_scripts work for executable

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

rgw: add --op-mask in radosgw-admin help info

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>

Merge PR #19411 into master

* refs/pull/19411/head:
osd: Step down RecoveryCtx from class to struct

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #19288 into master

* refs/pull/19288/head:
bluestore/BlueRocksEnv: Object initialization after deletion

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #19142 into master

* refs/pull/19142/head:
doc/dev/kernel-client-troubleshooting: Add kernel dynamic debuggin

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #19146 into master

* refs/pull/19146/head:
doc/dev/logs: Removes non-existence file path

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #19726 into master

* refs/pull/19726/head:
osd: Sanity check, if too full or not

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #24809 into master

* refs/pull/24809/head:
os/bluestore: omit redundant '/' in OSD path for ceph-bluestore-tool if
os/bluestore: improve error handling for migrate ops in
qa/standtalone/osd-bluefs-volume-ops: remove redundant code.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #21009 into master

* refs/pull/21009/head:
rados: fix error message for cache(-try)-flush-evict-all commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #24534 into master

* refs/pull/24534/head:
librados: Copy buffer on AIO read

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24545 into master

* refs/pull/24545/head:
os/bluestore: debug_omit_block_device_write isn't always respected.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24548 into master

* refs/pull/24548/head:
Objecter: add ignore cache flag if got redirect reply

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge PR #24599 into master

* refs/pull/24599/head:
os/bluestore/KernelDevice: Honor discard_granularity

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24742 into master

* refs/pull/24742/head:
osd: slightly refactor PrimaryLogPG::do_op.
osd: avoid unnecessary map lookups in PrimaryLogPG::do_op.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

ceph-volume systemd import main so console_scripts work for executable

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22132 from yehudasa/wip-menv

ceph-menv: mrun shell environment

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #24755 into master

* refs/pull/24755/head:
mgr/diskprediction: use global device_failure_prediction_mode setting

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #24764 from tspmelo/wip-jest-ci

mgr/dashboard: Reduce Jest logs in CI

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #24633 from Devp00l/wip-issue-36467

mgr/dashboard: Add a unit test form helper class

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Cleanup imports

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Fix links to documentations

- Added anchor to Grafana in URL of OSD, Overall Performance
- Fixed URL in Object Gateway, Buckets

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge PR #24668 into master

* refs/pull/24668/head:
mgr/DaemonState: fix get_config_defaults locking
mgr: adjust DaemonState device index when daemon metadata is updated
mgr/DaemonState: clear devices in set_metadata

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #24773 from sileht/no-scripts

ceph-volume: use console_scripts

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>

rbd-fuse: look for ceph.conf in standard locations

Fixes: http://tracker.ceph.com/issues/12219
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Renamed CdFormGroups inital test description

Fixes: https://tracker.ceph.com/issues/36467
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Use form helper in dashboard tests

Using form helper in pool, RGW user, user and role form tests and
although in the cd-validator tests.

Fixes: https://tracker.ceph.com/issues/36467
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #24715 from ricardoasmarques/fix-36564

mgr/dashboard: Fix ts error on iSCSI page

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add a unit test form helper class

This class helps testing forms, most methods were outsourced from the
pool form test suite.

Fixes: https://tracker.ceph.com/issues/36467
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #24735 from b-ranto/wip-rpm-dashboards

build/ops: rpm: Package grafana dashboards

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23569 from jcsp/wip-kubejacker

script/kubejacker: include cls libs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22833 from rjfd/wip-dashboard-jwt

mgr/dashboard: JWT authentication

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Landing Page: chart improvements
- Updated chart colors.
- Added border for chart slices (visual impairment help).
- Adjusted chart vertical centering.

Fixes: https://tracker.ceph.com/issues/35691
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #21243 from majianpeng/osd-misc-cleanup

osd/ReplicatedBackend: remove useless assert

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ReplicatedBackend: remove useless assert.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #13925 from nvedant07/bug_fix_1

osdmaptool/cleanup: Completed osdmaptool's usage

test: make readable.sh fail if it doesn't run anything

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #24791 from falcon78921/wip-docs-demodocs

doc: added demo document changes section

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

os/bluestore: omit redundant '/' in OSD path for ceph-bluestore-tool if
needed.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

mgr/dashboard: run-frontend-unittests: fix CEPH_ROOT initialization

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: frontend: JWT authentication implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: run-backend-api-request.sh: support for JTW tokens

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: backend: JWT based authentication

Signed-off-by: Ricardo Dias <rdias@suse.com>

os/bluestore: improve error handling for migrate ops in
ceph_bluestore-tool.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

mgr/dashboard: Fix ts error on iSCSI page

This error only happens until initiator is connected to the target.

Fixes: https://tracker.ceph.com/issues/36564
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge PR #24686 into master

* refs/pull/24686/head:
os/bluestore: show compress and buffered from WriteContext
os/bluestore: fix rename race with trim on replacement onode at old name

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

mgr/diskprediction: use global device_failure_prediction_mode setting

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #24229 into master

* refs/pull/24229/head:
common: drop BL_BACKWARD_COMPAT in bufferlist.
osd: OSDMap encoding uses bufferlist::contiguous_filler.
mds: encode_xattrs() uses buffer::list::contiguous_filler.
common: introduce contiguous_filler to optimize ENCODE_START.
common: optimize hole appending in bufferlist.
common: duplicate an encoding macro to suppress warnings.
common: drop backward iteration from bufferlist.

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #24787 into master

* refs/pull/24787/head:
Merge PR #24796 into nautilus
osd: fix heartbeat_reset unlock
Merge PR #24780 into nautilus
Merge PR #24761 into nautilus
Merge PR #24651 into nautilus
osd: fix race between op_wq and context_queue
test: Make sure kill_daemons failure will be easy to find
test: Add flush_pg_stats to make test more deterministic

qa/standtalone/osd-bluefs-volume-ops: remove redundant code.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #24763 from zmc/wip-36416

mgr/dashboard: Map dev 'releases' to master

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #24796 into nautilus

* refs/pull/24796/head:
osd: fix heartbeat_reset unlock

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: fix heartbeat_reset unlock

Fixes 51d8e2457d73c709bfa4f706793696b3ce704ff9, which moved to lock_guard
but didn't remove the unlock call on this exit path.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #24774 into master

* refs/pull/24774/head:
cmake: update dpdk drivers/modules to accomodate dpdk submodule
spdk: update to latest v18.07

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #24666 into master

* refs/pull/24666/head:
include/types: fixed compile warning for signed/unsigned comparison
osd/PrimaryLogPG: uncommitted dup ops should respond with logged return code
osd/PrimaryLogPG: propagate error return codes on object copy_get ops
osd/PGLog: optionally record error return codes for extra_reqids
osd/osd_types: include PG log return codes in object copy data

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24688 into master

* refs/pull/24688/head:
common: make ceph_abort store same crash info as ceph_assert
global: store assert msg in global and dump to crash meta
pybind/mgr: make 'ceph crash ls' output sorted list
log: don't clear ring when dump_recent is called
ceph-crash: make clear to user that 'posted' should be directory

Reviewed-by: Sage Weil <sage@redhat.com>