mgr: lock pg_map too for osd_pool_stats and notify_osdmap

Fixes: http://tracker.ceph.com/issues/36766
Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonServer: use with_osdmap_and_pgmap instead of nested calls

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ClusterState: add with_osdmap_and_pgmap

Several call sites need to lock both

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #25038 from yuriw/wip-yuriw-crontab-master

qa/tests: added "-n 7" to make sure mimic-x runs on built master branch

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tests: added "-n 7" to make sure mimic-x runs on built master branch

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge PR #25023 into master

* refs/pull/25023/head:
doc/governance: Add Sebastian to CLT

Reviewed-by: Sage Weil <sage@redhat.com>

doc/governance: Add Sebastian to CLT

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #24830 from dillaman/wip-rbd-pool-stats

rbd: expose pool stats summary tool

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24265 from trociny/wip-36091

osd: collect client perf stats when query is enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24214 from iridescent-rsy/bursting_io

librbd: reduce the TokenBucket fill cycle and support bursting io configuration

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22476 from dzafman/wip-23875

Removal of snapshot with corrupt replica crashes osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24806 into master

* refs/pull/24806/head:
os/filestore: fix merge_collection replay guards

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #24843 into master

* refs/pull/24843/head:
osd/osd_types: fix notify-ack string rendering

Reviewed-by: Neha Ojha <nojha@redhat.com>

rbd: added new 'pool stats' action

This provides a quick summary of provisioned pool usage.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: new pool init/stat API methods

The init method is a stub for handling new pool initialization. It
currently only handles setting the application tag. The stats method
will quickly calculate the number of images and provisioned space for
those images within the pool. Querying the pool stats on a pool with
10,000 images only required approximately 2 seconds as compared to
over 2 minutes for a "rbd ls -l" scan.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24998 from alfredodeza/wip-rm36470

ceph-volume enable  --no-systemd flag for simple sub-command

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24627 from Devp00l/wip-manage-ec-profiles

mgr/dashboard: Adds ECP management to the frontend

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #24459 into master

* refs/pull/24459/head:
os: Transaction uses append_hole() to minimize bl:_buffers inflation.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #24890 into master

* refs/pull/24890/head:
mon: fix 'log last' missing out latest event

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24904 into master

* refs/pull/24904/head:
common: point of this may be called after released

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24925 into master

* refs/pull/24925/head:
Avoid import _strptime failed
Avoid exception if remote plugin not enabled
Separate diskprediction local plugin from the diskprediction plugin

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24859 from jan--f/ceph-volume-inventory

ceph-volume: add inventory command

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: add inventory command

The inventory command provides information about a nodes disk inventory.
Existing logical volumes on a disk or one of its partitions are scanned
and reported.
The output can be formatted as plain text or json.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #24934 from votdev/improve_crush_map_viewer

mgr/dashboard: Improve CRUSH map viewer

Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #24489 from ricardoasmarques/wip-saml2

mgr/dashboard: SSO - SAML 2.0 support

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Show info button for crush rules in pool form

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Adds ECP management to the frontend

Now you can create, delete and get information about profiles inside
the pool form.

The erasure code profile form has a lot of tooltips to guide you through
the creation. It can create profiles with different plugins.

Fixes: https://tracker.ceph.com/issues/25156
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Adds ECP info endpoint

The new info endpoint will provide the frontend with the necessary
information it needs to create new profiles.

Fixes: https://tracker.ceph.com/issues/25156
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #24928 from rhcs-dashboard/36632-update-python-dependency

mgr/dashboard: update python dependency

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #24963 from dillaman/wip-ceph-iscsi

doc: tweak RBD iSCSI docs to point to merged tooling repo

Reviewed-by: Mykola Golub <mgolub@suse.com>

test: osd-scrub-snaps.sh: After snapshot removal wait for snaptrim to complete

Due to deliberate corruptions snaptrim_error means snaptrim is done

Signed-off-by: David Zafman <dzafman@redhat.com>

test: osd-scrub-snaps.sh: Testing with new --rmtype in ceph-objectstore-tool

Use --rmtype snapmap with new obj16 to remove snapmap only, check for repair message
Use --rmtype nosnapmap to remove obj5 while leaving snapmap behind

Signed-off-by: David Zafman <dzafman@redhat.com>

test: cleanup: Remove redundant cat of log and handle errors in create_scenario()

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: Add removal options to corrupt objects for testing

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge PR #21511 into master

* refs/pull/21511/head:
os/bluefs: set logfile w/ WRITE_LIFE_MEDIUM
os/bluefs: make super block w/ WRITE_LIFE_TIME_SHORT.
os/bluestore: BlueFS support write_life_time feature of SSD.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #24820 into master

* refs/pull/24820/head:
os/bluestore/BlueStore.cc: merge overlapping/adjacent regions before read

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #24828 into master

* refs/pull/24828/head:
qa/osd-bluefs-volume-ops: use ceph-bluestore-tool for fsck
qa/osd-bluefs-volume-ops: reduce space usage for the test case

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #24948 into master

* refs/pull/24948/head:
os/bluestore: fix wal dir presence verification

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24014 from yehudasa/wip-rgw-svc-2

rgw: initial RGWRados refactoring work

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

rgw: shutdown services only if initialized

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

librados_test_stub: implement move operations

Need to match whatever librados implements.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

ceph-volume tests ensure activate behavior with systemd disabling

Signed-off-by: Alfredo Deza <alfredo@deza.pe>

ceph-volume simple.activate factor out systemd interactions for easier testing

Signed-off-by: Alfredo Deza <alfredo@deza.pe>

Merge PR #24995 into master

* refs/pull/24995/head:
doc: use 'proxy' cache mode in "Removing a Writeback Cache" procedure

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24896 from tchaikov/wip-librados-cleanup

librados,rpm,deb: various fixes to address librados3 transition and cleanups in librados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: use 'proxy' cache mode in "Removing a Writeback Cache" procedure

The 'forward' mode is flawed as it allows the reordering of requests.

Signed-off-by: Mykola Golub <mgolub@suse.com>

test/rbd_mirror: fix linking

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc_notify: call set_enabled() when registering callback

svc_notify starts before svc_sysobj_cache, this way we make sure
svc_sysobj_cache gets the status callback triggered.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: sysobj_core: filter sysobj attrs

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

librados: add move operations to IoCtx

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: chained cache shouldn't access service after it was unchained

This fixes a cleanup ordering issue. We can now have the chained cache
removed before and after the service is destructed.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: AioThrottle uses RGWSI_RADOS::Obj

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RADOS::Pool::create() for single pool

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RADOS::Pool::lookup() uses Pool's pool name

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add Obj::aio_operate() for read ops

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: rgw_rados_ref uses default copy/move

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: fix svc_zone compile with is_started()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: fix list_realms() and list_zones()

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix chained cache put if cache isn't initialized

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc: support raw storage init

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: move services shutdown around

fixes ordering

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: services shutdown

Explicitly make sure services are being shut down in appropriate order.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: unregister chained cache on shutdown

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc: hold services in unique_ptr

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't try to canonicalize raw obj

and other minor fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: remove unneeded code

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't include rgw_zone.h in svc_zone.h

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc: due to popular demand, get rid of service registry

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: api fixes following rebase

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: more code removal and api adjustments

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: chained cache doesn't init if not caching

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc: remove SystemObject impl

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc: fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: services init fixes

First load all depending services, then initialize all.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: minor adjustments

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw-admin: api adjustments

now compiles

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc: more changes, add svc_sync_modules

radosgw actually compiles now

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: rgw_zone api adjustments

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: add missing files

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc_finisher: create service

and other fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc_cache: more work

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc_notify: initial work

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: sysobj service: initial cache conversion

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc_sysobj: split core out

Split core sysobj interface from sysobj service so that cache could be
implemented.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: zone, sysobj services: add missing system obj calls

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: sys_obj service, move zone init to svc_zone

still wip

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc: more zone work, add zone_utils, quota services

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: more refactoring work, initial work on svc_rados

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc: initial work on zone_svc

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: basic rgw services registry

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #24719 from Devp00l/wip-issue-26861

mgr/dashboard: Remove _filterValue from CdFormGroup

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #24333 into master

* refs/pull/24333/head:
test: update librados tests for CLI arg syntax
librados: update for CLI arg format change
pybind: update python callers of force flags
mon: convert remaining confirmation flags to CephBool
ceph_argparse: introduce CephBool arguments
test: remove quirky argparse case
mgr,mon: use CephChoices for confirmation flags
test: add cases for CLI's --key=val style
mon: remove dead "cluster_snap" command
pybind: enable --keyword=arguments in ceph_argparse
ceph.in: some cleanups
ceph.in: misc cleanups
common: remove unused 'avail' field from commands
mon: fix help string for `osd crush rule create-replicated`

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24930 from tspmelo/wip-multiple-actions

mgr/dashboard: Fix duplicate tasks

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #24974 from rhcs-dashboard/36712-cluser-hosts-fix

mgr/dashboard: fix for 'Cluster >> Hosts' page

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve CRUSH map viewer

Display the OSD state using a label.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: SAML 2.0 support

Fixes: https://tracker.ceph.com/issues/24268
Signed-off-by: Ricardo Dias <rdias@suse.com>
Signed-off-by: Ricardo Marques <rimarques@suse.com>

rpm,deb: libradospp1-dev should depend on librados-dev

because librados.hpp `#include`s librados.h

Signed-off-by: Kefu Chai <kchai@redhat.com>

examples/librados: link c++ samples against libradospp

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24530 from alimaredia/wip-rgw-dpp-gc-lc

rgw: refactor logging in gc and lc

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #24510 from pritha-srivastava/wip-sts-buffer-overflow

rgw: Fix for buffer overflow in STS op_post().

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>