crimson/osd: implement PGLSFilter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: OpsExecuter differentiates read and modifying operations.

This is necessary to pass ClsHello.BadMethods.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: implement CEPH_OSD_OP_GETXATTR and cls_cxx_getxattr().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: differentiate ENOENT during do_osd_call().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: implement cls_cxx_read2().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: implement cls_cxx_setxattr().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: implement CEPH_OSD_OP_SETXATTR.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os: port get_max_attr_name_length() from MemStore.

This method is an enabler for CEPH_OSD_OP_SETXATTR.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

objclass, osd: drop unused variable.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: implement cls_cxx_write_full().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: introduce OpsExecuter to uniform calling CEPH_OSD_OPS.

OSD has two entry points for executing CEPH_OSD_OP_*:
  1. the MOSDOp message handler,
  2. the Object Class API (cls_* and cls_cxx_* functions).

We definitely want to address these two users without code
duplication. However, exposing the entire PG to Obj Class
would break encapsulation. Moreover, there is difference
in life times between PG and sequence-of-operations-from-
MOSDOp.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: implement cls_cxx_stat().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: implement CEPH_OSD_OP_STAT.

This commit is enabler of cls_cxx_stat(), and thus the RecordHello
and ClsHello unit test.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: implement CEPH_OSD_OP_CALL.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: rename crimson_error and introduce namespace.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: exceptions derive from std::system_error now.

This change is be useful especially for CEPH_OSD_OP_CALL
which will be brought by further commits. The issue here
is that the Ceph Classes will be based on existing iface
handling errors with usual, int-based ret codes. These
codes needs to be glued with the existing handling  in
pg::do_osd_ops().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: put extra debugs into do_osd_op().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osd: switch ClassHandler::get_method() to std::string.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: mock OSD-specific parts of the objclass API.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

objclass, crimson: initialize the ClassHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

objclass, osd: dissect the OSD-specific parts from class_api.cc.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osd: improve formatting and refactor ClassHandler.h a bit.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #29167 into master

* refs/pull/29167/head:
client: return -eio when sync file which unsafe reqs has been dropped

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29572 into master

* refs/pull/29572/head:
mds: Reorganize class members in FSMap header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29796 from trociny/wip-journal-player-handle_cache_rebalanced2

journal: fix race between player shut down and cache rebalance

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29775 from trociny/wip-41229

librbd: always try to acquire exclusive lock when removing image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29459 from zy751713126/config_set

pybind/rbd: add config_set/get/remove api in rbd.pyx

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #29715 into master

* refs/pull/29715/head:
qa: fix broken ceph.restart marking of OSDs down
qa: add debugging failed osd-release setting

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29821 into master

* refs/pull/29821/head:
qa: stop DaemonWatchdog for each cluster in daemon roles

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #29575 into master

* refs/pull/29575/head:
objclass, osd: improve const-correctness of PGLSFilter.
common: add bl::contents_equal() override for void* + size_t.
osd: refactor manufacturing of PGLSFilter.
osd: don't carry PGLSFilter between multiple ops in MOSDOp.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #28727 into master

* refs/pull/28727/head:
test/crimson: resolve name collision
test: switch to ldout; let users specify mon debug level
test: add new ElectionLogic unit test framework
elector: const-ify a bunch of functions
elector: swap order of parameters in ElectionLogic::receive_propose
elector: Update Elector and ElectionLogic function documentation
elector: persist the epoch in bump_epoch()
elector: make some more ElectionLogic members private
elector: fix privacy and restore dout in Elector
elector: don't clear peer_info in bump_epoch()
elector: split ElectionLogic into its own compilation unit
elector: move all the elector callouts into the Elector
elector: make ElectionLogic private to Elector; undo most public shenanigans
elector: create declare_standlone_victory in Elector/Logic for Monitor
elector: make ElectionLogic::declare_victory private
elector: route _bump_epoch through the interface-to-be
elector: rename handle_propose_logic -> receive_propose
elector: hoist handle_victory into ElectionLogic
elector: hoist handle_ack into ElectionLogic
elector: hoist victory into ElectionLogic
elector: hoist expire into ElectionLogic
elector: hoist start into ElectionLogic
elector: hoist participating into ElectionLogic
elector: hoist init into ElectionLogic
elector: hoist defer into ElectionLogic
elector: split handle_propose in two and hoist into ElectionLogic
elector: hoist bump_epoch into ElectionLogic
elector: store accessors for ElectionLogic
elector: hoist Elector data bits out into a new ElectionLogic class
mon: Rearrange Paxos::dispatch to be a little cleaner

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #15183 into master

* refs/pull/15183/head:
kv/rocksdb: support rmrange unconditionally
cls/rgw: rgw_bi_log_trim() uses cls_cxx_map_remove_range()
cls/log: cls_log_trim() uses cls_cxx_map_remove_range()
test/cls: add cls_log.trim_by_marker test
test/cls: test_cls_log doesn't allocate ObjectOperations
test/cls: test_cls_log uses fixture for temporary pool
test/cls: add cls_rgw.bi_log_trim test
cls/rgw: expose cls_rgw_bilog_list/trim() for single shard
test/cls: test_cls_rgw uses cls_rgw_obj_key
test/cls: test_cls_rgw doesn't allocate ObjectOperations
test/cls: test_cls_rgw uses fixture for temporary pool
objclass: add cls_cxx_map_remove_range()
librados: add rados_write_op_omap_rm_range2()
osdc: add Objecter omap_rm_range()
osd: add CEPH_OSD_OP_OMAPRMKEYRANGE to do_osd_ops()
osd: add omap_rmkeyrange() to PGTransaction
os: add bufferlist overload for omap_rmkeyrange()
tracing: add do_osd_op_pre_omaprmkeyrange
rados: add CEPH_OSD_OP_OMAPRMKEYRANGE

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29778 from cbodley/wip-41212

vstart: move [client.rgw] config into [client]

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

mgr/dashboard: User Management E2E tests (#29641)

mgr/dashboard: User Management E2E tests

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: run-backend-api-tests.sh CI improvements (#29504)

mgr/dashboard: run-backend-api-tests.sh CI improvements

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #29590 from Aran85/fix_proc_replica_log

osd: merge replica log on primary need according to replica log's crt

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29747 from liewegas/wip-39546

osd/PeeringState: do not complain about past_intervals constrained by oldest epoch

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29624 from NancySu05/osdmonitor_markmedown

mon:C_AckMarkedDown has not handled the Callback Arguments

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29738 from ifed01/wip-ifed-alloc-cleanup

os/bluestore: minor improvements/cleanup around allocator

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #29614 from votdev/issue_41205

mgr/dashboard: Access control database does not restore disabled users correctly

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

Merge pull request #29146 from badone/wip-tracker-40835-OSDCap.PoolClassRNS-abort

osd/OSDCap: Check for empty namespace

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25697 from Aran85/fix-onode-trim

os/bluestore: more aggressive deferred submit when onode trim skipping

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #28488 from liuchang0812/show-pool-id-in-pool-ls-cmd

mon: show pool id in pool ls command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24636 from rzarzynski/wip-denc-container_base

denc: slightly optimize container_base::bound_encode

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29756 from Aran85/fix-repair-object

osd: clear PG_STATE_CLEAN when repair object

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #29806 into master

* refs/pull/29806/head:
mgr/BaseMgrModule: tolerate Int or Long for health 'count'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29298 from zhangsw/rgw-fix-bug-listobjv2-startafter

rgw: continuationToken or startAfter shouldn't be returned if not specified

Merge PR #29780 into master

* refs/pull/29780/head:
osd/PeeringState: semi-colon after DECLARE_LOCALS
osd/PeeringState: on_new_interval on child PG after split

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #29774 into master

* refs/pull/29774/head:
qa/standalone/scrub/osd-scrub-snaps: snapmapper omap is now 'm'

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #29807 into master

* refs/pull/29807/head:
mgr/pg_autoscaler: fix race with pool deletion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: stop DaemonWatchdog for each cluster in daemon roles

Fixes: https://tracker.ceph.com/issues/41398
Introduced-by: 08b99eef277b00a3ea423cbf085bd114a805813f
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: fix broken ceph.restart marking of OSDs down

Sage noticed `osd down` was not being performed. Bug was that the role
format had changed so splitting no longer worked correctly.

Fixes: https://tracker.ceph.com/issues/40773
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17719 from mikulely/fix-usage-stats

rgw: distinguish different get_usage for usage log

Reviewed-by: Robin H. Johnson <rjohnson@digitalocean.com>

qa: add debugging failed osd-release setting

See-also: https://tracker.ceph.com/issues/40773
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: run-backend-api-tests.sh CI improvements

As there is now a jenkins job to run this script
(see https://github.com/ceph/ceph-build/pull/1351),
this refactoring adapt the script to be run in a jenkins job as well as locally.

Signed-off-by: alfonsomthd <almartin@redhat.com>

Merge pull request #29544 from tchaikov/wip-doc-search-CSP

doc: always load resources via HTTPS

Reviewed-by: Tiago Melo <tmelo@suse.com>

doc: always load resources via HTTPS

Signed-off-by: Tiago Melo <tmelo@suse.com>

pybind/rbd: add config_image_set/get/remove test case

Signed-off-by: Zheng Yin <zhengyin@cmss.chinamobile.com>

pybind/rbd: add config_image_set/get/remove api in rbd.pyx

Signed-off-by: Zheng Yin <zhengyin@cmss.chinamobile.com>

Merge pull request #29755 from xiexingguo/wip-inc-recovery-4

osd: do not invalidate clear_regions of missing item at boot

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mgr/BaseMgrModule: tolerate Int or Long for health 'count'

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #29804 from alfredodeza/wip-rm41378

ceph-volume tests set the noninteractive flag for Debian

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #29744 into master

* refs/pull/29744/head:
qa/run-standalone.sh: fix python path
qa/standalone/mon/health-mute.sh: fix up rachet test
qa/standalone/mon/health-mute.sh: s/kill daemons/kill_daemons/

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #29749 into master

* refs/pull/29749/head:
mon/HealthMonitor: remove unused label

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29757 into master

* refs/pull/29757/head:
osd: always initialize local variable

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29763 into master

* refs/pull/29763/head:
qa/suites/rados: whitelist POOL_APP_NOT_ENABLED warning

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/pg_autoscaler: fix race with pool deletion

The pool_stats map comes from a get('df') that may not include a pool
because it was just deleted.

Fixes: https://tracker.ceph.com/issues/41386
Signed-off-by: Sage Weil <sage@redhat.com>

ceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge PR #28378 into master

* refs/pull/28378/head:
qa/tasks: introduce Thrasher base class
qa/tasks: Fix typo
qa/tasks: manage thrashers
qa/tasks: start DaemonWatchdog when ceph starts
qa/tasks: make watch and bark handle more daemons
qa/tasks: move DaemonWatchdog to new file

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29773 from dillaman/wip-41352

pybind/mgr/rbd_support: fix missing variable in error path

Reviewed-by: Mykola Golub <mgolub@suse.com>

journal: fix race between player shut down and cache rebalance

25a23364 was supposed to fix this race, but it was not enough:
there was still a window between `prefetch` is queued for
execution in handle_cache_rebalanced and is actually executed,
during which shut_down can be called and completed.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #29659 from jan--f/c-v-simple-functional-no-lvm-zap

ceph-volume: don't try to test lvm zap on simple tests

qa/tasks: introduce Thrasher base class

* Introduced a Thrasher base class.
* Updated thrashers to inherit from Thrasher.
* Replaced the magic variable e with Thrasher.exception as per the discussion.
  Now the exception variable sets by default as the thrashers are inheriting
  from the Thrasher class.

Fixes: https://github.com/ceph/ceph/pull/28378#discussion_r309337928
Fixes: https://tracker.ceph.com/issues/41133
Signed-off-by: Jos Collin <jcollin@redhat.com>

rgw: distinguish different get_usage for usage log

get_usage op via s3 endpoint are not the same as get_usage
via admin endpoint in the rgw usage log categories.

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

mon/HealthMonitor: remove unused label

move the whole sanity checks into `HealthMonitor::preprocess_command()`.

this change silences warning of:

warning: label 'reply' defined but not used [-Wunused-label]

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/crimson: resolve name collision

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd/PeeringState: semi-colon after DECLARE_LOCALS

This screws with emacs' auto-indent.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PeeringState: on_new_interval on child PG after split

This was broken as part of the PeeringState refactor, see
ace6655f955837b104f307183fc0b988b7880c04.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #29717 into master

* refs/pull/29717/head:
os/bluestore: prefix omap of temp objects by real pool

Reviewed-by: Sage Weil <sage@redhat.com>

qa/standalone/scrub/osd-scrub-snaps: snapmapper omap is now 'm'

...due to per-pool omap.

Fixes 91f533be71e5fd9b2c0135a5b54d663425a1d9c4

Fixes: https://tracker.ceph.com/issues/41353
Signed-off-by: Sage Weil <sage@redhat.com>

vstart: move [client.rgw] config into [client]

common rgw config was moved into [client.rgw] with the intent to
cover radosgw in multisite tests too, but this broke the default case
where radosgws are named after [client.rgw.port]. move the common stuff
to [client] so it does actually apply to both

Fixes: https://tracker.ceph.com/issues/41212
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #29759 into master

* refs/pull/29759/head:
qa/tasks: fixed typo in the comment

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29625 from kamoltat/wip-test-progress-fix-bug-#29385

mgr/progress/module.py: s/events/_events/

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29704 from tchaikov/wip-ceph-objectstore-tool

ceph-objectstore-tool: return 0 if incmap is sane

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #29655 from trociny/wip-40923

qa/workunits/rbd: stress test `rbd mirror pool status --verbose`

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr/rbd_support: fix missing variable in error path

Fixes: https://tracker.ceph.com/issues/41352
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: always try to acquire exclusive lock when removing image

We want it to use object map for fast removal.

Fixes: https://tracker.ceph.com/issues/41229
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #29376 from Songweibin/wip-rbd-display-id

rbd/action: display image id in rbd du/list output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29743 from smithfarm/wip-ceph-backport-https

script/ceph-backport.sh: carry https through to logical conclusion

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #29653 from lixiaoy1/AsyncOpTracker_dup

librbd: Remove duplicated AsyncOpTracker in librbd/Utils.h

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29094 from cbodley/wip-40806

radosgw-admin: bucket sync status not 'caught up' during full sync

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

osd/PeeringState: do not complain about past_intervals constrained by oldest epoch

The start of the required interval has a floor set by the oldest osdmap
epoch we have.  That can lead to an invalid/empty required interval
(because the start is >= the end), but the PG may still have past
intervals.  That can be cause by a slow PG deletion.

No need to complain about this harmless condition.

Fixes: https://tracker.ceph.com/issues/39546
Signed-off-by: Sage Weil <sage@redhat.com>

scripts/ceph-backport.sh: always use https://tracker.ceph.com

Completing the wave of fixes to this script in the wake of
https://tracker.ceph.com/issues/38764, this commit replaces
"http" with "https" in the comments and puts the Redmine endpoint
into a variable, along with some other cleanups.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #29762 from alfredodeza/bz-1738379

ceph-volume: use the OSD identifier when reporting success

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

qa/suites/rados: whitelist POOL_APP_NOT_ENABLED warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: User Management E2E tests

Test breadcrumbs for user management pages
Test creating, editing and deleting a user
Test creating, editing and deleting a role

Fixes: https://tracker.ceph.com/issues/41231
Fixes: https://tracker.ceph.com/issues/41232
Fixes: https://tracker.ceph.com/issues/41233
Signed-off-by: Adam King <adking@redhat.com>
Signed-off-by: Rafael Quintero <rquinter@redhat.com>

mgr/dashboard: login screen language dropdown enhancement (#29636)

mgr/dashboard: login screen language dropdown enhancement

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29737 from tspmelo/wip-label-alignment

mgr/dashboard: Vertically align the "Refresh" label

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Exclude some folders in the frontend project (#29708)

mgr/dashboard: Exclude some folders in the frontend project

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

ceph-volume devices.lvm zap use the identifier to report success

Signed-off-by: Alfredo Deza <adeza@redhat.com>