rgw: code for correctly decoding permission policy and tags
in a JSON object for a role.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: implementation of forward_request_to_master()
for role REST APIs.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: code changes to include tenant in replicated metadata
for roles.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: adding roles to non master ops list.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: removing svc_role.h and svc_role.cc, and
references to RGWSI_Role and cleaning up code.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: oids must not be prefixed in case of read_info
and store_info since that gets taken care of the meta_be->put()
and meta_be->get().

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: adding a metadata section for roles.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: moving data members of RGWRole into RGWRoleInfo.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: shifting back be_handler and module initialization to svc role
rados.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: using put() method of metadata backend to ensure that role
metadata is written to md log in method store_info().

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: cleanup.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: code for handling mtime, object tracker and attrs
needed for RGWRoleMetadataHandler.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: removing do_start() method from
RGWRoleMetadataHandler and class RGWSI_Role_Module.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rgw: role metadata handler invokes RGWRole interfaces.
Removing references to RGW services and RGW Ctl.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

svc: role: use the stat system obj call for checking presence

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: tools: introduce `rgw_stat_system_obj`

Similar to the read and write system_obj interfaces this function stats an
object and returns the mtime & attrs if set.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role/svc cosmetic comments & ws cleanup

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: use the list roles from role svc classes

This keeps the implementation of storage seperate from the RGWRole class. Also
drop the get_roles_by_path_prefix static function as this is currently
implemented in the backend and can be directly accessed from role ctl classes.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

svc: role: make string consts static

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

svc: role: introduce list roles by path

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: use svc->delete_role for deleting

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: use a err msg string in RoleInfo class for logging

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw_role: remove unrelated functions

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

svc: role: improve delete functions & introduce delete_role

This deletes all the role related objects

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

svc: roles: set objv_tracker to null for sysobj calls

Since these aren't tracked via MDLog and merely ancillary objects

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: move populate from svc->role

As populate creates uuids etc which are only needed in direct calls to rgw_admin

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: use create_role from svc

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

svc: role: introduce create role

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: json fixes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: use ot

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

svc: role: move prefix to Module

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role fixes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: backend handler definition

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: md handler get_meta_obj defn

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: define md handler for roles

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: add RGWRoleMetadataObject

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: metadata: provide a constructor for RGWMetadataObject

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: drop rados_ctl and use only role_ctl

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: finish role class impl

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: modify store_name & path

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: base role md handler decl

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

svc: introduce role_rados service

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: start using role_ctl

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: svc: role_rados:  stub out get_path

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw_role: define RGWRoleCtl

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: role: basic ctl class defn

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: split RGWRole into an Info Class containing members

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: implement delete oids

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

svc: role: implement read/write role path & name

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

svc: role: implement get/put/delete Roles

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

svc: role: fix real_time put param

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: RGWRole: add a missing RGWRados forward declaration

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: introduce RGW Role Rados service

This handles the metadata for RGW Roles with the RADOS backend

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: svc: introduce a service for rgw_role

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #46165 from rishabh-d-dave/qa-omit-sudo

qa/cephfs: set omit_sudo False when sudo is set to True

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>

Merge pull request #46168 from rishabh-d-dave/fix-caps-helper

qa/cephfs: fix minor bug in caps_helper.py's run_mon_cap_tests()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #40434 from rishabh-d-dave/fs-refactor-method-in-mount

qa/cephfs: modify get_key_from_keyfile() in mount.py

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #46522 from tchaikov/wip-crimson-logging

crimson/osd: reset logger before exit

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #46416 from tchaikov/wip-debian-dh-python3

debian: python3 related cleanups

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

crimson/osd: reset logger before exit

* extract the code to set logging fstream into a dedicated function
* do not reset logging until the end of the seastar application.

before this change, `reset_logger` is created in the
`if (auto log_file = local_conf()->log_file; !log_file.empty())` branch,
so its life cycle ends when the `if` block ends. in other words,
the cerr fstream is used for logging after the `if` block ends.
this is not the expected behavior.

after this changge, `reset_logger` is created out of the `if` block.
so we won't reset the logger back to `cerr` until the lambda passed to
`seastar::async()` exits.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #46483 from yaarith/rook-telemetry

mgr/telemetry: add Rook data

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #46417 from xxhdx1985126/wip-gc-parallel-live_extent_retrieval

crimson/os/seastore/segment_cleaner: parallel live extents retrieval

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #46480 from cfsnyder/wip-cfsnyder-device-classes-in-service-spec

python-common: allow crush device class to be set from osd service spec

Reviewed-by: Adam King <adking@redhat.com>

test/crimson/seastore: add test case for parallel extent retrieval

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/os/seastore/cache: make access to Transaction::read_set atomic

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #46505 from rhcs-dashboard/fix-backports_main-main

script/ceph-backport.sh: deal with main branch

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: nmshelke <NOT@FOUND>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #46186 from rhcs-dashboard/add-daemon-logs

mgr/dashboard: Add daemon logs tab to Logs component

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

Merge pull request #46283 from MrFreezeex/mixin-config

ceph-mixin: fix linting issue and add cluster template support

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #46454 from idryomov/wip-rbd-unlink-newest-snap-at-capacity

librbd: unlink newest mirror snapshot when at capacity, bump capacity

Reviewed-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #46434 from idryomov/wip-rbd-preserve-non-primary-snap

rbd-mirror: don't prune non-primary snapshot when restarting delta sync

Reviewed-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
Reviewed-by: Mykola Golub <mgolub@suse.com>

qa/cephfs: modify get_key_from_keyfile() in mount.py

CephFSMount.get_key_from_keyfile() should raise an exception instead of
returning None if key is not found in keyring file.

Fixes: https://tracker.ceph.com/issues/50010
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #46411 from pcuzner/add-serial-numbers

cephadm: Add server serial info to gather-facts

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #46444 from rkachach/fix_issue_55800

mgr/cephadm: check if a service exists before trying to restart it

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #46445 from rkachach/fix_issue_55801

mgr/cephadm: capture exception when not able to list upgrade tags

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #46481 from guits/cephadm-custom-names-osd-adoption

cephadm: fix osd adoption with custom cluster name

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #39002 from ceph/wip-rgw-multisite-reshard

rgw multisite: bucket reshard work in progress

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #45470 from ceph/wip-setx

run-backend-api-tests.sh: set -x for Jenkins job debugging

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #44684 from zenomri/wip-omri-tracing-compiled

tracer: set tracing compiled in by default

script/ceph-backport.sh: deal with main branch

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #43216 from k0ste/fix_47537

doc/mgr: Document wildcard to expose Prometheus metrics for all RBD pools and namespaces

python-common: allow crush device class to be set from osd service spec

Adds crush_device_class parameter to DriveGroupSpec so that device class
can be set via service specs with cephadm.

Fixes: https://tracker.ceph.com/issues/55813
Signed-off-by: Cory Snyder <csnyder@iland.com>

Merge pull request #46019 from yushu20171007/fix_issue_55422

common: notify all when max backlog reached in OutputDataSocket

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #46501 from rhcs-dashboard/fix-55826-master

qa: fix teuthology master branch ref

Reviewed-by: amathuria <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

qa: fix teuthology master branch ref

Fixes: https://tracker.ceph.com/issues/55826
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

doc/mgr: Document wildcard to expose Prometheus metrics for all RBD pools and namespaces

Fixes: https://tracker.ceph.com/issues/47537
Signed-off-by: Konstantin Shalygin <k0ste@k0ste.ru>

Merge pull request #46272 from sshambar/bug-55664

cephadm: preserve cephadm user during RPM upgrade

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #46488 from jtlayton/teuth-branch-fix

qa: remove .teuthology_branch file

qa: remove .teuthology_branch file

This was originally added to help support the py2 -> py3 conversion.
That's long since complete so we should be able to just remove this file
now.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #46487 from jtlayton/teuth-branch-fix

qa: fix .teuthology_branch file in qa/

test/rgw/multisite: enable zonegroup resharding feature

qa/tasks/rgw_multisite.py uses 'zonegroup set' to create zonegroups from
their json format. this doesn't enable any of the supported zonegroup
features by default, so this adds the 'enabled_features' field to the
json representations

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa: fix .teuthology_branch file in qa/

According to teuthology-suite:

  -t <branch>, --teuthology-branch <branch>
                              The teuthology branch to run against.
                              Default value is determined in the next order.
                              There is TEUTH_BRANCH environment variable set.
                              There is `qa/.teuthology_branch` present in
                              the suite repo and contains non-empty string.
                              There is `teuthology_branch` present in one of
                              the user or system `teuthology.yaml` configuration
                              files respectively, otherwise use `main`.

The .teuthology_branch file in the qa/ dir currently points at "master".
Change it to point to "main".

Signed-off-by: Jeff Layton <jlayton@redhat.com>

cephadm: fix osd adoption with custom cluster name

When adopting Ceph OSD containers from a Ceph cluster with a custom name, it fails
because the name isn't propagated in unit.run.
The idea here is to change the lvm metadata and enforce 'ceph.cluster_name=ceph'
given that cephadm doesn't support custom names anyway.

Fixes: https://tracker.ceph.com/issues/55654
Signed-off-by: Adam King <adking@redhat.com>
Co-authored-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #46474 from idryomov/wip-rbd-codeowners

CODEOWNERS: add RBD team

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

CODEOWNERS: add RBD team

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

crimson/os/seastore/segment_cleaner: retrieve different live extents in parallel

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

mgr/cephadm: capture exception when not able to list upgrade tags
Fixes: https://tracker.ceph.com/issues/55801
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

mgr/telemetry: add Rook data

Add the first Rook data collection to telemetry's basic channel.

We choose to nag with this collection since we wish to know the volume
of Rook deployments in the wild.

The next Rook collections should have consecutive numbers (basic_rook_v02,
basic_rook_v03, ...).

See tracker below for more details.

Fixes: https://tracker.ceph.com/issues/55740
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>

Merge pull request #46382 from rzarzynski/wip-crimson-op-tracking-3

crimson/osd: add support for historic & slow op tracking

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #46437 from cyx1231st/wip-seastore-tune-and-fixes

crimson/os/seastore/segment_cleaner: tune and fixes around reclaiming

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #46193 from ljflores/wip-zero-detection-off-by-default

os/bluestore: turn bluestore zero block detection off by default