librbd: use the correct error code when the exclusive lock isn't locked

If the client is currently blacklisted, use -EBLACKLISTED, otherwise
use -EROFS.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: helper to retrieve the correct error code for read-only op

When the exclusive lock is unlocked, the error code should be
-EBLACKLISTED when the client is blacklisted, otherwise -EROFS.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: reacquire lock should properly handle failed watcher

If the watch has been lost, assume the lock has been lost but attempt
to reacquire it if and when the watch is re-established.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: fix improper indentation of 'ceph_assert' statements

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: assume lock is unlocked if blacklisted or object deleted

This will ensure that it's possible to potentially re-acquire the
lock should the blacklist expire before the image is closed.

Fixes: http://tracker.ceph.com/issues/34534
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: watcher should internally track blacklisted state

Since it will periodically attempt to re-acquire the watch,
it will know when the RADOS client has been blacklisted and
when the blacklist has been removed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: attempt to recover lost image watcher upon all failures

For example, if an image is blacklisted and the blacklist eventually
expires, the image should recover its watch.

Fixes: http://tracker.ceph.com/issues/34534
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24160 from jdurgin/wip-rados-lead

doc/dev/index: update rados lead

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23662 from dillaman/wip-24412

librbd: support v2 cloning across namespaces

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #24059 from cbodley/wip-rgw-opstate-rm

rgw, cls: remove cls_statelog and rgw opstate tracking

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #24143 into master

* refs/pull/24143/head:
qa/workunits/cephtool/test_kvstore_tool.sh: run test in ., not /tmp

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #24007 from tianshan/fix_35905

rgw: async sync_object and remove_object does not access coroutine me…

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23610 from XiangxiangMeng/fix-rgw-bucket-policy

rgw: policy: fix s3:x-amz-grant-read-acp keyword error

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #24128 from ricardoasmarques/wip-optional-user-password

mgr/dashboard: User password should be optional

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge PR #24145 into master

* refs/pull/24145/head:
qa/workunits/mon/caps.sh: wait longer to time out

Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge pull request #23544 from joke-lee/wip-multiobjdelete-num-limit

rgw: delete multi object num limit

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #24001 from joke-lee/doc_life_cycle

rgw: update cls_rgw.cc and cls_rgw_const.h

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24114 from robbat2/wip-35990

rgw: fix chunked-encoding for chunks >1MiB

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #24110 from cbodley/wip-35986

rgw: abort_bucket_multiparts() ignores individual NoSuchUpload errors

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

cls: remove cls_statelog and tests

Signed-off-by: Casey Bodley <cbodley@redhat.com>

osd: remove statelog from osd_class_load_list config

Signed-off-by: Casey Bodley <cbodley@redhat.com>

common: remove config variables related to rgw opstate

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: remove dependencies on cls_statelog

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: remove RGWOpState and related classes

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: remove opstate commands

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: remove test for /admin/opstate rest apis

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: remove rest handlers for /admin/opstate

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: remove unused opstate params from copy/fetch/stat

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: remove opstate tracking from fetch_remote_obj()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18578 from gaosibei/wip-cleanup-duplicate-include

RGW: remove duplicate include header files in rgw_rados.cc

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #23927 from joke-lee/WIP-RGW-x-amz-version-id

rgw: return x-amz-version-id: null when delete obj in versioning suspended bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: User password should be optional

Fixes: https://tracker.ceph.com/issues/36031
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge PR #23092 into master

* refs/pull/23092/head:
common: drop RefCountObject.cc and improve inlineability.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24109 into master

* refs/pull/24109/head:
doc: update docs for device management
mgr: make devicehealth always-on
mgr/devicehealth: do not create metrics pool on get-device-metrics
mgr/devicehealth: converge OPTIONS and DEFAULTS
mgr/devicehealth: squelch health warnings for unused devices
mgr/devicehealth: show-health-metrics -> get-health-metrics

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #24005 from p-na/refactor-deletion-modal-component

mgr/dashboard: Refactoring of `DeletionModalComponent`

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #24149 into master

* refs/pull/24149/head:
unittest_bufferlist: add unit test for sha1 from multiple raw buffers

Reviewed-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
Reviewed-by: Myoungwon Oh <omwmw@sk.com>

githubmap: update contributors

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/rbd: test clone v2 across namespaces

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: permit cross-namespace when using clone v2

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: use helper method to create librados::IoCtxs

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: replace librbd::ParentSpec with cls::rbd::ParentImageSpec

The newer struct includes support for pool namespaces.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: invoke new parent class API methods

If the OSDs don't support the new methods, revert to the legacy
class API methods.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: replace existing attach parent calls with new state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: new attach parent state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: replace existing detach parent calls with new state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: new detach parent state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: refactor flatten state machine to new code style

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add new namespace compatible parent image cls methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: break cls_rbd_snap on-disk struct backwards compatability

The cls methods are now storing normalized parent overlaps within the snapshot
if the OSDs are required to be Nautilus or later. Therefore, we can safely
break the on-disk format compatibility for those newer OSDs.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: migrate legacy snapshot on-disk format to normalized parent

The parent reference should be written once under the "parent" key,
so all updates to a snapshot should ensure the new normalized format
is used if Nautilus or later OSDs are required.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: cls_rbd_snap on-disk struct supports parent overlap

This is the first step to fix the denormalization of the parent
image spec references between the HEAD revision and the snapshot
revisions.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: cls_rbd_parent on-disk struct supports namespaces

If the OSDs are required to be Nautilus or later, it will automatically break
backwards compatibility of the structure to support pool namespaces.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ChildImageSpec now optionally includes child pool namespace

It will break version compatibility if used against older OSDs that
don't support RBD pool namespace support.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: new ParentImageSpec type to support parent namespaces

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: break composite cls rbd helpers into individual helpers

With the forthcoming need to use new methods to retrieve a clone's parent,
use per-method helpers to invoke the correct method within the image
refresh state machine.

Also used this time to merge newer cls rbd methods that should be supported
by all non-EOLed OSDs.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: corrected namespace indentation within cls_client

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: not valid to have different parents between image snapshots

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

objclass: expose the required OSD release setting

This can be utilized to determine when it's safe to enable new
cls features.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24076 from ifed01/wip-ifed-fix-repairer

os/bluestore: fix bloom filter num entry miscalculation in repairer

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24085 from liewegas/wip-32731

osd/osd_types: fix pg_t::contains() to check pool id too

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23721 from batrick/log-entry-stack

log: avoid heap allocations for most log entries

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24111 from gregsfortytwo/wip-35987-connection-buffers

msg/async: clean up local buffers on dispatch

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #24124 from tchaikov/wip-35969

rpm: use updated gperftools

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

doc/dev/index: update rados lead

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23754 from p-na/down-osd-raises-error

mgr/dashboard: Fix OSD down error display

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge PR #24098 into master

* refs/pull/24098/head:
Rename "include/assert.h"

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

unittest_bufferlist: add unit test for sha1 from multiple raw buffers

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #24112 into master

* refs/pull/24112/head:
osd: avoid unncecessary ref-counting in OSD::enqueue_op.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

qa/workunits/mon/caps.sh: wait longer to time out

5s -> 5m to gives us more leeway for when the mons are thrashing.

Also, *only* set this timeout when we expect a timeout.  If we don't,
wait forever.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #24090 from cbodley/wip-crimson-concurrent-dispatch

crimson/net: concurrent dispatch for SocketMessenger

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/workunits/cephtool/test_kvstore_tool.sh: run test in ., not /tmp

Notably, we can't make a bluestore store on a tmpfs.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #24092 into master

* refs/pull/24092/head:
mgr: Change signature of PyString_AsString to match return

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24057 into master

* refs/pull/24057/head:
src/common: add a unit test (bufferlist.sha1())
osd, src/common: return sha1 value if zero-length buffer.
src/common/buffer.cc: remove unnecessary copy in sha1()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24072 into master

* refs/pull/24072/head:
osd/PG: kill __queue_query__
osd/OSD: deprecate __project_pg_history__

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24104 into master

* refs/pull/24104/head:
mgr/diskprediction: add prototype diskprediction module

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #24103 from b-ranto/wip-dashboard-http

mgr/dashboard: Do not require cert for http

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #24015 from ricardoasmarques/wip-update-selection-on-refresh

mgr/dashboard: More configs for table `updateSelectionOnRefresh`

Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Rename SSLCherryPyConfig

The class does not always set up SSL, dropping the SSL bit.

Fixes: http://tracker.ceph.com/issues/36069
Signed-off-by: Boris Ranto <branto@redhat.com>

mgr/dashboard: Do not require cert for http

The ceph dashboard currently requires a SSL certificate even if it is
not running in the SSL mode since it is always querying for the
certificate file/key pair.

This patch fixes the behaviour by querying for the certificate file/key
only if it is running in the SSL mode.

Fixes: http://tracker.ceph.com/issues/36069
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #24031 from rzarzynski/wip-common-bl-drop-0c

common: drop the unused zero-copy facilities in ceph::bufferlist

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24120 into master

* refs/pull/24120/head:
doc: Fix Typos of Developer Guide

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24105 from dillaman/wip-24516

librbd: object map improperly flagged as invalidated

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #23981 from cbodley/wip-35830

rgw: raise debug level on redundant data sync error messages

Merge pull request #24088 from dzafman/wip-35982

qa/standalone: Standalone test corrections

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

common: get_data() of buffer_raw isn't virtual anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: make the get_data() of buffer_raw interface final.

This is just to ensure the just dropped buffer::raw_pipe
was the solely user of this facility. After successful
validation, we can drop `virtual` on the method entirely.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: drop can_zero_copy() from ceph::buffer interfaces.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: drop zero_copy_to_fd() from ceph::buffer interfaces.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: drop write_fd_zero_copy() from bufferlist.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: drop buffer::raw_pipe altogether with its factory method.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: drop buffer::list::read_fd_zero_copy.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common, test: drop unit tests for 0c in ceph::bufferlist.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/diskprediction: add prototype diskprediction module

This module is written by Rick Chen <rick.chen@prophetstor.com> and
provides both a built-in local predictor and a cloud mode that queries
a cloud service (provided by ProphetStor) to predict device failures.

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #24087 from rzarzynski/wip-common-buffer_raw-drop-page-align-getters

common: drop/mark-as-final getters of buffer::raw for palign.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24061 from liewegas/wip-35923

osd/PG: write pg epoch when resurrecting pg after delete vs merge race

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23306 from penglaiyxy/msg_branch

msg: ceph_abort() when there are enough accepter errors in msg server

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23955 from wjwithagen/wjw-fix-ceph-helpers.sh

test: Start using GNU awk and fix archiving directory

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Cleanup

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Refactoring of `DeletionModalComponent`

- Simpler variable names:

    Examples:

- `actionDescription` and `itemDescription` instead of `metaType`
- `bodyTemplate` instead of `description`
- `validationPattern` instead of `pattern`

    Some of these variable names have been generalized to ease the
    unification/generalization of dialog components:

- `submitAction` instead of `deletionMethod`

- Removed unique `setUp` method.

    Benefits:

    - Creation of the component is done as intended by the developers of
    the `ngx-boostrap` package and as expected by developers which use
    the package. The `setUp` method does not have to be called anymore
    on the `DeletionModalComponent` exclusively but instead the
    component is instantiated as all other modals. Property assignment
    on the instantiated object isn't handled by the `setUp` method
    anymore but by the `modalService`.

    - With the removal of the `setUp` method, some tests could be
    removed as well.

    - No need to pass the reference of the created modal to the modal
    manually.

    Preserved:

    - The provided check within the `setUp` method, which checked if the
    component had been correctly instantiated, has been moved to the
    `ngOnInit` method of the component.

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

rpm: use updated gperftools

make sure we only build with the higher version of gperftools on
distros where both 2.4 and 2.6.1 are packaged. see
https://git.centos.org/summary/rpms!gperftools.git . at the time of
writing, gperftools 2.6.1 is packaged for CentOS/RHEL 7, if gperftools
(>= 2.4) is required by Ceph, and user already has this version
installed, when new Ceph packages are installed, the updated gperftools
2.6.1 version won't be installed as a dependency. when launching
Ceph compiled with tcmalloc enabled, we will have

symbol lookup error: ceph-osd: undefined symbol: _ZdaPvm

so, by bumping up the required version of gperftools, the updated
gperftools will be installed.

see https://software.opensuse.org/package/gperftools, openSUSE/SLE offer
2.5. so they are safe at this moment.

Fixes: http://tracker.ceph.com/issues/35969
Signed-off-by: Kefu Chai <kchai@redhat.com>