rgw: drop getting list-type when get_data is false

Signed-off-by: Jos Collin <jcollin@redhat.com>

rgw: Silence warning: control reaches end of non-void function

Build shows:
[ 53%] Building CXX object src/mds/CMakeFiles/mds.dir/JournalPointer.cc.o
ceph/src/rgw/rgw_rest_s3.cc: In member function ‘RGWOp* RGWHandler_REST_Bucket_S3::get_obj_op(bool)’:
ceph/src/rgw/rgw_rest_s3.cc:3588:5: warning: control reaches end of non-void function [-Wreturn-type]
   }   }
     ^
Fixes: 4ffc765c4c5debc665ade7769c4647c3a7278fd2
Fixes: http://tracker.ceph.com/issues/40747
Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #28800 from tchaikov/wip-crimson-mkfs

crimson: use given osd_fsid when mkfs

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #28774 from falcon78921/wip-doc-40575

doc: fixed broken link in Swift Settings section

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fixed broken link in Swift Settings section

Fixes: https://tracker.ceph.com/issues/40575
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

Merge pull request #28786 from tchaikov/wip-const-head

osd/PrimaryLogPG: more constness

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

crimson: use given osd_fsid when mkfs

MemStore does not have this feature, but CyanStore needs it. because
`qa/tasks/ceph.py` uses following steps when creating/starting an OSD

1. ceph-osd --mkfs --mkkey -i <osd_id> --monmap <monmap>
2. read the osd_fsid of osd.<osd_id>
3. ceph osd new <osd_fsid> <osd_id>

when we mkfs for the OSD, the osd_fsid is still unknown. so we cannot
use the configured one, as it is always empty. in that case, we need to
use a random uuid, and persist it to both ${osd_data}/fsid and
superblock.osd_fsid

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28719 from majianpeng/remove-unused-code-XioMessenger

msg/Message: Remove used code about XioMessenger.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28794 from tchaikov/wip-crimson-option-help-seastar

crimson/osd: add "--help-seastar" command line option

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/dev/crimson.rst: update with stuff related to seastar options

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28792 from tchaikov/wip-crimson-logging

common/config_values: set seastar logging level per that of ceph

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #28790 from tchaikov/wip-crimson-unknown-objectstore

crimson/osd: abort on unsupported objectstore type

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: add "--help-seastar" command line option

so we can

* have access to the available command line options offered by Seastar.
* tell if the executable we are playing around is ceph-osd or crimson-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev/crimson.rst: update logging related doc

as crimson's logging levels are now connected those of ceph, there is no
need to note down the difference between them anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/config_values: set seastar logging level as well

when the logging level of a subsys changes, update seastar accordingly

before this change, seastar logging levels of different logger are not
connected to ceph's logging levels of different subsystem.

after this change, they are connected. whenever a subsys's logging level
changes, the corresponding seastar logger is updated as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/dout.h: extract log level mapping out

so we can reuse it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: revert 9813f7d6

since we have a proper PeeringState implementation, there is no need to
fake an active+clean PG anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: abort on unsupported objectstore type

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add editor variables/options

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: implement "__ceph_abortf()"

the macro of `ceph_abort_msgf()` is implemented using it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add a space before function name

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: use get_osdmap_epoch()

always prefer using the interface.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/PrimaryLogPG: more constness

use `hobject_t::get_head()` instead of relying on the fact that head
object's snap is `CEPH_NOSNAP`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28780 from varshar16/wip-doc-update-sepia-link

doc: Updates link to Sepia lab

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28664 from linuxbox2/wip-rgwfile-faststats

rgw_file: introduce fast S3 Unix stats (immutable)

Merge pull request #28783 from cbodley/wip-rgw-object-lock-get-params

rgw: fix MalformedXML errors in PutBucketObjectLock/PutObjRetention

rgw_file: introduce fast S3 Unix stats (immutable)

For objects originating in S3/Swift, it is frequently acceptable
to synthesize c/mtime and size stats from meta attributes in the RGW
bucket index entry.

Synthetic stat behavior is not compatible with mutation of file
ownership and permissions via SETATTR.  In future, current xattr-based,
mutable, Unix stats may be moved to the bucket index.  For now, use
synthetic stats only when a new option rgw_nfs_s3_fast_attrs is present.

Fixes: http://tracker.ceph.com/issues/40456
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #27964 from Kriechi/fix-rgw-ldap-username

docs: fix rgw_ldap_dnattr username token

Merge pull request #28724 from ivancich/wip-abort-multipart-fix

rgw: multipart upload abort is best-effort

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28102 from albIN7/liv2

RGW:Listobjectsv2

rgw: fix MalformedXML errors in PutBucketObjectLock/PutObjRetention

RGWPutBucketObjectLock was not calling get_params(), so the 'data' it
was trying to parse was empty

RGWPutObjRetention was calling get_params() a second time, which
overwrote the 'data' from the first call

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #28491 into master

* refs/pull/28491/head:
doc/rados/operations: add safe-to-destroy check to OSD replacement workflow

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #28781 from ivancich/wip-fix-source-permissions

src/: remove execute permissions on nine source files

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28775 from neha-ojha/wip-doc-delete-sleep

doc/rados/configuration/osd-config-ref.rst: document osd_delete_sleep

Reviewed-by: Sage Weil <sage@redhat.com>

misc: remove execute permissions on nine source files

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #28565 from linuxbox2/wip-rgwfile-eof

rgw_file: fix readdir eof() calc--caller stop implies !eof

doc: Updates link to Sepia lab

Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #28761 from lixiaoy1/pmem_osd

os/bluestore: fix pmem osd build problem

Reviewed-by: Jason Dillaman <dllaman@redhat.com>

Merge pull request #28776 from badone/wip-minimimal-config-no-longer-valid

do_cmake.sh: Add a heading to the minimal config

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: fix pmem osd build problem

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>

do_cmake.sh: Add a heading to the minimal config

Post #28634 the minimal ceph config is invalid without a header.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #28771 from athanatos/sjust/wip-schedule-event-on-commit

crimson/osd/pg: start_operation for read_state, schedule_event_on_commit

Reviewed-by: Yingxin Cheng <yingxincheng@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/osd-config-ref.rst: document osd_delete_sleep

Signed-off-by: Neha Ojha <nojha@redhat.com>

crimson/osd/pg: start_operation for read_state, schedule_event_on_commit

This way they'll be holding the appropriate pipeline stage.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #28351 from trociny/wip-39968

 journal: add support for aligned appends

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28730 from trociny/wip-40519

rbd-mirror: handle duplicates in image sync throttler queue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28748 from trociny/wip-journal-player-handle_cache_rebalanced

journal: fix race between player shut down and cache rebalance

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc/rados/operations: add safe-to-destroy check to OSD replacement workflow

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #28706 from ajarr/wip-fs-subvolumes-enhancements

mgr/volumes: minor enhancements and fixes

Merge pull request #28653 from zhangsw/cleanup-doc-sample.conf

sample.ceph.conf: correct the default value of filestore merge threshold

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28749 from hydro-b/patch-1

doc/radosgw/compression.rst: fix typo

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/volumes: fix incorrect VolumeException arguments

Fixes: http://tracker.ceph.com/issues/40429
Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volumes: allow setting mode

... of fs subvolumes and subvolume groups during their creation.

Fixes: https://tracker.ceph.com/issues/40299
Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volumes: allow setting data pool layout

... of fs subvolumes and subvolume groups during their creation.

Fixes: https://tracker.ceph.com/issues/40431
Signed-off-by: Ramana Raja <rraja@redhat.com>

Merge pull request #28593 from smithfarm/wip-40398

cmake: set empty INSTALL_RPATH on crypto shared libs

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28623 from cyx1231st/wip-seastar-msgr-socket

test/crimson: add unit-test for ceph::net::Socket

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/net: added support for new errors

connection_aborted and invalid_argument.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>

test/crimson: add unit-test for ceph::net::Socket

Added unittest_seastar_socket to make sure that seastar network layer
works as expected.

Added missing error code invalid_argument and address_in_use.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>

doc/radosgw/compression.rst: fix typo

Signed-off-by: hydro-b <stefan@kooman.org>

Merge pull request #28747 from tchaikov/wip-more-than-activate

crimson/osd: update peering_state in PG::on_activate_complete()

Reviewed-by: Samuel Just <sjust@redhat.com>

rgw: minor code clean-up

Adding some const correctness and reformatting.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

rgw: allow multipart upload abort to proceed

Cleaning up after aborting unfinished multipart uploads is a
best-effort process. Due to the history of the multipart upload, not
all components may be found and the process needs to be robust to
that.

Presently, if a component is not found after the final call to
list_bucket_multiparts, the error propogates to the caller, but not if
the component is not found in an earlier call. This change does not
propogate that error up ever.

Additional logging is also added to multipart-related functions.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

journal: fix race between player shut down and cache rebalance

Signed-off-by: Mykola Golub <mgolub@suse.com>

crimson/osd: update peering_state in PG::on_activate_complete()

so that the PG can mark itself clean when it is activated and does not
need recovery.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: use get_osdmap_epoch()

would be better to use the interfaces of the PG class, instead of
relying on its implementation.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28634 from tchaikov/wip-22227

common, auth: use boost::spirit to parse ceph.conf, escape quotes in exported auths

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

common/ConfUtils: implement trim helpers using boost

remove unused trim_whitespace() and its tests. as it is not used anymore

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits: add auth caps allow command case

this case introduces multiple quotes in caps line
it will trigger the bug like http://tracker.ceph.com/issues/22227

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

auth/KeyRing: escape quotes around commands

so the exported auth can be re-imported.

Fixes: http://tracker.ceph.com/issues/22227
Signed-off-by: Kefu Chai <kchai@redhat.com>

src/: parser returns up to one error

since config parser bails out at seeing the first error, there is no
need to offer the interfaces to report multiple errors.

Signed-off-by: Kefu Chai <kchai@redhat.com>

src: use spirit based ceph.conf parser

before this change, the ceph.conf parser was implemented using a
hand-written state machine. which works fine. but it's hard to
understand and hence difficult to maintain.

in this change, it's rewritten using boost::spirit. which well, is also
a state machine under the hood, but the declarative pattern matching
rules help the human readers to understand the machinary.

before this change, the state machine is able to skip the lines with
invalid syntax and spit a warning message for each of them. after this
change, the parser will just bail out when it fails to match the rules.

and the line continuation marker, i.e., "\" is simply skipped, so it's
allowed to have nothing after the backslash in the value of key-value
pair.

changes are listed as following:

* invalid UTF-8 key/value is now treated as an error
* backslash at the very end is now allowed
* repeated settings are tolerated, and no warnings will be printed.
  the last one wins. before this change, we will print out warning
  messages at seeing lines with duplicated keys.
* malformed section header is now an error
* a setting line which does not parse is now an error
* unmatched quotes in setting value is now an error
* "\" following an empty line is allowed now.
* the format of error message is changed, so the cli test is updated
  accordingly.
* [global] section is not added by default. unless there is a single
  line in the conf file. this is only for the convenience of testing.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28695 from kamoltat/wip-mgr-progress-fix-bug-osd-in-out

mgr/progress: Bug fix complete event when OSD marked in

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #28689 from tchaikov/wip-crimson-add-me-to-crush

crimson/osd: add osd to crush when it boots

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #28688 from ifed01/wip-ifed-fix-no-compress

os/bluestore: load OSD all compression settings unconditionally.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #28660 from liewegas/wip-40451

osd/PG: do not queue scrub if PG is not active when unblock

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #28601 from rzarzynski/wip-crimson-os-omit-block-write

os/memstore, crimson/os: introduce memstore_debug_omit_block_device_write

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28510 from liuchang0812/fix-arg-parser

common: sort best-matched commond by req argument count

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28018 from hjwsm1989/wip-39625

mon: take the mon lock in handle_conf_change

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27627 from cxytt/fix-add-omap-tail

os/bluestore: create the tail when first set FLAG_OMAP

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #28718 from varshar16/wip-cephfs-shell-typo-mounting

cephfs-shell: Fix typo for mounting

Reviewed-by: Jos Collin <jcollin@redhat.com>

rbd-mirror: handle duplicates in image sync throttler queue

Fixes: http://tracker.ceph.com/issues/40519
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #28703 from tchaikov/wip-ceph.in-asan

ceph.in: do not preload asan even if not needed

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28725 from cbodley/wip-cmake-ubsan

cmake: fix WITH_UBSAN

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28395 from athanatos/sjust/wip-crimson-op

Add structures for tracking in progress operations

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson: fix state machine operations in advance_pg_to and pg creation

Both pg creation and advance_pg_to process statemachine events and
therefore need to be under the pg process pipeline stage.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #27285 from shangdehao1/wip-librbd-shared-read-only-cache-hook

librbd: shared read-only cache hook

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: refine re-connect implement and init method

- add new status to OpenImageRequest, including
  send_parent_cache and handle_parent_cache.
- refine re-connect to remove race condition.
- fixed read error bug

Signed-off-by: Dehao Shang <dehao.shang@intel.com>

doc: fix example command on creating ceph user

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

librbd: change init method of Parent cache to asynchronous

Signed-off-by: Dehao Shang <dehao.shang@intel.com>

librbd: add read_object method to ParentCache

Also remove SharedPersistentObjectCacher class

Signed-off-by: Dehao Shang <dehao.shang@intel.com>

librbd: refine hard code and remove dead code

Signed-off-by: Dehao Shang <dehao.shang@intel.com>

librbd: add lock to resovle race condition

possible race condition w/ multiple concurrent attempts to
re-create the cache session

Signed-off-by: Dehao Shang <dehao.shang@intel.com>

doc: better format for rbd persistent cache

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

librbd: remove parent cache config from ImageCtx

Signed-off-by: Dehao Shang <dehao.shang@intel.com>

librbd: cleanup parent cache

- remove compiler warning.
- add error log level.
- modify tool/immutable_obj_cache unit testing.

Signed-off-by: Dehao Shang <dehao.shang@intel.com>

librbd: modify Parent cache interface

follow the recent ObjectDispatchInterface to adjust parent cache hook

Signed-off-by: Dehao Shang <dehao.shang@intel.com>

doc: updating doc for rbd shared parent cache

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

librbd: move SharedPersistentObjectCacher to ParentCacheObjectDispatch

Signed-off-by: Dehao Shang <dehao.shang@intel.com>

librbd: add re-connect and infligh enable feature

- reconnect : when session don't work or RO daemon crash,
  parent cache will try to re-connect daemon.
- in-fligh enable RO daemon.

Signed-off-by: Dehao Shang <dehao.shang@intel.com>

tools: modify connect implement of RO cache

- In order to support re-connect, add async connect at CacheClient.
- sync connect is based on C_SaferCond + async_connect.

Signed-off-by: Dehao Shang <dehao.shang@intel.com>