rgw: rgwgc:process coredump in some special case。 Gc processes obja, objb, objc in order and pool of objb is deleted (obja and objc is in the same pool and pool exits). RGW will coredump as ctx->io_ctx_impl is an empty point during delete objc.
Fixes:http://tracker.ceph.com/issues/23199

Signed-off-by: zhaokun <develop@hikdata.com>

Merge pull request #17789 from shashalu/fix-bucket-link-dm

rgw: fix version bucket stats

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21622 from tianshan/misc_es_fix

rgw: some fix for es sync

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #24162 from zhangsw/cleanup-rgw-acl_op

rgw: remove repetitive conditional statement in RGWHandler_REST_Obj_S3

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24683 from linuxbox2/wip-v2-url-meta

rgw: handle S3 version 2 pre-signed urls with meta-data

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25142 from ivancich/wip-rgw-reshard-cleanup

rgw: reshard clean-up and associated commits

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #25406 from taoCH/wip-librgw-sync-user-regularly

rgw_file: user info never synced since librgw init

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #25423 from p-na/wip-pna-fix-pool-edit

mgr/dashboard: Fix renaming of pools

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #25208 from cbodley/wip-rgw-getobj-throttle

rgw: adapt AioThrottle for RGWGetObj
yay!

Merge PR #25345 into master

* refs/pull/25345/head:
qa/suites: fix bluestore links
qa/objectstore: bluestore -> bluestore-{bitmap,stupid}

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #25378 into master

* refs/pull/25378/head:
mgr/hello: add serve() method

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #25382 into master

* refs/pull/25382/head:
mon/OSDMonitor: include host, container info for 'osd find' command

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #25301 from wjwithagen/wjw-fix-dtor-pair

rbd: workaround for llvm linker problem, avoid std:pair dtor

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25265 from trociny/wip-rbd-fuse-namespace

rbd-fuse: namespace support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24968 from s0nea/wip-dashboard-osd-recv-speed-profiles

mgr/dashboard: add profiles to set cluster's rebuild performance

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Fix renaming of pools

The validation fails to allow dots in pool names and as such, it wasn't
allowed to create or rename pools with dots in their names.

Fixes: https://tracker.ceph.com/issues/37534
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #25409 from tchaikov/wip-public-iterator-traits

buffer: mark the iterator traits "public"

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #25266 from trociny/wip-24608

rbd-ggate: support namespaces

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

buffer: mark the iterator traits "public"

the iterator traits should be accessible from non-dervative classes.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25230 from runsisi/wip-cleanup-objecter

osdc/Objecter: no need null pointer check for op->session anymore

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #23550 into master

* refs/pull/23550/head:
auth: Kerberos authentication

Merge pull request #25166 from kungf/journal_remove_full

journal: allow remove set when jounal pool is full

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25175 from trociny/wip-36710

librbd: restart io if migration parent gone

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25260 from trociny/wip-24609

rbd-nbd: support namespaces

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25371 from trociny/wip-osd-query-subkey-types

 osd: support more dynamic perf query subkey types

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25151 from trociny/wip-osd-perf-query-limit

mgr: add limit param to osd perf query

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25379 from tchaikov/wip-37508

librbd: reset snaps in rbd_snap_list()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/suites: fix bluestore links

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #24882 from rzarzynski/wip-bl-hypercombined

common: hypercombined bufferlist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

rgw: store ref to librados::IoCtx in rgw::AioResult

the IoCtx has to outlive its aio completions

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add const overload of RGWSI_RADOS::Obj::get_ref()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: rgw_rados_ref contains rgw_raw_obj

rgw_rados_ref duplicated each of the fields from rgw_raw_obj

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: use AioThrottle for Object::Read::iterate() ops

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: clean up iterate_obj() callbacks

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add append_atomic_test() overload for RGWObjState

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add id to distinguish between Aio results

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: Aio reads return bufferlist in result

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: move aio throttles to namespace rgw

they're useful outside of putobj

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25365 from xiexingguo/wip-37493

osd/OSDMap: fix upmap mis-killing for erasure-coded PGs

Merge pull request #25262 from aclamk/wip-kvstore-tool-dump

ceph-kvstore-tool: dump fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25383 from dmick/wip-container-name

common/util.cc: add CONTAINER_NAME processing for metadata

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25400 from dillaman/wip-ceph-cli

ceph: do not raise StopIteration within generator

Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd: workaround for llvm linker proplem, avoid std:pair dtor

When linking with llvm linker on FreeBSD:
```
/usr/bin/ld.lld: error: undefined symbol: std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool>::~pair()
>>> referenced by Image.cc:349 (/home/jenkins/workspace/ceph-master/src/librbd/api/Image.cc:349)
>>>               Image.cc.o:(librbd::api::Image<librbd::ImageCtx>::list_children(librbd::ImageCtx*, cls::rbd::ParentImageSpec const&, std::__1::vector<librbd::linked_image_spec_t, std::__1::allocator<librbd::linked_image_spec_t> >*)) in archive ../../../lib/librbd_internal.a
```

Which is probably because the compiler calls a dtor on the just created pair.
But the FreeBSD linker does not find a dtor in the any of the libraries.
This rewrite build and links without the linker error.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

mgr/dashboard: show config option description via cd-helper

Fixes: https://tracker.ceph.com/issues/37406
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: add OSD recovery speed form

Add a form to set the OSD recovery speed. There are three predefined
priorities available: 'Low', 'Default' and 'High'.
By setting the priority the config options 'osd_max_backfills',
'osd_recovery_max_active', 'osd_recovery_max_single_start',
'osd_recovery_sleep' will be updated with predefined values.
It's also possible to set custom values.

Fixes: https://tracker.ceph.com/issues/35540
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #25392 from majianpeng/async-rdma

msg/async/rdma: Delete duplicate header file

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw_file: user info never synced since librgw init

Fixed: https://tracker.ceph.com/issues/37527

Signed-off-by: Tao Chen <chentao@umcloud.com>

qa/objectstore: bluestore -> bluestore-{bitmap,stupid}

The new default is bitmap, so we were testing bitmap twice.  Instead,
explicitly call out stupid and bitmap cases so a future default change
won't break coverage.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph: do not raise StopIteration within generator

This is implicit when a generator exits and under Python 3
it results in a "StopIteration" exception being raised when
"ceph --help" is run.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #24129 into master

* refs/pull/24129/head:
common: drop ::length() from other implementations of the OpQueue.
common, osd: kill OpQueue::length() to minimize inter-core traffic.

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>

Merge PR #25388 into master

* refs/pull/25388/head:
mon/OSDMonitor: do not no_reply on ready_to_merge message

Reviewed-by: João Eduardo Luís <joao@suse.de>

mgr/osd_perf_query: example how limit can be used

Signed-off-by: Mykola Golub <mgolub@suse.com>

osd: make use of osd perf query's limit param

Signed-off-by: Mykola Golub <mgolub@suse.com>

mgr: add limit param to osd perf query

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #25192 from ProgVal/doc-rbd-features

rbd: Document new RBD feature flags and version support

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: add config option bulk_set

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #25359 from zmc/wip-dashboard-fix-teuth-runner

mgr/dashboard: Fix PYTHONPATH for test runner

Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #25231 from rhcs-dashboard/37371-logs-links-permissions

mgr/dashboard: 'Logs' links permission in Landing Page

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #25322 from Devp00l/wip-issue-37468

mgr/dashboard: Document custom RESTController endpoints

Reviewed-by: Tatjana Dehler <tdehler@suse.com>

msg/async/rdma: Delete duplicate header file

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

rbd: bench snapshot if it was specified

Signed-off-by: Mykola Golub <mgolub@suse.com>

mgr/osd_perf_query: add query to test all supported subkeys

Signed-off-by: Mykola Golub <mgolub@suse.com>

osd: support more dynamic perf query subkey types

Signed-off-by: Mykola Golub <mgolub@suse.com>

common: drop the doubly-linked-list compatibility of bl::buffers_t.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: switch bl::buffers_t to a singly-linked list. Drop bi::list.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

bluestore/tools: Add option 'dump' to ceph-kvstore-tool to print both key and values.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #25335 from ifed01/wip-ifed-fix-enospc

os/bluestore: fix and unify log output on allocation failure

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25343 from liewegas/wip-36612

msg/async: do not trigger RESETSESSION from connect fault during connection phase

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

osd/OSDMap: fix upmap mis-killing for erasure-coded PGs

The up-set of erasure-coded PGs may include CRUSH_ITEM_NONE,
which as a result causes mis-killing of valid upmap items.

Fixes: https://tracker.ceph.com/issues/37493
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: ningtao <ningtao@sangfor.com.cn>

Merge pull request #25336 from dragonylffly/wip-silence-warning

crush/CrushWrapper: silence compiler warning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #25373 into master

* refs/pull/25373/head:
ceph_argparse: fix CephBool to CephChoices conversion

Reviewed-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: do not no_reply on ready_to_merge message

The C_ReplyMap will reply to this op.  Do not no_reply, or else an
ill-timed mon election will mean we do nothing for this request.

Fixes: http://tracker.ceph.com/issues/37512
Signed-off-by: Sage Weil <sage@redhat.com>

auth: Kerberos authentication

Signed-off-by: Daniel Oliveira <doliveira@suse.com> (github: oliveiradan)

Merge pull request #25361 from badone/wip-add-luminous-12.2.9-and-10

doc/releases: Add luminous releases 12.2.9 and 10

Reviewed-by: Noah Watkins <noahwatkins@gmail.com>

mon/OSDMonitor: include host, container info for 'osd find' command

Signed-off-by: Sage Weil <sage@redhat.com>

common/util.cc: add CONTAINER_NAME processing for metadata

Kubernetes clusters use POD_NAME; simple containers should be
distinguishable and don't have 'pods'.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

doc/rbd: Document new RBD feature flags and librbd/krbd version support

Sources:

layering:

* https://github.com/ceph/ceph/commit/f87c441f6891e5e72d5e712d9a19e45a1b9a97eb
* krbd: https://github.com/torvalds/linux/commit/770eba6e295fd36e43881176ee0644b9cc2803f1

striping:

* https://github.com/ceph/ceph/commit/2e403ef2c98bb64220060c06b435fb1ffe08294e
* krbd: https://github.com/torvalds/linux/commit/770eba6e295fd36e43881176ee0644b9cc2803f1

exclusive-lock:

* https://github.com/ceph/ceph/commit/fa96bb4db47c500947efcab51892f45a04b03d30
* krbd: https://github.com/torvalds/linux/commit/ed95b21a4b0a71ef89306cdeb427d53cc9cb343f

object-map:

* https://github.com/ceph/ceph/commit/7e751ce9125cbeb04b9551afff4b91c6255c03f9

fast-diff:

* https://github.com/ceph/ceph/commit/16ad44cebcc5d2464be38b95f16e0d63d2b29ed1

deep-flatten:

* https://github.com/ceph/ceph/commit/b556d3173ae4483e681bb21658a7d484a2627386

journaling:

* https://github.com/ceph/ceph/commit/750771c6ff07f6fcf0235172ec68ff4d907c621d
* not a default: https://github.com/ceph/ceph/blob/1a14963/src/test/cli/rbd/help.t#L229

data-pool:

* https://github.com/ceph/ceph/commit/c5f22900474e0eed88910341de57e17e1cc154f0
* description: https://tracker.ceph.com/issues/17409
* krbd: https://github.com/torvalds/linux/commit/7e97332ea9caad3b7c6d86bc3b982e17eda2f736

operations:

* https://github.com/ceph/ceph/commit/209f6fc5a44e2d068ce0dd8b3e9a25ee700ad0e0
* description: https://github.com/torvalds/linux/commit/e573427a440fd67d3f522357d7ac901d59281948
* krbd: https://github.com/torvalds/linux/commit/e573427a440fd67d3f522357d7ac901d59281948

migrating:

* https://github.com/ceph/ceph/commit/2f7c4ae3fc7ec7a2bfe99e116bf5e4f71f24c6a1

Signed-off-by: Valentin Lorentz <progval+git@progval.net>

librbd: reset snaps in rbd_snap_list()

in SnapIterator in rbd.pyx, rbd_snap_list_end() is called by
SnapIterator.__dealloc__(). and rbd_snap_list_end() frees
snaps->name if it is not nullptr.  but there is chance that
snaps->name is never initialized after snaps is allocated by
SnapIterator.__init__, in that case, we will free() a wild pointer.

Fixes: http://tracker.ceph.com/issues/37508
Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr: add "total" perf counter types

Signed-off-by: Mykola Golub <mgolub@suse.com>

mgr/hello: add serve() method

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Fix 'resource' typo

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Document custom RESTController endpoints

Extended HACKING.rst by the fact how to extend a RESTController with
a custom endpoint which has all permissions preserved.

Fixes: https://tracker.ceph.com/issues/37468
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #25305 from cbodley/wip-37440

qa/rgw: fix invalid syntax error in radosgw_admin_rest.py

Reviewed-by: Volker Theile <vtheile@suse.com>

os/bluestore: fix and unify log output on allocation failure

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

mgr/dashboard: 'Logs' links permission in Landing Page

- 'Logs' links in Landing Page cards' popovers
for cards 'Cluster Status' (when not in HEALTH_OK) and
'PG Status' appear only when the user has
the appropriate permissions.
- Some refactor.
- Added test: ensure that clickable text (popover) is shown
when there are health checks' messages.

Fixes: https://tracker.ceph.com/issues/37371
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #25366 from jtlayton/wip-orchestrator

orchestrator: add the ability to remove services

Merge pull request #25344 from sebastian-philipp/volumes-py3

mgr/volumes: Fix Python 3 import error

Reviewed-by: Ricardo Dias <rdias@suse.com>

ceph_argparse: fix CephBool to CephChoices conversion

Commit 525623b has introduced a regression that makes Ceph clients of
versions prior to Nautilus to receive invalid unicode characters in the
commands list JSON sent by the monitors.

The unicode characters are caused by memory corruption in a dangling
pointer stored by a std::string_view. The dangling pointer belonged to
an std::string object that was already destructed when the
std::string_view is used.
The fix in this PR makes the std::string object to live enough time for
the std::string_view to be used safely.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #24445 from jmolmo/wip_ansible_orchestrator

mgr/ansible: Ansible orchestrator module

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sébastien Han <seb@redhat.com>

common: bufferlist::buffers_t wraps bi::list now.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

include: buffer_raw.h uses C++11-compatible std::aligned_storage.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: factories of buffer::ptr_node produce std::unique_ptr now.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

client: create buffer::ptr_node directly.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

os/bluestore: KernelDevice creates buffer::ptr_node directly.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

tests: bench_bufferlist_alloc() gives a try to node_ptr.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: hypercombined bufferlist.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: proliferate ptr_node in the bufferlist implementation.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: switch bufferlist to use boost::intrusive for _buffers.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #25339 from jtlayton/wip-volumes

mgr/volumes: fix orchestrator remove operation

common: drop ::length() from other implementations of the OpQueue.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>