Merge pull request #14295 from theanalyst/release/10.2.7/notes

doc: release notes for v10.2.7 Jewel

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14441 from theanalyst/doc/changelog/v10.2.7

doc: add changelog for v10.2.7

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #14426 from tchaikov/wip-13562-kefu

misc: SCA fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13940 from Adirl/out

msg/async/rdma: Update fix broken compilation

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #14409 from Adirl/conn_info

msg/async/rdma: RDMA-CM, Pass specific ConnMgr info in constructor

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #14410 from Adirl/get_device

msg/async/rdma: RDMA-CM, get_device() by ibv_context

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13073 from chardan/jfw-wip-15475-admin_heap_command

osd: add "heap *" admin command

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14235 from nvedant07/bug_fix_3

rados: more info added to pool deletion error

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14022 from batrick/admin-socket-close-error

common: do not print error when asok is closed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14292 from joscollin/wip-radosacl-warning

tools: warning, ‘%.16x’ directive output truncated writing 16 bytes into a region of size 9.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14301 from Andrey1994/master

build: Add Virtuozzo Linux support

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14336 from Adirl/fix_sockaddr

src/msg/async: Update fix broken compilation for Posix

Reviewed-by: Haomai Wang <haomai@xsky.com>

doc: add changelog for v10.2.7

Signed-off-by: Abhishek Lekshmanan <alekshmanan@suse.com>

Merge pull request #14039 from dachary/wip-19307-set-deviceclass

mon/OSDMonitor: osd crush set-device-class

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14208 from dachary/wip-crush-uniform-weights

crush: bucket: crush_add_uniform_bucket_item should check for uniformity

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14209 from dachary/wip-crush-create-default-tunables

crush: builder: creating crush map with optimal configurations

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14401 from dillaman/wip-rbd-import-fixes

rbd: correct issues with image importing

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14437 from cernceph/dvanders_mgrrotate

ceph-mgr: rotate logs on sighup

Reviewed-by: Sage Weil <sage@redhat.com>

doc: release notes for v10.2.7 Jewel

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #14413 from hansbogert/patch-3

doc: rgw: remove mention of megabytes for quotas

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

logrotate.conf: also sighup the ceph-mgr

Tell ceph-mgr to reopen its logs postrotate.

Fixes: http://tracker.ceph.com/issues/19568
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

ceph_fuse.cc: fix 64/32bit detection

Fix for:

[src/ceph_fuse.cc:111]: (style) Condition 'sizeof(long)==4' is always false

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

lua_bufferlist.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

BitAllocator_test.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_librbd.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rest_s3.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

PrimaryLogPG.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/PG.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

BitAllocator.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #14430 from jcsp/wip-19566-monassert

mon: avoid segfault in wait_auth_rotating

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-mgr: reopen logs on SIGHUP

Wire up the sighup_handler to reopen logs, like the other daemons.

Fixes: http://tracker.ceph.com/issues/19568
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

Merge pull request #14428 from tchaikov/wip-ceph_objectstore_tool-kill-all

tests: ceph_objectstore_tool.py: kill all daemons

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #10636 from fullerdj/wip-djf-15069

cephfs: Permit recovering metadata into a new RADOS pool

Reviewed-by: John Spray <john.spray@redhat.com>

mon: avoid segfault in wait_auth_rotating

MonClient users should not be calling into MonClient
after calling shutdown().  However, MonClient should
assert out rather than proceeding to try and
follow a maybe-null pointer.

Related to http://tracker.ceph.com/issues/19566

Signed-off-by: John Spray <john.spray@redhat.com>

tests: ceph_objectstore_tool.py: kill all daemons

init-ceph kills all daemons that can be identified by get_name_list() by
default, so no need to list them explicitly. also, without this change,
mgr is not stopped when the test finishes.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/OSD.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rest_s3.cc: reorder checks in while loop

Fix for:

[src/rgw/rgw_rest_s3.cc:1224]: (style) Array index 'chunk_offset'
 is used before limits check.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

AsyncMessenger.cc: fix FORWARD_NULL

Use assert to ensure Message *m is not NULL.

Fix for:

CID 1399578 (#4 of 4): Dereference after null check (FORWARD_NULL)
 var_deref_model: Passing null pointer m to submit_message,
 which dereferences it.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_file.h: fix potential data race condition

Fix for:

CID 1399581 (#1 of 1): Data race condition (MISSING_LOCK)
 missing_lock: Accessing this->state.atime without holding lock
 rgw::RGWFileHandle.mtx. Elsewhere, "_ZN3rgw13RGWFileHandle5StateE.atime"
 is accessed with rgw::RGWFileHandle.mtx held 2 out of 3 times
 (2 of these accesses strongly imply that it is necessary).

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

cls_rgw_client.h: Close resource leak

Fix for:

CID 1395687 (#1 of 1): Resource leak (RESOURCE_LEAK)
CID 1395679 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable c going out of scope leaks the storage it
 points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mount.ceph.c: fix resource leak

Addressed issue:

CID 1219611 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable saw_name going out of scope leaks
 the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_ipaddr.cc: fix UNINIT issues around sockaddr_in struct

Fix for:

CID 1396088 (#1 of 1): Uninitialized scalar variable (UNINIT)
CID 1396090 (#1 of 1): Uninitialized scalar variable (UNINIT)
CID 1396091 (#1 of 1): Uninitialized scalar variable (UNINIT)
CID 1396092 (#1 of 1): Uninitialized scalar variable (UNINIT)
CID 1396096 (#1 of 1): Uninitialized scalar variable (UNINIT)
CID 1396097 (#1 of 1): Uninitialized scalar variable (UNINIT)
CID 1396099 (#1 of 1): Uninitialized scalar variable (UNINIT)
CID 1396101 (#1 of 1): Uninitialized scalar variable (UNINIT)
CID 1396103 (#1 of 1): Uninitialized scalar variable (UNINIT)
CID 1396104 (#1 of 1): Uninitialized scalar variable (UNINIT)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #14280 from tone-zhang/bug_19442

rgw: fix response header of Swift API

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #14378 from kestrels/wip-19454

arch: fix build on PowerPC with FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #14360 from joscollin/wip-Update-organizationmap

.organizationmap: Updated authors

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13803 from trociny/wip-18785

rbd-mirror: separate ImageReplayer handling from Replayer

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14406 from wjwithagen/wip-wjw-freebsd.readme

README.FreeBSD: Update the status

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: rgw: remove mention of megabytes for quotas

Remove mention of confusing reference to megabytes in opening paragraph of "quota management".
To remove further confusion, mention that setting a size for '--max-size' is in bytes by default.

Signed-off-by: Hans van den Bogert <hansbogert@gmail.com>

README.FreeBSD: Update the status

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #14397 from liewegas/wip-19541

osd/PG: only correct filestore collection bits on load

Reviewed-by: Kefu Chai <kchai@redhat.com>

msg/async/rdma: get_device() by ibv_context

This function will be used by RDMAConnCM

Issue: 995322
Change-Id: I52b684e8bbac14c1070801b2ba470c7c0dfe40b9
Signed-off-by: Amir Vadai <amir@vadai.me>

msg/async/rdma: Pass specific ConnMgr info in constructor

Pass specific information for RDMAConnTCP (and later for RDMAConnCM) in
the constructor.
This makes set_accept_fd() redundant.

Issue: 995322
Change-Id: I10b4a626f180858465a378e886a238fe187180d5
Signed-off-by: Amir Vadai <amir@vadai.me>

Merge pull request #14382 from zealoussnow/wip-0407

messages: fix return type name of MOSDMap

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14405 from wjwithagen/wip-wjw-gitignore

gitignore: Ignore rejects by patch

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14261 from liewegas/wip-15386

ceph_test_librados_api_misc: fix stupid LibRadosMiscConnectFailure.ConnectFailure test

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14130 from liewegas/wip-json-slash

common/escape: do not escape / in json

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #14198 from liewegas/wip-fs

qa/suites: drop 'fs' facet, and add 'objectstore' facet where missing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14255 from liewegas/wip-19430

osd/PrimaryLogPG: do not expect FULL_TRY ops to get resent

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/workunits/cephtool/test.sh: handle non-escaped-/ json

Signed-off-by: Sage Weil <sage@redhat.com>

gitignore: Ignore rejects by patch

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #14386 from tchaikov/wip-19515

mon/OSDMonitor: update creating epoch if target osd changed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14227 from ivancich/wip-perf-counter-race

common/perf_counters: fix race condition with atomic variables

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14365 from tchaikov/wip-another-mgr-command

mgr: move "osd pool stats" to mgr

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14370 from chardan/jfw-wip-simple_spinlock_atomic

common: Implements simple_spin_t in terms of std::atomic_flag.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14379 from tchaikov/wip-cmake-ctags

cmake: support optional argument for overriding default ctag excludes

Reviewed-by: Dan Mick <dmick@redhat.com>

tests: ceph_objectstore_tool.py: run vstart with MGR=1

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits: partial revert of commit 5b486d9

Testing for disk usage and diff extents for a sparsely written
imported RBD image cannot generically be handled across different
OSD object stores and RBD image features.

The only alternatives would include grepping the rbd CLI
debug log for specific invocations of aio_write or mocking
the rbd CLI for unit testing.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: import-diff was attempting to write empty chunks

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: corrected typo in rbd export v2 section header

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14374 from liewegas/wip-mgr-init

mgr: fix bugs in init, beacons

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14306 from liewegas/wip-bluestore-throttle-model

os/bluestore: unify throttling model

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

os/bluestore: make deferred_aggressive a counter

Multiple threads may run _osr_drain_preceding; use a count to
prevent the first finisher from prematurely disabling
aggressive mode.

Fixes: http://tracker.ceph.com/issues/19542
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14387 from wjwithagen/wip-wjw-test-osd-dup

test/osd/CMakeLists.txt: osd-dup.sh require BlueStore/AIO

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14348 from rzarzynski/wip-bs-bitmap-no-virts-in-hotspot

os/bluestore: avoid the VTABLE-related burden in BitMapAllocator's hotspot

Reviewed-by: Sage Weil <sage@redhat.com>

mon/MgrMonitor: send mgr beacons to leader

The leader has the last_beacon map that matters; make sure all
beacons go there.

At the same time drop the somewhat redundant checks in
preprocess_beaon and just send *everything* to prepare_beacon,
which is smart enough to do no update if there is none.

Fixes: http://tracker.ceph.com/issues/19502
Signed-off-by: Sage Weil <sage@redhat.com>

rbd-mirror A/A: separate ImageReplayer handling from Replayer

Fixes: http://tracker.ceph.com/issues/18785
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror A/A: InstanceReplayer class to acquire and release images

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

osd/PG: only correct filestore collection bits on load

PG::update_store_with_options() is called on load *and* when the pool
properties change, but if we only need to fix bits right at load time.
More importantly, but doing a second check on pool change, we may race
with a previously queued collection create that is not yet readable via
filestore, try to (synchronously) set the property again, and deadlock.

Fixes: http://tracker.ceph.com/issues/19541
Signed-off-by: Sage Weil <sage@redhat.com>

rbd-mirror: add set_remote_images method to ImageReplayer

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14366 from wjin/clean_mgr

mgr: remove unused function declarations

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/osd/CMakeLists.txt: osd-dup.sh require BlueStore/AIO

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

common: Implements simple_spin_t in terms of std::atomic_flag

Removes the requirement of machine-specific code in favor of the standard
library. See also PR #14337.

Signed-off-by: Jesse Williamson <jwilliamson@suse.de>

mon/OSDMonitor: update creating epoch if target osd changed

there is chance that the acting_primary of a PG being created changes
across osdmap updates, in that case we should update the new creating_pgs
with the mapping's epoch, not the stale one when the PG was originally
created. otherwise the new target OSD won't get the pg-create message.

Fixes: http://tracker.ceph.com/issues/19515
Signed-off-by: Kefu Chai <kchai@redhat.com>

messages: fix return type name of MOSDMap

Signed-off-by: Leo Zhang <nguzcf@gmail.com>

Merge pull request #14029 from dachary/wip-osd-crushmap

mon: osd crush set crushmap need sanity check

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: support optional argument for overriding default ctag excludes

so instead of excluding all submodules, one can just exclude boost and
rocksdb using:

  cmake -DCTAG_EXCLUDES="boost;rocksdb" ..

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14334 from tchaikov/wip-cmake-ctags

cmake: Add simple recursive ctags target for Ceph source only

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #14320 from Adirl/poll_blocking

msg/async/rdma: Make poll_blocking() poll for async events in additio…

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #14340 from oritwas/wip-rgw-fix-zone-create

radosgw-admin: use zone id when creating a zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14262 from guihecheng/rgw_file-fix-rmdir

rgw_file: fix missing unlock in unlink

Merge pull request #14216 from dongbula/add-multi-zonegroup-for-test

rgw: add --num-zonegroups option for multi test

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14354 from mogeb/cmake-lttng

cmake: build tracepoint libraries for vstart target

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/Mgr: unlock lock while blocking for metadata load

Fixes: http://tracker.ceph.com/issues/19516
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14115 from liewegas/wip-bluestore-metadata

os/bluestore: implement collect_metadata

Merge pull request #14304 from rchagam/18576_bug_fix

tracing: fix segv

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14328 from liewegas/wip-mgrmap-features

mgr: fix several init/re-init bugs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14333 from liewegas/wip-bluestore-misc

os/bluestore: misc fixes

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

msg/async/rdma: Make poll_blocking() poll for async events in addition to tx/rx

Issue: 995322
Change-Id: I62e75f4c4e574ba97e24b20a4d7b94c56c605bb5
Signed-off-by: Amir Vadai <amir@vadai.me>

Merge pull request #14210 from liewegas/wip-osd-dup

os: allow offline conversion of filestore -> bluestore (or anything else)

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #12175 from dongbula/fix-slavezonegroup-cannot-enable-bucket-versioning

rgw: fix slave zonegroup cannot enable the bucket versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>