osd: Test osd_find_best_info_ignore_history_les config in another assert

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 02a9a41f151a3d968bf8066749658659dc6e3ac4)

Merge pull request #6845 from dachary/wip-14019-infernalis

infernalis: libunwind package missing on CentOS 7

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

build/ops: enable CR in CentOS 7

To get libunwind from the CR repositories until CentOS 7.2.1511 is released.

http://tracker.ceph.com/issues/13997 Fixes: #13997

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 247ee6084b58861da601d349bdba739b252d96de)

Merge pull request #6634 from Abhishekvrshny/wip-13761-infernalis

unknown argument --quiet in udevadm settle

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6650 from Abhishekvrshny/wip-13830-infernalis

init script reload doesn't work on EL7

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #6477 from SUSE/wip-13705-infernalis

rbd : enable feature objectmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6474 from SUSE/wip-13619-infernalis

rbd clone issue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6633 from Abhishekvrshny/wip-13759-infernalis

rbd: pure virtual method called

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6632 from Abhishekvrshny/wip-13756-infernalis

QEMU hangs after creating snapshot and stopping VM

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6630 from Abhishekvrshny/wip-13754-infernalis

Avoid re-writing old-format image header on resize
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6396 from SUSE/wip-13342-infernalis

ceph upstart script rbdmap.conf incorrectly processes parameters

Reviewed-by: Sage Weil <sage@redhat.com>

init-rbdmap: fix CMDPARAMS

Fixes: #13214
Reported-by: Wyllys Ingersoll <wyllys.ingersoll@keepertech.com>
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 02113ac449cd7631f1c9a3840c94bbf253c052bd)

rgw: fix reload on non Debian systems.

When using reload in non-debian systems, /bin/sh's kill is used to send the HUP signal to the radosgw process.
This kill version doesn't understand -SIGHUP as a valid signal, using -HUP does work.

Fix: #13709
Backport: hammer
Signed-off-by: Hervé Rousseau <hroussea@cern.ch>
(cherry picked from commit 1b000abac3a02d1e788bf25eead4b6873133f5d2)

krbd: remove deprecated --quiet param from udevadm

This parameter has been removed since systemd 213, so this
effects Fedora 21+, Debian Jessie, and potentially future
releases of RHEL 7.

Fixes: #13560
Backport: hammer, infernalis
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4300f2a9fe29627eea580564ff2d576de3647467)

run_cmd: close parent process console file descriptors

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f46f7dc94139c0bafe10361622416d7dc343d31f)

WorkQueue: new PointerWQ base class for ContextWQ

The existing work queues do not properly function if added to a running
thread pool.  librbd uses a singleton thread pool which requires
dynamically adding/removing work queues as images are opened and closed.

Fixes: #13636
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3e78b18b09d75626ca2599bac3b9f9c9889507a5)

librbd: fixed deadlock while attempting to flush AIO requests

In-flight AIO requests might force a flush if a snapshot was created
out-of-band.  The flush completion was previously invoked asynchronously,
potentially via the same thread worker handling the AIO request. This
resulted in the flush operation deadlocking since it can't complete.

Fixes: #13726
Backport: infernalis, hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit bfeb90e5fe24347648c72345881fd3d932243c98)

tests: new test case to catch deadlock on RBD image refresh

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a9729d9553e7fb925509cad8d388cf52a9fede9c)

librbd: resize should only update image size within header

Previously, the whole RBD image format 1 header struct was
re-written to disk on a resize operation.

Fixes: #13674
Backport: infernalis, hammer, firefly
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d5be20b6d4646284571568ab28cbf45b0729390b)

librbd : fix enable objectmap feature issue

Fixes: #13558
Signed-off-by: xinxin shu <xinxin.shu@intel.com>
(cherry picked from commit b0536ebab4e1f34e1ed87fe5efbb00d0f7b48abb)

rbd: fix clone issue when we specify image feature

Fixes: #13553
Signed-off-by: xinxin shu <xinxin.shu@intel.com>
(cherry picked from commit da48dbb8f8c9417343d2ca7819c58b7c46ef7ad0)

9.2.0

Merge pull request #6444 from liewegas/wip-pg-key

osd/PG: tolerate missing epoch key

Reviewed-by: Samuel Just <sjust@redhat.com>

osd/PG: tolerate missing epoch key

An orphan PG may have an info attr but no epoch key.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6344 from dillaman/wip-13559-infernalis

librbd: potential assertion failure during cache read

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6363 from ceph/wip-init-rbdmap

Drop redhat-lsb-core dependency

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6366 from liewegas/wip-osd-fixboot-infernalis

osd: fix OSDService vs Objecter init order

Reviewed-by: Samuel Just <sjust@redhat.com>

osd: fix OSDService vs Objecter init order

This reverts c7d96a5ed1d2cb844622af29b13705b8f7be6be7, but still keeps
the Objecter init *after* we have authenticated.  This way we don't
crash when we get mon messages like MOSDPGCreate, and we also don't
request maps we aren't prepared to handle.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph.spec.in: We no longer need redhat-lsb-core

Drop the redhat-lsb-core dependency as it is no longer necessary on
fedora/rhel.

The other two init scripts do not use redhat-lsb-core either. The
init-ceph.in conditionally requires /lib/lsb/init-functions and does not
use any of the functions defined in that file (at least not directly).
The init-radosgw file includes /etc/rc.d/init.d/functions on non-debian
platforms instead of /lib/lsb/init-functions file so it does not require
redhat-lsb-core either.

Signed-off-by: Boris Ranto <branto@redhat.com>

init-rbdmap: Rewrite to use logger + clean-up

This patch rewrites the init-rbdmap init script so that it uses logger
instead of the log_* functions. The patch also fixes various smaller
bugs like:
* MAP_RV was undefined if mapping already existed
* UMNT_RV and UMAP_RV were almost always empty (if they succeeded) ->
  removed them
* use of continue instead RET_OP in various places (RET_OP was not being
  checked after the switch to logger messages)
* removed use of DESC (used only twice and only one occurrence actually
  made sense)

Signed-off-by: Boris Ranto <branto@redhat.com>

librbd: potential assertion failure during cache read

It's possible for a cache read from a clone to trigger a writeback if a
previous read op determined the object doesn't exist in the clone,
followed by a cached write to the non-existent clone object, followed
by another read request to the same object.  This causes the cache to
flush the pending writeback ops while not holding the owner lock.

Fixes: #13559
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: reproduce crash during read-induced CoW

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6356 from ceph/wip-fix-release-notes

doc/release-notes.rst: recovery isn't in the unified queue yet

doc/release-notes.rst: recovery isn't in the unified queue yet

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #6313 from ceph/wip-osd-compat

osd: allow firefly -> infernalis+ upgrades

Reviewed-by: David Zafman <dzafman@redhat.com>

osd: init objecter after we authenticate

Move objecter init until after we authenticate.  We ignore
osdmap messages while initializing anyway so it's pointless to
ask for them.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6248 from rzarzynski/wip-rgw-swift-acctmd-doc-infernalis

rgw, doc: remove remark for lack of custom account metadata of Swift API

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6212 from rzarzynski/wip-rgw-objexp-doc-infernalis

rgw, doc: object expiration of Swift API is supported now.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

osd/PG: make upgrade() use sequencer

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "os/FileStore: require upgrade to hammer before moving beyond"

This reverts commit 10cb507d9d52464b2fd28b3cc84a92328944e462.

Revert "osd: require an upgrade to hammer first"

This reverts commit c0f83df412c962a6b3a2f3cdd1d7c8c795ef1146.

Revert "ceph-objectstore-tool: require hammer upgrade"

This reverts commit b474991df74ab4d31b011bb6c5973fcd3c2e0179.

Revert "osd: drop support for pre-hammer pg metadata"

This reverts commit cd4e676e6d45c8166290ef834d73c2a0bda98fa2.

Revert "ceph-objectstore-tool: drop support for pre-pgmeta PGs"

This reverts commit b297e6d2b206c0da733ae9d4ac5fb321f113ae5d.

Revert "os: drop deprecated collection_* attr methods"

This reverts part of commit
893e00bc0fd793090a35d275919ca0348e9f05b0.

[we leave KeyValueStore untouched]

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "os/FileStore: fix version check"

This reverts commit 14d3a2b82bea668ee2e9898d0b612812d8f7e8cc.

Merge pull request #6249 from dillaman/wip-13274-infernalis

LTTng-UST tracing should be dynamically enabled

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6284 from dillaman/wip-13483-infernalis

qa: remove legacy OS support from rbd/qemu-iotests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: remove legacy OS support from rbd/qemu-iotests

RHEL7 derivatives were failing test 002 since they were using
legacy test cases for now unsupported OSes.

Fixes: #13483
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6273 from jdurgin/wip-13496

librbd: fix rebuild_object_map() when no object map exists

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: fix rebuild_object_map() when no object map exists

Enabling the object map feature and then attempting to rebuild it
results in an assert failure, since the number of objects was
accidentally passed to ObjectMap::aio_resize() instead of the size of
the image.

Fixes: #13496
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

ceph_context: remove unsafe cast for singletons

It was previously assumed that a CephContext singleton would
inherit from CephContext::AssociatedSingletonObject, but it was
not enforced.  This could result in unknown behavior when the
singleton is destroyed due to the implied virtual destructor.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6267 from dillaman/wip-13305-infernalis

ceph.spec.in: only run systemd-tmpfiles on ceph run directory

Reviewed-by: Sage Weil <sage@redhat.com>

ceph.spec.in: only run systemd-tmpfiles on ceph run directory

There is a kmod bug that affects RHEL7 derivatives which will
result in device permissions being reset back to defaults.

Fixes: #13305
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6265 from dillaman/wip-cmake-rbd-replay-infernalis

CMake: fix rbd_replay error

CMake: fix rbd_replay error

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: conditionally initialize the tracepoint provider

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados: conditionally initialize the tracepoint provider

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: conditionally initialize the tracepoint provider

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common: new tracing config options

Config options to enable LTTng-UST tracepoint providers for
the OSD, OSD objectstore, librados, and librbd.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tracing: merge tracepoint provider makefiles into main makefile

This results in the tracepoint provider shared libraries being
placed in the library path for unittests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tracing: dynamic tracepoint provider helper

The TracepointProvider class is a configuration observer.  When
tracing is enabled, it will dynamically load the associated
tracepoint provider.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

packaging: add new tracepoint probe shared libraries

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

ceph.spec.in: add new tracepoint probe shared libraries

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

lttng: move tracepoint probes to dynamic libraries

LTTng-UST initializes itself at program load, which means it is
currently always enabled.  This can lead to issues with SElinux
and AppArmor which might restrict access to the necessary device
files.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6261 from tchaikov/wip-12748-infernalis

osd: fix the snapshot reads of evicted tiering pool

Reviewed-by: Sage Weil <sage@redhat.com>

osd: fix the snapshot reads of evicted tiering pool

reset ssc->exsits in finish_ctx() if the ctx->cache_evict is true, and
the head is removed.

Fixes: #12748
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6215 from dillaman/wip-13429-infernalis

restore pre-infernalis API compatibility

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6221 from athanatos/wip-13434

Wip 13434

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6217 from dillaman/wip-13380-infernalis

cls: new force-promotion flag for class methods

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6230 from liewegas/wip-crushtool-err

mon/OSDMonitor: put crushtool error in log

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6235 from dzafman/wip-13462

osd: Correct the object_info_t::decode() version

Reviewed-by: Samuel Just <sjust@redhat.com>

doc/infernalis: hate hate

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: i hate rst

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: final infernalis notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge branch 'infernalis' of jenkins.front.sepia.ceph.com:ceph/ceph into infernalis

doc/release-notes: fix some attributions

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

doc/release-notes: infernalis notable changes

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "common, global: use lttng ust functions for handling fork-like calls"

This reverts commit 811898912f3e43d7312efe5d06c496e1e7e22fa4.

Merge pull request #6237 from ceph/wip-13458

rgw: location constraints should return api name

Reviewed-by: Sage Weil <sage@redhat.com>

rgw, doc: remove remark for lack of custom account metadata of Swift.

Backport: infernalis
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

9.1.0

Merge pull request #6245 from liewegas/wip-setuptools

debian/control: python-setuptools is a build dependency

Reviewed-by: Alfredo Deza <adeza@redhat.com>

debian/control: python-setuptools is a build dependency

cd ./ceph-detect-init ; python setup.py build
Traceback (most recent call last):
  File "setup.py", line 23, in <module>
    from setuptools import setup
ImportError: No module named setuptools

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: 9.1.0

Signed-off-by: Sage Weil <sage@redhat.com>

ReplicatedPG::maybe_handle_cache_detail: always populate missing_oid

Signed-off-by: Samuel Just <sjust@redhat.com>

ReplicatedPG::_rollback_to: handle block on full correctly

If we fail to promote the clone due to a full cache tier, we need
to block writes on the head just as if we were waiting on the
promote.  Generally therefore, we block writes on head if we block
an op on a full cache tier.  Add a new blocked set to support this.

Fixes: 13434
Signed-off-by: Samuel Just <sjust@redhat.com>

osd: Correct the object_info_t::decode() version

Caused by: 13b9dc70

Fixes: #13462
Backport: Hammer, Firefly

Signed-off-by: David Zafman <dzafman@redhat.com>

rgw: location constraints should return api name

Fixes: #13458
The S3 location constraints request should return the api name,
not the region name.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

mon/OSDMonitor: put crushtool error in log

Putting it in *ss is useless for mkpool callers who don't return
an error string.  Log it to the mon log, too.

Signed-off-by: Sage Weil <sage@redhat.com>

ReplicatedPG: allow maybe_handle_cache to return status detailing what happened

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #6220 from dreamhost/wip-pybind-rados-omap-binary

pybind/rados: Fix binary omap values.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

pybind/rados, get_omap_vals: Fix pydoc type.

pydoc type for filter_prefix did not match code.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

test: pybind/test_rados: add binary data.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

pybind/rados: Fix binary omap values.

The prior code caused binary omap values to be discarded. This fixes
them to use the same model as the xattr iterator, and correctly return
binary data as python strings, eg:
'object_prefix': '\x15\x00\x00\x00rbd_data.449d2ae8944a'

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #6193 from athanatos/wip-13240

ReplicatedPG::maybe_handle_cache: do not promote before checking full

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6190 from SUSE/wip-drop-my-conf-opt

ceph.spec.in: drop MY_CONF_OPTS

Reviewed-by: Ken Dreyer, <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

rgw: fix wrong etag calculation during POST on S3 bucket.

Fixes: #11241
Backport: infernalis, hammer, firefly
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
(cherry picked from commit 742906ab5ee644d5932570bf95342697b673a498)

Merge pull request #6183 from ceph/wip-fix-selinux-man-page

Fix selinux man page

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6191 from SUSE/wip-concentrate-buildrequires

ceph.spec.in: move BuildRequires out of subpackages

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>