librbd: separate journal::TagData predecessor data into new struct

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: include tag tid in bootstrap debug log messages

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: demote/promote image on same cluster

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10733 from dillaman/wip-16608

rbdmap: specify bash shell interpreter

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10735 from dillaman/wip-17030

librbd: always respond to "release lock" request if lock owner

Reviewed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #10712 from dillaman/wip-16740

librbd: permit disabling journaling if in corrupt state

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10711 from dillaman/wip-16984

librbd: fix possible inconsistent state when disabling mirroring fails

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10708 from dillaman/wip-16967

rbd: bench io-size should not be larger than image size

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10714 from dillaman/wip-16529

qa/workunits: support filtering cls_rbd unit test cases

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9688 from shylesh/wip-16280

ceph-disk: change ownership of initfile to ceph:ceph

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #10376 from onyb/wip-rbd-pypi

pybind: Rework rbd/setup.py for PyPI

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: always respond to "release lock" request if lock owner

Fixes: http://tracker.ceph.com/issues/17030
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10709 from cgwalters/doc-use-https-for-repos

doc/install: Use https:// for download.ceph.com

Reviewed-by: Dan Mick <dmick@redhat.com>

doc: added rbdmap man page to RBD restructured index

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbdmap: specify bash shell interpreter

Fixes: http://tracker.ceph.com/issues/16608
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10596 from cbodley/wip-rgw-period-puller

rgw multisite: fixes for period puller

Merge pull request #10330 from rzarzynski/wip-rgw-objdeler-fixes

rgw: multiple fixes for Swift's object expiration
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10619 from chengweiv5/fix-doc

doc: fix rados/configuration/osd-config-ref.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10138 from Yan-waller/yj-wip-rgwrados-0705

rgw: fix the error return variable in log message and cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10729 from SUSE/wip-rnotes-script-regex

script: improve ceph-release-notes regex

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #10705 from chengweiv5/doc-image-format

doc: format 2 now is the default image format

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

script: improve ceph-release-notes regex

Tolerate multiple spaces after ':', remove duplicate "fs", add "log" and
"msgr".

Signed-off-by: Nathan Cutler <ncutler@suse.com>

librbd: fix possible memory leak when image open fails

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10118 from weiqiaomiao/wqm-wip-bucketcreate

rgw: for the create_bucket api, if the input creation_time is zero, we should set it to 'now"
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10431 from ifed01/wip-bluestore-fix-csum-disable2

os/bluestore: proper handling for csum enable/disable settings

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10719 from stiopaa1/mon_pgmonitor_moveC_StatsStr

mon/PGMonitor: move C_Stats struct to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10208 from BlaXpirit/cmake-py3

Include Python 3 bindings into the cmake build and make packages for them

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mon/PGMonitor: move C_Stats struct to cc file

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #10685 from liewegas/wip-bluefs-extent

os/bluestore/bluefs_types: fix extent operator<<

Merge pull request #10676 from Yan-waller/yj-wip-rgwdocmultisite

rgw/doc: fix indent

Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa/workunits: support filtering cls_rbd unit test cases

Fixes: http://tracker.ceph.com/issues/16529
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: permit disabling journaling if in corrupt state

Fixes: http://tracker.ceph.com/issues/16740
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: new journal policy to disable initializing the journal

This will be used in the case where the journal is being disabled.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10448 from pritha-srivastava/wip_s3_v2_localauth_rework

rgw: Rework of s3 v2 local authentication code.

verfied:  f23, subset of s3tests

Merge pull request #10449 from ifed01/wip-bluestore-bluestore-perfcnt

os/bluestore: add perf counters for compression effectiveness and space utilization measurements

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10193 from yehudasa/wip-16435

rgw: fix issue 16435
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

librbd: fix possible inconsistent state when disabling mirroring

Fixes: http://tracker.ceph.com/issues/16984
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10706 from xiexingguo/xxg-wip-fix-mondbstore

mon/MonDBStore: fix assert which never fires

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10704 from chhabaramesh/master

os/bluestore/BitAllocator: batch is_allocated bit checks

Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #10689 from liewegas/wip-bluestore-hex-extents

os/bluestore: print bluefs_extents in hex

Reviewed-by: Varada Kari <varada.kari@sandisk.com>

doc/install: Use https:// for download.ceph.com

While the Ceph RPMs etc. are GPG signed, there are various
attacks against signed RPMs without signed/trusted *metadata*
that are elaborated on in https://theupdateframework.github.io/

Since download.ceph.com handles it today, let's make it the default.
(Might want to run this by the server admins though).

Signed-off-by: Colin Walters <walters@verbum.org>

rbd: bench io-size should not be larger than image size

Fixes: http://tracker.ceph.com/issues/16967
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: Rework of s3 v2 local authentication code.

The v2 local auth code has been reworked based on the new new authentication infrastructure.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

doc: format 2 now is the default image format

Signed-off-by: Chengwei Yang <yangchengwei@qiyi.com>

cmake: Move Python detection back outside of pybind

Signed-off-by: Oleh Prypin <oleh@pryp.in>

batch is_allocated bit checks

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

doc: fix command format

Signed-off-by: Chengwei Yang <yangchengwei@qiyi.com>

Merge pull request #10621 from tchaikov/wip-doc

doc: silence sphinx warnings

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

doc: fix typo maximum -> minimal

Signed-off-by: Chengwei Yang <yangchengwei@qiyi.com>

doc: replace cluster name with cluster-name to avoid confusing

Signed-off-by: Chengwei Yang <yangchengwei@qiyi.com>

Merge pull request #10701 from dx9/wip-run-make-cmake-args

run-make-check.sh: pass args to do_cmake.sh

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/MonDBStore: fix assert which never fires

The condition is always true and thus the assert never fires,
which is obvious not our plan.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rgw/doc: fix typo and indent.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

run-make-check.sh: pass args to do_cmake.sh

Signed-off-by: John Coyle <dx9err@gmail.com>

Lowered third example to keep PG count lower.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #10638 from linuxbox2/rgw-nfs-31

rgw file: remove busy-wait in RGWLibFS::gc()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw file: remove busy-wait in RGWLibFS::gc()

This is a background thread.  However, CPU is wasted.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

os/bluestore: print bluefs_extents in hex

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10670 from ceph/wip-cmake

cmake: cleanup Findgperftools.cmake

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #10632 from ryneli/fix_ceph_buffer_list

librados: use bufferlist instead of buffer::list in public header

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #10633 from dx9/wip-sig

global/signal_handler: use sig_str instead of sys_siglist

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore/bluefs_types: fix extent operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10634 from dx9/wip-bit-allocator

bluestore/BitAllocator: Fix deadlock with musl libc

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10675 from chiehtu/fix-16996

doc: fix broken link in SHEC erasure code plugin

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: fix whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10215 from ifed01/wip-bluestore-blob-release-fix

os/bluestore: refactor dirty blob tracking along with some related fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10680 from ifed01/wip-bluestore-somefix

os/bluestore: fix improper local var variable in collection_list meth…

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: fix improper local var variable in collection_list method causing StoreTest.Synthetic to fail.

Signed-off-by: Igor Fedotov <ifed@mirantis.com>

Merge pull request #10493 from Yan-waller/yj-wip-librados-0728

rados/client: fix typo

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #10669 from cbodley/wip-rgw-ldap-auth

rgw: fix for s3tests failure when ldap auth is not applied

rpm: Add packages for Python 3 bindings

Signed-off-by: Oleh Prypin <oleh@pryp.in>

debian: Add packages for Python 3 bindings

Signed-off-by: Oleh Prypin <oleh@pryp.in>

pybind: Make ceph_volume_client's syntax compatible with Python 3
(just so it passes the bytecompilation step)

Signed-off-by: Oleh Prypin <oleh@pryp.in>

cmake: Also build Python bindings for Python 3

Signed-off-by: Oleh Prypin <oleh@pryp.in>

rgw: don't overwrite auth result if ldap is not applicable

this was causing the following s3tests failure:

======================================================================
FAIL: s3tests.functional.test_s3.test_list_buckets_invalid_auth
----------------------------------------------------------------------
AssertionError: u'AccessDenied' != 'InvalidAccessKeyId'

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10667 from dillaman/wip-16980

rbd-mirror: potential race condition during failure shutdown

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

ceph-disk: change ownership of init file to ceph:ceph

Fixes: http://tracker.ceph.com/issues/16280
Signed-off-by: Shylesh Kumar <shmohan@redhat.com>

os/bluestore: collect 'dirty' blobs on per-transaction basis to avoid onode::blob_map enumeration

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: migrate to refcounting for Blob class instances

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: remove internal structures from Bluestore.h

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/blueStore: fix discard_unallocated for compressed blob

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #10486 from badone/wip-double-free-global-static-test

tests: Add test for global static non-POD segfault

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix broken link in SHEC erasure code plugin

Fixes: http://tracker.ceph.com/issues/16996
Signed-off-by: Albert Tu <albert.t@inwinstack.com>

cmake: cleanup Findxio.cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: cleanup Findleveldb.cmake

* and reference leveldb library and include dir with fullpath

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10629 from ceph/dillaman-rbd-glance-docs

doc: update rbd glance configuration notes

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10627 from dillaman/wip-16970

rbd-mirror: replaying state should include flush action

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

cmake: cleanup Findgperftools.cmake

* remove Findtcmalloc.cmake, use Findgperftools.cmake instead.
* tcmalloc is packaged in gperftools-dev, so we can check them in a single
  place.
* fix the check of heap-profiler.h, we should enable the check as long
  as tcmalloc is enabled or profiler is enabled.
* only check gperftools headers in "include/gperftools", and do not
  "include/google" anymore. as we only support the distros which
  shipping recent gperftools-dev package with "include/gperftools".
  and "google/*.h" are deprecated.
* set ALLOC_LIBS with GPERFTOOLS_TCMALLOC_LIBRARY, so we can link
  against tcmalloc with full path.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fix rbd map format in rbd manpage

Signed-off-by: Chengwei Yang <yangchengwei@qiyi.com>

librados: use bufferlist instead of buffer::list

Signed-off-by: Ryne Li <lizhenqiangsnake@gmail.com>

Merge pull request #10410 from xiexingguo/xxg-wip-bluestore-2016-07-23

os/bluestore: narrow lock scope for cache trim()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10362 from xiexingguo/xxg-wip-bluefs-2016-07-20

os/bluestore: fix potential access violation

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

rbd-mirror: potential race condition during failure shutdown

Fixes: http://tracker.ceph.com/issues/16980
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10425 from theanalyst/rgw/sync-paren-warn

rgw: kill a compile warning for rgw_sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10666 from ceph/wip-cmake

cmake: install platlib into a subdir of build-base dir

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: kill a compile warning for rgw_sync

killing the compile warning for

```
/ceph/src/rgw/rgw_sync.cc:1462:12:
warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wparentheses]
         if (can_adjust_marker) yield {
            ^
```

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10640 from vmcloud/libcephfs_jni_overflow

libcephfs: Fix the incorrect integer conversion in libcephfs_jni.cc

Reviewed-by: Noah Watkins <noahwatkins@gmail.com>

cmake: install platlib into a subdir of build-base dir

distutils' install command installs all stuff under $build-base, which
includes the temporary files. and rpm packaging does not like::
  Installed (but unpackaged) file(s)
and to support both py2 and py3, we should encode the python version
into the path, otherwise the built files will be overwritten.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10602 from smanjara/wip-16935

rgw: raise log levels for common radosgw-admin errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10110 from Yan-waller/yj-wip-rgwrados-0702

rgw: remove possible duplicate setting

Reviewed-by: Casey Bodley <cbodley@redhat.com>