mon/PGMonitor: fix potential underflow

h.h.size() could be 0(theoretically), so "h.h.size() - 1" could underflow.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/PGMonitor: drop redundant loading of dumpcontents

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/PGMonitor: skip scrub checking if we can

The PG-scrub checking may become expensive once the cluster
is big. Since the scrub-warn related options are defaulted to be off,
we should skip this checking when it is possible, which
is good for performance.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMonitor: drop redundant check of exists()

Which is aready included by the is_up() method.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMonitor: avoid duplicated assignment of oldest_map/newest_map

As build_incremental() has done this for us.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: avoid duplicated calls when determining osdmap boundary

We have caculdated the oldest/newest map epochs already,
so we can use it directly here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: get rid of snprintf when formatting output string

We can use the standard precision() and fixed method to achieve the same
effect.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: fix potential division-by-zero error

By checking pg_num against 0, we could avoid the risk of
division-by-zero error.

Also, we limit the loop to 50 or pg_num, whichever is smaller,
when trying to generate some pg_temps randomly.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #10431 from ifed01/wip-bluestore-fix-csum-disable2

os/bluestore: proper handling for csum enable/disable settings

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10719 from stiopaa1/mon_pgmonitor_moveC_StatsStr

mon/PGMonitor: move C_Stats struct to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10208 from BlaXpirit/cmake-py3

Include Python 3 bindings into the cmake build and make packages for them

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mon/PGMonitor: move C_Stats struct to cc file

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #10685 from liewegas/wip-bluefs-extent

os/bluestore/bluefs_types: fix extent operator<<

Merge pull request #10676 from Yan-waller/yj-wip-rgwdocmultisite

rgw/doc: fix indent

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10448 from pritha-srivastava/wip_s3_v2_localauth_rework

rgw: Rework of s3 v2 local authentication code.

verfied:  f23, subset of s3tests

Merge pull request #10449 from ifed01/wip-bluestore-bluestore-perfcnt

os/bluestore: add perf counters for compression effectiveness and space utilization measurements

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10193 from yehudasa/wip-16435

rgw: fix issue 16435
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10706 from xiexingguo/xxg-wip-fix-mondbstore

mon/MonDBStore: fix assert which never fires

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10704 from chhabaramesh/master

os/bluestore/BitAllocator: batch is_allocated bit checks

Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #10689 from liewegas/wip-bluestore-hex-extents

os/bluestore: print bluefs_extents in hex

Reviewed-by: Varada Kari <varada.kari@sandisk.com>

rgw: Rework of s3 v2 local authentication code.

The v2 local auth code has been reworked based on the new new authentication infrastructure.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

cmake: Move Python detection back outside of pybind

Signed-off-by: Oleh Prypin <oleh@pryp.in>

batch is_allocated bit checks

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #10621 from tchaikov/wip-doc

doc: silence sphinx warnings

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #10701 from dx9/wip-run-make-cmake-args

run-make-check.sh: pass args to do_cmake.sh

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/MonDBStore: fix assert which never fires

The condition is always true and thus the assert never fires,
which is obvious not our plan.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rgw/doc: fix typo and indent.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

run-make-check.sh: pass args to do_cmake.sh

Signed-off-by: John Coyle <dx9err@gmail.com>

Lowered third example to keep PG count lower.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #10638 from linuxbox2/rgw-nfs-31

rgw file: remove busy-wait in RGWLibFS::gc()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw file: remove busy-wait in RGWLibFS::gc()

This is a background thread.  However, CPU is wasted.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

os/bluestore: print bluefs_extents in hex

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10670 from ceph/wip-cmake

cmake: cleanup Findgperftools.cmake

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #10632 from ryneli/fix_ceph_buffer_list

librados: use bufferlist instead of buffer::list in public header

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #10633 from dx9/wip-sig

global/signal_handler: use sig_str instead of sys_siglist

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore/bluefs_types: fix extent operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10634 from dx9/wip-bit-allocator

bluestore/BitAllocator: Fix deadlock with musl libc

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10675 from chiehtu/fix-16996

doc: fix broken link in SHEC erasure code plugin

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: fix whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10215 from ifed01/wip-bluestore-blob-release-fix

os/bluestore: refactor dirty blob tracking along with some related fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10680 from ifed01/wip-bluestore-somefix

os/bluestore: fix improper local var variable in collection_list meth…

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: fix improper local var variable in collection_list method causing StoreTest.Synthetic to fail.

Signed-off-by: Igor Fedotov <ifed@mirantis.com>

Merge pull request #10493 from Yan-waller/yj-wip-librados-0728

rados/client: fix typo

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #10669 from cbodley/wip-rgw-ldap-auth

rgw: fix for s3tests failure when ldap auth is not applied

rpm: Add packages for Python 3 bindings

Signed-off-by: Oleh Prypin <oleh@pryp.in>

debian: Add packages for Python 3 bindings

Signed-off-by: Oleh Prypin <oleh@pryp.in>

pybind: Make ceph_volume_client's syntax compatible with Python 3
(just so it passes the bytecompilation step)

Signed-off-by: Oleh Prypin <oleh@pryp.in>

cmake: Also build Python bindings for Python 3

Signed-off-by: Oleh Prypin <oleh@pryp.in>

rgw: don't overwrite auth result if ldap is not applicable

this was causing the following s3tests failure:

======================================================================
FAIL: s3tests.functional.test_s3.test_list_buckets_invalid_auth
----------------------------------------------------------------------
AssertionError: u'AccessDenied' != 'InvalidAccessKeyId'

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10667 from dillaman/wip-16980

rbd-mirror: potential race condition during failure shutdown

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

os/bluestore: collect 'dirty' blobs on per-transaction basis to avoid onode::blob_map enumeration

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: migrate to refcounting for Blob class instances

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: remove internal structures from Bluestore.h

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/blueStore: fix discard_unallocated for compressed blob

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #10486 from badone/wip-double-free-global-static-test

tests: Add test for global static non-POD segfault

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix broken link in SHEC erasure code plugin

Fixes: http://tracker.ceph.com/issues/16996
Signed-off-by: Albert Tu <albert.t@inwinstack.com>

cmake: cleanup Findxio.cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: cleanup Findleveldb.cmake

* and reference leveldb library and include dir with fullpath

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10629 from ceph/dillaman-rbd-glance-docs

doc: update rbd glance configuration notes

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10627 from dillaman/wip-16970

rbd-mirror: replaying state should include flush action

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

cmake: cleanup Findgperftools.cmake

* remove Findtcmalloc.cmake, use Findgperftools.cmake instead.
* tcmalloc is packaged in gperftools-dev, so we can check them in a single
  place.
* fix the check of heap-profiler.h, we should enable the check as long
  as tcmalloc is enabled or profiler is enabled.
* only check gperftools headers in "include/gperftools", and do not
  "include/google" anymore. as we only support the distros which
  shipping recent gperftools-dev package with "include/gperftools".
  and "google/*.h" are deprecated.
* set ALLOC_LIBS with GPERFTOOLS_TCMALLOC_LIBRARY, so we can link
  against tcmalloc with full path.

Signed-off-by: Kefu Chai <kchai@redhat.com>

librados: use bufferlist instead of buffer::list

Signed-off-by: Ryne Li <lizhenqiangsnake@gmail.com>

Merge pull request #10410 from xiexingguo/xxg-wip-bluestore-2016-07-23

os/bluestore: narrow lock scope for cache trim()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10362 from xiexingguo/xxg-wip-bluefs-2016-07-20

os/bluestore: fix potential access violation

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

rbd-mirror: potential race condition during failure shutdown

Fixes: http://tracker.ceph.com/issues/16980
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10425 from theanalyst/rgw/sync-paren-warn

rgw: kill a compile warning for rgw_sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10666 from ceph/wip-cmake

cmake: install platlib into a subdir of build-base dir

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: kill a compile warning for rgw_sync

killing the compile warning for

```
/ceph/src/rgw/rgw_sync.cc:1462:12:
warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wparentheses]
         if (can_adjust_marker) yield {
            ^
```

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10640 from vmcloud/libcephfs_jni_overflow

libcephfs: Fix the incorrect integer conversion in libcephfs_jni.cc

Reviewed-by: Noah Watkins <noahwatkins@gmail.com>

cmake: install platlib into a subdir of build-base dir

distutils' install command installs all stuff under $build-base, which
includes the temporary files. and rpm packaging does not like::
  Installed (but unpackaged) file(s)
and to support both py2 and py3, we should encode the python version
into the path, otherwise the built files will be overwritten.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10602 from smanjara/wip-16935

rgw: raise log levels for common radosgw-admin errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10110 from Yan-waller/yj-wip-rgwrados-0702

rgw: remove possible duplicate setting

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9585 from vshankar/wip-15321

librbd: asynchronous v2 image creation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10034 from VictorDenisov/consistency_groups_images

librbd: add consistency groups operations with images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: update rbd glance configuration notes

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10643 from tchaikov/wip-cmake

cmake: fix wrong path introduced by bb163e9

Reviewed-by: Oleh Prypin <oleh@pryp.in>

Merge pull request #10557 from wjwithagen/wip-wjw-test-onexit

test: More portable use of mmap(MAP_ANON)

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: fix wrong path introduced by bb163e9

s/CYTHON_MODULE_LIB/CYTHON_MODULE_DIR/

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: raise log levels for common radosgw-admin errors

Currently while running radosgw-admin command we see these messages thrown at
the endusers since debug level is set to 0:

2016-08-02 14:48:28.687251 7f9e630639c0  0 Cannot find zone
id=7a2a89b9-4f5e-4f60-a29e-451ac9acc5a8 (name=us-west), switching to local
zonegroup configuration
2016-08-02 14:48:35.742185 7f9e630639c0  0 error read_lastest_epoch
.rgw.root:periods.a3db7884-2445-4bab-a165-6730f8573b8f:staging.latest_epoch

Raising the log level to 1.

Fixes: http://tracker.ceph.com/issues/16935
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #9848 from dx9/wip-ac-boost

configure.ac: add _LIBS variables for boost_system and boost_iostreams

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10478 from wjwithagen/wip-wjw-freebsd-cmake-excludes

src/CMakeLists.txt: remove unneeded libraries from ceph-dencoder target

Reviewed-by: Kefu Chai <kchai@redhat.com>

libcephfs: Fix the incorrect integer conversion in libcephfs_jni.cc

Signed-off-by: wenjunhuang <wenjunhuang@tencent.com>

Merge pull request #10568 from tchaikov/wip-cmake

cmake: specify distutils build path explicitly

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #10455 from wjwithagen/wip-wjw-clang-NULL

src/kv/MemDB.cc: the type of the parameter of push_back() does not match the ops's value_type

Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: specify distutils build path explicitly

so we don't need to replicate the logic in distutils/command/build.py
at different places.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10635 from linuxbox2/rgw-file-aiee

rgw_file: unlock() must precede out label

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw_file: unlock() must precede out label

In lookup_handle(...).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

bluestore/BitAllocator: Fix deadlock with musl libc

pthread_rwlock_wrlock() behaviour is undefined when calling thread holds lock. Added unlock() in shutdown() to avoid deadlock in destructor.

http://pubs.opengroup.org/onlinepubs/7908799/xsh/pthread_rwlock_wrlock.html

Signed-off-by: John Coyle <dx9err@gmail.com>

global/signal_handler: use sig_str instead of sys_siglist

musl libc does not provide sys_siglist[]

Signed-off-by: John Coyle <dx9err@gmail.com>

Merge pull request #10623 from tchaikov/wip-16940

admin/build-doc: bypass sanity check if building doc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #10417 from wjwithagen/wip-wjw-freebsd-clang

changes for Clang and yasm

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10232 from weiqiaomiao/wqm-wip-rgw-cleanbl

rgw:clean unuse bufferlist

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10613 from dillaman/wip-16956

rbd-mirror: potential assertion failure during error-induced shutdown

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: replaying state should include flush action

Fixes: http://tracker.ceph.com/issues/16970
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10622 from xiexingguo/xxg-wip-kill-cephfs-warn

libcephfs: kill compiling warning

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9744 from SUSE/wip-16345

rpm: fix shared library devel package names and dependencies

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

admin/build-doc: s/iquote /iquote/

currently this works fine, but we ran into this problem before. and
technically "-iquote<dir>" is the valid syntax, and "-iquote <dir>"
is not. see https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc,pybind: bypass sanity check if building doc

the empty .so file does not compile at all, so we need to bypass the
sanity check, if we are building docs. and what sphinx needs is just the
docstrings.

Fixes: http://tracker.ceph.com/issues/16940
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10612 from tchaikov/wip-16955

test/ceph_test_msgr: fix circular locking dependency

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #10620 from dachary/wip-16961-subman

subman: use replace instead of format

Reviewed-by: Kefu Chai <kchai@redhat.com>