test: fix compilation warnings

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9813 from jcsp/wip-kernel-doc

doc/cephfs: add advice about kernel versions

Merge pull request #9823 from dillaman/wip-16362

rbd-mirror: keep events from different epochs independent

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge branch 'wip-dmick-install-deps-master'

install-deps.sh: use mk-build-deps instead of processing control

Reviewed-by: Erwan Velu <erwan@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

install-deps.sh: use mk-build-deps instead of processing control

mk-build-deps is designed to handle installing build dependencies;
use that instead, so '|' indications are handled properly.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #9791 from ceph/wip-cmake-compile-flags

cmake: Wip cmake compile flags

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9481 from yehudasa/wip-rgw-hold-lease-1

rgw: stop bucket lease only after draining object sync operations

Reviewed-by: Casey Bodley <cbodley@redhat.com>

cmake: CMAKE_BUILD_TYPE defaulted to RelWithDebInfo

Default build type is RelWithDebInfo unless
CMAKE_BUILD_TYPE is specified when running the
`cmake` command.

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #9066 from yangdongsheng/rbd_bench

rbd: introduce a simple bench for read

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

cmake: eliminate FORTIFY_SOURCE warnings

"-werr" was added to CHECK_C_COMPILER_FLAG to
make sure FORTIFY_SOURCE is only turned on if
the flag generates no warnings.

Added "-U_FORTIFY_SOURCE" to CMAKE_C_FLAGS to
mimic PR #9761.

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #9728 from liewegas/wip-bluestore-encoding

os/bluestore: space efficient int encoding

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

rbd-mirror: keep events from different epochs independent

Fixes: http://tracker.ceph.com/issues/16362
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

os/bluestore/bluestore_types: delta encode for ref_map offsets

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: make blob csum info encoding conditional

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: use FLAG_CSUM to indicate if checksums are present

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: fsck: print intervals in hex

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: use varint for pextent count

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: varint or lba encode pextent, lextent, ref_map, onode

Certainly not done here, but this is a start.

Signed-off-by: Sage Weil <sage@redhat.com>

include/small_encoding: small encode/decode helpers

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9819 from dillaman/wip-16363

librbd: fix lockdep issue when duplicate event detected

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9816 from dillaman/wip-16198

librbd: potential race when replaying journal ops

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9788 from dillaman/wip-16230

rbd-mirror: potential race condition accessing local image journal

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9787 from batrick/mds-rename-enotempty

mds: improve comments for ENOTEMPTY checks

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9840 from xiexingguo/xxg-wip-fix-bmap-fm

os/bluestore: minor improvements for bitmap-freelist-manager

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9815 from xiexingguo/xxg-wip-fix-blue-cache

os/bluestore: cache relevant cleanups

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: make bytes_per_key a global member

So we don't have to figure it out everywhere.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: fix a typo(nonexistant->nonexistent)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: simplify BufferSpace::read() a little

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: simplify BufferSpace::read() a little

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: change type of length of region_t from uint32_t to uint64_t

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: add sanity check when try to rm_buffer

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9828 from liewegas/wip-bluestore-read-before-write

os/bluestore: limit read-amp on overwrite

Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #9826 from liewegas/wip-bluefs

os/bluestore/BlueFS: instrument which file types we write to

Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #9834 from tchaikov/wip-silence-warnings

test, bluestore: silence warnings

Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #9635 from liewegas/wip-make-dist

make-dist: include src/.git_version and src/ceph_ver.h

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9836 from dx9/wip-hardcoded-tcmalloc

cmake: fix hard coded tcmalloc lib

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: fix hard coded tcmalloc lib

Signed-off-by: John Coyle <dx9err@gmail.com>

Merge pull request #9670 from ceph/wip-debian-clean

debian: silence couple warnings

Reviewed-by: Dan Mick <dmick@redhat.com>

bluestore/BitAllocator: silence warnings of method hidden

is_allocated(int64_t *blocks, int64_t num_blocks, int blk_off) only gets
called in BitMapAreaIN and its child classes. so it's safe to move this
method down to BitMapAreaIN.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph_test_objectstore: remove unused function

its caller was removed in 8a42e61

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9832 from ceph/wip-cmake-librbd-unittest-fix

cmake: unittest_librbd missing a source file

cmake: unittest_librbd missing a source file

src/test/librbd/test_ConsistencyGroups.cc missing
from ${unittest_librbd_srcs}.

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #9531 from yehudasa/wip-16175

rgw: data sync debug logging

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9676 from efirs/fix_bitmap_freelist

os/bluestore: Fix enumeration when only single key in BitmapFreelistManager

os/bluestore/BlueFS: instrument which file types we write to

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: limit read-amp on overwrite

If we are doing a small overwrite over a blob with a large chunk size
(say, due to a large csum order), we are better off writing into a new
allocation than doing a large read/modify/write.  If the read amp will be
more than min_alloc_size, skip the read entirely and write into a new
blob.

Signed-off-by: Sage Weil <sage@redhat.com>

test: remove remaining references to compression_dir

Removed in 38c7f4a06673e4806d327931744338a7393b45ef

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9829 from ceph/wip-cmake-readme

cmake: updated CMake README instructions

Merge pull request #9814 from songbaisen/a11

mon: remove the repeat quorum size jugement

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9818 from ifed01/wip-zero-length-writes

os/store_test: uniform zero length writes handling

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: updated CMake README instructions

Updated testing and cluster start up instructions
for CMake in README.md

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #9605 from liewegas/wip-statfs

os: simplify store_statfs_t

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #9261 from prallabh/user_quota

rgw: Let the default quota settings take effect during user creation

Merge pull request #9817 from theanalyst/wip/boost-optional

rgw_swift: newer versions of boost/utility no longer include in_place
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

librbd: fix lockdep issue when duplicate event detected

Fixes: http://tracker.ceph.com/issues/16363
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

os/store_test: fix synthetic test cases to handle zero-writes properly, i.e. do not increase object size.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/memstore: zero-length write should not effect object size

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: zero-length write should not affect object size

Signed-off-by: Sage Weil <sage@redhat.com>

rgw_swift: newer versions of boost/utility no longer include in_place

boost > 1.58 no longer includes in_place in boost/utility, we need to
include in_place_factory explicitly. This causes build failures in
distros that ship with a higher version of boost. Since the only call is for
swift_ver_location, another possibility is to use emplace()
instead (though this requires boost ~ 1.56)

Fixes: http://tracker.ceph.com/issues/16391
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #9786 from pritha-srivastava/wip-multisite-error-msgs

rgw: modifying multi-site log messages.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

librbd: potential race when replaying journal ops

Fixes: http://tracker.ceph.com/issues/16198
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9812 from zhjwpku/cephx_protocol

doc: cephx_protocol typos

doc/cephfs: add advice about kernel versions

Signed-off-by: John Spray <john.spray@redhat.com>

  mon: remove the repeat quorum size jugement

  Signed-off-by:song baisen <song.baisen@zte.com.cn>

rgw: Add documentation for RGW default quota

Signed-off-by: Pavan Rallabhandi <PRallabhandi@walmartlabs.com>

Merge pull request #9188 from rzarzynski/wip-rgw-swift-versioning-new-2

rgw: improve support for Swift's object versioning.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

tools/rbd: introduce a simple bench for read

Currently, we have a bench-write for in rbd tool, but we don't have
a simple bench for read.

This patch add a new subcommand of bench, with an option of --io-type.
Then we can do a simple bench for write or read.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

tools/rbd: make bench-write invisible to user

rbd bench-write is going to be deprecated. This patch
makes it invisible in help message and manpage. But it
is still working.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

tools/rbd: introduce a new field of visible in Action Class

The new field means whether we want to show this Action in
command help. Sometimes, we want to make one action to be
deprecated and invisible to user, but we have to keep it
working for compatibility.

Then we need to support this action but make it invisible.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rgw: modifying multi-site log messages.

Increasing the log level and removing 'ERROR' from messages that appear during normal multi-site operation.
Fixes: http://tracker.ceph.com/issues/16121
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #9804 from ceph/revert-9801-msg_msg_types_entity_name_t_makeprivate

Revert "msg/msg_types.h: make members private in ..."

Reviewed-by: Kefu Chai <kchai@redhat.com>

Revert "msg/msg_types.h: make members private in ..."

Merge pull request #9801 from stiopaa1/msg_msg_types_entity_name_t_makeprivate

msg/msg_types.h: make members private in `entity_name_t`

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc:cephx_protocol

The goal of phase II is to provide C with a session key and ticekt
allowing it to communicate with S

Signed-off-by: Zhao Junwang <zhjwpku@gmail.com>

rgw: Do not bother to check against conf values during quota handling

Signed-off-by: Pavan Rallabhandi <PRallabhandi@walmartlabs.com>

rgw: Let the default quota settings take effect during user creation

Signed-off-by: Pavan Rallabhandi <PRallabhandi@walmartlabs.com>

msg/msg_types.h: make members private in ...

... entity_name_t class.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9778 from ifed01/wip-bluestore-blob-reduce

os/bluestore: reduce bluestore blob

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9750 from ifed01/wip-bluestore-csum-output-fix

os/bluestore: fix improper blob's csum visualization.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9756 from liewegas/wip-bluestore-diet

os/bluestore: remove unused bluestore fields

Reviewed-by: Allen Samuels <allen.samuels@sandisk.com>

debian: move libcephfs-jni to section "java"

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian: remove X-Python-Version section

it's obsolete and can be removed now,
see
https://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-specifying_versions

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian: add ${shlibs:Depends} to binary packages' Depends

rationale
* this fixes:
  warning: dpkg-gencontrol: package python-rbd: unused substitution
  variable ${shlibs:Depends}
* and to comply to debian policy. see
  https://www.debian.org/doc/debian-policy/ch-sharedlibs.html#s-dpkg-shlibdeps

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian: add jh_depends for ${java:Depends}

this fixes:
warning: dpkg-gencontrol: Depends field of package libcephfs-java:
unknown substitution variable ${java:Depends}

Signed-off-by: Kefu Chai <kchai@redhat.com>

makefile: add librgw_file_* as noninst_PROGRAMS

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9119 from runsisi/wip-fix-cmake-desc

cmake: update description for cmake module Findkeyutils.cmake

Reviewed-by: Ali Maredia <amaredia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9799 from stiopaa1/common_address_helper_changeScope

common/address_helper.cc:move variables closer...

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9797 from Yan-waller/yj-wip-rgwcompiling

rgw: kill compiling warning

Reviewed-by: Casey Bodley <cbodley@redhat.com>

common/address_helper.cc:move variables closer...

... to the point where they are first used.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

rgw: kill compiling warning

test/rgw/test_http_manager.cc: In member function ‘virtual void HTTPManager_SignalThread_Test::TestBody()’:
test/rgw/test_http_manager.cc:40:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_requests; i++) {
                       ^

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

Merge pull request #9764 from vasukulkarni/master

mention change of owner for /var/log/ceph

Merge pull request #9599 from sdieffen/feature15904

added -f flag for formatting to rados command line tool

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #9785 from cbodley/wip-rgw-httpmgr-select

rgw: fix deadlock in RGWHTTPManager when HAVE_CURL_MULTI_WAIT=0
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9765 from dx9/wip-run-cmake

cmake: add arguments to cmake invocation to support cache configuration

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #9634 from ceph/cmake-git-rev

Update git rev parsing in CMake

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #9601 from ceph/wip-cmake

cmake: add ceph_test_* used by qa units and ceph-qa-suite tasks

Reviewed-by: Ali Maredia <amaredia@redhat.com>

mds: improve comments for ENOTEMPTY checks

There are two checks in the code for ENOTEMPTY, clarify that the first check is
doing a fast check without locks.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #9725 from SUSE/wip-16327

rgw: check for -ERR_NOT_MODIFIED in rgw_rest_s3.cc
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: add pipe fd to set for select() in do_curl_wait()

when HAVE_CURL_MULTI_WAIT is 0, the pipe fd is never added to the
readfds for select(), so FD_ISSET() is always false. this prevents us
from ever trying to read from the fd, and the pipe's buffer eventually
fills up and deadlocks callers of RGWHTTPManager::signal_thread() when
they try to write to the pipe

Fixes: http://tracker.ceph.com/issues/16368
Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: add unit test for HTTPManager

unit test to detect a deadlock in RGWHTTPManager::signal_thread() when
HAVE_CURL_MULTI_WAIT=0

Signed-off-by: Casey Bodley <cbodley@redhat.com>