test_ImageReplayer.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_user.cc: remove unused code

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_sync.*: pass const parameter by reference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_op.h: init scalar field in ctor

Fix for:

CID 717368 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member is_truncated is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Capability.h: init scalar field in ctor

Fix for:

CID 1019612 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member before is not initialized in
  this constructor nor in any functions that it calls.
 uninit_member: Non-static class member seq is not initialized in
  this constructor nor in any functions that it calls.
 uninit_member: Non-static class member last_issue is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Capability.h: init scalar fields in ctor

Fix for:

CID 1138591 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member cap_id is not initialized in this
  constructor nor in any functions that it calls.
 uninit_member: Non-static class member issue_seq is not initialized in
  this constructor nor in any functions that it calls.
 uninit_member: Non-static class member mseq is not initialized in this
  constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/bit_vector.hpp: init scalar field in ctor

Fix for:

CID 1274314 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member m_header_crc is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

tools/cephfs/DataScan.h: init scalar field in ctor

Fix for:

CID 1313449 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member force_init is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_op.h: init scalar field in ctor

Fix for:

CID 1351689 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member total_size is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Capability.h: init non-static Export members in ctor

Fix for:

CID 717232 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member cap_id is not initialized
  in this constructor nor in any functions that it calls.
 uninit_member: Non-static class member wanted is not initialized
  in this constructor nor in any functions that it calls.
 uninit_member: Non-static class member issued is not initialized
  in this constructor nor in any functions that it calls.
 uninit_member: Non-static class member pending is not initialized
  in this constructor nor in any functions that it calls.
 uninit_member: Non-static class member seq is not initialized in
  this constructor nor in any functions that it calls.
 uninit_member: Non-static class member mseq is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/WatchNotifyTypes.h: init member vars in ctor

Fix for:

CID 1351734 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member result is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/rgw/rgw_rados.h: init some RGWPeriod member vars in ctor

Fix for:

CID 1353422 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
 uninit_member: Non-static class member cct is not initialized in this
  constructor nor in any functions that it calls.
 uninit_member: Non-static class member store is not initialized in
  this constructor nor in any functions that it calls.
CID 1353426 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
 uninit_member: Non-static class member cct is not initialized in this
  constructor nor in any functions that it calls.
 uninit_member: Non-static class member store is not initialized in
  this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

include/types.h: init members in ctor

CID 1351708 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member code is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/kstore/KStore.cc: init nid_last in ctor

Fix for:

CID 1351706 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member nid_last is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/fs/FS.h: init member vars in aio_t ctor

CID 1351707 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member offset is not initialized
  in this constructor nor in any functions that it calls.
 uninit_member: Non-static class member length is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSD.cc: init daily_loadavg in ctor

CID 1351705 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member daily_loadavg is not
  initialized in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/librbd/test_librbd.cc: fix OVERRUN

Fix for:

CID 1297862 (#1 of 1): Out-of-bounds access (OVERRUN)
 alloc_strlen: Allocating insufficient memory for the terminating
  null of the string.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rados.h: init non-static class members in ctor

Fix for:

CID 1353420 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
 uninit_member: Non-static class member cct is not initialized in
  this constructor nor in any functions that it calls.
 uninit_member: Non-static class member store is not initialized in
  this constructor nor in any functions that it calls.
CID 1353421 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
 uninit_member: Non-static class member cct is not initialized in
  this constructor nor in any functions that it calls.
 uninit_member: Non-static class member store is not initialized
  in this constructor nor in any functions that it calls.
CID 1353427 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
 uninit_member: Non-static class member cct is not initialized in
  this constructor nor in any functions that it calls.
 uninit_member: Non-static class member store is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-deploy.rst: document missing --mkfs option

Add missing option to the correct section to fix build
warnings. Resort the 'Options' section to alphabetical order.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/ConfUtils.cc: fix potential integer overflow

Fix for:

CID 1128394 (#1 of 1): Integer overflowed argument (INTEGER_OVERFLOW)
 overflow: Add operation overflows on operands line_len and 1UL.
 overflow_assign: Assigning overflowed or truncated value (or a value
  computed from an overflowed or a truncated value) to rem.
 overflow_sink: Overflowed or truncated value (or a value computed from an
  overflowed or truncated value) rem used as critical argument to function.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/msgr/test_async_driver.cc: fix UNINIT

Fix for:

CID 1254379 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use_in_call: Using uninitialized value c when calling write.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

erasure-code/shec/ErasureCodeShec.cc: use memset to fill integer VLAs

Fix for:

CID 1313448 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use: Using uninitialized value want[i + this->k]

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/exclusive_lock/AcquireRequest.cc: init lock_type

Fixup for:

CID 1351687 (#1 of 1): Uninitialized scalar variable (UNINIT)
 var_decl: Declaring variable lock_type without initializer
 uninit_use: Using uninitialized value lock_type.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/librgw_file.cc: remove not needed variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/bluestore/NVMEDevice.cc: init some member vars in ctor

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #11811 from tanghaodong25/reap-ioc-wip

os/bluestore: reap ioc when stopping aio_thread.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11832 from ceph/wip-python-rgw

deb: add python-rgw packages

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

debian: add python-rgw package

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11845 from xiexingguo/xxg-wip-bs-1109

os/bluestore: remove duplicated namespace of tx state

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11859 from ceph/wip-py-rgw

doc: add python-rgw doc

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #11823 from xiexingguo/xxg-wip-bluestore-1108

os/bluestore: refactor _do_write(); move initializaiton of csum out of loop

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #10748 from yuyuyu101/wip-dpdk8

msg/async: DPDKStack as AsyncMessenger backend

doc: add doc for pybind of rgw

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind: rename rgw_file.pyx to rgw.pyx

so the init function offered by rgw python extension is named "initrgw"
instead of "initrgw_file". and "initrgw" is the expected name by python
intepreter when loading an extension.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11847 from yuyuyu101/wip-fix-rgw-spec

ceph.spec: add pybind rgwfile

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph.spec: add pybind rgwfile

Signed-off-by: Haomai Wang <haomai@xsky.com>

os/bluestore: remove duplicated namespace of tx state

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

test/CMakeLists: only enable ceph_test_userspace when dpdk is true

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/dpdk: change error to warning

Signed-off-by: Haomai Wang <haomai@xsky.com>

update dpdk submodule to ceph/dpdk

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/Stack: allow to setup dpdkstack

Signed-off-by: Haomai Wang <haomai@xsky.com>

test_async_networkstack: fix unmatched create/delete file event

Signed-off-by: Haomai Wang <haomai@xsky.com>

test/test_async_networkstack: add dpdk backend test

Signed-off-by: Haomai Wang <haomai@xsky.com>

test: add userspace_event

Signed-off-by: Haomai Wang <haomai@xsky.com>

Event: don't wakeup dpdk driver since it's spinning on cpu

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/Event: add DPDKDriver type to allow userspace event

Like epoll, kqueue, select, dpdk backend also need to allow notify EventCenter
when a connection is ready to read/write. So we need to let DPDKStack can kick
EventCenter. DPDKDriver which introduced UserspaceEvent to simulate epoll
event achieve the goal.

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async: add dpdk stack plugin

Signed-off-by: Haomai Wang <haomai@xsky.com>

buffer: add new ptr type which destruct with passed deleter

deleter is a wrapper for lambda which can used to free resource associated
with object

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/Event: add Poller to allow event thread spin on Poller::poll

Signed-off-by: Haomai Wang <haomai@xsky.com>

Event: Add get_driver to return driver backend

Signed-off-by: Haomai Wang <haomai@xsky.com>

Throttler: Add reset support

Signed-off-by: Haomai Wang <haomai@xsky.com>

os/bluestore: move selectiong of csum/crr out of for loop

We don't have to trace the csum/compression_required_ration option changing
at such a small granularity, which costs too much.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: refactor _do_write()

By moving garbage collection logic into a separate method.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11681 from stiopaa1/mds_mdstableclient_addConsttoMemFun

mds/MDSTableClient.h: add const to member funct

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11818 from liewegas/wip-bluestore-aio-stalls3

os/bluestore: optional debug mode to identify aio stalls

Merge pull request #11826 from ovh/wip-doc-update-xattr-config-documentation

doc: Update filestore xattr config documentation.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11831 from ceph/wip-dang-rgw-pybind

RGW Python bindings - use explicit array

RGW Python bindings - use explicit array

Older Cython doesn't to implicit simple arrays, so use an explicit array
for stat output.

Revert change removing stat

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #11816 from cbodley/wip-17812

rgw: delete entries_index in RGWFetchAllMetaCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

os/bluestore: reap ioc when stopping aio_thread.

there is possibillity that reap_ioc don't excute when stopping
aio_thread. so add reap_ioc after aio_thread is stopped.

Signed-off-by: Haodong Tang <haodong.tang@intel.com>

Merge pull request #11817 from linuxbox2/wip-boost-3

boost: embedded

Merge pull request #11825 from ceph/wip-11820

pybind: fix build failure of rgwfile binding

Reviewed-by: Haomai Wang <haomai@xsky.com>

doc: Update filestore xattr config documentation.

Signed-off-by: Bartłomiej Święcki <bartlomiej.swiecki@corp.ovh.com>

pybind: fix build failure of rgwfile binding

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9624 from gaurav36/coverity-1355603-dead-code

coverity fix: fixing few coverity issue

Merge pull request #11798 from stiopaa1/mds_fsmap_preventcopyinginloop

mds/FSMap.cc: prevent unneeded copy of map entry

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11802 from stiopaa1/mds_fsmap_constReference

mds/FSMap.h: add const and reference

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11819 from stiopaa1/mds_migrator_addConstTomemFun

mds/Migrator.h: add const to member functions

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11807 from xiexingguo/xxg-wip-bs-1107

os/bluestore: speed up omap-key generation for same onode

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: speed up omap-key generation for same onode

This is useful if we have to batch-process omap kvs.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11754 from athanatos/wip-17789

osd/ReplicatedPG: do_update_log_missing: take the pg lock in the callback

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore/KernelDevice: debug support to detect stalled aios

This is additional bookkeeping to track the oldest aio in flight, so we
make it optional, and disabled by default.  It is useful for identifying
buggy kernel aio or bad devices.

Signed-off-by: Sage Weil <sage@redhat.com>

os/fs: add aio_list_t type

Signed-off-by: Sage Weil <sage@redhat.com>

os/fs: uninline aio methods

Signed-off-by: Sage Weil <sage@redhat.com>

mds/Migrator.h: add const to member functions

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

boost:  pull up boost to enable boost_python build

Pullup to just the merges on Sep 10 fixes an obvious compile
problem, but there are other relevant changes.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Adapt for ceph-manager

The ceph manager change added a separate Boost dependency
check, add it to the combined section.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

do_cmake.sh: set BOOST_J to $(nproc)

This lets us build boost with all processors.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

boost:  update boost to 8/29

Pull-up Boost to the commit which merged the upstream fix for
upstream Boost/iostreams bug #11860 (template arity warnings),
per discussion in core.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

cmake:  find and build bundled boost

Build Boost using the src/boost submodule, unless overridden
by -DWITH_SYSTEM_BOOST.

If -DBOOST_J=<n> is provided, builds Boost with <n> jobs.

Boost builds in the configured Build directory.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

boost: add submodule at tag boost-1.61.0

This implements option #4 for external boost, based on upstream
discussion.  In option #4:

1. boost is added as a submodule
2. builds default to using the attached boost module
3. building against a system-provided boost is supported, but
   must be configured explicitly

Because all of the boost components are attached as nested
submodules in the upstream boost repository, neither the nested
submodules nor the root boost submodule have been cloned into
modules in github.com/ceph (acked by Sage).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #11647 from ceph/wip-jlayton-cephfs

libcephfs client API overhaul and update

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11425 from weiqiaomiao/wqm-rgw-oplog

rgw: fix the field 'total_time'  of log entry in log show opt

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8712 from diwakargoel/master

doc: Correcting the sample python tempurl generation script.

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #11648 from cbodley/wip-17698

rgw: fix for bucket delete racing with mdlog sync

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #11598 from cbodley/wip-rgw-bucket-sync

rgw multisite: move lease up to RunBucketSync instead of child crs

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: delete entries_index in RGWFetchAllMetaCR

use a unique_ptr to ensure the entries_index is freed on destruction

Fixes: http://tracker.ceph.com/issues/17812
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11715 from dzafman/wip-test-ec-eio

Fix racey test by setting noout flag (tracker 17757)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11624 from yuyuyu101/wip-rgwfile-pybind

Wip rgwfile pybind

pybind/rgwfile: change description

Signed-off-by: Haomai Wang <haomai@xsky.com>

pybind/rgwfile: fixes hardcode way to find include dir

Signed-off-by: Haomai Wang <haomai@xsky.com>

pybind/rgwfile: raise Exception when failing to umount

Signed-off-by: Haomai Wang <haomai@xsky.com>

pybind/rgwfile: change FileHandler to FileHandle

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #11794 from rzarzynski/wip-rgw-17803

rgw: fix uncompressed object size deduction in RGWRados::copy_obj_data.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11797 from yuyuyu101/wip-bluestore-pending

os/bluestore: reap collection after all pending ios done

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11713 from stiopaa1/mds_mantle_passByConstRef

mds/Mantle: pass parameters by const ref

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11727 from stiopaa1/mds_mdsrank_addOverideToThreadClass

mds/MDSRank.h: add override to virtual function

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11781 from trociny/wip-qa-rbd-nbd-sudo

qa/workunits/rbd: simplify running nbd test under build env

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11806 from david-z/wip-remove-dup-log

mds: remove duplicated log in handle_client_readdir

mds: remove duplicated log in handle_client_readdir

Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>