qa/workunits/rbd: new mirroring forced promotion test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11153 from dillaman/wip-17317

test/rbd: fix possible mock journal race conditions

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11054 from rzarzynski/wip-rgw-postobj-nodynalloc

rgw: eradicate dynamic memory allocation in RGWPostObj.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11208 from tchaikov/wip-17380

osd: should not look up an empty pg

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #10949 from avnerbh/avnerbh-preforker-fix

preforker: prevent call to 'write' on an fd that was already closed

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: re-enable test_pg_scrub() test in ceph-helper.sh

this reverts d053705. i disabled this test in hope to bisect the
offending tests that fail the mysterious jenkins failure, which was
fixed by 6f3ce3a.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: should not look up an empty pg

this fixes the scrub timeouts we spotted in jenkins run and qa run.
it's a regression introduced by c2b7cdb

Fixes: http://tracker.ceph.com/issues/17380
Signed-off-by: Loic Dachary <loic@dachary.org>
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11205 from liewegas/wip-async-debug

msg/async: less verbose debug messages at debug_ms=1

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #11181 from linuxbox2/rgw-legacy-time

rgw_file:  pre-assign times

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11048 from jimifm/wip-FDCache-master

os/filestore/FDCache: fix bug when filestore_fd_cache_shards = 0

Reviewed-by: Sage Weil <sage@redhat.com>

msg/async: no need for ..."started." suffix

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: less noisy about EventCenter owner

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: make rx line look like normal debug

The DispatchQueue prints the traditional ==== line.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: make send debug line look like SimpleMessenger's

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async/AsyncConnection: less noisy about connection top

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async/EventCenter: less noisy debug on events

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11190 from tchaikov/wip-memstore-clone-fixture

tests: use a fixture for memstore clone testing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10587 from b-ranto/wip-logrotate-root

logrotate: Run as root/ceph

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #7338 from ceph/cls-lua

Lua object class support

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Ken Dreyer (Red Hat) <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10135 from david-z/wip-enhance-ceph-disk-bluestore

ceph-disk: support creating block.db and block.wal with customized size for bluestore

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #10371 from Yan-waller/yj-wip-msgsimple-0720

msg/simple: close the failed socket fds

https://github.com/ceph/ceph/pull/11052

Merge pull request #11052 from yuyuyu101/wip-shutdown-deadlock

OSD: avoid FileStore finisher deadlock in osd_lock when shutdown OSD

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11082 from aiicore/nearfull_flag_fix

mon: OSDMonitor: Missing nearfull flag set

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11089 from dachary/wip-16477-ceph-helper

cli: retry when the mon is not configured

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11199 from yuyuyu101/wip-17378

msg/async: remove fd output in log prefix

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11195 from xiexingguo/xxg-wip-csum-improve

os/bluestore: a few fixes about  the global csum setting

Reviewed-by: Sage Weil <sage@redhat.com>

msg/async: remove fd output in log prefix

it's tricky to prevent fd racing in logs, otherwise we need to provide with
lock to totally avoid this. But I don't think it deserve lock

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #11182 from ceph/wip-docs-rm-ink

remove old references to inktank premium support

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11194 from wjwithagen/wip-wjw-rocksdb-include

src/kv/CMakeLists.txt: force rocksdb/include to first include directory

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: fix race condition of global csum setting

The global csum_type option can be changed dynamically and
can be fixed by upgrading into std::atomic.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: drop the global bluestore_csum option

We don't need this option as it can be properly covered
by the bluestore_csum_type option.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: use blob internal settings for csum checking

This is because:
1. It is more accurate, as the global csum settings may change dynamically.
2. bluestore_blob_t::verify_csum() can handle both csum-on and csum-off correctly.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11151 from ceph/wip-bluestore-error-injection

os/bluestore: add error injection

Merge pull request #11184 from liewegas/wip-down-out

mon: wait 10m (not 5m) before marking down OSDs out

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #11186 from majianpeng/bluestore-allocator

os/bluestore: remove unused head file.

Reviewed-by: Sage Weil <sage@redhat.com>

rgw_file:  pre-assign times

Set unix timestamps based on RGW values for creation, modification
time for buckets, objects in stat requests.  Regard any saved
value of these in unix attributes as an overlay.

Fixes: http://tracker.ceph.com/issues/17367
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #11192 from xiexingguo/xxg-wip-bluestore-0922

os/bluestore: a few cleanups

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11152 from dillaman/wip-17198

librbd: potential deadlock closing image with in-flight readahead

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10506 from wjwithagen/wip-wjw-clock-realtime

FreeBSD: Define CLOCK_REALTIME_COARSE in compat.h

src/kv/CMakeLists.txt: force rocksdb/include to first include directory

 - Otherwise on FreeBSD older packages could be used for include
   this then can result to missing declared functions because
   includes and libs are out of sync

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

doc/rados/troubleshooting: remove references to premium support from inktank.com

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados: remove references to premium support from inktank.com

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #11147 from ukernel/wip-17286

client: fix readdir vs fragmentation race

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10393 from ukernel/wip-mds-log-flush

reduce unnecessary mds log flush

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11155 from dillaman/wip-17330

librbd: backward/forward compatibility for update_features

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11148 from dillaman/wip-17310

librbd: block name prefix might overflow fixed size C-string

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

os/bluestore: remove unused head file.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #11176 from zealoussnow/wip-0922

ceph_fuse: use sizeof get the buf length

Reviewed-by: Kefu Chai <kchai@redhat.com>

test_memstore_clone: use StoreTestFixture to do the setup/teardown

rename test case name from `MemStore` to `MemStoreClone`, because
if we use `TEST_F` to group the tests into a fixture, `MemStore`
will be used as the class name of the fixture, and hence conflicts
with the existing `MemStore` class under testing.

but the object store is created and destroyed for every test. before
this change, we do this for every test case. if it's a concern
we need to use SetUpTestCase() and TearDownTestCase() instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph_test_objectstore: fail the test if SetUp() fails

so it will issues a fatal error if anything goes wrong, instead of continuing
with the test cases.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph_test_objectstore: extract StoreTestFixture into store_test_fixture.cc

so it can be reused by other tests like test_memstore_clone.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: clean up finishers properly

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: move lookup() of SharedBlobSet back into .h

Because the lookup() method is simple and easy,
and requires no dout for debugging.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon: wait 10m (not 5m) before marking down OSDs out

5m isn't quite enough time for many boxes to reboot.  10m will do
a better job of masking transient crashes without having much
effect on time to repair.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: add error injection

This is used by the teuthology repair_test.py.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11122 from branch-predictor/bp-zlib-improvements

compressor/zlib: switch to raw deflate

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11108 from liewegas/wip-bluefs-dirty-fnodes

os/bluestore/BlueFS: log dirty files at sync time

ceph_fuse: use sizeof get the buf length

Signed-off-by: LeoZhang <nguzcf@gmail.com>

Merge pull request #11173 from tchaikov/wip-17306

test/cli/crushtool: fix the test of compile-decompile-recompile.t

Reviewed-by: Samuel Just <sjust@redhat.com>

test/cli/crushtool: fix the test of compile-decompile-recompile.t

should read the map from $TESTDIR,
it's a regression introduced by b2c0a07

Fixes: http://tracker.ceph.com/issues/17306
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11166 from xiexingguo/xxg-wip-bluestore-0921

os/bluestore: simplify calculation of collection key range

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11131 from ceph/rgw-lifecycle-testing

Rgw lifecycle testing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Lifecycle testing

Add new config option - rgw_lc_debug_interval - that modifies the
lifecycle algorithm.  It treats days as being the configured number of
seconds.  This allows lifecycle to be tested in minutes, rather than
days.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Lifecycle - Fix config locking

The lifecycle config lock was not setting a cookie.  This means that all
instances of that lock conflict, causing errors to be returned to the
client.

Set a unique cookie on each lock instance.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #11031 from yangdongsheng/du

rbd: return error if we specified a wrong image name for rbd du

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11163 from jcsp/wip-cmake-vstart

doc: Update developer docs for cmake paths

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

doc: update vstart guide for cmake-ish paths

Signed-off-by: John Spray <john.spray@redhat.com>

doc: update build instructions for cmake

Signed-off-by: John Spray <john.spray@redhat.com>

os/bluestore: simplify calculation of collection key range

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11083 from ifed01/wip-load-plugins-fix

common/compressor: add libcommon as a dependency for zlib and snappy p…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10975 from stiopaa1/osd_replicatedbackend_moveSomeClassesToCC

osd/PGBackend.h: move structs to .cc file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11073 from badone/wip-remove-check_disk_write_cache

FileJournal: Remove obsolete _check_disk_write_cache function

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #11060 from stiopaa1/mon_osdmonitor_removeAssign

mon/OSDMonitor.cc: remove use of boost assign

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11039 from stiopaa1/osd_inprogress_removingCount

osd/OSD.cc: remove unneeded searching of maps

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge branch 'wip-rocksdb-librados-test'

qa: add test of envlibrados for rocksdb

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

add qa test of envlibrados for rocksdb

Signed-off-by: Ryne Li <lizhenqiangsnake@gmail.com>

Merge pull request #11146 from tchaikov/wip-17307

os/filestore/FileJournal: error out if FileJournal is not a file

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #11103 from liewegas/wip-clone-range

ceph_test_objectstore: test clone_range and fix a few bugs

ceph_test_objectstore: disable filestore_fiemap

This very reliably triggers a test failure for
ObjectStore/StoreTest.Synthetic/1.

FIEMAP is bad!  Do not use it!

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: exercise clone_range from synthetic

Signed-off-by: Sage Weil <sage@redhat.com>

os/memstore: fix bug in BufferlistObject::write

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: clarify clone_range semantics

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11157 from cbodley/wip-memstore-clone

memstore: clone zero-fills holes from source range

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10991 from weiqiaomiao/wqm-wip-sync-error

rgw: fix failed to create bucket if a non-master zonegroup has a single zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10942 from JellevdK/master

systemd: add install section to rbdmap.service file

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test: add tests for memstore clone_range

Signed-off-by: Casey Bodley <cbodley@redhat.com>

memstore: clone zero-fills holes from source range

Signed-off-by: Casey Bodley <cbodley@redhat.com>

memstore: use const_iterator instead of const_cast

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11070 from VictorDenisov/consistency_groups_extract_module

librbd: extract group module from librbd/internal

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: update_features should handle Infernalis OSDs

Fixes: http://tracker.ceph.com/issues/17330
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls/rbd: set_features should adjust the mask to known features

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/rbd: fix possible mock journal race conditions

Fixes: http://tracker.ceph.com/issues/17317
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9844 from rzarzynski/wip-rgw-swift-website

rgw: add support for Static Website of Swift API

Reviewed-by: Casey Bodley <cbodley@redhat.com>

librbd: potential deadlock closing image with in-flight readahead

Fixes: http://tracker.ceph.com/issues/17198
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11149 from xiexingguo/xxg-wip-bluestore-0920

os/bluestore: narrow condition of sanity check when get_object_key()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11121 from stiopaa1/osd_osd_moveMembersToPrivate

osd/OSD.h: move some members under private

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10563 from stiopaa1/mon_healthmonitor_removeUnneededInclude

mon/HealthMonitor: remove unneeded include

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10515 from stiopaa1/mon_monitor_addOverride

mon/Monitor.h: add override to virtual functions

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10333 from stiopaa1/msg_EventSelect_RemoveIncFromHeader

msg/EventSelect: move includes to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10269 from stiopaa1/osd_watch_removeUnneededForwardDecl

osd/Watch.h: remove unneeded forward declaration

Reviewed-by: Sage Weil <sage@redhat.com>