qa: avoid using make install for fio

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/suites/rados: add perf suite

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa: add cbt task for performance testing

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #17177 from liupan1111/wip-fix-rm

osd: allow FULL_TRY after failsafe

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17220 from amitkumar50/cov-test-2

test: Check to avoid divide by zero

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17313 from mslovy/wip-fix-storetest

os: kstore fix unittest for FiemapHole

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17329 from xiexingguo/wip-21162

mon/OSDMonitor: make 'osd crush rule rename' idempotent

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17296 from amitkumar50/cov-test-4

test: assert check for negative returns

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17297 from amitkumar50/cov-os-2

os: Removing dead code from LFNIndex.cc

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17334 from chenliuzhong/modify-fault_rang-parameter-in-reshard

os/bluestore: ExtentMap::reshard - fix wrong shard length

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

Merge pull request #17337 from scienceluo/wip-luo-fix-cinder-conf-branch

doc: move glance_api_version option to the right place

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17341 from amitkumar50/cov-common-1

common: logically dead code inside shunique_lock.h

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16915 from theanalyst/doc-12.2.0-major-changes

doc: 12.2.0 major release announcements

doc: release notes: changelog for 12.2.0

Amalgamated all the dev & RC changelogs + the changelog we saw post
12.1.4 into one large 12.2.0 changelog.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #17308 from jcsp/wip-21157

mgr: fix crash in MonCommandCompletion

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #17312 from cbodley/wip-21155

qa: create_cache_pool no longer runs 'pool application enable'

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17125 from xiexingguo/wip-add-bucket-with-loc

mon/OSDMonitor: add location option for "crush add-bucket" command

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17239 from zmedico/PGPool-update-eliminate-expensive-union_of

osd/PG: PGPool::update: avoid expensive union_of

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17276 from yuyuyu101/wip-mon-loop

msg/async: don't stuck into resetsession/retrysession loop

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17288 from yuyuyu101/wip-21143

msg/async: don't kill connection if replacing

Reviewed-by: Sage Weil <sage@redhat.com>

common: logically dead code inside shunique_lock.h

Fixes the coverity issue:

** 1352097 Structurally dead code

CID 1352097 (#1 of 1): Structurally dead code (UNREACHABLE)
unreachable: This code cannot be reached: return ceph::shunique_lock<...

Signed-off-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #17254 from joscollin/wip-cleanup-rgw-override

rgw: add missing override in list_keys_init()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa: create_cache_pool no longer runs 'pool application enable'

Fixes: http://tracker.ceph.com/issues/21155
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17311 from hjwsm1989/wip-monop-tracker

mon: use ceph_clock_now if message is self-generated

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17330 from xiexingguo/wip-class-rename-idempotent

mon/OSDMonitor: make 'osd crush class rename' idempotent

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16863 from mikulely/fix-curl-referer

rgw: fix opslog can't record referrer when using curl as client

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #16860 from mikulely/fix-opslog

rgw: fix opslog cannot record remote_addr

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #16933 from amitkumar50/cov-rgw-3

rgw: Initialize is_master, max_aio, size

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #17185 from oritwas/wip-rgw-20777

rgw-admin: require --yes-i-really-mean-it when using --inconsistent_index

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: 12.2.0 major release announcements

Reordered the RC releases sections back to their respective components,
added a ceph-mon section, added links to documentation wherever
possible, and a few forgotten RGW announcements. Also cleared up the
PendingReleaseNotes upto this point

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

os/bluestore:reshard-fix wrong shard length

change fault_range parameter from 'needs_reshard_end' to 'needs_reshard_end-needs_reshard_begin'
be given needs_reshard_end may be out of bounds

Signed-off-by: chenliuzhong <liuzhong.chen@easystack.cn>

doc: move glance_api_version option to the right place

Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>

Merge pull request #16774 from yanyixing/fix-sidenav

mgr/dashboard: fix rbd's pool sub menu

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17283 from liupan1111/wip-final-nbd

rbd-nbd: cleanup for NBDServer shut down

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #17269 from liewegas/wip-health

mon: fix two stray legacy get_health() callers

Reviewed-by: Kefu Chai <kchai@redhat.com

Merge pull request #17328 from tchaikov/wip-kill-gcc-warning

test/unittest_pg_log: silence gcc warning

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17146 from shangzhong/wip-add-line-break

common: add line break for "ceph daemon TYPE.ID version"

Reviewed-by: huanwen ren <ren.huanwen@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17091 from liupan1111/wip-txc-add

os/bluestore: support calculate cost when using spdk.

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #17251 from Songweibin/wip-unused-variable

os/bluestore: drop unused param "what" in apply()

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/OSDMonitor: make 'osd crush class rename' idempotent

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

test/unittest_pg_log: silence gcc warning

this silences following warning:

In file included from ceph/src/test/osd/TestPGLog.cc:24:0:
ceph/src/googletest/googletest/include/gtest/gtest.h:430:16:
warning: ‘virtual void testing::Test::SetUp()’ was hidden
[-Woverloaded-virtual]
   virtual void SetUp();
                ^~~~~
ceph/src/test/osd/TestPGLog.cc:2703:8: warning:   by ‘void
PGLogTrimTest::SetUp(unsigned int, unsigned int, unsigned int)’
[-Woverloaded-virtual]
   void SetUp(unsigned min_entries, unsigned max_entries, unsigned
dup_track) {
        ^~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17326 from neha-ojha/wip-21023-contd

doc: fix order of options in osd new

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17180 from tchaikov/wip-literals-for-options

common/options: use user-defined literals for default values

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16066 from tanghaodong25/bluefs-sync-write

os/bluestore/BlueFS: don't need wait for aio when using _sync_write

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15479 from yonghengdexin735/wip-zzz-const

messages/MDentryLink: add const to member function

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17256 from scienceluo/wip-luo-add-log-branch

auth: add err reason for log info in load function

Reviewed-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: make 'osd crush rule rename' idempotent

Fixes: http://tracker.ceph.com/issues/21162
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: fix order of options in osd new

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #17307 from liewegas/wip-powercycle-slow-trim

qa/suites/powercycle/osd/whitelist_health: whitelist slow trimming

qa/suites/powercycle/osd/whitelist_health: whitelist more

"2017-08-26 16:09:27.704418 mon.a mon.0 172.21.15.169:6789/0 876 : cluster [WRN] MDS health message (mds.0): Behind on trimming (66/30)" in cluster log

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16857 from yanyixing/chartfix

pybind: fix chart size become bigger when refresh

Reviewed-by: John Spray <john.spray@redhat.com>

os: Removing dead code from LFNIndex.cc

Fixes the coverity issue:

** 1394832 Logically dead code
cannot_single: At condition r, the value of r
cannot be equal to 0.
dead_error_condition: The condition r must be true.
CID 1394832 (#1 of 1): Logically dead code (DEADCODE)
dead_error_line: Execution cannot reach the expression
-22 inside this statement: return r ? 0 : -22;.

Signed-off-by: Amit Kumar <amitkuma@redhat.com>

os:kstore fix unittest for FiemapHole

kstore always return [0, object_size] regardless of offset and length

Signed-off-by: Ning Yao <yaoning@unitedstack.com>

test: Check to avoid divide by zero

Fixes the coverity issue:

** 1219467 Division or modulo by zero
CID 1219467 (#1 of 1): Division or modulo by zero (DIVIDE_BY_ZERO)
74. divide_by_zero: In expression 100 / atoi(args[i + 1U]), division
 by expression atoi(args[i + 1U]) which may be zero has undefined behavior.

Signed-off-by: Amit Kumar amitkuma@redhat.com

test: assert check for negetive returns

Fixes the coverity issue:

** 1395371 Argument cannot be negative
1. negative_return_fn: Function open(file, 578, 448) returs a negative number.
2. var_assign: Assigning: signed variable fd = open

** 1395427 Argument cannot be negative
1. negative_return_fn: Function open(file, 578, 448) returns a negative number.
2. var_assign: Assigning: signed variable fd = open

Signed-off-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #17267 from linuxbox2/wip-lock-qlane

rgw_file:  fix LRU lane lock in evict_block()

mon: use ceph_clock_now if message is self-generated

Signed-off-by: huangjun <huangjun@xsky.com>

mgr: fix crash in MonCommandCompletion

Fixes: http://tracker.ceph.com/issues/21157
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #17293 from tchaikov/wip-jewel-x-singleton

qa/suites/rados/upgrade/jewel-x-singleton: tolerate sloppy past_intervals

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17019 from dachary/wip-doc-ceph-disk

doc: document ceph-disk prepare class hierarchy

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/upgrade/jewel-x-singleton: tolerate sloppy past_intervals

See-also: d5d5d7d1
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17284 from liupan1111/wip-nvme-cleanup

bluestore/NVMEDevice: code cleanup

Merge pull request #17253 from hjwsm1989/fix-qa-bug

qa: fix the potential delay of pg state change

Reviewed-by: Sage Weil <sage@redhat.com>

msg/async: don't kill connection if replacing

when existing connection is replacing, then meet send error, we can't kill
this existing connection. otherwise, when peer side reconnect, we will detect
reset condition and send resetsession to make peer discard messages.

Fixes: http://tracker.ceph.com/issues/21143
Signed-off-by: Haomai Wang <haomai@xsky.com>

bluestore/NVMEDevice: remove useless IOSegment, and set
dpdk_thread_adaptor as static.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@intel.com>

bluestore/NVMEDevice: remove useless aio_stop in class NVMEDevice

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

os/bluestore/NVMEDevice: remove not used BufferedExtents.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

rbd-nbd: don't need call "stop()" seperately, and ~NBDServer will do it.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>

Merge pull request #17055 from dillaman/wip-21017

mgr/dashboard: fix duplicate images listed on iSCSI status page

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17216 from Yan-waller/wip-walle-0824imagectx

test/librbd: fix mock method macro of set_journal_policy

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

msg/async: don't stuck into resetsession/retrysession loop

if exist connection's connect_seq is zero and state is STANDBY and peer
connection is new, we will reply with RETRYSESSION and make peer connection
incr connect_seq. Then we receive connect.connect_seq==1, but
existing->connect_seq is still zero, so RESETSESSION tag replied.

RESET->RETRY->RESET loop is forever

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #17275 from ceph/revert-17268-wip-21089

Revert "os/bluestore: compensate for bad freelistmanager size/blocks metadata"

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16494 from asomers/bin_bash

misc: Fix bash path in shebangs

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Revert "os/bluestore: compensate for bad freelistmanager size/blocks metadata"

Merge pull request #17268 from liewegas/wip-21089

os/bluestore: compensate for bad freelistmanager size/blocks metadata

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17271 from xiexingguo/wip-ls-by-class

mon/MonCommands: fix copy-and-paste error

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17272 from scienceluo/wip-luo-silence-buiding-warning-branch

common: silence jenkins's buiding warning in obj_bencher.cc

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17200 from optimistyzy/823_queue_depth

os/bluestore/NVMEDevice: fix the nvme queue depth issue

common: silence jenkins's buiding warning in obj_bencher.cc

Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>

mon/MonCommands: fix copy-and-paste error

Class is definitely required by default for the "crush rule ls-by-class"
command.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: allow FULL_TRY after failsafe

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>

Merge pull request #17270 from liewegas/wip-luminous-note

doc/release-notes: clarify purpose of require-osd-release

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

NVMEDevice: fix the nvme queue depth issue

We need to track the queue depth usage for each qpair to
prevent submit too many I/Os. If there are too many requests,
we need to poll the qpair, to make sure there is enough slots
for next I/O submit.

Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>

doc/release-notes: clarify purpose of require-osd-release

Signed-off-by: Sage Weil <sage@redhat.com>

mon: fix two stray legacy get_health() callers

These need to call the new method when appropriate.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: compensate for bad freelistmanager size/blocks metadata

This repairs bluestores created before http://tracker.ceph.com/issues/21089
was fixed in f6f1ae3724d593d3709d982c973ec18a25a47b6e.

In both cases, the freelistmanager's size is off by one block (4k).  In
one case, it is just a matter of fixing the size and twiddling the trailing
bit.  In the second case, the size delta causes freelistmanager to need
a new row, which means the blocks count also changes, and we have lots
of bits to zero (all but one in the new row).

Both are silently corrected by fsck in this patch.

Fixes: http://tracker.ceph.com/issues/21089
Signed-off-by: Sage Weil <sage@redhat.com>

rgw_file:  fix LRU lane lock in evict_block()

Found by "Supriti Singh" <Supriti.Singh@suse.com>.

Fixes http://tracker.ceph.com/issues/21141

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #17245 from jdurgin/wip-dup-perf

osd/PGLog: write only changed dup entries

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17225 from liewegas/wip-bluestore-fsck-leaks

os/bluestore: print leaked extents to debug output

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17242 from neha-ojha/wip-21023

mon: warn about using osd new instead of osd create

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17248 from tchaikov/wip-20987

mon/OSDMonitor: check last_scan_epoch instead when sending creates

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17246 from tchaikov/wip-20913

osd/PGBackend: release a msg using msg->put() not delete

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17029 from xiexingguo/wip-crush-rule-rename

mon: "ceph osd crush rule rename" support

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17103 from C2python/clear_attr

common: Destroy attr of RWLock after initialized

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17126 from xiexingguo/wip-nicenum

common/types: make numbers a bit nicer when displaying space usage

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17182 from tchaikov/wip-obj-bencher

common/obj_bencher: set {min,max}_iops if runtime < 1 sec

Reviewed-by: PCzhangPC <pengcheng.zhang@easystack.cn>

Merge pull request #17221 from xiexingguo/wip-pg-sparse-read

osd/PrimaryLogPG: fix sparse read won't trigger repair correctly

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17222 from xiexingguo/wip-propagate-eagain

osd/PrimaryLogPG: do_osd_ops - propagate EAGAIN/EINPROGRESS on failok

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17258 from liewegas/wip-21127

qa/standalone/scrub/osd-scrub-repair: fix grep pattern

Reviewed-by: David Zafman <dzafman@redhat.com>

qa/standalone/scrub/osd-scrub-repair: fix grep pattern

PGMap shows

    ss << pg_sum.stats.sum.num_objects_unfound
       << "/" << pg_sum.stats.sum.num_objects << " objects unfound (" << b << "%)";

but we were grepping for "1/1 unfound" instead of "1/1 objects
unfound".

Introduced by fe81b7e3a5034ce855303f93f3e413f3f2dc74a8.

Fixes: http://tracker.ceph.com/issues/21127
Signed-off-by: Sage Weil <sage@redhat.com>