mgr: add override in mgr subsystem

Fixes: http://tracker.ceph.com/issues/18922
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #13378 from liewegas/wip-luminous

qa/rados/upgrade/jewel-x-singleton: run luminous.yaml at the end

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #13365 from xiexingguo/wip-backoff-refleak

osdc/Objecter: fix possible OSDSession leak on wrong connection

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13363 from jdurgin/wip-req-luminous-error

osd: clarify REQUIRE_LUMINOUS error message

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/rados/upgrade/jewel-x-singleton: run luminous.yaml at the end

renumber too

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13369 from idryomov/wip-doc-kernel-vers

doc: kernel client os-recommendations update

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13229 from trociny/wip-18335

rbd-nbd: check /sys/block/nbdX/size to ensure kernel mapped correctly

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13292 from tchaikov/wip-script-sepia-bt-support-xenial

script/sepia_bt.sh: support xenial

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

osdc/Objecter: fix possible OSDSession leak on wrong connection

This is introduced by the newly added backoff logic.
Not sure if it will really happen, but just in case.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12342 from liewegas/wip-backoff

osd,osdc: pg and osd-based backoff

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/dev/rados-client-protocol: describe some of rados protocol, including backoffs

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/thrash: vary backoff settings

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: backoffs on individual objects

Do these midway down do_op.  Reorder the scrub waitlist after the
degraded and unreadable waitlists.

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: respect backoffs

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: PG-wide backoffs

Issue at top of do_request.  Release on activation or peering
interval change.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: dispatch OSDBackoff messages into PG workqueue

Signed-off-by: Sage Weil <sage@redhat.com>

osd: split backoffs on PG split

Signed-off-by: Sage Weil <sage@redhat.com>

osd: add pg helpers for managing backoffs

Link Backoff into Session, PG.  Tear them down on session
reset.

Signed-off-by: Sage Weil <sage@redhat.com>

include/ceph_features: RADOS_BACKOFF feature bit

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

common/hobject: use hash field for hash<>

No reason to waste CPU recalculating a hash value!

Signed-off-by: Sage Weil <sage@redhat.com>

osd: add MOSDBackoff message type

Signed-off-by: Sage Weil <sage@redhat.com>

osd: move Session class out of OSD

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: attach OSDSession to Connection

This lets us avoid an rbtree lookup.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: clarify REQUIRE_LUMINOUS error message

This should be visible at the default log level, so users have some
hope of figuring out the problem.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

doc: ref cephfs docs from os-recommendations

...instead of having potentially conflicting kernel
version information.

Signed-off-by: John Spray <john.spray@redhat.com>

doc: update krbd kernel recommendations

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #13249 from liewegas/wip-mon-smooth

mon: smooth io/recovery stats over longer period

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13264 from tchaikov/wip-print-failed-test-name

qa/workunits/rados/test.sh: print test name when it fails

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13362 from adamemerson/wip-oh-yeah

test: Fix mismatched sign comparison in histogram test

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13344 from gregsfortytwo/wip-osd-discussion-docs

Wip osd discussion docs

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #13342 from athanatos/wip-17831-18583-18809

osd/: don't leak context for Blessed*Context or RecoveryQueueAsync

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

test: Fix mismatched sign comparison in histogram test

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #13137 from jcsp/wip-18661

qa: fix race in Mount.open_background

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13099 from jcsp/wip-18663

qa/tasks: force umount during kclient teardown

Merge pull request #13281 from theanalyst/release/v12.0.0

doc: v12.0.0 release notes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13302 from YankunLi/lyk_clean_code

rgw: clean redundant code

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13304 from majianpeng/async-tracing

msg/async: cleanup code.

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13329 from Adirl/fix_febe

msg: Fix calls to Messenger::create with new parameter

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13351 from tchaikov/wip-init-ceph-newline

init-ceph: print trailing n in "status" output

Reviewed-by: Nathan Cutler <nculter@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13356 from rubenk/rgw-docs-fix-typo

doc: Fix typo and grammar in RGW config reference

Reviewed-by: Sage Weil <sage@redhat.com>

qa/workunits/rados/test.sh: print test name when it fails

we have

2017-02-04T16:15:46.090 INFO:tasks.workunit.client.0.mira032.stdout:error in 22088
2017-02-04T16:15:46.092 INFO:tasks.workunit.client.0.mira032.stderr:bash: line 1: 22092 Alarm clock             ceph_test_rados_api_aio 2>&1
2017-02-04T16:15:46.096 INFO:tasks.workunit.client.0.mira032.stderr:     22093 Done                    | tee ceph_test_rados_api_aio.log
2017-02-04T16:15:46.099 INFO:tasks.workunit.client.0.mira032.stderr:     22094 Done                    | sed "s/^/                  api_aio: /"
2017-02-04T16:15:46.102 INFO:tasks.workunit.client.0.mira032.stderr:+

if a unittest in rados/test.sh fails in teuthology.log, but it would
be desirable to have the failed test name in the line of "error in
22088".

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13258 from trociny/wip-18819

common: possible lockdep false alarm for ThreadPool lock

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13287 from guihecheng/rbd-fix

librbd: fix segfault on EOPNOTSUPP returned while fetching snapshot timestamp

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: Fix typo and grammar in RGW config reference

Signed-off-by: Ruben Kerkhof <ruben@rubenkerkhof.com>

Merge pull request #13322 from ceph/wip-include-workqueue

librbd: Include WorkQueue.h since we use it

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

init-ceph: print trailing n in "status" output

the version command is handled by HelpHook, which does not print newline
at the end of the returned JSON. so the output of "status" subcommand
looks like:

=== osd.1 ===
osd.1: running {"version":"Development"}=== osd.2 ===
osd.2: not running.

with this change, a newline is added to the end of version.

also use the "daemon" subcommand to talk to the asok, let the "ceph" cli
to figure out the asok path.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13338 from kshtsk/add-bc-to-deps

ceph.spec.in, debian/control: Add bc to build dependencies

Reviewed-by: Dan Mick <dan.mick@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13334 from tchaikov/wip-osd-scrub-repair-more-patient

tests: osd-scrub-repair.sh disable scrub backoff in test

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph.spec.in, debian/control: Add bc to build dependencies

The bc is missing for ceph-helpers.sh

Fixes: http://tracker.ceph.com/issues/18876
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>

OSD: update OSD backfill discussion comments

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #13309 from dachary/wip-mailmap

mailmap updates

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

tests: osd-scrub-repair.sh disable scrub backoff in test

there is good chance that the newly scheduled scrub fail to pass the
flip-coin check: the probability of scheduling a new scrub 1/3 every
time we flip the coin. so, instead hardwiring the probability to 1/3,
a new option named "osd_scrub_backoff_ratio" is introduced. so we can
override it when performing test.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13325 from liewegas/wip-fsck

ceph_test_objectstore: tolerate fsck EOPNOTSUPP too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13208 from ceph/wip-dalgaaf-sca-20170131

misc: SCA and Coverity Fixes

Reviewed-by: Sage Weil <sage@redhat.com>

msg: Fix calls to Messenger::create with new parameter

Update all calls to Messenger::create()
take 'ms_public_type' or 'ms_cluster_type' instead of 'ms_type'

issue: 971661

Change-Id: Ib234daf393d1838ef596dde35a6d83c1aeecaf10
Signed-off-by: Sarit Zubakov <saritz@mellanox.com>

Merge pull request #13306 from tchaikov/wip-18464

rocksdb: sync with upstream

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13313 from smithfarm/wip-18856

tests: update SUSE yaml facets in qa/distros/all

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

librbd: Include WorkQueue.h since we use it

We use m_work_queue of type ContextWQ in handle_update function but we
do not include common/WorkQueue.h that defines ContextWQ. This results
in dereference of an incomplete type and causes build error in latest
Fedora rawhide (future 26).

Fixes: http://tracker.ceph.com/issues/18862
Signed-off-by: Boris Ranto <branto@redhat.com>

script/sepia_bt.sh: support xenial

* support xenial
* support "rados" cli debugging
* do not include the ending "'" in the prog name sending to gdb

Signed-off-by: Kefu Chai <kchai@redhat.com>

RadosStriperImpl.cc: fix double free

Some variables have been already free'd by calling
release(), no need to call delete() again on them.

CID 1396200 (#1 of 1): Double free (USE_AFTER_FREE)
 double_free: Calling operator delete frees pointer
 getxattr_completion which has already been freed.
CID 1396206 (#1 of 1): Double free (USE_AFTER_FREE)
 double_free: Calling operator delete frees pointer
 getxattr_completion which has already been freed.
CID 1396214 (#1 of 1): Double free (USE_AFTER_FREE)
 double_free: Calling operator delete frees pointer
 stat_completion which has already been freed.
CID 1396218 (#1 of 1): Double free (USE_AFTER_FREE)
 double_free: Calling operator delete frees pointer
 stat_completion which has already been freed.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/ObjectCacher.cc: fix wrong self assignment

Fix for:

CID 1395468 (#1 of 1): Self assignment (NO_EFFECT)
self_assign: Assigning bh->last_write to itself has no effect.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/Checksummer.h: close resource leak

CID 1395784 (#2-1 of 2): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable state going out of scope
 leaks the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_main.cc: close leak in error case

ID 1398895 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable config going out of scope leaks
 the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rados.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_main.cc: reorder framework detection

Fix for:

CID 1398896: Resource leak (RESOURCE_LEAK)
 overwrite_var: Overwriting fe in fe = new
 RGWLoadGenFrontend(env, config) leaks the storage that fe
 points to.

Do not check again and again for framework if there is
already a match, make use of if/else. This should also
fix the RESOURCE_LEAK warning.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_compression.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_common.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_torrent.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

RDMAConnectedSocketImpl.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/OSDMonitor.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Mantle.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_lc.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_token.h: remove redundant and slow call to c_str()

Do not call c_str() if the parameter is already of type string.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

kv/MemDB.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

cls/rgw/cls_rgw.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #12539 from liewegas/wip-mon-pg-creates

mon: fix force_pg_create pg stuck in creating bug

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12923 from liewegas/wip-df-avail

mon/PGMap: factor mon_osd_full_ratio into MAX AVAIL calc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph_test_objectstore: tolerate fsck EOPNOTSUPP too

44c3ec81e9a638150b7d4aa5afb38e73ed92078f is problematic for filestore etc.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13278 from liewegas/wip-luminous

mon,osd: luminous feature bits, require flags, upgrade gates

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #12829 from ovh/wip-perf-counters-histogram

common, osd, tools: Add histograms to performance counters

Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: whu_liuchang <nfcj000@qq.com>
Reviewed-by: Alexey Sheplyakov <asheplyakov@mirantis.com>

Merge pull request #13296 from liewegas/wip-kill-sortbitwise

osd: remove sortbitwise thrashing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13097 from ceph/wip-osd-after-mon

systemd: Start OSDs after MONs

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13319 from smithfarm/wip-buildpackages-py

tests: drop buildpackages.py

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rocksdb: sync with upstream

and update the CMakeLists.txt accordingly, as upstream changed the
target name to build the static library.

5647d5e reverted the previous change by accident.

Fixes: http://tracker.ceph.com/issues/18464
Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: drop buildpackages.py

The buildpackages suite has been moved to teuthology. This cleans up a file
that was left behind by https://github.com/ceph/ceph/pull/13297

Fixes: http://tracker.ceph.com/issues/18846
Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/suites/rados/upgrade/jewel-x-singleton: upgrade mons first; box thrasher

Signed-off-by: Sage Weil <sage@redhat.com>

doc: v12.0.0 release notes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge remote-tracking branch 'gh/luminous'

Merge pull request #13218 from guits/ceph_detect_init

ceph-detect-init: Add docker detection

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: update SUSE yaml facets in qa/distros/all

Fixes: http://tracker.ceph.com/issues/18856
Signed-off-by: Nathan Cutler <ncutler@suse.com>

msg/async: cleanup code.

Only WITH_EVENTTRACE, it declare local paramter utime_t ltt_recv_stamp.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

12.0.0

mailmap: Yankun Li affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Ronak Jain affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Rishabh Kumar affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Pascal Bach affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: optimistyzy affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: John Lin affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Jamin W. Collins affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>