doc: Fixes parameter name in rbd configuration on openstack havana/icehouse

The documentation for using rbd together with openstack havana/icehouse
states that the parameter libvirt_disk_cachemodes should be added to
the nova.conf file. However, this is the only parameter that has no
legacy name with a 'libvirt_' prefix. (See
https://github.com/openstack/nova/blob/icehouse-eol/nova/virt/libvirt/driver.py#L252
for the configuration option)
Thus the configured disk_cachemodes were not applied, defaulting to
no caching.

Fixes: #17978
Signed-off-by: Michael Eischer <michael.eischer@fau.de>

Merge pull request #13017 from chuanhong-wang/dev_add_perfcount_for_backoff_throttle

osd: adding PerfCounters for backoff throttle

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #13274 from ifed01/wip-bluestore-perf-counters

os/bluestore: add more perf_counters to BlueStore

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13321 from liewegas/wip-kill-sortbitwise-harder

osd: kill sortbitwise

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13303 from Jing-Scott/fix-swift-cannot-disable-object-versioning

rgw: fix swift cannot disable object versioning

@Jing-Scott  updated, addressing @rzarzynski's change request

Merge pull request #13393 from ddiss/fix_rgw_fcgi_include

rgw: set FCGI_INCLUDE_DIR for cephd_rgw_base

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13382 from xiexingguo/wip-counter-name

os/bluestore: fix wal-queue bytes-counter to keep pace with others

Reviewed-by: Sage Weil <sage@redhat.com>

rgw: set FCGI_INCLUDE_DIR for cephd_rgw_base

cephd_rgw_base build currently fails with fastcgi enabled:
--
In file included from /home/david/ceph/src/rgw/rgw_request.h:13:0,
from /home/david/ceph/src/rgw/rgw_main.cc:53:
/home/david/ceph/src/rgw/rgw_fcgi.h:8:21: fatal error: fcgiapp.h:
No such file or directory
^
--

This is despite the fact that fastcgi was detected and located at
configure time:
build/CMakeCache.txt:FCGI_INCLUDE_DIR:PATH=/usr/include/fastcgi

Fix this by ensuring that the cephd_rgw_base build target correctly uses
FCGI_INCLUDE_DIR.

Fixes: http://tracker.ceph.com/issues/18918
Signed-off-by: David Disseldorp <ddiss@suse.de>

Merge pull request #13392 from Adirl/zero_inline

msg/async/rdma: zero wqe inline

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: zero wqe inline

we dont use wqe inline

issue: none

Change-Id: Ib187eec3bc79f8caf040fbbfbf0a915186cd58ba
Signed-off-by: Adir Lev <adirl@mellanox.com>

rgw: fix swift cannot disable object versioning

we should be able to disable object verioning by removing its X-Versions-Location
metadata header by sending an empty key value. this description can be found at
No.8 in http://docs.openstack.org/user-guide/cli-swift-set-object-versions.html.

Fixes: http://tracker.ceph.com/issues/18852
Signed-off-by: Jing Wenjun <jingwenjun@cmss.chinamobile.com>

Merge pull request #13380 from tchaikov/wip-sepia-bt

script/sepia_bt.sh: no need to pass version and sha1 anymore

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13368 from smithfarm/wip-doc-dev

doc: instructions for provisioning OpenStack VMs ad hoc

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13367 from tchaikov/wip-qa-jewel-x-singleton

test: Thrasher: update pgp_num of all expanded pools if not yet

Reviewed-by: Sage Weil <sage@redhat.com>

script/sepia_bt.sh: no need to pass version and sha1 anymore

* try to figure out the working directory from the core dump's path
* add -C directory option, so user is able to specify the directory
  for the downloaded packages, and extracted files.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: fix wal-queue bytes-counter to keep pace with others

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

test: Thrasher: restore changed options after done with thrash

Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: Thrasher: extract _set_config() method

Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: CephManager: add get_config() method

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: Thrasher: update pgp_num of all expanded pools if not yet

otherwise wait_until_healthy will fail after timeout as seeing warning
like:

HEALTH_WARN pool cephfs_data pg_num 182 > pgp_num 172

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13378 from liewegas/wip-luminous

qa/rados/upgrade/jewel-x-singleton: run luminous.yaml at the end

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #13365 from xiexingguo/wip-backoff-refleak

osdc/Objecter: fix possible OSDSession leak on wrong connection

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13363 from jdurgin/wip-req-luminous-error

osd: clarify REQUIRE_LUMINOUS error message

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/rados/upgrade/jewel-x-singleton: run luminous.yaml at the end

renumber too

Signed-off-by: Sage Weil <sage@redhat.com>

doc: instructions for provisioning OpenStack VMs ad hoc

Signed-off-by: Nathan Cutler <ncutler@suse.com>

osd/PrimaryLogPG: unconditionally set PGNLS next to hobject

Drop nibblewise workaround.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: drop get_sort_bitwise() accessors, flag

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: always bitwise

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: drop bitwise arg to MissingLoc::rebuild()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: make set_last_backfill() assume bitwise

Signed-off-by: Sage Weil <sage@redhat.com>

osd: BackfillInterval is always bitwise

Signed-off-by: Sage Weil <sage@redhat.com>

osd: fix add_source_info

Signed-off-by: Sage Weil <sage@redhat.com>

osd: fix write_blocked_by_scrub

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: drop bitwise from ScrubMap

Signed-off-by: Sage Weil <sage@redhat.com>

common/hobject: eliminate wonky compartors

Signed-off-by: Sage Weil <sage@redhat.com>

common/hobject: remove cmp_* comparators; add normal operators

Fix up callers.

Signed-off-by: Sage Weil <sage@redhat.com>

os: remove nibblewise sort order support

Signed-off-by: Sage Weil <sage@redhat.com>

osd: shutdown if sortbitwise flag is ever cleared

Note we do this only in activate_map so that we can (while booting)
skip past maps that don't have it set.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13369 from idryomov/wip-doc-kernel-vers

doc: kernel client os-recommendations update

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13229 from trociny/wip-18335

rbd-nbd: check /sys/block/nbdX/size to ensure kernel mapped correctly

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13292 from tchaikov/wip-script-sepia-bt-support-xenial

script/sepia_bt.sh: support xenial

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

osdc/Objecter: fix possible OSDSession leak on wrong connection

This is introduced by the newly added backoff logic.
Not sure if it will really happen, but just in case.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12342 from liewegas/wip-backoff

osd,osdc: pg and osd-based backoff

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/dev/rados-client-protocol: describe some of rados protocol, including backoffs

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/thrash: vary backoff settings

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: backoffs on individual objects

Do these midway down do_op.  Reorder the scrub waitlist after the
degraded and unreadable waitlists.

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: respect backoffs

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: PG-wide backoffs

Issue at top of do_request.  Release on activation or peering
interval change.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: dispatch OSDBackoff messages into PG workqueue

Signed-off-by: Sage Weil <sage@redhat.com>

osd: split backoffs on PG split

Signed-off-by: Sage Weil <sage@redhat.com>

osd: add pg helpers for managing backoffs

Link Backoff into Session, PG.  Tear them down on session
reset.

Signed-off-by: Sage Weil <sage@redhat.com>

include/ceph_features: RADOS_BACKOFF feature bit

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

common/hobject: use hash field for hash<>

No reason to waste CPU recalculating a hash value!

Signed-off-by: Sage Weil <sage@redhat.com>

osd: add MOSDBackoff message type

Signed-off-by: Sage Weil <sage@redhat.com>

osd: move Session class out of OSD

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: attach OSDSession to Connection

This lets us avoid an rbtree lookup.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: clarify REQUIRE_LUMINOUS error message

This should be visible at the default log level, so users have some
hope of figuring out the problem.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

doc: ref cephfs docs from os-recommendations

...instead of having potentially conflicting kernel
version information.

Signed-off-by: John Spray <john.spray@redhat.com>

doc: update krbd kernel recommendations

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #13249 from liewegas/wip-mon-smooth

mon: smooth io/recovery stats over longer period

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13264 from tchaikov/wip-print-failed-test-name

qa/workunits/rados/test.sh: print test name when it fails

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13362 from adamemerson/wip-oh-yeah

test: Fix mismatched sign comparison in histogram test

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13344 from gregsfortytwo/wip-osd-discussion-docs

Wip osd discussion docs

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #13342 from athanatos/wip-17831-18583-18809

osd/: don't leak context for Blessed*Context or RecoveryQueueAsync

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

test: Fix mismatched sign comparison in histogram test

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #13137 from jcsp/wip-18661

qa: fix race in Mount.open_background

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13099 from jcsp/wip-18663

qa/tasks: force umount during kclient teardown

Merge pull request #13281 from theanalyst/release/v12.0.0

doc: v12.0.0 release notes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13302 from YankunLi/lyk_clean_code

rgw: clean redundant code

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13304 from majianpeng/async-tracing

msg/async: cleanup code.

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13329 from Adirl/fix_febe

msg: Fix calls to Messenger::create with new parameter

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13351 from tchaikov/wip-init-ceph-newline

init-ceph: print trailing n in "status" output

Reviewed-by: Nathan Cutler <nculter@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13356 from rubenk/rgw-docs-fix-typo

doc: Fix typo and grammar in RGW config reference

Reviewed-by: Sage Weil <sage@redhat.com>

qa/workunits/rados/test.sh: print test name when it fails

we have

2017-02-04T16:15:46.090 INFO:tasks.workunit.client.0.mira032.stdout:error in 22088
2017-02-04T16:15:46.092 INFO:tasks.workunit.client.0.mira032.stderr:bash: line 1: 22092 Alarm clock             ceph_test_rados_api_aio 2>&1
2017-02-04T16:15:46.096 INFO:tasks.workunit.client.0.mira032.stderr:     22093 Done                    | tee ceph_test_rados_api_aio.log
2017-02-04T16:15:46.099 INFO:tasks.workunit.client.0.mira032.stderr:     22094 Done                    | sed "s/^/                  api_aio: /"
2017-02-04T16:15:46.102 INFO:tasks.workunit.client.0.mira032.stderr:+

if a unittest in rados/test.sh fails in teuthology.log, but it would
be desirable to have the failed test name in the line of "error in
22088".

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13258 from trociny/wip-18819

common: possible lockdep false alarm for ThreadPool lock

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13287 from guihecheng/rbd-fix

librbd: fix segfault on EOPNOTSUPP returned while fetching snapshot timestamp

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: Fix typo and grammar in RGW config reference

Signed-off-by: Ruben Kerkhof <ruben@rubenkerkhof.com>

Merge pull request #13322 from ceph/wip-include-workqueue

librbd: Include WorkQueue.h since we use it

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

init-ceph: print trailing n in "status" output

the version command is handled by HelpHook, which does not print newline
at the end of the returned JSON. so the output of "status" subcommand
looks like:

=== osd.1 ===
osd.1: running {"version":"Development"}=== osd.2 ===
osd.2: not running.

with this change, a newline is added to the end of version.

also use the "daemon" subcommand to talk to the asok, let the "ceph" cli
to figure out the asok path.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13338 from kshtsk/add-bc-to-deps

ceph.spec.in, debian/control: Add bc to build dependencies

Reviewed-by: Dan Mick <dan.mick@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13334 from tchaikov/wip-osd-scrub-repair-more-patient

tests: osd-scrub-repair.sh disable scrub backoff in test

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph.spec.in, debian/control: Add bc to build dependencies

The bc is missing for ceph-helpers.sh

Fixes: http://tracker.ceph.com/issues/18876
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>

OSD: update OSD backfill discussion comments

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #13309 from dachary/wip-mailmap

mailmap updates

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

tests: osd-scrub-repair.sh disable scrub backoff in test

there is good chance that the newly scheduled scrub fail to pass the
flip-coin check: the probability of scheduling a new scrub 1/3 every
time we flip the coin. so, instead hardwiring the probability to 1/3,
a new option named "osd_scrub_backoff_ratio" is introduced. so we can
override it when performing test.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13325 from liewegas/wip-fsck

ceph_test_objectstore: tolerate fsck EOPNOTSUPP too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13208 from ceph/wip-dalgaaf-sca-20170131

misc: SCA and Coverity Fixes

Reviewed-by: Sage Weil <sage@redhat.com>

msg: Fix calls to Messenger::create with new parameter

Update all calls to Messenger::create()
take 'ms_public_type' or 'ms_cluster_type' instead of 'ms_type'

issue: 971661

Change-Id: Ib234daf393d1838ef596dde35a6d83c1aeecaf10
Signed-off-by: Sarit Zubakov <saritz@mellanox.com>

Merge pull request #13306 from tchaikov/wip-18464

rocksdb: sync with upstream

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13313 from smithfarm/wip-18856

tests: update SUSE yaml facets in qa/distros/all

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

librbd: Include WorkQueue.h since we use it

We use m_work_queue of type ContextWQ in handle_update function but we
do not include common/WorkQueue.h that defines ContextWQ. This results
in dereference of an incomplete type and causes build error in latest
Fedora rawhide (future 26).

Fixes: http://tracker.ceph.com/issues/18862
Signed-off-by: Boris Ranto <branto@redhat.com>

script/sepia_bt.sh: support xenial

* support xenial
* support "rados" cli debugging
* do not include the ending "'" in the prog name sending to gdb

Signed-off-by: Kefu Chai <kchai@redhat.com>

RadosStriperImpl.cc: fix double free

Some variables have been already free'd by calling
release(), no need to call delete() again on them.

CID 1396200 (#1 of 1): Double free (USE_AFTER_FREE)
 double_free: Calling operator delete frees pointer
 getxattr_completion which has already been freed.
CID 1396206 (#1 of 1): Double free (USE_AFTER_FREE)
 double_free: Calling operator delete frees pointer
 getxattr_completion which has already been freed.
CID 1396214 (#1 of 1): Double free (USE_AFTER_FREE)
 double_free: Calling operator delete frees pointer
 stat_completion which has already been freed.
CID 1396218 (#1 of 1): Double free (USE_AFTER_FREE)
 double_free: Calling operator delete frees pointer
 stat_completion which has already been freed.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/ObjectCacher.cc: fix wrong self assignment

Fix for:

CID 1395468 (#1 of 1): Self assignment (NO_EFFECT)
self_assign: Assigning bh->last_write to itself has no effect.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/Checksummer.h: close resource leak

CID 1395784 (#2-1 of 2): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable state going out of scope
 leaks the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_main.cc: close leak in error case

ID 1398895 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable config going out of scope leaks
 the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rados.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_main.cc: reorder framework detection

Fix for:

CID 1398896: Resource leak (RESOURCE_LEAK)
 overwrite_var: Overwriting fe in fe = new
 RGWLoadGenFrontend(env, config) leaks the storage that fe
 points to.

Do not check again and again for framework if there is
already a match, make use of if/else. This should also
fix the RESOURCE_LEAK warning.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_compression.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>