doc: update mgr module guide for have_mon_connection

Signed-off-by: John Spray <john.spray@redhat.com>

mgr/dashboard: display warning if mgr load fails

Currently user would just see dead/stale data
and not realise anything was wrong unless they
looked at the JS console.

Signed-off-by: John Spray <john.spray@redhat.com>

mgr/dashboard: warn user if mon connection lost

So that if the mon cluster is down, we don't endlessly show
the same stale data.

Also remove the unused health elements that used to be displayed
in the top bar.

Signed-off-by: John Spray <john.spray@redhat.com>

mgr: expose have_mon_connection to python modules

So that they can warn their end users if the data
we have is probably stale.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #18809 from tchaikov/wip-21904

mgr/zabbix: ignore osd with 0 kb capacity

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Wido den Hollander <wido@42on.com>

Merge pull request #18793 from idryomov/wip-qa-cram-path

qa/tasks/cram: include /usr/sbin in the PATH for all commands

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18556 from joke-lee/validate_cors_rule_header_in_cors_option_request

rgw: add cors header rule check in cors option request

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18795 from idryomov/wip-krbd-enable-generic-050-448

qa/suites/krbd: enable generic/050 and generic/448

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18706 from cbodley/wip-rgw-policy-emplace

rgw: simplify use of map::emplace in iam

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #18770 from tchaikov/wip-doc-log-file

doc: fix desc of option "mon cluster log file"

Reviewed-by: John Spray <john.spray@redhat.com>

doc: fix desc of option "mon cluster log file"

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/zabbix: ignore osd with 0 kb capacity

Fixes: http://tracker.ceph.com/issues/21904
Signed-off-by: Ilja Slepnev <islepnev@gmail.com>

Merge PR #18807 into master

* refs/pull/18807/head:
ptl-tool: put common use-cases up front in README
ptl-tool: only create tag if new branch
ptl-tool: hint how to skip contributor

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ptl-tool: put common use-cases up front in README

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

ptl-tool: only create tag if new branch

It doesn't make sense to create tags for branches that already exist.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

ptl-tool: hint how to skip contributor

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18709 from ivancich/wip-bktidx-race-bugfix

rgw: fix BZ 1500904, stale bucket index entry remains after obj delete

qa/suites/krbd: enable generic/050 and generic/448

generic/448 has been failing on ext4 and is now fixed upstream.
The fix for generic/050 is in testing, scheduled for 4.15-rc1.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #18476 from majianpeng/objecter-misc-cleanup

osdc/Objecter: misc cleanups

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18768 from wumingqiao/wip-WaitActingChange

PG: fix name of WaitActingChange

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18789 from jcsp/wip-doc-config-set

doc: replace injectargs usage with "config set"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18769 from yaozongyou/wip-dashboard-format-label

mgr/dashboard: format tooltip's label as user friendly string

Reviewed-by: John Spray <john.spray@redhat.com>

doc: replace injectargs usage with "config set"

Cleaner and easier.  Also implicitly documents
the config set command, which hadn't been explicitly
called out in the docs before.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge PR #18689 into master

* refs/pull/18689/head:
cephfs: Do not check auth gid when not specified

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18386 into master

* refs/pull/18386/head:
ptl-tool: allow setting project/repo for GH req

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18658 from linuxbox2/wip-s3-post-20201

RGW: update S3 POST policy handling of Content-Type

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

qa/tasks/cram: include /usr/sbin in the PATH for all commands

/usr/sbin is not in non-login PATH on centos.  We already do this for
workunits, see commit 0e53f5f38b0c ("workunit: include /usr/sbin in the
PATH for all commands").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard: format chart's tooltip's label as friendly string

The default tooltip's label is a long number string, so format
the label as a user friendly string.

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>

Merge pull request #16238 from trociny/wip-copy-deep

librbd: generalized deep copy function

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

PG: fix name of WaitActingChange

the parent state of WaitActingChange is Primary, not Peering.

Signed-off-by: wumingqiao <wumingqiao@inspur.com>

qa/suites/rbd: enable fsx deep_copy test case

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

test/librbd: tweak fsx to optionally use deep-copy instead of clone

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

librbd: add deep_copy function to API

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

rbd-mirror: use generalized deep copy for image sync

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: generalized deep copy function

(based on rbd-mirror image sync)

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: export ImageOptions print operator

(so it could be used for logging image options)

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

test/librbd: set rand seed for all tests

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #18635 from majianpeng/bluestore-omap-flush

os/bluestore: remove unused parameters.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: remove unused paramerter.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #18541 from trociny/wip-mstart-bashizm

mstart.sh: remove bashizm in /bin/sh script

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18729 from iliul/fix-vstart-typo

vstart.sh: fix a typo

Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #18693 from XinzeChi/wip-dup-async-read-stat

osd/PrimaryLogPG: fix dup stat for async read

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-By: Kefu Chai <kchai@redhat.com>

vstart.sh: fix a spelling mistake

Signed-off-by: iliul <liul124@chinaunicom.cn>

Merge pull request #18724 from iliul/add-missing-blank-line

doc: add missing blank line

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18708 from dillaman/wip-22036

librbd: added preprocessor macro for detecting compare-and-write support

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18704 from dillaman/wip-rbd-iscsi-win

doc/rbd: tweaks to the Windows iSCSI initiator directions

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18712 from liu-chunmei/rm_redundant_dpdksource

dpdk: remove redundant dpdk submodule

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18711 from adamemerson/wip-system-includes

cmake: System Includes to silence warnings from submodules and libraries!

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18701 from tchaikov/wip-denc

denc: support enum with underlying type

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #18597 from tchaikov/wip-cmake-cleanup

cmake: cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>

doc: Add missing blank line

Signed-off-by: iliul <liul124@chinaunicom.cn>

build: Merge wip-system-includes from upstream

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Squashed 'src/dmclock/' changes from 1bbd42b8fe..d1ff03b667

d1ff03b667 Merge pull request #44 from adamemerson/wip-system-includes
4cc4b949ca build: Mark dependency includes as SYSTEM in dmclock
05096c1756 Merge pull request #43 from TaewoongKim/anticipation_conf
f356c45461 Add missing anticipation_timeout argument for PullPriorityQueue constructor
9896448ec5 Merge pull request #42 from tchaikov/wip-cmake
979899ef86 add travis CI on gnu/linux
8a3dabdbee cmake: the built archives are located in ${binary_dir}
ee15ced3e9 cmake: check for include in /usr/include also

git-subtree-dir: src/dmclock
git-subtree-split: d1ff03b667d9551478b2803ea533fc356ca441a9

build: Mark dependency includes as SYSTEM

It is not really our business to debug python, boost, or our other
dependencies. Mark them as system includes.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #18672 from jcsp/wip-21999

mgr: emit cluster log message on serve() exception

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18691 from liewegas/wip-fix-balancer

mgr/balancer: fix upmap; default balancer module enabled

rgw: add cors header rule check in cors option request

fix http://tracker.ceph.com/issues/22002

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>

Merge PR #18538 into master

* refs/pull/18538/head:
mds: clean up clog damage message when repairing
mds: flush after scrub repairs
mds: don't report repaired backtraces in damagetable
mds: cleaner scrub complete log msg

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18555 into master

* refs/pull/18555/head:
mds: fix inode count check in MDCache::check_memory_usage()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18612 into master

* refs/pull/18612/head:
mds: fix race in PurgeQueue::wait_for_recovery()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18624 into master

* refs/pull/18624/head:
mds: trim 'N' log segments according to how many log segments are there

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18646 into master

* refs/pull/18646/head:
mds: fix MDS_FEATURE_INCOMPAT_FILE_LAYOUT_V2 definition
mds: remove useless incompat feature definination in FSMap.h

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18654 into master

* refs/pull/18654/head:
ceph.in: pass RADOS inst to LibCephFS

Reviewed-by: Kefu Chai <kchai@redhat.com>
Tested-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18686 from xiexingguo/wip-bf-retry-interval

osd/PG: use osd_backfill_retry_interval for schedule_backfill_retry()

Reviewed-by: Sage Weil <sage@redhat.com>

remove redundant dpdk source folder, this one is not maintained since last Nov.2016,
the other one is located in src/spdk/dpdk, which is alwasy maintained and used by spdk.
so remove this src/dpdk folder, we only need maintain one copy.

Signed-off-by: chunmei <chunmei.liu@intel.com>

rgw: fix BZ 1500904, Stale bucket index entry remains after object deletion

We have a race condition:

 1. RGW client #1: requests an object be deleted.
 2. RGW client #1: sends a prepare op to bucket index OSD #1.
 3. OSD #1:        prepares the op, adding pending ops to the bucket dir entry
 4. RGW client #2: sends a list bucket to OSD #1
 5. RGW client #2: sees that there are pending operations on bucket
                   dir entry, and calls check_disk_state
 6. RGW client #2: check_disk_state sees that the object still exists, so it
                   sends CEPH_RGW_UPDATE to bucket index OSD (#1)
 7. RGW client #1: sends a delete object to object OSD (#2)
 8. OSD #2:        deletes the object
 9. RGW client #2: sends a complete op to bucket index OSD (#1)
10. OSD #1:        completes the op
11. OSD #1:        receives the CEPH_RGW_UPDATE and updates the bucket index
                   entry, thereby **RECREATING** it

Solution implemented:

At step #5 the object's dir entry exists. If we get to beginning of
step #11 and the object's dir entry no longer exists, we know that the
dir entry was just actively being modified, and ignore the
CEPH_RGW_UPDATE operation, thereby NOT recreating it.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

cephfs: Do not check auth gid when not specified

For auth caps that omit the gid, do not check for a gid match.

Fixes: http://tracker.ceph.com/issues/22009
Signed-off-by: Douglas Fuller <dfuller@redhat.com>

librbd: added preprocessor macro for detecting compare-and-write support

Fixes: http://tracker.ceph.com/issues/22036
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17923 from cbodley/wip-rgw-beast-update

rgw: pull up beast submodule and update frontend

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

doc/rbd: tweaks to the Windows iSCSI initiator directions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18522 from C2python/cleanup_to_string

rgw: Delete to_string functions. stringify defined in include/stringify.h can provide the same feature.

Reviewed-by: Adam Emerson <aemerson@redhat.com>

rgw: simplify use of map::emplace in iam

the piecewise_construct overloads of map::emplace() are only needed when
there's ambiguity around which arguments go to which constructor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18536 from ZVampirEM77/wip-fix-loadgen

rgw: loadgen fix generate random object name rgw crash issue

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18526 from tchaikov/wip-17454

mgr: In plugins 'module' classes need not to be called "Module" anymore.

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18678 from liu-chunmei/fix_dpdk_coremask

msg/async: fix ms_dpdk_coremask and ms_dpdk_coremask conflict

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18670 from liewegas/wip-min-in-4

qa/tasks/thrashosds: set min_in default to 4

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18671 from ifed01/wip-ifed-bluefs-empty-reclaiming-fix

os/bluestore: do not release empty bluefs_extents_reclaiming.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #18653 from dillaman/wip-21391

mgr/dashboard: added iSCSI IOPS/throughput metrics

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

denc: support enum with underlying type

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake,deb: remove the last trace of atomic_ops

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: fix indent

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: remove unused definitions

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: do not link ${XIO_LIBRARY} if not necessary

Signed-off-by: Kefu Chai <kchai@redhat.com>

ms_dpdk_coremask and ms_async_op_threads has some relationship,
dpdk library will create threasds as many as bit 1 number in
ms_dpdk_coremask and the thredas num should be equal ms_async_op_threads+1,
by ceph default confiuration,ms_async_op_threads =3,ms_dpdk_coremask =1,
so dpdk library can't create enough threads for dpdk workers. the sytem
will stuck at NetworkStack::start().so set ms_dpdk_coremask default 0xF.
and add assert to comapre the two configurations.

Signed-off-by: chunmei <chunmei.liu@intel.com>

Merge pull request #18695 from cooboos/wip-doc-wrongspell

doc/release-notes: fix typo 'psd' to 'osd'

Reviewed-By: Kefu Chai <kchai@redhat.com>

doc: correct wrong spell 'psd' to 'osd'

Signed-off-by: wangsongbo wangsongbo@cloudin.cn

osd/PrimaryLogPG: fix dup stat for async read

The async read would call do_osd_ops twice.

Signed-off-by: Xinze Chi <xinze@xsky.com>

pybind/mgr/mgr_module: fix calc_pg_upmaps

This was fixed in the object cleanup.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/balancer: enable module by default

It will still be "off".

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: loadgen fix rgw crash issue

When generating random bucket name and object name during
loadgen processing, RGW will crash. The reason is calling
"gen_rand_alphanumeric" with "cct = NULL".

Fixes: http://tracker.ceph.com/issues/22006
Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

Merge pull request #13992 from dreamhost/wip-19264

rgw: fix Swift container naming rules.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #18606 from adamemerson/wip-op-perm-refactor

rgw: Small refactor and two bug fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #14955 from aclamk/fix-civetweb-cryptopp

radosgw: Make compilation with CryptoPP possible.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18664 from oritwas/wip-rgw-21996

rgw: init oldest period after setting run_sync_thread
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18680 from tchaikov/wip-doc-mgr

doc/mgr: document facilities methods using `automethod` directive

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18685 from jcsp/wip-mgr-load-module

qa: fix mgr _load_module helper

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PG: use osd_backfill_retry_interval for schedule_backfill_retry()

These two options currently share the same value and I have no particular
preference, but using osd_backfill_retry_interval for the specific scenario
gives us more flexible choices.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

qa: fix mgr _load_module helper

I inadvertently broke this with the latest change
to the module ls output.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #18656 from ceph/ceph-volume-bluestore-tests

ceph-volume: adds functional testing for bluestore

Merge pull request #18399 from jcsp/wip-20950

mon: don't blow away bootstrap-mgr on upgrades

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Sage Weil <sage@redhat.com>