rbd-mirror: ensure forced-failover cannot result in sync state

If the local image already exists and the local image isn't
registered in the remote journal, it should be assumed that
it's already in the replaying state so it can verify the
ancestry to detect split-brains.

Fixes: http://tracker.ceph.com/issues/21559
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: forced-promotion should interrupt replay delay to shut down

Fixes: http://tracker.ceph.com/issues/21559
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17589 from myENA/feature/20883-adminapi_sync_stats_master

rgw: admin api - add ability to sync user stats from admin api

Merge pull request #17433 from C2python/check_action

rgw: Return Error if Bucket Policy Contians Undefined Action

Merge pull request #17969 from ceph/rbd-mirror-reboot

systemd: rbd-mirror does not start on reboot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17929 from renhwztetecs/renhw-wip-mgrmon-command_descs_prefix

mon/mgr: sync "mgr_command_descs","osd_metadata" and "mgr_metadata" prefixes to new mons

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17932 from jcsp/wip-21197

mgr: store declared_types in MgrSession

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17933 from jcsp/wip-17737

mgr: fix crashable DaemonStateIndex::get calls

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17944 from thmour/tmour_upmap

osd/OSDMap: upmap should respect the osd reweights

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17880 from amitkumar50/cov-rgw-9

rgw: Error check on return of read_line()

rbd-mirorr: does not start on reboot

The current systemd unit file misses 'PartOf=ceph-rbd-mirror.target',
which results in the unit not starting after reboot.
If you have ceph-rbd-mirror@rbd-mirror.ceph-rbd-mirror0, it won't start
after reboot even if enabled.
Adding 'PartOf=ceph-rbd-mirror.target' will enable
ceph-rbd-mirror.target when ceph-rbd-mirror@rbd-mirror.ceph-rbd-mirror0
gets enabled.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #17961 from tchaikov/wip-doc-local-pool-in-toc

doc/mgr: add "local pool" plugin to toc

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17949 from asomers/nproc_freebsd

do_{cmake,freebsd}: Don't invoke nproc(1) on FreeBSD

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17940 from jcsp/wip-21534

mon: update get_store_prefixes implementations

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/mgr: add "local pool" plugin to toc

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17842 from MahatiC/fix_example

examples: fix link order in librados example Makefile

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17633 from yonghengdexin735/wip-zzz-doc

doc/rbd: add info for rbd group

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17803 from amitkumar50/cov-librbd-5

test: address coverity false positives

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17800 from amitkumar50/cov-misc-8

test/librbd: initialize on_finish,locker,force,snap_id

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

do_{cmake,freebsd}: Don't invoke nproc(1) on FreeBSD

Use sysctl(8) instead.  Also, there's no longer any need for /bin/bash
on FreeBSD.

Signed-off-by: Alan Somers <asomers@gmail.com>

Merge pull request #17813 from liewegas/wip-21410-note

PendingReleaseNotes: note about upmap mapping change in luminous release notes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

PendingReleaseNotes: note about upmap mapping change in luminous release notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17809 from yaozongyou/fix-rgw-file-write-error

rgw_file: fix write error when the write offset overlaps

Merge pull request #17947 from batrick/ceph-osd-core-dbg

ceph-debug-docker.sh: add ceph-osd-dbg package

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-debug-docker.sh: add ceph-osd-dbg package

So we can debug ceph-osd cores.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

osd: upmap should respect osd reweights

Modify OSDMap::calc_pg_upmaps to take the osd reweight into account when
computing the size of each OSD.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
Fixes: http://tracker.ceph.com/issues/21538

test/cli/osdmaptool: test upmap with an out osd

Test upmap with one osd marked out.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>

Merge pull request #17810 from hjwsm1989/wip-21294

qa/ceph_manager: check pg state again before timedout

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17661 from yunfeiguan/master

librados: Fix a potential risk of buffer::list::claim_prepend(list& b…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17790 from joscollin/wip-test-warning-signed-unsigned

test: silence warning from -Wsign-compare

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17928 from xiexingguo/wip-more-interval-set-size

osd/PrimaryLogPG: calc clone_overlap size in a more efficient and concise way

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17528 from liewegas/wip-mgr-localpool

pybind/mgr/localpool: module to automagically create localized pools

Reviewed-by: John Spray <john.spray@redhat.com>

qa/workunits/mgr/test_localpool: simple test for localpool mode

Signed-off-by: Sage Weil <sage@redhat.com>

osdmaptool: add --mark-out option

Add a --mark-down <id> option which sets the reweight for a given
osd id to 0.0.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>

examples: fix link order in librados example Makefile

The library link order in librados example Makefile is incorrect
and as a result 'make' throws an error. This change fixes the order.

Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>

mon/mgr: sync mgr_command_descs table and mgr_metadata table

sync mgr_command_descs table and mgr_metadata table to new mons

Fixes: http://tracker.ceph.com/issues/21527
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mon/osd_metadata: sync osd_metadata table

sync osd_metadata table to new mons when add new mons

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mgr: fix crashable DaemonStateIndex::get calls

This function was recently fixed to return null
on missing entries: handle that properly.

Fixes: http://tracker.ceph.com/issues/17737
Signed-off-by: John Spray <john.spray@redhat.com>

mon: implement MDSMonitor::get_store_prefixes

Fixes: http://tracker.ceph.com/issues/21534
Signed-off-by: John Spray <john.spray@redhat.com>

mon/OSDMonitor: tidy prefix definitions

We should define them in one place to make it easy
when updating get_store_prefixes.

Fixes: http://tracker.ceph.com/issues/21534
Signed-off-by: John Spray <john.spray@redhat.com>

mon/osd_metadata: sync osd_metadata table

sync osd_metadata table to new mons when add new mons

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #17831 from badone/wip-require-osd-release-with-no-up-osds

mon: osd feature checks with 0 up osds

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17848 from liewegas/wip-ruleset-errors

mon/OSDMonitor: error out if setting ruleset-* ec profile property

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #17902 from tchaikov/wip-21474

qa/tasks: prolong revive_osd() timeout to 6 min

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17903 from lixiaoy1/add_func_log

os/bluestore: Add missing __func__ in dout

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #17905 from tchaikov/wip-osd_types-cleanup

osd/osd_types: object_info_t: remove unused function

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17927 from liewegas/wip-21511

qa/standalone/scrub/osd-scrub-snaps: adjust test for lack of snapdir objects

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #17919 into master

* refs/remotes/upstream/pull/17919/head:
qa: whitelist cache too large in client-limits

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #17918 into master

* refs/remotes/upstream/pull/17918/head:
qa: whitelist failover log warnings

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #17917 into master

* refs/remotes/upstream/pull/17917/head:
qa: ignore evicted client in client-recovery

Reviewed-by: Zheng Yan <zyan@redhat.com>

mon/OSDMonitor: error out if setting ruleset-* ec profile property

We change ruleset -> crush back in dc7a2aaf7a34b1e6af0c7b79dc44a69974c1da23.
If someone tries to use the old property, error out early, instead of
silently not doing the thing they thought they told us to do.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw_file: fix write error when the write offset overlaps.

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>

mgr: store declared_types in MgrSession

Because we don't (yet) properly prevent multiple sessions
from daemons reporting the same name (e.g. rgws), storing
it in the DaemonPerfCounters meant that one daemon's report
was referring to another daemon's set of reported types.

This should always have been a property of the session.

The behaviour will still be ugly when multiple daemons
are using the same name (stomping on each other's stats/statsu)
but it shouldn't crash.

Fixes: http://tracker.ceph.com/issues/21197
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #17850 from batrick/vstart-confg-path

qa: revert "qa: use config_path property instead of literal"

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17881 from joscollin/wip-mgr-signed-unsigned-warning

mgr: silence warning from -Wsign-compare

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PrimaryLogPG: calc clone_overlap size in a more efficient and concise way

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/standalone/scrub/osd-scrub-snaps: adjust test for lack of snapdir objects

The head_exists stuff is totally gone; those test failures go away.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17756 from jcsp/wip-19291

mgr: set explicit thread name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17808 from xiexingguo/wip-eqduot-vs-enospc

osd/PrimaryLogPG: prepare_transaction - fix EDQUOT vs ENOSPC

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17819 from neha-ojha/wip-cbt-teuthology-fix-pg-num

qa/suites/rados/perf: create pool with lower pg_num

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17827 from tchaikov/wip-mono-heartbeat

common: use mono clock for HeartbeatMap

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17844 from liewegas/wip-21259

os/bluestore: debug bluestore cache shutdown

Merge pull request #17845 from liewegas/wip-21408

os/bluestore: ignore 0x2000~2000 extent oddity from luminous upgrade

Reviewed-by: Varada Kari <varada.kari@gmail.com>

Merge PR #17853 into master

* refs/remotes/upstream/pull/17853/head:
mds: check ongoing catter-gather process before capping log

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17851 into master

* refs/remotes/upstream/pull/17851/head:
client: use unique_ptr for locks
client: use unique_ptr for Formatter

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #17843 into master

* refs/remotes/upstream/pull/17843/head:
ceph_volume_client: perform snapshot operations in

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17757 from ceph/wip-rgw-ansible-var

qa/tests: Add additional required ceph-ansible vars due to upstream changes

qa: whitelist cache too large in client-limits

Fixes: http://tracker.ceph.com/issues/21510
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: whitelist failover log warnings

Fixes: http://tracker.ceph.com/issues/21509
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: ignore evicted client in client-recovery

Fixes: http://tracker.ceph.com/issues/21508
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17910 from smithfarm/wip-21499

tests: point-to-point-x: upgrade client.1 to -x along with cluster nodes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17904 from carltonf/wip-fix-21498

ceph-disk: fix '--runtime' omission for ceph-osd service

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17770 from aclamk/fix-bluestore-fio-hash

bluestore/fio: Fixed problem with all objects having the same hash

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17901 from tchaikov/wip-cmake-gperftools

cmake: update the error message for gperftools bug

Add missed __func__ in dout

With __func__ it makes users to know where it prints the logs in source code.

Signed-off-by: Xiaoyan Li xiaoyan.li@intel.com

qa: point-to-point-x: upgrade client.1 to -x along with cluster nodes

The client.1 rgw in workload_x had not been upgraded to -x.

Fixes: http://tracker.ceph.com/issues/21499
Signed-off-by: Nathan Cutler <ncutler@suse.com>

osd/osd_types: object_info_t: remove unused function

legacy_object_locator_to_ps() is not used anymore, so drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17806 from tchaikov/wip-osdc-new-options

osdc/Journaler: use new style options

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17823 from xiexingguo/wip-clone-bytes

osd/osd_type: get_clone_bytes - inline size() for overlapping size

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17794 from hjwsm1989/wip-21425

qa/suites/rest/basic/tasks/rest_test: more whitelisting

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17744 from xiexingguo/wip-blue-eio

os/bluestore: propagate read-EIO to high level callers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17460 from dengquan/master

java/native: fix milliseconds to mtime/atime conversion

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/OSDMonitor: mon osd feature checks with 0 up osds

get_up_osd_features() returns 0 if no osds are up which causes feature
checks to fail.

Fixes: http://tracker.ceph.com/issues/21471
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #17422 from aclamk/fix-cmake-unsupported-allocator

cmake: in case of bad "ALLOCATOR" selected issue warning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17374 from jcsp/wip-mgr-config-tidy

mgr: use new style config opts + add metadata

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17309 from jcsp/wip-daemonserver-locks

mgr: locking fixes

Reviewed-by: Sage Weil <sage@redhat.com>

osdc/Journaler: use new style options

also use uint64_t for these options, as they are casted to uint64_t when
reading from conf.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17830 from Yan-waller/wip-walle-0920primarypg

osd/PrimaryLogPG: misc cleanups

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-disk: fix '--runtime' omission for ceph-osd service

f425a127b introduces a regression that ceph-disk omits "--runtime" when
enabling ceph-osd@$ID.service units for device-backed OSDs.

Fixes: http://tracker.ceph.com/issues/21498
Signed-off-by: Carl Xiong <cxiong@suse.com>

qa/tasks: prolong revive_osd() timeout to 6 min

bluestore_fsck_on_mount and bluestore_fsck_on_mount_deep are enabled by
default. and bluestore is used as the default store backend. it takes
longer to perform the deep fsck with verbose log. so prolong the
revive_osd()'s timeout from 150 sec to 360 sec.

Fixes: http://tracker.ceph.com/issues/21474
Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: update the error message for gperftools bug

it's a follow-up of #17788

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17891 from b-ranto/wip-selinux-lnk-getattr

selinux: Allow getattr on lnk sysfs files

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #17739 from liewegas/wip-filestore-configs

common/options: document filestore and filejournal options

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17740 from liewegas/wip-objecter-configs

common/options: document objecter, filer, and journal options

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17579 from liewegas/wip-kill-snapdirs

osd: kill snapdirs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17895 from ivancich/wip-fix-mclock-coverity

tests: fix uninitialized value found by coverity scan

Reviewed-by: Kefu Chai <kchai@redhat.com>

use ovh instead of vps

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

tests: fix uninitialized value found by coverity scan

Unit test for mclock priority queue had a default constructor, which
didn't initialize a value.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

osd: make shutdown debug conditional (and off by default)

Signed-off-by: Sage Weil <sage@redhat.com>

common/options: document filestore and filejournal options

Signed-off-by: Sage Weil <sage@redhat.com>