ceph-debug-docker.sh: add ceph-osd-dbg package

So we can debug ceph-osd cores.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17810 from hjwsm1989/wip-21294

qa/ceph_manager: check pg state again before timedout

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17661 from yunfeiguan/master

librados: Fix a potential risk of buffer::list::claim_prepend(list& b…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17790 from joscollin/wip-test-warning-signed-unsigned

test: silence warning from -Wsign-compare

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17928 from xiexingguo/wip-more-interval-set-size

osd/PrimaryLogPG: calc clone_overlap size in a more efficient and concise way

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17528 from liewegas/wip-mgr-localpool

pybind/mgr/localpool: module to automagically create localized pools

Reviewed-by: John Spray <john.spray@redhat.com>

qa/workunits/mgr/test_localpool: simple test for localpool mode

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17831 from badone/wip-require-osd-release-with-no-up-osds

mon: osd feature checks with 0 up osds

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17848 from liewegas/wip-ruleset-errors

mon/OSDMonitor: error out if setting ruleset-* ec profile property

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #17902 from tchaikov/wip-21474

qa/tasks: prolong revive_osd() timeout to 6 min

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17903 from lixiaoy1/add_func_log

os/bluestore: Add missing __func__ in dout

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #17905 from tchaikov/wip-osd_types-cleanup

osd/osd_types: object_info_t: remove unused function

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17927 from liewegas/wip-21511

qa/standalone/scrub/osd-scrub-snaps: adjust test for lack of snapdir objects

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #17919 into master

* refs/remotes/upstream/pull/17919/head:
qa: whitelist cache too large in client-limits

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #17918 into master

* refs/remotes/upstream/pull/17918/head:
qa: whitelist failover log warnings

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #17917 into master

* refs/remotes/upstream/pull/17917/head:
qa: ignore evicted client in client-recovery

Reviewed-by: Zheng Yan <zyan@redhat.com>

mon/OSDMonitor: error out if setting ruleset-* ec profile property

We change ruleset -> crush back in dc7a2aaf7a34b1e6af0c7b79dc44a69974c1da23.
If someone tries to use the old property, error out early, instead of
silently not doing the thing they thought they told us to do.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17850 from batrick/vstart-confg-path

qa: revert "qa: use config_path property instead of literal"

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17881 from joscollin/wip-mgr-signed-unsigned-warning

mgr: silence warning from -Wsign-compare

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PrimaryLogPG: calc clone_overlap size in a more efficient and concise way

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/standalone/scrub/osd-scrub-snaps: adjust test for lack of snapdir objects

The head_exists stuff is totally gone; those test failures go away.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17756 from jcsp/wip-19291

mgr: set explicit thread name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17808 from xiexingguo/wip-eqduot-vs-enospc

osd/PrimaryLogPG: prepare_transaction - fix EDQUOT vs ENOSPC

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17819 from neha-ojha/wip-cbt-teuthology-fix-pg-num

qa/suites/rados/perf: create pool with lower pg_num

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17827 from tchaikov/wip-mono-heartbeat

common: use mono clock for HeartbeatMap

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17844 from liewegas/wip-21259

os/bluestore: debug bluestore cache shutdown

Merge pull request #17845 from liewegas/wip-21408

os/bluestore: ignore 0x2000~2000 extent oddity from luminous upgrade

Reviewed-by: Varada Kari <varada.kari@gmail.com>

Merge PR #17853 into master

* refs/remotes/upstream/pull/17853/head:
mds: check ongoing catter-gather process before capping log

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17851 into master

* refs/remotes/upstream/pull/17851/head:
client: use unique_ptr for locks
client: use unique_ptr for Formatter

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #17843 into master

* refs/remotes/upstream/pull/17843/head:
ceph_volume_client: perform snapshot operations in

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17757 from ceph/wip-rgw-ansible-var

qa/tests: Add additional required ceph-ansible vars due to upstream changes

qa: whitelist cache too large in client-limits

Fixes: http://tracker.ceph.com/issues/21510
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: whitelist failover log warnings

Fixes: http://tracker.ceph.com/issues/21509
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: ignore evicted client in client-recovery

Fixes: http://tracker.ceph.com/issues/21508
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17910 from smithfarm/wip-21499

tests: point-to-point-x: upgrade client.1 to -x along with cluster nodes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17904 from carltonf/wip-fix-21498

ceph-disk: fix '--runtime' omission for ceph-osd service

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17770 from aclamk/fix-bluestore-fio-hash

bluestore/fio: Fixed problem with all objects having the same hash

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17901 from tchaikov/wip-cmake-gperftools

cmake: update the error message for gperftools bug

Add missed __func__ in dout

With __func__ it makes users to know where it prints the logs in source code.

Signed-off-by: Xiaoyan Li xiaoyan.li@intel.com

qa: point-to-point-x: upgrade client.1 to -x along with cluster nodes

The client.1 rgw in workload_x had not been upgraded to -x.

Fixes: http://tracker.ceph.com/issues/21499
Signed-off-by: Nathan Cutler <ncutler@suse.com>

osd/osd_types: object_info_t: remove unused function

legacy_object_locator_to_ps() is not used anymore, so drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17806 from tchaikov/wip-osdc-new-options

osdc/Journaler: use new style options

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17823 from xiexingguo/wip-clone-bytes

osd/osd_type: get_clone_bytes - inline size() for overlapping size

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17794 from hjwsm1989/wip-21425

qa/suites/rest/basic/tasks/rest_test: more whitelisting

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17744 from xiexingguo/wip-blue-eio

os/bluestore: propagate read-EIO to high level callers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17460 from dengquan/master

java/native: fix milliseconds to mtime/atime conversion

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/OSDMonitor: mon osd feature checks with 0 up osds

get_up_osd_features() returns 0 if no osds are up which causes feature
checks to fail.

Fixes: http://tracker.ceph.com/issues/21471
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #17422 from aclamk/fix-cmake-unsupported-allocator

cmake: in case of bad "ALLOCATOR" selected issue warning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17374 from jcsp/wip-mgr-config-tidy

mgr: use new style config opts + add metadata

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17309 from jcsp/wip-daemonserver-locks

mgr: locking fixes

Reviewed-by: Sage Weil <sage@redhat.com>

osdc/Journaler: use new style options

also use uint64_t for these options, as they are casted to uint64_t when
reading from conf.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17830 from Yan-waller/wip-walle-0920primarypg

osd/PrimaryLogPG: misc cleanups

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-disk: fix '--runtime' omission for ceph-osd service

f425a127b introduces a regression that ceph-disk omits "--runtime" when
enabling ceph-osd@$ID.service units for device-backed OSDs.

Fixes: http://tracker.ceph.com/issues/21498
Signed-off-by: Carl Xiong <cxiong@suse.com>

qa/tasks: prolong revive_osd() timeout to 6 min

bluestore_fsck_on_mount and bluestore_fsck_on_mount_deep are enabled by
default. and bluestore is used as the default store backend. it takes
longer to perform the deep fsck with verbose log. so prolong the
revive_osd()'s timeout from 150 sec to 360 sec.

Fixes: http://tracker.ceph.com/issues/21474
Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: update the error message for gperftools bug

it's a follow-up of #17788

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17891 from b-ranto/wip-selinux-lnk-getattr

selinux: Allow getattr on lnk sysfs files

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #17739 from liewegas/wip-filestore-configs

common/options: document filestore and filejournal options

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17740 from liewegas/wip-objecter-configs

common/options: document objecter, filer, and journal options

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17579 from liewegas/wip-kill-snapdirs

osd: kill snapdirs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17895 from ivancich/wip-fix-mclock-coverity

tests: fix uninitialized value found by coverity scan

Reviewed-by: Kefu Chai <kchai@redhat.com>

use ovh instead of vps

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

tests: fix uninitialized value found by coverity scan

Unit test for mclock priority queue had a default constructor, which
didn't initialize a value.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

osd: make shutdown debug conditional (and off by default)

Signed-off-by: Sage Weil <sage@redhat.com>

common/options: document filestore and filejournal options

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: can_create false for cache ops

Some cache ops do writes, but none of them *create* objects, so we
should not set can_create.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: drop odd maintain_ssc hack

This was added two years ago by b61f3e43f1c9d43daa0dedd7cbd0fe5787cbdfbb,
but I can see why it is needed here.  The test case for that
original bug (from a3442cf07841a7269713e30ca270f2f9c356a598) passes
fine.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: drop pre-snapset support for setting up snapmapper

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: drop head_exists corruption

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_rados_api_pool: make LibRadosPools.PoolList tolerate long pool names

There was a baked-in assumption that the first pool was <20 chars.
No more!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: object_info_t: remove legacy_snaps

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: drop last legacy_snaps users (just debug output!)

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: SnapSet: remove head_exists

Signed-off-by: Sage Weil <sage@redhat.com>

osd: drop last head_exists = true calls

No more users!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: find_object_context: drop !head_exists path

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: do_osd_ops: head_exists is always true for LIST_SNAPS

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: SnapSet: drop is_legacy

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: start_flush: drop legacy snapset handling

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: default head_exists = true

This will be helpful as we start removing code that would otherwise
set this to true in some convoluted way.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: finish_promote: drop legacy snapset handling

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: do_copy_get: drop legacy snapset path

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: _delete_oid: drop legacy snapset handling

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: do_osd_ops: drop legacy snapset handling from LIST_SNAPS

This is so much simpler, yay!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: on_local_recover: drop legacy snapset handling

This was there to deal with jewel peers, which are no more.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: _scan_snaps: drop legacy snapset handling

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: find_object_context: drop legacy snapset handling

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: issue_repop: drop obsolete condition on head_obc

The only user is trim_object, which sets obc = clone, head_obc =
head.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: snapset_obc -> head_obc

This is only used for trim_object, now!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: do_op: drop snapset_obc lookup on snapdir

We'll never find it.  This also means we can drop the goofy hack
to clear it for cache tiers.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: trim_object: ctx->snapset_obc -> head_obc

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: trim_object: simplify a bit more for post-snapdir world

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: trim_object: drop legacy SnapSet handling

Signed-off-by: Sage Weil <sage@redhat.com>

common: use mono clock for HeartbeatMap

to avoid problems when admin sets system clock.

see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53901 for the reason
why we cannot use atomic<time_point>.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Xinze Chi <xinze@xsky.com>

selinux: Allow getattr on lnk sysfs files

This showed up during downstream testing for luminous. We are doing
getattr on the sysfs lnk files and the current policy does not allow
this.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #17872 from joscollin/wip-osdmap-signed-unsigned-warning

osd: silence warnings from -Wsign-compare

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

mgr: silence warning from -Wsign-compare

Fixed the warning:

ceph/src/mgr/Mgr.cc: In lambda function:
ceph/src/mgr/Mgr.cc:413:42: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     for (unsigned int osd_id = 0; osd_id < osd_map.get_max_osd(); ++osd_id) {
                                   ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Jos Collin <jcollin@redhat.com>

osd: silence warnings from -Wsign-compare

Fixed the warning :

ceph/src/osd/OSDMap.cc: In member function ‘int OSDMap::validate_crush_rules(CrushWrapper*, std::ostream*) const’:
ceph/src/osd/OSDMap.cc:3278:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (pool.get_size() < (int)newcrush->get_rule_mask_min_size(ruleno) ||
         ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ceph/src/osd/OSDMap.cc:3279:18: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  pool.get_size() > (int)newcrush->get_rule_mask_max_size(ruleno)) {
  ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #17833 from mogeb/cache-tiering-doc

doc: update link to placing-different-pools

Reviewed-by: Kefu Chai <kchai@redhat.com>

mds: check ongoing catter-gather process before capping log

When deactivating mds, MDLog::trim() may start scatter-gather
process on mdsdir inode. Locker::scatter_writebehind() submits
log entry. So mds should make sure there is no scatter-gather
before capping log.

Fixes: http://tracker.ceph.com/issues/21467
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge PR #17849 into master

* refs/remotes/upstream/pull/17849/head:
qa: get config only on running MDS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

client: use unique_ptr for locks

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>