Merge pull request #17757 from ceph/wip-rgw-ansible-var

qa/tests: Add additional required ceph-ansible vars due to upstream changes

Merge pull request #17910 from smithfarm/wip-21499

tests: point-to-point-x: upgrade client.1 to -x along with cluster nodes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17904 from carltonf/wip-fix-21498

ceph-disk: fix '--runtime' omission for ceph-osd service

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17770 from aclamk/fix-bluestore-fio-hash

bluestore/fio: Fixed problem with all objects having the same hash

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17901 from tchaikov/wip-cmake-gperftools

cmake: update the error message for gperftools bug

qa: point-to-point-x: upgrade client.1 to -x along with cluster nodes

The client.1 rgw in workload_x had not been upgraded to -x.

Fixes: http://tracker.ceph.com/issues/21499
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #17806 from tchaikov/wip-osdc-new-options

osdc/Journaler: use new style options

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17823 from xiexingguo/wip-clone-bytes

osd/osd_type: get_clone_bytes - inline size() for overlapping size

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17794 from hjwsm1989/wip-21425

qa/suites/rest/basic/tasks/rest_test: more whitelisting

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17744 from xiexingguo/wip-blue-eio

os/bluestore: propagate read-EIO to high level callers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17460 from dengquan/master

java/native: fix milliseconds to mtime/atime conversion

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17422 from aclamk/fix-cmake-unsupported-allocator

cmake: in case of bad "ALLOCATOR" selected issue warning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17374 from jcsp/wip-mgr-config-tidy

mgr: use new style config opts + add metadata

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17309 from jcsp/wip-daemonserver-locks

mgr: locking fixes

Reviewed-by: Sage Weil <sage@redhat.com>

osdc/Journaler: use new style options

also use uint64_t for these options, as they are casted to uint64_t when
reading from conf.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17830 from Yan-waller/wip-walle-0920primarypg

osd/PrimaryLogPG: misc cleanups

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-disk: fix '--runtime' omission for ceph-osd service

f425a127b introduces a regression that ceph-disk omits "--runtime" when
enabling ceph-osd@$ID.service units for device-backed OSDs.

Fixes: http://tracker.ceph.com/issues/21498
Signed-off-by: Carl Xiong <cxiong@suse.com>

cmake: update the error message for gperftools bug

it's a follow-up of #17788

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17891 from b-ranto/wip-selinux-lnk-getattr

selinux: Allow getattr on lnk sysfs files

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #17739 from liewegas/wip-filestore-configs

common/options: document filestore and filejournal options

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17740 from liewegas/wip-objecter-configs

common/options: document objecter, filer, and journal options

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17579 from liewegas/wip-kill-snapdirs

osd: kill snapdirs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17895 from ivancich/wip-fix-mclock-coverity

tests: fix uninitialized value found by coverity scan

Reviewed-by: Kefu Chai <kchai@redhat.com>

use ovh instead of vps

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

tests: fix uninitialized value found by coverity scan

Unit test for mclock priority queue had a default constructor, which
didn't initialize a value.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

common/options: document filestore and filejournal options

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: can_create false for cache ops

Some cache ops do writes, but none of them *create* objects, so we
should not set can_create.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: drop odd maintain_ssc hack

This was added two years ago by b61f3e43f1c9d43daa0dedd7cbd0fe5787cbdfbb,
but I can see why it is needed here.  The test case for that
original bug (from a3442cf07841a7269713e30ca270f2f9c356a598) passes
fine.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: drop pre-snapset support for setting up snapmapper

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: drop head_exists corruption

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_rados_api_pool: make LibRadosPools.PoolList tolerate long pool names

There was a baked-in assumption that the first pool was <20 chars.
No more!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: object_info_t: remove legacy_snaps

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: drop last legacy_snaps users (just debug output!)

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: SnapSet: remove head_exists

Signed-off-by: Sage Weil <sage@redhat.com>

osd: drop last head_exists = true calls

No more users!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: find_object_context: drop !head_exists path

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: do_osd_ops: head_exists is always true for LIST_SNAPS

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: SnapSet: drop is_legacy

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: start_flush: drop legacy snapset handling

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: default head_exists = true

This will be helpful as we start removing code that would otherwise
set this to true in some convoluted way.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: finish_promote: drop legacy snapset handling

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: do_copy_get: drop legacy snapset path

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: _delete_oid: drop legacy snapset handling

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: do_osd_ops: drop legacy snapset handling from LIST_SNAPS

This is so much simpler, yay!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: on_local_recover: drop legacy snapset handling

This was there to deal with jewel peers, which are no more.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: _scan_snaps: drop legacy snapset handling

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: find_object_context: drop legacy snapset handling

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: issue_repop: drop obsolete condition on head_obc

The only user is trim_object, which sets obc = clone, head_obc =
head.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: snapset_obc -> head_obc

This is only used for trim_object, now!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: do_op: drop snapset_obc lookup on snapdir

We'll never find it.  This also means we can drop the goofy hack
to clear it for cache tiers.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: trim_object: ctx->snapset_obc -> head_obc

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: trim_object: simplify a bit more for post-snapdir world

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: trim_object: drop legacy SnapSet handling

Signed-off-by: Sage Weil <sage@redhat.com>

selinux: Allow getattr on lnk sysfs files

This showed up during downstream testing for luminous. We are doing
getattr on the sysfs lnk files and the current policy does not allow
this.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #17872 from joscollin/wip-osdmap-signed-unsigned-warning

osd: silence warnings from -Wsign-compare

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: silence warnings from -Wsign-compare

Fixed the warning :

ceph/src/osd/OSDMap.cc: In member function ‘int OSDMap::validate_crush_rules(CrushWrapper*, std::ostream*) const’:
ceph/src/osd/OSDMap.cc:3278:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (pool.get_size() < (int)newcrush->get_rule_mask_min_size(ruleno) ||
         ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ceph/src/osd/OSDMap.cc:3279:18: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  pool.get_size() > (int)newcrush->get_rule_mask_max_size(ruleno)) {
  ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #17833 from mogeb/cache-tiering-doc

doc: update link to placing-different-pools

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #17849 into master

* refs/remotes/upstream/pull/17849/head:
qa: get config only on running MDS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa: get config only on running MDS

Fixes: http://tracker.ceph.com/issues/21466
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17791 into master

* refs/remotes/upstream/pull/17791/head:
client: set client_try_dentry_invalidate to false by default

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17784 into master

* refs/remotes/upstream/pull/17784/head:
vstart: create MDSs before filesystems

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17783 into master

* refs/remotes/upstream/pull/17783/head:
mon/MDSMonitor: fix clog msgs on MDS rank add/rm

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17778 into master

* refs/remotes/upstream/pull/17778/head:
client: move Fh init to ctor

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #17768 into master

* refs/remotes/upstream/pull/17768/head:
ceph: do link/rename semantic checks after srcdn is readable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17807 from xiexingguo/wip-kill-add-interval-usage

osd/PrimaryLogPG: kill add_interval_usage

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17723 from amitkumar50/cov-librados-3

rados: Initialization of alignment

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: update 'ceph osd pool set' example

Signed-off-by: Mohamad Gebai <mgebai@suse.com>

Merge pull request #17747 from tchaikov/wip-qa

qa/standalone: respect $TEMPDIR

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #17828 from jdurgin/wip-21428

osd/PrimaryLogPG: request osdmap update in the right block

Reviewed-by: Sage Weil <sage@redhat.com>

doc: update link to placing-different-pools

Signed-off-by: Mohamad Gebai <mgebai@suse.com>

osd/PrimaryLogPG: drop unused variables

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

osd/PrimaryLogPG: avoid accessing value of map.end()

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

osd/PrimaryLogPG: request osdmap update in the right block

Fixes: http://tracker.ceph.com/issues/21428
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17785 from dzafman/wip-add-repair

test: Fix ceph-objectstore-tool usage check

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/osd_type: get_clone_bytes - inline size() for overlapping size

This is faster and simpler. A similar issue can be found at:
https://github.com/ceph/ceph/pull/17807

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17138 from Liuchang0812/wip-updata-osd-hostname-in-metaupdater

mgr: preventing blank hostname in DaemonState

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17738 from jtlayton/wip-jlayton-20988

common: lockdep fixes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17753 from tchaikov/wip-osd-cleanup

osd: cleanups

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17787 from tchaikov/wip-os-release-wo-version_id

common/util: do not print error if VERSION_ID is missing

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #17788 from tchaikov/wip-cmake-rocksdb-tcmalloc

cmake: identify the possible incompatibility of rocksdb and tcmalloc

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

cmake: error out if rocksdb is incompatible w/ tcmalloc

the commit d406f228 in gperf implements a c11 feature used by a
recent change in rocksdb: 16e03882, which uses aligned_alloc().
and 16e03882 in rocksdb was merged after v5.7 was tagged, while
16e03882 in gperf was merged after v2.6.1 was tagged.

because aligned_alloc() is not implemented by tcmalloc until the
not-yet-released 2.6.2, if we call aligned_alloc() in an application
linked against tcmalloc, what gets called will be the glibc's
aligned_alloc(). but if we free() the memory chunk allocated by
aligned_alloc(), the tcmalloc's implementation kicks in, then
InvalidFree() is called, because the memory chunk being freed was
allocated by tcmalloc. in short, "mixing allocators", quote from
Dan Mick.

in rocksdb, aligned_alloc() is used if _ISOC11_SOURCE is defined, this
makes sense, because aligned_alloc() is a C11 function. we could avoid
using it by not defining _ISOC11_SOURCE. but as long as _GNU_SOURCE is
defined, glibc defines _ISOC11_SOURCE. and libstdc++ requires
_GNU_SOURCE, because it uses a fair amount of GNU extensions.

Fixes: http://tracker.ceph.com/issues/21422
Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tests: skip test that use public bucket landsat-pds from AWS

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

qa/tests: update required ceph-ansbile vars

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

qa/tests: Fix ceph-ansible upstream vars

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #17795 from jdurgin/wip-21428

osd: request new map from PG when needed

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

osd: request new map from PG when needed

The fast dispatch refactor in 3cc48278bf0ee5c9535d04b60a661f988c50063b
eliminated the osdmap subscription in the ms_fast_dispatch path, which
meant ops could reach a PG without having the latest map. In a cluster
with few osdmap updates, where the monitor fails to send a new map to
an osd (it tries one random osd), this can result in indefinitely
blocked requests.

Fix this by adding an OSDService mechanism for scheduling a new osdmap
subscription request.

Fixes: http://tracker.ceph.com/issues/21428
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17607 from liewegas/wip-post-luminous-more-mon

mon: post-luminous cleanup (part 3 of ?)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16913 from liewegas/wip-20944

os/filestore: fix device/partition metadata detection

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #17427 from liewegas/wip-pg-num-limits

mon/OSDMonitor: implement cluster pg limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17760 from liewegas/wip-21410-b

Revert "osd/OSDMap: allow bidirectional swap of pg-upmap-items"

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15369 from rzarzynski/wip-rgw-custom-errmsgs-2

rgw: improve handling of Swift's error messages and limits

Merge pull request #12704 from rzarzynski/wip-rgw-per-storage-policy-stats

rgw: add support for Swift's per storage policy statistics

Merge pull request #17786 from neha-ojha/wip-cbt-teuthology-integration-cntd

qa/suites/rados/perf: add optimized settings

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: rename the configurables for metadata limits to start with rgw_.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common/options: document objecter, filer, and journal options

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: calculate and print Swift's X-Account-Storage-Policy-* headers.

Fixes: http://tracker.ceph.com/issues/17932
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: bucket linking stores also the info about a placement rule.

Fixes: http://tracker.ceph.com/issues/17932
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: convey placement rule in RGWBucketEnt and cls_user_bucket_entry.

Fixes: http://tracker.ceph.com/issues/17932
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: clean-up around and implement the move semantics in RGWBucketEnt.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osd/PrimaryLogPG: kill add_interval_usage

Use inline size() member of interval_set instead, which is faster and simpler.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>