tests: add require_jewel_osds to upgrade/hammer-x/tiering

Without this, the test tends to fail, but sometimes passes (apparently because
the "wait-for-healthy: true" in the last ceph.restart is racy - HEALTH_OK is
received before the MONs notice that all OSDs are running jewel without
require_jewel_osds.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13299 from dachary/wip-18848-jewel

jewel: tests: remove qa/suites/buildpackages

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

buildpackages: remove because it does not belong

It should live in teuthology, not in Ceph. And it is currently broken:
there is no need to keep it around.

Fixes: http://tracker.ceph.com/issues/18846
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 5a43f8d57925da227c95480501ceec10a29395d8)

Conflicts:
qa/tasks/buildpackages/common.sh (remove)
qa/tasks/buildpackages/make-rpm.sh (remove)

Merge pull request #13280 from athanatos/wip-revert-jewel-18581

Revert "Merge pull request #12978 from asheplyakov/jewel-18581"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Revert "Merge pull request #12978 from asheplyakov/jewel-18581"

See: http://tracker.ceph.com/issues/18809

This reverts commit 8e69580c97622abfcbda73f92d9b6b6780be031f, reversing
changes made to c05730ceb3387fb43c35937f0506297a34a44452.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #11872 from dachary/wip-17838-jewel

jewel: leak in RGWFetchAllMetaCR

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13182 from smithfarm/wip-18498-jewel

jewel: rgw: Realm set does not create a new period

Merge pull request #13180 from smithfarm/wip-18547-jewel

jewel: rgw: multisite: segfault after changing value of rgw_data_log_num_shards

Merge pull request #13177 from smithfarm/wip-18676-jewel

jewel: rgw: librgw: objects created from s3 apis are not visible from nfs mount point

Merge pull request #13175 from smithfarm/wip-18684-jewel

jewel: rgw: multisite: sync status reports master is on a different period

Merge pull request #13173 from smithfarm/wip-18710-jewel

jewel: rgw: slave zonegroup cannot enable the bucket versioning

Merge pull request #13171 from smithfarm/wip-18712-jewel

jewel: rgw: radosgw-admin period update reverts deleted zonegroup

Merge pull request #13001 from asheplyakov/jewel-bp/18559

jewel: rgw multisite: fix ref counting of completions

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12997 from asheplyakov/jewel-bp-18569

jewel: radosgw valgrind "invalid read size 4" RGWGetObj

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12764 from SUSE/wip-17342-jewel

jewel: tests: assertion failure in a radosgw-admin related task

Merge pull request #12738 from SUSE/wip-18286-jewel

jewel: rgw: multisite: coroutine deadlock in RGWMetaSyncCR after ECANCELED errors

Merge pull request #12678 from Abhishekvrshny/wip-18348-jewel

jewel: rgw ldap: enforce simple_bind w/LDAPv3 redux

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12622 from wido/jewel-15776

jewel: rgw: log name instead of id for SystemMetaObj on failure

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #12542 from dreamhost/wip-17076-jewel

jewel: rgw: Replacing '+' with "%20" in canonical uri for s3 v4 auth.

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12428 from SUSE/wip-18216-jewel

jewel: rgw-admin: missing command to modify placement targets

Merge pull request #12426 from SUSE/wip-18214-jewel

jewel: add max_part and nbds_max options in rbd nbd map, in order to keep consistent with

http://tracker.ceph.com/issues/17851#note-59

Merge pull request #12419 from SUSE/wip-18217-jewel

jewel: rgw sends omap_getvals with (u64)-1 limit

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12405 from jan--f/wip-18199-jewel

jewel: build/ops: install-deps.sh based on /etc/os-release

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12397 from linuxbox2/jewel-next

jewel: rgw: do not abort when accept a CORS request with short origin

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12316 from dachary/wip-18101-jewel

jewel: Add workaround for upgrade issues for older jewel versions

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12890 from dillaman/wip-18453-jewel

jewel: librbd: new API method to force break a peer's exclusive lock

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12302 from SUSE/wip-18135-jewel

jewel: build/ops: add hostname sanity check to run-{c}make-check.sh

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #12315 from dachary/wip-18102-jewel

jewel: rgw: Unable to commit period zonegroup change

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12314 from dachary/wip-18107-jewel

jewel: multisite: failed assertion in 'radosgw-admin bucket sync status'

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12313 from dachary/wip-18112-jewel

jewel: rgw: multisite requests failing with '400 Bad Request' with civetweb 1.8

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12258 from rzarzynski/wip-rgw-17931-jewel

jewel: rgw: add support for the prefix parameter in account listing of Swift API

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12156 from dachary/wip-17969-jewel

jewel: rgw: multisite upgrade from hammer -> jewel ignores rgw_region_root_pool

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11990 from asheplyakov/jewel-17908

jewel: rgw: for the create_bucket api, if the input creation_time is zero, we …

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11876 from dachary/wip-17839-jewel

jewel: rgw: the value of total_time is wrong in the result of 'radosgw-admin log show' opt

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11868 from dachary/wip-17512-jewel

jewel: multisite: metadata master can get the wrong value for 'oldest_log_period'

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11866 from dachary/wip-17709-jewel

jewel: rgw: multisite: coroutine deadlock assertion on error in FetchAllMetaCR

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11497 from rzarzynski/wip-rgw-slashinfo-jewel

jewel: rgw: add suport for Swift-at-root dependent features of Swift API

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11477 from dachary/wip-17119-jewel

jewel: rgw: multisite: assert(next) failed in RGWMetaSyncCR

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11476 from dachary/wip-17162-jewel

jewel: rgw: multisite doesn't retry RGWFetchAllMetaCR on failed lease

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11470 from dachary/wip-17514-jewel

jewel: rgw：bucket check remove _multipart_ prefix

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10661 from dachary/wip-16871-jewel

jewel: rgw: Have a flavor of bucket deletion in radosgw-admin to bypass garbage collection

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13187 from asheplyakov/jewel-bp-18729

jewel: cli: ceph-disk: convert none str to str before printing it

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13153 from smithfarm/wip-no-firefly-on-centos

jewel: tests: upgrade:hammer-x: install firefly only on Ubuntu 14.04

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13118 from smithfarm/wip-18274-jewel

jewel: Memory leaks in object_list_begin and object_list_end

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13025 from SUSE/wip-18605-jewel

jewel: ceph-disk prepare writes osd log 0 with root owner

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13106 from ceph/wip-cd-dev-option

qa/tasks: jewel backport -  ceph-deploy use of dev option

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #13062 from asheplyakov/jewel-bp-18379

jewel: msg/simple: clear_pipe when wait() is mopping up pipes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12978 from asheplyakov/jewel-18581

jewel: ReplicatedBackend: take read locks for clone sources during recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #12755 from Abhishekvrshny/wip-18284-jewel

jewel: Need CLI ability to add, edit and remove omap values with binary keys

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12677 from SUSE/wip-18104-jewel

jewel: mon: ceph osd down detection behaviour

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12291 from asheplyakov/jewel-18108

jewel: msg/simple/Pipe: error decoding addr

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11998 from Abhishekvrshny/wip-17877-jewel

jewel: FileStore: fiemap cannot be totally retrieved in xfs when the number of extents > 1364

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11991 from asheplyakov/jewel-17909

jewel: osd: limit omap data in push op

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #12045 from Abhishekvrshny/wip-17875-jewel

jewel: rgw: file: remove spurious mount entries for RGW buckets

Merge pull request #13161 from smithfarm/wip-lfn-upgrade-hammer

jewel: tests: add require_jewel_osds before upgrading last hammer node

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #12425 from SUSE/wip-18190-jewel

jewel: rbd-mirror: gmock warnings in bootstrap request unit tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13129 from smithfarm/wip-18558-jewel

jewel: rbd: bench-write will crash if --io-size is 4G

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13155 from smithfarm/wip-18494-jewel

jewel: rbd: [rbd-mirror] sporadic image replayer shut down failure

Merge pull request #13128 from smithfarm/wip-18633-jewel

jewel: rbd: [qa] crash in journal-enabled fsx run

http://tracker.ceph.com/issues/13512

Merge pull request #13104 from idryomov/wip-fio-unmap-devices-jewel

qa/tasks: backport rbd_fio fixes to jewel

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13156 from smithfarm/wip-18455-jewel

jewel: rbd: Attempting to remove an image w/ incompatible features results in partial removal

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13157 from smithfarm/wip-18434-jewel

jewel: rbd: Improve error reporting from rbd feature enable/disable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13168 from trociny/wip-18550-jewel

jewel: librbd: metadata_set API operation should not change global config setting

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12739 from SUSE/wip-18323-jewel

jewel: JournalMetadata flooding with errors when being blacklisted

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12416 from SUSE/wip-18219-jewel

jewel: msg: don't truncate message sequence to 32-bits

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13115 from smithfarm/wip-18404-jewel

jewel: mon: cache tiering: base pool last_force_resend not respected (racing read got wrong version)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13045 from SUSE/wip-18553-jewel

jewel: mon: peon wrongly delete routed pg stats op before receive pg stats ack

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13117 from smithfarm/wip-18280-jewel

jewel: mon: osd flag health message is misleading

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12320 from dachary/wip-18061-jewel

jewel: rgw:fix for deleting objects name beginning and ending with underscores of one bucket using POST method of js sdk.

Merge pull request #11759 from Abhishekvrshny/wip-17783-jewel

jewel: rgw: json encode/decode of RGWBucketInfo missing index_type field

Merge pull request #12153 from dachary/wip-18010-jewel

jewel: Cleanly reject session evict command when in replay

Merge pull request #12324 from dachary/wip-18103-jewel

jewel: truncate can cause unflushed snapshot data lose

Merge pull request #12783 from SUSE/wip-18413-jewel

jewel: cephfs: lookup of /.. in jewel returns -ENOENT

Merge pull request #12921 from xiaoxichen/wip-18520-jewel

Jewel: speed up readdir by skipping unwanted dn

Merge pull request #13119 from smithfarm/wip-18565-jewel

jewel: MDS crashes on missing metadata object

Merge pull request #13120 from smithfarm/wip-18551-jewel

jewel: ceph-fuse crash during snapshot tests

Merge pull request #13123 from smithfarm/wip-18282-jewel

jewel: monitor cannot start because of FAILED assert(info.state == MDSMap::STATE_STANDBY)

Merge pull request #13125 from smithfarm/wip-18195-jewel

jewel: cephfs: fix missing ll_get for ll_walk

Merge pull request #13126 from smithfarm/wip-18192-jewel

jewel: standby-replay daemons can sometimes miss events

tests: add require_jewel_osds before upgrading last hammer node

Note: this commit was inspired by
http://github.com/ceph/ceph-qa-suite/commit/50758a4810794d265c5d36a71d1e16799251a00d

As of 10.2.4, when upgrading a cluster from hammer to jewel, after the last
node is upgraded the MON will put the cluster into HEALTH_WARN and say: "all
OSDs are running jewel or later but the 'require_jewel_osds' osdmap flag is not
set". The release notes say:

    This is a signal for the admin to do "ceph osd set require_jewel_osds" – by
    doing this, the upgrade path is complete and no more pre-Jewel OSDs may be
    added to the cluster.

Fixes: http://tracker.ceph.com/issues/18719
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13049 from SUSE/wip-18433-jewel

jewel: rados bench seq must verify the hostname

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13040 from SUSE/wip-fs-thrash-jewel

jewel: tests: run fs/thrash on xfs instead of btrfs

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13029 from SUSE/wip-18611-jewel

jewel: cephfs: client segfault on ceph_rmdir path /

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #12875 from asheplyakov/jewel-bp-18485

jewel: osd/PG: publish PG stats when backfill-related states change

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12789 from SUSE/wip-18417-jewel

jewel: osd: leveldb corruption leads to Operation not permitted not handled and assert

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12761 from SUSE/wip-18402-jewel

jewel: tests: objecter_requests workunit fails on wip branches

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11947 from SUSE/wip-17884-jewel

jewel: mon: OSDs marked OUT wrongly after monitor failover

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11508 from SUSE/wip-17583-jewel

jewel: utime.h: fix timezone issue in round_to_* funcs.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Revert "use the create option during instantiation"

jewel cephfs still uses old Filesystem initializtion method

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

use dev option instead of dev-commit

Fixes: http://tracker.ceph.com/issues/18736
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #13169 from linuxbox2/jewel-mg-errortrans

jewel:rgw: fix interface compliance of RGWCivetWeb::write_data()

librbd: ensure owner lock is held before purging cache

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados: blacklist_add should wait for latest OSD map

This ensures that future operations against the OSDs force
a OSD map update to notice the blacklisted client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9242a2e4e1a5a9fcea48d8135b1589493fc28242)

librbd: prevent assertion failure when journal IO is blacklisted

Fixes: http://tracker.ceph.com/issues/18429
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c720f6e3704ed7e8cf41dffdb931dbb05d59a003)

librbd: ignore blacklist error when releasing exclusive lock

This ensures the journal and object map are properly closed so that the
image can be properly closed w/o failing any assertions.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 418dcf29cb8c450049047e09a4dad2941af87018)

librbd: fail immediately if the exclusive lock cannot be acquired

Fixes: http://tracker.ceph.com/issues/16988
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 47b89f4d48a0b99876462167389df28893a8d418)

librbd: add new lock_get_owners / lock_break_lock API methods

If the client application supports failover, let the application
force break the current lock and blacklist the owner. This is
required in case the current lock owner is alive from the point-of-view
of librbd but failover was required due to a higher level reason.

Fixes: http://tracker.ceph.com/issues/18327
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9a5a8c75a025143cee6f92f3dbc3a12f2b6a9ad7)

Conflicts:
src/pybind/rbd/rbd.pyx: trivial resolution
src/test/pybind/test_rbd.py: trivial resolution

librbd: separate break lock logic into standalone state machine

The current lockers are now queried before the lock is attempted to
prevent any possible race conditions when one or more clients attempt
to break the lock of a dead client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 23f60fee86d1ff9b261fbb6411746a2a9479cf19)

librbd: separate locker query into standalone state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 03533b912c59d5e433d0f006e1a063e014468ca5)

Conflicts:
src/test/librbd/exclusive_lock/test_mock_AcquireRequest.cc: trivial resolution

librbd/exclusive_lock/AcquireRequest.cc: init lock_type

Fixup for:

CID 1351687 (#1 of 1): Uninitialized scalar variable (UNINIT)
 var_decl: Declaring variable lock_type without initializer
 uninit_use: Using uninitialized value lock_type.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit da9ede90cd257ff605ab3ebfcc1aa995f655f1cc)