msg/simple/Pipe: manual backport of fix in PR#14795

Manual backport of errno fixup from PR#14795
(6f1037e22c2a304795895498cdc955e0ef80f8e3), as noted by
https://github.com/ceph/ceph/pull/13450#discussion_r114696885.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

msg/simple/Pipe: support IPv6 QoS.

Extend DSCP marking for heartbeat packets to IPv6, as commit
9b9a682fe035c985e416ee1c112fa58f9045a27c only implemented
support for IPv4.

Conflicts: Cherry-picked 91a29bc490fdfbbef0875fa620c7ba1a1a6492ae from master to avoid conflict.
Backport: jewel, luminious
Fixes: http://tracker.ceph.com/issues/18887
Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>
(cherry picked from commit 2d6021fbf7a728f73c2998be17e9224f14b83a30)

msg/simple: cleanups

should save the `errno` which may be changed by `ldout` and/or `<<` operator

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
(cherry picked from commit 91a29bc490fdfbbef0875fa620c7ba1a1a6492ae)
Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #12380 from dachary/wip-18183-jewel

jewel: cephfs metadata pool: deep-scrub error omap_digest != best guess omap_digest

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13130 from rjfd/wip-18608-jewel

jewel: librbd: allow to open an image without opening parent image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13233 from smithfarm/wip-18556-jewel

jewel: rbd: Potential race when removing two-way mirroring image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13113 from jcsp/wip-18679

jewel: mds: finish clientreplay requests before requesting active state

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13139 from jcsp/wip-18100

jewel: mds: fix MDSMap upgrade decoding

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #12490 from linuxbox2/jewel-rgw-header

jewel: rgw: add option to log custom HTTP headers (rgw_log_http_headers)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #12079 from rzarzynski/wip-rgw-17961

jewel: rgw: TempURL properly handles accounts created with the implicit tenant

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #12044 from Abhishekvrshny/wip-17886-jewel

jewel: rgw: multisite: ECANCELED & 500 error on bucket delete

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #13422 from smithfarm/wip-fix-hammer-x-fix

tests: remove extra indentation in exec block

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #13364 from tmuthamizhan/wip-ansible-jewel

qa: ceph-ansible smoke suite modified for jewel

tests: remove extra indentation in exec block

The exec block was indented by an extra space, causing

line 439, in parse_block_mapping_key
    "expected <block end>, but found %r" % token.id, token.start_mark)
yaml.parser.ParserError: while parsing a block mapping
  in "<string>", line 111, column 3:
      sequential:
      ^
expected <block end>, but found '<block sequence start>'
  in "<string>", line 126, column 4:
       - exec:
       ^

when trying to run upgrade/hammer-x

Reported-by: Yuri Weinstein <yweins@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13404 from smithfarm/wip-fix-hammer-x

tests: add require_jewel_osds to upgrade/hammer-x/tiering

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

tests: add require_jewel_osds to upgrade/hammer-x/tiering

Without this, the test tends to fail, but sometimes passes (apparently because
the "wait-for-healthy: true" in the last ceph.restart is racy - HEALTH_OK is
received before the MONs notice that all OSDs are running jewel without
require_jewel_osds.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

librbd: async method to check journal tag owner

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 953561f84140efcc870ec4a8f1ecb732b8e99409)

Conflicts:
src/librbd/Journal.cc - trivial context difference (master does not
                                have "class SafeTimerSingleton"),
                                added "typedef ::journal::Journaler Journaler;" in
                                C_IsTagOwner template to fix build failure

rbd-mirror: check image mirroring state when bootstrapping

Fixes: http://tracker.ceph.com/issues/18447
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 5fc5a8ac895524f05eed6e7db20b0dda3a8cb60f)

rbd-mirror: async request to test if image is primary

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 0a1cb35caacdf85029f31a0364dc07a5d7462f5f)

Conflicts:
src/tools/rbd_mirror/CMakeLists.txt - file doesn't exist in jewel
            (ported the patch to src/tools/Makefile-client.am)
src/tools/rbd_mirror/image_replayer/BootstrapRequest.cc - Journal is
             implemented by a typedef in jewel
src/tools/rbd_mirror/image_replayer/OpenLocalImageRequest.cc -
             no compat.h include in jewel, Journal is implemented by a typedef
             in jewel

Merge pull request #13331 from smithfarm/wip-18869-jewel

jewel: tests: SUSE yaml facets in qa/distros/all are out of date

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11410 from dachary/wip-17334-jewel

jewel: crushtool --compile is create output despite of missing item

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/cli/crushtool: fix the test of compile-decompile-recompile.t

should read the map from $TESTDIR,
it's a regression introduced by b2c0a07

Fixes: http://tracker.ceph.com/issues/17306
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 404043980d7882f320f48fbaed4b18f31bb15eb7)

crush/CrushCompiler: error out as long as parse fails

do not output compiled crush map if anything goes wrong when parsing
crush map.

Fixes: http://tracker.ceph.com/issues/17306
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b2c0a079dc074c907e1bc429861230096534f597)

qa: ceph-ansible smoke suite modified for jewel

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

Merge pull request #13358 from linuxbox2/wip-jewel-marker

jewel: rgw: fix use of marker in List::list_objects()

rgw: fix use of marker in List::list_objects()

Fixes: http://tracker.ceph.com/issues/18331
List marker is an index key, so treat it as such. This
fixes infinite loop in orphans find command.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit a5d1fa0587184f43c69d8e03114b58d43f320781)

tests: drop buildpackages.py

The buildpackages suite has been moved to teuthology. This cleans up a file
that was left behind by https://github.com/ceph/ceph/pull/13297

Fixes: http://tracker.ceph.com/issues/18846
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 6b7443fb50c117ee7f20d53bbc7530bb0eb7ebd5)

tests: update SUSE yaml facets in qa/distros/all

Fixes: http://tracker.ceph.com/issues/18856
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 0bd56e871a8549d4b0b1211f09dad2d1120fb606)

Merge pull request #13299 from dachary/wip-18848-jewel

jewel: tests: remove qa/suites/buildpackages

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

buildpackages: remove because it does not belong

It should live in teuthology, not in Ceph. And it is currently broken:
there is no need to keep it around.

Fixes: http://tracker.ceph.com/issues/18846
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 5a43f8d57925da227c95480501ceec10a29395d8)

Conflicts:
qa/tasks/buildpackages/common.sh (remove)
qa/tasks/buildpackages/make-rpm.sh (remove)

Merge pull request #13280 from athanatos/wip-revert-jewel-18581

Revert "Merge pull request #12978 from asheplyakov/jewel-18581"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Revert "Merge pull request #12978 from asheplyakov/jewel-18581"

See: http://tracker.ceph.com/issues/18809

This reverts commit 8e69580c97622abfcbda73f92d9b6b6780be031f, reversing
changes made to c05730ceb3387fb43c35937f0506297a34a44452.

Signed-off-by: Samuel Just <sjust@redhat.com>

librbd: allow to open an image without opening parent image

Fixes: http://tracker.ceph.com/issues/18325
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 61af1c25015de087a2423811548d975dd7d430b4)

Conflicts:
  src/librbd/ImageState.cc - added missing arg to RefreshRequest::create
  src/librbd/exclusive_lock/PostAcquireRequest.cc -
                                        deleted, does not exist in jewel
  src/librbd/image/OpenRequest.cc - added missing arg to
                                    RefreshRequest::create
  src/librbd/internal.cc - added missing arg to ImageState::open
  src/librbd/librbd.cc - added missing arg to ImageState::open
  src/test/librbd/exclusive_lock/test_mock_PostAcquireRequest.cc -
                                        deleted, does not exist in jewel
  src/test/rbd_mirror/image_replayer/test_mock_CreateImageRequest.cc -
                                        added missing arg to ImageState::open
  src/test/rbd_mirror/test_PoolWatcher.cc - added missing arg to
                                            ImageState::open

rbd-mirror: hold owner lock when testing if lock owner

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 82aa89668d666f434cd19ff444223017b5512c6a)

Merge pull request #11872 from dachary/wip-17838-jewel

jewel: leak in RGWFetchAllMetaCR

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13182 from smithfarm/wip-18498-jewel

jewel: rgw: Realm set does not create a new period

Merge pull request #13180 from smithfarm/wip-18547-jewel

jewel: rgw: multisite: segfault after changing value of rgw_data_log_num_shards

Merge pull request #13177 from smithfarm/wip-18676-jewel

jewel: rgw: librgw: objects created from s3 apis are not visible from nfs mount point

Merge pull request #13175 from smithfarm/wip-18684-jewel

jewel: rgw: multisite: sync status reports master is on a different period

Merge pull request #13173 from smithfarm/wip-18710-jewel

jewel: rgw: slave zonegroup cannot enable the bucket versioning

Merge pull request #13171 from smithfarm/wip-18712-jewel

jewel: rgw: radosgw-admin period update reverts deleted zonegroup

Merge pull request #13001 from asheplyakov/jewel-bp/18559

jewel: rgw multisite: fix ref counting of completions

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12997 from asheplyakov/jewel-bp-18569

jewel: radosgw valgrind "invalid read size 4" RGWGetObj

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12764 from SUSE/wip-17342-jewel

jewel: tests: assertion failure in a radosgw-admin related task

Merge pull request #12738 from SUSE/wip-18286-jewel

jewel: rgw: multisite: coroutine deadlock in RGWMetaSyncCR after ECANCELED errors

Merge pull request #12678 from Abhishekvrshny/wip-18348-jewel

jewel: rgw ldap: enforce simple_bind w/LDAPv3 redux

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12622 from wido/jewel-15776

jewel: rgw: log name instead of id for SystemMetaObj on failure

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #12542 from dreamhost/wip-17076-jewel

jewel: rgw: Replacing '+' with "%20" in canonical uri for s3 v4 auth.

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12428 from SUSE/wip-18216-jewel

jewel: rgw-admin: missing command to modify placement targets

Merge pull request #12426 from SUSE/wip-18214-jewel

jewel: add max_part and nbds_max options in rbd nbd map, in order to keep consistent with

http://tracker.ceph.com/issues/17851#note-59

Merge pull request #12419 from SUSE/wip-18217-jewel

jewel: rgw sends omap_getvals with (u64)-1 limit

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12405 from jan--f/wip-18199-jewel

jewel: build/ops: install-deps.sh based on /etc/os-release

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12397 from linuxbox2/jewel-next

jewel: rgw: do not abort when accept a CORS request with short origin

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12316 from dachary/wip-18101-jewel

jewel: Add workaround for upgrade issues for older jewel versions

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12890 from dillaman/wip-18453-jewel

jewel: librbd: new API method to force break a peer's exclusive lock

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12302 from SUSE/wip-18135-jewel

jewel: build/ops: add hostname sanity check to run-{c}make-check.sh

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #12315 from dachary/wip-18102-jewel

jewel: rgw: Unable to commit period zonegroup change

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12314 from dachary/wip-18107-jewel

jewel: multisite: failed assertion in 'radosgw-admin bucket sync status'

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12313 from dachary/wip-18112-jewel

jewel: rgw: multisite requests failing with '400 Bad Request' with civetweb 1.8

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12258 from rzarzynski/wip-rgw-17931-jewel

jewel: rgw: add support for the prefix parameter in account listing of Swift API

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12156 from dachary/wip-17969-jewel

jewel: rgw: multisite upgrade from hammer -> jewel ignores rgw_region_root_pool

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11990 from asheplyakov/jewel-17908

jewel: rgw: for the create_bucket api, if the input creation_time is zero, we …

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11876 from dachary/wip-17839-jewel

jewel: rgw: the value of total_time is wrong in the result of 'radosgw-admin log show' opt

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11868 from dachary/wip-17512-jewel

jewel: multisite: metadata master can get the wrong value for 'oldest_log_period'

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11866 from dachary/wip-17709-jewel

jewel: rgw: multisite: coroutine deadlock assertion on error in FetchAllMetaCR

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11497 from rzarzynski/wip-rgw-slashinfo-jewel

jewel: rgw: add suport for Swift-at-root dependent features of Swift API

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11477 from dachary/wip-17119-jewel

jewel: rgw: multisite: assert(next) failed in RGWMetaSyncCR

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11476 from dachary/wip-17162-jewel

jewel: rgw: multisite doesn't retry RGWFetchAllMetaCR on failed lease

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11470 from dachary/wip-17514-jewel

jewel: rgw：bucket check remove _multipart_ prefix

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10661 from dachary/wip-16871-jewel

jewel: rgw: Have a flavor of bucket deletion in radosgw-admin to bypass garbage collection

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13187 from asheplyakov/jewel-bp-18729

jewel: cli: ceph-disk: convert none str to str before printing it

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13153 from smithfarm/wip-no-firefly-on-centos

jewel: tests: upgrade:hammer-x: install firefly only on Ubuntu 14.04

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13118 from smithfarm/wip-18274-jewel

jewel: Memory leaks in object_list_begin and object_list_end

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13025 from SUSE/wip-18605-jewel

jewel: ceph-disk prepare writes osd log 0 with root owner

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13106 from ceph/wip-cd-dev-option

qa/tasks: jewel backport -  ceph-deploy use of dev option

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #13062 from asheplyakov/jewel-bp-18379

jewel: msg/simple: clear_pipe when wait() is mopping up pipes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12978 from asheplyakov/jewel-18581

jewel: ReplicatedBackend: take read locks for clone sources during recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #12755 from Abhishekvrshny/wip-18284-jewel

jewel: Need CLI ability to add, edit and remove omap values with binary keys

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12677 from SUSE/wip-18104-jewel

jewel: mon: ceph osd down detection behaviour

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12291 from asheplyakov/jewel-18108

jewel: msg/simple/Pipe: error decoding addr

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11998 from Abhishekvrshny/wip-17877-jewel

jewel: FileStore: fiemap cannot be totally retrieved in xfs when the number of extents > 1364

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11991 from asheplyakov/jewel-17909

jewel: osd: limit omap data in push op

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #12045 from Abhishekvrshny/wip-17875-jewel

jewel: rgw: file: remove spurious mount entries for RGW buckets

Merge pull request #13161 from smithfarm/wip-lfn-upgrade-hammer

jewel: tests: add require_jewel_osds before upgrading last hammer node

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #12425 from SUSE/wip-18190-jewel

jewel: rbd-mirror: gmock warnings in bootstrap request unit tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13129 from smithfarm/wip-18558-jewel

jewel: rbd: bench-write will crash if --io-size is 4G

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13155 from smithfarm/wip-18494-jewel

jewel: rbd: [rbd-mirror] sporadic image replayer shut down failure

Merge pull request #13128 from smithfarm/wip-18633-jewel

jewel: rbd: [qa] crash in journal-enabled fsx run

http://tracker.ceph.com/issues/13512

Merge pull request #13104 from idryomov/wip-fio-unmap-devices-jewel

qa/tasks: backport rbd_fio fixes to jewel

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13156 from smithfarm/wip-18455-jewel

jewel: rbd: Attempting to remove an image w/ incompatible features results in partial removal

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13157 from smithfarm/wip-18434-jewel

jewel: rbd: Improve error reporting from rbd feature enable/disable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13168 from trociny/wip-18550-jewel

jewel: librbd: metadata_set API operation should not change global config setting

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12739 from SUSE/wip-18323-jewel

jewel: JournalMetadata flooding with errors when being blacklisted

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12416 from SUSE/wip-18219-jewel

jewel: msg: don't truncate message sequence to 32-bits

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13115 from smithfarm/wip-18404-jewel

jewel: mon: cache tiering: base pool last_force_resend not respected (racing read got wrong version)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13045 from SUSE/wip-18553-jewel

jewel: mon: peon wrongly delete routed pg stats op before receive pg stats ack

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13117 from smithfarm/wip-18280-jewel

jewel: mon: osd flag health message is misleading

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12320 from dachary/wip-18061-jewel

jewel: rgw:fix for deleting objects name beginning and ending with underscores of one bucket using POST method of js sdk.

Merge pull request #11759 from Abhishekvrshny/wip-17783-jewel

jewel: rgw: json encode/decode of RGWBucketInfo missing index_type field

Merge pull request #12153 from dachary/wip-18010-jewel

jewel: Cleanly reject session evict command when in replay