qa: rgw/multifs: test on XFS, drop BTRFS

objectstore/filestore-xfs.yaml was overriding fs/btrfs.yaml anyway,
which was confusing.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa: rgw/verify: test on XFS, drop BTRFS

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #21156 from smithfarm/wip-22031-jewel

jewel: mds: FAILED assert(get_version() < pv) in CDir::mark_dirty

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21172 from smithfarm/wip-22384-jewel

jewel: tests: src/test/libcephfs/test.cc:376: Expected: (len) > (0), actual: -34 vs 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21163 from smithfarm/wip-22380-jewel

jewel: cephfs: client reconnect gather race

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20111 from pdvian/wip-22764-jewel

jewel: cephfs-journal-tool: add "set pool_id" option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21152 from smithfarm/wip-21489-jewel

jewel: tests: failures from pjd fstest

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21175 from smithfarm/wip-22970-jewel

jewel: mds: session reference leak

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #21208 from ukernel/jewel-23529

jewel: cephfs: fix tmap_upgrade crash

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21084 from pdvian/wip-23413-jewel

jewel: log: Fix AddressSanitizer: new-delete-type-mismatch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20622 from dzafman/wip-info-soid

jewel: follow-on: osd: be_select_auth_object() sanity check oi soid

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21158 from smithfarm/wip-21784-jewel

jewel: cli/crushtools/build.t sometimes fails in jenkins' make check run

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21135 from smithfarm/wip-23523-jewel

jewel: tests: unittest_pglog timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #20999 from pdvian/wip-23411-jewel

jewel: legal: remove doc license ambiguity

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20882 from dzafman/wip-23307

jewel: ceph-objectstore-tool command to trim the pg log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20763 from tchaikov/wip-23185-jewel

jewel: ceph.in: bypass codec when writing raw binary data

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #21234 from tchaikov/wip-jewel-rocksdb-without-jemalloc

jewel: rocksdb: disable jemalloc explicitly

Reviewed-by: Nathan Cutler <ncutler@suse.com>

rocksdb: disable jemalloc explicitly

rocksdb's makefile tries to detect jemalloc automatically and compiles
with it enabled if it's found, but if ceph disable jemalloc, it won't
link against jemalloc. this causes link failure if we compile ceph
without jemalloc enabled in a host with jemalloc library installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
src/kv/Makefile.am: this change is not cherry-picked from
master. as master does not use autotools anymore. and master is
using rocksdb's cmake script for building, the latter does not
jemalloc by default.

tools/cephfs: fix tmap_upgrade crash

MDSUtility::shutdown() is called by deconstructor of MDSUtility.
shouldn't call it manually.

introduced by commit 1c7b6b00 "jewel: cephfs-journal-tool: move
shutdown to the deconstructor of MDSUtility" make

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/23529

mds: fix compilation error ensuing from session ref leak fix

61b8b1d8420c46ca30d38d063e2dc97c9f28d7eb removes the local get_session functions
and replaces all calls to these functions with calls to mds->get_session()

Jewel has an extra call which is converted by this commit.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #21178 from tchaikov/wip-jewel-pr-21159

jewel: build/ops: drop deprecated --use-wheel option from pip invocations

Reviewed-by: Nathan Cutler <ncutler@suse.com>

src: s/--use-wheel//

--use-wheel was deprecated in favor of --only-binary in pip v7.0.0. and
--use-wheel was removed in a recent release of pip. but some packages
are source packages, so we cannot simply replace use-wheel with
only-binary. so a simpler approach is to drop --use-wheel option, as pip
respects --find-links, and will find the required package from the
wheelhouse.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a80cce4ba6afee921a7c6adce802c91e30f8c0a9)

Conflicts:
src/ceph-detect-init/CMakeLists.txt
src/ceph-disk/CMakeLists.txt: trivial resolution
src/pybind/mgr/dashboard/CMakeLists.txt: dashboard2 is not
in luminous, so drop this change.
src/tools/setup-virtualenv.sh:
src/ceph-detect-init/Makefile.am:
src/ceph-disk/Makefile.am: these files were dropped in
master, so update them also.

mds: reset connection's priv when marking down connection

mds won't get reset for down connection, so it needs to break
circle reference between connection and session.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 1526e0884a7b4d8d8d4f9ba37487a6d51da90992)

mds: fix session reference leak

"m->get_connection()->get_priv()" increases the session's reference
count by one. but we forget to release the reference at several places

Fixes: http://tracker.ceph.com/issues/22821
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 358f8a55f2e0614218909768f128c63e00d9de79)

Conflicts:
src/mds/Locker.cc - fbda12b1eefeebae1e1c64b01bc8f222c0d4c13d has not
been backported to jewel, so session->check_access() call has fewer arguments
than in master
src/mds/Server.cc - 3fe5e734be822492446a403ab176965d2be737fc has not
been backported to jewel, so there is no "bool blacklisted = false;" assignment
statement

test: pass correct buffer length to ceph_getdents

This test fails if the randvar r is 0.

Fixes: http://tracker.ceph.com/issues/22221
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ef5699441e056133967ffbf29076eb79594bf5e5)

mds: handle client reconnect gather race

Fixes: http://tracker.ceph.com/issues/22263
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit e4ecf26a8ef622f8290129a682afb1c64b3e7e00)

Conflicts:
    src/mds/Server.cc: need to remove 'reconnect_evicting', otherwise compilation fails

crushtool: print error message to stderr not dout(1)

in hope to fix the mysterious test failure where cli/crushtool/build.t
prints nothing when error message is expected.

Fixes: http://tracker.ceph.com/issues/21758
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit dc78d40af77ff8d5e4f3d39497e6e3c98c309306)

Conflicts:
src/tools/crushtool.cc (master has "crush.find_roots(&roots);", jewel
                               has "crush.find_roots(roots);")

mds: fix CDir::log_mark_dirty()

the 'will dirty' check is wrong because we don't always project fnode.

Fixes: http://tracker.ceph.com/issues/21584
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 461bbc7e89a9ed440478f30145158b4989c137d0)

mds: fix CDir::log_mark_dirty()

CDir::log_mark_dirty() moves dirfrag to current log segment's dirty
dirfrag list, but it does not submit any log event. Old log segments
(that include events which dirty the dirfrag) may get expired before
the dirfrag gets committed. If MDS crashes, the changes in expired
log segments get lost.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 7596f4b76a31d16b01ed733be9c52ecfffa7b21a)

ceph: do link/rename semantic checks after srcdn is readable

For hard link, source inode must not be directory. For rename,
types of source/destination inodes must match. If srcdn is replica
and we do these checks while it's not readble, it's possible that
wrong source inode is used in these checks.

Fixes: http://tracker.ceph.com/issues/21383
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 46962b253563a867707e7c5d7887abf2060cc4d7)

Conflicts:
src/mds/Server.cc (different placement of need_auth assignment in jewel)

Merge pull request #20418 from pdvian/wip-22965-jewel

jewel: rbd-mirror: fix potential infinite loop when formatting status message

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20524 from pdvian/wip-23065-jewel

jewel: test/librbd: utilize unique pool for cache tier testing

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20285 from smithfarm/wip-22175-jewel

jewel: rbd: possible deadlock in various maintenance operations

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20518 from smithfarm/wip-23076-jewel

jewel: osd: objecter sends out of sync with pg epochs for proxied ops

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

tests: remove TestPGLog ASSERT_DEATH test

Fixes: http://tracker.ceph.com/issues/23504
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit adcba97a41795c25de8b6dfe263a1014564a305f)

Conflicts:
src/test/osd/TestPGLog.cc (the code being removed is slightly different in jewel)

tests: remove TestPGLog EXPECT_DEATH tests

As per Sam Just advice, remove the EXPECT_DEATH tests to avoid
intermittent hang because they do not play well with threads.

Fixes: http://tracker.ceph.com/issues/18030
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit f1b457ff9826dd3adc7f335f028ee19ecfa9f1b8)

Conflicts:
src/test/osd/TestPGLog.cc (the code being removed is slightly different in jewel)

Merge pull request #20508 from smithfarm/wip-23066-jewel

jewel: ceph.restart + ceph_manager.wait_for_clean is racy

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #18743 from shinobu-x/wip-21923-jewel

jewel: core: Objecter::C_ObjectOperation_sparse_read throws/catches exceptions on -ENOENT

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20435 from pdvian/wip-22968-jewel

jewel: cephfs: osdc/Journaler: make sure flush() writes enough data

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20335 from pdvian/wip-22865-jewel

jewel: mds: fix scrub crash

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20333 from pdvian/wip-22863-jewel

jewel: cephfs-journal-tool: move shutdown to the deconstructor of MDSUtility

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20312 from pdvian/wip-22861-jewel

jewel: cephfs: osdc: "FAILED assert(bh->last_write_tid > tid)" in powercycle-wip-yuri-master-1.19.18-distro-basic-smithi

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19993 from pdvian/wip-22700-jewel

jewel: cephfs: fuse client: ::rmdir() uses a deleted memory structure of dentry leads …

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19961 from smithfarm/wip-22695-jewel

jewel: mds: fix dump last_sent

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19141 from shinobu-x/wip-22241-jewel

jewel: cephfs: Processes stuck waiting for write with ceph-fuse

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20271 from smithfarm/wip-20823-jewel

jewel: cephfs: client::mkdirs not handle well when two clients send mkdir request for a same dir

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

log: Fix AddressSanitizer: new-delete-type-mismatch

 If you directly call the operator new function, you must also directly
 call the operator delete function, and must manually call the
 destructor as well.

Fixes: http://tracker.ceph.com/issues/23324
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 0fe0c1173b1b3efcb407c22e736d0e2d94bf5840)

Conflicts:
src/log/Entry.h: Resolved for Entry destructor

Merge pull request #20982 from dzafman/wip-compact

jewel: Add compact operation to ceph-osdomap-tool

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

legal: remove doc license ambiguity

The license applicable to the files under doc/ was originally declared (in the
top-level file COPYING) to be "Creative Commons Attribution-ShareAlike (CC
BY-SA)" by ed0653b493a3f919a3abc37a0aa9b5aa29ae0b0e

This license declaration omitted a version number.

Some time later ef7418421b3748c712019c8aedd02b8005c1e1ea was merged, mentioning
CC-BY-SA-1.0 as one of the Ceph source code licenses. Although the purpose of that
commit was only to summarize the canonical license information from COPYING, it
unintentionally became the only place in the source code where the doc license
version was specified.

In March 2018, the Debian project warned that CC-BY-SA-1.0 does not meet its
criteria for inclusion in the "free" section of the Debian distribution. (For
that, at least version 3.0 must be used.)

This new commit removes the doc license ambiguity by setting it to CC-BY-SA-3.0
in all places where the license is mentioned.

The exact spelling of the license name is taken from https://spdx.org/licenses/

Fixes: http://tracker.ceph.com/issues/23336
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit e294dd59100d962753f4dee8f7156be5a7fba875)

Conflicts:
ceph.spec.in: resolved License

Merge pull request #21014 from dzafman/wip-remove-bc-jewel

jewel: test: Replace bc command with printf command

Reviewed-by: Nathan Cutler <ncutler@suse.com>

test: Replace bc command with printf command

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 293ac9895f86b1258434d0dc539a7d4b7046cb6d)

Merge pull request #20479 from pdvian/wip-23021-jewel

jewel: rgw: fix the max-uploads parameter not work

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20561 from smithfarm/wip-22894-jewel

jewel: rgw: ECANCELED in rgw_get_system_obj() leads to infinite loop

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20496 from pdvian/wip-23023-jewel

jewel: rgw: upldate the max-buckets when the quota is uploaded

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20749 from theanalyst/wip-jewel-curl-ssl

jewel: rgw: libcurl & ssl fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19635 from linuxbox2/wip-rgw-jewel-20101

jewel: rgw: S3 POST policy should not require Content-Type

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19488 from dplyakin/jewel-get-website-error-code

jewel: rgw: fix GET website response error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20293 from smithfarm/wip-22572-jewel

jewel: rgw: stale bucket index entry remains after object deletion

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20292 from smithfarm/wip-22018-jewel

jewel: rgw: segmentation fault when starting radosgw after reverting .rgw.root

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #20291 from smithfarm/wip-21954-jewel

jewel: rgw: list bucket which enable versioning get wrong result when user marker

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20269 from smithfarm/wip-21951-jewel

jewel: rgw: multisite: data sync status advances despite failure in RGWListBucketIndexesCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20262 from smithfarm/wip-21950-jewel

jewel: rgw: null instance mtime incorrect when enable versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20179 from shinobu-x/wip-21655-jewel

jewel: rgw: add ability to sync user stats from admin api

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #19057 from shinobu-x/wip-22188-jewel

jewel: rgw: add cors header rule check in cors option request

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18772 from smithfarm/wip-22028-jewel

jewel: rgw: boto3 v4 SignatureDoesNotMatch failure due to sorting of sse-kms headers

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-osdomap-tool: Add compact comand option

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b1f397f4fc3313ee0325722156759bfe32286c6b)

tools: Add pg log trim command to ceph-objectstore-tool

Add test script that verifies the command in qa/standalone/osd

Fixes: http://tracker.ceph.com/issues/23242
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b01e4ea5e2c1d2e9e997f53621aba73a7f7c9241)

Conflicts:
src/tools/ceph_objectstore_tool.cc
Different ops
Need ObjectStore::Sequencer
Use apply_transaction()
qa/standalone/special/ceph_objectstore_tool.py
Different ops

(cherry picked from commit b055cde916d00014e0d148f129934216f29252d4)

Conflicts:
src/common/legacy_config_opts.h src/common/options.cc
         Use src/common/config_opts.h in Jewel
src/test/ceph_objectstore_tool.py (different ops)
src/tools/ceph_objectstore_tool.cc (different ops, no PastIntervals)
src/test/osd/repro_long_log.sh
Moved from ../qa/standalone/osd
Use Jewel ceph-helpers.sh location
No ceph-mgr in Jewel
Single test that creates 51 write entries and trims to 30

ceph.in: bypass codec when writing raw binary data

See-also: 615f8f45
Fixes: http://tracker.ceph.com/issues/23185
Signed-off-by: Oleh Prypin <oleh@pryp.in>
(cherry picked from commit 8fc67075c6d7d4443747f53687e498439f80b57a)

Conflicts: this change was introduced to master by 8fc67075. but that commit
  involves a lot of py3 compatibility changes, which are not the focus of
  jewel or this fix. so i am only cherry-pick the change writing buffer
  returned by asock to stdout.

rgw: rgw_admin use raii for curl init/teardown

Since a lot of rgw-admin commands can exit early

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 4667937e4cece63751d5406d75c140418910a823)

rgw: call setup and release curl handles in setup_curl

Since rgw admin can also use it which will fail otherwise.

Fixes: http://tracker.ceph.com/issues/23203
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 08c4c05f5d939976c9e1607b9338f0a9b287708d)

rgw: rgw_admin also inits curl

since we use http manager which in turn uses curl and uses curl multi
interfaces. While curl is initialized at the first call of curl_easy_init() this
method isn't guaranteed to be safe when multiple threads may call the function
since curl_global_init isn't reentrant. Calling curl_global_init via
rgw::curl::setup_curl which additionally sets up ssl interfaces etc. when
openssl is used as curl's ssl backend. Similarly moving rgw target link to
accomodate this change.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 112ba0b4dae5972b520aec39fe99275a6dfa899d)

 Conflicts:
  src/rgw/CMakeLists.txt
  beast submodule related changes which are not a part of this patchset
  src/rgw/rgw_admin.cc
  trivial whitespace conflicts #

rgw: make frontend map optional for ssl init

Since rgw admin also needs to init curl globally where we don't have a frontend
map

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 55dde3df50a9735b81c6b31f5683ebda1dabd5e2)

rgw: setup locks for libopenssl

openssl <= 1.02 requires explicit callbacks for locking which libcurl doesn't
set. This causes random segmentation faults when openssl uses its global
structures across multiple threads. Providing a simple mutex lock/unlock
functions as a callback. We determine whether openssl is used for libcurl via
curl-config utility which should be installed as a part of our curl development
headers package. We also additionally check that the openssl version is < 1.1.0
which alleviates the need for these callbacks. In this patchset we have done the
following:

- move all curl related global init functionality under rgw::curl namespace
  since libcurl may need to set up various ssl libraries etc during its init
- introduce WITH_CURL_OPENSSL in cmake
  this checks the backend curl is deployed with using curl-config. Since curl
  devel is expected to be installed anyway, this binary should be available and
  can help identify the ssl backend curl was compiled with.
- we only setup the locks if beast/civetweb aren't terminated with ssl, since
  these libraries setup the locks anyway and we want to prevent double
  initialization of openssl. Also we pass in ~CURL_GLOBAL_SSL making curl not
  initialize openssl if civetwb/beast is initializing them. Unfortunately this
  flag is a noop from curl >= 7.57 wherein both the libraries will end up
  initializing openssl anyway, which might override certain settings like error
  strings if using openssl < 1.1

https://curl.haxx.se/libcurl/c/threadsafe.html
https://www.openssl.org/docs/man1.0.2/crypto/threads.html#DESCRIPTION

Fixes: http://tracker.ceph.com/issues/22951
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Jesse Williamson <jwilliamson@suse.com>
(cherry picked from commit 4e30277e4b38694e6f8d61925736cd7c2cff515f)

 Conflicts:
  - CMakeLists.txt
  libcryptopp drop in master conflict
  - src/rgw/rgw_main.cc
  conflict as we removed the ac related headers in Luminous

Conflicts:
  CMake isn't supported in jewel, while the CMake changes were cherried,
  CMakelists for rgw does not exist in jewel and this commit doesn't fix that.
  So though CMake changes are backported, cmake build is untested and expected
  to be broken
src/include/config-h.in.cmake
  src/rgw/rgw_main.cc

jewel: build/rgw: configure if curl is linked against openssl

Since the Luminous/master change was cmake only, making an explicit autoconf
test.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: reuse CURL* for keystone.

When using keystone with https (the recommended setting),
it is inefficient to start up a new SSL connection for
each and every operation.  Keeping a CURL* structure around
should reduce the cost of doing this.  This logic tries
to do so, but it also tries to free them fairly aggressively
(5-10 seconds).  This should still greatly reduce load on
keystone at peak times while not tying up excess resources.

Fixes: https://tracker.ceph.com/issues/23162
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit 0c5cee1977eac232045571f4850e64be93271aec)

 Conflicts:
  src/rgw/rgw_common.h
  trivial conflict as lowercase_dash_http_attr isnt a part of L

build/link against curl w/ openssl not nss or gnutls. (debian)

curl + gnutls has some big significant performance hits at least
when doing keystone validation.  nss has long-term memory growth
issues.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit cb12e1afd5beedf1b8c0b85e24e412e34c2fc427)

osd: Change a check to an assert() since it can't happen anymore

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 8e2b9a07e0551895809a3fc036aae557fadc74ba)

osd: Check for and automatically repair object info soid during scrub

Fixes: http://tracker.ceph.com/issues/20471
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 22f4d0cc010c5887b52847f050a361bc1ac20e40)

Merge pull request #20061 from cbodley/wip-jewel-rgw-log-trim

jewel: rgw: automated trimming of datalog and mdlog

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix ECANCELED handling in rgw_get_system_obj()

Fixes: http://tracker.ceph.com/issues/17996
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit e67a3b5b5c63ee3a34f06bdbe91d3256c6660a89)

Conflicts:
    src/rgw/rgw_rados.cc: used ctx.invalidate(obj) instead of ctx.raw.invalidate(obj)
        in RGWRados::SystemObject::invalidate_state()

Merge pull request #20557 from smithfarm/wip-really-disable-pidfile-test

test/CMakeLists: disable test_pidfile.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Client: mkdirs bugs when two clients send mkdir request for a same dir

Fixes: http://tracker.ceph.com/issues/20592
Signed-off-by: dongdong tao <tdd21151186@gmail.com>
(cherry picked from commit 80bdd3fb4afd262ee3073b4e8377da9238bdaef2)

Conflicts:
        src/client/Client.cc (jewel uses "uid, gid" instead of "perms")

test/CMakeLists: disable test_pidfile.sh

Too flaky, see http://tracker.ceph.com/issues/20975

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ff3de2304497544033837bb8d0c809a9e54a3e6e)

Conflicts:
    src/test/Makefile.am: applied change manually because jewel uses autotools

Merge pull request #20344 from pdvian/wip-22866-jewel

jewel: mon/OSDMonitor: fix dividing by zero in OSDUtilizationDumper

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20463 from dzafman/wip-23006

test: Adjust for Jewel quirk caused of differences with master

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20446 from smithfarm/wip-23010-jewel

jewel: Filestore rocksdb compaction readahead option not set by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20289 from smithfarm/wip-21239-jewel

jewel: test_health_warnings.sh can fail

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20284 from dzafman/wip-22912

jewel: ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pg

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #19611 from ukernel/jewel-22494

jewel: include/fs_types: fix unsigned integer overflow

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #20520 from smithfarm/wip-disable-test-pidfile

test/CMakeLists: disable test-pidfile.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

test/librbd: utilize unique pool for cache tier testing

Fixes: http://tracker.ceph.com/issues/11502
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c42c54d0ec643b272aff225134dd0eceae946af0)

Conflicts:
src/test/librbd/test_librbd.cc: Resolved in TEST_F

test/CMakeLists: disable test-pidfile.sh

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c4936f4493d34341aab8f01488b45dc981d4ef3f)

osd/PrimaryLogPG: cancel all objecter ops atomically

We want to avoid a situation like this:

- start proxy op A (epoch E)
- start proxy op B (epoch E)
- start proxy op C (epoch E)
- objecter sends none of these because target is down in epoch E
- osdmap update to E+1
- pg cancels requeues A, B
- objecter updates to E+1
- objecter sends C
- pg cancels/requeues C

Note that the key thing is that operations on each object are canceled
atomically.  On the interval change we do it all at once.  In the other
cases, we cancel everything on the given object together.

Fixes: http://tracker.ceph.com/issues/22123
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 93fd56ed039363c4f169259a0a560b968d1a0333)

Conflicts:
src/osd/PrimaryLogPG.cc (manually backported changes from src/osd/ReplicatedPG.cc)
src/osd/PrimaryLogPG.h (manually backported changes from src/osd/ReplicatedPG.cc)

osdc/Objecter: allow multiple ops to be canceled atomically

This allows multiple cancellations without worrying about a race with an
(objecter) osdmap update.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7459d4a4794eb092cdc3ee06af12b782c892193d)

qa/tasks/ceph.py: fix timing of wait-for-* and osd markdown

Mark down osds, *then* wait for them to come up or for the cluster to be
healthy!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 86c0d07e32205e2b6aa417a0e4ae03f0084a1888)

tasks/ceph.restart osd fix

ceph.restart should mark restarted osds down in order to avoid a
race condition with ceph_manager.wait_for_clean

Fixes: http://tracker.ceph.com/issues/15778
Signed-off-by: Warren Usui <wusui@redhat.com>
(manual cherry pick of 1b7552c9cb331978cb0bfd4d7dc4dcde4186c176)

Conflicts:
    qa/tasks/ceph.py (original commit was in ceph/ceph-qa-suite.git)

Merge pull request #17707 from liewegas/wip-17445

jewel: osd: also check the exsistence of clone obc for "CEPH_SNAPDIR" requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17883 from andyli029/wip-jewel-signal-handler-typo

jewel: core: global/signal_handler.cc: fix typo

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #19906 from jdurgin/jewel-filestore-split

jewel: HashIndex: randomize split threshold by a configurable amount

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20331 from dzafman/wip-21036

jewel: snapset xattr corruption propagated from primary to other shards

Reviewed-by: Josh Durgin <jdurgin@redhat.com>