ceph-volume: use our own testinfra suite for functional testing

This allows us to run a smaller subset of tests from the ones provided
by ceph-ansible and test only the things that pertain to ceph-volume.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit ccfc8e6dc42b55f14fa3b4ef210cb639b9a7e20b)

Merge pull request #26656 from jan--f/c-v-simple-activate-all-luminous

luminous: ceph-volume: add --all flag to simple activate

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #26065 from pdvian/wip-37989-luminous

luminous: MDSMonitor: missing osdmon writeable check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #25858 from ashishkumsingh/wip-37823-luminous

luminous: mds: log evicted clients to clog/dbg

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #25979 from pdvian/wip-37908-luminous

luminous: mds: wait shorter intervals if beacon not sent

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26215 from pdvian/wip-38102-luminous

luminous: mds: trim cache after journal flush

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26293 from pdvian/wip-38190-luminous

luminous: mds: broadcast quota message to client when disable quota

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26473 from pdvian/wip-38336-luminous

luminous: mds: fix potential re-evaluate stray dentry in _unlink_local_finish

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

ceph-volume: add --all flag to simple activate

This is intended to behave similarly to the lvm activate --all argument.
When passed, c-v will scan /etc/ceph/osd/ (or the location specified by
CEPH_VOLUME_SIMPLE_JSON_DIR) for json files (glob *.json) and call activate
for each file.  This should greatly ease the take-over of ceph-disk OSDs
with manual commands and deployment tools like DeepSea and ceph-ansible.
Also adds a simple unit test and modifies the
simple/centos7/filestore/activate functional test to use --all.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 05af94c6fca66083107e96e9aa1db87619e77e66)

 Conflicts:
src/ceph-volume/ceph_volume/tests/functional/simple/centos7/filestore/activate/test.yml
        resolved by deleting everything after conflict marker

Merge pull request #25611 from pdvian/wip-37561-luminous

luminous: rgw: rgwgc: process coredump in some special case

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25986 from ashishkumsingh/wip-37937-luminous

luminous: librgw: export multitenancy support

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #26310 from pdvian/wip-38193-luminous

luminous: Object can still be deleted even if s3:DeleteObject policy is set

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25738 from pdvian/wip-37648-luminous

luminous: rgw: feature -- log successful bucket resharding events

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25901 from ashishkumsingh/wip-37559-luminous

luminous: rgw: handle S3 version 2 pre-signed urls with meta-data

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #26131 from pdvian/wip-38038-luminous

luminous: rgw: fails to start on Fedora 28 from default configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #26192 from cbodley/wip-38079-luminous

luminous: rgw: bucket full sync handles delete markers

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #26263 from pdvian/wip-38148-luminous

luminous: rgw: "radosgw-admin bucket rm ... --purge-objects" can hang

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #26332 from smithfarm/wip-38232-luminous

luminous: rgw: fix radosgw linkage with WITH_RADOSGW_BEAST_FRONTEND=OFF

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #26587 from smithfarm/wip-doc-fix-luminous

luminous: doc: Fix default value of 'mon_osd_max_split_count'

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

doc: Fix default value of 'mon_osd_max_split_count'

This commit is not cherry-picked from master because the config
parameter mon_osd_max_split_count has been dropped in master

Signed-off-by: Kai Wagner <kwagner@suse.com>
(cherry picked from commit 17d503f5adfbfd96502877bcb55c42ba1eb34a5c)

Merge pull request #24906 from tchaikov/wip-23673-luminous

luminous: auth: ceph auth add does not sanity-check caps

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25711 from ashishkumsingh/wip-37756-luminous

luminous: osd/PrimaryLogPG: fix the extent length error of the sync read

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25722 from ashishkumsingh/wip-37690-luminous

luminous: tools: ceph-objectstore-tool: Dump hashinfo

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #25851 from ashishkumsingh/wip-37815-luminous

luminous: qa/workunits/rados/test_health_warnings: prevent out osds

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25853 from ashishkumsingh/wip-37833-luminous

luminous: objecter: avoid race when reset down osd's session

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #26305 from dzafman/wip-38207

luminous: mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_IN…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #26078 from ashishkumsingh/wip-37993-luminous

luminous: osd: keep using cache even if op will invalid cache

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26148 from ashishkumsingh/wip-38046-luminous

luminous: tests: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26236 from pdvian/wip-38105-luminous

luminous: osd/PG.cc: account for missing set irrespective of last_complete

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26261 from pdvian/wip-38142-luminous

luminous: os/bluestore: fixup access a destroy cond cause deadlock or undefine

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #25720 from ashishkumsingh/wip-36692-luminous

luminous: rbd-mirror: update mirror status when stopping

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25768 from ashishkumsingh/wip-26913-luminous

luminous: mgr: "balancer execute" only requires read permissions

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #25955 from pdvian/wip-37901-luminous

luminous: [journal] max journal order is incorrectly set at 64

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #26000 from pdvian/wip-37926-luminous

luminous: common: fix for broken rbdmap parameter parsing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #26286 from smithfarm/wip-38186-luminous

luminous: tests: krbd discard no longer guarantees zeroing

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

mds: fix potential re-evaluate stray dentry in _unlink_local_finish

If MDS handles cap release before _unlink_local_finish, then when MDS
processes _unlink_local_finish, eval_stray might be re-entered twice,
so this will cause crashing.

Fixes: http://tracker.ceph.com/issues/38263
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
(cherry picked from commit 72c93baac64211e9f02fa01e743451bf5d63d7a6)

Conflicts:
src/mds/MDCache.h : Resolved in notify_stray

Merge pull request #25990 from ukernel/luminous-37760

luminous: mds: handle fragment notify race

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26005 from ukernel/luminous-37481

luminous: mds: handle state change race

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26257 from batrick/i38084

luminous: mds: log new client sessions with various metadata

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26271 from batrick/i38104

luminous: client: session flush does not cause cap release message flush

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #26278 from ukernel/luminous-stale-caps

luminous: mds: optimize revoking stale caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26326 from alfredodeza/luminous-rm38229

luminous: doc: pin the version for "breathe" to 4.1.11

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

rgw: fix radosgw linkage with WITH_RADOSGW_BEAST_FRONTEND=OFF

The master commit 5c040d991510cb4ff0d74305889130e2d84fedc1 fixing issue
http://tracker.ceph.com/issues/23680 was backported to luminous for v12.2.11 by
a47e714e7f5ce803ba7d8986c5d954123b85fc8e which was included in
https://github.com/ceph/ceph/pull/24621, where it came as the first of a series
of five cherry-picks.

This, it turns out, was the wrong order - it should have come last since it
was a follow-up fix.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(partial manual backport of 5c040d991510cb4ff0d74305889130e2d84fedc1)

doc: pin the version for `breathe` to 4.1.11

It has the compatibility needed for Sphinx version 1.6.3

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #26295 from jan--f/25390-luminous

[luminous] ceph-volume: revert partition as disk

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

rgw: cleanup for RGWDeleteObj::verify_permission()

- if the 'Effect' field of user policy and bucket policy
  is set to 'Allow', RGWDeleteObj::verify_permission()
  should return 0 instead of true;

- as the same with bucket policy, if the 'Effect' field
  of user policy is set to 'Deny', RGWDeleteObj::verify_permission()
  should return -EACCES instead of false;

Signed-off-by: Enming.Zhang <zvampirem77@gmail.com>
(cherry picked from commit 86334645cf96c03515d30804e2df0bdf22916fb5)

Conflicts:
src/rgw/rgw_op.cc : Resolved in verify_permission

rgw: fix obj can still be deleted even if deleteobject policy is set

Fixes: http://tracker.ceph.com/issues/37403
Signed-off-by: Enming.Zhang <zvampirem77@gmail.com>
(cherry picked from commit 84879e915eb3f01b453aa2138cd44c09925e7cd1)

Conflicts:
src/rgw/rgw_op.cc : Resolved in verify_permission

ceph-volume: add inventory call to functional tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit a183a28c5e423c5658a37f7a1d3add9d64c6d683)

ceph-volume: add partition support to Device

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 05ac680cb37fd15fdeadaea86780bf1b4f2d566a)

Revert "ceph-volume tests.util ensure partitions are listed as top-level keys"

This reverts commit 6dc01772245e557b73551111f7c48cc60aec0893.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 3c1c32cbc2736d72295f1bd7921c4a3cd2f4727e)

Revert "ceph-volume util.disk include partitions as top level devices in get_devices"

This reverts commit 7f3c3598491e556ec1b93ea0a54f52144202fab6.
Fixes: http://tracker.ceph.com/issues/37506
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 50bc44f1c2327ad38504950fe62820a44ead9b14)

mds: broadcast quota message to client when disable quota

When disable quota by setting quota.max_files or quota.max_bytes
to zero, client does not receive the quota broadcast message.

Signed-off-by: Junhui Tang <tangjunhui@sangfor.com.cn>
Fixes: https://tracker.ceph.com/issues/38054
(cherry picked from commit 061ace2eb7c08d271db4d5d08ffa91ae832715f5)

Conflicts:
src/mds/MDCache.cc : Resolved in broadcast_quota_to_client

qa: update krbd tests for zeroout

Discard no longer guarantees zeroing, use BLKZEROOUT and "fallocate -z"
instead (blkdiscard(8) in xenial doesn't support -z).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 04f5b343f9987f0cf67509c2a76ddf7ef46288e5)

Conflicts:
qa/rbd/krbd_whole_object_zeroout.t
qa/suites/krbd/basic/tasks/krbd_whole_object_zeroout.yaml
- the modifications to these files are not relevant for luminous

mds: optimize resuming stale caps

If client doesn't want any cap, there is no need to re-issue stale
caps.

A special case is that client wants some caps, but skipped updating
'wanted'. For this case, client needs to update 'wanted' when stale
session get renewed.

Fixes: http://tracker.ceph.com/issues/38043
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit e824b3d2024db36789fdf579f0af9bf3bbe55d51)

 Conflicts:
src/client/Client.cc
src/mds/cephfs_features.h

client: avoid unnecessary wakeup when handling RENEWCAPS

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit c744bc1673ca0b4e50f41516fdf49c8560db073a)

client: set cap->wanted when adding new cap

This avoids unnecessary cap message if cap is added by open/create
request reply.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 314660a46a8a1af97f70b2bac05b2f6fa5d23bc4)

 Conflicts:
src/client/Client.cc
src/client/Client.h

client: don't wakeup cap waiters twice when mds recovered

Both kick_maxsize_requests() and wake_inode_waiters() wake up cap
waiters

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 5993a93e606d23ef311831863444294d7c64cc04)

 Conflicts:
src/client/Client.cc
src/client/Client.h

mds: optimize revoking stale caps

For caps that are not being revoked and don't have writeable range
and don't want exclusive caps or file read/write. there is no need
to call Locker::revoke_stale_caps(Capability*). Because these caps
don't need recover and don't affect eval_gather()/try_eval().

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit cb62030f0468fc04735c1b4cff73da779cb11ad8)

 Conflicts:
src/mds/CInode.cc
src/mds/Capability.h
src/mds/Locker.cc
src/mds/Migrator.cc

mds: put notable caps at the front of session's caps list

Notable Capabilities are ones that are being revoked, ones that
have writeable ranges and ones that want exclusive caps or want
file read/write.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit cb6e7184458e64720ad83c266e5e393a80c32697)

 Conflicts:
src/mds/Capability.cc

mds: track if client has writeable range in Capability

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 370ae1cb3e5dc07867d80e998082bc514e8fccfd)

 Conflicts:
src/mds/Capability.h
src/mds/Locker.cc
src/mds/MDCache.h
src/mds/Server.cc

mds: add session pointer to Capability

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 7c628472a86c6acebb20f0a2504744b10f250587)

 Conflicts:
src/mds/CInode.cc
src/mds/Capability.cc
src/mds/Capability.h
src/mds/Locker.cc

client: skip updating 'wanted' caps if caps are already issued

When reading cached inode that already has Fscr caps, this can avoid
two cap messages (one updats 'wanted' caps, one clears 'wanted' caps).

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit d20b260ecf3f323c87ad1e6865f87e2381444546)

 Conflicts:
src/client/Client.cc

client: sync 'retain caps' logical from kernel client

The main change is keeping CEPH_CAP_ANY_RD for unused file inodes

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 29034396398b1706db625e1abf2e6682be7130a5)

Merge pull request #26207 from vumrao/wip-vumrao-38108

luminous: mgr/DaemonServer: log pgmap usage to cluster log

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26186 from tchaikov/luminous-36737

luminous: build/ops: Allow multi instances of "make tests" on the same machine

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

client: flush cap release on session flush

MDS expects the completed cap release prior to responding to the session flush
for cache drop.

Fixes: http://tracker.ceph.com/issues/38009
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 59eca7a17816317d5064840745ea5e28d3349d1a)

Conflicts:
src/client/Client.cc

rgw: `radosgw-admin bucket rm ... --purge-objects` can hang...

This command can hang (i.e., enter an infinite loop) due to
problematic bucket index entries left as a result of bug
https://tracker.ceph.com/issues/38007 .

The fix is to ignore the false bucket index entries -- since they do
not represent actual objects -- and remove all actual objects in the
bucket, so that bucket itself can be removed.

This fixes the both code paths whether `--bypass-gc` is specified or
not.

Furthermore, to made these operations more efficient, the internal
listing of the bucket is done unordered. This would improve behavior
when removing buckets with a large number of objects.

Fixes: http://tracker.ceph.com/issues/38134
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 5b3dc38c628728651b26415ab3d2190cf06b9e39)

mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_INCONSISTENT does

Fixes: http://tracker.ceph.com/issues/38070
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a4b1574532ee4f789b30634f0eca03cc30ddcf8d)

os/bluestore: fixup access a destroy cond cause deadlock or undefine
              behavior.

Fixes: https://tracker.ceph.com/issues/37733
Signed-off-by: linbing <linbing@t2cloud.net>
(cherry picked from commit dbbde306484276251cbd7a7b7a6fde0093f98ba2)

Conflicts:
src/os/bluestore/BlockDevice.h : Resolved in try_aio_wake

mds: do not log internal journal error

MDS can't reach this code due to assert in C_MDS_session_finish.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0992ac1fa62edb6143efc80a6fea7b4704107705)

Conflicts:
src/mds/Server.cc

mds: clean up log level 2 messages

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit a225c266052d09ffcb066aead83a18dec918aa35)

Conflicts:
src/mds/MDSRank.cc

mds: log session open stats

This is to assist with tracking what clients are connecting and how long it
takes for them to open a session (a rough estimate of how long it takes to
mount).

Example line:

    2018-12-17 17:21:31.682 7f822703c700  2 mds.0.server New client session: addr="127.0.0.1:60552/2307908923",elapsed=0.013384,throttled=0.000071,status="ACCEPTED",root="/"

Fixes: http://tracker.ceph.com/issues/37678
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c89cd8cac16789ed98dc1487ec17f45adf03dd26)

Conflicts:
src/mds/Server.cc
src/mds/SessionMap.cc
src/mds/SessionMap.h
src/mds/events/ESession.h

osd/PG.cc: account for missing set irrespective of last_complete

When adding source info from another OSD, check if an object that needs
recovery is present in its missing set. If yes, do not include the OSD
as a missing loc.

Fixes: http://tracker.ceph.com/issues/37919
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit e6b1637e00a3e98e588e4b83a13db5703fb96fce)

cmake: use $CMAKE_BINARY_DIR for default $CEPH_BUILD_VIRTUALENV

to avoid conflict with other `make check` runs in the same host

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a8d6b2170dfa0838dad0d1b7b03fc8cf19e4eeff)

Conflicts:
- drop the change to mgr/*/run-tox.sh, as they don't exist in
luminous.
        - apply the change to ceph-disk/run-tox.sh
- apply the change to ceph-helpers.sh: as it set $PATH using
          $TMPDIR, which is set to /tmp if $TMPDIR is empty.

mds: trim cache after journal flush

Dropping cache should trim the cache only after journal flush otherwise dirty
inodes will remain pinned in cache.

Fixes: http://tracker.ceph.com/issues/38010
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d7f6c1583b4162fade434e376176f3068472f53f)

mgr/DaemonServer: log pgmap usage to cluster log

- change default of mon_cluster_log_file_level to debug
- log to clog->debug()

Fixes: https://tracker.ceph.com/issues/37886
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f64a69e5054fb319e9f717bbcb7ddd8541e01c6f)

 Conflicts:
src/common/options.cc
        dropped master .add_service, .add_see_also

12.2.11

rgw: bucket full sync handles delete markers

full sync uses CLS_RGW_OP_LINK_OLH_DM for delete markers instead of
trying to fetch them from the source zone

Fixes: http://tracker.ceph.com/issues/38007
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5ffd34c5a6a9bbdba086eb57cded915b879513d4)

Conflicts:
src/rgw/rgw_data_sync.cc sync tracing

test/rgw: add multisite test_suspended_delete_marker_full_sync()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8686b6429c520aff388fb64d46172d691d71950d)

test/rgw: add multisite test_delete_marker_full_sync()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit fba4ed72fd8b3ebec25b18ecce44ffd2717c0d91)

test/rgw: check_bucket_eq() uses bucket.list_versions()

get_all_versions() is documented as a lower-level api that doesn't
handle paging, ands suggests list_versions() instead. also caches the
results to avoid listing each bucket twice

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 22d1da72405cbfb522ac661139abf6a22722980c)

test/rgw: check_bucket_eq() supports delete markers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 305f6dd6e94498afa53554075009cb490803e255)

test: listen on random port in tests which start ceph-mon

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 94a84b6f5adad75b3cbf956447ee5d9caf1018d2)

test: write log file to current directory

instead of writing log files to '/tmp', write them to current directory.

by default, ctest runs the tests in ${CMAKE_CURRENT_BINARY_DIR}, so, in
this case, the log files are located in build/src/test/. this ensure
that current user has enough previledge to remove the log files.

this also allows multiple instances of 'unittest_log' to run in parallel.

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a13a6f1d0b39a541896591cd04fc1980bb1acfee)

Conflicts:
src/log/test.cc: trivial resolution

Merge pull request #26117 from ceph/backport-luminous-26084

luminous: ceph-volume: mark a device not available if it belongs to ceph-disk

Merge pull request #26114 from alfredodeza/luminous-rm37963

luminous ceph-volume fix TypeError on dmcrypt when using Python3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries

Fixes: http://tracker.ceph.com/issues/38025
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 4d87071bff8b4cdc1a8d167167362e98fc7f8fcf)

rgw ldap:  move conditional LDAPEngine init into it's ctor

rgw::auth::ExternalAuthStrategy defines an LDAPEngine in a
ctor-initializer list, making it too late to conditionally initialize
-it- in ExternalAuthStrategy's ctor.

Fixes: https://tracker.ceph.com/issues/24228
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 97e6c9dac7ca9050d7b16ac160120acbe6ddc33c)

Conflicts:
src/rgw/rgw_rest_s3.h : Added valid

rgw ldap: safe_read_file can return < 0

Avoid invalid string access when LDAP auth is enabled, but LDAP
secret is not set.  This is Casey's fix for the problem.

Fixes: https://tracker.ceph.com/issues/24228
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 65cd8accc127c6765f083a389455ce73b7517e11)

ceph-volume: mark a device not available if it belongs to ceph-disk

The `ceph-volume inventory` command will now show if a device is being
used by ceph-disk and mark it not available if so.

Fixes: https://tracker.ceph.com/issues/24871
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit d14a6b284c44a3f61c0dd8c60ccc474203003653)

ceph-volume tests verify behavior of create_dmcrypt_key in python 3

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 69613cc5a8b2e111e24b75ad5b69f924c89fd75b)

ceph-volume util.encryption ensure int is used for urandom (py3 compat)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0ab9fa62d6d46464b16f08d3a5bb3da50f5030e7)

Merge pull request #26088 from ukernel/luminous-37977

luminous: mds: fix infinite loop in OpTracker::check_ops_in_flight

Reviewed-By: Patrick Donelly <pdonelly@redhat.com>

mds: fix infinite loop in OpTracker::check_ops_in_flight

introduced by backport commit 02faf3d
"mds: don't report slow request for blocked filelock request"

Fixes: http://tracker.ceph.com/issues/37977
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Revert "mds: fix infinite loop in OpTracker::check_ops_in_flight"

This reverts commit adad2d873a37770e4910b04dfddf92f70964be4a.
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #26037 from yuriw/wip-yuriw-p2p-luminous

luminous: qa/tests: changed to `supported' distro and moved all `p2p` suites un…

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25867 from ashishkumsingh/wip-37827-luminous

luminous: mgr: fix crash due to multiple sessions from daemons with same name

osd: keep using cache even if op will invalid cache

the current wrong process on same obj in ec 4+2 pool with 16K stripe:

obj has 16K~16K with content-0
new op1 will write 8K~32K (16K~16K content-1)
op1 reserve 0K~48K in cache
op1 read 0~16K & 32K~48K from disk
new op2 will write 20K~24K (20K~4K content-2) *** (invalid_cache dur to clone)
op2 read 16K~32K from disk (content-0) ***** dur to not using_cache
op1 write 0~48K to disk (content-1)
op1 present 0~48K to cache (content-1)
op2 write 16~32K to disk (content-0 + content-2) *** here we drop the content1

even if the op will invalid cache, it should do that *after* itself,
the op should use the cache by itself.

Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>
(cherry picked from commit e2dbb8058261e8cc3a3fb00362e2785f0ac6cf18)

mds: log evicted clients to clog/dbg

Fixes: http://tracker.ceph.com/issues/37639
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7a81421453f76c6a67f28f749f1489737f7b58c5)

Conflicts:
src/mds/MDSRank.cc
  - Used 'std::stringstream' instead of 'CachedStackStringStream'
    and repllaced the 'ss.strv()' to 'ss.str()'.

MDSMonitor: add missing OSDMonitor writeable check

Fixes: http://tracker.ceph.com/issues/37929
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8ef68663d311f71d1598297c14fad39274ada41d)