mds: include session uptime when diplaying session list

Fixes: http://tracker.ceph.com/issues/35937
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b23a204cdde2bc5f34304cca3f1bac3496cf7a41)

mds: track average uptime of sessions

Average session age math improvements by Patrick.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit d2627b98d0c1477d664d00384ef033d323b26957)

Conflicts:
        src/mds/SessionMap.h

Merge pull request #23877 from smithfarm/wip-24842-luminous

luminous: qa: move mds/client config to qa from teuthology ceph.conf.template

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24086 from batrick/i35976

luminous: mds: configurable timeout for client eviction

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

Merge pull request #24376 from smithfarm/wip-35939-luminous

luminous: client: statfs inode count odd

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24378 from smithfarm/wip-36135-luminous

luminous: mds: rctime may go back

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24382 from alfredodeza/luminous-rm36247

luminous ceph-volume: skip processing devices that don't exist when scanning system disks

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24347 from pdvian/wip-35981-luminous

luminous: ceph-disk: compatibility fix for python 3

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #24311 from batrick/i35838

luminous: mds: use monotonic clock for beacon message timekeeping

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24323 from smithfarm/wip-36133-luminous

luminous: client: update ctime when modifying file content

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24328 from smithfarm/wip-24912-luminous

luminous: qa: multifs requires 4 mds but gets only 2

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa: remove check using method from master

Not essential we check this and it breaks tests in Luminous.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24329 from smithfarm/wip-32103-luminous

luminous: mds: allows client to create .. and . dirents

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-volume util.disk when there are no devices mapped, skip to the next one

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9b0f472abadde26fce2a603fca5c466ebb770d4a)

ceph-volume tests.util verify devices that don't exist don't break get_devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0176c1965e6ed532b38dfee03af0e9e3a85149d6)

Merge pull request #24136 from gregsfortytwo/wip-luminous-make-check

luminous: build/ops: rpm: selinux-policy fixes

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #24342 from tchaikov/lumious-21769

luminous: osd/ECBackend: don't get result code of subchunk-read overwritten

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mds: prevent rctime from going back

Fixes: http://tracker.ceph.com/issues/35916
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 920ef964311a61fcc6c0d6671b77ffe98522863d)

Conflicts:
src/mds/Server.cc
- luminous does not increment or decrement pi.inode.rstat.rsnaps

client: set f_files to the total number of files in the filesystem

Fixes: http://tracker.ceph.com/issues/24849
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 39467a2c95815a495d75a9ced119975bfe62616c)

Conflicts:
src/client/Client.cc

qa: fix kcephfs/recovery suite

This is a luminous-only commit.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #24242 from jonsger/luminous-backport-pr#23596

luminous: rgw: incremental data sync uses truncated flag to detect end of listing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24358 from alfredodeza/luminous-rm36249

luminous ceph-volume:  activate option --auto-detect-objectstore respects --no-systemd

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

qa: add qa helper methods from master

For Luminous. This is needed by tests in this branch.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: whitelist cap revoke warning

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 4367de377e68102f3c17c8dd85321c221d06d9dd)

doc: document cap revoke non-responders client eviction

Fixes: http://tracker.ceph.com/issues/25188
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 28a52d6fa14425fc877a69055dabe4e7c00f6b14)

test: validate client eviction for cap revoke non-responders

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit c0b1dacc9f9c9b5af07a1b83a0adb53d001c2b79)

mds: add counter for tracking cap non-responding clients

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 8f2de92712a98568b0d07a795f1158868caae550)

Conflicts:
src/mds/Server.cc
src/mds/Server.h

mds: evict clients that do not respond to cap revoke by MDS

By default, preserve old behaviour. When configured with a non
default value, evict clients that have not responded to cap
revoke by MDS for the configured amount of seconds.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 4cf7815cdcd8efbbb981ef45b3eabee387b4de21)

Conflicts:
src/common/options.cc
src/mds/MDSDaemon.cc
src/mds/MDSRank.h
src/mds/Server.cc
src/mds/Server.h

mds: pass timeout argument for fetching late clients

This would be required when fetching clients that have not
responded to cap revoke by MDS for a configured timeout
value.

Additionally, make member functions private which are called
from the Locker class itself.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 005cf6c76f2d25ee88dd6ac9d0c67cfa88a9d58e)

Conflicts:
src/mds/Locker.cc

mds: use monotonic clock in beacon

Also update other parts of MDS which interact.

Fixes: http://tracker.ceph.com/issues/26959
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e77a2f5da71f1ba62b69b8d0fb9f34abe426da79)

Conflicts:
src/mds/Beacon.cc
src/mds/Beacon.h
src/mds/MDCache.h
src/mds/Server.cc

mds: simplify beacon init

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b7448f8fe0e4d12dd1070de4c3380e21d9da310a)

Conflicts:
src/mds/Beacon.cc
src/mds/Beacon.h

Merge pull request #24089 from batrick/i26990

luminous: mds: curate priority of perf counters sent to mgr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Venky Shankar <vshankar@redhat.com>

Merge pull request #24310 from batrick/i36210

luminous: mds: runs out of file descriptors after several respawns

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-volume lvm.activate pass no_systemd args when detecting objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 29de6820a9a7108c8de41036add9332f9b910359)

ceph-volume tests.lvm verify activate works with --auto-detect-objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 60dbeb6184797ccffeaeea2e85ef2d7fb4a1c54c)

Merge pull request #24335 from alfredodeza/luminous-rm36098

luminous ceph-volume:  lsblk can fail to find PARTLABEL, must fallback to blkid

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24070 from smithfarm/wip-25025-luminous

luminous: cls_rgw test is only run in rados suite: add it to rgw suite as well

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24299 from pdvian/wip-35962-luminous

luminous: osd/PG: avoid choose_acting picking want with > pool size items

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #24308 from neha-ojha/wip-36239-luminous

luminous: osd/PrimaryLogPG: fix potential pg-log overtrimming

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

ceph-disk: compatibility fix for python 3

In python 3, dev is a string, but mounts_dev is bytes (because
/proc/mounts was opened with mode 'rb') so they can't compare
equal, resulting in is_mounted() returning None for mounted OSDs.
The safest fix for this we could come up with was to normalize
dev to a str using _bytes2str() (just in case), and open
/proc/mounts in mode 'r', so its lines are interpreted as strs.

Fixes: https://tracker.ceph.com/issues/35906
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 8e3d948ff4c3b19dc55b9a5ae63f5b63f0c476a3)

osd/ECBackend: don't get result code of subchunk-read overwritten

Fixes: http://tracker.ceph.com/issues/21769
Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 4692f995ba6967796d5dd2f5fd85fde8705a1ccc)

Conflicts:
src/osd/ECBackend.cc:
          master contains changes from
          33d79f303d945c8d6d2e9e9a471efc4df57617a6, but luminous
          does, so we don't need to consider the case of fragmented
          read.

mds: reorganize and curate perf log counters

Categorize perf counters into high (PRIO_{CRITICAL/INTERESTING}),
medium (PRIO_USEFUL) and low priority (PRIO_DEBUGONLY). High and
medium priority stats are transmitted to ceph mgr and included in
prometheus output.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
Fixes: http://tracker.ceph.com/issues/24004
(cherry picked from commit 48a33f6f06802fee998f60a2ec35d9041a7c51a6)

Conflicts:
src/mds/MDSRank.cc

mds: show session load average in dump session

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 682dc08cbd521342b4abc45fd85edb2dd7bc5682)

Conflicts:
    src/mds/SessionMap.h

mds: allow request load average decay rate configurable via admin socket

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 3809d96a5b53721b88602d662c4c1cd00dd727a0)

Conflicts:
src/mds/MDSRank.h
src/mds/SessionMap.h
src/mds/SessionMap.cc

mds: decay counter for tracking request load average

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 816d8aaea01aef81467e3df7a4b3ab3eff86d27a)

Conflicts:
src/common/options.cc
src/mds/SessionMap.cc
src/mds/SessionMap.h

mds: add perf counters to track current open/stale sessions

Signed-off-by: Venky Shankar <vshankar@redhat.com>

mds:improve mds perf counters can statistics filesystem operations number and latency

Dumping the latency of all operations which will help we analysis of
what kind of operation is the most waste of time which can do a simple
performence analysis.

Fixes: http://tracker.ceph.com/issues/22097
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
(cherry picked from commit 20760f26bae7d9c599019a0295d66386e4281af2)

Merge PR #23553 into luminous

* refs/pull/23553/head:
mds: report lagginess at lower debug
MDSMonitor: note beacons and cluster changes at low dbg level

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Conflicts:
    src/mds/Beacon.cc

ceph-volume simple.scan better detection for ceph-disk data devices

Uses the new ``CephDiskDevice`` to look into PARTLABEL from both lsblk
(the default) falling back to blkid, which in some cases has the right
value when lsblk fails.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c502e5f64aa4f0d832c14680e1731e58d8a3770b)

ceph-volume tests.util verify CephDiskDevice behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8f072eb3f940a6714089b37a6fbacd549420a454)

ceph-volume tests.conftest add blkid patching to device_info fixture

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5bfc8c26e639dba955f83f53893ce311e6e047f1)

ceph-volume util.device add a ceph-disk device object to detect types

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 200e6bfd934b34aa8175330aef32c51505ec3392)

ceph-volume tests.util verify new blkid API calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ead3c7e7d19635a741a9da6fc330ec12a68b3d77)

ceph-volume util.disk expand blkid's API calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dbde6a5b611fdd4bcd942df5dbab94ab5076fdff)

Merge pull request #24083 from batrick/i26851

luminous: ceph_volume_client: py3 compatible

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24313 from batrick/i36198

luminous: ceph-fuse: add SELinux policy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24119 from ukernel/luminous-22504

luminous: client: drop null child dentries before try pruning inode's alias

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24138 from pdvian/wip-35718-luminous

luminous: mds: print is_laggy message once

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24268 from smithfarm/wip-36101-luminous

luminous: qa: remove knfs site from future releases

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #24285 from iridescent-rsy/luminous-35958

luminous: librbd: fixed assert when flattening clone with zero overlap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24303 from vshankar/wip-35931

luminous: client: retry remount on dcache invalidation failure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

*: set missing CLOEXEC on opened fds

Otherwise these descriptors may leak across execve() during e.g. MDS respawn.

Fixes: http://tracker.ceph.com/issues/35850
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 86dbd9e755ae0472db3d8c04bc444c502193dc13)

Conflicts:
CMakeLists.txt
src/auth/Crypto.cc
src/ceph_osd.cc
src/common/CMakeLists.txt
src/common/Preforker.h
src/common/SubProcess.cc
src/common/admin_socket.cc
src/common/admin_socket_client.cc
src/common/blkdev.cc
src/crimson/CMakeLists.txt
src/global/signal_handler.cc
src/include/random.h
src/include/uuid.h
src/log/Log.cc
src/mon/MonitorDBStore.h
src/msg/simple/Accepter.cc
src/os/bluestore/BlueStore.cc
src/os/bluestore/bluestore_tool.cc
src/os/filestore/FileStore.cc
src/os/kstore/KStore.cc
src/test/objectstore/test_bluefs.cc

Merge pull request #24338 from alfredodeza/luminous-rm36251

luminous ceph-volume add new ceph-handlers role from ceph-ansible

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests.functional add new ceph-handlers role from ceph-ansible

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 956dc5e65c6e35e27ab6524d4fb9fdd05d616cf6)

mds: access last std::vector element with `back()`

... rather than accessing it as an array. also convert
to range loops whereever necessary. plus some style
fixes.

Fixes: https://tracker.ceph.com/issues/25113
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f9b8ad5793d8265f131566233080613f475004aa)

mds: disallow certain file operations to "." and ".." dirents

Also, fixup return value for file operations such as rmdir()
and rename() on these directories.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit bae6c9db656afeff1dede6253300d22082313187)

Conflicts:
src/mds/Server.cc
src/test/libcephfs/test.cc

qa: use 4 mds cluster for multifs tests

Fixes: http://tracker.ceph.com/issues/24899
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3b7233a402ec68094ae86333c3f4f09706d173d1)

client: update ctime when modifying file content

Fixes: http://tracker.ceph.com/issues/35945
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 60f73cde4e04be2685eaf7cd04e03c045e1f9977)

Conflicts:
src/client/Client.cc
- slightly different arguments to filer->zero() function call

qa: fix mon config section

Fixes: fc6e683832125a1042638f21b623e29d0d4c34d8
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 47e60908a3c07fe262a50a8073dbd8e309e17261)

qa: ignore normal slow IO warnings for cephfs

Generally the slow warnings we get are just over the threshold. These warnings
are related to deploying multiple Ceph daemons side-by-side. Let's see how we
do with two minutes.

Ignoring the warnings entirely is unsatisfactory as they serve as a useful
canary in the coal mine when you see warnings for ops > some unreasonably large
amount of time.

Fixes: http://tracker.ceph.com/issues/26900
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit fc6e683832125a1042638f21b623e29d0d4c34d8)

qa: create common conf for all cephfs suites

This will be followed by removing common CephFS configurations in the
ceph.conf.template in teuthology.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 73fa0efcbb15a47157615705043288f2913ff3bc)

Conflicts:
qa/suites/fs/32bits/overrides/debug.yaml
qa/suites/fs/basic_functional/overrides/debug.yaml
qa/suites/fs/basic_workload/overrides/debug.yaml
qa/suites/fs/bugs/client_trim_caps/overrides/debug.yaml
qa/suites/fs/multiclient/overrides/debug.yaml
qa/suites/fs/multifs/overrides/debug.yaml
qa/suites/fs/permission/overrides/debug.yaml
qa/suites/fs/snaps/overrides/debug.yaml
qa/suites/fs/upgrade/snaps/conf
qa/suites/fs/thrash/overrides/debug.yaml
qa/suites/fs/traceless/overrides/debug.yaml
qa/suites/fs/verify/overrides/debug.yaml
qa/suites/kcephfs/cephfs/overrides/debug.yaml
qa/suites/kcephfs/mixed-clients/overrides/debug.yaml
qa/suites/kcephfs/recovery/overrides/debug.yaml
qa/suites/kcephfs/thrash/overrides/debug.yaml

ceph-fuse: add to selinux profile

Fixes: http://tracker.ceph.com/issues/36103
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 2bfaac762d283a0e000074f79f704033afd6dead)

mds: report lagginess at lower debug

To help diagnose laggy daemons with performance tuned debug settings.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c8aff620c273fd4579b23e1647450b6f00c6bac7)

Conflicts:
src/mds/Beacon.cc

MDSMonitor: note beacons and cluster changes at low dbg level

These messages are essential for diagnosing the reason why the MDSMonitor is
kicking MDSs out of the MDSMap. They should also be rare enough that the extra
verbosity is not noticable.

Fixes: http://tracker.ceph.com/issues/26898
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d6f6b4f72aae3c869ed7ee40dc737bb2e2323d3c)

Conflicts:
src/mon/MDSMonitor.cc

ceph_volume_client: use integer division for pg_num

Otherwise a float is sent to the manager which is not the invalid format.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 33910303cb4940ca90cd76a5b4adb4de9c88d04c)

packages.yaml: install py3 compatible cephfs and rados bindings...

on Ubuntu and CentOS for upstream testing.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit c1efd59f618e24cf060d564ac0f21d5b0b57fd4a)

qa: test py2 and py3 for volume client

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit a5e3b0512c8c4e5ffbd960b903f40871d1867601)

qa: make test_volume_client.py py3 compatible

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit f28274dc70aa102e3c4523059a65e5da8c8a0426)

qa/ceph-volume: allow executing python pyloads using python3

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 2f35ade05f9910ef9bdaaa24de9593bd584073fa)

Conflicts:
qa/tasks/vstart_runner.py

cephfs.pyx: add py3 compatibility

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 5106582fc7edae7f39161cf89e566c020fcfa0ce)

ceph-volume-client: add py3 compatibility

Fixes: http://tracker.ceph.com/issues/17230
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit efdf1af41d198565c2993598b8e345683396b169)

Conflicts:
src/pybind/ceph_volume_client.py

Merge pull request #24084 from batrick/i24862

luminous: ceph_volume_client: allow atomic update of RADOS objects

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #24282 from pdvian/wip-35933-luminous

luminous:client:two ceph-fuse client, one can not list out files created by an…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24188 from pdvian/wip-35844-luminous

luminous: osdc/Objecter: fix split vs reconnect race

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24245 from dillaman/wip-36116-luminous

luminous: test/librbd: not valid to have different parents between image snapshots

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #24257 from pdvian/wip-35929-luminous

luminous: mon/OSDMonitor: invalidate max_failed_since on cancel_report

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #24259 from tchaikov/luminous-35969

luminous: rpm: use updated gperftools

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

msg: set O_NONBLOCK on file status flags

There is no NONBLOCK file **descriptor** flag.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7e91c8b5aa4548e15c01702f24aaf1f2fccf0468)

Conflicts:
src/msg/simple/Accepter.cc

Merge pull request #24307 from alfredodeza/luminous-batch-report-fix

luminous ceph-volume: fix journal and filestore data size in `lvm batch --report`

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

osd/PrimaryLogPG: fix potential pg-log overtrimming

In https://github.com/ceph/ceph/pull/21580 I set a trap to catch some wired
and random segmentfaults and in a recent QA run I was able to observe it was
successfully triggered by one of the test case, see:

```
http://qa-proxy.ceph.com/teuthology/xxg-2018-07-30_05:25:06-rados-wip-hb-peers-distro-basic-smithi/2837916/teuthology.log
```

The root cause is that there might be holes on log versions, thus the
approx_size() method should (almost) always overestimate the actual number of log entries.
As a result, we might be at the risk of overtrimming log entries.

https://github.com/ceph/ceph/pull/18338 reveals a probably easier way
to fix the above problem but unfortunately it also can cause big performance regression
and hence comes this pr..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3654d56985c67d15506fa37b56ef5b0c04e01a65)

Conflicts:
src/osd/PrimaryLogPG.cc: trivial resolution

ceph-volume: only run ceph-common once during tests

Using import_role we can have just one play and avoid rerunning
ceph-common over and over again in daemon specific plays.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a81386123e66e2d61391371cec4485b3148ff5a7)

ceph-volume: fail fast during tests

This also rsyncs the ceph-volume code to the testing vms before
a ceph.conf is generated because ceph-volume is needed now for this
to figure out the number of osds when you're using 'lvm batch'.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit eed244473d11a153cd7a37e0e9bfaf804a20db89)

ceph-volume: when using batch --report return journal/data size as ints

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a5e8bc97e70195cee54c4b66d10684d6cd3f5e38)

Merge pull request #24280 from alfredodeza/luminous-rm27210

luminous ceph-volume custom cluster names fail on filestore trigger

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24211 from pdvian/wip-35854-luminous

luminous: doc: remove deprecated 'scrubq' from ceph(8)

Reviewed-by: Nathan Cutler <ncutler@suse.com>

qa/suites/rgw/verify: drop ragweed.yaml

There is no ragweed task in luminous.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

client: retry remount on dcache invalidation failure

For some (unknown) reason, there have been reports of
ceph-fuse crash due to failure is remounting at the
time of invalidating kernel dentry cache. This issue
is not also reproducible yet.

Therefore, as suggested by Patrick and Zheng, for a
temporary workaround, client would ignore the failure
as the invalidation would be retried again. There is
a max cap on the number of consecutive remount failures
after which client would abort.

Fixes: http://tracker.ceph.com/issues/35931
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit d1471f070cd1ad9c0f773e00d2552161d1ad5955)

 Conflicts:
src/client/Client.cc
src/client/Client.h
src/common/options.cc

rgw: fix up indentation in incremental data sync

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 666bfe3a909ec83a3067398e7896527f99283e0a)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
Conflicts:
src/rgw/rgw_data_sync.cc: ident regarding current state

rgw: remove unused query for remote datalog info

Fixes: http://tracker.ceph.com/issues/26952
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a543df729a6ca415648678e2ae2ed46eceaba086)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
Conflicts:
src/rgw/rgw_data_sync.cc: adapt to ldout, as there is no tn->log

rgw: remove unused datalog trim detection logic

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0d47743e0f6b9f51848056885831910d0ea5b282)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
Conflicts:
src/rgw/rgw_data_sync.cc: trivial