install-deps.sh: Typo for krb5 on FreeBSD

The FreeBSD dependency install specifies devel/kbr5 instead of devel/krb5

Fixes: https://tracker.ceph.com/issues/40050
Signed-off-by: Thomas Johnson <NTmatter@gmail.com>

Merge PR #28058 into master

* refs/pull/28058/head:
mgr/zabbix: Fix raw_bytes_used key name

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26967 from qwren/qat-accel-doc

doc: QAT Acceleration for Encryption and Compression

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28099 from tchaikov/wip-blobhash

blobhash: do not use cast for unaligned access

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #28247 from tchaikov/wip-disable-asan-if-notfound

ceph.in: disable ASAN if libasan is not found

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Optimize iSCSI target edition (#27272)

mgr/dashboard: Optimize iSCSI target edition

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

blobhash: do not use cast for unaligned access

* remove the uncorrect comment. as std::hash<> does not apply to a
  customized type. see https://en.cppreference.com/w/cpp/utility/hash
* do not use cast for accessing an uint32_t by dereferencing (void *)
  or (char *) pointer. because the alignment requirement of `uint32_t`
  is stricter than that of `void*` or `char *`. we need to do an
  explicit memcpy() for accessing the uint32_t pointed by the pointer.
  see also https://www.kernel.org/doc/Documentation/unaligned-memory-access.txt
* instead of using a loop for mixing the last few bytes. use a switch-
  case. so GCC-9 won't complain with
```
../src/include/blobhash.h:38:10: warning: iteration 4 invokes undefined
behavior [-Waggressive-loop-optimizations]
   38 |       sh += 8;
      |       ~~~^~~~
../src/include/blobhash.h:36:12: note: within this loop
   36 |     while (len) {
      |            ^~~
```
  at seeing `sh += 8;`
* instead of mixing the last bits repeatly by derefencing `p` using
  `uint32_t`, while move it forwards with step size of `1`, mix
  the bits byte-wise.
* include <cstdint>. this header file is supposed to be self-contained.
* use `std::uint32_t` instead of using `uint32_t`. we cannot assume
  somebody is `using namespace std` or `using std::uint32_t` for us.
* mark the operator() `const noexcept`. see
  https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#c89-make-a-hash-noexcept
  for the rationale behind this

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27569 from batrick/gcc-9-wrn

CMakeLists: add std::move warnings in gcc9

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Add clients tab to NFS details (#28193)

mgr/dashboard: Add clients tab to NFS details

Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/zabbix: Fix raw_bytes_used key name

This patch fixes raw_bytes_used key which was renamed to stored_raw.
Also added key percent_used and fixed zabbix template to be fully
compatible with zabbix 3.0

Fixes: https://tracker.ceph.com/issues/39644
Signed-off-by: Dmitriy Rabotjagov <noonedeadpunk@ya.ru>

mgr/dashboard: Unify remaining action labels (#28162)

mgr/dashboard: Unify remaining action labels

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #27436 from simon-rock/simon_work

mds: avoid sending too many osd requests at once after mds restarts

Merge pull request #28224 from jan--f/pool-ls-detail-ec-profile

osd_types: add ec profile to plain text osd pool ls detail output

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28159 from batrick/i39599

common: fix parse_env nullptr deref

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28057 from wido/zabbix-pg-state

mgr/zabbix: Fix typo in key name for PGs in backfill_wait state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28025 from markhpc/wip-bs-aio-preadv

os/bluestore: Unify on preadv for io_uring and future refactor.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28010 from ifed01/wip-ifed-dump-large-onode

os/bluestore: dump onode that has too many spanning blobs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26341 from sebastian-philipp/dashboard-merge-orchclient

mgr/dashboard: Fix iSCSI + Rook issues

Reviewed-by: Ricardo Marques <rimarques@suse.com>

mds: avoid sending too many osd requests at once after mds restarts
Fixes: http://tracker.ceph.com/issues/40028
Signed-off-by: simon gao <simon29rock@gmail.com>

mgr/dashboard: Subscribe to changes when RequiredIf is used (#28231)

mgr/dashboard: Subscribe to changes when RequiredIf is used

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #28078 from sebastian-philipp/kubejacker-fix-mgr-plugins

scripts/kubejacker: Fix mgr_plugins target for centos

Reviewed-by: Tim Serong <tserong@suse.com>

ceph.in: disable ASAN if libasan is not found

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28155 from alimaredia/wip-remove-mon-valgrind-check-in-rgw-suite

qa: remove mon valgrind check in rgw verfiy suite

mgr/dashboard: Subscribe to changes when RequiredIf is used

FormControls that use 'requiredIf' validators depend on the state of other
controls.
To prevent that some of the state changes are not tracked, we now automatically
subscribe to the prerequsited controls and update the validation of the parent
control.

Fixes: https://tracker.ceph.com/issues/40017
Signed-off-by: Tiago Melo <tmelo@suse.com>

osd_types: add ec profile to plain text osd pool ls detail output

Fixes: http://tracker.ceph.com/issues/40009
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

mgr/dashboard: Unify the labels on all table actions objects

Fixes: http://tracker.ceph.com/issues/39102
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Unify action naming for NFS

Fixes: http://tracker.ceph.com/issues/39102
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Bug Fix Error on Mgr Dashboard NFS (#28075)

mgr/dashboard: Bug Fix Error on Mgr Dashboard NFS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

common: fix parse_env nullptr deref

Fixes: https://tracker.ceph.com/issues/39599
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

test: add test for parse_env

Test for https://tracker.ceph.com/issues/39599

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28169 from bk201/wip-39971

mgr/dashboard: fix some performance data are not displayed

Reviewed-by: Kanika Murarka <kmurarka@redhat.com>

Merge pull request #28181 from smithfarm/wip-39981

do_cmake.sh: use bash

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #28093 from dillaman/wip-39703

librbd: use custom allocator for aligned boost::lockfree::queue

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/rook: Raise proper exception in `describe_service`

`describe_service` is called often and raising
  assertions will pollute the log file with tracebacks.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Load iSCSI config from local store and orchestrator

As iSCSI is not supported by Rook, users need a way to manage
  ceph-iscsi without Rook knowledge

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Merge OrchClient with OrchestratorClientMixin

Fixed missing error propagation from
  the orchestrator to the dashboard

Adapted orchestrator integration in NFS

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #28147 from bk201/wip-39933

mgr/dashboard: fix backend error when updating RBD interlocked features

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Add clients tab to NFS details

Fixes: https://tracker.ceph.com/issues/39963
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix tcmu-runner perf counters page (#28154)

mgr/dashboard: Fix tcmu-runner perf counters page

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26518 from linuxbox2/wip-rgw-lc-prefix-2

rgw: lifecycle: alternate solution to prefix_map conflict

rgw: lifecycle: fix object iteration for >1 same-prefix rules

Rules of like-prefix must visit every matching object.  A next
logical step is to combine tag checking for rules of like-prefix.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: lifecycle: fix RGWLifecycleConfiguration::valid()

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: lifecycle: alternate solution to prefix_map conflict

A rule containing a tag filter but no constraining prefix is
legal, as is a sequence of >1 such rules.

As currently designed, enforcement requires rules to be
installed in prefix_map as well as rule_map, though only the
latter is encoded.

Fixes: http://tracker.ceph.com/issues/37879
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

do_cmake.sh: use bash

Fixes: http://tracker.ceph.com/issues/39981
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27996 from jan--f/package-prometheus-alerts-suse

ceph.spec.in: package prometheus default alerts for SUSE

Reviewed-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: fix some performance data are not displayed

Several Grafana dashboards were updated and their uids are changed.
Update corresponding uids in angular templates. Also fix that when a user
clicks performance details of an OSD, wrong OSD metrics are displayed.

Fixes: http://tracker.ceph.com/issues/39971
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #28094 from tchaikov/wip-install-deps-gcc-8

install-deps.sh: use gcc-8 on xenial and trusty

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28079 from tchaikov/wip-crimson-rados-cleanup

crimson: support pgnls and delete op

Reviewed-off-by: Samuel Just <sjust@redhat.com>

qa: remove mon valgrind check in rgw verfiy suite

memory leaks in the monitor are causing a significant
percentage of jobs run in the rgw verify suite to
fail even though the jobs succeeded before hand.

See: http://tracker.ceph.com/issues/38827 for
root cause

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #27653 from tspmelo/wip-nfs-pseudo

mgr/dashboard: Improve NFS Pseudo pattern message

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #28023 from rjfd/wip-fix-ceph-mgr-systemd

systemd: ceph-mgr: set MemoryDenyWriteExecute to false

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #28044 from dillaman/wip-librbd-io-perf

librbd: tweaks to increase IOPS and reduce CPU usage

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28067 from yuriw/wip-yuriw-crontab-master

qa/tests: added new client-upgrade-*-nautilus suites for jewel, luminous, mimic

mgr/dashboard: Fix tcmu-runner perf counters page

Fixes: https://tracker.ceph.com/issues/39954
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #27681 from tspmelo/wip-charts

mgr/dashboard: Unify the look of dashboard charts

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #28113 from hjwsm1989/fix-krb-compile-error

auth/krb: fix Kerberos compile error

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: fix backend error when updating RBD interlocked features

Now object-map and fast-diff features are interlocked. Allow
enabling/disabling these two features together without raising an already
enabled/disabled error.

Fixes: http://tracker.ceph.com/issues/39933
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

mgr/dashboard: Bug Fix Error on Mgr Dashboard NFS

This change fixes error on NFS Ganesha management dashboard that throws an error when request daemon list.

Fixes: https://tracker.ceph.com/issues/39709
Signed-off-by: Nur Faizin <nur.faizin91@gmail.com>

auth/krb: fix Kerberos compile error

Fixes: http://tracker.ceph.com/issues/39948
Signed-off-by: huangjun <huangjun@xsky.com>

Merge PR #28080 into master

* refs/pull/28080/head:
cephfs-shell: Remove whitespace after '('
cephfs-shell: Remove unused 'flag' variable
cephfs-shell: Fix flake8 invalid escape sequence warning
cephfs-shell: Remove unnecessary empty lines
cephfs-shell: Fix flake8 bare 'except' warnings

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27822 into master

* refs/pull/27822/head:
cephfs-shell: Fix mkdir relative path error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/tests: added new client-upgrade-*-nautilus suites for jewel, luminous, mimic

NOTE:

!!!! three suites below MUST use --suite-branch jewel, luminous, mimic (see https://tracker.ceph.com/issues/24021)
ref: https://github.com/ceph/ceph/pull/27983; https://github.com/ceph/ceph/pull/27934; https://github.com/ceph/ceph/pull/28027
--filter "ubuntu_16.04,centos_7.6,rhel_7.6" - test ONLY supported distro BEFORE mimic
--filter "ubuntu_16.04,ubuntu_18.04,centos_7.6,rhel_7.6" - test ONLY supported distro AFTER mimic

Merge after https://github.com/ceph/ceph/pull/28027

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

librbd: use custom allocator for aligned boost::lockfree::queue

If tcmalloc is in-use as the allocator and its version is less than 2.6.2,
it might be missing support for 'aligned_alloc'. This can result in the
glibc version of 'aligned_alloc' being used to allocate memory that is
then freed by tcmalloc -- resulting in a crash.

Fixes: http://tracker.ceph.com/issues/39703
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Unify the look of dashboard charts

Fixes: http://tracker.ceph.com/issues/39384
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28092 from tchaikov/wip-dashboard-package

mgr/dashboard: Update tar to v4.4.8

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28043 from jan--f/update-grafana-dashboards

Update grafana dashboards

Reviewed-by: kmurarka@redhat.com
Reviewed-by: epuertat@redhat.com

Merge PR #27377 into master

* refs/pull/27377/head:
qa: fs Ignore getfattr errors for ceph.dir.pin

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27594 into master

* refs/pull/27594/head:
mgr/volumes: allow creation/deletion of FS subvolume group snapshots
mgr/volumes: allow creation/deletion of FS subvolume groups
mgr/volumes: allow creation/deletion of snapshots of FS subvolumes
mgr/volumes: allow fetching path of FS subvolumes
mgr/volumes: use the fs_subvolume module
mgr/volumes: add fs_subvolume module

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27483 into master

* refs/pull/27483/head:
cephfs-shell: add quota management

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27753 into master

* refs/pull/27753/head:
cephfs-shell: Add stat command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27872 into master

* refs/pull/27872/head:
cephfs-shell: teuthology tests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

install-deps.sh: use gcc-8 on xenial and trusty

to be consistent with #27183 and https://github.com/ceph/ceph-build/pull/1286

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/volumes: allow creation/deletion of FS subvolume group snapshots

... using `ceph fs subvolumegroup snapshot create/rm` commands.

Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volumes: allow creation/deletion of FS subvolume groups

... using `ceph fs subvolumegroup create/rm` commands.
FS subvolume groups are parent directories of FS subvolumes. They
can be directly mapped to OpenStack Manila share groups.

Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volumes: allow creation/deletion of snapshots of FS subvolumes

... using `ceph fs subvolume snapshot create/rm` commands.

Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volumes: allow fetching path of FS subvolumes

... using `ceph fs subvolume getpath` command.

Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volumes: use the fs_subvolume module

... instead of src/pybind/ceph_volume_client.py to manage CephFS
subvolumes using `ceph fs subvolume` set of commands.

Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volumes: add fs_subvolume module

fs_subvolume module provisions and manages CephFS subvolumes, which are
CephFS subdirectories with a desired layout and quota. Its code is
heavily borrowed from, src/pybind/ceph_volume_client.py

Fixes: http://tracker.ceph.com/issues/39610
Signed-off-by: Ramana Raja <rraja@redhat.com>

monitoring: update Grafana dashboards

Fix various panels that used outdated metric names, cluncky or
unnecessary label_replace calls. Also unify the style of many panels.

Fixes: http://tracker.ceph.com/issues/39652
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

mgr/dashboard: Update tar to v4.4.8

This update fixes an Arbitrary File Overwrite issue (CVE-2018-20834).
see https://nvd.nist.gov/vuln/detail/CVE-2018-20834

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/zabbix: Fix typo in key name for PGs in backfill_wait state

Fixes: http://tracker.ceph.com/issues/39666
Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #28085 from rzarzynski/wip-crimson-no-atomics-in-refcntobj

crimson, common: RefCountedObj doesn't use atomics in Seastar builds.

Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: avoid bl manipulation on single object read/writes

If a read or write only spans a single object and buffer extent,
there isn't any need to build a new bufferlist from a full sub-bl.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: switch to lightweight striper for hot IO path

Optimize non-complex IO requests using the new lightweight object
striper that avoids heap allocations for small requests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: drop data oid parameter from IO path

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: helper method to generate data object OIDs on-demand

The new generator avoids multiple string length and copy operations
that are currently required in the Striper.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osdc/Striper: implemented a lightweight version of StripedReadResult

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osdc/Striper: implemented a lightweight version of file_to_extents

The previous versions of file_to_extents resulted in numerous heap
allocations for maps, vectors, and strings -- in addition to using
a CPU inefficient oid name formater.

librbd doesn't require the use of object_t and it would be better to
avoid any heap allocations on the IO path for the common case.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common/Finisher: avoid memory re-allocations for finisher queue

Since there is only a single thread associated with the Finisher,
swap between two queues to avoid the need to re-allocate a new
vector for each iteration through the loop. Also replace the
condition broadcast with a signal since there is only a single
thread to wake up.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27999 from trociny/wip-rbd-sparse_copyup

librbd: clone copy-on-write operations should preserve sparseness

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

crimson/osd: add crimson_debug_pg_always_active option

before crimson is able to peer, we should leave this option ON.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add preliminary delete support

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add pgnls support

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28071 from tchaikov/wip-crimson-perf-regres

crimson,osd: performance fixes

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson, common: RefCountedObj doesn't use atomics in SeaStar builds.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: Improve NFS Pseudo pattern message

Fixes: http://tracker.ceph.com/issues/39327
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #27270 from theanalyst/beast-fe-ipv6

rgw: normalize v6 endpoint behaviour for the beast frontend

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #26281 from smithfarm/wip-git-describe-long

make-dist: set version number only once

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #28045 from cbodley/wip-39635

cls/user: cls_user_set_buckets_info overwrites creation_time

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #27980 from cbodley/wip-24991

rgw: delete_obj_index() takes mtime for bilog

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>