mgr/rbd_support: rename "rbd_trash_trash_purge_schedule" oid

to "rbd_trash_purge_schedule"

Fixes: https://tracker.ceph.com/issues/45589
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge branch 'wip-joshd-fix'

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35015 from smithfarm/wip-45514

doc: drop obsolete cache tier options

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35083 from p-se/wip-pse-cephadm-prom-scrape-interval

mgr/cephadm: increase global scrape interval

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35092 from varshar16/wip-cephadm-mdcache-ganesha

 mgr/cephadm/nfs: Instead of CACHEINODE use MDCACHE

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34964 from dillaman/wip-45137

librbd: replace ImageRequestWQ with pluggable ImageDispatcher

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/cephadm: increase global scrape interval

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

Merge pull request #32682 from lorenzen-b1/patch-1

doc/architecture.rst: clarify scrubbing

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35091 from tchaikov/wip-crimson-cleanup

crimson/osd: avoid using variadic future

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #35081 into master

* refs/pull/35081/head:
qa: do not save/restore ip_forward sysctl

Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #34764 from ianwatsonrh/master

Docs: vault kv and transit engine updates

Reviewed-by: Casey Bodley <cbodley@redhat.com>

 mgr/cephadm/nfs: Instead of CACHEINODE use MDCACHE

As CACHEINODE will be deprecated soon.

Signed-off-by: Varsha Rao <varao@redhat.com>

crimson/osd: avoid using variadic future

it is deprecated by seastar. let's use future<tuple<...>> instead

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35025 from dillaman/wip-45518

librbd: copy API should not inherit v1 image format by default

Merge pull request #34789 from trociny/wip-rbd-nbd-quiesce

rbd-nbd: add quiesce/unquiesce hooks

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35023 from yuvalif/fix_issue_45513

cls/2pc_queue: fix timing issue in test

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34959 from yison/rwl-image-writeback-cache-seq10

rbd/cache: Replicated Write Log core codes - load existing cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34930 from trociny/wip-45409

rbd-mirror: wait for events to replay before shut down journal replay

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35042 from tchaikov/wip-cmake-remove-iwyu-cppcheck

cmake: remove cppcheck and iwyu targets

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #35070 from Devp00l/wip-45545-2

mgr/dashboard: Updates jest to fix cache failure

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34913 from athanatos/sjust/wip-seastore-journal

crimson/os/seastore: initial PR

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35052 from tspmelo/wip-unittest-warnings

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #35013 from matthewoliver/cephadm-iscsi-log-to-stderr

cephadm: enable logging to stderr in ceph-iscsi

Reviewed-by: Michael Fritch <mfritch@suse.com>

dec/dev/seastore.rst: add more detailed design information and status

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore: add initial cache support with tests

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore: introduce initial journal implementation and tests

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/gtest_seastar: add helpers for building seastar gtest unit tests

Also adds unsafe_get methods for errorator for use within unit tests.

Signed-off-by: Samuel Just <sjust@redhat.com>

qa: do not save/restore ip_forward sysctl

We don't really care if this is left on during the course of the mount.
These settings don't persist across tests anyway.

Fixes: https://tracker.ceph.com/issues/45525
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35077 from yuriw/wip-yuriw-crontab-master

qa/tests: reduced powercycle jobs amount

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: reduced powercycle jobs amount

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #32829 from Devp00l/wip-43765

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #35076 from yuriw/wip-yuriw-distro-master

qa/tests: run `nautilus-x` on `pacific` on `ubuntu 18.04`

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: run `nautilus-x` on `pacific` on `ubuntu 18.04`

...as we don't build `nauiltus` on `20.04`

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

librbd: QoS delayed dispatch needs to use op work queue

The QoS throttle will invoke its throttle complete callback within
the scope of the ImageCtx timer lock singleton. This would cause
the potential for deadlock if a future dispatch layer attempted
to acquire the same timer lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: corrected naming of singleton timer locker

It incorrectly included "Journal" in the name.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: correct misnamed parameters in io::ImageDispatcher

It was incorrectly using to object dispatch names instead of
image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: api::Io now properly passing the parent trace information

A copy-and-paste error from moving the original implentation from
ImageRequestWQ resulted in the parent trace being incorrectly
uninitialized.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32829 from Devp00l/wip-43765

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

librbd: ensure in-flight read-ahead + copy-on-read ops are finalized

Now that the image dispatcher isn't flushing AsyncOperations with each
flush request, we need to ensure that we flush all background ops prior
to fully closing the image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: remove io::ImageRequestWQ

This has been fully deprecated in favor of the new plugable image
dispatcher system.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: switch to new api::Io dispatch helper methods

This is another step in the removal of ImageRequestWQ in favor
of the new ImageDispatcher.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: created new api::Io helper methods

These will be invoked by the user API and wrap various user API
specific logic.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: exclusive-lock dispatch flush pseudo-handling

Ensure that the exclusive-lock layer tracks all in-flight IOs so that
it can flush the IOs from lower layers when the lock becomes required.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: qos dispatch flush pseudo-handling

Flush requests will be properly queued behind write operations to
ensure data consistency.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: generic helper for tracking in-flight IO and managing flush requests

Layers that potentially queue IOs but not all IOs will need to track all
in-flight IOs to properly ensure that a flush cannot complete while
older IO is still stuck in a queue.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: temporarily redirect ImageRequestWQ IO through ImageDispatcher

The next series of commits will work to remove ImageRequestWQ from librbd.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: complete ImageDispatchSpec upon AioCompletion finalization

This will allow ImageDispatchSpec to invoke any necessary finalizers
for the different dispatch layers and allows in-flight IO to be tracked
since the image dispatch layer finalizers won't be invoked until after
the AioCompletion fires.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: switched IO (un)block_write calls to new queue image dispatcher

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: queue image IO dispatch layer

This layer replaces the queue implemented by ImageRequestWQ and handles
blocking all write requests through the dispatch pipeline.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: exclusive lock image IO dispatch layer

This layer will handle acquiring the exclusive lock if required upon
incoming IO requests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: refresh image IO dispatch layer

This layer will handle refreshing the image if required upon incoming
IO requests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: QoS image io dispatch layer

Re-implement the existing QoS throttling behavior from ImageRequestWQ
as a new image IO dispatch layer. The existing QoS hooks in
ImageRequestWQ are temporarily hooked to the new layer.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: support a finish handler for image IO dispatch layers

This will be faster than allocating per-IO Context callbacks for
layers that need to track the in-flight state of particular IOs.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: init and shut down image IO dispatch layer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: initial implementation for image IO dispatcher

This is similar to the object IO dispatcher but it expects image extents
for IO operations.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add image dispatch layer to image dispatch spec

This will allow image extent-based IOs to skip layers. Also added helpful
enums for the start of user-based API requests and internal requests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: added io::ImageDispatchSpec::C_Dispatcher

This will assist with dispatching image extent-based IO requests
through the image IO dispatcher system (to be added in a later commit).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: move generic IO dispatcher logic to base class

This will allow re-use between the existing ObjectDispatcher and
the ImageDispatcher that will be added in a future commit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common: TokenBucketThrottle should use perfect forwarding for item

This will allow more than just a pointer to an item to be passed to
the callback.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Updates jest to fix cache failure

Fixes: https://tracker.ceph.com/issues/45545
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #31889 from ofriedma/wip-ofriedma-reshard-debug

rgw: Fix reshard log so we can know why the reshard is returning error

Merge pull request #34003 from AmnonHanuhov/wip-radosbench_4k_rw_yaml

test/crimson: Added CPU cycles per operation as an acceptance criteria to radosbench_4k_r/w yaml

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34981 from sebastian-philipp/doc-dev-cephadm-cuncurrency

doc/dev/cephadm: Note regarding network calls form CLI handlers

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #35066 from rzarzynski/wip-crimson-omap_get_keys-num

crimson/osd: fix key num handling in PGBackend::omap_get_keys().

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34668 from zdover23/wip-doc-ceph-orch-zap-45128

doc/ceph orch: Add ceph orch zap command

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #34944 from jschmid1/fix_osdspec_affinity_passing

mgr/cephadm: fix env_var passing to cephadm

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35030 from mgfritch/cephadm-shell-mount-special

cephadm: fix shell mount for special files

Reviewed-by: Matthew Oliver <moliver@suse.com>

crimson/osd: fix key num handling in PGBackend::omap_get_keys().

The patch fixes running-out-of-buffer when decoding the data
from `omap_get_keys()`. In some cases, the declared number
of encoded keys is greater by 1 from the actual causing execption
`buffer::end_of_buffer` to be thrown at decoder.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #35011 from dillaman/wip-45504

pybind/rbd: RBD.create() method's 'old_format' parameter now defaults to False

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #35022 from sebastian-philipp/cephadm-add-host-oserror

mgr/cephadm: Remove AsyncCompletion

Reviewed-by: Michael Fritch <mfritch@suse.com>

PendingReleaseNotes: note about security fix

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mgr: require all caps for pre-octopus tell commands

This matches the requirements for admin socket commands
sent via tell elsewhere.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mon: enforce caps for pre-octopus client tell commands

This affects only the commands whitelisted here - in particular
injectargs requires write access to the monitors.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34976 from changchengx/bufferlist_ut

test/bufferlist: fix and cleanups

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35056 from agayev/fix-do-make

do_cmake.sh: do not unnecessarily warn about debug build

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35029 from yuriw/wip-yuriw-subsets-master

qa/tests: reduce number of jobs for rados to 309

do_cmake.sh: do not unnecessarily warn about debug build

do_cmake.sh forgets to check the arguments passed to it and ends up warning
about debug build even if a user specifies a release build.

Signed-off-by: Abutalib Aghayev <agayev@gmail.com>

Merge pull request #34969 from pritha-srivastava/wip-rgw-gc-fix

cls/rgw_gc: Fixing the iterator used to access urgent data map

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Remove unit tests warnings

Fixes: https://tracker.ceph.com/issues/45536
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34965 from Devp00l/wip-45433

mgr/dashboard: Always use fast angular unit tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35041 from smithfarm/wip-tcmalloc-nit

doc: fix TCMalloc spelling

Reviewed-by: Zac Dover <zac.dover@gmail.com>

cmake: remove cppcheck and iwyu targets

these targets are not used by any other targets, and are not integrated with CI.
so let's drop them.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33121 from tspmelo/wip-angular-9

mgr/dashboard: Update to Angular 9

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mon: drop obsolete cache-tier options from contextual help

Fixes: https://tracker.ceph.com/issues/45514
Signed-off-by: Nathan Cutler <ncutler@suse.com>

rbd-nbd: don't abuse rbd_cache_writethrough_until_flush

If rbd_cache and rbd_cache_writethrough_until_flush options are
set, do not use flush for internal purposes unless the user
issued flush, so it won't unintentionally enable writeback cache
when it is still unsafe.

Signed-off-by: Mykola Golub <mgolub@suse.com>

doc: fix TCMalloc spelling

The google/tcmalloc page on GitHub refers to "TCMalloc", not
"tc_malloc".

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35010 from pcuzner/k8sevents-sanitise

mgr/k8sevents:sanitise kubernetes events

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34972 from tspmelo/wip-fix-45445

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #33886 from sebastian-philipp/doc-mon-command-api

doc: Add a generated reference of all mon commands.

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #35017 from tspmelo/wip-fix-fakefs

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34956 from tchaikov/wip-qa-tasks-cephfs-py3

qa/tasks/cephfs: remove sys.exc_clear() calls

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

librbd: load existing cache

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

Merge pull request #34857 from tchaikov/wip-qa-tasks-no-filter

qa/tasks: use list comprehension for checking the length

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

cephadm: enable logging to stderr in ceph-iscsi

All the other ceph servies when deployed are deployed with:

  --default-log-to-stderr=true --default-log-stderr-prefix=debug
  --default-log-to-file=false

Ceph-iscsi doesn't have these as commandline params but once
https://github.com/ceph/ceph-iscsi/pull/186 lands it'll have something
similar as config options. This patch sets these options

Signed-off-by: Matthew Oliver <moliver@suse.com>

mgr/dashboard: Add dashboard node script

This file should be used to agregate all external scripts we need.
Multiple flags can be used in the same call.

Available flags:

--env: Generates angular environment files.

--pre: Modifies 'angular.json' to enable the build of custom locales using
       angular --localize.
       Languages can be defined using the environment variable LANGS,
       if no value is provided all languages will be build.
       Default language is always build, even if not provided.
       p.e.: 'LANGS="pt" node cd --pre', will build EN and PT.
       For backward compatibility we accept both long and short version of
       languages, p.e.: 'pt' and 'pt-BR'

--post: Restores 'angular.json' to its original.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove ngcc warnings

Some packages are still not updated to be compatible with Angular 9
and can cause warnings.

I have whitelisted some that work well, so we don't see the warning every time.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update Node.js to latest LTS: v12.16.2

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update npm dependencies

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix SCSS imports

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix i18n configuration

Signed-off-by: Tiago Melo <tmelo@suse.com>