rbd: rename some MapOptions instances to unmap_options

It is no longer a global variable, so rename the ones that have to do
with options for unmapping (currently just "force").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit da4ffd834fb8326ca3c3e096de4024198c8f42f2)

Conflicts:
src/tools/rbd/action/Kernel.cc (backported unmap "udev" option)

rbd: fix default pool handling for krbd map/unmap

The default pool name does not get passed to the kernel since commit
96f05a7956b3 ("rbd: delay determination of default pool name").  The
kernel ends up interpreting the image name as the pool name (and the
snapshot name as the image name).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit c06338f33c9c8535562c99b6f2b779fbedbba058)

rbd: delay parsing of default kernel map options

It now connects to the cluster to retrieve any global config
overrides before processing the default map options.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 34f539d8af337ea5d02172bfeadaee578359e63b)

Conflicts:
src/tools/rbd/action/Kernel.cc
                (backported "ms_mode" and "udev" option, no quesce options)

rbd: 'list' action should connect to cluster before querying config

This will allow global config overrides for the max concurrent ops to
be applied prior to listing the images.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1527dcfa6d57a1644fa569a3270cb72f058fae44)

rbd: drop querying of RBD default features

These will be computed internally by librbd using any potential
global configuration overrides.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 596d892e6023aeedf2fa4b178708c712d3180269)

rbd: delay determination of default pool name

Wait until after the CLI has connected to the cluster and therefore
has received all global configuration overrides.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 96f05a7956b3d554e06330c6bb5743a6329d1c44)

rbd: let librados finish initializing the global context

librados will retrieve a bootstrap config update from the MONs
after connecting and then finish initializing the context. In
general, non-primitive config options cannot be updated after
initialization is complete. This prevents the rbd CLI from
supporting non-primitive config overrides from the MON.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f6c6887fe702190c65245502f04aba5941a4fa15)

mon/MonClient: apply MON config before finishing bootstrap

In general, non-primitive config options cannot be updated after
the service threads have been marked as running. In librados, this
happens after the MonClient bootstrap process. However, the configs
received from the MON were applied in a separate thread which can
race with librados marking the service threads as running.

Now the received MON configs will be applied within the context of
the bootstrap function call instead of deferring to a separate
thread.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 313aeaa55d67089e0a2e173718899e603e51ce2e)

Conflicts:
src/mon/MonClient.cc (trivial)

Merge pull request #41530 from rhcs-dashboard/wip-50884-octopus

octopus: mgr/dashboard: fix OSDs Host details/overview grafana graphs

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: wornet-mwo <NOT@FOUND>

Merge pull request #41508 from rhcs-dashboard/wip-50960-octopus

octopus: mgr/dashboard: fix API docs link

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #41199 from votdev/wip-50663-octopus

octopus: mgr/dashboard: disable NFSv3 support in dashboard

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #41642 from idryomov/wip-rbd-qemu-precise-repos-octopus

octopus: qa/tasks/qemu: precise repos have been archived

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #41031 from trociny/wip-48424-octopus

octopus: rgw: check object locks in multi-object delete

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

qa/tasks/qemu: precise repos have been archived

Fixes: https://tracker.ceph.com/issues/51033
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit dcd193c35eba7583613b805ab3941ff3ac5df745)

Merge pull request #41511 from ideepika/wip-49592-upgrade-octopus

octopus: qa/upgrade: disable update_features test_notify with older client as lockowner

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

octopus: qa/upgrade: disable update_features test_notify with older client as lockowner

 * with the recent support for async rbd operations from pacific+ when an
 older client(non async support) goes on upgrade, and simultaneously
 interacts with a newer client which expects the requests to be async,
 experiences hang; considering the return code for request completion to
 be acknowledgement for async request, which then keeps waiting for
 another acknowledgement of request completion.

 this if happens should be a rare only when lockowner is an old client
 and should be deferred if compatibility issues arises.

 * qa/upgrade: amend upgrade test workunits to use respective stable branches

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

15.2.13

mgr/dashboard: fix OSDs Host details/overview grafana graphs

Fixes: https://tracker.ceph.com/issues/50686
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 7d79efb0252e970b3621d1540e69e2115788dcc3)

dashboard: Fixed name clash when hostname similar to anaother

Fixes: #49769
Signed-off-by: Michael Wodniok <wodniok@wor.net>
(cherry picked from commit e97e27ebdbb394c01aa1cf502039a798d9c4e110)
(cherry picked from commit 9f729ff34d487adc7b38913b92fce4f5241b6c43)

mgr/dashboard: fix API docs link

Fixes: https://tracker.ceph.com/issues/50890
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit e79c9d3f51970fbe275087e92e468205f70e6f12)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/dashboard-help/dashboard-help.component.html
      - Adopt the master branch changes.

Merge pull request #41387 from votdev/wip-47471-octopus

octopus: mgr/dashboard: allow getting fresh inventory data from the orchestrator

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #40382 from singuliere/wip-48934-octopus

octopus: rgw/amqp: fix race condition in amqp manager initialization

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #41302 from MrFreezeex/wip-50712-octopus

octopus: librbd/mirror/snapshot: avoid UnlinkPeerRequest with a unlinked peer

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40388 from singuliere/wip-49024-octopus

octopus: rbd/bench: include used headers

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40387 from singuliere/wip-49766-octopus

octopus: librbd: allow interrupted trash move request to be restarted

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40386 from singuliere/wip-49772-octopus

octopus: librbd/io: send alloc_hint when compression hint is set

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #41321 from trociny/wip-50793-octopus

octopus: osd: don't assert in-flight backfill is always in recovery list

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41311 from k0ste/wip-50778-octopus

octopus: mgr/progress: ensure progress stays between [0,1]

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: allow getting fresh inventory data from the orchestrator

When there is a device change, a `ceph orch device ls --refresh` command
needs to be called so the orchestrator can invalidate its cache and
refresh all devices on all nodes. Currently, the call is asynchronous and
there is no way to determine is a refresh is done or not.

To allow doing a refresh in the Dashboard:
- The inventory device list is periodically updated with cached data.
- If the user clicks the refresh button, a refresh call is sent to the
  orchestrator. Thus if there are device changes, it will be revealed soon
  because of the periodical update.

Fixes: https://tracker.ceph.com/issues/44803
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 1147d6dca04fe6e45b385b549db37775fce54edd)

Conflicts:
src/pybind/mgr/dashboard/controllers/orchestrator.py
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/inventory/inventory.component.ts

Merge pull request #40945 from badone/wip-octopus-tracker-50414

octopus: qa/ceph-ansible: Update ansible version and ceph_stable_release

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #41157 from smithfarm/wip-50365-octopus

octopus: rgw: during reshard lock contention, adjust logging

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #40767 from smithfarm/wip-49472-octopus

octopus: qa: bump osd heartbeat grace for ffsb workload

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40743 from smithfarm/wip-50256-octopus

octopus: mds: trim cache regularly for standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40708 from smithfarm/wip-49475-octopus

octopus: test: use std::atomic<bool> instead of volatile for cb_done var

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #41247 from idryomov/wip-posix-memalign-fix-octopus

octopus: common/buffer: adjust align before calling posix_memalign()

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41237 from trociny/wip-50703-octopus

octopus: os/FileStore: fix to handle readdir error correctly

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41112 from k0ste/wip-50601-octopus

octopus: osd: compute OSD's space usage ratio via raw space utilization

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #40919 from neha-ojha/wip-50405-octopus

octopus: common/options/global.yaml.in: increase default value of bluestore_cache_trim_max_skip_pinned

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40296 from xijiacun/octopus

octopus: rgw: Use correct bucket info when put or get large object with swift

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39978 from singuliere/wip-49053-octopus

octopus: common/mempool: Improve mempool shard selection

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge remote-tracking branch 'origin/octopus-saved' into octopus

15.2.12

mgr/dashboard: fix cookie injection issue

Fixes: CVE-2021-3509
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit b39922818bc57cde1b016e9ad41908b18063b93b)

Conflicts:
src/pybind/mgr/dashboard/controllers/docs.py
        - Remove allow_empty_body and _with_token method

mgr/dashboard: fix base-href: revert it to previous approach

Fixes: https://tracker.ceph.com/issues/50684
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit b6f92922f5c80223fd288d98ce85405a650c0135)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/app.module.ts
     - Adopt the changes coming from master.

(cherry picked from commit fab19ddf55c1e3f1e61745a676785ff0309f11f2)

Merge pull request #40737 from Daniel-Pivonka/octopus-backport-40477

octopus: cephadm: fix failure when using --apply-spec and --shh-user

Merge pull request #40657 from badone/wip-octopus-fix-pytest-double-requirement

octopus: mgr/dashboard: Remove redundant pytest requirement

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: don't assert in-flight backfill is always in recovery list

In PrimaryLogPG::on_failed_pull, we unconditionally remove soid
from recovering list, but remove it from backfills_in_flight only
when the backfill source is the primary osd.

Fixes: https://tracker.ceph.com/issues/50351
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 9b78e00bdf3dd35478e678661606a59424621c4b)

pybind/mgr/dashboard: use setUpClass for initializeing class

instead of relying on __init__(), use setUpClass() to initialize class
for testing. it turns out in pytest > 4, __init__() is called for the
test class but the attributes of the instantiated class is in turn overriden.

so we have to use setUpClass to do this job.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 71979e9b46e21dc3d3cfc6f06f4a84c9b4c7ce78)

 Conflicts:
src/pybind/mgr/dashboard/tests/test_api_auditing.py:
          Differences in import lines
src/pybind/mgr/dashboard/tests/test_tools.py:
          Differences in import lines

tools/setup-virtualenv.sh: pass --use-feature=2020-resolver to pip

as long as pip supports this option, pass it to `pip install`

to silence warnings and errors like:

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

autopep8 1.5.4 requires pycodestyle>=2.6.0, but you'll have pycodestyle 2.5.0 which is incompatible.
pytest-cov 2.10.1 requires pytest>=4.6, but you'll have pytest 3.10.1 which is incompatible.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fa9e2bfd4b3648f08ed3a88ce737d432ab97cce1)

pybind/mgr/dashboard: move pytest into requirements.txt

before this change, pytest is included by both requirements-lint.txt
and requirements-test.txt. this fails the install-deps.sh script when
collecting the python package wheels:

ERROR: Double requirement given: pytest<4 (from -r requirements-test.txt (line 2)) (already in pytest (from -r requirements-lint.txt (line 12)), name='pytest')

also, since pytest is unconditionally imported in the source, for instance,
in pybind/mgr/dashboard/tests/test_ceph_service.py

it would be more straightforward just to include it in requirements.txt.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit eab195566d54122f826debd8efb7f36db78fa4e1)

 Conflicts:
src/pybind/mgr/dashboard/requirements-lint.txt: Additional
        package lines
src/pybind/mgr/dashboard/requirements-test.txt: No mock line

pybind/mgr/dashboard: s/pytest<4/pytest/

to address following failure:

    The user requested pytest<4
    pytest-cov 2.10.1 depends on pytest>=4.6

when building the target of "mgr-dashboard-virtualenv"

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 128778f25eb64cd334e062d627abdb23e0ef0e49)

 Conflicts:
src/pybind/mgr/dashboard/requirements-test.txt: No mock line

Merge pull request #41228 from ceph/wip-yuriw-octopus-p2p

octopus: qa/tests: advanced octopus initial version to 15.2.10

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/progress: ensure progress stays between [0,1]

If _original_pg_count is 0 then progress can be negative.

Fixes: https://tracker.ceph.com/issues/50591
Related-to: https://tracker.ceph.com/issues/50587
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 20990a94598d0249745e2ec25c9197d842119d92)

qa/tests: resolved comments

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #40783 from smithfarm/wip-50286-octopus

octopus: mon: check mdsmap is resizeable before promoting standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40779 from smithfarm/wip-50181-octopus

octopus: cephfs: client: only check pool permissions for regular files

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40778 from smithfarm/wip-50027-octopus

octopus: client: fire the finish_cap_snap() after buffer being flushed

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40777 from smithfarm/wip-49950-octopus

octopus: doc/cephfs/nfs: Add note about cephadm NFS-Ganesha daemon port

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40776 from smithfarm/wip-49934-octopus

octopus: test: reduce number of threads to 32 in LibCephFS.ShutdownRace

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40775 from smithfarm/wip-49752-octopus

octopus: doc: snap-schedule documentation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40774 from smithfarm/wip-49851-octopus

octopus: mds: fix race of fetching large dirfrag

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40773 from smithfarm/wip-49611-octopus

octopus: qa: add sleep for blocklisting to take effect

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40772 from smithfarm/wip-49560-octopus

octopus: qa: delete all fs during tearDown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40771 from smithfarm/wip-49518-octopus

octopus: cephfs: client: wake up the front pos waiter

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40770 from smithfarm/wip-49515-octopus

octopus: pybind/cephfs: DT_REG and DT_LNK values are wrong

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40765 from smithfarm/wip-49347-octopus

octopus: qa: for the latest kclient it will also return EIO

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40764 from smithfarm/wip-48878-octopus

octopus: mds: update defaults for recall configs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40763 from smithfarm/wip-48836-octopus

octopus: mount.ceph: collect v2 addresses for non-legacy ms_mode options

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40762 from smithfarm/wip-45853-octopus

octopus: tools/cephfs: don't bind to public_addr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40268 from kotreshhr/wip-49904-octopus

octopus: mgr/volumes: Retain suid guid bits in clone

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

librbd/mirror/snapshot: avoid UnlinkPeerRequest with a unlinked peer

CreatePrimaryRequest could create some UnlinkPeerRequest with an already
unlinked peer in a scenario where you have multiple peers. This request
will not remove the peer (as it's already not linked to the requested
peer) and will skip deletion of the mirror snapshot if another peer
remains. Eventually the code will go through an infinite recursive loop
between CreatePrimaryRequest and UnlinkPeerRequest and segfault.

This commit adds an extra condition to make sure to not submit a
UnlinkPeerRequest if the peer is not linked to the current snapshot. If
there is already no peer in the list it will submit a UnlinkPeerRequest
to remove the snapshot.

Fixes: https://tracker.ceph.com/issues/50439
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
(cherry picked from commit c6e2953fdb9e29cfb5fb4e04fd633862160cdb13)

Merge pull request #41057 from rhcs-dashboard/wip-50475-octopus

octopus: mgr/dashboard: Remove username and password from request body

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge PR #40766 into octopus

* refs/pull/40766/head:
doc/cephfs/nfs: Add rook pod restart note, export and log block example

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

rgw: sanitize \r in s3 CORSConfiguration's ExposeHeader

follows up on 1524d3c0c5cb11775313ea1e2bb36a93257947f2 to escape \r as
well

Fixes: CVE-2021-3524
Reported-by: Sergey Bobrov <Sergey.Bobrov@kaspersky.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 87806f48e7a1b8891eb90711f1cedd26f1119aac)

rgw: RGWSwiftWebsiteHandler::is_web_dir checks empty subdir_name

checking for empty name avoids later assertion in RGWObjectCtx::set_atomic

Fixes: CVE-2021-3531
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7196a469b4470f3c8628489df9a41ec8b00a5610)

Merge pull request #41252 from rhcs-dashboard/wip-50722-octopus

octopus: mgr/dashboard: fix base-href: revert it to previous approach

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix base-href: revert it to previous approach

Fixes: https://tracker.ceph.com/issues/50684
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit b6f92922f5c80223fd288d98ce85405a650c0135)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/app.module.ts
     - Adopt the changes coming from master.

msg/async/ProtocolV2: catch correct bad_alloc exception

We want buffer::bad_alloc, not std::bad_alloc.  Otheriwise, we end
up with a confusing error

  failed decoding of frame header: Bad allocation

from ProtocolV2::run_continuation(), printed after frame header is
successfully decoded.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 67bb6cf524975ea54d539c5b10ba83fa496a1ced)

common/buffer: adjust align before calling posix_memalign()

posix_memalign() requires alignment argument to be a multiple of
sizeof(void *).  Since it is an implementation detail of buffer,
it needs to be adjusted there -- buffer consumers have no way of
knowing that passing e.g. align == 4 is incorrect.

One place already does the adjustment, but only for align == 0.
The other just asserts.  Fix both and remove the "power of two"
assertion.  Let posix_memalign() return EINVAL and handle that
by throwing buffer::bad_alloc, as expected by the consumers.

Fixes: https://tracker.ceph.com/issues/50646
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit aa31ddf0e70b3b8ef8012e09cb3158f3db4dea1b)

os/FileStore: fix to handle readdir error correctly

Currently filestore code does not handle readdir error.
As man readdir(3) says, we need to check errno after readdir
returns NULL to determine if error happens or not.

This patch fixes the all readdir() calls to check errono and
handle it appropriately:
 - FileStore.cc ... abort if EIO error happens
 - BtrfsFileStoreBAckend.cc/LFNindex.cc
                ... return error to upper layer

Without this fixes, primary PG could fail to correctly perform
backfill operation and could lead data loss propagation described
in #50558.

Fixes: https://tracker.ceph.com/issues/50558
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
(cherry picked from commit 5a6c6267a182f859471ee629b490777ee1e970dd)

qa/tests: advanced octopus initial version to 15.2.10

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

mgr/dashboard: disable NFSv3 support in dashboard

Fixes: https://tracker.ceph.com/issues/49718
Related to: https://github.com/ceph/ceph/pull/40154

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 864a362389e988bc9aab23f66da608b4ceab74c0)

Merge pull request #41124 from aaSharma14/wip-50582-octopus

octopus: mgr/dashboard: OSDs placement text is unreadable

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #41017 from idryomov/wip-reset-authenticate-err-octopus

octopus: mon/MonClient: reset authenticate_err in _reopen_session()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40988 from trociny/wip-50479-octopus

octopus: os/FileStore: don't propagate split/merge error to "create"/"remove"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40838 from mgfritch/octopus-backport-39415

octopus: cephadm: Allow to use paths in all <_devices> drivegroup sections

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #40823 from mgfritch/octopus-backport-39259

octopus: mgr/cephadm: on ssh connection error, advice chmod 0600

Reviewed-by: Adam King adking@redhat.com

rgw: during reshard lock contention, adjust logging

When RGW fails to get a lock on a reshard log, we log it in such a way
that it looks like an error. Instead we'll make sure that the log
message is informational.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 6d3dee37791ad427a3435c493a1d7874ba075674)

mds: do not trim the inodes from the lru list in standby_replay

In standby_replay, if some dentries just added/linked but not get a
chance to replay the EOpen journals followed, if the upkeep_main() is
excuted, which will may trim them out immediately. Then when playing
the EOpen journals later the replay will fail.

In standby_replay, let's skip trimming them if dentry's linkage inode
is not nullptr.

Fixes: https://tracker.ceph.com/issues/50246
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 79bb44c1b9f1715378a9550a81984e949e454ff4)

mds: trim cache regularly for standby-replay

This change is slightly awkward because standby-replay MDS do not do all
the kinds of upkeep a normal active MDS does. In particular, it is not
going to recall client state from clients.

This diff also merges the extra recall_client_state in
MDCache::check_memory_usage into its only caller (the upkeep thread)
where it was also doing a recall. That's just a matter of merging the
recall flags. This has the added benefit of making
MDCache::check_memory_usage callable for all MDS daemons regardless of
state.

Fixes: https://tracker.ceph.com/issues/50048
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 19293d9b9d19c32af4de655cd59e206056b2417d)

mds: avoid spurious sleeps

Like trim_interval, don't sleep for small amounts of time. This avoids
spurious sleeps like:

    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s

Also, fix the same issue in the Client.

Fixes: https://tracker.ceph.com/issues/48753
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit eb47e990c33843b9baa366e2b2a187439210e680)

Conflicts:
src/client/Client.cc
- the code being changed does not exist in octopus

mds: remove extra heap release

We now regularly do this unconditionally in the MDS, see the upkeep
thread.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5a9d6c080d77c7e3644b02cab4f8c91900f4fe8f)

Conflicts:
src/mds/MDCache.cc
- octopus has a line, in->maybe_ephemeral_dist(false);, which is not there in
  master

mgr/dashboard:OSDs placement text is unreadable

While displaying the host pattern in the OSDs placement tab, it gets splited with semi-colons. Also adjusted the column size of Container Image ID and Placement columns.

Fixes: https://tracker.ceph.com/issues/50580
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 543b02436f18876a56757226c686a5c2c33c7c33)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/services.component.ts (PlacementPipe takes i18n as input param)

osd: compute OSD's space usage ratio via raw space utilization

Fixes: https://tracker.ceph.com/issues/50533
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 81c4d82be02ee14aff2849b3025a5dea6cb0327e)

Merge pull request #41061 from dvanders/50550

octopus: os/bluestore: be more verbose in _open_super_meta by default.

os/bluestore: be more verbose in _open_super_meta by default.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 4087f82aea674df4c7b485bf804f3a9c98ae3741)

mgr/dashboard: Remove username and password from request body

Fixes: https://tracker.ceph.com/issues/50451
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 273a776cad8065f568f17a05804aabd14625a1f0)

Merge pull request #40894 from rhcs-dashboard/wip-50349-octopus

octopus: mgr/dashboard: improve telemetry opt-in reminder notification message

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #39987 from aaSharma14/wip-49657-octopus

octopus: mgr/dashboard: test prometheus rules through promtool

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #40816 from rhcs-dashboard/wip-50170-octopus

octopus: mgr/dashboard: debug nodeenv hangs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>