Merge pull request #41416 from cfsnyder/wip-50730-octopus

octopus: rgw/rgw_file: Fix the return value of read() and readlink()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41415 from cfsnyder/wip-50640-octopus

octopus: rgw/sts: read_obj_policy() consults iam_user_policies on ENOENT

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41413 from cfsnyder/wip-50709-octopus

octopus: rgw: fix bucket object listing when marker matches prefix

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41412 from cfsnyder/wip-50464-octopus

octopus: rgw/notifications: delete bucket notification object when empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #40768 from smithfarm/wip-49513-octopus

octopus: cephfs: client: add ability to lookup snapped inodes by inode number

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #41647 from rhcs-dashboard/wip-51065-octopus

octopus: mgr/dashboard: fix bucket objects and size calculations

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #41530 from rhcs-dashboard/wip-50884-octopus

octopus: mgr/dashboard: fix OSDs Host details/overview grafana graphs

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: wornet-mwo <NOT@FOUND>

Merge pull request #41508 from rhcs-dashboard/wip-50960-octopus

octopus: mgr/dashboard: fix API docs link

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #41199 from votdev/wip-50663-octopus

octopus: mgr/dashboard: disable NFSv3 support in dashboard

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #41642 from idryomov/wip-rbd-qemu-precise-repos-octopus

octopus: qa/tasks/qemu: precise repos have been archived

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #41031 from trociny/wip-48424-octopus

octopus: rgw: check object locks in multi-object delete

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

qa/tasks/qemu: precise repos have been archived

Fixes: https://tracker.ceph.com/issues/51033
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit dcd193c35eba7583613b805ab3941ff3ac5df745)

mgr/dashboard: fix bucket objects and size calculations

Fixes: https://tracker.ceph.com/issues/51035
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 9f5ef98d9c88a91b80e622f16f7061eddff79b2c)

Merge pull request #41511 from ideepika/wip-49592-upgrade-octopus

octopus: qa/upgrade: disable update_features test_notify with older client as lockowner

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

octopus: qa/upgrade: disable update_features test_notify with older client as lockowner

 * with the recent support for async rbd operations from pacific+ when an
 older client(non async support) goes on upgrade, and simultaneously
 interacts with a newer client which expects the requests to be async,
 experiences hang; considering the return code for request completion to
 be acknowledgement for async request, which then keeps waiting for
 another acknowledgement of request completion.

 this if happens should be a rare only when lockowner is an old client
 and should be deferred if compatibility issues arises.

 * qa/upgrade: amend upgrade test workunits to use respective stable branches

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

test: add ceph_mksnap and ceph_rmsnap replacements

Signed-off-by: Jeff Layton <jlayton@redhat.com>

15.2.13

test: add test case for MDS privated inos accessing

Fixes: https://tracker.ceph.com/issues/50216
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 0ec4d781787df777bbfdeb742b394c73a0d76e08)

Conflicts:
src/test/libcephfs/test.cc
- octopus version of this file is missing lots of tests that were added to
  master post-octopus

mds: make the lost+found dir accessible from clients

Inode number 0x4 is reserved for the lost+found dir, and the apps
or recovery apps need to access it.

At the same time the 0x3 is reserved for the global snaprealm, which
will also be lookup by the clients when looking up the snaprealm inodes.

Here will make all the inode less than 100 accessible by clients.

Fixes: https://tracker.ceph.com/issues/50216
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 841071b3602aa2b8bcca57b95cdf671d48036a34)

Conflicts:
src/mds/Server.cc
- code being changed does not exist in octopus: change omitted

mds: move the inos 1,2 and 3 macros to ceph_fs.h

For inos 1 and 2 there has two places are defining them, to make
sure we won't miss any one of them, let's define them in only one
header file.

Fixes: https://tracker.ceph.com/issues/50216
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 8615807237458e8cebf9497a7431097cdc56c784)

Conflicts:
src/client/Inode.h
src/mds/CInode.h
src/mds/MDCache.cc
src/mds/Server.cc
        src/mds/SnapServer.h
- change MDS_INO_ROOT to CEPH_INO_ROOT in src/mds/SnapServer.h (octopus-only)

client: don't allow access to MDS-private inodes

Fixes: https://tracker.ceph.com/issues/50112
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 89c511356125f892477ef42bd14c0b447ff06106)

Conflicts:
src/client/Client.cc
- octopus wants ESTALE instead of CEPHFS_ESTALE

client: add testcase for ceph_ll_lookup_vino

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit a956c7986fd43da851918b87c71524f9a2bb1162)

Conflicts:
src/test/libcephfs/test.cc
- octopus is missing a bunch of tests, but this doesn't matter because the
  commit being cherry-picked did not touch those

client: add ceph_ll_lookup_vino

Add a new API function for looking up an inode via a vinodeno_t. This
should give ganesha a way to reliably look up snapshot inodes.

We do need to add some special handling for CEPH_SNAPDIRs. If we're
looking for one, then find the non-snapped parent, and then call
open_snapdir to get the snapdir inode.

Also, have the function check the local cache before calling the MDS
to look up an inode.

Fixes: https://tracker.ceph.com/issues/48991
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 70622079c2ec55222a139fa5042902e0b19bd839)

Conflicts:
src/client/Client.cc

client: make _lookup_ino take a vinodeno_t

Currently, it always leaves the snapid as 0. Rename it to
_lookup_vino and make it fill the snapid from the vinodeno_t
instead, but only when it's a "real" snapid.

Change the existing callers to pass in a vinodeno_t with the
snapid set to CEPH_NOSNAP.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 0e0bebd22dc1abea3f0132debc0b524a01fadf63)

Conflicts:
src/client/Client.cc

client: stop doing unnecessary work in ll_lookup_inode

It's not clear to me why we're looking up the parent and name of the
inode in ll_lookup_inode, as we don't actually do anything with them.
Just return once we get an inode reference.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 2b0b895e888c0f3dece3c1e717ca6e1552e588d8)

mgr/dashboard: fix OSDs Host details/overview grafana graphs

Fixes: https://tracker.ceph.com/issues/50686
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 7d79efb0252e970b3621d1540e69e2115788dcc3)

dashboard: Fixed name clash when hostname similar to anaother

Fixes: #49769
Signed-off-by: Michael Wodniok <wodniok@wor.net>
(cherry picked from commit e97e27ebdbb394c01aa1cf502039a798d9c4e110)
(cherry picked from commit 9f729ff34d487adc7b38913b92fce4f5241b6c43)

mgr/dashboard: fix API docs link

Fixes: https://tracker.ceph.com/issues/50890
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit e79c9d3f51970fbe275087e92e468205f70e6f12)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/dashboard-help/dashboard-help.component.html
      - Adopt the master branch changes.

Merge pull request #41387 from votdev/wip-47471-octopus

octopus: mgr/dashboard: allow getting fresh inventory data from the orchestrator

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #40382 from singuliere/wip-48934-octopus

octopus: rgw/amqp: fix race condition in amqp manager initialization

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #41302 from MrFreezeex/wip-50712-octopus

octopus: librbd/mirror/snapshot: avoid UnlinkPeerRequest with a unlinked peer

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40388 from singuliere/wip-49024-octopus

octopus: rbd/bench: include used headers

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40387 from singuliere/wip-49766-octopus

octopus: librbd: allow interrupted trash move request to be restarted

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40386 from singuliere/wip-49772-octopus

octopus: librbd/io: send alloc_hint when compression hint is set

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #41321 from trociny/wip-50793-octopus

octopus: osd: don't assert in-flight backfill is always in recovery list

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41311 from k0ste/wip-50778-octopus

octopus: mgr/progress: ensure progress stays between [0,1]

Reviewed-by: Neha Ojha <nojha@redhat.com>

rgw/rgw_file: Fix the return value of read() and readlink()

Fixes: https://tracker.ceph.com/issues/49189
Signed-off-by: Dai zhiwei <daizhiwei3@huawei.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
(cherry picked from commit bfd83e8fa142873a0bdf09a4d1ad1b04127f5885)

rgw: read_obj_policy() consults iam_user_policies on ENOENT

when the head object doesn't exist, read_obj_policy() has to decide
whether to return ENOENT or EACCES

when there's a bucket policy, we check whether it has s3ListBucket
permissions. when there's an assumed role, we also need to check
against the role's policies in s->iam_user_policies

Fixes: https://tracker.ceph.com/issues/49780
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5dc9375fa1888242f388f8b502f445f3ddc891f7)

 Conflicts:
src/rgw/rgw_op.cc

rgw: fix bucket object listing when marker matches prefix

When an iniitial marker that ends with a delimiter is provided, it
prevents listing of that "subdirectory" due to new logic at the cls
level to make listing more efficient. The fix catches that situation.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 220ef4b22d1d1667eb4f2c300a0b788e87b9067d)

rgw/notifications: delete bucket notification object when empty

Fixes: https://tracker.ceph.com/issues/49650
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 08e9846fd5a040887e2ddbfca36da97777fc76f6)

 Conflicts:
src/rgw/rgw_pubsub.cc
 - octopus does not have optional_yield arg

mgr/dashboard: allow getting fresh inventory data from the orchestrator

When there is a device change, a `ceph orch device ls --refresh` command
needs to be called so the orchestrator can invalidate its cache and
refresh all devices on all nodes. Currently, the call is asynchronous and
there is no way to determine is a refresh is done or not.

To allow doing a refresh in the Dashboard:
- The inventory device list is periodically updated with cached data.
- If the user clicks the refresh button, a refresh call is sent to the
  orchestrator. Thus if there are device changes, it will be revealed soon
  because of the periodical update.

Fixes: https://tracker.ceph.com/issues/44803
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 1147d6dca04fe6e45b385b549db37775fce54edd)

Conflicts:
src/pybind/mgr/dashboard/controllers/orchestrator.py
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/inventory/inventory.component.ts

Merge pull request #40945 from badone/wip-octopus-tracker-50414

octopus: qa/ceph-ansible: Update ansible version and ceph_stable_release

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #41157 from smithfarm/wip-50365-octopus

octopus: rgw: during reshard lock contention, adjust logging

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #40767 from smithfarm/wip-49472-octopus

octopus: qa: bump osd heartbeat grace for ffsb workload

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40743 from smithfarm/wip-50256-octopus

octopus: mds: trim cache regularly for standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40708 from smithfarm/wip-49475-octopus

octopus: test: use std::atomic<bool> instead of volatile for cb_done var

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #41247 from idryomov/wip-posix-memalign-fix-octopus

octopus: common/buffer: adjust align before calling posix_memalign()

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41237 from trociny/wip-50703-octopus

octopus: os/FileStore: fix to handle readdir error correctly

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41112 from k0ste/wip-50601-octopus

octopus: osd: compute OSD's space usage ratio via raw space utilization

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #40919 from neha-ojha/wip-50405-octopus

octopus: common/options/global.yaml.in: increase default value of bluestore_cache_trim_max_skip_pinned

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40296 from xijiacun/octopus

octopus: rgw: Use correct bucket info when put or get large object with swift

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39978 from singuliere/wip-49053-octopus

octopus: common/mempool: Improve mempool shard selection

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge remote-tracking branch 'origin/octopus-saved' into octopus

15.2.12

mgr/dashboard: fix cookie injection issue

Fixes: CVE-2021-3509
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit b39922818bc57cde1b016e9ad41908b18063b93b)

Conflicts:
src/pybind/mgr/dashboard/controllers/docs.py
        - Remove allow_empty_body and _with_token method

mgr/dashboard: fix base-href: revert it to previous approach

Fixes: https://tracker.ceph.com/issues/50684
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit b6f92922f5c80223fd288d98ce85405a650c0135)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/app.module.ts
     - Adopt the changes coming from master.

(cherry picked from commit fab19ddf55c1e3f1e61745a676785ff0309f11f2)

Merge pull request #40737 from Daniel-Pivonka/octopus-backport-40477

octopus: cephadm: fix failure when using --apply-spec and --shh-user

Merge pull request #40657 from badone/wip-octopus-fix-pytest-double-requirement

octopus: mgr/dashboard: Remove redundant pytest requirement

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: don't assert in-flight backfill is always in recovery list

In PrimaryLogPG::on_failed_pull, we unconditionally remove soid
from recovering list, but remove it from backfills_in_flight only
when the backfill source is the primary osd.

Fixes: https://tracker.ceph.com/issues/50351
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 9b78e00bdf3dd35478e678661606a59424621c4b)

pybind/mgr/dashboard: use setUpClass for initializeing class

instead of relying on __init__(), use setUpClass() to initialize class
for testing. it turns out in pytest > 4, __init__() is called for the
test class but the attributes of the instantiated class is in turn overriden.

so we have to use setUpClass to do this job.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 71979e9b46e21dc3d3cfc6f06f4a84c9b4c7ce78)

 Conflicts:
src/pybind/mgr/dashboard/tests/test_api_auditing.py:
          Differences in import lines
src/pybind/mgr/dashboard/tests/test_tools.py:
          Differences in import lines

tools/setup-virtualenv.sh: pass --use-feature=2020-resolver to pip

as long as pip supports this option, pass it to `pip install`

to silence warnings and errors like:

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

autopep8 1.5.4 requires pycodestyle>=2.6.0, but you'll have pycodestyle 2.5.0 which is incompatible.
pytest-cov 2.10.1 requires pytest>=4.6, but you'll have pytest 3.10.1 which is incompatible.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fa9e2bfd4b3648f08ed3a88ce737d432ab97cce1)

pybind/mgr/dashboard: move pytest into requirements.txt

before this change, pytest is included by both requirements-lint.txt
and requirements-test.txt. this fails the install-deps.sh script when
collecting the python package wheels:

ERROR: Double requirement given: pytest<4 (from -r requirements-test.txt (line 2)) (already in pytest (from -r requirements-lint.txt (line 12)), name='pytest')

also, since pytest is unconditionally imported in the source, for instance,
in pybind/mgr/dashboard/tests/test_ceph_service.py

it would be more straightforward just to include it in requirements.txt.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit eab195566d54122f826debd8efb7f36db78fa4e1)

 Conflicts:
src/pybind/mgr/dashboard/requirements-lint.txt: Additional
        package lines
src/pybind/mgr/dashboard/requirements-test.txt: No mock line

pybind/mgr/dashboard: s/pytest<4/pytest/

to address following failure:

    The user requested pytest<4
    pytest-cov 2.10.1 depends on pytest>=4.6

when building the target of "mgr-dashboard-virtualenv"

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 128778f25eb64cd334e062d627abdb23e0ef0e49)

 Conflicts:
src/pybind/mgr/dashboard/requirements-test.txt: No mock line

Merge pull request #41228 from ceph/wip-yuriw-octopus-p2p

octopus: qa/tests: advanced octopus initial version to 15.2.10

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/progress: ensure progress stays between [0,1]

If _original_pg_count is 0 then progress can be negative.

Fixes: https://tracker.ceph.com/issues/50591
Related-to: https://tracker.ceph.com/issues/50587
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 20990a94598d0249745e2ec25c9197d842119d92)

qa/tests: resolved comments

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #40783 from smithfarm/wip-50286-octopus

octopus: mon: check mdsmap is resizeable before promoting standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40779 from smithfarm/wip-50181-octopus

octopus: cephfs: client: only check pool permissions for regular files

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40778 from smithfarm/wip-50027-octopus

octopus: client: fire the finish_cap_snap() after buffer being flushed

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40777 from smithfarm/wip-49950-octopus

octopus: doc/cephfs/nfs: Add note about cephadm NFS-Ganesha daemon port

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40776 from smithfarm/wip-49934-octopus

octopus: test: reduce number of threads to 32 in LibCephFS.ShutdownRace

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40775 from smithfarm/wip-49752-octopus

octopus: doc: snap-schedule documentation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40774 from smithfarm/wip-49851-octopus

octopus: mds: fix race of fetching large dirfrag

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40773 from smithfarm/wip-49611-octopus

octopus: qa: add sleep for blocklisting to take effect

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40772 from smithfarm/wip-49560-octopus

octopus: qa: delete all fs during tearDown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40771 from smithfarm/wip-49518-octopus

octopus: cephfs: client: wake up the front pos waiter

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40770 from smithfarm/wip-49515-octopus

octopus: pybind/cephfs: DT_REG and DT_LNK values are wrong

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40765 from smithfarm/wip-49347-octopus

octopus: qa: for the latest kclient it will also return EIO

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40764 from smithfarm/wip-48878-octopus

octopus: mds: update defaults for recall configs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40763 from smithfarm/wip-48836-octopus

octopus: mount.ceph: collect v2 addresses for non-legacy ms_mode options

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40762 from smithfarm/wip-45853-octopus

octopus: tools/cephfs: don't bind to public_addr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40268 from kotreshhr/wip-49904-octopus

octopus: mgr/volumes: Retain suid guid bits in clone

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

librbd/mirror/snapshot: avoid UnlinkPeerRequest with a unlinked peer

CreatePrimaryRequest could create some UnlinkPeerRequest with an already
unlinked peer in a scenario where you have multiple peers. This request
will not remove the peer (as it's already not linked to the requested
peer) and will skip deletion of the mirror snapshot if another peer
remains. Eventually the code will go through an infinite recursive loop
between CreatePrimaryRequest and UnlinkPeerRequest and segfault.

This commit adds an extra condition to make sure to not submit a
UnlinkPeerRequest if the peer is not linked to the current snapshot. If
there is already no peer in the list it will submit a UnlinkPeerRequest
to remove the snapshot.

Fixes: https://tracker.ceph.com/issues/50439
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
(cherry picked from commit c6e2953fdb9e29cfb5fb4e04fd633862160cdb13)

Merge pull request #41057 from rhcs-dashboard/wip-50475-octopus

octopus: mgr/dashboard: Remove username and password from request body

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge PR #40766 into octopus

* refs/pull/40766/head:
doc/cephfs/nfs: Add rook pod restart note, export and log block example

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

rgw: sanitize \r in s3 CORSConfiguration's ExposeHeader

follows up on 1524d3c0c5cb11775313ea1e2bb36a93257947f2 to escape \r as
well

Fixes: CVE-2021-3524
Reported-by: Sergey Bobrov <Sergey.Bobrov@kaspersky.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 87806f48e7a1b8891eb90711f1cedd26f1119aac)

rgw: RGWSwiftWebsiteHandler::is_web_dir checks empty subdir_name

checking for empty name avoids later assertion in RGWObjectCtx::set_atomic

Fixes: CVE-2021-3531
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7196a469b4470f3c8628489df9a41ec8b00a5610)

Merge pull request #41252 from rhcs-dashboard/wip-50722-octopus

octopus: mgr/dashboard: fix base-href: revert it to previous approach

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix base-href: revert it to previous approach

Fixes: https://tracker.ceph.com/issues/50684
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit b6f92922f5c80223fd288d98ce85405a650c0135)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/app.module.ts
     - Adopt the changes coming from master.

msg/async/ProtocolV2: catch correct bad_alloc exception

We want buffer::bad_alloc, not std::bad_alloc.  Otheriwise, we end
up with a confusing error

  failed decoding of frame header: Bad allocation

from ProtocolV2::run_continuation(), printed after frame header is
successfully decoded.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 67bb6cf524975ea54d539c5b10ba83fa496a1ced)

common/buffer: adjust align before calling posix_memalign()

posix_memalign() requires alignment argument to be a multiple of
sizeof(void *).  Since it is an implementation detail of buffer,
it needs to be adjusted there -- buffer consumers have no way of
knowing that passing e.g. align == 4 is incorrect.

One place already does the adjustment, but only for align == 0.
The other just asserts.  Fix both and remove the "power of two"
assertion.  Let posix_memalign() return EINVAL and handle that
by throwing buffer::bad_alloc, as expected by the consumers.

Fixes: https://tracker.ceph.com/issues/50646
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit aa31ddf0e70b3b8ef8012e09cb3158f3db4dea1b)

os/FileStore: fix to handle readdir error correctly

Currently filestore code does not handle readdir error.
As man readdir(3) says, we need to check errno after readdir
returns NULL to determine if error happens or not.

This patch fixes the all readdir() calls to check errono and
handle it appropriately:
 - FileStore.cc ... abort if EIO error happens
 - BtrfsFileStoreBAckend.cc/LFNindex.cc
                ... return error to upper layer

Without this fixes, primary PG could fail to correctly perform
backfill operation and could lead data loss propagation described
in #50558.

Fixes: https://tracker.ceph.com/issues/50558
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
(cherry picked from commit 5a6c6267a182f859471ee629b490777ee1e970dd)

qa/tests: advanced octopus initial version to 15.2.10

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

mgr/dashboard: disable NFSv3 support in dashboard

Fixes: https://tracker.ceph.com/issues/49718
Related to: https://github.com/ceph/ceph/pull/40154

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 864a362389e988bc9aab23f66da608b4ceab74c0)

Merge pull request #41124 from aaSharma14/wip-50582-octopus

octopus: mgr/dashboard: OSDs placement text is unreadable

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #41017 from idryomov/wip-reset-authenticate-err-octopus

octopus: mon/MonClient: reset authenticate_err in _reopen_session()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40988 from trociny/wip-50479-octopus

octopus: os/FileStore: don't propagate split/merge error to "create"/"remove"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40838 from mgfritch/octopus-backport-39415

octopus: cephadm: Allow to use paths in all <_devices> drivegroup sections

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #40823 from mgfritch/octopus-backport-39259

octopus: mgr/cephadm: on ssh connection error, advice chmod 0600

Reviewed-by: Adam King adking@redhat.com

rgw: during reshard lock contention, adjust logging

When RGW fails to get a lock on a reshard log, we log it in such a way
that it looks like an error. Instead we'll make sure that the log
message is informational.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 6d3dee37791ad427a3435c493a1d7874ba075674)