qa/tasks/cephadm: include first mgr host when applying mgrs

This prevents the first mgr from being shut down due to lack of
appropriate placements.

trivial fix: removed octopus upgrade configs.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit c79fa6d780580f99b62117e54326a4ef4b7adfef)

qa/upgrade: replace ubuntu_latest to ubuntu_18.04

since mimic and nautilus does not support focal builds, keep using
ubuntu_18.

octopus only, since master and pacific support focal

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

qa/distros/supported-random-distro$: keep support for bionic

we still want to keep supporting 18.04 LTS for octopus, don't want to
use it for cephadm, since podman dropped support for bionic.

this is octopus only commit, since it's octopus specific support and we
dropped bionic in master.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

qa/workunits/cephadm/test_repos: adjust target version

15.2.7 is available for 20.04; 15.1.1 is not

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7fe3853af97120fc97406f5b8123bbaa4adc9c6e)

rados/cephadm/qa/distros: update to latest distros

- removes ubuntu_18.04 support for podman, instead we move to focal.
- use rhel_8.3 for all rhel_8 references
- use {centos/rhel}_8 instead of {rhel/centos}_latest: to keep things
  same in master and octopus since we use: rhel_8 and centos_8 as latest
  version symlinks, which differentiated after an octopus only commit.

this was not cherry picked from master as octopus had some of the
symlinks, not in sync with master, this commit does cleanup for them,
and tries to make them similar to master.

(updates introduced change by: 126217febf023a6f561fe88dbdb87b5b2f379798)
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

qa/distros/podman: preserve registries.conf

Otherwise we'll lose the mirror configuration.

trivial-fix:

specify location for container registry since we use sepia labs as our
container registery, which otherwise can't find registries.conf

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 917a9c49e55ab107c1e5b1d035398344bcd33858)

qa: new kubic distro files; use kubic podman for centos/rhel

The current centos/rhel version of podman (2.2.1) is broken.

- create new qa/distros/podman/* files that install kubic podman
- include centos/rhel variants
- adjust cephadm jobs to use new yaml files
- remove old qa/distros/all/*_podman.yaml files

trivial fix: we do not have cephadm/thrash suite in octopus(removed)
- distro(from octopus) renamed to 0-distro(from pacific)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 11beb7287084b56ac7cbd45ac3d28b3d9fa4cc5b)

qa/suites/rados/cephadm: add back centos+rhel with kubic podman

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit dbdd4d46e35d2fcf80a1b2cd9da77b6832c45aa3)

qa/suites/rados/cephadm: add 20.04 target

Link directly to the distro version... no need to use _latest here since
it obscures the podman vs docker difference.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 170dd14af4f72bdac2a43742d11fa194c84819f5)

qa/tests: replaced ubuntu_latest.yaml with ubuntu 20.04

resurrect 835f2f5f511e7363f4056e5137382982aac1bfbf, which was reverted by
1f885b6c2fdde1f3ab249a56f7b4b763556f4373

so all tests for `pacific` will be tested on the latest ubuntu version

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 55564dbd01e6020a40d9a18d080aa7b1b9f86d5e)

qa/distros: add centos 8.3

trivial fix: adding centos_8 symlink

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 74f04aa9e12bbe195506fe28d2a0d4e3d4ff200d)

qa/distros: add ubuntu 20.04 targets

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 24c2d2b1244d95c232f927dad1417717a2768ece)

qa/distros: add RHEL 8.3 and use RHEL 8.3 as default

Signed-off-by: rakeshgm <rakeshgm@redhat.com>
(cherry picked from commit 5eabe906e6d15dec48224fd29b45e1155e751314)

qa/distros: centos7 -> centos8, rhel7 -> rhel8

trivial fix: removing skipping log warning needed for previous versions

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1eb8198d0b25be38eae707fc79e8d13d2a3a9788)

Merge pull request #41087 from smithfarm/wip-50357-octopus

octopus: make-dist: refuse to run if script path contains a colon

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #41624 from cfsnyder/wip-50750-octopus

octopus: doc/rados/operations: s/max_misplaced/target_max_misplaced_ratio/

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #41766 from cfsnyder/wip-50727-octopus

octopus: rgw : add check empty for sync url

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41765 from cfsnyder/wip-51079-octopus

octopus: rgw: require bucket name in bucket chown

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41420 from cfsnyder/wip-50677-octopus

octopus: rgw: fix segfault related to explicit object manifest handling

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41419 from cfsnyder/wip-50423-octopus

octopus: rgw: send headers of quota settings

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41418 from cfsnyder/wip-50380-octopus

octopus: rgw/amqp/test: fix mock prototype for librabbitmq-0.11.0

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41417 from cfsnyder/wip-50643-octopus

octopus: rgw: allow rgw-orphan-list to process multiple data pools

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41416 from cfsnyder/wip-50730-octopus

octopus: rgw/rgw_file: Fix the return value of read() and readlink()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41415 from cfsnyder/wip-50640-octopus

octopus: rgw/sts: read_obj_policy() consults iam_user_policies on ENOENT

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41413 from cfsnyder/wip-50709-octopus

octopus: rgw: fix bucket object listing when marker matches prefix

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41412 from cfsnyder/wip-50464-octopus

octopus: rgw/notifications: delete bucket notification object when empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #40768 from smithfarm/wip-49513-octopus

octopus: cephfs: client: add ability to lookup snapped inodes by inode number

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #41647 from rhcs-dashboard/wip-51065-octopus

octopus: mgr/dashboard: fix bucket objects and size calculations

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #41530 from rhcs-dashboard/wip-50884-octopus

octopus: mgr/dashboard: fix OSDs Host details/overview grafana graphs

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: wornet-mwo <NOT@FOUND>

Merge pull request #41508 from rhcs-dashboard/wip-50960-octopus

octopus: mgr/dashboard: fix API docs link

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #41199 from votdev/wip-50663-octopus

octopus: mgr/dashboard: disable NFSv3 support in dashboard

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #41642 from idryomov/wip-rbd-qemu-precise-repos-octopus

octopus: qa/tasks/qemu: precise repos have been archived

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #41031 from trociny/wip-48424-octopus

octopus: rgw: check object locks in multi-object delete

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

rgw : add check empty for sync url

Fixes: https://tracker.ceph.com/issues/50103
Signed-off-by: caolei <halei15848934852@163.com>
(cherry picked from commit 3a4e0b79310b21eeee37043d5419887bb41c0cf6)

rgw: require bucket name in bucket chown

Checking and reporting missing the mandatory parameter avoid clueless error
message for bucket chown.

Signed-off-by: Zulai Wang <zl31wang@gmail.com>
(cherry picked from commit 158a1f4313c0fa206031ede6f48a26c0c7467d57)

qa/tasks/qemu: precise repos have been archived

Fixes: https://tracker.ceph.com/issues/51033
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit dcd193c35eba7583613b805ab3941ff3ac5df745)

mgr/dashboard: fix bucket objects and size calculations

Fixes: https://tracker.ceph.com/issues/51035
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 9f5ef98d9c88a91b80e622f16f7061eddff79b2c)

Merge pull request #41511 from ideepika/wip-49592-upgrade-octopus

octopus: qa/upgrade: disable update_features test_notify with older client as lockowner

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

octopus: qa/upgrade: disable update_features test_notify with older client as lockowner

 * with the recent support for async rbd operations from pacific+ when an
 older client(non async support) goes on upgrade, and simultaneously
 interacts with a newer client which expects the requests to be async,
 experiences hang; considering the return code for request completion to
 be acknowledgement for async request, which then keeps waiting for
 another acknowledgement of request completion.

 this if happens should be a rare only when lockowner is an old client
 and should be deferred if compatibility issues arises.

 * qa/upgrade: amend upgrade test workunits to use respective stable branches

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

doc/rados/operations: s/max_misplaced/target_max_misplaced_ratio/

max_misplaced with replaced by in target_max_misplaced_ratio
edbd592ee44e02a5328e1510879555c2f9dcfc9e, but the document was not
sync'ed. let's update it accordingly.

Fixes: https://tracker.ceph.com/issues/50745
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cc43462ab3d4b38cf0c496ae084b82c20ba449ae)

rgw: send headers of quota settings

Make the quota settings visible to s3 and swift users
rgw send headers of quota settings

Fixes: https://tracker.ceph.com/issues/47752
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit 706e077a79306f9c1efd66af7e5d9fdb474f629b)

 Conflicts:
src/rgw/rgw_op.cc

test: add ceph_mksnap and ceph_rmsnap replacements

Signed-off-by: Jeff Layton <jlayton@redhat.com>

15.2.13

test: add test case for MDS privated inos accessing

Fixes: https://tracker.ceph.com/issues/50216
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 0ec4d781787df777bbfdeb742b394c73a0d76e08)

Conflicts:
src/test/libcephfs/test.cc
- octopus version of this file is missing lots of tests that were added to
  master post-octopus

mds: make the lost+found dir accessible from clients

Inode number 0x4 is reserved for the lost+found dir, and the apps
or recovery apps need to access it.

At the same time the 0x3 is reserved for the global snaprealm, which
will also be lookup by the clients when looking up the snaprealm inodes.

Here will make all the inode less than 100 accessible by clients.

Fixes: https://tracker.ceph.com/issues/50216
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 841071b3602aa2b8bcca57b95cdf671d48036a34)

Conflicts:
src/mds/Server.cc
- code being changed does not exist in octopus: change omitted

mds: move the inos 1,2 and 3 macros to ceph_fs.h

For inos 1 and 2 there has two places are defining them, to make
sure we won't miss any one of them, let's define them in only one
header file.

Fixes: https://tracker.ceph.com/issues/50216
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 8615807237458e8cebf9497a7431097cdc56c784)

Conflicts:
src/client/Inode.h
src/mds/CInode.h
src/mds/MDCache.cc
src/mds/Server.cc
        src/mds/SnapServer.h
- change MDS_INO_ROOT to CEPH_INO_ROOT in src/mds/SnapServer.h (octopus-only)

client: don't allow access to MDS-private inodes

Fixes: https://tracker.ceph.com/issues/50112
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 89c511356125f892477ef42bd14c0b447ff06106)

Conflicts:
src/client/Client.cc
- octopus wants ESTALE instead of CEPHFS_ESTALE

client: add testcase for ceph_ll_lookup_vino

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit a956c7986fd43da851918b87c71524f9a2bb1162)

Conflicts:
src/test/libcephfs/test.cc
- octopus is missing a bunch of tests, but this doesn't matter because the
  commit being cherry-picked did not touch those

client: add ceph_ll_lookup_vino

Add a new API function for looking up an inode via a vinodeno_t. This
should give ganesha a way to reliably look up snapshot inodes.

We do need to add some special handling for CEPH_SNAPDIRs. If we're
looking for one, then find the non-snapped parent, and then call
open_snapdir to get the snapdir inode.

Also, have the function check the local cache before calling the MDS
to look up an inode.

Fixes: https://tracker.ceph.com/issues/48991
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 70622079c2ec55222a139fa5042902e0b19bd839)

Conflicts:
src/client/Client.cc

client: make _lookup_ino take a vinodeno_t

Currently, it always leaves the snapid as 0. Rename it to
_lookup_vino and make it fill the snapid from the vinodeno_t
instead, but only when it's a "real" snapid.

Change the existing callers to pass in a vinodeno_t with the
snapid set to CEPH_NOSNAP.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 0e0bebd22dc1abea3f0132debc0b524a01fadf63)

Conflicts:
src/client/Client.cc

client: stop doing unnecessary work in ll_lookup_inode

It's not clear to me why we're looking up the parent and name of the
inode in ll_lookup_inode, as we don't actually do anything with them.
Just return once we get an inode reference.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 2b0b895e888c0f3dece3c1e717ca6e1552e588d8)

mgr/dashboard: fix OSDs Host details/overview grafana graphs

Fixes: https://tracker.ceph.com/issues/50686
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 7d79efb0252e970b3621d1540e69e2115788dcc3)

dashboard: Fixed name clash when hostname similar to anaother

Fixes: #49769
Signed-off-by: Michael Wodniok <wodniok@wor.net>
(cherry picked from commit e97e27ebdbb394c01aa1cf502039a798d9c4e110)
(cherry picked from commit 9f729ff34d487adc7b38913b92fce4f5241b6c43)

mgr/dashboard: fix API docs link

Fixes: https://tracker.ceph.com/issues/50890
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit e79c9d3f51970fbe275087e92e468205f70e6f12)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/dashboard-help/dashboard-help.component.html
      - Adopt the master branch changes.

Merge pull request #41387 from votdev/wip-47471-octopus

octopus: mgr/dashboard: allow getting fresh inventory data from the orchestrator

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #40382 from singuliere/wip-48934-octopus

octopus: rgw/amqp: fix race condition in amqp manager initialization

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #41302 from MrFreezeex/wip-50712-octopus

octopus: librbd/mirror/snapshot: avoid UnlinkPeerRequest with a unlinked peer

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40388 from singuliere/wip-49024-octopus

octopus: rbd/bench: include used headers

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40387 from singuliere/wip-49766-octopus

octopus: librbd: allow interrupted trash move request to be restarted

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #40386 from singuliere/wip-49772-octopus

octopus: librbd/io: send alloc_hint when compression hint is set

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #41321 from trociny/wip-50793-octopus

octopus: osd: don't assert in-flight backfill is always in recovery list

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41311 from k0ste/wip-50778-octopus

octopus: mgr/progress: ensure progress stays between [0,1]

Reviewed-by: Neha Ojha <nojha@redhat.com>

rgw: fix segfault related to explicit object manifest handling

Fixes: https://tracker.ceph.com/issues/50467
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit ae00453515b17d25a5d42430e744855b020ccec6)

rgw/amqp/test: fix mock prototype for librabbitmq-0.11.0

also use extern C for to get compilation errors when
function prototype change

Fixes: https://tracker.ceph.com/issues/50291
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 2ba598ec4c294bd09d2df18ccd2096382e303d39)

rgw: allow rgw-orphan-list to process multiple data pools

We now allow multiple data pools to be specified either on the
command-line or at the prompt. The list should be space-separated.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit bde1278a5de3e9363c0f136b6ebc913bbb5e269d)

rgw/rgw_file: Fix the return value of read() and readlink()

Fixes: https://tracker.ceph.com/issues/49189
Signed-off-by: Dai zhiwei <daizhiwei3@huawei.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
(cherry picked from commit bfd83e8fa142873a0bdf09a4d1ad1b04127f5885)

rgw: read_obj_policy() consults iam_user_policies on ENOENT

when the head object doesn't exist, read_obj_policy() has to decide
whether to return ENOENT or EACCES

when there's a bucket policy, we check whether it has s3ListBucket
permissions. when there's an assumed role, we also need to check
against the role's policies in s->iam_user_policies

Fixes: https://tracker.ceph.com/issues/49780
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5dc9375fa1888242f388f8b502f445f3ddc891f7)

 Conflicts:
src/rgw/rgw_op.cc

rgw: fix bucket object listing when marker matches prefix

When an iniitial marker that ends with a delimiter is provided, it
prevents listing of that "subdirectory" due to new logic at the cls
level to make listing more efficient. The fix catches that situation.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 220ef4b22d1d1667eb4f2c300a0b788e87b9067d)

rgw/notifications: delete bucket notification object when empty

Fixes: https://tracker.ceph.com/issues/49650
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 08e9846fd5a040887e2ddbfca36da97777fc76f6)

 Conflicts:
src/rgw/rgw_pubsub.cc
 - octopus does not have optional_yield arg

mgr/dashboard: allow getting fresh inventory data from the orchestrator

When there is a device change, a `ceph orch device ls --refresh` command
needs to be called so the orchestrator can invalidate its cache and
refresh all devices on all nodes. Currently, the call is asynchronous and
there is no way to determine is a refresh is done or not.

To allow doing a refresh in the Dashboard:
- The inventory device list is periodically updated with cached data.
- If the user clicks the refresh button, a refresh call is sent to the
  orchestrator. Thus if there are device changes, it will be revealed soon
  because of the periodical update.

Fixes: https://tracker.ceph.com/issues/44803
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 1147d6dca04fe6e45b385b549db37775fce54edd)

Conflicts:
src/pybind/mgr/dashboard/controllers/orchestrator.py
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/inventory/inventory.component.ts

Merge pull request #40945 from badone/wip-octopus-tracker-50414

octopus: qa/ceph-ansible: Update ansible version and ceph_stable_release

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #41157 from smithfarm/wip-50365-octopus

octopus: rgw: during reshard lock contention, adjust logging

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #40767 from smithfarm/wip-49472-octopus

octopus: qa: bump osd heartbeat grace for ffsb workload

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40743 from smithfarm/wip-50256-octopus

octopus: mds: trim cache regularly for standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40708 from smithfarm/wip-49475-octopus

octopus: test: use std::atomic<bool> instead of volatile for cb_done var

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #41247 from idryomov/wip-posix-memalign-fix-octopus

octopus: common/buffer: adjust align before calling posix_memalign()

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41237 from trociny/wip-50703-octopus

octopus: os/FileStore: fix to handle readdir error correctly

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41112 from k0ste/wip-50601-octopus

octopus: osd: compute OSD's space usage ratio via raw space utilization

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #40919 from neha-ojha/wip-50405-octopus

octopus: common/options/global.yaml.in: increase default value of bluestore_cache_trim_max_skip_pinned

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40296 from xijiacun/octopus

octopus: rgw: Use correct bucket info when put or get large object with swift

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39978 from singuliere/wip-49053-octopus

octopus: common/mempool: Improve mempool shard selection

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge remote-tracking branch 'origin/octopus-saved' into octopus

15.2.12

mgr/dashboard: fix cookie injection issue

Fixes: CVE-2021-3509
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit b39922818bc57cde1b016e9ad41908b18063b93b)

Conflicts:
src/pybind/mgr/dashboard/controllers/docs.py
        - Remove allow_empty_body and _with_token method

mgr/dashboard: fix base-href: revert it to previous approach

Fixes: https://tracker.ceph.com/issues/50684
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit b6f92922f5c80223fd288d98ce85405a650c0135)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/app.module.ts
     - Adopt the changes coming from master.

(cherry picked from commit fab19ddf55c1e3f1e61745a676785ff0309f11f2)

Merge pull request #40737 from Daniel-Pivonka/octopus-backport-40477

octopus: cephadm: fix failure when using --apply-spec and --shh-user

Merge pull request #40657 from badone/wip-octopus-fix-pytest-double-requirement

octopus: mgr/dashboard: Remove redundant pytest requirement

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: don't assert in-flight backfill is always in recovery list

In PrimaryLogPG::on_failed_pull, we unconditionally remove soid
from recovering list, but remove it from backfills_in_flight only
when the backfill source is the primary osd.

Fixes: https://tracker.ceph.com/issues/50351
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 9b78e00bdf3dd35478e678661606a59424621c4b)

pybind/mgr/dashboard: use setUpClass for initializeing class

instead of relying on __init__(), use setUpClass() to initialize class
for testing. it turns out in pytest > 4, __init__() is called for the
test class but the attributes of the instantiated class is in turn overriden.

so we have to use setUpClass to do this job.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 71979e9b46e21dc3d3cfc6f06f4a84c9b4c7ce78)

 Conflicts:
src/pybind/mgr/dashboard/tests/test_api_auditing.py:
          Differences in import lines
src/pybind/mgr/dashboard/tests/test_tools.py:
          Differences in import lines

tools/setup-virtualenv.sh: pass --use-feature=2020-resolver to pip

as long as pip supports this option, pass it to `pip install`

to silence warnings and errors like:

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

autopep8 1.5.4 requires pycodestyle>=2.6.0, but you'll have pycodestyle 2.5.0 which is incompatible.
pytest-cov 2.10.1 requires pytest>=4.6, but you'll have pytest 3.10.1 which is incompatible.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fa9e2bfd4b3648f08ed3a88ce737d432ab97cce1)

pybind/mgr/dashboard: move pytest into requirements.txt

before this change, pytest is included by both requirements-lint.txt
and requirements-test.txt. this fails the install-deps.sh script when
collecting the python package wheels:

ERROR: Double requirement given: pytest<4 (from -r requirements-test.txt (line 2)) (already in pytest (from -r requirements-lint.txt (line 12)), name='pytest')

also, since pytest is unconditionally imported in the source, for instance,
in pybind/mgr/dashboard/tests/test_ceph_service.py

it would be more straightforward just to include it in requirements.txt.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit eab195566d54122f826debd8efb7f36db78fa4e1)

 Conflicts:
src/pybind/mgr/dashboard/requirements-lint.txt: Additional
        package lines
src/pybind/mgr/dashboard/requirements-test.txt: No mock line

pybind/mgr/dashboard: s/pytest<4/pytest/

to address following failure:

    The user requested pytest<4
    pytest-cov 2.10.1 depends on pytest>=4.6

when building the target of "mgr-dashboard-virtualenv"

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 128778f25eb64cd334e062d627abdb23e0ef0e49)

 Conflicts:
src/pybind/mgr/dashboard/requirements-test.txt: No mock line

Merge pull request #41228 from ceph/wip-yuriw-octopus-p2p

octopus: qa/tests: advanced octopus initial version to 15.2.10

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/progress: ensure progress stays between [0,1]

If _original_pg_count is 0 then progress can be negative.

Fixes: https://tracker.ceph.com/issues/50591
Related-to: https://tracker.ceph.com/issues/50587
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 20990a94598d0249745e2ec25c9197d842119d92)

qa/tests: resolved comments

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #40783 from smithfarm/wip-50286-octopus

octopus: mon: check mdsmap is resizeable before promoting standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40779 from smithfarm/wip-50181-octopus

octopus: cephfs: client: only check pool permissions for regular files

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40778 from smithfarm/wip-50027-octopus

octopus: client: fire the finish_cap_snap() after buffer being flushed

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40777 from smithfarm/wip-49950-octopus

octopus: doc/cephfs/nfs: Add note about cephadm NFS-Ganesha daemon port

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40776 from smithfarm/wip-49934-octopus

octopus: test: reduce number of threads to 32 in LibCephFS.ShutdownRace

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>