removing the image-feature for img0

krdb_data_pool python3 syntax and adding the image-feautre

Merge pull request #33102 from hkadam134/fs-ec-fix

deleting distros

deleting distros

Signed-off-by: hkadam134 <hyelloji@redhat.com>

Merge pull request #32509 from ceph/revert-32264-WIP-removedistro

Revert "[qa]: Removing distros othern than rhel 7.x and rhel 8.x"

Revert "[qa]: Removing distros othern than rhel 7.x and rhel 8.x"

Merge pull request #32264 from racpatel/WIP-removedistro

[qa]: Removing distros othern than rhel 7.x and rhel 8.x

Merge pull request #32468 from Manohar-Murthy/wip-manohar

removing unnecessary ubuntu,centos distros

removing unnecessary ubuntu distros

Merge pull request #32330 from racpatel/wip-centosremove

[qa] : removing centos related TestCases

Merge pull request #32329 from racpatel/wip-hardcodeos

[qa] : removing ubuntu specific test cases - rbd-nbd

Merge pull request #32328 from racpatel/wip-ubunturemove

[qa] cephfs-shell is only available as a package on Ubuntu so removing for RHEL

remove centos reference

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

Merge pull request #32306 from racpatel/wip-removeubuntu

 removeing ubuntu

remove references to os_type "centos"

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

removing ubuntu specific test cases - rbd-nbd

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

cephfs-shell is only available as a package on Ubuntu, removing
test case as its not applicable to RHEL

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

 removeing ubuntu

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

Removing distros othern than rhel 7.x and rhel 8.x

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

Merge pull request #32156 from racpatel/WIP-fsfix

qa: fix suite config for cephfs/bugs

qa: fix malformed suite config

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

Merge branch 'rh-nautilus' of https://github.com/ceph/ceph into rh-nautilus

Merge pull request #32147 from TejasC88/wip-ubuntu-remove

Changes to remove references to os_type "ubuntu" and "centos" hardcode

Merge branch 'wip-ubuntu-remove' of https://github.com/TejasC88/ceph into wip-ubuntu-remove

Changes to remove references to os_type "ubuntu" and "centos" hardcodings

Signed-off-by: TejasC88 <tchandra@redhat.com>

Changes to remove references to os_type "ubuntu" and "centos" hardcodings

Merge pull request #31939 from TejasC88/wip-tejas2

symlink to add beast.yaml to rgw singleton suite

symlink to add beast.yaml to rgw singleton suite

Merge pull request #31820 from racpatel/rh-nautilus

removing radom-distro$

removing random-distro
Merge branch 'rh-nautilus' of https://github.com/ceph/ceph into rh-nautilus

removing radom-distro$

Merge pull request #31803 from racpatel/WIP-PythonTOsh

[qa] fixing 'python -c' in tasks/ceph.manager.py so it works for python 2 and python 3

[qa] fixing 'python -c' in tasks so it works for python 2 and python 3 both

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

Merge pull request #31634 from rakeshgm/rh-nautilus

distro changes

removed random distro softlink

Signed-off-by: rakeshgm <rakeshgm014@gmail.com>

added rhel_7.7 distro yaml

Signed-off-by: rakeshgm <rakeshgm014@gmail.com>

Merge pull request #31423 from rakeshgm/add-rhbuild-version

Add rhbuild versions file

added rhbuild supported verisions

Signed-off-by: rakeshgm <rakeshgm014@gmail.com>

Merge pull request #31395 from racpatel/WIP-ConfigDownStream

[qa] : added downstream_config dir

added downstream_config dir

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

Merge pull request #31391 from racpatel/WIP-rhel8rpm

[qa] : added rhel8 rpm

added rhel8 rpm

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

Merge pull request #31390 from racpatel/WIP-rmUnsupported

added/removed supported distros softlinks

added/removed supported distros softlinks

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

Merge pull request #31389 from racpatel/WIP-rhel8Latest

rhel 8.0 is latest

rhel 8.0 is latest now

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

Merge pull request #31388 from ceph/revert-31386-WIP-rhel8-latest

Revert "rhel 8.0 is latest now"

Revert "rhel 8.0 is latest now"

Merge pull request #31386 from racpatel/WIP-rhel8-latest

rhel 8.0 is latest now

rhel 8.0 is latest now

Signed-off-by: Rachanaben Patel <racpatel@redhat.com>

Merge pull request #31237 from racpatel/WIP-nautilus-rhel8distro

added rhel_8 distro yamls

added rhel_8 distro yamls

14.2.4

ceph-volume tests create a test file for checking unicode output

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests create a shell test for functional unicode

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests verify new logging fallback and encodings in terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume create a logger for the terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: instantiate the new terminal logger in main()

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume terminal remove unicode stream handler

This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

14.2.3

Merge pull request #29945 from trociny/wip-41475-nautilus

nautilus: pybind/mgr/rbd_support: ignore missing support for RBD namespaces

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29975 from theanalyst/nautilus-beast-endpoint-fix

nautilus: rgw: asio: check the remote endpoint before processing requests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29977 from yuriw/wip-yuriw-41513-nautilus

qa/tests: adding mgr.x into the restart/upgrade sequence before monitors

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: adding mgr.x into the restart/upgrade sequence before monitors

Fixes: https://tracker.ceph.com/issues/41513
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

rgw: asio: check the remote endpoint before processing requests

`socket.remote_endpoint()` can throw exceptions corresponding to errors in the
`getpeername` syscall, make sure these are handled.

Fixes: CVE-2019-10222, https://tracker.ceph.com/issues/40018
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit caa653196856ecdf50519a9a33195d5c4e3372af)

Conflicts:
src/rgw/rgw_asio_frontend.cc
conflicts due to missing yield ctx in nautilus

pybind/mgr/rbd_support: ignore missing OSD support for RBD namespaces

During upgrades, the OSDs might not yet have been upgraded yet and
therefore might be missing namespace support.

Fixes: https://tracker.ceph.com/issues/41029
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 10c0c806819741012e1c440c38fac17682125d9a)

pybind/rbd: new OperationNotSupported exception

This will help catch errors where the backing OSDs do not support
a given RBD feature.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 97c1a120b7e7d934dd5d2456fd1fb4c1d0e891ec)

Merge pull request #29918 from badone/wip-nautilus-tracker-41518-grafana-server

nautilus: qa/ceph-ansible: Disable dashboard

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #29899 from alfredodeza/nautilus-rm41378

nautilus ceph-volume tests set the noninteractive flag for Debian

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

nautilus: qa/ceph-ansible: Disable dashboard

Recently a dashboard playbook was added which will fail without a
grafana server so disabling the dashboard

Fixes: https://tracker.ceph.com/issues/41521
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 131f5a145adf21d402e39ac2c151664eb8ccec76)

ceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89231c9a60a46ba0691f787efbfe5ab35f05c679)

Merge pull request #29805 from yuriw/wip-yuriw-41384-nautilus

qa/tests: changed running rbd tests test_librbd_python.sh from…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: changed running rbd tests test_librbd_python.sh from tag: v14.2.2

Fixes: https://tracker.ceph.com/issues/41384
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #29801 from smithfarm/wip-41263-nautilus

nautilus: rgw_file:  advance_mtime() takes RGWFileHandle::mutex unconditionally

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

rgw_file: dont deadlock in advance_mtime()

Fixes: https://tracker.ceph.com/issues/41173
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 0d3338c2d2271bbf6f4c7774ad175f7ee42d07f6)

Merge pull request #28862 from liewegas/wip-bluefs-extents-nautilus

nautilus: os/bluestore/bluefs_types: consolidate contiguous extents

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #29191 from mynaramana/patch-1

doc/rados: Correcting some typos in the clay code documentation

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/rados: Correcting some typos in the clay code documentation
Signed-off-by: Myna <mynaramana@gmail.com>

Merge pull request #29551 from pdvian/wip-40882-nautilus

nautilus: rbd: Reduce log level for cls/journal and cls/rbd expected errors

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29722 from dillaman/wip-39499-nautilus

nautilus: librbd: snapshot object maps can go inconsistent during copyup

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #29723 from dillaman/wip-40511-nautilus

nautilus: librbd: improve journal performance to match expected degredation

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #29725 from dillaman/wip-41078-nautilus

nautilus: mgr/rbd_support: support scheduling long-running background operations

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #29745 from liewegas/wip-bluestore-no-cgroup-nautilus

nautilus: os/bluestore: do not set osd_memory_target default from cgroup limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

os/bluestore/bluefs_types: consolidate contiguous extents

If we allocate a new extent that is contiguous with the last extent,
just extend it.  This avoids having long vectors of continguous extents
when a single large extent would suffice--especially with log files.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4f3ce9bc201c20d10d739d42e3f7018408f110a6)

os/bluestore: do not set osd_memory_target default from cgroup limit

On the aarch64 box I'm testing, this gives us a value of
7378697629483768832, which is not what we want.

I think we are better off relying on this limit being explicitly set via
environment variables (POD_* by kuberentes/rook) or via the command line.

This partially reverts 5c6b533697814af8acfd9e731a2599b2294687ef, but not
all of it, since we wan to keep the option itself, as it is now used by
common/config.cc when dealing with the POD_MEMORY_LIMIT env var.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9346d3c3bcde7d0a68e21a554eb7fc836df78ba9)

Merge pull request #29650 from pdvian/wip-40945-nautilus

nautilus: mgr/dashboard: RGW User quota validation is not working correctly

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #29678 from pdvian/wip-40948-nautilus

nautilus: core: Better default value for osd_snap_trim_sleep

Reviewed-by: Neha Ojha <nojha@redhat.com>

librbd: tweaks to improve throughput for journaled IO

Increase the per-object append flush bytes to 1MiB to mimic the 16MiB
in-memory writeback cache. Finally, default enable the new journal
writethrough until flush option.

Performance results comparing no journal, pre-change, and post-change:

no journal original modified
4K 40310.84 7223.73 27427.27
32K 21530.37 3256.39 10284.05
256K 3159.67 564.73 1627.16
4M 223.03 48.03 91.91

Fixes: http://tracker.ceph.com/issues/40072
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5f3b043896762fa97eb35648f8639004c45f5978)

librbd: new rbd_journal_object_writethrough_until_flush option

When set to true, the journal will not attempt to batch appends until
after it receives the the first flush request from the user.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 172ad649fb8a2410bdc078a5752b2ca09dc03ac2)

Conflicts:
src/librbd/io/ImageRequest.cc: trivial resolution

journal: support dynamically updating recorder flush options

Default to disabling writeback-style append flushes unless overridden
by a call to 'set_append_batch_options'.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c0322a13c83f590067a120212620ebba15fc8661)

journal: fix broken append batching implementation

The original flush_interval/bytes/age batching options no longer
actually do anything. Integrate these better with the max in-flight
AIO handling.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a4e160e410ab3743515338510cb1e369a641152f)

journal: improve logging on recorder append path

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 84e48ddbf58d4028c4195d9c80634f895f62f2be)

journal: wait for in flight advance sets on stopping recorder

Before object overflow detection optimization (83461c42b) the
overflow was detected when trying to send data, so when an object
set was advanced we always had some data and flush always
completed later. But now we detect overflow on client side, it
may happen when buffer is empty and flush may complete
when "advance object set" is still in flight.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 340726ec89c4a2b57905aa1be328d55c96758fee)

journal: optimize object overflow detection

Previously to detect overflow we were sending journal append
requests until -EOVERFLOW is returned by osd. This means that we
had at least one waste (rejected) request per object set (though
there may be more if the number of in-flight appends is not
limited).

We can easily predict when the osd will start to return
-EOVERFLOW and avoid such additional requests.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 83461c42b03cf7d407ac3c5d02043cca1e908015)

Merge pull request #27684 from liewegas/wip-rgw-pgs-nautilus

nautilus: rgw: remove rgw_num_rados_handles; set autoscale parameters or rgw metadata pools

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29478 from xiaoxichen/wip-41002

nautilus: cephfs: client: unlink dentry for inode with llref=0

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29724 from dillaman/wip-40888-nautilus

nautilus: doc/rbd: initial live-migration documentation

Reviewed-by: Mykola Golub <mgolub@suse.com>

rgw: pass mostly_omap flag when opening/creating pools

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit af30e3a512dd1e5c51639d1f823b99ed9b43fb50)

rgw/rgw_rados: pass mostly_omap flag when opening/creating pools

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c4b830069372302ee93687d417d04ad39f600c4e)

rgw: move rgw_init_ioctx() to rgw_tools.cc

move rgw_init_ioctx() out of rgw_rados.cc so that it doesn't need to be
duplicated by services/svc_rados.cc

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1cf90269057ed18636879edef28b6e038352e335)

 Conflicts:
src/rgw/rgw_gc.cc
include conflicts with scope_guard, added rgw_tools include as well which
introduces rgw_init_ioctx