rgw: fix nginx-rgw docs

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit d73b879ac169c46f2dfeba0f4ca7f3a8af272a53)

rgw: create rgw-cache example directory

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit 94c81f3094222d49a9c12d088667f58926b1cbf5)

rgw: fix nginx cache docs

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit 892e9fa5fe2a8759da50e62d48632dff6ad81c30)

use ceph::split

Signed-off-by: Or Friedmann <orfriedma@redhat.com>
(cherry picked from commit 737e1aa95686d8c4a429cac9ea5f388bcda04bac)

rgw: fixes to override_range_hdr

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit e6e1dd868140e7ed356e007d4bf95a5bd2415035)

fix docs

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit a4ae27242a2426cefd602ba053ae565e3c96ff0b)

rgw: Adding data cache and CDN capabilities

This feature is meant to add data cache feature to the RGW.
It is using Nginx as a cache server.
This feature adds 2 new apis, Auth api and Cache api.

Some Performance tests using hsbench:
16K objs:

RGW direct access:
Mode: GET, Ops: 3001, MB/s: 46.89, Lat(ms): [ min: 30.4, avg: 33.2, 99%: 34.7, max: 35.2 ]
Nginx access (objs have not been cached)
Mode: GET, Ops: 1363, MB/s: 21.30, Lat(ms): [ min: 63.8, avg: 73.8, 99%: 78.1, max: 86.6 ]
Nginx access (objs have been cached)
Mode: GET, Ops: 2446, MB/s: 38.22, Lat(ms): [ min: 36.9, avg: 41.0, 99%: 43.9, max: 45.9 ]

512K objs:
RGW direct access:
Mode: GET, Ops: 1492, MB/s: 746.00 Lat(ms): [ min: 60.4, avg: 66.7, 99%: 73.5, max: 75.9 ]

Nginx access (objs have not been cached)
Mode: GET, Ops: 1382, MB/s: 691.00, Lat(ms): [ min: 64.5, avg: 72.1, 99%: 77.9, max: 82.8 ]

Nginx access (objs have been cached)
Mode: GET, Ops: 2947, MB/s: 1473.50, Lat(ms): [ min: 3.3, avg: 32.7, 99%: 62.2, max: 72.1 ]

2M objs:
RGW direct access:
Mode: GET, Ops: 613, MB/s: 1226.00, Lat(ms): [ min: 143.6, avg: 162.0, 99%: 180.2, max: 190.1 ]

Nginx access (objs have not been cached)
Mode: GET, Ops: 462, MB/s: 924.00, Lat(ms): [ min: 180.2, avg: 215.0, 99%: 243.2, max: 248.3 ]

Nginx access (objs have been cached)
Mode: GET, Ops: 1392, MB/s: 2784.00, Lat(ms): [ min: 3.0, avg: 5.3, 99%: 18.8, max: 30.2 ]

10M objs:
RGW direct access:
Mode: GET, Ops: 135, MB/s: 1350.00, Lat(ms): [ min: 191.1, avg: 265.8, 99%: 373.1, max: 382.8 ]

Nginx access (objs have not been cached)
Mode: GET, Ops: 120, MB/s: 1200.00, Lat(ms): [ min: 302.1, avg: 428.8, 99%: 561.2, max: 583.7 ]

Nginx access (objs have been cached)
Mode: GET, Ops: 281, MB/s: 2810.00, Lat(ms): [ min: 3.2, avg: 8.3, 99%: 16.9, max: 25.6 ]

gdal_translate 4GiB image gdal_translate -co NUM_THREADS=ALL_CPUS /vsis3/hello/sat.tif

Nginx (have not cached):
real 0m24.714s
user 0m8.692s
sys 0m10.360s

Nginx (have been cached):
real 0m21.070s
user 0m9.140s
sys 0m10.316s

RGW:
real 0m21.859s
user 0m8.850s
sys 0m10.386s

The results are showing that for objects larger than 512K the cache will increase the performance by twice or more.

For small objs, the overhead of sending the auth request will make the cache less efficient

The result for cached objects in the 10MB test can be explained by net limit of 25 Gb/s(it could reach more)

In Gdal (image decoder/encoder over s3 using range requests) the results were not that different because of Gdal single cpu encoding/decoding.
Gdal have been chosen because of the ability to check the smart cache of the nginx.
https://www.nginx.com/blog/smart-efficient-byte-range-caching-nginx/

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit 1bcd212fd260e8436a2fe8858068c3613b30caed)

Merge pull request #36327 from kotreshhr/fs-volumes-misc-octopus

octopus: mgr/fs/volumes: misc fixes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #36299 from smithfarm/wip-46712-octopus

octopus: mgr/nfs: Check if pseudo path is absolute path

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #36220 from smithfarm/wip-46632-octopus

octopus: mgr/volumes: fix "ceph nfs export" help messages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #36346 from s0nea/wip-46753-octopus

octopus: mgr/dashboard: wait longer for health status to be cleared

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36588 from jan--f/wip-46911-octopus

octopus: ceph-volume: run flake8 in python3

Merge pull request #35925 from smithfarm/wip-45889-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #36563 from s0nea/wip-46896-octopus

octopus: mgr/dashboard: fix tasks.mgr.dashboard.test_rbd.RbdTest.test_move_image_to_trash error

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #36562 from s0nea/wip-46895-octopus

octopus: mgr/dashboard/api: increase API health timeout

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

ceph-volume: run flake8 in pyhton3

Fixes: https://tracker.ceph.com/issues/46897
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 03ec346fa410b8f2816f42d6f0fc5204ff9f1ccf)

Merge pull request #36558 from jan--f/wip-46785-octopus

octopus: ceph-volume: add drive-group subcommand

Merge pull request #35795 from callithea/wip-45449-octopus

octopus: mgr/dashboard: The max. buckets field in RGW user form should be pre-filled

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #36419 from bk201/wip-46794-octopus

octopus: mgr/dashboard: fix regression in delete OSD modal

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36436 from yuriw/wip-yuriw-cleanup-octopus

qa/tests: moved client-upgrade-octopus-pacific under new dir

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #36460 from bk201/wip-46788-octopus

octopus: mgr/dashboard: Configure overflow of popover in health page

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: fix tasks.mgr.dashboard.test_rbd.RbdTest.test_move_image_to_trash error

The API call is a task and the response status is determined by whether
the call is completed within a pre-defined duration (2 seconds) or not.
We should also allow the status when the call takes longer.

Fixes: https://tracker.ceph.com/issues/46812
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit d208d67f9fc92873ba9d63190d7e5f5c202395a8)

mgr/dashboard/api: increase API health timeout

Fixes: https://tracker.ceph.com/issues/46601
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit cd9f680b1121d0a8cb14f260a21b0cab4dbeee0a)

ceph-volume: dependency on python-ceph-common

Since e5b585d15de8b07e0a179344d4187582a5c069f2 ceph-volume depends on
python-ceph-common. This commit introduces this dependency for the
ceph-osd rpm (which includes ceph-volume) and installs the dependency
for tox runs.

Fixes: https://tracker.ceph.com/issues/46772
Fixes: e5b585d15de8b07e0a179344d4187582a5c069f2
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit cb432fe41d4ea8cb71aa592e0727d2da1978121f)

python-common, spec.in: Add missing pyyaml dependency

Fixes: cd6a488ab2ca036dd4fb36751b938f605e97e1c8
Fixes: https://tracker.ceph.com/issues/46759
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0ada965fafb36094194347deebd4778f47bbe407)

 Conflicts:
src/python-common/setup.py
        picked both changes

ceph-volume: add drive-group subcommand

This new subcommand takes a drive group specification as json and deploys
the OSDs accordingly.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Fixes: https://tracker.ceph.com/issues/46689
(cherry picked from commit e5b585d15de8b07e0a179344d4187582a5c069f2)

Merge pull request #36291 from dzafman/wip-46707-octopus

octopus: osd: Cancel in-progress scrubs (not user requested)

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #36123 from aclamk/wip-bluefs-log-replay-rescue-octopus

octopus: bluestore: Rescue procedure for extremely large bluefs log

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #36450 from sebastian-philipp/octopus-backport-35374-35667-36225-35839-36239-36301-36262-36012-36285-36235-36217-3572

octopus: cephadm batch backport August (1)

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #36382 from ceph/fix-rbdmirroring-dropdown-menu

octopus: mgr/dashboard: fix rbdmirroring dropdown menu

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36340 from smithfarm/wip-46742-octopus

octopus: osd: fix crash in _committed_osd_maps if incremental osdmap crc fails

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #36267 from Devp00l/wip-46693-octopus

octopus: mgr/dashboard: Always use fast angular unit tests

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36389 from smithfarm/wip-46798-octopus

octopus: rgw: append obj: prevent tail from being GC'ed

Reviewed-by: Casey Bodley <cbodley@redhat.com>

octopus: mgr/dashboard: fix rbdmirroring dropdown menu

Fixes: https://tracker.ceph.com/issues/46566
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
This seems to be the same situation as https://tracker.ceph.com/issues/44458,
but in a different part of the frontend. Since we are not backporting
ngx-bootstrap to octopus, this has to be manually fixed in octopus.

mgr/dashboard: Configure overflow of popover in health page

Fixes: https://tracker.ceph.com/issues/46657
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit b15b1fe33964ef697aa1179f69188773389aa246)

Conflicts:
      src/pybind/mgr/dashboard/frontend/src/app/ceph/dashboard/info-card/info-card-popover.scss
      - The order of CSS properties changed in the master.

Merge pull request #35879 from ceph/wip-octopus-32493

octopus: ceph-volume: don't use container classes in api/lvm.py

Merge pull request #36409 from shyukri/wip-46148-octopus

octopus: ceph-volume: fix lvm functional tests

Merge pull request #36410 from shyukri/wip-46112-octopus

octopus: ceph-volume: report correct rejected reason in inventory if device type is invalid

mgr/orchestrator: fix placement of '1' activating dry-run bug

when applying a placement of '1' for mon, mgr, rbd-mirror, crash, alertmanager, grafana, node-exporter, prometheus, or mds a dry-run was activated

Fixes: https://tracker.ceph.com/issues/46819
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit 9f7c8f71fc19bf0b123dd2e76ea8025ed0b8ccc9)

cephadm: Fix error bootstraping with '--registry-json'

We need to force setting registry auth options because
these options are unknown before the mgr first starts
and registers them.

Fixes: https://tracker.ceph.com/issues/46777
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 6f8aa25f645cdec1514daecc9b36264fa572234a)

mgr/cephadm: skip Dashboard configuration if Dashboard is disabled

Fixes: https://tracker.ceph.com/issues/46045
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 95a66fed20ed71b31fa5548f50566f9ec780f298)

mgr/orch: disallow unimplemented format types

unimplemented format types can cause the mgr to hang:
$ ceph orch ls --format xml

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit b7eaefa2ee1b415c60744802c75cd34adc093ccc)

mgr/cephadm: track dependency in tox.ini

* track dependency for running tox tests with tox.ini. the fix
  test is not performed by "make check", so `install-deps.sh`
  is not responsible for preparing the autopep8 wheels. hence
  we can just put it in tox.ini
* remove "python_version >= '3'" as we do not support python2

this change addresses a regression introduced by
7d8a3b60321da10af6f4557a9d69eda09ca2c620, which fails the rpmbuild like:

RPM build errors:
    Installed (but unpackaged) file(s) found:
   /usr/share/ceph/mgr/requirements-fix.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a196672b77bf5f1806e0c1c8b906052cf45a6afe)

mgr/cephadm: Introduce tox and autopep8

This PR introduces tox and autopep8 to format the Python code according to PEP8. It does NOT include any lint related things.

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 7d8a3b60321da10af6f4557a9d69eda09ca2c620)

Conflicts:
src/pybind/mgr/tox.ini

cephadm: Deploying alertmanager image is broken

Fixes: https://tracker.ceph.com/issues/46814
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit e401946531d6b151cd696c3ea74fa75dcd209f43)

mgr/cephadm: correct to_json output from str->dict

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 4b8384f0638b6138abed5264e3c9688350fa9987)

cephadm: add unwrap_ipv6 helper method

When we pass in a mon-ip that is ipv6 we want it wrapped, this is so it
can be properly inserted into the mon_addrv address.
But there are times we need to unwrap it to test it's a valid ipv6
address.

This patch adds a helper method `unwrap_ipv6` which takes a string and
returns it either unwrapped or as it is, so it's harmless to other types
of ips.

This allows us to check a wrapped ipv6 base_ip with the networks on the
host.

Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit 13d3bad01954a25dd49b7a7d71bd74dd931f09c5)

build/debian: remove osd_support

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 8ff2824beb78dfd03710e94302f038ea70fb4561)

doc/cephadm: adjust osd removal docs

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 9606f4d6023d2663469b0857ce879ea1ae41c166)

mgr/cephadm: add tests for new remove/replace logic

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit e78f5c8faeedf014aa8590ff47ebbd517a77204a)

mgr/cephadm: implement osd remove/replace logic in cephadm

Fixes: https://tracker.ceph.com/issues/44548
Fixes: https://tracker.ceph.com/issues/45594
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit a0f68c488f74c13534b1b6452a8018f4bb2802d1)

mgr/osd_support: remove module and all traces

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit a55c1dd0fac826978edf22e448c3456e7634cbc3)

Conflicts:
src/mon/MgrMonitor.cc

mgr/cephadm: add qa tests for daemon restart

Added tests to cove daemon stop/start and restart

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 5ff0a1d2daa7d36c9c3284523269c4a1a7daa437)

mgr/cephadm: fix call to cephadm for daemon restarts etc

The call currently passes the daemon_type as the first
parameter - but the function expects the hostname. This
results in failures when attempting daemon restarts through the ceph orch daemon <action> <daemon_id>
command.

Fixes: https://tracker.ceph.com/issues/46740
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 94e418581c9126dc199def6078d6a8449bdec8df)

mgr/cephadm: Add migration to keep the service names consistent

After 15.2.4, we unified some service IDs: MONs, MGRs etc no longer have a service id.
Which means, the service names changed:

mon.foo -> mon
mgr.foo -> mgr

This fixes the data structure consistency

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f2c0b2efae730c972f088a9d25247375c229499f)

mgr/cephadm: minor cleanup in test_migration

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 775d5bedad2703014878d7744d23a25d13bd9bde)

doc/mgr/orchestrator: Fix formatting and examples in create OSDs

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 195f28a6148c41c7502027b96e8f154a99342acf)

mgr/orchestrator: For OSDs, the service_id is the osdspec_affinity

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d04bfb846b78a0a53777a32526eae935e318a606)

mgr/cephadm: get_daemons_by_service should use dd.service_name

daemon_name does not necessarily starts with the service_name.

Especially not for OSDs.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 051361e22246120331795b70513beaaff29ceb51)

python-common: don't assert when building rados url

breaks the `orch ls --format yaml` command

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 26aa14820196143c25d57b4d5517adf6500b19f9)

mgr/cephadm: revamp ceph.conf distribution scheduling

Having an in-memeory list doesn't work properly: Especially
when loading the mgr module, we didn't knwo if we should
deploy confs or not.

Now we only distribute ceph.confs, if there is a new mon_map.
We also store that info now in the config store

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit aa4c3b2a4e9a37f0f92f50d3bafbbd63dfb6fefa)

mgr/cephadm: Add test verifying the initializaiton order

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 537412cc96be40bc4ab920eb5625f7d1338f4569)

vstart: infer the mon public_network

set the mon public_network when deploying with the cephadm flag

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6442662bbe45af2591427e9522eef45604cebbfa)

mgr/orch: allow for multiline OrchestratorEvent message

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit a10861255087b1122f69b6a863d5374ec113bb58)

mgr/cephadm: streamline rgw deployment

cephadm will create realm, zonegroup, and zone if needed before creating rgw service

fixes: https://tracker.ceph.com/issues/43681
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit 131001c453dc3c5d14c56691cdf25353e0127882)

mgr/cephadm: add `orch host ok-to-stop` command

$ ceph orch host ok-to-stop host1
It is presumed safe to stop host host1

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit d6fa2e23015c405775c89c2f1bd55d10c0d8a710)

mgr/cephadm: return HandleCommandResult from ok_to_stop

- return output from the result of the ok_to_stop command
- log ok-to-stop result during all invocations

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 2521a7cef1a5ac0d42f26aabe597a89926814152)

mgr/orch: add errno to OrchestratorError

add errno to OrchestratorError and ServiceSpecValidationError exceptions

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 60b99dcdc767d5520f4dddafe124de4404ee7459)

cephadm: use src/mypy.ini instead

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 148ca82f7483f6564d23472f25733fc1eb743c5e)

Conflicts:
src/cephadm/tox.ini

cephadm: Add tcmu-runner container when deploying ceph-iscsi

Currently when we deploy ceph-iscsi via cephadm it doesn't include a
running tcmu-runner. Which means initiators will be able to login but
you wont see the LUNS on the initiator.

This patch deploys an additional tcmu-runner container along side the
ceph-iscsi container that just runs the tcmu-runner service.

Fixes: https://tracker.ceph.com/issues/46540
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit eb604d3338b32a7a11e36e05df6556dbd95ac4a0)

mgr/orch: Add some more type annotations

Made `orch.Completion` a generic type

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 2310d7fbd092dc28e053f18e354ed01514061610)

mgr/cephadm: allow use of authenticated registry

Add option to use custom authenticated registry during
bootstrap as well as a registry-login command in order
to let user change authenticated registry login info

Fixes: https://tracker.ceph.com/issues/44886
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit a63b030ded2f3ea885883530eb06549356655955)

mgr/cephadm: re-add `apply_drivegroups()`

Fixes: d348d7bf8d3663140c089937b62a0b316b69176b
Fixes: https://tracker.ceph.com/issues/46681
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 6e8f40e4a413e5de291a32185196e31fbef49c90)

doc/cephadm: `status` doesn't show a progress

Fixes: https://tracker.ceph.com/issues/45858
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1125a805bd7c4c4663c7805d3f713d6dd143dcce)

mgr/cephadm: create OSDs in parallel

Should speedup day 1 OSD creation by something like factor 10

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit bd356bae1474884726ff295b23a3a02951a9aedc)

mgr/cephadm: move `forall_hosts()` to utils module

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 031365d357f02cdc5a0743c31672cbc392df8816)

doc/mgr/orchestrator: add missing OSD service_id

example for deploying multiple specs via yaml was missing the service_id

Fixes: https://tracker.ceph.com/issues/46377
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 7906460483c473b639f19d7452e899635552fd82)

python-common: clean-up ServiceSpec.service_id handling

service_id is required for iscsi, mds, nfs, osd, rgw.

any other service_type (mon, mgr, etc.) should not contain a service_id

Fixes: https://tracker.ceph.com/issues/46175
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit eecc8fcbcc2ee88e4c873579c69f2012dac8c652)

doc/cephadm: Fix typos in 'Cephadm Concepts' document.

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 7f5244f6ec44339f0bea8c201ff86dec1b9bb5b5)

doc: add notes about --dry-run

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit a8e21a114bb25bec9b666db61412b2c1768680dc)

mgr/cephadm: rework --dry-run/previews

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit d348d7bf8d3663140c089937b62a0b316b69176b)

Conflicts:
src/pybind/mgr/orchestrator/module.py

mgr/cephadm: adapt tests to have OSDSPEC_AFFINITY

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 917e043c7c7b94a40e6867bb6fd25ff005be76b3)

mgr/cephadm: re-enable osdspec affinity passing

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit edbb4f2db4834800eb3d005e92ad9497d3451668)

qa/tests: moved client-upgrade-octopus-pacific under new dir

The motivation for this chnage is to enable more developers to use and run upgrades tests in more simplified way as "-s upgrades/"

"client-upgrade-octopus-pacific" doesn't fit this logic as it need to use suites from older branches, so that's why they are separated.

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #36332 from yuriw/wip-yuriw-octopus-p2p-octopus

qa/tests: changed rbd/test_librbd_python.sh to run on v15.2.4

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #36143 from rhcs-dashboard/wip-46576-octopus

octopus: mgr/dashboard: add API team to CODEOWNERS

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #36226 from bk201/wip-46351-octopus

octopus: mgr/dashboard: Re-enable OSD's table autoReload

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #36277 from s0nea/wip-46590-octopus

octopus: mgr/dashboard: Use same required field message accross the UI

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36322 from rhcs-dashboard/wip-46736-octopus

octopus: mgr/dashboard: cpu stats incorrectly displayed

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36369 from bk201/wip-44458

octopus: mgr/dashboard: right-align dropdown menu of column filters

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: fix regression in delete OSD modal

The autofocus attribute of `preserve` input makes the form become touched
after the view is loaded. Removing it to avoid
ExpressionChangedAfterItHasBeenCheckedError.

Fixes: https://tracker.ceph.com/issues/46413
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
Co-authored-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit 92f94bd71290d6e2016f56812c417d972cd7b632)

ceph-volume: cleanup code

Simplify the logic and fix a typo.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
(cherry picked from commit 0169b72fff48134ef01802ade38c55281b9f4510)

Conflicts:
src/ceph-volume/ceph_volume/util/device.py
Accept incoming change on octopus

ceph-volume: fix lvm functional tests

This is related to https://github.com/ceph/ceph-ansible/pull/5413 and
adjusts the tests to work with the related ansible fix.

Fixes: https://tracker.ceph.com/issues/46131
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit a0134bb06508738e83456af7e440f418153053b0)

rgw: append obj: prevent tail from being GC'ed

append object tail gets GC'ed otherwise as the state has a manifest similar to
atomic obj processor, but if the manifest exists and the position is correct, it
is not an overwrite and shouldn't be GC'ed

Fixes: https://tracker.ceph.com/issues/42670
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 2dad52a18f889cca4309e78a88c6c5c2c04d4947)

Merge pull request #36247 from dillaman/wip-46674

octopus: librbd: new 'write_zeroes' API methods to suppliment the `discard` APIs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

mgr/dashboard: right-align dropdown menu of column filters

When the value of a filter is too long, the text of value will run out
of the viewport. Right-aligning the menu makes the text visible.

NOTE: This change can't be backported from master because in master we had
switched the bootstrap library to ng-bootstrap and the menu is already
right-aligned in the master.

Fixes: https://tracker.ceph.com/issues/44458
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

mgr/dashboard: Updates jest to fix cache failure

Fixes: https://tracker.ceph.com/issues/45545
Signed-off-by: Stephan Müller <smueller@suse.com>
 Conflicts:
src/pybind/mgr/dashboard/frontend/package-lock.json
- Changed through package change
src/pybind/mgr/dashboard/frontend/package.json
- Conflict because the jest version was even older than what the PR in
master was replacing

mgr/dashboard: Always use fast angular unit tests

This will replace the current approach having a fast testing workaround
that didn't fully work on every test. Now using ng-bullet's optimization
works with every test, therefore there is no need to keep the old
configuration files and use the slow testing on Jenkins.

Fixes: https://tracker.ceph.com/issues/45433
Signed-off-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit 46e20a5e855beb1e9ff822b0c211a3af3a928d0c)

mgr/dashboard: wait longer for health status to be cleared

Because of reasons the cluster needs more time to recover from
HEALTH_WARN while changes are made by `test_pool_update_metadata`.
Lets wait several times for the cluster status to be HEALTH_OK
again.

Fixes: https://tracker.ceph.com/issues/46573
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 739b365a3f6be9557ccb784819d4ad9ad524880f)

qa/suites/rados/thrash/crc-failures: randomly inject bad incremental osdmap crc

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f9399b508c6584e539e323da858372fa7115be3b)