mgr/nfs: fix flake8 missing whitespace around parameter equals error

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 57dba10325b23e52524b2a2c82381b1b5a020964)

mgr/nfs: annotate _cmd_nfs_* methods return value

Fixes: 7c57d1634
Fixes: https://tracker.ceph.com/issues/50783
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 1ec90c804319b32d86212f7402cac93fb14cb1d2)

doc/cephfs/nfs: add section about ganesha logs

Fixes: https://tracker.ceph.com/issues/50008
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 6ff3c1da87d8a4fe3750065a554e348e1882f240)

doc/cephfs/nfs: Replace volume/nfs with nfs

Recently, nfs related code was moved out of volumes plugin[1]. So using the
name volume/nfs for the interface is not appropriate.

[1] https://github.com/ceph/ceph/pull/40526

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit ff11cdb4bc7cfc0a51ca047e6b7c8c686075b2ec)

doc/cephfs/nfs: add note about export management with volume/nfs interface only

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 923e689e0828ac5186123a2ffc57db01465cbdd0)

spec: add nfs to spec file

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 8ab49501719f4d3d997a3b4e0f500ac985e9558f)

mgr/nfs: Don't enable nfs module by default

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 2a99da961adac780ac45dd9e1073d5e695c0c1f5)

mgr/nfs: check for invalid chars in cluster id

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 07e7263936191f6be07d7e03be2de16aa8bffd58)

mgr/nfs: Use CLICommand wrapper

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 7c57d1634049484b3037d297e0955b14dcac9654)

mgr/nfs: reorg nfs files

This patch does the following:
- removed export dir and moved its files to nfs dir
- renamed nfs.py to export.py

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 6045c4690b75b8be8969c67d5a5d5caf6196abea)

mgr/nfs: Check if transport or protocol are list instance

This is to fix "int object is not iterable” error.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit ef4d7d7f2c6faa4d755d2da9d5f0fd88c9f81c09)

mgr/nfs: reorg cluster class and common helper methods

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit f8adac3e709f85707910d09acb8667fd861a2fa9)

mgr/nfs: move common export helper methods to ExportMgr class

This reorganization is to support RGW exports in future PR.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit c6288a69a02434c5419d5b92c8e79d8206fad350)

mgr/nfs: move validate methods into new ValidateExport class

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit be22f0592e3123bf121b572cbb22fc8d5d329d4f)

mgr/nfs: add custom exception module

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 3a08139452f6ebeff9eda6d2743ac27741d07380)

mgr/nfs: create new module for export utils

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit d064306dd3afcbab620c5489654359bb2f911c3d)

mgr/nfs: rename fs dir to export

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 23c63ccb7b5cdff834bed0ab7302ed3eb51ea1c8)

mgr/volumes/nfs: Move nfs code out of volumes plugin

This PR is to support integration of dashboard with volumes/nfs plugin.

Fixes: https://tracker.ceph.com/issues/50080
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit af9336316f978a9f67faafad2eb3e608ed2616b1)

Merge pull request #41005 from mgfritch/pacific-backport-40411

pacific: mgr/volumes/nfs: drop type param during cluster create

Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #40996 from batrick/i50258

pacific: qa: convert some legacy Filesystem.rados calls

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #40855 from batrick/i50254

pacific: mds: standby-replay only trims cache when it reaches the end of the replay log

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #40685 from singuliere/wip-50183-pacific

pacific: client: fix the opened inodes counter increasing

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #41402 from rhcs-dashboard/wip-50881-pacific

pacific: mgr/dashboard: update frontend deps due to security vulnerabilities

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: update frontend deps due to security vulnerabilities.

Fixes: https://tracker.ceph.com/issues/50800
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 7ee75880c1c600a25f05f98c280be87758579901)

Conflicts:
  - src/pybind/mgr/dashboard/frontend/package-lock.json
    Resolved conflicts.

mgr/dashboard: get SASS vars values in TS through CssHelper.

Now SASS vendor variables' values can be retrieved by var name via CssHelper.

We were relying on the angular webpack config that has changed in the latest v10.x and v11.
See: https://github.com/angular/angular-cli/issues/19622#issuecomment-744395101

Fixes: https://tracker.ceph.com/issues/50800
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 7310e15a9bb704c3b214b9b5373807b1167df009)

Merge pull request #41324 from rhcs-dashboard/wip-50772-pacific

pacific: mgr/dashboard: fix OSDs Host details/overview grafana graphs

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: wornet-mwo <NOT@FOUND>

Merge pull request #41200 from votdev/wip-50662-pacific

pacific: mgr/dashboard: disable NFSv3 support in dashboard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #41298 from rhcs-dashboard/wip-50543-pacific

pacific: mgr/dashboard: fix HAProxy (now called ingress)

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41296 from rhcs-dashboard/wip-50769-pacific

pacific:  mgr/dashboard: RGW buckets async validator performance enhancement and name constraints

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41136 from ideepika/wip-pacific-40572

pacific: qa/config/rados: add dispatch delay testing params

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: add Services e2e tests

Fixes: https://tracker.ceph.com/issues/50567
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Introducing e2e tests for service creation for Ingress and RGW service types.

mgr/dashboard: disable NFSv3 support in dashboard

Fixes: https://tracker.ceph.com/issues/49718
Related to: https://github.com/ceph/ceph/pull/40154

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 864a362389e988bc9aab23f66da608b4ceab74c0)

Merge remote-tracking branch 'origin/pacific-saved' into pacific

16.2.4

mgr/dashboard: fix cookie injection issue

Fixes: CVE-2021-3509
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit b39922818bc57cde1b016e9ad41908b18063b93b)

systemd: remove `ProtectClock=true` for `ceph-osd@.service`

Ceph 16.2.0 Pacific by https://github.com/ceph/ceph/commit/9a84d5a introduce following new systemd restriction:

    ProtectClock=true
    ProtectHostname=true
    ProtectKernelLogs=true
    RestrictSUIDSGID=true

BTW, `ceph-osd@.service` failed with `ProtectClock=true` unexpectly, also see:

  - <https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/TNBGGNN6STGDKARAQTQCIPTU4KLIVJQV/>
  - <https://serverfault.com/questions/1059317/bluestore-var-lib-ceph-osd-ceph-2-block-read-bdev-label-failed-to-open-var-l>

This PR intruduce:

  - Remove `ProtectClock=true` for our systemd service templates

Fixes: https://tracker.ceph.com/issues/50347
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
(cherry picked from commit 85bc551b179d940a50cbdfd0c20848e3187c70a6)

mgr/dashboard: fix set-ssl-certificate{,-key} commands

- Now create-self-signed-cert command relies on set-ssl-certificate{,-key} commands.
- Simplify the command testing and increase the test coverage.

Fixes: https://tracker.ceph.com/issues/50519
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit fb20990bca55ffe1712a905c3a9741e8df004e74)
(cherry picked from commit 3bdbbb95a05b26e84e09a550d0bde3facf3a76d4)

mgr/dashboard: fix base-href: revert it to previous approach

Fixes: https://tracker.ceph.com/issues/50684
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit b6f92922f5c80223fd288d98ce85405a650c0135)
(cherry picked from commit 6b9c8ca076ef63df64ee12f89132ed2f46c4b739)

mgr/dashboard: fix OSDs Host details/overview grafana graphs

Fixes: https://tracker.ceph.com/issues/50686
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 7d79efb0252e970b3621d1540e69e2115788dcc3)

dashboard: Fixed name clash when hostname similar to anaother

Fixes: #49769
Signed-off-by: Michael Wodniok <wodniok@wor.net>
(cherry picked from commit e97e27ebdbb394c01aa1cf502039a798d9c4e110)

Merge pull request #41313 from ceph/wip-yuriw-p2p-pacific

pacific: qa/tests: corrected point versions to reflect latest releases

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tests: corrected point versions to reflect latest releases

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #41208 from ceph/wip-yuriw-p2p-pacific2

pacific: qa/tests: initial checkin for pacific-p2p suite (2)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #41079 from tchaikov/pacific-rocksdb

pacific: rocksdb: pickup fix to detect PMULL instruction

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41232 from neha-ojha/wip-50381-pacific

pacific: systemd: remove `ProtectClock=true` for `ceph-osd@.service`

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41182 into pacific

* refs/pull/41182/head:
ceph_test_rados_api_service: stop threads before asserting
ceph_test_rados_api_service: more retries for servicemkap

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #40729 into pacific

* refs/pull/40729/head:
qa/distros/podman: preserve registries.conf

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #41219 into pacific

* refs/pull/41219/head:
mgr/cephadm: misc. autopep8 changes
mgr/cephadm: add timeout when removing iscsi gateway.conf
mgr/cephadm: mock subprocess calls in tests that remove iscsi daemons
qa/tasks/cephadm_cases: longer wait for osd to start
mgr/cephadm: apply hostname/addr checks to 'orch host set-addr' too
mgr/cephadm: make 'host add' idempotent
mgr/cephadm: set host crush location based on HostSpec
python-common: add location property to HostSpec, + tests
doc/cephadm: rewriting "starting the upgrade"
doc/cephadm: rewrite "monitoring the upgrade"
doc/cephadm: rewrite "cancelling the upgrade"
doc/cephadm: rewrite "potential problems"
doc/cephadm: rewrite "using customized con..."
Add Rocky Linux to supported DISTRO_NAMES
mgr/cephadm: remove haproxy+keepalived container image from service spec
mgr/cephadm:  extend disk selector test which disk size may be decimal
mgr/cephadm: When device size contains the decimal, it can not match size exactly.
mgr/dashboard: fix HAProxy (now called ingress)
mgr/cephadm: Purge iscsi configuration from pool and ceph config

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #40813 from rhcs-dashboard/wip-50034-pacific

pacific: build(deps): bump node-notifier from 8.0.0 to 8.0.1 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: Detailed error messages in rgw bucket name validation

Explain the rgw bucket name constrains for each bucket name validation
errors.

Fixes: https://tracker.ceph.com/issues/50516
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit ab04e536684024c8d3613907d4bcd72fddf2ef20)

mgr/dashboard: RGW buckets async validator performance enhancement

The rgw bucket creation form has the Name field which have an async
validator. The validator calls all the bucket name and check if the
entered name is unique or not. This happens on every keystroke. So if
100 or more buckets are there, then the async validation can be real
    slow and causes misvalidations in different fields.

I changed the validation logic and did some cleanups to improve the
performance of the async validation.

Fixes: https://tracker.ceph.com/issues/50514
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 005327c4e12fef8d7054894d9df021c0b3c53e19)

Merge pull request #40982 from rhcs-dashboard/wip-50427-pacific

pacific: mgr/Dashboard: Remove erroneous elements in hosts-overview Grafana dashboard

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #41259 from petrutlucian94/wip-50726-pacific

pacific: Update boost url, fixing windows build

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40484 from gregsfortytwo/wip-stretch-mon-location-pacific-base

pacific: Improve mon location handling for stretch clusters

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41203 from rhcs-dashboard/wip-50680-pacific

pacific: mgr/dashboard: avoid data processing in crush-map component

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41197 from rhcs-dashboard/wip-50674-pacific

pacific: mgr/dashboard: fix bucket versioning when locking is enabled

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

rgw: sanitize \r in s3 CORSConfiguration's ExposeHeader

follows up on 1524d3c0c5cb11775313ea1e2bb36a93257947f2 to escape \r as
well

Fixes: CVE-2021-3524
Reported-by: Sergey Bobrov <Sergey.Bobrov@kaspersky.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 87806f48e7a1b8891eb90711f1cedd26f1119aac)

rgw: RGWSwiftWebsiteHandler::is_web_dir checks empty subdir_name

checking for empty name avoids later assertion in RGWObjectCtx::set_atomic

Fixes: CVE-2021-3531
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7196a469b4470f3c8628489df9a41ec8b00a5610)

Merge pull request #41251 from rhcs-dashboard/wip-50723-pacific

pacific: mgr/dashboard: fix base-href: revert it to previous approach

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #40054 from alimaredia/wip-rgw-dpp-pacific

Pacific: Add DoutPrefixProvider for RGW Log Messages in Pacfic

Reviewed-by: Casey Bodley <cbodley@redhat.com>

win*.sh,cmake: Fix Windows linking errors

The Windows build is hitting linking errors after
bumping the Boost version to 1.75. The issue is that Boost
is now setting the zlib dependecy using INTERFACE_LINK_LIBRARIES,
which means that it's no longer located using the standard
"find_package" mechanism.

In order for the linker to locate zlib, we'll add it to the
linker search path.

[1] https://github.com/boostorg/boost_install/issues/47

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit 0af38d179e48a48354a863e2b7f6478ecd5db318)

win32*.sh: ensure that the build dir exists

The Windows build scripts try to use the build dir before
actually creating it.

We'll have to move up the "mkdir" command a few lines.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit c6404e15cb2cde046224433403349e7437f3044e)

win32*.sh: bump boost version to 1.75

We're updating the Windows build scripts to use Boost 1.75, the same
as the Linux build.

We can now drop the custom Boost patches that were needed in order
to use mingw. Those have been merged upstream in the meantime.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit b683c55283f79fef2de349d08736fe1781e8f6b0)

win32*.sh: improve logging

We'll use a log file when building the Ceph dependencies. Also, we'll
log some details about the operation that's about to be performed.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit 9cad7db13a1176ac3b3fee89983d6d21420f5429)

win32*.sh: fix boost url

The boost download location has changed, breaking the Windows build.
Ths commit updates the boost download url.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit f1a8044f1da0f7171ee189b1c0c5f1e519694777)

mgr/dashboard: fix base-href: revert it to previous approach

Fixes: https://tracker.ceph.com/issues/50684
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit b6f92922f5c80223fd288d98ce85405a650c0135)

systemd: remove `ProtectClock=true` for `ceph-osd@.service`

Ceph 16.2.0 Pacific by https://github.com/ceph/ceph/commit/9a84d5a introduce following new systemd restriction:

    ProtectClock=true
    ProtectHostname=true
    ProtectKernelLogs=true
    RestrictSUIDSGID=true

BTW, `ceph-osd@.service` failed with `ProtectClock=true` unexpectly, also see:

  - <https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/TNBGGNN6STGDKARAQTQCIPTU4KLIVJQV/>
  - <https://serverfault.com/questions/1059317/bluestore-var-lib-ceph-osd-ceph-2-block-read-bdev-label-failed-to-open-var-l>

This PR intruduce:

  - Remove `ProtectClock=true` for our systemd service templates

Fixes: https://tracker.ceph.com/issues/50347
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
(cherry picked from commit 85bc551b179d940a50cbdfd0c20848e3187c70a6)

Merge pull request #41212 from neha-ojha/wip-41211-pacific

pacific: qa/suites/rados/standalone: remove mon_election symlink

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mgr/cephadm: misc. autopep8 changes

Some of the file in mgr/cephadm are getting changed
every time I want to run the mgr tox tests to check
changes and it's inconvenient to have to check the
files out every time

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 7a3ec28cc47ffba044f3820deee77f70b496fce7)

mgr/cephadm: add timeout when removing iscsi gateway.conf

We already catch TimeoutExpired exceptions here but we don't
actually specify a timeout

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit ade46fda791ee2c2e1cac1c2376d56acf7ae4766)

mgr/cephadm: mock subprocess calls in tests that remove iscsi daemons

tests that remove iscsi daemons use subprocess.run to make
a call to rados to clear out the gateway.conf. We don't want
the tests to actually make that call.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit c5b984b10c2d6f2eef02e9dbf74aea64bf11f19a)

rgw: add DPP's to logging for most ops

This commit contains the following major
changes:

- Most log output lines for major ops now have
DoutPrefixProviders.

- Create new macro ldpp_subdout. This macro is meant
as a replacement for lsubdout for headerfiles that
do not define the rgw subsys.

- Changes to RGWObjManifest begin and end iterators.

- Make it so that rgw_dencoder.cc should only contain the
logic related to encoding.

Also add dpp to ldouts and lderr's already using
req_state and replace sync_env/env->dpp's with dpp's

Signed-off-by: Ali Maredia <amaredia@redhat.com>
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6cfc2ce2ce669e139acb682898c53c5602075e31)

Merge pull request #41204 from rhcs-dashboard/wip-50652-pacific

pacific: mgr/dashboard: Generate NPM dependencies manifest

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

qa/tasks/cephadm_cases: longer wait for osd to start

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a103fd6955bb89f857f56a4ec73d77c3a8818d78)

mgr/cephadm: apply hostname/addr checks to 'orch host set-addr' too

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b5d39be1881775f141dfb3f10ebe4ae3601b75e6)

mgr/cephadm: make 'host add' idempotent

Do not let prime_empty_host blow away inventory.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a8acc354dc7dc1245f5030a751f4fc5ddb689b18)

mgr/cephadm: set host crush location based on HostSpec

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2f2c997652539795845184f938a70e67d245705c)

python-common: add location property to HostSpec, + tests

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d2a9a35993e11b726e94d5dec9a5fe098f6e6eba)

doc/cephadm: rewriting "starting the upgrade"

This PR fixes the prompts in this section and
makes the sentences a tiny bit less onerous for
the reader.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 9dc63ab22aa14f0790f11f672a3f865abfa738cc)

doc/cephadm: rewrite "monitoring the upgrade"

This PR adds correctly-formatted prompts and
makes slight alterations to the readability
of the text.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit f4c16f05b51fa4b2ff0095f331f7d2a4e1e3ca68)

doc/cephadm: rewrite "cancelling the upgrade"

This PR corrects a single prompt in the section
"Cancelling the Upgrade".

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 8d531a754890f054c3533c0a985599b10acf8dc7)

doc/cephadm: rewrite "potential problems"

This PR adds correct prompts and slightly
reduces the cognitive load of the reader.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 39c05445f211d3033571afa907443d5e7eb88f91)

doc/cephadm: rewrite "using customized con..."

This PR rewrites the text in "Using Customized
Container Images" so that it is just a bit
clearer, and it also formats the prompt in the
text correctly.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 630f0c8ad305c074c399c35b9547a9a6a501af3b)

Add Rocky Linux to supported DISTRO_NAMES

Rocky Linux is a RHEL clone. I did a test-installation of ceph pacific on Rocky Linux RC1 with cephadm. As far as I can see, everything works as expected.

Signed-off-by: Dennis Körner <koerner@netzwerge.de>
(cherry picked from commit 188e6b3fc56960ea731c71e57af64eb8c1fc76ea)

mgr/cephadm: remove haproxy+keepalived container image from service spec

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 729ebaabd539f1a09c7cfcdc69dff1e2cb409b0d)

mgr/cephadm:  extend disk selector test which disk size may be decimal

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
(cherry picked from commit 9acd4be8bd56c9d0e848a336be4552b705654a62)

mgr/cephadm: When device size contains the decimal, it can not match size exactly.

for example, osd.yaml:
service_type: osd
service_name: test
placement:
  host_pattern: '*'
spec:
  data_devices:
    size: 3.6TB

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
(cherry picked from commit fd30aff672a1908a37163472f9c6478eca4e98e2)

mgr/dashboard: fix HAProxy (now called ingress)

Fixes: https://tracker.ceph.com/issues/50319
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Support from Dashboard UI to create Ingress service type.

(cherry picked from commit 21318e8fa965c352e00d84c04ee072dd9fe45e4f)

mgr/cephadm: Purge iscsi configuration from pool and ceph config

Remove gateway.conf from iscsi pool when service is removed
Remove iscsi ceph config keys
Remove iscsi dashboard gateways config from dashboard

fixes: https://tracker.ceph.com/issues/48930

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 1b9e3edcfd1c1a3dd02d6eb14072494f57b086a8)

Merge remote-tracking branch 'origin/pacific-saved' into pacific

Merge PR #40557 into pacific

* refs/pull/40557/head:
win32*.sh: disable libcephsqlite when targeting Windows

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mgr/dashboard: fix bucket versioning when locking is enabled

Fixes: https://tracker.ceph.com/issues/50545
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit e90ce333c69725f870acc6665c5e9bb4aa33de3f)

qa/suites/rados/standalone: remove mon_election symlink

The standalone tests need parameters to be passed as ceph_args to
override defaults.

This was just doubling the number of standalone tests being run in each rados
run with no effect!

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit c8e48c5c2517c5a28705a99339e99fa8fd8d3e96)

qa/tests: try focal instead of bionic

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

qa/tests: initial checkin for pacific-p2p suite (2)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

mgr/dashboard: Generate NPM dependencies manifest

A txt file with all the dependencies and its version & url link.
Fixes: https://tracker.ceph.com/issues/50515
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 825ea98915bcb0ab4bbaefb478ee0a0b8e506933)

mgr/dashboard: avoid data processing in crush-map component

Fixes: https://tracker.ceph.com/issues/49236
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 5454426a0a72ab9a44388c81ede8c2f20e35de6e)

16.2.3

cephadm: normalize image digest in 'ls' output too

We were previously only normalizing the 'inspect' output.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 06d849acfb77a189abf6e9c26b27aaaefdd77f14)

Merge PR #41151 into pacific

* refs/pull/41151/head:
mgr/cephadm: ceph-volume verbose only when fails
qa/workunits/cephadm/test_cephadm: test zap-osds
cephadm: add --zap-osds argument to rm_cluster
cephadm: implement zap-osds --fsid ... command
doc/cephadm: add podman version note to install
mgr/cephadm: check hostname resolution before adding host
cephadm: provide a way to checkhost connection without /etc/hosts passed the shell
doc/cephadm: remove /etc/hosts from list of hostname resoltion methods
qa/suites/rados/cephadm/smoke-roleless: test client-keyring
qa/tasks/cephadm.py: adjust client.admin key mode; place on all hosts
cephadm: distribute client.admin keyring+conf to label:_admin on bootstrap
doc/cephadm: document the default 'admin' label
mgr/cephadm: 'ceph orch client-keyring ...' commands to manage keyring files
mgr/cephadm: reimplement ceph.conf pushing
mgr/cephadm: use _write_remote_file for ceph.conf
mgr/cephadm: _write_remote_file helper
mgr/cephadm: add placementspec for which hosts get ceph.conf
mgr/cephadm: skip ok-to-stop for mons in upgrade if < 3 mons
mgr/cephadm: don't allow upgrade start with less than 2 mgrs
cephadm: re-assimilate user provided conf after mgr created
cephadm: allow several public networks be matched
mgr/cephadm: The command of 'ceph orch daemon restart mgr.xxx' may case mgr daemon loop to restart
doc/cephadm: add a single word
doc/cephadm: adding "device" to a sentence
doc/cephadm: rewrite "nfs.rst"
mgr/cephadm: s/_hosts_with_daemon_inventory/_schedulable_hosts/
mgr/cephadm: don't remove daemons from hosts in maintenance or offline  mode
mgr/cephadm: default status for daemons on maintenance hosts to stopped
qa/tasks/cephadm: fix ctx archive check for teuthology
python-common: use OrderedDict instead of Set to remove duplicates from host labels list
mgr/cephadm: less noise about osd specs
mgr/cephadm: kick serve loop when adding/removing labels
mgr/cephadm: do not place osds on _no_schedule hosts
doc/cephadm: document _no_schedule label
mgr/cephadm: fix 'orch ls' count to reflect schedulable hosts
mgr/cephadm: do not schedule on _no_schedule hosts
doc/cephadm: osd.rst -- removing colons
doc/cephadm: osd: rewrite "additional opts"
doc/cephadm: rewrite "advanced osd s. specs"
doc/cephadm: rewrite "delcarative state" in osd.rst
mgr/MgrStandby: fix config observer
mgr/MgrStandby: respawn if mgr_standby_modules changes
qa/tasks/mgr/test_dashboard: skip test_standby if mgr_standby_modules=false
qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
qa/suites/rados/cephadm/smoke-singlehost: test --single-host-defaults
cephadm: add --single-host-defaults option to bootstrap
mgr/cephadm: allow mgr colo if mgr_standby_modules=false
mgr/MgrStandby: add mgr_standby_modules option
cephadm: ignore apparmor if profiles file is empty

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>