mgr/cephadm: ingress: support nfs

- use consistent hashing
- statically map across ranks
- disable backend checks so that clients don't move

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 51f0dedbbb996fdaca4ba866791c2ac86d4429c6)

mgr/cephadm: do not reconfigure daemons on deleted services

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 444663b740da4e91cdf318f2661fc5cc0b76557c)

mgr/cephadm: nfs: shell out to rados tool for conf creation

This avoids any hangs due to rados.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 917fb597673ade3b436854d17aef559447540a85)

mgr/cephadm: nfs: add rank to grace file from mgr module

Do the grace file manipulation from the mgr module.  For add, this isn't
especially important, but for remove it is very important.  Clean out
old ranks from the grace table before we record that the rank has been
purged from the rank_map.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 74169045a9b212d4b549159a8f7870ed6e9119ea)

mgr/cephadm: nfs: bind ganesha to appropriate ip:port

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ae4ab5d2041856c8a891e25df610b88bc07b14f1)

mgr/cephadm: enable ranked daemons for nfs

Use ranked daemons for NFS. Ganesha does not like it if multiple
instances start up with the same rank, but we need stable ranks so that
a rank can "fail over" to a new instance of a new daemon on another host
(with the same rank) for NFS client reclaim to work.

Specify a nodeid of '{service_name}.{rank}' for ganesha.

Include a unique id in the daemon_id just because this avoids some issues
with the create/destroy ordering, and because the daemon_id doesn't matter
much anymore since we are using a stable rank.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a7d65f00ea554078f6e8f058e9ec137230902964)

mgr/cephadm: support creation of daemons with ranks

- we need to assign all names and update the rank_map before we start
creating daemons.
- if we are using ranks, we should delete old daemons first, and
fence them from the cluster (where possible).

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e8b07982b50e35a546a52b092cac5a825c37981d)

mgr/cephadm: make _plan show removed daemon names

This is more informative than just the hostnames.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1b0980752f159e93a72d73094125dd8d4ec582cc)

mgr/cephadm/schedule: assign/map ranks

If we are passed a rank_map, use it maintain one daemon per rank, where
the ranks are consecutive non-negative integers starting from 0.

A bit of refactoring in place() so that we only do the rank allocations
on slots we are going to use (no more than count).

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit de03942f806cb16e00259f2397446afb9322c3f1)

mgr/cephadm: add rank[_generation] properties

DaemonDescription
CephadmDaemonDeploySpec
DaemonPlacement
unit.meta
get_unique_name() (we include it in the daemon_id)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5e8f184be8091fdea8d7856a243dca913180e2a9)

mgr/cephadm/inventory: store optional rank_map along with specs

The rank_map is a bit of state to keep track of which ranks are
occupied by which generation and daemon_id.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d0d2232c0542170b0908d8d79d6fc50eaa053d94)

mgr/cephadm: include service_name is generated DaemonDescription

This makes 'orch ls' match up daemosn to services (and probably cleans up
other bits and pieces) when the old daemon id -> service name calc code
can't do its thing.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 11ff4842a207c82dd4ec8254aa370962cde24c82)

mgr/orchestrator: include service_name in DaemonDescription dump

('orch ps')

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 15e5c0a3accf8284d22df3b8f98b0127eec02834)

mgr/cephadm/inventory: fix deleted check

Look in dict, not encoded JSON string

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a282c3c249fcd901bf8005f9f8e581ceb0ff6225)

mgr/cephadm: simplify

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 991515aaaa19eeee93f6b98b6721ed2b3efbff1b)

mgr/cephadm/schedule: make placement shuffle deterministic

hash(str) is non-deterministic, probably because it is using the internal
object ID or something and not the string content?

In any case, explicitly hash the string content and use that instead.

Also, sort the input pre-shuffle to ensure that variations in the original
host list ordering don't screw with the result.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit adceaa9b28278601c56a7db1c3f42eaa592ec4d1)

mgr/cephadm: document CephadmService flags

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2fa80d8769a5b4ab1238a9b18627b9e28a540b66)

doc/cephadm: fix prompts in service-management.rst

This PR formats the prompts in service-managment.rst
properly.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit db3af8e3d2c6aceced0176930717546496147a3b)

qa/tasks/cephadm: Include bootstrap registry options for downstream
- registry-url, registry-username and registry-password bootstrap options are
supported now. This is needed to access monitoring service container images.
- usage of RHEL distribution based cephadm in download_cephadm task.

Signed-off-by: sunilkumarn417 <sunnagar@redhat.com>
(cherry picked from commit 364fb5899bc87ca0da802293946ef8f2dcbd3c4e)

qa/suites/rados: include rook test in rados

This just to make sure we don't break mgr/orchestrator.

Note that we already symlink ../orch/cephadm, so this makes rados
include all of orch/.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5db5c8c292e70a643d91f19e1b21da3be65ecb34)

Merge pull request #41646 from rhcs-dashboard/wip-51063-pacific

pacific: mgr/dashboard: fix bucket objects and size calculations

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41320 from trociny/wip-50794-pacific

pacific: osd: don't assert in-flight backfill is always in recovery list

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41312 from k0ste/wip-50779-pacific

pacific: mgr/progress: ensure progress stays between [0,1]

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #41236 from trociny/wip-50702-pacific

pacific: os/FileStore: fix to handle readdir error correctly

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41125 from sseshasa/wip-50606-pacific

pacific: osd: Override recovery, backfill and sleep related config options during OSD and mclock scheduler initialization.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #41113 from k0ste/wip-50602-pacific

pacific: osd: compute OSD's space usage ratio via raw space utilization

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #40566 from singuliere/wip-49992-pacific

pacific: common/mempool: only fail tests if sharding is very bad

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41468 from batrick/i50873

pacific: mon,doc: deprecate min_compat_client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40689 from singuliere/wip-50014-pacific

pacific: debian,cmake,cephsqlite: hide non-public symbols

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: fix bucket objects and size calculations

Fixes: https://tracker.ceph.com/issues/51035
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 9f5ef98d9c88a91b80e622f16f7061eddff79b2c)

Merge pull request #41633 from rhcs-dashboard/wip-51036-pacific

pacific: mgr/dashboard: pass Grafana datasource in URL

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41528 from rhcs-dashboard/wip-50968-pacific

pacific: mgr/dashboard: show RGW tenant user id correctly in 'NFS create export' form

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge PR #41053 into pacific

* refs/pull/41053/head:
cephfs-top: set the cursor to be invisible
cephfs-top: self-adapt the display according the window size
cephfs-top: use the default window object from curses.wrapper()
cephfs-top: improve the output

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41389 into pacific

* refs/pull/41389/head:
qa/tasks/nfs: add test to check if cmds fail on not passing required arguments
mgr/nfs: fix flake8 missing whitespace around parameter equals error
mgr/nfs: annotate _cmd_nfs_* methods return value
doc/cephfs/nfs: add section about ganesha logs
doc/cephfs/nfs: Replace volume/nfs with nfs
doc/cephfs/nfs: add note about export management with volume/nfs interface only
spec: add nfs to spec file
mgr/nfs: Don't enable nfs module by default
mgr/nfs: check for invalid chars in cluster id
mgr/nfs: Use CLICommand wrapper
mgr/nfs: reorg nfs files
mgr/nfs: Check if transport or protocol are list instance
mgr/nfs: reorg cluster class and common helper methods
mgr/nfs: move common export helper methods to ExportMgr class
mgr/nfs: move validate methods into new ValidateExport class
mgr/nfs: add custom exception module
mgr/nfs: create new module for export utils
mgr/nfs: rename fs dir to export
mgr/volumes/nfs: Move nfs code out of volumes plugin

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40634 into pacific

* refs/pull/40634/head:
qa: check mounts attribute in ctx

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41169 into pacific

* refs/pull/41169/head:
pacific: qa: drop the distro~HEAD directory from the fs suite.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #41629 from ideepika/wip-pr-41514

pacific: qa/upgrade: conditionally disable update_features tests

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: pass Grafana datasource in URL

PR https://github.com/ceph/ceph/pull/24314 added support for
specifying the Grafana datasource via $datasource template variable, but
this hadn't been used from the Dashboard side so far.

As per https://grafana.com/docs/grafana/latest/variables/#templates, by
adding `var-datasource=Dashboard1`, Dashboard can specify the
datasource.

Fixes: https://tracker.ceph.com/issues/51026
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 957c9c304bca8ef8416041f96b0e2d5bb0f0a0dc)

qa/upgrade: conditionally disable update_features tests

with the recent support for async rbd operations from pacific+ when an
    older client(non async support) goes on upgrade, and simultaneously
    interacts with a newer client which expects the requests to be async,
    experiences hang; considering the return code for request completion to
    be acknowledgement for async request, which then keeps waiting for
    another acknowledgement of request completion.

    this if happens should be a rare only when lockowner is an old client
    and should be deferred if compatibility issues arises.

see also: 541230475d3b25ab18c4eb9bc5011060462594a6(octopus)

Signed-off-by: Deepika <dupadhya@redhat.com>
(cherry picked from commit 9c0b239d70cdeb0c5705fa89fa5aac111152671b)

Merge pull request #41428 from rhcs-dashboard/wip-50774-pacific

pacific: mgr/dashboard: ingress service creation follow-up

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41427 from rhcs-dashboard/wip-50882-pacific

pacific: mgr/dashboard: fix ESOCKETTIMEDOUT E2E failure

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #41505 from rhcs-dashboard/wip-50959-pacific

pacific: mgr/dashboard: drop container image name and id from services list

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41507 from rhcs-dashboard/wip-50962-pacific

pacific: mgr/dashboard: fix API docs link

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41440 from rhcs-dashboard/wip-50864-pacific

pacific: mgr/dashboard: Fix for query params resetting on change-password

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

mgr/dashboard: show RGW tenant user id correctly in 'NFS create export' form.

Fixes: https://tracker.ceph.com/issues/50909
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 440d6d643c172252790791ec38ce27364ccc183e)

mgr/dashboard: fix API docs link

Fixes: https://tracker.ceph.com/issues/50890
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit e79c9d3f51970fbe275087e92e468205f70e6f12)

Merge pull request #41494 from aaSharma14/wip-50902-pacific

pacific: mgr/dashboard: provide the service events when showing a service in the UI

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: drop container image name and id from services list

Fixes: https://tracker.ceph.com/issues/50889
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Dropping container image name and id columns from services list.

(cherry picked from commit ee98d73a231ce99e9a5a74ed602724c2111ee93d)

mgr/dashboard: provide the service events when showing a service in the UI

When service deployment failures occur, events are generated and associated with the service. This PR intends to add these events as Daemon Logs and Service Logs to the UI for troubleshooting and doing diagnostics for the same.

Fixes: https://tracker.ceph.com/issues/49262
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 00ce7c90ef19f02beea9431ab360d23744efde2d)

Merge PR #41463 into pacific

* refs/pull/41463/head:
cephadm: write config files as utf-8
cephadm: reduce `noqa` usage
cephadm: apply autopep8 changes
cephadm: introduce autopep8
cephadm: remove `verify-prereqs` subcommand
cephadm: add HAProxy sysctl settings
cephadm: move keepalived sysctl settings
cephadm: apply osd sysctl settings
mgr/cephadm: warn users if they try to pass a daemon to 'orch rm'
qa/suites/orch/rook: disable centos for now
qa/suites/orch/rook/smoke: initial smoke suite
qa/tasks/rook: ROOK_HOSTPATH_REQUIRES_PRIVILEGED=true on centos
qa/tasks/rook: simplify shutdown
qa/tasks/rook: archive logs
qa/tasks/rook: more orderly cluster teardown
qa/tasks/rook: deploy ceph via rook on top of kubernetes
qa/tasks/kubeadm: install kubernetes with kubeadm
qa/suites: move rados/cephadm -> orch/cephadm; symlink
qa/tasks/cephadm: add whitespace between functions
qa/tasks/cephadm: clean up ctx.manager setup
mgr/cephadm: pin haproxy image to 2.3.z
mgr/cephadm: fix issue with missing prometheus alerts
mgr/cephadm: check version in upgrade check
mgr/cephadm: Fix OSD replacement in hosts with FQDN host name
mgr/cephadm: fix prometheus jinja template
doc/cephadm: rewrite "disabling monitoring"
mgr/cephadm: induce retune of osd memory on osd creation
qa/tasks/cephadm.conf: autotune osd memory by default
mgr/cephadm: do not autotune when _no_autotune_memory label is present
mgr/cephadm: autotune osd memory
common/options: add osd_memory_target_autotune
mgr/cephadm: report memory usage, request (limit) in 'orch ps'
doc/cephadm/host-management: document _admin group
mgr/orchestrator: fix help formatting
mgr/cephadm/schedule: fix filter message
mgr/cephadm: Use `cephfs-mirror` caps profile

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #41281 from idryomov/wip-rbd-pwl-status-pacific

pacific: rbd: don't attempt to interpret image cache state json

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

Merge pull request #41244 from idryomov/wip-rbd-pwl-reopen-pacific

pacific: librbd/cache/pwl: fix parsing of cache_type in create_image_cache_state()

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

Merge pull request #41195 from idryomov/wip-rbd-qemu-iotests-focal-pacific

pacific: qa/workunits/rbd: use bionic version of qemu-iotests for focal

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

mon,doc: deprecate CephFS min_compat_client

Instead admins should specify specific features to require.

Fixes: https://tracker.ceph.com/issues/50819
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d0e5a2bfac3a05bc60a34a815264b1f5baec944b)

doc: add alternate_name cephfs feature to table

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5180c39db5621224359e62cf812877c4b1f2b068)

cephadm: write config files as utf-8

If we don't specify the encoding, python may use something like 'latin-1',
which cannot handle unicode characters.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit fa2cf2987cccea9f152c04862fc684e146c4692e)

cephadm: reduce `noqa` usage

- E722 do not use bare 'except'
- W504 line break occurred after a binary operator

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fa75096e68110e8673d3c1cfede92c50af97a676)

cephadm: apply autopep8 changes

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6eaf23231d6048e6e583eff850395666b370c631)

cephadm: introduce autopep8

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 0efb4b546c6c24d8a5d109a31d859c6ada77556e)

cephadm: remove `verify-prereqs` subcommand

sysctl settings are now managed per-host, rather than per-daemon

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit bd308816746431331a047ab5f39d47272125685a)

cephadm: add HAProxy sysctl settings

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 465e7ef243f85db27642b2cd626841e9e4be848e)

cephadm: move keepalived sysctl settings

from container pre-start to sysctl file

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 04127a0d2226c0499af53503a936134aa744a2b3)

cephadm: apply osd sysctl settings

These were added to ceph-salt (ceph/ceph-salt@800dbb2)
.. but let's make them available to all consumers of cephadm!

Fixes: https://tracker.ceph.com/issues/47873
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 141d0cc5d3a0aedecd1f5755015581d8b63030ca)

mgr/cephadm: warn users if they try to pass a daemon to 'orch rm'

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit b07efcb4192ad5988baf3bb92df42cea877a8351)

qa/suites/orch/rook: disable centos for now

We are hitting an selinux denial:

type=AVC msg=audit(1621353663.595:6016): avc: denied { write } for pid=28812 comm="cephcsi" name="pids.max" dev="cgroup" ino=2933 scontext=system_u:system_r:container_t:s0:c43,c177 tcontext=system_u:object_r:cgroup_t:s0 tclass=file permissive=1

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ad8096e0a0493795a57c44493581feb98b014cdd)

qa/suites/orch/rook/smoke: initial smoke suite

- os: 18.04, 20.04, centos, centos+kubic
- nodes: 1 or 3
- rook: 1.6.2 or master
- k8s: 1.21 always (for now)
- net: calico always (flannel + centos is wonky)
- workload: none, rados bench

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ab33ca365567eeed7881e034bbc78e7cd66432d4)

qa/tasks/rook: ROOK_HOSTPATH_REQUIRES_PRIVILEGED=true on centos

This fixes OSD creation on centos.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f765677b8138ab5b57cd4d4113f402d89c3d9dfd)

qa/tasks/rook: simplify shutdown

For some reason deleting common.yaml sometimes fails.  Not really
sure why, but since we will tear down kubernetes anyway this
cleanup isn't really needed.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 929d6ad8ca7f319a4ebcefde0202ca5f993d626b)

qa/tasks/rook: archive logs

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e5148b3f59022068498334d056756de03c350fef)

qa/tasks/rook: more orderly cluster teardown

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 539cd362dde182bc78f750b62750d3fac82e8748)

qa/tasks/rook: deploy ceph via rook on top of kubernetes

This assumes that k8s is installed and kubectl works.

The ceph container to use is selected the same way the cephadm
task does it.

All scratch devices are consumed as OSDs.

A ceph.conf and client.admin keyring are deployed on all test
nodes, so normal tasks should work (if/when packages are installed).

Fixes: https://tracker.ceph.com/issues/47507
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4574ed70f42002028e2d30e8aa04fbff95daf728)

qa/tasks/kubeadm: install kubernetes with kubeadm

- install k8s with kubeadm
- initial support for flannel only
- remove taint from bootstrap/master node
- create PVs for all scratch_devs + a 'scratch' SC
- kubeadm.kubectl task

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ebf841a82f3939441fa118fa689c8e290ef31cfe)

qa/suites: move rados/cephadm -> orch/cephadm; symlink

Move cephadm under orch/ top-level suite.  Symlink so that we
still include it in a rados run.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5b25f8a2e51ccdea1f5409a4b8e55f8e625c8193)

# Conflicts:
# qa/suites/rados/cephadm
 - included thrash-old-clients, which is different in master

qa/tasks/cephadm: add whitespace between functions

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b6d1536a67428b4ffa0df3791e9eff667736c765)

qa/tasks/cephadm: clean up ctx.manager setup

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 267cefc0401d078e247d387d93a60840f6b53978)

mgr/cephadm: pin haproxy image to 2.3.z

The 2.4 version fails for reasons that aren't immediate clear:

May 17 15:28:42 dael conmon[3007013]: [NOTICE]   (8) : haproxy version is 2.4.0-6cbbecf
May 17 15:28:42 dael conmon[3007013]: [NOTICE]   (8) : path to executable is /usr/local/sbin/haproxy
May 17 15:28:42 dael conmon[3007013]: [ALERT]    (8) : Cannot open configuration file/directory /var/lib/haproxy/haproxy.cfg : Permission denied

Even with mode 777 on those files, there is an error about the chroot to
/var/lib/haproxy.

For now, just stick with a working version.

Fixes: https://tracker.ceph.com/issues/50830
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b94c8dedd8aa77aeb0b4957e88197ac89370ffb4)

mgr/cephadm: fix issue with missing prometheus alerts

Files passed as configuration to the cephadm binary had not been created
and mapped to the container, if those files weren't included in the
required files section inside cephadm. This prevented optional file
includes in the configuration.

The configuration file for the Prometheus default alerts is not
mandatory and hence wasn't included in the required files section, still it
needs to be added to the container by cephadm.

This change enables optional files to be included in the configuration
for monitoring components, so that those files are created and mapped
within the container.

Note that a `required_files` variable has been removed at one position
in these changes, though it wasn't used to ensure that required files
were included in the configuration at that point anyway. The test which
ensures that all required files are passed is somewhere else.

Fixes: https://tracker.ceph.com/issues/49856
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 38f9846d13ae2c5f12bde3397d2826b1cf43759b)

mgr/cephadm: check version in upgrade check

Fixes: https://tracker.ceph.com/issues/50784
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit e1979fb5bda09d7028d53a64b36cca5206e9e3a4)

mgr/cephadm: Fix OSD replacement in hosts with FQDN host name

Resolves: https://tracker.ceph.com/issues/50805

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit b1feb876d3bbe8798be062c61d523dde09d281d7)

mgr/cephadm: fix prometheus jinja template

The prometheus yaml file shouldn't use "tab" character but only spaces.

Fixes: https://tracker.ceph.com/issues/50717
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 2d8995e1ded22e680f778c4ef73ddd86592df21c)

doc/cephadm: rewrite "disabling monitoring"

* Simplified the text.
* Correctly formatted the prompts.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit b67b6112ac2b2b21360fca2ffd7ed0136a5d4929)

mgr/cephadm: induce retune of osd memory on osd creation

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 821a57ceabc70e8f4dce11fd04ec20ffead3789b)

qa/tasks/cephadm.conf: autotune osd memory by default

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e5c88e2c49266447334aa42b90a3f8004ea31eab)

mgr/cephadm: do not autotune when _no_autotune_memory label is present

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1e4d20a592aa6ca111201b0a3edc4dd619885f93)

mgr/cephadm: autotune osd memory

- set osd_memory_target_autotune=true to enable
- tuning is periodic (check every 10m by default)
- tuned values are reflected by osd_memory_target config options scoped
  to the host
- only make a change if it appears that we will affect at least 1 of the
  relevant OSDs
- attempt to clean out conflicting options.  (This is imperfect, since any
  manner of weirdly-scoped config options could be responsible; we only
  attempt to clean out one scoped directly to the osd name.)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 85ea078787d864bd8ae199f9517a9d2dbaf33c7c)

common/options: add osd_memory_target_autotune

This does not cherry-pick 6029a0e256cb10a8c35b3bbdc9a22ad679e6fe03
because options.cc is autogenerated in master branch.

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: report memory usage, request (limit) in 'orch ps'

Fill in from {osd,mon}_memory_target if no container limit is set.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4a8182a60658bfbd7034d8eb03e54dc1b154b165)

doc/cephadm/host-management: document _admin group

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit eb3c38577a4424080e4f297f79374d7be5aed9fa)

mgr/orchestrator: fix help formatting

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3ad3f06d2cdb5b700f8642814c32a3680bbd0dd5)

mgr/cephadm/schedule: fix filter message

This is now only used when scheduling mons.  (Units now enable the kernel
features needed instead of checking for them during placement.)  Move the
message to the filter itself.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d5aba1ec65a0ca49c6dbb1bb8975e5209cb020b3)

mgr/cephadm: Use `cephfs-mirror` caps profile

The profile is required to establish a peer
connection, but was added after this got merged

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit f95ded3f902d383c166dfde9bd178f5bc9563036)

Merge pull request #41122 from idryomov/wip-rbd-nuke-source-spec-file-pacific

pacific: rbd: --source-spec-file should be --source-spec-path

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

mgr/dashboard: Fix for query params resetting on change-password

Fixes: https://tracker.ceph.com/issues/50857
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 09307ab3bdd9ce27f746c6c1fa8018f807388182)

qa/tasks/nfs: add test to check if cmds fail on not passing required arguments

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 94644d1e5f68a0dc02686cbe826d34751b005f93)

mgr/nfs: fix flake8 missing whitespace around parameter equals error

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 57dba10325b23e52524b2a2c82381b1b5a020964)

mgr/nfs: annotate _cmd_nfs_* methods return value

Fixes: 7c57d1634
Fixes: https://tracker.ceph.com/issues/50783
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 1ec90c804319b32d86212f7402cac93fb14cb1d2)

doc/cephfs/nfs: add section about ganesha logs

Fixes: https://tracker.ceph.com/issues/50008
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 6ff3c1da87d8a4fe3750065a554e348e1882f240)

doc/cephfs/nfs: Replace volume/nfs with nfs

Recently, nfs related code was moved out of volumes plugin[1]. So using the
name volume/nfs for the interface is not appropriate.

[1] https://github.com/ceph/ceph/pull/40526

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit ff11cdb4bc7cfc0a51ca047e6b7c8c686075b2ec)

doc/cephfs/nfs: add note about export management with volume/nfs interface only

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 923e689e0828ac5186123a2ffc57db01465cbdd0)

spec: add nfs to spec file

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 8ab49501719f4d3d997a3b4e0f500ac985e9558f)