mgr/volumes: Remove dependency on dashboard ganesha module

Instead of importing ganesha module, necessary classes are directly used in
volumes nfs module.

Signed-off-by: Varsha Rao <varao@redhat.com>

vstart: Use random port instead of default Ganesha port

Signed-off-by: Varsha Rao <varao@redhat.com>

vstart: Use "NFS" instead "GANESHA" for starting nfs ganesha clusters

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes: Improve readability of ganesha common config

Signed-off-by: Varsha Rao <varao@redhat.com>

vstart: Add note about mounting cephfs exports

Signed-off-by: Varsha Rao <varao@redhat.com>

src/vstart: Set CEPH_CONF environment variables

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/fs: Update default ganesha conf options

Signed-off-by: Varsha Rao <varao@redhat.com>

doc: Add document about fs nfs interface to create CephFS exports

Signed-off-by: Varsha Rao <varao@redhat.com>

vstart: Update fs nfs export create command

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes: Update nfs export create and delete command

Remove fill_keys function and add key to nfsconfig object.
Export create and delete methods that no longer require ganeshaconf object to
be passed.

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes: Remove create_rados_pool method

Pool is created by cluster command.

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes: Changes to nfs export delete and create

Fixes: https://tracker.ceph.com/issues/44193
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes: While creating nfs-ganesha user update its cap

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes: Update caps for new user created and add it's key to keyring

Fixes: https://tracker.ceph.com/issues/44193
Signed-off-by: Varsha Rao <varao@redhat.com>

vstart: Update vstart according to cluster create command

Fixes: https://tracker.ceph.com/issues/44193
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes: Add Ganesha common config

Fixes: https://tracker.ceph.com/issues/44193
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes: Create NFSConfig class

Move the ganesha config functions into this class.

Fixes: https://tracker.ceph.com/issues/44193
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes: Add command to create nfs-ganesha clusters

ceph fs nfs cluster create <cluster_id> [--size=1]

cluster_id: Name of the nfs cluster

This command creates a common recovery pool for all Ganesha daemons, creates
new user and nfs cluster. Orchestrator needs to be enabled.

Fixes: https://tracker.ceph.com/issues/44193
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr: Create pool for nfs ganesha recovery

Fixes: https://tracker.ceph.com/issues/44193
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr: Add "ceph fs nfs delete <export_id>" to delete exports

Fixes: https://tracker.ceph.com/issues/44193
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr: Add command to create exports for nfs-ganesha

'ceph fs nfs create': This command creates export objects for nfs-ganesha.
vstart ganesha daemon fetches the export block from rados object.

Fixes: https://tracker.ceph.com/issues/44193
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #34229 from Yan-waller/wip-walle-fixsparsereadlength

osd/PrimaryLogPG: fix SPARSE_READ stat

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34342 from ideepika/fixes-44862

mon: calculate min_size on osd pool set size

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34219 from yanghonggang/bluefs-tool

os/bluestore: Don't pollute old journal when add new device

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #34143 from tchaikov/wip-mgr-disable-dne-module

mon/MgrMonitor: show different error message when disabling a dne module

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #34366 from SUSE/wip-mgr-fix-python-traceback

mgr/PyModule: fix missing tracebacks in handle_pyerror()

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34337 from majianpeng/throttle-remove-lock

common/Throttle: Don't lock for atomic type update.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34381 from rhcs-dashboard/fix-44721-master

rpm: add python3-saml as install dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34409 from adamemerson/wip-namespace-osd

osd: build without `using namespace` declarations in headers

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34460 from majianpeng/cmakefile-fix

cmake: remove duplicated code.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34451 from tchaikov/wip-standalone-pgid

qa/standalone/scrub: s/$(pgid)/${pgid}/

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34245 from rzarzynski/wip-bug-24995

mgr: synchronize ClusterState's health and mon_status.

Reviewed-by: Tim Serong <tserong@suse.com>

cmake: remove duplicated code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #34368 from majianpeng/msg-remove-unsued-code

msg, common/Throttle: remove unsued code.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34369 from smithfarm/wip-43895

cmake: really stop at top of source code tree

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34383 from matthewoliver/older-tox

tox: Fix the tox.ini's to support older versions of tox

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34427 from smithfarm/wip-44964

spec: address some warnings raised by RPM 4.15.1

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33945 from rishabh-d-dave/fs-qa-vstart_runner.LocalRemote.sh

qa/vstart_runner: update vstart_runner.LocalRemote.sh

Reviewed-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #34456 from ceph/wip-crontab-master

qa/tests: removed ceph-deploy from master and octopus (ceph-deploy is…

qa/tests: removed ceph-deploy from master and octopus (ceph-deploy isn't written to run on el8)

Signed-off-by: yuriw <yuri.weinstein@gmail.com>

Merge pull request #34439 from jschmid1/osdspec_encrypted_docs

docs/cephadm: <encrypted> is a global flag

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34053 from matthewoliver/iscsi-cephadm

cephadm: Add ceph-iscsi

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34357 from mgfritch/mgr-orch-test-warn

mgr/orch: fix python3 DeprecationWarning

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

qa/standalone/scrub: s/$(pgid)/${pgid}/

to address the test failures like
```
2020-04-07T15:44:58.693 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed:  ceph pg dump
pgs
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed:  pgid
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh: line 498: pgid: command not found
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

rpm: add python3-saml as install dependency

`python.*-saml` is required for Ceph-Dashboard SSO support (optional
feature).

Fixes: https://tracker.ceph.com/issues/44721
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #34396 from tspmelo/wip-npm-hacking

mgr/dashboard: Update HACKING.rst

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34412 from jan--f/c-v-fix-simple-scan-log-string

devices/simple/scan: Fix string in log statement

doc/cephadm: <encrypted> is a global flag

Signed-off-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #34406 from liewegas/fix-cephadm-upgrade-start

qa/suites/rados/cephadm/upgrade: start from v15.2.0

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34310 from badone/wip-restful-api-dont-pass-empty-dict-to-get

qa/suites/rados/rest: don't pass empty dict as data arg

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #34302 from badone/wip-specify-rule-for-pool-creation

Wip specify rule for pool creation

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

ceph.spec.in: remove extra tokens at end of %endif directive

RPM 4.15.1 warns: extra tokens at the end of %endif directive

Heed the warning.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: version an Obsoletes

RPM 4.15.1 warns: It's not recommended to have unversioned Obsoletes.

Heed the warning.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

osd: Build ceph-osd without using namespace declarations in headers

This is part of a series of commits to clean up using namespace at top
level in headers.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

{common, rgw}: Build ceph-osd without namespace pollution in headers

This is part of a series of commits to clean up using namespace at top
level in headers.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

include/ceph_assert: qualify __assert_warn call in assert_warn macro

A macro never knows where it will be expanded.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

perfglue: Build ceph-osd without using namespace declarations in headers

This is part of a series of commits to clean up using namespace at top
level in headers.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

cls: Build ceph-osd without using namespace declarations in headers

This is part of a series of commits to clean up using namespace at top
level in headers.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #34239 from p-se/wip-pse-fix-false-root-vol-full-alert

monitoring: root volume full alert fires false positives

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34240 from krig/grafana-dashboards-fixes

mgr/dashboard: Repair broken grafana panels

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

devices/simple/scan: Fix string in log statement

A multiline string enclosed by () gained a comma, making it into a
tuple, which has no format() method.

Fixes: https://tracker.ceph.com/issues/44949
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

mgr/dashboard: Update HACKING.rst

This will update information regarding npm.

Fixes: https://tracker.ceph.com/issues/44927
Signed-off-by: Tiago Melo <tmelo@suse.com>

common: move to_string and to_integer overloads into root namespace

Since that's where the type is defined, that's where overloads taking
it should be defined.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #34374 from sebastian-philipp/mgr-fix-tox-cephadm

pybind/mgr: Fix `run_tox.sh mgr -- cephadm`

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34269 from yuvalif/fix_cls_queue_logs

cls/queue: remove error message when queue is not initialized

Merge pull request #33985 from zhengchengyao/clone_metadata

librbd: children should inherit parent's stripe

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #34072 from shangdehao1/fix_nfs_bug

qa/tasks/qemu: fix nfs setup and teardown bug in qemu task

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34371 from smithfarm/wip-43896

doc/releases/nautilus: restart OSDs to make them bind to v2 addr

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #31480 from ukernel/wip-20

client: auto reconnect after blacklisted

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #33915 from lxbsz/client_assert_failedddd

Client: fix Finisher assert failure

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #34386 from gregsfortytwo/wip-33279-revert-sudo

Wip 33279 revert sudo

Reviewed-by: Douglas Fuller <dfuller@redhat.com>

Merge pull request #34110 from ukernel/wip-44680

mds: don't assert empty io context list when shutting down

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #34385 into master

* refs/pull/34385/head:
doc/dev/cephadm: a few notes on developing with cephadm

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #34281 from vshankar/wip-44677

mgr: force purge normal ceph entities from service map

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34306 from ukernel/wip-44771

ceph-fuse: don't get mount options from /etc/fstab when doing remount

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #34296 into master

* refs/pull/34296/head:
qa/suites/rados/cephadm/smoke-roleless: add smoke test
qa/tasks/cephadm: add 'roleless' mode

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #34384 into master

* refs/pull/34384/head:
cephadm: create /var/run/ceph dir via unit.run, not unit file

Reviewed-by: Michael Fritch <mfritch@suse.com>

qa/suites/rados/cephadm/upgrade: start from v15.2.0

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #34405 from ceph/wip-yuriw-cron-master2

qa/tests: fixed typo

qa/tests: fixed typo

Signed-off-by: yuriw <yuri.weinstein@gmail.com>

Merge pull request #34404 from ceph/wip-yuriw-cron-master

qa/tests: added client upgrades tests for octopus, removed mimic-x/ma…

qa/tests: added client upgrades tests for octopus, removed mimic-x/masted as not needed

Signed-off-by: Yuri Weinstein <yweinstein@redhat.com>

doc/dev/cephadm: a few notes on developing with cephadm

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33502 from yison/rwl-image-writeback-cache-seq3

rbd/cache: Replicated Write Log core codes part 3

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34362 from tchaikov/wip-rbd-with-pmem

cmake, librbd: fix build with pmem and cleanups

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34332 from cyx1231st/wip-seastar-messenger-lossy-peer

crimson/net: enable features for lossy peer connections of heartbeat

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34319 from clyso/patch-2

doc/mgr/orchestrator: update cephadm shell proposed alias entry

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34318 from clyso/patch-1

doc/mgr/orchestrator: add path to ssh-copy-id instructions

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34367 from sebastian-philipp/codeowners-cephadm-update

CODEOWNERS: update cephadm paths

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34390 from tchaikov/wip-github-codeowners

.github/CODEOWNERS: add ceph/crimson

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #34325 from tspmelo/wip-npm-update-20-03

mgr/dashboard: Update all npm packages

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34377 from votdev/issue_44914_rgw_debug_info

mgr/dashboard: Add more debug information to Dashboard RGW backend

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #34333 from changchengx/mail_org_update

mailmap: update mail org relationship

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/mgr/orchestrator: update cephadm shell proposed alias entry

prior change i get following error when using alias as "ceph -v":
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
/usr/bin/docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "/usr/bin/ceph -v": stat /usr/bin/ceph -v: no such file or directory": unknown.
after:
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
ceph version 15.2.0 (dc6a0b5) octopus (rc)

Signed-off-by: Tobias Fischer <tobias.fischer@clyso.com>

doc/mgr/orchestrator: add path to ssh-copy-id instructions

by default ssh key will be placed under /etc/ceph - so it should be included in examples

Signed-off-by: Tobias Fischer <tobias.fischer@clyso.com>

Merge pull request #34227 from p-se/wip-pse-fix-alert-space-prediction

monitoring: alert for prediction of disk and pool fill up broken

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

.github/CODEOWNERS: add ceph/crimson

so we can add reviewers in a more efficient way for crimson related
changes

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: specify CC when compiling pmem

make always assume that `cc` is available. but we cannot ensure this,
and furthermore, we need to use the compiler specified by user. so
specify `CC` variable when compiling pmem. and reindent the code to fix
the formatting.

Signed-off-by: Kefu Chai <kchai@redhat.com>

librbd/cache: undefine dout_subsys before defining it

otherwise we could have following warning from compiler
```
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:42:
../src/librbd/cache/ImageWriteback.cc:13: warning: "dout_subsys" redefined
   13 | #define dout_subsys ceph_subsys_rbd
      |
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:39:
../src/librbd/cache/ReplicatedWriteLog.cc:23: note: this is the location of the previous definition
   23 | #define dout_subsys ceph_subsys_rbd_rwl
      |
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:43:
../src/librbd/cache/rwl/ImageCacheState.cc:12: warning: "dout_subsys" redefined
   12 | #define dout_subsys ceph_subsys_rbd_rwl
      |
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:42:
../src/librbd/cache/ImageWriteback.cc:13: note: this is the location of the previous definition
   13 | #define dout_subsys ceph_subsys_rbd
      |
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: expose pmem::pmemobj from rbd_internal

because `librbd/cache/rwl/Types.h` includes `libpmemobj.h`, without this
change, `cache/test_mock_ReplicatedWriteLog.cc` will fail to compile as
`libpmemobj.h` might not exist in the default directories compile search
for header files. and hence we could have following failure
```
ceph/src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:11:
86 /root/luyuan/community/github/ceph/src/librbd/cache/rwl/Types.h:8:10: fatal error: libpmemobj.h: No such file or directory
87 8 | #include <libpmemobj.h>
88 | ^~~~~~~~~~~~~~
89 compilation terminated.
90 src/test/librbd/CMakeFiles/unittest_librbd.dir/build.make:1814: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/cache/test_mock_ReplicatedWriteLog.cc.o' failed
91 make[3]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/cache/test_mock_ReplicatedWriteLog.cc.o] Error 1
92 make[3]: *** Waiting for unfinished jobs....
93 CMakeFiles/Makefile2:20379: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/all' failed
94 make[2]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/all] Error 2
95 CMakeFiles/Makefile2:20391: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/rule' failed
96 make[1]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/rule] Error 2
97 Makefile:5375: recipe for target 'unittest_librbd' failed
```

Signed-off-by: Kefu Chai <kchai@redhat.com>