mount: fix the debug log when keyring getting secret failed

Getting one strang debug log:
Could not discover monitor addresseskeyring.get_secret failed

Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #31963 from yison/rwl-image-writeback-cache-seq2

rbd/cache: Replicated Write Log core codes part 2

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #33461 into master

* refs/pull/33461/head:
cephadm: Simplify is_available
cephadm: Improve type signature of is_available
cephadm: Fix type errors found by mypy
cephadm: Enable and start chrony using any packager
cephadm: Add Packager for Zypper (openSUSE / SLES)
cephadm: Normalize mypy annotations

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33486 into master

* refs/pull/33486/head:
qa/suites/rados/cephadm/smoke: run a few orch commands
mgr/orch: fix 'host ls'

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33475 into master

* refs/pull/33475/head:
qa/suites/rados/cephadm/upgrade: change start version

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33091 into master

* refs/pull/33091/head:
qa/suites/rados: disable device scraping
qa/standalone/ceph-helpers: disable device monitoring
qa/tasks/ceph.py: add pre-mgr-commands option for ceph task
mgr/devicehealth: set default monitoring to 'on'

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33235 into master

* refs/pull/33235/head:
os/bluestore/BlueFS: fixed printing stats

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33362 into master

* refs/pull/33362/head:
common/config :remove duplicaed code.
common/config: remove useless code.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33404 into master

* refs/pull/33404/head:
selinux: Allow ceph to setsched

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/cephadm/smoke: run a few orch commands

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: fix 'host ls'

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33472 into master

* refs/pull/33472/head:
cephadm: allow users to provide their dashboard cert during bootstrap

Reviewed-by: Sage Weil <sage@redhat.com>

qa/suites/rados/cephadm/upgrade: change start version

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33469 into master

* refs/pull/33469/head:
mgr/cephadm: fix upgrade ok-to-stop condition check

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #33473 from liewegas/remove-memstore-test

test/CMakeLists: disable memstore make check test

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33106 from Yan-waller/wip-walle-fixbackfillundersized

osd/PG: restart peering for undersized PG on any down stray peer coming back

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #33401 from liewegas/wip-octopus-notes

doc/release/octopus: note about upgrade times

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #33476 into master

* refs/pull/33476/head:
mgr/volumes: fix placement default value

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/volumes: fix placement default value

Fixes: https://tracker.ceph.com/issues/44244
Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: Simplify is_available

There is no need to return a closure to call
from is_available, it can just do the work
directly.

Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>

cephadm: Improve type signature of is_available

Specify the function signature expected by `is_available`
using the mypy Callable type.

Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>

cephadm: Fix type errors found by mypy

Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>

Merge pull request #33412 from idryomov/wip-krbd-fsx-build-deps

qa: rbd_workunit_suites_fsx: install build dependencies

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test/CMakeLists: disable memstore make check test

This is flaky, with frequent failures like

[ RUN      ] ObjectStore/StoreTest.CompressionTest/0
2020-02-21T19:06:13.940+0000 7f5cdErrors while running CTest
Build step 'Execute shell' marked build as failure

Tracked by https://tracker.ceph.com/issues/44243

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33468 into master

* refs/pull/33468/head:
debian: fix ceph-mgr-modules-core files

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #33457 into master

* refs/pull/33457/head:
qa/workunits/cephadm: separate out test_adoption.sh; fix

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33464 into master

* refs/pull/33464/head:
mgr/BaseMgrModule: drop GIL for is_authorized check
mgr/ActivePyModules: drop GIL to register/unregister clients

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cephadm: allow users to provide their dashboard cert during bootstrap

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>

Merge PR #33466 into master

* refs/pull/33466/head:
cmake: exclude unittest_alloc_aging from "all"

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/cephadm: fix upgrade ok-to-stop condition check

Broke this in 1072a7e3b5026c1e605dd45967fa555d2e6a0d14

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33415 into master

* refs/pull/33415/head:
mgr/cephadm: progress for upgrade

Reviewed-by: Michael Fritch <mfritch@suse.com>

debian: fix ceph-mgr-modules-core files

Remove the files from ceph-mgr that are added to ceph-mgr-modules-core.

2020-02-21T17:59:56.031 INFO:teuthology.orchestra.run.smithi060.stdout:dpkg: error processing archive /tmp/apt-dpkg-install-YpwLHV/044-ceph-mgr_15.1.0-1037-ga6b324b-1bionic_amd64.deb (--unpack):
2020-02-21T17:59:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: trying to overwrite '/usr/share/ceph/mgr/alerts/__init__.py', which is also in package ceph-mgr-modules-core 15.1.0-1037-ga6b324b-1bionic

Broken by 589626464d475b4ad96905fbddf11a5782a040f0

Signed-off-by: Sage Weil <sage@redhat.com>

cmake: exclude unittest_alloc_aging from "all"

link it against GTest::Main, which was remove by
96a0439faf551dfae45676072eddd5f37a157289.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cephadm: Enable and start chrony using any packager

Add method to enable services to Packager, and call it
from `command_prepare_host`.

Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>

Merge PR #33432 into master

* refs/pull/33432/head:
mgr/orchestrator: functools.partial doesn't work for methods

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33258 from sebastian-philipp/orchestrator-get_hosts-HostSpec

mgr/orchestrator: get_hosts return `HostSpec` instead of `InventoryDevice`

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #32552 from Devp00l/wip-42617-new-tree

mgr/dashboard: Reload all CephFS directories

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #32680 from votdev/wip-force-pwd

mgr/dashboard: Enforce password change upon first login

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #33193 from cbodley/wip-44068

rgw: data_sync_source_zones only contains 'exporting' zones

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

mgr/BaseMgrModule: drop GIL for is_authorized check

This is making a call all the way to the mon!  We need to drop the GIL.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ActivePyModules: drop GIL to register/unregister clients

We need to drop the GIL since we are taking ActivePyModules::lock.

Fixes: df507cde8d71063d5873a42f668156e4c32e86f9
Fixes: https://tracker.ceph.com/issues/44241
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33046 from ivancich/wip-listing-w-namespaces

rgw: fix bug with (un)ordered bucket listing and marker w/ namespace

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #33397 from aclamk/fix-disable-alloc-aging

cmake: Removed unittest_alloc_aging from make check

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33256 from tchaikov/wip-cmake-cares

cmake: should expose ${C-ARES_BINARY_DIR} from c-ares

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cephadm: Add Packager for Zypper (openSUSE / SLES)

Adds basic support for using zypper as the packager.

Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>

Merge pull request #31981 from petrutlucian94/windows

Windows support [part 1]

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

cephadm: Normalize mypy annotations

Try to use `# type: ...` consistently.

Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>

qa/workunits/cephadm: separate out test_adoption.sh; fix

- fix adoption test to handle different legacy fsids
- separate it out into a different script

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33430 from rosinL/wip-testosdscrub

test/TestOSDScrub: fix mktime() error

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33444 into master

* refs/pull/33444/head:
mgr/cephadm: no gibberish suffix for some services
mgr/cephadm: prometheus: scrape all mgrs
mgr/cephadm: make prometheus scrape all node-exporters
cephadm: reset-failed on reconfig
cephadm: fix reconfig for monitoring daemons

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>

Merge PR #33447 into master

* refs/pull/33447/head:
mgr/cephadm: fix upgrade wait loop

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #33435 from tchaikov/wip-viva-la-module

deb,rpm,doc: s/plugin/module/

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #33422 from tchaikov/wip-ceph-mgr-essentials

rpm,deb: package always-enabled plugins in a separated package

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #33088 from liewegas/wip-cephadm-doc

doc/bootstrap: add mds and rgw steps to bootstrap

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

osd/PeeringState: restart peering on any previous down acting member coming back

One of our customers wants to verify the data safety of Ceph during scaling
the cluster up, and the test case looks like:
- keep checking the status of a speficied pg, who's up is [1, 2, 3]
- add more osds: up [1, 2, 3] -> up [1, 4, 5], acting = [1, 2, 3], backfill_targets = [4, 5],
  pg is remapped
- stop osd.2: up [1, 4, 5], acting = [1, 3], backfill_targets = [4, 5], pg is undersized
- restart osd.2, acting will stay unchanged as 2 belongs to neither current up nor acting set,
  hence leaving the corresponding pg pinning undersized for a long time until all backfill
  targets completes

It does not pose any critical problem -- we'll end up getting that pg back into active + clean,
except that the long live DEGRADED warnings keep bothering our customer who cares about data
safety more than any thing else.

The right way to achieve the above goal is for:

boost::statechart::result PeeringState::Active::react(const MNotifyRec& notevt)

to check whether the newly booted node could be validly chosen for the acting set and
request a new temp mapping. The new temp mapping would then trigger a real interval change
that will get rid of the DEGRADED warning.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

qa/tasks/cephadm: `orch host ls` now returns hostnames

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: NodeAssignment.get_hosts_func now returns List[str]

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/orchestrator: get_hosts return `HostSpec` instead of `InventoryDevice`

This fixes a bad symmetry problem:

previsously:
```python
def add_host(host: HostSpec): ...
def get_hosts() -> List[InventoryDevice]: ...
```

now:
```python
def add_host(host: HostSpec): ...
def get_hosts() -> List[HostSpec]: ...
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/orchestrator: functools.partial doesn't work for methods

Use `partialmethod` instead:

```
def decorator_partial(f):
     return partial(f)

def decorator_lambda(f):
     return lambda *args, **kwargs: f(*args, **kwargs)

class C:
     @decorator_partial
     def f(self, arg): pass

     decorator_lambda
     def g(self, arg): pass

C().f(1)
TypeError: f() missing 1 required positional argument: 'arg'

C().g(1)
None
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #33453 from tchaikov/wip-seastar

seastar: pickup change to add pthread linkage

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #33452 from tchaikov/wip-docker-centos8

test: remove Dockerfile for centos7 and add Dockerfile for centos8

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

cmake: drop Threads::Threads linkage from libcrimson

since seastar is now linked against pthread by itself.

Signed-off-by: Kefu Chai <kchai@redhat.com>

seastar: pickup change to add pthread linkage

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/centos-8: change to adapt to centos8

for more context, see

https://github.com/CentOS/sig-cloud-instance-images/issues/22
https://wiki.centos.org/AdditionalResources/Repositories/CR
https://fedoraproject.org/wiki/EPEL

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/centos-8: add centos8 Dockerfile

and drop centos-7 Dockerfile, as we've moved over centos8 for better
python3 support

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #33441 into master

* refs/pull/33441/head:
mgr/orchestrator: add ability to parse placementspec from strings
mgr/volumes: add arg to fs volume create for mds daemons placement

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #33443 into master

* refs/pull/33443/head:
cephadm: prepare-host: do not create Packager unless we need it

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #33445 into master

* refs/pull/33445/head:
cephadm: fix adoption safety check

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33446 into master

* refs/pull/33446/head:
cephadm: pull: strip newline from version string

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #32882 from trociny/wip-41559-2

rbd: add 'mirror snapshot schedule' commands

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #33450 into master

* refs/pull/33450/head:
doc: add CephFS Octopus release notes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33368 from trociny/wip-44161

rbd-mirror: don't expect image map is always initialized

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #33411 from trociny/wip-44159

rbd-mirror: improve detection of blacklisted state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #33426 into master

* refs/pull/33426/head:
mgr/dashboard: fix tasks.mgr.dashboard.test_rgw suite

Reviewed-by: Sage Weil <sage@redhat.com>

doc: add CephFS Octopus release notes

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/cephadm: fix upgrade wait loop

This was succeeding after 4 tries even though it wasn't ok-to-stop.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: fix adoption safety check

The 'type' file is only required for *osds*.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: pull: strip newline from version string

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33437 into master

* refs/pull/33437/head:
cephadm: allow multiple get_parm() calls

Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/cephadm: no gibberish suffix for some services

Lots of services only need to appear once per node; no need to create a
unique name.  (In fact, most of these *won't* run more than once per
node because they bind to a standard port.)

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: prometheus: scrape all mgrs

Assume that all standby mgrs are running on the same port as the
active.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: make prometheus scrape all node-exporters

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: reset-failed on reconfig

Sometimes we had a bad config that reconfig will fix, so we need to
reset-failed before restart.  If there was not failure this is harmless.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: fix reconfig for monitoring daemons

We shouldn't check for port usage here.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orchestrator: add ability to parse placementspec from strings

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>

mgr/volumes: add arg to fs volume create for mds daemons placement

add placement arg for mds daemons created by fs volume create

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>

mgr/cephadm: progress for upgrade

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: prepare-host: do not create Packager unless we need it

Otherwise, bootstrap may fail on an unrecognized/unsupported distro even
though all the dependencies are present.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #31338 from ivancich/wip-clean-up-ordered-list

rgw: clean up ordered list

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33338 from smithfarm/wip-qa-cephadm-smoke-opensuse

qa/cephadm/smoke: run on opensuse_15.1

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

qa/cephadm: "ls -l" cephadm after downloading it

For better debugging when the download goes awry.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rbd: add 'mirror snapshot schedule' commands

Signed-off-by: Mykola Golub <mgolub@suse.com>

mgr/rbd_support: rename 'dump' command to 'list'

and remove the old 'list' command -- we can live with one command.

Signed-off-by: Mykola Golub <mgolub@suse.com>

qa/cephadm.py: better check for cephadm download success

When the URL passed to "curl --silent ..." does not exist, the
resulting file will be populated with the string:

    404: Not Found

If that (or something similar) happens, the file size will be
suspiciously low, like < 1000 bytes. Fail the test immediately in this
case.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/tasks/cephadm.py: lop off .git more robustly

The line

    rest.replace('.git/', '/')

was added to accommodate weird folks who run teuthology-suite
with an option like this:

    --suite-repo https://github.com/ceph/ceph.git/

but they might just as well give the option like this:

    --suite-repo https://github.com/ceph/ceph.git

Signed-off-by: Nathan Cutler <ncutler@suse.com>

cephadm: allow multiple get_parm() calls

This fixes deployment of monitoring components when the config-json is
passed via stdin.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #31994 from pritha-srivastava/wip-rgw-assume-role-web-id-doc

rgw: adding documentation for AssumeRoleWithWebIdentity.

mgr/rbd_support: return 'rbd mirror snapshot schedule status' as json

Signed-off-by: Mykola Golub <mgolub@suse.com>

doc/mgr/dashboard.rst: s/plugin/module/

Signed-off-by: Kefu Chai <kchai@redhat.com>

rpm,deb: s/plugin/module/

Signed-off-by: Kefu Chai <kchai@redhat.com>