mgr/ssh: run c-v with podman when getting inventory

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: simplify ssh connection management

- Use a single instance of the config and identity files for the whole
module.  There's no need to create these for *every* connection--it just
pollutes /tmp.
- Drop the SSHConnection wrapper, since the temp files are tied to the
daemon lifecycle now.
- Prefix the tmp files so I can tell wtf is going on.
- Always connect to root@host, to avoid remoto's localhost detection
feature.  This ensures we have a consistent connection model and user.
(The daemon might be running as user ceph and try to connect to localhost,
but end up running the command as the wrong user and/or inside the
ceph-mgr container.)

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: use ceph-daemon for deploying mon

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: allow --mon-network for deploying new mon (vs specifying IP)

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: --config-and-keyring (not key)

Signed-off-by: Sage Weil <sage@redhat.com>

common/options: add 'image' config option

Defaults to 'ceph/daemon-base'.

Signed-off-by: Sage Weil <sage@redhat.com>

test_ceph_daemon: specify image name

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: add --ssh to enable+configure ssh orchestrator

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: use ssh identity from config-key, if present

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: hardcode default ssh_config

Use this unless either ssh_config or ssh_config_file are present.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: store ssh identity in mon config-key store

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: --privileged arg for 'exec'

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: make deploy work for osd (do a c-v prepare)

This is sufficient to deploy an OSD that is based on ceph-volume lvm.
YMMV if it's not an lvm-based OSD.

Run the OSD container privileged so we can open the raw block device.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: make shell privileged

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: move get_container_mounts to a helper

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: pass full path for entrypoint

Sometimes we run things in /usr/sbin...

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: make id portion of 'shell' optional

This lets you start up a 'generic' container of a particular class,
without a data mount, but with the appropriate other mounts and privilege
levels.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-volume: accept --no-tmpfs argument for bluestore

When activating a bluestore inside a container, we want to (be able to)
make the osd dir metadata persistent inside the container.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: 'unit' command

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: fix run command to use call(), not check_output()

Signed-off-by: Sage Weil <sage@redhat.com>

src/ceph-daemon: whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: add 'enter', 'exec' commands

'enter' is really just 'exec' + bash

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: bind config to default location

This is one less thing on the command line, and an easier time when inside
the container.

Signed-off-by: Sage Weil <sage@redhat.com>

test_ceph_daemon.sh: test deploy mds too

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: generate ssh keys

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: --config, not --conf

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: long lines

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: add --config to bootstrap

Settle on --config|-c too, not --conf.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: add 'shell' command

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: do not import subprocess symbols directly

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: add mons with 'deploy mon.x ...'

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: add 'ls'

Eventually we should teach this to understand rook mons+osds and
ceph-ansible containerized daemons.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: simplify uid/gid a bit

Assume user ceph and group ceph exist inside the container.  We still need
the uid/gid for creating daemon directories, though, and for the bootstrap,
where we are creating files on the host and passing them through.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: fix libudev

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: autodetect uid/gid from container image

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: default to empty log files, log to stderr (systemd journal)

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: rm-{daemon,cluster}

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: fix bootstrap config

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: fix args.fsid usage

Pass it explicitly, since it only sometimes comes from args.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: be careful overwriting live files

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: slurp some options over from the standard systemd unit

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: add ceph.target and ceph-$fsid.target units

Signed-off-by: Sage Weil <sage@redhat.com>

test_ceph_daemon.sh: stupid test script

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: bootstrap and deploy (mgr) work

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: initial checkin

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-mon: fix debug print of public_addr

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #30542 into master

* refs/pull/30542/head:
doc: cleanup CephFS Landing Page

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Show smartctl summary in OSD tab (#30378)

mgr/dashboard: Show smartctl summary in OSD tab

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29922 from smithfarm/wip-41524

cmake: set empty RPATH for some test executables

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge PR #30416 into master

* refs/pull/30416/head:
client: add procession of SEEK_HOLE and SEEK_DATA in lseek.

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30653 from tchaikov/wip-os/drop-kinetic

os: remove KineticStore

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28569 from alimaredia/wip-rgw-ioctx-operate-replace

rgw: replace direct calls to ioctx.operate()

Merge pull request #30655 from tchaikov/wip-cmake/crimson/pass-args

cmake: pass arguments to crimson tests

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

client: add procession of SEEK_HOLE and SEEK_DATA in lseek.

Fixes: https://tracker.ceph.com/issues/42107
Signed-off-by: Shen Hang <harryshen18@gmail.com>

Merge pull request #30613 from joke-lee/rgw-ceph-dencoder-add-RGWPeriodLatestEpochInfo

ceph-dencoder: add RGWPeriodLatestEpochInfo support.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

cmake: pass arguments to crimson tests

otherwise they will take all CPU cores and all memory

Signed-off-by: Kefu Chai <kchai@redhat.com>

build/ops: set empty RPATH for some test executables

Fixes: https://tracker.ceph.com/issues/41524
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30587 from sebastian-philipp/python-common-readme

python-common: Add small Readme

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

Merge PR #30341 into master

* refs/pull/30341/head:
cephfs-shell: better complain info, when deleting non-empty directory

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge PR #30559 into master

* refs/pull/30559/head:
mds: Reorganize class members in MDBalancer header

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #30580 into master

* refs/pull/30580/head:
client: don't ceph_abort on bad llseek whence value
client: remove Inode dir_contacts field

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30407 from joscollin/wip-B41841-yes-really-mean-it

mgr/volumes: protection for `fs volume rm` command

doc: cleanup CephFS Landing Page

Fixes: https://tracker.ceph.com/issues/41952
Signed-off-by: Milind Changire <mchangir@redhat.com>

os: remove KineticStore

kinetic-cpp-client and kinetic project as a whole are not acitvely
supported or maintained anymore. see

- https://github.com/Kinetic/kinetic-cpp-client
- http://www.openkinetic.org

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #30552 into master

* refs/pull/30552/head:
cmake: fix libtsan detection
test: librados startup/shutdown racer test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #30630 from tchaikov/wip-build-on-centos8

install-deps,rpm,do_cmake: build on RHEL/CentOS 8

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #30531 from alimaredia/wip-rgw-maven-version-update

qa: bump maven repo version in s3a_hadoop.py

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30615 from xiexingguo/wip-progress-time-remaining

mgr/progress: estimated remaining time for events

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30591 from xiexingguo/wip-balancer-throttler

mgr/balancer: upmap_max_iterations -> upmap_max_optimizations; behave as it is per pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29674 from athanatos/sjust/wip-fio-trace

os/bluestore: expand lttng tracepoints, improve fio_ceph_objectstore backend

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

cls: CLS_CLIET_HIDE_IOCTX changes in cls headers

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: various ioctx.operate refactoring changes

- remove unused #include for cls_client headers
- refactor cls_timeindex_trim call in
rgw_object_expirer_core.cc
- add more CLS_CLIENT_HIDE_IOCTX in header files
- move target_compile_definitions to rgw_a target
- make repeat cls helper functions static

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: overload cls_rgw functions that do io_ctx.operate()

Call the new overloaded functions and call
rgw_rados_operate after them.

Signed-off-by: Ali Maredia <amaredia@redhat.com>

cls: remove unused cls_rgw_reshard_get_head()

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: replace ioctx.operate() with rgw_rados_operate()

For all operate calls in:
rgw_op.cc
rgw_orphan.cc
rgw_reshard.cc
rgw_rados.cc

Signed-off-by: Ali Maredia <amaredia@redhat.com>

cmake: fix libtsan detection

This variable is case-sensitive.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #30620 from votdev/issue_42069_validate_bucket_name

mgr/dashboard: Bucket names cannot be formatted as IP address

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kai Wagner <kwagner@suse.com>

Merge pull request #30647 from s0nea/wip-dashboard-fix-typo

mgr/dashboard: fix small typos in description message

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29925 from sebastian-philipp/run_tox-overwrite-build-dir

script/run_tox.sh: Don't overwrite the build dir

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: fix two small typos

Remove unnecessary blanks.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

script/run_tox.sh: Don't overwrite the Ceph's bulid dir

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #30639 from tchaikov/wip-crimson-readable

crimson/osd: implement readable/lease related methods

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30644 from majianpeng/osd-remove-unused-func

osd: remove unused function

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30632 from xiexingguo/wip-kick-recovery-pri

osd/PrimaryLogPG: always use strict priority ordering for kicked recovery ops

Reviewed-by: Yan Jun <yan.jun8@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30642 from Rjerk/wip-fix-doc

doc/radosgw: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PrimaryLogPG: add closing ")" in log

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd/pg: implement schedule_renew_lease()

see also d883db70282c406edaceb72a90ecddccd97a0161

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: implement ShardServices::get_hb_stamps()

PeeringState::proc_lease() requires a valid hb_stamps[0], which is in
turned maintained by `pl`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd/pg: implement queue_check_readable() and recheck_readable()

see also 7aec060e0aff4588ff51f744dd194d1c0f7793c2

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/ceph_time: do not handle negative case

this change partially reverts 353a0e5f, unlike `signedspan`,
`timespan::rep` is unsigned. so no need to handle negative case.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Bucket names cannot be formatted as IP address

In general, bucket names should follow domain name constraints:
- Bucket names must be unique.
- Bucket names cannot be formatted as IP address.
- Bucket names can be between 3 and 63 characters long.
- Bucket names must not contain uppercase characters or underscores.
- Bucket names must start with a lowercase letter or number.
- Bucket names must be a series of one or more labels. Adjacent labels are separated by a single period (.). Bucket names can contain lowercase letters, numbers, and hyphens. Each label must start and end with a lowercase letter or a number.

Fixes: https://tracker.ceph.com/issues/42069
Signed-off-by: Volker Theile <vtheile@suse.com>

osd: remove unused function

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge PR #30396 into master

* refs/pull/30396/head:
doc: document mds journaling

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30463 into master

* refs/pull/30463/head:
vstart_runner: allow the use of it with kernel mounts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/radosgw: fix typo

Signed-off-by: Liu Lan <liulan@umcloud.com>

mgr/progress: estimated remaining time for events

Fixes: https://tracker.ceph.com/issues/40419
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30638 from uweigand/cmake-atomic-16byte

cmake: Test for 16-byte atomic support on IBM Z

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: Test for 16-byte atomic support on IBM Z

On IBM Z the Boost tagged pointer implementation cannot use
"pointer compression" as there are no unused bits in an address;
the whole 64-bit address space is available to user space code.

Instead, Boost uses 16-byte atomics.  This is always supported
on IBM Z, but depending on the particular compiler (version)
it may require linking against libatomic.  The existing checks
in CheckCxxAtomic.cmake do not catch this, however, as they only
test for (up to) 8-byte atomic support.

Fixed by adding a test for 16-byte atomic support on IBM Z.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>

Merge pull request #30617 from xiexingguo/wip-weird-daemon-key

mgr: fix weird health-alert daemon key

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr: fix weird health-alert daemon key

Was:
  19 slow ops, oldest one blocked for 34 sec, daemons [osd,2,osd,4] have slow ops.

Now:
  153 slow ops, oldest one blocked for 38 sec, daemons [osd.3,osd.4,osd.5] have slow ops.

Fixes: https://tracker.ceph.com/issues/42079
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>