cephadm: Remove assignment to member variable in ServiceSpecs

Remove unnecessary assignment to member variable 'preview_only', this is done in the constructor of the derived ServiceSpec class.

Fixes: https://tracker.ceph.com/issues/47384
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #34777 from nSedrickm/grafana-iframe

mgr/dashboard: increase Grafana iframe height to avoid scroll bar

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #37057 from tchaikov/wip-bump-seastar

seastar,crimson: bump seastar to use upstream master

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #37042 from xxhdx1985126/wip-47357

crimson/osd: avoid accessing WaitForObjectRecovery::pi when no pulling occurred

Reviewed-by: Kefu Chai <kchai@redhat.com>

seastar: bump seastar to use upstream master

before this change we are using a fork which includes a fix to address
the FTBFS with GnuTLS < 3.5.6

after this change, we are using the lastest upstream which already
contains this fix.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: print notify_id instead of its address

this addresses the FTBFS like

    /usr/include/fmt/core.h:902:19: error: static assertion failed: formatting of non-void pointers is disallowed
      902 |     static_assert(!sizeof(T), "formatting of non-void pointers is disallowed");
          |                   ^~~~~~~~~~

also, i think it is more helpful in this context to have the
notify id instead of the address, as we can use the id to reference
the notification printed by other logging messages.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: cast pointers to void* before printing them

to address the FTBFS like:

../src/crimson/os/seastore/lba_manager/btree/btree_range_pin.cc:110:14:   required from here
/usr/include/fmt/core.h:902:19: error: static assertion failed: formatting of non-void pointers is disallowed
  902 |     static_assert(!sizeof(T), "formatting of non-void pointers is disallowed");
      |                   ^~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: instantiate future_type<> with future_stored_type

to accommodate the change on seastar's side, which started to use
monostate for representing `void` return type, and hide this convention
using future_stored_type_t.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: move continuation_base_from_future out of internal namespace

otherwise we have

../src/crimson/common/errorator.h:1090:8: error: specialization of ‘template<class Future> struct seastar::continuation_base_from_future’ in different namespace [-fpermissive]
 1090 | struct continuation_base_from_future<Container<::crimson::errorated_future_marker<Values...>>> {
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

since seastar has moved its continuation_base_from_future() out
of seastar::internal into seastar namespace

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: print entity_addr_t not the in4_addr in it

for two reasons:

* we don't have operator<<(ostream&, in4_addr&) defined.
* we don't have any fmt::format() facility specialized for in4_addr
* to print entity_addr_t instead of in4_addr is more correct in
  the sense that we should support IPv6

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/crimson: include seastar/util/later.hh for seastar::now()

seastar::now() is extracted into seastar/util/later.hh in a recent
change in seastar, so update accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: include seastar/core/when_all.hh for when_all()

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: use temporary RecoveryInfo for recovering object delete ops

Right now, recover_delete use WaitForObjectRecovery::pi, which may not be available
if there were no pulling, to pass recovery_info, so use temporary recovery_info instead

Fixes: https://tracker.ceph.com/issues/47357
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/osd: set push future an exception when there are errors during pushing

Fixes: https://tracker.ceph.com/issues/47357
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/net: do not print non-void* pointer

to address the failure like

fmt/core.h:902:19: error: static assertion failed: formatting of non-void pointers is disallowed
  902 |     static_assert(!sizeof(T), "formatting of non-void pointers is disallowed");
      |                   ^~~~~~~~~~

and i searched around, couldn't find any other
statement printing the address of message, so
i think it's not of much use to print their
addresses in a single place, unless we can cross
check them in different places.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: use seastar::make_ready_future()

instead of seastar::now(). as seastar::now() was moved into
seastar/util/laster.hh.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37049 from yuriw/wip-yuriw-crontab-master

qa/tests: removed extra line

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tests: removed extra line

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #36230 from trociny/wip-43174-2

os/bluestore: fix collection_list ordering

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #36955 from agayev/zbc-to-zbd

os/bluestore: Switch from libzbc library to libzbd library

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36950 from ceph/wip-bz1872983

ceph-volume: simple scan should ignore tmpfs

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #36976 from rhcs-dashboard/fix-47285-master

mgr/dashboard: fix duplicate logs

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #37003 from xxhdx1985126/wip-47310

crimson/osd: drop repop if osdmap has increased

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36974 from jan--f/snap-schedule-keep-with-empty-retention

snap-schedule: don't prune all on empty retention

crimson/osd: add can_discard_op method for detecting "discardable" client requests

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/osd: drop repop if it can be discarded

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/osd: rename is_valid_rep_op_reply to can_discard_replica_op

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #36903 from sebastian-philipp/python-common-simplify-hsp-to-json

python-common: Make HostPlacementSpec.to_json() more friendly

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #36994 from ktdreyer/boost-context-s390x

rpm: enable WITH_BOOST_CONTEXT for s390x

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #36432 from sebastian-philipp/cephadm-repo_digest

mgr/cephadm: convert tags to repo_digest

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #37015 from tchaikov/wip-seastar-packet

common/buffer_seastar: reserve the packet slots in ctor

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #36946 from zdover23/wip-docs-36796-fix

doc/cephadm: Document mgr/cephadm/manage_etc_ceph_ceph_conf

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #36995 from smithfarm/wip-47305

rpm: on SUSE, podman is required for cephadm to work

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #36992 from uweigand/fix-encode-float

include/encoding: Fix encode/decode of float types on big-endian systems

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36829 from changchengx/osdmap

crush,osd: refine crushmap comment and implementation

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36947 from pdvian/wip-mgr-get-rule-root

mgr/mgr_module : Fix keyerror for op if rule steps first element is opcode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36972 from smithfarm/wip-47112

rpm,deb: drop /etc/sudoers.d/cephadm

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36937 from tchaikov/wip-mgr-cleanup

mgr: use range-based loop and cleanups

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #36999 from neha-ojha/wip-47309

qa/*/mon/mon-last-epoch-clean.sh: mark osd out instead of down

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37008 from tchaikov/wip-crimson-pg-tell

crimson/osd: add "ceph tell <pgid> <command>" support

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

common/buffer_seastar: reserve the packet slots in ctor

for better performance, and simpler this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37010 from tchaikov/wip-crimson-osd-net

crimson/net: move implementation into .cc file

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #37001 from tchaikov/wip-crimson-basic

qa/suites: add crimson-rados/basic tests

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: add "WITH_ZBD" option

for enabling Zoned Block Device support using libzbd

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: Switch from libzbc library to libzbd library

Libzbc maintainers recommend switching to libzbd, which is lighter and supports
both ZNS SSDs and HM-SMR HDDs.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>

Merge PR #36600 into master

* refs/pull/36600/head:
mds: make caps dump more readable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

crimson/osd: add "ceph tell <pgid> <command>" support

* add an abstract class of `PGCommand` for `ceph tell <pgid> <command>`
* add two sample implementations for the pg tell commands.
  - "query"
  - "mark_unfound_lost"

Signed-off-by: Kefu Chai <kchai@redhat.com>

crush/CrushWrapper: refine type_map/name_map comment

type_map is the item(bucket & device) type id and its type name.
name_map is the item(bucket & device) id and its name.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

crush/CrushWrapper: use "new_bucket_id" as var name of the added bucket

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

crush/CrushWrapper: align variable name with comment

In the prolog of "for loop" statement, it writes "add child in the
location with 0 weight"

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

osd/OSDMap: no need to give same locations per time to create bucket

Suggested-by: Kefu Chai <kefu@redhat.com>
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

crush/CrushWrapper: refine comment to improve readability

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

osd/OSDMap: assert osd_weight default initialized value is CEPH_OSD_OUT

refer: http://www.cplusplus.com/reference/vector/vector/resize/
Explicitly specify initialization value to vector::resize operation.

Suggested-by: Kefu Chai <kefu@redhat.com>
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

osd/OSDMap: use class member instead of parameter to reset container size

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

osd/OSDMap: avoid re-initialize osd state and weight done in set_max_osd

In set_max_osd function, container's osd_weight and osd_vector have been
assigned with the right state/weight value.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

crush: correct typo

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

crush: no need to double check root bucket id

root bucket id should be definitely less than 0. assert(r < 0) here.
There's no possible to find device items in find_nonshadow_roots.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

include/encoding: Fix encode/decode of float types on big-endian systems

Currently, floating-point types use "raw" encoding, which means they're
simply copied as byte stream.

This means that if the decoding happens on a machine that differs in
byte order from the source machine, the returned value will be
incorrect. As one effect of this problem, a big-endian OSD node cannot
join a cluster where the MON node is little-endian (or vice versa),
because the OSDMap (incremental) structure contains floating-point
values, and as a result of this conversion problem, the OSD node will
crash with an assertion failure as soon as it receives any OSDMap update
from the MON.

This should be fixed by always encoding floating-point values in
little-endian byte order just as is done for integers. (Note that this
still assumes source and target machines used the same floating-point
format except for byte order. But given that nearly all platforms these
days use IEEE binary32/binary64 for float/double, that seems a
reasonable assumption.)

Fixes: https://tracker.ceph.com/issues/47302
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>

Merge pull request #36924 from ddiss/sigabbrev_np

cmake: detect and use sigdescr_np() if available

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #36929 from tchaikov/wip-python-common-no-unicode

python-common: s/unicode/str/

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

crimson/net: move implementation into .cc file

for faster compilation.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37007 from xxhdx1985126/wip-47313-47311

crimson/osd: accurately judge whether to do drop_recovery_read on recovering objs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37004 from tchaikov/wip-discard-repop

crimson/osd: discard op if pg changes or sender is down

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/osd: add PG::mark_unfound_lost()

it is a stub implementation for supporing asock command

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add PG::dump_primary()

for implementing asock command "ceph tell 1.0 query"

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: drop unused header

osd_admin.h is referenced in the .cc file, so drop it in the header

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add OSD::get_pg()

for implementing asock commands.

and add `PGMap::get_pg()` as a helper

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: refactor PrimaryLogPG::do_command("query")

we should let recovery_state.dump_peering_state() close the formatter
section, because it's recovery_state.dump_peering_state(), who opens it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/PGStateUtils: do not forward declare unused class

PG is not referenced in this header

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: print pgid in OSD::load_pg()

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: fix indent

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: don't do push related operations if there's no shard to push

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/osd: accurately judge whether to do drop_recovery_read on recovering objs

Fixes: https://tracker.ceph.com/issues/47313
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/osd: discard op if pg changes or sender is down

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites: add crimson-rados/basic tests

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37000 from xxhdx1985126/wip-47226

crimson/osd: construct log_keys_debug when reading pglog entries

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: construct log_keys_debug when reading pglog entries

Fixes: https://tracker.ceph.com/issues/47226
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #36990 from tchaikov/wip-doc-prompt

doc/dev: use prompt directive when appropriate

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/dev: drop unused label

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev/developer_guide/tests-unit-tests: use literal "make check"

the ``make check`` is better than a hyper link here.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev: fix broken "make check" link

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev: link to label not file

it's less error prone, and can help us avoid broken links if we move a
file around

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/*/mon/mon-last-epoch-clean.sh: mark osd out instead of down

The test should mark the OSD out to check if only "in" OSDs are considered by
the osdmap trimming logic.

Fixes: https://tracker.ceph.com/issues/47309
Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #36983 into master

* refs/pull/36983/head:
client: make the osdmap wait under the mount_state reader check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36580 into master

* refs/pull/36580/head:
Client: defer puting the inode
client: defer cancelling the timer event
client: remove _sync_write_commit which makes no sense any more

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36850 into master

* refs/pull/36850/head:
mds: add performance counter for cap messages

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #36464 into master

* refs/pull/36464/head:
client: get rstat from auth mds to reduce forwarding requests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/dev: use prompt directive when appropriate

for 2 reasons:

* sphinx renders codeblock using python syntax highlighting by default,
  so it's not surprising that it highlight keywords like "export" in
  command line samples. so to render command line code blocks, we'd
  better specify the syntax explicitly for better rendering result.
* with the help of "prompt" directive, user is able to copy and paste
  the command without the prompt. for instance, with the default
  "::" directive, user will copy "$ ceph df", which is not very
  convenient, but with "prompt" directive, user only copies
  "ceph df".

Signed-off-by: Kefu Chai <kchai@redhat.com>

rpm: enable WITH_BOOST_CONTEXT for s390x

We've upgraded to Boost version 1.73
(1169dee2592ec41777207fcf5742b5e936a100fc), and this version provides
working support for boost::context for s390x. Enable this in the RPM
packaging.

Fixes: https://tracker.ceph.com/issues/47304
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

rpm: on SUSE, podman is required for cephadm to work

On SUSE, cephadm is tested with podman only.

Fixes: https://tracker.ceph.com/issues/47305
Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph-volume: simple scan should ignore tmpfs

When simple scan is ran against a ceph-volume
OSD, util.encryption.legacy_encrypted returns
tmpfs. We want to avoid creating a Device
object with tmpfs and ignore the OSD as it's
not a ceph-disk created OSD.

Resolves: rhbz#1872983

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #36824 from SchoolGuy/feature/replace-png-with-svg

mgr/dashboard: Exchange PNGs for SVGs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34831 from rhcs-dashboard/41060-API-wrong-error-code

mgr/dashboard: REST API returns 500 when no Content-Type is specified

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/mgr_module: fix keyerror for op

If rule steps first element is opcode.

Fixes: https://tracker.ceph.com/issues/47272
Signed-off-by: Prashant Dhange <pdhange@redhat.com>

Merge pull request #36059 from liu-chunmei/seastore-extentmap-tree

crimson/seastore: add extent map tree

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #36865 from ishanrai05/global-styles-fix

mgr/dashboard: fix style being applied globally

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mds: make caps dump more readable

Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>

client: make the osdmap wait under the mount_state reader check

It will make no sense to check this if the unmouting is already
going on.

https://tracker.ceph.com/issues/47293
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #36489 from zdover23/wip-doc-dev-merge-request-2020-aug

doc/dev: add proper merge commit format

Merge pull request #36977 from tchaikov/wip-47290

mon/OSDMonitor: only take in osd into consideration when trimming osd…

Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>