doc: Updated dashboard iSCSI configuration, added labels

Added note about the requirement for the latest ceph-iscsi version
3 to the dashboard documentation. Added some doc references
and replaced some URLs in the iSCSI docs with reST labels instead.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #27050 from theanalyst/build-doc-warns

admin/build-doc: keep-going when finding warnings

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26534 from majianpeng/skip-calc-crc-header-for-async-msg

msg/async: skip repeat calc crc header in Message::encode

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #27009 into master

* refs/pull/27009/head:
ceph_release: the next release will be octopus

Reviewed-by: Kai Wagner <kwagner@suse.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #26514 from dillaman/wip-38381

pybind/rados: fixed Python3 string conversion issue on get_fsid

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26985 from rhcs-dashboard/38768-fix-sparkline-component

mgr/dashboard: fix sparkline component

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #27034 from jan--f/mgr-prometheus-remove-scrape-duration

pybind/mgr/prometheus: remove scrape_duration metric

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #26701 from adamemerson/wip-trip-the-light-clangtastic

fix clang build failures, and clean up warnings

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26836 from p-na/wip-pna-fix-osd

 mgr/dashboard: "1 osds exist in the crush map but not in the osdmap" breaks OSD page

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #26717 from ifed01/wip-ifed-objectstore-tool-fix

tools: no-mon-config switch for ceph-objectstore-tool.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26926 from rouming/epoll-lost-epollet

msg/async/EventEpoll: set EPOLLET flag on del_event()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26934 from sebastian-philipp/doc-rados-mon_command

doc/rados/api/python: Add documentation for mon_command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26562 from familyuu/log

mgr/dashboard: Add date range and log search functionality

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #27065 from tchaikov/wip-crimson-mgr-client

crimson/osd: report pg_stats to mgr

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: connect OSD to mgr

so OSD is able to report to mgr periodically using mgr::Client

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: include MonClient.h in .cc

mon::Client's definition is only necessary in .cc

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add facilities for reporting pg stats

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mgr: add mgr client

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd/pg: accessors for pg stats/state

so OSD can collect pg stats and report them to mgr

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: create msgrs in main.cc

update PG to hold references of messengers instead of pointers to them,
as their lifecycle will be managed in main.cc

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27045 from smithfarm/wip-38810

rpm: fix "rhel <= 7" conditional

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #27066 from tchaikov/wip-crimson-sharded-stop

crimson/osd: create msgrs in main.cc

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os: use transparent comparator in xattr

so we don't need to allocate a temporary string for looking up OI or SS
attribute

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: cancel timers before restart

otherwise we will rearm already-armed timer when booting up OSD again.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: call do_shutdown() in SocketMessenger::stop()

so sharded<SocketMessenger> can stop the messenger service with stop()

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: create msgrs in main.cc

messengers are sharded<Service>. we should not create them in another
sharded service's start() method. to ensure the ordering of stop of
sharded services, we should create the sharded services in main().
and register their stop() method in the proper order.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27064 from tchaikov/wip-crimson-pg-init

crimson/osd: init PG with more info

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: init primary state in PG::read_state()

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add PG::read_state()

to load pg_info and past_intervals

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: skip fast_info if it is not around

fast_info is optional. for instance, there is chance that it
is the first time the info is written to store, in that case,
there is no delta, i.e. fast info yet.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: pass pgid, osdmap and msgr to PG's ctor

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27062 from tchaikov/wip-crimson-ceph-opts-in-argv

crimson/osd: pass unknown args to ConfigProxy::parse_args()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #27003 from tchaikov/wip-crimson-for-classic

osd,messages: changes for preparing for crimson-osd

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #27026 from tchaikov/wip-remove-MPGStats/had_map_for

messages,osd: remove MPGStats::had_map_for

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27059 from tchaikov/wip-crimson-heartbeat

crimson/{osd,heartbeat}: allow heartbeat to have access to authorizer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: pass unknown args to ConfigProxy::parse_args()

these args are not recognized by seastar's app_template's option parser,
so they are likely to be the ceph options and should be fed to
ConfigProxy::parse_args()

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: do not pass ceph options to seastar

the program_option parser used by seastar::app_template does not allow
unrecognized options. but ceph options can be specfied by being passed
to ceph applications as command line options, for instance, we can
specify the "key" or "keyfile" when creating an objectstore using
"--mkfs", like:

  ceph-osd --mkfs --key <key>

in this change, all options known by seastar's app_template are
enumerated and stored into a separated vector. so it can be passed to
"app". and the unknown ones are passed to ceph functions.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: add ConfigProxy::parse_argv()

it's only used by crimson-osd for parsing ceph options.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27061 from tchaikov/wip-crimson-mkfs

crimson/osd: call engine().exit(0) after mkfs

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

ceph_release: the next release will be octopus

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #26987 into master

* refs/pull/26987/head:
msg/msg_types: use inet_ntop(3) to render IP addresses

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #26672 into master

* refs/pull/26672/head:
osd: correct a local variable type

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

crimson/osd: init OSD::store in its ctor

less repeating this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: unmount store in OSD::stop()

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: print message after done with mkfs

so user is aware of what's accomplished.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: call engine().exit(0) after mkfs

* crimson/osd/main.cc:
  if crimson-osd is launched as a daemon, we can always stop it using
  SIGINT, and seastar::reactor does stop itself when handling this signal.

  but if we start crimson-osd as an ordinary command line tool, we should
  stop it explicitly after it is done with its job. so call
  seastar::engine().exit().
* crimson/osd/osd.cc:
  do not stop unitialized services in OSD::stop(). OSD is initialized as
  a sharded service in main.cc, so we have to stop it before stopping
  engine. if OSD is used for mkfs, the internal services like heartbeat,
  monc, mgrc are not initialized at all, so we should not stop them in
  this case. and in theory, user could stop crimson-osd when it boots,
  so we need to check the pointers for null before dereferencing them.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #26652 into master

* refs/pull/26652/head:
osd: clear osd op reply output only when writes success

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26604 into master

* refs/pull/26604/head:
qa/workunits/rest: Better detection of rest url

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27010 into master

* refs/pull/27010/head:
mon/OSDMonitor: allow 'osd pool set pgp_num_actual'

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #26812 into master

* refs/pull/26812/head:
msg/async V2: optimize check state by replace.

Reviewed-by: Ricardo Dias <rdias@suse.com>

crimson/osd: correct the osd_ping's message type

s/CEPH_MSG_PING/MSG_OSD_PING/

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: implement Heartbeat::ms_handle_reset()

reconnect to peer if the connection is reset

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: add AuthService

AuthService is introduced to allow Dispatchers to access to
authorizers, even if it is not chained with a Dispatcher implementing
Dispatcher::ms_get_authorizer(). in this case, we need to grant access to
Heartbeat class. it has its own messengers dedicated for heartbeat
traffic. it's mon::Client which provides the facilities of authorization
via Dispatcher interface.

we could just cast mon::Client to ceph::common::Dispatch for accessing
Dispatcher::ms_get_authorizer(), but i want to make this explicit using
AuthService. as the consumers of Dispatch inteface is messenger and
ChainedDispatcher not the domain specific classes.

in future, we need to either implement Auth{Client,Server} or adapt to
this machinery for msgr V2.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: implement Dispatcher::ms_get_authorizer()

we need this method for supporting cephx authentication

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/{net,osd}: make ms_get_authorizer() sync

the authorizer manager does not perform (significant) i/o for building
an authorizer. see CephXTicketHandler::build_authorizer(). what it does
is but read random bytes using getentropy(3) which uses getrandom(2).
getrandom(2) could potentially block if the system just boots and does
not have enough randomness. but i think it's safe to assume that we have
enough entrophy when crimson-osd starts.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27006 from tchaikov/wip-crimson-os

crimson/os: do not fail if fsid file exists when mkfs

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #26996 from neha-ojha/wip-2-36739

osd/PG: skip rollforward when !transaction_applied during append_log()

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26056 into master

* refs/pull/26056/head:
mds: check earlier if directories are already exported
mds: dont print auth trees if they are too many
mds: dont print subtrees if they are too many/big

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #26662 into master

* refs/pull/26662/head:
qa: update kclient testing to RHEL 7.6

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #26817 into master

* refs/pull/26817/head:
mds: avoid revoking Fsx from loner during directory fragmentation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #26895 into master

* refs/pull/26895/head:
mds: convert unnecessary list usage to vector
mds: convert get_*dirfrags to use vector
mds: convert iterator loop to generic loop
mds: list to std::list
mds: constantize dump_loads

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #26931 from trociny/wip-38706

librbd: make flush be queued by QOS throttler

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27049 from trociny/wip-rbd-deep-copy-cleanup

librbd: fix typo in deep_copy::ObjectCopyRequest::compute_read_ops

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

crimson/os: do not fail if fsid file exists when mkfs

instead of passing a random uuid down to CyanStore::mkfs(), let it
create one if necessary.

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: implement CyanStore::get_attr()

it is used for retrieving object_info and snapset attributes.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #27042 into master

* refs/pull/27042/head:
qa: add krbd_discard_granularity.t test
qa: update and rename krbd_discard_1b.t

Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tools: admin/build-doc: keep-going when finding warnings

This allows the build to fail, but also report a list of warnings so that
multiple runs are not required to find all the errors in doc changes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #27004 from tchaikov/wip-crimson-osdmap

crimson/osd: crimson/osd: do not load fullmap.0

Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

librbd: fix typo in deep_copy::ObjectCopyRequest::compute_read_ops

The second arg for interval_set insert method is the inserting
interval lenth, while the end position was provided. It still
worked correctly, because the end position value is always larger
than the truncated length.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #26973 from smithfarm/wip-nautilus-backports

script: enable nautilus in backport scripts

Reviewed-by: Sebastian Wagner <swagner@suse.com>

rpm: fix "rhel <= 7" conditional

We do not build for RHEL 6 and below, so "rhel == 7" is just as good, while
also avoiding the bug.

In addition, "rhel == 7" is the conditional we are using everywhere else in the
spec file for this purpose.

Fixes: https://tracker.ceph.com/issues/38810
Signed-off-by: Nathan Cutler <ncutler@suse.com>

EventEpoll: refactor del_event() a bit

The main purpose of the patch is to set 'ee' members only on EPOLL_CTL_MOD
path, deletion path does not requie any of the members set.

This will make code a bit better.

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>

EventEpoll: set EPOLLET flag on del_event()

When new fd is being added to epoll fd EPOLLET (edge triggered) flag
is set, i.e. we assume that in and out works in edge triggered mode.

But on del_event(), when eventually epoll_ctl(EPOLL_CTL_MOD) is
invoked, EPOLLET flag is cleared, i.e. level triggered mode is
enabled.

Do not lose EPOLLET.

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>

qa: add krbd_discard_granularity.t test

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge PR #27019 into master

* refs/pull/27019/head:
doc/releases/nautilus: add reference to msgr2 config update section
doc/cephfs/nfs: update ceph.conf example
doc/releases/nautilus: final upgrade note updates

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Add date range and log search functionality

Fixes: http://tracker.ceph.com/issues/37387t st
Signed-off-by: guodan1 <guodan1@lenovo.com>

Merge pull request #26772 from sebastian-philipp/orchestrator-rm-stateful-rule

mgr/orchestrator: Remove `(add|test|remove)_stateful_service_rule`

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge tag 'v14.2.0'

v14.2.0

messages,osd: remove MPGStats::had_map_for

MPGStats::had_map_for was added back in 7844d0e5, the last release that
still checks this field was mimic -- monitor sends OSD incremental
osdmaps if the monitor finds that the pg_stats' had_map_for is greater
than 30 and the epoch is less than that of latest osdmap.

but DaemonServer as the new consumer of MPGStats does not check
had_map_for anymore -- it simply updates the cluster state with the
pg_stats reported by OSD. and we directs OSD to mgr for sending pg_stats
since mimic. so, we can safely drop the support of had_map_for in
octopus, as it has been 2 releases.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26977 from p-na/wip-pna-fix-tox-env-vars

mgr/dashboard: Fix env vars of `run-tox.sh`

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26095 from bbc/s3secretcache

rgw: Added caching for S3 credentials retrieved from keystone

Merge pull request #26564 from theanalyst/wip-reshard-lc-fixes

rgw: LC: handle resharded buckets

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27001 from linuxbox2/wip-rgw-delimchar

rgw: fix signed char truncation in delimiter check

Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa: update and rename krbd_discard_1b.t

Passing 1 for alloc_size is no longer allowed.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge remote-tracking branch 'gh/nautilus'

Merge pull request #26457 from mogeb/osd-stop-steady-clock

osd: use steady clock in prepare_to_stop()

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix env vars of `run-tox.sh`

Fixes: http://tracker.ceph.com/issues/38798
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: "1 osds exist in the crush map but not in the osdmap" breaks OSD page

Fixes: http://tracker.ceph.com/issues/36086
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

pybind/mgr/prometheus: remove scrape_duration metric

Prometheus itself adds this metric to every scrape target.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

mgr/dashboard: Clean up of Osd::list()

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

doc/rados/api/python: Add documentation for mon_command

Co-authored-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

14.2.0

doc/releases/nautilus: add reference to msgr2 config update section

Signed-off-by: Sage Weil <sage@redhat.com>

doc/cephfs/nfs: update ceph.conf example

and reference 'ceph config generate-minimal-conf'

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #26956 into nautilus

* refs/pull/26956/head:
doc/releases/nautilus: more detail around ceph-volume step

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #27013 from badone/wip-nautilus-CA-change

qa/ceph-ansible: Move to Nautilus

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #27011 from liewegas/wip-redirect-mimic

qa/suites/upgrade/mimic-x/parallel: enable all classes

Reviewed-by: Myoungwon Oh <omwmw@sk.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27018 from liewegas/wip-38294

qa/suites/upgrade/luminous-x: disable c-o-t import/export tests between versions

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #26960 into nautilus

* refs/pull/26960/head:
ceph_release: nautilus is will be 'stable' for v14.2.0

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge PR #27017 into master

* refs/pull/27017/head:
osd: transpose two wait lists in comment

Reviewed-by: Sage Weil <sage@redhat.com>