Revert "exporter: set stats_period to 0 on first response"

Merge pull request #46758 from rhcs-dashboard/fix-stats-period-on-first-response

exporter: set stats_period to 0 on first response
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

exporter: set stats_period to 0 on first response

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: minor fixes and cleanups

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: improving logic for stats_period + cleanups

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

expoter: add configs

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: argument parsing

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

exporter: guard metrics string

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

exporter: move ceph_exporter.cc to src/exporter

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

expoter: addressing http server fixes

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: fix run-tox-cephadm/mgr

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: addressing cephadm cleanups/suggesstions

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: cleanups

- DaemonMetricCollector singleton with static variable
- some iterator references
- DaemonMetricCollector chagne timer to steady_timer with chrono
- AdminClientSocket handle error when send_request.
- update_sockests now clears the list of sockets before adding new ones.
- use string_body instead of dynamic_body in http::response

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

exporter-cephadm: increase number of docker.io occurences

This is just a temporary change, as in future once the feature branch is merged in master the exporter
would be included in default master ceph image
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: cephadm tox fixes

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: update package description

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: fix lint errors + remove exporter entity

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: Cleanups on DaemonMetricCollector and google styling

1. Formatted all exporter files to google c++ styling.
2. Refactored send_requests a bit to increase readabilty.
3. Removed unused functions + variables.
4. Moved fs::path socketdir to a static const char* SOCKETDIR.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

exporter: fix metrics names + scrape all exporter endpoints in cephadm

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: make keep-alive connection based on client

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: ignore mgr perf counters + change port to 9926 + cleanups

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: quote label values

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

mgr/cephadm: adding ceph exporter container

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: add ceph-exporter package

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: exporter dir and http_server.h

* move all related files to the exporter to src/exporter
* move http_server logic to a new file under src/exporter/http_server.*

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

exporter: httpserver and collector threads

* Start http server on another thread.
* Expose metrics plain text (string) so that the /perf_counters endpoint
  can retrieve it.
* Introduce stats_period so that the collector updates the metrics every
  'stats_period' seconds.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

exporter: expose perf_counters at http endpoint

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: add http server

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

exporter: parse perfcounters

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

exporter: socket magic

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

exporter daemon

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #45012 from nSedrickm/grafana-iframe-ux-update

mgr/dashboard: display helpfull message when the iframe-embedded Grafana dashboard failed to load

Reviewed-by: Sarthak Gupta <sarthak.dev.0702@gmail.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

Merge pull request #46512 from rhcs-dashboard/rbd-mirroring-force-resync

mgr/dashboard: rbd force resync from front-end

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #46639 from rhcs-dashboard/mirroring-page-fixes

mgr/dashboard: improve edit site name action in rbd-mirroring

Reviewed-by: Sarthak Gupta <sarthak.dev.0702@gmail.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #46533 from rhcs-dashboard/dictionaryIscsi

mgr/dashboard: iterate through copy of items

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #44900 from nmshelke/fix-54111

mon: verify data pool is already not in use by any file system

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #46295 from dparmar18/tracker3998

mds: split up mdstypes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #46363 from dparmar18/tracker_55548

qa/cephfs: fallback to older way of get_op_read_count

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #46412 from lxbsz/wip-55778

client: choose auth MDS for getxattr with the Xs caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #46560 from lxbsz/wip-55824

qa: wait rank 0 to become up:active state before mounting fuse client

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>

Merge pull request #46635 from nmshelke/fix-55822

mgr/volumes: remove incorrect 'size' from output of 'snapshot info'

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #46636 from nmshelke/fix-55759

mgr/volumes: subvolume ls command crashes if groupname as '_nogroup'

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #46699 from neesingh-rh/wip-56065

qa: TestMDSMetrics.test_delayed_metrics failure

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>

Merge pull request #45144 from dvanders/doc_overrides

common/options/osd: clarify the non-zero overrides

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #46630 from rzarzynski/wip-pglog-trim-dups

tools: ceph-objectstore-tool is able to trim solely pg log dups' entries

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #46433 from rhcs-dashboard/rbd-mirroring-replay

mgr/dashboard: move replaying images to Syncing tab

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: nSedrickm <NOT@FOUND>

Merge pull request #39980 from badone/wip-ceph_test_lazy_omap_stats-improve-scrubbing-calls-2

test/lazy-omap-stats: Various enhancements

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mgr/dashboard: rbd force resync from fornt-end

Signed-off-by: Sarthak0702 <sarthak.dev.0702@gmail.com>

mgr/dashboard: rbd force resync from fornt-end

Signed-off-by: Sarthak0702 <sarthak.dev.0702@gmail.com>

Merge pull request #46698 from guits/cv-hide-luks-key-in-log

ceph-volume: do not log sensitive details

Merge pull request #45284 from tobias-urdin/doc-memory-profiling-valgrind-massif

doc: Add alternative memory profiling to doc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #46712 from zdover23/wip-doc-2022-06-15--master-to-main-dev-guide-basic-workflow-title

doc/dev: s/master/main/ in title

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #45690 from sleepinging/fix-fio-windows-crash

test/fio/fio_ceph_messenger: fix str_to_ptr() crash at windows

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

ceph-volume: do not print the secret of osd keyring

during osd preparation, ceph-volume logs the secret of the osd keyring to file
```
[2022-06-15 12:31:17,466][ceph_volume.process][INFO  ] Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-0/keyring --create-keyring --name osd.0 --add-key AQAM0aliR5JvDRAAQBu0stWl9ZhZrcjijg2BIQ==
[2022-06-15 12:31:17,481][ceph_volume.process][INFO  ] stdout creating /var/lib/ceph/osd/ceph-0/keyring
added entity osd.0 auth(key=AQAM0aliR5JvDRAAQBu0stWl9ZhZrcjijg2BIQ==)
```

This shouldn't be logged nor printed on terminal.

Fixes: https://tracker.ceph.com/issues/56071
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-volume: do not print luks key encryption

During osd activation, ceph-volume logs the luks key to its log file.

```
[2022-06-15 12:50:35,180][ceph_volume.process][INFO  ] Running command: /usr/bin/ceph --cluster ceph --name client.osd-lockbox.51d0770d-403d-4f81-93e6-e99f627f246c --keyring /var/lib/ceph/osd/ceph-0/lockbox.keyring config-key get dm-crypt/osd/51d0770d-403d-4f81-93e6-e99f627f246c/luks
[2022-06-15 12:50:35,522][ceph_volume.process][INFO  ] stdout ut9NjMK6YtMh1BLMJZ/mE2A7zTNyrp9pW1kHV8F2ipfz1BIX9MkEWhdYB2Azm1JPZ1d7ahIjBMUbrC/Iqqr2jQhP3MIsDzUYj1enw+sw7LeVvGPf0qNUdKmEGu5tUmvtQ+5pbk4T/9PF36kT6vCHKfNML/3fL6nnY8FDySrI4LY=
[2022-06-15 12:50:35,522][ceph_volume.process][INFO  ] Running command: /usr/sbin/cryptsetup --key-size 512 --key-file - --allow-discards luksOpen /dev/ceph-83c307d3-710b-4197-8ecd-0484e17395e3/osd-block-51d0770d-403d-4f81-93e6-e99f627f246c a9HhDO-MiYD-DtYm-SKJf-nO1d-5O3u-FmcCrd
```

Fixes: https://tracker.ceph.com/issues/56066
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #45926 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-moment-2.29.3

mgr/dashboard: bump moment from 2.29.1 to 2.29.3 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: improve edit site name action in rbd-mirroring

Fixes: https://tracker.ceph.com/issues/55896
Signed-off-by: Nizamudeen A <nia@redhat.com>

doc/dev: s/master/main/ in title

This changes "master" to "main" in a title. If we lived in an
ideal world, this would have been a part of PR#46678.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #46705 from zdover23/wip-doc-2022-06-15--master-to-main-dev-guide-merging

doc/dev_guide: s/master/main in merging.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #46678 from zdover23/wip-doc-2022-06-14-dev-guide-basic-workflow-master-to-main

doc/dev: s/master/main/ in basic workflow

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #46632 from anthonyeleven/anthonyeleven/osd-activate-typos

src/ceph-volume/ceph_volume/activate: Improve usage message text

doc/dev_guide: s/master/main in merging.rst

This changes the branch name "master" to the branch name
"main" in merging.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #44861 from Matan-B/wip-matanb-doc-gdb

doc/dev: Debuggging with gdb

Merge pull request #46582 from ceph/rhel86

qa: Default to RHEL8.6 instead of 8.5

Merge pull request #46029 from kamoltat/wip-ksirivad-fix-notify-rank-removed

mon/Elector: notify_rank_removed erase rank from both live_pinging and dead_pinging sets for highest ranked MON

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #45858 from ganeshmaharaj/ganeshma/gcc-12-libcephsqlite

libcephsqlite: ceph-mgr crashes when compiled with gcc12

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #45750 from m-ildefons/boost-b2-jobs

cmake: configure boost build with concurrent jobs

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa: Default to RHEL8.6 instead of 8.5

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #46364 from rkachach/fix_issue_54581

mgr/nfs: validate virtual_ip parameter

Reviewed-by: John Mulligan <jmulligan@redhat.com>

doc/dev: Debuggging with gdb

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #46648 from zmc/cephadm-sysctl-noop

Merge pull request #46664 from tchaikov/wip-doc-gantt

doc: render release with mermaid gantt

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #46669 from tchaikov/wip-crimson-deferred

crimson/osd: use seastar::deferred_stop()

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

qa: TestMDSMetrics.test_delayed_metrics failure

TestMDSMetrics.test_delayed_metrics is failing due to
the absence of omit_sudo parameter in the remote.run()
of set_inter_mds_block() in qa/tasks/cephfs/filesystem.py

Fixes: https://tracker.ceph.com/issues/56065
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>

Merge pull request #45614 from lxbsz/wip-54653

ceph-fuse: add dedicated snap stag map for each directory

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>

doc: render release with mermaid gantt

for better readability

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

admin/doc-requirements: bump sphinx to 4.5.0

also pin sphinx-autodoc-typehints to 1.18.3

to address following error:

ERROR: sphinx-autodoc-typehints 1.18.3 has requirement Sphinx>=4.5, but you'll have sphinx 4.4.0 which is incompatible.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

ceph-fuse: add dedicated snap stag map for each directory

This will fix the fino colliding bug, which is caused when the
snapid is later than 0xffff.

From mds 'mds_max_snaps_per_dir' option, we can see that the max
snapshots for each directory is 4_K, and in ceph-fuse we have
around 64_K, which is from 0xffff - 2, stags could be used to make
the fake fuse inode numbers for each directory.

Fixes: https://tracker.ceph.com/issues/54653
Signed-off-by: Xiubo Li <xiubli@redhat.com>

ceph-fuse: return EINVAL if get invalid fino instead of assert

All the snap ids of the finos returned to libfuse from libcephfs
will be recorded in the map of 'stag_snap_map', and will never be
erased before unmounting. So if libfuse passes invalid fino the
ceph-fuse should return EINVAL errno instead of crash itself.

Fixes: https://tracker.ceph.com/issues/54653
Signed-off-by: Xiubo Li <xiubli@redhat.com>

ceph-fuse: reserve stag number 1 for snapdirs

There have two stags will be reserved, 0 for CEPH_NOSNAP and 1 for
CPEH_SNAPDIR.

This will always make sure that for the nonsnap and snapdir inode
numbers to be consistent for all the ceph-fuse mounts.

Signed-off-by: Xiubo Li <xiubli@redhat.com>

mds-client: make the fake inos option unchangeable in runtime

If the flags is empty then in option.h in can_update_at_runtime()
it will return true. That means this opetion could be changed in
runtime, which is buggy. Because if this is false, ceph-fuse will
use its own fake inos instead of libcephfs'. If this is changed
during runtime, we will hit inos dosn't exist assert bugs.

Fixes: https://tracker.ceph.com/issues/54653
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #46629 from anthonyeleven/anthonyeleven/46195_formatting

doc/man/8: Tweak formatting and wording in ceph.rst

doc/dev: s/master/main/ in basic workflow

This PR changes "master" to "main" in the
basic_workflow.rst file. I have even changed
"master" to "main" in some terminal output from
several years ago. This isn't historically ac-
curate, of course, but my hope is that this change
will prevent someone in the future from being con-
fused about why an antiquated branch name is ref-
erred to.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #46362 from pdvian/wip-logm-mgrstat

mon/LogMonitor, MgrStat: Report logm and mgrstat committed version

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #46279 from jianwei1216/osd-peering-complete-log-patch

osd: add log for pg peering and activiting complete

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/osd: use seastar::deferred_stop()

trade `seastar::defer()` for `seastar::deferred_stop()` to
simplify the code a little bit.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

cephadm: Make sysctl conf writing idempotent

By checking the system's actual settings against our desired settings
before writing out the config file.

Fixes: https://tracker.ceph.com/issues/56026
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #46509 from aclamk/wip-pglog-test-leak

test/pglog: Fix memory leak

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #46438 from pdvian/wip-scrub-starts

osd/scrub: Reintroduce scrub starts message

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #46428 from sseshasa/wip-fix-55773-have-pending

mon/OSDMonitor: Ensure kvmon() is writeable before handling "osd new" cmd

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #45984 from trociny/wip-51835

mgr: relax "pending_service_map.epoch > service_map.epoch" assert

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #46285 from s0nea/config-fqdn-active

mgr/cephadm: try to get FQDN for active instance

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>

Merge pull request #46506 from asm0deuz/mod_iscsi_systemd

Modify how Iscsi tcmu-runner container is started within systemd

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #44894 from guits/add_dockerfile_keepalived

cephadm: change default keepalived/haproxy container images

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #46588 from guits/cluster-fsid-common

ceph-volume: make is_valid() optional

Reviewed-by: Teoman ONAY <tonay@redhat.com>

Merge pull request #46586 from SUSE/wip-suse-default-jaeger-off

ceph.spec.in: turn jaeger off by default for SUSE distros

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #46370 from rhcs-dashboard/stop-poll-page-inactive

mgr/dashboard: stop polling when page is not visible

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #46475 from rhcs-dashboard/01-hosts-failure

mgr/dashboard: fix drain e2e failure

Reviewed-by: Sarthak Gupta <sarthak.dev.0702@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #44567 from lxbsz/client_reply

client: do nothing when get a stale reply

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #46313 from zenomri/wip-omri-tracing-lua

rgw: add SetAttribute and AddEvent functions for TraceMetaTable in Lua

Reviewed-by: yuvalif