mgr/prometheus: Fix metric types from gauge to counter

Affected metrics:
- ceph_pool_rd
- ceph_pool_rd_bytes
- ceph_pool_rw
- ceph_pool_rw_bytes

Fixes: https://tracker.ceph.com/issues/51868
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit ede63b2eff23d824425da69ed304049c1c232ffd)

Merge pull request #43168 from linuxbox2/pacific-perl-semicolon

workunits/rgw: semicolon terminates perl statements

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

workunits/rgw: semicolon terminates perl statements

Fixes a lexical error in one line of code added in
90e9307ab0a52da260bc1ebb50329cd8ff942eb9, removing the dependency
on lsb_release, on 8/16/2021.

Fixes: https://tracker.ceph.com/issues/52613
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 221fdb858be083a981697cdb59c19f0659be3f1d)

Merge pull request #43151 from ceph/pacific-systemd

pacific: debian/control: dh-systemd is part of debhelper now

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #43131 from ceph/remove-cython

pacific: debian/control: remove cython from Build-Depends

Reviewed-by: Kefu Chai <kchai@redhat.com>

pacific: debian/control: dh-systemd is part of debhelper now

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=822670

Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit 4f21d41b73e30b71c9222841751dcebf898613cc)

debian/control: remove cython from Build-Depends

as cython3 is enough. and we've dropped the python2 support.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit bcd7f3ac0f299855516df871a56ef26c1ee21bc9)

Merge pull request #43100 from ifed01/wip-ifed-fix-migrate-pac

pacific: os/bluestore: fix bluefs migrate command

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42976 from idryomov/wip-51419-pacific

pacific: common/buffer: fix SIGABRT in  rebuild_aligned_size_and_memory

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #43116 from guits/wip-52560-pacific

pacific: ceph-volume: fix lvm activate arguments

Merge pull request #43110 from guits/wip-52554-pacific

pacific: ceph-volume: fix lvm migrate without args

ceph-volume: fix lvm activate arguments

When using the `lvm activate` subcommand without any osd ID or osd FSID
then ceph-volume fails.

Currently we can either activate with:
 - both osd ID and osd FSID
 - only the osd FSID (because it's unique)

The remaining scenarios aren't covered and generate a stack trace:
 - only the osd ID
 - no osd ID nor osd FSID

This ends up with an error like:

-->  UnboundLocalError: local variable 'tags' referenced before assignment

Fixes: https://tracker.ceph.com/issues/50665
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit b77ed5f99d3cb14a46bb36bb74e4136c22edf48a)

ceph-volume: fix lvm migrate without args

When running the `lvm migrate` subcommand without any args then the
ceph-volume command fails with a stack trace.

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 59, in newfunc
    return f(*a, **kw)
  File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 151, in main
    terminal.dispatch(self.mapper, subcommand_args)
  File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
    instance.main()
  File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/main.py", line 46, in main
    terminal.dispatch(self.mapper, self.argv)
  File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
    instance.main()
  File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/migrate.py", line 520, in main
    self.migrate_osd()
  File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 16, in is_root
    return func(*a, **kw)
  File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/migrate.py", line 403, in migrate_osd
    if self.args.osd_id:
AttributeError: 'Migrate' object has no attribute 'args'

That's because we're exiting the parse_argv function but we continue to
execute the migrate_osd function. We should instead exit from the main function.

This update the parsing argument to have the same code than new-db and
new-wal classes.
Now the parsing is done in the make_parser function but the argv testing is
done in the main function allowing to exit the program and displaying the
help message when no arguments are provided.

Fixes: https://tracker.ceph.com/issues/51811
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 98de3306127cb758e5517a322f6da9e636f91036)

Merge pull request #43091 from guits/wip-52545-pacific

pacific: ceph-volume: support no_systemd with lvm migrate

Merge pull request #43098 from guits/wip-52547-pacific

pacific: ceph-volume: remove --all ref from deactivate help

Merge pull request #42925 from lxbsz/flush_mdlog_pacific

pacific: client: flush the mdlog in unsafe requests' relevant and auth MDSes only

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42923 from lxbsz/pg_num-pacific

pacific: qa: increase the pg_num for cephfs_data/metadata pools

Reviewed-by: Ramana Raja <rraja@redhat.com>

os/bluestore: fix bluefs migrate command

After migrating DB volume to a slow one RocksDB still
needs to be provided with slow.db path to properly access relevant files under db.slow subfolder.
Without that specification it tries to access them under 'db' one which
results in "not-found" error.

Fixes: https://tracker.ceph.com/issues/40434
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 90852d9b6f0da7967121200c9a1c56bed1929d2d)

qa/osd-bluefs-volume-ops: reproduce bluefs migrate bug

Reproduces: https://tracker.ceph.com/issues/40434
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 0b0f8ef12fcc53666809b5bea6d24d07a8777425)

Merge pull request #43023 from ifed01/wip-ifed-fix-bluefs-replay-crc-pac

pacific: os/bluestore: accept undecodable multi-block bluefs transactions on log

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42990 from rhcs-dashboard/wip-52325-pacific

pacific: pybind/mgr: Fix IPv6 url generation

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

ceph-volume: remove --all ref from deactivate help

Until the `--all` feature is implemented.

This was partially removed in c13901f but not from the help command.

Fixes: https://tracker.ceph.com/issues/50109
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 3a385bc38686e45c734ffaa182a23837aaceaaa2)

Merge pull request #43087 from guits/wip-52538-pacific

pacific: ceph-volume: fix raw list with logical partition

Merge pull request #43089 from guits/wip-52542-pacific

pacific: doc/ceph-volume: add lvm migrate/new-db/new-wal

ceph-volume: support no_systemd with lvm migrate

The `ceph-volume lvm migrate/new-db/new-wal` commands don't support
running on non systemd systems or within containers.
Like other ceph-volume commands (lvm activate/batch/zap or raw activate)
we also need to be able to use the --no-systemd flag.

Fixes: https://tracker.ceph.com/issues/51854
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 96d84acf5aedc9cf462230fd5fb685f0562a1371)

doc/ceph-volume: add lvm migrate/new-db/new-wal

The inital PR was implementing those new commands only adds the man page
and not the online documentation.

Fixes: https://tracker.ceph.com/issues/51814
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 22de626220e6145fce8e1fd963802170f099f733)

Merge pull request #43076 from guits/wip-52525-pacific

pacific: ceph-volume: pvs --noheadings replace pvs --no-heading

ceph-volume: test the lsblk command in list.py

Let's test we use the expected args when we build the `lsblk` command
to list the devices present on the node.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 959ea33c0ba118984d678403b234e7628ccac3dc)

ceph-volume: fix raw list with logical partition

This is a regression introduced by 9212420, when the host is using a
logical partition then lsblk reports that partition as a child from the
physical device.
That logical partition is prefixed by the `└─` character.

This leads the `raw list` subcommand to show the lsblk error on the stderr.

```
$ ceph-volume raw list
{}
 stderr: lsblk: `-/dev/sda1: not a block device
```

The lsblk command output looks like:

```
$ lsblk --paths --output=NAME --noheadings
/dev/sda
└─/dev/sda1
/dev/sdb
/dev/sdc
/dev/sdd
```

Using the `--list` option with lsblk solves the issue.

```
$ lsblk --list --paths --output=NAME --noheadings
/dev/sda
/dev/sda1
/dev/sdb
/dev/sdc
/dev/sdd
```

Fixes: https://tracker.ceph.com/issues/52504
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 7a6486bf4ddb2d789cc16047d8163baa8473e253)

Merge pull request #42937 from batrick/i51977

pacific: client: make sure only to update dir dist from auth mds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #43007 from ifed01/wip-ifed-fix-52311-pac

pacific: os/bluestore: fix using incomplete bluefs log when dumping it.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #43001 from ktdreyer/pacific-52449-rm-lsb-release

pacific: Remove dependency on lsb_release

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #43000 from ktdreyer/pacific-52471-rm-gdbm

pacific: ceph.spec.in: drop gdbm from build deps

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42987 from MrFreezeex/wip-52461-pacific

pacific: rbd-mirror: add perf counters to snapshot replayer

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #42980 from idryomov/wip-50522-pacific

pacific: rbd: fix default pool handling for nbd map/unmap

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

Merge pull request #42979 from idryomov/wip-52063-pacific

pacific: rbd-mirror: fix potential async op tracker leak in start_image_replayers

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #42972 from idryomov/wip-51867-pacific

pacific: pybind/rbd: fix mirror_image_get_status

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #43029 from sebastian-philipp/pacific-backport-42535-42682-42252-42293-42816-42589-42590-42592-42690-42405-

pacific: cephadm: September batch 1

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

ceph-volume: pvs --noheadings replace pvs --no-heading

Fixes: https://tracker.ceph.com/issues/52482
Signed-off-by: FengJiankui <fengjiankui@inspur.com>
(cherry picked from commit 7083c4ffc63274bdd3103e9c5296bc70acbe745f)

cephadm: Add test_command_list_networks

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8b768153b8829692fe5ad833a8f86c70f8101360)

cephadm: test_networks: Move tests from test_cephadm

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 419dfd6afae96739efe52a979603b257dd000568)

cephadm: list-networks: Avoid duplicated IPs

Fixes: https://tracker.ceph.com/issues/52083
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit b65822fc8cd3cbf541d50e9ee69d77ef9d1e9cfd)

cephadm: Fix TestCephVolume.test_noop

Fixes `PermissionError: [Errno 13] Permission denied: '/var/lib/ceph/760b...5a5fbb'`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 1978cd33238351045a8b7d856a1e35e36221f09a)

cephadm: Set tcmalloc env var for Ceph daemons

We don't need to set the TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES environment
variable for non Ceph daemons.

Fixes: https://tracker.ceph.com/issues/52334
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit c08c56324fd53a60518cd75859fe8ec8771e4104)

cephadm:Add listening ports to gather-facts output

This patch adds tcp and udp listening ports to the data
returned by gather-facts. This can be used to check port
availability prior to trying to deploying daemons, to
catch port conflicts earlier. IPv4 and IPv6 are supported

Fixes: #52038
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 0c4e88993bc04d248b1616ba4d46ebe24bf17381)

cephadm: check for openntpd.service as time sync service

openntpd is an alternative implementation of time synchronization
by the openbsd project and is packaged for debian and ubuntu
since at least jessie / 18.04 with the service named openntpd.service

Signed-off-by: Oleander Reis <oleander.reis@hostserver.de>
(cherry picked from commit dadf786e77f0adc2f62fd91a70f4ed31a82c1e32)

mgr/cephadm: make scheduler able to accomodate offline/maintenance hosts

Fixes: https://tracker.ceph.com/issues/51027
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 01a580b3082f83347181bf835b9ef698cecfc015)

cephadm: better port stripping

in case the specified port number contains leading zeros

Signed-off-by: Yuxiang Zhu <vfreex@gmail.com>
(cherry picked from commit 496a2c40c87858c188467b7dbacf5e2385ad521c)

cephadm: propagate environment variables to subprocesses

so that I can use an http(s) proxy for external network access when running cephadm.

e.g.
```sh
http_proxy=http://proxy:8080 https_proxy=http://proxy:8080 cephadm pull
```

Signed-off-by: Yuxiang Zhu <vfreex@gmail.com>
(cherry picked from commit 8f72f774892be6aec458452fc2678578f2436e90)

cephadm: Fix bootstrap error with IPv6 mon-ip

Fix the following error by removing the enclosing square brackets in `ctx.mon_ip`.

This issue also occurs on pacific.

```sh
$ sudo cephadm --verbose bootstrap --mon-ip fd49:277:42:2020:c4ba:e6ff:fedf:604

/usr/bin/ceph: stderr > ssh -F ssh_config -i ~/cephadm_private_key root@[fd49:277:42:2020:c4ba:e6ff:fedf:604]
Traceback (most recent call last):
  File "/usr/sbin/cephadm", line 3776, in prepare_ssh
    cli(args)
  File "/usr/sbin/cephadm", line 4076, in cli
    return CephContainer(
  File "/usr/sbin/cephadm", line 3263, in run
    out, _, _ = call_throws(self.ctx, self.run_cmd(),
  File "/usr/sbin/cephadm", line 1453, in call_throws
    raise RuntimeError('Failed command: %s' % ' '.join(command))
RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16 -e NODE_NAME=ceph-node2 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/fee89f24-f3af-11eb-87b1-c6bae6df0604:/var/log/ceph:z -v /tmp/ceph-tmpnjeaui27:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpjfm82r9d:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16 orch host add ceph-node2 [fd49:277:42:2020:c4ba:e6ff:fedf:604]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/sbin/cephadm", line 8230, in <module>
    main()
  File "/usr/sbin/cephadm", line 8218, in main
    r = ctx.func(ctx)
  File "/usr/sbin/cephadm", line 1759, in _default_image
    return func(ctx)
  File "/usr/sbin/cephadm", line 4142, in command_bootstrap
    prepare_ssh(ctx, cli, wait_for_mgr_restart)
  File "/usr/sbin/cephadm", line 3778, in prepare_ssh
    raise Error('Failed to add host <%s>: %s' % (host, e))
__main__.Error: Failed to add host <ceph-node2>: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16 -e NODE_NAME=ceph-node2 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/fee89f24-f3af-11eb-87b1-c6bae6df0604:/var/log/ceph:z -v /tmp/ceph-tmpnjeaui27:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpjfm82r9d:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16 orch host add ceph-node2 [fd49:277:42:2020:c4ba:e6ff:fedf:604]
Releasing lock 140115256165328 on /run/cephadm/fee89f24-f3af-11eb-87b1-c6bae6df0604.lock
Lock 140115256165328 released on /run/cephadm/fee89f24-f3af-11eb-87b1-c6bae6df0604.lock
```

Signed-off-by: Yuxiang Zhu <vfreex@gmail.com>
(cherry picked from commit d3f5aaf40267a168f9709be0292b160638297999)

mgr/cephadm/upgrade: Upgrade status: We also upgrade non-ceph daemons

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit bcd127b46a097d18f43ee829ff8b97033902b05e)

doc/cephadm: rewrite client-setup.rst

This improves the text in client-setup.rst.

We should make certain that the technical details
in this file remain current in July 2021. This
file was origingally written in November 2019.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 18a15891df23222a60affdf9cfcd5760cfb0fa05)

mgr/dashboard: set rgw credentials: fix api tests

Fixes: https://tracker.ceph.com/issues/44605
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit a682b9d7a49fbd50c69494f947a6c0565ad0c78e)

mgr/dashboard: run-frontend-e2e-tests.sh: remove unneeded rgw setting

Fixes: https://tracker.ceph.com/issues/44605
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 8e482f36f2b4eca228f8990690414c0ec513bf88)

mgr/dashboard: rgw service creation form: add realm and zone to service spec.

Align rgw service id pattern with cephadm: https://github.com/ceph/ceph/pull/39877
  - Update rgw pattern to allow service id for non-multisite config.
  - Extract realm and zone from service id (when detected) and add them to the service spec.

Fixes: https://tracker.ceph.com/issues/44605
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 0575844192502ded32962b75a91cf51de22e97e6)

mgr/dashboard: connect-rgw: rename to set-rgw-credentials; refactoring

- Rename the dashboard command to better reflect its behavior.
- Rename '_radosgw_admin' method to 'send_rgwadmin_command' for consistency with
  'send_mon_command' and move it to the mgr_module.py .
- Cleanup: remove unneeded rgw settings.
- Better error handling and test coverage.

Fixes: https://tracker.ceph.com/issues/44605
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 6e20ef1dd35f3681d14cd4e08ca63eb20edc2c88)

mgr/dashboard: connect-rgw: adaptation and test coverage

- Align Dashboard with cephadm: configure credentials using the same logic.
- Fix: create a 'dashboard' user per realm (before: only on 1st realm).
- Lint fixes, test coverage, method renaming to better reflect behavior and method visibility.

Fixes: https://tracker.ceph.com/issues/44605
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 0fcf0a7827cf4e8748a382613f9c8d1715c4a1e8)

mgr/cephadm: re-check dashboard <-> rgw creds when rgw daemons created/destroyed

We don't always know when a realm is created/destroyed, but we can use
service config and purge to cover most such cases.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 91e185d956495885575f052848204169937efde7)

mgr/dashboard: add 'dashboard connect-rgw' command

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 478c53ba68feb0448d9b4bb8be2f5012f7d85b9b)

doc/mgr/dashboard: simplify dashboard+rgw config docs

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 599116a0680296776445d614c55b63f145a700b6)

cephadm: no need to explicitly enable prometheus module

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4dc860feef4802d0432c54f3a40584ad7e9138bf)

mgr/cephadm: enable prometheus module before deploying prometheus

The mon will restart the mgr when the module is enabled, so we don't
really have to do anything here.  The raise is there just in case the
mgr doesn't immediately get the new mgrmap and respawn, although there is
likely no harm done if we continue to deploy prometheus in the meantime,
even if we're interrupted partway through.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 50dc1d0decb2fbf7b9129bddba940969410be5cd)

mgr/cephadm: drop daemon_id arg to CephadmService.config()

Unused (and nonsensical since this is *service* config).

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e9dabdab6540d949bd3401fef66ec583368b33de)

doc/cephadm: no need to manually enable the prometheus module

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit afa509ec572b1099c2e8d8c69787a764a1fcc44a)

doc/install/containers: quay.io!

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d1c744ee534a6196964ef546f4e9bbd302907dfe)

mgr/cephadm: add ability to remove offline host

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 219887301dfdeac55c75a7105bf3a2851fe56387)

os/bluestore: accept undecodable multi-block bluefs transactions on log
replay.

We should proceed with OSD startup when detecting undecodable bluefs
transaction spanning multiple disk blocks during log replay.
The rationale is that such a transaction might appear during unexpected
power down - just not every disk block is written to disk. Hence we can
consider this a normal log replay stop condition.

https://tracker.ceph.com/issues/52079

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 2ba5cb7865c2a7452ee2abda5e0986ce9c42f4b9)

Merge pull request #42939 from batrick/i51832

pacific: mds: META_POP_READDIR, META_POP_FETCH, META_POP_STORE, and cache_hit_rate are not updated

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42938 from batrick/i51932

pacific: mds: MDCache.cc:5319 FAILED ceph_assert(rejoin_ack_gather.count(mds->get_nodeid()))

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42936 from batrick/i51198

pacific: msg: active_connections regression

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42935 from batrick/i51935

pacific: mds: add debugging when rejecting mksnap with EPERM

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42956 from ifed01/wip-ifed-fix-huge-omap-rename-pac

pacific: os/bluestore: cap omap naming scheme upgrade transaction.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42534 from liewegas/use-quay-pacific

pacific: cephadm: use quay, not docker

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #42889 from rhcs-dashboard/wip-52378-pacific

pacific: mgr/dashboard: stats=false not working when listing buckets

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

os/bluestore: fix using incomplete bluefs log when dumping it.

BlueFS superblock might contain incomplete list of physical extents for
bluefs log. Hence we should alway replay ops for ino 1 to get them properly.

Fixes: https://tracker.ceph.com/issues/52311
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 47632fe9baf7f20921c3e6888e29c4478fa5ff2d)

Merge pull request #42974 from s0nea/wip-52308-pacific

pacific: mgr/dashboard: don't notify for suppressed alerts

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

rpm, debian: remove lsb-release

Nothing requires this any more, so we can remove it from the RPM
and Debian packaging.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit cbc0127f3435a1c62344df377e19577c4931b376)

 Conflicts:
debian/control - trivial ordering

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

make-debs.sh: remove lsb_release

The lsb_release utility brings in a lot of other dependencies. Remove
it from the make-debs script.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit 92f73a2bb94d7ff645072b98c2b9c7eea8208782)

test/docker-test: remove lsb_release

The lsb_release utility brings in a lot of other dependencies. Remove
it from the docker-test script.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit 24af043ee57419bd28d72ffc85cd1078df9af123)

workunits/rbd: remove lsb_release

The lsb_release utility brings in a lot of other dependencies. Remove
it from the RBD workunit script.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit 65f4d6eb3664a4cc6120031679b8368cbc02a4a5)

 Conflicts:
qa/workunits/rbd/qemu-iotests.sh - trivial fix

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

workunits/rgw: remove lsb_release

The lsb_release utility brings in a lot of other dependencies. Remove
it from the RGW workunit Perl scripts.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit db66f6f35735d2aae2170af3520d222cf748ef9b)

ceph.spec.in: drop gdbm from build deps

neither gdbm nor libgdbm is used when building ceph, so let's drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 9d05d7337460322b78d676bf32a52c4519c7ae1f)

Merge pull request #42898 from smithfarm/wip-52380-pacific

pacific: cmake: exclude "grafonnet-lib" target from "all"

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #42897 from ifed01/wip-ifed-fix-spurious-read-errors-warn-pac

pacific: os/bluestore: respect bluestore_warn_on_spurious_read_errors setting

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42888 from callithea/wip-51951-pacific

pacific: Don't persist report data

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

os/bluestore: cap omap naming scheme upgrade transactoin.

We shouldn't use single per-onode transaction for such an upgrade when onode's omap list is huge. This results in similarly sized WAL/SST files which are inefficient, might cause high memory usage and sometimes error-prone.

Fixes: https://tracker.ceph.com/issues/49170
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit e897fa243c1dd38329733b452872616023f14ac8)

Merge pull request #42969 from idryomov/wip-krbd-escape-match-sysattr-pacific

pacific: krbd: escape udev_enumerate_add_match_sysattr values

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

mgr/{prometheus,restful}: Fix url generation again

Fixes

```
======================================================================
ERROR: test_standby (tasks.mgr.test_prometheus.TestPrometheus)
----------------------------------------------------------------------
urllib3.exceptions.LocationParseError: Failed to parse: http://172.21.15.71:7789metrics

```

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 45361ea9b3dab917a455bc37bc96aabd0bf64fcf)

mgr/cephadm: fix generation of wrong IPv6 urls

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 3d96f79b7401b9a391782b9592547be4d6e4ae56)

mgr/restful: Fix url generation for IPv6 hosts

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 98091591e312d27f25e630306c31399180fb90ce)

mgr/prometheus: Fix url generation for IPv6 hosts

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 0b8701b9140c6c42d70c11c15c7618341e55f6fa)

mgr/dashboard: Fix redirect to active MGR

Fixes: https://tracker.ceph.com/issues/52117
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e367796d05a44097edd738326b06ca41e28c48fb)

pybind/mgr: move build_url from dashboard to mgr_util

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 29883d68ef696a361b4fec0af22505835170a74d)

rbd-mirror: add perf counters to snapshot replayer

Add bytes replayed and statistics about number of snapshots synchronized
and how long it took to do so.

Fixes: https://tracker.ceph.com/issues/50973
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
(cherry picked from commit cf9317b58f966c64554f46092fc4162b5e8fd5fb)

qa: wait pgs to be clean before using the pools

Or in some use cases, like the mds-full tests, we will hit the
"PG_AVAILABILITY" warning.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit a448d1c3ee161dc1910ff9fd71a82f7eb0020526)

qa: ignore PG_RECOVERY_FULL and PG_DEGRADED for mds-full

Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 3456ff26289915b1cdbc534274f269fc603edacd)

qa: wait more time since there have many more pgs than before

Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 999c787ac60beb0d28f09d574242ec64eee06686)

qa: do not multiple the full ratio twice

The cluster has already multiple the full ratio before returning
the "max_avail".

Fixes: https://tracker.ceph.com/issues/50984
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit ba3833a6221eca47bc6536cc9b2ac72b2c5fcf04)

qa: do not raise for kclient for _fsync test

For kclient, the write() will return -ENOSPC instead of the fsync().

Fixes: https://tracker.ceph.com/issues/45434
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit a96ee419088a1550bb530ed16fac44ab1db1367a)

qa: use the pg autoscale mode to calcuate the pg_num

Setting the pg_num to 8 is too small that some osds maybe not covered by the
pools, some osds maybe overloaded. Remove the hardcodeing pg_num here and let
the pg autoscale mode to calculate it as needed, and at the same time set the
pg_num_min to 64 to avoid the pg_num to small.

If ec pool is used, for the test cases most datas will go to the ec pool and
the primary replicated pool will store a small amount of metadata for all the
files only, so set the target size ratio to 0.05 should be enough.

Fixes: https://tracker.ceph.com/issues/45434
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit c1cea71299300ed2fd2a34d5c10a1948f3016031)

Cherry-pick notes:
- Fixed Invalid command: missing required parameter pg_num(<int[0-]>)