ceph.spec.in: Requires python3-scikit-learn only on fedora and suse

so far, python3-scikit-learn is not packaged by RHEL8/CentOS8 or EPEL8.

see
https://src.fedoraproject.org/rpms/python-scikit-learn/branches?branchname=master

this change should address the regression introduced by
d13b441cdb63e37ad3b8ba36cbd3edc33b1e4bbd

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35648 from ideepika/fix-44948

ceph.spec: fix mgr diskprediction_local dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35722 from ideepika/drop-el7

ceph.spec: drop support for el7 rpm builds

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35534 from varshar16/wip-doc-fix-SyntaxWarning

doc/scripts/gen_state_diagram.py: Fix literal comparison syntax warnings

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35677 from sebastian-philipp/_refresh_hosts_and_daemons_forall_hosts

mgr/cephadm: speedup _refresh_hosts_and_daemons by 10x

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #30592 into master

* refs/pull/30592/head:
qa: fix flake8 warnings
doc: add documentation for new ephemeral pinning feature
mds: enable ephemeral pinning by default
pybind/mgr/volumes: wire up pinning subvolumes/subvolumegroups
qa: adapt tests for empty pinned dir export
qa: break export pin tests into discrete tests
qa: add more ephemeral pin tests
qa: add tests for ephemeral pinning
mds: add maximum random ephemeral pin percentage
mds: replicate random pin state
mds: finish implementation of ephemeral pins
mds: do string equality comparison
mds: add ephemeral pinning for subtrees
mds: trim pinned and empty subtrees
mds: refactor remove_subtree
mds: allow export of pinned directory if empty
mds: reduce subtree processing verbosity
mds: skip export of empty directories
mds: remove frozen export pin from queue
mds: simplify for loop construction
mds: add debug messages for export queue processing
qa: refactor _wait_subtree and _get_subtree
qa: use status from wait_for_daemons
qa: quietly print json output from asok commands

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35736 from ivancich/wip-orphan-list-timestamp-fix

rgw: orphan-list timestamp fix

Reviewed-by: Ali Maredia <amaredia@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

qa: fix flake8 warnings

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc: add documentation for new ephemeral pinning feature

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: enable ephemeral pinning by default

This feature is enabled by default for Pacific. This commit will be
dropped in the backport to Octopus.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

pybind/mgr/volumes: wire up pinning subvolumes/subvolumegroups

Fixes: https://tracker.ceph.com/issues/41541
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: adapt tests for empty pinned dir export

Previously, empty pinned directories were not migrated.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: break export pin tests into discrete tests

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add more ephemeral pin tests

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add tests for ephemeral pinning

Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>

mds: add maximum random ephemeral pin percentage

This new config is designed to prevent creating too many subtrees from a
random ephemeral pinning policy.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: replicate random pin state

This is slightly evil in its current form. The MDS should use locks to
transmit state changes but right now it's just set when the CInode is
replicated. This replication of this state marker is necessary for
failover situations where we want the randomly pinned subtree to remain
pinned across failovers.

Note: this problem does not exist for the ephemeral distributed pins
because simple knowledge of the immediate parent's setting (which is
replicated normally) is sufficient to determine if the CInode is
ephemerally distributed. Ditto for regular export pins.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: finish implementation of ephemeral pins

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: do string equality comparison

The string::find method would return true for ceph.dir.pin even for the
other ephemeral pin xattr names. For this reason, it was never possible
to actually turn ephemeral pins on!

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: add ephemeral pinning for subtrees

This PR introduces inode xattrs export_ephemeral_random and
export_ephemeral_distributed which enables two different metadata
distribution strategies - the first being suitable for a more depthwise
scaling of metadata (height of the tree keeps increasing) and the latter
for horizontal scaling (many subtrees under a single parent).
export_ephemeral_distributed applies is not hierarchical. Any direct
descendant directory (i.e. a child directory) has an ephemeral export
pin applied to it according to a consistent hash of the child directory
inode number. export_ephemeral_distributed is hierarchical like
"export_pin". Any CDir loaded into the cache may be ephemerally pinned
to a random rank. Like "export_ephemeral_distributed", the random rank
is determined by a consistent hash.

The metadata distribution strategies are facilitated by using John
Lamping and Eric Veach's Jump Consistent Hashing as the consistent hash
algorithm. This hashing algorithm eliminates the need to store the data
structures representing the consistent hash cluster state and performs
as well as Akamai's original implementation providing a fairly uniform
distribution. This algorithm only works for distributed systems with
numbered buckets (nodes) arranged in ascending order and cluster resizes
does not produce any holes in the arrangement of nodes i.e (0, 1, 2, 3)
--[removing node 1]--> (0, 1, 2). CephFS satisfies these conditions as
the MDSs are arranged as numbered ranks and cluster modifications does
not produce any holes in the resulting arrangement of ranks.

Fixes: https://tracker.ceph.com/issues/41302
Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: trim pinned and empty subtrees

Before export (and ephemeral) pinned subtrees are stuck in cache
forever.

Add qa test for checking export pinned directories can be trimmed.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: refactor remove_subtree

Search each map only once as necessary.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: allow export of pinned directory if empty

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: reduce subtree processing verbosity

and some mild loop refactoring.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: skip export of empty directories

Note: empty as in no cached sub-entries.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: remove frozen export pin from queue

The export already belongs on the node id so there's no need to keep it
in the queue.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: simplify for loop construction

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: add debug messages for export queue processing

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: refactor _wait_subtree and _get_subtree

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use status from wait_for_daemons

Avoid an extra `fs dump` call.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: quietly print json output from asok commands

Pretty print output once. Use --format=json so the stdout on teuthology
is not pretty printed, taking hundreds of lines.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35752 into master

* refs/pull/35752/head:
qa: fix flake8 error

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa: fix flake8 error

    ./tasks/vstart_runner.py:33:1: F401 'io.BytesIO' imported but unused

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35665 into master

* refs/pull/35665/head:
vstart_runner.py: improve a conditional in _do_run

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #33879 into master

* refs/pull/33879/head:
client: fix snap directory atime

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35518 into master

* refs/pull/35518/head:
mds: fix filelock state when Fsc is issued

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #35522 into master

* refs/pull/35522/head:
vstart_runner: set default values of stdout and stderr to None

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35532 into master

* refs/pull/35532/head:
Revert "Revert "qa/suites/rados/mgr/tasks/module_selftest: whitelist mgr client getting backlisted""

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge PR #35540 into master

* refs/pull/35540/head:
qa/cephfs: don't pass cmd args from run_as_user as str
qa/cephfs: refactor run_as_root() to user run_as_user()

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35582 into master

* refs/pull/35582/head:
mds: reset heartbeat in EMetaBlob replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35619 into master

* refs/pull/35619/head:
mds: reset sequence number when new rank0 mds is available
mds: dump mds name when processing mds map

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rgw: orphan-list timestamp fix

When creating intermediate and output files, the rgw-orphan-list
script uses a timestamp using the `date` command. The hour was
inserted with "%k" but that padds with a space rather than a zero. So
that's changed to "%H".

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #35208 from prateek2211/refactor

rgw: Replace boost::string_ref/view with std::string_view

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35331 from yuvalif/fix_issue_43536

rgw/pubsub: remove deprecated radogw-admin pubsub creation commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #35604 into master

* refs/pull/35604/head:
CODEOWNERS: added doc-writers team

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #35739 into master

* refs/pull/35739/head:
mgr/volumes/nfs: Reorder imports

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35716 from rzarzynski/wip-bl-rval-claim_append

common, msgr/async: bufferlist::claim_append() doesn't require intermediaries anymore

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35643 from sebastian-philipp/cephadm-fix-already-in-use-msg

mgr/cephadm: fix broken "already in use" exception message

Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35633 from matthewoliver/cephadm_bootstap_ipv6

cephadm: Set ms bind ipv6 when mon-ip is ipv6

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35589 from jschmid1/osd_servicedescription_fix

mgr/cephadm: exclude osds without osdspec_affinity

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35655 from batrick/i46081

cephadm: restrict mds caps to cephfs pools

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35651 from sebastian-philipp/cephadm-unit-run-set-e

cephadm: unit.run: add `set -e`

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #35654 from neha-ojha/wip-44186

pybind/mgr/pg_autoscaler/module.py: do not update event if ev.pg_num== ev.pg_num_target

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35710 from rhcs-dashboard/wip-46144-fix-sort-ui

mgr/dashboard: fix UI for Device class column in OSDs list

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35742 from theanalyst/doc/releases/oct-update

doc: releases: updates for octopus

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35721 from bk201/wip-43971

mgr/dashboard: fix Source column i18n issue in RBD configuration tables

Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #35734 from tspmelo/wip-fix-tooltip

mgr/dashboard: Fix display of tooltips inside tables

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

doc: releases: octopus: fixup changelog

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

doc: releases: update timeline for octopus

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mgr/volumes/nfs: Reorder imports

This patch reorders import to fix "ImportError: cannot import name 'TimedOut'".
As flake8 requires similar import types to be grouped together.

Signed-off-by: Varsha Rao <varao@redhat.com>

CODEOWNERS: added doc-writers team

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #35473 from aclamk/fix-45903-bluefs-log-growth

os/bluestore: fix bluefs log growth.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

doc: update releases table

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

doc: releases: octopus: add v15.2.3 release notes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

doc: releases: add v15.2.2 release notes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #35572 from p-se/wip-pse-prometheus-mod-cache

mgr/prometheus: improve cache

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #35155 from yuvalif/verify_wraparound_in_cls_q

cls/queue: add unit test to verify behaviour when queue wraps

Reviewed-by: Pritha Srivastava <prsivas@redhat.com>

Merge pull request #35674 from tchaikov/wip-qa-tox

qa,cmake: enable flake8 and import-tasks tests

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #35580 from xiexingguo/wip-46024

common, osd: add sanity checks around osd_scrub_max_preemptions

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #35662 from neha-ojha/wip-42820

mgr/DaemonServer.cc: make 'config show' on fsid work

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35656 from ifed01/wip-ifed-fix-spillover-again

os/bluestore: fix spillover test case

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35416 from yanghonggang/master

os/bluestore: add blkin traces to BlueStore

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #35689 from changchengx/options

common: use template shortcuts like is_unsigned_v

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35660 from mgfritch/orch-yaml-safe-load

mgr/orch: read untrusted input using the yaml SafeLoader

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Stephan Müller <smueller@suse.com>

qa/tasks/cephfs: flake8 fixes

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35621 from rzarzynski/wip-crimson-alien_memory

crimson: experimentally run AlienStore with Seastar's default allocator

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35727 from rzarzynski/wip-crimson-no-msgrv2.1

crimson/msgr: don't advertise the on-wire format v2.1.

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35719 from tchaikov/wip-cephadm-sans-el7

qa/suites/rados: do not test with el7

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #32855 from jdurgin/wip-message-cap

osd: make message cap option usable again

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>

mgr/dashboard: Fix display of tooltips inside tables

Hover event was not being propagated into the components inside the table.

Fixes: https://tracker.ceph.com/issues/45968
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35732 from neha-ojha/wip-perf-distro

qa/suites/perf-basic: remove supported-all-distro

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

qa/suites/perf-basic: remove supported-all-distro

d4a04809fd7fd8aaf447005f76a1090db99d75c5 did not serve it's purpose, we
need to remove supported-all-distro to only run on ubuntu_latest

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35678 from sebastian-philipp/cephadm-availabe-ssh-set

mgr/cephadm: make `orch status` fail, if ssh key is not set

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #35645 from sebastian-philipp/python-common-remove-old-cephadm-orch.py

python-common: remove old `cephadm_orchestrator.py`

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35242 from bk201/wip-38234

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

qa/tasks/cephfs/test_nfs.py: flake8 fixes

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits/mon: flake8 fixes

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/standalone: flake8 fixes

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/vstart_runner: flake8 fixes

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/ragweed: flake8 fixes

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/mgr/test_orchestrator_cli: flake8 fixes

Signed-off-by: Kefu Chai <kchai@redhat.com>

tasks/mgr/dashboard: flake8 fixes

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks: flake8 fixes

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: drop flake8-py2 test

as we've dropped py2 support, no need to run flake8 with python2
anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add TOX_ENVS keyword to test with listed test envs

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #35702 into master

* refs/pull/35702/head:
vstart: just use ganesha.nfsd that's in $PATH

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #35724 into master

* refs/pull/35724/head:
vstart: Add ip value to nfs mount information

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35646 into master

* refs/pull/35646/head:
qa/tasks/cephfs/nfs: Poll for max 60 seconds to ensure removal of ganesha services
pybind/mgr/volumes/nfs: Remove unused cephfs module
mgr/volumes/nfs: Ignore if ganesha disconnects

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

crimson/msgr: don't advertise the on-wire format v2.1.

We don't support it yet.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os/alienstore: make the thread pool private.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>