Zac Dover [Sat, 13 Aug 2022 23:14:56 +0000 (09:14 +1000)]
doc/rados/operations: add prompts to operating.rst
This commit adds ".. prompt:: bash $"-style prompts to operating.rst.
This brings this file up to the standard established in 2020 when
Kefu added support for the ".. prompt::" directive.
This commit is a part of an initiative to modernize the presentation
of all BASH commands in the RADOS documentation.
The progress of this project can be tracked here:
https://tracker.ceph.com/issues/57108
Ilya Dryomov [Fri, 12 Aug 2022 09:10:45 +0000 (11:10 +0200)]
rbd: remove incorrect use of std::includes()
- std::includes() requires sorted ranges but command specs aren't
sorted
- std::includes() purpose is to check whether the second range is
a subsequence of the first range but here the size of the second
range is always equal to the size of the first range, which means
that, had the ranges been sorted, std::includes() would have checked
straight equality
Ilya Dryomov [Fri, 12 Aug 2022 09:10:45 +0000 (11:10 +0200)]
rbd: find_action() should sort actions first
The order in which objects with static storage duration in
different TUs are initialized is undefined. If the compiler
chooses to initialize Shell::Action objects in action/Trash.cc
before Shell::Action objects in action/TrashPurgeSchedule.cc,
all "rbd trash purge schedule ..." commands get shadowed by
"rbd trash purge" command:
$ rbd trash purge schedule list
rbd: too many arguments
The confusing error arises because "rbd trash purge" takes a single
positional argument. "schedule" gets interpreted as <pool-spec> and
"list" generates an error.
Kefu Chai [Fri, 12 Aug 2022 05:06:25 +0000 (13:06 +0800)]
mgr/dashboard: bump up more-itertools
before this change, more-itertools tries to import Sequence from
collections, this leads us to failures like:
```
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
__import__(pkg_name)
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/__init__.py",
line 9, in <module>
import cherrypy
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/cherrypy/__init__.py",
line 76, in <module>
from . import _cprequest, _cpserver, _cptree, _cplogging, _cpconfig
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/cherrypy/_cprequest.py",
line 11, in <module>
from cherrypy import _cpreqbody
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/cherrypy/_cpreqbody.py",
line 135, in <module>
import cheroot.server
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/cheroot/server.py",
line 96, in <module>
from .workers import threadpool
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/cheroot/workers/threadpool.py",
line 20, in <module>
from jaraco.functools import pass_none
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/jaraco/functools.py",
line 8, in <module>
import more_itertools
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/more_itertools/__init__.py",
line 1, in <module>
from more_itertools.more import * # noqa
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/more_itertools/more.py",
line 3, in <module>
from collections import Counter, defaultdict, deque, Sequence
ImportError: cannot import name 'Sequence' from 'collections'
(/usr/lib/python3.10/collections/__init__.py)
ERROR: InvocationError for command
/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/bin/python3
-m dashboard.controllers.docs
/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/tmp/openapi.yaml
(exited with code 1)
```
after this change, more-itertools is pin'ed at the latest stable
at the time of writing, which includes the fixes including
https://github.com/more-itertools/more-itertools/commit/30a861bc5a4f53a9ba73923c9048a3632a0f9d18
.
please note, more-itertools dropped python3.3 support. but neither
do us support this python version, so we should be safe.
Yingxin Cheng [Sun, 7 Aug 2022 08:05:42 +0000 (16:05 +0800)]
crimson/os/seastore: construct TransactionManager classes after device mount
To construct TransactionManager after all the devices are discoverred.
Also, it makes the following cleanups possible:
* Cleanup SeaStore and TransactionManager factory methods.
* Decouple TransactionManager from SegmentManagerGroup.
* Drop the unnecessary tm_make_config_t.
* Drop the unnecessary add_device() methods.
Casey Bodley [Wed, 10 Aug 2022 22:23:34 +0000 (18:23 -0400)]
ceph.spec.in: install gcc-toolset-11-libatomic-devel in x86_64 also
otherwise after enabling gcc-toolset-11, cmake fails with:
- Performing Test HAVE_LIBATOMIC - Failed
CMake Error at cmake/modules/CheckCxxAtomic.cmake:66 (message):
Host compiler /opt/rh/gcc-toolset-11/root/usr/bin/g++ requires libatomic,
but it is not found
Adam King [Wed, 10 Aug 2022 15:44:55 +0000 (11:44 -0400)]
Merge pull request #46400 from rkachach/fix_issue_55733
mgr/cephadm: adding dynamic prometheus configuration based on http_sd_config
Reviewed-by: Adam King <adking@redhat.com> Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com> Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Kefu Chai [Wed, 10 Aug 2022 14:35:14 +0000 (22:35 +0800)]
ceph.spec.in: %enable_devtoolset11 only if the macro is defined
there is chance that we are using `yum-builddep` to prepare the
build dependencies. in that case, gcc-toolset-11-build is not
installed. it's like a chicken-egg dilemma, but the point is
`yum-builddep` is able to pull in the gcc-toolset-11-build. once
gcc-toolset-11-build is installed, we will have the %enable_devtoolset11
rpm macro.
Ernesto Puerta [Fri, 5 Aug 2022 08:56:36 +0000 (10:56 +0200)]
.github/workflows: add create-backport action
Currently there's a cron job in a teuthology VM running a script to find all trackers in needs-backports and to create their corresponding backport trackers. This is done through the [Backport Bot](https://tracker.ceph.com/users/12172) Redmine account.
This PR intends to run this cron job task as a periodic Github Action.
pybind/mgr: maximum recursion depth exceeded in comparison
Original issue from https://github.com/python/typing_extensions/issues/10
In case typing-extensions is upgrade to 4.1.1
(https://pypi.org/project/typing-extensions/4.1.1/, the final Python 3.6
supported version), Ceph MGR will failed with pg_autoscaler as below:
[root@cp-nightsky ~]# pip list | grep typing-extensions
typing-extensions 4.1.1
[root@cp-nightsky ~]# journalctl -xef -u ceph-mgr@cp-nightsky.service
May 12 06:16:39 cp-nightsky.novalocal systemd[1]: Started Ceph cluster manager daemon.
-- Subject: Unit ceph-mgr@cp-nightsky.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit ceph-mgr@cp-nightsky.service has finished starting up.
--
-- The start-up result is done.
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.181+0000 7fe3eb521700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'pg_autoscaler' while running on mgr.cp-nightsky: maximum recursion depth exceeded in comparison
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.182+0000 7fe3eb521700 -1 pg_autoscaler.serve:
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.182+0000 7fe3eb521700 -1 Traceback (most recent call last):
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 206, in serve
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: self._maybe_adjust()
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 423, in _maybe_adjust
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: ps, root_map, pool_root = self._get_pool_status(osdmap, pools)
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 325, in _get_pool_status
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: self.log.debug('skipping empty subtree %s', cr_name)
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/logging/__init__.py", line 1296, in debug
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: self._log(DEBUG, msg, args, **kwargs)
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/logging/__init__.py", line 1443, in _log
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: exc_info, func, extra, sinfo)
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/logging/__init__.py", line 1413, in makeRecord
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: sinfo)
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/logging/__init__.py", line 277, in __init__
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if (args and len(args) == 1 and isinstance(args[0], collections.Mapping)
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/abc.py", line 193, in __instancecheck__
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: return cls.__subclasscheck__(subclass)
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/abc.py", line 228, in __subclasscheck__
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/abc.py", line 228, in __subclasscheck__
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/typing.py", line 1154, in __subclasscheck__
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: return super().__subclasscheck__(cls)
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/abc.py", line 228, in __subclasscheck__
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib/python3.6/site-packages/typing_extensions.py", line 426, in __subclasscheck__
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib/python3.6/site-packages/typing_extensions.py", line 426, in __subclasscheck__
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib/python3.6/site-packages/typing_extensions.py", line 426, in __subclasscheck__
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: [Previous line repeated 239 more times]
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib/python3.6/site-packages/typing_extensions.py", line 421, in __subclasscheck__
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if self.__extra__ in subclass.__mro__:
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: RecursionError: maximum recursion depth exceeded in comparison
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.291+0000 7fe3de3c7700 -1 client.0 error registering admin socket command: (17) File exists
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.291+0000 7fe3de3c7700 -1 client.0 error registering admin socket command: (17) File exists
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.291+0000 7fe3de3c7700 -1 client.0 error registering admin socket command: (17) File exists
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.291+0000 7fe3de3c7700 -1 client.0 error registering admin socket command: (17) File exists
May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.291+0000 7fe3de3c7700 -1 client.0 error registering admin socket command: (17) File exists
Solution provided by
https://github.com/python/typing_extensions/issues/10#issuecomment-1131767191
not working:
Reviewed-by: Kefu Chai <tchaikov@gmail.com> Reviewed-by: Daniel Gryniewicz <dang@redhat.com> Reviewed-by: Matt Benjamin <mbenjamin@redhat.com> Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Kefu Chai [Mon, 8 Aug 2022 14:41:17 +0000 (22:41 +0800)]
pybind/mgr/dashboard: do not use distutils.version.StrictVersion
replace `distutils.version.StrictVersion` with
`pkg_resources.parse_version()`
as the former is deprecated, see https://peps.python.org/pep-0632/.
let's use `pkg_resources` instead. this change also addresses
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1010894.
we have this issue when testing with an ubuntu jammy test node.
see https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1967139
When a network mount is present in `/proc/mounts` but for any reason
the corresponding server is down, this function hangs forever.
In a cluster deployed with cephadm, the consequence is that
it triggers `ceph-volume inventory` commands that hang and stay in D
state.
The idea here is to use a thread with a timeout to abort the call if the
timeout is reached.
`get_mounts()` is now a method of a class so we can exclude a path
altogether during the whole `inventory` execution (otherwise,
ceph-volume would try to access it as many devices there is on the
host which could slow down the inventory execution)
Michaela Lang [Thu, 4 Aug 2022 17:14:49 +0000 (19:14 +0200)]
- mgr/cephadm: provide an additional hint when running into I/O closed exception from execnet seen as
Error EINVAL: Can't communicate with remote host `127.0.0.1`, possibly because python3 is not installed there: cannot send (already closed?)
Kefu Chai [Mon, 8 Aug 2022 12:40:52 +0000 (20:40 +0800)]
ceph.spec.in: add libatomic to BuildRequires on fedora
otherwise we'd have failures like
/opt/compiler-explorer/gcc-trunk-20220808/bin/../lib/gcc/x86_64-linux-gnu/13.0.0/../../../../x86_64-linux-gnu/bin/ld:
/tmp/ccVlMbVh.o: in function `std::atomic<tagged_ptr>::store(tagged_ptr,
std::memory_order)':
/opt/compiler-explorer/gcc-trunk-20220808/include/c++/13.0.0/atomic:273:
undefined reference to `__atomic_store_16'
when generating the building system using CMake on fedora 36.
Ronen Friedman [Sun, 20 Mar 2022 13:20:59 +0000 (13:20 +0000)]
osd/scrub: modify scrub behaviour under no-scrub
Fix no-scrub & nodeep-scrub related code to match requirements:
- deep scrubs should be allowed to execute when no-scrub is set;
- some initiated scrubs (i.e. not periodic ones) might be changed
from the requested 'deep' to 'shallow'.
projects / ceph.git / log