When running the `lvm migrate` subcommand without any args then the
ceph-volume command fails with a stack trace.
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 59, in newfunc
return f(*a, **kw)
File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 151, in main
terminal.dispatch(self.mapper, subcommand_args)
File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
instance.main()
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/main.py", line 46, in main
terminal.dispatch(self.mapper, self.argv)
File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
instance.main()
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/migrate.py", line 520, in main
self.migrate_osd()
File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 16, in is_root
return func(*a, **kw)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/migrate.py", line 403, in migrate_osd
if self.args.osd_id:
AttributeError: 'Migrate' object has no attribute 'args'
That's because we're exiting the parse_argv function but we continue to
execute the migrate_osd function. We should instead exit from the main function.
This update the parsing argument to have the same code than new-db and
new-wal classes.
Now the parsing is done in the make_parser function but the argv testing is
done in the main function allowing to exit the program and displaying the
help message when no arguments are provided.
The `ceph-volume lvm migrate/new-db/new-wal` commands don't support
running on non systemd systems or within containers.
Like other ceph-volume commands (lvm activate/batch/zap or raw activate)
we also need to be able to use the --no-systemd flag.
This is a regression introduced by 9212420, when the host is using a
logical partition then lsblk reports that partition as a child from the
physical device.
That logical partition is prefixed by the `└─` character.
This leads the `raw list` subcommand to show the lsblk error on the stderr.
```
$ ceph-volume raw list
{}
stderr: lsblk: `-/dev/sda1: not a block device
```
libudev uses fnmatch(3) for matching attributes, meaning that shell
glob pattern matching is employed instead of literal string matching.
Escape glob metacharacters to suppress pattern matching.
pybind/rbd: explain why "primary" isn't exposed in mirror_image_status_list()
"primary" is part of mirror image info (rbd_mirror_image_info_t) and
is exposed in mirror_image_get_status(). mirror_image_status_list(),
even though it is often thought of as an equivalent of repeated calls
to mirror_image_get_status(), doesn't actually fetch the mirror image
info.
pybind/rbd: actually append site_status dict to remote_statuses
Using += operator is wrong -- only site_status keys get appended
(and repeatedly at that in case there is more than one remote site
as the keys are added one by one).
Will Smith [Fri, 23 Jul 2021 19:18:12 +0000 (15:18 -0400)]
rbd: Fix mirror_image_get_status in rbd python bindings
When retrieving the status of a mirrored image from the Python rbd
library, a TypeError is raised.
*To Reproduce:*
Set up two Ceph clusters for block storage, and configure image
mirroring between their pools. Create a least one image with mirroring
enabled, then run the following script on either cluster (once the image
exists everywhere):
with rados.Rados(conffile=CONF_PATH) as cluster:
with cluster.open_ioctx(POOL_NAME) as ioctx:
with rbd.Image(ioctx, IMAGE_LABEL) as image:
image.mirror_image_get_status()
```
This will result in the following stack trace:
```
Traceback (most recent call last):
File "repo-bug.py", line 10, in <module>
image.mirror_image_get_status()
File "rbd.pyx", line 3363, in rbd.requires_not_closed.wrapper
File "rbd.pyx", line 5209, in rbd.Image.mirror_image_get_status
TypeError: list indices must be integers or slices, not str
```
Conflicts:
src/pybind/mgr/dashboard/controllers/rgw.py
- rgw-list endpoint doesn't have daemon_name parameter in octopus, so adopted
changes accordingly.
Deepika Upadhyay [Wed, 23 Jun 2021 05:12:38 +0000 (10:42 +0530)]
mon/PGMap: DIRTY field as N/A in `df detail` when cache tier not in use
'ceph df detail' reports a column for DIRTY objects under POOLS even
though cache tiers not being used. In replicated or EC pool all objects
in the pool are reported as logically DIRTY as they have never been
flushed .
we display N/A for DIRTY objects if the pool is not a cache tier.
See Rook issue https://github.com/rook/rook/issues/7940 for full
information.
Ceph bluestore disks can sometimes appear as though they have "phantom"
Atari (AHDI) partitions created on them when they don't in reality. This
is due to a series of bugs in the Linux kernel when it is built with
Atari support enabled. This behavior does not appear for raw mode OSDs on
partitions, only on disks.
Changing the on-disk format of Bluestore OSDs comes with
backwards-compatibility challenges, and fixing the issue in the Kernel
could be years before users get a fix. Working around the Kernel issue
in ceph-volume is therefore the best place to fix the issue for Ceph.
To work around the issue in Ceph volume, there are two behaviors that need
adjusted:
1. `ceph-volume inventory` should not report that a partition is
available if the parent device is a BlueStore OSD.
2. `ceph-volume raw list` should report parent disks if the disk is a
BlueStore OSD and not report the disk's children, BUT it should still
report children if the parent disk is not a BlueStore OSD.
Using only the exit status of `ceph-bluestore-tool show-label` to
determine if a device is a bluestore OSD could report a false negative
if there is a system error when `ceph-bluestore-tool` opens the device.
A better check is to open the device and read the bluestore device
label (the first 22 bytes of the device) to look for the bluestore
device signature ("bluestore block device"). If ceph-volume fails to
open the device due to a system error, it is safest to assume the device
is BlueStore so that an existing OSD isn't overwritten.
Tim Serong [Thu, 5 Aug 2021 08:48:57 +0000 (18:48 +1000)]
mgr/cephadm: pass --container-init to `cephadm deploy` if specified
This (sort of) reverts a small part of 442904bf2da, in order to
work correctly with 82d30afc413. The problem is that the latter
commit sets CONTAINER_INIT=False, which means we need to explicitly
pass --container-init to cephadm if we want that behaviour turned on.
There's a kink though: --container-init is only valid with cephadm's
"adopt", "bootstrap" and "deploy" commands; it's not a global option.
The cephadm mgr module doesn't invoke "adopt" or "bootstrap", only
"deploy", so we're only setting it in that specific case.
Fixes: https://tracker.ceph.com/issues/52064 Signed-off-by: Tim Serong <tserong@suse.com>
David Caro [Wed, 16 Jun 2021 08:32:15 +0000 (10:32 +0200)]
monitoring/grafana/cluster: use per-unit max and limit values
The value we get is a perunit, so the limits and the max value should
be over 1, not 100. Note that the value being shown was correct, it
was the gauge that was not showing the correct indicators.
octopus-only: this change was introduced in: c2486c7239f2efff1f87a0c6064ccbf792e90bf0
as a linking unintentional typo while creating symlink, the parent
commit was octopus only hence we don't require it in other branches.
Fixes failures like the following with rhel 8.3 in octopus
```
2021-08-03T17:32:19.328 INFO:tasks.workunit.client.0.smithi148.stdout:No match for argument: libarchive-3.3.3
2021-08-03T17:32:19.338 INFO:tasks.workunit.client.0.smithi148.stderr:Error: Unable to find a match: libarchive-3.3.3
2021-08-03T17:32:19.376 DEBUG:teuthology.orchestra.run:got remote process result: 1
2021-08-03T17:32:19.377 INFO:tasks.workunit:Stopping ['rados/test_envlibrados_for_rocksdb.sh'] on client.0...
```
Follow-up to https://github.com/ceph/ceph/pull/42421
Improvements and some adaptations related to the jenkins job.
Fixes: https://tracker.ceph.com/issues/51612 Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 65b75000b7694cb3cbe617bbec28c513a2522be8)
Conflicts:
doc/dev/developer_guide/dash-devel.rst
- Put changes in HACKING.rst as this file does not exist in the octopus branch.
src/pybind/mgr/dashboard/ci/cephadm/bootstrap-cluster.sh
- Resolve conflict originated by code that is deleted anyway.
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Patrick Donnelly [Fri, 30 Jul 2021 02:35:13 +0000 (19:35 -0700)]
Merge PR #42537 into octopus
* refs/pull/42537/head:
mon/MDSMonitor: propose if FSMap struct_v is too old
mon/MDSMonitor: give a proper error message if FSMap struct_v is too old
qa: add tests for fs dump of epoch and trimming
qa: add file system support for dumping epoch
mon/MDSMonitor: return mon_mds_force_trim_to even if equal to current epoch
mon: add debugging for trimming methods
mon: fix debug spacing
Reviewed-by: Ramana Raja <rraja@redhat.com> Reviewed-by: Neha Ojha <nojha@redhat.com>
Patrick Donnelly [Thu, 15 Jul 2021 01:02:20 +0000 (18:02 -0700)]
mon/MDSMonitor: propose if FSMap struct_v is too old
To flush older versions which may still be an empty MDSMap (for clusters
that have never used CephFS), we need to force a proposal so older
versions of the struct are trimmed.
This is the main fix of this branch. We removed code which processed old
encodings of the MDSMap in the mon store via 60bc524. That broke old
ceph clusters which never used CephFS (see cited ticket below). This is
because the initial epoch is an empty MDSMap (back in Infernalis/Hammer)
that is never updated. So, the fix here is to just do proposals
periodically until all of the old structs are automatically trimmed by
the mons.
Fixes: 60bc524827bac072658203e56b1fa3dede9641c5 Fixes: https://tracker.ceph.com/issues/51673 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 56c3fc802ee8848ba85da4300adcc2ee8bd95416)
Conflicts:
src/mds/FSMap.cc: adjust for octopus which decodes old MDSMaps
src/mon/MDSMonitor.h: trivial conflicts
Patrick Donnelly [Wed, 14 Jul 2021 20:31:21 +0000 (13:31 -0700)]
mon/MDSMonitor: return mon_mds_force_trim_to even if equal to current epoch
The PaxosService code already excludes the value returned by
PaxosService::get_trim_to as the upper bound of the range of epochs to
trim. Without this fix, you need to set mon_mds_force_trim_to to one
greater than the epoch you want to trim _and_ force the current epoch to
be one greater than that; the net result being that you can only force
trimming up to 2 epochs behind the current epoch.
This change is helpful for resolving issue 51673, but not strictly
necessary.
Related-to: https://tracker.ceph.com/issues/51673 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d9dc2f11d56fb4341ba5823f8d17459d10f3b2c1)
Conflicts:
src/common/options/mon.yaml.in: drop doc change
Sometimes, it can happen that the osds being destroyed in those tests
are not yet marked as 'down' for some reason. Let's add some retries on
those tasks to avoid CI failures.
rpm: drop use of $FIRST_ARG in ceph-immutable-object-cache
The use of $FIRST_ARG was probably required because the SUSE-specific
%service_* rpm macros were playing tricks on the shell positional parameters.
This is bad practice and error-prone, so let's assume that no macros should do
that anymore and hence it's safe to assume that positional parameters remain
unchanged after any rpm macro call.
qa/workunits/mon/test_mon_config_key: use subprocess.run() instead of proc.communicate()
the loop of proc.communicate() on python3.6, where we always are able to
get something out of stdout and/or stderr PIPEs. and the `stdout` and
`stderr` keep growing until out of memory. and teuthology considers
the command crashed after a while.
Since inject_facts_as_vars is set to false in the ansible.cfg file then we
have to update the references to use ansible_facts[<thing>] instead of
ansible_<thing>.
projects / ceph.git / log