Jason Dillaman [Tue, 30 Jun 2020 12:44:38 +0000 (08:44 -0400)]
librbd: add 'write_zeroes' public C/C++ API methods
Unlike the existing 'discard' option which is more of a hint to
attempt to release space, the new 'write_zeroes' APIs will ensure
that the entire provided extent is fully zeroed.
When the radosgw-admin radoslist was running, the portion where it
lists the incomplete multipart uploads did not handle the marker
correctly between calls to the ordered list op. This fixes those
issues and does some additional clean-up.
Marcus Watts [Tue, 10 Mar 2020 04:34:01 +0000 (00:34 -0400)]
rgw: fix loop problem with swift stat on account.
Loop logic in RGWStatAccount::execute was failing to change the
marker before repeating a call to rgw_read_user_buckets(). This resulted
that in the case that a user had more than 1000 buckets, this routine
would loop forever.
Fixes: https://tracker.ceph.com/issues/44671 Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit c9f3cf6ed6a649812d47dc1fa1357ae0749dbcbf)
mgr/volumes: Deprecate protect/unprotect CLI calls for subvolume snapshots
Subvolume snapshots required to be protected, prior to cloning the same.
Also, protected snapshots were not allowed to be unprotected or removed,
if there were in-flight clones, whose source was the snapshot being
removed.
The protection of snapshots explicitly is not required, as these can be
prevented from being removed based only on the in-flight clones checks.
This commit hence deprecates the additional protect/unprotect requirements
prior to cloning a snapshot.
In addition to deprecating the above, support to query a subvolume for
supported features, via the info command, is added. The feature list
is set to "clone" and "auto-protect", where the latter is useful to
decide if protect/unprotect commands are required or not.
Conflicts:
src/pybind/mgr/volumes/fs/operations/versions/subvolume_v1.py
- required manual resolution during cherry-pick to delete unused functions
doc/cephfs/fs-volumes.rst
- added line space for correcting multi-line command rendering
rgw: orphan list teuthology test & fully-qualified domain issue
Sometimes when teuthology machines are provisioned, the command
`hostname --fqdn` does not provide a fully qualified domain name but
instead just the hostname (e.g., smithi149 instead of
smithi149.front.sepia.ceph.com). This prevents the teuthology test for
rgw-orphan-list from running successfully [for example, the hostname
was for some reason mis-interpreted as the bucket name in the
request].
This commit checks whether the hostname derived from `hostname --fqdn`
contains any '.'s and if it does not, it will append
".front.sepia.ceph.com" to the hostname. This is a hack, but until
teuthology machines are configured appropriately it seems to be a
reasonable work-around.
rgw: rgw-orphan-list -- fix interaction, quoting, and percentage calc
The interactive mode wasn't working due to prompts going to stdout
instead of stderr. If a space should appear in temporary file, it will
generate a shell error, so quoting was added. Furthermore if there are
no objects found in a pool, a divide by zero error will be
generated. This commit addresses these issues.
J. Eric Ivancich [Tue, 23 Jun 2020 21:05:59 +0000 (17:05 -0400)]
rgw: orphan-list timestamp fix
When creating intermediate and output files, the rgw-orphan-list
script uses a timestamp using the `date` command. The hour was
inserted with "%k" but that padds with a space rather than a zero. So
that's changed to "%H".
qa/rgw: integration test for `rgw-orphan-list` & `radosgw-admin radoslist`
Add teuthology test for `rgw-orphan-list` in a new tool suite under
rgw. It only needs to be tested under one configuration. And the new
tool sub-suite can be used by other tooling int he
future. radosgw-admin `radoslist` is tested indirectly through
`rgw-orphan-list` and therefore does not need its own test.
J. Eric Ivancich [Tue, 21 Apr 2020 15:28:58 +0000 (15:28 +0000)]
qa/rgw: allow the rgw teuthology task to capture/set dns names
A teuthology workunit might want to use the rgw task, setting the
rgw-dns-name and/or rgw-dns-s3website-name configuration options to
the fully-qualified domain name. Existing code implies that setting
these configuration options to the empty string will do that. However
the current logic does not support that given it has Python
conditionals that treat the empty string as false. This fixes that.
Now the following teuthology tasks YAML will work as expected:
Adds a radosgw-admin subcommand and possibly a bucket name (if not,
all buckets are assumed) and walks the bucket(s) listing(s) and the
manifest (if it exists) for each object in the listing to generate the
rados objects that represent the rgw objects in the bucket.
Also adds a tool named rgw-orphan-list that will produce a list in a
local file of what appear to be rgw orphans.
NOTE: This is not a cherry-pick from master because the feature was
originally written for an older version of Ceph and is being gradually
forward-ported to newer and newer versions.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Ernesto Puerta [Mon, 11 May 2020 18:33:25 +0000 (20:33 +0200)]
mgr/dashboard: work with RBD images v1
Add support for RBD Image Format v1:
- This format lacks ID field, required for dashboard. Instead,
RBD image `block_name_prefix` is used as unique ID (together with pool
id and namespace)
- Additionally, `image_format` is now exposed.
- In the front-end side:
- Copy action on a v1 image will cause the image to be copied to v2
format.
- List doesn't allow Move to Trash on v1 images,
- Details section now shows `image_format` for images,
- Edit Form disables flags not supported for v1 (`deep-flatten`,
`layering`, `exclusive-lock`).
- Protect does not work on v1 images or v2 images created from v1
ones.
Avan Thakkar [Tue, 25 Feb 2020 08:49:10 +0000 (14:19 +0530)]
mgr/dashboard: add popover list of managers in landing page Fixes: https://tracker.ceph.com/issues/42979 Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit cdfeb1d196c7d47340baae2be5910b90c889e778)
Conflicts:
src/pybind/mgr/dashboard/controllers/health.py
-removed few lines as those lines were removed in the master branch too
src/pybind/mgr/dashboard/frontend/src/app/ceph/dashboard/health/health.component.html
-added the braces missing
there is chance that we have a PG just created but fails to peered
before a mgr module retrieves the health report from mgr. in that
case, the "last_peered" field is not set, as that pg has not peered. but
normally, the newly created PG will be active+clean in couple seconds
which is way under the default setting of mon_pg_stuck_threshold (60
seconds).
so in this change, if the "last_whatever" is not set, we also use the
"last_changed" as a reference to see if the PG is healthy, and only
consider PG stuck if the last_changed is also too old.
Improve cache by running requests in a thread and prevent multiple
requests to Ceph from multiple sources (e.g. Prometheus instances) which
increase load on the manager.
Fixes: https://tracker.ceph.com/issues/45554 Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit fa69d8e1112d2688e2979ab10c303d1facb6bc76)
Conflicts:
src/pybind/mgr/prometheus/module.py
- line 1096 if condition block changed
- second commit "mgr/prometheus: enable mypy type checking for prometheus module"
is not required for the backport as discussed with Patrick
who is the creator of the original PR.
Patrick Donnelly [Tue, 17 Dec 2019 23:10:34 +0000 (15:10 -0800)]
qa: add upgrade test for volume upgrade from legacy
This tests that volumes created using the ceph_volume_client.py library
continue to be accessible/function via the Nautilus/Octopus ceph-mgr
volumes plugin.
Fixes: https://tracker.ceph.com/issues/42723 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0c8899c985763826fb77fed599eeea5624263a35)
Conflicts:
qa/tasks/cephfs/fuse_mount.py
- use the coding format with nautilius
- use `require-osd-release nautilus` for this version
mgr/dashboard: Prometheus query error in the metrics of Pools, OSDs and RBD images Fixes: https://tracker.ceph.com/issues/45068 Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 47b515c09496da8fc326300bab6618250466effe)
This commit adds the dmcrypt support in `ceph-volume raw` mode.
Note about `ceph-volume raw list` change:
Given `lsblk -J` (json output) option isn't available on all OS, I came up with
adding '--inverse' option to the existing command which allows us to get the
mapper devices list in that command output. Not listing root devices containing
partitions shouldn't have side effect since we are in `ceph-volume raw`
context.
example:
running `lsblk --paths --nodeps --output=NAME --noheadings` doesn't allow to
get the mapper list because the output is like following :
adding `--inverse` is a trick to get around this issue, the counterpart is that
we can't list root devices if they contain at least one partition but this
shouldn't be an issue in `ceph-volume raw` context given we only deal with
raw devices.
Conflicts:
- src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-configuration-list/rbd-configuration-list.component.html
Formatting change in the master, use the code in the master to fix. Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
projects / ceph.git / log