Jason Dillaman [Thu, 7 May 2020 20:25:50 +0000 (16:25 -0400)]
librbd: generic helper for tracking in-flight IO and managing flush requests
Layers that potentially queue IOs but not all IOs will need to track all
in-flight IOs to properly ensure that a flush cannot complete while
older IO is still stuck in a queue.
Jason Dillaman [Tue, 14 Jul 2020 22:49:30 +0000 (18:49 -0400)]
crush/CrushWrapper: rebuild reverse maps after rebuilding crush map
The Objecter will crash when localized reads are enabled and two threads
attempt to rebuild the (invalidated) reverse maps concurrently. This
should address the issue for the Objecter use-case without the need to
add additional locking.
Fixes: https://tracker.ceph.com/issues/44311 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8b866794f5b3674c5e3ad9adceb5e3230d55a0e0)
Switch the default path for the immutable object cache from "/tmp" to
"/tmp/ceph_immutable_object_cache" to prevent the deletion of all
files in the "/tmp" directory and sub-directories for unconfigured
daemons.
Jason Dillaman [Fri, 17 Jul 2020 14:25:20 +0000 (10:25 -0400)]
immutable-object-cache: fix error handling during start up
Previously the daemon would crash if it couldn't remove all the
files from the specified cache directory. It would also crash if
it could not access its specified domain socket file.
Fixes: https://tracker.ceph.com/issues/45169 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f6dcb6c3dbd45574e1a7ca3f5e8f1f52a0d7ae1e)
Jason Dillaman [Wed, 22 Jul 2020 15:25:56 +0000 (11:25 -0400)]
librbd: flush all queued object IO from simple scheduler
Normally IO is tracked via the AioCompletion's async_op but the
scheduler will "complete" writes while the IO might be still
executing. Therefore, prior to shutting down this dispatch layer
we need to wait for all IO to complete.
Fixes: https://tracker.ceph.com/issues/46668 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 75ff8fd14dccaa7d2f11ba8a561ad3c0f410585c)
Jason Dillaman [Tue, 28 Jul 2020 01:14:18 +0000 (21:14 -0400)]
librbd: update hidden global config when setting pool config override
The new "dev"-level global config setting will be updated when any
pool-level config override is updated. librbd clients will detect
the new global-level config update and trigger a refresh. This avoids
the need for potentially tens of thousands of librbd clients
registering a watch on the pool metadata object or periodically polling
the pool metadata object for updates.
Fixes: https://tracker.ceph.com/issues/46694 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f45df9fe786e8057c491c082e840483759d67e9e)
Conflicts:
src/common/options.cc
- "rbd_quiesce_notification_attempts", "rbd_default_snapshot_quiesce_mode", and
"rbd_plugins" options have not been backported to Octopus, yet
Jason Dillaman [Mon, 27 Jul 2020 19:31:09 +0000 (15:31 -0400)]
librbd: initial config watcher implementation
The config watcher will initially observe all "rbd_" configuration
updates received from the MON that have not been locally overridden
at the pool and/or image level.
rgw: policy: reuse eval_principal to evaluate the policy principal
Since the other edge case when no Principal or a NotPrincipal is supplied also
must be accounted for, which is already done in eval_principal function. Also
reraising the error as Effect::Pass in line with the previous output, though an
Effect::Deny would also work here.
The commit adds 2 different parts to show the
Telemetry activation notification in the dashboard:
1. The Telemetry activation notification component
itself. It contains the definition of the
notification panel.
2. The Telemetry notification service. The service
is needed to be able to show/hide the
notification from:
* the component itself (e.g. when clicking the
button button)
* the Telemetry configuration component (when
enabling/disabling Telemetry)
* the navigation component (to set the css-
classes accordingly)
Fixes: https://tracker.ceph.com/issues/45464 Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit f7e457952ba5780b4b7fc3a1e4290c02d4e5d70d)
Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/telemetry/telemetry.component.ts
A form loading directive has been introduced in master: https://github.com/ceph/ceph/pull/34746
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.spec.ts
The test configuration has been improved in master: https://github.com/ceph/ceph/pull/34965
src/pybind/mgr/dashboard/frontend/src/app/shared/components/telemetry-notification/telemetry-notification.component.html
src/pybind/mgr/dashboard/frontend/src/app/shared/components/telemetry-notification/telemetry-notification.component.spec.ts
The alert component has been migrated from ngx-bootstrap to ng-boostrap in master: https://github.com/ceph/ceph/pull/35297
TestBed.get has been replaced by TestBed.inject in master: https://github.com/ceph/ceph/pull/34934
src/pybind/mgr/dashboard/frontend/src/app/shared/services/telemetry-notification.service.spec.ts
TestBed.get has been replaced by TestBed.inject in master: https://github.com/ceph/ceph/pull/34934
The backport contains one commit less (39a26ae4b2a1b154c88a414c4abee1c04808effa
is missing) than the original pull request because the migration from alert to
ngb-alert (https://github.com/ceph/ceph/pull/35297) has not been backported.
Tatjana Dehler [Tue, 26 May 2020 13:41:02 +0000 (15:41 +0200)]
mgr/dashboard: reset pwd notification value
Reset the password notification value if the
user logs out. Otherwise if a another user logs
in, who doesn't see the password expiration
notification (because his password is not going to
expire soon), will see a blank bar instead.
Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/shared/components/pwd-expiration-notification/pwd-expiration-notification.component.html
The alert component has been migrated from ngx-bootstrap to ng-boostrap in master: https://github.com/ceph/ceph/pull/35297
The API call is a task and the response status is determined by whether
the call is completed within a pre-defined duration (2 seconds) or not.
We should also allow the status when the call takes longer.
Jan Fajerski [Thu, 30 Jul 2020 09:46:00 +0000 (11:46 +0200)]
ceph-volume: dependency on python-ceph-common
Since e5b585d15de8b07e0a179344d4187582a5c069f2 ceph-volume depends on
python-ceph-common. This commit introduces this dependency for the
ceph-osd rpm (which includes ceph-volume) and installs the dependency
for tox runs.
Fixes: https://tracker.ceph.com/issues/46772 Fixes: e5b585d15de8b07e0a179344d4187582a5c069f2 Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit cb432fe41d4ea8cb71aa592e0727d2da1978121f)
Matt Benjamin [Wed, 5 Aug 2020 14:45:45 +0000 (10:45 -0400)]
rgwlc: fix conditional decode of legacy lc op structures
Fixes the case where an upgraded radosgw operates on an unupgraded
OSD, so likely these cases do not execute.
Fixes: https://tracker.ceph.com/issues/46838 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 3814ff2cc4503330d7094bcdcaf7c5ea630ce5b9)
Matt Benjamin [Wed, 22 Jul 2020 17:58:00 +0000 (13:58 -0400)]
rgwlc: fix error return from backward-compat decode
Exception handling in backward-compat case returned EIO even when
the actual backward-compat decode succeeded.
Fixes: https://tracker.ceph.com/issues/46677 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit fa3036168b67a191d7ac37d0d34409f5b3d87b5d)
Matt Benjamin [Fri, 22 May 2020 21:43:46 +0000 (17:43 -0400)]
rgwlc: add lifecycle perf counters
Historically, RGW lifecycle processing has not provided perfcounters.
A single expire counter was added in recent parallel lifecycle changes,
but granular counters for
current expiration
non-current expiration
delete-marker expiration
current transition
non-current transition
mpu aborts
are arguably much more useful, and can be summed in reports.
Fixes: https://tracker.ceph.com/issues/45667 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 57d0bcf57d7d5774b63465ae46dafd0253afde04)
Matt Benjamin [Wed, 29 Apr 2020 12:00:55 +0000 (08:00 -0400)]
rgw: lc: restore unordered listing
With current sharding strategy, a delete marker is always coresident
with all corresponding objects, including noncurrent, so unordered
listing -is- compatible with lookahead.
The incoming version of -new- clients required to be one
higher to trigger the new result format.
Yes, this means that "radosgw-admin lc list" against an
un-upgraded OSD will display Unix epoch for lc-entry
start_time (harmless, and in general will not happen).
Matt Benjamin [Fri, 10 Apr 2020 14:47:46 +0000 (10:47 -0400)]
rgwlc: give radosgw-admin lc process more freedom to run
Makes two primary changes:
1. allows the admin lc process to re-run a bucket (but not
when a session is ACTIVE, to avoid overwriting session state)
2. to avoid confusion when rgw_lc_debug_interval is set and is small,
don't force lc process to exit early, allowing one run to clear all
expired objects during debug and test
octopus: mgr/dashboard: fix rbdmirroring dropdown menu
Fixes: https://tracker.ceph.com/issues/46566 Signed-off-by: Avan Thakkar <athakkar@redhat.com>
This seems to be the same situation as https://tracker.ceph.com/issues/44458,
but in a different part of the frontend. Since we are not backporting
ngx-bootstrap to octopus, this has to be manually fixed in octopus.
Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/dashboard/info-card/info-card-popover.scss
- The order of CSS properties changed in the master.
Kefu Chai [Sun, 2 Aug 2020 01:31:01 +0000 (09:31 +0800)]
mgr/cephadm: track dependency in tox.ini
* track dependency for running tox tests with tox.ini. the fix
test is not performed by "make check", so `install-deps.sh`
is not responsible for preparing the autopep8 wheels. hence
we can just put it in tox.ini
* remove "python_version >= '3'" as we do not support python2
Matthew Oliver [Thu, 30 Jul 2020 03:18:48 +0000 (13:18 +1000)]
cephadm: add unwrap_ipv6 helper method
When we pass in a mon-ip that is ipv6 we want it wrapped, this is so it
can be properly inserted into the mon_addrv address.
But there are times we need to unwrap it to test it's a valid ipv6
address.
This patch adds a helper method `unwrap_ipv6` which takes a string and
returns it either unwrapped or as it is, so it's harmless to other types
of ips.
This allows us to check a wrapped ipv6 base_ip with the networks on the
host.
Paul Cuzner [Wed, 29 Jul 2020 04:00:37 +0000 (16:00 +1200)]
mgr/cephadm: fix call to cephadm for daemon restarts etc
The call currently passes the daemon_type as the first
parameter - but the function expects the hostname. This
results in failures when attempting daemon restarts through the ceph orch daemon <action> <daemon_id>
command.
Fixes: https://tracker.ceph.com/issues/46740 Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 94e418581c9126dc199def6078d6a8449bdec8df)
projects / ceph.git / log