From: Ramana Raja Date: Wed, 24 Dec 2025 10:24:50 +0000 (-0500) Subject: mgr/rbd_support: Fix "start-time" arg behavior X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=5882c17bbf8c816875d81a21bad3438684446f39;p=ceph.git mgr/rbd_support: Fix "start-time" arg behavior The "start-time" argument, optionally passed when adding or removing an mirror image snapshot schedule or a trash purge schedule, does not behave as intended. It is meant to schedule an initial operation at a specific time of day in a given time zone. Instead, it offsets the schedule’s anchor time. By default, the scheduler uses the UNIX epoch as the anchor to calculate recurring schedule times, and "start-time" simply shifts this anchor away from UTC, which can confuse users. For example: ``` $ # current time $ date --universal Wed Dec 10 05:55:21 PM UTC 2025 $ rbd mirror snapshot schedule add -p data --image img1 1h 19:00Z $ rbd mirror snapshot schedule ls -p data --image img1 every 15m starting at 19:00:00+00:00 ``` A user might assume that the scheduler will run the first snapshot each day at 19:00 UTC and then run snapshots every 15 minutes. Instead, the scheduler runs the first snapshot at 18:00 UTC and then continues at the configured interval: ``` $ rbd mirror snapshot schedule status -p data --image img1 SCHEDULE TIME IMAGE 2025-12-10 18:00:00 data/img1 ``` Additionally, the "start-time" argument accepts a full ISO 8601 timestamp but silently ignores everything except hour, minute, and time zone. Even time zone handling is incorrect: specifying "23:00-01:00" with an interval of "1d" results in a snapshot taken once per day at 22:00 UTC rather than 00:00 UTC, because only utcoffset.seconds is used while utcoffset.days is ignored. Fix: Similar to the handling of the "start" argument in the FS snap-schedule manager module, require "start-time" to use an ISO 8601 date-time format with a mandatory date component. Time and time zone are optional and default to 00:00 and UTC respectively. The "start-time" now defines the anchor time used to compute recurring schedule times. The default anchor remains the UNIX epoch. Existing on-disk schedules with legacy-format "start-time" values are updated to include the date Jan 1, 1970. The `snap schedule ls` output now displays "start-time" with date and time in the format "%Y-%m-%d %H:%M:00". The display time is in UTC. Fixes: https://tracker.ceph.com/issues/74192 Signed-off-by: Ramana Raja --- diff --git a/PendingReleaseNotes b/PendingReleaseNotes index e5f12413838..8060eb62b6e 100644 --- a/PendingReleaseNotes +++ b/PendingReleaseNotes @@ -28,6 +28,12 @@ estimates. This feature is enabled by default for relevant commands including scan_extents, scan_inodes, and other state-changing operations. Related Tracker: https://tracker.ceph.com/issues/63191 +* RBD: Fixed incorrect behavior of the "start-time" argument for mirror + snapshot and trash purge schedules, where it previously offset the schedule + anchor instead of defining it. The argument now requires an ISO 8601 + date-time. The `schedule ls` output displays the start time in UTC, including + the date and time in the format "%Y-%m-%d %H:%M:00". The `schedule status` + output now displays the next schedule time in UTC. >=20.0.0 diff --git a/doc/man/8/rbd.rst b/doc/man/8/rbd.rst index 311167c5f6c..348d8d79716 100644 --- a/doc/man/8/rbd.rst +++ b/doc/man/8/rbd.rst @@ -1037,7 +1037,7 @@ To restore an image from trash and rename it:: To create a mirror snapshot schedule for an image:: - rbd mirror snapshot schedule add --pool mypool --image myimage 12h 14:00:00-05:00 + rbd mirror snapshot schedule add --pool mypool --image myimage 12h 2020-01-14T11:30+05:30 Availability ============ diff --git a/doc/rbd/rbd-mirroring.rst b/doc/rbd/rbd-mirroring.rst index 2f0ec7c622e..7602191ca81 100644 --- a/doc/rbd/rbd-mirroring.rst +++ b/doc/rbd/rbd-mirroring.rst @@ -428,10 +428,11 @@ image name; interval; and optional start time:: rbd mirror snapshot schedule add [--pool {pool-name}] [--image {image-name}] {interval} [{start-time}] The ``interval`` can be specified in days, hours, or minutes using ``d``, ``h``, -``m`` suffix respectively. The optional ``start-time`` can be specified using -the ISO 8601 time format. For example:: +``m`` suffix respectively. The optional ``start-time`` must be specified in +the ISO 8601 time format. If no UTC offset is provided, UTC is assumed. For +example:: - $ rbd --cluster site-a mirror snapshot schedule add --pool image-pool 24h 14:00:00-05:00 + $ rbd --cluster site-a mirror snapshot schedule add --pool image-pool 24h 2020-01-14T11:30+05:30 $ rbd --cluster site-a mirror snapshot schedule add --pool image-pool --image image1 6h To remove a mirror-snapshot schedules with ``rbd``, specify the @@ -441,12 +442,13 @@ corresponding ``add`` schedule command. To list all snapshot schedules for a specific level (global, pool, or image) with ``rbd``, specify the ``mirror snapshot schedule ls`` command along with an optional pool or image name. Additionally, the ``--recursive`` option can -be specified to list all schedules at the specified level and below. For -example:: +be specified to list all schedules at the specified level and below. + +Schedule start times are always displayed in UTC. For example:: $ rbd --cluster site-a mirror snapshot schedule ls --pool image-pool --recursive POOL NAMESPACE IMAGE SCHEDULE - image-pool - - every 1d starting at 14:00:00-05:00 + image-pool - - every 1d starting at 2020-01-14 06:00:00 image-pool image1 every 6h To view the status for when the next snapshots will be created for @@ -456,11 +458,12 @@ image name:: rbd mirror snapshot schedule status [--pool {pool-name}] [--image {image-name}] -For example:: +The next schedule time is always displayed in UTC. For example:: $ rbd --cluster site-a mirror snapshot schedule status SCHEDULE TIME IMAGE - 2020-02-26 18:00:00 image-pool/image1 + 2026-01-24 06:00:00 image-pool/image1 + Disable Image Mirroring ----------------------- diff --git a/qa/workunits/rbd/cli_generic.sh b/qa/workunits/rbd/cli_generic.sh index bd2a50d7c49..8cef48b238c 100755 --- a/qa/workunits/rbd/cli_generic.sh +++ b/qa/workunits/rbd/cli_generic.sh @@ -1167,12 +1167,21 @@ test_trash_purge_schedule() { expect_fail rbd trash purge schedule remove -p rbd dummy expect_fail rbd trash purge schedule remove -p rbd 1d dummy - rbd trash purge schedule add -p rbd 1d 01:30 + rbd trash purge schedule add -p rbd 1h 2100-01-01T19:00Z + test "$(rbd trash purge schedule ls -p rbd)" = 'every 1h starting at 2100-01-01 19:00:00' + for i in `seq 12`; do + rbd trash purge schedule status -p rbd | grep '2100-01-01 19:00:00' && break + sleep 10 + done + test "$(rbd trash purge schedule status -p rbd --format xml | + xmlstarlet sel -t -v '//scheduled/item/schedule_time')" = '2100-01-01 19:00:00' + rbd trash purge schedule rm -p rbd - rbd trash purge schedule ls -p rbd | grep 'every 1d starting at 01:30' + rbd trash purge schedule add -p rbd 1d 2020-01-14T07:00+05:30 + rbd trash purge schedule ls -p rbd | grep 'every 1d starting at 2020-01-14 01:30:00' expect_fail rbd trash purge schedule ls - rbd trash purge schedule ls -R | grep 'every 1d starting at 01:30' - rbd trash purge schedule ls -R -p rbd | grep 'every 1d starting at 01:30' + rbd trash purge schedule ls -R | grep 'every 1d starting at 2020-01-14 01:30:00' + rbd trash purge schedule ls -R -p rbd | grep 'every 1d starting at 2020-01-14 01:30:00' expect_fail rbd trash purge schedule ls -p rbd2 test "$(rbd trash purge schedule ls -p rbd2 -R --format json)" = "[]" @@ -1193,18 +1202,18 @@ test_trash_purge_schedule() { test "$(rbd trash purge schedule status -p rbd --format xml | xmlstarlet sel -t -v '//scheduled/item/pool')" = 'rbd' - rbd trash purge schedule add 2d 00:17 - rbd trash purge schedule ls | grep 'every 2d starting at 00:17' - rbd trash purge schedule ls -R | grep 'every 2d starting at 00:17' + rbd trash purge schedule add 2d 2020-01-14T05:47+05:30 + rbd trash purge schedule ls | grep 'every 2d starting at 2020-01-14 00:17:00' + rbd trash purge schedule ls -R | grep 'every 2d starting at 2020-01-14 00:17:00' expect_fail rbd trash purge schedule ls -p rbd2 - rbd trash purge schedule ls -p rbd2 -R | grep 'every 2d starting at 00:17' - rbd trash purge schedule ls -p rbd2/ns1 -R | grep 'every 2d starting at 00:17' + rbd trash purge schedule ls -p rbd2 -R | grep 'every 2d starting at 2020-01-14 00:17:00' + rbd trash purge schedule ls -p rbd2/ns1 -R | grep 'every 2d starting at 2020-01-14 00:17:00' test "$(rbd trash purge schedule ls -R -p rbd2/ns1 --format xml | xmlstarlet sel -t -v '//schedules/schedule/pool')" = "-" test "$(rbd trash purge schedule ls -R -p rbd2/ns1 --format xml | xmlstarlet sel -t -v '//schedules/schedule/namespace')" = "-" test "$(rbd trash purge schedule ls -R -p rbd2/ns1 --format xml | - xmlstarlet sel -t -v '//schedules/schedule/items/item/start_time')" = "00:17:00" + xmlstarlet sel -t -v '//schedules/schedule/items/item/start_time')" = "2020-01-14 00:17:00" for i in `seq 12`; do rbd trash purge schedule status --format xml | @@ -1222,18 +1231,18 @@ test_trash_purge_schedule() { xmlstarlet sel -t -v '//scheduled/item/pool'))" = 'rbd2 rbd2' test "$(echo $(rbd trash purge schedule ls -R --format xml | - xmlstarlet sel -t -v '//schedules/schedule/items'))" = "2d00:17:00 1d01:30:00" + xmlstarlet sel -t -v '//schedules/schedule/items/item'))" = "2d2020-01-14 00:17:00 1d2020-01-14 01:30:00" rbd trash purge schedule add 1d - rbd trash purge schedule ls | grep 'every 2d starting at 00:17' + rbd trash purge schedule ls | grep 'every 2d starting at 2020-01-14 00:17:00' rbd trash purge schedule ls | grep 'every 1d' rbd trash purge schedule ls -R --format xml | - xmlstarlet sel -t -v '//schedules/schedule/items' | grep '2d00:17' + xmlstarlet sel -t -v '//schedules/schedule/items' | grep '2d2020-01-14 00:17:00' rbd trash purge schedule rm 1d - rbd trash purge schedule ls | grep 'every 2d starting at 00:17' - rbd trash purge schedule rm 2d 00:17 + rbd trash purge schedule ls | grep 'every 2d starting at 2020-01-14 00:17:00' + rbd trash purge schedule rm 2d 2020-01-14T00:17:00 expect_fail rbd trash purge schedule ls for p in rbd2 rbd2/ns1; do @@ -1272,9 +1281,17 @@ test_trash_purge_schedule() { expect_fail rbd trash purge schedule remove -p rbd 1d dummy expect_fail rbd trash purge schedule remove dummy expect_fail rbd trash purge schedule remove 1d dummy - rbd trash purge schedule ls -p rbd | grep 'every 1d starting at 01:30' + expect_fail rbd trash purge schedule add -p rbd 30m 00:15 + expect_fail rbd trash purge schedule add -p rbd 30m 00:15+05:30 + expect_fail rbd trash purge schedule add -p rbd 30m 2020-13-14T00:15+05:30 + expect_fail rbd trash purge schedule add -p rbd 30m 2020-01-32T00:15+05:30 + expect_fail rbd trash purge schedule add -p rbd 30m 2020-01-14T25:15+05:30 + expect_fail rbd trash purge schedule add -p rbd 30m 2020-01-14T00:60+05:30 + expect_fail rbd trash purge schedule add -p rbd 30m 2020-01-14T00:15+24:00 + + rbd trash purge schedule ls -p rbd | grep 'every 1d starting at 2020-01-14 01:30:00' rbd trash purge schedule ls | grep 'every 2m' - rbd trash purge schedule remove -p rbd 1d 01:30 + rbd trash purge schedule remove -p rbd 1d 2020-01-14T01:30 rbd trash purge schedule remove 2m test "$(rbd trash purge schedule ls -R --format json)" = "[]" @@ -1352,6 +1369,16 @@ test_mirror_snapshot_schedule() { expect_fail rbd mirror snapshot schedule remove -p rbd2/ns1 --image test1 dummy expect_fail rbd mirror snapshot schedule remove -p rbd2/ns1 --image test1 1h dummy + rbd mirror snapshot schedule add -p rbd2/ns1 1h 2100-01-01T19:00Z + test "$(rbd mirror snapshot schedule ls -p rbd2/ns1)" = 'every 1h starting at 2100-01-01 19:00:00' + for i in `seq 12`; do + rbd mirror snapshot schedule status -p rbd2/ns1 | grep '2100-01-01 19:00:00' && break + sleep 10 + done + test "$(rbd mirror snapshot schedule status -p rbd2/ns1 --format xml | + xmlstarlet sel -t -v '//scheduled_images/image/schedule_time')" = '2100-01-01 19:00:00' + rbd mirror snapshot schedule rm -p rbd2/ns1 + rbd mirror snapshot schedule add -p rbd2/ns1 --image test1 1m expect_fail rbd mirror snapshot schedule ls rbd mirror snapshot schedule ls -R | grep 'rbd2 *ns1 *test1 *every 1m' @@ -1403,15 +1430,15 @@ test_mirror_snapshot_schedule() { done rbd mirror snapshot schedule status | grep 'rbd2/ns1/test1' - rbd mirror snapshot schedule add 1h 00:15 - test "$(rbd mirror snapshot schedule ls)" = 'every 1h starting at 00:15:00' - rbd mirror snapshot schedule ls -R | grep 'every 1h starting at 00:15:00' + rbd mirror snapshot schedule add 1h 2020-01-14T04:30+05:30 + test "$(rbd mirror snapshot schedule ls)" = 'every 1h starting at 2020-01-13 23:00:00' + rbd mirror snapshot schedule ls -R | grep 'every 1h starting at 2020-01-13 23:00:00' rbd mirror snapshot schedule ls -R | grep 'rbd2 *ns1 *test1 *every 1m' expect_fail rbd mirror snapshot schedule ls -p rbd2 - rbd mirror snapshot schedule ls -p rbd2 -R | grep 'every 1h starting at 00:15:00' + rbd mirror snapshot schedule ls -p rbd2 -R | grep 'every 1h starting at 2020-01-13 23:00:00' rbd mirror snapshot schedule ls -p rbd2 -R | grep 'rbd2 *ns1 *test1 *every 1m' expect_fail rbd mirror snapshot schedule ls -p rbd2/ns1 - rbd mirror snapshot schedule ls -p rbd2/ns1 -R | grep 'every 1h starting at 00:15:00' + rbd mirror snapshot schedule ls -p rbd2/ns1 -R | grep 'every 1h starting at 2020-01-13 23:00:00' rbd mirror snapshot schedule ls -p rbd2/ns1 -R | grep 'rbd2 *ns1 *test1 *every 1m' test "$(rbd mirror snapshot schedule ls -p rbd2/ns1 --image test1)" = 'every 1m' @@ -1424,7 +1451,14 @@ test_mirror_snapshot_schedule() { expect_fail rbd mirror snapshot schedule remove 1h dummy expect_fail rbd mirror snapshot schedule remove -p rbd2/ns1 --image test1 dummy expect_fail rbd mirror snapshot schedule remove -p rbd2/ns1 --image test1 1h dummy - test "$(rbd mirror snapshot schedule ls)" = 'every 1h starting at 00:15:00' + expect_fail rbd mirror snapshot schedule add 30m 04:30 + expect_fail rbd mirror snapshot schedule add 30m 04:30+05:30 + expect_fail rbd mirror snapshot schedule add 30m 2020-13-14T04:30+05:30 + expect_fail rbd mirror snapshot schedule add 30m 2020-01-32T04:30+05:30 + expect_fail rbd mirror snapshot schedule add 30m 2020-01-14T25:30+05:30 + expect_fail rbd mirror snapshot schedule add 30m 2020-01-14T04:60+05:30 + expect_fail rbd mirror snapshot schedule add 30m 2020-01-14T04:30+24:00 + test "$(rbd mirror snapshot schedule ls)" = 'every 1h starting at 2020-01-13 23:00:00' test "$(rbd mirror snapshot schedule ls -p rbd2/ns1 --image test1)" = 'every 1m' rbd rm rbd2/ns1/test1 diff --git a/src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py b/src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py index 665fe1656c1..02e2b7882eb 100644 --- a/src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py +++ b/src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py @@ -4,7 +4,7 @@ import rados import rbd import traceback -from datetime import datetime +from datetime import datetime, timezone from threading import Condition, Lock, Thread from typing import Any, Dict, List, NamedTuple, Optional, Set, Tuple, Union @@ -338,7 +338,7 @@ class MirrorSnapshotScheduleHandler: self.condition = Condition(self.lock) self.module = module self.log = module.log - self.last_refresh_images = datetime(1970, 1, 1) + self.last_refresh_images = datetime(1970, 1, 1, tzinfo=timezone.utc) self.create_snapshot_requests = CreateSnapshotRequests(self) self.stop_thread = False @@ -370,7 +370,7 @@ class MirrorSnapshotScheduleHandler: pool_id, namespace, image_id = image_spec self.create_snapshot_requests.add(pool_id, namespace, image_id) with self.lock: - self.enqueue(datetime.now(), pool_id, namespace, image_id) + self.enqueue(datetime.now(timezone.utc), pool_id, namespace, image_id) except (rados.ConnectionShutdown, rbd.ConnectionShutdown): self.log.exception("MirrorSnapshotScheduleHandler: client blocklisted") @@ -381,7 +381,7 @@ class MirrorSnapshotScheduleHandler: def init_schedule_queue(self) -> None: # schedule_time => image_spec - self.queue: Dict[str, List[ImageSpec]] = {} + self.queue: Dict[datetime, List[ImageSpec]] = {} # pool_id => {namespace => image_id} self.images: Dict[str, Dict[str, Dict[str, str]]] = {} self.schedules = Schedules(self) @@ -393,7 +393,7 @@ class MirrorSnapshotScheduleHandler: self.schedules.load(namespace_validator, image_validator) def refresh_images(self) -> float: - elapsed = (datetime.now() - self.last_refresh_images).total_seconds() + elapsed = (datetime.now(timezone.utc) - self.last_refresh_images).total_seconds() if elapsed < self.REFRESH_DELAY_SECONDS: return self.REFRESH_DELAY_SECONDS - elapsed @@ -405,7 +405,7 @@ class MirrorSnapshotScheduleHandler: self.log.debug("MirrorSnapshotScheduleHandler: no schedules") self.images = {} self.queue = {} - self.last_refresh_images = datetime.now() + self.last_refresh_images = datetime.now(timezone.utc) return self.REFRESH_DELAY_SECONDS images: Dict[str, Dict[str, Dict[str, str]]] = {} @@ -421,7 +421,7 @@ class MirrorSnapshotScheduleHandler: self.refresh_queue(images) self.images = images - self.last_refresh_images = datetime.now() + self.last_refresh_images = datetime.now(timezone.utc) return self.REFRESH_DELAY_SECONDS def load_pool_images(self, @@ -473,13 +473,10 @@ class MirrorSnapshotScheduleHandler: pool_name, e)) def rebuild_queue(self) -> None: - now = datetime.now() - # don't remove from queue "due" images - now_string = datetime.strftime(now, "%Y-%m-%d %H:%M:00") - + now = datetime.now(timezone.utc) for schedule_time in list(self.queue): - if schedule_time > now_string: + if schedule_time > now: del self.queue[schedule_time] if not self.schedules: @@ -494,7 +491,7 @@ class MirrorSnapshotScheduleHandler: def refresh_queue(self, current_images: Dict[str, Dict[str, Dict[str, str]]]) -> None: - now = datetime.now() + now = datetime.now(timezone.utc) for pool_id in self.images: for namespace in self.images[pool_id]: @@ -536,13 +533,11 @@ class MirrorSnapshotScheduleHandler: if not self.queue: return None, 1000.0 - now = datetime.now() - schedule_time = sorted(self.queue)[0] + now = datetime.now(timezone.utc) + schedule_time = min(self.queue) - if datetime.strftime(now, "%Y-%m-%d %H:%M:%S") < schedule_time: - wait_time = (datetime.strptime(schedule_time, - "%Y-%m-%d %H:%M:%S") - now) - return None, wait_time.total_seconds() + if now < schedule_time: + return None, (schedule_time - now).total_seconds() images = self.queue[schedule_time] image = images.pop(0) @@ -616,7 +611,7 @@ class MirrorSnapshotScheduleHandler: continue image_name = self.images[pool_id][namespace][image_id] scheduled_images.append({ - 'schedule_time': schedule_time, + 'schedule_time': schedule_time.strftime("%Y-%m-%d %H:%M:00"), 'image': image_name }) return 0, json.dumps({'scheduled_images': scheduled_images}, diff --git a/src/pybind/mgr/rbd_support/schedule.py b/src/pybind/mgr/rbd_support/schedule.py index 4968700ad82..173ef7e6d5e 100644 --- a/src/pybind/mgr/rbd_support/schedule.py +++ b/src/pybind/mgr/rbd_support/schedule.py @@ -1,11 +1,11 @@ -import datetime import json import rados import rbd import re -from dateutil.parser import parse -from typing import cast, Any, Callable, Dict, List, Optional, Set, Tuple, TYPE_CHECKING +from datetime import date, datetime, timezone, timedelta +from dateutil.parser import parse, isoparse +from typing import Any, Callable, Dict, List, Optional, Set, Tuple, TYPE_CHECKING from .common import get_rbd_pools if TYPE_CHECKING: @@ -269,36 +269,45 @@ class Interval: class StartTime: - def __init__(self, - hour: int, - minute: int, - tzinfo: Optional[datetime.tzinfo]) -> None: - self.time = datetime.time(hour, minute, tzinfo=tzinfo) - self.minutes = self.time.hour * 60 + self.time.minute - if self.time.tzinfo: - utcoffset = cast(datetime.timedelta, self.time.utcoffset()) - self.minutes += int(utcoffset.seconds / 60) - - def __eq__(self, start_time: Any) -> bool: - return self.minutes == start_time.minutes + def __init__(self, dt: datetime) -> None: + self.dt = self._to_utc(dt) + + @staticmethod + def _to_utc(dt: datetime) -> datetime: + if dt.tzinfo is None: + return dt.replace(tzinfo=timezone.utc, second=0, microsecond=0) + return dt.astimezone(timezone.utc).replace(second=0, microsecond=0) + + def __eq__(self, other: Any) -> bool: + return self.dt == other.dt def __hash__(self) -> int: - return hash(self.minutes) + return hash(self.dt) def to_string(self) -> str: - return self.time.isoformat() + return self.dt.strftime("%Y-%m-%d %H:%M:00") @classmethod - def from_string(cls, start_time: Optional[str]) -> Optional['StartTime']: + def from_string(cls, + start_time: Optional[str], + allow_legacy: bool = False) -> Optional['StartTime']: if not start_time: return None try: - t = parse(start_time).timetz() + dt = isoparse(start_time) except ValueError as e: - raise ValueError("Invalid start time {}: {}".format(start_time, e)) + if not allow_legacy: + raise ValueError("Invalid start time {}: {}".format(start_time, e)) + + try: + t = parse(start_time).timetz() + except ValueError as e: + raise ValueError("Invalid legacy start time {}: {}".format(start_time, e)) + + dt = datetime.combine(date(1970, 1, 1), t) - return StartTime(t.hour, t.minute, tzinfo=t.tzinfo) + return cls(dt) class Schedule: @@ -320,33 +329,35 @@ class Schedule: start_time: Optional[StartTime] = None) -> None: self.items.discard((interval, start_time)) - def next_run(self, now: datetime.datetime) -> str: + def next_run(self, now: datetime) -> datetime: schedule_time = None - for interval, opt_start in self.items: - period = datetime.timedelta(minutes=interval.minutes) - start_time = datetime.datetime(1970, 1, 1) - if opt_start: - start = cast(StartTime, opt_start) - start_time += datetime.timedelta(minutes=start.minutes) - time = start_time + \ - (int((now - start_time) / period) + 1) * period - if schedule_time is None or time < schedule_time: - schedule_time = time + + for interval, start_time in self.items: + period = timedelta(minutes=interval.minutes) + anchor_time = start_time.dt if start_time else datetime(1970, 1, 1, tzinfo=timezone.utc) + + if anchor_time > now: + candidate_time = anchor_time + else: + q, r = divmod(now - anchor_time, period) + candidate_time = anchor_time + (q + bool(r)) * period + + if schedule_time is None or candidate_time < schedule_time: + schedule_time = candidate_time + if schedule_time is None: raise ValueError('no items is added') - return datetime.datetime.strftime(schedule_time, "%Y-%m-%d %H:%M:00") + + return schedule_time def to_list(self) -> List[Dict[str, Optional[str]]]: - def item_to_dict(interval: Interval, - start_time: Optional[StartTime]) -> Dict[str, Optional[str]]: - if start_time: - schedule_start_time: Optional[str] = start_time.to_string() - else: - schedule_start_time = None - return {SCHEDULE_INTERVAL: interval.to_string(), - SCHEDULE_START_TIME: schedule_start_time} - return [item_to_dict(interval, start_time) - for interval, start_time in self.items] + return [ + { + SCHEDULE_INTERVAL: interval.to_string(), + SCHEDULE_START_TIME: start_time.to_string() if start_time else None + } + for interval, start_time in self.items + ] def to_json(self) -> str: return json.dumps(self.to_list(), indent=4, sort_keys=True) @@ -358,8 +369,9 @@ class Schedule: schedule = Schedule(name) for item in items: interval = Interval.from_string(item[SCHEDULE_INTERVAL]) - start_time = item[SCHEDULE_START_TIME] and \ - StartTime.from_string(item[SCHEDULE_START_TIME]) or None + # Allow loading 'start_time' values in legacy format for backwards compatibility + start_time = StartTime.from_string( + item.get(SCHEDULE_START_TIME), allow_legacy=True) schedule.add(interval, start_time) return schedule except json.JSONDecodeError as e: diff --git a/src/pybind/mgr/rbd_support/trash_purge_schedule.py b/src/pybind/mgr/rbd_support/trash_purge_schedule.py index abc50ec394f..b9774d18e3d 100644 --- a/src/pybind/mgr/rbd_support/trash_purge_schedule.py +++ b/src/pybind/mgr/rbd_support/trash_purge_schedule.py @@ -3,7 +3,7 @@ import rados import rbd import traceback -from datetime import datetime +from datetime import datetime, timezone from threading import Condition, Lock, Thread from typing import Any, Dict, List, Optional, Tuple @@ -21,7 +21,7 @@ class TrashPurgeScheduleHandler: self.condition = Condition(self.lock) self.module = module self.log = module.log - self.last_refresh_pools = datetime(1970, 1, 1) + self.last_refresh_pools = datetime(1970, 1, 1, tzinfo=timezone.utc) self.stop_thread = False self.thread = Thread(target=self.run) @@ -51,7 +51,7 @@ class TrashPurgeScheduleHandler: pool_id, namespace = ns_spec self.trash_purge(pool_id, namespace) with self.lock: - self.enqueue(datetime.now(), pool_id, namespace) + self.enqueue(datetime.now(timezone.utc), pool_id, namespace) except (rados.ConnectionShutdown, rbd.ConnectionShutdown): self.log.exception("TrashPurgeScheduleHandler: client blocklisted") @@ -64,7 +64,7 @@ class TrashPurgeScheduleHandler: try: with self.module.rados.open_ioctx2(int(pool_id)) as ioctx: ioctx.set_namespace(namespace) - rbd.RBD().trash_purge(ioctx, datetime.now()) + rbd.RBD().trash_purge(ioctx, datetime.now(timezone.utc)) except (rados.ConnectionShutdown, rbd.ConnectionShutdown): raise except Exception as e: @@ -72,7 +72,7 @@ class TrashPurgeScheduleHandler: pool_id, namespace, e)) def init_schedule_queue(self) -> None: - self.queue: Dict[str, List[Tuple[str, str]]] = {} + self.queue: Dict[datetime, List[Tuple[str, str]]] = {} # pool_id => {namespace => pool_name} self.pools: Dict[str, Dict[str, str]] = {} self.schedules = Schedules(self) @@ -84,7 +84,7 @@ class TrashPurgeScheduleHandler: self.schedules.load() def refresh_pools(self) -> float: - elapsed = (datetime.now() - self.last_refresh_pools).total_seconds() + elapsed = (datetime.now(timezone.utc) - self.last_refresh_pools).total_seconds() if elapsed < self.REFRESH_DELAY_SECONDS: return self.REFRESH_DELAY_SECONDS - elapsed @@ -96,7 +96,7 @@ class TrashPurgeScheduleHandler: self.log.debug("TrashPurgeScheduleHandler: no schedules") self.pools = {} self.queue = {} - self.last_refresh_pools = datetime.now() + self.last_refresh_pools = datetime.now(timezone.utc) return self.REFRESH_DELAY_SECONDS pools: Dict[str, Dict[str, str]] = {} @@ -112,7 +112,7 @@ class TrashPurgeScheduleHandler: self.refresh_queue(pools) self.pools = pools - self.last_refresh_pools = datetime.now() + self.last_refresh_pools = datetime.now(timezone.utc) return self.REFRESH_DELAY_SECONDS def load_pool(self, ioctx: rados.Ioctx, pools: Dict[str, Dict[str, str]]) -> None: @@ -137,13 +137,10 @@ class TrashPurgeScheduleHandler: pools[pool_id][namespace] = pool_name def rebuild_queue(self) -> None: - now = datetime.now() - # don't remove from queue "due" images - now_string = datetime.strftime(now, "%Y-%m-%d %H:%M:00") - + now = datetime.now(timezone.utc) for schedule_time in list(self.queue): - if schedule_time > now_string: + if schedule_time > now: del self.queue[schedule_time] if not self.schedules: @@ -156,7 +153,7 @@ class TrashPurgeScheduleHandler: self.condition.notify() def refresh_queue(self, current_pools: Dict[str, Dict[str, str]]) -> None: - now = datetime.now() + now = datetime.now(timezone.utc) for pool_id, namespaces in self.pools.items(): for namespace in namespaces: @@ -194,13 +191,11 @@ class TrashPurgeScheduleHandler: if not self.queue: return None, 1000.0 - now = datetime.now() - schedule_time = sorted(self.queue)[0] + now = datetime.now(timezone.utc) + schedule_time = min(self.queue) - if datetime.strftime(now, "%Y-%m-%d %H:%M:%S") < schedule_time: - wait_time = (datetime.strptime(schedule_time, - "%Y-%m-%d %H:%M:%S") - now) - return None, wait_time.total_seconds() + if now < schedule_time: + return None, (schedule_time - now).total_seconds() namespaces = self.queue[schedule_time] namespace = namespaces.pop(0) @@ -273,7 +268,7 @@ class TrashPurgeScheduleHandler: continue pool_name = self.pools[pool_id][namespace] scheduled.append({ - 'schedule_time': schedule_time, + 'schedule_time': schedule_time.strftime("%Y-%m-%d %H:%M:00"), 'pool_id': pool_id, 'pool_name': pool_name, 'namespace': namespace