Patrick Donnelly [Mon, 23 Feb 2026 15:29:13 +0000 (10:29 -0500)]
Merge PR #67135 into main
* refs/pull/67135/head:
pybind: remove compile_time_env parameter from setup.py files
pybind/rados,rgw: replace Tempita errno checks with C preprocessor
pybind/cephfs: replace deprecated IF with C preprocessor macro
Reviewed-by: Ilya Dryomov <idryomov@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
cls/rgw_gc/cls_rgw_gc: read config via cls_get_config
Commit https://github.com/ceph/ceph/commit/3877c1e37f2fa4e1574b57f05132288f210835a7
added new way to let CLS gain access to global configuration (`g_ceph_context`).
`cls_rgw_gc_queue_init` method is not using the new CLS call of `cls_get_config`
but instead directly uses `g_ceph_context`.
Crimson OSD implementation does **not** support `g_ceph_context` which results in a (SIGSEGV)
crash due to null access. Switching to `cls_get_config`, similarly to `cls_rgw.cc`, would allow
both OSD implementations to access the conf safely.
The above approach is well-defined due to the two orthogonal implementations of objclass.cc.
Classical OSD uses `src/osd/objclass.cc` While Crimson OSD uses `src/crimson/osd/objclass.cc`.
mgr/DaemonServer: Re-order OSDs in crush bucket to maximize OSDs for upgrade
DaemonServer::_maximize_ok_to_upgrade_set() attempts to find which OSDs
from the initial set found as part of _populate_crush_bucket_osds() can be
upgraded as part of the initial phase. If the initial set results in failure,
the convergence logic trims the 'to_upgrade' vector from the end until a safe
set is found.
Therefore, it would be advantageous to sort the OSDs by the ascending number
of PGs hosted by the OSDs. By placing OSDs with smallest (or no PGs) at the
beginning of the vector, the trim logic along with _check_offlines_pgs() will
have the best chance of finding OSDs to upgrade as it approaches a grouping
of OSDs that have the smallest or no PGs.
To achieve the above, a temporary vector of struct pgs_per_osd is created and
sorted for a given crush bucket. The sorted OSDs are pushed to the main
crush_bucket_osds that is eventually used to run the _check_offlines_pgs()
logic to find a safe set of OSDs to upgrade.
pgmap is passed to _populate_crush_bucket_osds() to utilize get_num_pg_by_osd()
for the above logic to work.
Implement a new Mgr command called 'ok-to-upgrade' that returns a set of OSDs
within the provided CRUSH bucket that are safe to upgrade without reducing
immediate data availability.
The command accepts the following as input:
- CRUSH bucket name (required)
- The CRUSH bucket type is limited to 'rack', 'chassis', 'host' and 'osd'.
This is to prevent users from specifying a bucket type higher up the tree
which could result in performance issues if the number of OSDs in the
bucket is very high.
- The new Ceph version to check against. The format accepted is the short
form of the Ceph version, for e.g. 20.3.0-3803-g63ca1ffb5a2. (required)
- The maximum number of OSDs to consider if specified. (optional)
Implementation Details:
After sanity checks on the provided parameters, the following steps are
performed:
1. The set of OSDs within the CRUSH bucket is first determined.
2. From the main set of OSDs, a filtered set of OSDs not yet running the new
Ceph version is created.
- For this purpose, the OSD's 'ceph_version_short' string is read from
the metadata. For this purpose a new method called
DaemonServer::get_osd_metadata() is used. The information is determined
from the DaemonStatePtr maintained within the DaemonServer.
3. If all OSDs are already running the new Ceph version, a success report is
generated and returned.
4. If OSDs are not running the new Ceph version, a new set (to_upgrade) is
created.
5. If the current version cannot be determined, an error is logged and the
output report with 'bad_no_version' field populated with the OSD in question
is generated.
6. On the new set (to_upgrade), the existing logic in _check_offline_pgs() is
executed to see if stopping any or all OSDs in the set as part of the upgrade
can reduce immediate data availability.
- If data availability is impacted, then the number of OSDs in the filtered
set is reduced by a factor defined by a new config option called
'mgr_osd_upgrade_check_convergence_factor' which is set to 0.8 by default.
- The logic in _check_offline_pgs() is repeated for the new set.
- The above is repeated until a safe subset of OSDs that can be stopped for
upgrade is found. Each iteration reduces the number of OSDs to check by
the convergence factor mentioned above.
7. It must be noted that the default value of
'mgr_osd_upgrade_check_convergence_factor' is on the higher side in order to
help determine an optimal set of OSDs to upgrade. In other words, a higher
convergence factor would help maximize the number of OSDs to upgrade. In this
case, the number of iterations and therefore the time taken to determine the
OSDs to upgrade is proportional to the number of OSDs in the CRUSH bucket.
The converse is true if a lower convergence factor is used.
8. If the number of OSDs determined is lower than the 'max' specified, then an
additional loop is executed to determine if other children of the CRUSH
bucket can be added to the existing set.
9. Once a viable set is determined, an output report similar to the following is
generated:
A standalone test is introduced that exercises the logic for both replicated
and erasure-coded pools by manipulating the min_size for a pool and check for
upgradability. The tests also performs other basic sanity checks and error
conditions.
The output shown below is for a cluster running on a single node with 10 OSDs
and with replicated pool configuration:
mgr/DaemonServer: Modify offline_pg_report to handle set or vector types
The offline_pg_report structure to be used by both the 'ok-to-stop' and
'ok-to-upgrade' commands is modified to handle either std::set or std::vector
type containers. This is necessitated due to the differences in the way
both commands work. For the 'ok-to-upgrade' command logic to work optimally,
the items in the specified crush bucket including items found in the subtree
must be strictly ordered. The earlier std::set container re-orders the items
upon insertion by sorting the items which results in the offline pg check to
report sub-optimal results.
Therefore, the offline_pg_report struct is modified to use
std::variant<std::vector<int>, std::set<int>> as a ContainerType and handled
accordingly in dump() using std::visit(). This ensures backward compatibility
with the existing 'ok-to-stop' command while catering to the requirements of
the new 'ok-to-upgrade' command.
Afreen Misbah [Mon, 16 Feb 2026 13:57:24 +0000 (19:27 +0530)]
mgr/dashboard: Add health check panel
Fixes https://tracker.ceph.com/issues/74958
- adds helath check panel in overview dashboard
- updates tests
- refactors component as per modern Angular convention
- using onPush CDS in Overview component
- using view model pattern to aggregate data for rendering
Ramana Raja [Wed, 24 Dec 2025 10:24:50 +0000 (05:24 -0500)]
mgr/rbd_support: Fix "start-time" arg behavior
The "start-time" argument, optionally passed when adding or removing an
mirror image snapshot schedule or a trash purge schedule, does not
behave as intended. It is meant to schedule an initial operation at a
specific time of day in a given time zone. Instead, it offsets the
schedule’s anchor time. By default, the scheduler uses the UNIX epoch as
the anchor to calculate recurring schedule times, and "start-time"
simply shifts this anchor away from UTC, which can confuse users. For
example:
```
$ # current time
$ date --universal
Wed Dec 10 05:55:21 PM UTC 2025
$ rbd mirror snapshot schedule add -p data --image img1 1h 19:00Z
$ rbd mirror snapshot schedule ls -p data --image img1
every 15m starting at 19:00:00+00:00
```
A user might assume that the scheduler will run the first snapshot each
day at 19:00 UTC and then run snapshots every 15 minutes. Instead, the
scheduler runs the first snapshot at 18:00 UTC and then continues at the
configured interval:
```
$ rbd mirror snapshot schedule status -p data --image img1
SCHEDULE TIME IMAGE
2025-12-10 18:00:00 data/img1
```
Additionally, the "start-time" argument accepts a full ISO 8601
timestamp but silently ignores everything except hour, minute, and time
zone. Even time zone handling is incorrect: specifying "23:00-01:00"
with an interval of "1d" results in a snapshot taken once per day at
22:00 UTC rather than 00:00 UTC, because only utcoffset.seconds is used
while utcoffset.days is ignored.
Fix:
Similar to the handling of the "start" argument in the FS snap-schedule
manager module, require "start-time" to use an ISO 8601 date-time format
with a mandatory date component. Time and time zone are optional and
default to 00:00 and UTC respectively.
The "start-time" now defines the anchor time used to compute recurring
schedule times. The default anchor remains the UNIX epoch. Existing
on-disk schedules with legacy-format "start-time" values are updated to
include the date Jan 1, 1970.
The `snap schedule ls` output now displays "start-time" with date and
time in the format "%Y-%m-%d %H:%M:00". The display time is in UTC.
Fixes: https://tracker.ceph.com/issues/74192 Signed-off-by: Ramana Raja <rraja@redhat.com>
common/options: change osd_target_transaction_size from int to uint
Change osd_target_transaction_size from signed int to unsigned int to
match the return type of Transaction::get_num_opts() (ceph_le64).
This change:
- Eliminates compiler warnings when comparing signed/unsigned values
- Enables automatic size conversion (e.g., "4_K" → 4096) via y2c.py
for improved administrator usability
- Maintains type consistency throughout the codebase
Kefu Chai [Tue, 17 Feb 2026 11:41:32 +0000 (19:41 +0800)]
librbd/pwl: fix memory leaks in discard operations
Fix memory leak in librbd persistent write log (PWL) cache discard
operations by properly completing request objects.
ASan reported the following leaks in unittest_librbd:
Direct leak of 240 byte(s) in 1 object(s) allocated from:
#0 operator new(unsigned long)
#1 librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>::discard(...)
/ceph/src/librbd/cache/pwl/AbstractWriteLog.cc:935:5
#2 TestMockCacheReplicatedWriteLog_discard_Test::TestBody()
/ceph/src/test/librbd/cache/pwl/test_mock_ReplicatedWriteLog.cc:534:7
Plus multiple indirect leaks totaling 2,076 bytes through the
shared_ptr reference chain.
Root cause:
C_DiscardRequest objects were never deleted because their complete()
method was never called. The on_write_persist callback released the
BlockGuard cell but didn't call complete() to trigger self-deletion.
Write requests use WriteLogOperationSet which takes the request as
its on_finish callback, ensuring complete() is eventually called.
Discard requests don't use WriteLogOperationSet and must explicitly
call complete() in their on_write_persist callback.
Solution:
Call discard_req->complete(r) in the on_write_persist callback and
move cell release into finish_req() -- mirroring how C_WriteRequest
handles it. The complete() -> finish() -> finish_req() chain ensures
the cell is released after the user request is completed, preserving
the same ordering as write requests.
Test results:
- Before: 2,316 bytes leaked in 15 allocations
- After: 0 bytes leaked
- unittest_librbd discard tests pass successfully with ASan
os/Transaction: change get_num_ops() return type to uint64_t
Change Transaction::get_num_ops() to return uint64_t instead of int
to match the underlying data.ops type (ceph_le<__u64>) and eliminate
compiler warnings about signed/unsigned comparison.
Fixes warning in ECTransaction.cc:
```
/home/kefu/dev/ceph/src/osd/ECTransaction.cc: In constructor ‘ECTransaction::Generate::Generate(PGTransaction&, ceph::ErasureCodeInterfaceRef&, pg_t&, const ECUtil::stripe_info_t&, const std::map<hobject_t, ECUtil::shard_extent_map_t>&, std::map<hobject_t, ECUtil::shard_extent_map_t>*, shard_id_map<ceph::os::Transaction>&, const OSDMapRef&, const hobject_t&, PGTransaction::ObjectOperation&, ECTransaction::WritePlanObj&, DoutPrefixProvider*, pg_log_entry_t*)’:
/home/kefu/dev/ceph/src/osd/ECTransaction.cc:589:25: warning: comparison of integer expressions of different signedness: ‘int’ and ‘__gnu_cxx::__alloc_traits<std::allocator<unsigned int>, unsigned int>::value_type’ {aka ‘unsigned int’} [-Wsign-compare]
589 | if (t.get_num_ops() > old_transaction_counts[int(shard)] &&
```
Imran Imtiaz [Fri, 20 Feb 2026 10:57:15 +0000 (10:57 +0000)]
mgr/dashboard: add schedule_level to image API for pool/cluster snapshot schedule
Add optional schedule_level param (image|pool|cluster) to
PUT /api/block/image/{image_spec}. Removes more-specific schedules
before setting at the chosen level. Backward compatible when omitted.
Fixes: https://tracker.ceph.com/issues/75043 Assisted-by: Cursor AI Signed-off-by: Imran Imtiaz <imran.imtiaz@uk.ibm.com>
mon/OSDMonitor: remove unused crush rules after erasure code pools deleted
When erasure code pools are created, a corresponding Crush rule is concurrently added to the Crush map.
However, when these pools are subsequently deleted, the associated rule persists within the Crush map.
In the event that a pool is re-created with the same name, the rule already exists.
However, if any modifications were made to the rule prior to the pool's deletion,
the new pool will inherit these modifications.
The proposed solution involves the automatic deletion of the Crush rule when a pool is deleted,
but only if no other pools are utilizing that particular rule.
Github allows to add a instructions file to each repo
(.github/copilot-instructions.md) to improve the behavior
of Copilot Reviews and Agent.
These instructions can also be customized per path, filetype, etc.:
https://docs.github.com/en/copilot/how-tos/configure-custom-instructions/add-repository-instructions
This commit was authored through a Github Agent session: https://github.com/ceph/ceph/tasks/edeca07b-eabd-477c-917a-a18e72a0e2c2
This commit makes it log the http error with the code and the reason
in sessionservice_discover() and log the error code along with the
body in query() for 5xx responses.
node-proxy: encapsulate send logic in dedicated method
Move the "send data to mgr when inventory changed" logic from main()
into a dedicated method _try_send_update().
This flattens the reporter loop and keeps main() to a single call under
the lock.
- use warning for bad request in the API,
when thread is not alive and for retry failure,
- use error for OOB load failure,
- use info for backoff interval,
- use debug in send attempts and for member fetch
this commit fixes mypy errors by adding explicit types for get_path
and get_* getters methods, extending SystemBackend with
start/shutdown and declaring _ca_temp_file on NodeProxyManager
node-proxy: split out config, bootstrap and redfish logic
refactor config, bootstrap, redfish layer, and monitoring:
this:
- adds a config module (CephadmCofnig, load_cephadm_config and
get_node_proxy_config) and protocols for api/reporter.
- extracts redfish logic to redfish.py
- adds a vendor registry with entrypoints.
- simplifies main() and NodeProxyManager().
This commit renames CONFIG to DEFAULTS and add load_config() with
deep merge, refactor Config to use path + defaults and makes
node-proxy config path configurable via bootstrap JSON or env.
projects / ceph.git / log