N Balachandran [Mon, 5 Jan 2026 15:48:01 +0000 (21:18 +0530)]
rgw/bucket-logging: support for EC pools
Log buckets can now be created within erasure-coded (EC) pools.
To support append operations, a temporary log record object is initially
created in the replicated default.rgw.log pool. This object is then copied
to the EC pool upon log record commitment.
All implicit log commit operations will execute asynchronously. A new
BucketLoggingManager class is responsible for processing these pending
commits at set intervals. Explicit commit operations, however, will
continue to be performed synchronously.
Yuval Lifshitz [Mon, 3 Nov 2025 11:20:07 +0000 (11:20 +0000)]
rgw/logging: deleteting the object holding the temp object name on cleanup
* in case of prefix per source this would prevent leaking this object
* in case of share prefix, it would prevent data loss when other source
buckets will try to commit an already comitted temporary object
* when updatign the "last committed" attribute, the object must exist.
this is so that commit without rollover (in case of cleanup) won't
recreate the deleted object
* some refactoring of try-catch code to have less nesting
Yuval Lifshitz [Thu, 30 Oct 2025 09:45:48 +0000 (09:45 +0000)]
rgw/logging: make sure source bucket is in the target's list
if a target log bucket is deleted, and then recreated with the same name,
the attribute holding the sources will not exist.
this is fixing it, but updating the list anytime we try to flush
or add a record.
if the source is already in the list, there will be no update to the attribute.
Yuval Lifshitz [Sun, 12 Oct 2025 14:14:36 +0000 (14:14 +0000)]
rgw/logging: fix race condition when name update returns ECANCELED
* when we get ECANCELED indication from the name set operation we should
bail out and not continue with the rollover
* this fix revealed a hidden bug where we do not check the existing temp
name when we do conf change cleanup (rollover)
rgw/logging: add error message when log_record fails
when log_record fails in journal mode due to issues in the target
bucket, the result code that the client get will be confusing, since
there is no indication that the issue is wit hte target bucket and not
the source bucket on which the client was operating.
the HTTP error message will be used to convey this information.
Kefu Chai [Wed, 24 Dec 2025 05:55:26 +0000 (13:55 +0800)]
debian/control: add iproute2 to build dependencies
Test scripts like qa/tasks/cephfs/mount.py expect the ip command to be
available in the container environment. Without it, tests fail with:
```
/bin/bash: line 1: ip: command not found
File "/ceph/qa/tasks/cephfs/mount.py", line 96, in cleanup_stale_netnses_and_bridge
p = remote.run(args=['ip', 'netns', 'list'],
...
teuthology.exceptions.CommandFailedError: Command failed with status 127: 'ip netns list'
```
Add iproute2 to the debian package build dependencies when the
<pkg.ceph.check> build profile is enabled. This ensures the package is
available during container-based builds, since buildcontainer-setup.sh
→ script/run-make.sh → install-deps.sh → debian/control → generated
dependency package chain respects build profiles configured via
`FOR_MAKE_CHECK` and `WITH_CRIMSON` environment variables set in
Dockerfile.build.
David Galloway [Tue, 16 Dec 2025 22:08:00 +0000 (17:08 -0500)]
install-deps: Replace apt-mirror
apt-mirror.front.sepia.ceph.com has happened to always work because we set up CNAMEs to gitbuilder.ceph.com.
That host is making its way to a new home upstate (literally and figuratively) so we'll get rid of the front subdomain since it's publicly accessible anyway and add TLS while we're at it.
Thomas Lamprecht [Wed, 31 Jul 2024 07:48:08 +0000 (09:48 +0200)]
debian/control: record python3-packaging dependency for ceph-volume
Commit 0985e201 added `from packaging import version` to
ceph_volume/util/encryption.py. On Debian and its derivatives, the
packaging module is provided by the python3-packaging package.
Add python3-packaging to ceph-volume's runtime dependencies to ensure
this import is satisfied.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com> Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
(cherry picked from commit a6ab6c63853d071f7383b0ce51d278de92fc3960)
Afreen Misbah [Tue, 21 Oct 2025 16:37:46 +0000 (22:07 +0530)]
mgr/dashboard: Carbonize the Change Password Form
Fixes https://tracker.ceph.com/issues/73193
- using carbon based stylings, typography and components
- used grid layout for form arrangement
- breadcrumb is slightly off, which needs to be fixed by applying grid layout to the app shell
Nitzan Mordechai [Wed, 22 Oct 2025 05:41:56 +0000 (05:41 +0000)]
tasks/cbt_performance: Tolerate exceptions during performance data updates
If an exception occurs during the POST request to update CBT performance,
log the error instead of failing the entire job. This ensures that
intermittent update failures do not block the main workflow.
Afreen Misbah [Mon, 17 Nov 2025 05:01:45 +0000 (10:31 +0530)]
mgr/dashboard: Set max subsystem count to 512 rather than 4096
Fixes https://tracker.ceph.com/issues/73867
- regression from https://github.com/ceph/ceph/pull/64477/files
- removing frontend valdations as this values are volatiel and require changes every release. Nvmeof is seeting these and validating as well.
Kefu Chai [Tue, 21 Oct 2025 03:25:00 +0000 (11:25 +0800)]
debian: Use system packages for cephadm bundled dependencies
Configure the Debian build to use CEPHADM_BUNDLED_DEPENDENCIES=deb,
which instructs the cephadm build script to bundle dependencies from
system-installed Debian packages instead of downloading from PyPI.
This change addresses build failures in restricted network environments
where Debian build tools do not permit internet access. By leveraging
the Debian package support added in commit 9378a2988e1, the build now
uses python3-markupsafe, python3-jinja2, and python3-yaml packages
that are already installed as build dependencies.
This approach mirrors the existing RPM packaging workflow, ensuring
consistent behavior across different distribution package formats.
Kefu Chai [Tue, 21 Oct 2025 03:26:25 +0000 (11:26 +0800)]
cephadm/tests: Add tests for deb bundled dependencies
Add container definitions and test cases for building cephadm with
Debian package dependencies. The new test_cephadm_build_from_debs
function mirrors the existing RPM test structure, verifying that:
- Build succeeds when required Debian packages are installed
- Build fails when packages are missing
- Bundled packages are correctly identified as sourced from 'deb'
- All expected packages (Jinja2, MarkupSafe, PyYAML) are included
- The zipapp contains expected package directories
Test environments include Ubuntu 22.04 and 24.04 with and without
the required python3-jinja2, python3-yaml, and python3-markupsafe
packages.
Kefu Chai [Tue, 14 Oct 2025 13:04:42 +0000 (21:04 +0800)]
cephadm/build: Add Debian package support for bundled dependencies
Extends the cephadm build script to support bundling dependencies from
Debian packages in addition to pip and RPM packages. This allows building
cephadm on Debian-based distributions using system packages.
Key changes:
- Add 'deb' to DependencyMode enum to enable Debian package mode
- Implement _setup_deb() to configure Debian dependency requirements
- Add _install_deb_deps() to orchestrate Debian package installation
- Add _gather_deb_package_dirs() to parse Debian package file listings
and locate Python package directories (handles both site-packages and
dist-packages directories used by Debian)
- Add _deps_from_deb() to extract Python dependencies from installed
Debian packages using dpkg/apt-cache tools
- Fix variable reference bug in _install_deps() (deps.mode -> config.deps_mode)
The Debian implementation follows a similar pattern to the existing RPM
support, using dpkg-query and dpkg -L to locate installed packages and
their files, with special handling for Debian naming conventions
(e.g., PyYAML -> python3-yaml).
Kefu Chai [Mon, 10 Nov 2025 04:11:08 +0000 (12:11 +0800)]
cephadm: fix zip_root_entries population in version command
The 'cephadm version --verbose' command was returning an empty
zip_root_entries list because it relied on the private '_files'
attribute of zipimport.zipimporter, which is not reliably populated
across Python versions.
This commit fixes the issue by using the zipfile module to properly
read the archive contents via the loader.archive path. This ensures
that zip_root_entries is correctly populated with the root-level
directories in the zipapp.
This fix is necessary for the cephadm build tests to properly validate
that all expected packages and modules are included in the built zipapp.
Kefu Chai [Mon, 10 Nov 2025 04:10:46 +0000 (12:10 +0800)]
cephadm/tests: fix _dist_info function logic error
The _dist_info helper function had a logic error where it was checking
if 'entry.startswith(entry)' instead of 'entry.startswith(name)'. This
caused the function to always evaluate incorrectly when checking for
.dist-info or .egg-info entries in the zipapp.
This bug was preventing the test assertions from properly validating
that package metadata directories are included in the built cephadm
zipapp.
Kefu Chai [Wed, 15 Oct 2025 07:46:26 +0000 (15:46 +0800)]
debian/control: Add libxsimd-dev build dependency for vendored Arrow
In commit e8460cbd, we introduced the "pkg.ceph.arrow" build profile to
support building with system Arrow packages. However, neither Debian nor
Ubuntu currently ships Arrow packages.
Since WITH_RADOSGW_SELECT_PARQUET is always enabled in debian/rules,
Arrow support is required for all builds. When the pkg.ceph.arrow profile
is not selected, the build uses vendored Arrow. With the recent change to
use AUTO mode for xsimd detection, Arrow will attempt to find system xsimd
>= 9.0.1. Adding libxsimd-dev as a build dependency ensures it's available
for Arrow to detect and use, reducing build time on supported distributions.
On distributions with insufficient xsimd versions (< 9.0.1), Arrow will
automatically fall back to its bundled version.
Kefu Chai [Wed, 15 Oct 2025 07:46:22 +0000 (15:46 +0800)]
cmake/BuildArrow: Use AUTO mode for xsimd dependency detection
Arrow requires xsimd >= 9.0.1 according to arrow/cpp/thirdparty/versions.txt.
Previously, we unconditionally set -Dxsimd_SOURCE=BUNDLED, forcing the use
of Arrow's vendored xsimd regardless of system package availability.
This commit changes to -Dxsimd_SOURCE=AUTO, which allows Arrow's
resolve_dependency mechanism to automatically:
1. Try to find system xsimd package
2. Check if version >= 9.0.1
3. Use system version if found and sufficient
4. Fall back to bundled version otherwise
This reduces build time and dependencies on systems with sufficient xsimd,
while maintaining compatibility with older distributions.
Distribution availability:
- Ubuntu Noble (24.04): libxsimd-dev 12.1.1 (✓ will use system)
- Ubuntu Jammy (22.04): libxsimd-dev 7.6.0 (✗ will use bundled)
- Debian Trixie (13): libxsimd-dev 13.2.0 (✓ will use system)
- CentOS Stream 9: xsimd-devel 7.4.9 (✗ will use bundled)
projects / ceph-ci.git / log