John Mulligan [Fri, 25 Apr 2025 15:22:26 +0000 (11:22 -0400)]
mgr/dashboard: add an option to control the dashboard crypto caller
Add a mgr config option `crypto_caller` that lets a ceph user override
the default behavior of using the remote crypto caller. Supported
values are `internal` and `remote`.
John Mulligan [Fri, 25 Apr 2025 15:06:41 +0000 (11:06 -0400)]
mgr/cephadm: always use the internal cryptocaller
The cephadm modules needs to use python cryptography module for ssh (via
asyncssh) and thus there's no need to use the remote crypto caller in
cephadm. Configure cephadm to always use the internal cryptocaller.
John Mulligan [Fri, 25 Apr 2025 15:05:46 +0000 (11:05 -0400)]
python-common/cryptotools: catch all failures to read cert
Previously, the internal crypto caller would catch (and convert) some
errors when reading the cert but not all cases. Move the logic to catch
the errors to a common location and do it once consistently.
John Mulligan [Thu, 24 Apr 2025 18:36:58 +0000 (14:36 -0400)]
python-common/cryptotools: unify and organize all endpoint functions
Lightly reorganize and make the "endpoint" functions in cryptotools.py more
consistent and uniform. Use small functions for input and output
handling so that the handling is done the same way throughout. Pass a
pre-constructed crypto caller via the args to then endpoint functions.
Make generating the private key it's own named function rather than
one single (and only) function with overloaded behavior controlled by
a cli switch.
John Mulligan [Wed, 23 Apr 2025 15:23:43 +0000 (11:23 -0400)]
pybind/mgr: fix test case in test_tls.py
Why violate the typing in a test? mypy never noticed this because tests
are not type checked but there seems to be no need to turn a str into
bytes to pass to a function that is typed only as taking str!
John Mulligan [Wed, 23 Apr 2025 15:25:07 +0000 (11:25 -0400)]
python-common/cryptotools: fix error path in verify tls function
The remote verify_tls function was not raising errors when it should.
Fix the function so that it always returns an object when it succeeds or
fails gracefully. Always parse that function in the crypto caller class.
John Mulligan [Wed, 16 Apr 2025 18:56:28 +0000 (14:56 -0400)]
python-common/cryptotools: create CrytpoCaller interface class
Create a class to act as a common shim between the cryptotools external
functions and the mgr. It provides common conversion mechanisms and
could possibly act as an abstraction in case we decide to make
the external function calls in different ways in the future.
Paulo E. Castro [Sat, 5 Apr 2025 20:47:55 +0000 (21:47 +0100)]
pybind/mgr: Hack around the 'ImportError: PyO3 modules may only be initialized once per interpreter process' issue.
Fixes: https://tracker.ceph.com/issues/64213 Signed-off-by: Paulo E. Castro <pecastro@wormholenet.com>
(cherry picked from commit 717d0a6f3530ad3e07f4423002810327b2addcf1)
Kefu Chai [Wed, 24 Dec 2025 05:55:26 +0000 (13:55 +0800)]
debian/control: add iproute2 to build dependencies
Test scripts like qa/tasks/cephfs/mount.py expect the ip command to be
available in the container environment. Without it, tests fail with:
```
/bin/bash: line 1: ip: command not found
File "/ceph/qa/tasks/cephfs/mount.py", line 96, in cleanup_stale_netnses_and_bridge
p = remote.run(args=['ip', 'netns', 'list'],
...
teuthology.exceptions.CommandFailedError: Command failed with status 127: 'ip netns list'
```
Add iproute2 to the debian package build dependencies when the
<pkg.ceph.check> build profile is enabled. This ensures the package is
available during container-based builds, since buildcontainer-setup.sh
→ script/run-make.sh → install-deps.sh → debian/control → generated
dependency package chain respects build profiles configured via
`FOR_MAKE_CHECK` and `WITH_CRIMSON` environment variables set in
Dockerfile.build.
David Galloway [Tue, 16 Dec 2025 22:08:00 +0000 (17:08 -0500)]
install-deps: Replace apt-mirror
apt-mirror.front.sepia.ceph.com has happened to always work because we set up CNAMEs to gitbuilder.ceph.com.
That host is making its way to a new home upstate (literally and figuratively) so we'll get rid of the front subdomain since it's publicly accessible anyway and add TLS while we're at it.
Thomas Lamprecht [Wed, 31 Jul 2024 07:48:08 +0000 (09:48 +0200)]
debian/control: record python3-packaging dependency for ceph-volume
Commit 0985e201 added `from packaging import version` to
ceph_volume/util/encryption.py. On Debian and its derivatives, the
packaging module is provided by the python3-packaging package.
Add python3-packaging to ceph-volume's runtime dependencies to ensure
this import is satisfied.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com> Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
(cherry picked from commit a6ab6c63853d071f7383b0ce51d278de92fc3960)
Afreen Misbah [Tue, 21 Oct 2025 16:37:46 +0000 (22:07 +0530)]
mgr/dashboard: Carbonize the Change Password Form
Fixes https://tracker.ceph.com/issues/73193
- using carbon based stylings, typography and components
- used grid layout for form arrangement
- breadcrumb is slightly off, which needs to be fixed by applying grid layout to the app shell
Nitzan Mordechai [Wed, 22 Oct 2025 05:41:56 +0000 (05:41 +0000)]
tasks/cbt_performance: Tolerate exceptions during performance data updates
If an exception occurs during the POST request to update CBT performance,
log the error instead of failing the entire job. This ensures that
intermittent update failures do not block the main workflow.
Afreen Misbah [Mon, 17 Nov 2025 05:01:45 +0000 (10:31 +0530)]
mgr/dashboard: Set max subsystem count to 512 rather than 4096
Fixes https://tracker.ceph.com/issues/73867
- regression from https://github.com/ceph/ceph/pull/64477/files
- removing frontend valdations as this values are volatiel and require changes every release. Nvmeof is seeting these and validating as well.
Kefu Chai [Tue, 21 Oct 2025 03:25:00 +0000 (11:25 +0800)]
debian: Use system packages for cephadm bundled dependencies
Configure the Debian build to use CEPHADM_BUNDLED_DEPENDENCIES=deb,
which instructs the cephadm build script to bundle dependencies from
system-installed Debian packages instead of downloading from PyPI.
This change addresses build failures in restricted network environments
where Debian build tools do not permit internet access. By leveraging
the Debian package support added in commit 9378a2988e1, the build now
uses python3-markupsafe, python3-jinja2, and python3-yaml packages
that are already installed as build dependencies.
This approach mirrors the existing RPM packaging workflow, ensuring
consistent behavior across different distribution package formats.
Kefu Chai [Tue, 21 Oct 2025 03:26:25 +0000 (11:26 +0800)]
cephadm/tests: Add tests for deb bundled dependencies
Add container definitions and test cases for building cephadm with
Debian package dependencies. The new test_cephadm_build_from_debs
function mirrors the existing RPM test structure, verifying that:
- Build succeeds when required Debian packages are installed
- Build fails when packages are missing
- Bundled packages are correctly identified as sourced from 'deb'
- All expected packages (Jinja2, MarkupSafe, PyYAML) are included
- The zipapp contains expected package directories
Test environments include Ubuntu 22.04 and 24.04 with and without
the required python3-jinja2, python3-yaml, and python3-markupsafe
packages.
Kefu Chai [Tue, 14 Oct 2025 13:04:42 +0000 (21:04 +0800)]
cephadm/build: Add Debian package support for bundled dependencies
Extends the cephadm build script to support bundling dependencies from
Debian packages in addition to pip and RPM packages. This allows building
cephadm on Debian-based distributions using system packages.
Key changes:
- Add 'deb' to DependencyMode enum to enable Debian package mode
- Implement _setup_deb() to configure Debian dependency requirements
- Add _install_deb_deps() to orchestrate Debian package installation
- Add _gather_deb_package_dirs() to parse Debian package file listings
and locate Python package directories (handles both site-packages and
dist-packages directories used by Debian)
- Add _deps_from_deb() to extract Python dependencies from installed
Debian packages using dpkg/apt-cache tools
- Fix variable reference bug in _install_deps() (deps.mode -> config.deps_mode)
The Debian implementation follows a similar pattern to the existing RPM
support, using dpkg-query and dpkg -L to locate installed packages and
their files, with special handling for Debian naming conventions
(e.g., PyYAML -> python3-yaml).
Kefu Chai [Mon, 10 Nov 2025 04:11:08 +0000 (12:11 +0800)]
cephadm: fix zip_root_entries population in version command
The 'cephadm version --verbose' command was returning an empty
zip_root_entries list because it relied on the private '_files'
attribute of zipimport.zipimporter, which is not reliably populated
across Python versions.
This commit fixes the issue by using the zipfile module to properly
read the archive contents via the loader.archive path. This ensures
that zip_root_entries is correctly populated with the root-level
directories in the zipapp.
This fix is necessary for the cephadm build tests to properly validate
that all expected packages and modules are included in the built zipapp.
Kefu Chai [Mon, 10 Nov 2025 04:10:46 +0000 (12:10 +0800)]
cephadm/tests: fix _dist_info function logic error
The _dist_info helper function had a logic error where it was checking
if 'entry.startswith(entry)' instead of 'entry.startswith(name)'. This
caused the function to always evaluate incorrectly when checking for
.dist-info or .egg-info entries in the zipapp.
This bug was preventing the test assertions from properly validating
that package metadata directories are included in the built cephadm
zipapp.
Kefu Chai [Wed, 15 Oct 2025 07:46:26 +0000 (15:46 +0800)]
debian/control: Add libxsimd-dev build dependency for vendored Arrow
In commit e8460cbd, we introduced the "pkg.ceph.arrow" build profile to
support building with system Arrow packages. However, neither Debian nor
Ubuntu currently ships Arrow packages.
Since WITH_RADOSGW_SELECT_PARQUET is always enabled in debian/rules,
Arrow support is required for all builds. When the pkg.ceph.arrow profile
is not selected, the build uses vendored Arrow. With the recent change to
use AUTO mode for xsimd detection, Arrow will attempt to find system xsimd
>= 9.0.1. Adding libxsimd-dev as a build dependency ensures it's available
for Arrow to detect and use, reducing build time on supported distributions.
On distributions with insufficient xsimd versions (< 9.0.1), Arrow will
automatically fall back to its bundled version.
Kefu Chai [Wed, 15 Oct 2025 07:46:22 +0000 (15:46 +0800)]
cmake/BuildArrow: Use AUTO mode for xsimd dependency detection
Arrow requires xsimd >= 9.0.1 according to arrow/cpp/thirdparty/versions.txt.
Previously, we unconditionally set -Dxsimd_SOURCE=BUNDLED, forcing the use
of Arrow's vendored xsimd regardless of system package availability.
This commit changes to -Dxsimd_SOURCE=AUTO, which allows Arrow's
resolve_dependency mechanism to automatically:
1. Try to find system xsimd package
2. Check if version >= 9.0.1
3. Use system version if found and sufficient
4. Fall back to bundled version otherwise
This reduces build time and dependencies on systems with sufficient xsimd,
while maintaining compatibility with older distributions.
Distribution availability:
- Ubuntu Noble (24.04): libxsimd-dev 12.1.1 (✓ will use system)
- Ubuntu Jammy (22.04): libxsimd-dev 7.6.0 (✗ will use bundled)
- Debian Trixie (13): libxsimd-dev 13.2.0 (✓ will use system)
- CentOS Stream 9: xsimd-devel 7.4.9 (✗ will use bundled)
projects / ceph.git / log