Dan Mick [Wed, 13 Aug 2025 19:16:45 +0000 (12:16 -0700)]
pybind/mgr/dashboard/frontend: add NPM_CACHEDIR envvar, use in bwc
Add an optional NPM_CACHEDIR environment variable to serve as the
cache parameter for npm in the dashboard frontend build. The idea
is to allow it to persist across builds so that we decrease the load
on registry.npmjs.org, which has been throttling our requests when
using build-with-container.py, and also hopefully improve the time
of the frontend npm operations.
build-with-container.py also grows a --npm-cache-path option to allow
setting it for container builds and passing the envvar to the build.
Fixes: https://tracker.ceph.com/issues/72298 Signed-off-by: Dan Mick <dan.mick@redhat.com>
rgw/restore: Update expiry-date of restored copies
As per AWS spec (https://docs.aws.amazon.com/AmazonS3/latest/API/API_RestoreObject.html),
if a `restore-object` request is re-issued on already restored copy, server needs to
update restoration period relative to the current time. These changes handles the same.
Note: this applies to only temporary restored copies
Patrick Donnelly [Tue, 12 Aug 2025 18:43:43 +0000 (14:43 -0400)]
Merge PR #64821 into main
* refs/pull/64821/head:
.github: only run redmine-upkeep actions from ceph/ceph.git
script/redmine-upkeep: add transform to resolve merged issue
script/redmine-upkeep: set default filter name/priority
script/redmine-upkeep: raise exception for PUT failures
script/redmine-upkeep: finish transform after application
script/redmine-upkeep: indicate log location in comments
script/redmine-upkeep: check envvar to see if running as action
script/redmine-upkeep: bullet issue list
script/redmine-upkeep: add stronger note on upkeep-failed tag in failure message
script/redmine-upkeep: do not raise commennt if upkeep-failed already present
script/redmine-upkeep: correct filter out of upkeep-failed
.github/workflows: allow redmine-upkeep to write comments
Patrick Donnelly [Wed, 16 Jul 2025 18:28:59 +0000 (14:28 -0400)]
script/redmine-upkeep: add transform to resolve merged issue
Few things:
- Add priority to transforms. Largely this is to have the "merged"
transformation run first to update the "Merge Commit" field of the ticket
before any other transform intends to look at that field. This avoids
duplicating logic to set the Merge Commit field.
- Fix a bug where the github API cannot be trusted to indicate the Merge Commit
for a PR. When the branch is rename or changed, the github backend clearly
gets confused and gives the "HEAD" commit instead.
- Add new transform to resolve tickets that are merged to either Resolved or
Pending Backport status.
* Note: filters on TAGS cannot be combined. There is some restructuring to deal with that.
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
John Mulligan [Tue, 1 Jul 2025 23:08:02 +0000 (19:08 -0400)]
mgr/cephadm: support tls creds via uri in service spec
Support populating the cert data sent to the cephadm binary using
special `URI:` prefixed strings instead of putting the cert data itself
in the smb service spec. This avoids having an extra copy of the cert
floating around but still matches the behavior of other services where
cephadm writes the certs into files.
In the future we may be able to avoid even putting the data in here as
sambacc can use rados apis - but for simplicity and matching other
services we will send the data this way for now.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 1 Jul 2025 21:13:34 +0000 (17:13 -0400)]
mgr/smb: remove unused generate_smb_service_spec function
Remove the generate_smb_service_spec as it was unused (it was used
early on for a debug dump command). Leaving it would have meant
keeping it up-to-date with the real spec generating function and
unnecessary extra work.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 2 Jul 2025 21:27:45 +0000 (17:27 -0400)]
mgr/smb: add tls credential support to staging
Add the necessary cross-check functions to staging.py for tls credential
resources. Add prune function for tls resources because they support
linked_to_cluster.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 1 Jul 2025 23:08:24 +0000 (19:08 -0400)]
mgr/smb: add a uri lookup func to the MonKeyConfigStore
Add a new lookup_uri function to the MonKeyConfigStore - this allows the
store to return an entry given a URI. The URIs in the stores are
typically used to communicate to components outside the mgr module. But
there are occasions that we have a uri and want to look it up instead
of using a key.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 1 Jul 2025 21:14:55 +0000 (17:14 -0400)]
mgr/smb: extend the mon key store to support raw data
Previously, a new protocol for raw data entries was added. Extend the
MonKeyStoreEntry to meet this protocol so that it can be used to retain
raw text/data blobs.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 1 Jul 2025 15:37:33 +0000 (11:37 -0400)]
mgr/smb: add support for tls credential resoruces to sqlite store
The sqlite store needs to be updated for each top-level resource type.
Update it for the newly added tls credential type. Configure it so that
it works similarly to the join auth resource such that the cert data is
not stored in the sqlite db - only in the layered mon store.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 1 Jul 2025 21:15:10 +0000 (17:15 -0400)]
mgr/smb: add a RawConfigEntry protocol type
Previously, the ConfigEntry type was created to be general interface for
serializing structures to JSON and persist them in a store.
However, there are times we want to retain data in a store that is not
serialized JSON - but just raw data. Create a new protocol for that
purpose.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 2 Jul 2025 22:15:16 +0000 (18:15 -0400)]
mgr/smb: add a remote control configuration to cluster resource
Add the remote control component to a field of the cluster resource.
A remote control side car can be explicitly enabled or implicitly
enabled by specifying the tls creds.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 2 Jul 2025 22:15:03 +0000 (18:15 -0400)]
mgr/smb: create a new resource component for remote control
Remote control is a new optional feature of the samba-container. It runs
a small "sidecar" service that provides a (g)RPC interface that bridges
certain runtime operations, like disconnecting a particular client,
from the cloudy-gRPC world to samba's existing tooling.
Add a new type that can be used to configure this sidecar service.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 2 Jul 2025 21:45:42 +0000 (17:45 -0400)]
mgr/smb: add a new tls source component resource
Add a new resource component for referencing a tls credential resource.
This works pretty much like the join auth resource in that a cluster
refers to a tls credential resource by name - avoiding having to have
potentially sensitive info stored in the cluster resource.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 2 Jul 2025 21:44:45 +0000 (17:44 -0400)]
mgr/smb: add a new tls credential resource type
Add a new TLS credential resource type that can currently store one
TLS certificate, TLS key, or TLS CA certificate. This is a new top-level
resource so that they can be use across clusters or managed
independently by different people and or private information can be kept
out a cluster resource stored in a file committed somewhere.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 2 Jul 2025 21:41:26 +0000 (17:41 -0400)]
mgr/smb: add a new BigString helper type for serializing yaml
Add a new BigString type that serves to help serialize resources to
YAML, causing it to use the multi-line literal style. A BigString is
a string so it should be transparent to other serializations, but
note that it should only be used in "simplified" structures.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 2 Jul 2025 21:42:40 +0000 (17:42 -0400)]
mgr/smb: use a private constant for hidden value indicator
When the password filter hides a value we replace it with a string
of asterisks - indicating that there was a value here. Move it
to a constant so it can be reused later.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 2 Jul 2025 22:06:30 +0000 (18:06 -0400)]
mgr/smb: add a new generalized enum for source references
The join auth and users and groups sources typically specify a
source as a reference to a (local) resource. Right now this is
really the only supported value. Since they're all basically the same
add a new generic SourceReferenceType enum for future source
resource components.
Note that python doesn't allow subclassing enums that have values
assigned. This is the reason for the copy paste here.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 2 Jul 2025 21:47:56 +0000 (17:47 -0400)]
mgr/smb: add a new TLSCredentialType enum
Add a new TLSCredentialType to enums.py. This will be used in the future
to support a new resource type that manages TLS/SSL credential objects (
certs, keys, etc).
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 30 Jun 2025 23:11:34 +0000 (19:11 -0400)]
mgr/cephadm: add a helper function for setting up smb config
Add a private helper function for copying values into the config_blobs
and similar dictionaries that are serialized and passed to cephadm for
configuration of smb. The helper automates the familiar pattern of
```
if value:
config_blobs[key] = value
```
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 9 Jun 2025 17:19:06 +0000 (13:19 -0400)]
python-common/deployment: add remote control sidecar config params
Add parameters to the smb service spec for configuring the remote
control sidecar. The feature always controls the deployment (or not) of
the sidecar and the ssl/tls config params will be used to set up (m)TLS
for the sidecar.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 9 Jun 2025 17:18:49 +0000 (13:18 -0400)]
cephadm/smb: add support for remote control sidecar
Add support for the remote control sidecar. The remote control server
is provided by the sambacc project. It is a small grpc server that
allows remote systems to get live information about connected clients
and disconnect clients or shares from the servers. The gRPC server
requires being run in mTLS mode to make changes.
Adding sidecar required adding a bunch of support for TLS/SSL certificate
management. I copied some of what other ceph containers do, like map
in a dedicated dir having certs. I expect some more stuff in the smb
daemon will require TLS support so I tried to make it a little more
generic than would be strictly needed by just this sidecar.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 16 Jul 2025 20:17:07 +0000 (16:17 -0400)]
cephadm: update type of populate_files first argument
The populate_files function's first argument was typed as `str` but
all places that str is used in that function can also use a
pathlib.Path. Update the type of that argument to match.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 30 Jun 2025 21:46:10 +0000 (17:46 -0400)]
mgr/smb: improve error handling when _customize_resource is wrong
While working on other changes I noticed that if _customize_resource
does not return a resource object (something fairly easy to do like
accidentally delete a line) the error it produces is somewhat hard
to debug. Check the return from the function returns a non-false
object to provide better error reporting.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 30 Jun 2025 21:18:36 +0000 (17:18 -0400)]
mgr/smb: add a wrapper_type attribute for resourcelib fields
Add a new wrapper_type attribute for resourcelib fields that will be
used for better YAML representations in a future change. The
wrapper_type attribute will be applied during the simplification
phase to prep the resulting object for serialization. It can be set
using the special _customize_resource function.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Update the build.sh script such that the environment variable
CUSTOM_CEPH_REPO_URL is passed to the container build command
as a build argument. This allows a developer or system outside of the
ceph CI to build container images similar to those produced by the
CI but using custom RPM packages.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a CUSTOM_CEPH_REPO_URL build argument to the Containerfile.
This configures the container to consume ceph packages from the repo
file indicated by this URL. This allows constructing a container image
"the ceph way" without requiring the use of CI built packages.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
ftbfs:
```
/home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/cls/fifo/detail/fifo.h:630:14: error: no member named 'parse' in namespace 'ceph'; did you mean 'pause'?
630 | auto n = ceph::parse<decltype(m.num)>(num);
| ^~~~~~~~~~~
```
Nizamudeen A [Fri, 8 Aug 2025 06:42:20 +0000 (12:12 +0530)]
mgr/dashboard: fix memory leak in prometheus service
Prometheus API calls in the Cluster Utilization call is subscribed in
the for loop multiple times but this is not properly unsubscribed. As we
stay in the dashboard page for longer time, it produces a significant
memory leak which eventually lags the UI. Attempting to fix it by
properly handling the subscription
Fixes: https://tracker.ceph.com/issues/72511 Signed-off-by: Nizamudeen A <nia@redhat.com>
crimson/os/seastore: mkfs_managers() into coroutines
* txn is now created only once and not in each repeated
iteration as before, with_repeat_trans_intr will reset the txn
if needed.
* in order for us to return a seastar::future, we would have to handle
each and every errorator. Instead, change the return value to
errorator and let the caller handle the error.
* The lambda which is passed to with_trans_intr returns mkfs_iertr
since this is an extended version of base_iertr.
Introduce a mkfs_ertr for the reason above. Future commits would
unify the new mkfs_ertr with seastore_mkfs_ertr.
projects / ceph.git / log