John Mulligan [Fri, 14 Feb 2025 19:51:03 +0000 (14:51 -0500)]
doc: document the new container build tool and link to it in README
Add a new markdown file in the root of the tree, ContainerBuild.md, that
can serve as a basic introduction to the new container build tools
recently merged to ceph.
Add a small 'breadcrumb' section to the project README.md to help find
this new document.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 20 Feb 2025 00:17:30 +0000 (19:17 -0500)]
script/build-with-container: add support for overlay dir
The source dir (aka homedir, default /ceph) is mounted in the container
read-write. This is needed as the various ceph build scripts expect to
write things into the tree - often this is in the build directory - but
not always. This can lead to small messes and/or situations that are
confusing to debug, especially if one is jumping between distros often.
Add an option to use an overlay volume for the homedir - by default we
enable a persistent overlay with a supplied "upper dir" where files that
were written will appear. One can also enable a temporary overlay that
forgets the writes when the container exits - maybe useful when doing
experiments in 'interactive' mode.
To use this option run the command with the `--overlay=<dir>` option.
For example: `./src/script/build-with-container.py -b build.inner
--overlay-dir build.ovr`. This will create a directory
`build.ovr/content` automatically and all new files will appear there.
For example the build directory will appear at
`build.ovr/content/build.inner`.
To use the temporary overlay use a `-` as the directory name. For
example: `./src/script/build-with-container.py -b build.inner
--overlay-dir -`
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 20 Feb 2025 14:50:49 +0000 (09:50 -0500)]
script/build-with-container: skip dnf cache dir volume mounts on docker
When using docker the --volume option is not available during build
(docker [buildx] build), unlike podman. Since passing these volumes must
be conditional on them being set up I see no way to handle this short of
just disabling the option on docker. Log the fact that it's being
skipped - the only other issue is that we pointlessly set up some dirs
and the build may be a bit slower.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 19 Feb 2025 18:20:36 +0000 (13:20 -0500)]
script/build-with-container: remove default --volume arg from ctr build
On the original github pr #59841 user fayak kindly informed us that the
--volume option was not supported by docker build. Since this section
was a leftover from a previous way of constructing the builder image and
was no longer needed we simply removed it.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 19 Feb 2025 18:20:01 +0000 (13:20 -0500)]
script/build-with-container.py: build builder image with --pull=always
Construct the builder image using the --pull=always flag to initiate a
pull of the base image (centos, ubuntu, etc) in order to avoid using a
stale base image. Since the script automatically (by default) avoids
building if a matching tag is in local container storage it is handy to
use a fresh base when it *is* time to build something. Otherwise, you
end up in a situation like I sometimes do - using a months old base
unintentionally.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Fri, 14 Feb 2025 19:50:42 +0000 (14:50 -0500)]
script/build-with-container: add a common packages target
Add a `packages` target to build-with-container.py that requests a build
of packages, whatever package type is native to the distro selected.
For example `./src/script/build-with-container.py -d ubuntu22.04 -e
packages` will automatically select a deb packages build where
`./src/script/build-with-container.py -d centos9 -e packages` will
trigger rpm packages to be built. The underlying package-type specific
targets remain unchanged.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Fri, 14 Feb 2025 16:44:35 +0000 (11:44 -0500)]
script/build-with-container: support custom tag suffixes
Previously, one could use the `--tag` option to completely override the
container tag generated by the script. However, there are cases where
one may want to add information to the tag rather than override it.
Allow the tag value to start with a plus (+) character that indicates
that the remainder of the string is to be suffixed to the generated tag.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a command line option --base-branch that allows the user to supply a
custom base branch name. git doesn't make determining this easy so we
always assume a base branch of 'main' by default - but this option lets
one change that.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Fri, 14 Feb 2025 16:24:29 +0000 (11:24 -0500)]
src/script: rename CEPH_BRANCH to CEPH_BASE_BRANCH for build container
Previously, we were passing build argument of CEPH_BRANCH, but that was
a bit misleading as we expect the current branch to vary a bit (as users
will be using branches to develop and test the code). What we actually
care about is the base branch ('main', 'squid', etc) as that is fed into
our bootstrap script and we want the option to simple variations based
on the name of said base branch.
Rename CEPH_BRANCH to CEPH_BASE_BRANCH for clarity.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a new --current-branch argument that lets the user supply a name for
the current branch. This allows the automatic tag generation to avoid
calling git - something useful if the tree is not using a git checkout
(like a tarball). It also allows you to pull a temporary branch in git
but ignore it and act like the temporary branch is the base branch.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 11 Feb 2025 23:36:13 +0000 (18:36 -0500)]
script/build-with-container: add more distro aliases
Add a system to define distro name aliases and use that to define some
additional aliases, primarily to match ubuntu codenames rather than
version numbers. Requested by Zack.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Patrick Donnelly [Tue, 25 Feb 2025 21:36:12 +0000 (16:36 -0500)]
Merge PR #61889 into main
* refs/pull/61889/head:
msg/async/Stack: include cleanup
msg/Message: include cleanup
msg/async/crypto_onwire: include cleanup
msg/async/ProtocolV1: add missing includes
msg/async/frames_v2: add missing includes
msg/msg_types: add missing includes
Casey Bodley [Wed, 8 Jan 2025 19:52:03 +0000 (14:52 -0500)]
rgw/s3: CreateBucket extension for layout type and shard count
extend s3's CreateBucketConfiguration [1] with a custom BucketIndex element
that can override rgw's default bucket index type and shard count
to create an indexless bucket:
<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<BucketIndex>
<Type>Indexless</Type>
</BucketIndex>
</CreateBucketConfiguration>
to create a normal pre-sharded bucket:
<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<BucketIndex>
<Type>Normal</Type>
<NumShards>1023</NumShards>
</BucketIndex>
</CreateBucketConfiguration>
Dhairya Parmar [Tue, 25 Feb 2025 10:58:06 +0000 (16:28 +0530)]
qa: remove unreacheable/redundant code from test_data_scan.py
call to self.fs.journal_tool(["journal", "reset", "--force", "--yes-i-really-really-mean-it"], 0)
is already made below it, the same call under if False is unreachable.
Zac Dover [Tue, 25 Feb 2025 04:57:11 +0000 (14:57 +1000)]
doc/releases: correct squid release order
Put the releases of Squid in descending order. This change alters the
order of the Squid releases so that it is the same as the order of the
other Ceph releases.
1.Added option kmip in KMS dropdown in configuration page RGW
2.Made API dynamic for setting form data for Vault, kmip both for (S3 and
KMS)
3.Both Getdata and setdata API for RGW config page is modified
Casey Bodley [Mon, 24 Feb 2025 21:50:39 +0000 (16:50 -0500)]
doc/rgw: clarify path-style vs virtual-hosted-style access
instead of referring to "vhost-style", copy the "path-style" and
"virtual-hosted-style" language from https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html
expand the FQDN acronym to avoid potential confusion
"The second method is deprecated by AWS" had incorrectly referred to the
vhost-style method - clarify that it refers to path-style access
Patrick Donnelly [Mon, 24 Feb 2025 21:41:00 +0000 (16:41 -0500)]
Merge PR #61697 into main
* refs/pull/61697/head:
osdc/ObjectCacher: include cleanup
osdc/Objecter: include cleanup
osdc/Journaler: include cleanup
osdc/Striper: add missing includes
osdc/Filer: include cleanup
crush/CrushWrapper: un-inline methods to fix linker errors
client/Client: add missing include and forward declaration
These changes allow the pool to be configured to disable mirroring on
the default namespace if required (init-only mode). It also allows the
default namespace to be mirrored to a non-default namespace on the
remote pool.
Sachin Prabhu [Sun, 9 Feb 2025 12:38:14 +0000 (12:38 +0000)]
cephadm/smb: add ctdb ports to smb daemon endpoint
When setting up a clustered SMB service, make sure that the port for the
ctdb service is opened up on the firewall. This port is used by the ctdb
servers to communicate with each other.
Ilya Dryomov [Thu, 20 Feb 2025 15:38:41 +0000 (16:38 +0100)]
qa/workunits/rbd: add a test for force promote with a user snapshot
Add a reproducer for the crash on a bad variant access which was fixed
in commit 7d75161051da ("librbd: fix a crash in get_rollback_snap_id").
The reproducer deliberately works around many other issues with force
promote in snapshot-based mirroring: stopping rbd-mirror daemon
shouldn't be necessary (let alone with SIGKILL), get_rollback_snap_id()
and its caller can_create_primary_snapshot() are flawed and can pick
the wrong snapshot to roll back to or skip rollback when it's actually
required, the user snapshot in this scenario should be removed as part
of force promoting because it's incomplete and won't be usable after
the image is promoted, etc.
projects / ceph.git / log