Zac Dover [Thu, 7 Aug 2025 05:03:22 +0000 (15:03 +1000)]
doc/cephfs: edit troubleshooting.rst
Follow up on comments made by Anthony D'Atri in
https://github.com/ceph/ceph/pull/64832 and make other small changes to
increase the ease of reading this text.
Adam C. Emerson [Fri, 18 Apr 2025 07:27:36 +0000 (03:27 -0400)]
rgw: Add run_coro utility
A convenience function for turning coroutines that return values and
use exceptions, `error_code`, or similar into `int`-returning
functions that take references to out parameters.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Wed, 6 Aug 2025 20:02:32 +0000 (16:02 -0400)]
common/async: Update `use_blocked` for newer asio
Reimplement with `initiate` rather than the old style. This
necessitates getting rid of the old `async::Completion` in anything
that was calling it, and other changes.
Also, use disposition for error handling.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Ronen Friedman [Wed, 6 Aug 2025 05:38:07 +0000 (00:38 -0500)]
osd/scrub: do not limit operator-initiated repairs
'auto-repair' scrubs are limited to a maximum of
'scrub_auto_repair_num_errors' damaged objects.
However, operator-initiated repairs should not be limited
by that number. Alas, a bug in a previous commit
(97de817ad1c253ee1c7c9c9302981ad2435301b9) modified the
code in such a way that it applied the
'scrub_auto_repair_num_errors' limit to all repairs,
including operator-initiated ones. This commit fixes that.
Ville Ojamo [Wed, 6 Aug 2025 05:05:49 +0000 (12:05 +0700)]
doc/install: Linkify mention of ceph.conf and use ref for links
Linkify first mention of config file to ceph.conf docs in:
- install-storage-cluster.rst
- manual-deployment.rst
- manual-freebsd-deployment.rst
Use ref instead of an external link in:
- clone-source.rst
- get-packages.rst
- index_manual.rst
- install-storage-cluster.rst
- manual-deploymen.rst
- manual-freebsd-deployment.rst
Only where a label already exists at the destination.
Delete the old link definition if one was used previously.
That should be about all external links in install/ that can use
existing labels for ref.
Fix an instance of "the a" into just "a" that is consistent with other
similar mentions in manual-freebsd-deployment.rst.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Adam C. Emerson [Mon, 30 Jun 2025 20:54:46 +0000 (16:54 -0400)]
rgw/datalog: Manage and shutdown tasks properly
This is slightly ugly but good enough for now. Make sure we can block
when shutting down background tasks.
Remove a few `driver` parameters that are unused. This lets us
simplify the IAM Policy and Lua tests and not construct stores we
never use. (Which is good since we aren't running them under a cluster.)
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Fri, 30 May 2025 20:54:45 +0000 (16:54 -0400)]
neorados: Hold reference to implementation across operations
Asynchrony combined with cancellations keeps leading to occasional
lifetime issues, so follow the best-practices of Asio I/O objects by
having completions keep a reference live.
The original NeoRados backing implements Asio's two-phase shutdown
properly.
The RadosClient backing does not, because it shares an Objecter with
completions that do not belong to it. In practice I don't think this
will matter since librados and neorados get shut down around the same
time.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
* kafka: pass full broker list to consumer in tests
* kafka: use ip instead of localhost
* kafka: make sure topic exists before consumer start
* kafka: fix zookeeper and broker conf in tests
* kafka: verify receiver in the test
* kafka: tests were not running (Fixes: https://tracker.ceph.com/issues/72240)
* kafka: failover tests were failing (Fixes: https://tracker.ceph.com/issues/71585)
* simplify basic tests run command
* v2 migration tests were not running
* fix failing migration tests
Bill Scales [Fri, 1 Aug 2025 15:17:58 +0000 (16:17 +0100)]
doc: erasure coding enhancements for tentacle
* Document new pool flag allow_ec_optimizations
* Reference new conf setting osd_pool_default_flag_ec_optimizations
* Add section describing Erasure Code Optimizations
Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>
Ville Ojamo [Tue, 5 Aug 2025 15:34:26 +0000 (22:34 +0700)]
doc/rados: Remove obsolete fs-recomm links
2 files linked to filesystem-recommendations.rst which was removed
around the year 2017.
I understand this was relevant only for Filestore. So simply remove the
references to this file & the link definition if one was used.
Ville Ojamo [Tue, 5 Aug 2025 14:45:05 +0000 (21:45 +0700)]
doc/rados: Use ref instead of relative external links
Instead of external links use :ref: where dst labels exist already in:
operations/erasure-code.rst
operations/pools.rst
troubleshooting/troubleshooting-osd.rst
Use link text generation where it is reasonably close to previous manual
link text.
Delete some unused link definitions.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Zac Dover [Tue, 5 Aug 2025 11:24:41 +0000 (21:24 +1000)]
doc/cephfs: edit troubleshooting.rst
Edit "Stuck in up:replay" under the "Stuck During Recovery" section of
doc/cephfs/troubleshooting.rst. I had planned to edit the entire "Stuck
During Recovery" section in a single commit, but I think that the
material is too involved for that.
Ville Ojamo [Tue, 5 Aug 2025 06:08:09 +0000 (13:08 +0700)]
doc/radosgw: Small improvements in kmip.rst
Major rewrite of the last section that is a copypasta from vault.rst:
- "engines" are relevant only to Hashicorp Vault and not KMIP
- leave only 1 copy of the 2 identical CLI examples
- talk about KMIP and not Vault, it is an alternative to Vault
Also fix other mention of "Vault" into "KMIP".
Auto-generate contents list instead of hardcoding it.
Use '=' in ceph.conf example, I believe ':' cannot be used.
Capitalize "Ceph", "Python", "KMIP", "OpenSSL", "PyKMIP" consistently.
Call it consistently "Ceph Object Gateway".
Format "pykmip" in italic when referring to the binary.
Hyphen in "PEM-encoded" along with capitalization.
Use double backticks for data.
Spell out a lonesome number "1" in text.
Fix typo "correspondent" to "corresponding".
Promptify CLI commands.
Use title case consistently in section titles.
Linkify mention of Ceph configuration file.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Ville Ojamo [Tue, 5 Aug 2025 05:14:13 +0000 (12:14 +0700)]
doc/radosgw: Small improvements in s3-notification-compatibility.rst
Attempt a small fix to a grammatical error in a sentence.
It should also refer to "below" and not "above", probably.
End full sentences in full stops.
Indent an unordered list consistently so that it renders consistently
with the same bullets.
Also indent all various blocks at the same columns consistently.
Wrap lines before column 80 while we're at it.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Ville Ojamo [Mon, 12 May 2025 09:01:44 +0000 (16:01 +0700)]
doc/radosgw: Use ref for hyperlinks, 2nd batch
Use validated ":ref:" hyperlinks instead of "external links" in "target
definitions" when linking within the Ceph docs:
- Add a label at beginning of referenced files if missing.
- Remove unused "target definitions".
- Updated links targeting files: compression encryption keystone
Cleaned hyperlinks usage in kmip.rst:
- Some links were using anonymous links (double underscore) unnecessarily.
- Some links were not using backticks, add for consistency.
- Move anonymous link definition to after the ordered list to avoid
unnecessary empty line between list items.
Use an already existing label for 2 intra-docs links that used full URLs.
Use an already existing label for intra-docs link instead of a file name
reference in s3/authentication.rst.
The rendered PR should look the same as the old docs, only differing in
the source RST.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Ville Ojamo [Mon, 4 Aug 2025 06:06:38 +0000 (13:06 +0700)]
doc/cephadm: Small improvements in services/tracing.rst
Use ref instead of a full URL link and add label for it in
doc/jaegertracing/index.rst.
Capitalize "Ceph", "Jaeger", "ElasticSearch" consistently.
Start sentences with capital case consistently.
Fix a typo.
Wrap lines a bit before column 80.
Use an ordered list instead of hardcoding list numbers in separate
paragraphs.
Don't use ordered list for items that do not both fit under the text
paragraph introducing the list.
Rewrite the sentences to be more consistent and hopefully more correct.
Add articles that I believe should be there, also for consistency with
the previous paragraph.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Naman Munet [Tue, 22 Jul 2025 17:08:42 +0000 (22:38 +0530)]
mgr/dashboard: user accounts enhancements
fixes: https://tracker.ceph.com/issues/72072
PR covers:
1) Displaying account name instead of account id in bucket list page & bucket edit form for account owned buckets
2) non-root account user can now be assigned with managed policies with which they can perform operations
3) The root user indication shifted next to username in users list rather than on Account Name with a new icon.
Alex Ainscow [Mon, 14 Jul 2025 15:55:40 +0000 (16:55 +0100)]
osd: Optimised EC avoids ever reading more than K shards (if plugin supports it).
Plugins which support partial reads, should never need more than k shards
to read the data, even if some shards have failed. However, rebalancing commonly
requests k + m shards, as very frequently all shards are moved. If this occurs
and all k + m shards are online, the read will be achieved by reading ALL shards
rather than just reading k shards. This commit fixes that issue.
The problem is that we don't want to change the API to the old EC, so we cannot
update the plugin behaviour here. Instead, the EC code itself will reduce
the number of shards it tells minimum_to_decode about.
In a comment we note that bitset_set performance could be improved using _pdep_u64.
This would require fiddly platform-specific code and would likely not show
any performance improvements for most applications. The majority of the calls to
this function will be with a bitset that has <=n set bits and will never enter this
if statement. When there are >n bits set we are going to save one or more read I/Os,
the cost of the for loop is insignificant vs this saving. I have left the comment
in as a hint to future users of this function.
Further notes were made in a review comment that are worth recording:
- If performance is limited by the drives, then less read I/Os is a clear advantage.
- If performance is limited by the network then less remote read I/Os is a clear advantage.
- If performance is limited by the CPU then the CPU cost of M unnecessary remote
read I/Os (messenger+bluestore) is almost certainly more than the cost of doing an
extra encode operation to calculate the coding parities.
- If performance is limited by system memory bandwidth the encode+crc generation
has less overhead than the read+bluestore crc check+messenger overheads.
Longer term this logic should probably be pushed into the plugins, in particular
to give LRC the opportunity to optimize for locality of the shards. Reason for
not doing this now is that it would be messy because the legacy EC code cannot
support this optimization and LRC isn't yet optimizing for locality
Incorporate into doc/cephfs/ceph-dokan.rst the suggestions made by
Anthony D'Atri in https://github.com/ceph/ceph/pull/64737, and make a
few other small improvements to the English language in that file.
Adam King [Wed, 30 Jul 2025 19:51:11 +0000 (15:51 -0400)]
mgr/cephadm: don't use list_servers to get active mgr host for prometheus SD config
Having a lot of calls into list_servers causes issues with
the core ceph mgr on large clusters. Additionally, we were
using it purely to get the active mgr's host here, which
cephadm should be able to do without needing a mgr api call
Adam King [Wed, 30 Jul 2025 19:49:20 +0000 (15:49 -0400)]
mgr/cephadm: add interval control for stray daemon checks
Primarily to avoid running list_servers (which we kind of
need to do stray daemon checks since the whole point is
to check against a source that isn't cephadm). It was
found on larger clusters calling into list_servers
often can cause issues with the core ceph mgr
projects / ceph.git / log