Zac Dover [Mon, 4 Mar 2024 10:41:16 +0000 (20:41 +1000)]
doc/rados: link to pg setting commands
Link to the instructions for manually setting the number of PGs per
pool, from the mention of placement groups. These instructions are
included here in response to a request from Ronen Friedman on the
occasion of the removal of links to the PGcalc (see
https://github.com/ceph/ceph/pull/55899#pullrequestreview-1912940118).
Zac Dover [Sun, 3 Mar 2024 10:28:00 +0000 (20:28 +1000)]
doc/rados: remove PGcalc from docs
Remove mention of the "PG calc" tool from the documentation. I have
removed all mention of this in one fell swoop to help posterity restore
mention of this tool if we decide we need to do so.
Zac Dover [Fri, 1 Mar 2024 12:11:14 +0000 (22:11 +1000)]
doc/install: add manual RADOSGW install procedure
Add a manual RADOSGW installation procedure to
doc/install/manual-deployment.rst. This procedure was developed by Janne
Johansson and reported to the ceph-users mailing list on 29 Jan 2024
here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/LB3YRIKAPOHXYCW7MKLVUJPYWYRQVARU/
Co-authored-by: Janne Johansson <icepic.dz@gmail.com> Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 565bc9503838906995fa48f59debcd2843775b18)
Zac Dover [Thu, 29 Feb 2024 08:08:10 +0000 (18:08 +1000)]
doc/glossary: improve "MDS" entry
Improve the entry for "MDS" in doc/glossary.rst by linking to the
"ceph-mds" man page and mentioning the relationship between clients and
MDS (or MDSes).
Zac Dover [Mon, 26 Feb 2024 10:03:48 +0000 (20:03 +1000)]
doc/rados: add "change public network" procedure
Add a procedure to /doc/rados/operations/add-or-rm-mons.rst that
explains how to change the public_network in a Ceph cluster deployed
with cephadm. This procedure was developed by Eugen Block, and can be
seen in its original form here:
https://heiterbiswolkig.blogs.nde.ag/2024/02/22/cephadm-change-public-network/
Zac Dover [Fri, 23 Feb 2024 16:05:42 +0000 (02:05 +1000)]
doc/rbd: repair ordered list
Fix the numbering in an ordered list. The numbering was thrown off
because a ".. prompt" directive was improperly indented (it wasn't
indented at all).
See https://github.com/ceph/ceph/pull/55540#discussion_r1500051264
Kefu Chai [Sat, 3 Feb 2024 05:46:05 +0000 (13:46 +0800)]
debian/cephadm.postinst: stop using adduser --gecos
--gecos option of adduser is deprecated in debian/bookworm, and
will be removed in debian/trixie,
see https://manpages.debian.org/bookworm/adduser/adduser.8.en.html.
so to be future-proof, let's switch to `usermod --comment`. please
note, since we still need to support ubuntu/jammy which is used in
our CI, and `adduser` shipped by ubuntu/jammy does not support
`--comment` yet, so we cannot use this option.
Kefu Chai [Wed, 17 Jan 2024 15:47:39 +0000 (23:47 +0800)]
debian/cephadm.postinst: specify --home when adduser
quote from adduser/NEWS.Debian.gz:
> System user home defaults to /nonexistent if --home is not specified.
> Packages that call adduser to create system accounts should explicitly
> specify a location for /home (see Lintian check
> maintainer-script-lacks-home-in-adduser).
so let's follow this change in adduser. otherwise "cephadm"
would have a $HOME at `/nonexistent`.
Kefu Chai [Wed, 17 Jan 2024 15:36:12 +0000 (23:36 +0800)]
debian/ceph-common.postinst: set user directory using adduser
now that adduser allows us to set its home directory, we can do
this using adduser instead of using usermod. this change also
silences the warning from lintian
"maintainer-script-lacks-home-in-adduser". lintian complains if
`adduser --system` is called without passing `--home` option.
also, take this opportunity to s/-c/--comment/ in the command line
of `usermod`, for better readability.
Kefu Chai [Wed, 17 Jan 2024 15:09:02 +0000 (23:09 +0800)]
debian/control: add adduser to Depends of cephadm and ceph-common
in `debian/ceph-common.postinst` and `debian/cephadm.postinst`, we
use `adduser --system` to create the system user when configuring
the corresponding package.
before this change, the dependency is not listed in the runtime
`Depends` section of ceph-common and cephadm.
in this change, the dependency is added. this is also suggested
by Securing Debian Manual, see
https://www.debian.org/doc/manuals/securing-debian-manual/bpp-lower-privs.en.html
Daniel Persson [Sun, 3 Dec 2023 09:46:56 +0000 (09:46 +0000)]
mgr/dashboard: Updated test dependencies
Seemed that the test dependencies was separated in two different requirements files
one for the testing and one for linting. Added the JWT dependency in the linting file
as well.
Daniel Persson [Wed, 29 Nov 2023 09:39:51 +0000 (09:39 +0000)]
mgr/dashboard: Simplify authentication protocol
By removing the dependency to PyJWT we also remove the dependency to the cryptographic library which
in the dashboard module will create a crash. In newer implementations of the library PyO3 is used to run
rust code in order to encrypt with Elliptic Curves. This is never used in the dashboard communication so
a much simpler implementation where we only use the hmac sha256 algorithm to create the signed JWT message
could be used.
Fixes: https://forum.proxmox.com/threads/ceph-warning-post-upgrade-to-v8.129371 Signed-off-by: Daniel Persson <mailto.woden@gmail.com>
(cherry picked from commit c616a9d017b5fcc85bb5c1556bccf4c77cc3899e)
Redouane Kachach [Thu, 22 Feb 2024 09:04:11 +0000 (10:04 +0100)]
mgr/rook: adding empty calls to upgrade_ls and upgrade_status
added empty calls to upgrade_ls and upgrade_status to avoid
dashboard errors when entering the view Cluster > Upgrade. Empty
calls are used because we don't support the upgrade functionality
in rook as we do for normal Ceph deployments. In case of rook user
has to follow a different process to upgrade Ceph.
Afreen [Tue, 13 Feb 2024 10:26:09 +0000 (15:56 +0530)]
mgr/dashboard: Handle errors for /api/osd/settings
Fixes https://tracker.ceph.com/issues/62089
issue:
=====
/api/osd/settings returns "TypeError: string indices must be
integers" sometimes.
The result is coming from `osd dump` command which instead of returning
an object returns an error message which then displays error on
dashboard.
fix:
====
Added a try-catch block to handle error and updated frontend code to
handle those
Redouane Kachach [Thu, 22 Feb 2024 08:48:06 +0000 (09:48 +0100)]
mgr/rook: removing all the code related to OSDs creation/removal Fixes: https://tracker.ceph.com/issues/64211 Signed-off-by: Redouane Kachach <rkachach@redhat.com>
Patrick Donnelly [Thu, 15 Feb 2024 15:28:32 +0000 (10:28 -0500)]
mds: reverse MDSMap encoding of max_xattr_size/bal_rank_mask
Commit e134c890 adds the bal_rank_mask with encoded (ev) version 17. This was
merged into main Oct 2022 and made it into the reef release normally.
Commit 7b8def5c adds the max_xattr_size also with encoded (ev) version 17 but
places it before bal_rank_mask. This is problematic as there were no plans to
backport e134c890 to quincy or pacific so piggybacking on the ev 17 bump would
not work and otherwise would require the backports to be done as a set to
ensure consistency (including with the kernel client).
However, the real issue is that 7b8def5c was not merged until after reef was
already cut. This required 7b8def5c to be backported separately in [1] which
was not merged until after v18.2.1 (current reef HEAD as of this commit).
Ultimately, this means that there are reef versions (v18.2.[01]) in the wild
which expect bal_rank_mask to be encoded at ev17 and not (max_xattr_size,
bal_rank_mask). Adding to the complications, the kernel client has already
merged code [2] expecting max_xattr_size for ev17.
It was decided in a github discussion [3] to move bal_rank_mask to ev18 to
avoid updating the kernel client which was done in the main branch via 36ee8e7e
and update the reef max_xattr_size backport with the same change (d8cebd67).
Unfortunately, this breaks upgrades v18.2.[01] to newer reef versions or to
main. The reason is that monitors will encode v17 with bal_rank_mask
(max_xattr_size is not merged yet) and send that to upgraded mgrs (which are
upgraded first). The mgr will attempt to decode bal_rank_mask as a uint64_t
(max_xattr_size) but fail because an empty (by default) bal_rank_mask is simply
encoded as a signed 32-bit integer. Consequently, the mgr will fail decoding
with:
failed to decode message of type 45 v1: End of buffer [buffer:2]
Of course the problem does not stop there, even if the mgr were able to handle
this, the monitors/mds/clients would fail in similar fashion.
So the only choice left is to fix max_xattr_size to be encoded at ev18.
Fortunately, v18.2.2 has not been released nor has any max_xattr_size backport
to quincy/pacific been merged. The main downside will be that kernels will
wrongly decode ev17 (which is already true for ceph clusters running
v18.2.[01]). A follow-up kernel fix will be required.
Ilya Dryomov [Mon, 12 Feb 2024 12:07:22 +0000 (13:07 +0100)]
librbd: refactor merge() for SparseBufferlistExtent
- pass left.length + right.length instead of bl.length()
for consistency and to avoid circumventing the assert in
SparseBufferlistExtent constructor
- claim_append() takes an lvalue reference, no need to move
- follow the pattern used in split()
Ilya Dryomov [Mon, 12 Feb 2024 10:00:45 +0000 (11:00 +0100)]
librbd: fix split() for SparseExtent and SparseBufferlistExtent
SparseExtents and SparseBufferlist are typedefs for interval_map. In
both cases, split() handler is broken: for the former the extent isn't
actually split and for the latter incorrect bufferlist is attached to
the split extent.
Fortunately, both SnapshotDelta as produced by ObjectListSnapsRequest
and SparseBufferlist used in a couple of places seem to be collections
where only disjoint intervals are inserted and splitting doesn't occur
(at least in the common case). But still, this is a landmine waiting
for someone to step on it.
Zac Dover [Mon, 19 Feb 2024 08:41:45 +0000 (18:41 +1000)]
doc/cephfs: edit add-remove-mds
Disambiguate a note in doc/cephfs/add-remove-mds.rst to help readers
distinguish between cases in which they might want to use an automated
tool such as cephadm to deploy MDSes and cases in which they might want
to manually deploy MDSes.
See: https://github.com/ceph/ceph/pull/45639
Tracker: https://tracker.ceph.com/issues/54551
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 39ad6264aa1c97b04974e04046033887325ed2b2)
Milind Changire [Mon, 19 Feb 2024 12:06:34 +0000 (17:36 +0530)]
Merge PR #52751 into wip-61794-reef
* refs/pull/52751/head:
PendingReleaseNotes: support for subvolumes and subvolume groups in snap_schedule
snap_schedule/tests: fix db upgrade issue
qa: add yaml for on demand subvol version testing
qa: add test cases for testing --subvol and --group arguments
mgr/volumes: conditionalize subvolume upgrade
mgr/volumes: ensure correct init of v1 subvol
mgr/snap_schedule: add subvol and subvol group arguments to cli
mds/snap_schedule: add subvolume group column management
mgr/volumes: add remote helper methods to fetch subvolume info
mgr/snap_schedule: catch all exceptions for cli
Patrick Donnelly [Thu, 15 Feb 2024 20:22:42 +0000 (15:22 -0500)]
Merge PR #54952 into reef
* refs/pull/54952/head:
qa: configure balancer for multi-mds workloads
qa: create qa subvolumes in named subvolumegroup
qa: do not rely on default max_mds value
qa: add automate_balance to dashboard qa schema
doc/cephfs: add docs for balance_automate
doc/cephfs: use bash prompt for shell code
mds: add balance_automate fs setting
projects / ceph.git / log