Venky Shankar [Thu, 30 Nov 2023 05:58:10 +0000 (11:28 +0530)]
Merge PR #53431 into main
* refs/pull/53431/head:
qa: add test cases to verify error reporting works as expected
mgr: fix some doc strings in object_format.py
mgr/tests: test returning error status works as expected
mgr: make object_format's Responder class capable of responding err status
mgr/nfs: report proper errno with err status
Reviewed-by: John Mulligan <jmulligan@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
... when checking whether a rbd_support module command fails after
blocklisting the module's client.
In tests that check the recovery of the rbd_support module after its
client is blocklisted, the rbd_support module's client is
blocklisted using the `osd blocklist add` command. Next,
`osd blocklist ls` command is issued to confirm that the client is
blocklisted. A rbd_support module command is then issued and expected
to fail in order to verify that the blocklisting has affected the
rbd_support module's operations. Sometimes it was observed that before
this rbd_support module command reached the ceph-mgr, the rbd_support
module detected the blocklisting, recovered from it, and was able to
serve the command. To reduce the race window that occurs when trying to
verify that the rbd_support module's operation is affected by client
blocklisting, get rid of the `osd blocklist ls` command.
Fixes: https://tracker.ceph.com/issues/63673 Signed-off-by: Ramana Raja <rraja@redhat.com>
Adam C. Emerson [Fri, 2 Dec 2022 06:55:47 +0000 (01:55 -0500)]
rgw: Remove `RGWSI_RADOS` from `RGWSI_MDLog`
Simply use the RADOS handle and `rgw_rados_ref` directly.
Also move `async_processor` out from `RGWSI_RADOS` and into
`RGWServices_Def`. This is as good a place as it for any, for now, as
it's reachable by everyone who needs it and exists through the
lifetime of the process.
Eventually it's going to go away due to coroutinization, anyway.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Fri, 2 Dec 2022 04:25:54 +0000 (23:25 -0500)]
rgw: Distentangle `rgw_rados_ref` from `RGWSI_RADOS`
`RGWSI_RADOS` has no reason to exist. `rgw_rados_ref`, on the other
hand, is fairly useful. Move `rgw_rados_ref` to
`store/rados/rgw_tools.h` along with a function to create one.
Then clean up the fallout.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Rishabh Dave [Thu, 23 Nov 2023 10:40:00 +0000 (16:10 +0530)]
qa/cephfs: improvements for name generators in test_volumes.py
Generate a name that is shorter and easier to remember.
Also, write a simpler, faster & better helper method for generating
unique names. This method will also have shorter and more concise name,
so this will be easier to type and easier to read.
Fixes: https://tracker.ceph.com/issues/63680 Signed-off-by: Rishabh Dave <ridave@redhat.com>
Zac Dover [Tue, 28 Nov 2023 05:08:48 +0000 (06:08 +0100)]
doc/rados: improve "Ceph Subsystems"
Improve the English in the subsection "Ceph Subsystems" in the section
"Subsystem, Log and Debug Settings" [sic] in
doc/rados/troubleshooting/log-and-debug.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
Josh Soref [Thu, 23 Jun 2022 08:11:09 +0000 (04:11 -0400)]
rgw: rewrite rgw_admin help text
* The line wrapping was fairly random.
* There were cases where words appeared to be doubled.
- In fact one was an argument
- the other was the start of the help text.
The `ceph-mgr` package lost its dependency on `python3-bcrypt` when
the dependencies got moved from d/control to a dh_python3 compatible
requires file. Add it again as the bcrypt module is still used there.
Otherwise one gets errors when, e.g., calling `ceph -s` after a fresh
installation:
> 13 mgr modules have failed dependencies
> Module 'balancer' has failed dependency: No module named 'bcrypt'
> Module 'crash' has failed dependency: No module named 'bcrypt'
> Module 'devicehealth' has failed dependency: No module named 'bcrypt'
> Module 'iostat' has failed dependency: No module named 'bcrypt'
> Module 'nfs' has failed dependency: No module named 'bcrypt'
> Module 'orchestrator' has failed dependency: No module named 'bcrypt'
> Module 'pg_autoscaler' has failed dependency: No module named 'bcrypt'
> Module 'progress' has failed dependency: No module named 'bcrypt'
> Module 'rbd_support' has failed dependency: No module named 'bcrypt'
> Module 'restful' has failed dependency: No module named 'bcrypt'
> Module 'status' has failed dependency: No module named 'bcrypt'
> Module 'telemetry' has failed dependency: No module named 'bcrypt'
> Module 'volumes' has failed dependency: No module named 'bcrypt'
Fixes: https://tracker.ceph.com/issues/63637 Fixes: ef19547e83e ("debian: add .requires for specifying python3 deps") Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Casey Bodley [Wed, 15 Nov 2023 20:29:35 +0000 (15:29 -0500)]
encoding: add round_trip_encode()/decode() for chrono types
the default encodings for chrono types were made to be
backward-compatible with utime_t, so truncated seconds to 32 bits
adds new functions that encode these chrono types using their underlying
representation, which for ceph::real_time and ceph::timespan is
'nanoseconds as uint64_t'
Casey Bodley [Wed, 22 Nov 2023 19:42:07 +0000 (14:42 -0500)]
rgw/quota: remove base class AsyncRefreshHandler
instead of using a virtual `allocate_refresh_handler()` to create an
object to call the virtual `AsyncRefreshHandler::init_fetch()` on, just
provide one virtual `init_refresh()` to start the operation. this avoids
the need for `AsyncRefreshHandler` altogether
Casey Bodley [Wed, 22 Nov 2023 19:29:23 +0000 (14:29 -0500)]
rgw/quota: use intrusive_ptr for RefCountedWaitObject
instead of relying on async_refresh_response/fail to manage ref counts
manually, just stash an intrusive_ptr in AsyncRefreshHandler for the
duration of the async operation
Casey Bodley [Wed, 22 Nov 2023 16:20:26 +0000 (11:20 -0500)]
rgw/sal: unify ReadStatsCB for async user/bucket stats
`rgw::sal::Bucket::read_stats_async()` had returned stats as
`map<RGWObjCategory, RGWStorageStats>`, but the only caller in
rgw_quota.cc just summed up the categories into a single instance
of `RGWStorageStats`. moving this summation up into rgw_rados'
`RGWGetBucketStatsContext` allows us to unify these sal callbacks
around `RGWStorageStats` alone
the `RGWGetUserStats_CB`/`RGWGetBucketStats_CB` callbacks had inherited
from `RefCountedObject` and required manual reference counting. switch
to `boost::intrusive_ptr` for scope-based shared ownership
Rishabh Dave [Thu, 2 Nov 2023 15:15:27 +0000 (20:45 +0530)]
mgr/vol: fix flake8 warnings
Fix warnings printed by the command "flake8 --select=F,E9
--exclude=venv,.tox src/pybind/mgr/volumes/".
While working with mgr/vol, syntax errors are not printed anywhere. Any
attempt to run vstart.sh after such faulty patch causes vstart.sh to go
in a infinite loop. And running "ceph fs volume" command prints "no such
command exists". This doesn't tell the actual issue and causes
confusion.
When flake8 is run, the issue is not immediately apparent due to so many
warnings. Therefore, fix these warnings so that it becomes easier to
spot such critical issues here onwards.
Note: mypy uses comments like "#type: Dict" for type checking and
therefore counts 'from typechecking import Dict' as not unsued. But
flake8 doesn't recognize type hints embedded in comment. Therfore,
switch to actually using type hints instead of adding type hints to
comments.
Rishabh Dave [Fri, 10 Nov 2023 14:53:33 +0000 (20:23 +0530)]
qa/cephfs: don't write guest keyring to file "keyring"
Writing guest keyring to CWD's file named "keyring" will over-write
build/keyring on developer's machine which will make the cluster
inoperatable and also fail the test.
Fixes: https://tracker.ceph.com/issues/63506 Signed-off-by: Rishabh Dave <ridave@redhat.com>
Casey Bodley [Thu, 23 Nov 2023 05:12:29 +0000 (00:12 -0500)]
cmake/rocksdb: make sure dependencies build before rocksdb
some of rocksdb's dependencies may not have built by the time its
ExternalProject starts, so it can fail with missing headers or
libraries. for example, `uring::uring` may itself be an ExternalProject,
and its include directory won't exist until it starts building:
```
[89/1345] Performing configure step for 'rocksdb_ext'
FAILED: src/rocksdb_ext-prefix/src/rocksdb_ext-stamp/rocksdb_ext-configure build/src/rocksdb_ext-prefix/src/rocksdb_ext-stamp/rocksdb_ext-configure
...
CMake Error in CMakeLists.txt:
Imported target "uring::uring" includes non-existent path
"build/src/liburing/src/include"
in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include:
* The path was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and references files it does not
provide.
...
[91/1345] Performing download step (git clone) for 'liburing_ext'
Cloning into 'liburing'...
```
use `add_dependencies(rocksdb_ext)` to make sure all of its dependencies
are available before starting the build
Rishabh Dave [Sun, 19 Nov 2023 11:42:51 +0000 (17:12 +0530)]
mgr/vol: add comments in async_cloner
It's not obvious where the clone creation is performed. It's especially
obfuscated because we call the function/method for cloning indirectly.
Therefore, add comments where necessary and mark it with heavy rating.
Same goes for all other "subvolume snapshot clone" operations.
Lucian Petrut [Fri, 18 Aug 2023 12:25:52 +0000 (12:25 +0000)]
common: Windows Unicode CLI support
Windows CLI arguments use either ANSI (main()) or UTF-16 (wmain()).
Meanwhile, Ceph libraries expect UTF-8 and raise exceptions when
trying to use Unicode CLI arguments or log Unicode output:
rbd.exe create test_unicode_șțăâ --size=32M
terminate called after throwing an instance of 'std::runtime_error'
what(): invalid utf8
We'll use a Windows application manifest, setting the "activeCodePage"
property [1][2]. This enables the Windows UCRT UTF-8 mode so that
functions that receive char* arguments will expect UTF-8 instead of ANSI,
including main(). One exception is CreateProcess, which will need the
UTF-16 form (CreateProcessW).
Despite the locale being set to utf-8, we'll have to explicitly set
the console output to utf-8 using SetConsoleOutputCP(CP_UTF8).
In order to use the UTF-8 locale, we'll have to switch the mingw-llvm
runtime from msvcrt to ucrt.
This also fixes ceph-dokan crashes that currently occur when non-ANSI
paths are logged.
projects / ceph.git / log