Cory Snyder [Wed, 2 Nov 2022 20:06:23 +0000 (20:06 +0000)]
rgw: add 'inline_data' zone placement info option
Adds a new RGW zone placement info option to control whether
an object's first data chunk is placed in the head object.
This allows admins to make a tradeoff between optimizing for
PUT/GET performance vs. DELETE performance for some cluster
configurations.
Ronen Friedman [Mon, 24 Jan 2022 13:19:01 +0000 (13:19 +0000)]
osd/scrub: report replicas slow to repond to scrub requests
Implemented timeouts:
1: Slow-Secondary Warning:
Once at least half of the replicas have accepted the reservation, we
start reporting any secondary that takes too long (more than <conf>
milliseconds after the previous response received) to respond to the reservation
request.
(Why? because we have encountered real-life situations where a specific
OSD was systematically very slow to respond (e.g. 5 seconds in one case) to
the reservation requests, slowing the scrub process to a crawl).
2: Reservation Process Timeout:
We now limit the total time the primary waits for the replicas to
respond to the reservation request. If we do not get all the responses
(either Grant or Reject) within <conf> milliseconds, we give up and release all the
reservations we have acquired so far.
(Why? because we have encountered instances where a reservation request
was lost - either due to a bug or due to a network issue.)
Kefu Chai [Sat, 10 Dec 2022 02:31:21 +0000 (10:31 +0800)]
rgw: drop rgw_obj_key::to_str()
rgw_obj_key::to_str() is mainly used by operator<<(ostream&, ..), so
we can just implement it with the specialization of
fmt::formatter<rgw_obj_key>. and let operator<<(ostream&, ..) to
call into fmt::format(..):
1. for better readability and
2. for probably better performance -- we don't need to do deep copy
for constructing a `std::string` from a `char[]`.
3. for better standard compliance -- we don't need to use variable-length
array in C++ code. it is a part of C99 standard. but not a C++ standard.
Kefu Chai [Thu, 8 Dec 2022 04:38:07 +0000 (12:38 +0800)]
rgw: define FMT_HEADER_ONLY using CMakeLists.txt
less repeatings this way. and the macro definition can be populated
to the targets linking against `rgw_common`. this is more maintainable,
as rgw executables and libraries always link against `rgw_common`.
Kefu Chai [Fri, 2 Dec 2022 06:10:02 +0000 (14:10 +0800)]
fmt: pickup v9.1.0
fmt 9.0.0 dropped automatic `std::ostream` insertion operator discovery
when `fmt/ostream.h` to prevent ODR violations. instead of defining
`FMT_DEPRECATED_OSTREAM`, we took efforts to specialize
`fmt::formatter<..>` to be compatible with the new fmtlib. to avoid
breaking the build with fmt v9 and up, let's bump up the fmt submodule.
Kefu Chai [Fri, 2 Dec 2022 09:25:24 +0000 (17:25 +0800)]
rgw: pass fmt::format_string to pretty_print()
otherwise we'd have following compiling failure:
In file included from /var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.cc:13:
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.h:345:28: error: call to consteval function 'fmt::basic_format_string<char, std::basic_string<char> &, rgw_obj_key &, unsigned long &, std::basic_string<char> &>::basic_format_string<char[59], 0>' is not a constant expressio
n
fmt::print(*env->ostr, std::forward<T>(t)...);
^
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.cc:4136:8: note: in instantiation of function template specialization 'pretty_print<const char (&)[59], std::basic_string<char> &, rgw_obj_key &, unsigned long &, std::basic_string<char> &>' requested here
pretty_print(sc->env, "Syncing object s3://{}/{} version {} in sync from zone {}\n",
^
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.cc:4070:3: note: in instantiation of member function 'RGWBucketSyncSingleEntryCR<rgw_obj_key, rgw_obj_key>::operate' requested here
RGWBucketSyncSingleEntryCR(RGWDataSyncCtx *_sc,
^
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.cc:4352:27: note: in instantiation of member function 'RGWBucketSyncSingleEntryCR<rgw_obj_key, rgw_obj_key>::RGWBucketSyncSingleEntryCR' requested here
yield spawn(new SyncCR(sc, sync_pipe, entry->key,
^
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.h:345:44: note: function parameter 't' with unknown value cannot be used in a constant expression
fmt::print(*env->ostr, std::forward<T>(t)...);
^
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.h:343:53: note: declared here
void pretty_print(const RGWDataSyncEnv* env, T&& ...t) {
^
Kefu Chai [Fri, 2 Dec 2022 09:23:23 +0000 (17:23 +0800)]
rgw: specialize fmt::formatter<> for rgw_obj_key
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<rgw_obj_key> is defined so the tree can
compile with fmt v9.
Adam King [Fri, 9 Dec 2022 19:20:47 +0000 (14:20 -0500)]
pybind/mgr: partial reversion of #49359
When creating #49359 I was testing on an outdated
branch and didn't realize part of what I was fixing
had already been fixed in #49321. Basically ended up
changing what a variable "mypy_args" is set to but that
variable is no longer being used. It has no actual effect
but we should remove the extraneous code.
Adam King [Fri, 9 Dec 2022 16:10:36 +0000 (11:10 -0500)]
mgr/pybind: fix mypy arg parsing
On the new tox version it is treating each line as a new command
so it will do something like "mypy --config-file=../../mypy.ini"
as one command and then "-m balancer" as a totally separate command.
The first one immediately fails as it doesn't include any modules
to test. Adding backslashes to the ends of the lines gets it to
handle the lines as one long command
RGW - Zipper - Rename rgw::sal::Store to rgw::sal::Driver
All along, we've had an overload of the term "store". It was the base
class of the entrypoint, and it was the lowest layer in a stack. This
renames the base class to Driver. So, the primary elements of zipper
that to work are now Drivers, and they come in two different flavors:
Filters live in the middle of the stack, and do not need to implement
every API, but pass through instead. And Stores live a the bottom of
the stack, and must implement the entire API.
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Kefu Chai [Thu, 8 Dec 2022 16:49:37 +0000 (00:49 +0800)]
pybind/mgr: do not test with py37
as we always test with ubuntu jammy, which does not provide python3.7:
py37: skipped because could not find python interpreter with spec(s): py37
so there is no point testing with python3.7.
also, in tox v4, it is not able to handle "key = value" anymore, where
value has newlines in it. so we need to find a better way passing
command line options to the test command.
Kefu Chai [Thu, 8 Dec 2022 06:42:42 +0000 (14:42 +0800)]
qa: set locale to C.UTF-8 in tox.ini
as ansible is using UTF-8 encoded characters in the file names, so,
to avoid failures like:
File "/home/jenkins-build/build/workspace/ceph-pull-requests/qa/.tox/py3/lib/python3.10/site-packages/pip/_internal/utils/unpacking.py", line 217, in untar_file
with open(path, "wb") as destfp:
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 137-140: ordinal not in range(256)
we have to set a locale which is able to handle UTF-8.
see also https://github.com/ceph/teuthology/pull/1671
Kefu Chai [Thu, 8 Dec 2022 10:25:20 +0000 (18:25 +0800)]
pybind/mgr: drop cython from requires
cython is not required for running tox commands.
this should address the test failure like:
ROOT: will run in automatically provisioned tox, host /home/jenkins-build/build/workspace/ceph-pull-requests/build/mgr-virtualenv/bin/python3.10 is missing [requires (has)]: cython
Kefu Chai [Thu, 8 Dec 2022 06:53:33 +0000 (14:53 +0800)]
*: s/whitelist_externals/allowlist_externals/
as allowlist_externals was introduced in
tox v4.0. see
https://github.com/tox-dev/tox/commit/5e33fda1a40ffb4973de3d607a572891eb3cb2d2 , but
this option was backported to 3.18 as an alias of whitelist_externals, so we don't need
to specify the minversion to 4.0 in this change.
as we started using tox 4.0 and up (v4.0.2 in specific). tox complains
and fails like:
alerts-lint: failed with promtool is not allowed, use allowlist_externals to allow it
alerts-lint: FAIL code 1 (9.25 seconds)
see https://tox.wiki/en/latest/faq.html#tox-4-removed-tox-ini-keys
and https://tox.wiki/en/latest/config.html#allowlist_externals
it'd be nice to use a more inclusive language also. so, in this change,
s/whitelist_externals/allowlist_externals/ in all tox.ini in this
project.
projects / ceph.git / log