Nizamudeen A [Fri, 19 Jul 2024 13:44:36 +0000 (19:14 +0530)]
mgr/test_orchestrator: add apply_mds command in the test_orch
so that when we are testing things with test_orchestrator as the orch
back-end and do volume creation, it won't raise the below error
```
2024-07-19T13:40:46.349+0000 7f2d11a00640 0 [test_orchestrator ERROR orchestrator._interface]
Traceback (most recent call last):
File "/ceph/src/pybind/mgr/orchestrator/_interface.py", line 139, in wrapper
return OrchResult(f(*args, **kwargs))
File "/ceph/src/pybind/mgr/orchestrator/_interface.py", line 618, in apply
return raise_if_exception(reduce(merge, [fns[spec.service_type](spec) for spec in specs], OrchResult([])))
File "/ceph/src/pybind/mgr/orchestrator/_interface.py", line 618, in <listcomp>
return raise_if_exception(reduce(merge, [fns[spec.service_type](spec) for spec in specs], OrchResult([])))
File "/ceph/src/pybind/mgr/orchestrator/_interface.py", line 762, in apply_mds
raise NotImplementedError()
NotImplementedError
2024-07-19T13:40:46.349+0000 7f2d11a00640 0 [volumes ERROR volumes.fs.fs_util] Failed to create MDS daemons
Traceback (most recent call last):
File "/ceph/src/pybind/mgr/volumes/fs/fs_util.py", line 55, in create_mds
orchestrator.raise_if_exception(completion)
File "/ceph/src/pybind/mgr/orchestrator/_interface.py", line 242, in raise_if_exception
raise e
NotImplementedError
2024-07-19T13:40:46.350+0000 7f2d11a00640 0 [volumes INFO volumes.module] Finishing _cmd_fs_volume_create(name:foo, placement:1 ) < "None"
2024-07-19T13:40:46.350+0000 7f2d11a00640 0 [dashboard ERROR exception] Internal Server Error
Traceback (most recent call last):
File "/ceph/src/pybind/mgr/dashboard/services/exception.py", line 47, in dashboard_exception_handler
return handler(*args, **kwargs)
File "/usr/lib/python3.9/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
return self.callable(*self.args, **self.kwargs)
File "/ceph/src/pybind/mgr/dashboard/controllers/_base_controller.py", line 263, in inner
ret = func(*args, **kwargs)
File "/ceph/src/pybind/mgr/dashboard/controllers/_rest_controller.py", line 193, in wrapper
return func(*vpath, **params)
File "/ceph/src/pybind/mgr/dashboard/controllers/cephfs.py", line 65, in create
raise RuntimeError(
RuntimeError: Error creating volume foo with placement {'placement': {}, 'unmanaged': False}:
```
Fixes: https://tracker.ceph.com/issues/67044 Signed-off-by: Nizamudeen A <nia@redhat.com>
Redouane Kachach [Fri, 17 May 2024 13:19:35 +0000 (15:19 +0200)]
mgr/cephadm: adding new cephadm service mgmt-gateway
adding mgmt-gateway, a new cephadm service based on nginx, to act as
the front-end and single entry point to the cluster. This gateway
offers unified access to all Ceph applications, including the
Ceph dashboard and monitoring tools (Prometheus, Grafana, ..),
while enhancing security and simplifying access management
through nginx.
```
fedora :: pere/scrapper-replayer/build 130 » ./bin/replayer -h
All options:
Options:
-h [ --help ] produce help message
-i arg (=input.txt) Input file (output of op_scraper.py)
--ceph-conf arg (=ceph.conf) Path to ceph conf
--io-depth arg (=64) I/O depth
--parser-threads arg (=16) Number of parser threads
--worker-threads arg (=16) Number of I/O worker threads
--pool arg (=test_pool) Pool to use for I/O
```
Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>
Afreen Misbah [Sat, 22 Jun 2024 02:18:33 +0000 (07:48 +0530)]
mgr/dashboard: fix service form count bugs
Fixes https://tracker.ceph.com/issues/66608
- for services which do not have a count set default count to be null, otherwise the previous selected service's count is used which is wrong
- make count null when label is selected for placement
Afreen [Fri, 31 May 2024 07:54:27 +0000 (13:24 +0530)]
mgr/dashboard: Introduce NVMe/TCP navigation
Fixes https://tracker.ceph.com/issues/66346
- adds NVMe/TCP tab under Block nav
- adds overview page for NVMe/TCP nav
- overview page lists gateways
- add default error page when no nvmeof service running
- added unit tests
- fixes service page e2e test
authentication.rst described the steps to generate a v2 signature,
without reference to aws docs. replace that with sections that reference
aws docs for v2 and v4 signatures. list which values of the request
header x-amz-content-sha256 are supported for v4
Matt Benjamin [Sat, 22 Jun 2024 20:12:54 +0000 (16:12 -0400)]
rgw_cksum: address review comments
* remove rgw_cksum_pipe state enum, not needed [Casey review]
* remove a format that just took a single string substitution
and passed it to an iostream [Casey review]
* use boost::to_upper* [Casey review]
* remove unused RGW_ATTR_CKSUM_ALGORITHM decl [Casey review]
* negate error code values in two places [Casey review]
* split cksum digests from base type decls
* resolve comment when checksum requested but not available
* remove redundant memset
* remove junk from rgw_blake3_digest.h
* s/ldpp_dout + fmt::format/ldpp_dout_fmt/g;
* fix conditional return of parts_count
from RGWRados::Object::prepare(). A value for parts_count should
be returned iff a *multipart* object manifest exists.
* remove /tmp output test
* finish moving ceph_crypto headers out of rgw_cksum.h
* consume the optional in multipart_parts_count
* target_attrs can be a reference (but not const)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Mon, 15 Apr 2024 20:44:58 +0000 (16:44 -0400)]
rgw_cksum: implement POST upload checksums
* properly transform pseudo headers in PostObj
* enable cksum verify in PostObj
* match checksum headers in match_policy_vars
* fixup add POST headers to environment
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Sun, 5 May 2024 18:27:06 +0000 (14:27 -0400)]
rgw_cksum: fix ReadOp comment
parts_count now returns a value for parts_count whenever the
target object is a multipart upload. this has no additional
overhead, since this can be read off the manifest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Wed, 10 Apr 2024 21:58:15 +0000 (17:58 -0400)]
rgw_cksum: multipart upload checksums
includes commits:
* fixes init-multipart header return
* introduce checksum to SAL MultipartPart interface
* thread optional checksum through DataProcessor
* code complete multipart checksum verify
* fix formatter
* fix ckecksum format for multipart objects in GET/HEAD ops
* always return parts_count from ReadOp::prepare() if applicable
This behavior is used when returning the checksum of a multipart
upload object.
* tweak conditional multipart_parts_count
* add checksum output to ListMultipart
* fix nil-return from GetHeaderCksumResult
* re-arm truncated if re-entering list-parts
* complete-multipart w/list-parts
* validate supplied checksum in CompleteMultipart
* verify checksum type against initial checksum algorithm
* rgw_op: suppress more x-amz headers
* final fixes and cleanups
* remove unused t0
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Tue, 24 Sep 2019 21:12:02 +0000 (17:12 -0400)]
rgw: add checksum and digest machinery
Adds new Blake3 digest format (native), a concrete type to
represent digests, and static_visitor machinery to unify varying
checksum computations.
This framework, together with new trailing checksum header support,
is used to implement S3 additional checksum verification. Parts of
the AWS content checksum API work build on a prior contribution from
imtzw <tongzhiwei_yewu@cmss.chinamobile.com>.
Thank you!
Fixes: https://tracker.ceph.com/issues/42080 Fixes: https://tracker.ceph.com/issues/63951
squashed commits:
* rgw_cksum: add trival test vectors for sha-format digests
Computed digests match those produced by sha1sum, sha256sum,
and sha512sum utilities.
* rgw_cksum: add test vectors for blake3
Tests the same input strings with digests validated by
b3sum (https://crates.io/crates/b3sum).
* rgw_ckum: switch to accel crc32c
The internal Ceph convention appears to be to omit a final
xor where ceph_crc32c is used, but it's required for compatibility
with AWS implementations.
* rgw_cksum: add XXH3 digest
* rgw_cksum: write class encoder for rgw::digest::Cksum
* rgw_cksum: also reverse crc32c (REBASEME)
Mark noticed that the crc32c output was being tested against a
byteswapped value (crc32c also needs byteswap on LE).
* rgw_cksum: add digest::Cksum serde tests
* rgw_cksum: fix main(...) linkage
(so we run our main unit and not the one in gmock
* rgw_cksum: convenience extensions for integration with RGW/S3
* introduce rgw_cksum unique_ptr factory
* rgw_cksum: mark string transform accessors const
* rgw_cksum: fixup unittest_rgw_chksum compilation--all existing tests pass
* rgw_cksum: hook up put-object checksum workflow
* tweaks to report on content checksum mismatch
* rgw_cksum: match SDK as well as general checksum header
* make it more efficient
* initialize RGWPutObj_Cksum::digest
* rgw_cksum: write parse_cksum_type w/const char* arg
* initialize _type correctly; doing armored wrong
* fix expected checksum header name, clean up verify
* fix output on checksum verify fail, cleanup
* introduce Cksum::to_armor(); all AWS cases pass
* oops, extra 0-byte at end of to_armor() result
* use to_armor() with decoded checksums (i.e., for all S3 presentation)
* remove unnecessary finalize() in RGWPutObj_Cksum dtor
* RGWPutObj_Cksum::Factory fixes
* fixes test_object_checksum_sha256
* choose preferred checksum algorithm header if both are present
* log verified checksums in RGWPutObj::execute at 16
* checksum not needed in policy condition
* fix checksum trailing header format
* move Blake3 to rgw_digest_blake3.h
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Adam King [Thu, 27 Jun 2024 21:09:20 +0000 (17:09 -0400)]
mgr/rgw: fix setting rgw realm token in secondary site rgw spec
This was setting a field called "rgw_token" in the rgw spec
but this is not a real field in rgw specs. Instead we should
be setting "rgw_realm_token" which is what the field is
actually called.
Setting this nonexistent field causes the spec to be deleted
the first time cephadm needs to convert it from a json string
back into a python object (which happens whenever the module
restarts or the active mgr changes) which then causes all the
rgw daemons attached to the service to be removed
Fixes: https://tracker.ceph.com/issues/66824 Signed-off-by: Adam King <adking@redhat.com>
Document how to manually pass the search domain to "mon_dns_srv_name" in
doc/rados/configuration/mon-lookup-dns.rst.
This commit is made in response to a request by Lander Duncan that was made on the [ceph-users] mailing list, and can be seen here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/F7V4CWLIYCAJ4JXI2JLNY6QPCFPR4SLA/
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
projects / ceph.git / log