* Controller tests: cherrypy config: authentication disabled by default; ability to pass custom config (e.g. enable authentication).
* Auth controller: add tests; test that unauthorized request fails when authentication is enabled.
* DocsTest: clear ENDPOINT_MAP so the test_gen_tags test becomes deterministic.
* pylint: disable=no-name-in-module: fix imports in tests.
Fixes: https://tracker.ceph.com/issues/53083 Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit a6aeded5141ec3a959a86aa4002ccf5ac8d0a523)
Avan Thakkar [Wed, 20 Oct 2021 14:07:26 +0000 (19:37 +0530)]
mgr/dashboard: gather facts should only be fetched when orch backend is cephadm
Fixes: https://tracker.ceph.com/issues/52981 Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Gather facts in UI should only be fetched if there is orch available and
get_facts feature is available for that orch backend.
Alfonso Martínez [Thu, 26 Aug 2021 10:05:54 +0000 (12:05 +0200)]
mgr/dashboard: NFS exports: API + UI: integration with mgr/nfs; cleanups
mgr/dashboard: move NFS_GANESHA_SUPPORTED_FSALS to mgr_module.py
Importing from nfs module throws AttributeError because as a side effect the dashboard module is impersonating the nfs module.
https://gist.github.com/varshar16/61ac26426bbe5f5f562ebb14bcd0f548
mgr/dashboard: 'Create NFS export' form: list clusters from nfs module
mgr/dashboard: frontend+backend cleanups for NFS export
Removed all code and references related to daemons. UI cleanup and adopted unit-testing for
nfs-epxort create form for CEPHFS backend. Cleanup for export list/get/create/set/delete endpoints.
mgr/dashboard: rm set-ganesha ref + update docs
Remove existing set-ganesha-clusters-rados-pool-namespace references as
they are no longer required. Moreover, nfs doc in dashboard doc is
updated accordingly to the current nfs status.
mgr/dashboard: add nfs-export e2e test coverage
mgr/dashboard: 'Create NFS export' form: remove RGW user id field.
- Improve bucket typeahead behavior.
- Increase version for bucket list endpoint.
- Some refactoring.
mgr/dashboard: 'Create NFS export' form: allow RGW backend only when default realm is selected.
When RGW multisite is configured, the NFS module can only handle buckets in the default realm.
mgr/dashboard: 'Create service' form: fix NFS service creation.
After https://github.com/ceph/ceph/pull/42073, NFS pool and namespace are not customizable.
- Allow only existing buckets.
- Refactoring:
- Moved bucket validator from bucket form to cd-validators.ts
- Split bucket validator into 2: bucket name validator and bucket existence (that checks either existence or non-existence).
None of these values are required for listing nfs cluster by mgr/nfs module.
Instead directly list available cluster names
mgr/dashboard: add comment to remove listing of daemons
As the configs are per cluster. There is no need to list daemons per cluster.
mgr/dashboard/controllers/nfsganesha: Add comments to update/remove status endpoint
This endpoint can be updated in suggested way or even removed. As it was
initially[1] introduced to check if dashboard pool and namespace configuration was
set.
Varsha Rao [Mon, 16 Aug 2021 05:39:18 +0000 (11:09 +0530)]
mgr/nfs: don't log fsal keys
FSAL keys are written to export objects. Don't log it, instead log the initial
export before generating keys and user config object mostly will not contain
export block. It's okay to log user config object content.
Sage Weil [Wed, 28 Jul 2021 14:29:47 +0000 (10:29 -0400)]
mgr/dashboard: consume mgr/nfs via mgr.remote()
Stop using the dashboard version of the Ganesha config classes; consume
mgr/nfs instead via remote().
mgr/nfs/export: return Export from _apply_export
Future callers will want this.
mgr/nfs: new module methods for dashboard consumption
Add some new methods that are easy for the dashboard API to consume. These
are very similar to the CLI methods but do now have the @CLICommand and
related decorators, and have slightly different interfaces (e.g., returning
the created/modified Export dict).
mgr/dashboard: remove old ganesha code (and tests)
Nizamudeen A [Mon, 25 Oct 2021 07:25:35 +0000 (12:55 +0530)]
mgr/dashboard: Proper error handling in module status guard
I've recently introduced a check that verifies if the orch_backend we need for a feature matches the orch_backend of the cluster. But to verify this I need to call configOpt API and some user's don't have the permission to access it. So it'll show this Access Denied page.
[lint:tsc ] cypress/integration/orchestrator/01-hosts.e2e-spec.ts(29,13): error TS2339: Property 'clickHostTab' does not exist on type 'HostsPageHelper'.
also change "host" to "hostname" to be more consistent
1. The device preview disappearing when going to next step and coming back to the previous step
2. Even when clearing the device preview, the Storage Capacity count and the drive group spec doesn't get cleared.
3. Expanding the cluster without selecting any devices gives a 400
error.
4. Renamed "Delete Host" to "Remove Host"
5. Generalizing most of the sub component code
Fixes: https://tracker.ceph.com/issues/52499 Fixes: https://tracker.ceph.com/issues/51991 Signed-off-by: Nizamudeen A <nia@redhat.com> Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit b914f59ff123b9e006d7d11dc786e967c0271dbf)
Nizamudeen A [Sun, 4 Jul 2021 13:16:45 +0000 (18:46 +0530)]
mgr/dashboard: Cluster Creation Add Host Section and e2es
Add host section of the cluster creation workflow.
1. Fix bug in the modal where going forward one step on the wizard and coming back opens up the add host modal.
2. Rename Create Cluster to Expand Cluster as per the discussions
3. A skip confirmation modal to warn the user when he tries to skip the
cluster creation
4. Adapted all the tests
5. Did some UI improvements like fixing and aligning the styles,
colors..
- Used routed modal for host Additon form
- Renamed the Create to Add in Host Form
Avan Thakkar [Tue, 1 Jun 2021 12:55:15 +0000 (18:25 +0530)]
mgr/dashboard: Create Cluster Workflow welcome screen and e2e tests
A module option called CLUSTER_STATUS has two option. INSTALLED
AND POST_INSTALLED. When CLUSTER_STATUS is INSTALLED it will allow to show the
create-cluster-wizard after login the initial time. After the cluster
creation is succesfull this option is set to POST_INSTALLED
Also has the e2e codes for the Review Section
Fixes: https://tracker.ceph.com/issues/50336 Signed-off-by: Avan Thakkar <athakkar@redhat.com> Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit b9f38cadc46338c9f1527beeaaba3d42639badf9)
Nizamudeen A [Thu, 30 Sep 2021 07:47:15 +0000 (13:17 +0530)]
mgr/dashboard: Taking care of deprecation warnings and other errors
1. deprecation group is deprecated: This API is not typesafe and can result in issues with Closure Compiler renaming. Use the `FormBuilder#group` overload with `AbstractControlOptions` instead.
2. cypress/integration/rgw/users.po.ts(113,23): error TS2345: Argument of type 'void' is not assignable to parameter of type 'string'.
3. Cannot find module 'isomorphic-form-data' from 'node_modules/swagger-ui/dist/swagger-ui.js'
Nizamudeen A [Tue, 29 Jun 2021 09:30:34 +0000 (15:00 +0530)]
mgr/dashboard: Angular v11 Migration
Fixes: https://tracker.ceph.com/issues/50268 Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit de2648501838891144a1cd85dc52527e1a28726f)
Conflicts
src/pybind/mgr/dashboard/frontend/package.json
- Resolved the conflicts
src/pybind/mgr/dashboard/frontend/package-lock.json
- Generated a new package-lock.json file
Kefu Chai [Sat, 30 Oct 2021 03:18:17 +0000 (11:18 +0800)]
admin/doc-requirements.txt: pin Sphinx at 3.5.4
* pin Sphinx at 3.5.4
* pin docutils at 0.18
at least the combination of these two versions
is known to compile.
to address the bug reported at
https://sourceforge.net/p/docutils/bugs/431/
the backtrace looks like:
/home/jenkins-build/build/workspace/ceph-pr-docs/build-doc/virtualenv/lib/python3.8/site-packages/sphinx/util/docutils.py:285:
RemovedInSphinx30Warning: function based directive support is now
deprecated. Use class based directive instead.
warnings.warn('function based directive support is now deprecated. '
Exception occurred:
File
"/home/jenkins-build/build/workspace/ceph-pr-docs/build-doc/virtualenv/lib/python3.8/site-packages/docutils/writers/html5_polyglot/__init__.py",
line 445, in section_title_tags
if (ids and self.settings.section_self_link
AttributeError: 'Values' object has no attribute 'section_self_link'
Nathan Cutler [Wed, 20 Oct 2021 10:51:02 +0000 (12:51 +0200)]
rgw/tracing: unify SO version numbers within librgw2 package
The librgw2 package contains several SO files. Two of those - librgw_op_tp.so
and librgw_rados_tp.so - had a different version number than the main librgw.
This was a violation of the openSUSE Shared Library Packaging Policy [1] but it
also seems like a "violation" of common sense.
* APIVersion:
* Moved to a separate file
* Added doctests
* Added sentinel values:
* DEFAULT = 1.0
* EXPERIMENTAL = 0.1
* NONE = 0.0
* Added to_mime_type() helper method
* Controllers.__init__:
* Added type hints
* Replaced string versions with APIVersions
* Feedback controller:
* Replaced with EXPERIMENTAL (probably it should be NONE)
Fixes: https://tracker.ceph.com/issues/52480 Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
Conflicts:
src/pybind/mgr/dashboard/controllers/__init__.py
- Remove the current changes and keep the incoming new changes
src/pybind/mgr/dashboard/controllers/crush_rule.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/controllers/docs.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/controllers/feedback.py
- Deleted the file since feedback module isn't backported to pacific
src/pybind/mgr/dashboard/controllers/host.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/openapi.yaml
- Generated a new openapi yaml file
src/pybind/mgr/dashboard/tests/__init__.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_docs.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_host.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_tools.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_versioning.py
- Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/controllers/crush_rule.py
- Removed the MethodMap decorator which updates the version of the
enpoint to 2.0 because those changes which caused that version
updating were not backported to pacific
Patrick Donnelly [Tue, 14 Sep 2021 17:02:12 +0000 (13:02 -0400)]
test/libcephfs: put inodes after lookup
Otherwise, the client umount will hang due to inability to trim the
inodes looked up using the low-level interface. This results in slow-op
warnings and an eviction:
2021-09-11T17:23:31.097+0000 7f99c3522700 0 log_channel(cluster) log [WRN] : evicting unresponsive client smithi176 (9756), after 303.924 seconds
2021-09-11T17:23:31.097+0000 7f99c3522700 10 mds.0.server autoclosing stale session client.9756 172.21.15.176:0/3891214934 last renewed caps 303.924s ago
mgr/dashboard: make modified API endpoints backward compatible
Fixes: https://tracker.ceph.com/issues/52480 Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Introducing APIVersion class to handle versioning for API-endpints and making
them backward compatible.
The test is failing on deleting a host because the agent daemon is
present in that host. Its not possible to simply delete a host. We need
to drain it first and then delete it.
where the numbers of scrubbed object, clones, dirty and omap are always
less than the total number of corresponding numbers, if the PG contains
object(s) whose hash happens to be 0xffffffff.
in this change, if the calculated hash of the upper bound is greater
than the maximum possible number represented by uint32_t, in addition to
setting the hash of the upper bound hobj to 0xffffffff, we also set the
nspace of hobj of the upper bound to "\xff", so that the upper bound
is greater than an hobj whose hash happens to be 0xfffffff. please note,
the nspace of "\xff" is not an ascii string, so it's not likely to be
less than a real-world nspace of an hobj.
with this new *greater* upper bound, we are able to include the previous
missing hobj when listing the objects in a PG. so the scrub won't be
annoyed when the number of objects does not match.
Mykola Golub [Mon, 30 Aug 2021 06:58:04 +0000 (07:58 +0100)]
osd: re-cache peer_bytes on every peering state activate
peer_bytes is used for backfill reservation request and may be
reset if backfill is interrupted, and we want it set back before
continuing backfill and re-sending the reservation request.