From c068fa130371beb411a3adc4af7e90fb84d9190d Mon Sep 17 00:00:00 2001 From: Joshua Schmid Date: Thu, 7 Nov 2019 10:33:26 +0100 Subject: [PATCH] mgr/ssh: move and adapt documentation for ssh-orchestrator Signed-off-by: Joshua Schmid --- doc/mgr/ssh.rst | 6 --- src/pybind/mgr/ssh/HACKING.rst | 90 ++++++++++++++++++++++++++++++++ src/pybind/mgr/ssh/README.md | 93 ---------------------------------- 3 files changed, 90 insertions(+), 99 deletions(-) create mode 100644 src/pybind/mgr/ssh/HACKING.rst delete mode 100644 src/pybind/mgr/ssh/README.md diff --git a/doc/mgr/ssh.rst b/doc/mgr/ssh.rst index 1d1e96631bd..1b904061451 100644 --- a/doc/mgr/ssh.rst +++ b/doc/mgr/ssh.rst @@ -37,9 +37,3 @@ To clear this value use the command: :: # ceph ssh clear-ssh-config - -Development ------------ - -Instructions for setting up a development environment can be found in the Ceph -source tree at `src/pybind/mgr/ssh/README.md`. diff --git a/src/pybind/mgr/ssh/HACKING.rst b/src/pybind/mgr/ssh/HACKING.rst new file mode 100644 index 00000000000..7987bab456f --- /dev/null +++ b/src/pybind/mgr/ssh/HACKING.rst @@ -0,0 +1,90 @@ +Development +----------- + + +There are multiple ways to set up a development environment for the SSH orchestrator. +In the following I'll use the `vstart` method. + +1) Make sure remoto is installed (0.35 or newer) + +2) Use vstart to spin up a cluster + + +:: + + # ../src/vstart.sh -n --ssh + +*Note that when you specify `--ssh` you have to have passwordless ssh access to localhost* + +It will add your ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub to `mgr/ssh/ssh_identity_{key, pub}` +and add your $HOSTNAME to the list of known hosts. + +This will also enable the ssh mgr module and enable it as the orchestrator backend. + +*Optional:* + +While the above is sufficient for most operations, you may want to add a second host to the mix. +There is `Vagrantfile` for creating a minimal cluster in `src/pybind/mgr/ssh/`. + +If you wish to extend the one-node-localhost cluster to i.e. test more sophisticated OSD deployments you can follow the next steps: + +From within the `src/pybind/mgr/ssh` directory. + + +1) Spawn VMs + +:: + + # vagrant up + +This will spawn three machines. +mon0, mgr0, osd0 + +NUM_DAEMONS can be used to increase the number of VMs created. (defaults to 1) + +If will also come with the necessary packages preinstalled as well as your ~/.ssh/id_rsa.pub key +injected. (to users root and vagrant; the SSH-orchestrator currently connects as root) + + +2) Update the ssh-config + +The SSH-orchestrator needs to understand how to connect to the new node. Most likely the VM isn't reachable with the default settings used: + +``` +Host * +User root +StrictHostKeyChecking no +``` + +You want to adjust this by retrieving an adapted ssh_config from Vagrant. + +:: + + # vagrant ssh-config > ssh-config + + +Now set the newly created config for Ceph. + +:: + + # ceph ssh set-ssh-config -i + + +3) Add the new host + +Add the newly created host(s) to the inventory. + +:: + + + # ceph orchestrator host add + + +4) Verify the inventory + +:: + + # ceph orchestrator host ls + + +You should see the hostname in the list. diff --git a/src/pybind/mgr/ssh/README.md b/src/pybind/mgr/ssh/README.md deleted file mode 100644 index 10f268cd278..00000000000 --- a/src/pybind/mgr/ssh/README.md +++ /dev/null @@ -1,93 +0,0 @@ -# dev environment setup - -1. start vms with _only_ the ceph packages installed - -In `src/pybind/mgr/ssh` run `vagrant up` to create a cluster with a monitor, -manager, and osd nodes. The osd node will have two small extra disks attached. - -2. generate an `ssh_config` file for the vm hosts - -Execute `vagrant ssh-config > /path/to/ssh_config` to generate a ssh -configuration file that contains hosts, usernames, and keys that will be used by -the bootstrap cluster / ssh orchestrator to establish ssh connections to the -vagrant vms. - -3. install ssh orchestrator dependencies - -The primary dependency is the `remoto` package that contains a Python SSH client -for connecting to remote nodes and executing commands. - -Install with `dnf install python3-remoto`. The version must be >= 0.0.35. At the -time of writing this version is being packaged and is not available. To install -from source: - -``` -git clone https://github.com/ceph/remoto -cd remoto -python3 setup.py sdist -pip3 install --prefix=/usr dist/remoto-0.0.35.tar.gz -``` - -4. start the bootstrap cluster (in this case a `vstart.sh` cluster) - -Start with a network binding to which the vms can route traffic: - - `vstart.sh -n -i 192.168.121.1` - -The following is a manual method for finding this address. TODO: documenting a -automated/deterministic method would be very helpful. - -First, ensure that your firewall settings permit each VM to communicate with the -host. On Fedora, the `trusted` profile is sufficient: `firewall-cmd ---set-default-zone trusted` and also allows traffic on Ceph ports. Then ssh into -one of the vm nodes and ping the default gateway, which happens to be setup as -the host machine. - -``` -[nwatkins@smash ssh]$ vagrant ssh mon0 -c "getent hosts gateway" -192.168.121.1 gateway -``` - -5. setup the ssh orchestrator backend - -Enable and configure the ssh orchestrator as the active backend: - -``` -ceph mgr module enable ssh -ceph orchestrator set backend ssh - -# optional: this document assumes the orchestrator CLI is enabled -ceph mgr module enable orchestrator_cli -``` - -Configure the ssh orchestrator by setting the `ssh_config` option to point at -the ssh configuration file generated above: - -``` -ceph config set mgr mgr/ssh/ssh_config_file /path/to/config -``` - -The setting can be confirmed by retrieving the configuration settings: - -``` -[nwatkins@smash build]$ ceph config get mgr. -WHO MASK LEVEL OPTION VALUE RO -mgr advanced mgr/orchestrator_cli/orchestrator ssh * -mgr advanced mgr/ssh/ssh_config_file /home/nwatkins/src/ceph/src/pybind/mgr/ssh/config * -``` - -An SSH config file can also be provided through standard input that avoids the -need to have an accessible file path. Use the following command: - - -``` -ceph ssh set-ssh-config -i -``` - -The next set of instructions we should move to the docs folder - -ceph orchestrator host add osd0 -ceph orchestrator host add mgr0 -ceph orchestrator host add mon0 -ceph orchestrator device ls -ceph orchestrator mgr update 3 mgr0 mgr1 -- 2.39.5