--- /dev/null
+Development
+-----------
+
+
+There are multiple ways to set up a development environment for the SSH orchestrator.
+In the following I'll use the `vstart` method.
+
+1) Make sure remoto is installed (0.35 or newer)
+
+2) Use vstart to spin up a cluster
+
+
+::
+
+ # ../src/vstart.sh -n --ssh
+
+*Note that when you specify `--ssh` you have to have passwordless ssh access to localhost*
+
+It will add your ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub to `mgr/ssh/ssh_identity_{key, pub}`
+and add your $HOSTNAME to the list of known hosts.
+
+This will also enable the ssh mgr module and enable it as the orchestrator backend.
+
+*Optional:*
+
+While the above is sufficient for most operations, you may want to add a second host to the mix.
+There is `Vagrantfile` for creating a minimal cluster in `src/pybind/mgr/ssh/`.
+
+If you wish to extend the one-node-localhost cluster to i.e. test more sophisticated OSD deployments you can follow the next steps:
+
+From within the `src/pybind/mgr/ssh` directory.
+
+
+1) Spawn VMs
+
+::
+
+ # vagrant up
+
+This will spawn three machines.
+mon0, mgr0, osd0
+
+NUM_DAEMONS can be used to increase the number of VMs created. (defaults to 1)
+
+If will also come with the necessary packages preinstalled as well as your ~/.ssh/id_rsa.pub key
+injected. (to users root and vagrant; the SSH-orchestrator currently connects as root)
+
+
+2) Update the ssh-config
+
+The SSH-orchestrator needs to understand how to connect to the new node. Most likely the VM isn't reachable with the default settings used:
+
+```
+Host *
+User root
+StrictHostKeyChecking no
+```
+
+You want to adjust this by retrieving an adapted ssh_config from Vagrant.
+
+::
+
+ # vagrant ssh-config > ssh-config
+
+
+Now set the newly created config for Ceph.
+
+::
+
+ # ceph ssh set-ssh-config -i <path_to_ssh_conf>
+
+
+3) Add the new host
+
+Add the newly created host(s) to the inventory.
+
+::
+
+
+ # ceph orchestrator host add <host>
+
+
+4) Verify the inventory
+
+::
+
+ # ceph orchestrator host ls
+
+
+You should see the hostname in the list.
+++ /dev/null
-# dev environment setup
-
-1. start vms with _only_ the ceph packages installed
-
-In `src/pybind/mgr/ssh` run `vagrant up` to create a cluster with a monitor,
-manager, and osd nodes. The osd node will have two small extra disks attached.
-
-2. generate an `ssh_config` file for the vm hosts
-
-Execute `vagrant ssh-config > /path/to/ssh_config` to generate a ssh
-configuration file that contains hosts, usernames, and keys that will be used by
-the bootstrap cluster / ssh orchestrator to establish ssh connections to the
-vagrant vms.
-
-3. install ssh orchestrator dependencies
-
-The primary dependency is the `remoto` package that contains a Python SSH client
-for connecting to remote nodes and executing commands.
-
-Install with `dnf install python3-remoto`. The version must be >= 0.0.35. At the
-time of writing this version is being packaged and is not available. To install
-from source:
-
-```
-git clone https://github.com/ceph/remoto
-cd remoto
-python3 setup.py sdist
-pip3 install --prefix=/usr dist/remoto-0.0.35.tar.gz
-```
-
-4. start the bootstrap cluster (in this case a `vstart.sh` cluster)
-
-Start with a network binding to which the vms can route traffic:
-
- `vstart.sh -n -i 192.168.121.1`
-
-The following is a manual method for finding this address. TODO: documenting a
-automated/deterministic method would be very helpful.
-
-First, ensure that your firewall settings permit each VM to communicate with the
-host. On Fedora, the `trusted` profile is sufficient: `firewall-cmd
---set-default-zone trusted` and also allows traffic on Ceph ports. Then ssh into
-one of the vm nodes and ping the default gateway, which happens to be setup as
-the host machine.
-
-```
-[nwatkins@smash ssh]$ vagrant ssh mon0 -c "getent hosts gateway"
-192.168.121.1 gateway
-```
-
-5. setup the ssh orchestrator backend
-
-Enable and configure the ssh orchestrator as the active backend:
-
-```
-ceph mgr module enable ssh
-ceph orchestrator set backend ssh
-
-# optional: this document assumes the orchestrator CLI is enabled
-ceph mgr module enable orchestrator_cli
-```
-
-Configure the ssh orchestrator by setting the `ssh_config` option to point at
-the ssh configuration file generated above:
-
-```
-ceph config set mgr mgr/ssh/ssh_config_file /path/to/config
-```
-
-The setting can be confirmed by retrieving the configuration settings:
-
-```
-[nwatkins@smash build]$ ceph config get mgr.
-WHO MASK LEVEL OPTION VALUE RO
-mgr advanced mgr/orchestrator_cli/orchestrator ssh *
-mgr advanced mgr/ssh/ssh_config_file /home/nwatkins/src/ceph/src/pybind/mgr/ssh/config *
-```
-
-An SSH config file can also be provided through standard input that avoids the
-need to have an accessible file path. Use the following command:
-
-
-```
-ceph ssh set-ssh-config -i <path to ssh_config>
-```
-
-The next set of instructions we should move to the docs folder
-
-ceph orchestrator host add osd0
-ceph orchestrator host add mgr0
-ceph orchestrator host add mon0
-ceph orchestrator device ls
-ceph orchestrator mgr update 3 mgr0 mgr1
projects / ceph.git / commitdiff