mon: fix mon_keyvaluedb application

mon: fix mon_keyvaluedb application

In 42a6b0efe1a3269c8e6c10e89f92bc0f28923af2 I mistakenly thought that
create_or_open was the path taken for normal open (not during mkfs) and
made it assume a missing kv_type meant leveldb.  In reality, this is the
only path where mon_keyvaluedb is ever used (during mkfs), and
create_or_open is only called during mkfs.  The bug I was (probably?)
trying to fix was that regular open() did not write out a kv_type file
(with the assumption of leveldb) if it was missing.  As a result, the
previous fix was forcing all mons to be leveldb.

Fix this by reverting the create_or_open hunk (so that we express
mon_keyvaluedb on mkfs), and fixing the normal open path to write kv_type
if it is missing.  This effectively switches the mon back to rocksdb by
default (and allows teuthology to test both rocksdb and leveldb by setting
the option).

Signed-off-by: Sage Weil <sage@redhat.com>