pam: C implementation for conversation and cleanup
This commit adds in a C implementation for the pam_conv we will use in
login.go as well as adding three CleanupFuncs that will be used with
pam_set_data(). It also adds copyInfoSecret() which should be paired
with freeSecret().
Now that we can distinguish between lacking encryption support and
lacking fscrypt metadata, "fscrypt status" can now display this
additional information.
Almost all actions only need to to check that the fscrypt metadata
exists (this is handled by the Mount methods). Only "fscrypt encrypt"
need to be sure the filesystem also supports encryption, so this check
is added.
This commit splits two pieces of functionality. Detecting if the fscrypt
metadata exists is now in CheckSetup() and checking if the filesystem
supports encryption is now in CheckSupport().
This commit changes all the internal import paths from `fscrypt/foo` to
`github.com/google/fscrypt/foo` so that it can be built once we release
externaly. The documentation in README.md is updated accordingly.
Also, the README has a note noting that we do not make any guarantees
about project stability before 1.0 (when it ships with Ubuntu).
This commit updates the README and Makefile to get them ready for
external release. This includes adding some common pitfalls, including
example usage, and allowing for tarball creation.
This commit adds in the "fscrypt metadata add-protector-to-policy" and
the "fscrypt metadata remove-protector-from-policy" subcommands. These
commands allow for the creating of policies protected by multiple
protectors.
This command adds in the "fscrypt metadata" command. This command allows
advanced users to manipulate the metadata directly instead of just
creating a policy or protector as an option when encrypting a directory.
As some of these methods will require certain flags, error handling for
this case is also added. As the change passphrase method must indicate
when a old vs new password is necessary, additional KeyFuncs are added
which add this indicator.
This commit adds in the status command, which has 2 functions, allowing
the user to query the state of the entire system or a specific
filesystem.
This commit also adds in the purge command to remove all policy keys
corresponding to a filesystem. This (along with getting the unlock
status for the status commands) uses additional keyctl functionality in
the crypto and actions packages.
This commit adds in the framework for adding commands and subcommands to
the fscrypt tool. This commit adds in the "setup", "encrypt", and
"unlock" commands. Additional information can be found by running:
fscrypt <command> --help.
This commit defines how flags are parsed and errors are handled. It also
creates an extensible framework for prompting the user for information.
This commit changes the error handling for the actions package to use
the error handling library github.com/pkg/errors. This means replacing
"errors" with "github.com/pkg/errors", reworking some of the error
values, and wrapping some errors with additional context.
This commit also changes the Protector/Policy API, moving most of the
package functionality into Protector or Policy methods. These types are
now "locked" when they are queried from the filesystem, and Unlock()
must be used to get their corresponding keys. Note that only certain
operations will require unlocking the keys. Certain unnecessary
functions and methods are also removed.
This CL also fixes two bugs reported by Tyler Hicks in CreateConfigFile.
CPU time is used instead of wall time, and kiB is used instead of kB.
This commit changes the error handing for the crypto, filesystem,
metadata, pam, and util packages to use the error handling library
github.com/pkg/errors. This means elimination of the FSError type, an
increased use of wrapping errors (as opposed to logging), switching
on the Cause() of an error (as opposed to its value), and improving our
integration tests involving TEST_FILESYSTEM_ROOT.
This commit also fixes a few bugs with the keyring code to ensure that
our {Find|Remove|Insert}PolicyKey functions are always operating on the
same keyring. The check for filesystem support has been moved from the
filesystem package to the metadata package. Finally, the API for the
filesystem package has been slightly modified:
* filesystem.AllFilesystems() now returns all the filesystems in
sorted order
* certain path methods are now public
O_SYNC is also removed for writing the metadata. We don't get that much
from syncing the metadata, as the actual file data could also be
corrupted by and IO error. The sync operation is also occasionally very
slow (~3 seconds) and can be unfriendly to battery life.
This commit adds in the github.com/pkg/errors package for handling
errors. This allows us to continue to add context to our errors as they
move up the call stack, but still be able to switch on their root cause.
This commit makes the callbacks for getting keys easier to understand.
Functions which need keys now take a KeyFunc callback. This callback
contains a ProtectorInfo parameter (basically a read-only version of
metadata.ProtectorData) and a boolean which indicates if the call is
being retried. The documentation is also updated to say which functions
will retry the KeyFunc.
For selecting a protector, there is now an OptionFunc callback which
takes a slice of ProtectorOptions. A ProtectorOption is a ProtectorInfo
along with additional information about a linked filesystem (if
applicable).
This commit also adds in methods for getting the protector options for a
specific filesystem or policy. It also adds a function for getting the
policy descriptor for a specific path.
This commit updates the vendored x/sys/unix to the current version. This
means that our vendored version of x/sys/unix no longer contains custom
patches. A few changes are needed in InsertPolicyKey() as the final API
differed slightly from our patches.
This commit adds in the FindPolicyKey and RemovePolicyKey functions to
complement the InsertPolicyKey function. The existing functions were
also refactored slightly.
As new encryption modes are being added to the kernel that use 128 bit
keys (see https://patchwork.kernel.org/patch/9741913), we will need the
encryption modes to be more descriptive.
This change breaks backwards compatibility for the protobuf, but that's
fine because we have not released yet.
filesystem: change support detection and bug-fixes
Instead of checking if the filesystem type is correct, we now detect if
a filesystem supports encryption by trying to read a policy on its root
directory. The error returned tells us if there is support or not.
This commit also fixes a bug in the use libblkid. Throughout all of
fscrypt, cannonicalizePath() is used before any path comparison or
lookup. However, the canonical device path in the blkid cache may differ
from our idea of a canonical path. Additional blkid functions are needed
to perform the necessary translation. This is noted in the documentation
of makeLink().
Finally, this commit makes a few API changes. AllSupporedFilesystems()
now returns an error, and a GetProtector() method now replaces the
GetLinkedProtector() and GetEitherProtector() methods. A PathSorter has
also been added so Mounts can be sorted in a reliable order.
This commit adds in the fscrypt/pam package. This package will hold all
functionality related to Linux Pluggable Authentication Modules (PAM).
Right now this package uses cgo to mock a PAM conversation, allowing the
function to check if a provided passphrase actually belongs to a user.
Due to the nature of cgo callbacks, global state of the key to check is
necessary for this function. This commit also addresses some issues
about building the cgo components. Now, only the minimal linking flags
are included in the go files. Additional linker flags may now be
necessary to build a static binary of fscrypt. This is addressed in the
Makefile and README.
Finally, this commit fixes a bug where the tests would not run correctly
due to shared global state on the testing filesystem. Fixed, by having
all the tests run sequentially.
crypto/ssh/terminal gets us information about the terminal, such as
its dimensions and state. It also allows us to manipulate the terminal.
For example, we can put the terminal in raw mode when a passphrase needs
to be entered.
This commit also add the github.com/urfave/cli package which we will use
to write the command line tool component of fscrypt. This tool allows
for us to easily use commands and subcommands.
Note that this is actually the upstream repository with two PRs applied:
This commit adds in the Policy structure. This structure represents an
unlocked policy key and its associated data. Policies can add or remove
Protectors, apply encryption policies to filesystem directories, and
provision a key into the kernel keyring.
This commit adds in the Protector struct to the actions package. This
struct represents an unlocked Protector. They can be created from a
context or they can be unlocked using some provided data. In either
case, the data is provided via a callback mechanism.
This commit adds in the actions package. This package will be the
highest-level interface to the fscrypt packages. The public functions
in this package will be called directly from cmd/fscrypt.
The actions added in this commit pertain to creating and reading the
fscrypt global config file "fscrypt.conf". The challenging part about
creating this file is finding the correct hashing parameters for the
desired time target.
The getHashingCosts() function finds the desired costs by doubling the
costs and running the passphrase hash until the target is exceeded.
Then, a cost estimate is obtained using a linear interpolation between
the last two costs (and their time results).
This commit adds in the filesystem subpackage. The goal of this package
is to provide and interface for adding to and removing from the metadata
storage for a given filesystem. This is primarily done in filesystem.go.
To facilitate this functionality, mountpoint.go exposes an interface for
querying the system about the current mounted filesystems and their
information. Note that this operation is done with a lazy loading
mechanism.
To refer to other filesystems, we use link files that can be parsed by
libblkid. The README is also updated to account for this new dependancy.
This package uses the FSError type under the hood so that error messages
will include the filesystem name, but callers can still check for
specific error instances.
This changes the crypto package so it now builds in light of the changes
to the util and metadata package. This commit also improves the error
handling, adds tests, and makes it so recovery keys now correspond to
Policy keys (as they are used to recover a directory in the absence of
any metadata).
The only feature addition here is the ability to compute descriptors.
For backwards compatibility, we keep the same descriptor algorithm used
before (double SHA512).
This commit adds in IsValid() checks for the metadata structures that
let us enforce stronger invariants than those imposed by the protobuf
package. The main uses of this will be to check that metadata is valid
before writing it to the filesystem, and to check that the filesystem
contains valid metadata before returning it to the user. These functions
also will log the exact reason if the validity checks fail.
To have these checks in the metadata package, all of the various
constants have been moved to a single metadata/constants.go file. The
uses of these constants were changed accordingly.
Finally, this commit standardizes our use of errors so that they always
begin with an appropriate prefix.
This commit changes how we handle InvalidInput and System errors.
Instead of having formatting functions, the now just wrap a string and
should be created with fmt.Sprintf or similar. We also move all of the
error related code into a single place.
Finally, the utils package gets additional functionality with MinInt64
and GetUsername, and the UnderlyingError function gets better logging.
Note that this will break packages that depend on it. For instance,
metadata and crypto currently do not build. This is fixed in a later
commit.
This commit moves most of the documentation about contributing to
fscrypt into CONTRIBUTING.md and updates the legal disclaimer.
It also updates the README.md to include all of fscrypt's planned
functionality and dependencies. Finally, the makefile is updated to
include more documentation, versioning support, and a different location
for the output file.
Joe Richey [Fri, 3 Mar 2017 01:32:50 +0000 (17:32 -0800)]
crypto: passphrase hashing with Argon2
This commit adds in the PassphraseHash function which hashes the
provided passphrase (in key form) using Argon2id. This cost parameters
for Argon2id and that salt are both fed into the function. It also
includes tests and benchmarks for the passphrase hashing.
Joe Richey [Thu, 2 Mar 2017 22:01:20 +0000 (14:01 -0800)]
crypto: reading and writing recovery keys
This commit adds in the concept of recovery codes: human-readable
strings that contain the necessary information to rederive a
cryptographic key. These keys look like:
73PZBXVP-DKJX7SKV-NNTFIC7A-QEGRPZUX-4K5ORRH2-MTKMKP3B-HFCA====
They are input or output directly to a io.Reader or io.Writer
respectively. This prevents the data from passing through unsecured
memory before it gets to its destination. Of course, if the provided
io.Reader or io.Writer is insecure, there is nothing we can do. In most
cases the provided io.Reader or io.Writer will be stdin or stdout. In
some rare cases you might want to pipe the output to another key.
This commit also adds tests and benchmarks for encoding/decoding
recovery codes. It also tests that encoding/decoding will fail in the
correct situations. A benchmark is also added to measure the effect of
locking the keys in memory.
Joe Richey [Thu, 2 Mar 2017 19:58:07 +0000 (11:58 -0800)]
crypto: secure key wrapping/unwrapping
This commit adds in the ability to use the WrappedKeyData from the
metadata package to wrap and unwrap cryptographic keys of any length.
This makes use of several cryptographic primitives:
- Unsalted, SHA256-based HKDF for key stretching
- AES256 in CTR mode for encryption
- SHA256-based HMAC for authentication
Note that the key wrapping/unwrapping uses an "Encrypt then MAC" scheme
for doing authenticated unwrapping. This means we can detect if bogus
metadata has been given. This package also standardizes the length for
fscrypt's internal keys.
This CL is the first to add benchmarks, which can be run with:
go test -bench=. ./...
This commit adds in the golang.org/x/crypto/hkdf package which contains
the HMAC-based Extract-and-Expand Key Derivation Function (HKDF) as
defined in RFC 5869.
This package will be used later in the crypto package to stretch a
single cryptographic key into multiple keys.
Joe Richey [Thu, 2 Mar 2017 19:58:07 +0000 (11:58 -0800)]
crypto: add secure random reader using getrandom
This commit adds in RandReader, a cryptographically secure io.Reader
that will fail when the os has insufficient randomness. This is done
using the getrandom() syscall in non-blocking mode.
see: http://man7.org/linux/man-pages/man2/getrandom.2.html
Any kernel new enough to have filesystem encryption will also have this
syscall.
This RandReader is preferable to the one provided by the standard
library in crypto/rand. See the bugs:
https://github.com/golang/go/issues/11833
https://github.com/golang/go/issues/19274
This will be removed when go updates the crypto/rand implementation.
Joe Richey [Thu, 2 Mar 2017 19:47:07 +0000 (11:47 -0800)]
crypto: insert key into keyring from go
This commit adds in the ability to insert Keys into the kernel keyring
from go code. This is done via a patched version of x/sys/unix. We
also expose the specific requirements for keys that will be placed in
the keyring, namely PolicyKeyLen. The legacy services are also exposed.
Joe Richey [Thu, 2 Mar 2017 19:22:43 +0000 (11:22 -0800)]
crypto: Key struct for secure buffers
This commit adds in the crypto package, which will hold all
of the security primitives for fscrypt. This first component deals with
securely handling keys in memory. To do this in a consistent way across
fscrypt, we introduce the Key struct.
Any sensitive memory (like keys, passwords, or recovery tokens) in
fscrypt will be held in a Key. No code outside of the crypto package
should access the Key's data directly. Convenience functions and methods
are provided to construct keys from io.Readers (either with fixed length
or with variable length) and to access information about the Keys.
The most important property of Keys is that the data is locked in memory
on construction, and the data is unlocked and wiped when Wipe is called.
This happens either by something like "defer key.Wipe()" or through the
finalizer.
Joe Richey [Thu, 2 Mar 2017 18:38:33 +0000 (10:38 -0800)]
metadata: get and set policies from go
This commit adds in the ability to get and set policy data from go using
the GetPolicy and SetPolicy functions. This is done via a patch of the
x/sys/unix package that exposes the filesystem encryption structures.
Note that not all the fields of the PolicyData protocol buffer are
needed to get and set policies. The wrapped_policy_keys are not used and
will be written and read by other components of fscrypt.
To run the policy tests, the environment variable BASE_TEST_DIR must be
set to a directory for testing on a filesystem that supports encryption.
This commit adds in the golang.org/x/sys/unix package. This package
provides a low-level interface to unix syscalls. We will uses this
package instead of the built-in "syscall" package because the syscall
package is locked down (https://golang.org/pkg/syscall) and is not
exposing any new kernel functionality.
In fact, this is actually a patched version of the x/sys/unix package
pending review (first part: https://go-review.googlesource.com/c/37943).
The version included in this commit exposes all of the filesystem
encryption kernel interfaces to Go code.
Joe Richey [Thu, 2 Mar 2017 18:15:23 +0000 (10:15 -0800)]
metadata: introduce protobuf structures
This commit adds in the metadata package. The primary purpose of this
package is to provide the on-disk metadata structures in the form of
protocol buffers. This includes:
- Policy metadata structure
- Protector metadata structure
- Config file structure
- All necessary sub-structures (wrapped keys, parameters, etc)
This commit also adds in an example usage of the Config structure, which
represents the structure of the global config file. All the package
does at this point is convert between the Config structure and a JSON
representation.
Here we introduce govendor, which is described more in the README. This
means we will have all of our Go dependencies in the vendor
subdirectory. This means we will have no Go source dependencies, only
dependencies on the build tools (Go and govendor). The README describes
this in detail.
Note that we commit the generated files.
see: https://blog.golang.org/generate
This commit adds in the two protocol buffer libraries for Go. The
github.com/golang/protobuf/proto package will let Go code read and write
protocol buffers. The github.com/golang/protobuf/jsonpb package lets Go
code output a JSON representation of protocol buffers.
These packages are stored in the vendor directory, meaning that they
will be imported instead of any installed system packages.
Joe Richey [Thu, 2 Mar 2017 00:57:27 +0000 (16:57 -0800)]
util: convenience utilities for fscrypt
This commit adds in the util package. This package provides
two functions for creating errors. These functions are:
- InvalidInputErrorF - bad input from user or caller
- SystemErrorF - low level failure
It also adds in a small function for converting Go byte slices into C
void pointers. This will be very useful for interoperating with C.
Joe Richey [Fri, 31 Mar 2017 17:14:20 +0000 (10:14 -0700)]
cmd/fscrypt: Initial stub program and docs
This commit adds in a stub fscrypt program. The binary just tells the
time and the tests do nothing, but the Makefile will build them! This
commit also adds documentation to the README that explains how to get,
build, run, test, format, lint, and install the code.
Also note that the executable is now in the cmd/fscrypt directory. The
library implementing the core functionality will be at the root. This
is essentially point 2 of https://medium.com/@benbjohnson/structuring-applications-in-go-3b04be4ff091
Joe Richey [Thu, 30 Mar 2017 21:39:09 +0000 (14:39 -0700)]
Initial Documentation for fscrypt
This commit includes the Apache 2.0 License and a README with
documentation on how to checkout the code, some context about Linux
filesystem encryption, and documentation stubs for fscrypt and
fscryptctl. Also adds in a simple .gitignore so we don't commit build
files.
projects / fscrypt.git / log