This essentially worked for non-dev mode before but would create storage
directories that weren't actually used or needed, and it didn't work in
dev mode because it made assumptions about the presence of a worker auth
kms or not. This fixes those behaviors.