I’ve de-railed (or rather de-activesupported) some of my gems in favor of dry-rb and I’m very happy ever since. However, I’m not sure how best to deal with secrets. Keeping encrypted secrets in the repo comes in very handy for CI or deployment to Kube etc. So far, I’m using require 'active_support/encrypted_configuration' for this, but I’d really like to kiss it goodbye.
It’s not very hard to code something similar in PORO, but I guess I’m not the only one dealing with a bunchload of secrets in dry-land. How do you manage this? Or how does Hanami 2 manage this?
In case there’s no infrastructure for this yet, how about something like dry-secrets?
@alassek I somehow missed your answer, but since the topic popped up again, I’d like to pick this thread up after a long while:
I wasn’t suggesting encrypted attributes (which would be a ROM feature), but rather the ability to commit encrypted secrets into the repo and then decrypt them at runtime using only one decrypt-key to unlock all secrets (API keys, tokens etc).
When deploying my de-railed projects, I’m falling back to settings such secrets using one by one environment variables again. So for every API token, I need one corresponding environment variable. In a cloudy world, just setting one environment variable with the decryption key is much cleaner.
I’m wondering, where something similar would fit in dry-land.
A new dry-credentials gem?
Extending the dry-config gem?
Somewhere else?
I’m no fan of overloading things so the first option would be my choice.
Also, is it okay for a dry-gem to rely on ActiveSupport::EncryptedConfiguration? I have a gut feeling towards “no”. Even though you can load just that bit, ActiveSupport as a dependency is heavy and implementing it from scratch shouldn’t be too hard.
ROM is not merely a database layer; it is a general-purpose data mapper. I imagine this functionality could be built as an extension to rom-yaml. There is no extant implementation of this currently that I know about.
I don’t think that a Dry gem built with ActiveSupport would be welcomed. I know how Peter feels about it, and I feel similarly. I am quite content building systems in a plain Ruby dialect, and I would consider an ActiveSupport dependency a nonstarter.
That said, let’s not ignore the potential problems introduced by this convenience. My company was recently impacted by the CircleCI security incident.
We were using encrypted credentials in the way you describe
We had neglected to use different keys by environment
We had to roll every secret in all environments as a result
Yes, we fucked up here. But we did so because Rails made it easy. As annoying as it may seem, taking a more 12-factor route for your secrets might not be the worst thing you could do.
I don’t think that a Dry gem built with ActiveSupport would be welcomed. I know how Peter feels about it, and I feel similarly.
So I thought, I don’t particularly like it when gems pull ActiveSupport. I’ve taken a quick look under the hood of ActiveSupport::EncryptedConfiguration (AS:EC) and there’s nothing going on that couldn’t be easily done with just some openssl and base64.
We had neglected to use different keys by environment
I get your point, but rest assured that I consider AS:EC overengineered to begin with. Two things I don’t like about it for safety and bloat reasons:
AS:EC creates key-files. To prevent them from being committed to the repo, it fiddles with .gitignore. IMO, keys must not be persisted to the filesystem, I’d create the encrypted file and just print the key to emphasize this.
AS:EC features multiple named encrypted files (names normally in sync with app environments), but when no name is given, it falls back to the one-size-fits-all credentials.yml.enc. Not only do you have to remove those (and the fiddling in .gitignore) if you accidentally forget to mention the name, but it makes unsafe use like what you’ve mentioned more likely. I’d not implement that fallback but enforce you always work on a specified name. When these credentials are embedded in a framework like Hanami, the framework may decide to hardwire the encrypted files names to be in sync with app environments which makes such an “everything in one file” solution impossible. (Or at least you’d have to copy the main encrypted file for each app environment whenever you modified a credential… high enough a safety bar I’d say.)
I’m quite sure it’s possible to implement the idea in a lightweight and safer way and I’m also quite sure I’d like to have that e.g. for my bot “framework” which talks to so many other parties, handling all the keys in individual environment variables for cloud deployment is beyond a nightmare.
Question though: Provided the community is happy with the implementation, would the very concept stand a chance to become a part of dry-rb? Your unfortunate experience makes it less probable it appears. Depending on this, I’d bootstrap this bit as dry-credentials or chose some other name… renaming is no fun
PS:
ROM is not merely a database layer; it is a general-purpose data mapper. I imagine this functionality could be built as an extension to rom-yaml
Maybe, however, does it really make sense to implement encryption for one particular data type (YAML)? Encryption feels like a different level to me. But I must admit, I have not much insight when it comes to ROM.
You might want to take a look at symmetric-encryption, it looks like you could transparently encrypt any data format you like. It has very minimal dependencies. (Though I will note that you definitely want to not use the aes-256-cbc cipher – looks like aes-256-gcm is available).
I don’t think there’s anything stopping you from writing your own dry-credentials lib, but as for official inclusion I can’t speak to that.
That is a fair point. I certainly have no great love for YAML, and have to deal with it much more than I would like. I’ve toyed with the idea of creating a more robust rom-files gem that has pluggable adapter support for yaml, toml, csv, json, and whatever else.
@alassek Turns out, symmetric-encryption is quite an overkill as well. The tiny bit required for such a gem, however, is not complicated and with some inspiration from ActiveSupport, openssl is not hard to handle. At least I’m quite sure my implementation is just as safe as others. This way, there are no non-stdlib runtime dependencies at all now.
I’ve modeled the interface similar to dry-configurable, no frills yet easy to integrate in any kind of app. I’ll play with it in some of my pet projects and port bridgetown_credentials to it.
I’ll take some time later to look more closely at this, although at first blush I would like to be able to integrate your credentials into dry-configurable settings somehow. Perhaps that’s an exercise best left to the user, but it would be nice if there was a non-tedious way to do this.
Picking Marshal as the default serializer is… dangerous. I can see why you would make that choice, but it’s worth calling out the potential drawbacks.
I dedicated a dog walk to giving this some thought and ultimately decided against it:
Settings and credentials are quite different in nature: Settings are more dynamic and may change at runtime, credentials should not. This is why settings are implemented on classes and instances, credentials only on classes.
Merging credentials into settings may cause name collisions (both use method chains to query).
Separation of the two is not such a bad thing since it forces the developer to decide what goes where.
If you still want to access credentials via settings, it shouldn’t be difficult to add a method_missing delegation.
However, I’m certainly open for suggestions. Separating things by default still seems the better approach, but adding a command to merge wouldn’t be very difficult.
