We’re using dry-system and dry-transaction as the basis for a few of our data transforming microservices. The overall architecture is the same for every service:
listen to a pubsub queue for a message
take the payload of the message, which is a path, and retrieve the file at that path
convert the file from JSON to a data structure
validate that data structure
instantiate a Google Datastore Entity from the data structure
persist the entity
go back to listening
Now our idea was to extract a gem that implements everything except steps 4 and 5 so that new services only have to implement validation and entity instantiation themselves.
How would one best go about and structure the gem so that users of the gem can easily inject the relevant steps without too much effort?
I recently added support for component providers. It seems like this is what you could use. The feature is described right here. This means you can simply create a gem, which registers itself as a component provider, then define whatever you need and that’s it. In your microservices you add this gem as a dependency, require it, and use your container to boot dependencies provided by your gem. We have support for configuration too, so each component can describe its settings (keys + types + optional default values), and you can configure your components from within your app.
This component thing looks interesting, though I am not sure that this is quite what I need. But maybe I’ve been thinking in the wrong direction, I am slowly getting used to the dry.rb way
I’ve set up a simplified example of how the gem code is structured currently here, and a sample client that overrides two steps here. It works, but it feels too… manual compared to the gem code.
Ideally, there would be a way to either overwrite parts of the registered components, or use some sort of symbolic link, like Application[‘validate’] points to either ‘transformer.validate’ or ‘myapp.validate’
You can nest systems by importing one into another. ie MyApp::Container.import(core: Core::Container). In the future we’ll provide a way to cherry-pick which parts should be imported. Maybe that’s closer to what you need.
Okay, in case anyone else runs into the same problem, I have updated the demo repositories with a working solution that seems to work fine. I just had to remember that you can also manually register components, and make the transaction refer to those instead of the auto-registered ones. Now the client just has to make sure to register the missing components themselves.
Thanks for this component/container providing and nested imports feature ( and the question that prompted the answer ) ! It’s something I needed today to solve a problem. dry-rb keeps on giving, and I’m working on something cool to give back by year’s end.
Sending you encouragement!
I’d like to see an example using dry-system in the context of creating a gem.
It would be helpful because learning the way a Ruby gem is structured can be tricky, and involves understanding concepts like require and $LOAD_PATH. Both those concepts are related to dry-system, so an example illustrating using dry-system in the context of a gem would make a nice bridge for people who took the time to figure out how a Ruby gem works, which is probably a lot of people
Your time is precious, but wanted to express interest in this
