We’re using dry-system and dry-transaction as the basis for a few of our data transforming microservices. The overall architecture is the same for every service:
- listen to a pubsub queue for a message
- take the payload of the message, which is a path, and retrieve the file at that path
- convert the file from JSON to a data structure
- validate that data structure
- instantiate a Google Datastore Entity from the data structure
- persist the entity
- go back to listening
Now our idea was to extract a gem that implements everything except steps 4 and 5 so that new services only have to implement validation and entity instantiation themselves.
How would one best go about and structure the gem so that users of the gem can easily inject the relevant steps without too much effort?
I recently added support for component providers. It seems like this is what you could use. The feature is described right here. This means you can simply create a gem, which registers itself as a component provider, then define whatever you need and that’s it. In your microservices you add this gem as a dependency, require it, and use your container to boot dependencies provided by your gem. We have support for configuration too, so each component can describe its settings (keys + types + optional default values), and you can configure your components from within your app.
Let me know if this makes sense
This component thing looks interesting, though I am not sure that this is quite what I need. But maybe I’ve been thinking in the wrong direction, I am slowly getting used to the dry.rb way
I’ve set up a simplified example of how the gem code is structured currently here, and a sample client that overrides two steps here. It works, but it feels too… manual compared to the gem code.
Ideally, there would be a way to either overwrite parts of the registered components, or use some sort of symbolic link, like Application[‘validate’] points to either ‘transformer.validate’ or ‘myapp.validate’
You can nest systems by importing one into another. ie
MyApp::Container.import(core: Core::Container). In the future we’ll provide a way to cherry-pick which parts should be imported. Maybe that’s closer to what you need.
Okay, in case anyone else runs into the same problem, I have updated the demo repositories with a working solution that seems to work fine. I just had to remember that you can also manually register components, and make the transaction refer to those instead of the auto-registered ones. Now the client just has to make sure to register the missing components themselves.
Thanks for this component/container providing and nested imports feature ( and the question that prompted the answer ) ! It’s something I needed today to solve a problem. dry-rb keeps on giving, and I’m working on something cool to give back by year’s end.
@solnic it would be awesome if you could share some code samples on how a gem would register itself as a component provider using dry-system?
This is what I came with so far https://gist.github.com/gottfrois/5e8edd540910c3ee8276eb2c76d8ff1a
OK I’ll put together an example and let you know here
Is there an example available somewhere?
Not yet, didn’t have time to do it. I’ll get to it eventually.
Came here after checking out:
Sending you encouragement!
I’d like to see an example using dry-system in the context of creating a gem.
It would be helpful because learning the way a Ruby gem is structured can be tricky, and involves understanding concepts like require and $LOAD_PATH. Both those concepts are related to dry-system, so an example illustrating using dry-system in the context of a gem would make a nice bridge for people who took the time to figure out how a Ruby gem works, which is probably a lot of people
Your time is precious, but wanted to express interest in this