Matching optional params schema against a nested schema

Support
#1

Hi folks,

I’m using dry-schema with some form data, and my understanding is that empty string values in a params schema are treated as nil. But I’m finding that if the attribute value is optional and expected to be matched against a nested schema, empty string values are still validated against the nested schema.

Here’s an initial attempt:

inner_schema = Dry::Schema.Params do
  required(:path).filled(:string)
end

outer_schema = Dry::Schema.Params do
  required(:foo).maybe(inner_schema)
end

I’d expect the first three of these test cases to pass, and the last one to fail, but the first one actually fails as well because the path value is missing:

outer_schema.("foo" => "").success? #=> false, but we want it to be true
outer_schema.("foo" => nil).success? #=> true
outer_schema.("foo" => {"path" => "bar"}).success? #=> true
outer_schema.("foo" => {"other" => "bar"}).success? #=> false

I feel like it’s doing the validation before it realises the empty string is the equivalent of nil for params? I’ve tried a few other variations, but none work to what I need:

# explicit inner schema rather than something reusable
# fails the first test case
outer_schema = Dry::Schema.Params do
  required(:foo).maybe(:hash) do
    required(:path).filled(:string)
  end
end

outer_schema.("foo" => "")
# => #<Dry::Schema::Result{:foo=>{}} errors={:foo=>{:path=>["is missing"]}}>

# explicit empty comparison fails third test case, because
# it seems to require symbol keys - surprising for a Params schema?
outer_schema = Dry::Schema.Params do
  required(:foo).maybe do
    empty?.or(inner_schema)
  end
end

outer_schema.("foo" => {"path" => "bar"})
# => #<Dry::Schema::Result{:foo=>{"path"=>"bar"}} errors={:foo=>{:path=>["is missing"]}}>

outer_schema.("foo" => {:path => "bar"}).success? # => true

# wrap the schema in a hash() call, and it doesn't care about the inner values
# at all, so even what should be *invalid* is considered valid.
outer_schema = Dry::Schema.Params do
  required(:foo).maybe do
    empty?.or(hash(inner_schema))
  end
end

outer_schema.("foo" => {"other" => "bar"})
# => #<Dry::Schema::Result{:foo=>{}} errors={}>

I’m not sure if any of these approaches should actually be working and thus I’ve found a bug, or if I’m trying to do something that isn’t technically possible. Any guidance would be greatly appreciated :slight_smile:

#2

Whether or not any of the above approaches are buggy, I’ll leave that to project maintainers to decide. I have found a way around the problem, though:

outer_schema = Dry::Schema.Params do
  KEYS = %i[ foo ]

  before(:value_coercer) { |result|
    result.to_h.each_with_object({}) { |(key, value), hash|
      hash[key] = KEYS.include?(key) && value == "" ? nil : value
    }

  required(:foo).maybe(inner_schema)
end

I’m not applying the logic to all keys, as I’ve found it didn’t work with arrays of nested schemas (albeit from very brief testing - I didn’t want to go deep in debugging), so instead I’m just keeping the workaround to where it’s necessary.

#3

This is because in Params an empty string is turned into an empty hash :frowning: It uses to_hash from dry-types. I actually don’t remember why we decided to do this but your usecase clearly shows that it wasn’t a good idea. Using maybe in this case cannot work.

Here’s how you can workaround this:

require "dry/schema"

inner_schema = Dry::Schema.Params do
  required(:path).filled(:string)
end

outer_schema = Dry::Schema.Params do
  required(:foo).value { empty? | hash(inner_schema) }
end

puts outer_schema.(foo: { path: "something" }).inspect
#<Dry::Schema::Result{:foo=>{:path=>"something"}} errors={}>

puts outer_schema.(foo: { path: "" }).inspect
#<Dry::Schema::Result{:foo=>{:path=>""}} errors={:foo=>{:path=>["must be filled"]}}>

puts outer_schema.(foo: "").inspect
#<Dry::Schema::Result{:foo=>{}} errors={}>

puts outer_schema.(foo: nil).inspect
#<Dry::Schema::Result{:foo=>nil} errors={}>

Please report an issue. This is something we should address in 2.0.0.

#4

Thanks for that suggested solution @solnic - it’s a bit neater than mine! - though the app I’m working within expects either nil or a hash matching the nested schema, so the translation of an empty string to an empty hash doesn’t quite work out in my particular case.

I can stick with the value coercer for the moment, but certainly if there are suggestions on something neater that returns nil rather than an empty hash, I’d love to hear about them :slight_smile:

#5

I just realized this works:

require "dry/schema"

inner_schema = Dry::Schema.Params do
  required(:path).filled(:string)
end

outer_schema = Dry::Schema.Params do
  required(:foo).maybe(:hash, inner_schema)
end

puts outer_schema.(foo: { path: "something" }).inspect
#<Dry::Schema::Result{:foo=>{:path=>"something"}} errors={}>

puts outer_schema.(foo: { path: "" }).inspect
#<Dry::Schema::Result{:foo=>{:path=>""}} errors={:foo=>{:path=>["must be filled"]}}>

puts outer_schema.(foo: "").inspect
#<Dry::Schema::Result{:foo=>nil} errors={}>

puts outer_schema.(foo: nil).inspect
#<Dry::Schema::Result{:foo=>nil} errors={}>

This is because a hash? check is applied separately from applying your schema. I can see how that’s confusing though - using an inner schema should probably imply a hash? check too :thinking:

#6

The one catch with this new suggestion is that it expects symbol keys for the inner params values - which is not the case for the form data I’m dealing with. Yes, I could wrangle the data further… :man_shrugging:

puts outer_schema.(foo: { path: "something" }).inspect
# #<Dry::Schema::Result{:foo=>{:path=>"something"}} errors={}>

puts outer_schema.(foo: {"path" => "something" }).inspect
# #<Dry::Schema::Result{:foo=>{"path"=>"something"}} errors={:foo=>{:path=>["is missing"]}}>
#7

Ah damn, that’s because the key map (which does the key conversion prior applying rules) doesn’t include the inner schema. Oh boy, that feels like a bug actually - sorry but could you report this too? :bowing_man: