How can I report a failure in a dry-types constructor?

I am processing some input from a web form of the shape “3 days” or “1 month”, etc. We use a third-party gem to do the conversion of the string to a duration as seconds. That gem returns nil when it fails. We are wrapping the gem inside of a custom type.

How can I indicate to Dry-Types (and ultimately Dry-Validation) that a failure has occurred?

# frozen_string_literal: true

require 'dry-validation'

ONE_DAY = 24 * 60 * 60
TWO_DAY = 2 * ONE_DAY

# Represents some external code I cannot change
def duration_parsing_placeholder(v)
  if v == "1 day"
    ONE_DAY
  elsif v == "2 days"
    TWO_DAY
  else
    nil # indicates an error
  end
end

module Types
  include Dry.Types
end

Types::Duration = Types::Integer.constructor do |input|
  if input == "" || input == "0"
    # Want empty and zero strings to count as unsetting the value
    nil
  else
    # If this returns nil, however, that's a failure to parse, and should be reported
    duration_parsing_placeholder(input)
  end
end

class MyContract < Dry::Validation::Contract
  params do
    required(:duration).maybe(Types::Duration)
  end

  rule(:duration) do
    next unless value
    key.failure "Duration must be less than 1 day" if value > ONE_DAY
  end
end

# Correct
validation = MyContract.new.call(duration: '1 day')
raise "failed" unless validation.success?
raise "failed" unless validation[:duration] == 86400

# Correct
validation = MyContract.new.call(duration: '2 days')
raise "failed" unless validation.failure?
raise "failed" unless validation.errors[:duration] == ["Duration must be less than 1 day"]

# Correct
validation = MyContract.new.call(duration: '0')
raise "failed" unless validation.success?
raise "failed" unless validation.key? :duration
raise "failed" unless validation[:duration] == nil

# Correct
validation = MyContract.new.call(duration: '')
raise "failed" unless validation.success?
raise "failed" unless validation.key? :duration
raise "failed" unless validation[:duration] == nil

# Does not pass
validation = MyContract.new.call(duration: 'cows go moo')
raise "failed" unless validation.failure?
raise "failed" unless validation.errors[:duration] == ["Duration must be ..."]

Check out dry-types 1.0 release notes: https://github.com/dry-rb/dry-types/releases/tag/v1.0.0, the “Added” section. We should add docs about this, though it’s a kind of “advanced” level.

Thanks!

If a block is passed, you must call it on failed coercion, otherwise raise a type coercion error

This doesn’t seem to work. If I apply the code you linked against the runnable example in the original post:

Types::Duration = Types::Integer.constructor do |input, &block|
  if input == "" || input == "0"
    nil
  else
    v = duration_parsing_placeholder(input)

    if v.nil?
      if block
        block.call
      else
        raise Dry::Types::CoercionError.new("My custom message")
      end
    end

    v
  end
end

Then the specific coercion message is lost:

validation = MyContract.new.call(duration: 'cows go moo')
p validation.errors[:duration]
# => ["must be an integer"]

It’s unclear what the block is to be used for compared to the exception. In the absence of documentation or suggestive variable names, it looks like the block would be used for a fallback coercion. I don’t want a fallback — coercion has failed and I wish to abort.

If I remove the block.call and only raise the exception, nothing catches it and the entire program aborts — not ideal for a HTTP endpoint.

This makes the exception handling your job

I don’t see how to make it “my job” because I’m not calling the type constructor myself. It’s somewhere in the pile of code from dry-{validations,schema,logic,types}. It would be highly appreciated if you could edit the runnable example in the original post to demonstrate how I can make it “my job”.

Why would I read the release notes when I’m not upgrading dry-types, but instead starting fresh?

That would be appreciated!

I’m a bit thrown though, because this doesn’t feel like it should be “advanced”. dry-types bills itself as a “Flexible type system for Ruby with coercions and constraints”. Shouldn’t the failure modes of coercions / conversions / constraints be the normal case? Is there a different library that you recommend for treating these types of failures as first-class concepts?

You could do this:

# frozen_string_literal: true

require 'dry/validation'
require 'date'

module Types
  include Dry::Types()

  MyDate = Types::Date.constructor do |value|
    ::Date.today + 1 if value.eql?('1 day')
  end
end

class MyContract < Dry::Validation::Contract
  params do
    required(:duration).filter(:string, :filled?).value(Types::MyDate)
  end
end

contract = MyContract.new

puts contract.(duration: nil).inspect
#<Dry::Validation::Result{:duration=>nil} errors={:duration=>["must be a string"]}>

puts contract.(duration: "").inspect
#<Dry::Validation::Result{:duration=>""} errors={:duration=>["must be filled"]}>

puts contract.(duration: "foo bar").inspect
#<Dry::Validation::Result{:duration=>nil} errors={:duration=>["must be a date"]}>

puts contract.(duration: "1 day").inspect
#<Dry::Validation::Result{:duration=>#<Date: 2019-07-31 ((2458696j,0s,0n),+0s,2299161j)>} errors={}>

You can read more about filter rules here.

Calling a block is faster since it doesn’t involve raise/rescue.

This is not supported, these messages are not passed to dry-validation.

This is why I called it “advanced”, you either use &block and call it on failure or raise a CoercionError. In the latter case, &block must be removed because if it’s there dry-types decides you’re using the block option. It’s done like this for performance reasons and I know it’s confusing.

You’re citing dry-types reference, from its point of view dry-schema and dry-validation don’t exist. “your job” in that context means if you have “&block” in the constructor signature then dry-types doesn’t try to rescue from broad exception types like TypeError or ArgumentError because it can hide some bug in you coercion logic. It “believes” you call block.call where needed and have rescues where needed. Does it make more sense now?

Thanks!

Some follow-up clarification questions…

Based on the link, this is a feature of dry-schema. As I understand it, that means that every contract I create that has a field of “duration” type needs to repeat the code for .filter(:string, :filled?).value(Types::MyDate). These two things will always want to be combined.

1. Is there a way to avoid the repetition of .filter(:string, :filled?).value(Types::MyDate)?

In my case, I don’t have a trivial regex that I can apply to the input; the only way I know if it is valid is to parse it. This means that I need to parse it once in the filter to see if it’s valid, then parse it again in the type constructor.

2. Is there a way to avoid parsing the data twice without reimplementing the internals of the third-party gem as a regex?

Your example uses existing predicates like :string and :filled? in the filter. None of the built-in predicates appear to fit my case.

3. How do I create a custom predicate that calls my parsing function in order to use a filter rule?

In order for end users to have an idea of what is actually valid, the error message needs to be descriptive, not just something like “must be a duration”.

4. How can I customize the error message when the filter / predicate fails?

I’m going to express some frustration here; feel free to ignore this if you don’t want to read it. It’s not intended as a personal attack, but I understand that this kind of criticism often raises blood temperatures. Please read this in good faith.


As an end-user, I don’t care about the lines between dry-{types,logic,validation,schema}. My goal is to take some input from a user, convert it to the correct types in the correct form, and inform the user of any issues encountered along the way. I want to avoid rewriting code other people have written and avoid duplicating my own code.

I don’t want to use dry-types to define a type, but the implementation of dry-{validation,schema,types,logic} imposes that on me. Being new to these gems, I’ll assume that there’s a reason for that.

As an open-source maintainer, I get the benefit of modularity — smaller libraries are more easily tested and are composable. However, end-user documentation tends to greatly suffer in these cases.

Take a gander at the dry-validation documentation with a fresh set of eyes and you realize that you have to completely learn the surface API of 3+ gems to make effective use of dry-validation. Except it’s not the entire surface area, but just some of it (unclear on which portions), and you have to know the implementation of one gem in terms of the other.

For this specific topic, I’m told that to report an error to my users, I need to call the (undocumented) block argument to #constructor, but that ignores the fact that dry-{validation,schema} is the one passing the block argument and (presumably) just ignoring it. The forest has been ignored in favor of the trees.

Reform has (had?) the same philosophy of splitting up one conceptual gem into many gems. It was one of the single-most painful things to deal with there — do you use reform or representable or disposable or twin to specify an option? Does this configuration from N layers down in the stack actually show through at layer N+1?

I want to use dry-* because the stated project goals mesh fairly well with my worldview, but I keep running into issues that cause harsh mental corrections. These make me think “does anyone actually use this project in the real world?”. I know that such a question is unfair, and what it really means is that everyone else uses it differently from me, and that’s OK.

Again, I apologize for the rant, and I hope that something I’ve said can somehow help y’all improve the dry ecosystem.

We’re open to constructive criticism so this is absolutely fine.

The assumption we make is that you want to learn how to use these gems separately. They are all useful standalone in many, many different situations. Some high-level abstractions that hide these “details” may be useful, so we may introduce them eventually, possibly in Hanami 2.0 rather than in a dry-rb lib though.

The reason why dry-rb libs are separated is not to ease lives of maintainers (I actually think it adds more work for us). Re-usability and composability is the actual reason. For example at work I’ve recently built a whole backend system where I only used dry-schema, because there was no need for dry-validation. As a fast sanitizer/type-checker it works better because it’s faster (which was very important in my project). Another example is how dry-types is used in so many places in rom-rb/dry-rb that I’ve lost count already.

We’re fully aware of this issue. In fact, I recently started exploring possible solutions and I’ll be rebuilding dry-rb.org website using Antora as a result. I have an idea to write embeddable documentation, so ie various portions of dry-schema docs would be included in dry-validation docs, or various portions of dry-validation docs would be included in hanami-validation docs. Not sure if it’s a reasonable idea but we’ll see.

Improving docs is one of the highest priorities these days.

This wasn’t the right solution, hence the confusion. Using types for validation has been always a big no-no (this needs to be clearly explained in the docs…but it’s not).

Yeah this can easily go bonkers. I believe we’re in better shape, despite your issue in this thread.

dry-rb organization is 4 years old. dry-validation has been recently rewritten from scratch. dry-types 1.0.0 was released just a couple of months ago. This whole ecosystem is fairly fresh and is still shaping up. We’re building these gems, changing them, improving them based on our real-world experience and feedback from the people like you. There’s always a lot to improve and as I mentioned, the documentation (despite getting better and better) is still missing a lot which I believe is one of the biggest actual issues, rather than how things work and the fact you need to learn 3 gems instead of one to handle complex validation use cases.

YMMV :man_shrugging:

1 Like

No, not yet.

The filter rules should check if the value is in the expected state for the gem to try to coerce it - which I think is just checking if it’s a non-empty string. Then the gem tries to coerce the value, if it fails you get nil back, and since you specify that the output value must be a date, you’ll get an error message. This way you do not parse the value more than once using the gem, the type is applied only once, after filter rules are applied.

You can provide custom error messages for duration key:

en:
  dry_validation:
    errors:
      rules:
        duration:
          filled?: 'custom message 1'
          string?: 'custom mesage 2'
          date?: 'custom message 3'

Thank you for reading and responding to my rant!

Yes, the documentation does communicate this effectively.

It is not. That’s a core point of this question — I literally do not know what the possible inputs for this third-party gem are. It attempts to parse human language into a duration. As selected examples, I know that 1 works but cow does not. I attempted to communicate this:

I’d appreciate feedback on how I could have presented that information in a more accessible way. I also numbered and bolded my questions in an attempt to prevent them from being overlooked, but I don’t see where #3 was addressed:

Presumably once I have defined a custom predicate to be used in the filter, I can set up an error message for it in the same way, via YAML.

I provided you with a working example here - this is all you really need. You do know that the gem needs a non-empty string before you can try to coerce it into a date, that’s all you need to filter out any unexpected values. Then the gem can try to coerce and if it returns nil you will get a nice validation error that the output value is not a date (and you can customize the error message if you want).