CPAN is a software modules repository famous for the sheer number of solutions it offers free of charge. Not all of them do different things though, and in some cases its pushing TIMTOWTDI to its limits. Sometimes choosing the right module for the job is easy, but when its not you have to compare them in one way or another.
While speed of execution may not be the most important factor in choosing a solution, it can speak for the code quality and feature design. Two programs written in the same language doing exactly the same thing should have quite comparable run times, but feature creep, code bloat or inefficient algorithms could increase that time beyond acceptable level.
In this article, I'm going to present code and results for a benchmark of a couple of CPAN form frameworks / validation libraries. Disclaimer: I'm the author of one of them, but I try my best to stay objective.
Benchmark setup
We're going to see how the following libraries fare against each other in validating a hash reference:
Side note: if I missed anything notable, let me know and I will add it to this article
- Data::MuForm
A form framework that was recommended to me on #perl IRC. Even though I haven't used it in any serious project, it seems pretty comprehensive and is capable of more than just validating data (like rendering). It has some maintenance issues, but has seen some activity lately.
- Form::Toolkit
Moose-based, role-heavy framework that can be easily extended by creating more Moose classes and roles. It focuses on validating the data and doesn't care from where it came from. Was not updated in years, but still passes all tests and noone has reported any issues.
- Form::Tiny
My take on data validation, inspired by Laravel validation system, Form::Toolkit and Type::Tiny. It does not contain any field validation code, and instead depends on Type::Tiny constraints to deliver them.
- HTML::FormHandler
Very similar to Data::MuForm, but seems to have more rendering and other non-validation capabilities. The most ++'ed module of them all.
- Valiant
Recent addition to CPAN, presented at the last conference. Inspired by Ruby on Rails and meant to be used together with Moo. Marked as early release in the documentation.
- Validate::Tiny
Possibly the smallest validation library on CPAN. Will act as our control group, as it should be as fast as using no library at all.
The code used for benchmark looks like this:
use Benchmark qw(cmpthese);
my $data = { ... };
cmpthese(-5, {
form_toolkit => sub {
my $form = FToolkit->new;
$form->fill_hash($data);
die if $form->has_errors;
},
form_tiny => sub {
my $form = FTiny->new;
$form->set_input($data);
die unless $form->valid;
},
data_muform => sub {
my $form = FMuForm->new;
die unless $form->check(data => $data);
},
html_formhandler => sub {
my $form = FHtmlHandler->new;
$form->process(params => $data);
die unless $form->validated;
},
validate_tiny => sub {
my $form = FValidateTiny->new;
die unless $form->valid($data)->{success};
},
valiant => sub {
my $form = FValiant->new($data);
$form->validate;
die unless $form->valid;
},
});Contents of $data will change in each case.
Case #1: it's just a field, bro!
This will be the most basic hash reference with just a single value:
my $data = {
a => 2
};We don't check for the value here, we just want 'a' existence in $data to be ensured.
Data::MuForm
Data::MuForm greets us with a has_field DSL keyword:
use Moo;
use Data::MuForm::Meta;
extends 'Data::MuForm';
has_field 'a' => (
required => 1,
);Form::Toolkit
Since Form::Toolkit is heavy Moose-centric, we see a lot of familiar Moose code and no DSL:
use Moose;
extends 'Form::Toolkit::Form';
sub build_fields {
my ($self) = @_;
$self->add_field('a')
->add_role('Mandatory');
}
__PACKAGE__->meta->make_immutable();Form::Tiny
Similar to Data::MuForm, but less boilerplate:
use Form::Tiny;
form_field 'a' => (
required => 1,
);HTML::FormHandler
This looks exactly like Data::MuForm, but the boilerplate changes a bit:
use HTML::FormHandler::Moose;
extends 'HTML::FormHandler';
has_field 'a' => (
required => 1,
);
no HTML::FormHandler::Moose;Valiant
As far as I understand, Valiant only acts on Moo attributes, so we also need to declare it with Moo:
use Moo;
use Valiant::Validations;
has 'a' => (
is => 'ro',
);
validates a => (
presence => 1,
);Validate::Tiny
I had to create a bit of extra code here, and decided to use Moo not to give any unfair advantage that might come from avoiding to bless:
use Moo;
use Validate::Tiny ':all';
my $rules = {
fields => [qw(a)],
checks => [
[qw(a)] => is_required(),
],
};
sub valid
{
my ($self, $data) = @_;
return validate($data, $rules);
}Results
Rate html_formhandler form_toolkit data_muform form_tiny valiant validate_tiny
html_formhandler 684/s -- -26% -88% -97% -99% -99%
form_toolkit 925/s 35% -- -83% -96% -98% -99%
data_muform 5589/s 717% 504% -- -76% -89% -96%
form_tiny 23306/s 3308% 2419% 317% -- -54% -83%
valiant 50834/s 7334% 5395% 810% 118% -- -63%
validate_tiny 136039/s 19795% 14605% 2334% 484% 168% --As expected, Validate::Tiny is speedy, but since it is very bare-bones it required significantly more mental gymnastics than the other options. Form::Tiny and Valiant are in the same order of magnitude, and so are Form::Toolkit and HTML::FormHandler. Valiant is the fastest option among the heavier frameworks.
Case #2: Don't be so negative
With the most basic case out of the way, we can check for something useful - whether the value is a number, and if the number is positive:
my $data = {
a => 2
};We're not going to use any PositiveInt checks, and check manually for $value > 0 - to test how well the frameworks handle custom checks.
Data::MuForm
Validating a single filed in Data::MuForm is done by creating a validate_<field> method in the class. Error must be added manually. Integer type is built into the library:
use Moo;
use Data::MuForm::Meta;
extends 'Data::MuForm';
has_field 'a' => (
type => 'Integer',
required => 1,
);
sub validate_a {
my ($self, $field) = @_;
unless ($field->value > 0) {
$field->add_error('must be positive');
}
}Form::Toolkit
Form::Toolkit does not actually seem to support custom coderef checks, but I think we were supposed to use Moose roles instead. Lets see how well that goes:
package Form::Toolkit::FieldRole::IsPositive {
use Moose::Role;
with qw'Form::Toolkit::FieldRole';
after 'validate' => sub {
my ($self) = @_;
unless ($self->value > 0) {
$self->add_error('must be positive');
}
}
}
use Moose;
extends 'Form::Toolkit::Form';
sub build_fields {
my ($self) = @_;
$self->add_field('Integer', 'a')
->add_role('Mandatory')
->add_role('IsPositive');
}
__PACKAGE__->meta->make_immutable();Form::Tiny
Latest Form::Tiny has a DSL keyword that adds a validator to the last defined field:
use Form::Tiny;
use Types::Standard qw(Int);
form_field 'a' => (
type => Int,
required => 1,
);
field_validator 'must be positive' => sub {
shift() > 0
};HTML::FormHandler
Again, same as Data::MuForm. This will be a recurring theme, as those two were created by the same person:
use HTML::FormHandler::Moose;
extends 'HTML::FormHandler';
has_field 'a' => (
type => 'Integer',
required => 1,
);
sub validate_a {
my ($self, $field) = @_;
unless ($field->value > 0) {
$field->add_error('must be positive');
}
}
no HTML::FormHandler::Moose;Valiant
All valiant validators are added after the validates DSL keyword, here we need numericality and with:
use Moo;
use Valiant::Validations;
has 'a' => (
is => 'ro',
);
validates a => (
presence => 1,
numericality => {
is_integer => 1,
},
with => sub {
my ($self, $attribute_name, $value, $opts) = @_;
$self->errors->add($attribute_name, 'must be positive')
unless $value > 0;
},
);Validate::Tiny
Validate::Tiny only needed to gain two scalars in checks array. Since it knows no types, I added one from Type::Tiny together with the value test:
use Moo;
use Validate::Tiny ':all';
use Types::Standard qw(Int);
my $rules = {
fields => [qw(a)],
checks => [
[qw(a)] => is_required(),
a => sub {
my $value = shift;
return 'must be an int' unless Int->check($value);
return 'must be positive' unless $value > 0;
return;
}
],
};
sub valid
{
my ($self, $data) = @_;
return validate($data, $rules);
}Results
Rate html_formhandler form_toolkit data_muform form_tiny valiant validate_tiny
html_formhandler 671/s -- -50% -88% -97% -98% -99%
form_toolkit 1346/s 101% -- -76% -94% -95% -99%
data_muform 5572/s 731% 314% -- -75% -81% -94%
form_tiny 21853/s 3158% 1523% 292% -- -25% -78%
valiant 28997/s 4223% 2054% 420% 33% -- -71%
validate_tiny 100002/s 14810% 7328% 1695% 358% 245% --HTML::FormHandler, Data::MuForm and Form::Tiny did not move an inch compared to the last benchmark. Valiant and Validate::Tiny seems to have slowed down slightly. Valiant still remains the fastest among the full-fledged frameworks. Suprisingly, Form::Toolkit seems to have speeded up.
Case #3: Are we in your type?
Its time to move to more advanced stuff: now we want a to be a hash reference, containing b, which is an integer number, and c, which is a string:
my $data = {
a => {
b => 5,
c => 'text',
},
};Use any built in types available, and if there isn't one, use Type::Tiny instead.
Data::MuForm
Data::MuForm has Compound type specifically for that. Also, both Integer and Text types are built in:
use Moo;
use Data::MuForm::Meta;
extends 'Data::MuForm';
has_field 'a' => (
type => 'Compound',
required => 1,
);
has_field 'a.b' => (
type => 'Integer',
required => 1,
);
has_field 'a.c' => (
type => 'Text',
required => 1,
);Form::Toolkit
I searched through the documentation a couple of times, but Form::Toolkit doesn't seem to have a way to do nested fields. It seemed to have nested forms, but those only check if the value is a form, not actually validate it nor fill it. Because of that I have to disqualify it. I could write my own field, but that would be going a bit too far.
Form::Tiny
Nested fields are one of the main features of Form::Tiny, achieved with a dot:
use Form::Tiny;
use Types::Standard qw(Str Int);
form_field 'a.b' => (
type => Int,
required => 1,
);
form_field 'a.c' => (
type => Str,
required => 1,
);HTML::FormHandler
Same as Data::MuForm:
use HTML::FormHandler::Moose;
extends 'HTML::FormHandler';
has_field 'a' => (
type => 'Compound',
required => 1,
);
has_field 'a.b' => (
type => 'Integer',
required => 1,
);
has_field 'a.c' => (
type => 'Text',
required => 1,
);
no HTML::FormHandler::Moose;Valiant
I understand Valiant can have a hard time here - Moo properties do not nest after all. However, it still seems to support them, even if it's getting a bit crowded in the validates hash. Also, Valiant does not seem to have its own check for string value (which is pretty much anything else than undef and a reference I believe), so I used one from Type::Tiny:
use Moo;
use Valiant::Validations;
use Types::Standard qw(Str);
has 'a' => (
is => 'ro',
);
validates a => (
presence => 1,
hash => [
[b => presence => 1, numericality => { is_integer => 1 }],
[c => presence => 1, check => Str],
]
);Validate::Tiny
While Validate::Tiny does not understand nesting, I figured out I can just weld two validate calls together, plus a couple of Type::Tiny validate calls on top of that:
use Moo;
use Validate::Tiny ':all';
use Types::Standard qw(Str Int);
my $rules2 = {
fields => [qw(b c)],
checks => [
[qw(b c)] => is_required(),
b => sub {
Int->validate(shift);
},
c => sub {
Str->validate(shift);
},
],
};
my $rules = {
fields => [qw(a)],
checks => [
[qw(a)] => is_required(),
a => sub {
my $value = shift;
return 'not a hash'
unless ref $value eq 'HASH';
my $validated = validate($value, $rules2);
return if $validated->{success};
return $validated->{error};
},
],
};
sub valid
{
my ($self, $data) = @_;
return validate($data, $rules);
}Results
Rate html_formhandler data_muform form_tiny valiant validate_tiny
html_formhandler 513/s -- -78% -96% -97% -99%
data_muform 2377/s 363% -- -83% -84% -94%
form_tiny 13827/s 2596% 482% -- -6% -63%
valiant 14674/s 2761% 517% 6% -- -61%
validate_tiny 37188/s 7150% 1464% 169% 153% --Validating a hash reference has slowed down all the systems noticeably. The gap between Validate::Tiny and HTML::FormHandler is shrinking as the complexity of the validation task increases.
Case #4: Just a couple of things left
The last case I prepared is an array of hashes:
my $data = {
a => [{
b => 5,
c => 'text',
}, {
b => -1,
c => 'another text',
}, ({
b => 1000,
c => 'and another',
}) x N]
};This should not only test the framework's ability to validate such structure, but also whether its performance goes down linearly with data amount, or exponentially. We're going to change the number N at the end a couple of times.
Data::MuForm
Similarly to the Compound type used in the last benchmark, we can use the Repeatable type, which handles both arrays of arrays and arrays of hashes:
use Moo;
use Data::MuForm::Meta;
extends 'Data::MuForm';
has_field 'a' => (
type => 'Repeatable',
required => 1,
);
has_field 'a.b' => (
type => 'Integer',
required => 1,
);
has_field 'a.c' => (
type => 'Text',
required => 1,
);Form::Tiny
Form::Tiny uses * to mark an array, borrowed from Laravel:
use Form::Tiny;
use Types::Standard qw(Str Int);
form_field 'a.*.b' => (
type => Int,
required => 1,
);
form_field 'a.*.c' => (
type => Str,
required => 1,
);HTML::FormHandler
Same as Data::MuForm:
use HTML::FormHandler::Moose;
extends 'HTML::FormHandler';
has_field 'a' => (
type => 'Repeatable',
required => 1,
);
has_field 'a.b' => (
type => 'Integer',
required => 1,
);
has_field 'a.c' => (
type => 'Text',
required => 1,
);
no HTML::FormHandler::Moose;Valiant
Valiant still stands heroically with the help of array validator, but the indentation level is getting out of hand:
use Moo;
use Valiant::Validations;
use Types::Standard qw(Str);
has 'a' => (
is => 'ro',
);
validates a => (
presence => 1,
array => {
validations => [
hash => [
[b => presence => 1, numericality => { is_integer => 1 }],
[c => presence => 1, check => Str],
]
],
}
);Validate::Tiny
The solution here did not change that much, other than an additional if and a foreach loop:
use Moo;
use Validate::Tiny ':all';
use Types::Standard qw(Str Int);
my $rules2 = {
fields => [qw(b c)],
checks => [
[qw(b c)] => is_required(),
b => sub {
Int->validate(shift);
},
c => sub {
Str->validate(shift);
},
],
};
my $rules = {
fields => [qw(a)],
checks => [
[qw(a)] => is_required(),
a => sub {
my $value = shift;
return 'not an array'
unless ref $value eq 'ARRAY';
for my $el ($value->@*) {
return 'not a hash'
unless ref $el eq 'HASH';
my $validated = validate($el, $rules2);
return $validated->{error}
unless $validated->{success};
}
return;
},
],
};
sub valid
{
my ($self, $data) = @_;
return validate($data, $rules);
}Results
For N=3 (5 elements)
Rate html_formhandler data_muform valiant form_tiny validate_tiny
html_formhandler 119/s -- -69% -87% -98% -99%
data_muform 385/s 223% -- -57% -92% -97%
valiant 898/s 654% 133% -- -81% -92%
form_tiny 4851/s 3975% 1160% 440% -- -56%
validate_tiny 11081/s 9209% 2779% 1134% 128% --For N=38 (40 elements)
Rate data_muform html_formhandler valiant form_tiny validate_tiny
data_muform 14.0/s -- -30% -96% -98% -99%
html_formhandler 20.0/s 43% -- -94% -97% -99%
valiant 355/s 2433% 1674% -- -55% -78%
form_tiny 786/s 5511% 3830% 122% -- -51%
validate_tiny 1609/s 11386% 7944% 354% 105% --For N=100 (102 elements)
Rate data_muform html_formhandler valiant form_tiny validate_tiny
data_muform 2.37/s -- -73% -99% -99% -100%
html_formhandler 8.85/s 273% -- -95% -97% -99%
valiant 187/s 7786% 2013% -- -43% -69%
form_tiny 328/s 13741% 3608% 76% -- -46%
validate_tiny 613/s 25742% 6823% 228% 87% --Valiant starts up slower than before, but its losing the least performance as the number of elements grow. Data::MuForm scales the worst among the competitors in this test, losing way over 20 times the performance with just eight as many elements. Even though it's been faster than HTML::FormHandler in the most tests, the last two benchmarks put it at the end of the list.
Conclussion
The big lead that Validate::Tiny had in the first test has been reduced substantially as the complexity of the task grew. That was to be expected: the more ad-hoc code is crafted the less performant it will be compared to an optimized module.
Valiant was very performant, but the syntax becomes mixed when we want to nest fields - top level fields are defined as Moo attributes, while nested fields are only mentioned in the validation hash. Having chosen the Moo properties as the fields to be validated, I believe the focus of the module is most likely elsewhere.
Form::Tiny has shown pretty consistent performance and needed the least code.
Form::Toolkit lacked in features and required too much time to instantiate. Creating custom Moose classes and roles is not the best way to add basic features. I really like its idea though.
Last but not least, Data::MuForm and HTML::FormHandler showed acceptable performance for everything other than a large set of elements. We have to take into account that those are probably the most feature-complete form modules on CPAN.
Edit history
- 2021.10.20
Added HTML::FormHandler
Comments? Suggestions? Send to feedback@bbrtj.eu