CPAN is a software modules repository famous for the sheer number of solutions it offers free of charge. Not all of them do different things though, and in some cases its pushing TIMTOWTDI to its limits. Sometimes choosing the right module for the job is easy, but when its not you have to compare them in one way or another.
While speed of execution may not be the most important factor in choosing a solution, it can speak for the code quality and feature design. Two programs written in the same language doing exactly the same thing should have quite comparable run times, but feature creep, code bloat or inefficient algorithms could increase that time beyond acceptable level. Some modules can also implement interesting optimization strategies, while others may not or could not due to different design goals.
In this article, I am going to present benchmark results ran on my machine. You should probably try it out yourself on yours too! Source code for these benchmarks can be viewed at https://github.com/bbrtj/perl-validator-benchmark.
Benchmark setup
We're going to see how the following libraries fare against each other in validating a hash reference:
- Data::MuForm
A form framework that was recommended to me on #perl IRC. Even though I haven't used it in any serious project, it seems pretty comprehensive and is capable of more than just validating data (like rendering). It has some maintenance issues, but has seen some activity lately.
- Data::Sah
A very interesting module that lets you compile your rules into a single expression in Perl, JavaScript or human language. It implements a schema language for validating structures called Sah.
- Form::Toolkit
Moose-based, role-heavy framework that can be extended by creating more Moose classes and roles. It focuses on validating the data and doesn't care from where it came from. Was not updated in years, but still passes all tests and noone has reported any issues.
- Form::Tiny
Lightweight data validator inspired by Laravel validation system, Form::Toolkit and Type::Tiny. It does not contain any field validation code, and instead depends on Type::Tiny constraints to deliver them.
- HTML::FormHandler
Very similar to Data::MuForm, but seems to have more rendering and other non-validation capabilities. The most ++'ed module of them all.
- JSON::Schema::Modern
A comprehensive perl implementation of a validator using json schema.
- JSON::Schema::Tiny
A slimmed down version of JSON::Schema::Modern from the same author.
- Type::Tiny
Pure type-based check. Other validators may already use Type::Tiny, but here we just construct a pure Type::Tiny nested structure and validate with that. Yes, that's possible.
- Valiant
Recent addition to CPAN, presented at the last conference. Inspired by Ruby on Rails and meant to be used together with Moo. Marked as early release in the documentation.
- Validate::Tiny
Possibly the smallest validation library on CPAN. May be basic, but thanks to that you have full control over what's going on, and hopefully better performance.
- Validator::LIVR
Perl implementation of Language Independent Validation Rules.
- Whelk
Not actually a full validation framework, but rather lightweight engine with JSON Schema-like validation for use in the Whelk API framework.
- Json::Schema::Validate
Newest take on JSON schemas, with some interesting features like generating a pure JS check.
System info
Following benchmarks were run on Thinkpad T480 (Intel i7-8650U) running Slackware 15.0. The machine was plugged in to AC and a performance cpu scaling mode was enabled, which keeps CPU cores at maximum frequency all the time. We use Dumbbench with 1% target precision and a reasonable number of initial runs. Machine was not under any other significant load during the benchmark runtime.
To equalize the playing field, each framework will bless some kind of object, even if it does not require any object to run. It will use that object to cache its internal state, whatever it may be.
Perl 5.42.0 (built by perlbrew, not threaded) was used with latest available CPAN dependencies installed by Carmel. Some optional XS modules like Type::Tiny::XS were installed prior to benchmarking.
Case #1: a single field
This will be the most basic hash reference with just a single value:
{
a => 2
}We don't check for the value here, we just want 'a' existence in $data to be ensured.
Results
Rate Error margin Speedup vs previous
HtmlFormHandler 3215/s 0.06% --
JsonSchemaModern 5842/s 0.05% 81%
JsonSchemaTiny 18640/s 0.03% 219%
JsonSchemaValidate 24871/s 0.11% 33%
DataMuForm 26088/s 0.04% 4%
FormToolkit 57237/s 0.03% 119%
FormTiny 70707/s 0.02% 23%
ValidateTiny 112522/s 0.04% 59%
Valiant 115774/s 0.06% 2%
Whelk 277315/s 0.13% 139%
DataSah 526814/s 0.11% 89%
ValidatorLivr 720051/s 0.02% 36%
TypeTiny 1086200/s 0.02% 50%Case #2: object creation
This is the same as the previous benchmark, but there's a twist - we do not cache the constructed object, it has to be created anew for each validation. This lets us judge how each framework would perform in a scenario where it's hard or impossible to cache the object for later.
Note that this isn't completely fair, since some frameworks may store form state in global variables (attached to the class, not the object), Form::Tiny being one example.
Results
Rate Error margin Speedup vs previous
DataSah 603/s 0.01% --
JsonSchemaModern 913/s 0.01% 51%
FormToolkit 946/s 0.1% 3%
HtmlFormHandler 1120/s 0.04% 18%
DataMuForm 5290/s 0.02% 372%
JsonSchemaValidate 6889/s 0.07% 30%
JsonSchemaTiny 16471/s 0.03% 139%
ValidatorLivr 37547/s 0.04% 127%
Valiant 48167/s 0.09% 28%
FormTiny 61203/s 0.08% 27%
Whelk 63192/s 0.02% 3%
ValidateTiny 78231/s 0.03% 23%
TypeTiny 104405/s 0.12% 33%Case #3: multiple fields
A little more complex case, which involves five fields which are all required and string:
{
a => 'test1',
b => 'test2',
c => 'test3',
d => 'test4',
e => 'test5',
}Results of this case can be used to determine how efficiently each framework is traversing a flat structure.
Results
Rate Error margin Speedup vs previous
JsonSchemaModern 1431/s 0.02% --
HtmlFormHandler 1615/s 0.03% 12%
JsonSchemaTiny 2300/s 0.04% 42%
JsonSchemaValidate 6992/s 0.03% 204%
DataMuForm 8936/s 0.03% 27%
FormToolkit 19726/s 0.1% 120%
Valiant 24882/s 0.01% 26%
ValidateTiny 28978/s 0.07% 16%
FormTiny 32307/s 0.02% 11%
Whelk 97600/s 0.01% 202%
ValidatorLivr 140270/s 0.01% 43%
DataSah 149084/s 0.02% 6%
TypeTiny 324131/s 0.03% 117%Case #4: array of nested hashes
The last case is an array of 100 hashes:
{
a => [{
b => 5,
c => 'text',
}, {
b => -1,
c => 'another text',
}, {
b => 1000,
c => 'and another',
}, # and 97 hashes more
]
}This should not only test the framework's ability to validate such structure, but also whether its performance goes down linearly with data amount, or exponentially.
Results
Rate Error margin Speedup vs previous
DataMuForm 1.9/s 0.06% --
HtmlFormHandler 7.6/s 0.05% 300%
JsonSchemaModern 29/s 0.19% 281%
JsonSchemaTiny 44/s 0.2% 51%
JsonSchemaValidate 118/s 0.07% 168%
Valiant 175/s 0.05% 48%
FormToolkit 289/s 0.15% 65%
ValidateTiny 461/s 0.41% 59%
FormTiny 822/s 0.2% 78%
Whelk 1506/s 0.04% 83%
DataSah 2190/s 0.24% 45%
ValidatorLivr 2241/s 0.58% 2%
TypeTiny 9100/s 0.15% 306%Conclusion
Left as an exercise for the reader. Now that the numbers are out, you can do with them as you please.
Changelog
- Thu Aug 24 2023: Updated the frameworks and redone the benchmarks, added benchmark machine specs
- Wed Mar 5 2025: Added Whelk, updated the frameworks and redone the benchmarks
- Tue Nov 25 2025: Added Json::Schema::Validate, improved other JSON Schema implementations, updated the frameworks and redone the benchmarks
- Wed Nov 26 2025: All benchmarks have been revisited to ensure more accurate and fair assessment of performance
This article has been rewritten. The old version can be viewed here
Comments? Suggestions? Send to bbrtj.pro@gmail.com
Published on 2021-10-19