autopkgtest - Debian Perl Group autopkgtest implementation and team practices


The autopkgtest package, which provides as-installed testing of Debian packages, is integrated with pkg-perl packages as of version 3.5.1 (September 2014). The integration is intended to allow for continuous integration testing at

This document describes the implementation and best practices for the team.



 Testsuite: autopkgtest-pkg-perl

in the source package section of debian/control will activate an implicit test control file that's currently embedded in the autopkgtest package. This test control file has an indirection layer that pushes all the test logic into the pkg-perl-autopkgtest package, maintained inside the team. The entry point for the logic is the runner ( script, which runs tests in /usr/share/pkg-perl-autopkgtest/*.d directories.

Thanks to the uniformity of CPAN distributions, this implicit control file works with most (currently about 75%) of the pkg-perl packages. The rest will need package specific tweaks, as detailed below.

An explicit test control file that corresponds to the implicit one described above is included in the pkg-perl-autopkgtest package in /usr/share/doc/pkg-perl-autopkgtest/examples and can be viewed at

In case you want other tests, or need other dependencies or restrictions, you can add an explicit test control file and list them according the the spec manually. You can use the aforementioned default-tests-control file as a starting point.

The current tests activated by Testsuite: autopkgtest-pkg-perl are listed below. If some of them have expected failures, they can be skipped by listing the tests in debian/tests/pkg-perl/SKIP. The file can contain empty lines and comment lines starting with the # sign.



This test re-uses the build time test suite present in most CPAN distributions to check that changes in the package's dependencies haven't introduced regressions in the package functionality.

The package and its build dependencies are installed, the t/ directory and files are copied into a temporary directory, and the build time test suite is run against the installed modules. The temporary directory is used to make sure that the installed modules get used instead of the source tree.

If the test suite needs files outside the t/ directory, those can be listed in the debian/tests/pkg-perl/smoke-files. Empty lines and #-style comment lines are supported. Note that this overrides the default list of t/ and, so you normally need to include those too.

Some common test files are removed from the temporary directory; they currently include pod.t, pod-coverage.t, boilerplate.t, 04critic.t, and 97_meta.t. If the file debian/tests/pkg-perl/smoke-skip exists, it is used as a list of files to be removed. Empty lines and #-style comments are supported.

The environment variables AUTOMATED_TESTING and NONINTERACTIVE_TESTING are set to 1 by default. These can be overridden and others can be added in the configuration file debian/tests/pkg-perl/smoke-env. Empty lines and comments are allowed. The file syntax is one line per variable setting, in the form KEY=VAL; every such line is eval'ed with a shell so you can compute VAL dynamically, for example: DBDPG_INITDB=$(pg_config --bindir)/initdb.

The tests are run with the prove --merge option, which merges and synchronizes stderr and stdout from the test scripts. In some rare cases this option can break the test suite. It can be disabled by unsetting PKG_PERL_PROVE_ARGS in debian/tests/pkg-perl/smoke-env.

For the benefit of tests looking for something in lib/ or blib/, dummy module files are installed there and included in a dummy MANIFEST file. An empty MANIFEST.SKIP is also provided.

If the file debian/tests/pkg-perl/smoke-setup exists and is executable, it will be executed right before running the tests, with the working directory set to the package source and $TDIR set to the test running directory. The testing gets aborted if this program returns a nonzero value. Please use this facility responsibly and only as a last resort if the other hooks don't work for you.

If the file debian/tests/pkg-perl/smoke-cleanup exists and is executable, it will be executed when the smoke tests exit, with the working directory set to the package source and $TDIR set to the test running directory.

The smoke test by default runs prove with the --recurse option on the t/ directory (prior to 0.37 only the t/*.t tests were run). In case a package should run other test scripts, debian/tests/pkg-perl/smoke-tests can be used to declare which files should be used instead.

Example: t/*/*.t



This test ensures that the runtime dependencies are sufficient for the most basic usage of the module. Warnings from such basic usage are considered failures because they can be quite annoying on consumers of the module.

The package and its runtime dependencies are installed, and the command perl -w -MFoo::Bar -e1 is executed, where Foo::Bar is the name of the main module in the package. If this exits with a nonzero status, or if there is output on stderr, the test is considered failed. The test is repeated with PERL_DL_NONLAZY=1 set in the environment; this makes sure there are no unresolved symbols in binary modules.

The name of the module is read from META.json or META.yml, but it can be overridden in debian/tests/pkg-perl/use-name, which also allows listing multiple module names, one per line. The test is skipped if the name isn't present in one of the above sources.

There are some rare cases where warnings are to be expected even from the most basic usage. Such expected warnings can be listed in debian/tests/pkg-perl/use-whitelist. Each line in this file is interpreted as a regular expression.



This test checks that the runtime dependencies and recommendations cover all the requirements of the modules shipped in the package, and that the modules are syntactically valid Perl.

The test first installs the package and its runtime dependencies, including recommendations. All the .pm files in the package are then run through a syntax check with perl -wc If the command exits with a nonzero status, the test is considered failed. Standard error output is not treated as a failure. If there are other files that you'd like to check, you can specify those as regular expressions in debian/tests/pkg-perl/syntax-extra. Don't include the match delimiters (//). Empty lines and #-style comments are supported.

If some of the module files have expected problems passing this test, they can be skipped by listing them in debian/tests/pkg-perl/syntax-skip. The lines are matched as fixed substrings (not regular expressions). Empty lines and #-style comments are supported.

Packages that Suggests other packages are exempt from this test, unless they include a (possibly empty) syntax-skip file. This is because some of the module files are expected to need the suggested packages to work, but there is no mechanism to install them in the environment provided by autopkgtest.

To test all module files despite a Suggests, an empty syntax-skip (with optional comments) can be used since pkg-perl-autopkgtest version 0.45.

Even with the implicit test control file, all the packages would need source uploads to include the Testsuite header, which would take quite a while to happen for all the team packages (currently numbering almost 3000). To speed up adoption, the current plan is to make use a whitelist of known good pkg-perl packages that it will treat as having the Testsuite: autopkgtest-pkg-perl header. This whitelist is maintained at and is based on test runs of all our packages such as

The whitelist is considered a temporary measure that can be dropped once we have Testsuite: autopkgtest-pkg-perl headers in all our packages.

As of May 2015, there are 2029 whitelisted packages, 378 packages that declare Testsuite: autopkgtest-pkg-perl, and 604 non-whitelisted pkg-perl packages that currently don't pass the default checks and therefore need manual work.

The list of packages needing manual work is available at and current as of 20150524, but is not actively updated at the moment (volunteers welcome.)

Team practices

Once the set of known good packages has stabilized and is live on, we should do a mass commit in our git repository adding Testsuite: autopkgtest-pkg-perl headers to the known good packages.

A team specific lintian check (activated by lintian --profile pkg-perl) triggers on the lack of the Testsuite header and points to this document. Coupled with the mass commit above, affected packages are expected to be those that need tweaking to get the tests working. Maintainers encountering the lintian warning are encouraged to fix those (other team members are happy to help with that!), but they are also naturally free to ignore the warning and postpone the addition of the header.

When uploading a package with a Testsuite header, uploaders should adopt a habit of verifying that the tests pass so as not to cause unnecessary work to the volunteers.

Bugs related to autopkgtest failures should be filed with

 Usertags: autopkgtest

so that they show up on the tracking list at;

See for more information about usertags.


To set up a sbuild chroot for autopkgtest (formerly known as adt-run), follow the following steps.

chroot type

You need to setup a chroot type that supports ephemeral sessions for autopkgtest to run properly. It will fail otherwise.

One option is to use lvm-snapshot, so you need to create a logical volume and mount it first. For instance, assuming that /dev/sdb1 is empty and available to be used as a physical volume:

    sudo apt-get install lvm2
    sudo pvcreate /dev/sdb1
    sudo vgcreate sbuild /dev/sdb1
    sudo lvcreate -L2G -n sid sbuild
    sudo mkfs.ext4 /dev/sbuild/sid
    sudo mkdir -p /srv/sbuild/sid
    sudo mount /dev/sbuild/sid /srv/sbuild/sid

A simpler alternative would be to use directory type, which doesn't need any previous setup before it is created, just a regular directory (ephemeral sessions will be setup through union filesystem).


Note that sbuild uses schroot underneath to manage the chroots.

See Enrico Zini's blog entry at and/or sbuild-createchroot(8) for details.


You can use eatmydata do improve the speed of your builds inside the chroot pretty easily.

See for more details.


An example package using debian/tests/pkg-perl/SKIP: libtest-file-sharedir-perl

An example package using debian/tests/pkg-perl/smoke-skip: libtext-csv-xs-perl

An example package using debian/tests/pkg-perl/smoke-files: libnet-ldap-perl

An example package that uses debian/tests/pkg-perl/use-name: libnet-radius-perl

An example package using debian/tests/pkg-perl/syntax-skip: libplack-perl

An example package using debian/tests/pkg-perl/smoke-env: libsvn-notify-mirror-perl

An example package disabling prove --merge in debian/tests/pkg-perl/smoke-env: libterm-progressbar-perl

An example package using debian/tests/pkg-perl/smoke-setup: libnet-sftp-sftpserver-perl

An example package using debian/tests/pkg-perl/smoke-cleanup: libmemcached-libmemcached-perl


Running autopkgtest against a package in the archive, assuming your schroot name is sid-amd64-sbuild:
  autopkgtest libfoo-bar-perl -- schroot sid-amd64-sbuild # since autopkgtest 4.0
  adt-run libfoo-bar-perl --- schroot sid-amd64-sbuild

(Make sure that schroot source:sid-amd64-sbuild has a deb-src entry in sources.list and that you have run apt-get update there at least once before trying this example.)

Running autopkgtest on a .changes file:
  autopkgtest libfoo-bar-perl_0.01_amd64.changes -- schroot sid-amd64-sbuild # since autopkgtest 4.0
  adt-run libfoo-bar-perl_0.01_amd64.changes --- schroot sid-amd64-sbuild
Running autopkgtest with the control information in the current directory but the actual package from the archive:
  autopkgtest -B . -- schroot sid-amd64-sbuild # since autopkgtest 4.0
  adt-run -B .// --- schroot sid-amd64-sbuild

(The source tree doesn't need to be built, the binary package is installed from the archive with apt.)

See /usr/share/doc/autopkgtest/README.running-tests.rst.gz for more information.



Copyright (c) 2014-2022 by the individual authors and contributors noted above. All rights reserved. This document is free software; you may redistribute it and/or modify it under the same terms as Perl itself

Perl is distributed under your choice of the GNU General Public License or the Artistic License. On Debian GNU/Linux systems, the complete text of the GNU General Public License can be found in `/usr/share/common-licenses/GPL' and the Artistic License in `/usr/share/common-licenses/Artistic'.