277 lines
12 KiB
HTML
277 lines
12 KiB
HTML
<!DOCTYPE html>
|
|
<html>
|
|
<head>
|
|
<title>ProFTPD: Testsuite</title>
|
|
</head>
|
|
|
|
<body bgcolor=white>
|
|
|
|
<hr>
|
|
<center><h2><b>ProFTPD: Testsuite</b></h2></center>
|
|
<hr>
|
|
|
|
<p>
|
|
The proftpd source code is accompanied by a testsuite, for testing various
|
|
features and functionality. At present, the testsuite is divided into two
|
|
main parts: the API-level tests, and the integration tests. The API tests
|
|
are written in C, and exercise the various APIs that proftpd provides to
|
|
module developers. The integration tests are currently written in Perl, and
|
|
seek to check a compiled <code>proftpd</code>'s run-time behavior.
|
|
|
|
<p>
|
|
In order to run the testsuite, you must configure your proftpd build so that
|
|
the testsuite is prepared. You do this by using the <code>--enable-tests</code>
|
|
configure option, along with your other build options, <i>e.g.</i>:
|
|
<pre>
|
|
$ ./configure --enable-tests ...
|
|
</pre>
|
|
|
|
<p>
|
|
<b>API Tests</b><br>
|
|
The API tests use the <code>libcheck</code> library; see:
|
|
<pre>
|
|
<a href="http://check.sourceforge.net/">http://check.sourceforge.net/</a>
|
|
</pre>
|
|
This library must be installed on your machine in order to run the API tests.
|
|
When the <code>--enable-tests</code> configure option is used, proftpd's
|
|
configure script will check for libcheck, and will fail unless the library
|
|
is present. If no <code>libcheck</code> is found and you've used
|
|
<code>--enable-tests</code>, you will see:
|
|
<pre>
|
|
checking for tcase_create in -lcheck... no
|
|
configure: error: libcheck support, required for tests, not present -- aborting
|
|
</pre>
|
|
Note that, if necessary, the <code>--with-includes</code> and
|
|
<code>--with-libraries</code> configure options can be used to specify the
|
|
locations of the <code>libcheck</code> headers and libraries.
|
|
|
|
<p>
|
|
To run the testsuite, use the <code>make check</code> target:
|
|
<pre>
|
|
$ ./configure --enable-tests ...
|
|
$ make
|
|
...
|
|
$ make check
|
|
...
|
|
$ ./api-tests
|
|
Running suite(s): pool
|
|
array
|
|
str
|
|
sets
|
|
timers
|
|
table
|
|
var
|
|
event
|
|
env
|
|
version
|
|
feat
|
|
netaddr
|
|
netacl
|
|
class
|
|
regexp
|
|
expr
|
|
scoreboard
|
|
modules
|
|
100%: Checks: 140, Failures: 0, Errors: 0
|
|
</pre>
|
|
The <code>make check</code> will also go on to run the integration tests,
|
|
if the API tests all pass.
|
|
|
|
<p>
|
|
If one of the API tests fails, you will see an error message like:
|
|
<pre>
|
|
99%: Checks: 140, Failures: 1, Errors: 0
|
|
api/env.c:53:F:base:env_get_test:0: Failed to handle null arguments
|
|
-------------------------------------------------
|
|
FAILED 1 test
|
|
|
|
Please send email to:
|
|
|
|
proftp-devel@lists.sourceforge.net
|
|
|
|
containing the `api-tests.log' file (in the tests/ directory)
|
|
and the output from running `proftpd -V'
|
|
-------------------------------------------------
|
|
</pre>
|
|
Please do follow those instructions; you will be helping in proftpd
|
|
development!
|
|
|
|
<p>
|
|
<b>Expected Failures</b><br>
|
|
Unfortunately, due to variations in system configurations, it's possible
|
|
that some of the API tests will fail, without it necessarily being an
|
|
indication of a regression or problem.
|
|
|
|
<p>
|
|
Most notably, some of the most brittle tests make assumptions about the
|
|
host's <code>/etc/hosts</code> file, particularly the list of names
|
|
associated with the address <code>127.0.0.1</code>. If the
|
|
<code>/etc/hosts</code> file has a name other than "localhost" or
|
|
"localhost.localdomain" immediately after the 127.0.0.1 address, <i>e.g.</i>:
|
|
<pre>
|
|
127.0.0.1 some.other.name localhost
|
|
</pre>
|
|
Then the API tests may fail with errors like this:
|
|
<pre>
|
|
98%: Checks: 152, Failures: 2, Errors: 0
|
|
api/netaddr.c:344:F:base:netaddr_get_dnsstr_test:0: Expected 'localhost or localhost.localdomain', got '<i>some.other.name</i>'
|
|
api/netacl.c:535:F:base:netacl_match_test:0: Failed to positively match ACL to addr: No such file or directory
|
|
</pre>
|
|
If this is the case, it is an <i>expected failure</i>; please do not report
|
|
the issue as a bug. (Unfortunately, I have not been able to think of a good
|
|
solution/workaround for these situations other than ignoring the test failure,
|
|
but I would rather not do that if possible.)
|
|
|
|
<p>
|
|
<b>Integration Tests</b><br>
|
|
The current integration tests are written in Perl, and rely on the
|
|
<code>Test-Unit</code> Perl package. Specifically, you <b>must</b> use
|
|
<code>Test-Unit-0.14</code>, <b>not</b> <code>Test-Unit-0.25</code>. Not sure
|
|
which version of <code>Test-Unit</code> you have, if at all? Run the following
|
|
command:
|
|
<pre>
|
|
$ perl -MTest::Unit -e 'print $Test::Unit::VERSION, "\n";'
|
|
0.14
|
|
</pre>
|
|
|
|
<p>
|
|
To run the integration tests manually, use the <code>tests.pl</code> script:
|
|
<pre>
|
|
$ cd tests/
|
|
$ perl tests.pl
|
|
t/logins.....................ok
|
|
t/commands/user..............ok
|
|
t/commands/pass..............ok
|
|
t/commands/pwd...............ok
|
|
t/commands/cwd...............ok
|
|
t/commands/cdup..............ok
|
|
t/commands/syst..............ok
|
|
t/commands/type..............ok
|
|
t/commands/mkd...............ok
|
|
t/commands/rmd...............ok
|
|
t/commands/dele..............ok
|
|
t/commands/mdtm..............ok
|
|
t/commands/size..............ok
|
|
t/commands/mode..............ok
|
|
t/commands/stru..............ok
|
|
t/commands/allo..............ok
|
|
t/commands/noop..............ok
|
|
t/commands/feat..............ok
|
|
t/commands/help..............ok
|
|
t/commands/quit..............ok
|
|
t/commands/rnfr..............ok
|
|
t/commands/rnto..............ok
|
|
t/commands/rest..............ok
|
|
t/commands/pasv..............ok
|
|
t/commands/epsv..............ok
|
|
t/commands/port..............ok
|
|
t/commands/eprt..............ok
|
|
t/commands/nlst..............ok
|
|
t/commands/list..............ok
|
|
t/commands/retr..............ok
|
|
t/commands/stor..............ok
|
|
t/commands/appe..............ok
|
|
t/config/displayconnect......ok
|
|
t/config/displaylogin........ok
|
|
t/config/maxloginattempts....ok
|
|
t/config/serverident.........ok
|
|
All tests successful.
|
|
Files=37, Tests=141, 160 wallclock secs ( 4.33 cusr + 4.74 csys = 9.07 CPU)
|
|
</pre>
|
|
|
|
<p>
|
|
Some of the integration require root privileges in order to perform the
|
|
test properly, <i>e.g.</i> for
|
|
<code><Anonymous></code>/<code>DefaultRoot</code> logins, <i>etc</i>.
|
|
If you do <i>not</i> run `make check' as root, those tests which require
|
|
root privileges are automatically skipped. Thus for doing a full regression,
|
|
run the integration tests as the root user.
|
|
|
|
<p>
|
|
<b>Adding New Tests</b><br>
|
|
The following information is for those who are interested in adding new
|
|
tests, updating existing tests, or otherwise helping with the testsuite
|
|
development.
|
|
|
|
<p>
|
|
The API tests driver, <code>api-tests</code>, honors the
|
|
<code>PR_TEST_SUITE</code> environment variable. This is useful for running
|
|
just the test cases associated with a particular suite, such as <i>env</i> or
|
|
<i>pool</i>. Only <b>one</b> suite can be specified using the
|
|
<code>PR_TEST_SUITE</code> environment variable:
|
|
<pre>
|
|
$ make check
|
|
$ cd tests/
|
|
$ PR_TEST_SUITE=pool ./api-tests
|
|
Running suite(s): pool
|
|
100%: Checks: 5, Failures: 0, Errors: 0
|
|
</pre>
|
|
This way, when you are adding to an existing test case or adding a new test
|
|
case to an existing suite, you need not run the entire API testsuite in order
|
|
to run your changes.
|
|
|
|
<p><a name="FAQ">
|
|
<b>Frequently Asked Questions</b><br>
|
|
<p>
|
|
<font color=red>Question</font>: I have installed the <code>check</code>
|
|
library, and I am using the <code>--enable-tests</code> configure option, but
|
|
the configure command fails with:
|
|
<pre>
|
|
checking check.h usability... yes
|
|
checking check.h presence... yes
|
|
checking for check.h... yes
|
|
checking for tcase_create in -lcheck... no
|
|
configure: error: libcheck support, required for tests, not present -- aborting
|
|
</pre>
|
|
What am I missing?<br>
|
|
<p>
|
|
<font color=blue>Answer</font>: To properly answer this, we would need to look
|
|
in the log file generated by the configure script, <i>i.e.</i> the
|
|
<code>config.log</code> file.
|
|
|
|
I suspect that looking in that <code>config.log</code> file, you would see
|
|
something like this:
|
|
<pre>
|
|
configure:20568: checking for tcase_create in -lcheck
|
|
configure:20603: gcc -o conftest -g2 -g -O2 -fdebug-prefix-map=/home/hille/devel/proftp/pftp_build/proftpd-dfsg=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -fno-omit-frame-pointer -Werror=implicit-function-declaration -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include/mysql -I/usr/include/postgresql -L$(top_srcdir)/lib -Wl,-z,relro -rdynamic conftest.c -lcheck -ldl >&5
|
|
/usr/lib/gcc/i686-linux-gnu/7/../../../i386-linux-gnu/libcheck.a(check.o): In function `tcase_create':
|
|
(.text+0x1ce): undefined reference to `floor'
|
|
/usr/lib/gcc/i686-linux-gnu/7/../../../i386-linux-gnu/libcheck.a(check.o): In function `tcase_set_timeout':
|
|
(.text+0x513): undefined reference to `floor'
|
|
/usr/lib/gcc/i686-linux-gnu/7/../../../i386-linux-gnu/libcheck.a(check.o): In function `check_get_clockid':
|
|
(.text+0xc1d): undefined reference to `timer_create'
|
|
(.text+0xc48): undefined reference to `timer_delete'
|
|
/usr/lib/gcc/i686-linux-gnu/7/../../../i386-linux-gnu/libcheck.a(check_pack.o): In function `ppack':
|
|
(.text+0x5b1): undefined reference to `__pthread_register_cancel'
|
|
(.text+0x5fc): undefined reference to `__pthread_unregister_cancel'
|
|
collect2: error: ld returned 1 exit status
|
|
</pre>
|
|
|
|
Newer versions of the <code>check</code> library have dependencies on other
|
|
libraries, which need to be declared explicitly. Thus to address these
|
|
dependency linker errors, you would need to re-run your configure command and
|
|
provide those linker dependencies:
|
|
<pre>
|
|
$ make clean
|
|
$ ./configure LIBS="-lm -lrt -pthread" ...
|
|
</pre>
|
|
<i>Note</i> that even after doing the above, you <i>still</i> may have the
|
|
configure command fail to detect the <code>check</code> library. There are
|
|
reports that using <code>check</code> 0.10.0 and later <i>also</i> require the
|
|
<code>subunit</code> library, and thus you may need to use:
|
|
<pre>
|
|
$ make clean
|
|
$ ./configure LIBS="-lsubunit -lm -lrt -pthread" ...
|
|
</pre>
|
|
|
|
<p>
|
|
<hr>
|
|
<font size=2><b><i>
|
|
© Copyright 2017-2018 The ProFTPD Project<br>
|
|
All Rights Reserved<br>
|
|
</i></b></font>
|
|
<hr>
|
|
|
|
</body>
|
|
</html>
|