OyjlTest Code and Tools Testing

API testing for prototyping and regression checking in CI. More...

Collaboration diagram for OyjlTest Code and Tools Testing:

Macros
#define	TEST_RUN(prog, text, do_it)
#define	OYJL_TEST_NAME   argv[0]
#define	OYJL_TEST_MAX_COUNT   64
#define	OYJL_PRINT_SUB_LENGTH   "OYJL_PRINT_SUB_LENGTH"
#define	PRINT_SUB_BASIC(result_, ...)
	Register status and print info of sub test. More...
#define	PRINT_SUB(result_, ...)
#define	PRINT_SUB_INT(result_, count_, ...)
#define	PRINT_SUB_PROFILING(result_, integer_, duration_, term_, ...)
#define	OYJL_TEST_WRITE_RESULT(mem, size, hint, suffix)
	Store test data. More...
#define	OYJL_TEST_MAIN_SETUP
	setup Oyjl to your needs by defining this macro More...
#define	OYJL_TEST_MAIN_FINISH
	end your test program as you need by defining this macro More...

Enumerations
enum	oyjlTESTRESULT_e {   oyjlTESTRESULT_SYSERROR, oyjlTESTRESULT_FAIL, oyjlTESTRESULT_XFAIL, oyjlTESTRESULT_SUCCESS,   oyjlTESTRESULT_UNKNOWN }

Functions
oyjlTESTRESULT_e	oyjlTestRun (oyjlTESTRESULT_e(*test)(void), const char *test_name, int number)
const char *	oyjlIntToString (int integer)
const char *	oyjlProfilingToString (int integer, double duration, const char *term)
const char *	oyjlPrintSub (int space, int right, const char *format,...)
	print test results in a canonical way More...
const char *	oyjlPrintSubProfiling (int space, int integer, double duration, const char *term, const char *format,...)
	print test results with profiling in a canonical way More...
oyjlTESTRESULT_e	displayFail ()
int	main (int argc, char **argv)
	simple start function for testing program More...

Variables
FILE *	zout
int	oyjl_print_sub_length = 51
	default for printed columns More...
oyjlTESTRESULT_e	oy_test_last_result = oyjlTESTRESULT_UNKNOWN
	Result of last sub test. More...
char *	oyjl_print_sub = 0
int	verbose = 0
	print more results, when the -v argument is passed to the test program. More...

Detailed Description

API testing for prototyping and regression checking in CI.

The API is designed to be easily useable without much boilerplate. The implementation is declared and defined in header only. No extra linking is needed, except of libm and libc where required.

Define somewhere in your test.c file a TESTS_RUN with your test functions like:

#define TESTS_RUN \
  TEST_RUN( testVersion, "Version matching", 1 ); \
  TEST_RUN( testJson, "JSON handling", 1 ); \
  TEST_RUN( testFromJson, "Data Writers", 1 ); \
  TEST_RUN( testJsonRoundtrip, "Data Readers", 1 );

Then include simply the oyjl_test_main.h header and it defines a main() function for you to handle command line parsing, statistics and summary printing after test program finish.

Include oyjl.h before oyjl_test_main.h, in case you need it.

A complete test file is the self test. It includes the oyjl_test_main.h and implicitely other oyjl_test.h headers and compiles without additional dependencies:

Macro Definition Documentation

OYJL_PRINT_SUB_LENGTH

#define OYJL_PRINT_SUB_LENGTH   "OYJL_PRINT_SUB_LENGTH"

Macro to override te name of the OYJL_PRINT_SUB_LENGTH environment variable. The environment variable will be used to detect the desired print width inside sub tests. The oyjl_print_sub_length is set accordingly.

Referenced by oyjlTestRun().

OYJL_TEST_MAIN_FINISH

#define OYJL_TEST_MAIN_FINISH

end your test program as you need by defining this macro

Place this macro in front of your #include oyjl_test_main.h. The macro is called as last entry inside oyjl_test_main.h defined main().

#define OYJL_TEST_MAIN_FINISH printf("\n    My Test Program finished\n\n");

OYJL_TEST_MAIN_SETUP

#define OYJL_TEST_MAIN_SETUP

setup Oyjl to your needs by defining this macro

The macro is called as first entry inside oyjl_test_main.h defined main().

#include <oyjl.h> // declare oyjlDebugVariableSet()
int my_debug_variable = 0;

#define OYJL_TEST_MAIN_SETUP oyjlDebugVariableSet( &my_debug_variable ); printf("\n    My Test Program\n");
#include <oyjl_test_main.h> // inject code from OYJL_TEST_MAIN_SETUP into main() start

The above example let your code set your own debug variable. So you can see if the -v -v option was set, as that increases *oyjl_debug += 1. Or you can e.g. redefine zout to default to a FILE pointer instead of stdout.

Referenced by main().

OYJL_TEST_MAX_COUNT

#define OYJL_TEST_MAX_COUNT   64

Macro to override inbuild default of 64 maximum array length for storing test results.

OYJL_TEST_NAME

#define OYJL_TEST_NAME   argv[0]

Macro to set the global test name. It will be used as file name component. in OYJL_TEST_WRITE_RESULT.

OYJL_TEST_WRITE_RESULT

#define OYJL_TEST_WRITE_RESULT	(		mem,
			size,
			hint,
			suffix
	)

Value:

if(mem) { \
  char * fn = (char*)malloc((hint?strlen(hint):0)+64); \
  sprintf( fn, "%s-%d-%d-%s-%s.%s", OYJL_TEST_NAME, oyjl_test_number, oy_test_current_sub_count, hint?hint:"", oyjlTestResultToString(oy_test_last_result,0), suffix ); \
  oyjlWriteTestFile( fn, mem, size ); \
  if(verbose) fprintf(zout, "%s\n", fn); \
  if(oy_test_last_result == oyjlTESTRESULT_FAIL || oy_test_last_result == oyjlTESTRESULT_XFAIL) { \
    char * fns = (char*)malloc((hint?strlen(hint):0)+64); \
    sprintf( fns, "%s-%d-%d-%s-%s.%s", OYJL_TEST_NAME, oyjl_test_number, oy_test_current_sub_count, hint?hint:"", oyjlTestResultToString(oyjlTESTRESULT_SUCCESS,0), suffix ); \
    oyjlBT_test \
    FILE * fp = fopen(fns, "r"); \
    if(fp && strcmp(suffix, suffix) == 0) { \
      char * diff = (char*)malloc((hint?strlen(hint)*2:0)+128); \
      sprintf( diff, "diff -aur %s %s", fns, fn ); \
      system(diff); \
      free(diff); \
      fclose(fp); \
    } \
    free(fns); \
  } \
  free(fn); \
}

Store test data.

Write test data to a file for easy error comparision. Use the macro after the PRINT_SUB() of the belonging sub test. The macro will generate the file name containing test related infos, like test number and sub test number, status and possibly your provided hint. That way you can easily find your test data in the writeable test directory. In case one test was successful followed by a failed test and the suffix is "txt", then the diff of the successful and the failed file will be shown.

char * text = myApiTesting(); // shall return "good result"
if(strcmp(text,"good result") == 0) // good
  PRINT_SUB_INT( oyjlTESTRESULT_SUCCESS, strlen(text), "myApiTesting()" );
else // fail
  PRINT_SUB( oyjlTESTRESULT_FAIL, "myApiTesting()" );
OYJL_TEST_WRITE_RESULT( data, strlen(text), "myApiTesting", "txt" )

Parameters

mem	memory pointer
size	size of mem to write to file
hint	optional text to place inside the file name
suffix	file name suffix

PRINT_SUB

#define PRINT_SUB	(		result_,
			...
	)

Value:

{ PRINT_SUB_BASIC( result_, \
    oyjlPrintSub(-1,-1, ## __VA_ARGS__ ) ); \
}

Like PRINT_SUB_BASIC but with variable columns size.

PRINT_SUB_BASIC

#define PRINT_SUB_BASIC	(		result_,
			...
	)

Value:

{ oy_test_last_result = result_; \
  if((result_) < oyjlTESTRESULT_SUCCESS && (result_) < result) OYJL_TEST_START \
  if((result_) < result) \
    result = result_; \
  fprintf(stdout, ## __VA_ARGS__ ); \
  fprintf(stdout, " .. %s", oyjlTestResultToString(result_,1)); \
  if((result_) <= oyjlTESTRESULT_FAIL) \
    fprintf(stdout, " !!! ERROR !!!" ); \
  fprintf(stdout, "\n" ); \
  ++oy_test_sub_count; \
  ++oy_test_current_sub_count; \
}

Register status and print info of sub test.

Print a custom line to stdout followed by the status. Register state.

The PRINT_SUB_BASIC macro remembers the first file position of similar strongly failed sub tests. As macros count the last closing brace ')', the line number is set to (__LINE__ - 1). So it is suggested to place the status macro in one line to let the position fall in front or use a two line macro with falling the debug position in the start of the macro. Here two examples:

int i = 4; // debugging hint will show this line
if(i != 2) PRINT_SUB(oyjlTESTRESULT_FAIL, "i = %d", i);

if(i == 2)
{
  PRINT_SUB_INT( oyjlTESTRESULT_SUCCESS, i,
  "i =" );
}
else
{
  // debugging hint will point to line below
  PRINT_SUB( oyjlTESTRESULT_XFAIL,
  "i = %d", i );
}

Parameters

result_	use oyjlTESTRESULT_e for
...	the argument list to fprint(stdout, ...)

See also

PRINT_SUB PRINT_SUB_INT PRINT_SUB_PROFILING

PRINT_SUB_INT

#define PRINT_SUB_INT	(		result_,
			count_,
			...
	)

Value:

{ PRINT_SUB_BASIC( result_, \
    oyjlPrintSub(-1, count_, ## __VA_ARGS__ ) ); \
}

Like PRINT_SUB_BASIC but with variable columns size and right side integer print.

PRINT_SUB_PROFILING

#define PRINT_SUB_PROFILING	(		result_,
			integer_,
			duration_,
			term_,
			...
	)

Value:

{ PRINT_SUB_BASIC( result_, \
    oyjlPrintSubProfiling(-1, integer_, duration_, term_, ## __VA_ARGS__ ) ); \
}

Like PRINT_SUB_BASIC but with variable columns size and right side profiling print.

TEST_RUN

#define TEST_RUN	(		prog,
			text,
			do_it
	)

Value:

oyjlTESTRESULT_e prog(void); \
{ \
  if(argc > argpos && do_it) { \
      for(i = argpos; i < argc; ++i) \
        if(strstr(text, argv[i]) != 0 || \
           atoi(argv[i]) == oyjl_test_number ) \
          oyjlTestRun( prog, text, oyjl_test_number ); \
  } else if(list) \
    printf( "[%d] %s\n", oyjl_test_number, text); \
  else if(do_it) \
    oyjlTestRun( prog, text, oyjl_test_number ); \
  ++oyjl_test_number; \
}

macro to register a test

See also

TESTS_RUN

Parameters

prog	test function: oyjlTESTRESULT_e (*test)(void)
text	name of the test
do_it	enable the test - usually 1

Enumeration Type Documentation

oyjlTESTRESULT_e

enum oyjlTESTRESULT_e

test result categories

Enumerator
oyjlTESTRESULT_SYSERROR	system error
oyjlTESTRESULT_FAIL	test failed and will error
oyjlTESTRESULT_XFAIL	non critical failure
oyjlTESTRESULT_SUCCESS	tested and verified feature
oyjlTESTRESULT_UNKNOWN	unknown error

Function Documentation

displayFail()

oyjlTESTRESULT_e displayFail

(

)

helper to check for availablity of display environment for test status

References oyjlTESTRESULT_FAIL, and oyjlTESTRESULT_XFAIL.

main()

int main	(	int argc	,
		char **	argv
	)

simple start function for testing program

The OYJL_TEST_MAIN_SETUP macro can be used to do something initially.

Use the verbose variable in your test code to enable additional result printing. The test command argument '-v' will set the verbose variable to 1, default is 0.

The OYJL_TEST_MAIN_FINISH macro can be used to do something after all tests.

Examples:

image2pdf.c, image_display.cpp, oymd5icc.c, tutorial1.c, tutorial_json_options.c, and tutorial_taxi.c.

References OYJL_TEST_MAIN_SETUP, oyjlTESTRESULT_UNKNOWN, and zout.

oyjlIntToString()

const char* oyjlIntToString

(

int

integer

)

helper to print a number

oyjlPrintSub()

const char* oyjlPrintSub	(	int	space,
		int	right,
		const char *	format,
			...
	)

print test results in a canonical way

The funktion is not reentrant. You need to copy the result if calling repeatedly.

Parameters

[in]	space	number line width: use -1 for default of 51
[in]	right	number for right alignment: use -1 to omit
[in]	format	printf format
[in]	...	variable argument list for format

Returns

return static string in oyjl_print_sub

Version

Oyjl: 1.0.0

Date

2021/09/29

Since

2021/09/29 (Oyjl: 1.0.0)

References OYJL_CREATE_VA_STRING, oyjl_print_sub_length, and oyjlRegExpReplace().

oyjlPrintSubProfiling()

const char* oyjlPrintSubProfiling	(	int	space,
		int	integer,
		double	duration,
		const char *	term,
		const char *	format,
			...
	)

print test results with profiling in a canonical way

The funktion is not reentrant. You need to copy the result if calling repeatedly.

Parameters

[in]	space	number line width: use -1 for default columns or 51
[in]	integer	arg like for oyjlProfilingToString()
[in]	duration	arg like for oyjlProfilingToString()
[in]	term	arg like for oyjlProfilingToString()
[in]	format	printf format
[in]	...	variable argument list for format

Returns

return static string in oyjl_print_sub

Version

Oyjl: 1.0.0

Date

2021/10/08

Since

2021/10/08 (Oyjl: 1.0.0)

References OYJL_CREATE_VA_STRING, oyjl_print_sub_length, and oyjlRegExpReplace().

oyjlProfilingToString()

const char* oyjlProfilingToString	(	int	integer,
		double	duration,
		const char *	term
	)

helper to print tempo

oyjlTestRun()

oyjlTESTRESULT_e oyjlTestRun	(	oyjlTESTRESULT_e(*)(void)	test,
		const char *	test_name,
		int	number
	)

run a test and print results on end

Parameters

test	test function
test_name	short string for status line
number	internal test number

Handle columns width by checking OYJL_PRINT_SUB_LENGTH envar or using terminal window size by asking linux ioctl or use inbuild default.

Print header line and title.

Print sumarisation of sub test.

References OYJL_PRINT_SUB_LENGTH, and oyjlTESTRESULT_UNKNOWN.

Variable Documentation

oy_test_last_result

oyjlTESTRESULT_e oy_test_last_result = oyjlTESTRESULT_UNKNOWN

Result of last sub test.

oyjl_print_sub

char* oyjl_print_sub = 0

Storage location of last return from oyjlPrintSub() and inside the PRINT_SUB() family respectively.

oyjl_print_sub_length

int oyjl_print_sub_length = 51

default for printed columns

See also

OYJL_PRINT_SUB_LENGTH

Referenced by oyjlPrintSub(), and oyjlPrintSubProfiling().

verbose

int verbose = 0

print more results, when the -v argument is passed to the test program.

zout

FILE* zout

FILE descriptor for printed inbetween results

A good default might be stdout for a CLI program.

Examples:

tutorial_json_options.c.

Referenced by main().