This document defines the plan for functional testing of POSIX Clocks and Timers. First a list of the features that will and will not be tested is given. Then, an attempt is made to break down the features into possible testing scenarios. Not all of these may be implemented. The assertions.xml file will have the additional detail on which were chosen for implementation. Features To Be Tested ===================== The following features will be tested: clocks - Testing that clocks can accurately keep time in certain scenarios. sleep - Testing that processes can go to sleep for specified periods of time. timers - The following types of timers will be tested. absolute timer, no repeat think: alarm clock relative timer, no repeat think: stopwatch absolute timer, repeats think: grandfather clock relative timer, repeats think: repeating stopwatch Also, timer overruns will be counted. Features Not To Be Tested ========================= API testing of the clocks/timers functions is considered to be done by the conformance section. Load and other stress testing will be done in the stress section. A few error handling tests will be created, but the majority will not for this initial release. (Over time, these may be added from regression tests, or regression tests may suggest additional error handling tests to create.) Approach ======== In order to generate the test cases listed below, the following approaches were taken: - If it was not covered in the conformance area fully, determined the correct behavior of the features listed above and generated test scenarios to verify this behavior happened. - Determined certain critical values that may cause failures to appear and generated test scenarios setting clocks or timers to these critical values. - Determined race conditions that could appear or time dependencies for things happening and generated test cases to expose these issues. Test Case Scenarios =================== Some potential test case scenarios are listed after each functional area. Clocks ------ - Set clock time to: 0, a large number, all the dates that were critical for Y2K testing, set to time before daylight savings (and then after daylight savings). ==> This is probably multiple test cases. - Call clock_gettime() mulitple times and ensure the time is always increasing (call also at nsec increments to ensure both nsec and sec are always increasing). - Ensure we can tell time within acceptable jitter (over some period of time). - Test for a few different clocks. - Test after setting the time. - Send a signal while setting the time. - Kill the process (example of previous with SIGKILL) when setting the time. - Have a timer expire when setting the time. - Set the time from two different processes. Sleep ----- - Mimic the non-repeating timers tests as they apply to sleep. Timers ------ Non-repeating (relative and absolute) - Set timer expiration to: 0, a large number, all the dates that were critical for Y2K testing, set to time before daylight savings (and then after daylight savings). ==> This is probably multiple test cases. Some are absolute timers only. - Have the clock change underneath a timer and ensure it expires correctly. - [Relative only] Set up multiple timers to expire after the same duration and ensure they expire in the order set up. (May need to nanosleep for CLOCKRES nanoseconds in between set ups to get correct results. Also, even if this fails, not necessarily a "bug" since many factors influence expiration time.) - Test out all the various events that can be sent for a timer expiration. - Set two timers to expire at the same time, different events, in the same process. - Set up two timers to expire for the same time in two different processes. - Set up one repeating timer to expire after X time and one non-repeating to expire after 2*X time. Repeating (relative and absolute) - Perform all non-repeating timer tests on repeating timers. - Ensure that these are still going off at the same time, over time. Overruns - Have multiple timers in a process blocked and ensure overrun count for each is correct. - Have a repeating timer blocked for a period of time and ensure overrun count is correct. - Ensure overrun count and timer_gettime() return correct values when SIGEV_NONE is used in ev.sigev_notify. - Try to call timer_getoverrun() as an overrun is happening. Contributors: julie.n.fleischer REMOVE-THIS AT intel DOT com geoffrey.r.gustafson REMOVE-THIS AT intel DOT com rolla.n.selbak REMOVE-THIS AT intel DOT com majid.awad REMOVE-THIS AT intel DOT com salwan.searty REMOVE-THIS AT intel DOT com george REMOVE-THIS AT mvista DOT com