What to look for: Control

• Start by stimulating simple sequences.
• Do the interfaces go through the appropriate phases?
• Are the FSMs behaving as expected? (Observe internal state)
• Have all the transitions in the state diagram been observed …
• … for all the reasons that might trigger them?
• Termination — does it?
• (test coverage – have you got it covered?

e.g. a new processor processor fed with ‘NOP’s

• resets and starts
• performs fetch/decode/execute sequence
• can stall if memory is slow
• …

‘Families’ of tests

• Control may include a test for data stability even though the value is not checked
• Is the number of cycles performed ‘control’ or ‘data’?

Thinking up test cases

Consider an ALU in a pipelined processor design. This performs a given set of operations, typically one per clock cycle although pipeline stalls and flushes may interfere with this. May want to check:

• Normal data flow through unit
• That the output retains data when stalled
• If the output is stalled, does the input stall?
  • This may be under local or remote (external) control.
• Is data validity passed through correctly?
• Is an attempt to stall the output ignored if the stage contains invalid data?
  • this would be an optimization for performance
• Does the unit ADD, SUB, AND, etc.?
  • signed, unsigned values, shifts …
• If an input is unused (e.g. MOV ‘second operand’) is it genuinely ignored?
  • A use for ‘don't care’ test values, perhaps?
• …

Each test is simple in itself but they can be numerous, even in a ‘simple’ unit! It is likely that this list is not exhaustive.

Example: An interface between blocks

In the laboratory system the pixel output interface from the drawing engine has the following timing rules.

There are two control signals {req, ack} with an associated bundle of data. Following an active clock edge:

• if req is not asserted, ack should not be asserted.
• if req is asserted, ack may be asserted to indicate the acceptance of data.
• ack will be asserted for a single clock cycle for each data item accepted.

In the figure:

• The first transfer goes through immediately.
• The second transfer has to wait for one cycle.
• The third and fourth transfers do not wait and follow each other as closely as possible — this takes two clock cycles because ack must pulse.

Notes:

• This interface has been specified to be quite simple to use.
• Small delays have been incorporated into the figure for clarity.
• This is not a full ‘handshake’ interface as data bursts can go through without separate request transitions.

Do the tests cover all these cases?

Is (for example) the data held stable when the second transfer waits?

What to look for: Data

Just because a design runs through appropriate control sequences this doesn't mean that the output data is correct.

In an RTL abstraction the details of the ‘data’ are not really considered. However it is important to check that the data are also correct.

• Data values can be compared with independently generated test results
• Some data faults become readily apparent
  • out of range values
  • the wrong number of operations (cycles) are performed
  • ‘silly’ graphics drawing … assuming the values can be viewed
• Self-testing may be possible
  • a processor can be tested by running some software
  • it will probably crash if there is an error in (e.g.) instruction decode

If control signals are interacting correctly the data can be checked. Probably the first thing to establish is that the data is stable when it is expected to be. It is not much use if a pipeline's control signals stall correctly but the data tries to keep flowing.

The data can be sampled for correctness at the time an interacting unit would do so. This requires some expected results for comparison, which could be generated in a HDL test harness or imported from another model. Note that the expected results should have been verified in some way.

Data crossing interfaces should be deducible (if not directly available) in a higher level model. Such a model can be used in various ways to aid testing.

• to generate stimuli for a HDL model
• to generate a file of expected test results
• as part of a cosimulation where it interacts with the HDL simulation

A high-level model may also be able to generate internal variable sequences with little extra effort.

Exhaustive tests

It is possible to test some units exhaustively, i.e. try every possible case. These are usually combinatorial units as each bit of buried state doubles the number of tests required.

There are 2⁽⁸⁺⁸⁾ = 65536 possible input combinations. It is quite feasible to test this and it is possibly the simplest thing to do.

Counterexample: 32 × 32 multiplier block

There are 2^(32 + 32) = 18446744073709551616 possible input combinations.

Checking at one per nanosecond would still require over 500 years to complete.

Exhaustive tests are feasible for control circuits, not for (most) data.

Representative samples

If exhaustive testing is impossible some sample data are needed. There are various ways of generating these:

• hand generated
• algorithmically generated
• random
• biased random
• …

Each method has merits and drawbacks. Example: a set of random inputs to a divider may be unlikely to test the ‘divide by zero’ case. Random inputs could be biased so that (for example) small numbers are more likely if a block which counts leading zeros is under test.

Data timing

Timing checks require some more detail in the model. Logic timings are difficult to estimate at an early simulation stage; however there are some timing checks which it may be worth modelling early.

A good example is reading a memory. On the laboratory board there is external SRAM to provide capacity for the frame buffer. This SRAM has an access time of 55 ns – which is more than one clock period (40 ns). A simple HDL RAM model might supply data as soon as the address was valid. In this case the data could be latched (accidentally) after one cycle and appear to work.

Even more worryingly, the 55 ns is a worst-case time; a real SRAM device may meet the 40 ns deadline … under some conditions. This could lead to many field failures, e.g. when the weather warms up or the battery is below half charge.

A more sophisticated model, which forces data ‘unknown’ until the relevant time has elapsed, would expose this fault because the ‘unknown’s would be latched and propagated.

What to look for: Timing

Although it may not be possible practical to predict units' timing relationships, even on a cycle-by-cycle basis, it may be useful to monitor some sequence properties for verification.

Examples:

• Is the correct number of operations performed?
  • e.g. when drawing is the number of pixels plotted correct?
• Is something happening?
  • Is a timeout detector appropriate?
• …

Back (up) to testing.

Forward to test coverage.