Warning Letter: Multiple mistakes in the study resulted in errors in the calculations, use of formulas, and statistical analyses. (ucm623581)

 Calculations could not be validated due to the PDF format (rather than Excel format) of the amended report, which did not include underlying calculations and formulas utilized.

View the original warning letter

Validation Methodology: Documenting and Testing Functions Directly

Calculations represent one of the highest risk components of any computer validation project. When results are displayed on a form or report, users tend to accept those results as correct. Validation has to ensure that the results are actually correct. One common method to validate calculations is to enter a few known values and verify that the expected results are returned. However, even if the results of a calculation are correct in a few cases that does not mean the underlying formula is correct. For that reason, Ofni Systems’ preferred method is to directly verify the function or formula.

The User/Functional Requirements is the appropriate place to document what each formula is supposed to calculate. All of the components of the function should be explained, including any Boolean logic that is used to capture errors or unusual conditions. For example:

• Force = Mass x Acceleration
• With valid input, Current = Voltage / Resistance. If Resistance = 0, then Current is NULL.

The Design Specification is the appropriate place to document the actual calculation. It is a good practice for the author of the Design Specification to document the arguments in the calculation to ensure that the calculation does represent the intended calculation; if the author notices that the calculation does not match the requirement, this should be brought to the appropriate person’s attention. For example:

• Force = Mass x Acceleration. In Excel this is expressed as Cn = (An*Bn), where Column A is Force, Column B is Acceleration and n is row number.
• Current = IF (Resistance = 0) THEN 0 ELSE (Voltage / Resistance)

The test protocol then ties the User/Functional Requirements to the Design Specification. The protocol procedure describes the requirement. The tester documents the calculation in the results along with the individual components of the calculation. The tester then verifies that the calculation accurately describes the requirement. This does require a tester who is has enough experience to review the formulas, compare them with the requirement and verify that the formula meets the requirement. It also requires that the functions are directly accessible by the tester. The advantage of this method is that the function itself has been directly verified and validated, rather than a few selected inputs.