V&V and SQA: Part 2, Requirements for Production-Grade Software
The requirements for release of software for production-grade applications include:
Requirement I: Documentation. Adequate documentation must exist that:
1. Describes the theory and assumptions made in developing all mathematical models and analytical and numerical solution methods for the code.
2. Describes the code structure, logic, and overall execution procedures.
3. Describes in detail how to use the code.
4. Describes how to install the code on a user’s computer system.
Requirement II: Verification. The code must be verified to assure:
1. The coding is correct with respect to the code specification document.
2. The numerical solution methods are consistent, stable, and convergent.
3. The code is correctly solving the equation set of the models.
4. Mathematical methodologies demonstrate expected theoretical performance of the solution methods.
Requirement III: Validation. The code must be validated to perform the analysis required of it by:
1. Comparison of code predictions with relevant experimental data.
2. Comparison of code results with results from other, approved, calculational techniques.
3. Assuring that code results are consistent with the physical phenomena of interest.
Requirement IV: Qualification. The code must be demonstrated to be qualified for use in its intended applications areas by:
1. Demonstrating that the software and its users can correctly and accurately apply the software to analyses for which it is designed.
2. Providing users with sufficient guidance and training for use of the code and its output.
Requirement V: Quality Assurance. The production-level versions of the software must be maintained under approved quality assurance procedures.
Various aspects of the requirements listed above for release of software for production use are usually conducted as software is developed. However, in many cases, these activities are generally conducted by personnel who are developing the software and thus are not independent and, in addition, the results are sometimes not formally documented. In other cases, the requirements may be conducted by organizations independent of the developers, but they are not conducted with the exact version of the software that will be released.
Formal V&V and SQA procedures, on the other hand, must be applied by an independent group, in a thorough and systematic manner to a fixed product. And, the results of the procedures must be documented in such a way that they can be easily and completely understood by any other independent organizations that may have an interest in the software.
Code Specification Documents
If software is developed from a fresh start under formal software design and development procedures almost all of the production-level requirements will be natural products of the software design process. Many software codes for complex real-world problems, unfortunately, have not been developed under formal, controlled software design processes. In these cases the documentation must be developed prior to conducting the independent V&V procedure; a situation that has been known to occur on more than one occasion. In the absence of products from formal software-design procedures, the code documentation is considered to be the specification documents for the software.
Specification documents are required in order that the as-built software can be verified to conform to the specifications. Software designed, developed and constructed from fresh start under formal software-development concepts will, of course, be based on formal specifications. Almost all software developed for inherently complex physical phenomena and processes, on the other hand, is not developed under fresh-start conditions. Instead the software has evolved, many times over time spans measured in decades. Under these conditions, evolution of the documentation of the software, as the software evolves, is critically important. In the absence of formal specification documents, the software documentation becomes the specification documents.
In general, Requirement I above (documentation) requires a level of detailed documentation that is seldom attained for software that does not undergo verification, validation, and quality assurance procedures. Each and every equation and mathematical relationship used in the software must be written somewhere. Documentation for production-level V&V and SQA software must be comprehensive and usually includes at the minimum the following set of manuals:
Volume 1: Model Theory and Solution Methods Manual. A theory manual in which the derivations of each and every equation, both the continuous and discrete formulations, and the solution methods applied to the discrete approximations are given in sufficient detail that the final equations for the models can be obtained by independent interests.
Volume 2: Computer Code Manual. A computer code manual in which the code is described in sufficient detail that independent outside interests can understand the source code.
Volume 3: User’s Manual. A user’s manual that describes how to develop the input for the code, perform the calculations and understand the results form the calculations.
Volume 4: Verification and Validation Manual. A manual or reports in which the verification and validation of the basic functions of the software and example applications are given.
Volume 5: Qualification Manual. Additional manuals or reports in which the models and methods, software and user are demonstrated to be qualified for application to analyses of the intended application areas.
Other reports and papers can be used to supplement the above documentation. These then become a part of the official record of the Design Review. The specific volume numbers given above are for citations in the following discussions.
The technical evaluation criteria and associated success metrics for the production-level software requirements are discussed in the following paragraphs.
Documentation Technical Evaluation Criteria and Success Metrics
An independent review of the documentation to ensure that the production-level software requirements of Requirement I have been met could use the following statement of the documentation requirements and technical evaluation criteria and success metrics to show that the requirement has been met:
Requirement I: Part 1. Documentation Requirement. Is the model documentation sufficient to provide an understanding of the basis and derivation of all aspects of the model?
Technical Evaluation Criteria and Success Metric. A positive response to at least one of the following is required:
a. The model is correctly derived based on the stated assumptions.
b. The model matches the original references.
c. The model is standard practice for this type of application.
d. The model is satisfactory based on sound expert judgment.
Specification Document. Volume 1 of the computer code manuals, the theory and numerical analysis manual, will be the specification document for this requirement.
Requirement I: Part 2. Documentation Requirement. Adequate documentation exists which describes the code structure, logic, and overall execution procedures.
Technical Evaluation Criteria and Success Metric. A positive response to the following technical evaluation criterion is required:
Is the specification document sufficient to provide an understanding of the code structure and overall execution procedures?
Specification Document. Volume 2 the Computer Code Manual will be the specification document for this requirement.
Requirement I: Part 3. Documentation Requirement. Is the documentation sufficient to describe in detail how to use the code?
Technical Evaluation Criteria and Success Metric. A positive response to the following is required:
Are the input requirements for the code adequately described such that the user can provide input for each variable?
Specification Document. Volume 3 of the code manual, the User’s Manual, will be the specification document for this requirement.
Requirement I: Part 4. Documentation Requirement. Adequate documentation exists which describes how to install the code on a user’s computer system.
Technical Evaluation Criteria and Success Metric. The instructions given with the software distribution must be complete and accurate for installation of the code.
The adequacy of the installation documentation will be established through a survey of the staff at the various sites at which the code has been installed. A positive response to the question, “Were you able to install the software using the information provided with the distribution you received, except for site-specific requirements? If not, why?” is required.
Specification Document. The installation instructions sent with the distribution of the software will be the specification document for this production-level use requirement.
Completion of these activities will ensure that all individual models and methods in the code are completely and correctly documented.
Models Methods Software 
Hello! eeebfda interesting eeebfda site! I’m really like it! Very, very eeebfda good!
In fact no matter if someone doesn’t know after that its up to other users that they will assist, so here it occurs.