Lesson : Test and Evaluation Overview
Why Test and Evaluate?
Information is critical to managing an acquisition program. This includes information on the capabilities and limitations of the system and risks affecting system cost, development schedule, and performance. The overall goal of T&E is to reduce risk by providing crucial information to decision makers. T&E Mandates
DODD 5000.01 provides principles and policies governing test and evaluation in the Defense
Acquisition System.
DODI 5000.02 identifies the flow of the T&E activities within the development process and states that test activities shall be part of a Test and Evaluation strategy to:
●
●
●
Provide information regarding risk and risk mitigation.
Provide empirical data to validate models and simulations.
Permit an assessment of the attainment of technical performance specifications and system maturity. T&E Customers
These Customers:
Acquisition Managers
Use T&E Information to:
●
●
●
●
Evaluate the acquisition program for risk management.
Determine if system performance meets thresholds.
Determine if the system has the potential to meet objectives.
Determine the technical changes needed to keep the program on track. 1
Feb 2013
Designers and
Engineers
●
●
●
●
End-users
●
●
●
●
Evaluate system design and performance.
Identify if design changes are needed.
Provide feedback into Systems Engineering Process.
Update modeling and simulation.
Learn how the system performs under operational conditions.
Utilize operational performance characteristics to improve tactics and warfighting capability.
Obtain information to help plan for training and logistics support.
Understand full performance envelope and capabilities.
Defining Test and Evaluate
TERM
DEFINITION
Test
A program, procedure, or process to obtain, verify, or provide data for determining the degree to which a system (or subsystem) meets, exceeds, or fails to meet its stated requirements. These requirements can be stated in terms of thresholds and objectives. Testing:
●
●
●
Evaluate
Obtains raw data.
Measures specific, individual performance factors (e.g., measuring range). Is resource intensive.
Reviewing, analyzing, and assessing data obtained from testing to project system performance under operational conditions. Evaluating:
●
●
●
Produces analyzed information from test data, modeling and simulation, or other sources.
Draws conclusions by looking at how the factors interact.
Is intellectually intensive.
T&E Feedback Loop
T&E is the essential feedback loop contained within the Systems Engineering Process. T&E ensures that the system meets the capability needs. The T&E process is repeated as the system evolves from models to components to production articles and complete systems.
2
Feb 2013
Test and Evaluation Process Overview
The T&E process has five steps that occur as we evaluate important information within the acquisition process. Five Steps in the T&E Process
STEP
DESCRIPTION
1
Identify critical issues and data requirements.
2
Pre-test engineering analysis by the evaluation and the development of an evaluation plan. Expected outcomes are predicted.
3
The test is planned and conducted. Data are retrieved and analyzed. Data may be collected from other means such as Modeling and Simulation (M&S), training, etc.
3
Feb 2013
4
A post-synthesis step in which an evaluation report of data is compiled. Predictions from
Step 2 are compared with actual measured outcomes.
5
T&E results are balanced with other available program information. The appropriate programming decision is made.
T&E and the Acquisition Life Cycle
T&E activities occur throughout the acquisition life cycle. T&E reveals information about a program and measures performance of a system against established capability needs. The PM coordinates
T&E activities appropriate to each phase and milestone of an acquisition program.
