Acoustic range managers need a better system for identifying high-value decision points before conducting test events. When this research was conducted, a qualitative process model that represents the acoustic range decision process did not exist.

This research focused on modeling the decision process of the Naval Undersea Warfare Center (NUWC) Division, Keyport, underwater acoustic tracking range. The model had to be capable of replicating a simplified version of an event, so test planners can see the identification of high-value decision nodes and use this information to make better decisions regarding risk management and mitigation. Research focused on developing a basic understanding of the decision methods, developing an appropriate model, and then simulating the decision process to identify high-value decision points.

Range managers relied on heuristics to predict problem areas, which do not always provide useful risk analysis tools for new testing programs that may require different evaluation constructs. Additionally, the range did not employ quantitative risk analysis techniques, which could have identified problem areas before testing. The research focused on providing a formal model that could provide these capabilities and addressed the following question: Can formally modeling the business process of conducting test range events expose previously overlooked ambiguities and identify high-value decision points?

Next, a proposed decision process capable of representing a simplified range event was formalized and used the following measures of merit to evaluate the effectiveness of the model:

  • The number of high-value decision points reflected in the formalized process model exceeds the number of high-value decision points in current informal process models of range event activity. High-value decision points reflect specific places in the test process that, if improperly executed, could result in loss of test data or prematurely abort the test.

  • The formal model displayed at a minimum the decision points found in the simplified model. The formal model improved on the simplified model by providing more decision points, representing the test event more accurately.

Model variables included the number of decision nodes and the number of uses of a decision node during the simulation run. Multiple uses of the same decision node indicated a high-value decision point. Confirmation of high-value decision nodes occurred during model validation. The following two points summarize the risks and uncertainties:

  • Since the acoustic range studied had never modeled their decision process, the ability to replicate the informal process as a formal model via a simulation program was uncertain. Ultimately, a suitable program was identified that adequately represented the range decision process.

  • Similarly, the ability of the formal model to identify high-value decision points in the test decision process was unknown. In the course of building the formal model, methods provided by the software program allowed counting decision node usage, which provided information on the value of each decision in the process model.

Current business process modeling (BPM) methods were evaluated. The simulation of the range process model was evaluated using the selected process model software. Aspects observed included logical flow of events, event duration, decision probabilities, number of uses of each decision node, accuracy of feedback loops, and overall utility of the model to range managers.

The research evaluated current modeling approaches used by the acoustic range through the collection of relevant decision and process data used to control range events. The main sources were existing range control policy documents and information collected from range managers. A simplified block-diagram model representing a routine acoustic range event was developed, as well as a formal process model using simulation software capable of replicating the simplified process model. Conduct of multiple model runs determined that the model could identify high-value decision nodes.

This work was done by William Carlson for the Naval Postgraduate School. For more information, download the Technical Support Package (free white paper) below. NPS-0018

This Brief includes a Technical Support Package (TSP).
Formal Process Modeling to Improve Human-Decision-Making During Test and Evaluation Range Control

(reference NPS-0018) is currently available for download from the TSP library.

Don't have an account? Sign up here.