Sufficiently Complex Data

​Most applications testing, both performance and in development environments, is being done today utilizing production data that has been extracted utilizing an ETL (Extract Transform Load) process and then manually modified to create specific use cases.  For example for cyber applications, most testing is being done by replaying network traffic.  Due to the labor intensity of this process, use case coverage is generally very low and most of the business logic and workflow rules go untested.  This is where the concept of sufficiently complex data comes in.  Test data should be of large enough volumes to cover peak processing volumes and have sufficient complexity to cover almost all of the business logic and workflow rules.  Utilizing large amounts of sufficiently complex test data will exercise algorithms at peak processing volumes to expose failures before moving to the production environment and enable precision error measurement for ambiguous, true and false errors.  Systems can then be optimized for the cost of errors versus the cost to correct.