I’ve been writing a lot about integrated verification flows the last several weeks. I’ve had lots about how techniques fit together but I haven’t talked much about what holds everything together. This is where shared development objectives come in. Fundamental to an integrated verification flow is establishing shared objectives within the team. When I say shared objectives, I’m talking about objectives that represent tangible, demonstrable value at an integrated system level that require input across an entire team.
A test plan is a good place to establish shared objectives. In this post, we’ll work out a method for test planning that takes advantage of the dependencies between verification techniques, all for the purpose of creating tangible outcomes. Easy in theory but challenging in practice, it involves bottom-up application of a top-down test plan with the primary motivator being early and continuous delivery of product level outcomes.
Top-down test planning starts at the highest level of abstraction. In the model we captured in Portable Stimulus And Integrated Verification Flows, that’d be a system level use case addressed with integrated HW/SW tests. That integrated HW/SW test is decomposed into prerequisite tests that address various scenarios, those scenario tests are decomposed into prerequisite tests for various transactions and so on down to prerequisite unit tests to address various details.
Tests that address the feature, feature set and system abstractions should appear in a test plan because they can be meaningfully traced to a specification and therefore relevant for representing progress from a product perspective. These would include the integrated HW/SW tests, portable stimulus based tests, constrained random tests and selected directed tests. Finer directed tests and unit tests wouldn’t end up in the test plan. They’re implementation dependant and therefore dynamic; they’re identified as implementation decisions are made and are too low level to be directly linked to a specification.
The combination of tests in the test plan, their intent and how they blend together builds on what we discussed in A Verification Where And How. That’s where I presented each test technique, where they’re best suited and how each test technique transitions to the next.
Once you have tests mapped out, development proceeds bottom-up branch-by-branch. Unit tests happen first, followed by directed tests and so on, all feeding up to the integrated HW/SW tests.
To circle back to the idea of collaboration and shared objectives… a passing integrated HW/SW test is a shared objective. It is multidisciplinary in that it requires hardware design and verification, software design and test and integrated HW/SW platform specialists. It also represents the aggregation of value produced across the entire development hierarchy from individual subsystems up to an integrated device. Finally, this type of shared objective is meaningful in that it is tangible from a customer perspective, the only perspective that truly matters.
In case it’s not obvious, thinking about a test plan in this way makes it more of a development plan than test plan. It can be used to focus and coordinate effort across an entire team for the purposes of producing tangible outcomes as quickly as possible.
-neil