Neil Johnson, Principal Consultant – XtremeEDA Corp.
Bryan Morris, VP Engineering – XtremeEDA Corp.
Contents
Introduction
There are opportunities for positive transformation in any IC development team. While people and process related initiatives can have a tremendous complementary impact to the technical advancements already occurring in IC development, their value is often overlooked. Technology driven advancements such as coverage driven verification and the proliferation of OVM as a functional verification platform have been critical to keeping pace with exploding design size and complexity. However the difference between success and failure will frequently depend on how effectively people can embrace these new technologies, and what processes are put in place to ensure their optimal use. The technology itself is rarely the decisive factor for a successful or failed product delivery.
In the world of agile software development delivering value to a customer is the primary objective, and teamwork and efficient processes are the essential components. This article identifies the characteristics common to effective agile development teams, and how a team that embraces these characteristics–in combination with using cutting edge technology–can accelerate their schedule, improve their product quality, and increase customer and employee satisfaction.
Providing Value to a Customer
Providing value to a customer early and often is the primary objective of an Agile team.
The CEO and sales force probably say them but how often are the words “value” and “customer” used together in ASIC development? While a team might realize who the customer is, the concept of value is probably not well understood nor used as a basis for discussion during development.
Too often, teams are concerned with planning and executing according to a requirements specification. That is important, but creating a plan and then executing the plan cloud the purpose of a project. The purpose of any project is to provide value to a customer. If– or more likely is when–your plan has to change to provide value to a customer, change it. A team that understands the needs of a customer and are concerned with meeting those needs will have happy customers. And as much as it hurts sometimes to make customers happy, realize that a company cannot survive without them.
The entire team needs to understand the thoughts and objectives of its customer. Everyone needs to understand what they are building and why they are building it. If a team is constantly asking itself, “Is this good for the customer?”, the resulting product is more likely to satisfy your customer’s needs.
Most Agile methodologies advocate regular interaction with the customer to ensure a product vision is relayed directly to the development team. If possible, it is even recommended that the customer be onsite and available throughout development, so that the feedback loop is as short as possible.
Visibility and Effective Communication
IC development teams, in general, rely heavily on written documentation, bug tracking systems and other formal means for communication. Most ignore the merits of communication in other forms; and unfortunately, teams choose to ignore its most effective form: verbal communication.
Visibility and effective communication are also characteristics of an Agile team. Verbal communication is most effective for conveying intent so an Agile flow creates frequent opportunities to ask directly for information or clarification. Arrange your workspace so that conversation between team members is easy and convenient. When cube walls get in the way, try removing them! Co-locating a team in a single room is also very common for Agile teams (Beck 2004) to ensure people and information are always close at hand. Cockburn 2007 recommends a one bus length rule between team members. Any further than that and meaningful communication degrades drastically.
When a conversation is not possible or appropriate, the next best thing is communication that is as interactive as possible in order to keep response times short (Cockburn 2007). Keep in mind that telephone, Instant Messenger and the like are generally more effective for communicating intent than email, written documentation or bug tracking systems. Where information is required by the entire team, use forms that are as visible as possible (e.g. Intranet wikis).
A Picture or 1000 Words
Documentation is necessary. However, it should have three key elements to be truly effective: correct, complete, and concise. While written document can be correct and complete, it is sometimes difficult to be concise.
Software engineers have evolved diagrams that can concisely and accurately design intent using graphical capture The most advanced design capture method is using Unified Modeling Language (UML). Using UML engineers can capture requirements, architecture, synchronization and timing. From the saying “a picture speaks a thousand words”, a UML diagram and significantly reduce your project’s documentation requirements. The underlying rules, enforce a UML to be precise – reducing the likelihood of ambiguity common in the written text.
As “Agile” also implies ability to adapt quickly, it is generally faster to update a set of UML diagrams for any major architectural changes over a written text.
Many agile teams use UML to capture design intent. While there many free tools that allow UML capture they lack the “flow” connected software to hardware. Mentor’s BridgePoint xtUML provides a consistent approach that can be used by an IC development for embedded systems. BridgePoint goes beyond that static capture of UML, and provides the ability for your software team to accurately model and test your system and provide the ability to dynamically analyze the system.
Regular Reflection
The end-of-project post mortem is an ineffective technique for critiquing the qualities and actions of the team that have either made it successful, lead it to failure, or left it somewhere in between. A post-mortem may lead to corrective action, but it cannot be applied until the next project.
A better approach is to establish a review process at regular intervals throughout the project to measure and critique a team’s progress. If required, a team can take corrective action to the benefit of their current project, as well as improving the process for future projects. Agile teams continuously reflect on their product quality and process efficiency.
The Cross-functional IC Development Team
IC development teams are typically divided into functional sub-teams. Design, verification, physical design and software are the most common functional splits. Between them, there are usually spikes of cooperation during times of bug fixing and crisis management, Cooperation also tends to increase as a project nears conclusion. These functional sub-teams, however, usually set their own priorities and work toward their own goals on a daily and weekly basis. And while they do not operate in complete seclusion from each other, it is fair to say that meaningful cooperation is relatively limited during normal development.
A critical feature of an Agile team is its cross-functional composition. While each team member has an area of expertise, there are no formal lines drawn based on function. The entire team works toward common goals on a daily basis and collectively the team assumes a “get it done” attitude. The team’s common goal of delivery at regular intervals brings steady motivation for regular cooperation. It also creates a penchant for crisis aversion that is not present between independent sub-teams. Agile teams use the most effective means of communication possible to convey and clarify intent–which is often verbal communication–and reach beyond their area of expertise when necessary.
In a time where specialization and organization into functional sub-teams is the norm, IC development teams stand to benefit from working as a cross-functional team with a renewed emphasis on shared goals and cooperative problem solving.
Software + Hardware = System
In SoC development, embedded software is a functional team that often operates independently of design and verification. However, because the software team typically acts as proxy for the end user, it is better to have them involved in specification and implementation of the hardware throughout development. One way to involve the software team earlier is to build an ESL model.
While one benefit of ESL modeling is to enable parallel development of software and hardware, in many cases it only encourages the formation of yet another independent team: the modeling team. We believe that simply adding another team to enable parallel development without changing how teams interact provides little benefit.
We are suggesting a different approach where software, modeling and hardware experts work together to incrementally build a system. As the system architect defines features, the modeling expert incrementally populates the ESL model with the logic required for each feature. The model can then be leveraged by the software experts–for driver or application development–and the design and verification experts–for implementation and test of the hardware.
Leveraging the model twice is only a byproduct of this approach. The real benefit is it creates a tight feedback loop between each group of experts and regular opportunities to confirm design intent. The entire team runs in lockstep throughout development; guided by the system architect and the ESL model. There is constant collaboration between hardware and software where cooperatively they implement and test the vision of the system architect. The architect can see the end product incrementally grow in functionality and use the functioning system to refine his/her vision.
These feedback loops and cooperation cannot reasonably exist without a cross-functional team armed with tools that move focus toward the system and away from its pieces.
Incremental Development and the Steel Thread
Incremental development is an approach where functionality is built and tested as a thin slice through the entire development flow, start to finish. A product starts as a small yet functional subset and steadily grows as features are completed. Progress is based on code that works rather than code that’s written. Working code is a great metric for measuring real progress and demonstrating and clarifying implementation decisions.
One barrier to incremental development is there is often a considerable amount of effort required to identify, implement and test a feature that performs some significant functionality and provides some recognizable value to a customer (Andrea 2001). Such a feature is sometimes called a “steel thread” because it often represents a very thin but significant end-to-end path through a device.
It is likely that a team new to incremental development not only lacks the skill to scope and plan a first increment, but that the sheer number of tasks spawned in the first increment can be difficult to coordinate effectively. It is important, therefore, that the development team does what it can to prepare for the challenge.
Most IC development teams are familiar with the concept of “basic sanity”. Generally speaking, the basic sanity milestone is the first time an IC performs some simple function of recognizable value. Therefore, basic sanity has similar functionality to that of a first increment–or the first “steel thread” for the project. That value in this “steel thread” is apparent not only to the development team, but quite likely to other stakeholders, including the customer.
An effective way to mentally prepare for a first increment is to identify the simplest set of functions through a retrospective exercise where the criteria for basic sanity of a legacy product are analyzed. Similarly, for new products decide on the minimum set of functions that would demonstrate “basic sanity”. Any logic that is not specifically required for basic sanity should be ignored. In cases where unnecessary functionality cannot be completely ignored, the team should understand where and how it may be simplified (Schwaber 2004). After identifying the minimum required for the basic sanity, the team can go through a planning exercise to understand how they could implement and test basic sanity functionality as fast as possible. This defines the goal and tasks for steel thread for the first iteration.
With the completion of the first steel thread, incrementally adding functionality should be a more straightforward assignment because the steel thread gives the team a functioning starting point from which they can confidently add new features. No doubt though, the first and all subsequent incremental improvements will require practice, planning and great communication from a cross-functional team to be successful!
Incremental Development with OVM
Verification in incremental development–verifying a steel thread in particular–requires two criteria for success:
- a flexible environment that is easily deployed
- functionality that is easy to build and integrate
A verification environment based on OVM can satisfy both these criteria.
To start, the ovm_component class provides a defined, but extensible run-flow. Very little effort is required to have an operational–albeit non-functional–verification environment.
In terms of functionality, OVM provides mature, reusable verification components allowing a team to quickly build functionality. Verification component base classes for monitors (ovm_monitor), scoreboards (ovm_scoreboard) and sequence generators (ovm_sequence) provide the means to quickly create the active stimulus and checking components needed to verify a steel thread.
In addition, the OVM library contains a robust and fully functional set of components for reporting, object synchronization and object generators-to name a few.
From an integration stand-point, the tight coupling between the run-flow and OVM-based verification components provides considerable benefits when it comes to integrating new components. The common virtual interface of each component enables the run-flow to assume control of each component by virtue of a simple parent-child relationship established at instantiation. Furthermore, the fact that OVM leverages Transaction Level Modeling (TLM) and a common transaction class (ovm_data) to standardize communication between components makes inter-component, communication and synchronization relatively straightforward.
Lastly, OVM has an active user community providing a wealth of additional resources that can leveraged in a functional verification environment.
The well defined yet flexible OVM library makes it very well suited to building the framework and functionality required for verification in incremental development.
Coverage Closure
If an IC development team uses functional coverage closure as a prerequisite to sign-off, an agile team would do the same as a prerequisite to an increment being complete. The functional coverage model and associated tests can also be built incrementally to provide an objective measure that the functional requirements of each increment have been implemented and exercised. To be clear, the entire functional coverage model is not completed; it is only completed to a point where it confirms the functional goals of the increment.
A great side effect to incremental coverage closure is that it should be much easier for a team to comprehend, specify and implement the coverage model in small pieces as opposed to one large chunk. This is especially true when the model can be so dependant on supporting code infrastructure which is always difficult to envision early in development.
Regular Regression Testing
Regression testing is critical for teams to maintain a working code base. It is also a vital prerequisite for incremental development. A regression suite should be run regularly to ensure the incremental addition of new features have not inadvertently broken existing functionality. While this is common practice for many IC development teams, what might not be common is regression testing that starts on day one of development. Regression testing on day one is an effort to prevent bugs from even entering the code base instead of resolving to find them later.
If a team wants to prevent the accumulation of bugs in the code base–and use working code as an objective progress metric–early and regular regression testing should be considered mandatory.
Summary
This article highlights just a few ways an IC development team can make an Agile transformation. Agile software development teams have been using–and benefiting from– these and many others for more than a decade.
Agile methods, are relatively unknown and untried in IC development. The time has come, however, for organizations to recognize the success of Agile methods in other fields and to consider the merits of agile development approaches in IC development.
And while Agile methods can provide great new frameworks and tools… the details of when and how they are used are still up to us!
References
[1] Agile IC Development with Scrum, Neil Johnson, Bryan Morris, SNUG San Jose 2010.
[2] Agile Project Management With Scrum, Ken Schwaber, Microsoft Press, 2004.
[3] Agile Software Development: The Cooperative Game (2nd edition), Alistair Cockburn, Addison-Wesley, 2007.
[4] Extreme Programming Explained: Embrace Change (2nd edition), Kent Beck, Addison-Wesley Professional, 2004.
[5] Managing The Bootstrap Story In An XP Project, Jennitta Andrea, http://www.jennittaandrea.com/, 2001.