With Cadence recently releasing an eUnit test framework with Specman, I figured now would be a good time to suggest a 2014 new year’s resolution for each of the Big 3:
Make functional verification manageable by releasing a SystemVerilog unit test framework with your simulator.
Don’t worry if that sounds complicated because it isn’t. In a couple months, tops, you could have everything you need to package a first release. All it takes is 4 easy steps.
NOTE (to Mentor/Synopsys): Cadence is winning this race so far. If I had to guess, I’d say that their recent additions to Specman mean they’re considering something similar for SystemVerilog if not well into its development.
1. Read a book on TDD: Ultimately, you’re building a framework so that people can do test-driven development (TDD) of design and verification IP or unit test design and verification IP. Assuming you you’re not overly familiar with TDD, the first step is all about motivation and understanding. Most of us techie-types like curling up with a good book now and again so that’s where I’d start with your new year’s resolution. Let an expert tell you about TDD; what it is and how to do it. It’s the same in any language so the book you choose doesn’t really matter though I’ll recommend three here. Test-Driven Development By Example by Kent Beck is a good place to start. Two favorites of mine for the C/C++ crowd are Modern C++ Programming with Test-Driven Development by Jeff Langr and Test-Driven Development for Embedded C by James Grenning. Pick one; read it front to back.
1a. Do the examples: Every book I’ve seen on TDD is about half words, half code examples and the examples are always available online. More than any other technique I’ve come across, TDD expertise comes from practice and not from learning which means doing the examples that come along with your book is quite important. Another benefit of TDD is that it is (mechanically) very simple so the examples you come across will be equally simple but entirely practical.
2. Find a favorite unit test framework: I’m pretty sure there are about 841 different unit test frameworks in existence and most of them are designed to simplify the process of writing and running focused, purposeful unit tests in pretty much the same way. That means you can cross “reinvent the wheel” off your list. There will be no reinvention of any wheels. Your new year’s resolution involves building tried-and-true so spend some time finding an existing framework that looks and feels right to you. I’ve spent some of my off-hours practicing with JUnit (Java) and GoogleTest (C++). Both are very well used in the software world while Java and C++ are close enough to SystemVerilog that the usage model fits reasonably well. There are others, but these are good to start.
3. Build your framework: If you’ve spent the time on steps 1 and 2, building your new year’s product should be the easy part. All you’re doing here is porting something that already exists to a different language. It ain’t brain surgery. Remember that reinventing the wheel is not part of the deal. Take existing idea, implement in different language, done.
4. Eat your own dogfood: If building the framework was the easy part, I’d say using it will end up being the hard part. Using your framework will be the most important step of all. TDD is not complicated whiteboard theory, it’s practice. Use your framework to build design and testbench IP so you have an idea of how your customers are going to use it and how it can be tweaked to become more productive. Practice makes better.
That’s it. Pretty easy, no? A couple months of book learning, practice and coding will get you a first release. With a little practice you gain the wisdom. A little wisdom and a few versions down the road you’ll have customers using it. Just be aware of a few pitfalls you may hit along the way…
Unit test frameworks are user friendly. It should take someone totally new to your framework about 10 minutes to write and run a test. That’s is. This is not UVM, it’s unit testing. Simplicity is power and getting started should be simple. If what you have is not simple, then you probably need to start again.
Unit tests should look like executable documentation. I’ve heard of software teams that “document” their code with unit tests. That means your API is clean and free of clutter. If your framework is cluttered or organized such that you cannot read unit tests like documentation, then it’s time to start again.
Your framework should not be a UVM plugin. That’d be a tremendous fail for a couple reasons. First is that UVM is complicated so you’re breaking the “unit test frameworks are user friendly” guideline. Second, TDD and unit testing are techniques that will not go away. UVM will. If you want your framework to remain relevant for as long as Verilog is relavant, avoid dependencies on UVM. If you find yourself building a UVM-based framework, start again.
TDD and unit testing are for design AND verification. When it comes to new verification techniques, designers have been stuck with the short straw the last few years in that new test techniques have been geared entirely toward the verification crowd. That’s not the way it works for unit testing. If your unit test framework is not designer-friendly, start again.
Plan B. Why yes, there is a plan B. Instead of investing your time in a new SystemVerilog test framework, why not just ship SVUnit with your simulator. It follows all the guidelines above and it’s open-source. Use it, encourage your customers to use it and when the time comes that you know you can do better, then go do better ;).
Follow AgileSoC.com
That’s it! If you’re at Cadence, Mentor or Synopsys and your job involves helping people write and/or verify SystemVerilog code, consider this your new year’s resolution. Overnight shipping from Amazon.com means you can get started tomorrow!
…and with UVM currently setting the standard for usability in hardware verification, there’s really no harm in just going for it ;).
-neil