About a month has passed since the great unit testing discussion in the Verification Academy booth at DAC in Austin. It was my second year in the booth. Very grateful for having had the opportunity. It was a lot of fun!
The format involved me cycling through a bunch of questions and comments on unit testing as a series of topics during which time discussion would erupt with me and audience.
At least that was the plan!
Turned out we didn’t really need any guidance. People in the audience came with more than enough questions to keep things rolling.
Not getting through these topics, though, feels to me like wasting opportunity so I figure a good back up plan would be to pass them along as topics that anyone can use to facilitate discussion. To that end, here’s a few from my slide deck with my own added commentary. If you’re a unit testing advocate, a design or verification team lead, manager, or anyone else with an interest in unit testing, I hope you can use them to motivate discussion within your teams.
Hamilton Carter: “Holy crap unit testing is addictive!”
Me: This is a fun comment to start that I absolutely agree with it. Unit testing – or TDD specifically – has become a bit of an addiction for me. I find myself much more productive and producing code of much higher quality which has been great. But the addiction comes from working in a cycle that regular ends in a new passing test (aka: positive feedback). Those passing tests increase confidence in your code and quickly become something you can’t live without… like a (good) addiction.
Jean-Marc Tremblay: “These SVUnit tests have not only been useful at helping me find bugs in these classes, but also at showing the group how they could use them.”
Me: We have two great points here. First is the acknowledgement that unit testing with SVUnit helps eliminate bugs, but that’s an obvious benefit. The second point is truly notable because it isn’t as obvious. Unit test suites can become a platform for demonstrating code and spreading knowledge to colleagues. This is a side effect that I’ve seen as well. After the whiteboard, unit tests are very effective for demonstrating code; far better than more complex directed or constrained tests.
Tudor Timisescu: “Even if setting up the testing infrastructure takes some effort, I find it pays out in the end when I want to make small changes in the implementation, without changing the behavior (i.e. refactor stuff). This, to me, makes it all worthwhile.”
Me: There are other 2nd order benefits to unit testing. Unit tests provide a very useful backstop against which developers can confidently refactor and improve their code without the risk of breaking functionality. Change the code all you want. As long as the tests keep passing, the code still works!
Stan Sokorac: “By far the most common response I get is I don’t have time in the schedule for this, as if this is some extracurricular activity that is taking away from real work.”
Me: This is an unfortunate but real concern that I regularly hear as well. To me it’s a sign hardware teams believe it’s possible to prioritize schedule over quality, like we can somehow punt on quality to meet schedule. Ultimately though, quality trumps schedule every time because tape-out doesn’t happen without quality, regardless of what the schedule says. My opinion: unit testing is a great way to prioritize quality. And if we prioritize quality, the schedule will take care of itself.
Gareth Williams: “What’s an appropriate level of granularity for unit test functionality? Or maybe expressed in a different way, what’s an acceptable relative burden of maintaining unit-tests for new code?”
Me: This is a good question because unit testing is effort and we’d like to be able to gage that effort. My opinion on how far you go has everything to do with responsibility and confidence. My code is my responsibility and I need to be confident it works before I dump it on anyone else. For me personally, confidence comes from near exhaustive unit testing and continuous maintenance to accommodate change. I accept a very high level of burden because for me that’s just what it takes. For someone that writes better code, maybe that burden is lower. But in my mind the responsibility is the same. Code needs to work and the person writing it needs to do whatever it takes to see that’s the case. Accept the burden.
There’s five topics to get you started. If you’re looking for others, the entire recorded session from DAC is posted on Verification Academy. You can login for 36 minutes of video that includes a short slide presentation with about 20 minutes of Q&A. Between this and the video, hopefully I’ve given you enough to get your own unit testing discussions going!
-neil