When I hear the word shame, there is only one thing that comes to mind: Slapshot. If you’re Canadian, chances are you already know what I’m talking about. To everyone else, Slapshot is a hockey movie and shame comes from an interview where Jim Carr is talking penalties with Denis Lemieux, goalie for the Charlestown Chiefs. Denis comes up with this to describe high-sticking, slashing, tripping, hooking and spearing…
“…all bad. you do dat you go to du box… you know. Uh… 2 minutes by yourself and you feel shame… you know. And then you get free.”
[youtube_sc url=n_w4MV_LwMw width=640 height=480]
The reason I bring this up is because I recently heard the word shame used in a chat with James Grenning at the Agile2014 conference. He told me about shame factors that motivate embedded developers toward test-driven development. One of his shame factors was the length time that passed between writing a piece of code and knowing whether or not it’s correct (i.e. testing it). Most would agree that it’s best to cut that time down to be as short as possible yet many developers settle. They wait hours, days or longer before testing thereby (shamefully) dismissing the importance of immediate feedback and maintaining a high quality code base.
Initially, I felt like shame was a pretty harsh word for what James described. But it didn’t take long for me to relate with my own shame factor. If you’re a verification engineer, you may be able to relate, too.
I run an RTL test, I believe I’ve found a bug in the RTL, I take it to the designer and a couple hours later he/she bounces it right back at me.
“You’re right. There is a bug”, they say, “but it’s in your testbench, not in my design”.
That’s it. That’s shame for me. I hate the idea that I can use a bug I’ve created to waste someone else’s time and when it happens I feel like a tool. Indeed, I go to the penalty box (i.e. my cubicle) and feel shame.
Feeling that shame regularly was part of what motivated me to start using TDD. I got tired of wasting other people’s time. I realized that trying harder wasn’t the answer so I tried a new technique entirely.
Thankfully, getting better is exactly what’s happened. Sure, I still create defects and I still feel that shame of wasting other people’s time, but it happens much less often than in the past. Basically, TDD helps me write code I can feel good about while the defects that linger remind me that there’s still room for improvement.
There it is… the shame of creating defects and how TDD can help you “get free”.
-neil
Agreed! After all, the designer’s job is not to find bugs in the testbench that you wrote. It’s your job to find bugs in the design that they wrote.
An interesting metric would be to track number of testbench bugs found by designers vs. testbench bugs found by verification engineers – one would certainly expect to see TDD to result in fewer testbench bugs found by designers, which means they can focus on what they’re supposed to be working on.
@Erik: I looked up your LinkedIn profile and got some important context to find out that this comment was written by a verification engineer and not a design engineer. If it were written by a design engineer, it may have triggered a slight of offense from me to this comment, because I could have interpreted the attitude as similar to that coming from some particular designers that I have worked with in the past that were very poor partners with the verification team.
Unfortunately I have worked with my fair share of bad designers who feel it is fine for them to write whatever, and then just shovel the shit downhill — any problems from then on, even silly mistakes, are the verification team’s or always someone else’s problem. On the complete opposite end of the spectrum, there are also some designers I’ve worked with that always take full responsibility, and are even a bit hard on themselves every time an RTL bug is found, even though this is an expected event in the course of a project.
“It’s your job to find bugs in the design that they wrote” – as a verification engineer, I take great pride in this being my primary duty, but when I think of some of the best designers that I have had good fortune to work with, in those cases, I would reword this to something more nuanced (and wordy) to “the verification engineer and the designer form a partnership to produce the highest quality design that is needed – there may still be minor bugs, but if the two of you have done a good job, they should not impact the time to market or other important success criteria for the product”. In cases, where I am not fortunate enough to be working with a designer who views this as a partnership, then obviously it still holds that my job is to find bugs in their design.
All that being said, I am definitely a fan of many years of unit testing or TDD of my testbench code as I develop, often going through many short, tight loops to quash bugs or issues before it makes contact with a single line of RTL, and like the author, I do feel shame when a sighting is root caused to a bug in my testbench code, but fortunately this doesn’t happen all that frequently because I am doing so much TDD in my development.
However, I think most good designers will be understanding that this occasionally will still happen, despite the discipline, preventative measures and the best intentions of the most seasoned and rigorous verification engineers. All human-created things will have bugs, and the testbench and/or verification IP components these days are very complex things, often surpassing that of most design units. But it’s not black and white: the fact that bugs will slip through in the testbench is not an excuse to not use TDD in order to minimize this happening as much as possible, since I have seen poorly run projects that have been overwhelmed by false failures due to testbench bugs, and it’s definitely not pretty.
Anyway, I think I’m preaching to the choir a bit, but just sharing my thoughts.
P.S. As a Canadian, props on the hockey penalty box metaphor.
I recently experienced lot of this shame myself. I set out to build a golden model for the DUT. Project cycles has come and gone. But i’ve not been able to iron out all the bugs in the Testbench.
There were many times i wished how i could test my code first before running it with RTL.
Could you elaborate more on how TDD works with testbench development.
I’m very curious. And I do not want to feel this shame in the future.
yes… not being able to iron out all the bugs in the testbench… I can relate.
A practical starting point is the examples I have in the svunit release package (I use SVUnit for unit testing my testbench components and I have the examples in there to give people a head start).
For more of the theory, we did a series of posts on TDD back in november 2011. You can find links to these posts on the SVUnit project page.
If all that looks good, I’d suggest going through a TDD book to learn the mechanics. I can suggest a couple if you’d like.
-neil