Unit Testing And TDD: Too Different To Compare

Here’s another post inspired by James Grenning and his “TDD with C” tutorial. Going through an abbreviated version of that tutorial last year in Salt Lake City at Agile2011 was what got me started with TDD. It inspired TDD month on www.AgileSoC.com last November. It encouraged me to get knee deep into SVUnit and TDD of verification IP. It also, in part, motivated my Agile2012 submission: TDD and A New Paradigm For Hardware Verification.

My second time through it was at ESC in San Jose this past March. This time around, I found myself fascinated by some skepticism in the audience when a fellow piped up with the comment “I see the value of unit testing but it’s hard for me to see the extra value of TDD”.

It was easy to see it wasn’t the first time James had seen this kind of skepticism during one of his talks. His response was “My goal is to show the value of TDD… but if all I can do is convince you there’s value in unit testing and that’s as far as you get, that’s great, too. That’s a start.” Continue reading

Functional Verification Doesn’t Have To Be A Sideshow

This is another one of those challenge-the-way-we-think-about-functional-verification posts. The motivation behind it comes from a few different places.

First is a blog posted on eetimes designline by Brian Bailey a couple weeks back called Enough of the sideshows – it’s time for some real advancement in functional verification! In that post, Brian exposes a few techniques – constrained random verification in particular – that have failed to live up to the hype and praise that’s been heaped upon them over the last several years. That’s an industry expert suggesting we re-think the direction we’re going with functional verification tools so if you haven’t read it, I’d suggest doing so. In case you missed it, Brian also inspired the snappy title of this post :). Continue reading

Unicorns, Leprechauns and Reusable Verification IP

There are times when I use agilesoc.com to step out on a limb and challenge the general consensus in hardware development. This post would definitely qualify as one of those times.

I don’t think the reusable verification IP we’ve been building is as reusable as we think it is. I don’t think reusable IP is reused as many times as we’d like (if at all). Nor do I think reusable IP is as valuable as we think it is.

There… I said it. Continue reading

(Wasted) Effort Spent In Verification

I just spent a few minutes reading Harry Foster’s analysis of a functional verification study commissioned by Mentor Graphics and carried out by Wilson Research Group in 2010. There’s lots of good information in Harry’s analysis – there’s 9 posts in all – and I’d highly recommend people take a look if they haven’t already. It’ll be worth your time. The posts go back to June 26th, 2011… which, admittedly, makes me a little slow on the uptake :). Continue reading

Survey: Project Planning in Hardware and Embedded Systems Development

If you’re currently working in hardware and/or embedded systems development, we’d appreciate your participation in the following survey:

Project Planning in Hardware And Embedded Systems Development Survey

Are people in the hardware and embedded systems communities confident that their approach to project planning gives them a reasonable chance of success? We don’t know… but we’d like to.

We want you to know, too.

The survey is designed to measure confidence in the project planning approaches we use in hardware and embedded systems development. It is two pages of multiple choice questions covering tendencies in project planning and demographics. The survey takes roughly 5 minutes to complete and data is collected anonymously. Data and analysis will be published and freely/publicly available at the conclusion of the survey. The survey is being conducted by in partnership by me, Neil Johnson, and Catherine Louis of CLL-Group, LLC.

Project Planning in Hardware And Embedded Systems Development Survey

We’d also appreciate people’s efforts in spreading the word. Please help us by posting a link to the survey or to this page on Linkedin, Facebook, Twitter, your company intranet and/or anywhere else you go to share news and information with colleagues.

Thanks very much for your participation and thanks for helping make this survey a success!

For more information, you can contact me at neil.johnson@agilesoc.com.

-neil

UVM Express Step 2: SVUnit with Covergroups and UVM Agents

I can honestly say that as of a couple weeks ago, I’ve gone further with SVUnit than I thought was realistic when I first starting looking at it. Having done more TDD, written more tests and recently finished step 2 of my UVM Express example, I’ve come to the conclusion there’s value in unit testing anything! That includes coverage groups and UVM agents… which I talk about here.

Continue reading

Why I Like UVM Express

This week Mentor released an extension to UVM called UVM Express. Normally, when someone announces an extension to the UVM, it involves more code or more tools. Not so in this case. With the library passing 67,000 lines of code (can that be right??), Mentor isn’t just piling on more code. UVM Express is an “extension” that helps people use what’s already there.

Here’s a few excerpts from the UVM Express page on Mentor’s verification academy with some additional commentary:

The UVM Express is a collection of techniques, coding styles and UVM usages that are designed to increase the productivity of functional verification. The techniques include raising the abstraction level of tests, writing tests using BFM function and task calls, adding functional coverage, and adding constrained-random stimulus generation.

Seasoned (aka: skeptical) verification engineers that have seen their share of new product announcements promising “increased productivity in functional verification” and suggesting “raising the level of abstraction” might be tapping the back button by this point but I’d encourage those skeptics to read on. I think the “revelations” appear in the next few sentences. Continue reading

By Example: Test-driven Development of Verification IP

I’ve been putting a lot of time into developing new examples for SVUnit lately and as of wednesday last week, I’ve finished another to show how people can use SVUnit to do test-driven development of verification IP.

This particular example involves development of an APB master BFM. APB isn’t the most complicated of bus protocols but it’s a very good subject for an example like this because the code and tests are easy to understand (there’s an APB example in the latest UVM release also, presumably for the same reason).

UPDATE: for people looking at UVM Express that was announced by Mentor Graphics on Feb 22, this is my interpretation of how TDD can be applied at the lowest layer development of the BFM. Examples showing the addition of functional coverage and completion of an agent that includes sequence generation are still to come. You can see UVM Express on the Mentor website by going here.

Continue reading

Unit Testing UVM Components: The Making Of

Today we take another step into the practical with a demonstration of how SVUnit can be used to test UVM components.

In the 3rd installment of the SVUnit Demo Series, I take people through a simple – yet complete – example of what’s required to test a UVM component within the SVUnit framework (if you haven’t seen the video yet, I’d recommend watching it here before reading on). The example I put together, and more importantly the plumbing under the hood required to make it work took me quite a while to put together for reasons that I don’t really get into in the video… but I will talk a bit about them here.

The usage model for SVUnit involves sequentially running a series of classes or modules through a corresponding list of unit test methods. In UVM however, due to the tight coupling between the phase methods in all uvm components and the instance of the uvm_root that ultimately drives the invocation of each phase method, uvm components are run in parallel.

One usage model wanting to be sequential and the other coded for parallelism gives us two models that are fundamentally at odds… so for unit testing UVM components, I had to get creative :).

I wouldn’t consider myself a UVM expert so the best way forward wasn’t immediately obvious. First I attempted to disable the components I wasn’t interested in by temporarily removing them from the component hierarchy. That was a dead end however because of the local access restrictions. My second idea was to replace components with a dud that effectively had no implementation… which means if the dud ran in place of an actual component, nothing would end up happening. That too was a dead end because of the same local access restrictions of the component hierarchy (in hindsight I should have known right away this was a no-go but I was still learning!).

The last implementation that ended up working involved creating and adding a new uvm_domain that ran in parallel with the existing UVM domain (where the run phases are stored). In the video, I talk about idle components that do nothing and a single unit under test that is driven through the run phases, iteratively if necessary. The idle components end up being idle because the newly created domain they’re assigned to – after of course going through the common phases of build, connect, end of elaboration and start of simulation – ends up raising an objection in the pre_reset_phase. That objection effectively means that components assigned to the domain never advance further. That’s what idle means… never advancing further than pre_reset.

I was pleasantly surprised to see that components could be assigned to different domains at any time which means I could activate and deactivate components whenever I wanted simply by changing which domain they’re assigned to.

When a component is deactivated, it is assigned to the new idle domain by…

function void svunit_deactivate_uvm_component(uvm_component c);
  c.set_domain(svunit_idle_uvm_domain::get_svunit_domain(), 1);
endfunction

Similarly, where I show a component being activated, it is being assigned to the normal uvm domain by…

function void svunit_activate_uvm_component(uvm_component c);
  c.set_domain(uvm_domain::get_uvm_domain(), 1);
endfunction

That seems to be the trick to being able to run uvm components sequentially in SVUnit without adding any new control or requirements to the components themselves, something I was looking to avoid from the outset.

If you want to take a closer look at the code, you can become an SVUnit project member and early adopter by getting a hold of me at neil.johnson@agilesoc.com.

Bonus marks go to the person that looks at the code and can suggest a better way of doing the same thing!

-neil

Work Smarter, Not Harder

You’ve been working 12 hour days for the last 4 months. You’re coming off a night of only 4 hours sleep because all you could think about was the all-hands meeting with the CEO, CFO and CTO the next morning where you’d be reminded that you’re 3 months behind schedule. Your alarm sounds. You drag yourself into work. You’re there in the meeting – just barely – leaning back half asleep. After listening for 30 minutes about how important <this release> is the company, you finally hear it…

We need to find a way to work smarter (not harder).

The ultimate bit of useless advice… work smarter, not harder – or some variation thereof.

Continue reading