Turning Documentation Into (Unit Testing) Action

A hypothetical for design engineers… what if there were an online tool useful for both documenting your RTL and bootstrapping a testbench. Would you use it?

The tool is Wavedrom. It’s an open source tool hosted at wavedrom.com. You may already use it for documentation. I’ve used it in the past for documenting BFM behaviour. It’s accessible, easy to use and the output is clear. Highly recommended.

If you haven’t seen Wavedrom before you should load it up to see what it can do. By default, it comes up with a simple req/ack data transfer to illustrate the basics. The input is JSON which is pretty easy to work with. Output can be exported as PNG or SVG.

If you want to try something from scratch to see what it’d look like, you can paste in this APB write transaction… Continue reading

Unit Testing UVM Sequences

Next to unit testing UVM drivers, which was the topic of Testing UVM Drivers (Without The Sequencer), the second most popular question for which I had no good answer has been “How can I use SVUnit to test my UVM sequences?”.

Thankfully, SVMock seems to have made life easier here as well. With SVMock we can isolate a sequence from sequencer and driver such that it can be unit tested on it’s own before it’s used as part of a larger system. Here’s an example of how it works. Continue reading

Testing UVM Drivers (Without The Sequencer)

So. A couple weeks ago I introduced SVMock. It’s a mocking framework for use with SVUnit that makes it easier to isolate, check and control behaviour of Systemverilog classes. Unsurprisingly, the response to that announcement averaged out to tepid. A few people were immediately interested. I’m sure a huge number of people didn’t care, probably because mocking has never been on their unit test radar. Then there were people in the middle who were interested but I didn’t give them enough to get over the great-but-now-what hurdle.

This post is for the last group, the people that reckon SVMock can help them write better unit tests but don’t quite see how. The test subject to get the point across: the uvm_driver. Continue reading

SVMock Version 0.1

A few times over the last while people have suggested I add a mocking framework to SVUnit. Took me a while, but I finally got around to it. SVMock version 0.1 is now available on GitHub. I’m actively working on it but the base functionality is there and ready for use.

If you’re new to mocking, in the context of unit testing it’s a technique for replacing dependancies of some unit-under-test to make it easier to test. Mocks increase isolation so you can focus on development without the hassle of including large chunks of code, infrastructure or IP you don’t necessarily care about. Mocks inject new sample points and assertions that make it easier to capture interactions with surrounding code. They also offer increased control by substituting potentially complex dependancy usage models for direct control over interactions. In short, mocking helps you focus on what you care about and ignore the stuff you don’t.

Now to SVMock… Continue reading

Lessons Learned From An SVUnit Early Adopter

Funny thing happened today.

After reaching out to people last week for SVUnit success stories, to my pleasant surprise I found one in my inbox this morning. It was from SVUnit early adopter Manning Aalsma of Intel (formerly of Altera). When I say early adopter, I mean early. Looking back, I found the first email he ever sent me still in my inbox requesting an early version of SVUnit. Timestamp on that was Jan 8, 2012!

I’m happy to have people like Manning using and advocating for the framework and the techniques that go with it. Here’s what he had to say about he and his teammates using SVUnit.

Thanks Manning!

-neil


Hi Neil,

Catching up on a little reading, I came across your post from a week ago or so:

http://agilesoc.com/2018/06/11/is-svunit-a-legitimate-verification-framework/

I thought I’d share my experience so far. Continue reading

Is SVUnit A Legitimate Verification Framework?

Is SVUnit a legit verification framework?

I get that question periodically from folks who are looking into incorporating SVUnit into their verification flow. Of course it’s always phrased a little differently depending on who’s asking – How many users are there? What is the bug rate? What teams have integrated it into their verification flow? Have people published papers about it? Is it actively being developed? Do others contribute to the development? – but the intent behind the question always feels the same. We developers want to know that others have blazed the trail before us, that the tools we’re considering have a proven track record, that the major bugs and issues are long since fixed and that tools are truly ready before we get started with them.

Unfortunately, it’s a tough question to answer. Being that SVUnit is open-source and usage is basically anonymous, unless people reach out to me personally I can’t make any definitive claims.

That said, I’m confident we have enough anecdotal evidence for a solid yes. SVUnit is a legitimate test framework for design and verification engineers looking for an alternative that addresses low level code quality.

Here’s a few stats to support that yes: Continue reading

The Time It Takes To Find A Bug

How much time passes between writing a line of RTL or testbench code and knowing that it works. I know it varies, but on average how long would it take? A line of code is written, time passes, then a test is run that either verifies it’s correct or discovers a bug. Does that time pass as minutes? As days? Maybe weeks? Months?

I’m preparing for a talk at DAC next week in the Verification Academy booth and in going through my material I’ve started asking myself that question. It’s the 3rd year in a row the folks at Mentor have invited me for a session in the booth. The last 2 years have been a lot of fun and I’m hoping for the same this year. This year’s session is called Add Unit Testing To Your Verification Toolbelt. As the name suggests, the focus will be on how we can use unit testing and test-driven development to improve the quality of the hardware we build.

I believe quite strongly in the value of unit testing and test-driven development because the quality of what I produce as a verification engineer is noticeably better because of them. I believe so strongly in TDD in particular that I know I get a little preachy at times. It’s the most important engineering skill I’ve ever learned and I don’t shy away from telling people about it.

I always think of rigour first when it comes to benefits. Testing code as you write it a few lines at a time is a rigorous process to say the least. But during the run-up to DAC I’m starting to think it’s timing that’s the real key. With TDD, the amount of time that passes between writing and testing a line of code is minutes, sometimes less. A design and test cycle measured in minutes is almost too short a time to think about anything else. I focus on getting exactly 1 thing right. There’s no thinking back, no back-and-forth with a teammate that found a bug, flipping through my logbook for clues as to what I’ve done or editing bug reports. In fact, with TDD there’s hardly any context switching at all. I write a line or 2, test them, if they’re broken I fix them, then I move to the next line. Most of the bugs I create are killed immediately; which coincidentally is the best time to kill them.

I’ve seen a lot of people talk about how bugs become more costly to fix the longer they live. For me, TDD has been the most effective way to respond.

If you’re at DAC next week in Austin and using TDD to kill bugs fast sounds good – or great! – I hope to see you at Mentor’s Verification Academy Booth at 11am on Monday. It’ll be a half hour presentation with lots of time for discussion. We’ll talk about TDD, unit testing, SVUnit and all the frustration that magically vanishes because of them!

Here’s a link to the session info for mine and others in the booth on VerificationAcademy.com.

See you at DAC!

-neil

I Can’t Hide From Unit Tests

There are times when I try and convince myself that unit testing my verification IP may not be necessary. Believe it or not, I can put together a pretty convincing argument. On a good day I shake it off and do the right thing. Other times I’m just convincing enough. I know it won’t end well, but I skip the unit tests and go straight to the code.

Here’s a couple arguments I’ve used on myself to get into trouble. Continue reading

Verilog/VHDL Mixed Language Unit Testing

 

SVUnit supports mixed language Verilog/VHDL unit testing.

Finally.

Download the Latest Version of SVUnit from GitHubIt didn’t take much to do it but for some reason it took me a loooooooong time to get at it. It involves a new switch to the runSVUnit command line. Users can now specify a ‘-m <vhdl file list>’ where vhdl file list is all their VHDL files and extra VHDL-related command line switches. It’s been tested with Modelsim and Incisive. Probably works with VCS though I can’t say for sure because I haven’t seen it with my own eyes. If someone wanted to give VCS a try and let me know what happens, I’d appreciate it!

The updated runSVUnit usage is (emphasis on -m|–mixedsim <vhdlfile>):

screen-shot-2016-09-14-at-9-01-56-pm

As an example, if you’re unit testing a mixed language design with Modelsim where VHDL files in your design are listed in myVHDL.f and your Verilog files are in myVerilog.f, your command line would be…

runSVUnit -s modelsim -m myVHDL.f -f myVerilog.f

As always, if there’s any trouble with it you can log an issue on github, send me an email at neil.johnson@agilesoc.com or leave something in the comments. If you were waiting on mixed language support to get started with SVUnit, now is your chance. Just scroll up and hit the big blue button to get started!

Happy unit testing!

-neil