It’s finally time to see if TDD is a viable technique for writing RTL with verilog. But first, a little backstory…
For the Agile2014 conference in Orlando this past summer, Soheil and I built an Agile hardware/software co-development demo using a Xilinx FPGA with an ARM dual core Cortex-A9 to show how TDD could be used to write embedded software, drivers and RTL (i.e. TDD of a complete system).
As it stands now, the software and drivers running on the ARM core are complete. We built Conway’s Game of Life with all the supporting software and drivers necessary to create a series of video frames, then DMA those frames to the FPGA.
On the FPGA, we have a pipeline of IP blocks that transfers the software built video frames to an HDMI connector on the development board. Starting from a video DMA controller, frames travel block to block over a series of streaming AXI interfaces ultimately being displayed on the monitor connected to the HDMI.
We don’t have it shown in the graphic, but we also inserted a module into the IP pipeline that changes the colour of certain pixels before they go to the HDMI controller. For the demo this was an absolute minimum because it was hardware logic we could build with TDD. This absolute minimum was enough to impress a few software people that saw the demo, which was good. Hardware designers, though, would probably have laughed at our TDD-built module… mainly because it was laughably simple and doesn’t even come close to proving TDD of RTL is viable.
So that’s where we were as of August 2014… we had a demo that showed TDD is viable for writing embedded software while in all honesty, we were pretending TDD was viable on the hardware side.
No more pretending to do TDD of RTL. It’s time for the real thing.
Over the last month, I’ve been hunkered down doing a real proof-of-concept to show that TDD is a viable technique for writing RTL. I’m replacing the trivial module we originally had in the demo with something far more substantial: a module that does some real processing of the video frames as they pass from the video DMA to the HDMI controller. Here’s the initial block diagram and plan I scribbled down a month ago…
Ultimately, the completed design will add a glow of configurable depth around living cells created by the software. The ingress logic pulls frames from the video DMA and pushes them to a memory on the FPGA. The egress logic pulls processed frames from the memory and sends them down the line to the HDMI controller. Between the ingress and egress logic there’s an engine that reads pixels loaded in memory, modifies the pixels to add the glow, then writes them back to the memory.
The focus is TDD. As you can see by the steps I have listed in the plan, though, part of this proof-of-concept is showing that incremental development of RTL is also viable and that refactoring RTL as I go is not only manageable but beneficial. Step 1 ignores the processing in the middle and focuses on the ingress/egress logic (which basically equates to a FIFO). In subsequent steps, I add a narrow outline, then a wider glow, and finally the configurable glow.
Admittedly, TDD of RTL for a verification engineer is a bit of a stretch… but hey… someone has to do it :).
As of right now, I’ve completed step 1 (the ingress and egress logic) and am working my way through step 2 (a single pixel outline of live cells). I’ve learned quite a bit so far and in the next few weeks, I’ll share some of that.
So far, I’m pleased to say that everything is going quite well :).
-neil
I’ve been using TDD successfully over several hardware projects now.
My preferred flow is to use the python unittest module to create functional tests as part of the specification. We then create a loosely timed Python model of the system and get the tests passing against the model. Then we write the RTL, running the same tests against the simulated SystemVerilog RTL using Cocotb. When we have enough to move onto real hardware we run the same unittests against the real hardware.
We can then fill out the remaining functionality, running all the tests against model, simulated RTL and hardware automatically as part of our regression.
As soon as we’re confident in the behaviour of the models we can start creating randomised tests, using the model as a reference for validating the RTL behaviour both in sim and on hardware. We track code/functional coverage of the model and RTL and use this to determine when we’re happy to release.
Using Python it’s easy to mock things, or programatically generate tests.
Even for fairly complex designs using significant percentages of the latest and greatest FPGAs we typically run the entire unittest suite (thousands of tests) against the models in a couple of seconds, against the sim takes a little longer but it’s still only a couple of minutes. For the randomised testing we have a variety of different run lengths for post-check-in and overnight etc.
I’m not sure it’s possible to develop faster than this flow – we get optimal re-use of the tests, validating the model and then the RTL. We get to do architectural exploration using Python and re-use these models for verification. Sometimes we have achieved things in a matter of days that seem to take other companies multiple months 🙂
I’m actually training up my team to do TDD for Verification. We just had a big discussion today on it, so your timing is fortuitous. They see huge value to it. Next stop: RTL designers!
erik, that’s great. nice to hear your new team is interested.
-neil
I like to to TDD with formal checking tools (e.g. jaspergold).
Write properties directly from specs, then do the rtl. Works a treat on small incremental pieces of the design.
Things like systemverilog interfaces are good for this also – write the assertions from the interface spec, then add the rest of the rtl.
The assertions/covers are always there from then on, and get “instant” testing in the formal proof tool, and once the verification team have the test benches up and going, they’re automatically included.