I’ve been learning Clojure over the past few months. But that’s only slightly relevant to this post. As part of my learning process, I’ve been listening to recorded lectures from several Clojure conferences. There was one lecture by by Rich Hickey (creator of the Clojure language) called Hammock Driven Design which I found quite interesting. It’s about 40 minutes long, but I think it’s well worth a viewing.
My main take-away from this lecture – you may be able to guess the main thesis of this lecture from that title – is that for the hard problems that we need to solve we need allow ourselves (specifically our brains) the time to think about the problem.
If I could summarize the talk in a couple of points:
- State the problem you’re trying to solve. Rich’s thesis is that we don’t create features, we solve problems. I like that description very much.
- Understand the problem ask yourselves all the questions about the problem you’re trying to solve, the limitations and constraints the problem space is bound to. Never stop asking yourself questions. Be your own best critic.
- Do the Research regarding the problem: gather facts, earlier work, etc. Use a diverse sets of ‘inputs’ as possible.
- Give your brain a chance to percolate through these ideas, winnow out the unworkable solutions and let the good ones percolate to the surface.
There was a lot more to the talk and he incorporates some research on how to prime your brain, and then trust that it will do what it does best: finding the best solution for the problem given the constraints you have.
An interesting statement (for me at least) that he kept coming back to was the value in writing the problem down, capturing the facts, constraints, earlier research. The very act of capturing this information allows your brain to absorb all the necessary information it needs to do its job. This statement resonates strongly with me because I’ve always been a visual person and a writer-down-of-things. The very act of writing and drawing diagrams solidifies things and makes it easier for me to spot errors, inconsistencies, and the cool, elegant stuff as well.
This capture phase I do pretty well. I’ve always liked to do as much analysis as possible before I put my fingers to keyboards. But this is a phase I certainly short-circuit more often than I like to admit. From my viewpoint, I think that many of us do that far too often. Or (hopefully a smaller set) skip that step entirely.
Too often the “code” becomes the “design document”. While the project seems to be successful with a lot of early ‘wins’ by creating the code. Usually that missing supporting documentation comes back to haunt the team at some later in the project – usually at the worst time.
Just to be clear, I don’t think there needs to be a BUFD (“BIG Up-front Analysis/Design”) phase. Instead, I’m suggesting a JEUFD (“JUST ENOUGH Up-front Analysis/Design”) phase. I believe that the time you take to do the analysis (problem statement, facts, constraints, limitations) and capture and carefully scrutinize your design will pay for itself in the implementation and integration stage. Principally, by reducing the likelihood that you won’t have to refactor (or rework) later in the project, and times spent in debug – when you can least afford it. This lies at the heart of Neil’s “Defect Prevention” Professional Quest.
2.1. Design First
Design First – it sounds so basic.
The value of taking a pencil to paper and create a design means that I’m taking time to consider all the options before I start coding. Ultimately, it saves time because I can eliminate the options that I believe are not maintainable, extensible, or just won’t work! But I think more importantly it allows the team to work as a team – rather than a group of talented people working on the same project. Capturing your design in a consistent and concise method provides an opportunity to discuss with your peers the value of your design. They can offer (ideally) constructive feedback, or suggest ways on how something they’ve created can help you. But you can’t share what you don’t have captured. If you’re sharing it when you’ve already implemented there is too much inertia to overcome to change it.
As you build this simple design document as a team you’ll find that the pieces will at worst be more consistent than they would be without documentation, and at best will make sure that the individual pieces are of higher quality and will integrate more easily. Another side-benefit of this approach is that you’ll have the team understand at high-level the entire design, and how their piece must integrate in.
Now, I’ll try and relate all the above back to using an Agile process. Based on comments I’ve seen or heard a prevailing belief that agile methodologies lack discipline and documentation. Since it’s iterative the thinking is you can always learn how to do things better and ‘fix’ it in a subsequent iteration. I’d like to convince you that, in fact, agile methodologies enforce discipline, principally by creating opportunities to discuss and review progress, but just as important you can still be agile and create useful, consistent, and evolving documentation. That’s key: it must be documentation that adds value to the team and project, and is evolves in lock-step with the overall design.
In order to add value, time and effort must be used to create concise documentation that is useful. The one question that may pop up now is how detailed does this design documentation need to be? Of course, it depends on size of the feature you’re designing, but generally if the initial design view is more than six pages long then you’ve probably got too much detail.
Ideally, you want to capture an initial view and as with other agile discussions this will be incrementally added to as more facts are understood. The process that I recommend is:
- Create a document blending text and diagrams stating the following:
- state the problem you’re going to solve;
- known facts;
- known constraints and limitations;
- any unknowns.
- Capture an initial view of the design, but with just enough detail that commensurate the statements listed above.
- Scrutinize and be critical about your own work.
- Share it with others and demand feedback.
- As more details emerge: rinse and repeat (i.e., start at the top of this process).
Do you think I missed anything here?
How Do I Capture This?
Now, if you’ve actually watched the Hammock Driven Design you’ll have notice that the presenter makes some cryptic and somewhat disparaging remarks about UML (Unified Modelling Language) diagrams. I’m not really sure where that comes from, but I suspect it comes from folks who believe that UML diagrams are all you need. I don’t. But I do believe that can add a great deal of value to the design process. They are standardized methodology of capturing design intent, and system behaviour.
For the past several years, I’ve been using UML diagrams along with a minimal amount of glue text to pull together a just good-enough view of the verification environment I was definitely. I actually use a sub-set of the UML diagrams (class, sequence, user story) as they seem to be most effective way to communicate class relationships (the Static Class UML diagram), how the classes work together (dynamic Sequence diagram) and a concise way to capture behaviour of particular feature (using the user story UML diagram). I’ve also found that the Sequence diagram is particularly good at explaining the transactions that make up my verification “sequences” e.g., uvm_sequencer.
I recently wrote a for SNUG Canada 2012 a paper describing how I use UML to do initial design capture. I’ve been using the plantuml tool to capture UML diagrams quickly and easily in a textual format (in a text editor which lowers the barrier) using a domain specific language (DSL), and plantuml creates very usable diagrams. I highly recommend you investigate the tool. If you know UML already the ramp-up is quite small, and if you don’t UML… well you should investigate that first.
If you’re interested in this paper just email me at firstname.lastname@example.org and send it your way.