Agile mashup in practice

“Traditional methodologies are a bunch of stick-in-the-muds who’d rather produce flawless documentation than a working system that meets business needs, Lightweight, er, ‘agile’ methodologists are a bunch of glorified hackers who are doing to be in for a heck of a surprise when they try to scale up their toys into enterprise-wide-software.”

– Jim Highsmith, in “Great methodologies Debate: Part 2″, Cutter IT journal

That says, none of the methodologies is perfect alone. Since traditional/formal models like Waterfall, Incremental,RAD model, Evolutionary, Spiral, Component-Based development etc are almost obsolete and impractical. They lost their feasibility in most cases for their time-consuming and expensive nature. Besides, few developers have the necessary background to apply formal methods, they often need extensive training on the model and methods. Furthermore, these models does not support a good communication with technically unsophisticated customers.

Hence come the agile methodologies. To produce the product right we encounter many implementation in this agile era, there are many paths originated from the agile development practice.

Now, which to choose ? Should you choose one and stick to that ? In my experience I found choosing a single methodologies is quite impractical, partly because none can match up with the unique project requirement, partly because there are way too many methodologies defined

Lets look at the core blocks of some major agile practices.

Extreme Programming (XP)
Planning:
Planning goes along with the creation of user stories. Stories are kept in index cards with priority value set by the customer. The xp team assigns a cost in terms of estimate time to complete. If the cost is more than three weeks, the story is broken into smaller ones. Needless to say, as agile goes, stories can be added any time!

Stories which will be included are set with customer’s approval along with the delevary date. The stories with highest value and the most risky ones are implemented first.

Project velocity:
Project velocity is calculated after each release. It is a simple calculation based on how many storeis are implemented in the release. Project velocity helps estimating the next release and understand the estimation accuracy.

Design and Spikes:
XP design provides only the implementation guidelines for a story as it is written. Specifying extra functionality is highly discouraged. Do only the things that is required for the current sotry in next release. If encountered a difficult design problem, create a spike solution i.e. an operational prototype.

One thing that you must need is the use of CRC cards. These cards also contains the release and story information. They are the only design artifacts produced by XP.

Coding:
Having done with the story and desing, now dive into development ? No, wait ! First write up some test cases and unit tests. Once the unit tests are created, focus only on what must be implemented. XP also encourages pair programing.

Testing:
As mentioned earlier,testing starts before coding. Tests should be done using a framework which will enable regression testing as the iteration goes on.

Adaptive software development(ASD)

It is based on team self-organization and collaboration. It is almost a formal incremental and iterative method which gives high value on learing practice.

Speculation:
First statge of ASD is speculation when the project goal, constraints etc are decided.

Collaboration:
Collaboration in this methodology does not mean simply being a motivated team working together. It is based on trust, the team met must trust each other to
1. Criticize without anmosity
2. Assist without resentment
3. Communicate to solve a problem

Learning:
Learning is practiced by formal technical review of the component built by the team members.

The role of a team lead in agile environment

I think pure waterfall model is impractical. It is as impractical as the pendulum in pure physics where the thread is of no weight and no friction exists. And that is what you can never have in the lab.

We, the software developers know there is always ‘friction’ and nothing is ‘massless’. The ever changing nature of the human mind and limitation to perceive what it needs results in modification in the requirement. That why we are agile, we don’t just fall.

Now, trust and sense of responsibility are at the base of agile methodologies among some other. Here, every member in a team is expected to be self driven and self motivated. They are given task and set free to accomplish that at their own choice. Formal managing, let alone macro managing, hardly gets its place. Then what would the team-lead do ? Should s/he dive into developing? or is there indeed any need of a team leader?

Actually, there is an unavoidable need of the team-lead-role. And s/he should never dive into developing unless it is that required. Rather, he(please add an ’s’ before ‘h’, I’m lazy.) role is to keep all his mets connected as well observe aspect of their work.

Team-Lead role is almost similar to the role of the wicket keeper in cricket; cheering up the players and catch all the balls behind. A team lead should also look for the quality and security of the code component. As well, he will also ensure development path is not deviated from the path. As we are talking about agile methodologies, change may come, but that change should be verified by the team lead.

Besides, the team lead should accumulate the knowledge learned by each of the team members. He is to somehow implement the notion of adaptive software development(ASD proposed by Jim Highsmith) i.e. focus on collaboration and learning. He is to arrange an meeting to share the knowledge of the team mets. It can be in lunch time, can be after lunch hour or whenever seems appropriate. Remember, there is no need to prepare presentation as it often time consuming and deviates the concentration from the development. Main idea is to use any wheel if developed already by a team.

One point must be concerned about at this regard is to maintain the motivation and positive team spirit. Say two techniques are developed by two of the team members for similar type of problem. Both the developers are smart and techniques are smart too. Now, to server in future which method is to be used? The team lead should carefully analyze(don’t take too much time, if not resolved drop it till a right time) the two techniques with participation of them and expectedly he should sort out the right one and has to convince both of them positively.

Always keep in concern to maintain the team spirit and motivation.

Interesting Code Review "Facts"

The benefit of code reviews is undeniable.

Yet, code reviews don’t happen in most software development organizations. And for good reason. The way code reviews are typically handled uses an enormous amount of people and time resources. Even with system quality improvements developers may not feel the effort is worth the cost.

It turns out you don’t need a high overhead code review process to get results. At one company I worked at I created a very low overhead automated code review system that enabled every line of code to be reviewed before it was checked-in.

Here are some ideas of how code reviews can be made practical in your organization.

1. Interesting Code Review “Facts”
2. Code Review Standard
3. Perspective Based Reviews
4. Deciding If Code Reviews Work
5. Are Code Reviews Necessary With Pair Programming?
6. Code Review Resources

---

My initial impression of code reviews is that they were heavy and bloated and a waste of time. I was wrong on all counts. Here are some interesting facts I found in the research:

1. Using two inspectors is as effective as using four or more.
2. Using an asyncronous process is as effective as a meeting based process for code reviews. For design reviews a meeting based approach may be more effective.
3. One review session is as effective as multiple inspection sessions.
4. Scenario or perspective based reviews were more effective than add-hoc and check-list based reviews.
5. Inspections are effective.
6. Inspecting upstream process like requirements and designs is very effective.

The implication is that the complexity and overhead of Fagan like code inspections are not needed and a semi-automated method will yield improved system quality. Requirements and HLDs seem to benefit from a more structured inspection process, which makes a certain amount of sense.

---

Code Review Standard

This is the list of some policies I would use to create a code review standard for my organization. Take what’s here, read the review on code research, and figure out what will work best for your organization.

The key point to keep in mind is that the heavy weight nature of code reviews is not true. You can create a code review system for your group that is light enough that you can review every change in your entire system.

Every Line of Code Must Be Reviewed Before it is Checked-In

Do not allow code to be checked-in unless it has been reviewed, fixed, and rereviewed. If you can’t do this then your review process isn’t fast or light enough. It can be done.

Reviewing code after it has checked-in is next to useless as everyone is exposed to the unreviewed code.

Code Must Be Integrated, Compiled, Unit Tested, and System Tested Before Review

Spending time on code that is just going to change or doesn’t work is a massive waste of time. The entrance requirements for a code review are:

• Code should compile without errors or warnings (according to coding standard).
• Code should be already be integrated with its parent branch.
• Code should be fully unit tested.
• Code should already pass the system tests where possible.

Develop Perspective Based Reviews

For the issues that are important in your software consider developing a Perspective Review for each issue. Perhaps special hotspots for you are internationalization, memory corruption, memory usage, threads, semaphores, etc. An example can be found in Semaphore Perspective Code Review.

You can then use the perspectives to assign roles to review team members.

Use Meetingless Asyncronous Reviews

You don’t need to hold a big meeting for a review. Everyone can review the code when they can find time. Simpy have a done-by date when the review must be completed.

Everyone on the review must review the code. If a person can’t perform the review then the team needs to elect someone else to perform a similar role in the review.

Meetingless aysncronous reviews are fast and light, yet they are still effective. With the right tool support you can easily review every line of code in your system.

Use Between 2 and 5 Relevant Reviewers

Too many reviewers wastes everyones time. Keep the number of reviewers small and relevant to the code being reviewed.

Assign Reviewers Roles and Perspectives

It’s almost impossible to review a lot of different issues in a lot of code. People get tired and they stop noticings things.

The way around the getting tired issue is to use perspective reviews. Create a perspective for each important issue category your are concerned about. Assign the perspectives to people on the review team. Because they are only reviewing for issues in their perspective they will do a better job because they can stay more concentrated. This doesn’t mean they can’t find other issues as well, but they responsible for their perspective.

For example, if using semaphores correctly is important in your software and the code has semaphores, then assign someone the role of reviewing semaphore usage. An example can be found in Semaphore Perspective Code Review.

All Review Communication Should Go to the Review List and be Logged

Part of the benefit of a review is that people learn about the system being reviewed. This learning feature is facilitated by broadcasting email communication between the review team and saving all communication so it can be read by other people later.

Do Not Redesign in the Review

Make a note and schedule design issues for for a later time.

Developers always think they can do stuff better. Take these issues off line unless the issue is that requirements are not being met. Requirements not being met is not the same as you could have done it better.

Do Not Cover Coding Standards Violations in the Review.

Send violations via email or in person.

Talking about violations only gets everyone angry and is a waste of time.

Code Is Rereviewed Until it Passes

Code isn’t reviewed once and then forgetten. Any changes made have to be rereviewed. If you think this is too slow then your process isn’t light enough.

No reviewing all changes makes the process useless as people will just ignore suggestions or introduce new bugs in any changes.

All Issues Must be Fixed, Marked as Not an Issue, or Marked as Bug

Any issue brought up to a developer must be handled. A developer just can’t ignore issues because they think it’s stupid. Every issue must:

• Fixed.
• Marked as Not an Issue. If the developer and the reviewer can’t decide between themselves if an issue should be fixed or not, then the review team gets to decide. If there is only one reviewer then bring a manager in or another team memember.

Automate Your Code Review System

You can make your process light enough by building it into your build system. If your process isn’t light enough work on until it is.

Review All Code on Private Branch Before a Merge

Code developed on a private branch doesn’t need to reviewed during development. But before the code is merged into a parent branch all code changes must go through the complete review process. For this reason, development of a large scale feature, may still want to perform reviews on the private branch because that can speed up the merge process.

Of course, try not to have branches separate from the mainline, but for large features that take a long time to develop you will often need separate branches.

Review the Right Scope of Changes

You don’t have to review every line of code in every module that has changed. Certainly if a module is new it must be completely reviewed. Other than that you may be able to just review the changed code. Though just reviewing changed code isn’t always possible. If you are performing a semaphore perspective review, for example, then you will need to go look at the code within in the scope of the sempahore as well.

Stick to Reviewable Issues

Develop your list of what issues can be reviewed and how they are to be reviewed. Usually this is in the form of check lists and perspective reviews. Don’t allow reviews on other items without changing what can be reviewed. Otherwise people spend endless time on off-topic arguments.

Review Team Reponsible for Deciding Issues

If there is a conflict on any part of the review then the review team is responsible for handling it. That’s the only way the review process will be light enough to work.

Keep it Cool

Nobody is perfect. The attitude of the review should never be personal, it should always be professional, with the goal of improving the system and the people building the system. Keep your tempers.

Don’t blame people for bugs. Work together to make things better. No finger pointing! Not ever!

Meetingless reviews can help keep the anger down, but it can make it worse too. When people are in the same room anger can ramp up really quick. And we know in email it’s very easy to say something that can be take wrong. Raise the awareness of these issues in your team.

A good rule is to Never Assume An Attack. If you find yourself getting angry, assume it’s a misunderstanding, not an attack.

No Managers

Unless a manager has something to add to the reviews then they shouldn’t be involved. Issues should be decided by the review team. Managers always have to run to a different meeting, they don’t have time slots open for meetings, and they generally don’t add technical input. So you don’t need managers as part of the review process.

If a Bug Was Not Caught in a Review Figure Out Why

If a bug happens after a review then track down why each bug wasn’t found and then change your development process somehow to try and prevent that bug in the future.

This is not always possible as running full tests are often impossible, but it should be mostly possible.

I would create a bug for each bug to track down why it wasn’t caught. Because this issue is more serious than just the review. It means the unit test, the system test, and the review did not work.

Review Upstream Documents Too

Reviewing product requirement documents, specs, standards, etc can provide excellent return on value. Make sure those products are reviewed as well.

Feed Leasons Learned Into the Team and Documentation

If issues come up during the review that everyone in the team would benefit from, then have a way to make wisdom public.

I would recommend a development wiki where you can write documentation on anything useful that people come up with.

Plug Reviews into Your Source Code Control System

I have done this through change check-in comments. Each change has to be reviewed before it is checked-in. The submit comment must contain a review ID that points to some document containing the review status for the change that is about to be submitted. The code is prevented from being submitted without a valid passed review. If you are able to automate your code review system all this works quite quickly and painlessly.

---

Perspective Based Reviews

Before my reseach into code reviews I had never before heard about perspective based reviews . Perspective based reviews are done from the point-of-view of various project stakeholders. This is as apposed to everyone doing an add hoc or checklist based review. A perspective would be a user, requirements person, hardware interface, exception handling, mutex usage, memory usage, etc. The thought is they are more effective because reviewers are concentrating on a particular issue instead of trying to review everything. Do one thing well i guess.

Reviews are driven by scenarios. A scenario tells the insepector how to go about reading the documentation from particular perspective and what to look for. Clearly there’s a lot to the scenarios.

Perhaps people could be assigned in various review roles. Or maybe some people can perform reviews only from a certain perspective.

---

Deciding If Code Reviews Work

How do you know if code reviews are working for you? Look for

• Increased in test execution rate. The tests suites runs through its test suite. Unstable software can’t do this.
• Increased pass rate.
• Reduction of critical bugs in QA.
• Reduction of field issues.
• Subjective improvements in quality.

Try to measure the issues to verify if code reviews are working for you.

---

Are Code Reviews Necessary With Pair Programming?

An issue that comes up is the relationship between code reviews and pair programming . Do you still need code reviews if you pair program?

If you don’t pair program then you definitely should perform code reviews. That one is easy. Unit tests and system tests simply aren’t good enough to replace code reviews.

After that it depends.

It depends on the product you are developing, it’s importance, it’s complexity, and how critical it is that there be no errors.

Pair programming is an excellent form of active code review, but there are many products where I don’t think two eyes are enough. Some systems are complex enough, some domains are complex enough, and some conseuqneces are severe enough that I want more eyeballs on them before the code is let loose in a real code running system.

When pair programming is used you could decide what code should be reviewed by subsystem or by the type of change involved.

For example, code in a key algorithm may need to be reviewed by the impacted parties. Perhaps a drive change needs to be reviewed. Perhaps when a piece of code is written new or completely rewritten, it should be reviewed. Perhaps if semaphores are added to code that would trigger a review because it changes the system a great deal.

Your mileage may vary of course.

Charecteristics of commercial software

Commercial Softwares – A term with whom most of the software professional, including me, are familiar with. There are thousands of different commercial software in different areas; even though all of them share some common properties such as: built by many, expandable, maintainable, understandable, and stable.

Lets see what are those:

• Built by many: Software that is built by a team inherently means that no single person has firsthand knowledge of all of the code. It also means that good communication among the team members is critically important to the software’s success. Communication has to start at the beginning; everyone on the team must share an understanding of what exactly is being built, for whom, and when it is needed. By understanding the customers and their requirements, you can always go back and ask yourself, “Is this feature I’m adding really necessary?” Understanding both the business and product requirements can help you avoid the dreaded feature creep or increasing scope that often delays products. Communication has to continue throughout the process: conveying interface designs so that integration among components runs smoothly; accurately and completely documenting your code; and conveying new member functions and their purpose to the Software Quality Assurance (SQA) and Documentation groups.

• Expandable: You must be able to add new features quickly and extend existing features in commercial software. These requests often seem to come when an influential customer finds that they need just one more thing to solve their problem. Even worse is when a limitation is discovered only after the product is released. Whether or not code can be easily expanded to accommodate these requests is often a function of the design. While it may be as easy as adding a new member function, significant new features require that the design be extended. Put simply, you have to think ahead when designing and writing code. Just implementing to a specification, with no thought to future expansion or maintainability, makes future changes much more difficult. A further complication is that most software won’t allow features to be deprecated once the product is in use. You can’t simply remove a class that was ill conceived, as it is certain that if you do so, you’ll find a whole host of customers using it. Backward compatibility is often a key design constraint.

• Maintainable: No matter how many unit tests are written, automated tests are run, or functional tests are executed, it is inevitable that bugs will be found after the product has been released. For commercial software, the goal of all of the rigorous testing and preparation is to ensure that any post-release bugs have minor consequences to the product. What becomes crucial is that these bugs be easily and quickly corrected for the next release. It is also likely that the person who implemented a particular component won’t be the person responsible for fixing the bug, either because enough time has passed that he has been assigned to a different project, or there are designated engineers responsible for support and maintenance. Given these constraints, maintainability must be designed into the software. There has to be an emphasis on solid design practices and clarity in coding styles, so that the software can be efficiently maintained without further complicating it.

• Understandable: Commercial software often includes visible software interfaces. If you are building an embedded system library, it needs to be understandable to the application engineers within your company. If you are building a software library, the interface you present must be easily understood by your customers so that they can use it to build their own applications. This doesn’t just mean that naming conventions must make sense, but more importantly that your design and your use of language elements, like templates, are clear and appropriate.

• Stable: Commercial software must be stable; that is, it must be able to run for extended periods of time without crashing, leaking memory, or having unexplained anomalies.