Personal Software Process — The Good, The Bad, and the SEI

I recently took the Personal Software Process (PSP) training governed by the Software Engineering Institute (SEI). The PSP is the process-for-the-individual that supports the Team Software Process (TSP). Both were created by the late Watts Humphrey. While it was relatively fresh I thought I’d jot down some thoughts about the PSP as well as some advice to the SEI.

There are several key ideas in the PSP that are extremely powerful and compelling:

  1. Proxy-Based Estimating: The idea here is that when estimating you categorize “chunks” of work into various sizes and use your historical performance data as a “proxy” for the effort estimate. This idea alone is worth the price of admission, and if you work in an organization that relies more on up-front planning than on continuous refinement of schedules, this may be the only way to make it work over time.
  2. High quality results: The PSP as a whole very much focuses on the notion of the cost of quality. Loosely translated, the idea is that high quality software is actually cheaper to produce in the long run because the cost of fixing errors is so high. Classic Deming, but difficult to argue with if you’ve ever seen a project buried under the weight of its defect backlog.
  3. Using Earned Value (EV) to Track Progress: EV is a staple of standard project management procedures (refer to the PMI if you’re interested), but it entirely relies on starting with good estimates. The simplest form of EV is to simply mark each item in the schedule as only “done” or “not done”; none of this 72.4% complete nonsense. But in any event, knowing where you are in a project is unarguably useful.

Disciplined processes such as the PSP take a great deal of heat from the hacking community, or the toy-problem community, but I think you need to look at it with an honest appraisal of how effective your personal way of developing is; how accurate are your estimates? do you actually break down your estimates to items that are sized such that they can be estimated accurately? how buggy is what you release to QA? or to the customer? What is the cost (both to you and the company) of the bugs that get that far? Ruthless honesty is uncomfortable here, so you may want to add a little ruth back in just to make it palatable…

The PSP doesn’t deal with many critical issues such as scope and requirements creep — these are left for the Team Software Process. Still the PSP is worth checking out, if for no other reason than to broaden the scope of how you think about doing a project.

However, the SEI seems to be bent on some key barriers to acceptance of the PSP by a wider community. In order to appreciate the valuable things about the PSP, there is a lot you have to look past:

  • Pedantic Presentation: The PSP process is presented as a long list of things you must do, that you simply cannot do without in order to use it. This is in sharp contrast to methods such as Agile that provide a toolbox approach, and are much easier to adapt to. Further, much of the PSP book is devoted to working through the math that is used in adapting the estimation model — which is fine but gets in the way of the important material.
  • Outdated in Many Respects: The PSP insists on doing software inspections on paper, multiple pass inspections with checklists, logging time for the “compile” step, and other steps that are quite simply just grossly out-of-date. Even the insistence on doing inspections on paper — though supported by some empirical data — will simply never be widely adopted at this point. The PSP also does not incorporate newer thinking about unit testing or test-driven development, nor does it make a distinction between manual and automated tests. In many ways, it’s like a little process island from the early ’80s…
  • Defect Categorization is Weak and Poorly Defined: The PSP defines a “defect” as anything discovered after the fact that requires a change to an artifact, whether code, design docs, or whatever. This definition is just silly. A typo is not the same as a functional error. A defect found after release is qualitatively different than many/most defects found during code inspections (e.g. coding standard violations, typos, minor refactorings, etc). And although the PSP lets you classify “defects”, all the process quality metrics are based on raw, uncategorized defect counts.
  • Copy-and-Edit is not Reuse: Finally, the PSP seems to view code reuse as simple copy-and-paste-and-edit. How to fold an active reuse program into the PSP or TSP is not at all obvious, and the oblique way the PSP does it is not useful at all. This may be, again, just another example of how outdated much of the PSP is.

Without addressing these problems in the PSP, I doubt that it will ever achieve widespread interest or acceptance as compared to, say, Agile or Lean processes.


About jeffkotula

Software engineer, musician, and so on...
This entry was posted in Process, Software and tagged . Bookmark the permalink.

2 Responses to Personal Software Process — The Good, The Bad, and the SEI

  1. jeffkotula says:

    The cooking analogy is good.

    One of the problems is that there are two groups with very different goals. Developers are focused on building the thing. Managers are focused on keeping track of the building process. Worse yet, neither of the groups is very good at addressing the concerns of the other, sometimes not even recognizing the _legitimacy_ of the others! The places I’ve seen the best progress is when the developers take the issue in hand and first address the needs of the managers, and only after building up some good will do they start trying to get their own issues addressed.

    None of this has anything in particular to do with PSP/TSP/SEI but are really the meta-issues surrounding the whole problem.

  2. I feel like a lot of software process discussion reads like a recipe does, with many similar problems.

    A recipe doesn’t tell you how to find quality / affordable ingredients at the grocery store. (The tools you will use, as you execute the process.) It doesn’t tell you how to do substitutions for ingredients, if you have an abundance of one and a lack of another. It hints to you that it’s okay to do things out of order (like pre-heating the oven before you chop the onions), but doesn’t really help you estimate how long each of the steps will take. It doesn’t help you scale the recipe up to serving a banquet hall full of hungry guests, or reduce the recipe down to a single serving for one.

    If you’re already an expert cook, new recipes are valuable to you, and you can imagine how the different mixtures of ingredients will apply. You can imagine how each guest will respond to the dish. You can probably come up with wine pairings that will delight.

    Similarly, if you’re already an expert at getting things done in a team, then you can imagine how a new process will work with (or against) the people, to help or hurt them, as they try to get things done.

    If you try to directly apply a recipe for 2, for a banquet of 2000 guests, you will run into serious problems. You will be tempted to directly scale up, following each step, in sequence, just as it had originally been written. You won’t realize that you cannot just make one, much larger hamburger, to feed a large group. (Yes, that’s an overly simple example.) You won’t know what’s a good batch size to cook all at once. You can estimate how long it takes to grill one burger, but does that really tell you anything useful at all about how long it would take to grill 2000? How would you decide when to have multiple grills, multiple cooks? Is it going to be a buffet service, family style, or served individually? Will each guest get their choice of entrees, or is everyone eating the same thing, regardless of preference? The answers to those questions would result in very different atmospheres for the guests, and very different workflows for the chefs and servers.

    Reading a “cooking for one” book would be completely different from a “how to cook at Yankee Stadium,” would be completely different from “how to cater a wedding.” They’re very, very different. I’m not sure software process discussions are appropriately scoped to different audiences.

    And so that’s what frustrates me about software process discussions. They’re not very helpful, and they often cause a lot of harm, because they make you feel like you can directly apply their recipe to your situation, just by scaling things appropriately… But that’s the hard part! You can’t just keep doubling the recipe. At a certain point, you need an entirely different recipe.

    Okay, Jeff, I know you also don’t like everyone’s attitude that their situation is different…

    But I’ve worked now at a few companies that did things very, very differently from each other. And it would be very hard to write one book that could address all of their situations collectively. Worse, the discussions I’ve seen about software process haven’t been very inventive. Waterfall, and agile. Okay, great, that frames the conversations you’ll have in certain ways, but it really doesn’t address the full pipeline of how you will deploy and maintain features, in the hands of customers…

    In case you can’t tell, I’m finding this hard to speak about in generalities, without stupid metaphors that are probably more misleading than helpful… Here are some more specific complaints:

    I haven’t seen many software development process books start with the questions like, many branches (for each project) or no branches? How do you expose an API to external customers (over time, as it changes)? What happens when you committed to release a feature in version 7, and at the last minute, you need to ship it in version 6? What kinds of tests are appropriate, and how much effort should you invest in them, at each of their different scales (unit tests, integration tests, etc.)? How do you handle a firm deadline with flexible content… especially when you have no idea how long any of the content will take? How do you deploy a critical fix, in the midst of normal of development? How do you know what’s happening in production? Do you fix things first, or add new features first (and “fix everything” when you’re done?) How do you not pay too much attention to that one squeaky customer that you should probably ignore more often? How do you monitor the health of the department itself, and the morale of the people in it? What’s a good ratio of developers to testers, given where the product is at in its development / maintenance process? How do you document the design of the old parts of the system that have changed some, but not a lot? The project has to ship on time, and it’s not going to make it – now what? You lost a critical developer, and they left a bunch of critical code that no one understands… now what?

    If you were teaching a kid how to handle a car, you’d probably teach them how to change a tire. Because flats are going to happen, and you can’t always just use your cellphone to ask for help.

    Well, each of those situations I described are really going to happen to you, if you develop software long enough. And I feel like people are fundamentally not prepared to handle those situations.

    A real software process should start with questions like, which of those possible outcomes would be the most damaging to your company, and which are the most likely? Okay, we just found out that some of them would destroy your company, and are very likely… Well, let’s not put off the discussion of how to handle them, until they happen – let’s put managing those eventualities front and center in how we design our process.

    And on the flip side, how do you create an environment of “flow,” where everyone can just get their work done, without thinking about how they’ll get it done. Uninterrupted productivity. That question depends entirely on which kinds of problems you want to flow well. It’s tempting to say “adding features and fixing bugs,” but that thinking is like saying “We want to win the war!” It’s a vision, it’s not a strategy. It’s not broken down to a sufficient granularity to make meaningful plans.

    Okay, hopefully this wasn’t too much of a rant, and possibly sparked some useful thinking…

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s