By Alan on 2010-01-30 12:49:53Z

Yesterday, I imagined a dialogue between a customer consulting with an engineer/architect as it might be applied to constructing a building. Only I reversed the order of history, and wondered how it would be different if the science and art of building construction was as new as software engineering. Yesterday, we listened to the conversation between the customer and a traditional developer (of buildings). Today, we’ll listen to the conversation between a different customer and an agile developer.

The Agile Process applied to People Containers

Customer: “I’m getting married in three months, and I need you to put together a container where I can protect my wife and myself from the wolves and mountain lions. But first, I’ll have to get money from my dad, so I need to know how much this will cost.”

Agile Developer: “Well, we know that upfront designs are usually estimated poorly. I know about this one developer that was asked to build a people container, and he made a big upfront design—the old-school way—and when they finally delivered the container, that customer was really upset to find out that he’d bought a big hanging birdcage to put his family in! It was a total disaster. The customer had to get it lowered from the tree, and cover it with wood panels, put in vents, and so on. The original estimate was $10,000, and he wound up paying three times that, and not only that, but the project took twice as long as the original estimate of three months. So, the way I work is this: We’ll pick a period of time, let’s say about two weeks, and each period we’ll commit to completing something which is in line with your needs. Each period, you’ll get a clearer idea of what we’re building, and you can refine it until it’s exactly what you need.”

Customer: “What? Why can’t I just tell you what I want, and then you can go away and make it? You don’t really need me to help make it, right?”

Agile Developer: “Wrong. I totally need you to help make it. In fact, I will predict that if I do it that way you just suggested, you’ll get something that isn’t quite right, and then you’ll have to spend more money than you planned to fix it. Look, I’m not saying you need to do any of the actual construction. But every day, we’ll get together for a very, very short meeting so you can see how we’re doing and meeting our goals. Every two weeks, we’ll actually have a finished product. Now, naturally, the first couple of times we do this, the finished product won’t really be useful, the way the final product will, but it will definitely be a step in the right direction, because you’ll be there to help us understand what it is you really want.”

Customer: “That sounds great and all, except for one thing. I don’t know what I want yet. I just know what I want it to do.”

Agile Developer: “Perfect! That’s all we need from you. You tell us all the things you need it to do. Then we’ll prioritize, and make sure it takes care of the most important things first. When you think it’s finally doing everything you asked, then it’s done.”

Customer: “So it could go on for a really long time?”

Agile Developer: “That’s up to you.”

Customer: “But I said I need it before I get married in three months! How do I know it will be done before then?”

Agile Developer: “Because you are really the force driving what we do in each two-week cycle. You know your priorities better than we do, so you can make sure that each two-week’s goals will get us closer to your final goal, and since you know your schedule better than us, too, you can decide which goals to keep and which ones you can do without.”

Customer: “Yeah, that makes some sense. Only, how much is this going to cost?”

Agile Developer: “Again, that’s something you control. You should know how much value you need to get from this container."

Customer: “You mean, like how much this is worth to me? Well, see the problem is that it’s worth everything to me. It’s for my wife. But like I said, I’ll need to get the money from my dad. He’s an old-fashioned guy, and has spent his whole life in caves or up in trees. He thinks people containers are just really expensive problems waiting to happen, but we’re running out of room in the cave, so he’s agreed to let me do this. But he says I have to keep the cost down. All my friends tell me that agile developers are great at that.”

Agile Developer: “I see. So, then we have to stick with how much money you think you can get from your dad.”

Customer: “Uh… yeah, only I’m not sure how well he’ll take me saying ‘Dad, I need money, only I don’t know exactly what I’m going to get when it’s done.’ “

Agile Developer: “Well, look, you have a fixed deadline, right? And our average rate is about $1,000 per week. You want to finish just a little early, rather than late. So we can plan for ten weeks, five cycles of building. That’s $10,000.”

Customer: “That helps a little. But I still don’t know what I’ll get, other than that I’ll know what it will do, right? My dad is going to want to know I’m getting something more than a couple of those wooden boxes like they use for burying people.”

Agile Developer: “I can see you’re a little apprehensive about this. But rest assured, even by the first or second cycle, you’ll have a much clearer picture of what you’re getting than you could have right now, even if we tried our best to figure it out now.”

Epilogue

Near the beginning of this conversation, I suggested an ending to the story with the customer and the traditional developer. It wasn’t very flattering to the traditional developer, but the truth is that most traditional software projects are delivered over budget and late. See almost any of Steve McConnell’s works if you would like to see abundant research supporting that claim.

But what was the outcome of the agile developer? Bear in mind, both of these stories are totally fabricated from my imagination, but I believe I can predict a likely outcome.

Before beginning the project, the customer was asked to list his priorities for success. He came up with this list:

1. Big enough for him and his wife
2. Protection from predators like wolves and mountain lions.
3. Insulation from the elements (rain, cold, sunburn, wind)
4. Comfortable enough for sleep
5. Could be expanded for kids

Iteration 1: The team decided that the “big enough” requirement was largely an architectural requirement, and not something they could actually build and deliver on its own. However, they could certainly start to deliver something that offered protection from predators. They agreed, after discussing the possibilities amongst themselves and the customer, that an obstacle like a very high wall or very deep ditch could not be built in time and would be prohibitively expensive. The suggestion was raised to create an alarm system instead which could wake the customer (if he was sleeping) with enough time to take action. The customer reluctantly agreed, since he couldn’t really afford what he wanted.

The team committed to a “continuously constructed” tactic. Rather than having large chunks of work in an “under construction” state, they planned to have an actual container by the end of the first iteration, and to keep it inhabitable every day. By the end of the first two weeks, the team had built a meter long section of bells hanging from trip wires, and had placed a large flat rock on the ground about 50 meters from the bells. The customer was able to stand on the rock and hear the bells, and judged that he could probably keep his sleep light enough to be awakened by the bells and take action. The customer was both a little perplexed and pleased when the team celebrated their success at creating an inhabitable container. It didn’t look much like a container to him.

The cost for this iteration was $3,000, 50% more than what was originally estimated, but that was because $1,000 went to securing the large area of land (10,000 square meters, or about 2.5 acres) where the container would be built. The team assured the customer that they were making great progress and that, in all likelihood, they would finish under budget and ahead of schedule.

Iteration 2: Part of the team continued working on the alarm wires and bells. They needed to build over 300 meters of wires to create a circle that was at least 50 meters from the center. They determined they were completing 10 meters a day, so they focused on working the part of the circle deemed most likely to encounter predators, and set the goal for this iteration at 100 meters (they were not working weekends!).

Meanwhile, the rest of the team focused on the container. Clearly, the rock would not work to insulate the customer from any elements, but it would work well to anchor the actual container. The team and the customer discussed ways to insulate a person from the elements. The rain was seen as the hardest and most important problem to solve. Several members of the team had experience working on water container projects, designed to catch and keep rainwater for later use. They knew the best way to create a large water container was to dig a large hold in the ground and line it with something waterproof. The best waterproof material created so far was concrete. Unfortunately, it would be too expensive and difficult to apply for this project. But the team felt that the same principle could be applied here. The plan was to create a large upside-down bowl, and cover it with a waterproof liner.

So the team, sticking with their “continuously constructed” plan, began building the frame of a large, upside-down bowl. The goal for this iteration was to have the general frame of the bowl complete. At the end of the two weeks, they had created a circle three meters wide of stripped saplings on the ground lashed together with rope, and three arcs of saplings tied onto the circle and joining at the top. The customer could now begin to see something that seemed a little like a people container, and was pleased.

The cost for this iteration was $2,500, or 25% more than what was originally estimated. $500 went to purchasing stripped saplings, because the team did not have time to cut and strip them themselves. So far, the customer was delighted that he chose an agile developer.

Iteration 3: The wires and bells team had built 100 meters of the alarm system. With a little over 200 meters to go, they predicted they would speed up their progress with experience and could complete it by the end of the next iteration, but only committed to completing 100 meters this iteration.

The container team set a goal of completing the upright arcs of saplings around the circle. They were still trying to decide which waterproof material to use. They’d considered tanned animal hides, large tree leaves, and woven corn husks. They settled on tree leaves as the most practical given the time and expense allotted. However, the largest tree leaves available were only a little over 1/4 meter large. The team realized that meant the arcs around the circle would have to be spaced about 1/4 meter apart. With nearly 10 meters around the edge of the circle, that came to 40 half arcs, or 20 full arcs.

Three full arcs had already been built, so seventeen more needed building. It was taking the team a full day to build each arc, and they still needed to start fastening the leaves to the arcs. So they cleverly decided to start attach each new arc within 1/4 meter of an existing arc. That would allow another member of the team to start attaching leaves to a pair of adjacent arcs sooner than if they had waited until all the arcs were in place. They committed to building nine arcs for this iteration, leaving the remaining eight for the next iteration.

Unfortunately, there were not enough people on the team to work simultaneously on the wires and bells, the arcs, and the leaves. And they still hadn’t tried to tackle the requirement for comfortable sleep. They couldn’t push out the schedule, so they decided to hire more people to fasten on the leaves. The new team was brought on board and shown what to do.

The cost for this iteration was $3,000. The unplanned $1,000 was used to bring in the additional workers to keep the project on schedule. Nevertheless, the customer was getting very excited. He could see the outline of his new container being formed. He could hear the alarm bells periodically tested. He started telling people what a great move it was to use the agile developer. He had to spend some time convincing his dad that it would be worth more than the $10,000 he’d originally agreed to, but because he could actually show some good results every two weeks, his dad agreed to another $5,000. After all, every day, he’d been able to walk inside this new container and imagine living in it in safety and comfort with his bride.

Iteration 4: The wires and bells team completed 105 meters during the last iteration, and felt they could continue to accelerate their pace for this iteration. So they set a goal of 110 meters, which would bring them to completion.

The container team had completed twelve arcs so far, and had planned to complete the remaining eight this iteration. However, as they began, the customer noticed a new requirement that he hadn’t noticed before. He needed a way to easily get into or out of the container. The agile developer took this problem in stride. It was exactly to address this sort of thing that agile development had been created. Everyone had known that the customer would probably get clearer about his requirements as he was able to more clearly see the solution every day.

So the team brainstormed, and came up with a few solutions. One was just to cut some arcs a little short of where they would come down to the circle, but nobody was sure they could properly fasten a beam that would hold them in place since they wouldn’t be fastened to the circle at the bottom. Another was to bend the arcs at one section so that they left an opening wide enough to walk through. A third suggestion was to dig a tunnel under the circle through which the customer could crawl into and out of the container; it was discarded as soon as someone pointed out that it would fill with water at the first rain. The best and cheapest option seemed to be leaving a gap around the outside by bending the arcs away from the opening.

A few big problems came up during this iteration. First, during one of their daily meetings, the team realized that the leaf-fastening team was going to have a problem. The arcs were parabolic, and fastened to a circle that was three meters wide. While this had created a serendipitous outcome of yielding a stable structure, it meant that the top of the structure was about four meters off the ground. The leaf-fasteners had been working from the base of the structure, and working up as high as they could get—about two meters. They would need to find a way to reach the higher parts of the arc.

Second, the wire and bells team were having another issue. Birds kept landing on the wires or the posts holding up the wires and ringing the bells. They’d been continuous testing the bells to make sure they kept working, but hadn’t foreseen the need to test against false alarms.

Third, just as the arc-fastening team was fastening the fourth of the last eight arcs, some things happened unexpectedly and suddenly. At the top of the structure, the arcs had been merely laid across each other at the center where they all crossed. They were too far off the ground to lash them together. Now, with sixteen saplings laying on top of each other, a few slipped sideways a little. With all the tension present in the bent saplings, and with the pulling apart of some of the arcs to make space for the opening, what had been a flat circle on the ground suddenly became a wobbly oval. Furthermore, the gap for the opening was folded in, and what had been planned to be a meter-wide opening was now about one-half meter wide.

The cost for this iteration was $2,000. Although no new unplanned costs came into play, the customer was a little alarmed to see that he wasn’t as close as he thought he was to the outcome he’d planned. It wasn’t making him feel much better that the agile developer seemed not at all upset by the setbacks. The agile developer reminded the customer that all projects encounter setbacks; what made agile better than traditional methods is that the customer and the agile team could more easily adapt and respond to these setbacks appropriately.

Iteration 5: With only a month to go before his wedding, the customer was getting nervous. The alarms bells were going off all the time, even when they shouldn’t, while the wires and bells team tried to figure out how to solve the false alarm problem. The top half of the container was still uncovered, and the only plausible way to cover it would be to roll the whole container onto its side, which presented an additional risk of bending the container even more than it was. And the container would probably never sit completely flat on the ground.

Iteration 6: The wires and bells team continually tried to make the case that they’d met their requirement, and that the new requirement to avoid false alarms would need new planning, more money, and probably a newer technology.

The leaf-fastening team came up with a clever way to fasten the leaves from the inside of the container by constructing a crude scaffold. They managed to complete their task, but everyone on the team noticed that the leaves were beginning to dry and would become brittle before long. While they would still keep out the sun, rain, and wind while brittle, they would clearly not last more than a few months. They said nothing, deciding to let the next team deal with the customer when he noticed that he would have a requirement for durability in the waterproofing.

The arc-building team, which was still the core team, worked long and hard (including weekends) to fasten lashings across the arcs inside the container. Eventually, the pulled the container back into more-or-less the shape it had been. The opening was covered with a tanned animal hide.

The customer was out of time. He realized had decided to postpone the requirement of a comfortable place to sleep. The lashings across the insides of the container made it difficult to walk around in, let alone lay down comfortably. When he presented the container to his bride, she was pleased, but not very impressed.

Iteration 8: The customer stopped the project. Although he could have stopped the project at iteration 6 and called it successful at meeting the schedule, he really was not happy with what he would have had to live with. He’d spent nearly $20,000 so far, and although it was technically true that the container fulfilled the requirements he’d set early in the project, he realized that he was still a long ways from getting what he’d dreamed of getting two months ago.

The agile developer maintained that the project had been successfully executed, and implored the customer not to give up too quickly. After all, they were still much further ahead than that other customer who’d hired the traditional developer and got a bird cage. They’d spent less, they’d done it in less time, and they were much closer to completing the solution that the customer really wanted.

The container did what it was supposed to. It protected them from predators by ringing bells when wolves or mountain lions brushed against the wires surrounding the container. It kept out the sun, the rain, the wind, and the cold. And it was possible to find a spot to lie down and get a comfortable night’s sleep on some animal furs and hay.

The customer was familiar with the basic problems. The leaves were starting to crack, and had to be replaced frequently. The alarm bells were routinely ignored; the customer and his new wife would take turns keeping watch at night instead. And the container was crisscrossed with lashings that had to be avoided and retightened periodically. Plus, it wobbled and rattled loudly when the wind blew even a little bit.

But it also had revealed many more problems that the customer would have to resolve some other day. For one thing, it was extremely dark inside, especially when the opening was covered. Fortunately, they thought of how flammable the leaves and saplings were before they tried to bring a fire inside to illuminate it. Even if they could have safely made a fire inside, there would be no place for the smoke to go.

After all this, the customer wished he’d known from the beginning that the container project really would cost him the $50,000 and six months that his brother had paid for his container. Even though he wouldn’t have been able to afford the $50,000, he would have felt better going to sleep in the cheap wooden body-burying boxes that would allow him to save his money for what he really wanted.

By Alan on 2010-01-29 07:10:02Z

As I was saying yesterday, most customers can not fully and vividly grasp the complexity of the software required to deliver even a rudimentary solution to their problems. Most customers understand this by now, and are willing to accommodate whatever path we recommend towards solving their problem. They don’t care too much whether we prefer traditional or agile methods. Proponents of agile methods tend to suggest that at least with their method, customers with the same money and the same initial vision wind up with their needs met most closely than with traditional methods, and I agree that seems more likely to be true.

On the other hand, in the real world, most customers need to be able to justify large expenses before they commit the money, not after. And no matter whether we start with agile or traditional methods, they won’t have a clear idea of the actual outcome by the time they need to make the decision whether to spend the money.

On an agile project, we could ask the customer to commit to only small steps at a time. For what will ultimately be a $100,000 piece of software, we ask for commitment to only $10,000 at a time. We’ll deliver exactly what we said we would with the first the $10,000, and the second $10,000. The customer will develop faith in our ability to produce what they want, and we repeat until we’re done, eight iterations later.

But before that customer agreed to pay the first $10,000, we will have had to explain that $10,000 wouldn’t really meet their needs, and that several more iterations would be needed. How many? We don’t really know. We guess, and we point out that it’s more likely to cost less and work better than traditional methods. How much? The customer really needs to know whether they are embarking on a $25,000 project or a $400,000 project.

Steve McConnell is an expert that every software developer should get to know. As he writes in his book, Software Estimation: Demystifying the Black Art (Best Practices (Microsoft)), “Many businesses value predictability more than development time, cost, or flexibility. Be sure you understand what your business values the most.”

Most businesses—for that matter, most people—want to know before they agree to commit any money to a project at all that they will get the value from the project that they need to get in order to make it worth the money. An agile process can’t guarantee that outcome from the beginning with any more certainty than a traditional process. It simply promises to deliver successive versions of something that the customer agrees is more valuable than previous versions.

I’ve compared building software to building actual buildings, but there’s a great big discrepancy in that metaphor. It’s this discrepancy that agile methods attempt to address, and it’s because of this discrepancy that no actual buildings are ever built using agile processes. Creating a building involves the aggregation of a number of tasks, each of which have already been done many times before by a wide variety of people. It’s well known how long it takes to get permits, draw up blueprints, hire subcontractors, prepare the land, lay the foundation, erect the frame, cover the roof, install wiring, ductwork, and plumbing, put in sheetrock, tiles, and ceiling panels, lighting, windows, lay carpet, and paint. Those tasks have to be done for virtually every building ever made. The tools used for one building are the same tools for all buildings. And nearly every part of the building is something that the customer can tangibly grasp.

In contrast, building software is frequently filled with tasks which the developers have, at best, only read about. A new development technology is not like an improvement on a screwdriver; better, but you still use it the same way when turning screws. It’s more like switching from a screwdriver to a rivet gun. And then from a rivet gun to a laser welder. And more often than not, we have to find a way to use the laser welder when an ordinary screwdriver might have worked better.

To complicate it, the relationships between components in software are not as obvious or well understood as the relationships between components in a building. That has to be worked out each and every time with each software project. And finally, most of what makes a software project complete is never directly seen or controlled by the customer.

If we reverse these comparisons—that is, if we had not been making building for thousands of years, but had been building software the way we do for that long, and were now trying to apply software development concepts into building construction—then buyers of houses and buildings would face a pretty crazy process. The customer would come to you and say they wanted a building that they could put their family and their stuff in, and they wanted to know how much it would cost.

The Traditional Process applied to People Containers

Customer: “I’m getting married in three months, and I need you to put together a container where I can protect my wife and myself from the wolves and mountain lions. But first, I’ll have to get money from my dad, so I need to know how much this will cost.”

Traditional Developer: “A container in three months? I need to know a few things first. How much time each day will you and your wife need to spend in the container? And how long does the container have to last?”

Customer: “Well, mostly we need it at night when we’re asleep and can’t be on guard for predators. How long? I don’t know. I guess a couple of years, when I will have money of my own and can get a better one built.”

Traditional Developer: “There a lot of options, and the costs range widely. For example, I could make a couple of small wooden boxes with padded lining for you to sleep in. They’re only about $2,500 a pop. Less if you use cheap wood, like pine, which you might be able to do since you only need it for 2 years. But some people get creeped out with those, because they’re basically the same thing they’ve been using when they put dead people in the ground. Or, we could go to the high end, and put together a large ball made out of cast iron with a watertight door, put in an airhose and a pneumatic pump, a mechanism that can raise and lower it every night into the lake. But that’s kinda pricey; it could cost about $1,500,000. I think that’s what Bill Gates uses. Maybe I’m thinking of Larry Ellison.”

Customer: “I can’t spend that much, but my wife will hate the wooden boxes. What can I get for about $10,000?”

Traditional Developer: “I think we can build you a platform that has been suspended from a large tree, and we’ll put a bunch of iron bars around the sides and top to keep out the mountain lions that might try to jump from the tree. You’ll have a rope ladder you can pull up when you don’t need it. Although, you should know that we’re trying a new thing with the iron bars. We’d been just tying them together with rope, but the rope tended to fray in a few months and the bars would come loose. So we’re planning to use a new technology to keep them together. I think it’s called “wedding.” No, it’s “welding.” They’re using it in those other projects – the ones that connect roads on both sides of a river. Bridges. We haven’t tried it with people containers yet, but I’m sure it will work great here.”

Customer: “And I can get that in three months for $10,000?”

Traditional Developer: “Um… yeah, that sounds do-able. Obviously, I’ll need to get more requirements from you before we really commit to the schedule.”

Customer: “That’s fine, I just needed to know how much I needed to ask from my Dad. I’ll call you back when I have the money.”

 

Out of time for today. I’ll be back tomorrow with the same conversation, only this time with the Agile People Container Developer.

By Alan on 2010-01-28 07:16:39Z

I’m not a big fan of meetings usually, but there are some times when they’re absolutely essential. Unfortunately, I’ll be the only one attending this project kick-off meeting, so I get to have my cake and eat it too.

The agenda for this meeting is fairly simple. The goal I have today is to set the next goal.

If I were following a traditional large project model (waterfall, RUP, MSF), I’d probably set the next goal so that it was as far away as I could reasonably make it. I would have whoever is going to pay for this project describe their aspirations, and how quickly they wanted it, and how much resources they would devote to get it. I’d want them to be fairly clear about those aspirations, because ultimately, I’ll be attempting to guess (only I’ll call it “estimating”) how much work it will take to deliver a solution in the time they want it, and whether the available resources are enough. Usually, this means telling the customer that they will either have to scale back their ambition, or give it more time, or spend more on resources, none of which makes anyone happy. I’ll need a lot of experience at doing similar work if I want to be any good at all at guessing the time frame and work required.

If were following something more like an agile project model, I’d probably meet with whoever is going to pay for this project, have them describe their aspirations, the time frame by which they’d like to see what they might call a finished product, and find out how many resources would be available. I would not spend much time getting clear on the goals right now; rather, I will trust that the customer will be able to decide when they’ve been met. I’ll probably give some feedback about my guess for the success we might have at meeting the expectations in time with the available resources. But my main goal will be to make sure the customer understands that we’ll take incremental steps towards that goal, without looking much beyond the immediate step we’re on. I want to make sure the customer understands that our ability to predict project outcomes is extremely limited early in the project, but will get progressively better.

Probably the biggest problem with the first approach is that my experience has taught me that over the lifetime of the project the requirements will grow. the resources will turn out to be not as valuable as I’d hoped, but the schedule won’t change to accommodate those problems. I’ll share my observations with the customer; some will be okay with that and some won’t. To counter it, what I usually try to do is estimate on the very high side, just to see how much tolerance the customer has for finding out the project will cost more or take longer or deliver less than they wanted. Interestingly, those very high estimates usually turn out to be close to what happens. (Perhaps I should be estimating even higher?)

This is the sort of quandary that agile methodology seeks to avoid. By avoiding committing to a fully worked out plan early on, I’m pushing the management of expectations back onto the customer.

I think software development processes has similarities to building construction. I’m the general contractor, responsible for delivering a building which is only in someone’s imagination right now. Let’s say we’re building a house.

If I followed traditional process models, then I’d sit with the customer, maybe bring an architect (or the architect could bring me to sit with the customer), and we’d work out how much the customer was willing to spend, what kinds of features and qualities they wanted in the house, and how quickly they wanted it. In houses, I would probably tell the customer when I could deliver it. In software, the customer usually has a quarterly or annual goal to make and will know how quickly they must have it. While the first meeting would not accomplish much, it would only take a few meetings to gel an idea of the scope of the house and the budget. Ultimately, I’m going to give an estimate to the customer for the cost of the house and the time it will take to build it.

If I followed an agile process model, then one of the first things I’d do with the customer is to make it clear that I won’t be giving much of an estimate. Rather, I’ll lay out the plan to just start building a house, a chunk of work at a time. When the customer decides that enough time and money has been spent on building the house, and that it’s complete enough for their satisfaction, then we’ll end the project. I can assuage the feelings of uncertainty the customer will undoubtedly have by referring to previous successful projects and having previous customers describe how I solidly met my intermediate goals.

Software development has a lot that is different from building construction, however, and this is why agile methods work better than my brief descriptions might suggest.

Most people can vividly imagine a house that hasn’t been built yet. We’ve all lived in at least one, we’ve all been inside dozens if not hundreds. And, as houses go, the overall patterns in the architecture are classifiable, we have countless reference works from which to pull ideas, and any new house is probably not going to have some extraordinarily novel approach to solving a fundamental need that houses fulfill. More importantly, we’re all experienced enough with houses that we can, without expert guidance, guess that some outlandish feature of a house could be prohibitively expensive or not work as well as we’d like. For example, suppose that I wanted to put my house on a turntable so that it could face east in the morning and west at night. Without consulting anyone, I can guess that it will be extremely expensive, and may wind up causing more problems than it solves.

In contrast, few people can vividly imagine software that hasn’t been built yet. While some aspects of software are shared by most applications, most people have only experienced the user interface part and have real grasp of what happens under the hood, or what it might take to put all that stuff under the hood in the first place. So how can we help customers understand all this?

I don’t know yet. Unfortunately, I’m out of time. So, with the goal of this meeting unmet, I’ll have to adjourn and return to this tomorrow.

By Alan on 2010-01-27 07:20:22Z

Now, I will describe The Work.

Project Code Names

For a long time (longer than I care to admit), I’ve been toying with a project idea that, for lack of a better project name, I called “Sumatra.” (I like code names more than final product names because, although I like to imagine that I’m a great marketer, I realize that I may not actually be one.) Sumatra works as a code name for me because I have so few other projects. But now that I’ll be discussing it more openly, I’ve decided to rename it. Sumatra is now Project Cheeky Moose, thanks so some suggestions from a random project name generator.

I created a CodePlex location for this project. It’s at http://cheekymoose.codeplex.com/. I’m not very familiar with CodePlex; I hope it works well for what I’m going to try to do. Did I pick it because it’s run by Microsoft? Pretty much, yes. I don’t have, at this early stage, many requirements for what I’ll need for sharing code, documentation, and so on. I know I needed it to support SVN, and that the licensing model wasn’t onerous. CodePlex will work fine for now.

About Cheeky Moose

The very short description of Cheeky Moose (hereinafter “CM”), aside from being a vehicle to showcase technologies and techniques, is to connect real people across various networks and technologies.

Well, an iPhone does that, and I’m clearly not making an iPhone. So a little more detail is called for.

Cheeky Moose works a little like a big brother. Yep, I know that has all kinds of negative overtones, thank you very much George Orwell. That’s why this project makes for a terrible business idea. The privacy implications and potential litigation opportunities just stink. Nevertheless, creating this kind of application gives me all sorts of ways to hook into lots of different technologies, and, if it does what I think it will do, actually would be very useful in the way a real-life big brother could—or should—be.

So what does it actually do? Cheeky Moose does its best to track your actual physical self, and what you are spending your time on while connected to the Internet, and it can report to you or whomever you select what you are doing and what you have done in ways you totally control. It tries to manage your online identity and presence automatically, and reduce the number of separate identities that you need to manage independently. It helps you stay connected to other real-world people as much or as little as you desire by using information they have knowingly shared with you about their online identities. Finally, at some point, it will help you quickly find people you haven’t met who would be willing to help you solve a problem or take some action for a cause.

Is all this useful? I don’t know. As I mentioned in a previous post, this is very likely a solution in search of a problem, although I can assure you I conceived this idea out of frustration from

• trying to manage my time by keeping track of where it has gone,
• trying to automatically let other people know whether it’s okay to bug me right now with an instant message, text message, or phone call,
• trying to let people know that I usually prefer email over phone calls for business communication, and phone calls over email for casual social communication,
• trying to schedule things with people who live in different time zones or countries, respecting their preferred hours or recognized holidays,
• trying to wrangle a couple of hundred different accounts on the Internet,
• trying to remember who I have met on the Internet (wondering if I know them anywhere else), and
• trying to find someone who could and was willing to help me with small problems or projects.

I suppose each issue could have its own solution, and indeed I will build Cheeky Moose so that each need is addressed separately, but the more I think about solutions for any one of these needs, the more I realize that the single solution that leaps to mind is a very deep and personal form of a personal assistant. But, I want this “virtual assistant” to be relatable to me more like a protector and better able to advise me than some hired flunky. I’m trusting it with a lot of very personal information! It had better live up to the trust I put in it.

That said, there will certainly be a limit on how much I actually do share using Cheeky Moose. I know that every system is potentially vulnerable, and I have no wish to unnecessarily increase the risk of having my identity stolen, or ruining my friendships, and so on.

Tomorrow: project kick off

By Alan on 2010-01-26 07:13:17Z

The Work

Normally, for a software project, I would begin by trying to establish an understanding of what value the final solution represents. What need is it going to fulfill? Why is it needed at all? How will we know that it was worth the time it took to build it?

Indeed, I will do this exercise for this work, but I won’t be as skeptical of its estimated value as I might be normally. I will be guilty of creating a solution whether it really solves a problem or not. That is usually not a great way to design a solution.

However, this is a learning opportunity and a teaching opportunity. I hope that the solution I’ll design will have enough similarity to truly valuable solutions that the technologies and techniques I use will be transferable to real world solutions. Partly, this is so that I will have developed some skills that I can use in later projects myself, and partly it is to show you one way to use these technologies and techniques in the full context of the development of a complete solution.

So much for the disclaimer. Now, what technologies and techniques are these? The list is pretty ambitious. I may have to skip some eventually, but let’s cross that bridge when we get to it. Mostly, as I said before, these will be Microsoft-oriented, but I will venture into non-Microsoft waters occasionally.

Products

In no particular order:

• Microsoft .NET Framework
• Windows Communication Foundation (WCF)
• Windows Workflow Foundation (WF)
• Windows Presentation Foundation (WPF)
• Windows Identity Foundation (WIF)
• Amazon Web Services (AWS)
• Google App Engine
• SharePoint
• BizTalk
• SQL Server
• Windows Azure
• SQL Azure
• C# programming language
• F# programming language
• Visual Basic programming language
• JScript programming language
• Java programming language
• .NET Regular Expressions
• XHTML
• CSS
• PowerShell
• AJAX
• ASP.NET
• ASP.NET MVC Framework
• Windows Forms
• Silverlight
• LINQ
• ADO.NET Entity Framework
• WCF Data Services
• Code name “M” programming language
• SQL Server Modeling Services (formerly code name “Oslo”)
• Windows Server AppFabric (formerly code name “Dublin” and “Velocity”)
• MSBuild
• Managed Extensibility Framework
• Windows Live Messenger
• Microsoft Office SDK
• Windows Mobile
• NMock
• Rhino Mocks

Techniques

Again, no particular order:

• Test-driven design
• Well-known patterns
• Iterative development
• Service-oriented architecture
• Continuous Integration

Features and Qualities

• Secure
• Separation of concerns
• Loose coupling
• Accessible
• Extensible
• Interoperable
• Testable
• Maintainable
• Manageable
• Reliable
• Responsive
• Stable

I think that will be enough for now (only mild sarcasm intended).

Tomorrow, I’ll cover exactly what the work will do.

By Alan on 2010-01-25 06:57:03Z

The Worker

Once upon a time, I fancied that I would get a certification. Specifically, the Microsoft Certified Solution Developer (MCSD) certification. (Those are now outdated.) In the process of studying, I learned about the Microsoft Solutions Framework (MSF). Oddly, I have not yet come across a developer or team utilizing it. However, I like to refer to it because I really like to have a shared understanding of common terms and ideas. Although we all know about patterns, I avoid trying to call things patterns because I think the term “pattern” is overused.

MSF describes five phases of the software development lifecycle: Envisioning, Planning, Building, Stabilizing, and Deploying. We are at the Envisioning phase right now. The goal now is to try to reach an agreement on what will be accomplished by this effort. Normally, you would do this with a team of stakeholders. As I’m the only stakeholder, I’ll just to apply decisions somewhat arbitrarily.

What I’m going to be looking for is a description of what function the solution will perform, how it is valuable (and how will I know that it is), what general strategy it will employ. In short, how will I know whether the final solution is what I want. Basically, I just need to describe in fairly specific terms what outcome I want, without describing too much how I think it should be made. I can talk about the overall cost constraints (or, in my case, the time constraints), or the political realities (that I’ll generally favor Microsoft technologies), or the goals I have in non-functional requirements.

Because I’m journaling all this and because I intend to use this project to learn new technologies as well as teach, my goals specifically do include using some technologies without knowing exactly how they are essential to the final solution. I do not recommend including any specific technologies in the Envisioning phase. Those are decisions best made in the Planning phase. For my part on this project, I won’t consider the project a failure if I wind up deciding not to use some technology I thought I would earlier.

Tomorrow, I’ll cover my vision of The Work, and lay out some of the technologies I would like to include. I’ll also, for completeness’s sake, lay out some of the methodologies and development patterns I would like to use while building The Work.

 

By Alan on 2010-01-25 06:30:11Z

One of the things I’d really like to reveal with this journal is the full perspective of the software development process, all the way from the conception of the idea, to the final delivery of a product that I could feel good about handing over to someone else. (Like many artists, I have some difficulty ever calling software totally finished.)

One perspective is to consider the technologies I use. Why do I select one technology over another? How do I use it? Why do I use it and not a different method requiring a different technology altogether?

I’ll let you know right away that I will be using Microsoft technologies primarily. This is because I’ve spent many years getting to know and use these technologies, not because they always make the best choice. As much as I’m able, I’ll contrast the Microsoft technology with other technologies, and I’ll attempt to design the system in such a way that you could, if you were building a similar system, substitute a different technology.

My experience suggests that technologies evolve more rapidly than our practical design and usage. I may choose to use technology X in a part of the system now, because it seems to be the right choice, but as I build my system, the technology will reach a new version which has been extended to include new functionality, or it will be in a direction slightly different than I anticipated. I don’t want to completely redesign my system, and I don’t want to let it permanently depend on old version of the technology. So, I try to isolate the application of the technology from the dependency my system has on it.

Another perspective is to consider the development techniques I use. How do I plan my time? How do I set goals? How would I delegate work to a team or to subcontractors? How much documentation do I do? What do I document? What do I do with my documents? How long do I spend making documents versus writing code? How much testing do I do? Do I do test-driven design, or model-driven design, or something else? How much consideration do I give to the intangible, non-functional requirements like extensibility, scalability, and reusability? What patterns will I apply and reject over different aspects of creating this system?

For the moment, those two perspectives are enough. They’re pretty broad! I could say that from one perspective, I’m highlighting the work in progress, and from the other perspective I’m illuminating the worker. For brevity’s sake, I’ll begin to label these perspectives as “The Work” and “The Worker.” But keep in mind that these are merely two different perspectives of the same thing.

If metaphors help (and they do for me), I’ll give you one. Imagine that I’m setting out to build a house.When I label something as “The Work,” you could think of it like I’m describing the plans and construction of the house itself. How many stories will it have? Is it brick or wood? How large are the windows? Hardwood floors or carpets? What’s the floor plan? Who makes the sinks and faucets, and how do I put them in? How do I make it so that I don’t have to rip open ceilings and walls to add a ceiling fan with two switches in the wall where there used to be only a single light and switch?

And when I label something as “The Worker,” you could think of it like I’m describing the skills and techniques needed by the architect of the house, and the general contractor who is hired to direct the team that will build the house, and the individual concrete pourers, carpenters, electricians, plumbers, tile layers, and so on.

Summary:

• Sections denoted with “The Work” are focused on the tools and technologies I apply to creating the software itself.
• Sections denoted with “The Worker” are focused on the skills, techniques and methods I use in creating the software.

By Alan on 2010-01-25 05:48:08Z

I’m not sure where this will go, but I know how it will begin.

Inspired both by the movie Julie & Julia, and, oddly enough, Conan O’Brien’s admonition to not be cynical, I have committed finally to a project that has long been on my mind.

Every day, for two hours, I will work on a software project that has been a fancy of mine for years. I will journal all aspects of it here, with an eye towards revealing all of the things I learn, as well as sharing and teaching the things I know. I intend to write this as though it were to be the content of a book I could publish, and ultimately, I do hope that I can create a book from this.

Mostly, I want to put down what I have always found lacking in other books. A picture of good software development as seen from all perspectives. Managing the lifecycle. Choosing the right technologies. Applying appropriate patterns. And this is the most important part: Seeing mistakes and how they get detected and corrected.

I’ve read many books on software development. Not as many as some people I know, but quite a few. And rarely do I see the author correcting him/herself. I only get to see the finished product. I admit, I don’t know how useful that really would have been, but it’s always left me with the feeling that everybody else that I read knows everything and that I know next to nothing. Naturally, this has humbled me. And yet I know that I recognize good software, and I can even create good software with enough time (I’ll never win any speed awards).

So for all the future developers that, at the time they read this, believe that maybe I have something useful to impart, I want you to know that I will be showing you all the work, in full, mistakes and all. As much as I can, anyway. I don’t mean the typographical errors; hopefully, I have few of those. No, I mean the mistakes of thinking that I knew how to apply a pattern. Or that I knew how to apply a new technology. Or that the architecture I create is as effective as I hope it will be.

It will probably take me more time to create my project than it would ordinarily, as I plan to document my thoughts as I go, which will take considerably more time. I hope that the documentation will make it easier for me, as I go, to recall why I made certain decisions, to keep to the plan, and hopefully, even though I have no readers at the moment, allow for others to help me spot those mistakes as early in the project as possible.

So, what is this fantabulous project I’m going to work on that I’ve cherished for so many years? I have had, I’ll admit, the conceit that I could turn this project into a profitable venture. However, the more I consider it, the more I believe it would be doomed to failure. Not because the idea is bad or not as useful as I believe. It’s because in order for it to succeed, people will have to be very slightly different than they are, or else the people owning the business have to be much braver and much more prepared to deal with potential negative fallout than I am.

The short description of the project: The software will connect and keep connected real people to real people across all of the methods we have of being connected to people via technology.

Does it sound like a social network, like FaceBook or LinkedIn? It’s not. Sounds like Microsoft Communicator? It’s not. Sounds like Twitter? It’s not. What it does have in common with all those systems is that it can keep you connected with all of the real-world people you know and connect with over those systems. What it doesn’t have in common is that it also helps you locate people that are not in your pre-existing network but are willing to help you.

I thought, at first, that would be a great idea. At least for me it would. But there are all kinds of privacy implications of a tool like this. For one thing, it will need to be able to represent you in all of those systems you connect to. In other words, it will need your passwords. That’s a non-starter for a lot of people. A lot of people.

Furthermore, one of the main things it would do is share what you are up to with the people you trust to know what you are doing. (For those of you who tweet “I’m going to Puerto Vallarta for 2 wks,” don’t be surprised when your house gets robbed while you’re away.) While that’s great for the people you trust, there are also countless marketers who would pay dearly to know what you do, too, but not because they plan to rob you (although I guess some would argue that they actually do!). They want to know because they actually might have something to offer you that you would, if you knew about it at the right time, actually want to buy. It’s just that they never know when that will be, and they never know exactly who that is at any given moment.

And, if there ever are any plans to turn my project in to an actual business, it would have to be paid for, somehow. And whoever pays for it is going to be suspected of wanting to harvest all that incredibly valuable data for purposes which may collide with our desires. I suppose it could be funded entirely out of donations, like Wikipedia. And if that turns out to be the case, then it’s good that I fully document this system, the ultimate open-source project, and hope that it stays generally secure from attempts to hack it.

So, there you have it. Two hours a day, for as long as it takes. This could take months, or years. Depends a little on whether I’m as good as I’d like to believe I am. Mistakes will be revealed. Foolish misunderstandings of my part will be evident. And, hopefully, someone, somewhere, will get something useful from my journal. For my sake, I hope that I come across as someone who has something worthwhile to share skills that are valuable.

And so it begins, come what may.

By Alan on 2009-10-25 08:29:48Z

My cousin wrote me and a few of her friends, asking for advice on how to buy a computer.

… I'm putting some feelers out there to see what you all might recommend for what to include in on an AWESOME system. I would LOVE to get a Mac, but then we'd have to buy our programs all over again, like Photoshop and Lightroom as well as MS Office (does Mac have MS Office compatible?)

Boys, I need your help! Please shoot me off some advice about what makes a system AWESOME. We know we want a big screen and will need to include a printer in the purchase…. I want it to be awesome for editing my photography and a large memory is a plus too. Also please let me know anything about PC vs Mac. I have wanted to convert to Mac for a long while, but not sure about what it takes to work on a Mac in a PC majority world. Any advice or tidbits at all would be awesome on either PC or Mac.

This is what I wrote back:

Macs are good computers, but you tend to get less awesomeness for the money. Why? Because Apple keeps very tight control over the available software and peripherals to make sure everything works more or less the way it is supposed to. Macs are best for people who don't think of themselves as very technical (like being interested in the difference between CPU speed and bus speed). There is a way to run some or most Windows programs on Mac; I believe that includes Office. I can't really give you much advice over which Mac to pick to get if you go that way.

I favor Dell computers for PCs. After that, I think Lenovo makes good portable computers. I'm not as much a fan of Hewlett Packard; HP tends to put in their own additions on top of Microsoft Windows that, in my opinion, add only bells and whistles while making other features less stable. Dell is happy to give you a basic, working PC, and let Windows do whatever it wants.

When I select a computer, I try to get a balance between these factors:

1. Manufacture reliability / quality
2. Lower price
3. Higher CPU speed (with multiple cores and/or raw speed)
4. How much RAM (memory) is included
5. Higher bus speed
6. Available RAM expandability

It's important to get these items as good as you can because you will probably keep the computer for a while, and these are things that you can't really change over the life of the computer. You can add more RAM (see #4), but only up to the limit of #6, and usually running out of RAM becomes the main reason why older computers appear to run slower over time.

Once I have used those to narrow my selection down to a short list, I consider how much I want to spend on these things, in about this order:

1. Larger and/or faster hard disk
2. High quality, high resolution, and larger-sized monitor
3. Video display adapter
4. Printer/scanner
5. Optical disc player/burner
6. Speakers

Each of those things are very easily replaced or expanded after you have your computer. Very large hard disks are not that expensive, but you should be careful not to get a giant hard disk that is very slow. The hard disk can also make your computer appear sluggish over time, so it's important to get a moderately fast hard disk. However, like I said, you can replace hard disks, although replacing your main hard disk (the C: drive) can be a lot of work.

I tend to stay low-tech on things like keyboards and mice. My keyboard has a cord and is about 12 years old. My mouse doesn't, but it's not a blue-laser wonder mouse, either.

Don't be afraid to ask the tech guys at the stores lots of questions, and if they aren't explaining things to where you REALLY understand them, don't feel bad about asking to talk to someone else. it's YOUR decision, not theirs.

 

By Alan on 2009-10-09 03:06:41Z

I was working on a .NET 2.0 Web Application, and had attempted to add a ScriptManager control to a Master page from the Toolbox, from the ASP.NET AJAX Extensions (1.6.0something). Visual Studio 2005 refused to do it, giving me this error message in a dialog box:

Control cannot be created because Visual Studio cannot find the control's type in the control's assembly

I Binged that error message, and found something that was close: Rick Strahl had an error in his web.config file. However, I did not have an error in my web.config file.

What I did have, however, was a <system.web>/<compilation> section that used the configSource attribute to use a separate .config file for that section. I find it helpful to have a separate .config file for almost every section in system.web. Unfortunately, Visual Studio doesn't care for me doing that. Once I brought all the child elements of the compilation element back into the web.config file, I could easily add the ScriptManager to my Master page. I'm not sure why -- I had already manually added the assembly to the <compilation> section, so VS didn't need to add it again, nor why the error message is as cryptic as it is considering the root cause.