Budget Builder Analysis - Conclusions





Overview	BB Use	Power	Participation	Design	Methods	Next Steps

Conclusions

System Design Issues

A recent article about lessons learned in web development presents four pitfalls that developers of web-based systems should be aware of (Dennis 1998). Unfortunately, we learned nearly the same lessons they did, avoiding only one of the problem areas they outline. Their conclusions are listed as the four following headings, while the explanations are from the Budget Builder project.

Don't Chase New Technologies
We were originally misled by the hype and promise of using the new programming language *Java to develop the budget building module* of our site. It wasn't until several months into the project that we learned that Java was in a rudimentary state, without support for many common features such as scrolling and printing that were essential to the project. There were good reasons for trying to use Java, most notably that it would have allowed a more interactive version of the tool. However, pursuing a Java version of the site used up money that might have been better spent on other tasks, and slowed down development of the site by taking up meeting and work time on features that were never completed. Now that Java supports some of the earlier missing features, has matured as a programming language, and has more widespread support, we may want to revisit the decision to abandon Java for HTML.

Users Aren't Techies
This was the one pitfall we were aware of from the beginning and did a good job of avoiding. However, there are many ways in which this trap can disguise itself.

"Don't assume that users are technical" is a maxim in the human-computer interaction field, but it can be hard to always keep in mind, especially when focused on a myriad of technical details. As a designer, it is very easy to get caught up in the technical sophistication of a particular feature, such as linking the web site to databases or generating charts on-the-fly. The more difficult the feature was to implement, the more pride a designer takes in its success. However, the usefulness or usability of a particular feature is not necessarily linked to its technical complexity, and many times, the reverse is true. While designing, it is important to be aware of that distinction and hold the user's needs and abilities in the forefront.

From the beginning of the project, non-technical users were assumed to be the major users of the site, so we designed its features accordingly. We included navigation aids, online help and glossary information, and avenues for users to send us feedback. In addition, the Seattle Public Schools developed a training manual and offered training classes for their principals.

One important and often overlooked aspect of the technical design process concerns the hardware and software used by the designer. To develop the Internet site most easily, the designer wants to use the fastest computer hardware, speediest Internet connection, and latest software possible. This allows rapid creation of new versions and quick testing of changes. As a designer, it is very easy to get used to such a high level of capability. However, very few end users are likely to have equipment at the same level. The designer must constantly keep in mind the realities of the users and design accordingly.

In our case, we assumed that users would have slow connections to the Internet, older computers, small screens, and older versions of Web browsing software. While the university computer on which I did most of the work was relatively fast, and had a fast Internet connection, my home computer was substantially slower, had a smaller screen, and relied on a slow modem to connect to the Internet—precisely the conditions that we were designing for. Regularly testing the Budget Builder with my home computer helped to keep our wider audience in mind.

Training is Critical
One of the important shortcomings in this project was our failing to provide adequate training for many of the site's potential users. Because we agreed to focus our work on the principals' version of the Budget Builder, there was no training of people outside of the district. The public meetings we held were demonstrations more than training sessions. For the principals as well, training was minimal, although at a somewhat higher level. Jay Iman of the Seattle Public Schools developed a training manual and offered training classes for the principals. The classes were poorly attended, however, since the principals were extremely busy. It is likely that other methods for training principals will need to be tried, such as sending experienced users out to the schools for one-on-one consultations.

Another crucial piece of the training link was left out as well. The budget analysts in the Seattle Schools budget office were assigned the task of assisting the principals with the Budget Builder, as well as their normal task of assisting with budget development. However, the budget analysts received no training in the use of the Budget Builder and received less formal support than the principals. It was assumed that the since the analysts were familiar with the budgeting process and forms, and were computer literate, that they would have no problems with using the Budget Builder. While I didn't hear about any problems directly from the analysts, they never gave any feedback during the design process outside of development meetings.

Training is crucial, although fostering it was challenging in this situation, particularly since anyone with access to the Internet was a possible user. Since it is impossible to train all potential users of the Budget Builder, more effort needs to go into providing a clear and simple user interface.

In addition to the technical training, Cross City is planning to sponsor school-based budgeting training sessions for anyone in the community who wishes to learn more about the topic. The most important piece of this training will not involve direct use of the Budget Builder, but will instead be focused on school-based budgeting principles and grassroots activism. Use of the Budget Builder as one tool to achieve school-based budgeting goals will occur later.

It is certain that with its limited resources, Cross City will not be able to directly train every community member interested in using the Budget Builder. For that reason, training will likely take the form of individuals who are familiar with the Budget Builder training each other informally. That likelihood reinforces the necessity of improving the Budget Builder interface so that it can be used with minimal training and support.

Web Information Systems Are Not Just Collections of Web Pages
*My failure to recognize that the Budget Builder was a rather complicated software program*, rather than a collection of Web pages, was a primary reason that there was *no structured user testing of the site*. Although user testing for readability is always a good idea (Morkes and Nielsen, 1997; Nielsen 1997, 1998), a site editor with a good knowledge of technical communication principles can be reasonably confident that a site will be understandable with standard editing methods. However, "understanding requirements is more difficult for Web information systems than for traditional Web development because users interact with the system much more than with simple Web pages. Unlike Web page development, in which users play small roles, user participation is as critical to Web system development as it is for traditional system development." (Dennis 1998) Future improvements to the Budget Builder will be accompanied by more structured user testing, to give clearer guidance to the development process earlier on.

One reason why I didn't fully appreciate the systemic nature of the project was the Seattle Schools inability to articulate what their budgeting system was for many months. In many early design meetings, I asked, "What does a complete budget consist of?" and usually got different, overlapping answers. We decided at the end of 1996 that the Budget Builder wouldn't be ready to use for the Spring of 1997. That had two effects: First, the route we took for the Budget Builder involved a highly simplified view of school budgets. This continued the impression that the Budget Builder was a simpler project than it turned out to be. Second, because we weren't able to figure it out for them, the Seattle Schools budget office had to determine what a complete budget was made of. We acted as an outside catalyst for conversations within their organization that should have happened earlier. As Andy said, "If they hadn't had us, they would have had to invent us." They eventually produced a set of *budget forms and guidelines* (known affectionately as "The Green Book," after the color of its cover) that described everything a principal needed to develop a complete budget.

A result of more clearly defining the existing SPS budgeting system was that the longer we worked on the project, the more the Budget Builder grew to look like their internal systems. Although we started out working from a clean slate, the site got more and more complex over time—partly out of necessity, because it's a complicated system—but also because SPS didn't attempt to simplify their systems. In the second year, we tried to make the budgeting section of the Budget Builder become an electronic version of the Green Book. Eventually, it approached the Green Book's complexity enough that Chris Warden of Cross City said, "The Budget Builder has become a data entry tool for the Seattle Schools!" She recognized that by increasing the complexity, we had lost the ability to do simple, straightforward budgets that had existed in our earlier version. We ended up restoring the simplified version and allowed people to work on whichever version they choose: the simplified version for planning or the Green Book version for submitting budget office-approved budgets.