``We couldn't say anything,'' says Lynda, a senior technical writer who designed Improv's revolutionary hypertext documentation system. ``We stoically stood there and took it, [thinking] `Go ahead, I'll get the last laugh.' ''
In retrospect, there was no way that the people at the user group meeting could have known how powerful or popular Lotus' secret next-generation spreadsheet would be. They had no way of knowing that, less than a year later, businesses would be buying NeXT computers in order to run Improv. After all, they had never heard about Fluffy Bunny.
* * *
The idea that eventually grew into Improv started in 1986, when the Lotus Development Corporation took an empty library on the first floor of its building at 161 First Street, Cambridge, and set up the Advanced Technology Group.
ATG's charter was to come up with new ideas that Lotus might be able to turn into products. There were three projects and three researchers: One person was exploring a platform independent development environment. Another person was developing an artificial intelligence-based system that would automatically design new spreadsheets. And then there was ``modeler.''
One of the hardest things to do with a computerized spreadsheet like Lotus' 1-2-3 is to lay out the initial modeler. What should go in the rows? What should go in the columns? Fundamentally, even the most basic spreadsheet program will let the user put a number or a formula in any cell desired. It's a lot of power --- a lot more than you need for most applications.
Enter Pito Salas, a bright developer in the Advanced Technology Group. Pito's project was to look at a variety of complicated models that had been built with conventional spreadsheets and see what they had in common. Pito looked at financial statements, revenue forecasts, and future tax estimates, to name a few. He discovered that most spreadsheets have many patterns in them. If a spreadsheet program could be taught to understand those patterns, he realized, he could make it easier to build and use complicated models.
Within a few months, Pito had come up with the fundamental idea at the core of Improv: that the raw data in a spreadsheet, the way that the user views the data, and the formulas used to perform calculations can all be separated from each other. The formulas should be general, so that the user can type something like ``PROFIT=PRICE-COST'' and have the spreadsheet calculate every PROFIT cell from its corresponding PRICE and COST cells. The user should be able to rearrange the views to highlight the information and relations that she is trying to convey. And the data itself should be put into a multi-dimensional database. A slick interface should sit on top to make it easy to get information in and out.
By the end of the summer, Pito had a demonstration program working on an IBM/PC that showed his basic concepts. It was really more a slide show than a computer program, admits Pito: the math was all faked. There the project waited until February 1987, when Pito hired Glenn Edelson --- a hot programmer whose job description was to take Pito's idea for general formulas and see if they could be implemented in C++.
``All those [ideas] I had described with hand waving,'' recalls Pito. ``I had no clue how to build it. Glen came in and we started working.''
Things clicked: it soon became obvious that the technology for calculating general formulas could work, and that multi-dimensionality was a powerful way to model complicated financial problems. In the spring of 1987, Pito hired Bonnie Sullivan to write a specification for a project and Jeff Anderholm®MDNM¯ to see if there was really a market need for a ``high-end financial modeling application.''
That summer, Modeler went on tour.
The ``road show,'' as Pito called it, took the Modeler demo program to six different companies. At each stop they spent a day with a few analysts who did high-end financial modeling.
``We had a paradigm shift on that trip,'' recalls Pito. ``It was really cool... They told us we had something a lot hotter than we thought. We said that these ideas might be turned into a high-end financial modeling package of interest to perhaps 10 percent of your company, and they said `No, no, no! What you have is the next generation spreadsheet --- something that will be used by half of our company!' ''
Despite the enthusiastic comments, Lotus still took more than a year to decide to turn Pito's ideas into a product. ``You can be at the prototype stage where you understand that the concepts are viable, but you still require a huge amount of work to work out all of the various detailed cases,'' says Allen.
The decision was finally made in September 1988 to go ahead with the ``Back Bay'' project (named after a Boston neighborhood known for its townhouses, boutiques and fine restaurants). After experimenting with interfaces and a database engine under DOS and the Macintosh operating system, the group decided that the product would be based on OS/2 and Microsoft's Presentation Manager. They even picked a mascot --- Fluffy Bunny --- and started up an underground newsletter, ``Fluffy Bunny Goes to Back Bay.''
* * *
In October 1988, Steve Jobs came to Lotus to show off his new computer. After the talk, Lotus' top management did a private show-and-tell for Steve of their most interesting products that were under development.
Pito showed Steve a clunky, character-based, primitive spreadsheet, but all of the elements of the future were there: there were formulas at the bottom of the spreadsheet, rather than integrated in the cells; it was multi-dimensional; and the user could instantly call up different views of the same data set.
Immediately, Jobs wanted Back Bay for the NeXT.
``He kept getting more excited; he was the most excited person in the room,'' remembers Pito.
Back Bay fit right in with Steve's vision, says Allen: ````Right from the start, he was looking for something new... It might have been better financially for his company to get 1-2-3 [ported to the NeXT], but that would have compromised his vision.... [Back Bay] was attractive because it was a new kind of spreadsheet.''
Likewise, the Back Bay group was excited by the NeXT. But when Jobs went back to California, the Back Bay group continued working on the OS/2 product --- or at least they tried. Writing applications for the buggy initial releases of OS/2 and Presentation Manager was no picnic. And the allure of Steve's black cube kept creeping back into everyone's minds. Four months later, on February 7, 1989, Frank King left a Post-It note for Pito Sales: ®MDUL¯WHAT WAS ON IT?®MDNM¯
``I think that Jim Manzi really believed in us,'' recalls Steve Jobs. ``When the Improv team recommended to proceed on NeXT (after their failure on OS/2 and PM), Jim gave the green light.''
Another reason that Lotus decided to go with NeXT, says Jeff Anderholm®MDNM¯, was that the NeXT didn't run 1-2-3, Lotus' cash cow. ``We didn't have to worry about any [marketing] conflict with 1-2-3.''
Rather than try to port the code that they had written for Presentation Manager to the NeXT, the group resolved to rewrite the entire user interface from scratch, taking advantage of everything that the NeXT had to offer. ``If we had tried to write PM code on the NeXT, or NeXT calls in PM, we would have really limited what we could do with the product,'' explains Allen.
Says Pito: ``We decided at the beginning that we wanted to come up with a generally great NeXT product.''
Moving to the NeXT also gave the Back Bay group the freedom to rethink some of their basic ideas about the user interface. ``We knew that we had a lot of work to do in the UI area, and it seemed that it was quick to do [on the NeXT],'' recalls Allen.
A few days after the decision, Steve Jobs sent a huge bouquet of flowers to Cambridge.
``It was like he was wooing us,'' says Lynda. ``It must have been three feet tall!''
Perhaps more importantly, NeXT sent technical wizard Bruce Blumberg to Lotus, where he gave the Back Bay developers a special one-week class on NeXT application development.
Fluffy Bunny was off and running.
* * *
The program that evolved into Improv was an application program with an identity crisis. The core of the system was the original multi-dimensional database engine and the general formula evaluator written in C++. All of the code dealing with the user interface, however, was written in Objective C. Gluing them together were ordinary functions written in C. Nobody knew if two different object oriented languages could safely be mixed in a single application. The Back Bay group tried it, and it worked.
Indeed, one of the unexpected advantages of using different languages was that it segregated different aspects of the programs, with a well-defined interface between the two. That, in turn, made it easier to track down bugs.
The best part about developing a user interface with Interface Builder, many of Improv developers agree, wasn't the ability to design a single interface quickly: it was the ability to try out a lot of them. ``It let us develop a lot of bad UIs,'' says Pito. ``We could have an idea and in an hour we would have it on the screen.''
For example, an early version of Improv's ``item dispenser'' was simulated with Interface Builder and multiple windows. Other times, developers used the program merely as an electronic blackboard to sketch out ideas before writing the code to make the ideas work.
Perhaps the single most important and unique aspect of Improv's user interface is its ``category tiles,'' which the program uses to represent different dimensions of a multi-dimensional data set. A user can change the way that data is presented by simply clicking-and-dragging the category tiles around a view.
But in January 1989, Improv didn't have category tiles. Instead, all of the view rearrangement was done with menu commands. One command, called ``promote,'' rearranged the ordering of dimensions --- in much the same operation that moving a category tile once to the left does with today's Improv. Other operations, named ``demote'' and ``perspective,'' performed other transformations.
``It had all of these arbitrary restrictions,'' says Paul. Then the group hit upon the idea to use icons. ``We realized that if we represented these things as icons, all these manipulations could be represented by moving icons from one place to another.''
The category tiles also ``broke through a lot of barriers in understanding for new users,'' says Pito, by finally letting users visualize and directly manipulate the dimensions of the multi-dimensional database.
But where should the icons for the categories go? After trying a lot of different ideas, the developers decided to create a special icon window.
``And then Steve Jobs came,'' says Paul, remembering Job's visit in April 1989. ``I remember he said, `okay, who is the guy who came up with the picture?... It's very important that you be here.' ''
Jobs then said that the category manipulation had to be more direct. ``You have to be able to touch the categories and move them around. Having them off in a separate window is too removed,'' Paul remembers. ``He didn't even want to have the tiles; he wanted to just move them around. He's really a fanatic for direct manipulation, and it really shows.''
Jobs didn't have an answer, says Paul, but ``one of the benefits of that [meeting] was we junked the idea of the extra panel,'' and put the category tiles on the worksheet itself.
* * *
In a way, Steve Jobs himself became an icon on the Improv development landscape. ``He would come in periodically, tell us what he liked, and what he didn't like,'' remembers Paul. Sometimes Jobs would jump up and down and say ``It's ®MDUL¯got®MDNM¯ to be this way!''
``We didn't do everything he wanted, but he pushed us in the right direction a lot of the time,'' says Paul.
For example, in one issue of Fluffy Bunny goes to Back Bay, Fluffy Bunny is warned: ``sometimes formulas are friendly, but sometimes they are not Very Nice.''
Formula conflicts were the problem: Since Improv has general formulas, it's possible that a single cell can be specified by more than one formula. Suppose you have a spreadsheet charting the business operations of a bicycle shop for the past two years: you might have one formula that says PROFIT=PRICE-COST; a second might say that TOTAL=groupsum(YEARS). But how should the total profit be calculated--by subtracting the total price from the total cost, or by computing the sum of the profits for each year? Conflict.
The Back Bay group thought that the possibility for formula conflicts was a real weakness with the program. There seemed to be no way to automatically resolve conflicts. When Improv detected a conflict, it invalidated both formulas and complained to the user: sometimes, formulas were not very nice.
``Jobs said that `this is not a weakness, it is a great feature!' '' remembers Paul. ``With 1-2-3, you can copy and blow away a formula.'' The fact that Improv could automatically detect conflicts and tell the user about them was a major step forward. ``We sort of switched our threads, [and started] thinking of it as a positive feature.''
As the product matured, many commands held over from the OS/2 version were eliminated. One command was ``disable formula,'' which turned off evaluation of formulas. Another was ``compile formula'' which compiled formulas into the computer's native language. (Today's Improv compiles all formulas when they are typed.)
``It looks like a logical line from concept to birth, but it goes all over the place,'' says Pito. ``A whole bunch of stuff was left on the cutting-room floor.''
Meanwhile, the Help system and the Presentation Builder were developed as separate application programs. (The Help system was eventually integrated with Improv during the summer of 1990.) Forty people, including 13 developers, were working on the project. All of the pieces were beginning to fall into place.
* * *
Throughout the summer of 1990, as the Back Bay group struggled to get Improv working, NeXT struggled to get the NeXT 2.0 operating system completed. Originally the NeXT project was called ``Warp 9,'' because that was how fast the NeXT development team had to get it out. (``But Captain, the engines won't go faster than Warp 8!'') As the summer progressed, the team received a variety of pre-releases: Warp 2, Warp 3, Warp 3i, and finally Magenta. ``Not only was it a new versions: it had a new numbering scheme!'' exclaims Paul Kleppner.
Other changes --- and bugs introduced in intermediate releases --- were far more serious. One time a bug with the memory allocation scheme caused the malloc() function --- and all of Improv --- to run incredibly slow. Another time a bug with the Mail application prevented people from sending documents.
``We came close to backing out [of the update] a number of times,'' says Robby.
Throughout it all, NeXT was there, trying to help what was to become their star application: ``Bruce Blumberg supported us through the whole project,'' says Paul. ``He was there on the phone and frequently in the office.''
Then, a few weeks before the roll-out of the new NeXT machines, the NeXTStep ``Text'' object changed completely --- a real problem for Improv's Presentation Builder. ``We had spent a fair amount of time optimizing to the old one,'' says Allen. Days before the announcement, nobody knew if Improv's graphs would have text on them or not.
They did.
During the following months, the newly-named Improv group shipped out their beta-release of the program. The final version started shipping in February 1991.
Once again, Steve Jobs sent flowers to the developers. Only this time, it was truckloads:
``There was one bouquet that was five feet tall --- the size of a short doc writer,'' says Lynda. ``There was forsythia, lilacs, and other very out-of-season flowers.''
A dozen large bouquets came, and many small ones: one with just long stemmed roses, another with just birds of paradise. ``It seemed like they just kept coming in,'' says Robby.
* * *
What's next?
``Right now, we are all kind of relaxing,'' says Allen. ``Just trying to catch our break.'' Although Lotus might eventually port Improv to another platform, the company doesn't have any plans to do so before the end of 1991.
Foremost on the Improv group's agenda is listening to user's reactions. Lotus has set up a ``hotline you can call to drop a flame on the Improv development team,'' says Jeff, something that is very unusual for Lotus. The company has also set up a user newsletter and is conducting extensive interviews with users to find out what they like about the program and what they want changed.
``It's not the typical Lotus where we sit in our Ivory tower and run ads,'' he says. ``We need the feedback.''
Already, users have made it very clear that Improv needs macros and an ``undo'' capability. But more than that, says Jeff: ``We have to figure out what we can add to the product to help people learn it, because you have to unlearn what you know about conventional spreadsheets like Excel and 1-2-3. Our challenge is to convince people that the benefits of this new spreadsheet are worth the cost of switching.''
-30-