Copland (operating system)

Copland was a project at Apple Computer to create an updated version of the Macintosh operating system. It was to have introduced protected memory, preemptive multitasking and a number of new underlying operating system features, yet still be compatible with existing Mac software. Development began in 1994 and was underway in earnest by 1995, when the system started to be referred to as System 8 or Mac OS 8.

During development the system accrued new features and the completion date continued to slip into the future. In 1996, Apple's newest CEO, Gil Amelio, hired Ellen Hancock away from National Semiconductor and put her in charge of engineering in an effort to try to get development back on track. Instead, she decided it was best to cancel the project outright and try to find a suitable third-party system to replace it. Development officially ended in August 1996, and after a short search they announced that Apple was buying NeXT in order to use their NeXTSTEP operating system as the basis of a new Mac OS.

Hancock also suggested that Apple should work on improving the existing System 7 while the new system matured. This process would eventually lead to the release of an unrelated operating system using the name Mac OS 8. The new operating system based on NeXTSTEP shipped in 2001 as Mac OS X.

Background

Pink

In March 1987,^[1] technical middle managers at Apple held an offsite meeting to plan the future course of Mac OS development.^[2] Ideas were written on index cards; features that seemed simple enough to implement in the short term (like adding color to the user interface) were written on blue cards, longer-term goals like multitasking on pink cards, and "far out" ideas like an object-oriented file system on red cards.^[3][4] Development of the ideas contained on the blue and pink cards was to proceed in parallel, and at first the two projects were known simply as "blue" and "pink".^[5] Apple intended to have the "blue" team (which came to call themselves the "Blue Meanies" after characters in Yellow Submarine)^[6] release an updated version of the existing Macintosh operating system in the 1990–1991 timeframe, and the "pink" team to release an entirely new OS around 1993.

The "blue" team delivered what became known as System 7 on May 13, 1991, but the "pink" team suffered from second-system effect and continued to slip its release into the indefinite future. Some of the reason for this can be traced to problems that would become widespread at Apple as time went on; as "pink" became delayed, engineers on the project jumped ship to work on "blue" instead.^[7] This left the "pink" team constantly struggling for staffing and suffering from the problems associated with high employee turnover. Management ignored these sorts of technical development issues, leading to continual problems delivering working products.

At the same time, the recently-released NeXTSTEP was generating intense interest in the developer world. Features that were originally part of "red" were folded into "pink" and the red project (also known as "Raptor")^[8] was eventually cancelled. This problem was also common at Apple during this period; in order to chase the next big thing, managers would allow new features to be added to their projects with little oversight, leading to enormous problems with feature creep. In the case of "pink", development eventually slowed to the point that the project appeared moribund.

Taligent

On April 12, 1991, Apple CEO John Sculley demoed Pink running on an PS/2 Model 70 to a delegation from IBM in a top-secret mission. The system was not fully functional, but looked a lot like System 7 running on a PC. IBM proved extremely interested and over the next months the two companies formed an alliance to further development of the system. These efforts became public in early 1992, under the new name "Taligent".^[9] At the time, Sculley summed up his concerns with Apple's own ability to ship Pink when he stated "We want to be a major player in the computer industry, not a niche player. The only way to do that is to work with another major player."^[10]

Infighting at the new joint company was legendary and the problems with Pink within Apple soon appeared to be minor in comparison.^[11] Apple employees made T-shirts graphically displaying their prediction that the result would be an IBM-only project,^[12] a prediction that came true on December 19, 1995 when Apple officially pulled out of the project.^[13] IBM continued working with Taligent, and eventually released its application development portions under the new name "CommonPoint". This saw little interest and the project disappeared from IBM's catalogs within months.

Business as usual

By modern standards, the original Mac OS was not an operating system at all, but a collection of application support libraries. There was nothing equating to a kernel that was responsible for mediating application access to limited resources like the CPU or hard drive. Instead, applications were placed in control of the entire system, using the libraries to handle common chores. Originally intended to support a single user running a single application at a time on a non-networked monochrome machine with a single floppy disk drive for storage and printing to a dot matrix printer, placing the applications in control of the system made sense, and allowed the developers to improve performance compared to a system with a kernel.^[14] The introduction of MultiFinder in 1987 greatly confused matters, placing the current application in control of not only the system, but other applications as well.

Further complicating matters was the way that new functionality was added to the system through the early 1990s. As many of these were optional expansions, they were generally added through a patching mechanism known as CDEVs and INITs. Third party developers also made use of this mechanism to add features like screensavers and a hierarchal Apple menu, independently of Apple. Some of these third-party control panels became almost universal, like the popular After Dark screensaver package.^[15]

This patching mechanism had no central control, so it was not at all difficult for two extensions to interfere with each other. As they were operating at the level of the underlying OS (equivalent to a kernel patch on a modern system), these often resulted in system crashes. These sorts of problems would normally be protected against through the use of protected memory of some sort. However, as the original model of the Mac was a single-user system, the developers had made a fateful choice to share certain important bits of data between the applications and the system libraries. This was widely used within QuickDraw for instance, which allowed the system to have reasonable performance in spite of the limited hardware it ran on. This made the addition of a protected memory system very difficult - not impossible, but at the price of greatly reduced performance.

While Taligent collapsed, system development had continued on in a business as usual fashion. During the early 1990s Apple released a series of major new packages to the system; among them were QuickDraw GX, OpenTransport, OpenDoc, PowerTalk and many others. Some of these were larger than the operating system itself. Problems with stability that had existed even with small patches grew along with the size and requirements of these packages, and by the mid-1990s the Mac had a reputation for instability and constant crashing.^[14]

Another try

The problems with the underlying system were well known within Apple, but as long as Taligent was still a going concern, it was difficult to gain any traction. Several new projects started during this time, notably the Star Trek project which reached demo quality. Invariably these projects were cancelled for one reason or another. However, while Apple was struggling to modernize their system, others were not suffering from the same problems. By 1994 the press buzz surrounding the upcoming release of Windows 95 started to grow to a crescendo, often with questions about Apple's response to the challenge it presented.^[4] The press turned on the company, often introducing Apple's new projects as failures in the making.

Given this pressure, the collapse of Taligent, and the growing problems with the existing operating system, after System 7.5 was released in autumn 1994, Apple management decided that the decade-old operating system had run its course. A new system addressing these problems was needed, and soon. In order to regain the trust of the press and the industry as a whole, the project simply had to ship.

Since so much of the existing system would be difficult to re-write, Apple developed a two-stage approach to the problem. In the first stage, the existing system would be moved on top of a new kernel-based OS with built-in support for multitasking and protected memory. The existing libraries, like QuickDraw, would take too long to be re-written for the new system and would not be converted, so user-facing programs would not be able to take advantage of these features. Once the new kernel in place and this basic upgrade was released, development would move on to re-writing the older libraries into new forms that could run directly over the new kernel.^[16][17] At that point, applications would gain the modern features that were available from Microsoft in Windows NT since 1993.

As System 7.5 was code-named "Mozart", the next-generation operating system that was intended to address the looming architectural issues was dubbed "Copland" after composer Aaron Copland. The follow-on system, "Gershwin", would complete the process of moving the entire system to the new platform.

Design

Copland was designed to run the Mac OS on top of a microkernel named Nukernel, which would handle basic tasks such as application startup and memory management, leaving all other tasks to a series of semi-special programs known as servers. For instance, networking and file services would not be provided by the kernel itself, but by servers which would be sent requests though interapplication communications.^[18] Copland consisted of the combination of Nukernel, various servers, and a suite of application support libraries to provide implementations of the well-known Macintosh programming interface.^[14]

Application services would be offered through a single program known officially as the "Cooperative Macintosh Toolbox environment",^[14] but universally referred to as the blue box. The blue box encapsulated an existing System 7 operating system inside a single process and address space. Mac programs would run inside the blue box much as they did under System 7, as 'co-operative tasks' that used the non-re-entrant Toolbox calls. A worst-case scenario was that an application in the blue box would crash, taking down the entire box with it. This would not result in the system as a whole going down, however, and the blue box could be restarted.

Copland runtime architecture. The purple boxes show threads of control, while the heavy lines show different memory partitions. In the upper left is the blue box, running a number of System 7 applications (blue) and the toolbox code supporting them (green). Two “headless” applications are also running in their own spaces, providing file and web services. At the bottom are the OS servers, running in the same memory space as the kernel, indicating co-location.

New applications, those written with Copland in mind, would be able to directly communicate with the system servers and thereby gain many advantages in terms of performance and scalability. They could also communicate with the kernel to “spin off” separate applications or threads, which would run as separate processes in protected memory, as in most modern operating systems. However, these separate applications could not use non-re-entrant calls like QuickDraw, and thus could have no user interface. Apple suggested that larger programs could place their user interface in a normal Macintosh application, which would then start "worker threads" externally.^[14]

Another key feature of Copland was that it would be completely PowerPC "native". System 7 had been ported to the PowerPC (PPC) with great success; large portions of the system ran in PPC code, including both high-level functionality, such as the majority of the user interface "toolbox" managers, and low-level functionality, such as interrupt management. However there was enough 68k code left in the system, and especially user applications, that the operating system had to map some data between the two environments. In particular, every call into the Mac OS required a mapping between the 68k's interrupt system and the PPCs. Removing these mappings would greatly improve general system performance; at WWDC'96, engineers claimed that the performance of system calls would be as much as 50% faster.^[19]

Copland was also based on the newly-defined Common Hardware Reference Platform, or CHRP, which standardized the Mac hardware to the point where it could be built by different companies and run other operating systems (Solaris and AIX were two of many mentioned). This was a common theme at the time; many companies were forming groups to define standardized platforms to offer an alternative to the "Wintel" platform that was rapidly becoming dominant - examples included 88open, ACE and the AIM alliance.^[19]

The challenge in Copland would be getting all of this functionality to fit into an ordinary Mac. System 7.5 already used up about 2.5 megabytes (MB) of RAM, and at the time this was a significant portion of the total RAM in most machines. Copland would be running what was essentially a complete copy of System 7.5 (in the blue box) and an entirely separate operating system running “under” it as well. Copland therefore was to use a Mach-inspired memory management system and rely extensively upon shared libraries, with the goal being for Copland to be only some 50% larger than 7.5.

Development

The Copland project was first announced in March 1995.^[20] Parts of Copland, most notably an early version of the new file system, were demonstrated at Apple's Worldwide Developers Conference in May 1995. Apple also promised that a beta release of Copland would be ready by the end of the year, for full release in early 1996.^[20][21] Gershwin would follow the next year.^[22] Throughout the year, Apple released a number of mock-ups to various magazines showing what the new system would look like, and commented continually that the company was fully committed to this project. By the end of the year, however, the developer release was nowhere in sight.^[21]

Copland's "open file" dialog box. Note the preview area on the right. The "stacked folders" area on the left was intended to provide a visual path to the current selection, but this was later abandoned as being too complex. The user is currently using a "favorite" location shortcut.

As had happened in the past during the development of Blue/Pink, developers within Apple soon started abandoning their own projects in order to work on the new system. Middle management and project leaders fought back; they would claim that their project was vital to the success of the system, and move it into the Copland development stream—that way it could not be canceled and their employees removed to work on Copland.^[23] This process took on momentum over the next year.

"Anytime they saw something sexy it had to go into the OS," said Jeffrey Tarter, publisher of the software industry newsletter Softletter. "There were little groups all over Apple doing fun things that had no earthly application to Apple's product line." What resulted was a vicious cycle: As the addition of features pushed back deadlines, Apple was compelled to promise still more functions to justify the costly delays. Moreover, this Sisyphean pattern persisted at a time when the company could scarcely afford to miss a step.^[20]

Soon the project looked less like a new operating system and more like a huge collection of new technologies; QuickDraw GX, SOM and OpenDoc became core components of the system, while completely unrelated technologies like a new file management dialog box (the "open dialog") and "themes" support appeared as well. The feature list grew much faster than the features could be completed, a classic case of creeping featuritis.^[20]

An industry executive noted that "The game is to cut it down to the three or four most compelling features as opposed to having hundreds of nice-to-haves, I'm not sure that's happening."^[24]

As the "package" grew, testing it became increasingly difficult and engineers were commenting as early as 1995 that Apple's announced 1996 release date was hopelessly optimistic: "There's no way in hell Copland ships next year. I just hope it ships in 1997."^[24]

Developer release

At WWDC '96, Apple's new CEO, Gil Amelio, used the keynote to talk almost exclusively about Copland, now known as System 8. He repeatedly stated that it was the only focus of Apple engineering and that it would ship to developers at the end of summer with a full release planned for late fall. Very few, if any, demos of the running system were shown at the conference. Instead, various pieces of the technology and user interface that would go into the package (such as a new file management dialog) were demonstrated. Little of the core system's technology was demonstrated and the new file system that had been shown a year earlier was absent.

There was one way to actually use the new operating system, by signing up for time in the developer labs. This did not go well:

"There was a hands-on demo of the current state of OS 8. There were tantalizing glimpses of the goodies to come, but the overall experience was awful. It does not yet support text editing, so you couldn’t actually do anything except open and view documents (any dialog field that needed something typed into it was blank and dead). Also, it was incredibly fragile and crashed repeatedly, often corrupting system files on the disk in the process. The demo staff reformatted and rebuilt the hard disks at regular intervals. It was incredible that they even let us see the beast."^[25]

After a number of people at the show complained about the microkernel's lack of sophistication, notably the lack of symmetric multiprocessing — a feature that would be exceedingly difficult to add on to an already existing system within a few months — Amelio came back on stage and announced that they would be adding that to the feature list.

In August 1996, "Developer Release 0" was sent to a small number of selected partners.^[20] Far from demonstrating improved stability, it often crashed after doing nothing at all, and was completely unusable for development. In October, Apple moved the target delivery date to 'sometime', hinting that it might be 1997. One of the groups most surprised by the announcement was Apple's own hardware team, who had been waiting for Copland to allow the PowerPC to truly shine. Members of Apple's software QA team suggested, jokingly, that given current resources and the number of bugs in the system they could clear the program for shipping some time around 2030.

Cancellation

Later that summer, the situation was no better. Amelio complained that Copland was "just a collection of separate pieces, each being worked on by a different team ... that were expected to magically come together somehow."^[26] Hoping to salvage the situation, Amelio hired Ellen Hancock away from National Semiconductor to take over engineering and get Copland development back on track.^[27]

After a few months on the job, Hancock came to the conclusion that the situation was hopeless; given current development and engineering, she felt Copland would never ship. Instead, she suggested that the various user-facing technologies in Copland be rolled out in a series of staged releases, instead of a single big release. To address the aging infrastructure below these technologies, Hancock suggested looking outside the company for an entirely new operating system. Apple officially canceled Copland in August 1996.^[22] While the CD envelopes for the developer's release had been printed, the discs themselves had not been mastered.

After lengthy discussions with Be and rumors of a merger with Sun Microsystems, many were surprised at Apple's December 1996 announcement that they were purchasing NeXT and bringing Steve Jobs on in an advisory role.^[28] Amelio quipped that they "choose Plan A instead of Plan Be."^[29] The project to port OpenStep to the Macintosh platform was named Rhapsody and was eventually released as Mac OS X, which, prior to the Intel release of Mac OS X 10.4 (Tiger), also used the 'blue box' concept in the form of Classic to run applications written for older versions of Mac OS.

Following Hancock's plan, development of System 7.5 continued, with a number of technologies originally slated for Copland being incorporated into the base OS. Apple embarked on a buying campaign, acquiring the rights to various third-party system enhancements and integrating them into the OS. The Extensions Manager, hierarchical Apple menu, collapsing windows, the menu bar clock, sticky notes — all were developed outside of Apple. Stability and performance was improved by Mac OS 7.6, which dropped the "System" moniker.^[30] Eventually, many features developed for Copland, including the new Finder and support for themes (the default Platinum was the only theme included) were rolled into Mac OS 7.7, which was rebranded as Mac OS 8.

With the return of Jobs, this rebranding also allowed Apple to exploit a legal loophole to terminate third-party manufacturers' licenses to System 7 and effectively shut down the Macintosh clone market.^[31] Later, Mac OS 8.1 finally added the new filesystem and Mac OS 8.6 updated the nanokernel to handle preemptive tasks. Its interface was Multiprocessing Services 2.x and later, but there was still no process separation and the system still used cooperative multitasking between processes. Even a process that was Multiprocessing Services-aware still had a portion that ran in the blue box, a task that also ran all single-threaded programs (and the only task that could run 68k code).

A number of features originally seen in Copland demos, including its advanced Find command, built-in Internet browser, and support for video-conferencing, have reappeared in recent releases of Mac OS X as Spotlight, Safari, and iChat AV, respectively, although the implementation and user interface for each feature is completely different.

Hardware requirements

According to the documentation included in the developer release, Copland supported the following hardware configurations:^[32]

• NuBus-based Macintoshes: 6100/60, 6100/60AV (no AV functionality), 6100/66, 6100/66 AV (no AV functionality), 6100/66 DOS (no DOS functionality), 7100/66, 7100/66 AV (no AV functionality), 7100/80, 7100/80 AV (no AV functionality), 8100/80/ 8100/100/ 8100/100 AV (no AV functionality), 8100/110
• Nu-Bus-based Performas: 6110CD, 6112CD, 6115CD, 6117CD, 6118CD
• PCI-based Macintoshes: 7200/70, 7200/90, 7500/100, 8500/120, 9500/120, 9500/132
• Drives formatted with Drive Setup (other initialization software may work; if you have trouble, try reinitializing with Drive Setup 1.0.4 or later).
• For builds up to and including DR1, the installer is set to ensure you have System 7.5 or later on a hard disk 250MB or greater.
• Monitors connected to either built-in video or a card set to 256 colors (8-bit) or Thousands (16-bit).

See also

• Mac OS history

Notes

1. ^ Singh puts it in March 1988
2. ^ Linzmayer 2004, pg. 55
3. ^ Carlton, pg. 96-98
4. ^ ^{a b} Singh, pg. 2
5. ^ Carlton, pg. 167
6. ^ Carlton, pg. 169
7. ^ Carlton, pg. 99
8. ^ Singh 2007, pg. 4
9. ^ "'Pink' may get a pink slip", Business Week, 1993, pg. 40
10. ^ Linzmayer 2004, pg. 69
11. ^ Linzmayer 2004, pg. 70 & 230
12. ^ Gordon Thygeson, "Apple T-shirts: a yearbook of history at Apple computer", Pomo Pub, 1997, pg. 44-48
13. ^ Linzmayer 2004, pg. 81
14. ^ ^{a b c d e} Dierks, 1995
15. ^ Adam Engst, "After Dark Returns for Mac OS X", TidBITS, 9 June 2003
16. ^ Michael J. Miller (October 4, 1995). "Beyond Windows 95". PC Magazine. http://www.guidebookgallery.org/articles/beyondwindows95. Retrieved 2006-07-23. 
17. ^ Henry Bortman; Jeff Pittelkau (January 1997). "Plan Be". MacUser. http://www.bebox.nu/articles.php?s=articles/199701xx-PlanBe. Retrieved 2006-07-23. 
18. ^ Francis 1996, pg. 32
19. ^ ^{a b} Francis 1996, pg. 9
20. ^ ^{a b c d e} "Mac's new OS: Seven years in the making" cnet, 21 March 2001
21. ^ ^{a b} Crabb 1995
22. ^ ^{a b} "The Long and Winding Road", MacWorld, 1 September 2000
23. ^ Jake Widman, "Lessons Learned: IT's Biggest Project Failures", Computerworld, 9 Oct 2008
24. ^ ^{a b} Burrows, 1995
25. ^ "WWDC'96: Looking for the Future", MacTECH, Volume 12 Issue 9 (August 1996)
26. ^ Gil Amelio and William Simon, "On the Firing Line", Harper, 1998
27. ^ Carlton, pg. 402
28. ^ Dawn Kawamoto, Mike Yamamoto and Jeff Pelline, "Apple acquires Next, Jobs", cnet 20 December 1996
29. ^ Linzmayer 2004, pg. 277
30. ^ Singh 2007, pg. 6
31. ^ Beale, Steven (October 1997). "Mac OS 8 Ships with No License Deal". Macworld 14 (10): pp. 34–36 
32. ^ How to Install Mac OS 8 (D11E4), "Hardware Supported" section

References

• Peter Burrows, "Apple's Copland: NEW! IMPROVED! NOT HERE YET!",BusinessWeek, 18 December 1995
• Tim Dierks, "Copland: The Mac OS Moves Into the Future", develop, Issue 22 (June 1995)
• Steve Falkenburg, "Planning for Mac OS 8 Compatibility", develop, Issue 26 (June 1996)
• Geoff Duncan, "OS Directions: Marconi, Copland, and Gershwin", TidBITS, Issue 256 (12 December 1994)
• Jim Carlton, "Apple: the inside story of intrigue, egomania, and business blunders", Times Business/Random House, 1997, ISBN 0812928512
• Don Crabbe, "Copland, Where Are Thee?", MacTECH, Volume 11 Issue 12 (December 1995)
• Owen Linzmayer, "Apple Confidential 2.0: the definitive history of the world's most colorful company", No Starch Press, 2004
• Tony Francis, "Mac OS 8 Revealed", Addison-Wesley, 1996
• Amit Singh, "Mac OS X internals: a systems approach", Addison-Wesley, 2007

External links

• Apple's Copland project: An OS for the common man Article covering Copland development
• "A Time Machine trip to the mid-'90s", MacWorld article with screen snaps from Copland
• Apple Copland Reference Documentation