Macsyma

Macsyma is a computer algebra system that was originally developed from 1968 to 1982 at MIT as part of Project MAC and later marketed commercially. It was the first comprehensive symbolic mathematics system and one of the earliest knowledge based systems; many of its ideas were later adopted by Mathematica, Maple, and other systems.

Development

The project was initiated in July, 1968 by [http://acl.ldc.upenn.edu/J/J83/J83-3008.pdf Carl Engelman] ,
William A. Martin (front end, expression display, polynomial arithmetic)and
Joel Moses (simplifier, indefinite integration: heuristic/Risch). Bill Martin was in charge of the project until 1971, and Moses ran it for the next decade. Engelman and his staff left in 1969 to return to The MITRE Corporation.Later major contributors to the core mathematics engine were: Yannis Avgoustis (special functions),David Barton (algsys),Richard Bogen (special functions),Bill Dubuque (limits, Grobner, TriangSys, indefinite integration: Bronstein, power series, number theory, special functions, functional equations, pattern matching, sign queries),
Richard Fateman (rational functions, pattern matching, arbitrary precision floating-point),Michael Genesereth (comparison, knowledge database),Jeff Golden (simplifier, language, system),
R. W. Gosper (definite summation, special functions),Charles Karney (plotting),John Kulp,Ed Lafferty (ODE solution, special functions),Stavros Macrakis (real/imaginary parts, system),Barry Trager (algebraic integration, factoring, Grobner),
Paul Wang (polynomial factorization, limits, definite integration),David Y. Y. Yun,Gail Zacharias (Grobner),andRich Zippel (power series, polynomial factorization, number theory, combinatorics).

Macsyma was written in Maclisp, and was, in some cases, a key motivator for improving that dialect of Lisp in the areas of numerical computing, efficient compilation and language design. Maclisp itself ran primarily on PDP-6 and PDP-10 computers, but also on the Multics OS and on the Lisp Machine architectures. Macsyma was one of the largest, if not the largest, Lisp programs of the time.

Commercialization

In 1979, in response to a request from Richard Fateman, then a professor at UC Berkeley, MIT provided a temporary license for Macsyma code. This was almost immediately brought up on the new VAX-11/780 computer using a dialect of Maclisp, Franz Lisp, written by Fateman's group. MIT reluctantly licensed versions of VAX Macsyma from Berkeley to about 50 universities, starting with Caltech, under condition that the licenses would be revoked when negotiations for a proper license were complete. Indeed this happened when the negotiation with Symbolics (see below) was complete. Symbolics was reluctant to license the VAX product, since VAX constituted competitive hardware to their own Lisp Machines and therefore suppressed the VAX software for five years. UC Berkeley also brought up copies of Macsyma on Motorola 68000-based systems, most notably Sun workstations. Symbolics suppressed those as well. At the same time Fateman worked to change the (now revoked) temporary license for Macsyma into something more permanent.

Ultimately, the United States Department of Energy imposed on MIT the requirement that they release a copy into the National Energy Software Center (NESC) library in 1982, while allowing MIT to assert restrictions in terms of high price and no redistribution. This was intended to protect the technology transfer to Symbolics. (Such restrictions have been since lifted, 2002 or so.) This so-called DOE Macsyma had been rewritten at MIT in a pre-Common Lisp dialect called NIL lisp, incompletely implemented on VAX/VMS, an unpopular system in academia, where Berkeley VAX Unix was common. DOE Macsyma formed the basis for the much-later open source Maxima system.

In 1981, Moses and Richard Pavelle, an MIT staffer and proponent of applying Macsyma to engineering and science, proposed to form a company to commercialize Macsyma. However, MIT policy prevented MIT personnel from profiting directly from MIT developments. In early 1982, Macsyma was licensed by MIT to Arthur D. Little, Inc., which became the broker for Macsyma and soon licensed Macsyma to Symbolics in late 1982. It appears that Symbolics was not so much interested in the code as in keeping Macsyma out of the software catalog of its competitor in the Lisp Machine business, LMI. The business arrangement between Symbolics and Arthur D. Little required a royalty payment to ADL of 15% of Macsyma gross sales. This led to speculation on the desire of MIT and ADL to see MACSYMA prosper. The development of Macsyma continued at Symbolics despite the fact that it was seen as a diversion from the sales of Lisp machines, which Symbolics considered to be their main business. Macsyma sales and the leveraged sales of Lisp Machines reached 10% of overall sales at Symbolics within two years. Despite resistance from many in Symbolics, Macsyma was released for DEC VAX computers and Sun workstations using Berkeley's Franz Lisp in 1986 and 1987.

However, problems in Symbolics and within the Macsyma group led to the decline of Macsyma. In the first half of 1986, Macsyma revenues were lower than in the first half of 1985, in a growing industry. Wolfram's SMP program and Waterloo Maple were growing at this time, although MACSYMA was easily superior to these other packages in symbolic math.

Pavelle ran the Macsyma division at Symbolics until early 1986. In the second half of 1986 Richard Petti became the manager of the Macsyma business. Macsyma cut headcount but expanded its sales force and marketing, and focused its developers more on features that customers asked for. (For example, the Grobner algorithm developed in the 1970s at MIT was not installed in a shipping Macsyma until 1987.) In 1987, Macsyma annual revenues roughly doubled. MACSYMA became more user friendly: documentation and on-line help were reorganized and expanded; some command names were changed to be more mnemonic. Petti argued to Symbolics management that Macsyma was a 'strategic business unit' that should be funded based on its performance and potential, not based on the prospects of the main workstation business. However, Symbolics cut the Macsyma headcount after this period of sharp growth; Symbolics tried to use Macsyma as a tiny cash cow to fund the losses of the workstation business that was 30 times its size.

The greatest product weakness of Macsyma was its numerical analysis. Mastering symbolic math was a herculean task; but numerical capabilities were critical to get a piece of the much larger engineering and lower-end scientific markets. At MIT Macsyma had a link to the IMSL (now Visual Numerics) numerical libraries, but this link was severed when Macsyma moved to Symbolics. Lisp developers at Symbolics generally believed that numerical analysis was an old technology that was not important for Lisp applications, so they declined to invest in it. Double precision arithmetic in Macsyma (on a PC version) was about six times slower than Fortran. Also Macsyma's matrices were implemented as lists of lists, which cost another factor of perhaps 5-10 in speed for key algorithms. Macsyma did not have many of the basic algorithms of numerical linear algebra, such as LU decomposition.

In 1987-88, the Macsyma group tried to build a PC Macsyma with Gold Hill Lisp. (Earlier, Symbolics had killed its own project to build a Lisp compiler for standard computers to avoid competing with Lisp machine sales. This was a controversial move that, by some accounts, was undertaken without approval of senior management. Symbolics also declined to cooperate with Sun to make Lisp available on Sun workstations for the same reason.) Gold Hill Lisp was too unstable, and its weak architecture made it impossible for Gold Hill to eliminate the bugs. This was a crucial failure for Macsyma. It meant that Macsyma could not respond on PCs when Mathematica appeared on Apple computers in mid-1988. Macsyma appeared on Windows PCs in August 1989 using the CLOE Lisp from Symbolics. However, the Macsyma staff was too small and lacked the mix of skills needed to add the kind of graphics, notebook interface and better numerics that Mathematica had.

By 1989, it was clear to Petti that Symbolics would implode due to poor product strategy, and that it would take Macsyma with it. (Some product issues: (1) Symbolics' software was designed for MIT-class software developers without enough concessions to others. (2) Symbolics' world-class software was losing share in the market due to dependence on very high-cost hardware. (3) In moving to VLSI hardware in the mid-1980s, Symbolics converted from 36-bit word size to a 40-bit, without justification from the market for the enormous development cost of this change.) Macsyma could not assemble a buy-out team due to lack of cooperation from MIT. After keeping reasonably quiet since 1986 about the product issues, Petti tried in late 1988 to persuade Symbolics to adopt a software-only or board-level strategy; but the fourth president in four years would hear nothing of it. In 1990 Petti left Symbolics for a start-up.

Macsyma, Inc.

Macsyma, Inc., was founded in 1992 by Russell Noftsker (chairman, who had co-founded Symbolics) and Richard Petti (president, who had turned around Macsyma for a time at Symbolics) by raising funds and purchasing the rights to Macsyma from the ailing Symbolics. Although the market was growing fast, Macsyma sales in 1991 and early 1992 were still falling rapidly. Macsyma's market share in symbolic math software had fallen from 70% in 1987 to 1% in 1992. By 1993, market growth had slowed and the market had standardized on Mathematica and Maple. The competitors had development staffs that were 4-8 times as large as that of Macsyma Inc. throughout the 1990s.

In early 1995 the company shipped Macsyma 2.0.5, with these improvements: (1) On Wester's large test of symbolic math, Macsyma 2.0.5 scored 10% better than Maple and 15% better than Mathematica. Instead of being very slow, it was on average faster than Mathematica and almost as fast as Maple. (2) Macsyma 2.0.5 had a better notebook interface than either competitor, with formatted math display, text formatting, hypertext links, and the best scientific graphics interface in the math software industry. (3) Reviewers unanimously agreed that Macsyma had the best help system in the industry (including hypertext, demos, examples, function templates, and later Mathtips natural language query). Macsyma also had greatly improved its paper documentation. (4) Although Macsyma 2.0.5 was still very slow at numerics, it had a greatly strengthened portfolio of numerical analysis and linear algebra routines. (In 1996 Macsyma added LAPACK which greatly increased the speed of most numerical linear algebra.) A review in PC Magazine raved about the product, saying "Macsyma is the leader in the mathematical program marketplace."

This accomplishment was possible because Macsyma Inc. had more than its share of world-class developers, including Jeff Golden (for most symbolic math), Bill Gosper (for special functions, summations and other areas), Howard Cannon (VP of development, for all software), and occasional consulting from Bill Dubuque (for integration and equation solving). Other developers made major contributions in numerical analysis, graphics, and help systems.

The company could not continue indefinitely to outperform staffs 4-8 times as large and reverse the market momentum. Market share did not increase above 2%, because the competitors were entrenched in all key accounts, converting to a new product is very costly in learning time, and market growth declined. Also, starting in 1992 or 1993, Mathsoft engaged in a Pyrrhic strategy of spending $10 million on direct mail at very low prices, which won much of the remaining growth in the symbolic math market, just when Macsyma Inc. was struggling to rebuild its world-class product.

In 1999, Macsyma were acquired by Tenedos LLC, a holding company that previously had purchased Symbolics. The holding company has not re-released or resold Macsyma, but Macsyma continues to be distributed by Symbolics.

Available versions

There is also a GPL-licensed version, called Maxima, based on the 1982 version of the DOE Macsyma, subsequently adapted for Common Lisp and enhanced by William Schelter. It is under active development, and can be compiled under several Common Lisp systems. Downloadable executables for Linux, Microsoft Windows, Mac OS X and other systems, including graphical user interfaces are available. Maxima does not include any of the numerous enhancements made to the commercial version of Macsyma between 1982-1999 (at least 50 man-years of work), but includes numerous additional features and several alternative front ends. While this has resulted in some incompatibilities between Macsyma and Maxima, programs written in Common Lisp or in the Macsyma algebraic language can often be run, with only minor changes, in either system.

External links

* [http://www.math.utexas.edu/pipermail/maxima/2003/005861.html Richard Petti's summary of the history of commercial Macsyma] "(remarks on the history of Macsyma by a cofounder of Macsyma, Inc.)"
* [http://www.math.utexas.edu/pipermail/maxima/2003/005884.html] more from Richard Petti."