Software brittleness problem

The software brittleness problem is a consequence of the well-known fact that computers carry out commands, rather than exhibiting initiative.

A computer system with true initiative would have large amounts of practical knowledge about the real world. When it could not carry out some instruction, it would look for an alternative method of accomplishing an equivalent result.

Instead, as of this date (2008) programs usually just fail with an error message.

There are several approaches to solving the brittleness problem.

Restricting the problem domain

Methods that limit the user's possible behaviors are arguably the most successful approach currently known to reducing software brittleness. In this way, the program can be designed to cope with all possible behaviors, and then tested to see if it meets its design.

Goal seeking software

In this design, the user specifies an acceptable outcome. The computer program has a set of operations it can combine. Crucially, the software must be able to try out different combinations of the operations without causing harm. When it finds a sequence that produces the acceptable result, the program is done.

Goal seeking is fine when the computer can identify every possible undesired result, but this is rarely possible. People are far more skilled at recognizing undesirable outcomes than software is.

Also, producing software to identify every possible bad solution is at least as complex as simply finding a solution. See the article on the frame problem for more on this.

Knowledge-based systems

In this design, goal seeking is combined with a large number of facts about the real world, and a logic engine to reason about the facts. This is a basic attempt to add to the computer's ability to recognize undesirable results.

At least one such software (see Cyc) has been under development for more than a decade (as of 2003).Whilst Cyc has captured far more domain knowledge than any other similar system, it is unclear how successful it has been at meeting its original goals.

Neural networks

This plan would substitute a self-training neural network for the knowledge base of goal-seeking software. No method is generally known to accomplish this.

Evolutionary systems

This plan uses a software ecology that breeds software to solve the identified problem. It is a goal-seeking system, so it can theoretically solve the brittleness probelem. As with other solutions, the problem is that there must be an identified method of rejecting all possible bad solutions.