Quality of models

Modeling is an integral part of many technical fields, including engineering, economics, and software engineering. In this context, a model is a formal representation of an organizational system, such as a business model or a formal description of software in UML.

The SEQUAL framework

For evaluating the quality of models, the semiotic quality framework (SEQUAL) developed by John Krogstie [http://www.palgrave-journals.com/ejis/journal/v15/n1/pdf/3000598a.pdf] , is introduced. The framework is grounded in semiotics, particularly the semiotic theory of Charles W. Morris.

With a focus on the Quality Framework for modeling languages.

Sets in the Quality Framework:
* A: Actors that develop or have to relate to (parts of) the model. Can be persons or tools.
* L: What can be expressed in the modeling language
* M: What is expressed in the model
* D: What can be expressed about the domain (area of interest)
* K: The explicit knowledge of the participating persons
* I: What the persons in the audience interpret the model to say
* T: What relevant tools interpret the model to say
* G: The goals of the modeling

Physical quality

Externalization: KM = Ø – Is it possible to externalize knowledge by using the model language?

Internalizability: Model persistence and availability.

Basically: Is the model language able to express the model domain?

Externalization is presenting the modeller's concept in some model form for others to make sense of it. Other people can have look on it and can discuss. How other people perceives the model is a matter of internalization. After perceiving the model in their own way they can discuss and change their mind accordingly. To make sense others, it is better to have some model language in common. Physical quality refers to the possibility of externalizing models by using model language that should be available and of course in persistence manner to be internalized by audiences.

How available is the model to audience? Availability depends on distributability, especially when members of the audience are geographically dispersed. Then, a model which is an electronically distributable format will be more easily distributed than one which must be printed on paper and sent by ordinary mail or fax. It may also matter exactly what is distributed, e.g. the model in an editable form or merely in an output format.

How persistent is the model, how protected is it against loss or damage? This also includes previous versions of the model, if these are relevant. E.g. for a model on disk, the physical quality will be higher if there is a backup copy, or even higher if this backup is on another disk whose failure is independent of the originals. Similarly, for models on paper, the amount and security of backup copies will be essential.

Empirical quality

To evaluate empirical quality, the model should be well externalized.

Main aspects: Ergonomics, readability, layout, information theory.

Basically: Is the model easily readable?

Empirical quality deals with the variety of elements distinguished, error frequencies when being written or read, coding (shapes of boxes) and ergonomics for Computer-Human Interaction for documentation and modelling-tools. Ergonomics is the study of workplace design and the physical and psychological impact it has on workers. This quality is related to readability and layout. There are different factors that have an important impact on visual emphasis like size, solidity, foreground/background differences, colour (red attracts the eye more than other colours),change(blinking or moving symbols attract attention), position and so on.

For graph aesthetics there may be different consideration(Battista, 1994, Tamassia, 1988) like angles between edges not be too small, minimize the number of bends along edges, minimize the number of crossings between edges, place nodes with high degree in the centre of the drawing, have symmetry of sons in hierarchies, have uniform density of nodes in the drawing, have verticality of hierarchical structures and so on.

yntactical quality

Syntactic quality is the correspondence between the model M and the language extension L of the language in which the model is written.

Syntactic correctness: Are all statements in the model according to the syntax and vocabulary of the language?

Syntactic invalidity: Are words or graphemes not part of the language used?

Syntactic completeness: Does the model contain all the statements which would be correct and relevant about the domain? DM = Ø

To assure the syntactic quality of the model, syntax checks should be done with the help of the following two main directions:

1.Error prevention: this type of checks adapts the principles of syntax directed editors. Thus, only modelling constructs that are defined in the language's vocabulary are available through the editor. Also, when a drawing session violates a syntax rule of the language, the modelling session should be temporarily interrupted in order to restore the legal model. This type of checks is controlled by the tool.

2.Error detection: During a modelling session, some syntactical errors--- syntactic incompleteness --- should be allowed on a temporary basis. For instance, although the DFD language requires that all processes are linked to a flow, it is difficult to draw a process and a flow simultaneously. Syntactical completeness has to be checked upon user's request. So, in contrast to implicit checks where the tool is ``forcing" the user to follow the language syntax, explicit check can only detect and report on existing errors. The user has to make the corrections.

emantic quality

What is expressed in the model?The semantic goals of this framework are:
* Validity; if all the statements in the model are correct and related to the problem. MD = Ø
* Completeness; if the model contains all relevant and correct statements to solve this problem. DM = Ø

Perceived semantic quality

Perceived semantic quality is the relation between an actor's interpretation of a model and his/her knowledge of the domain.
* Perceived validity IK = Ø
* Perceived completeness KI = Ø

Pragmatic quality

Pragmatic quality is the correspondence between the model and people's interpretation of it. Comprehension is the only pragmatic goal in the framework. It is very important that people that read the model, understand it. No solution is good if no-one understands it.

Pragmatic quality relates to the effect the model have on the participants and the world. Four aspects is treated specifically

1.That the human interpretation of the model is correct relative to what is meant.

2.That the tool interpretation is correct relative to what is meant to be expressed in the model. 3.That the participants learn based on the model.

4.That the domain is changed (preferably in a positive direction relative to the goal of modelling).

ocial quality

The goal for the social quality is agreement. Agreement about knowledge, interpretation and model. Agreement is achieved if perceived semantic quality and comprehension are achieved. There is relative agreement and absolute agreement.For the three agreement parts (knowledge, interpretation and model) we can define:
* Relative agreement in the three above agreement types; all K, I and M are consistent.
* Absolute agreement in the three above agreement types; all K, I and M are equal.

Knowledge quality

Degree of internalization of existing organizational reality.

Knowledge in domain is "complete": DK = Ø.Knowledge in domain is "valid": KD = Ø.

Activities for improvement:
* Stakeholder identification
* Knowledge source identification
* Research and investigation
* Participant selection
* Participant training
* Problem definition

Language quality

To receive good language quality it is important that:
* The language is appropriate to the domain.
* The language is appropriate to the participants' knowledge of modeling languages
* The language appropriate to express the knowledge of the participants

If the language quality is good, it will improve the participants' interpretation and other technical actors' interpretation. For additional detail, see the quality of modelling languages

Organizational quality

The organizational quality of the model relates to:
* That all statements in the model contribute to fulfilling the goals of modeling.(Organizational goal validity)
* That all the goals of modeling are addressed through the model.(Organizational goal completeness)

External links

* [http://www.idi.ntnu.no/~krogstie/publications/2003/quality-book/b3-quality.pdf "Quality of conceptual models" by John Krogstie]
* [http://www.idi.ntnu.no/~krogstie/publications/2001/ifipwc/ifip2001-article.pdf "A semiotic approach to quality in requirements specifications" by John Krogstie]