Audio equipment testing

Audio equipment testing is done to provide consumers with an idea of what they are looking for and to make the process of equipment selection easier. The results are published in specialty electronics magazines, online, and in other media. Many people involved in the development or use of audio gear have an engineering background and attempt to bring a scientific perspective to evaluating audio gear. They are concerned with measurements using test equipment and would ideally like to see double-blind testing used to compare competing products. On the other hand, some reviewers believe that not all of the characteristics that produce excellence in sound reproduction are measured by the current tests. Audio reviewers in this camp also claim that double-blind testing does not provide the kind of relaxed extended-listening environment needed to evaluate an audio component. The testing methods used to evaluate equipment can be roughly divided into two groups. The two opposing factions are called objectivists, who believe that all perceivable differences in audio equipment can be measured scientifically and subjectivists, who believe that the human ear is capable of hearing details and differences which can not be directly measured. [ [http://www.theaudiocritic.com/back_issues/The_Audio_Critic_16_r.pdf "Basic Issues of Equipment Reviewing and Critical Listening: Our Present Stance,"] Peter Aczel, "The Audio Critic," issue number 16, page 31 (PDF page 25), accessed 2007-05-18.]

Objectivists

Objectivists believe that audio components, accessories, and treatments must pass rigorously-conducted double-blind tests and meet specified performance requirements to meet the claims made by their adherents.

*Objectivists point out that every properly conducted and interpreted double-blind test has failed to support subjectivists' claims of significant or extremely subtle sonic differences between devices if measurements alone predict that there should be no sonic differences between the devices when listening to music. [cite news | url = http://www.mastersonaudio.com/audio/20020901.htm | title = The Ongoing Debate about Amplifier "Sound" |work = Ian G. Masters | date = September 1 2002 | journal = mastersonaudio.com ] [ [http://www.biline.ca/critic1.htm Paste This in Your Hat! - What Every Audiophile Should Know and Never Forget] , Peter Aczel, Biline.ca, Accessed 2007-05-11]
*Objectivists feel that some subjectivists lack engineering training, technical knowledge, and objective credentials, but nevertheless praise a product's innovation and performance. [http://www.biline.ca/critic3.htm]
*Objectivists reject concepts that while superficially based on accepted physical principles, apply them to circumstances where they are irrelevant. The skin effect, for instance, which relates the efficiency of cables to the frequency transmitted, is often applied to audio frequencies where it is insignificant [http://www.st-andrews.ac.uk/~www_pa/Scots_Guide/audio/skineffect/page1.html] .
*Objectivists believe that some subjectivists' practices seem driven by fashion—e.g., the late eighties' vogue for marking the edges of CDs with a green felt marker [cite web
url=http://www.snopes.com/music/info/greening.htm
title=Bewaring of the Green
publisher=Snopes.com
date=May 15 - June 15 1990] or suspending cables above the floor on small racks—and bear no relation to well-known laws of physics.
*Subjectivists often reject attempts to categorize differences in sound using measurements despite evidence of its effectiveness. It has shown that by tailoring the transfer function of a particular amplifier, it is possible to make it sound indistinguishable from another amplifier. [ [http://www.audio-ideas.com/interview/carver.html Interview with Sunfire's Bob Carver] Gordon Brockhouse, Audio Ideas Guide, Accessed 2007-05-11]
*Measured-audio distortion is immensely higher in electromechanical components such as microphones, turntables, tonearms, phono cartridges, and loudspeakers than in purely electronic components such as preamplifiers and power amplifiers, making it logically more difficult for objectivists to accept that very subtle differences in the latter can have an appreciable effect on overall musical-reproduction quality.

British audio equipment designer Peter Baxandall, who may be considered an objectivist, has written, "I ... confidently maintain that all first-class, competently designed amplifiers, tested under completely fair and carefully-controlled conditions, including the avoidance of overloading, sound absolutely indistinguishable on normal programme material no matter how refined the listening tests, or the listeners, may be; and that when an inferior amplifier is compared with a very good one and a subjective quality difference is genuinely and reliably established, it is always possible, by straightforward scientific investigation, to find a rational explanation for this difference." Baxandall also proposed a "cancellation test", which he claimed would prove his point. [Baxandall, Peter J. "Audible amplifier distortion is not a mystery". Wireless World, November 1977, pp. 63.]

Subjectivists

One statement that has influenced some audiophiles' values is from Harry Pearson, long-time editor of "The Absolute Sound": [cite web
url=http://www.enjoythemusic.com/tas/
title=The Absolute Sound
accessdate=2007-01-23 ]

"We believe that the sound of music, unamplified, occurring in a real space is a philosophic absolute against which we may judge the performance of devices designed to reproduce music."

*Subjectivists will rely on demonstrations and comparisons, but believe there are problems in applying double-blind methods to comparisons of audio devices. They believe that a relaxing environment and sufficient time measured in days or weeks is necessary for the discriminating ear to do its work. [ [http://www.hificritic.com/downloads/Archive_1.pdf The Objective Subjective Review Debate] , Martin Colloms, 1991, Retrieved on 2007-05-09 from hificritic.com]

*Subjectivists believe that careful individual listening is an appropriate tool for discovering the true worth of a device or treatment, and will generally acquire equipment that suits their own listening or style preferences as opposed to measurable equipment performance.

Some audiophile-equipment designers and consumers are obsessed over seemingly irrelevant details. Many components, for instance, are able to reproduce frequencies higher than the limit of human hearing—20 kHz. [ cite news | url = http://www.dizziness-and-balance.com/disorders/hearing/hearing.html | title = Hearing Loss | work = Timothy C. Hain, MD. | date = February 26 2006 | journal = Chicago Dizziness and Balance ] Some sources, such as FM radio, will not reproduce frequencies higher than 15 or 16 kHz.

Experienced listeners can be relied upon for valid subjective advice on how equipment sounds. British Hi-fi critic, Martin Colloms, writes that "the ability to assess sound quality is not a gift, nor is it the feature of a hyperactive imagination; it is simply a learned skill", which can be acquired by example, education and practice [ [http://www.hificritic.com/downloads/Archive_7.pdf Working in the Front Line:An approach to equipment reviewing] , Martin Colloms, Stereophile, Vol.14 No.1, January 1991 Retrieved on 2007-05-09 from hificritic.com] . In any event, the eventual purchase decision will be made by the end-user, whose "perception is reality" and can be influenced by factors other than the equipment's actual performance.

Opposing viewpoints

Objectivists attack Vacuum-tube amplifiers as vastly inferior because, in addition to their substantially higher total harmonic distortion, they require rebiasing, are less reliable, generate more heat, are less powerful, and are usually more expensive. [http://www.mastersonaudio.com/letters/200202.htm] Subjectivist believe that while tubed electronics are less linear than solid-state electronics at high-signal levels, they are much more linear at low-signal levels — less than one watt. Most musical signals spend most of the time at these low levels.

Objectivists claim that digital sound is superior to analog sound because it has no clicks, pops, wow, flutter, audio feedback, or rumble, has a higher signal-to-noise ratio, has a wider dynamic range, has less total harmonic distortion, and has a flatter and more extended frequency response. [http://www.soundstageav.com/mastersonaudio/20050415.htm] [http://www.mastersonaudio.com/audio/20030101.htm] Subjectivists however claim that the process of converting a bit-stream to an analog waveform requires heavy filtering to remove spurious high-frequency information and that it should be expected that such filtering should involve some signal degradation and a large amount of phase shift in the passband. They point out that commonly-used consumer-grade digital-to-analog converters (DACs) exhibit very poor linearity at low levels. Both problems, at first dismissed, were then addressed by such solutions as digital filtering, oversampling, and the use of DACs operating at 20-bit (or higher) resolution. The introduction of the new higher-bandwidth high-resolution music formats is a tacit admission of the reality of this issue. Musician Neil Young, for example, is a harsh critic of the sound of the original CD format but has approved of the sound of the newer SACD format with its greater safety margin between its ideal behavior and the requirements set by the limits of human hearing.

Objectivists consider total harmonic distortion to be an accurate measure of sound quality.Subjectivist however claim that total harmonic distortion has been proven by scientific testing to correlate poorly with perceived sound quality. The type of distortion is more significant. For instance, distortion by even harmonics has been shown to be less objectionable than distortion by odd harmonics.

Subjectivists believe that sound quality is degraded by large levels of negative feedback in amplifiers. Objectivists claim that negative feedback is beneficial to amplifier stability and produced good test results using steady-state waveforms. Subjectivists however believe that the application of negative feedback is inherently problematic for constantly-changing waveforms such as those that occur in music.

Subjectivists claim that there is a limit to what can be tested using Objective measurements. High-end audio companies which do rely on quantitative evaluations guard their measurement techniques as trade secrets. These are far more complex than the techniques which are in the public domain e.g. total harmonic distortion, transient intermodulation distortion. Subjectivist point out that objectivists since the 1970s no longer tout distortion measurements in their advertisements as there is a general consensus that an amplifier with 0.01% total harmonic distortion may not sound "better" than one with 0.1% total harmonic distortion - especially if the lower distortion is achieved with (excessive) feedback.

Overall, the subjectivists' world is looked upon by objectivists as being a hotbed of gullibility and fraud, its marketing engine driven primarily by either a constant desire for one-upmanship or a more benign desire to tinker with equipment. In particular, the tinkering drive is fed by wild claims for minor parts of the system such as cables. Objectivists, however, are often harshly dismissed by subjectivists as meter men — people who simply refuse to recognize what the subjectivists consider obvious. The debate is rather heated in certain quarters, and even James Randi chimed in on the issue. [http://www.randi.org/jr/112604yes.html#4]

Difficulty of testing

It is difficult, but very important, to match sound levels before comparing systems, as minute increases in loudness—more than 0.15 dB [ [http://www.theaudiocritic.com/back_issues/The_Audio_Critic_29_r.pdf "Our Last Hip Boots Column,"] Peter Aczel, "The Audio Critic," issue number 29, Summer/Fall 2003, page 5 (PDF page 6), accessed 2007-07-05.] or 0.1 dB [ [http://www.soundandvisionmag.com/assets/download/AmpSpekerInterface.pdf "The Amp/Speaker Interface: Are Your Loudspeakers Turning Your Amplifier into a Tone Control?"] E. Brad Meyer, "Stereo Review," June 1991, page 54, accessed 2007-07-05.] —have been demonstrated to cause perceived improvements in sound quality.

Listening tests are subjected to many variables, and results are notoriously unreliable. Thomas Edison, for example, showed that large audiences responded favorably when presented both live performances by artists and reproductions by his recording system, [cite web
url=http://inventors.about.com/library/inventors/bledisondiscphpgraph2.htm
title=The History of the Edison Disc Phonograph
publisher=About.com] which today would be regarded as primitive in quality.

Similarly, results of component evaluation between various listeners or even the same listener under different circumstances cannot be easily replicated or standardized.

Similarly, the acoustic behavior of the listening room—the interaction between loudspeakers and the room's acoustics—and the interaction between an electromechanical device (loudspeaker) and an electronic device (amplifier) are subjected to many more variables than between electronic components. Thus the "difference" in sound quality between amplifiers is actually the ability of an amplifier to interface well with loudspeakers or a lucky combination of loudspeaker, amplifier, and room that works well together [ [http://www.soundandvisionmag.com/assets/download/AmpSpekerInterface.pdf The Amp/Speaker interface] , Brad Meyer, Sound & Vision Magazine, Accessed 2007-05-11] .

The introduction of switching apparatus, with either metal connection (mechanical switches) or electronic processing (solid-state switches), may, some believe, obscure the differences between the two signal sources being tested.

References

External links

* [http://www.theaudiocritic.com/ The Audio Critic] - Thirty-year publication, now online only, with in-depth independent verification of vendors' claims.
* [http://www.stereophile.com "Stereophile"] - Largest, oldest, and most read subjectivist magazine includes online reviews and articles.