Do Reviews with Ratings Help?

[MylesBAstor]

Do you like or is it useful when a reviewer sums up an equipment review using a rating system employing a numerical value, certain number of stars, percentage improvement, etc.? Or gives the DUT five stars for tonality, 10 for soundstaging, etc.

[Johnny Vinyl]

What is a DUT?

[cpp]

Stars, numbers, or percentages Just provide me the a truthful and unbiased closing summary on the product without "fluff" . Be sure to outline the pro’s and con’s there. Summaries also help to gel together longer reviews into concise opinions.

[tom_hankins]

They can help, but I really like comparisons to other equipment.

[Johnny Vinyl]

They can help, but I really like comparisons to other equipment.

I'm the opposite. Comparing to other gear is too subjective an exercise that carries no weight for me. I do like a comparison to the previous model however.

[1morerecord2clean]

Numerical values might help but I find the absence of detailed disclosure of wires used, racks and mechanical isolation used or not, and specifically what electronics used to be a hurdle in reviews. Yes, I know that a laundry list of associated equipment is often at the end of a review but I have rarely read if all the equipment available to the reviewer was tried or if only some was used and if in fact some of that equipment was used with the reviewed component.

[Chris F]

What is a DUT?

DUT is short for "Device Under Test".

Personally for analog source components like cartridges/turntables/phono stages I find nothing helps more then a sound sample. One each from the "reference setup" and then with the new component in place. I really really like it when Fremer does group reviews with sound clips and I find them extremely extremely useful.

Sure the clips are far from the final word and you lose a bit during the A/D and then D/A process but the essence definitely remains. If the review says "great transient attack", "great tonality" etc... I should still be able to get that from a sound clip.

[Johnny Vinyl]

Thanks for the decyphering.

[JCOConnell]

With the latest MQA debacle, I'm not sure I can trust any reviews any more when they can conflict so strongly.

[Rob]

Do you like or is it useful when a reviewer sums up an equipment review using a rating system employing a numerical value, certain number of stars, percentage improvement, etc.? Or gives the DUT five stars for tonality, 10 for soundstaging, etc.

breaking it down like Steve Rochlin's reviews? I rather see a composite score taking into consideration all aspects because the reader is less likely to misinterpret the reviewer's intent, that's what descriptive writing is for. Jerry Seigel ranks components under review from 1 to 10 LPs. IMO, that system has some validity because a 5 LP score doesn't mean its necessarily bad, it's based on the reviewer's biases and system synergy. it leaves room for the reader to decide whether or not it merits further investigation.

I dislike Stereophile's 'Class system' the most (e.g. A, B, C) its confusing for the short timer that's building a system based on these rankings, its pure rubbish.

[fcrowder]

I find ratings such as those in Stereophile's Class System or Absolute Sound's Recommended Components to be virtually useless as so many different products are often listed in the same general class. It also provides no information on component compatability or even what power cords or interconnects work well. A well written review if read careully is much more useful, at least if it is honest. Assembling a system is an art not an exercise in picking components from a class A listing.

[atmasphere]

I'm going to say 'no' to this one.

Way back in the 1970s Kawasaki came out with a line of 3 cylinder 2-stroke motorbikes. They got excellent reviews and rated highly.

In time, it was sorted out that the 750cc version was a bit overpowered for the frame. If you accelerated through a turn you were OK but if you let off on the throttle, the frame would unflex from where the motor had wound it up and you would wind up in the ditch. The bike was soon notorious.

Ratings loose their value over time due to a variety of issues. if you give something a '9' because something else is better, that plus item had better well hold its value! That does not always happen, even though gear that holds a lessor rating continues to rate on the used market. We can all think of examples. Just my opinion here but I think a numeric rating will get dated while a simple expression of what is good about a piece will not.

[Steve Lefkowicz]

A friend in college had the 500CC triple from this line (the H1 or H3, I forget). Fast as all get out, but a death trap otherwise. I much preferred the Norton Combat 750 twin I had at the time.
Neither bike was terribly reliable.

[Rust]

I had an H2 Mk IV when I was a kid and it was the E-ticket ride (for those of you who know what an E-ticket is). And it wasn't quite as bad as some pundits suggested. Well, at least until I stuck a set of expansion chambers on it and finally got the jetting right. Then it was purely evil wicked fun..

[Steve Lefkowicz]

They are only relative to that particular reviewer's past reviews. We used to have stars (quality) and checks (value) at Listener. I remember reviewing the Rega P3 and gave it a certain number of checks and stars. It started a long internet discussion, because Herb Reichert (one of my two favorite audio writers, along with Artie) had reviewed a different more expensive table and gave it a half a star lower rating. You'd think we had just told people their kids were ugly. That star discrepancy totally invalidated all of our reviews to some people.

Then again, I only look at reviews as a personal tale of how a product worked in my system. I don't expect anyone else to know or like the music I play or like (why I don't often mention specific tracks in my reviews) and I don't particularly care for what goes on inside the box (I spend all day at work dealing with complex technology). That to me is the designers choice and I just want to see what the results are.

The famous naval architect Bob Perry (quite the audiophile too) used to review sailboats for Sail magazine. but he reviewed the engineering and design, on paper, never actually sailing them. Great articles that gave great insight to what yacht design was all about. However, they never gave an indication of how they sailed or what it was like to be on the water in one. I also subscribed to Practical Sailor, a great monthly newsletter that reviewed sailboats strictly on the water in terms of how they worked. Totally the opposite of Perry's articles. It was really nice getting both viewpoints.

I consider myself more of the Practical Sailor type than the Bob Perry type, though it is nice to have both articles to read.

Odd points, before I knew Perry was an audiophile and occasional letter writer to Positive Feedback (in the old paper days) he was who my wife and I had decided would design our first custom sailboat if we ever could afford one. Practical Sailor was owned by the same company that bought Listener.

[MylesBAstor]

What is a DUT?

Device Under Test.

[MylesBAstor]

breaking it down like Steve Rochlin's reviews? I rather see a composite score taking into consideration all aspects because the reader is less likely to misinterpret the reviewer's intent, that's what descriptive writing is for. Jerry Seigel ranks components under review from 1 to 10 LPs. IMO, that system has some validity because a 5 LP score doesn't mean its necessarily bad, it's based on the reviewer's biases and system synergy. it leaves room for the reader to decide whether or not it merits further investigation.

I dislike Stereophile's 'Class system' the most (e.g. A, B, C) its confusing for the short timer that's building a system based on these rankings, its pure rubbish.

Or like Martin Colloms does.

Does anyone remember (know MEP does) when Dave Wilson used to draw diagrams?

[Rust]

A few comments from the cheap seats.

I rather like what some of the British publications do, review five or six similar components in around the same price range. For instance, integrated amplifiers. All tested with the same source, wires and speakers. Pretty much as the typical consumer would use it.

Note that typically such reviews are of relatively affordable gear. You know, stuff I could actually buy myself.

Such reviews wouldn't work for more expensive gear for a variety of reasons. At a certain level gear becomes so idiosyncratic that system matching is a must. At that point the review becomes less about the actual DUT and more about system tuning/matching to determine ultimate performance of the system as a whole.

I think both styles of reviewing are relevant albeit for different audiences and price points.