Posted by
Tom Hammond-2 on
Oct 01, 2007; 12:26pm
URL: http://elecraft.85.s1.nabble.com/Sherwood-on-ARRL-Testing-Methodology-tp454007p454008.html
Bill:
I'm assuming you saw Rob Sherwood's recent posting to the FT-2000
reflector. It was recently re-printed to the elecraft reflector but
Bill Tippett, W4ZV.
Here's some comments on that article from Eric Swartz, WA6HHQ, of Elecraft.
73,
Tom
__________________________
Hi Bill,
I discussed the issue of the missing IMDDR3 numbers (Third Order
Dynamic Range) with Michael Tracy at the ARRL a couple of weeks ago.
I expressed my strong concern that they had dropped these from the
review data and were only posting a footnote showing how to calculate
them from the IMD level and MDS. I pointed out that we, and many
others, use the IMDDR3 numbers as the primary IMD Dynamic range
comparison between rigs and that the IP3 numbers were not as useful
for receiver comparisons, since they can artificially be inflated by
turning on the attenuator, or making a receiver have low sensitivity (deaf).
They agreed that they could add the IMDDR3 numbers back in explicitly
in the data table for future reviews. There is no sinister plot here.
:-) They absolutely did NOT remove them at the request of any
manufacturer. Quite the opposite. They were almost unintentionally
removed in the post lab test formatting at the layout level for the
review to save space as the review was rushed to print. After
discussing this with Michael and Joel Hallas, the reviews manager,
they said that they would add them back in for future reviews.
Also, the multi-level IP3 numbers were in direct response to requests
to the ARRL from several members of the advisory group that Rob
mentions below. The primary reason for doing so, as mentioned in the
sidebar of the FT-2000 review, was to show how some radios depart
from the ideal IP3 curve at different levels. Personally, as I
mentioned above, I feel that IP3, while useful for testing stand
alone amplifier stages, is not useful for receiver comparisons. (Rob
also alludes to this in his comments.) The same IMD level test is run
for IMDDR3 and IP3. The IP3 number is just calculated differently
from the exact same IMD data. The best comparison still is the full
IMD Dynamic range, IMDDR3, which can not be as easily manipulated by
artificially changing a receiver's sensitivity, as it can for IP3. I
can make the K2 have a huge IP3 by reducing its front end gain by
another 10 dB. Would anyone want that? No.
The better test is to just take the signal generator level necessary
to create an IMD product equal to the MDS, and compare that generator
level to the MDS for the receiver. The net difference is the IMDDR3.
Michael Tracey and the ARRL test lab are above reproach in my
opinion. They take great pains to try to make accurate and useful
measurements of each rig, and they are ethical to an extreme. Michael
spends an incredible amount of time and care making each set of test
measurements. They do not change any data for big advertisers and
they buy the rigs they test on the open market. All of the recent
changes were made as a result of input from the test advisory group
that Rob is a member of. I'm sure there is still a lot of room for
discussion and improvement of the tests they run, but the ARRL does a
great service for the amateur community with their tests (as does
Rob). Their ONLY goal with these latest changes is to make the data
even more useful and repeatable between rigs.
73, Eric WA6HHQ
-----
Bill Tippett wrote:
> Rob Sherwood gave his permission to post this on the
>FT-2000 list. Since that is public information I am posting
>it here also. BTW I agree with his comments on the "new"
>methodology.
>
> 73, Bill W4ZV
>
>
>What has gone wrong with the ARRL's new Product Reviews in QST?
>
>For several years I participated as part of a group of hams who were
>trying to improve the testing of radios reviewed in QST. Several
>reviews in the past had included questionable data on receiver
>performance. A sincere effort was initiated to correct testing
>problems within the lab, and hopefully to also improve the "hands
>on" portion of the report.
>....
>
>What has changed in the ARRL reports?
>
>Some of the changes are of minor interest, like measuring the noise
>figure of an HF radio. Noise figure is generally used by VHF and
>UHF enthusiasts, but adding these data points certain hurts
>nothing. Is noise figure, or noise floor, or sensitivity a
>significant issue in today's receivers? The 75A-4 has an excellent
>noise floor, as reported in the January 2006 QST Annual Vintage
>Issue. Few of us have such a quiet location that atmospheric and
>galactic noise don't overshadow the noise floor of a modern receiver.
>
>What we did get was additional confusion in the ethereal world of
>third-order intercept (IP3), in place of real dynamic-range
>data. The League used to measure it one way, then a second way, and
>now three ways. Is this supposed to be helpful? The old way
>(measured at the noise floor) was acceptable. The second way
>referenced an imprecise S5, now defined as -97 dBm, and a third new
>way at 0 dBm. Zero dBm is really strong, something we don't likely
>ever see, unless we are working Field Day or Multi-Multi contests
>from near-by transmitters.
>(I am assuming we are not living in Europe with their 5 megawatt AM
>broadcast transmitters.)
>
>0 dBm is S9 + 73 dB, assuming any S meter reads that level
>accurately. (The Flex 5000A would actually do that.) On my IC-781,
>0 dBm reads S9 + 50 with 30 dB of internal attenuation, or something
>like S9 + 80 dB with the attenuators off, if the S meter would read
>that high, which is does not. What happens when you put two 0 dBm
>signals into an IC-781 at 20 kHz spacing? The IMD reads S9 + 18 dB.
>At 2 kHz spacing the IMD reads S9 + 60 dB! The 781 is not a radio
>with performance problems, so what do these new and improved
>measurements really mean?
>
>If you look at the FT-2000 chart for IP3 at 2 kHz with the preamp
>off, you see the IP3, measured at the noise floor, is -19 dBm. This
>is not a good number, particularly since a Yaesu radio with "IPO"
>enabled (no preamp) is similar to most other radios with the 10 dB
>attenuator enabled. Yet if you measure the FT-2000 at 0 dBm, the
>IP3 calculates out to +15 dBm, which sounds good. This new
>information is meaningless at best, or misleading at its worst. Why
>is the IP3 so high at 0 dBM? Because the inter-modulation is so
>strong (S9 + 60 dB) the AGC has basically turned the gain of the radio off.
>
>Most operators will run an FT-2000 with preamp 1 enabled, since it
>gives a reasonable noise floor, sensitivity and AGC threshold. Yet
>no information is available with this typical setting for the newly
>touted IP3 reporting method, which at 2 kHz would be about -30 dBm
>for the League's sample. (The FT-2000 I measured was considerably
>worse.) To get a meaningful dynamic-range number, the reader now has
>to subtract two numbers. Why is this important data now missing, or
>at least obfuscated? Could it be the big advertisers in QST didn't
>like seeing 2 kHz dynamic-range numbers that are typically around 70
>dB? Only the League could take a measured 2 kHz dynamic range of 69
>dB at 2 kHz and calculate it into a +15 dBm intercept at 0
>dBm. Talk about smoke and mirrors!
>
>Th
_______________________________________________
Elecraft mailing list
Post to:
[hidden email]
You must be a subscriber to post to the list.
Subscriber Info (Addr. Change, sub, unsub etc.):
http://mailman.qth.net/mailman/listinfo/elecraftHelp:
http://mailman.qth.net/subscribers.htmElecraft web page:
http://www.elecraft.com_______________________________________________
Elecraft mailing list
Post to:
[hidden email]
You must be a subscriber to post to the list.
Subscriber Info (Addr. Change, sub, unsub etc.):
http://mailman.qth.net/mailman/listinfo/elecraft
Help:
http://mailman.qth.net/subscribers.htmElecraft web page:
http://www.elecraft.com