Posted by
Bill W4ZV on
Oct 01, 2007; 12:08am
URL: http://elecraft.85.s1.nabble.com/Sherwood-on-ARRL-Testing-Methodology-LONG-tp453987.html
Rob Sherwood gave his permission to post this on the
FT-2000 list. Since that is public information I am posting
it here also. BTW I agree with his comments on the "new"
methodology.
73, Bill W4ZV
What has gone wrong with the ARRL's new Product Reviews in QST?
For several years I participated as part of a group of hams who were
trying to improve the testing of radios reviewed in QST. Several
reviews in the past had included questionable data on receiver
performance. A sincere effort was initiated to correct testing
problems within the lab, and hopefully to also improve the "hands on"
portion of the report.
As time went on, however, it appeared to me that the group had become
fixated on minutia, and at the same time the League was unwilling to
look at real problems in new radios being offered to the amateur
radio operator. I have not contributed lately in the steering
committee, as I felt I was banging my head against the wall.
Some of the nonsense coming out of the League has been around for a
long time, like the following quote from the 2004 review of the Icom
IC-7800. "I was able to hear calling US stations on back scatter
that I don't believe I would have heard on the '930." Did the
reviewer bother to turn on his TS-930? No, he just assumed he was
hearing something unusual on the receiver being reviewed (or hyped) in QST.
Has anything improved in 2007? The latest October review of the
FT-2000D (200 watt version of the FT-2000 that was earlier reviewed
in February) states the following: "Why would I need a 200 W
transceiver? After using it for a while, I was quite impressed with
the extra punch the '2000D offered during routine CW and SSB contacts
compared to the 100 W version." How could anyone tell a difference
of 3 dB, especially compared to operation of the FT-2000 eight months
ago? This kind of reporting is drivel.
What has changed in the ARRL reports?
Some of the changes are of minor interest, like measuring the noise
figure of an HF radio. Noise figure is generally used by VHF and UHF
enthusiasts, but adding these data points certain hurts nothing. Is
noise figure, or noise floor, or sensitivity a significant issue in
today's receivers? The 75A-4 has an excellent noise floor, as
reported in the January 2006 QST Annual Vintage Issue. Few of us
have such a quiet location that atmospheric and galactic noise don't
overshadow the noise floor of a modern receiver.
What we did get was additional confusion in the ethereal world of
third-order intercept (IP3), in place of real dynamic-range
data. The League used to measure it one way, then a second way, and
now three ways. Is this supposed to be helpful? The old way
(measured at the noise floor) was acceptable. The second way
referenced an imprecise S5, now defined as -97 dBm, and a third new
way at 0 dBm. Zero dBm is really strong, something we don't likely
ever see, unless we are working Field Day or Multi-Multi contests
from near-by transmitters. (I am assuming we are not living in
Europe with their 5 megawatt AM broadcast transmitters.)
0 dBm is S9 + 73 dB, assuming any S meter reads that level
accurately. (The Flex 5000A would actually do that.) On my IC-781,
0 dBm reads S9 + 50 with 30 dB of internal attenuation, or something
like S9 + 80 dB with the attenuators off, if the S meter would read
that high, which is does not. What happens when you put two 0 dBm
signals into an IC-781 at 20 kHz spacing? The IMD reads S9 + 18
dB. At 2 kHz spacing the IMD reads S9 + 60 dB! The 781 is not a
radio with performance problems, so what do these new and improved
measurements really mean?
If you look at the FT-2000 chart for IP3 at 2 kHz with the preamp
off, you see the IP3, measured at the noise floor, is -19 dBm. This
is not a good number, particularly since a Yaesu radio with "IPO"
enabled (no preamp) is similar to most other radios with the 10 dB
attenuator enabled. Yet if you measure the FT-2000 at 0 dBm, the IP3
calculates out to +15 dBm, which sounds good. This new information
is meaningless at best, or misleading at its worst. Why is the IP3
so high at 0 dBM? Because the inter-modulation is so strong (S9 + 60
dB) the AGC has basically turned the gain of the radio off.
Most operators will run an FT-2000 with preamp 1 enabled, since it
gives a reasonable noise floor, sensitivity and AGC threshold. Yet no
information is available with this typical setting for the newly
touted IP3 reporting method, which at 2 kHz would be about -30 dBm
for the League's sample. (The FT-2000 I measured was considerably
worse.) To get a meaningful dynamic-range number, the reader now has
to subtract two numbers. Why is this important data now missing, or
at least obfuscated? Could it be the big advertisers in QST didn't
like seeing 2 kHz dynamic-range numbers that are typically around 70
dB? Only the League could take a measured 2 kHz dynamic range of 69
dB at 2 kHz and calculate it into a +15 dBm intercept at 0 dBm. Talk
about smoke and mirrors!
The League is also going to differentiate between blocking (gain
compression) and phase noise limited (a typical problem with
synthesized radio). A narrow band audio spectrum analyzer is needed
to measure blocking this new way. (I used this method on my Flex
5000A report because of the phase noise.) The ear is not going to
hear what the analyzer sees, but the League may have made an
improvement here. At least the two measurements now will be differentiated.
What is the League completely missing?
Most new DSP radios have serious problems in QRN, and with any kind
of transient impulse noise. Has QST reported on these
problems? They have not said a word. I gave a talk on this subject
at the 2007 Dayton Hamvention, to try to point out that all is not
well in the current state of radio design. The IC-7000 is a prime
example of a radio that is nearly useless in QRN, as is the
FT-2000. Every DSP-chip based radio designed in the last few years
has an AGC problem to some extent. Fast rise-time noises are
improperly handled by the AGC, drastically exaggerating the impulse noise.
I recently finalized an AGC test, using an HP fast-rise-time pulse
generator. It basically approximates an electric fence. The
generator was set for one pulse per second. The rise time was < 10
nanosecond, with a duration of around 1 microsecond. The level was
set to 1 volt peak, to propagate a pulse well into the HF
spectrum. The first radio tested with this new method was the
FT-2000, with preamp 1 enabled. This produces a rather typical CW
noise floor of -124 dBm, an SSB sensitivity reading of 0.3 uV, and an
excellent AGC threshold of 1.3 uV. With a reference non-DSP IC-781
that has similar specifications, the S meter on the pulse test read
less than S1, barely moving the S meter. On the FT-2000 the impulse
noise read S7, pulse after pulse after pulse.
While many hams seem oblivious to these AGC problems, at least some
operators are voicing their concern. I was pleased to hear from a
new ham at a recent Colorado hamfest describe his observations on his
IC-7000. Even though he had no past frame of reference from an
analog radio, he noted how strangely his Icom reacted to the
slightest click or tick. Merely turning on a light switch would kick
his S meter up many S units. I noticed the exact same problem two
years ago on all the DSP radios coming though my lab and ham shack.
When I queried the League on their review of the IC-7000, saying they
totally missed the AGC problems on transient noise and QRN, they
simply said they listened to it in December when there was no
QRN. What is their excuse this time on the FT-2000D? This radio had
to have been evaluated during the summer of 2007 when there was
plenty of thunderstorm static.
What did the FT-2000D review happen to say about the dynamic range
numbers with the different roofing filters? After giving a full
paragraph to explaining why narrower roofing filters are usually
helpful, the League simply said, "We noticed little difference in
performance between the 3 and 6 kHz roofing filers in any of the
FT-2000s tested, though, at any signal spacing."
On the FT-2000 data recently posted on my web site, the dynamic range
actually dropped from 90 dB at 20 kHz with the 6 kHz roofing filter
to 81 dB with the 3 kHz roofing filter. At 2 kHz spacing there was
minimal difference, 63 dB (3 kHz filter) and 61 dB (6 kHz
filter). Dynamic range numbers in the low 60s are not acceptable for
serious operators.
Finally, one more bizarre comment from the "hands-on" QST
reviewer. The u-Tune unit adds modest selectivity in the front end,
and significant insertion loss, as seen by the degraded noise
floor. Yet the reviewer found the u-Tune unit to be helpful "on 20
meters before the band closed with the u-Tune unit switched on." One
wonders why a little added RF selectivity and 10 dB insertion loss
would help when the band was fading out. If this statement is
accurate, which I question, there is something seriously wrong with
this radio beyond AGC and roofing-filter problems, a subject totally
ignored by the review.
When will the day come when the information in QST is more than a
fluff review, and a free multi-page advertisement for the manufacturer?
73,
Rob Sherwood
NC0B
Rev C1
_______________________________________________
Elecraft mailing list
Post to:
[hidden email]
You must be a subscriber to post to the list.
Subscriber Info (Addr. Change, sub, unsub etc.):
http://mailman.qth.net/mailman/listinfo/elecraft
Help:
http://mailman.qth.net/subscribers.htmElecraft web page:
http://www.elecraft.com