Sherwood on ARRL Testing Methodology

[Bill W4ZV]

W5WVO:
 >I don't know what the test configuration of the SDR-5000 was -- I
don't think
it uses any discrete hardware roofing filters, does it? -- but Elecraft is
already claiming that the IMDDR3 at 5 kHz spacing (400 Hz CW roofing filter)
will be greater than 100 dB. (See K3 Specs page) I don't believe Eric would
have let that spec be on a public web page unless he was pretty certain it
would still be true when the final numbers come out.  :-)

         The QSD design of the SDR-1000/5000 is completely different than
that used by the Orion/K3, so it does not need roofing filters.  Specs
like IMD/BDR are determined by the resolution and linearity of the ADC
in the sound card.  Although Sherwood did not publish 2 kHz
measurements, he did say the IMD/BDR performance is basically
independent of signal spacing, so 2 kHz IMD performance is
likely 96 dB also.  BDR at 100 kHz was measured at 123 dB,
which will also likely apply at 2 kHz.

         Phase noise was reported as:

Phase noise (normalized) at 10 kHz spacing:    123 (flat) dBc *
* Phase noise does not fall off at 6 dB per octave as expected.  Flex
believes the present phase noise limitations are caused by A/D clock jitter.

By contrast the K3's phase noise is:

Rig     1kHz    2   10   20   50   100   1M
K3      -110  -119 -136 -140 -143 -144  -150
Bottom line is that QSD-based SDRs are very different beasts, so
our assumptions based on classical designs are probably wrong.

         It's also dangerous to assume Elecraft's measurements
will be identical to ARRL/Sherwood.  There are often differences
due to different test methodologies, people and equipment.  For
example Ten-Tec maintains Orion's 5 kHz IMD spec is "101 dB typical",
but nobody else has ever measured more than about 95 dB, so I
would wait for independent measurements on the K3.

                                 73,  Bill  W4ZV

_______________________________________________
Elecraft mailing list
Post to: [hidden email]
You must be a subscriber to post to the list.
Subscriber Info (Addr. Change, sub, unsub etc.):
 http://mailman.qth.net/mailman/listinfo/elecraft   

Help: http://mailman.qth.net/subscribers.htm
Elecraft web page: http://www.elecraft.com

[Don Wilhelm-4]

As measurements become more precise, it becomes more and more important
to recognize what Bill is pointing out.  A slight difference in test
setup can result in a different result.  These measurements are done at
the sub-microvolt level and it does not take much to create a difference.

Test equipment must be calibrated, and the calibration tolerance should
be known.  Traceable calibration is one thing, but the tolerance limits
of that calibration are also important - not all calibration labs are equal.

Even with calibrated equipment and the same test setup, two different
equipment operators may yield two different results.  As an example,
consider an instrument having a display for readout (like an
oscilloscope), the trace has a finite width, and one operator may place
the cursor on the midpoint of a trace width while another may place it
at one edge yielding two different values - how much they differ depends
on the resolution used, brightness of the trace, scale illumination, how
well the display was focused, etc.

One good step in the right direction would be to report the region of
uncertainty for all measurements. For me, that is a piece of information
that becomes more critical as the measured values become smaller.  The
ARRL lab may do that calculation in-house (I haven't asked), but they do
not state it in their published reports.

So for now, when I see comparison data between two receiver that vary
only by a dB or so, I usually figure that is close enough to ignore the
difference (I usually do consider 3 dB or more difference to be
significant).

73,
Don W3FPR
 
Bill Tippett wrote:
> snip...
>         It's also dangerous to assume Elecraft's measurements
> will be identical to ARRL/Sherwood.  There are often differences
> due to different test methodologies, people and equipment.
>  
_______________________________________________
Elecraft mailing list
Post to: [hidden email]
You must be a subscriber to post to the list.
Subscriber Info (Addr. Change, sub, unsub etc.):
 http://mailman.qth.net/mailman/listinfo/elecraft   

Help: http://mailman.qth.net/subscribers.htm
Elecraft web page: http://www.elecraft.com

[David Cutter]

At these incredibly low signal levels a very good screened room is required.  Even in the 70's I was using a copper enclosure to measure sensitivity levels within a double screened room.  Extraneous radiation from the test equipment itself disturbed our measurements.  We had to fit blanking plugs to unused outlets on various bits of gear.  This is not for the home lab.

David
G3UNA

>
> From: Don Wilhelm <[hidden email]>
> Date: 2007/10/01 Mon PM 03:53:59 BST
> To: [hidden email]
> Subject: Re: [Elecraft] Sherwood on ARRL Testing Methodology
>
> As measurements become more precise, it becomes more and more important
> to recognize what Bill is pointing out.  A slight difference in test
> setup can result in a different result.  These measurements are done at
> the sub-microvolt level and it does not take much to create a difference.
>
> Test equipment must be calibrated, and the calibration tolerance should
> be known.  Traceable calibration is one thing, but the tolerance limits
> of that calibration are also important - not all calibration labs are equal.
>
> Even with calibrated equipment and the same test setup, two different
> equipment operators may yield two different results.  As an example,
> consider an instrument having a display for readout (like an
> oscilloscope), the trace has a finite width, and one operator may place
> the cursor on the midpoint of a trace width while another may place it
> at one edge yielding two different values - how much they differ depends
> on the resolution used, brightness of the trace, scale illumination, how
> well the display was focused, etc.
>
> One good step in the right direction would be to report the region of
> uncertainty for all measurements. For me, that is a piece of information
> that becomes more critical as the measured values become smaller.  The
> ARRL lab may do that calculation in-house (I haven't asked), but they do
> not state it in their published reports.
>
> So for now, when I see comparison data between two receiver that vary
> only by a dB or so, I usually figure that is close enough to ignore the
> difference (I usually do consider 3 dB or more difference to be
> significant).
>
> 73,
> Don W3FPR
>  
> Bill Tippett wrote:
> > snip...
> >         It's also dangerous to assume Elecraft's measurements
> > will be identical to ARRL/Sherwood.  There are often differences
> > due to different test methodologies, people and equipment.
> >  
> _______________________________________________
> Elecraft mailing list
> Post to: [hidden email]
> You must be a subscriber to post to the list.
> Subscriber Info (Addr. Change, sub, unsub etc.):
>  http://mailman.qth.net/mailman/listinfo/elecraft   
>
> Help: http://mailman.qth.net/subscribers.htm
> Elecraft web page: http://www.elecraft.com
>

-----------------------------------------
Email sent from www.virginmedia.com/email
Virus-checked using McAfee(R) Software and scanned for spam

_______________________________________________
Elecraft mailing list
Post to: [hidden email]
You must be a subscriber to post to the list.
Subscriber Info (Addr. Change, sub, unsub etc.):
 http://mailman.qth.net/mailman/listinfo/elecraft   

Help: http://mailman.qth.net/subscribers.htm
Elecraft web page: http://www.elecraft.com