BIMA Memoranda No. 19 OPTIMIZED CALIBRATION Melvyn Wright, 2-Oct-91 SUMMARY This memo explores the optimum calibration of millimeter array data. The tradeoff between integration time on source and calibration, and the choice of calibrators is discussed. For strong sources, or at short baselines, calibration errors become more important. On-line calibrations of instrumental subsystems made with high SNR reduces the number of parameters needed to define the gain and passband functions. The gain calibration can be improved by fitting antenna gains to both sidebands simultaneously. This facilitates the selfcalibration and analysis of relative positions of images in both sidebands. For strong sources, or line and continuum observations, special care needs to be taken with the passband calibration. 1) MEASURED CORRELATION The measured correlation for antennas i and j, V'ij, are related to the true source visibilities, Vij, by V'ij(t,f) = Gij(t,f) * Vij(t,f) + R(t,f) ...................(1) where Gij(t,f) is the complex instrumental gain as a function of time and frequency, and R(t,f) is additive noise from the atmosphere and electronics. Since we wish to determine Vij(t,f), we must estimate Gij(t,f) with the best SNR and fidelity. 2) GAIN CALIBRATION Observation of a strong, unresolved, and achromatic calibration source whose true visibility is unity measures Gij(t,f) directly. Usually we cannot observe a calibrator simultaneously with the source and we must make a number of approximations. We usually determine the geometry of the array and correct the measured correlations so that the gain is independent of source HA, DEC, elevation, FOCUS, air temperature, and other instrumental and environmental parameters. We then assume that the gain is a slowly varying function of time, and observe unresolved calibrators, usually quasars, interspersed with the observations of the source. The instrumental gain versus time determined from the calibrators is then interpolated and used to correct the source observations. 2.1) INTEGRATION TIME ON CALIBRATOR. The calibrator must be observed for sufficient time to determine the gain (amplitude and phase) at each calibration interval. In order to obtain the best SNR we observe the calibrator with the maximum available bandwidth. The rms noise dS(Jy) = Jy/K * Tsys / (Bandwidth * integration time). For the BIMA array , using 200 Jy/K, Tsys = 500 K, Bandwidth = 320 MHz and 5 minutes integration time, we obtain dS = 0.3 Jy for each baseline and for each sideband. For a 2 Jy calibrator we obtain a SNR = 6 in 5 minutes. The rms phase error in the calibration is then 10 degrees. The error in the corrected data is dG * V + R, where dG is the error in the gain determination. The errors in the gain function will depend on the SNR on the calibration source, and on the number of parameters fitted to determine the function G. Since errors in the gain have a longer coherence in the corrected data, they have a more pernicious effect on the synthesised maps, and should be determined so that dG * V is smaller than the random noise R. Clearly, for a strong source, or at short baselines where the source visibility is high, it becomes more important to determine the gains accurately. As we discuss later it is possible to improve the gain determination by using an antenna based calibration, and using both sidebands. 2.2) INTERVAL BETWEEN CALIBRATIONS. The gain has two major contributions: i) a slowly varying instrumental component, mostly thermal in origin, which we try to calibrate by observing quasars at intervals. ii) an atmospheric component which can vary on all time scales. The choice of calibration interval is a compromise between determining the gain function whilst obtaining adequate integration time on the source. We try to minimize the error in the calibrated visibility (dG*V + R). The quasar calibration is usually taken at 30-60 min intervals and samples instrumental and atmospheric effects on this time scale. The effects of more rapid gain variations are discussed later. It is best to first remove all discontinuities in the instrumental gain (e.g. when an antenna is focussed) so that as few parameters as possible are required to define the gain function. The calibration time interval is of the same order as the spatial separation of source and calibrator. 2.3) SEPARATION OF SOURCE AND CALIBRATOR There are several factor to consider here. i) array geometry. There are phase errors, phi = dB.s + B.ds, where B is the baseline vector and s is the unit vector in the calibrator direction. We can include in dB residual errors due to other incompletely calibrated aspects of array geometry such as offsets between the antenna axes. Typically dB ~ 0.1 wavelength (.001 ns), and ds < 0.01 arcsec. On a 1000 ns baseline (1e5 wavelength) B.ds < 2e-2, so that the dominant error is due to dB. The phase error on the source is phi' = dB.s', so that the error in the calibrated source data dphi = phi' - phi = dB.(s' - s) where dphi is in turns. So we can get phase errors ~ 18 degrees using a calibrator 30 degrees (or 2 hours) from the source direction. For the best position information we use several calibrators weighted to have a mean position as close to the source direction as possible. (Wright etal., 1990) ii) atmospheric coherence. The data are corrected for the phase shift through a curved atmosphere. The atmosphere contains structures on all size scales. Calibrators close to the source will suffer similar phase shifts. This is hard to quantify, but inspection of the residual errors in the baseline calibration will indicate the magnitude of phase differences between quasars in different directions. These are generally < 0.1 radian rms, and if the baseline is well determined it is usually better to use a strong calibrator within 30 degrees rather than a nearby weak one which requires excessive integration time to obtain a reasonable SNR. iii) antenna slew time Time spent between between source and calibrator is lost integration time. Hat Creek antennas slew at 80 degrees per minute, so this is usually not a problem. In the normal pointing mode sources north of the latitude are observed with elevation gt 90 and sources south of latitude with elevation .lt. 90 degrees. If the source and calibrator have declinations on opposite sides of the latitude, then the pointing mode can be forced to minimize the slew time. A drawback to this procedure is that the pointing is not as well determined, and pointing errors may increase for unusual azimuth and elevation ranges. 3) PASSBAND CALIBRATION If the source and calibrator data are not obtained in the same frequency interval, then we must also determine the instrumental frequency response, or passband function. The quasar calibrator is usually observed with as wide a bandwidth as possible to obtain the best SNR. If narrow band, or spectral line observations of the source are required, then the SNR for the quasar is usually insufficient to determine the gain in each narrow band channel, and we must make further assumptions. If the instrumental frequency response is not a function of time, then we can write: Gij(t,f) = G'ij(t) * Pij(f),.........................(2) where G'ij(t) is the gain versus time and Pij(f) is the gain versus frequency, or instrumental passband. We can determine the passband by observing a strong achromatic source, and normalizing the observed spectra in the narrow band channels w.r.t. the gain in the wideband channel which is used to determine the gain versus time. If the gain versus time is varying rapidly, then the best SNR in the passband is obtained by normalizing the passband observation on a short time scale (e.g. normalize each record separately). If the gain versus time is more stable, then better SNR will be obtained by normalizing the passband observation w.r.t. a time averaged gain function. 3.1) PASSBAND ERRORS The error in the passband corrected uv-data is dV = (dG * P + G * dP) * V + R Where the source visibility is high we must take special care to minimize the passband errors dP. Special processing may be required, e.g., for planet observations. The passband calibrator need not be unresolved on all baselines, but must have sufficient SNR to determine the gains of the narrow band channels. The passband source should not be so resolved that the source structure changes the measured visibility spectra. E.g., a planet observation near a null in the visibility is not a good choice. The SNR on Planets and the strongest quasars is usually not sufficient to determine the passband directly for the individual channels, and we must again fit some function to the spectra in order to average channels to gain SNR. In order to fit as few parameters as possible to obtain the best SNR we must first remove the fine scale frequency structure from the instrumental frequency response. This can be done separately for the correlator and delay system. 3.2) CORRELATOR PASSBAND The correlator bandpass is defined by inserting low-pass filters with the required bandwidth. These have considerable frequency structure which must be calibrated with high SNR. Since the output from the correlator is the product of the input signals, we can do this by correlating a strong wideband signal. In practice we combine this calibration with a measurement of the overall bandpass for each antenna by injecting the IF from antenna i into all the correlator inputs and measuring the power spectrum Ei(f)Ei*(f) from each antenna in turn. The cross products for each pair of antennas, sqrt( Ei(f)*Ei*(f) * Ej(f)*Ej*(f) ) = abs( Ei(f) * Ej(f) ) is then used to correct the measured spectrum for the subsequent data. In principle this calibration includes everything except the phase of Ei(f) * Ej*(f) due to components in front of the point where the common signal is injected into the correlator. Since the system is relatively wideband before this point, only a slowly varying phase as a function of frequency should remain to be determined by the astronomical passband calibration. In practice the components which split the IF for the calibration may introduce some dispersion, and we also fit a slowly varying amplitude to the astronomical passband calibration. This calibration is made each time a new source is observed, so that it also includes the dispersion in the delay used at the start of each observation. 3.3) DELAY DISPERSION Each delay step may introduce dispersion and ideally we should correct the measured spectrum for the frequency response of each delay. The larger delays have a longer coherence in the uv-data and are most important to correct. This correction is best done on-line when the actual delays used for each antenna are known. On strong sources such as planets at short baselines we can see the measured spectrum change due to changes in the inserted delay. Since the delay is inserted at the IF frequency, the spectral change in the upper and lower sideband are the same at the same IF frequency. This offers the possibility of correcting, e.g. the upper sideband based on the spectrum obtained in a line-free lower sideband. This correction is coded into the MIRIAD uvcal task. The usb and lsb must first be corrected for the overall passband. This should be done by fitting as few parameters as possible; in particular it is best to fit the whole sideband with one continuous function to avoid introducing discontinuities between the correlator sections. 4) FLUX DENSITY CALIBRATION It is conventional to calibrate the correlated visibilities in intensity units (Jy). The flux density calibration is usually done by first determining the quasar fluxes in Jy. Since the quasars typically vary on time scales of days to months, this is done by interpolating the quasar fluxes from their measured values w.r.t. planet observations. The gains determined from the quasar observations then convert the measured visibilities to Jy. 4.1) ANTENNA TEMPERATURES The correlations are measured in units of antenna temperature (K) by calibrating the correlator output using signals from hot and cold loads placed in front of the receiver. The measured antenna temperatures are also corrected for atmospheric attenuation, so that they are equal to the signal that would have been received above the atmosphere. This calibration is made on-line every 500 seconds using the chopper wheel method. The gains of each sideband may differ due to different atmospheric opacities in the upper and lower sideband, or due to an unequal sideband response in the mixer. Since the antenna temperature calibration is a double sideband calibration, we correct the calibrated antenna temperatures based on an atmospheric model. The receiver calibration is described in more detail in the BIMA users guide p71. 5) ANTENNA GAINS Most of the gain is associated with each antenna and the atmosphere over each antenna. If we can determine the gains associated with each antenna, rather than with each baseline, then we need fit fewer parameters. For n antennas, there are n(n-1)/2 baselines, and so by using the data from all baselines to determine the antenna based gains we gain sqrt((n-1)/2) in SNR. Actually there are only n-1 phases which can be determined since only the phase difference between antennas is determined, so we gain a little more SNR. The output of the correlator is the product of the signals from each pair of antennae. Thus we can write: V' = gi*conjg(gj) * V + R .........................(3) We can solve for the antenna gains in a optimum way by performing a least squares fit to the calibrator data minimizing Sum{ [|gi*conjg(gj) V - V'|/sigma]**2} ............(4) Here gi, gj are the unknown antenna gains, V' is the observed source visibility, and V is the true source visibility. Sigma is the noise estimated from the system temperature, bandwidth, and integration time. Note that resolved sources as well as quasars can be used in the gain determination provided that a good model of the source structure is known (e.g. for planets). The gains are determined at intervals fixed by the averaging time in the summation. This is exactly the same algorithm as used for self-calibration. For the a-priori calibration of the data we use the a-priori positions and flux densities of the calibrators to estimate the true source visibility. For self-calibration we use a model of the source itself to estimate the true source visibility. 5.1 CLOSURE ERRORS AND CORRELATOR OFFSETS There may also be closure errors, Cij(t,f), and offsets Oij(t,f) associated with the correlator itself. These are usually small in a digital correlator, but may be significant in a wide bandwidth analog correlator. In general we can write: V'ij(t,f) = gi(t) * gj*(t) * pi(f) * pj*(f) * Cij(t,f) * Vij(t,f) + R(t,f) + O(t,f)..........................(5) where gi(t) is the gain and pi(f) is the frequency dependence for antenna i, and gj*(t) and pj*(f) represent the complex conjugate of the gains for antenna j. Correlator offsets are removed by phase switching the inputs to the correlator. Closure errors which are not a function of time and frequency can often be calibrated and removed from the data, but may be a function of the signal level and care should be taken with the calibration. If the closure errors are a function of frequency, then they can be included in the passband calibration, by making a baseline based passband calibration. If the closure errors are a function of time, then we must also determine the gains versus time for each baseline. 6) ATMOSPHERIC PHASE FLUCTUATIONS The atmosphere produces gain fluctuations on all time scales. Time scales longer than the quasar calibration interval will be included in the gain determined from the quasar observations provided that the source and the calibrator are close enough in the sky that they are sampling the same atmospheric fluctuation. More rapid gain variations are not removed by the a-priori calibration. Variations in refractive index change the phase of the signal arriving at each antenna. An interferometer measures the difference in the phase of the wavefront arriving at each antenna, so that the correlated phase noise will also depend on the spatial coherence of the atmosphere. Most atmospheric phase fluctuations will not be removed by the quasar calibration, and the data quality on strong sources is limited by atmospheric phase fluctuations rather than by receiver noise. The atmospheric phase fluctuations will also reduce the amplitude of the correlated signal if the integration time is longer than the coherence time of the atmosphere. 6.1) PHASE STRUCTURE FUNCTION The phase structure function D(b,T) = <(phi(x,t) - phi(x+b,t+T))**2> is often used to measure the interferometer phase fluctuations, where b is the interferometer baseline, and T is the time interval between measurements. For Kolmogorov turbulence D(b,T) is expected to scale as b**5/3 and the rms phase as ~ b**5/6. The empirically determined rms phase for good observing conditions at Hat Creek scales as baseline ** (0.7 +/- 0.1) . (Wright and Welch, 1989) The time scale of the fluctuations is ~ d/v where d is the size of a coherence patch and v is the wind velocity along the baseline. At mm wavelengths the phase fluctuations are mostly due to water vapour. Both the water vapour content and the turbulence increase in summer observing conditions, and the time scale of phase fluctuation is often shorter than the integration time leading to a loss of coherence on longer baselines. 6.2) CORRELATION WITH TOTAL POWER VARIATIONS Water vapour emission produces an increased emission of 1 K per 100 degrees of phase shift at 1-3 mm wavelength. If the variations in total power due to atmospheric fluctuations are larger than the variations due to gain instabilities, and variable ground radiation, then we can estimate the phase errors and correct the data. We have developed software tools to estimate the correlation between atmospheric phase and total power fluctuations, and correct the data. A receiver gain stability of about 10-4 is required, and the variations in total power from ground radiation must be smaller than about 0.2 K on the time scale of the atmospheric fluctuations (Bima memo no.3) 7) DOUBLE SIDEBAND CALIBRATION The atmospheric phase fluctuations are proportional to frequency. Thus the phase variations in upper and lower sideband closely track each other and both sidebands can be used to determine the gain function with a gain in SNR. In order to use both sidebands we must first remove the phase difference between the sidebands. [phi(usb)-phi(lsb) = 2*phi(lo2 + IF)]. This is expected to be a constant, or slowly varying function of time, and can be determined with high SNR since the phase of lo1, and most of the atmospheric fluctuations are removed when we take the difference. The instrumental gain versus time can be fitted to the vector average of both sidebands after removing the phase difference between the sidebands. In addition to the gain in SNR, using the same gain function for upper and lower sideband means that the remaining phase fluctuations in the upper and lower sideband are the same and a self calibration can use both sidebands to further improve the images. For example a strong signal in one sideband can be used to calibrate a weak signal in the other sideband. Note that the absolute position information is lost in selfcal, but the relative positions of features in both sidebands are preserved if a double sideband calibration is used. 8) SELF-CALIBRATION This is the same algorithm as used for the a-priori calibration of the antenna gains. For self-calibration we use a model of the source itself to estimate the antenna gains. (See Cornwell and Fomalont, 1989). Miriad supports multichannel selfcalibration. If the antenna gains are assumed to be independent of frequency, then a multichannel model can be used to determine the gains. If the a-priori calibration applied the same gain correction to both sidebands (see above), then multichannel models from both sidebands can be used simultaneously to determine the antenna gains. When observations of several fields are interleaved (as in the observing task MINT), then models of all the fields can be used to determine gain variations on time intervals longer than the cycle time to observe all fields. (BIMA memo 14) 9) CONTINUUM SUBTRACTION There are some problems with subtracting the continuum from channels containing both line and continuum emission. Since the array response (synthesized beam) is a function of frequency, the sidelobe pattern is different in line and off-line channels. If the difference in the sidelobe pattern is larger than the random noise on the images then some care must be taken in subtracting the continuum (See van Gorkom and Ekers, 1989). Several algorithms for continuum subtraction interpolate the structure of off-line channels to estimate the structure in line channels. This can be done either in the image or uv domain. One of the more successful methods is to subtract linear fits to the real and imaginary parts of the uv-data. If the line and continuum are in different correlator windows it becomes important that there be no discontinuities in the passband correction. i.e. it is best to fit a single function with as few parameters as possible to all windows, rather than separate functions to each window. CONCLUSIONS 1 - There is clearly a tradeoff between integration time on source, and time spent calibrating. For strong sources, or at short baselines, calibration errors become more important. 2 - The SNR on the gain calibration can be increased by fitting antenna gains to both sidebands simultaneously, by first correcting the uv-data for closure errors and wideband gains. 3 - The data can be corrected on-line for the instrumental frequency response due to the delay system, and filters in the hybrid correlator. 4 - The passband calibration should be fit as an antenna based function of frequency to all spectral windows simultaneously. 5 - The best passband normalization will depend on the gain stability. If the gain is varying rapidly with time, then normalization w.r.t. the amplitude and phase of the wideband channels in each record will give the best SNR. If the gain is more stable, then better SNR will be obtained by normalizing w.r.t. the time averaged gain function. In either case the SNR is improved by using a double sideband reference. FOR FURTHER READING Calibrating BIMA Data with MIRIAD: A Cookbook, Christine Wilson, Bima memo 16. Dispersion in the Coaxial Cables. W.J.Welch, Bima memo 4. Self-calibration. Cornwell, T., and Fomalont, E.B., 1989, Synthesis Imaging in Radio Astronomy, ASP Conference series. Vol 6, p185. A-Priori Self-Calibration. Melvyn Wright and Bob Sault, Bima memo 14. Phase Prediction from Total Power Measurements. Melvyn Wright. Bima memo 3. Multichannel calibration. Van Gorkom, J.H., and Ekers, R.D., 1989, Synthesis Imaging in Radio Astronomy, ASP Conference series. Vol 6, p341. Absolute Positions of SiO Masers, Wright,M.C.H., Carlstrom,J.E., Plambeck,R.L., and Welch,W.J., 1990, A.J. 99, 1299 Interferometer measurements of atmospheric phase noise, Wright, M.C.H., and Welch, W.J., 1989, IAU/URSI Symposium on Radio Astronomy Seeing, Wang and Baldwin Eds.