Revision as of 16:45, 22 June 2009 by Teuben (Talk | contribs)

Jump to: navigation, search



Data Reduction/User Notes

NOTE: To add a page of your own, edit this page and add a line
[[Your page name here]]
or click the internal link button: Ab This will bring the link up in red. If you click on the link, a new page will be created and you can start editing it.

Baseline Issues

MIRIAD Routines

This section contains PACS notes for Miriad, in particular for those maintaining their own MIRIAD version:

Peter will normally update the OVRO and CARMA machines, as well as the 32 bit version ('astroload miriad') and 64 bit version ('astroload -v 64 miriad') on the UMD machines. For anybody else, check the procedure below if that is what you do:

New/modified software

A summary of the software components modified for PACS:

  • gpbuddy: writes out (unwrapped) phaseatm from a gain solution (that came from selfcal) [V1.7 and up]
  • gplist: handle 23 ants (SZA files now use 23, CARMA still uses nants=15). Can also list subsets of ants now.
  • uvcal: has new options=atmcal to apply the atmphase to a dataset. Make sure you have a good freqatm value in your dataset (for now)
  • uvwide: SZA data do not have narrow flags, this program was updated for this. [25-nov or later]
  • bldelay: baseline based
  • panda: baseline based
  • maxdim: new program that just displays your miriad version and the MAX parameters. MAXBUF increased.
  • selfcal/mselfcal/mfcal: to handle the large number (around 10,000) integrations. maxdim.h also updated, see below. Roundoff issues were uncovered mid-March in (m)selfcal, please make sure you have that version.

Although selfcal works fine, mselfcal crashes on freeing up memory (apr 2, 2009), being looked into.

  • listobs: handle SZA data [V1.25 and up]

MIRIAD/SZA data issues

The procedure to convert SZA data into miriad format is still changing, here are some known issues. Most of these are to be resolved in the new filler that writes data from Dec 11 onwards:

Version 1.1 issues (17 Dec and after)

  • now have properly dimensioned systemp, though some data don't have systemp (e.g. pacs 19)
  • some data create an extra hierarchy data/eml/mirData/ in the current directory. Simple manually move it.

Version 1.1 issues (before Dec 17)

  • systemp is erroneously written with the wrong dimension. New version of listobs (V1.26) has been patched to look at wsystemp in this case.
  • There are no linelength data (phasem1), but this is a known issue and won't be added for a while.

Version 1.0 issues (before Dec 11)

  • Data not always time sorted (solved in 1.1). If the data is not time sorted, mfcal won't work. Rest should be ok. See comments on 3QSO below.
  • The antpos uv variable is not correct. (fixed in 1.1)
  • The freq variable is missing, listobs now works around that (fixed in 1.1)
  • There is no version keyword in the data. (fixed in 1.1)
  • Only wide band data are flagged (fixed in 1.1), you must run
      uvwide vis=SZA narrow=t
  • missing NOISE source data.(zrcx252.Arp220_12CO.1.mir.tar)
  • (now resolved) prior to about Dec 3, data had the wrong sign in the U-V coordinates. You can solve that by using scale < 1 in gpbuddy (V1.9). To check if your data has the wrong sign of the phases, check a similarly oriented antenna pair using UV list and see in which quadrant the UV coordinates are, viz.
 uvlist vis=SZA select='ant(21)(23),time(15:00,15:01),-source(noise)'
 uvlist vis=CAR select='ant(2)(4),time(15:00,15:01),-source(noise)'
  • systemp's are constant, but this is a feature. Need to double check if the array is properly written as systemp(nants,nwide), and not the other way around.
  • 3QSO track (zrct015.3QSO_3C273.3mm.1.mir).
    • no wide band data
    • first few integrations after a source change are bad.

There are actually two versions of the 3QSO dataset (zrct015.3QSO_3c273.3mm.1.mir.tar.gz and zrct015.3QSO_3C273.3mm.1.mir.tar.gz) that have a 30MB difference in size. The latter (upper case C) one is smaller and sorted in time, but has lost the wide band data (a feature of uvsort). In the following procedure

  uvwide vis=zrct015.3QSO_3c273.3mm.1.mir narrow=t
  uvsort vis=zrct015.3QSO_3c273.3mm.1.mir out=sza1
  uvwide vis=sza1 out=sza2 nwide=16

all flags are correct, including the bad data after a source change.

Reminder on updating MIRIAD (old style)

It is assumed you have a CVS enabled release of MIRIAD (i.e. you have files such as $MIR/CVS)

  • MAXDIM: two header files describing maximum sizes were updated: $MIRINC/maxdim.h and $MIRINC/maxdimc.h. Edit this files and modify the MAXBUF parameter in both files to at least 10000000 (4 times the amount is the memory used by programs that use MAXBUF).
 $EDITOR maxdim.h  maxdimc.h

and if to properly update the CVS status of these files (they changed on Dec 4, 2008) a one time is needed to old MIRIAD installs:

 mkdir tmp
 mv maxdim.h  maxdimc.h tmp
 cvs update
 mv tmp/max*h .
  • CVS: always query your source tree first and look for C (conflicts) and M (locally modified) files. The U (to be updated) files will be new ones you will get:
 cvs -nq update

and if happy with this, issue the real update

 cvs update
  • Old style updates: (you can give these commands from any directory and it assumed you have write permission in $MIR)
 mir.install subs prog

for a blanket re-install of the library and programs. You need this if e.g. maxdim.h was changed, and/or your miriad does not use shared libraries. This is also the safest way to update. On most computers this will only take 2-3 minutes. If however you know exactly what the impact of a new file is, you can pick any of the following examples, which will take 2-3 seconds:

 mir.subs fitsio.for uvio


 mir.prog itemize gpbuddy

Notice a file extension is optional, and multiple arguments can be given. Watch the output carefully for fatal compile or link errors.

Either way, a quick way to check if you had no disastrous compile issues, count the number of binaries in $MIRBIN, there should be about 400:

 ls $MIRBIN | wc

The output of a single mir.prog cgdisp command should be clean, and not give any compile or linking problems. If so, you may have either corrupt miriad library, or a misconfigured system to begin with.

  • New style updates: (these can take 10 minutes)
 cd $MIR
 make install


  • uvflag:
   uvwide vis=$VIS narrow=t

fixes the narrow band flags by copying them from the wide band flags. This issue will probably disappear when the SZA data filler is updated to write the narrow flags directly.

  • puthd: manually need to put freqatm in the dataset that came out of gpbuddy (phaseatm needs it in uvcal). Ideally selfcal would write the freq0 variable. E.g.
   puthd vis=$VIS/freqatm value=30.1 type=double
  • gpbuddy time ranges: if carma data (vis=) and sza data (vis2=) do not share the same time range, erroneous phases can be applied. Sampling can be slighly different, but there are caveats to using a long interval for phaseatm: UV variables only remember the past, selfcal gains are interpolated.

Memos and papers

Draft material and figures

The summer 2009 PACS analysis is maintained on Google Sites

Personal tools