Tests of SPARC and disk performance on standard MIRIAD tasks A. Rudolph P. Teuben 8 July 1991 Abstract -------- Tests were run on three of the Maryland SPARC workstations (a SPARC2, an IPC, and a SPARC1), running three standard MIRIAD tasks, INVERT, CLEAN, and RESTORE. These tests were run both using each machine's local disk and using disks on the other machines, in order to test the relative performance of the machines, as well as the importance of working on a local disk. Also, running time versus map size was measured for the three tasks, and a test was run with another user on the machine. The results of these tests is presented. Introduction ------------ Since Maryland (and presumably eventually Berkeley and Illinois) has acquired three SPARC workstations for general use (LMC, a SPARC2, FORNAX, an IPC, and SMC, a SPARC1) with disks on each machine for data storage, the questions of the importance of using the fastest possible machine and of the importance of using a local or a non-local disk have yet to be quantified. Therefore tests were run on the three of the Maryland SPARC workstations running three standard MIRIAD tasks, INVERT, CLEAN, and RESTORE. These tests were run both using each machine's local disk and using disks on the other machines. In this way, the relative performance of the machines was tested, as well as the importance of working on a local disk. In addition a test was run with another user on the SPARC2, to test the effect of sharing the CPU. The data set consisted of 10 configurations of data, totalling about 2600 visibilites. INVERT was run to make 56 128x128 maps. These maps were then CLEANed at a 5% gain to 1.5 times the rms measured in the maps without signal (1 Jy) which gave between 250-450 CLEAN components per plane. Finally, RESTORE was run on these same maps. In the map-size test, INVERT was run to make 56 256x256 maps, CLEAN was run with the same gain (5%) to the same cutoff (1 Jy), which led to 600-1000 clean components per plane. With the exception of the multiple-user test, all tests were run with no other users on the system. The tests were run between 0800 and 0930 on a Monday morning. Therefore, the ethernet traffic was low, and these tests may not accurately reflect the total cost of working on a non-local disk. Comparison of Machines ---------------------- Table 1 shows the expired time for all the tests conducted, measured using the UNIX cshell built-in command "time". The numbers in parentheses are the ratio of each time to a "fiducial time", which is the time it took a task to run on a SPARC2, working with 128x128 maps, and no other users on the machine. As expected, the SPARC2 is twice as fast as the SPARC1, and about 66% faster than the IPC, which, in turn, is about 25% faster than the SPARC1. However, adding another user to the SPARC2 degrades the performance by a factor of 2-3 (depending on the task), so that it is no better than being a single user on a SPARC1, and worse than being a single user on an IPC. Increase in Running Time with Map Size -------------------------------------- To test the increase in running time with map size, 256x256 maps were made with the same data. INVERT took only 1.5 times longer (due to an overhead not related to map size), while CLEAN took 9.2 times longer, RESTORE took 5.6 times longer, and the three tasks together took 6.9 times longer (45 minutes v. 6.5 minutes). The factor of almost ten for CLEAN is a combination of a factor of 4 due to the larger area of the CLEAN box, and a factor of 2-3 due to the larger number of CLEAN components necessary to get to a given CLEAN cutoff. It is obviously more important to be the only user on the machine when running these larger tasks. Comparison of Local v. Non-local Disks -------------------------------------- Under the heading for each machine (see Table 1) are listed the expired times for the tasks to run on a local disk and on one or more non-local disks (the disk with the same name as the machine is local, e.g. /lmc is local to LMC). The greatest degradation, for all machines, occured running RESTORE (40-220%), the next greatest occured running INVERT (5-100%), and the least degradation occured running CLEAN (< 5%). These differences exist because RESTORE and INVERT are relatively I/O intensive, whereas CLEAN is a very CPU-intensive task (CPU efficiencies above 90% are common). The results also show that, the faster the machine, the worse the degradation caused by using a non-local disk. For the SPARC2, INVERT took 60% longer, and RESTORE took twice as long running on the non-local disk as on the local disk. On the IPC these tasks took 30 and 40% longer, and on the SPARC1, these tasks took 5% and 44% longer respectively. Since CLEAN requires the largest amount of CPU time, and was relatively unaffected by choice of disk, the overall degradation of the three tasks together was only 5%, 12%, and 37%, for the SPARC1, the IPC, and the SPARC2 respectively, much less than the loss of going to a slower machine. In order to test if the speed of the machine serving the non-local disk affected the performance, the tasks were run on the IPC using both a disk attached to the SPARC1 and to the SPARC2. For INVERT and CLEAN, the effect was negligible (2-3%), but for RESTORE, using the disk on the SPARC1 caused a 24% degradation compared to using the disk attached to the SPARC2. The reason for this degradation is not well understood. In any case, the effect on the overall performance of the three tasks was only 4%. Conclusions ----------- The overall conclusion of these tests is that it is much more important to use a faster machine than to use a local disk. However, the faster the machine, the greater the degradation of performance caused by using a non-local disk. Also, it is clear that one must be the only user on a machine to get the full benefit from the CPU. One other user on a SPARC2 means that each person has the equivalent of a SPARC1, and with more users the problem will get worse. Running on the SPARC2, another user causes CLEAN to take 9 minutes as opposed to 4.5 minutes, which already begins to be a barrier to running this task a number of times (often desireable to be able to optimize the CLEANing parameters). For 256x256 maps, CLEAN took 40 minutes, with only one user. Thus, when working with these larger maps, working on the SPARC2 as the only user, and with a local disk, becomes a necessity. TABLE 1 ------------------------------------------------------------------------------------------------------------------------------------ | | | Machine | LMC (SPARC2) | FORNAX (IPC) | SMC (SPARC1) | | | Disk | /lmc /smc | /fornax /lmc /smc | /smc /fornax | | | ------------------------------------------------------------------------------------------------------------------------------------ | | | INVERT | | | | | | 128x128 (no user) | 105 (1.0) 173 (1.6) | 175 (1.7) 230 (2.2) 236 (2.2) | 221 (2.1) 232 (2.2) | | | 128x128 (user) | 228 (2.2) ---- | ---- ---- ---- | ---- ---- | | | 256x256 (no user) | 150 (1.4) ---- | ---- ---- ---- | ---- ---- | | | ------------------------------------------------------------------------------------------------------------------------------------ | | | CLEAN | | | | | | 128x128 (no user) | 263 (1.0) 275 (1.0) | 430 (1.6) 435 (1.6) 444 (1.7) | 533 (2.0) 535 (2.0) | | | 128x128 (user) | 533 (2.0) ---- | ---- ---- ---- | ---- ---- | | | 256x256 (no user) | 2434 (9.2) ---- | ---- ---- ---- | ---- ---- | | | ------------------------------------------------------------------------------------------------------------------------------------ | | | RESTORE | | | | | | 128x128 (no user) | 25 (1.0) 55 (2.2) | 47 (1.9) 66 (2.6) 82 (3.3) | 57 (2.3) 82 (3.3) | | | 128x128 (user) | 93 (3.7) ---- | ---- ---- ---- | ---- ---- | | | 256x256 (no user) | 139 (5.6) ---- | ---- ---- ---- | ---- ---- | | | ------------------------------------------------------------------------------------------------------------------------------------ | | | TOTAL | | | | | | 128x128 (no user) | 393 (1.0) 537 (1.4) | 652 (1.7) 731 (1.9) 762 (1.9) | 811 (2.1) 849 (2.2) | | | 128x128 (user) | 854 (2.2) ---- | ---- ---- ---- | ---- ---- | | | 256x256 (no user) | 2723 (6.9) ---- | ---- ---- ---- | ---- ---- | | | ------------------------------------------------------------------------------------------------------------------------------------ Notes to the Table: All times are actual elapsed clock time in seconds. Numbers in parentheses are the ratio of the listed time to the fiducial time (a SPARC2 on a local disk making 128x128 maps, with no other users on the machine).