Disk I/O timings: SCSI vs. IDE vs. USB2.0

The following table summarized a number of experiments copying a 1.7GB directory/file between 3 differents disks: SCSI, IDE and USB2.0. The desktop is a Dell Precision 610, a Xeon 500 MHz with 128MB memory, with the following 3 disks attached (see output dmesg below)
  1. /dev/sda: (8GB SCSI):  QUANTUM  ATLAS 10K 9WLS  on a Adaptec aic7890/91 Ultra2 SCSI adapter
  2. /dev/hda: (60GB IDE):  MAXTOR 6L060J3 on an "old" on-board UDMA/33 IDE controller, as well as a UDMA/66 controller
  3. /dev/sdb: (60GB IDE):  QUANTUM  FIREBALLP AS on a USB2.0 SiiG PCI card controller
The two IDE disks are actually identical MAXTOR 60GB drives, baught a few months apart, but one labeled as Maxtor, the other as Quantum. The experiments were done on a system with X running, but no other activity.

In the table below the "source" disk is the column on the left, the "destination" disk the row on top. For example the17th experiment, (17), copied a directory from hda to sda, and took 175". The measurements with hdparm of course give a nice upper limit to the throughput via that specific device.  A cache read was also performed. All 3 drives produced identical results ranging from 120-130 MB/s.

Note added in proof: a firewire card was added to the same machine, and the same drive used in the same configuration. Basically this combination gave me about 20MB/sec, instead of about 10MB/sec on USB2.0
        size=1700MB
 sda (SCSI)  ext3
 hda (IDE/UDM66)  ext3
 sdb (USB2.0)  ext2
sda (SCSI)
 hdparm:  23.0 MB/s



tar: 215"                 (1,14)



cp -a:  178"/ 161"     (12)
 cp -a: 231"         (13)

cp:  157"                 (6)
 cp:  98" / 76"             (2,5)
 cp: 201"                (4)




hda  (IDE)
hdparm: 22.0/ 40.0 MB/s


tar:   284" / 216"         (15)

cp -a:   175"           (17)
cp -a;:   225"       (14)

cp:          89"/ 76"     (16)
 cp:    264" / 186"        (3)
 cp:          203"       (18)


untar:   318" / 245"     (11)
sdb (USB2.0)

hdparm: 9.5 MB/s



tar:         387"        (10)



 cp -a:    484"         (19)

cp:     181"                (8)
 cp:   182"                     (9)
 cp:          293"         (7)





The figure below is the output of 'xload', as the above mentioned experiments were done. Lines are drown at loads of 1,2,3 and 4. Comparing for example (3) and (6) shows the clear difference that IDE (3) has on the load, where SCSI (6) keeps the load fairly small.

cpu load for various experiments


This system was outfitted with redhat 7.3, and it should be noticed that the (SiiG) PCI USB2.0 controller was nicely recognized, and correctly initialized upon booting the system. Here is the first bootlog, the one without the UDMA/66 controller.
Linux version 2.4.18-3 (bhcompile@daffy.perf.redhat.com) (gcc version 2.96 20000731 (Red Hat Linux 7.3 2.96-110)) #1 Thu Apr 18 07:37:53 EDT 2002
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 0000000007ffe000 (usable)
 BIOS-e820: 0000000007ffe000 - 0000000008000000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
 BIOS-e820: 00000000ffe00000 - 0000000100000000 (reserved)
On node 0 totalpages: 32766
zone(0): 4096 pages.
zone(1): 28670 pages.
zone(2): 0 pages.
Kernel command line: initrd=initrd.img hdc=ide-scsi root=/dev/sda2 BOOT_IMAGE=vmlinuz auto
ide_setup: hdc=ide-scsi
Initializing CPU#0
Detected 497.441 MHz processor.
Console: colour VGA+ 80x25
Calibrating delay loop... 992.87 BogoMIPS
Memory: 126092k/131064k available (1119k kernel code, 4588k reserved, 775k data, 280k init, 0k highmem)
Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)
Inode cache hash table entries: 8192 (order: 4, 65536 bytes)
Mount-cache hash table entries: 2048 (order: 2, 16384 bytes)
Buffer cache hash table entries: 4096 (order: 2, 16384 bytes)
Page-cache hash table entries: 32768 (order: 5, 131072 bytes)
CPU: Before vendor init, caps: 0383fbff 00000000 00000000, vendor = 0
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 512K
CPU: After vendor init, caps: 0383fbff 00000000 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU:     After generic, caps: 0383fbff 00000000 00000000 00000000
CPU:             Common caps: 0383fbff 00000000 00000000 00000000
CPU: Intel Pentium III (Katmai) stepping 02
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
POSIX conformance testing by UNIFIX
mtrr: v1.40 (20010327) Richard Gooch (rgooch@atnf.csiro.au)
mtrr: detected mtrr type: Intel
PCI: PCI BIOS revision 2.10 entry at 0xfca2e, last bus=2
PCI: Using configuration type 1
PCI: Probing PCI hardware
Unknown bridge resource 0: assuming transparent
PCI: Using IRQ router PIIX [8086/7110] at 00:07.0
isapnp: Scanning for PnP cards...
isapnp: Card 'CS4236B'
isapnp: 1 Plug & Play card detected total
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society NET3.039
Initializing RT netlink socket
apm: BIOS version 1.2 Flags 0x03 (Driver version 1.16)
Starting kswapd
VFS: Diskquotas version dquot_6.5.0 initialized
pty: 2048 Unix98 ptys configured
Serial driver version 5.05c (2001-07-08) with MANY_PORTS MULTIPORT SHARE_IRQ SERIAL_PCI ISAPNP enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A
ttyS01 at 0x02f8 (irq = 3) is a 16550A
Real Time Clock Driver v1.10e
block: 240 slots per queue, batch=60
Uniform Multi-Platform E-IDE driver Revision: 6.31
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PIIX4: IDE controller on PCI bus 00 dev 39
PIIX4: chipset revision 1
PIIX4: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings: hda:DMA, hdb:pio
    ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdc:DMA, hdd:pio
hda: MAXTOR 6L060J3, ATA DISK drive
hdc: Hewlett-Packard CD-Writer Plus 9500b, ATAPI CD/DVD-ROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
blk: queue c035e6a4, I/O limit 4095Mb (mask 0xffffffff)
hda: 117266688 sectors (60041 MB) w/1819KiB Cache, CHS=7299/255/63, UDMA(33)
ide-floppy driver 0.99.newide
Partition check:
 hda: hda1
Floppy drive(s): fd0 is 1.44M
FDC 0 is a National Semiconductor PC87306
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
ide-floppy driver 0.99.newide
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP, IGMP
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 8192 bind 8192)
Linux IP multicast router 0.06 plus PIM-SM
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
RAMDISK: Compressed image found at block 0
Freeing initrd memory: 256k freed
VFS: Mounted root (ext2 filesystem).
SCSI subsystem driver Revision: 1.00
kmod: failed to exec /sbin/modprobe -s -k scsi_hostadapter, errno = 2
PCI: Found IRQ 11 for device 02:0a.0
PCI: Sharing IRQ 11 with 02:06.0
PCI: Sharing IRQ 11 with 02:0e.0
PCI: Found IRQ 11 for device 02:0e.0
PCI: Sharing IRQ 11 with 02:06.0
PCI: Sharing IRQ 11 with 02:0a.0
scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.5
        <Adaptec aic7890/91 Ultra2 SCSI adapter>
        aic7890/91: Ultra2 Wide Channel A, SCSI Id=7, 32/253 SCBs

scsi1 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.5
        <Adaptec aic7880 Ultra SCSI adapter>
        aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs

  Vendor: QUANTUM   Model: ATLAS 10K 9WLS    Rev: UC81
  Type:   Direct-Access                      ANSI SCSI revision: 03
scsi0:A:0:0: Tagged Queuing enabled.  Depth 253
  Vendor: NEC       Model: CD-ROM DRIVE:466  Rev: 1.06
  Type:   CD-ROM                             ANSI SCSI revision: 02
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
(scsi0:A:0): 80.000MB/s transfers (40.000MHz, offset 31, 16bit)
SCSI device sda: 17783249 512-byte hdwr sectors (9105 MB)
 sda: sda1 sda2 sda3
Journalled Block Device driver loaded
PCI: Found IRQ 10 for device 00:0e.0
PCI: Sharing IRQ 10 with 00:11.0
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
00:0e.0: 3Com PCI 3c905 Boomerang 100baseTx at 0xdc80. Vers LK1.1.16
PCI: Found IRQ 10 for device 00:11.0
PCI: Sharing IRQ 10 with 00:0e.0
00:11.0: 3Com PCI 3c905B Cyclone 100baseTx at 0xdc00. Vers LK1.1.16
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
Freeing unused kernel memory: 280k freed
Adding Swap: 257032k swap-space (priority -1)
usb.c: registered new driver usbdevfs
usb.c: registered new driver hub
usb-uhci.c: $Revision: 1.275 $ time 07:43:07 Apr 18 2002
usb-uhci.c: High bandwidth mode enabled
PCI: Found IRQ 11 for device 00:07.2
PCI: Sharing IRQ 11 with 02:06.1
usb-uhci.c: USB UHCI at I/O 0xdce0, IRQ 11
usb-uhci.c: Detected 2 ports
usb.c: new USB bus registered, assigned bus number 1
hub.c: USB hub found
hub.c: 2 ports detected
usb-uhci.c: v1.275:USB Universal Host Controller Interface driver
PCI: Found IRQ 11 for device 02:06.0
PCI: Sharing IRQ 11 with 02:0a.0
PCI: Sharing IRQ 11 with 02:0e.0
usb-ohci.c: USB OHCI at membase 0xc88bc000, IRQ 11
usb-ohci.c: usb-02:06.0, NEC Corporation USB
usb.c: new USB bus registered, assigned bus number 2
hub.c: USB hub found
hub.c: 3 ports detected
PCI: Found IRQ 11 for device 02:06.1
PCI: Sharing IRQ 11 with 00:07.2
usb-ohci.c: USB OHCI at membase 0xc88cf000, IRQ 11
usb-ohci.c: usb-02:06.1, NEC Corporation USB (#2)
usb.c: new USB bus registered, assigned bus number 3
hub.c: USB hub found
hub.c: 2 ports detected
hub.c: USB new device connect on bus2/1, assigned device number 2
usb.c: USB device 2 (vend/prod 0xc0b/0xb001) is not claimed by any active driver.
ehci-hcd.c: $Revision: 1.1 $ USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-hcd.c: block sizes: qh 96 qtd 96 itd 160 sitd 64
PCI: Found IRQ 11 for device 02:06.2
PCI: Sharing IRQ 11 with 01:00.0
hcd.c: ehci-hcd @ 02:06.2, PCI device 1033:00e0 (NEC Corporation)
hcd.c: PCI latency reduced to max 34
hcd.c: irq 11, pci mem c890ac00
usb.c: new USB bus registered, assigned bus number 4
ehci-dbg.c: ehci_start hcs_params 0x2395 dbg=0 cc=2 pcc=3 ports=5
ehci-dbg.c: 02:06.2: ehci_start portroute 1 0 1 0 0 
ehci-dbg.c: ehci_start hcc_params 0x0002 caching 0 uframes 256/512/1024
ehci-hcd.c: reset 80002 cmd (park)=0 ithresh=8 period=1024 Reset HALT
ehci-hcd.c: init 10000 cmd (park)=0 ithresh=1 period=1024 HALT
ehci-hcd.c: USB 2.0 support enabled, EHCI rev 0.95
hcd.c: 02:06.2 root hub device address 1
hub.c: USB hub found
hub.c: 5 ports detected
usb.c: USB disconnect on device 2
ehci-hub.c: GetStatus port 1 status 0x1803 POWER speed=2 CSC CONNECT
ehci-hub.c: 02:06.2 port 1 high speed
ehci-hub.c: GetStatus port 1 status 0x1005 POWER speed=0 PE CONNECT
hub.c: USB new device connect on bus4/1, assigned device number 2
usb.c: USB device 2 (vend/prod 0xc0b/0xb001) is not claimed by any active driver.
Initializing USB Mass Storage driver...
usb.c: registered new driver usb-storage
scsi2 : SCSI emulation for USB Mass Storage devices
ehci-q.c: ep 2-in qtd token 800d8d40 --> status -32
hcd.c: giveback urb c7d81660 status -32
  Vendor: QUANTUM   Model: FIREBALLP AS      Rev: A1Y.
  Type:   Direct-Access                      ANSI SCSI revision: 02
Attached scsi disk sdb at scsi2, channel 0, id 0, lun 0
SCSI device sdb: 117266688 512-byte hdwr sectors (60041 MB)
 sdb: sdb1
WARNING: USB Mass Storage data integrity not assured
USB Mass Storage device found at 2
USB Mass Storage support registered.
EXT3 FS 2.4-0.9.17, 10 Jan 2002 on sd(8,2), internal journal
ad1848/cs4248 codec driver Copyright (C) by Hannu Savolainen 1993-1996
ad1848: No ISAPnP cards found, trying standard ones...
cs4232: set synthio and synthirq to use the wavefront facilities.
kjournald starting.  Commit interval 5 seconds
EXT3 FS 2.4-0.9.17, 10 Jan 2002 on ide0(3,1), internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS 2.4-0.9.17, 10 Jan 2002 on sd(8,1), internal journal
EXT3-fs: mounted filesystem with ordered data mode.
scsi3 : SCSI host adapter emulation for IDE ATAPI devices
  Vendor: HP        Model: CD-Writer+ 9500b  Rev: 1.06
  Type:   CD-ROM                             ANSI SCSI revision: 02
hdc: DMA disabled
parport0: PC-style at 0x378 (0x778) [PCSPP,TRISTATE,EPP]
parport0: irq 7 detected
ip_conntrack (1023 buckets, 8184 max)
(scsi1:A:5): 20.000MB/s transfers (20.000MHz, offset 15)
Attached scsi CD-ROM sr0 at scsi1, channel 0, id 5, lun 0
Attached scsi CD-ROM sr1 at scsi3, channel 0, id 0, lun 0
sr0: scsi3-mmc drive: 17x/40x cd/rw xa/form2 cdda tray
Uniform CD-ROM driver Revision: 3.12
sr1: scsi3-mmc drive: 24x/32x writer cd/rw xa/form2 cdda tray
  
Peter Teuben; 14-may-2002