(Abstrakt in Deutsch siehe unten)

Problems with USB-2 on linux

This is a worst case scenario. You store data on a USB-disk and data connection not only is slow but also the stored data is corrupt and reading one and the same data block gives sometimes wrong data back to you. And sometimes I did not noted it, because I did not get an error messages. The only hint that something was wrong was a "reset high speed USB device" message in the log file /var/log/messages and subsequent I/O errors. This happened with different PCs, USB devices or USB-PATA Adapters and different ATA-disks and USB-Sticks. Only few USB devices seem to work properly, but confidence to linux and USB-2 was gone. Reading seems to be ok, but after writing further reading and writing operations may(!) fail.

The only solution I had was unload the ehci_hcd module. And I tried every new linux kernel until 2.6.24 (Jan 2008) because I thought on a buggy USB-2 driver (ehci_hcd) which must be fixed sometimes. After trying kernel 2.6.24 without success I made more systematic experiments. And I found the problem!
It was the USB front connection of my PC. I had only errors if I used the front connections, but it was working fine if I used rear connections of the same PC. This is probably connected to the bad electric conditions for the 480MHz USB-2 signals which have to travel between rear and front connectors. Have a look to the shots:

rear USB connections inside USB connections
front USB female

Two additional connectors and more or less suitable cables cause the problem. Note the the front panel shows the "Hi-Speed" USB logo. Kabel was labeled as E164571 (UL) 2725 TYPE CM 75° 28AWG/2C for the rear connector.

Reproducing the problem

To reproduce the problem, you can use extension cords. Plug some of them together to get longer cables until you get failures for USB-2 transfers. I had to plug 11 (7m) USB extension cables together to create USB errors on my USB stick, but only 7 cables (5m) for my USB disk to get errors.

Silent data corruption?

Why linux commands do not report the error?
... 

Example log:
Feb  8 15:02:50 joerg kernel: usb 5-4: new high speed USB device using ehci_hcd and address 3
Feb  8 15:02:50 joerg kernel: usb 5-4: configuration #1 chosen from 1 choice
Feb  8 15:02:50 joerg kernel: scsi29 : SCSI emulation for USB Mass Storage devices
Feb  8 15:02:55 joerg kernel: scsi 29:0:0:0: Direct-Access     ST52520A 5585             60   PQ: 0 ANSI: 2 CCS
Feb  8 15:02:55 joerg kernel: sd 29:0:0:0: [sda] 5009760 512-byte hardware sectors (2565 MB)
Feb  8 15:02:55 joerg kernel: sd 29:0:0:0: [sda] Write Protect is off
Feb  8 15:02:55 joerg kernel: sd 29:0:0:0: [sda] Assuming drive cache: write through
Feb  8 15:02:55 joerg kernel: sd 29:0:0:0: [sda] 5009760 512-byte hardware sectors (2565 MB)
Feb  8 15:02:55 joerg kernel: sd 29:0:0:0: [sda] Write Protect is off
Feb  8 15:02:55 joerg kernel: sd 29:0:0:0: [sda] Assuming drive cache: write through
Feb  8 15:02:55 joerg kernel:  sda: sda1 sda2 sda3 sda4
Feb  8 15:02:55 joerg kernel: sd 29:0:0:0: [sda] Attached SCSI disk
Feb  8 15:02:55 joerg kernel: sd 29:0:0:0: Attached scsi generic sg0 type 0

dd if=/dev/sda of=/dev/null bs=1024k count=100  # 14s 7.3MB/s
if=/dev/sda bs=1024k count=100 | md5sum         # 14s 8cb98f18695a1e733b75d3d6df91dcce
/sbin/mkfs.vfat /dev/sda3                       #  1s $?=0
mount /dev/sda3 /media/sda3                     #  1s $?=0
dd if=/dev/zero of=/media/sda3/aaa bs=1024k count=32 # ok
umount /media/sda3                              #  5s $?=0 damn, now its working

unplug/plug to other connector (3th try)

Feb  8 15:18:07 joerg kernel: usb 5-1: new high speed USB device using ehci_hcd and address 6
Feb  8 15:18:07 joerg kernel: usb 5-1: configuration #1 chosen from 1 choice
Feb  8 15:18:07 joerg kernel: scsi32 : SCSI emulation for USB Mass Storage devices
Feb  8 15:18:12 joerg kernel: scsi 32:0:0:0: Direct-Access     ST52520A 5585             60   PQ: 0 ANSI: 2 CCS
Feb  8 15:18:12 joerg kernel: sd 32:0:0:0: [sda] 5009760 512-byte hardware sectors (2565 MB)
Feb  8 15:18:12 joerg kernel: sd 32:0:0:0: [sda] Write Protect is off
Feb  8 15:18:12 joerg kernel: sd 32:0:0:0: [sda] Assuming drive cache: write through
Feb  8 15:18:12 joerg kernel: sd 32:0:0:0: [sda] 5009760 512-byte hardware sectors (2565 MB)
Feb  8 15:18:12 joerg kernel: sd 32:0:0:0: [sda] Write Protect is off
Feb  8 15:18:12 joerg kernel: sd 32:0:0:0: [sda] Assuming drive cache: write through
Feb  8 15:18:12 joerg kernel:  sda: sda1 sda2 sda3 sda4
Feb  8 15:18:12 joerg kernel: sd 32:0:0:0: [sda] Attached SCSI disk
Feb  8 15:18:12 joerg kernel: sd 32:0:0:0: Attached scsi generic sg0 type 0
Feb  8 15:18:13 joerg kernel: usb 5-1: reset high speed USB device using ehci_hcd and address 6
                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~ bingo!
Feb  8 15:18:20 joerg last message repeated 5 times
Feb  8 15:18:20 joerg kernel: sd 32:0:0:0: [sda] Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK,SUGGEST_OK
Feb  8 15:18:20 joerg kernel: end_request: I/O error, dev sda, sector 5009696
Feb  8 15:18:20 joerg kernel: printk: 236 messages suppressed.
Feb  8 15:18:20 joerg kernel: Buffer I/O error on device sda, logical block 626212
Feb  8 15:18:25 joerg kernel: usb 5-1: reset high speed USB device using ehci_hcd and address 6
Feb  8 15:19:32 joerg last message repeated 3 times

dd if=/dev/sda bs=1024k count=100 | md5sum   # 29s 376e45b2400e4ebceb5337c556ccd780
-- no errors in /var/log/messages! ---
dd if=/dev/sda bs=1024k count=100 | md5sum   # 28s c209980e6f0fcfac09b14c4885924247
echo $?                                      # 0 (no error!)
Feb  8 15:24:11 joerg last message repeated 2 times # hah! we had an error! 

dd if=/dev/sda bs=1024k count=100 of=/dev/null # 33s $?=0
Feb  8 15:25:49 joerg last message repeated 4 times

dd if=/dev/sda bs=1024k count=100 | md5sum   # 29s 8cb98f18695a1e733b75d3d6df91dcce OK
dd if=/dev/sda bs=1024k count=100 | md5sum   # hangs?
Feb  8 15:27:09 joerg kernel: usb 5-1: reset high speed USB device using ehci_hcd and address 6
Feb  8 15:28:40 joerg last message repeated 2 times
Feb  8 15:29:51 joerg last message repeated 3 times
Feb  8 15:30:45 joerg last message repeated 3 times
Feb  8 15:30:45 joerg kernel: sd 32:0:0:0: [sda] Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK,SUGGEST_OK
Feb  8 15:30:45 joerg kernel: end_request: I/O error, dev sda, sector 119040
Feb  8 15:30:45 joerg kernel: Buffer I/O error on device sda, logical block 14880
Feb  8 15:30:45 joerg kernel: Buffer I/O error on device sda, logical block 14881
Feb  8 15:30:45 joerg kernel: Buffer I/O error on device sda, logical block 14882
Feb  8 15:30:45 joerg kernel: Buffer I/O error on device sda, logical block 14883
...
Feb  8 15:31:45 joerg kernel: usb 5-1: reset high speed USB device using ehci_hcd and address 6
After 211s we get a I/O-error from dd. Thats ok, but why so late?

... not finished ... Seems there is a problem with the error reporting back to the application? (Author: Joerg Schulenburg, Feb 2008)

Abstract in german:

Wenn Sie unter Linux (und Windows?) Probleme beim Speichern auf USB2-Geraeten haben, koennte es an schlechten USB Verbindungen (Stecker und Kabel) und damit verbundenen Uebertragungsfehlern liegen. Bei mir fuehrte das unter anderem zu unbemerkten Datenfehlern und jahrelanger Frustration, da ich USB-2 unter Linux nicht traute. Nun habe ich die Ursache ausfindig gemacht und will helfen, das Problem endgueltig zu beseitigen. (Joerg Schulenburg, Feb 2008)