Frequently Asked Questions
Linux (page4)
Q16: With Extreme Performance under Linux all of the CPU utlization numbers (CPU
util, WIO, SYS, USR) show zeros?
A16: Linux may not install the system utility 'iostat' by default. For
Redhat, it is included in the sysstat package and must be installed for the
CPU utilization numbers to work. Look in Applications/System for the sysstat
package.
Q17: For Extreme Performance the CPU utilization is nearly 100% and WIO (Waiting
on IO) is 0?
A17: You are probably running Performance in FileIO mode. When using Raw
SCSI (uscsi, sg, etc) the ioctls block until the operation is complete. WIO
measures the time spent while blocked. Linux doesn't provide a WIO statistic
in its version of 'iostat' so it is left blank.
In FileIO mode we use polled Posix Asyncronous IO, which means that the read
or write operation is issued, and then we check the status of the IO and
reissue when it's complete. If more than 1 tag is specified, then multiple
read or write operations can be going at once.
Since it is an active polling loop, and we're interested in achieving the
highest performance, the loop never sleeps and runs as quickly as possible
to minimize the amount of time between IO completion and reissuing for that
tag.
Q18: Why can't I run FileIO mode on my disks for Linux?
A18: Linux file handles are not interchangable between different drivers.
When doing a bus probe we set up all of the devices controlled by the sg
(scsi generic) driver. Unfortunately, you can't open a handle to the sg
device and issue a filesystem read to this as you can on Solaris and
Windows.
In order to run with FileIO mode you'll need to set up File devices for the
program to operate on. These are binary files on mounted system disks that
get accessed as though they were SCSI devices. See Q13 for details.
Q19: Why are the performance numbers for Linux unrealistically high in FileIO
mode?
A19: Though Extreme Performance supports the ability to specify flags to
bypass buffering (O_RSYNC, O_DSYNC) for File devices, Linux only supports
O_DSYNC (write syncronization.) As a result, when running with Read and/or
Write Cache Enabled on Linux the performance numbers will be very high as
the performance basically amounts to memory copies to and from the OS file
buffer.
If you have specified Write Cache Disabled, then the O_DSYNC flag is used
when opening the file, and the OS buffering is bypassed, resulting in
numbers that reflect the performance capabilities of the underlying
hardware. It is not currently possible to bypass the OS buffering for Read
operations, which makes using these performance numbers for qualitative
tests questionable.
It may be possible to recompile the OS to dramatically reduce the size of
the buffers used so that they become less of a factor. We have not done so,
and can't help you with that, but it may be something you're interested in
investigating if you need FileIO numbers for Linux.
Q20: Why are your random performance numbers lower than IOmeter?
A20: Each tag in Xperf runs in its own thread. The random number
generator, unless told otherwise, inherits the random seed from the
launching process. With 4 tags all following the same "random" pattern
on the same disk, we were seeing a scenario where the first tag would
do the actual IO with head movement, but the following 3 (which were all
asking for the same data) would simply pull it out of cache. This resulted
in unrealistically high performance numbers. After fixing this bug by
having each tag using its own random seed, performance decreased as it should.
We suspect that IOmeter has each tag following the same sequence, and as a result
is showing bogus performance data which only gets better the more tags that get
added, rather than experiancing diminishing returns for added tags.
Q21: Where is the documentation?
A21: Documentation is available in pdf format in the install directory. Use
Acrobat Reader 4.0+ to view the documentation.
Additional Linux Information
Linux 2.4.1 or later is required so that we can access the improved generic
scsi interface ioctls (sg). Testing has been performed on Redhat 7.2.
Motif is Required to run. Download and compile from
www.opengroup.org or
purchase compiled binaries from a vendor.
Low level scsi access is ultimately controlled by the driver. We can't
guarentee that all drivers do everything correctly as Linux is a
work-in-progress and drivers are evolving on a constant basis.
To Turn on Direct IO (no kernal memory copy): echo > 1 >
/proc/scsi/sg/allow_dio
To change max xfer size (?) modify /proc/scsi/sg/def_reserved_size
|
|
|
