Kernel Asynchronous I/O (AIO) Support for Linux

Overview

AIO enables even a single application thread to overlap I/O operations with other processing, by providing an interface for submitting one or more I/O requests in one system call (io_submit()) without waiting for completion, and a separate interface (io_getevents()) to reap completed I/O operations associated with a given completion group.

Support for kernel AIO has been included in the 2.6 Linux kernel.

What Works?

• AIO read and write on raw (and O_DIRECT on blockdev)
• AIO read and write on files opened with O_DIRECT on ext2, ext3, jfs, xfs

What Does Not Work?

• AIO read and write on files opened without O_DIRECT (i.e. normal buffered filesystem AIO). On ext2, ext3, jfs, xfs and nfs, these do not return an explicit error, but quietly default to synchronous or rather non-AIO behaviour (i.e io_submit waits for I/O to complete in these cases). For most other filesystems, -EINVAL is reported.
• AIO fsync (not supported for any filesystem)
• AIO read and write on sockets (doesn't return an explicit error, but quietly defaults to synchronous or rather non-AIO behavior)
• AIO read and write on pipes (reports -EINVAL)
• Not all devices (including TTY) support AIO (typically return -EINVAL)

Additional functionality is available HERE

• Buffered filesystem AIO, i.e. AIO read and write on files opened without O_DIRECT, on ext2, ext3, jfs, xfs, reiserfs.
• Does not include AIO fsync
• AIO read and write on pipes (from Chris Mason)

Q. Where can I find AIO libraries that utilize kernel AIO on Linux ?

Q. Is AIO functionality available for 2.4 kernels?

Reading About the Linux Kernel AIO Design

• Asynchronous I/O Support Linux 2.5, Ottawa Linux Symposium 2003
  Covers the AIO design for 2.6, including the filesystem AIO implementation that's currently in the -mm tree, and some performance results for O_DIRECT AIO using rawiobench.
  
• AIO Design Notes written at the time of the 2.4 AIO implementation, These are a little out-of-date, but provide some idea and background on various issues, alternative design approaches and considerations involved in an AIO implementation and describes the design of the 2.4 AIO implementation.
• Round 3: aio vs /dev/epoll compares epoll, AIO poll and AIO read behaviour. The full paper is included in the OLS 2002 Proceedings.

Future Work

• Performance Tuning
• Vector AIO

Benchmarks and Testing

• Chris Mason's aio-stress benchmark issues a stream of AIO requests to one or more files, where one can vary several parameters like I/O unit size, total I/O size, depth of iocbs submitted at a time, number of concurrent threads, type and pattern of I/O operations, etc. It reports the overall throughput attained and average io submission latency.
• rawiobench is another microbenchmark which can issue sequential or random disk I/O. It supports both synchronous and asynchronous I/O for O_DIRECT and buffered filesystem AIO and reports throughput and CPU utilization results.

• Daniel McNiel's (OSDL) aiocp test.
• fsx-linux is now enabled for AIO. Just compile it with -DAIO.

Performance Results

• Filesystem AIO aiostress results with and without fsaio patches