Ask HN: What are some good resources for learning about low level disk/file IO?

1

OSXFUSE

I lead a project that included shipping a filesystem driver and a virtual disk on Windows. What I did to learn the lower-level APIs, and perform initial testing on the driver, was write a "mirror" drive. The user-mode code pointed to a folder on disk, the driver made a virtual disk drive, and all reads and writes in the virtual disk drive went to the mirror folder. On Windows, you can implement something like that using Dokany, Dokan, or Winfsp. On linux, there's the Fuse API. On Mac, there's MacFUSE. <i>Even if you don't do a "mirror" drive, understanding the callbacks that libraries like Dokany, Dokan, Winfsp, and Fuse do helps you understand how IO happens in the driver.</i> Many IO methods provided in popular languages provide abstractions above what the OS does. (For example, the Windows kernel has no concept of the "Stream" that's in your C# program. The "Stream"'s Position property is purely a construct within the .Net framework.) https://dokan-dev.github.io/ https://github.com/dokan-dev/dokany Another place to start is the OS's documentation itself. For example, you can start with Window's CreateFileA function. This typically is what gets called "under the hood" in most programming languages when you open or create a file: https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea.

#Storage #Cloud Storage #OS & Utilities 31 social mentions

2

Dokany

I lead a project that included shipping a filesystem driver and a virtual disk on Windows. What I did to learn the lower-level APIs, and perform initial testing on the driver, was write a "mirror" drive. The user-mode code pointed to a folder on disk, the driver made a virtual disk drive, and all reads and writes in the virtual disk drive went to the mirror folder. On Windows, you can implement something like that using Dokany, Dokan, or Winfsp. On linux, there's the Fuse API. On Mac, there's MacFUSE. <i>Even if you don't do a "mirror" drive, understanding the callbacks that libraries like Dokany, Dokan, Winfsp, and Fuse do helps you understand how IO happens in the driver.</i> Many IO methods provided in popular languages provide abstractions above what the OS does. (For example, the Windows kernel has no concept of the "Stream" that's in your C# program. The "Stream"'s Position property is purely a construct within the .Net framework.) https://dokan-dev.github.io/ https://github.com/dokan-dev/dokany Another place to start is the OS's documentation itself. For example, you can start with Window's CreateFileA function. This typically is what gets called "under the hood" in most programming languages when you open or create a file: https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea.

#Cloud Storage #CD Image Creator #Web Service Automation 3 social mentions

3

Dokan

I lead a project that included shipping a filesystem driver and a virtual disk on Windows. What I did to learn the lower-level APIs, and perform initial testing on the driver, was write a "mirror" drive. The user-mode code pointed to a folder on disk, the driver made a virtual disk drive, and all reads and writes in the virtual disk drive went to the mirror folder. On Windows, you can implement something like that using Dokany, Dokan, or Winfsp. On linux, there's the Fuse API. On Mac, there's MacFUSE. <i>Even if you don't do a "mirror" drive, understanding the callbacks that libraries like Dokany, Dokan, Winfsp, and Fuse do helps you understand how IO happens in the driver.</i> Many IO methods provided in popular languages provide abstractions above what the OS does. (For example, the Windows kernel has no concept of the "Stream" that's in your C# program. The "Stream"'s Position property is purely a construct within the .Net framework.) https://dokan-dev.github.io/ https://github.com/dokan-dev/dokany Another place to start is the OS's documentation itself. For example, you can start with Window's CreateFileA function. This typically is what gets called "under the hood" in most programming languages when you open or create a file: https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea.

#Cloud Storage #Storage #CD Image Creator 2 social mentions

4

fio

Not specifically addressing your question, but when you get to the point of wanting to start doing some experiments you may find that 'fio' [1] is very handy. [1] <a href="https://github.com/axboe/fio">https://github.com/axboe/fio</a>.

#Hard Drive Tools #Utilities #Monitoring Tools 14 social mentions