Fragmentation - Julio Merino (jmmv.dev)

Reinstalled Mac OS X in multiple partitions, again

Past weekend, for some strange reason, I decided to dump all the MBP's hard disk contents and start again from scratch. But this time I decided to split the disk into multiple partitions for Mac OS X, to avoid external fragmentation slowdowns as much as possible. I already did such a thing back when the MBP was new. At that time, I created a partition for the system files and another for the user data. However, that setup was not too optimal and, when I got the 7200RPM hard disk drive six months later, I reinstalled again in a single partition. Just for convenience. But external fragmentation hurts performance a lot, specially in my case because I need to keep lots of small files (the NetBSD source tree, for example) and files that get fragmented very easily (sparse virtual machine disks). These end up spreading the files everywhere on the physical disk, and as a result the system slows down considerably. I even bought iDefrag and it does a good job at optimizing the disk layout... but the results were not as impressive as I expected. This time I reinstalled using the following layout: System: Mounted on /, HFS+ case insensitive, 30GB.Users: Mounted on /Users, HFS+ case insensitive, 50GB.Windows: Not mounted, NTFS, 40GB.Projects: Mounted on /Users/jmmv/Projects, HFS+ case sensitive, 30GB.Windows had to go before Projects so that the MBR partition table was constructed correctly; otherwise Windows failed to start after installation. The Projects partition holds those small files as well as the virtual machines. And Users keeps all the personal stuff such as photos, music and documents, which are mostly static. Using this layout, the machine really feels a lot faster. Applications start quickly, programs that deal with personal data such as iPhoto and iTunes load the library faster, and I don't have to deal with stupid disk images to keep things sequential on disk. However, the price to pay for such layout is convenience, because now the free disk space is spread in multiple partitions.

July 5, 2008 · Tags: <a href="/tags/fragmentation">fragmentation</a>, <a href="/tags/mac">mac</a>, <a href="/tags/macos">macos</a>
Continue reading (about 2 minutes)

How to disable journaling on a HFS+ volume

Mac OS X’s native file system (HFS+) supports journaling, a feature that is enabled by default on all new volumes. Journaling is a very nice feature as it allows a quick recovery of the file system’s status should anything bad happen to the machine — e.g. a power failure or a crash. With a journaled file system, the operating system can easily undo or redo the last operations executed on the disk without losing meta-data, effectively avoiding a full file system check.

April 9, 2007 · Tags: <a href="/tags/fragmentation">fragmentation</a>, <a href="/tags/hfs">hfs</a>, <a href="/tags/journal">journal</a>, <a href="/tags/macos">macos</a>
Continue reading (about 2 minutes)

CVS and fragmentation

First of all, happy new year to everybody! I've recently got a MacBook Pro and, while this little machine is great overall, the 5400 RPM hard disk is a noticeable performance bottleneck. Many people I've talked to say that the difference from 5400 to 7200 RPM should not be noticeable because:These 2.5-inch drives use perpendicular recording, hence storing data with a higher bit density. This means that, theorically, they can read/write data more quickly achieving speeds similar to 7200 RPM drives.Modern file systems prevent fragmentation, as described here for HFS+.To me, these two reasons are valid as long as you manage large files: the file system will try to keep them physically close and the disk will be able to transfer sequential data fairly quickly. But unfortunately, these ideas break when you have to deal with thousands of tiny files around (or when you flood the drive with requests from different applications, but this is not what I want to talk about today). The easiest way to demonstrate this is to use CVS to manage a copy of pkgsrc on such drives. Let's start by checking out a fresh copy of pkgsrc from the CVS repository. As long as the file system has a lot of free space (and has not been "polluted" by erased files), this will run quite fast because it will store all new files physically close (theorically in consecutive cylinders). Hence, we take advantage of the higher bit densities and the file system's file allocation policy. Just after the check out operation (or unarchiving of a tarball of the tree), run an update (cvs -z3 -q update -dP) and write down the amount of time it takes. In my specific tests, the update took around 5 minutes, which is a good measure; in fact, it is almost the same I got in my desktop machine with a 7200 RPM disk. Now start using pkgsrc by building a "big" package; I've been doing tests with mencoder, which has a bunch of dependencies and boost, which installs a ton of files. The object files generated during the builds, as well as the resulting files, will be physically stored "after" pkgsrc. It is likely that there will be "holes" in the disk because you'll be removing the work directories but not the installed files, which will result in a lot of files stored non-contiguously. To make things worse, keep using your machine for a couple of days. Then, do another update of the whole tree. In my specific tests, the process now takes around 10 minutes. Yes, it has doubled the original measure. This problem was also present with faster disks, but not as noticeable. But do we have to blame the drive for such a slowdown or maybe, just maybe, it is CVS's fault? The pkgsrc repository contains lots of empty directories that were once populated. However, CVS does not handle such entries very well. During an update, CVS recreates these empty directories locally and, at the end of the process, it erases them provided that you passed the -P (prune) option. Furthermore, every such directory will end up consuming, at least, 5 inodes on the local disk because it will contain a CVS control directory (which typically stores 3 tiny files). This continuous creation and deletion of directories and files fragment the original tree by spreading the updated files all around. Sincerely, I don't know why CVS works like this (anyone?), but I bet that switching to a superior VCS could mitigate this problem. A temporary solution can be the usage of disk images, holding each source tree individually and keeping its total size as tight as possible. This way one can expect the image to be permanently stored in a contiguous disk area. Oh, and by the way: Boot Camp really suffers from the slow drive because it creates the Windows partition at the end of the disk; that is, its inner part, which typically has slower access times. (Well, I'm not sure if it'd make any difference if the partition was created at the beginning.) Launching a game such as Half-Life 2 takes forever; fortunately, when it is up it is fast enough. Update (January 9th): As "r." kindly points out, the slower part of the disk is the inner one, not the outer one as I had previously written (had a lapsus because CDs are written the other way around). And the reason is this: current disks use Zone Bit Recording (ZBR), a technique that fits a different amount of sectors depeding on the track's length. Hence, outer (longer) tracks have more sectors allocated to them and can transfer more data in a single disk rotation.

January 7, 2007 · Tags: <a href="/tags/cvs">cvs</a>, <a href="/tags/fragmentation">fragmentation</a>, <a href="/tags/performance">performance</a>
Continue reading (about 4 minutes)

Posts: Fragmentation

Reinstalled Mac OS X in multiple partitions, again

How to disable journaling on a HFS+ volume

CVS and fragmentation

Featured software

Featured posts