The recent push to get OS-X ready for release has included an impressive effort to get its foundation, the Darwin kernel, ready to go. This work was recently released to the public both as programming source code and in a functional package as Darwin 1.0. In the following paragraphs, I'll take a look at some of the more significant aspects of this release.

Getting Darwin

The new release of Darwin, version 1.0, became available about a month ago. The initial release could only install on New World ROM machines (anything later than a beige G3). A few weeks after that, the 1.02 release followed. In addition to adding updated versions of a couple of pieces of software, this release allowed installation onto the beige G3s, as well as unsupported installations onto all PCI PowerMacs, including clones. The installer is available as a single, 130MB file or in 10MB segments; either leaves a single disk image with the installer on your desktop. The installer won't work off this image, though, so the files have to be copied onto a local disk to work.

Earlier releases of Darwin required a free hard disk, as the installer wiped out and reformatted it as a single UFS Darwin partition of set size and put the remaining space into a single HFS+ partition. The new installer is significantly more civilized. It will install onto any HFS+ partition on either SCSI or IDE hard disks (though note that it doesn't include drivers for most SCSI PCI cards, so not every SCSI disk will work). The installation wipes that partition clean, but leaves the rest of the disk intact; the installer can even be run off a second partition on the same disk. Note that the installer only works properly with disks formatted by Apple's Drive Setup, the latest version of which is thoughtfully included. Those who have used third-party formatting software will still have to wipe their drive clean.

The installer works fairly quickly and painlessly; with the above conditions met and the Mac started with extensions off, installation is likely to go off without a hitch. The installer also includes a utility which lets you select a startup partition. Choose the Darwin partition, and your Mac will automatically boot into that unless the option key is held down. The utility also allows selecting a partition with a traditional MacOS system, letting you boot into that without intervention.

How has Darwin evolved:

There are other significant changes in this version of Darwin that set it apart from its predecessors. Most significantly, this is the first release to use the 3.0 version of the Mach microkernel, the one that will serve as the foundation for OS-X (previous versions used 2.5, which was the basis for NeXTStep and OS-X server). In addition, the release marks the end of NeXT's DriverKit as the basis for writing drivers to communicate with the hardware that Mach is running on. In its place is I/O Kit, which will provide the structure for drivers in OS-X. I/O kit works in parallel with a new and flexible method for generating extensions to the kernel that can be readily loaded and unloaded dynamically. This makes adding drivers and changing things like network configurations easier, and is absolutely necessary for hot-swappable devices such as USB and FireWire hardware. All these changes have been available to paid members of Apple's Developer program, but this is the first time these technologies have been made available to the general, non-paying public. Also significantly, Darwin simply ignores hardware that it doesn't know how to work with, so that the drives on my Adaptec SCSI card are invisible to it, but the presence of the card isn't enough to stop Darwin from booting. This is critical, as it will allow people to test drivers for hardware from within Darwin, rather than requiring separate development and test machines.

The last addition that stands out is the fact that all of the compiled software is in the form of a fat binary. This does not mean that Apple's trying to support outdated hardware based on Motorolla's 68K processor, as was the case with software packaged as a fat binary for the traditional MacOS. Rather, this software contains the instructions necessary for it to work on both PowerPC and Intel compatible x86 processors. A lot of speculation went in to guessing whether Apple was hoping to use this to get the MacOS onto Intel hardware. Although this is possible in the long term, more immediately it could greatly increase the number of open source developers looking at and working with Darwin code, as most open source development takes place on x86 based hardware.

Making it available and making it work, however, are two different things. Several things are needed before reports of Darwin running on Intel hardware will make the news. To begin with, all computers have to read critical information of specific regions of the hard disk before they can begin loading the operating system, and no mechanism exists to put this information on a disk in an Intel computer at the moment. This information has to inform the computer how to load the OS into memory. Apple's currently reworking the PowerPC version of this software, so there won't be a lot of effort expended here before the updated version becomes publicly available. Finally, all the drivers that help Darwin communicate with the hardware it's on have been written with Apple components in mind. Although some of these components are shared with x86 based machines (ATI sells a lot of graphics chips to the other guys, too), a lot of drivers will have to be written before anything can be done with Darwin on the Intel side. Getting all of these problems sorted out may take a while, but having them solved will be critical for getting a wider community of developers involved in this work.

What's with Darwin?

Darwin is currently a very bare-bones Unix style operating system. You get booted into a slow-scrolling, text-based screen with access to little more than Unix utilities available to you. It can handle a static IP address or configure itself via DHCP, but doesn't currently support modem connections to the outside world. Included in the software package are the GNU compiler and debugger, versions of which are used by other open source operating systems. In theory, these would allow just about any Unix utility to be ported to Darwin. In practice, however, the effort involved in porting can range from trivial to overwhelming, depending on a variety of factors. The developer mailing list has had some reports of success, and others of running into significant roadblocks. Either way, as a single text based terminal is a pretty limited operating environment, and work within it will probably be limited to Unix afficianados for the near term.

Yet another interesting thing about this release, though, is that you don't have to work within its environment in order to work with it. As mentioned above, Darwin now installs onto an HFS+ partition that gets mounted on the desktop by default. Conversely, other HFS partitions are now visible within Darwin. Thus, if your only access to the internet is by modem, you can download necessary files in the MacOS, and then have access to them from within Darwin. Perhaps more significantly, you can have access to the Darwin files and directory structure from within the MacOS, allowing you to familiarize yourself with things before venturing into the world of the text terminal. In fact, you don't even have to install Darwin to poke around in its directories; the installer includes the full directory structure in a disk image in its "configurations" folder. Mount that, and you can examine the Darwin directory organization to your heart's content.

And that organization is a weird beast, a hybrid of Unix and the upcoming OS-X structure. The other drives visible within Darwin now appear as files on the root level of the Darwin partition; although they appear to be aliases, they are not recognized as such by the 8.6 Finder. Some of the critical Unix directories, like /etc and /usr, also appear to be aliases; opening these files with a text editor reveals the path to their OS-X replacements, which are now in the /private directory. Also lurking in /private is a file that acts as swap space for virtual memory, something which is normally its own partition on Unix systems. This is a more flexible system, allowing for installation without repartitioning a disk to add the swap space and letting more swap space be created as needed. Unfortunately, it's balanced by virtual memory suffering from congestion and fragmentation on a heavily used disk.

Another interesting exercise this setup allows is to look for files on the Darwin disk with Sherlock. Wondering if a specific Unix program is present? Just fire up Sherlock and find out. Since a lot of the documentation for the software is being made in HTML, you can simply search for files with a ".html" in the name; doing so gave me access to the entire manual for the Apache web server with web browsers that aren't even available for OS-X, much less Darwin. For entertainment, index for find by content, then go looking for words like "internal" and "confidential" and you'll find all sorts of evidence that many programmers at Apple never expected their work to wind up being open sourced.

The last interesting thing that this allows is for people who have no knowledge of Unix to start programming for OS-X now, within the current MacOS. Most Unix software reads its preferences from structured (and, in many cases, annotated and documented) text files. By programming a GUI editor for one of these text files in an OS-X compatible development environment (and remember, things as simple as AppleScript will work in OS-X, and carbon-based REALbasic has been demonstrated), you can make Unix software accessible to people who never intend to touch a command line.

Where's Darwin headed?

When I wrote this in early May, I concluded that the two main things that were inhibiting Darwin's use are its inability to run on Intel hardware and its lack of a windowing system. Apparently, some of the Apple employees who work on Darwin felt the same way. In a posting to the Darwin mailing lists, they announced that their presentation at the World Wide Developer's Conference included both these things. Booting on Intel went off without a hitch in the demo (though there is only support for very specific hardware at the moment). The windowing system shown was the Unix standard, X Windows, which can work either on the host machine or over a network. It wasn't clear which of the two was demonstrated. The second would only be useful if a user has a network with a second machine that runs a X Window client, while the first would allow all users to benefit from being able to work in multiple windows and with all the X-based software out there. Hopefully, all this will be made clear to non-developers such as myself after the conference ends and everyone involved catches up on the sleep they missed getting things ready for presentation.

In the mean time, developers outside of Apple are also getting busy. Several bugs have been caught and squashed, and software is getting ported both to and from other Unixes, such as Linux. A couple of people have expressed interest in getting work on drivers going, but it's not clear whether that will get moving before Apple releases its own drivers with the OS-X public beta. The MkLinux developers are watching kernel development closely; David Gatwood, a key figure in the MkLinux camp, has stated that they intend to switch from the previously used version of Mach to Darwin, and port it back to NuBus based PowerMacs. This will both spread Darwin to earlier machines than Apple plans to and allow MkLinux to stay current with Apple's hardware offerings from the moment they are released. It also raises the possibility that OS-X users will be able to run both a BSD and a Linux environment in addition to the MacOS.

With Darwin, Apple's also making a lot of its work available to the Open Source community developers. They're working to bring their compiler in line with the current GNU release, which would undoubtedly bring significant PowerPC optimizations and Alti-Vec enabling to the world at large. No other kernels currently support HFS+, but everyone now has access to the code to do so. A full implementation of AppleTalk is available for all to see. It's not clear at this point whether the coders working on other OSes will take advantage of this information, but it is clear that Apple's given away a significant amount of information.

Darwin 1.0 is a significant release any way it's looked at. It doesn't have a lot there for current Mac users who just want email and word processing, but nearly everyone else should find something there to intrigue them.