Thursday, March 14, 2013

JPSoftware's Take Command and OpenAFS

I have been a user of Rex Conn's replacement command processors since the early days of 4DOS.  When I switched to OS/2 and began work on OS/2 C-Kermit, 4OS2 was there for me.  When I added REXX language support to OS/2 C-Kermit, 4OS2 added it as well.  When I moved to Windows NT, there was 4NT waiting for me.  In 2003 I began my work on OpenAFS for Windows (WinAFS) which at the time was implemented as a locally SMB server proxy to the AFS name space.  Before I started work on the WinAFS client, the only method of accessing the AFS name space was by use of Windows drive letter mappings.  It wasn't possible to consistently access the AFS name space via a UNC path.  It wasn't until the OpenAFS 1.3.66 release in July 2004 that it became possible to live entirely in a UNC \\AFS\cellname\path\ world except that the Microsoft command processor (cmd.exe) does not permit UNC paths to be the current directory.  4NT on the other hand supported UNC paths as the current directory for years and it was a natural fit.  Drive letter mappings suddenly became no longer necessary for my day to day activities.

For those readers that are not long time AFS users there are some important things to understand about the AFS name space.  Unlike a Windows file share, the UNC path \\server\share\ does not refer to a single on-disk volume on the specified machine.  Instead with AFS UNC paths \\afs\cell\ refers to the root directory of a volume named root.cell in the specified AFS cell.  AFS UNC paths are location independent and do not signify on which physical machines the data is stored.  In fact, root.cell is in most cases a geographically replicated volume. In addition to directories and files, AFS supports mount points and symlinks as first class file system types.  An AFS mount point is an object that refers to the root directory of another AFS volume and symlinks can refer to any absolute or relative file path.

The AFS name space can therefore be viewed as a directed graph of volumes joined to other volumes where each volume contains a directory tree.  Volumes can be either read/write or read-only snapshots of a read/write volume.  Volumes can be assigned quotas or can be permitted to grow to fill the entire partition on which they are stored.  AFS volumes can be migrated from server to server while in use and the amount of free space can change as a result of the volume being moved.  The AFS name space is therefore a challenge to use when it is accessed via the SMB protocol.

SMB file shares were designed prior to the existence of NTFS Junctions and NTFS Symlinks (added in Vista and Server 2008).  The assumption is that there is only one volume on one partition located at the other end of a UNC path.  Obtaining the free space is most often performed using GetDiskFreeSpace which can only refer to root directories and not GetDiskFreeSpaceEx which can refer to arbitrary paths.  Even the MSDN documentation for these APIs states that the reason to use the Ex version is to avoid unnecessary arithmetic whereas the most important reason for using the Ex version in my opinion is that it works with complex name spaces constructed by NTFS junctions and AFS mount points.

Since the AFS name space is made up of a potentially infinite number of volumes joined together via mount points and volumes can sometimes be read/write and other times be read-only, how should the WinAFS SMB server respond when it is asked to report the total disk space and total free disk space?  Its impossible to provide an accurate value for either of these.  As a result the AFS SMB server would simply lie.  It would report an arbitrarily large number for the partition size and the free space.  Free space was always reported even when there was absolutely none.

Which brings us back to JPSoftware and 4NT.  While it wasn't possible for arbitrary volume information to be obtained via the Win32 API, the AFS fs command obtains this information via the afs path ioctl interface.  In September 2005 Rex Conn added OpenAFS specific knowledge and functionality to 4NT 7.0:
  1. The command parser understands UNIX style inputs /afs/ and automatically converts them to UNC notation \\afs\\user\jaltman when the first component matches the AFS "NetbiosName".
  2. The command language contains @AFSCELL, @AFSMOUNT, @AFSPATH, @AFSSYMLINK, @AFSVOLID, @AFSVOLNAME functions which operate on paths and return AFS specific data.
  3. Free space computations use AFS volume information so it is accurate even when the Win32 GetVolumeInformation() call executed over SMB would not be.
Over the last five years as the AFS Redirector has been developed 4NT (now called Take Command) has been a constant companion.  One of my favorite features of Take Command directory listings is its awareness of Reparse Points.  For example:
As you can see, directory listing expand the target of NTFS Junctions and Symlinks providing the target information.  I have for the longest time wanted this behavior for AFS.   Unfortunately, up until a late TC 14.03 build, Take Command did not understand how to parse the AFS Reparse Point data.  Now that it does we get the same useful output:

Although not shown, symlink to file targets are displayed as well.
With the release of Take Command 15.0 and OpenAFS 1.7.22 the circle has now been completed.  Not only can Take Command display AFS mount point and symlink targets, but Take Command's MKLINK command can be used to create symlinks to both files and directories, and the DEL and RMDIR commands can be used to remove them.

Take Command's GLOBAL command can either cross [/J] or not cross [/N] junctions as specified.

Finally, Take Command properly uses GetVolumeInformationByHandle() to obtain volume information.  As a result the built-in AFS functions operate even when AFS is accessed via an NTFS directory symlink.

I recommend Take Command for any user of OpenAFS that relies upon the command shell.

For further information on Take Command visit the JP Software web site at

No comments: