Files On-Demand support

The ShellBoost Files On-Demand feature can be used without creating a Shell Namespace Extension. However, the two techniques can be combined to have a Shell Namespace Extension that displays on-demand files and folders.

ShellBoost supports the Windows 10 Files On-Demand feature (starting with  1709, or “Fall Creators Update”) with the following architecture:

Files On-Demand support - Picture 41

OnDemandSynchronizer is a class that sits between the Windows File System (NTFS only) and an implementation of the ShellBoost’s IRemoteFileSystem interface:

/// <summary>
/// Represents an external file system resources.
/// A remote resource can be a file or a directory.
/// </summary>
public interface IRemoteFileSystem
{
    /// <summary>
    /// Enumerate the remote resources.
    /// </summary>
    /// <param name="context">The operation context.</param>
    /// <param name="parentPath">The directory resource path.</param>
    /// <returns>A list of remote resources</returns>
    IEnumerable<IRemoteResource> EnumResources(RemoteOperationContext context, string parentPath);
    
    /// <summary>
    /// Deletes a remote resource.
    /// </summary>
    /// <param name="context">The operation context.</param>
    /// <param name="path">The resource path.</param>
    void DeleteResource(RemoteOperationContext context, string path);
    
    /// <summary>
    /// Gets a remote resource information.
    /// </summary>
    /// <param name="context">The operation context.</param>
    /// <param name="path">The resource path.</param>
    /// <returns>An instance of the IRemoteResource type.</returns>
    IRemoteResource GetResource(RemoteOperationContext context, string path);
    
    /// <summary>
    /// Updates a remote resource information.
    /// </summary>
    /// <param name="context">The operation context.</param>
    /// <param name="path">The resource path.</param>
    /// <param name="resource">The resource information.</param>
    /// <param name="properties">The extra resource properties.</param>
    void UpdateResource(RemoteOperationContext context, string path, IRemoteResource resource, IReadOnlyDictionary<string, object> properties);
    
    /// <summary>
    /// Uploads a remote resource content from a stream.
    /// </summary>
    /// <param name="context">The operation context.</param>
    /// <param name="path">The resource path.</param>
    /// <param name="input">The input stream.</param>
    void UploadResource(RemoteOperationContext context, string path, Stream input);
    
    /// <summary>
    /// Downloads a remote resource content to a stream.
    /// </summary>
    /// <param name="context">The operation context.</param>
    /// <param name="path">The resource path.</param>
    /// <param name="offset">The content offset.</param>
    /// <param name="count">The content byte count.</param>
    /// <param name="output">The output stream.</param>
    void DownloadResource(RemoteOperationContext context, string path, long offset, long count, Stream output);
 
    /// <summary>
    /// Creates a remote directory resource.
    /// </summary>
    /// <param name="context">The operation context.</param>
    /// <param name="path">The directory path.</param>
    void CreateDirectory(RemoteOperationContext context, string path);
 
    /// <summary>
    /// Gets a property value by its name. This provides a simple extension mechanism to the Remote File System implementation.
    /// </summary>
    /// <param name="name">The property name.</param>
    /// <param name="value">The value.</param>
    /// <returns><c>true</c> if the property was read, <c>false</c> otherwise.</returns>
    bool TryGetPropertyValue(string name, out object value);
}

 

/// <summary>
/// Defines a remote resource. A resource is a file or a directory.
/// </summary>
public interface IRemoteResource
{
    /// <summary>
    /// Gets or sets the display name.
    /// </summary>
    /// <value>The display name.</value>
    string DisplayName { get; set; }
    
    /// <summary>
    /// Gets or sets the resource content type.
    /// </summary>
    /// <value>The type of the content.</value>
    string ContentType { get; set; }
    
    /// <summary>
    /// Gets or sets the resource content length.
    /// </summary>
    /// <value>The content length.</value>
    long ContentLength { get; set; }
    
    /// <summary>
    /// Gets or sets the resource ETAG.
    /// </summary>
    /// <value>The ETAG.</value>
    string ETag { get; set; }
    
    /// <summary>
    /// Gets or sets the resource last write time in UTC coordinates.
    /// </summary>
    /// <value>The last write time in UTC coordinates.</value>
    DateTime LastWriteTimeUtc { get; set; }
 
    /// <summary>
    /// Gets or sets the creation time in UTC coordinates.
    /// </summary>
    /// <value>The creation time  in UTC coordinates.</value>
    DateTime CreationTimeUtc { get; set; }
 
    /// <summary>
    /// Gets or sets the resource attributes.
    /// The Directory file attribute values determines if the resource is a file or a directory.
    /// </summary>
    /// <value>The attributes.</value>
    FileAttributes Attributes { get; set; }
}

Note the term remote may be misleading. You can implement IRemoteFileSystem with anything, “remote” or not. The AmalgaDrive sample contains an implementation of IRemoteFileSystem that’s based on WebDAV.

The OnDemandSynchronizer instance acts as a proxy between the Windows and the “remote” file system, and handles the following actions. In most cases, the synchronizer will do that periodically. The SyncPeriod read/write property defines the period. If set to 0, synchronization will stop. The synchronizer doesn’t work much on events to make sure synchronization still works if its process is stopped and restarted later:

Registration to the Windows File System (when running) and to the Windows Shell (in the registry). This should be done at install time, and uninstall time only.

New item or new folder.

Deleted item or folder.

Modified item or folder. On the file system side, the synchronizer will use dates and size. On the remote file system side, the synchronize uses IRemoteResource’s ETag value. This is an arbitrary string that uniquely identifies a resource in a directory. In the AmalgaDrive sample, the WebDAV class implements ETAG using dates and size.

Download and Upload from and to the remote file system.

Registration

Once you have an implementation of the IRemoteFileSystem interface, you must register a directory as being a “sync root”. This requires talking to the Windows File System and to the Windows Shell. This sync root will be the top of tree of on-demand files and directories. You register the sync root using ShellBoost’s OnDemandSynchronizer class:

// OnDemandRegistration is a congifuration class. You can keep as is for default configuration.
// myDirectoryPath is the sync root path.
OnDemandSynchronizer.EnsureRegistered(myDirectoryPath, new OnDemandRegistration());

You can also unregister a sync root like this, but you should only do that when uninstalling your program from a machine.

OnDemandSynchronizer.Unregister(myDirectoryPath, new OnDemandRegistration());

Synchronization

When your program starts (make sure you registered a sync root before that, at setup time), the synchronizer will run only if you set the SyncPeriod to a non-zero value:

// fileSystem must be an instance of IRemoteFileSystem
synchronizer = new OnDemandSynchronizer(myDirectoryPath, fileSystem);
synchronizer.SyncPeriod = mySyncPeriod; // start sync

That’s it, the synchronizer will now run automatically, using a timer.

In this screenshot we see a directory below “AmalgaDrive” which contains 1 directory and 5 files. Only two files are present locally on the disk (note they are images, as the Windows Shell often tries to download images, without being asked to, to be able to display icons or thumbnails). The 3 other files are just “placeholders”.

Synchronization - Picture 44

If the end-user tries to open one (double-clicking on it), the Windows File System will ask to the OnDemandSynchronizer instance to fetch the content of the file (and itself will ultimately ask it to the IRemoteFileSystem implementation), because the OnDemandSynchronizer instance registered itself to Windows when it started. Note if the synchronizer class is not running, the end-user will get this type of message from the Windows Shell, which is expected:

 - Picture 45

Combining with a Namespace Extension

You can perfectly use the file on-demand feature from a Shell Namespace Extension written with ShellBoost. However, only the physical shell items (or folders) will be able to support on-demand feature as it relies technically on Windows NTFS storage. You can still combine virtual shell item with physical on-demand shell items, though. The OnDemandSynchronizer instance can be hosted in the same .exe as the Shell Namespace Extension server.

Here are the same files, but displayed in the context of the AmalgaDrive namespace extension:

Combining with a Namespace Extension - Picture 46

The “Status” column has been added by code to the folder, like this:

public OnDemandRootFolder(OnDemandShellFolderServer server, ShellItemIdList idList, DirectoryInfo info)
    : base(idList, info)
{
    // since this is a FileSystem oriented NSE, we let base properties pass through
    ReadPropertiesFromShell = true;
 
    /// We just add the sync status column.
    AddColumn(Props.System.StorageProviderUIStatus, SHCOLSTATE.SHCOLSTATE_ONBYDEFAULT);
}
 

System.StorageProviderUIStatus is a Windows property that represents the status folder. It’s a property of type Icon. Refer to the “Item icon properties” sub-chapter of Item properties / folder columns for more on this. The only difference is, here, the two properties (System.StorageProviderUIStatus , StorageProviderState) are already defined by Windows, so for each item, we just have to set their value according to the item sync status.

The ReadPropertiesFromShell property set to true instructs the ShellItem instance to fall back to the physical file’s Shell built-in properties for properties not handled by the ShellItem instance itself. It guarantees maximum shell integration (menus, etc.).