Windows Lesson 8 - NTFS File System


In this lesson we will learn a little bit about NTFS. We will see how it compares to other file systems and see some of the advances features of NTFS and how we can implement them.

File System History

The file system controls how data is stored and accessed on your drives. Most files systems use a hierarchical system of directories containing files. File systems have evolved over the years. In earlier operating systems we had FAT (File Allocation Table), and FAT was a basic way to access our data. FAT improved over the years, but was lacking in some features needed to push the capabilities of the operating systems farther. When Windows NT was in development, the limitations of FAT were apparent, so a new file system was created called NTFS. All modern versions of Windows use NTFS as their default file system on hard drives, but we still use a version of FAT on our removable USB drives. Here is a breakdown of the basics of the file systems, even including HPFS. HPFS was a file system used in OS/2 which was co developed with IBM before they split off and created Windows NT.

  • FAT - File Allocation Table

    • Primary file system for DOS and non NT versions of Windows (Windows 3.1, 95, 98, ME)

    • Widely supported, works on Windows, Linux, and OS X.

    • No journaling support

    • Many versions of FAT

      • FAT16

        • File size limit: 2 GB (Windows 95 OSR2 added extensions to change this to 4 GB by using an unsigned number)

      • FAT32

        • File size limit: 4 GB (Introduced with Windows 95 OSR2)

        • exFAT

          • File size limit: 128 PB

    • HPFS - High Performance File System

      • Primary files system for OS/2 but was developed by Microsoft and IBM and supported in Windows NT 3.1 and 3.5. Support for formatting new drives as HPFS was dropped in Windows 3.5.1.

      • No journaling support.

      • File size limit: 7.68 GB

    • NTFS - New Technology File System

      • Primary file system for Windows NT based versions of Windows. (Windows NT, 2000, XP, Vista, 7, 8, 10)

      • File size limit:

        • Windows 8 / Windows Server 2012 and later: 256 TB

        • Windows 7 / Windows Server 2008 R2 and earlier: 16 TB

      • Features

        • Security - You can control who can access your files and folders.

        • Compression - You can set a file or folder to be compressed to save space.

        • Encryption - You can encrypt your data to further protect it.

        • Disk Quotas - You can limit how much data people can store.

        • Journaling - A log of changes that can help recover from a crash.

        • Shadow Copy - This allows us to keep previous versions of our data.

        • Resize partitions - You can reduce or increase the size of an NTFS partition.

File and Folder Attributes

There are four basic attributes that can be associated with a file or folder in NTFS. They are Read Only, Hidden, Archive and System. We consider these to be basic attributes because they also exist in FAT. If you display the properties of a file you can see two attributes. Read Only and Hidden.

Read Only will let someone view the file, but not update or make changes. Hidden files are more interesting. You can set a file as hidden and it will disappear. You won't be able to see the file when using Windows Explorer. In the image below we have a file that's hidden and we can't see it.

Windows Explorer has a way to easily turn on hidden files so you can see them. Once you turn on hidden files the hidden files will show with a slightly dimmer icon than normal to indicate it's hidden.

If you want to prevent this behavior you can set a file or folder as hidden and system. If a file is hidden and system it won't show when you display hidden files. Unfortunately you can't modify the system attribute using Windows Explorer. You have to use the attrib command from PowerShell. In the screenshot below we use the attrib command to see what attributes are set on our files and folders. We turn off the hidden attribute then we turn on hidden and system on our file.

Once you set a file as both system and hidden it won't show in Windows Explorer when you turn on hidden files.

These files can still be made visible by showing protected operating system files in folder options. It's dangerous to run this way since you may accidentally delete an actual system file causing problems. When you turn on system files you will see more than the file you hid.

The archive attribute is used to determine if a file or folder has changed since it was backed up. When you create a file the archive attribute is turned on.

Once the file is backed up the archive attribute is turned off. It won't be turned back on until the file is changed. The archive attribute will let the backup software know it's been updated and needs to be backed up.

Advanced Attributes

There are two advanced attributes, Compress and Encrypt. These are only available on NTFS partitions and are only in effect while the data exists on the NTFS partition. If you move data that is either encrypted or compressed to a USB drive formatted with an exFAT file system they will no longer be encrypted or compressed. The advantage of these advanced attributes is how they work in a transparent way. The applications don't need to know how to compress or encrypt the data, the file system takes care of it seamlessly.

You can only select one of the attributes at a time. You can't have a file or folder be both compressed and encrypted. The animated gif below shows how only one can be selected at a time. The confusing thing about this is the choice of checkboxes instead of radial buttons. Usually checkboxes mean more than one can be selected and radial buttons mean only one can be selected. Boo...

Encrypting a file is easy, in the properties of the file click Advanced button and then select Encrypt contents to secure data. When you encrypt a file or folder, a lock in Windows Explorer. This lets you easily tell if your data is encrypted.

When you encrypt data using the advanced attribute the data is encrypted with a File Encryption Key (FEK). That FEK is stored with the file twice in two fields. The Data Decryption Field and the Data Recovery Field both contain a copy of the FEK. The Data Decryption Field is encrypted with the Public Key of the user, and the Data Recovery Field is encrypted with the Public Key of the data recovery agent. By default the data recovery agent is the administrator, but this can be changed.

The advantage of this is the data can still be accessed if the original user leaves the organization. As long as the recovery agent can access the FEK the data will still be accessible. This setup also allows you to share encrypted data. You can choose which uses can access the data and a new field will be added to the file that contains the FEK and is encrypted with their Public Key.

When you copy or move data into an encrypted folder the data becomes encrypted. Any new data created in an encrypted folder is encrypted as well.

Setting the Compress attribute on a file or folder will reduce the amount of space the files or folders take on the drive. The file system handled the compression and expansion automatically as you open and close compressed files. The data is only compressed when it sits on the disk. If you try to email a compressed file it will be expanded when it's attached to the email.

If we look at our file called Large Document we'll see it takes up 384KB on the disk at rest. Compressing a file is easy, in the properties of the file click Advanced button and then select Compress contents to save disk space.

After it's compressed the file is shrunk to 144K on disk. The actual size of the file hasn't changed. When you open it the file system automatically expands the file so it can be read.

When you compress a file or folder, a double arrow icon overlay is added to the icon. This lets you easily tell if your data is compressed.

When you compress a folder or file the results will vary. Some file types are already compressed so there won't be a big gain when you compress them at the file system level. It's important to understand that this is not the same as zipping your data. If you create a zip file all the data is compressed in a portable format that can be carried to other computers. This level of compression is on the file system and is gone when you remove it from the file system. So if you have 4.5 GB of data and you put it in a compressed folder it may end up taking 3.0 GB of space. This doesn't mean you can copy it to a 4 GB USB flash drive. Once the data leaves the file system the compression is lost.

The inheritance of the compressed attribute on a folder works a little bit differently than it did with encryption. If you move a file from the same partition into a compressed folder it won't inherit the compression. If you copy a file into a compressed folder it will inherit the compression. Also any new file created in a compressed folder will be compressed.

Disk Quotas

Disk Quotas were introduced in Windows 2000 Server. At the time you could only change the quotas for each drive. Every folder on the drive had the same quota settings. With Windows Server 2003 R2 Microsoft introduce File Server Resource Manager (FSRM) allowing us to control quotas at the folder lever. Before we can manage quotas at the folder level we need to install FSRM.

Once installed you can use FSRM to set up quotas for your folders. There are a few built in quota templates that may work in your environment. If not you can create your own Quote Template to define how much space a user can use. Below we're creating a template with a 1 GB limit.

If we look at the network drives on a client before we set a quote we can see the total size of the server's drive as the capacity of the network drive. This may give an illusion that they have exclusive access to this space and they may store more data then they should.

On the server we can create a new quota that links a shared folder to a quota template.

You can setup multiple shared folders using different templates. Below we have three shared folders linked to the 1GB limit and one shared folder linked to a 2GB template.

Once the quotas are applied the users will see the drive capacities change to the quota limit.

There are some settings you can change when setting up a quota. You can change the limit and set the limit to a hard or soft limit. If you set it to a hard limit the user will get an out of space message when they try to go above the limit. A soft limit is used to keep track of who is using the most space. You can also configure what happens when users hit a certain percent of their space. You can have it email the user, or admin, to let them know, or have it record to the event viewer.

Data counts against your quota if you own it. Each file and folder in an NTFS volume contains a field designating someone as the owner. If you take ownership of files and folders the quota data won't be accurate.

Distributed File System

Distributed File System (DFS) allows you to store data on multiple servers and have it replicate automatically. If you have multiple sites and people travel between the sites you could create a DFS to hold their data. That way they are always accessing the data locally and it replicates automatically to it. The end result is a single namespace that contains shortcuts to other servers, or replicated data.

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10