Microsoft has figured out why the Windows 10 October 2018 Update deleted data from some systems and produced a fixed version. The severity of the bug caused the company to cease distribution of the update last week; the fixed version is now being distributed to Windows Insiders for testing, ahead of a resumption of the wider rollout.
Microsoft is advising anyone affected by the bug to contact support. The bug caused files to be deleted, meaning that the only guaranteed way of restoring them is to retrieve them from a backup (if you have one). However, undelete software can often recover recently deleted files, and Microsoft's support can apparently assist with this process at no cost for those bitten by the problem. In the meantime, the advice is to use affected PCs as little as possible; minimizing disk activity helps maximize the chance of a successful undeletion.
The software giant claims that only a small number of users were affected and lost data, and has published an explanation of the problem.
Known unknowns
Certain directories, including Documents, Desktop, Pictures, and Screenshots, are known as "Known Folders." Known Folders are special, in that software can ask the operating system for the folder's path, without having to hardcode any names. For example, an application wanting to save files in the Documents folder can ask the operating system for FOLDERID_Documents and the operating system will translate this to the actual location on disk. Typically this would be the "Documents" directory in the user profile.
However, it doesn't have to be. The storage location of the Known Folders can be changed, a capability called Known Folder Redirection (KFR). This is useful to, for example, move a large Documents folder onto a different disk. Software asking for the Documents Known Folder location will be given the redirected location so it'll seamlessly pick up the redirection and use the correct place. This is why programs shouldn't just hardcode the path; it allows this kind of redirection to work.
Redirecting one or more Known Folders does not, however, remove the original folder. Moreover, if there are still files in the original folder, redirecting doesn't move those files to the new location. Using KFR can thus result in your files being split between two locations; the original folder, and the new redirected folder.
The October 2018 Update tried to tidy up this situation. When KFR is being used, the October 2018 Update will delete the original, default Known Folder locations. Microsoft imagined that this would simply remove some empty, redundant directories from your user profile. No need to have a Documents directory in your profile if you're using a redirected location, after all. The problem is, it neither checked to see if those directories were empty first, nor copied any files to the new, redirected location. It just wiped out the old directory, along with anything stored within it. Hence the data loss.
Adding insult to injury, there are ways in which Windows users could have enabled KFR without really knowing that they did so or meaning to do so. The OneDrive client, for example, can set up KFR for the Documents and Pictures folders, if you choose to enable automatic saving of documents and pictures to OneDrive. The current OneDrive client will set up KFR and then move any files from their original location to the new OneDrive location. Older versions of the OneDrive client, however, would set up KFR but leave existing files in the old location. The October 2018 Update would then destroy those files.
This scenario is not especially esoteric or extraordinary, and it's not some difficult interaction with third-party software or hardware. KFR is widely used, and the situation in which both the old and new locations contain files is easy enough to arrive at; software such as the OneDrive client used to do it, and when enabling KFR for the Documents folder through Explorer, you can elect not to move files to the new location. It beggars belief that this "tidying" feature was developed without any consideration of what might happen if the old folder still had files in it. And adding insult to injury, the only reason that the October 2018 Update deletes those original folders is aesthetics. Some Windows users had apparently complained that the old, empty folders were still left even after enabling KFR. Microsoft wanted to "tidy up" those empty folders by deleting them. The rest, like affected users' files, is history.
In response the company has promised to update the Feedback Hub tool so that the severity of bugs can be indicated. Many people reported this data loss bug, but none of the reports received many upvotes, with Microsoft accordingly disregarding those bugs. If the bugs had been marked as causing data loss—the highest severity possible—then they may have received the additional attention that they deserved. Microsoft hasn't, however, explained why this update didn't receive any kind of "release preview" distribution or testing. There are no guarantees that this would have caught the bug, but it would have meant that an extra round of people installed the update onto their systems, and who knows, one of their bug reports might have got lucky.
[contf] [contfnew] 
Ars Technica
[contfnewc] [contfnewc]
