Flag - or Solve - a Windows Tree Problem

[kphilo]

Good Morning!

Thanks for the software! I was about to start something similar for myself, when I stumbled upon your program. Well worth every cent!

I'd like to propose an enhancement I believe would be very beneficial to all us happy clients! Of course, I may live in such a black hole that I'm the only one interested in this idea. But I'm open to feedback from you and your community. (It never hurts to get cross checks on reality!)

SUGGESTION: UVK ver. 6.7.0.0 under XP Pro for following;

I ran into a bit of a snag a couple of weeks ago and thought I'd suggest the following enhancement. <Sorry this isn't a bit more timely -some details are going to be a bit foggy....>

THE SYMPTOM(S):
About a terabyte of data/programs over multiple hard drives of various sizes - the discovered problem actually had mutilple instances, but I'll just focus on the 1 example which allowed me to ID the issue.

Working on a XP Pro system with a clean report from virus scan. I had a variety of permission and access errors that I had to address. I used the "unlock all files" feature. Needed to do a reboot, so I did: still had locked files. Same with resetting all permissions to defaults. I decided to delete files in a temp dir in order to reduce processing times. Some files needed to be deleted on re-boot. Seemed strange for a temp dir that had scanned clean. But, ever vigilant, I checked running processes, etc and couldn't find anything that should effect the directory. Oh well, who knows what secrets lurk in the hearts of computers? Re-booted. Took a look. No files deleted. Attempted this several times. Same results every time. Repeatedly re-did unlock & permissions prior to new attempts to del tree. Same results. Checked Win XP points: Yes, the list of files to delete were in the proper place(s). Reboot. No files deleted. Furthermore, they were no longer listed as pending deletion (anywhere). Commands always seemed to work. No error messages. Event log(s) did not report anything. No matter how many times I tried, nothing was ever accomplished.

THE PROBLEM & SOLUTION:
The temp directory was actually labeled as a temp backup location of files. Newest of which was several years old. There was a second copy in an other location - presumably the original. Other files were in the temp backup as well and also had originals in other locations. All files were related to the same subject and all were consolidated into a more cohesive structure in the backup dir. (Presumed it was an organizational thing prior to moving the backup elsewhere.) I noted the entire tree was compressed, but this turned out to be a red herring. The real problem turned out to be that when the files were moved into the "new" tree structure, max path name limits were exceeded! << I later confirmed that the various "sub-trees" were just copied from their original locations into an already opened branch in the temp backup dir.>> I was forced down to the bottom of the tree in order to delete "segments" with short enough path lengths - one at a time using del .\*.* /f /s. Then I'd move a ways up the tree and try again. If it worked, fine. If not, move down the tree a ways. Repeat until entire dir finally deleted.

SUGGESTION:
It's been a long time since I've done any coding (before C+). But I still remember that not everything is as easy as you'd think! IF it isn't too much trouble, maybe the delete - and other file/dir commands - could do a count on path length. If max length met or exceeded, at least throw up an error message on the display. That alone would be very helpful. Although I've seen this sequence happen a few times before, there had been enough other problems with this system that I just figured I'd missed something. The suggested message would have saved me a fair bit of time. I considered a stand alone tool to do the check and report Pass/Fail. But that would only get used when I already suspected the problem. Not as functional as ID'ing the problem for me.

IF it isn't too much trouble, it would be even nicer if the command didn't just throw a message. Instead, when the limit is reached (or reasonably close) the command would work it's way down the tree keeping pointers to segments of the tree within max length bounds. The command would work its way down the tree, keeping pointers to segments of paths shorter than max, all the way to the bottom of the tree. Then it would perform its function, starting at the bottom of the tree (highest point within limits) and work its way up - performing it's action on each segment before it moved up to the next block and repeated the process incrementally till the "top" was reached. At completion a message about exceeding max path length (with total path length found?) would not only enhance realization of your tool's quality, but could serve as a flag about potentially similar problems elsewhere.

I know it's some bloat to code. But size cost vs functionality seems worth it to me (for all that's worth!). I kinda envision a small routine called prior to the command to check for path length size. If max not met/exceeded, perform as usual. If path too long, routine would then call a recursive sub-routine to pass back (an array) of pointers into the tree that do not exceed the limits. The routine would then call the command (recursively?), passing segments of the tree to process the array from the bottom up. Takes incrementally longer, but not as long as manually traveling to the bottom of the tree and manually working our way back up.

Please feel free to contact me if I can provide more info or whatever. About mid-week I'll be out of touch for about 7 - 10 days, so please accept apology for being potentially non-communicative.

I'd like to once more say thanks for a nice quality tool. I'm looking forward to continuing to become more proficient with it. (It takes a while longer when you don't get to use it every day.)

[Fred]

kphilo, file paths are limited to 260 characters unless you use the unicode version of the API and prepend "\\?\" to the path. For instance, a path like C:\very\long\path\... would become \\?\C:\very\long\path\... and would be deleted on reboot without any problem.

That is what UVK is doing, so it should not have any problem deleting files with very long paths.

[kphilo]

kphilo, file paths are limited to 260 characters unless you use the unicode version of the API and prepend "\\?\" to the path. For instance, a path like C:\very\long\path\... would become \\?\C:\very\long\path\... and would be deleted on reboot without any problem.

That is what UVK is doing, so it should not have any problem deleting files with very long paths.

Good Morning, Fred!

First off, I feel just as sorry you're working on the weekend as I am!

Thanks for the information. After you're input on \\? format, I do remember seeing something about this in the past, but had obviously forgotten it. I think I'm faced with a learning opportunity here, and appreciate it very much. My question is, if UVK is using the \\?\c:\very\long\path\... why were the files not being deleted? As stated, not trying to be difficult, just hoping to understand better. That way, when I next encounter a similar situation, I won't get so frustrated.

And thanks again for all your work. I appreciate it!

KP

[Fred]

Hi kphilo, my guess is that the files were deleted, and re-created when the computer boots up.

I can not make a real assumption unless you give me a very long path to test out.

XP had some known bugs in the "Delete on reboot" feature (only in some situations). Since you're willing to learn, I advise you to read carefully the following article:
http://msdn.microsoft.com/en-us/library ... s.85).aspx

I think you will find some answers there.

Best wishes.

[kphilo]

Thanks, Fred!

I'm off to the link!

EDIT: I'm Back...

I'm sure I need to study this a bit more, but could this be part of my problem?

*** Start Quote ***
The system deletes a directory that is tagged for deletion with the MOVEFILE_DELAY_UNTIL_REBOOT flag only if it is empty. To ensure deletion of directories, move or delete all files from the directory before attempting to delete it. Files may be in the directory at boot time, but they must be deleted or moved before the system can delete the directory.

The move and deletion operations are carried out at boot time in the same order that they are specified in the calling application. To delete a directory that has files in it at boot time, first delete the files.
*** End Quote ***

Directories were populated before & after all attempts to delete them. But I confess, it never occurred to me that I'd need to delete files in the dir before deleting the dir.

The system is long gone out the door at this time. I'll probably play around a bit with the lab system to see what damage I can do. But I'm going to be away for a while - a nice remote area without even cell phones! I'll check back in a couple of weeks, so I won't even start til then (unless I just can't sleep for curosity's sake).

[Fred]

What you quoted text means is that files inside a directory need to be scheduled for removal on reboot before the parent directory. UVK is already doing that, so no worry.

I just thought about something else: You were probably trying to move the files to the recycle bin! If that was the case then it is normal that the removal operation fails for long paths. Only files whose paths do not exceed 260 characters can be moved to the recycle bin! In that case you need to check "Delete files permanently" before the removal process.

[kphilo]

Thanks, again, Fred!

I was reading the change log after downloading the current versions...

Then I understood correctly, delete files, then directory.

As for moving to the recycle bin... I don't think so, but that was a couple weeks ago. I remember thinking that as long as there was all this redundant data that was in a temp directory, reclaim space (several gigabytes!). Of course, there is always the grain of salt: If I'm working on a system, something isn't right! I try and make it unthinking habit to start with chkdsk and sfc (after I'm fairly sure malware is gone), but it wouldn't be the first time that I've had sfc report no errors, erroneously! As indicated, I'm going to make myself a note to play on the lab system when I get back. If I find anything of interest, I'll let you know. Otherwise, I've just got a new chalk mark under "experience".

Don't have too much fun!
KP