2008-01-26

More Disk Space Tweaking...

UPDATE 1: DO NOT TRY THIS! Even though everything is working ok on my system, I've just noticed a shortcut icon problem that I believe is related to the order I've done something. I'll have to test it a bit more and will update this post where needed to prevent that from happening but it may take a couple days before I update the post!

Update 2: Apparently it's Windows Installer that is to blame. Not only it completely ignores the junction, it goes even further: deletes the junction (causing all contents to be lost!) and creates a new INSTALLER folder in your Windows subdirectory. So, this particular folder CANNOT be moved or the next time you use msi installer (which is just about any installer or patcher these days), you'll LOSE all that was transferred.

It's always "fun" to see Microsoft NOT playing by their own book!

Or, how to keep your C: drive mean and lean and still don't throw away those lengthy patches and cached install files...
This is not entirely related to Delphi as the techniques used here are more general and thus you can extend this to your own liking. It is however triggered by the fact that I hate to see wasted space on my C: drive, which, as I've told you before, Delphi's current installer likes to do! :)
In short, you'll learn how to relocate some huge folders to another partition or even another hard-disk. This is especially useful for temporary or cache folders as those serve no useful purpose 99,99% of the time but still use up disk space.

Theory

The whole process revolves around Windows XP and NTFS formatted partitions: you can create junctions, which are links to folders or whole disks and access those as if they were native folders on another partition. For instance, you could relocate C:\Documents and Settings\All Users\Application Data to D:\MyData and whenever a program tries to access the first folder it is in fact accessing physical data from the MyData folder in drive D:

To ease up the process, I created a small batch file in 4NT (sorry, but I don't do CMD in years and don't feel like going back!). 4NT has a nice little command MKLNK (and also a newer one MKLINK which is *NOT* what we'll use, so just keep it in mind that there are two very SIMILAR but DISTINCT commands!). The MKLNK command allows you to take advantage of mount points and create/delete them. In the end of this post, I'll list the whole batch file I used. As you can see, using ALIASes and a few 4NT variables and functions makes the whole batch a lot better than if I were to try it out using CMD only!)

Now, I'm sure that there is a MKLNK-alike utility or two around, even though I haven't searched for any, so feel free to ignore the batch and try doing the commands on your own, although you'd probably be doing yourself a big favor by actually trying out 4NT! ;)

Limitations

As mentioned in the batch file itself, which you can download from here, you need to be aware of the following:

  • Non-English versions of Windows XP may use different names for Application Data and Local Settings;
  • You have to edit the SET %RootFolder= line to set your own root folder. I used T:\Patches but you can use anything else (and even modify the batch to spread things around multiple folders instead of sub-folders of this one)
  • This batch file is a first version. In order to keep things simple, it does NOT check for available free space on the target folder but it does warn at start that it doesn't do that and that you SHOULD make sure you have enough or else you risk being left with some stuff moved and other not quite so... I do believe, but have NOT tested, that it should be able to resume from where it left if you run out of free space and then run the batch again, but, I repeat, I have NOT tested that!
Let's get busy!

So, let's start digging into the batch file:

01-24

You should edit line 5 and, if running on a non-English OS, change line 9 and comment lines 15 to 17.

Line 20 defines an alias to test for the existence of a folder name, create it if not found (creating parent folders if needed) and then changes to that folder. It's not exactly needed here as it's only used in one place, but I copy/pasted it from my aliases file.

Line 23 defines another alias to echo a blank line, then echo the passed parameters slightly indented.

The %+ symbol equals "whatever command separator is configured on this system". 4NT allows several commands to be executed on a single line, which is useful, among other places, in aliases.

25-30

Now we get to the main alias, the one doing most of the work in line 30.
Here we test if the 1st parameter is already a junction: if that's the case, then the "TRUENAME" (which is a 4NT function that returns the actual full path and "sees" through junctions, substs, net shares and symlinks) and we compare that to the "FULL" name, that is, the name after expanding any % variables and processing any ..\ or drive aliases. If they're equal, then no junction is in place.
Here's an example to make it clearer:
Imagine that:
  • %windir% equals C:\WINDOWS
  • %windir%\$hf_mig$ has been redirected to T:\Patches\$hf_mig$
  • %1 equals %windir%\system32\..\$hf_mig$ (this is intentionally complicated to illustrate)

ECHO %@TRUENAME[%1] would return T:\Patches\$hf_mig$
ECHO %@FULL[%1] would return C:\Windows\$hf_mig$

If the two match (which is false in this case), then no junction or subst is in place. Seeing that you can only SUBST a whole drive letter and not a folder, it is assumed by the batch that if it fails you already have a junction in place and you get a warning (the ELSE part). If the two strings match, then no junction is in place so it MOVEs the source to the destination, showing only /Totals instead of all files moved, moving /Hidden files, not complaining on /Errors, and moving whole /Subdirectories. Finally, the junction is created to "fake" the system into believing that the whole folder exists but it now points to a different drive.

31-44 

Line 36 shows a message box prompting the user to acknowledge that the batch will NOT check for free space and asking to proceed.

Lines 39-41 stop some services that interfere with moving some of these folders. If you add to the list below, you should first use an utility like HANDLE from SysInternals to check if some program or service is keeping that folder or anything inside open and add the service to the list and maybe even use PSKILL to terminate the application if it's not a service. (Edit: on checking SysInternals website for the links to add here, I noticed a Junction utility that you can use if you don't/can't/won't use 4NT!)

Line 43 creates the root folder if needed.

45-59

This is the "bulk" of the work, relocating several folders from my system to the cache folder. Pretty self-explanatory (ignore the color bug in some of the lines: that's UltraEdit not syntax highlighting BTM files properly!)

60-75

Finally, I also disable (/CACHESIZE=0) and remove existing entries (/PURGECACHE) in the dreaded %windir%\dllcache folder. You may want to remove or comment out this line if you rather keep half a gigabyte of duplicate dlls on your disk.

Lines 69-75 remove ALIASes and SETs used by this batch. Not really needed, but I like to clean up after myself! :)

And that's all for now. There are more folders that can be safely added to this list, and, on my system, this saved me 2.5 GB worth of C: space. EDIT: Even not counting Installer, I still save 1.6 GB worth of data...

End Notes

If you're thinking of doing this to, say, the user profile directory, you'll need to do it from another account and not have logged that account in the current work session. But I would advise you NOT to do it! I once did it, and even in a more "crude" way requiring messing around with the registry, but then I found that doing it saves a LOT of trash! So, upon restoring a working C: image, I'd have my profile all set with all the trash accumulated from all the programs installed/removed/changed.

You can download the batch file from here. (Yes, I know I've linked it above, but it's easier to find it here if you come back later!)

2008-01-20

Help needed with PDF Processing Library...

After two days of searching, clicking through hundreds of links, downloading tens of files, trying, browsing, etc, I decided it was about time to give up and ask for help!

I need to process some PDFs, mostly "black-and-whitening" them, that is, converting color boxes and text into black and white. It's not the same as "printing" in black and white though, as I want to replace boxes filled with some color with ones with no color fills and change the corresponding white or light colored text with black text.

Example:

From this:Source to this: Destination

I'd like to do this in Delphi, but I can do in a million other languages such as Perl, PHP, Python, Java, C# or whatever. I have one restriction though: the library has to be either free or open-source, but this restriction can be waived if there's nothing free or open-source that I can use to solve the problem...

Does anyone know of a library that will allow me to do that? Open a PDF file, loop through all the objects and change characteristics as I see fit, and finally saving the edited file

If you know such a library/tool, please leave a comment... Thank you.