Portable Delphi 2007 for Win32 - Part I

[EDIT] Even though you can do all listed here, there are a few minor 'gotchas' that you should be aware of, so browse through the 2nd part before following this to the letter, especially if you use the command line rather than a graphical tool to delete the folders. I'm currently typing Part II into Windows Live Writer, so check back in a couple hours if it's not online yet when you're reading this... Unfortunately I ran into an internet problem and then changed my mind and decided to split the work and use WinWord to create a PDF file to upload with the whole start to finish process and only keep the "blah-blah" on the post so that you can more easily follow the process without waving through a ton of blah-blah. But real-life has a way of messing up one's plans! I have been sick for a few days (an annoying flue that kept me in bed and hardly willing to touch a computer). I'm feeling better now and will finish work on this as promised, but it most likely won't happen before the next weekend as I have a lot of work stuff to catch up during the week.[/EDIT]

Ever wanted to carry your development environment with you on a flash memory stick? Well, using Thinstall you can, albeit it comes with a hefty price tag of 5.000 USD... Maybe now that VMWare has bought the company they'll wise up and release a cheaper personal version! I would like one, and have e-mailed them already stating that. If you would too, don't forget to bring it to their attention! (Edit: Thinstall is now known as VMWare Project NorthStar)

There seem to be some cheaper alternatives, and I even tried Xenocode's Virtual Application Studio (VAS for short) but, even though it has a nice visual interface, it has a few fatal flaws. I couldn't find a way to have multiple entry points and even something as simple as selecting the main entry point is painful with an application that has several thousand files as they use a simple non-typing lookup and they display ALL files to be selected. And if your app (or app combo) somehow has the same exe name in two different folders, then you can't tell them apart in that combo-box... But the main problem against VAS was it failing a simple test: virtualizing ActiveState's Perl 5.8.8... That's something that Thinstall did without a glitch...

This is a two-part article, but don't worry: this time both parts are already written before publishing the first one! But I also like to tease a bit, so Part II will only come out in a couple days! That and because I took my notes on a text file while doing things and have to get them into WLW for proper formatting! :)

Thinstalling (TM) Delphi for Win32 2007 (TM) Professional

Those reading my blog will know that I'm a bit anal about files and disk space in my computer: I'm a software developer (among other things), so I like to be in control! It is me and not the machine nor the OS or any given application that should be in control of what goes on on my computer...

In light of that, I have a plethora of small and some not so small utilities that I have in a "ready-to-run" state on a different partition, so that, after a fresh install, I can run those immediately. I also use Delphi (currently the "2007 for Win32" version) and if you couple all that with the lousy installer and/or lousy install script that Delphi ships with you can understand why I have targeted it a few times before.

I also hate the way my system gets noticeably slower as I add programs. That's no big surprise: the big junkyard of Windows, usually known as Registry, easily starts to hold a gazillion of entries slowing it down. Add Windows Installer to the mix (and very few major-named products these days use an installer that is not based on MSI technology) and you can see where all this goes...

MSI technology is one of those nice-in-theory concepts with all it's self-healing that allows a "regular Joe" to keep running, say, Microsoft Office, even after he blissfully deletes some required folders because upon running an office application it will detect that some files are missing and will re-install them. Now, this is bad news for those like me that like to tweak things. Something as simple as deleting some readme's in 30 different languages will trigger the self-healing process and next time you run your app, they'll be back in place... Also, messing up with the MSI files left behind on your disk will usually result in bad news, from missing application icons (why on earth don't they use the proper .EXEs for the shortcut icons?), to more serious problems with self-healing kicking in and asking for original media to put things back there and you not being able to run the app until you please the installer...

So, I decided to try a new angle and this time I decided to experiment with an old tool but one which I only recently got in contact with: Thinstall. In a short sentence, it's VMWare for single applications, that is, it's a virtualization system that allows you to run applications without installing them first. Just double click a "simple" executable file and it will run as if you had installed it on your machine.

So, what are the pluses and minuses of Thinstalling applications?

+ No more re-install whenever you switch machines or re-format your C: drive or reset it to a "working" ghost image;
- You do need to re-activate when you switch machines; (that's an area where those using pirated and cracked versions have an edge over us, licensed users, but that's the price to pay when companies don't trust their customers and with MSFT setting the way, other companies keep following like lambs...)
+ You can run Delphi 2007 for Win32 from a 1 GB USB flash drive even on a machine with NO .Net installed and from a restricted guest account;
+ You can run it from a network share with no concurrency problems;
- Updating a "Thinstalled" app requires a bit more work (more on that on Part II);
- By default, a thinstalled program will have R/W access to "My Documents" and "Desktop" only: every other file/folder accessed, if changed, will be modified in a local copy on a "sandbox"; you can however configure per folder/drive which ones will be writeable or not. Or you can configure your thinstalled app to run in "merged" mode, but that can be tricky if the host computer has a similar version of the app installed.
+ Thankfully, network and removable drives are used R/W by default so if your files are not on a local disk, you don't need to make further changes to modify them.
- You can't Thinstall apps that install device drivers, such as PalmOne apps or a Virtual CD/DVD emulator or even some Anti-Virus.
+ You can Thinstall applications that install and depend on services as those will be virtualized and started as well when you run your app.

In my current setup, using a 4 GB flash drive, I can fit pretty much everything I need, being 1 GB for a ghost image of Win XP SP3 RC2 with nothing but Win XP + AV + , another one for Delphi 2007 and still have 2 GB left for other thinstalled applications and some data files. This allows me to be back in a working condition within 10 minutes, even if something seriously messed up my hard disk big-time...

So, how does one get to Thinstall Delphi 2007 for Win32?

After you get hold of a copy (you can get a trial from Thinstall's website), you do the following:

0 - Start with a CLEAN machine: I recently re-installed my C: partition with Windows XP SP2 and, after changing some settings, I proceeded to make a ghost of that partition to a bootable 2 GB USB flash disk (only uses less than 600 MB after a bit of tweaking). Using a CLEAN machine is important as is choosing the "oldest" one, that is, if you plan on running it on Windows 2000 and XP, don't use XP for the CLEAN machine as it may already have some files that the 2000 will not and those won't be installed and thus won't be captured.

0b- It's also VERY important to do it all in one go and refrain from using non-related programs, such as surfing the net or doing downloads or even cleaning up your temporary folders: whatever you do may get added to the thinstall project and needlessly clutter it.

1. Run "Setup Capture" and do the pre-install capture.
   
2. Install Delphi (including .net runtime and SDK); as for the shortcuts location, I used the option for "All Users" so it would not clutter my relocated "My Documents".
   
3. Launch and activate (if you want the install to be pre-activated, e.g., you're the only one using it); if you don't activate, when you run it you'll have 30 days to do it later, although I believe, but haven't tested, that those 30 days start counting from the day you INSTALLED it and not from the date you first run the Thinstalled version.
   
4. Configure any settings and/or install components and/or install updates.
   
5. Run "Setup Capture" and do the post-install capture.
   
6. Select the "shortcuts" to create and main package name. When you Thinstall an application, you create a single executable file but you can have alternative "entry points" into that file to run other applications that are part of the same capture. For instance, you can have an independent shortcut for running the Help Documentation without first launching Delphi. These are dependent of the "main" package where all the files/registry settings are stored. On my system, and for some reason (GUID conflict?), it suggests to store the capture in "ATI - Software Uninstall Utility". I opted instead to name the capture folder "Delphi 2007".
   
7. Wait a few minutes while it copies over 31.000 files using 3+ GB for my Professional SKU full install.
   
8. On my system, it failed to copy "C:\Documents and Settings\LocalService\NTUSER.DAT" and "C:\Documents and Settings\NetworkService\NTUSER.DAT": just ignore as those should not be copied anyway... This may be a symptom of me not following step 0b! :)
   
9. IMMEDIATELY SAVE this captured installation, that is, save the "Delphi 2007" sub-folder inside the Captures folder: this will be needed later if you want to upgrade your installation. I copied it to my external hard-drive and named it "Delphi 2007 (Full)" to distinguish from a copy that I'll make later after the tweaking is done.
   
10. Now we can start tweaking the thinstallation!
    

Tweaking the Thinstallation

You can do the remaining process on any machine: all you need is access to the thinstall folder and respective Capture sub-folder where you stored your "Delphi 2007" capture. I opted to use the same machine as I will quickly restore it to a clean state by booting from a flash drive and running ghost from there.

1. Go to your "Delphi 2007" capture folder and open up "Package.ini";
   
2. Uncomment (remove the starting ";") the "CompressionType=Fast" line and comment the other one (";CompressionType=None"); this will instruct Thinstall to use a quick compression when creating this application. Whenever in doubt, check Thinstall's online help file for available options.
   
3. Edit the "Sandbox=" line and use a descriptive name. (I used SandboxName=Delphi 2007). Whenever you run a Thinstalled application, any changes to registry and files that are not in the "merged" folders/keys will be saved in a local sandbox. By default, that is in your current user's Application Data\Thinstall\<sandboxname> folder, so, in a typical Win XP install, with a user named Joe and the suggested sandbox name above, this will be "C:\Documents and Settings\Joe\Application Data\Thinstall\Delphi 2007". It is possible to configure the application to clear this folder on exit, but you can also do a more interesting thing: if you create a "Thinstall" folder in the same place where you have your Thinstalled application, that folder will be used to store the Sandbox. This is particularly useful to do when running an application from an USB flash drive as in this way, not only your settings are not stored on the local computer, they will "travel" alongside the application on your flash drive.
   3b- I like to keep my apps in the same drive, a partition of my HDD and a copy of everything on a 4 GB flash drive. In both cases, a neat way to keep all your settings (read: sandboxes) nicely grouped in one place is to add the following line next to the previous one:
   SandboxPath=LocalSandbox\..\..\..\..\.Settings
   This way, and as long as your apps are not stored more than 3 levels deep, all the sandboxes will be created under the .Settings folder in the root of the drive where the applications are. So you can have something like:
   X:\Multimedia\Image\Image Editing App 1.exe
   X:\Multimedia\Image\Image Editing App 2.exe
   X:\Multimedia\Audio\Audio Editing App 1.exe
   X:\Dev\RAD Studio 2007\Delphi 2007.exe
   and, assuming the Sandboxes use the same name as the executables, you'll have the sandboxes created like this:
   X:\.Settings\Image Editing App 1\
   X:\.Settings\Image Editing App 2\
   X:\.Settings\Audio Editing App 1\
   X:\.Settings\Delphi 2007\
   
4. At this point, I did my first "build" which is as simple as invoking the BUILD.BAT file: 1 hour and 22 minutes later (on a 3 year old laptop!), I had a 2.000.782.941 bytes "Delphi 2007.dll" file along with a few .exe entry points for a total of 1.86 GB. Of course, this includes at least the huge 600 MB file with the .NET framework SDK install files. We'll get rid of those soon!
   
   4b- Don't worry about two things: the app won't show the proper icon (we'll use a workaround for an explorer bug soon), and it may take a LOAD of time launching (we'll get to that fix soon too!). If you want to test it out quickly, just launch something like the Command Prompt or the Documentation.
   
5. Thinstall captures are very easy to process: 3 "registry change" files are created containing all the registry changes to HKLM, HKCU and HKU; several "virtual" folders are created such as "%desktop%" which equals the current user's desktop folder and some %drive_x% folders for each drive where changes where made other than in one of the "standard" folders, that is, if the program only makes changes to, say, "C:\Program Files\CodeGear...", you will see no %drive_c% folder created, but if you change the default install to, say, "C:\Delphi2007", then you'll see a %drive_c%" folder with whatever was created/modified there.
   
6. We can safely get rid of uninstall files(*) as there's no point in uninstalling stuff from a Thinstalled app: even though the changes would be recorded in the local sandbox, and it would appear uninstalled, you'd save nothing by doing so and by deleting the sandbox, it will revert back to the "default" state. Thinstall also already ignores changes to some self-healing registry keys, so you can remove stuff without being prompted with a "re-install".
   (*) As we'll see in Part II, there are some exceptions, with Delphi being one of them!
   
7. With that in place, let's delete (you DID make that full copy already, didn't you?!) the following folders/files:
   - %ProgramFilesDir%\Microsoft Visual Studio 8\Microsoft .NET Framework 2.0 SDK - ENU (569,8 MB)
   - %SystemRoot%\Microsoft.NET\Framework\v2.0.50727\Microsoft .NET Framework 2.0 (76,4 MB)
   - %SystemRoot%\Microsoft.NET\Framework\v2.0.50727\Microsoft Visual J# 2.0 Redistributable Package (15,6 MB)
   - %SystemRoot%\Installer (93,9 MB - this was unrelated to Delphi 2007 and probably because I somehow ran something else: I've had this capture on "hold" for a couple days...)
   - %Common AppData%\{6AF0EFC6-B937-4704-A430-319EB93F4C12} - Delete all sub-folders but leave the files as they're needed for the "Check for Updates" option. (423 MB) This one will be in %Local AppData% instead of %Common AppData% if you used the "Just for me" shortcuts option when installing.
   - %Common AppData%\{135F413C-658B-4923-B950-5F2B786BC8DA} (341 MB - Help update setup files)
   - %Profile% (nothing useful)
   - %Common Programs% and %Desktop% - These hold the shortcuts as created by the installer, but we'll use a different set
   - HKEY_USERS.txt (on my system, there's only a handful entries and none Delphi related, probably for me breaking step 0b!)
   
8. Now let's open up Package.ini and fix a few more things:
   - Fix the broken Icon for "Delphi 2007.exe". Find the section [Delphi 2007.exe] and duplicate it. Now do a find/replace (on the whole text file) finding "Delphi 2007.exe" and replacing with "Delphi 2007.dll". Next, change one of those two sections to read [Delphi 2007.exe]. You'll need a few more changes so just check that they look like this:
   
   [Delphi 2007.exe]
   Shortcut=Delphi 2007.dll
   Source=%ProgramFilesDir%\CodeGear\RAD Studio\5.0\bin\bds.exe
   CommandLine="%ProgramFilesDir%\CodeGear\RAD Studio\5.0\bin\bds.exe" -pDelphi
   Shortcuts=%Programs%\CodeGear RAD Studio
   
   [Delphi 2007.dll]
   ReadOnlyData=bin\Package.ro.tvr
   Source=%ProgramFilesDir%\CodeGear\RAD Studio\5.0\bin\boreditu.dll
   
   The "problem" you fixed is that Explorer will try loading the whole .exe file in memory to show the icon, and with an .exe several hundred MBs it will fail. We trick it by storing the "bulk" of the information in a .dll or .bin or .dat or whatever non-exe file and then adding a smaller shortcut to Delphi 2007.exe, as per the Thinstall help section on this topic.
   
   - You can cleanup quite a few more entries in here, or just leave them with Disabled=1 so they won't have shortcuts created alongside with the main executable. I left the following entry points enabled:
   Check for Updates, Delphi 2007, RAD Studio Command Prompt, RAD Studio Documentation, Rave Reports.
   
9. You may want to find and delete all the "Icon=" lines as they force some applications to use an icon stored in an MSI instead of the one in the EXE and deleting the Install cache will get rid of the icons.
   [EDIT]In order to keep the RAD Studio Command Prompt and the Documentation icons, move those two *.ICO files elsewhere or copy them from the "full" version if you have already deleted them from step 7 above and update the Icon= line accordingly. You can, for instance, name them RAD Studio Command Prompt.ico and Documentation.ico and place them straight in the project root directory so that it's clear what they're used for.
   [/EDIT]

1. Also, while you're at it, check the lines with "Shortcuts=" (notice that it's PLURAL Shortcuts with an S at the end: not to be mistaken with the similar but different meaning option Shortcut!).
   In my install, I set all the shortcuts into a Thin\genre\app type of folder, for instance, I use Shortcuts=%Programs%\Thin\Dev\Delphi 2007, so that all shortcuts created (for the sections that don't have Disabled=1), will be stored in a Thin -> Dev -> "Delphi 2007" folder. (More on that on Part II).
   
2. The speed problem seemed to go away as soon as the "main" executable file dropped in size: didn't test to see if it was related to being over 2 GB of uncompressed data or to having a few huge files in there, but it works ok in a 1.7 GB uncompressed version as well as in this 1 GB compressed one...
   
3. Now you can re-build by simply invoking the batch again. On the same machine as above, and because Thinstall caches the files from the previous compilation, it took only 10 minutes to generate the brand new 1.032.472.412 bytes file. This means that, together with the few extra shortcuts or alternative entry points, it now uses up only 984 MB, so it perfectly fits on a 1 GB flash drive. (And we'll trim that further in Part II).

That's all for now. In Part II, we'll tweak a lot more by removing the .net related help content. I will also explain how to update your Thinstalled "Delphi 2007.exe" file so that you can incorporate patches, add/remove/update some components or simply modify default settings. Finally, I'll show you how easy it is to have the proper file type associations so that you can simply double-click an *.dpr file and open up your Thinstalled Delphi. Until then, get your own (30 day trial of Thinstall)* and start playing!

* EDIT: Thanks to Xsintill for pointing out the VMWare NorthStar beta (which is essential the full Thinstall project) but has two things better than the Thinstall trial: 1) lasts until 1st of June 2008 (roughly 70 days from now); 2) The whole registration process for the trial is automated and less restrictive in the choice of e-mails. So go grab that beta and start playing...