The PDMAN tools are NOT officially supported by the Hewlett-Packard Company. Please do not call the HP Response Center and expect to get answers to problems related to the tools. PDMAN tools were written and are maintained on a "best effort" basis by the authors. Any bug reports or enhancement requests can be emailed to us, but there is no guarantee that they will be added to PDMAN. Neither Hewlett-Packard, nor the authors of PDMAN are responsible for problems or system downtime caused by the use of PDMAN tools. Feel free to take advantage of these powerful utilities, but use them at your own risk.

Note that the specific syntax of PDMAN tools described in this paper may not match the current release of the tools. Please reference the installed tools themselves before using them. The authors reserve the right to modify the tools at our own discretion and without any notice to end users. This software has no warranties.

Many times in working with large enterprise HP implementations, the request comes from the local system administrators for an easy way for them to have all the latest HP-UX patches at their local site. Requests for tapes of all the patches or ways to "mirror" the HP public patch server have been made. In addition to that, there are questions about the best way to structure and manage the multitudes of SD-UX depots that contain mixes of various patch bundles for easier and more organized patch distribution throughout their UNIX environment.

The Patch Depot MANager (PDMAN) tools grew out of these types of requests.

This paper (and related PDMAN tools) addresses the following issues:

• How to automatically grow a local patch depot containing all the latest HP-UX patches and keep it current

• How to avoid re-bundling different mixes of patches into many depots, but still have the capability of distributing any combination of patches to any machine easily and conveniently

• How to clean up a master patch depot and migrate older patches to an archive depot

• How to easily determine differences in patch levels between two machines or depots

• How to quickly find which depot a patch or software package exists on, given a whole set of depots to search

• How these tools relate to and compliment other HP patching tools and services

To be as proactive as possible in keeping current with HP-UX patches should be the goal in any company’s HP environment. Organizations that have accomplished this have reaped the benefits of reducing unplanned downtime. Staying on top of patching is not a typical System Administrator’s favorite thing to do. By automating some of the more mechanical aspects of patching through the PDMAN tools, administrators can focus on which patches to implement and perhaps take some of the drudgery out of patching activities.

The following are features of the PDMAN tools:

• All the latest HP-UX patches are available locally (currently 10.X and 11.X).

• All information and utilities necessary to implement these tools in an HP-UX environment are publicly available to HP customers.

• Downloaded patch files are validated via the MD5 security fingerprinting utility – you are assured that the patch you download is the same patch HP created. See section below for more details.

• Verbose logging of commands allows problems to be isolated quickly.

• Totally command line driven for complete control of commands.

• PDMAN utilities are non-interactive so they can be implemented in a batch mode, or can be part of further patching automation efforts.

• Built in manual pages for commands via the "-M" option.

• Compatible with the HP Electronic Support Center Custom Patch Manager – the CPM package_info file can be used to feed the PDMAN mk_vbundle command.

• Written in K-Shell scripting – allows easier customization if necessary.

• One file (pdman_env) where global variables can be modified if necessary.

• Built in traps to allow graceful shutdown of the tools.

• PDMAN tools are available for no charge.

The depot_update utility is provided to grow a local master patch depot from scratch. The catalog files on the public HP patch FTP site is checked to see what the latest patches are to download. The command syntax is,

depot_update [-M] [-v] [-I] –r hpux_release –d master_depot

As with other PDMAN tools, you can see the built-in manual page for the command using the "-M" option,

/opt/pdman/bin/depot_update -M

Assuming the PDMAN tools have been installed properly with enough disk space allocated in the proper areas, you can initialize and build a local master patch depot with the following command (to run in the background):

nohup /opt/pdman/bin/depot_update –I –r 10 –d /pdman/master_10X_patches \

> /dev/null 2>&1 &

where,

-I	Indicates this is the initial run of depot_update for this particular depot. An error will occur if an existing depot is specified in the –d option in combination with this –I option.
-r 10	Specifies that this will be an HP-UX 10.X depot (as opposed to an 11.X one)
-d /pdman/master….	Is the full path name of the HP-UX 10.X patch depot that you wish to initialize

This command’s progress and output can be monitored via the PDMAN depot_update log file found in /var/opt/pdman/logs/depot_update.10.log. The logs are allocated by which PDMAN command is being run and upon which type of depot by release number. For example, to monitor the command, do the following,

tail –f /var/opt/pdman/logs/depot_update.10.log

If for some reason you need to terminate a depot_update command that is running, DO NOT USE YOUR SYS ADMIN KILLER INSTINCT and kill the process with a –9 hard kill option! Use a –15 type of kill and the command will shut down gracefully with full cleanup. Using the hard kill can cause unnecessary temporary files to be left around and also could cause an incomplete patch to be copied to your master depot.

An initial build can take several days of running to get all the patches, depending on the speed of your box, network performance and what type of problems you might encounter during the downloads.

Every run after the original initialization of a depot should NOT have the –I option specified. Restart depot_update as many times as you wish to get a clean download going. Each successive run of depot_update will check which patches are already on the master local patch depot and will download only the ones that have not yet been loaded.

If you want to see the output from the command interactively without having to monitor the log file, use the –v option. All output will be captured in the PDMAN log files, regardless of whether the –v option is specified or not.

Once all current HP-UX patches have been loaded into the master local patch depot, run depot_update via CRON daily to keep your master local depot up to date with the latest released patches.

The depot_update utility uses a publicly available file "fingerprint" utility called MD5 that is a stronger algorithm than a standard check sum of a file. MD5 comes as part of the PDMAN distribution, so you really do not have to do anything extra to account for this. If you already have installed and built your own copy of MD5 at your site and wish to use it, simply edit the pdman_env file to specify the location of the desired MD5 utility.

The folks at HP that post the patches on the public FTP site have provided a file (/export/patches/hp-ux_patch_sums) that contains all the MD5 codes for every patch that is posted. The PDMAN tools use these codes to ensure that every patch that has been downloaded is complete AND is the same patch the HP intends you to get.

If you get MD5 type of errors while using the depot_update utility, one of the following conditions could apply:

• For some reason the FTP did not download the patch correctly and the downloaded file is incomplete.

• The network connection between your local site and the public HP FTP Patch server is down.

• Your local machine cannot resolve the name of the public HP FTP Patch server.

• PDMAN was able to download the initial patch file, but an error occurred during the unshar of the patch to extract the *.depot file from it.

• An incorrect MD5 code was posted in the patch sums file that does not match the downloaded patch. If you discover this, notify HP of this situation.

• You found a bug in the PDMAN tools (heaven forbid!)

In most cases of an MD5 error, the patch will download successfully the next time depot_update is run. In all cases where there is an MD5 error, the patch will not get copied into your local master depot.

Now that you have grown a complete local master patch depot of all the current HP-UX patches, the next step is to take advantage of this valuable local resource for patching activities. It is very convenient to have only one depot to have to worry about and everyone will know where to look to find the patches they want. The question is, how do you easily select the only patches you want to install off of this massive depot?

You could go into an interactive swinstall session and select the patches you want to install, but if you have more than a handful to install, this could be very labor intensive (currently there are over 1450 HP-UX 10.X patches). On the other hand, you could go into an interactive swinstall session and do a "Match What Target Has" selection. The problem with this is that you will be installing potentially hundreds of patches, many of which you do not want installed on your target machine.

The reality is, you only want to select patches that have been identified via a thorough patch analysis of your target machine to be patched. The mechanism to do this is to create a "Virtual Patch Bundle" using the mk_vbundle command.

All the mk_vbundle utility requires is a text file containing the names of the patches to apply, the name of the local master patch depot, and the name of an output file into which to place the newly created "Virtual Patch Bundle". The syntax is:

mk_vbundle [-M] [-t sess_template] –p patchfile –d master_depot –o vbundle

For example, to make a virtual patch bundle,

/opt/pdman/bin/mk_vbundle –p /tmp/patchfile –d /pdman/master_10X_patchess \

–o /tmp/vbundle.out

where,

-p /tmp/patchfile	Is the name of a file containing the names of the patches you want to install. Lines beginning with a "#" and blank lines are ignored. As long as the patch name is the first field of each line, the file can be processed by mk_vbundle. This file could look something like this: PHKL_11342 PHKL_14332 PHNE_12345 Or, the file could look like this: #--------------------------------------------- # Patch List to fix network errors #--------------------------------------------- PHKL_11342 Fixes panic condition PHKL_14332 Another patch description PHNE_12345 A network patch description Another format for this file is that it can be a "package_info" file that was generated via the HP Electronic Support Center Custom Patch Manager.
-d /pdma/master_10X…	The full path of the local master patch depot
-o /tmp/vbundle.out	The name of the file where you want the virtual patch bundle to be generated. This will actually be a simple text file that will consume very little space on your machine.

You may be wondering what this magic "Virtual Patch Bundle" is or what it looks like. The file generated will actually be an SD-UX swinstall "session" file. The SD-UX session files provide an extremely powerful command line capability for SD-UX tools, but are frequently overlooked and are rarely taken advantage of.

An swinstall session file contains lines for every possible swinstall option as well as the list of selections made during the run of an swinstall session. To see an example of one on your own HP-UX 10.X system, look at

/.sw/sessions/swinstall.last

and you will see a file containing all the information from your last run of the swinstall command.

If there were any errors detected during the invocation of the mk_vbundle utility, the vbundle.out may still be generated, but the word "ERROR" will appear in the file where there were patches that the patch specification could not be generated. These error cases may have to be manually resolved by editing the file, or modifying your patchlist and re-running mk_vbundle.

The –t sess_template switch is provided as an optional parameter to specify a custom built session file. The default one is /opt/pdman/bin/session_template, but this can be copied and modified to set the desired swinstall switches. One example of this would be to include newer swinstall options that are now available in HP-UX 11.00.

Assuming you created a "Virtual Patch Bundle" named vbundle.out and you put this bundle on the machine you want to patch, you can use this file to patch the machine with the following command. Notice that the "–p" option is specified which will only go through the analysis phase of the install WITHOUT going to the execution phase:

swinstall –p –S vbundle.out –s patchsrv:/pdman/master_10X_patches

Note that any command line switches specified in the swinstall command take precedence over switches set within the "Virtual Patch Bundle" file. When you ready to really do the install, remove the "-p" option from the command line.

The mk_vbundle command uses an swinstall session file template that comes with the PDMAN utilities. However, you can use the "-t" switch to specify your own custom template of a session file if you wish. Use the session.template found under /opt/pdman/bin as an example.

Some final comments on the mk_vbundle utility concern the origin of the list of patches specified in the "-p" option of the command. This list can come from any one of several sources:

• An HP Response Center patch analysis

• An HP ESC Custom Patch Manager via the package_info file

• A local patch guru at your company

The key element among all of these sources is that the list must be carefully put together to include not only patches appropriate to the machine you intend on patching, but also must include all dependencies as well. A good patch analysis should be conducted by a person with very good "patch intuition". Patch intuition means that the person putting the list together must know a lot about patching, have the appropriate tools to determine the correct mix and lastly have a good "gut feel" for what should be included versus what should not be included.

The reason this concept of "patch intuition" is being mentioned is because there is NO "patch intuition" built into the PDMAN tools. The "Virtual Patch Bundle" generated by mk_vbundle will include the patches you tell it to include – nothing more, nothing less. Therefore, if you give mk_vbundle a list of inappropriate patches, it will generate an inappropriate "Virtual Patch Bundle". Inappropriate patches may or may not be caught by the swinstall command depending on the circumstances. Make sure that wherever you choose to get your list of patches for the "Virtual Patch Bundle" that it is from a reliable source.

An important part of depot maintenance is a periodic cleaning of your depots. The "depot_clean" utility has several options to clean your depot, depending on what type of cleanup you desire. The command syntax follows:

depot_clean [-M] [-p] [-v] [-a arch_depot] –o|-c|-f patchlist -r hpux_release

–d master_depot

Any time you specify the "-p" option, this is a preview mode that allows you to see what will happen without actually doing any cleaning up.

The "-v" option turns on verbose mode to see everything that is happening. Whether the "-v" is specified or not, all depot_clean logging can be found in /var/opt/pdman/logs.

If the "-a arch_depot" option is specified, then the patches will be copied to the depot indicated by the arch_depot parameter before being removed from the main depot. If the "-a" option is omitted, the patches will be removed without any archiving.

You must specify AT LEAST ONE of the following switches. Using more than one of these at the same time will result in each cleaning method being executed to create a combined list of patches to be removed/archived:

-c	Tells depot_clean to use the HP supported "cleanup" utility with the switch to clean up all superceded patches from a depot.
-o	Specifies that depot_clean should simply compare what is on master depot with what is found in the latest patch catalogs at the HP public FTP patch site and clean off any that are no longer posted by HP from master_depot. The hpux_release parameter needs to be specified so PDMAN knows which patch catalog to compare the depot to (i.e. –r 10 would mean to look at the HP-UX 10.X catalog while –r 11 would mean to look at the HP-UX 11.X catalog)
-f patch_list	Give depot_clean a list of patches in a text file that should be cleaned off of master_depot. This file has the same format as the one that depot_update accepts with its –p patch_list parameter.

The following switches are required:

-r hpux_release	Tells which catalog to look in, as well as which log file to write to in /var/opt/pdman/logs
-d master_depot	This is the full path of the depot that you would like to clean (required parameter).

To archive off patches from the master depot that are no longer in the current HP patch catalogs (preview first),

/opt/pdman/bin/depot_clean –v –p –r 10 –o –a /pdman/master_10X_archive \

–d /pdman/master_10X_patches

To archive off patches from the master depot that are superceded by other patches that are already on your depot (preview first),

/opt/pdman/bin/depot_clean –v –p –r 10 –c –a /pdman/master_10X_archive \

–d /pdman/master_10X_patches

To actually execute the removal/archives, omit the "-p" option from the command line.

You may wonder why you cannot simply use swremove or the cleanup utility to do cleaning of your depots. The answer is that you can use these, however, PDMAN keeps track of patch names and bundle names of product upgrades that are released as patches. Using the PDMAN depot_clean utility will maintain these relationships whereas using the standard swremove and cleanup commands will not. Therefore, it is recommended that you use depot_clean in a PDMAN environment in order to keep track of things correctly. Look in the /var/opt/pdman/products directory to see the types of things that are maintained by the PDMAN utilities concerning bundled products that are released as patches.

A couple of other useful utilities are included in the PDMAN commands. If you have ever gone through the process of trying to find which of your depots contains a patch you are looking for, you will appreciate the "swfind" command. Given a patch name or software product name and a list of potential depots, "swfind" gives back a report of which depots contain which patches or software. For example,

swfind –p product_list –d depot_list

would use product_list to search the depot_list and give back which software is on which depot(s).

Another utility is the swdiff command that can be used to find differences in software products between two SD-UX depots,

swdiff [-l level] depot_or_host1 depot_or_host2

If you are having problems, there are several places to look at what is happening. First look in

/var/opt/pdman/logs

and you will see log files for the different PDMAN commands (depot_clean.10.log and depot_update.10.log). If you do not see what you need to see, another place to look is in the depot itself. The SD-UX commands put an swagent.log file within each depot that has information about all the SD commands that operated on that depot. An example would be for a depot called /pdman/master_10X_patches, look in

/pdman/master_10X_patches/swagent.log

Also note that this file would be a good one to trim periodically, since it tends to grow quite large after a while. A final place to look would be in the standard SD log files found under the /var/adm/sw directory.

Lastly, pay attention to the messages that come from commands that do not have logs in /var/opt/pdman/logs (e.g. mk_vbundle).

The PDMAN utilities are provided for no charge at the following location:

http://www.interex.org/topic/9000/patchtools.htm

Or you can contact one of the authors via email.

In summary, the PDMAN tools should help system administrators in a large enterprise HP-UX environment to access and implement HP-UX patches more easily and efficiently. The following is a brief wrap-up of the PDMAN tools. For more complete details, see the README file that is distributed with the PDMAN tools as well as the Manpages for the individual PDMAN commands. Here is a quick rundown:

• Install the PDMAN utilities

• If the intent is to use PDMAN for HP-UX 10.X and 11.X depots on the same machine, choose a machine with HP-UX 11.X running on it.

• Master local patch depot should have at least 4GB disk space available to start with (current 10.X patches alone take approximately 3.2GB)

• /var/opt/pdman/tmp must have 300 MB disk space available as a temporary staging area to bring down patches from the public HP FTP site.

• FTP must be available on the machine to download the patches from the public HP FTP site

• Initialize and grow the local master patch depot with the depot_update command

• Always use kill –15 when running PDMAN commands. Use kill –9 only as a last resort!

• Set up a cron job to continue to update the local master patch depot periodically (once a day)

• When ready to patch, get text list of patch names (via an expert patch analysis effort) and create a "Virtual Patch Bundle" using the mk_vbundle command.

• Use swinstall to install the patches using the "Virtual Patch Bundle" to select patches off of the master patch depot for the target machine. Use the "-p" switch first to preview the install.

• Use depot_clean to clean out your local master patch depot periodically and possibly archive them to an archive depot.

• Use swfind to find products or patches on a given set of SD-UX depots

• Use swdiff to compare the products and patches between two depots or two machines.

• Periodically trim the /var/adm/sw log files as well as the /var/opt/pdman/logs log files. These are the files that can fill up space if left unchecked.

Good luck and happy patching – if there is such a thing!!!