Patching Quick Reference

Here is a quick reference for the output of currentPatchingStatus.csh, which is run nationally and reported by Sanford Garrard’s on the weekly email to awips_status list-serve.  See Part 2 for instructions on the patching scripts.

These summaries are updated in this folder:
Hazard Services Patching Status folder

The naming is HSmmddyy
Also, the most recent one is usually checked into the OverridesInstaller package as the file globalPatchSummmary.txt

Part 1: Possible Messages in the national file:

1. Patch utility has not been pulled down.

(SVN updates need to be run on the LX, scripts get run on the DX)

1. Summary: The patch utility has not been downloaded from the SCP
2. Action: Download and install the patch utility from the SCP 
3. Action: Download and install the OverridesInstaller from the SCP 

2. There is no internal record for any patch.

It is strongly recommended that confirmAll.csh be run before

rerunning currentPatchingStatus.csh.

4. Summary: There are no records from the patch utility script. Patches may have been installed manually.
5. Action: run confirmAll.csh (user awips on dx3) and rerun currentPatchingStatus.csh

3. Internal records indicate no operational instances of any patch.

        It is strongly recommended that confirmAll.csh be run before

        rerunning currentPatchingStatus.csh.

6. Summary: There are no records from the patch utility script. Patches may have  been installed manually.
7. Action: run confirmAll.csh (user awips on dx3) and rerun currentPatchingStatus.csh

4. Internal records indicate no operational instances of any patch.

1. Summary: There are no records from the patch utility script
2. Action: install necessary patches with patchInstall.csh (user awips on dx3)

5. Patching is completely up to date.

1. Summary: All patches valid for the build have been installed
2. Action: None

6. Deprecated patches still installed:

1. Summary: Patches that now have baseline fixes are still installed.
2. Action: Run patchWithdraw.csh for the listed patches (user awips on dx3)

7. Needed patches not installed:

1. Summary: The listed patches are valid, but not installed.
2. Action: Review the available patches and install with patchInstall.csh (user awips on dx3)

8. Installed patches that need to be updated:

1. Summary: Some patches have been updated since they were initially made available.
2. Action: reinstall the patch with patchInstall.csh (user awips on dx3)

1. Run ./patchInstall.csh nnnnn -r -k

9. 2020_10_07 missing

1. Summary: If three missing patches are listed, the tech order bundle has not been installed

1. Action: run patchInstall.csh 2020_10_07 -r (user awips on dx3)

2. If less than three missing patches are listed, run command(s) shown by running currentPatchingStatus.csh -c

10. Reference documents

1. SCP Patching Manual
2. Quick Reference for Confirming Patch Status

11. Support

1. Jim Ramer (james.e.ramer@noaa.gov)
2. Mark Armstrong (mark.armstrong@noaa.gov)

Example Output from one of the weekly summaries.

Currently these summaries land in this folder:
https://drive.google.com/drive/folders/1swjPnGC8g3FALZP8ZU49QAWqq-rUG63l

These summaries are in documents named like HSmmddyy

ARX

AWIPS release is 19.3.5

Site ARX is at Hazard Services Release 19.3.5

Deprecated patches still installed: 73055@

Needed patches not installed: 76103 76438@ 75208@ 75298@ 73986 57738 76047 75491@ 78126@ 77893@ 82656 80166 75403@ 78706@ 75740 79810 79548

Installed patches that need to be updated: 80308 82905

2020_10_07 missing 82656

Helpful hints:

In these status outputs, the characters @ (at sign), o (lowercase o), or w (lowercase w) are not part of any patch ids; they are information about the state of that patch. See usage information in comments at the top of currentPatchingStatus.csh for more information.

Just because there is a directory present in /localapps/runtime/HazardServices/Patches with content for a patch, it does not necessarily follow that one needs to install that patch.  At a minimum, those directories need to be part of the Patch utility subversion repository for as long as it might be necessary to support being able to withdraw those patches at any WFO.

Running currentPatchingStatus.csh will tell you which patches you need but are not installed.  If you want to know which patches are installed, run recordSummary.csh.

Part 2: Update and Run PatchUpdate scripts: Sites will want to check and patch HS periodically to make sure they have the latest updates. (Consolidated from Above)

Pre-Install Steps (One time Setup completed above in Part 1.1)

• Initial install of OverridesInstaller from VLAB (most likely complete)

• If directory /localapps/runtime/HazardServices/OverridesInstaller/ does not exist, then this step is needed.

• Initial install PatchesPackage from VLAB (most likely complete)

• If directory /localapps/runtime/HazardServices/Patches/ does not exist, then this step is needed.

• dx1(awips): chmod -R 775 /localapps/runtime/HazardServices  

Step 1: Update Patching Scripts
             (Assumes these are already pulled down from SCP)

• lx1(awips): cd  /localapps/runtime/HazardServices/Patches
• lx1(awips): ./updatePackage.csh
• IF updatePackage.csh DOES NOT EXIST, THEN:
• lx1(awips): ( umask 002 ; svn update )
• lx1(awips): ./updatePackage.csh^[a][b]
• If updatePackage.csh fails to run, the most common reason is problems with your Patches and OverridesInstaller subversion repositories. When this happens, see the section about troubleshooting approaches.

Step 2 Check your current patching status

• dx3(awips): cd  /localapps/runtime/HazardServices/Patches
• dx3(awips): ./confirmAll.csh
• dx3(awips): ./currentPatchingStatus.csh -c
• The script will output a list of commands to run to update patches. Run them one at a time.  Pay close attention to any detailed information that needs to be followed; if an install or withdraw operation fails to complete, this means you have encountered either a “covering” conflict or a direct conflict.

• If no commands are listed, then you are up to date.
• Note that when you are requested to run commands with a -k option, you may see a line of output like the following; this is not a problem.
  RECORD WAS OUT OF DATE IN THIS REGARD.
• As patching commands run, you will often see a message like this:
  CHANGE IN region LEVEL FILE(S) MADE THROUGH FILE SYSTEM,
  BOUNCE CAVE TO HAVE THIS TAKE EFFECT.
  This applies both to the functioning of Hazard Services and to how these files are represented in the localization perspective.

• dx3(awips): ./currentPatchingStatus.csh -c (Run again to make sure you are done)  

• If no commands are listed, you are up to date

Dealing with covering conflicts when installing a patch.

The text that follows is an example of the kind of response one sees if a “covering” conflict is encountered:

If this is encountered, two steps need to be taken.  First, respond
-m -y
in order to complete the installation of the region level files for the patch. Once this is done, the patch will still not be in effect when generating hazards for the site(s) for which there are covering conflict files (e.g. OAX in the example above). The patch will be in effect immediately when generating hazards for other sites.

Second, check the status of the covering conflicts involved, and clean them up if necessary, by running the commands that follow.

cd /localapps/runtime/HazardServices/OverridesInstaller
./coverSniffer.py nnnnn

If any of the methods referred to by coverSniffer.py have a * on them, this means that the covering method is a duplicate of the method being covered.  This is very easy to clean up, just run:

./coverScrubber.csh nnnnn

If there are methods referred to by coverSniffer.py that do not have a * on them, they need to have the patch functionality in the region level override merged into the covering level method. Running the command that follows will provide assistance in performing this task:
./coverScrubber.csh -m nnnnn

This should produce a directory named like the following with proposed updated overrides:
/localapps/runtime/HazardServices/mergesRootPPPPP/
Note that PPPPP is not literal, nor is it the patch id; it is a process id to ensure unique naming.

Run the following command to check whether any of the proposed updated overrides have text merge conflicts:
./manageCoverConflicts.csh nnnnn -v

If text merge conflicts exist, edit the indicated files to deal with those conflicts. There is guidance about dealing with text merge conflicts here.  Do not follow any of the instructions shown that involve running clearBackupOverrides.csh.  As you edit the proposed updated overrides, leave in place any comments like the following:
#@# Merged automatically with the same method in

Once you are sure there are no remaining text merge conflicts, you can install the proposed updated overrides. If there is just one or two, which will usually be the case with a cover conflict for a specific patch, it may be just as easy to paste them into the localization perspective. The proposed updated override has the entire content of the override, so remove the entire existing override before pasting in the updated content.  The advantage of this is that manageCoverConflicts.csh installs site overrides directly through the file system, which can leave them out of sync with what your Cave sees and won’t share the update by way of Backup Services.  Otherwise run manageCoverConflicts.csh without the -v option, e.g.:
./manageCoverConflicts.csh nnnnn

If there are any doubts about how to complete the cleanup of a covering conflict, contact someone for support. If you are dealing with covering conflicts for multiple patches, be sure to deal with them one at a time. And once the conflict has been fully resolved and the updated overrides have been installed, the /localapps/runtime/HazardServices/mergesRootPPPPP directory should be removed.

Dealing with direct conflicts while updating patches.

A direct conflict while updating a patch means the patching action is attempting to update or remove an “unknown” instance of that method.  In this context, “known” means one of the following:

• The same as the method that is in the patch.
• The same as the previous method that was in the patch (assuming the current instance of the patch represents an update).
• The same as the current base level instance of that method.

A direct conflict is essentially anything that is not a “covering” conflict. When you experience a direct conflict the first thing you need to do is to identify the current override file that is in conflict. In the file /tmp/hsAutoinstall_awips_logfile.txt, any file identified for which the path starts with /awips2/edex/data/utility/common_static/ represents an override file with a direct conflict.

When a direct conflict is encountered, the patch scripting will pause before completion, and will ask you to respond to this:
  Enter new installer args to use:

First use the localization perspective to make a temporary copy of any conflicting override at the user or workstation level.

Next, if one needs to complete the installation of a patch, respond:

  -a -y

and if one is trying to complete a withdraw of a patch, respond:

  -e -y

Finally, you need to inspect the content of your temporary copy to determine if there is any logic present that is not patch logic and you wish to restore. If there is no logic you wish to restore, just delete the temporary copy.  If there is logic you wish to restore, how you restore it depends on whether you were installing or withdrawing a patch.  If you were withdrawing a patch, what you restore should be your local effects applied to the base level instance of the method(s) being restored. If you were installing a patch, what you restore should be your local effects applied to the patch instance of the method(s) being restored.  The file(s) with the patch instance will also be identified in /tmp/hsAutoinstall_awips_logfile.txt.  In either case, the file so restored should typically be installed at the site level.  Any region level instance(s) should be left in place to implement any needed patch(es) to backup sites.

Troubleshooting problems with subversion.

The various troubleshooting approaches here are only to get your set of patch utility files up to date with the latest contents of the VLab SCP subversion repository.  These commands should be run on an lx host.  To actually install or withdraw patches, be sure to run those commands on dx3 as user awips.

Troubleshooting approach # 1

Troubleshooting approach # 2

Troubleshooting approach # 3

Troubleshooting approach # 4

Troubleshooting approach # 5

Troubleshooting approach # 6