Chapter 3: Exercises and Examples     contents  ⇑ ↑  ^   v  ↓  ⇓

3.0 Configuration     contents ⇑  ↑  ^   v  ↓  ⇓

3.0.1 Settings     contents ⇑ ↑  ^  ─  +   v  ↓  ⇓

By default initial setting, we mean the setting file that is used on startup in the situation where this had not been specifically defined. The initial setting can be specified by creating a user version of StartUpConfig.py where the item "defaultSettings" is set to the desired initial settings file. This can now also be done using SETUP->Manage Settings->Make My Current Default in the console.

NOTE: PUTTING A FILE IN THE DIRECTORY THAT StartUpConfig.py LIVES IN THAT IS NOT PRECISELY NAMED StartUpConfig.py WILL CAUSE A CATASTROPHIC FAILURE STARTING UP HAZARD SERVICES. (HOPEFULLY, THIS IS TEMPORARY BEHAVIOR).

(The new Hazard Services Overrides Installer available in the SCP has safeguards against this problem.)

Be aware that there is now a ‘universal generic base’ for all settings files, CommonSettings.py, found in /awips2/edex/data/utility/common_static/base/HazardServices/settings/config/. Also be aware that the way specific settings files interoperate with CommonSettings.py is still in flux.

It is possible to use the localization perspective to manipulate settings. However, for most things one might want to do with settings it makes more sense to use the Edit Setting: GUI (at SETUP->Manage Settings->Edit/Filter… in the console). See section 5.0 of the User Guide for more information about this. There are exceptions to this, including promoting custom settings to site level , configuring CommonSettings.py, and removing a base level default setting file.  For completeness, we will show some examples of using the localization perspective to manipulate settings.

3.0.1.1 Modifying an existing Settings file     contents ⇑ ↑  ^  ─  +   v  ↓  ⇓

• Locate the Settings file in the Hazard Services section of the Localization Perspective e.g. Hydrology_All.py
• Open the BASE version in the editor. Let’s suppose for the sake of argument that what you want to do is remove “Stream” from the set of default visible columns for that setting. Here is the part of the base Hydrology_All.py which defines this:

• From the Browser, Click MB3 over the BASE version and select Create Override File-->Site

• Copy and paste the top level Settings dictionary into the Override file e.g.
  Hydrology_All = {
  }
  (if the site file is already there, then for this exercise you are going to add only the "visibleColumns" section to it)
• Copy and paste the section you would like to modify e.g. “visibleColumns” and then modify as desired:

• Save the file in the Localization perspective and your change will be enacted.

Now you can restart Hazard Services and verify that the Stream column is gone, right? Wait a minute, the Stream column is still there...what happened? This is a result of the fact that this file is subject to incremental override. Incremental override is why one does not have to copy the whole base file to the site file. However, for incremental override, the default behavior for two lists at the same namespace is to merge the lists rather than to replace the list in the base file with the list in the site file. You have two options in the site file that can change this default behavior, as follows:

3.0.1.2 Creating a new Settings file   contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

This can be done in two ways; through the Edit Settings: dialog or through the localization perspective. Using the Edit Settings: dialog has the advantage of it being very hard to shoot yourself in the foot. Using the localization perspective has the advantage that the resulting settings file can be made ‘incremental’ (e.g. you only define the things you want to change.)

Here is how this can be done through the Edit Settings: dialog.

• Choose SETUP -> Manage Defaults -> Edit / Filter … from the top of the Hazard Services Console to open the Edit Settings: dialog.
• In the Display Name: text field, enter the desired title of your new setting.
• Using the selectors in the various tabs (Hazards Filter, Console, Recommenders, Maps/Overlays), make any desired changes to the settings information.
• Hit the Save As … button at the bottom of the dialog; this will pop a dialog where the new file name for the setting can be entered.
• Enter the new file name for the setting in the text field and then hit OK.

Here is how this can be done through the localization perspective:

• Go into the localization perspective and open Hazard Services -> Settings.
• Identify the existing default setting that is most similar to the new setting you wish to create.
• Open that setting file, and on the BASE level selector, choose Copy To->New File… using a third button popup.
• Enter a unique file name with a .py extension.
• Open the newly created setting file, which should only have a USER level selector.
• Double click on the USER level selector for the new setting file to bring the file into the edit area.
• Change the “displayName”: attribute to what you want the title of that setting to be in the console.
• This is very important; change the variable name in the first line (before the = { ) to be exactly the same as the setting file name without the .py extension.
• Make any other desired changes to the settings information.
• Hit ctrl-S to save the new setting file, and go back to the D-2D or Hydro perspective.
• Choose SETUP->Load… and invoke your newly created setting file.
• Choose SETUP->Manage Defaults->Edit… and make any necessary changes to configure the new setting.
• Hit the Save button.

3.0.1.3 Promoting a settings file to Site level.   contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

It is perfectly reasonable to use the dialog to initially create a new settings file, and then to later go into the localization perspective to further modify the settings information. Also, be advised that both of these means for creating a new settings file will create a USER level settings file. If it is desired to make the new settings file a SITE level settings file, this can be accomplished through the localization perspective as follows:

• Go into the localization perspective and open Hazard Services -> Settings.
• Find the entry for your new settings file and open it.
• Choose Copy To -> Site using a third button popup over the USER entry for your new settings file.
• Choose Delete using a third button popup over the USER entry for your new settings file.

3.0.1.4 Console Time Window.   contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

A recently added feature is to allow a setting file to specify the range of hours offset from the current time for which to display hazards in the Console. This currently defaults to essentially an infinite time range in all default settings files. Some sites or users may find it more convenient now for default settings files to make ALL Status values visible and apply a reduced size console time window. What follows is an override for CommonSettings.py that can accomplish this.

With this example override file, hazards valid in the future will appear in the Console essentially forever, but previous hazards will appear in the Console only for the last 8 hours (e.g. one shift). If there is a desire to have this apply to all users, installing this as a Region level override is a reasonable way to do this.

Note that the Console Time Window can be controlled using the Console tab of the Edit Settings: dialog.

3.0.1.5 Remove default base setting file.   contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

New settings files created by the user that are not overrides of an existing default base setting file are straightforward to remove.  If they are implemented at the User level, this can be done either using the localization perspective or by way of SETUP->Manage Settings->Delete.  At the Site level, one can still use the localization perspective, but at the Configured or Region level this needs to be done using the file system.  However, consider the situation where one wants to remove a settings file that was delivered as Base level default settings file.  This cannot be done by overriding or undoing an override of the setting file itself; this must be done using an override of StartUpConfig.py.  Suppose one wanted to remove the Hydrology_ESF settings file.  Here is an override of StartUpConfig.py that can accomplish this:

CAUTION!!! It is very important that one provide a list (can be empty) of both “settings” and “recommenders” in each turn off set, even if one is not removing any items of that type. Failure to do so will result in a catastrophic failure starting Hazard Services.

3.0.2 StartUpConfig       contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

NOTE: PUTTING A FILE IN THE DIRECTORY THAT StartUpConfig.py LIVES IN THAT IS NOT PRECISELY NAMED StartUpConfig.py WILL CAUSE A CATASTROPHIC FAILURE STARTING UP HAZARD SERVICES. (HOPEFULLY, THIS IS TEMPORARY BEHAVIOR).

(The new Hazard Services Overrides Installer available in the SCP has safeguards against this problem.)

Here we show how to modify an entry in StartUpConfig.py. Note that StartUpConfig.py is subject to incremental override, and so one only need specify the items to change; there is no need to copy the whole base file.

• Locate the StartUpConfig file in the Hazard Services tab of the Localization Perspective and open the BASE version
• Open the BASE version in the editor and locate the section you would like to modify e.g. “Console”
• From the Browser, Click MB3 over the BASE version and select Create Override File-->Site
• Choose the Site version just created in the localization file browser to bring it into the localization file editor. Put in the following content, and it will make ‘Hydrology_All.py’ be the settings file used by Hazard Services upon starting:

• Save the file in the Localization perspective with ctrl-S and your change will be enacted.

Note that the console control SETUP->Manage Settings->Make My Current Default is just like entering your current setting as the default setting for an override of StartUpConfig.py. Also note that this operation results in an incremental version of StartUpConfig.py being written out.

3.0.3 Hazard Types       contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

3.0.3.1 Modifying a Hazard Type entry     contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

Here we show how to modify a Hazard Type entry. These entries occur in the file HazardTypes.py. Note that HazardTypes.py is subject to incremental override, and so one only need specify the items to change; there is no need to copy the whole base file.

• Locate the HazardTypes.py file in the Hazard Services / Hazard Types section of the Localization Perspective
• Open the BASE version in the editor and locate the item you would like to modify e.g. “defaultDuration” for FA.A
• From the Browser, Click MB3 over the BASE version and select Create Override File-->Site
• Choose the Site version just created in the localization file browser to bring it into the localization file editor. Put in the following content; it will change the default duration for FA.A hazards:

• Save the file in the Localization perspective with ctrl-S and your change will be enacted

Here is a related example for a hazard type that has a list of durations to pick from. Suppose that it is desired to allow Flash Flood warnings of a shorter duration (this is often a convenient way to test the behavior of hazards as they approach their end time). Here are two possibilities for what the content of the override file could look like in that case:

Because HazardTypes.py is subject to incremental override, the default behavior for two lists at the same namespace is to merge the lists rather than to replace the list in the base file with the list in the site file. The first example override file tells the system to prepend  '15 min',  

‘30 min’, and  '45 min' to the list of durations, whereas the second example starts by stipulating that the supplied override list should replace the base list.

3.0.3.2 Completely removing an existing Hazard Type.     contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

In this section we describe how to completely remove an existing default hazard type from Hazard Services. We make the assumption that if one wants to remove a user defined hazard type from Hazard Services, one can simply undo the steps described in section 4.2 which were used to add that hazard type in the first place.

In the section entitled disabling the ability to issue a hazard type, we describe how to make a hazard type unissuable. Making a hazard type unissuable retains the ability to view hazards of that type from neighboring offices and the ability to manage that hazard type in the settings. The changes documented in this section result in eliminating all ability for Hazard Services to interoperate with hazards of that type. This means hazards of that type from neighboring offices cannot be viewed, nor can this hazard type be managed in the settings.

Here we will demonstrate specifically how to eliminate the Flood Advisory (FL.Y) hazard type.
We will start by eliminating it from HazardCategories.py. The approach is to create a site override for HazardCategories.py containing the following:

HazardCategories.py is subject to incremental override, and so there is no need to replicate the whole file, we just need to mark FL.Y for deletion from its category, "Hydrology". Were an override for HazardCategories.py already present, its new content would need to be a merge of the immediately preceding example and its current content. If an override file for HazardCategories.py had been used to reassign FL.Y to a different category, updating the override file would need to include removing FL.Y from its new category as well.

Hazard Service’s decisions about which hazard types are important to the console are completely driven by the content of your current setting, and the code that manages the settings files is smart enough to completely throw on the floor any hazard type that does not have a category.  Functionally removing a hazard type from Hazard Services does not require removing the main entry for that hazard type from HazardTypes.py. In fact, attempting to do this would likely cause more problems than it would solve and therefore is not recommended.

This does not mean that no changes at all are necessary in HazardTypes.py. Once a hazard type has been ‘orphaned’ from all categories, one needs to remove all occurrences of that hazard type from 'replacedBy': lists.  This means an override for HazardTypes.py needs to be created where there is an entry for each hazard type for which the hazard type being removed appears in the base version’s 'replacedBy': list.  Again using FL.Y as an example, such an override would contain the following:

(In 20.2.1, work against #75201 will eventually mean the updates for the ‘replacedBy’ lists will be unnecessary.) Since a single hazard is being removed from 'replacedBy' in these instances, one could technically use _override_remove_ instead of _override_remove_list_.  However,  _override_remove_ only removes one item, and having _override_remove_list_ in place sets you up to also remove any additional items you might add to the end of list, making it harder to overlook this.

 As mentioned, it is true that removing a hazard type from all categories functionally removes that hazard type from the console. The same cannot be said, however, for any recommenders that might generate hazards of that type. Nor is this true for the logic that displays hazards.

For the sake of argument, let us stipulate (not true) that the RiverFloodRecommender recommends hazards only of type FL.Y and no other types. Therefore, all references to that recommender need to be removed from all our settings files. Plus, we want to remove all references to “FL.Y” from the settings files. These four commands allow one to easily identify any setting files for which this is an issue:

> touch /tmp/bbb

> cd /awips2/edex/data/utility/common_static

> find . -name '*.py' -path '*/settings/*' -exec grep 'RiverFloodRecommender' '{}' /tmp/bbb \;

> find . -name '*.py' -path '*/settings/*' -exec grep 'FL.Y' '{}' /tmp/bbb \;

For any base settings file identified where there is no corresponding site override, one needs to create a site override containing what follows. The italicized text is not literal, but rather is the name of the settings file minus the .py extension. Please note that while we are using the file system to identify those settings files that we need to change, creating and/or updating the site override file should be done in the localization perspective.

For any override settings file where there is no corresponding base file, one simply needs to remove any instance of "RiverFloodRecommender" from the "toolbarToolNames" , plus any occurrence of “FL.Y”. The same goes for any override settings file where there is an entry of “_override_replace_”: True, at the top level. Otherwise the modified override settings file needs to be a merge of the immediately preceding example and its current content.

In addition to removing a default recommender from all settings, one can in theory also completely remove it from the system, which needs to be done by way of StartUpConfig.py. See section 3.0.1.5 for more information.

As previously mentioned, it is not actually true that the RiverFloodRecommender recommends hazards only of type FL.Y. Therefore, if one were actually completely removing the hazard type FL.Y from Hazard Services, one needs to go into the recommender itself and disable its ability to interoperate with hazards of type FL.Y. For this particular recommender, this is detailed in section 3.3.3. For other recommenders that recommend multiple hazard types, the details would be different. It is also possible for a hazard type to have no associated recommender. In that case, none of the override settings files would need any entries for "toolbarToolNames".

3.0.4 FilterIcons.py     contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

The purpose of FilterIcons.py is to allow icons to be placed on the left side of the console tool bar that indicate when hazard events meeting certain criteria have been filtered out. One can see the default version of FilterIcons.py in the localization perspective under Hazard Services -> Hazard Categories. What follows is a sample override for this file. This file is subject to incremental override, and so to eliminate the default entry the list of entries begins with '_override_replace_' . If there was a desire to only add new entries, those entries could be placed in the override file and the default entry would still be in effect as well. The image referred to with the 'normalIcon' element is what shows up when no events meeting the criteria have been filtered out.; the 'coloredIcon' element indicates the image shown when there ARE events being filtered out that meet the criteria.

3.0.5 Highway Mile Markers in Flash Flood Warnings     contents ⇑ ↑  ^   v  ↓  ⇓

Note that it was recently discovered that certain implementations of mileMarkers.xml and mileMarkers.vm can result in problematic content for PointMarkerConfigs.py after running parseWarngenTemplate.py with the -m option.  This can manifest as route entries with an empty string for the “type”: element.  If there are any problems with the mile marker functionality, first download the most recent version of the parseWarngenTemplate.py script (see  section HH)  and try running parseWarngenTemplate.py again.

Hazard Services by default supports the ability to list the mile markers of impacted highways in flash flood warnings. Hazard Services leverages much of the same scripting and data structures to do this that warnGen does. For both Hazard Services and warnGen, this functionality is dependent on creating a table in postgres for each route.   These tables appear in the mapdata schema of the maps database.  For warnGen, the name of each table that supports the mile marker functionality is placed in an xml file; for Hazard Services, each table needs to be inserted into an override for the file PointMarkerConfigs.py.  

Here is an example of an entry in PointMarkerConfigs.py that supports the mile marker functionality for one route:

Note that the content in blue is the structure within which the content for each route exists, the content in black is what is specific to the route for this example.  Also note that the site ‘oax’ is embedded in the table name entry. Unlike dam break or burn scar information, there is no formal segregation in postgres according to CWA for this information.  Therefore it is a good idea to always embed the CWA name that the route exists within as part of the table name.

For now lets assume the mile marker functionality is already working in warnGen, and has been implemented for all the markers along all the routes you care about at your site.  If this is the case, there should already be a postgres table for each route, and so one only need create an override for PointMarkerConfigs.py to leverage those.  A python script has been updated to do most of this work for you.  To invoke this script, become user awips on dx3, and run these commands:

> cd /awips2/edex/scripts/HazardServices/

> /awips2/python/bin/python parseWarngenTemplate.py -m

With the -m option, parseWarngenTemplate.py reads the files mileMarkers.vm and mileMarkers.xml from site/LLL/warngen/ and outputs an override for PointMarkerConfigs.py in the following directory (LLL is not literal, but is your primary site):

common_static/configured/LLL/HazardServices/python/events/productgen/geoSpatial/

You can run parseWarngenTemplate.py with no arguments to get usage information.  You can also supply a backup site id on the command line to run it for any arbitrary backup site. Note that if this conversion step fails to create a useable override of PointMarkerConfigs.py, it is always possible to compose these entries for yourself, following the example shown.  If for some reason you want to add mile markers for an additional route not yet implemented in your warnGen, that will also require composing one or more of these entries.

If after running the default conversion, the mile marker functionality in hazard services is working without any problems, then it would be reasonable to promote the configured level of PointMarkerConfigs.py to the site level and remove the configured level.  The only reason to keep the configured level file around would be if you think that you may need to rerun the conversion for some reason.

If you decide to have both a configured and site version of PointMarkerConfigs.py, there are some things to be aware of.  The set of route table entries is a list, and so the default behavior is to add the entries in the site version to those in the configured version.  If adding new route entries is all you want to do with the site override, then the default behavior will be fine.  If in any way you wish to use a site override to “fix” something in the configured override, you will need to use control strings in the site override, following one of these approaches:

Note that the italicized text is not literal. For the first approach, it is as if the configured version is not even there, except that it provides you with a record of the raw conversion. The other two approaches allow you to apply incremental overrides to the individual route table entries, using either the “tableName” or “displayName” value to define the namespace.

In the case where the mile marker functionality is already working in warnGen, all the postgres tables referred to by the version of PointMarkerConfigs.py created here should already be in postgres.  If mile markers for some or all of the routes are not showing up in FFW text, this means that some or all of the postgres tables are not installed.  If this is the case, here is what to do.  First, identify all the tables referred to in all your overrides of PointMarkerConfigs.py using these commands, best run as user awips on dx3:

> cd /awips2/edex/data/utility/common_static

> grep tableName */LLL/HazardServices/python/events/productgen/geoSpatial/PointMarkerConfigs.py

Next, identify all the tables in the mapdata schema of the maps database.  The command that follows will do this (leave off the -U awips -hdx1f in a dev environment):

> psql -U awips -hdx1f -d maps -c '\dt mapdata.*'

For any table referred to in an override of PointMarkerConfigs.py that is not in postgres, one either needs to remove all entries that refer to it in any overrides of PointMarkerConfigs.py, or add it to postgres.  To add such a table to postgres, one first needs a file that contains a list of mile markers for a single route in a very specific format.  What follows is an example of such a file.  The name of the file does not matter except that it helps keep this information organized in a meaningful way for the person managing this.

highway92_oax.id :

The script used to install this information as a postgres table is available only on dx1, and should be run as user awips. The full path to this script is: /awips2/database/sqlScripts/share/sql/maps/importMarkersInfo.sh

The importMarkersInfo.sh script is very particular about this format, so it is recommended to not deviate from it. If you have already successfully installed mile markers for warnGen, then you will likely already have a set of these ID files that you can reuse for creating mile marker postgres tables for Hazard Services if necessary. The first line in the ID file always contains the number of entries in the file. The ‘p’ and the ‘|1’ are literal and should always be placed in each line exactly as shown. The first and fifth whitespace delimited columns are sequence numbers and should always count up. Just before the vertical bar are the mile marker values; these are arbitrary strings. The latitude and longitude values are self explanatory.

Here is how this script is run to create a postgres table:

> importMarkersInfo.sh /path/to/markerfile.id mapdata tableNameInPointMarkerConfigs.py

mapdata is literal. This results in a new table being created in the mapdata schema of the maps database with the same name as the final argument. Unlike dam break or burn scar information, there is no formal segregation in postgres according to CWA for this information.  Therefore it is a good idea to always embed the CWA name that the route exists within as part of the table name.

The very first time importMarkersInfo.sh is called for a given table name, you might see a message like:
    NOTICE:  table "highway92_oax" does not exist, skipping
This seems bad, but does not necessarily mean the script failed. It may mean simply that it ‘skipped’ the step of deleting the table before reinstalling it.

3.0.6 Other arbitrary Markers in Flash Flood Warnings.     contents ⇑ ↑  ^   v  ↓  ⇓

Hazard Services implements the ability to mention when the location of other arbitrary markers occur within the bounds of a Flash Flood Warning. Here we will present an idealized example of this. It starts with a site override of PointMarkerConfigs.py:

With this in place, the Additional Locations: section of the Hazard Information Dialog for a Flash Flood Warning will have a checkbox labeled "Public Events" (e.g., the value of the “pointSpecificLabel" key). In order for this to have useful functionality, the importMarkersInfo.sh script must be used to create a new table in the mapdata schema of the maps database with the name "publicevents" (e.g. the value of the "tableName" key). Suppose the following invocation of importMarkersInfo.sh was used:
> importMarkersInfo.sh /tmp/publicEvents.id mapdata publicevents

Furthermore, suppose that the contents of publicEvents.id were the following:

Finally, suppose that one created an FFW where the warning box contained both of those latitude/longitude points and the "Public Events" checkbox was activated in the Hazard Information Dialog. The resulting text product would contain the following small paragraph:

This includes the following Public Events...

        Little League Finals and Regional Tractor Pull.

3.1 Recommenders     contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

Please see this presentation to become familiar with the Recommender Framework. Also, please see the section on how to configure a Recommender Dialog using megawidgets.

3.1.1 Modifying an existing Recommender     contents ⇑ ↑  ^  v  ↓  ⇓

• Locate the Recommender you want to modify in the Hazard Services / Recommenders section of the Localization Perspective e.g. RiverFloodRecommender.py
• From the Browser, Click MB3 over the BASE version and select Create Override File-->Site.
• A skeleton Recommender class will appear and you can override methods from the BASE class. At the end of the RiverFloodRecommender class is a set of methods designated as typical overrides: “getWarningThreshold”, “isWatch”, “isWarning”, “isAdvisory”. To change the warning threshold hour from 24 to 20, include the altered method in your override file:

• Save the file in the Localization perspective and your change will be enacted

3.1.2 Interdependencies in a Recommender.     contents ⇑ ↑  ^  v  ↓  ⇓

Just as with metadata, it is possible for megawidgets associated with a recommender to be subject to an interdependency script. Here is an example of a recommender customization that makes use of an interdependency script. What this does is add one or more extra buttons to the River Flood Recommender GUI that activate any arbitrary list of gauges you want. These can be individual gauges or a set of river groups (hence the terminology “super group”). For any specific WFO, the logic in assignSuperGroups() would need to be customized for that WFO; the rest of the logic should work fine exactly as is. This example is for WFO OAX, which has rivers in Nebraska, rivers in Iowa, and the Missouri River which flows along the border. Note that the super group buttons can only turn on gauges, not turn them off. It would be possible to update this logic to be able to turn off super group gauges, but that would require work beyond modifying the assignSuperGroups() method. There are some commented out diagnostics in this code example; those could be uncommented to help understand what is going on if difficulties were encountered implementing this.

3.1.3 Creating a new Recommender     contents ⇑ ↑  ^  v  ↓  ⇓

Consider a case where a particular dam was at imminent risk of failing and producing catastrophic impacts. Here we demonstrate how one can add a new recommender for the purpose of having a recommendation for that specific dam ready to go on the very top of the Tools menu.  For this example, we are using a dam called “Old River Dam”, which is an actual dam that was configured for warnGen in a sample damInfo.vm and damInfoBullet.xml we were given from a WFO for testing.  First thing is to create an instance of the new recommender. Go into the localization perspective to:
    Hazard Services->Recommenders->DamLeveFlood.py->BASE
Using the third button popup on that selector, choose Copy To->New File… In the dialog that comes up entitled ‘Rename File’, type in the name of the new recommender, which for this example is OldRiverSpecial.py, and hit OK. What follows is the content of OldRiverSpecial.py for this example.  Of course, if you were doing this for one of your own dams, the name of the recommender and some of these changes that follow would be different. Also, this creates a user level override. Once these changes were tested, they would need to be promoted to site level overrides if the new behavior was something that was meant to support operations. We show the differences from the default dam break recommender, DamLeveeFlood.py, in the customary green.

OldRiverSpecial.py:

The next thing is to register this new recommender and give it a suitable title on the Tools menu.  This is done with an override of CommonSettings.py. Here we leverage the default incremental override behavior of lists to keep this override very simple.

Another thing needed is to supply a DamMetaData entry for this new dam key, "Old River Dam Special". First we show the existing entry for this dam so that one can understand what is being done different for the new entry:
"Old River Dam": {

            "riverName": "Old River",

            "damName": "Old River Dam",

            "cityInfo": "Manawa, located just below",

            "dropDownLabel": "Old River Dam",

            "featureID": "OldRiverDam"

        },

What follows is the override for DamMetaData.py.  Note that the value for "damName", which is put into product text, is still the same.  Also note that the the "ibwType"  is set to 'catastrophic', which will cause it to start out as being an EMERGENCY hazard, and that a response scenario has been added. That scenario was made up to support this example; any scenario for operations would of course need to be valid for the situation, and it is not mandatory to add one.

Finally, this recommender needs to be added to any settings that it is important for. While straightfoward, this can be the most challenging aspect of this because it can be tedious. Over time, many users will create their own user instance of some of the default settings, or perhaps their own customized settings, by way of the Edit Setting: dialog.  The way these user settings files are written out, the list of recommenders to have under the Tools menu for the setting is specified in a completely non-incremental way.  That means that once such a user instance of a setting is created, it becomes impossible to affect the list of applicable recommenders for that setting by way of a Region, Configured, or Site override of that settings file. So in the case where a new customized setting is introduced, the only practical way to deal with making it available is to have each user go into the Recommenders tab of the Edit Setting: dialog, and manually add the new recommender to each setting where it is important, and then save the updated instance of those settings.  While doing this, a particularly critical recommender can be moved to the very top of the list of applicable recommenders, so that it is right at the user’s fingertips.

3.2 Hazard Metadata     contents ⇑ ↑  ^  v  ↓  ⇓

At the beginning of hazard metadata modules, generally a variable self.hazardEvent is set for the class; this contains a local working copy of the hazard event the metadata is being generated on behalf of. It is very common for metadata logic to inspect the contents of this data structure during the creation of the metadata. However, metadata logic should never change the contents of self.hazardEvent.

Another caution; if one is using the “refreshMetadata” feature to modify an existing megawidget that retains the same "fieldName" attribute, great care must be taken if one is also changing the "fieldType". In general this is not recommended.  More about this in section 3.2.7.

When errors involving executing metadata show up in alertviz or console logs, the way they are reported is a string representation of the entire returned metadata (which can be a huge amount of text), followed by a description of the error encountered.  So one needs to pay close attention when scanning such an error message for the error description. Note that if a method in a metadata class constructs a proper megawidget element but neglects to actually return it, this will not cause a python compile error.  Instead the metadata will execute and an error message as just described will be output;  the error description in this case might only make a vague reference to some of the content being “null”.

3.2.1 Examples of changes in Calls to Action     contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

In this section we will present some customization examples for calls to action. When making modifications to calls to action, it is very important to be mindful of the distinction between the getCTA_Choices() method and the getCTAs() method. getCTA_Choices() is where the list of all calls to action that apply to a hazard is specified. getCTAs() packages those choices as a checkbox menu and controls which are activated by default.  Also be aware that the methods that create the megawidget element for each specific CTA are neither in CommonMetaData.py nor in any of the individual metadata modules; they are all in the module CallsToActionAndImpacts.py.

3.2.1.1 Change the wording of a Call to Action  contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

This example is to change the wording of the “Nightime flooding” call to action. First, one needs to find which python module this call to action is defined in. These commands will accomplish this (see Introduction for more information about searching code for functionality):

> cd /awips2/edex/data/utility/common_static/base/HazardServices

> find . -name '*.py' -exec grep -i 'Nighttime flooding' '{}' \; -print

In this case the file identified will be CallsToActionAndImpacts.py; the base version of all call to action methods are defined here. Search the module CallsToActionAndImpacts.py for the string “Nightime flooding”, and you will see that the method involved is ctaNightTime(). To implement the wording change for all hazard types, create an override file for CallsToActionAndImpacts.py with the following (change from base in green):

Now suppose there was a desire to change the wording of this call to action for only one hazard type; for sake of argument let’s say convective FFWs. Here is what this override would look like in this case:

What is happening here is that the traceback utility is used to determine whether this method is being invoked by the MetaData_FF_W_Convective.py metadata class, and updating the productString accordingly if this is the case.

3.2.1.2 Change list of Calls to Action for a Hazard     contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

This customization example will be to change the the default call to action choices for a particular hazard type. Specifically, we will add the "Turn around...don't drown" call to action to the list available for the Areal Flood Watch. Use commands similar to those in the previous example to learn that the default version of this call to action method is implemented CallsToActionAndImpacts.py; inspect CallsToActionAndImpacts.py to learn that the method name is ctaTurnAround(). The specific metadata method for Areal Flood Watch is MetaData_FA_A.py. To make this change, you need to provide an override for the getCTA_Choices() method of MetaData_FA_A.py as follows (change from base in green):

3.2.1.3 Change Calls To Action activated by default     contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

This customization example will be to make the "Stay away" call to action be activated by default for an Areal Flood Watch. First, one needs to locate the call to action method in CallsToActionAndImpacts.py (ctaStayAway) and note the value of the identifier specified in that method, which is "stayAwayCTA". Then we need to create an override for the getCTAs() method of MetaData_FA_A.py. Note that the default instance of the getCTAs() method for MetaData_FA_A.py is actually in CommonMetaData.py. The content of this override follows (change to base instance in green):

Of course, if there were already a site version of MetaData_FA_A.py, the instance of getCTAs() shown would be added to that. The only code added to this is the code in green, which makes the CTA with the identifier of "stayAwayCTA" the one that is turned on by default for a new hazard. The two lines of code immediately preceding that allows an existing set of activated CTAs selected by the forecaster to be propagated to the next product in the life cycle.

3.2.1.4  Remove an existing default Call To Action  contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

In this example we remove any calls to action referring to arroyos from those available for convective Flash Flood warnings. For this example this will be approached differently from other examples where the set of available calls to action were changed; we will show how to implement this by overriding getCTAs(). Note that the base version we are overriding here is originally from CommonMetaData.py.

What follows is new site version of MetaData_FF_W_Convective.py for this purpose. The code added or changed is in green as before. The point here is not to necessarily recommend this as a superior approach to implementing this in getCTA_Choices(); rather to show that this is possible and to point out an advantage of this approach. The main advantage is that any future update of the getCTA_Choices() method in the base MetaData_FF_W_Convective.py will be far less likely to require a manual merge to both pull in any new CTAs but still filter out arroyo related CTAs. Note again that the default instance of getCTAs() for this hazard type is actually in CommonMetaData.py.

3.2.1.5 Add a new Call To Action  contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

In this final customization example related to calls to action, we show how to add a new call to action. First we need to add a new method for the content of the call to action. These methods always go into an override of CallsToActionAndImpacts.py:

To complete this, the new call to action content method needs to be added to the getCTA_Choices() method in the metadata file for the desired hazard type(s). Here is what such an override would look like added in MetaData_FA_A.py:

3.2.2 Changing default Hydrologic Cause     contents ⇑ ↑  ^  ─  +  v  ↓  ⇓

The purpose of this customization example is to change the default hydrologic cause for non-convective flash flood warnings to be “Ice Jam” in winter and “Snow Melt” otherwise. The file being overridden here is MetaData_FF_W_NonConvective.py (changes to base in green).

3.2.3 Adapt the metadata to the VTEC life cycle stage.   contents ⇑ ↑  ^  v  ↓  ⇓

At times there might be a desire to have the content of the metadata adapt to the stage in the VTEC life cycle (e.g. NEW, CON, etc.) the product being issued will have. This already happens to some degree, mostly where a distinction is made between hazards that are ending and hazards that are not ending.  Recent changes make it straightforward to have the metadata adapt more precisely to the specific stage in the VTEC life cycle that applies to the hazard being generated. These changes mean that nearly all the ambiguity with respect to the life cycle has been eliminated.  This can now be determined by checking the content of the following metadata member variables:
       self.hazardStatus  self.isCANset  self.isEXPset

The table that follows shows the state of those variables for each VTEC life cycle stage:

Note that there is one instance of ambiguity; one cannot distinguish between CON and EXT.  The main reason this ambiguity still exists is that when transitioning from issuing a CON to issuing an EXT (by changing Add to End Time), the metadata will never be reexecuted and so there will never be an opportunity to actually change the metadata in response to this.

Another aspect of the impending product being issued that could possibly impact the design of the metadata is whether or not it is a correction. This is also now completely unambiguous; a metadata member variable exists, self.isCORClicked, which will be True if a correction is being issued. Everything in the previous table will apply to the product that is about to be issued, regardless of whether or not it is a correction.

Here is a very simple override that leverages this. The question of whether the immediate cause should be modifiable for a correction is something that has spawned endless debate, and we are not necessarily recommending this. However, for the sake of argument, suppose one did want to lock out changes to the immediate cause for a correction.  Here is an override that can implement this:

CommonMetaData.py:

There are some downsides to the ambiguity between CON and EXT.  Because an EXT is an update that does not end the hazard, the information needed will most of the time be very similar to what is needed for a CON.  However, it is true that for some hazard types, the product format used for an EXT is much closer to what is used for a NEW than for a CON. This is because an EXT brings new future times under threat, which can necesitate things like EAS activation.  So some aspects of the information needed for an EXT may be more like that needed for a NEW.

Therefore, let us suppose that a situation was encountered where it was important to have the metadata for an EXT be different than that for a CON.  There is a solution available, albeit a clunky one.  First, create the override that follows for CommonMetaData.py, which introduces a new boolean, self.isEXTset:

CommonMetaData.py:

Then there needs to be a way to get the metadata to reexecute after changing Add to End Time.  Here is where the clunkiness comes in.  The override that follows demonstrates this for MetaData_FA_Y.py; it adds a button to the top of the Details section labelled “For EXT’. The idea is that after making a choice in Add to End Time, the user hits that new button and the metadata reexecutes.  Certainly not an ideal long term solution.

MetaData_FA_Y.py:

3.2.4 Modifying the Product Staging Dialog     contents ⇑ ↑  ^  v  ↓  ⇓

The Product Staging Dialog is a dialog that sometimes comes up after choosing Preview but before the Product Editor appears. The purpose of this dialog is to allow the user to make choices that are for the product as a whole rather than for one particular hazard; this typically  occurs for hazard types where a single product can possibly contain text for multiple specific hazards.

Files that define metadata for a hazard type have the hazard phensig for the hazard type encoded into the metadata file name (e.g. MetaData_FA_W.py). Files that define metadata for a Product Staging Dialog have the NNN of the product(s) to generate encoded into the metadata file name (e.g. MetaData_FFA_FLW_FLS.py).

Here we have a customization example where all the synopsis options that refer to snow melt are commented out; one might do this if one was at a WFO in a place like Southern Florida. We also make it so the category increase synopsis results in a first guess for the language describing the rivers involved. This is done by overriding the methods execute(), synopsisCategoryIncrease(), and getSynopsisChoices() in MetaData_FFA_FLW_FLS.py (changes from base in green).

3.2.5 Change default search parameters for impacts section of flood warning products     contents ⇑ ↑  ^  v  ↓  ⇓
         
Here we describe how to change the default search parameters for the impacts sections of point flood warning products. When the CrestsAndImpactsUtil.py module was introduced, several small methods were added that specifically carry default values for several of these selectors. For the sake of argument, suppose we want to change the default value of 'Maximum Depth Below Flood Stage' from -3 to -5.  The following override to CrestsAndImpactsUtil.py will accomplish this:

class CrestsAndImpactsUtil(object):     def getDepthBelowFloodStageDefaultValue(self, parm):         #return -3         return -5

This override assumes that the exact same defaults are desired for FL.W, FL.A, and FL.Y hazards. If it was desired to have different defaults among these hazards, this would require an override to either MetaData_FL_W.py, MetaData_FL_A.py, or MetaData_FL_Y.py. Here we show one way to apply this same change to only FL.W hazards.

First generate any point flood warning recommendation. Click on a recommendation in the Hazard Services console to pop the Hazard Information Dialog. Choose the Impacts Statement tab and then click on Impacts Search Parameters to expand that dialog. As mentioned, for the purpose of this exercise we are going to change the default for 'Maximum Depth Below Flood Stage'. If there was a different element one wanted to make such a hazard type specific change for, viewing the dialog would allow one to find an appropriate string to search for.So now use the following commands to learn where this dialog is being constructed (see Introduction for more information about searching code for functionality):

> cd /scratch/albatross/awips2/edex/data/utility/common_static/base
> touch /tmp/bbb
> find HazardServices -name '*py' -exec grep 'Maximum Depth Below' '{}' /tmp/bbb \;

This tells us that HazardServices/hazardMetaData/CommonMetaData.py is where this dialog is constructed. Inspecting CommonMetaData.py, one learns that the method which constructs this dialog is getStageWindow(). So to change the default search parameters one needs to create a site override of this method. This could be in a site version of CommonMetaData.py, which would impact all hazard types using this dialog, or it could be in MetaData_FL_W.py, which would impact only this particular hazard type. For this exercise, we will implement this in MetaData_FL_W.py. What follows is the user override of MetaData_FL_W.py that implements this (changes from base in green; note that very little is a copy of the base).

There are several things going on here that are unique in comparison to other customization solutions shown in this document. Instead of completely replacing the existing method for obtaining the metadata, we directly call the method in the base class CommonMetaData to get an initial version of the metadata, and then modify that resulting metadata. This has the advantage that if a new base version of CommonMetaData::getStageWindow() appears in a future release, there is a very good chance that no manual merge will be required. We include in the method a simple way for someone to optionally track the behavior of this new version of getStageWindow() in a temporary file. This is set up so that if there is a runtime error in the method updateDefaultValues(), the traceback will also end up in that temporary file. This can be helpful because it then becomes possible for someone not working in eclipse to nonetheless debug the behavior of updateDefaultValues() method. Even if one is working in eclipse, it can sometime be very difficult to identify one traceback of interest from the 'firehose' of information that can appear in the eclipse console.

3.2.6 Change precision for additional rainfall selectors.     contents ⇑ ↑  ^  v  ↓  ⇓

In the Hazard Information Dialog for convective Flash Flood Warnings, there are selectors for specifying additional rain that is expected to fall. The current default behavior of those selectors is for them to only allow rainfall to even whole inches. This customization example explains how to change the precision allowed for these selectors. First it is necessary to identify the module where the default behavior for this is invoked. We use the labeling string for those selectors and search the metadata modules using these commands:

> cd /awips2/edex/data/utility/common_static/base/HazardServices/
> touch /tmp/bbb
> find . -name '*Meta*py' -exec grep -i 'additional rain' '{}' /tmp/bbb \;

This tells us that the default megawidget specification for this menu is in CommonMetaData.py. Inspecting the base version of CommonMetaData.py tells us that the method that puts together this megawidget specification is called additionalRain(). We could override this method for CommonMetaData.py, which would change this behavior for all product types. For this example we choose to implement an override for additionalRain() in MetaData_FF_W_Convective.py, so that the change only applies to that hazard type. What follows is the override for MetaData_FF_W_Convective.py, with changes in green:

With this override, we specify 2 digit precision, which means it is possible to specify rainfall values to the nearest 0.01 inches. This degree of precision, which is unrealistic for any forecast rainfall amount, is nonetheless required to enable the incrementDelta value of a quarter inch. One could make the argument that it makes no sense to increase the precision of the additional rainfall specifier without also doing the same for the ‘rain so far’ specifier; this is left as an exercise for the user.

3.2.7 Interdependency scripts.     contents ⇑ ↑  ^  ─  + v  ↓  ⇓

Interdependency scripts are a means by which a choice made by a user in the Hazard Information Dialog can result in automatically changing the state of other elements in the Hazard Information Dialog. While interdependency scripts are the most efficient way to accomplish this, they are not the only way. One can also define the "refreshMetadata" key to be True for a megawidget element, and this will result in the execute() method for the metadata to be rerun when that element is changed. (To see an example of invoking this refresh logic, examine the getHydrologicCause() method of MetaData_FF_W_NonConvective.py.) Also note that while interdependency scripts can result in changing the state of other elements, invoking the refresh logic is the only way for one choice to result in adding, removing, or changing the label of some other element.  Also, one does not have access to the hazard event inside an interdependency script.

Note that the method applyInterdependencies() is a static method of metadata modules. This particular static module method is overridable, but only because the logic that executes the metadata python is hardwired to do so. Therefore, be advised that it is not possible to introduce any additional static module methods in any of the metadata modules through override. Also be aware that many instances of applyInterdependencies() start with something similar to the following:

The exact name of the module from CommonMetaData may be different than applyInterdependencies.  The thing to keep in mind is if you have a similar construct in an override MetaData module, then at times one must put import CommonMetaData at the top of the override file.

Under the next four headings are examples of using an interdependency script. There is more information about interdependency scripts in the megawidget document.

3.2.7.1 Workaround for problems with floating point spinners.   contents ⇑ ↑  ^  ─  + v  ↓  ⇓

One common feature of the metadata for areal flood hazards is the use of floating point spinners to enter observed or expected rainfall amounts. Ideally these spinners would also allow the user to type in a number, but that feature has never worked well.  This customization example leverages an interdependency script to pair floating point spinners with a text entry field to at least obtain the desired functionality. Here we apply this to the Rain so far: entries for the convective FFW; it is left as an exercise for the reader to apply this to other situations if desired. Note that the extra 30 spaces added to the "inches of rain have fallen" label is to avoid annoying extra whitespace after the Text widgets.

MetaData_FF_W_Convective.py:

3.2.7.2 Changing other default state based on the Source selection.
             contents ⇑ ↑  ^  ─  + v  ↓  ⇓

This example focuses on the state of a checkbox labeled Flash Flooding occurring that appears in the Hazard Information Dialog for convective Flash Flood warnings. Here the goal is to change its default value depending on which value is selected for the set of radio buttons under Source:. If the user actually changes the state of the Flash Flooding occurring checkbox from the Hazard Information Dialog, thereafter the user’s choice for the checkbox will be honored and this defaulting logic will be disabled.

The override that follows for MetaData_FF_W_Convective.py contains an interdependency script that invokes this behavior:

3.2.7.3 Sharing methods between MetaData class and interdependency script.
             contents ⇑ ↑  ^  ─  + v  ↓  ⇓

There may be times when, in order to avoid duplicating code, it would be desirable to share methods between the code in a MetaData class and the code in an interdependency script. This is a bit tricky, because many of the typical classes used for broadly shared methods (such as TextProductCommon.py) are not importable by the MetaData modules. The module GeneralUtilities.py, which is importable by MetaData modules, is a reasonable place to put such shared methods. This module is simply a collection of static methods rather than a class.

Here will be one of the rare instances where we will demonstrate a point with an override that does nothing that is actually useful; this is just an example of how to structure the code such that a MetaData class and an interdependency script can share a method.  This override example will function, but will only output the file /tmp/testSharedMethod.txt to demonstrate the method sharing; it changes no actual behavior for the user.

GeneralUtilities.py:

MetaData_FA_Y.py:

3.2.7.4 Source, Event Type, and Immediate Cause interdependencies.
         contents ⇑ ↑  ^  ─  + v  ↓  ⇓

Here we are just going to refer to another document where this has been discussed in gory detail. This is because our plan is to formalize this functionality in a later release, and so this section will at a minimum change a great deal, and may possibly be temporary.

3.2.8 Disclaimers that clarify what is editable.     contents ⇑ ↑  ^  ─  + v  ↓  ⇓

A Hazard Services ticket (#45995) has been written to address some confusing aspects of the appearance of the Impact Statement tab in the Hazard Information Dialog for River hazards. The problem is that the individual impact statements appear to be editable in the Hazard Information Dialog but are not. It may be some time before this ticket is completed, but in the mean time we present this customization that shows disclaimers that make it clear what is editable and what is not. The file being overridden is CommonMetaData.py. This customization is large because it overrides some methods with alot of code, but the changes themselves (in the customary green) are very simple.

3.2.9 Showing Type Source on the Hazard Information Dialog.     contents ⇑ ↑  ^  v  ↓  ⇓

This only applies to river flood hazards. Issuing a river flood hazard depends on having observed and forecast stage or flow data for a gauge. Most gauges have multiple potential sources of both observed and forecast stage and/or flow data. These are referred to as a “Type Source”, and by default these are always shown in the Graphical Time Editor.  This override allows this information to also be shown directly on the Hazard Information Dialog. This involves very simple overrides to four modules; CommonMetaData.py, MetaData_FL_A.py,   MetaData_FL_W.py, and  MetaData_FL_Y.py.  Here we only show the overrides to CommonMetaData.py and MetaData_FL_A.py, with changes from base in the customary green. The overrides for MetaData_FL_W.py and  MetaData_FL_Y.py are so completely analogous to the override for MetaData_FL_A.py that we leave those as an exercise for the reader.

CommonMetaData.py:

MetaData_FL_A.py:

3.2.10 Including gauge data in areal flood hazards.     contents ⇑ ↑  ^  v   ↓  ⇓

Hazard Services has functionality that allows one to include river gauge data in areal flood hazards.  Typically, river gauge data is only shown in river hazard products, so this functionality is deactivated by default.  However, it is very simple to activate through the following override of CommonMetaData.py:

This results in radio buttons labeled River Information and With Impacts being added in the Locations Affected section of the Hazard Information Dialog for areal flood hazards.  Activating the River Information radio button populates the list of Gauges to Include with any for which either the gauge itself or its reach polygon overlaps the area of the hazard. Activating the With Impacts radio button creates a hierarchical choice list below the Locations Affected section; this hierarchical choice list allows one to select gauges and impacts to mention for the gauge.

There are two other methods one can override that are binary switches controlling other aspects of this capability. These are hideMissingDataGauges() in CommonMetaData.py and mentionMissingDataGauges() in SectionLevelMethods.py.  It is left to the reader to read the comments on the Base level instances of these methods to learn what they do.

Also note that this capability does not currently work correctly for discharge gauges. Ticket #73784 is being worked to address this, but it will be some time before this fix appears formally in a Hazard Services release. Current thinking is that this will not be something of broad interest, so we will not include a patch here.  Put a question on the hazard services list server if this patch is needed.