Chapter 1: Customization of Hazard Services    contents  ⇑ ↑   ↓ ⇓

Hazard Services is delivered with a set of baseline hazard generation functionality. However, it is clear that “one size” does not fit all national sites spanning diverse meteorological regimes and topographies. Therefore, Hazard Services is highly configurable and this section will help Focal Points to customize the system to address their local requirements.

We describe the various recommenders, product generation files, and configuration files showing how they are used by the system. We include use cases and instructions for customization.

1.0 Localization -- How to set up Hazard Services for your Site   contents ⇑ ↑  ^  v  ↓ ⇓

Hazard Services needs to be localized to your site. This document covers Hazard Services-specific steps needed and assumes that more general localization steps such as porting over a Hydro Database, setting up FFMP, and localizing EDEX have been taken care of.

The Localization Perspective in CAVE provides access to the user-configurable files for various applications. There is a directory for Hazard Services which contains configuration files, recommenders, and product generators. Each file has a BASE version which can be viewed in the Localization editor by clicking on it. A copy of this file can be made to produce a SITE, Desk, Workstation, or User version. If these versions exist, they are used instead of the BASE version in a hierarchical manner. For example, if a SITE version exists, it is used instead of the BASE. If a User version exists, it is used instead of the SITE or BASE. Overrides can be partial in the sense that the override file contains only what is to be changed from the base file. In this way, specific customizations can be made.

Be aware that certain files under Hazard Services in the localization perspective are NOT subject to override. Even though the localization perspective will let you make override copies of them, those override copies will be completely non-functional. Currently, these are the files we are aware of that are not subject to override, but there may be others:

ProductInterface.py

The base version of many files now have a comment at the top that says whether the file is overridable. As far as can be determined, when present, that comment is always accurate about whether the file in question is overridable.

Also, be aware that even when a file is overridable, it might not be overridable at every level that the localization perspective allows you to copy it to.  For example, the file backupSites.xml is only overridable at the SITE level, even though the localization perspective allows you to copy it to other levels.  This is different from interoperability.xml, where the localization perspective only allows one to copy it to the SITE level.

Finally, be aware that most files in the localization perspective have a Copy To -> Region option, but that option never does anything useful; more on this in the next section.

Overrides

In Hazard Services, there are various kinds of Overrides to files.

• Incremental Override: Some files consist of simple Python dictionaries or lists and are overridden by a process we call “incremental” override. Examples of these files are the Settings files (e.g. Hydrology_All.py) and HazardTypes.py. Although you will see examples of overriding files, a complete set of documentation for “incremental override” of Hazard Services configuration files can be found in the Incremental Override for Configuration Data document.
• Class-based Override: Other files contain Python classes and their overrides are “class-based”. A “skeleton” class is defined in the override file and the user adds selected Python methods from the class to “override” or add to the original methods. Overriding of these files is similar to the GFE and is covered in that training.
• XML Override: Some configuration files (e.g. backup sites) are “xml” format and are overridden as other “xml” files in AWIPS 2.
• Python Method Override: Some files consist only of Python methods e.g. Megawidget Side Effects. To override these files, you must copy the entire method into your override file.

There may be times when the more conventional non-incremental override behavior is desired even though the file is subject to incremental override. This is very simple to achieve with a single extra entry placed at the very beginning of the override file content; the form of this entry depends on whether the file is a dictionary or list at the top level. For a dictionary this extra entry is "_override_replace_" : True, ; for a list it is "_override_replace_", . Then the content that follows can be a complete copy of the base, modified in whatever way is desired.

The Localization Perspective under the Hazard Services tab (or in directories under ...utility/common_static/base/HazardServices/) contains the baseline and localization files for Hazard Services. You will find the following tabs in alphabetical order. They are listed here in logical groupings:

• Configuration of Graphical User Interface

• Startup Config (incremental override)
• Alerts (incremental override)
• Settings (incremental override)

• Except for interoperability.xml and backupSites.xml

• Hazard Types and metadata

• Hazard Types (incremental override)
• TODO: Hazard Type Color Table ??
• Hazard Categories (incremental override)
• Hazard metadata (class-based override)

• Except for HazardMetaData.py, which is incremental.

• Megawidget Side Effects (python method override)

• Recommenders and Product Generation

• Recommenders (class-based override)
• Product Generator Table (incremental override)
• Product Generators (class-based override)
• Product Formats (class-based override)
• Utilities (class-based override)

• Except for BurnScarMetaData.py, DamMetaData.py, and ProductEditorFields.py, which are incremental.

Megawidgets

Hazard Services supports a set of user-defined graphical user interface components, called Megawidgets which are used throughout Hazard Services customization. Examples will be seen throughout this document and complete documentation can be found in the Megawidget section.

1.0.0 Exceptional override behavior worth noting.  contents ⇑ ↑  ^  ─  + v  ↓ ⇓

In this section we describe certain override behaviors that may be non-intuitive for the user.

1.0.0.1 Region level override.  contents ⇑ ↑  ^  ─  + v  ↓ ⇓

By far the overwhelming majority of overrides will be performed at the Site or User level. However, it is also possible to override files at the Region level. There are some things about how Region level overrides work that are non-standard compared to other overrides. For that reason, and because Region level overrides are broader in scope than other overrides, it is recommended that only Focal Points and/or ITOs implement Region level overrides. That being said, the broad scope is exactly why Region level overrides are useful in some instances. Region level files occur after Base but before any other level in the localization hierarchy. A single Region level override file can impose a behavior change for all users, all workstations, and all sites the WFO has any responsibility for. Of course, any Region level override can be further overridden though Site, User, or any other level of override.

Note: it seems very likely that region level overrides will be leveraged in the future for implementing functionality “turnkeys”.  A turnkey refers to a functionality change (usually that implements a policy change) that is meant to be invoked systemwide at a coordinated time, rather than its timing being driven by when a particular software release is installed at each individual site. This means that all things being equal, there should be a very good reason to implement a change by way of a region level override, so as to not interfere with any future turnkeys.  This does not mean never use a region level override, it just means to consider carefully what your other options might be.

There are six different two character region designators; AR, CR, ER, PR, SR, and WR. The designator in effect for your Cave instance determines the paths to Hazard Services region level overrides. The designator in use depends on which region the localization being used for your Cave instance is in, not the region of the site you might be doing service backup for.  The vast majority of WFOs never do service backup across region boundaries, and so this is a distinction without a difference.  However there are some exceptions. Suppose one is at WFO GJT (in Central Region, CR), and you are performing service backup for WFO SLC (in Western Region, WR).  If you run your Cave on behalf of your primary site of GJT, and use the Hazard Services service backup capability to backup site SLC, the region designator in effect will be CR.  However, suppose you had decided to back up SLC by starting a Cave instance on behalf of site SLC (see section 4.5 for more information).  In that case, the region designator in effect will be WR. Our goal here is not to either defend or criticize this behavior, but to explain unambiguously what the code is doing right now.

If one looks in the localization perspective, most files have a Copy To -> Region option, but this option never does anything useful. To override something at the Region level, the file must be created directly in the file system in the appropriate place under /awips2/edex/data/utility/. Usually just placing an override file in the file system under utility/ is not enough to get CAVE (or EDEX) to recognize the presence of a new or changed override file. One usually needs to either create or modify the file in the localization perspective, send it into EDEX through the thrift client, or restart EDEX to get any changes recognized. Region level files behave differently, at least with respect to CAVE. For Region level files, placing the file in the file system and restarting CAVE is always enough to get CAVE (and therefore the Hazard Services application) to recognize the change.

Because it is not possible to update a Region level file in the cannonical way, for now it is recommended that Region level files be avoided in the case where the desired impact is on how EDEX behaves. Furthermore, given that Region level overrides are essentially global in scope, they should be as incremental as possible. That is, they should contain only the methods or data items being directly impacted by the override, and should NEVER be modified copies of entire Base level files.

Here is a simple hypothetical example of a Region level override. Suppose at site OAX there was a desire to have all hand drawn convective Flash Flood warnings (whether for their own site or for any backup site) have an initial duration of 2 hours, instead of the 3 hours in the default configuration. Here is how that could be implemented as a Region level override. One needs to first note that the region designation for site OAX is CR, because they are in Central Region. To implement this, the file HazardTypes.py would be created with the content indicated in the following directory:

/awips2/edex/data/utility/common_static/region/CR/HazardServices/hazardTypes/

1.0.0.2 Configured level override.  contents ⇑ ↑  ^  ─  + v  ↓ ⇓

Configured level files manifest after region level files and before site level files in the localization hierarchy.

Configured level overrides are notable in that they cannot be manipulated using the localization perspective; only viewed.  Therefore, any updates to configured level overrides need to be done directly through the file system.  One should also note that certain GFE configured level files for backup sites can end up being removed when a backup site GFE is shut down (this behavior may get fixed in a future release).  This is important to Hazard Services because there are certain GFE configured level files used by Hazard Services at run time (see section 1.2.6 for more on this).

A Cave restart is necessary to get the system to recognize an update to a configured level file.

1.0.0.3 Python class overrides for Service Backup.  contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

This section refers specifically to the behavior of overrides in the context of using the Change Site popup present under the little downward triangle on the right side of the console. (See section B for how that is set up). This section also refers specifically to python class files (e.g., MetaData_FL_Y.py) as opposed to files subject to incremental override, such as HazardTypes.py.

There have been many instances of this behavior being inconsistent.  One workaround for that is to do service backup by way of starting a whole Cave for a backup site.  Note that some additional configuration is required before using Hazard Services in this way; see section 4.5 for details. Our long term goal is certainly to make this unnecessary in all but a few very limited circumstances, but in the short term this workaround may be helpful.

Consider a site level override of a python class for your primary site.  Also assume that your system has no overrides of that same python class for any of your backup sites.  If one uses the Change Site popup to perform service backup, the same override behavior implemented for the primary site by that file will nonetheless manifest during service backup. This is not a bug; it is by a previous design.

Now consider that a site override for that same python class is later implemented for one of your backup sites.  Most often this will not be because of an action taken directly by anyone on your own A-II system; rather this would most often be because AWIPS II Backup Service delivered a change made by someone at one of your backup sites.  Also assume the method(s) implemented in the backup site override of that class are different from the method(s) implemented by the primary site override.  This will result in method(s) from your primary site override reverting to the base (or possibly region/configured) level behavior, while the method(s) in the backup site override will start directly controlling new behavior. Note that the same relationship exists between a pair of configured level overrides for both the primary and backup site.

This is a challenging concept to get your head around, so we will restate this for clarification. In this table, LLL refers to a python class file implemented as site level override for the primary site, and BBB refers to a python class file implemented as a site level override for the backup site.  The little c suffix means a configured level file. The columns with x’s just indicates whether a given possible python class override is present.  Again, this refers only to python class files (not incremental override files) and only for the case where one is performing service backup by way of the Change Site popup.

Some final thoughts on this; again, this only applies to python class files.  When doing service backup by way of the Change Site popup, adding a new override file for the backup site can cause a file that used to be part of the active localization hierarchy to no longer be part of that hierarchy.  Another way of stating this is that adding a new override file for the backup site can impact the behavior of methods not implemented in that override file. When running Hazard Services for the primary site (or doing service backup by restarting all of Cave for the backup site) this can never happen.

A user reading this might come away with the notion that one should always just restart all of Cave for the backup site when performing service backup.  This is not what is being recommended.  First, using a Cave restart is slower and less efficient and could end up being a problem in a time critical warning situation.  Second, the behavior of Cave restart service backup can be different from Change Site popup service backup, and those differences may not be what you want.

1.0.0.4 Python incremental overrides for Service Backup.  contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

This section refers specifically to the behavior of overrides in the context of using the Change Site popup present under the little downward triangle on the right side of the console. (See section B for how that is set up). This section also refers specifically to python files subject to incremental override (e.g., HazardCategories.py) as opposed to python class files, such as SectionLevelMethods.py.

For incremental override files, the localization hierarchy is much more straightforward than for class based files.  The configured and site level files included in the localization hierarchy depend only on which site is selected on behalf of the Change Site popup.  Put another way, the configured and site level files included can never be a mix of the Cave site and the Change Site site as for class python files; they can only be for the Change Site site.

1.0.1 Recommended Steps    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

• StartUpConfig.py: Override StartUpConfig.py recommended fields as designated in the base level file.
• Shape file based polygons for river reaches: River hazards (FL.W, FL.A, FL.Y, HY.S) are issued for a river gauge. However, those hazards are not just valid for that single point, but for some distance along the river upstream and downstream from the gauge, which is referred to as a river reach. By default Hazard Services will use the same low resolution polygons for the river reaches that RiverPro does. In Hazard Services it is possible to supply higher resolution river reach polygons for some or all of the gauges. This is done by running the commands that follow on dx3 as user awips:
   > cd /awips2/edex/scripts/HazardServices/
   > ./ingestshapefiles.sh -r -s shapefilecontainer -w BBX -c altIDattribute
  For the -w option, BBX is not literal but is the CWA of a backup site, and this is not needed for the default site. The shape file container is a directory, tar file, tgz file, or zip file that contains one or more shapefiles. In this context, each shape supplied should represent an updated river reach polygons for each gauge you want to update, identified by the gauge id (e.g. DCTN1). The gauge id can be specified by an attribute called literally name, an attribute whose name is supplied with the optional -c option, or by the name of the shape file itself if the shape file has exactly one reach polygon.
  Be advised that the form of the ingestshapefiles.sh command just shown can add new river reach polygons or update an existing one, but will never remove one.  If there is a need to remove a river reach polygon, this requires using the -e or -n option. Run this script with no arguments to see usage information for these options.
• Site-specific geometries: Ingest shapefiles for Dam Inundation and Burnscar areas. See section D for a minimal implementation, see detailed instructions in the next section.

1.0.2 Configuring Hazard Services for predefined dam break and burn scar impact areas.    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

Note that currently (Sept 2019) a supporting file (cleanUpShapeFiles.py) is apparently not being distributed with releases.  For now, one can download a copy of that file here.

This section exists for three purposes. First, for some sites the automated conversion of warnGen dam break and burn scar configuration using parseWarngenTemplate.py and ingestshapefiles.sh (see section D) may be insufficient to allow Hazard Services to satisfactorily issue warnings for these hazards. Second, there may be sites for which warnGen had never been configured to interoperate with predefined impact areas for dam breaks and/or burn scars; in that case the site will need to directly configure Hazard Services for this purpose. Third, even if the automated conversion initially ends up being sufficient, eventually (especially for burn scars) there will be a need to update the configuration concerning predefined impact areas for dam breaks and burn scars.

It should also be noted that predefined impact areas for dam breaks have been generalized since 18.1.1. Predefined impact areas can be defined that apply to nearly any flavor of non-convective Flash Flood warning. The automated warngen conversion logic will handle only dam breaks; a predefined impact area for an ice jam, for example, must be configured separately from anything handled by the conversion scripts.

There will be some offices that never use the warnGen conversion for configuring Hazard Services for predefined dam break and burn scar impact areas.  For those offices the script parseWarngenTemplate.py is not important.  What is important in the sections that follow are the parts of 1.0.2.1 that discuss ingestshapefiles.sh, plus sections 1.0.2.2, 1.0.2.3, and 1.0.2.6; sections 1.0.2.4 and 1.0.2.5 will be unimportant for offices that never use the warnGen conversion.

1.0.2.1 What parseWarngenTemplate.py and ingestshapefiles.sh do.

 contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

We begin this discussion by describing what these scripts do in more detail. Both scripts output usage information if run with no arguments. Both parseWarngenTemplate.py and ingestshapefiles.sh can potentially be run on behalf of your primary site and each backup site for both dam breaks and burn scars. Both are meant to be run on dx3 as user awips, and can be found in /awips2/edex/scripts/HazardServices/.

By default, parseWarngenTemplate.py reads the files damInfoBullet.xml and damInfo.vm to convert warnGen dam break configuration, and reads the files burnScarInfoBullet.xml and burnScarInfo.vm for burn scars. By default it is assumed these files reside in the directory common_static/site/LLL/warngen/. However, one can provide an alternate full path to either (or both) the vm component or the xml component. Here are the files that parseWarngenTemplate.py attempts to write out:

common_static/configured/LLL/shapefiles/hazardServices/damInfo.{dbf,shp,shx}

common_static/configured/LLL/HazardServices/python/textUtilities/DamMetaData.py
common_static/configured/LLL/shapefiles/hazardServices/burnScar.{dbf,shp,shx}

common_static/configured/LLL/HazardServices/python/textUtilities/BurnScarMetaData.py

Most commonly, parseWarngenTemplate.py will be run only during the initial setup of Hazard Services and then never again. One possible exception would be if the previously mentioned warnGen configuration files were updated and there was a desire to migrate that updated information to Hazard Services; in theory this would be necessary only during a limited transition period and so should be rare.

By default, ingestshapefiles.sh reads the aforementioned shapefiles output by parseWarngenTemplate.py, and writes out the impact areas into the hazardservicesarea table in postgres. This table is within the mapdata schema of the maps database. ingestshapefiles.sh also writes out a merged shapefile into the directory edex_static/site/LLL/shapefiles/hazardServices/. When ingestshapefiles.sh is run to install impact areas for dam breaks and burn scars (-d and -b options), it will inform the user as to which impact areas end up with no associated metadata. For those dam break/burn scar impact areas which have no metadata, idealized metadata will be output in /tmp; the script will inform the user as to the exact path to the idealized metadata.  Note that for options not associated with dam breaks or burn scars, idealized metadata is never output. Running ingestshapefiles.sh with a -i option summarizes impact areas already in postgres for the specified site. The expectation is that ingestshapefiles.sh will be rerun any time one needs to add, remove, or modify the boundary of any of these impact areas.

 ingestshapefiles.sh is not limited to processing only warnGen converted shape files from parseWarngenTemplate.py. The user can supply a shape file container with a -s option. A shape file container is a directory, .tar file, .tgz file, or .zip file containing one or more shape files.  Any single character argument to -s will result in an attempt to read back the merged shape file from edex_static/site/LLL/shapefiles/hazardServices/.  If user supplied shape files are specified, by default ingestshapefiles.sh will still try to also read in warnGen converted shape files at the same time; supplying a -u argument suppresses this.

Each impact area shape is primarily identified by its name attribute. If the name attribute is not present, each shape can be identified by an attribute specified with a -c option. If neither of these apply, the shape file name can be used if there is only one shape in a given shape file. A mix of these identification methods can be used in a single invocation of ingestshapefiles.sh as long as it is unambiguous which method is being used for each impact area shape. Note that shape files written by parseWarngenTemplate.py and the merged shapefile written out by ingestshapefiles.sh itself will always have a name attribute.

By default, running ingestshapefiles.sh will not delete existing impact areas stored in postgres, it will only add new impact areas. An exception to this is if multiple impact areas are encountered with the same name attribute (or proxy). When redundant naming is encountered, a shape from the current invocation of ingestshapefiles.sh will always displace a shape already existing in postgres. After this is considered, a shape from a user supplied shape file will get precedence over one converted from warnGen. Finally, naming redundancies among the shapes in a user supplied set of shape files will be eliminated, but there is no reliable way to control which shape will get precedence. Therefore, this is not recommended and ingestshapefiles.sh will warn the user when this occurs.

If there is a need to delete an impact area stored in postgres, ingestshapefiles.sh must be run with the -e or -n option.  Run ingestshapefiles.sh script with no arguments to see usage information for these options.

1.0.2.2 How Hazard Services uses configuration information for impact areas.

 contents ⇑ ↑  ^  ─  + v  ↓ ⇓

At runtime, Hazard Services reads DamMetaData.py, BurnScarMetaData.py, and the hazardservicesarea postgres table. The postgres table is meant to carry only the geographic boundary of each impact area; the rest of the metadata associated with the impact areas are in DamMetaData.py and BurnScarMetaData.py.

The directory common_static/base/HazardServices/python/textUtilities/ is where the base versions of DamMetaData.py and BurnScarMetaData.py reside; one can use the comments in the base versions as a quick reference for understanding the contents of these files. Functionally, the base versions of these files are mostly just stubs, except for the default non-dam specific (generic) entries in DamMetaData.py. The generic dam metadata has a key of simply “Dam” in DamMetaData.py. For each impact area, if a key does not exist in the dam specific entry but does exist in the generic entry, the generic element for that key gets applied to the dam at runtime with substitutions made for variables like ${riverName}. There is nothing in BurnScarMetaData.py analogous to the generic entry in DamMetaData.py. When run for dams, parseWarngenTemplate.py will generate its own version of generic dam metadata based on the warnGen configuration for the site. Mostly the metadata text values are meant to be directly inserted into generated text products. An exception is the “dropDownLabel”; if present, it provides the description of the dam or burn scar that goes into the choice list for the recommender. Otherwise that description defaults to the value of the primary key of the entry. The primary key of the entry is also what matches the ‘name’  attribute associated with the impact area in postgres.  The primary key of the entry usually is exactly the same as the “damName”/”burnScarName” element, which is the text used to identify the impact area as products are formatted. However, the “damName” or ”burnScarName” element can be overridden to be different than the primary key.

There are currently three other elements for a DamMetaData entry that function beyond just providing a simple text replacement. The most important is "hydrologicCause", which results in a different default initial selection for that selector on the Hazard Information Dialog. The "hydrologicCause" is important because for non-convective FFWs, it functions as a sub-subtype. Current meaningful values for this are 'levee', 'floodgate', 'icejam', 'ijbreak', 'glacier', 'volcano', and 'volcanoLahar'. If not present things default to being for a dam break.

There are two other elements that provide default initial selections in the Hazard Information Dialog. These are "ibwType" and "source". The meaningful values for "ibwType" are 'general', 'considerable', and 'catastrophic'. The meaningful values for "source" are 'countySource', 'localLawEnforcementSource', 'corpsOfEngineersSource', 'bureauOfReclamationSource', 'damOperatorSource', 'publicSource', 'gaugesSource', and 'civilAirPatrolSource'.

What follows is a portion of a sample dam break warning product generated by Hazard Services, with text values that flow directly from the metadata indicated with the key of the metadata element. An asterisk indicates something not directly from the metadata, but rather looked up on behalf of that element in the metadata.

Now the same for a burn scar warning.

Be advised that the structure of the metadata is largely based on warnGen examples examined to date. Our understanding of what needs to happen to every metadata element from the warnGen configuration is not yet complete (especially for burn scars). Hopefully we can close the loop on this as the formatting for these products gets exposed to more testing by field forecasters and we get more feedback.

1.0.2.3 Situation where warnGen was never configured for predefined impact areas. contents ⇑ ↑  ^  ─  + v  ↓ ⇓

Now we discuss what to do if your site had never configured warnGen to interoperate with predefined impact areas for dam breaks and/or burn scars. Of course, it is possible to approach this by first configuring your warnGen for this functionality and then running the conversion as in section D. Here we will not go into any detail about how to configure warnGen for this functionality, this page has some information about this.

This all starts with creating a shapefile (link?) or shapefiles that contain a polygon (impact area shape) for each entity (predefined dam and/or burn scar). However this is done, the shape file container (directory, tar file, tgz file, or zip file) for dam breaks needs to be completely separate from the container for burn scars. The path to the shapefile container is specified with the -s option for ingestshapefiles.sh. There are three different ways to approach connecting each polygon with each specific entity. The most straightforward is to put the primary identifying name of the entity in a text attribute literally called name. If some other text attribute already contains the primary id of each entity, one can use the -c option on the ingestshapefiles.sh command to specify this attribute. Finally, for shapefiles containing exactly one impact area, the name of the shapefile itself can be used as the primary id, as long as the shapefile does not contain the attribute name or an alternate attribute as identified with the -c option. The container may contain a mix of shape files with all three entity identification strategies, as long as it is unambiguous for each entity how this identification is occurring.

Here are the commands needed to perform this processing for dam breaks and burn scars respectively. The colored text is optional, needed only for backup sites or if an attribute other than name is providing the primary entity id.

ingestshapefiles.sh -d -s pathtodamshpcontainer -w BBX -c altIDattribute
ingestshapefiles.sh -b -s pathtoscarshpcontainer -w BBX -c altIDattribute

In the case where site supplied shapefiles are used to configure your impact areas, you will want to keep these in a safe place. The following directories would be reasonable places to keep these shape files (LLL is not literal, but rather is a site id):

  /localapps/HazardServices/LLL/shapefiles/damBreak/

  /localapps/HazardServices/LLL/shapefiles/burnScar/
You will want to have separate directories for your primary site and each backup site. In theory, one can always grab the merged shape files from the backup site in question that live in edex_static/site/BBX/shapefiles/hazardServices/.  However, having your own safe copy to work with is not a bad idea, particularly if you performed the conversion locally based on the warnGen files from the backup site.  At some point in the future (especially for burn scars) it will become necessary to modify these shapefiles and rerun ingestshapefiles.sh.

Properly running ingestshapefiles.sh to output the impact areas in postgres will provide a working albeit bare bones functionality for either dam breaks or burn scars. The products generated as a result of this will end up with many of the |* *| must edit markers, and as such will require a fair amount of hand editing to issue. The way to deal with this is to provide site overrides for DamMetaData.py and/or BurnScarMetaData.py; these would reside in common_static/site/LLL/HazardServices/python/textUtilities/ (under Hazard Services -> Utilities in the localization perspective) . The previous section should provide guidance on how to construct the metadata.  For backup sites, it is possible that this metadata would have been obtained by staging the set of files mentioned in section B.  If this is not the case, then creating this metadata needs to be done independently for each backup site.

Note that each time ingestshapefiles.sh is run for dams or burn scars, it will inform the user as to which impact areas have no metadata entries.  For those dam break/burn scar impact areas which have no metadata, idealized metadata will be output in /tmp; the script will inform the user as to the exact path to the idealized metadata.  Normally, the idealized metadata will be installed as a site override, or merged into an existing site override as appropriate.  The idealized metadata entries will still need some massaging before they are optimum for creating the proper product text.

1.0.2.4 Addressing problems with initial conversion of warnGen configuration. 

contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

There are two main types of problems to address. First is if the conversion fails; the most common way this can fail is if some of the assumptions listed in section D are not true. For example, if the vm and xml files are not in common_static/site/LLL/warngen/ or have non-standard file names. Suppose the dam break vm and xml for your primary site are together in common_static/configured/LLL/warngen/. Then the proper command would be:

> /awips2/python/bin/python parseWarngenTemplate.py -d \

   /awips2/edex/data/utility/common_static/configured/LLL/warngen/damInfoBullet.xml

In this example, the full path for the vm file is not supplied. However, parseWarngenTemplate.py is smart enough to try the default vm file name in the same directory as the input xml file.

Another thing to look for is if damInfoBullet.xml or burnScarInfoBullet.xml are not well formed xml. In theory if this functionality is working in warnGen they would have to be, and parseWarngenTemplate.py will complain about this if they are not. If parseWarngenTemplate.py does not report writing out shape files, this could be because pyshp 1.2 is not available, or because there is an issue with how the impact area polygons are encoded in the warngen xml file. Here we have examples of the files input to parseWarngenTemplate.py. The hope is that by presenting these examples to compare against, it might be possible to modify your input files such that the parseWarngenTemplate.py script can successfully process them.

In the case where the warnGen vm and xml is well formed but parseWarngenTemplate.py cannot process it properly, there are two possible explanations. One possibility is that running the conversion exposes problems in the vm or xml that also negatively impacts warnGen. In that case the expectation is that the fixes would be made directly in the warnGen directories and then parseWarngenTemplate.py would be run again in the default manner. Another possibility is that the vm or xml work correctly in warnGen but contain unanticipated syntax that nonetheless cause parseWarngenTemplate.py to fail. In that case, first copy all the input files to a separate directory and make the modifications there. Then parseWarngenTemplate.py can be run as in the immediately preceding example, where the specific path to the vm and xml files are given. If it is necessary to make Hazard Services specific modifications to the vm or xml files in order to get parseWarngenTemplate.py to work right, you will want to keep those modified files in case it is necessary to rerun the conversion. The following directory would be a reasonable place to keep these modified warnGen configuration files:

  /localapps/HazardServices/LLL/warnGen/

Remember that parseWarngenTemplate.py outputs DamMetaData.py and BurnScarMetaData.py as configured level localization files. Therefore a restart of CAVE (and sometimes possibly EDEX) is necessary for that information to become available to Hazard Services. Alternatively, one can implement these as site overrides and then remove the configured level files; this should only be done if you are 100% sure that there will never be a need to rerun the warnGen->Hazard Services conversion ever again.

The conversion can also fail if for some reason the ingestshapefiles.sh script cannot interoperate with postgres. Here is a command that can be run to verify what ingestshapefiles.sh has written out to postgres (the -w BBX is necessary only for a service backup site):

 ingestshapefiles.sh -i -w BBX

If no postgres entries at all show up then there is probably some kind of permission problem with writing to postgres. If there are some entries, but not all the expected impact areas show up, then it is likely you will need to reexamine the contents of the files input to parseWarngenTemplate.py.

The second type of problem to address is that the conversion basically works, but the products generated by Hazard Services for dam break and/or burn scar warnings do not completely suit the needs of your WFO. The only way to know this for sure is to run Hazard Services in practice mode and test previewing warnings and watches for all the predefined impact areas. If there are problems with the parts of text directly connected to the metadata (as described in section 1.0.2.2), then this can likely be addressed with a site override to DamMetaData.py and/or BurnScarMetaData.py. These files are subject to incremental override and so the site override file need only contain the things you want to add or change. For example, suppose the river name was not right for one dam. Then the site override for DamMetaData.py could contain the following (italicized text not literal):

This example also helps explain the rationale for writing out configured level files from parseWarngenTemplate.py. During the period shortly after starting to use Hazard Services, there will be a desire to keep warnGen fully functional. If the dam break information in warnGen were updated, one could rerun the conversion (overwriting existing configured level files), and the correction to the one river name as a site level file would remain in effect.

It is also possible for the conversion to basically work, but that one or more impact areas either don’t make it into the shape file written out by parseWarngenTemplate.py, or do not have the boundaries you want. If ingestshapefiles.sh is run such that a shape file container is provided with the -s argument, the script will also still try to read the shapefile output by parseWarngenTemplate.py. If there are additional new impact areas in the container, they will be added to the ones obtained from warnGen. In this case, a full set of metadata for the new impact area(s) should be placed in a site override of DamMetaData.py or BurnScarMetaData.py (as in the last paragraph of section 1.0.2.3). If there are impact areas in the container with the same primary id (whether by name attribute, alternate attribute, or shape file name) as an existing impact area from warnGen, then the boundary in the user supplied shape file container will replace the boundary from warnGen. This can be used to either correct or add detail to an existing impact area boundary converted from warnGen.

Each time parseWarngenTemplate.py and ingestshapefiles.sh is run in a manner that successfully produces some sort of output, the exact form of the command will be appended to a file named wgn2hazSer_commands.txt. wgn2hazSer_commands.txt is written into the home directory of the account that is used to run those scripts, which will be ~awips/ if the scripts are run as designed. To the extent that it is necessary to run the commands parseWarngenTemplate.py and/or ingestshapefiles.sh with different arguments than recommended in section D, the exact form of these commands used is therefore recorded, just in case there is a need to run these commands again.

Finally, it is possible that situations will arise where the default formatting logic for either dam breaks or burn scars will not be capable of formatting products that suit the needs of a particular WFO, regardless of the state of the metadata or the geography of the impact areas. If and when that happens, it will be possible that this could be addressed with customizations to the formatters. There are many other examples in this document of customizing the product formatting to meet other needs. For now, we will defer trying to present a specific example of how one might do this in the context of dam break and/or burn scar warnings until such an instance actually arises.

1.0.2.5 Maintaining impact area configuration.  contents ⇑ ↑  ^  ─  + v  ↓ ⇓

Here we describe our current thoughts about how one might go about maintaining your impact area configurations beyond the initial installation of Hazard Services. Because aspects of Hazard Services may evolve in unanticipated ways, for now this is speculation, albeit informed speculation.

As previously mentioned, there will be a transition period immediately after a site starts to use Hazard Services operationally. During this transition period, the legacy hazard applications (including warnGen) will need to remain fully functional. How one maintains the impact area configuration during the transition will depend on whether a site’s warnGen had been properly configured to deal with predefined impact areas before the transition, and whether the warnGen -> Hazard Services conversion for the impact area configuration was used successfully.

Assuming the conversion was used successfully, one straightforward means of managing the impact area configuration during the transition would be to first make updates to this for warnGen, and then rerun the conversion. Any site overrides for DamMetaData.py or BurnScarMetaData.py can just stay in place, but to the extent you had initially run the parseWarngenTemplate.py and/or ingestshapefiles.sh commands with different arguments from the defaults shown in section D, you would want to reuse that modified set of arguments (recorded in ~awips/wgn2hazSer_commands.txt).

If the conversion was not used and your impact area configurations were handled as in section 1.0.2.3, then whether or not one is still in the transition period will not have much to do with how one manages the impact area configuration. Going forward, this will be managed by updating site overrides for DamMetaData.py or BurnScarMetaData.py, modifying the shapefiles stored under /localapps/HazardServices/LLL/shapefiles/{damBreak,burnScar}/ (or wherever you decided to keep them), and rerunning ingestshapefiles.sh.

Even if a site had originally successfully used the conversion to initialize their Hazard Services impact area configurations, eventually there will be a need to migrate to a paradigm where the impact area configurations are managed completely independent of any warnGen configuration. This migration needs to operate on both shapefiles and metadata.

These commands illustrate the simplest way to migrate how shapefiles are handled:

 > cd /awips2/edex/data/utility/common_static/configured/LLL/shapefiles/hazardServices/

 > mv * /my/safe/temporary/directory

 > cd /localapps/HazardServices/LLL/shapefiles/damBreak

 > mv * /my/safe/temporary/directory

 > cd /localapps/HazardServices/LLL/shapefiles/burnScar

 > mv * /my/safe/temporary/directory

 > cd /awips2/edex/data/utility/edex_static/site/LLL/shapefiles/hazardServices/

 > cp *dam* /localapps/HazardServices/LLL/shapefiles/damBreak

 > cp *burn* /localapps/HazardServices/LLL/shapefiles/burnScar

What this does is move out of the way any shape files that were specific to managing the impact area configurations in the context of warnGen, and replaces them with the merged shape file that was last output by ingestshapefiles.sh.

How one migrates the metadata (DamMetaData.py and BurnScarMetaData.py) depends on whether there is only a configured level file, only a site level file, or both. For either DamMetaData.py or BurnScarMetaData.py, if there is only a site level file there is nothing to do. If there is only a configured level file, then implement the configured level file as a site level file and then remove the configured level file; the advantage of this is that in the future it will never be necessary to restart CAVE or EDEX to deal with a configured level file update. Where both a configured and site level file exist, one needs to implement a merge of the site level file with the configured level file as the site level file and then remove the configured level file. The following commands run on dx3 will provide the necessary merged content:

> cd /awips2/edex/scripts/HazardServices/

> ./mergedHSconfig.py DamMetaData.py

> ./mergedHSconfig.py BurnScarMetaData.py

Once this migration is complete, the impact area configurations can be managed as if they had never originated from warnGen. One completely plausible scenario for some sites is that the dam break impact areas are initially handled through the conversion and need to undergo this migration, but burn scars do not because there were no active fire years immediately prior to Hazard Services going operational, and therefore no burn scar impact areas to convert.

1.0.2.6 Predefined impact areas for other flash flood hazards.  contents ⇑ ↑  ^  ─  + v  ↓ ⇓

If one looks at the Hazard Information Dialog for non-convective Flash Flood warnings, one sees a choice list menu labeled Hydrologic Cause:. In the context of non-convective Flash Flood warnings, the hydrologic cause functions like a sub-subtype. What follows is an example override of DamMetaData.py that could be used to setup a predefined impact area for an ice jam hazard:

There are two different ways this metadata entry can be utilized. One approach is to use ingestshapefiles.sh to ingest a shape with a name attribute of Cold Bend Ice Jam. Then one can run the Dam/Levee Break Flood Recommender and choose this feature.

Another approach is to draw a feature on the spatial display that approximates the area impacted by the flooding from this ice jam and choose a Type value of FF.W.NonConvective in the Hazard Information Dialog. Next scroll down to the Dam or Levee: text field in the Hazard Information Dialog; clear that field out and enter Cold Bend Ice Jam into that text field. This will result in a Hydrologic Cause value of Ice jam break being chosen, and all other meaningful information from that metadata entry being transferred into the Hazard Information Dialog.

1.0.3 Optional Steps     contents ⇑ ↑  ^   ─ + v  ↓ ⇓

• Extension Area Configuration: Sites bordering an ocean, the Gulf of Mexico, or the Great Lakes may want to create a site override for HazardTypesUtils.py where non-zero extension distances are set. See section A for more detail.
• Settings: Create Site-Specific Settings. It may be useful for your site to have site-specific Settings to aid in the Forecast Process. For example, you may want a Setting that views only Warnings and Advisories, filtering out Watches. Or you may want a Setting to view only issued and ending hazards.
• General Configuration: Baseline versions of the Recommenders, Product Generators, Hazard MetaData (appearing in the Hazard Information Dialog), Hazard Types and Categories will work for your site “out-of-the-box.” However, as you work with the system, you will want to appropriately localize. This document will give you the information you need to do so.
• Python Language Tutorials: Configuration of Hazard Services involves Python configuration files. If you are not already familiar with Python and Class-based programming there are various tutorials that will give you the background needed:

• Codeacademy -- gives problems and checks them for you
• Google tutorial -- has videos which is also very helpful
• Official Python Website 
• Learn Python
• Tutorials Point

1.1 StartUp Configuration   contents ⇑ ↑  ^  v  ↓ ⇓

The StartUpConfig.py file contains various items listed below. More detailed information can be found in the documentation within that file.

• Dissemination order for products
• Recommender to use when creating hazards from gage points
• Console Settings and Time Line navigation options
• Hazard Information Dialog layout and tab options
• Specific default Alerts
• Excluding certain features of the system from being exposed to the user

1.1.1 Startup like functionality in settings files.   contents ⇑ ↑ ^  v  ↓ ⇓

There are certain things that one can configure in CommonSettings.py where the resulting behavior appears to the user more like start up configuration than like settings behavior. An example of this is configuring "visibleSites" and "possibleSites" in CommonSettings.py.
"possibleSites" contains a list of sites that one can choose to display in the console for your site. "visibleSites" contains a list of sites that are displayed in the console at start up. The choices in "possibleSites" are accessed from the top of the console under SETUP->Manage Defaults->Edit/Filter… in the Site IDs section of the Hazards Filter tab.

At one time it was absolutely necessary to define "visibleSites" and "possibleSites" in an override.  Now,  if they are both an empty list as is true in the base version of CommonSettings.py, then your primary site is assumed to be in both. However, exercise caution when customizing either “possibleSites” or “visibleSites”; if they are set to something besides an empty list that does not contain your primary site, then you can shoot yourself in the foot and create a scenario where you are unable to display any or your own site’s hazards.

The file CommonSettings.py is found under Hazard Services->Settings->config

 in the localization perspective.

1.1.2 Default setting for service backup site.  contents ⇑ ↑  ^  v ↓ ⇓

Here we discuss how one can specify a particular setting to use when first switching to a backup site. Note that to accomplish this, one generally needs to directly create a file in the file system; this is challenging to do using the localization perspective. One needs to create an instance of the file StartUpConfig.py file in the following directory:

  /awips2/edex/data/utility/common_static/site/BBB/HazardServices/startUpConfig/

where BBB is not literal, but is the id of the backup site in question. The contents of that file would be:

Again, the italicized text is not literal, but is the name of the setting file (minus .py extension) that you want to be the default setting to use when doing service backup for site BBB.  Note that this can either be a custom setting or a default setting.  

The reason this generally needs to be done directly through the file system has to do with how the localization perspective handles files for backup sites (see section 4.5 for more on this). In a nutshell, the localization perspective lets you modify existing override files for a backup site, but does not let you create new instances of files for a backup site. If for some reason you already had a StartUpConfig.py file for site BBB, you would of course need to merge the indicated content into that file, and it would be possible to accomplish this using the localization perspective.

1.2 Hazard Configuration   contents ⇑ ↑  ^  v ↓ ⇓

The baseline VTEC hazard types, categories, and metadata are set up in the files discussed in this section. All of these files can be overridden to adjust modifiable attributes of existing hazards or to add new hazard types.

1.2.1 Hazard Types   contents ⇑ ↑  ^  v  ↓ ⇓

The Hazard Types are stored in a localization file (HazardTypes.py) identifying all the hazards and basic information about each. (This is similar to the VTECTable in legacy operations.) It’s stored as a:

• Python Dictionary of dictionaries with

• Key: phen, sig, sub-type (optional). Examples:

• TO.W
• FF.W.Convective
• FF.W.NonConvective (e.g. Dam Failure)

• Fields: Please see the HazardTypes.py file for descriptions of the fields

• headline -- 'FLASH FLOOD WARNING'

Note that the Hazard Types file contains almost 100 hazard types and is quite large. Using incremental override to add a new hazard type, the SITE level file could simply contain something like:

Also, since Hazard Services will eventually implement a National Hazard Database, there is the possibility that with user-defined hazard types there could be duplicate hazard types with different meanings which could lead to confusion for Forecasters viewing hazard information from other sites. This problem needs to be addressed, perhaps as a National registry for hazard types.

1.2.2 Hazard Categories   contents ⇑ ↑  ^  v  ↓ ⇓

Hazard Types are assigned user-configurable Hazard Categories which are defined in another localization file containing a dictionary of Python List entries. The dictionary is a mapping from a category name to a list of hazard types (and possibly subtypes).

The Hazard Categories make it easier to select the desired Hazard Type in the Settings dialog and the Hazard Information Dialog. Incremental override could be used on this file as well, to add to or modify the BASE configuration. For example, to move hydrology watches into a new category called, literally, “Hydrology Watches”, the SITE override file might contain either of the following:

The file HazardCategories.py is subject to incremental override. This means that when a list in an override is at the same namespace as a list in the base, the default behavior is to combine the override list with base list. So to remove items from an existing list in the base, one needs to either specifically remove those items (first example) or first stipulate that the base list should be ignored before presenting the override list (second example).

1.2.3 Hazard Metadata   contents ⇑ ↑  ^  v  ↓ ⇓

The Hazard Metadata is user-configurable per Hazard Type and appears in the Hazard Information Dialog. The configuration for the metadata consists of these types of files:

• HazardMetaData.py : One Python file which designates the metadata file name for each hazard type (or set of hazard types that share common metadata definitions). Here is the current default content of HazardMetaData.py so you can see the format:

• Metadata Python files which contain the fields, choices, and display format for all the metadata to appear in the Hazard Information Dialog and subsequently in the products. For example, the MetaData_FF_W_Convective.py file is shown in Appendix 3.

Hazard Metadata is specified as megawidgets so that they can be displayed easily in the Hazard Information Dialog. For information on megawidgets see Chapter 2.

Overriding metadata files. Using the class-based approach, the override files need only contain the methods that need to be changed or added. As a simple example, suppose one wanted to have a more descriptive GUI label for one Call to Action. The override file would need only contain:

CallsToActionAndImpacts.py:

The green is what was added to the default instance of this method.

1.2.4 Hazard Type Color Table     contents ⇑ ↑  ^  v  ↓ ⇓

This functionality will eventually interoperate with the GFE Hazard Grid color table.

1.2.5 Hazard Expiration Alerts      contents ⇑ ↑  ^  v  ↓ ⇓

Focal points can configure when and how alerts, based on hazard expiration, manifest themselves. This is controlled in a file named HazardAlertsConfig.py; the base configuration is shown here. Please refer to that document in the discussion below. At a later time, there may be a GUI available to assist in this process. For now, overrides of this configuration can be created via the Localization Perspective.

The structure of HazardAlertsConfig.py is fairly complex. The complexity is mostly because this file is a highly nested set of lists inside of dictionaries inside of lists inside of dictionaries. This makes the task of defining the namespace for the purposes of incremental override challenging. Fortunately, the incremental override capability has all the tools necessary to rigorously define the namespace within this file.

The basic unit of information in this file is an alert configuration entry, which associates one or more expiration alerts with a hazard type or list of hazard types. Here is an example of one alert configuration entry:
            {

                "description": "NonConvective Flash Flood Warning",

                "hazardTypes": ["FF.W.NonConvective"],

                "criteria": [

                    {"name": "1st warning",

                     "expirationTime": 50,

                     "units": "percent_completed",

                     "manifestations": ["Console"],

                     "color": {"red": 255, "green": 255, "blue": 0, "alpha": 255},

                     "blinking": False,

                     "italic": False,

                    },

                    {"name": "2nd warning",

                     "expirationTime": 90,

                     "units": "minutes",

                     "manifestations": ["Console"],

                     "color": {"red": 255, "green": 0, "blue": 0, "alpha": 255},

                     "blinking": False,

                     "italic": False,

                    },

                    {"name": "3rd warning",

                     "expirationTime": 15,

                     "units": "minutes",

                     "manifestations": ["Popup"],

                    },

                ]

            },

This entry applies to the event types listed in "hazardTypes": [ ] , which for this example is only "FF.W.NonConvective". Multiple alert triggers can be associated with an entry, and these are listed in "criteria": [ ]. Alert triggers with a manifestation of "Console" control the appearance of the Time to Expiration column in the Hazard Services Console. Alert triggers with a manifestation of “Popup” result in a small informational window popping up at the appropriate time. The first "criteria" here means turn the Time to Expiration column yellow when the event is half over (50% completed). The second "criteria" means turn the Time to Expiration column red when the event is 90 minutes from expiring. The last "criteria" causes a popup to appear noting the expiration of the event 15 minutes before expiration.

The user will note that HazardAlertsConfig.py is not simply a list of alert configuration entries. Rather, those entries are found in a list at "eventExpiration": { "configuration": [ ] } . This is because theoretically this file might eventually contain information about alerts not based on event expirations.

Let us suppose there was a desire to change the color of the console alerts only for convective FFWs. Here is an override file that could accomplish this:

The "_override_by_key_description_" string that appears at the beginning of the "configuration": list helps us manage the namespace for incremental override. What this does is specify that the value of the key "description" within the individual list members is used to define the namespace. Using this strategy means that each alert configuration entry needs to have a unique value for "description". One can see that approach is also applied to the list of "criteria", but with the key "name". With the namespace rigorously defined, the only other information we need to supply is the new color for the alerts; orange for the "1st warning" and magenta for the "2nd warning". We could actually leave the value for “alpha” out as well, since it is being left at 255 just as for the base file.

Note that as this is written (December 2018) overrides to HazardAlertsConfig.py do not take effect immediately after making them in the localization perspective; one must stop and restart Hazard Services to have them take effect.

1.2.6. Hazard Services use of existing GFE configuration information.

           contents ⇑ ↑  ^  v  ↓ ⇓

Hazard Services makes use of some existing GFE configuration files. What follows are the paths to these files after a standard A-II installation (LLL stands for a site ID):

common_static/configured/LLL/gfe/python/SiteCFG.py
cave_static/configured/LLL/gfe/userPython/textUtilities/DefaultCityLocation.py
cave_static/configured/LLL/gfe/userPython/textUtilities/DefaultAreaDictionary.py

Note that if one is in service backup by way of the Change Site popup, one must have instances of these files installed in directories for the backup site.  If this is not true, then product generation in service backup mode will fail.  For SiteCFG.py, one can always copy your primary site version into the backup site directory and that will be fully functional. For immediately neighboring sites, one can usually do this with DefaultCityLocation.py and DefaultAreaDictionary.py as well.  However, for a more distant site it is always necessary to obtain the default version of DefaultCityLocation.py and DefaultAreaDictionary.py from the site being backed up.

Now we want to say something about how the information in these GFE configuration files gets used. Here we mention only items Hazard Services actually uses, these data structures may contain other configuration items. Note that DefaultAreaDictionary.py does not appear to impact county based products.

SiteCFG.py is a per site (i.e. WFO id) dictionary with the following contents:

'wfoCityState': Used to describe the location of issuing WFOs in MND headers.
'wfoCity': Used to describe the location of issuing WFOs in headlines and first bullets.
'fullStationID': 4 character WMO ID for each WFO. Legacy_Base_Generator.py uses it for VTEC processing, whereas ProductLevelMethods.py uses it to help encode the first line of a product (e.g. WUUS55 KBOU 171915). The WMO ID in the first line of a product sort of functions as the node for product transmission.

DefaultCityLocation.py is a per UGC (e.g. COC001) dictionary containing lists of cities and
their corresponding lat/lons.

Hazard Services uses this to list cities in non-polygon products as long as "accurateCities" is not set to False for the product type in HazardTypes.py. (Currently for IOC all active hazard types without polygons do have this set to False).

DefaultAreaDictionary.py: is a per UGC dictionary with the following contents:

'fullStateName': Plain language name of state, used in headlines, first bullets, and elsewhere.
'partOfState': Used to describe part of a state counties are in; like FE_AREA from old CitiesInfo shapefile.
'ugcCities': Used to list cities in non-polygon products if "accurateCities" is set to False for the product type in HazardTypes.py.

Normally in Hazard Services, files that are simple global python variable initializations can be directly overridden through incremental override. Because these are GFE configuration files, they are not subject to incremental override. However, one can create a total replacement site override; that is assuming that the same change you want for Hazard Services is also desired in GFE.

A caution about overriding DefaultAreaDictionary.py and DefaultCityLocation.py; the information is often defined redundantly and must be overridden in multiple places. For example, in either of these files if one wanted to override the cities for Sarpy county, NE, one would override this information on behalf of three keys, 'NEC153' (the county code), 'NEZ053' (the colocated zone) and 'NE_Sarpy'.

If there is a desire to override some aspect of this information for Hazard Services only, then one needs to override a method. The most straightforward thing is to override the module where this data is presented to clients, which is Bridge.py. What makes this desirable is that all downstream clients of the affected changes automatically see the same change. The methods in Bridge.py that return these types of data to clients is shown in the table that follows:

1.3 Recommenders     contents ⇑ ↑  ^  v  ↓ ⇓

The Forecaster can use Recommenders to augment both situational awareness and hazard operations. Recommenders are code modules which produce hazard events based on a variety of input data, including but not limited to hydrometeorological observations, model forecasts, forecaster input, other hazards, and the Hazard Services session state. Session state refers to information such as the Hazard Services current selected time and index of the frame of time-matched data loaded in CAVE. Forecaster input can be obtained through a dialog defined by the recommender and/or the Forecaster’s interaction with the Hazard Services spatial display. Based on these input data, the recommender may produce pending or potential hazards which are displayed in the Hazard Services spatial display and console. In short, recommenders can quickly analyze a vast variety of data and notify the Forecaster of places in the area of forecast responsibility where criteria for hazardous conditions may be met.

There are various components composing the Hazard Services Recommendation process. The distribution and refinement of logic and function across these components needs to be optimized according to the essential considerations such as performance, reliability, usability, and maintainability.

1.3.1 Recommender Framework     contents ⇑ ↑  ^  v  ↓ ⇓

Recommenders are run from within the Recommender Framework. This framework is responsible for loading recommenders from the Localization File System, retrieving meta-information about them, and obtaining any dialog definitions for required user input or any required input from the Hazard Services spatial display.

1.3.2 Recommender Interface    contents ⇑ ↑  ^  v  ↓ ⇓

In order to be run from the Recommender Framework, a recommender must implement the following four methods from the RecommenderTemplate.py interface:

• defineScriptMetadata - Returns a dictionary which defines basic information about the recommender, such as author, script version, and description.
• defineDialog - Defines a dialog that will be presented to the user prior to the recommender’s execute routine. This returns a dictionary with the following key-value pairings; those specified in italics are optional:

• fields: List of maps, each defining a megawidget (see Chapter 2.0 for more information on megawidgets).
• valueDict: Map pairing attribute identifiers with their starting values. The attributes are the parameters the user may alter. For example, an attribute might be “damName”, and its starting value could be the name of a dam.
• title: Optional title of the dialog.
• runToolTriggers: Optional list of megawidget identifiers indicating which of the megawidgets defined within fields are to trigger tool execution when invoked, if any.

• sideEffectsScript: Optional string which, if provided, is used as the Python script that defines the applySideEffects() method, allowing invocations of megawidgets to alter one another’s mutable properties (see Section 2.1.2).
• minimumTime: Minimum possible time that may be specified using any TimeScale megawidgets included in the dialog, specified as epoch time in milliseconds. This is not required if no TimeScales are found within the fields.
• maximumTime: Maximum possible time that may be specified using any TimeScale megawidgets included in the dialog, specified as epoch time in milliseconds. This is not required if no TimeScales are found within the fields.
• minimumVisibleTime: Minimum visible time for any TimeScale megawidgets included in the dialog, specified as epoch time in milliseconds. This is not required if no TimeScales are found within the fields.
• maximumVisibleTime: Maximum visible time for any TimeScale megawidgets included in the dialog, specified as epoch time in milliseconds. This is not required if no TimeScales are found within the fields.

• defineSpatialInfo - Determines spatial information needed by the recommender.
• execute - Runs the recommender. Accepts a set of events, Hazard Services session state, user input from a dialog (if any), and user input from the Hazard Services Spatial Display (if any). This returns a list of recommended events.

The Recommender Framework provides access to each of these four methods. So, a client can query a recommender’s defineDialog method, receive the dialog megawidget definition map, build the dialog, retrieve the user input from the dialog, pass this input to the recommender’s execute method, and receive recommendations.

Note: If one is creating a new recommender or modifying an existing recommender, one needs to be careful about how one goes about sharing information among these four top level methods. It is not a good idea to set a member variable in the recommender class on behalf of one of these four methods and then try to pick up that value while executing a different top level method. This type of communication should be done by adding to or modifying the set of attributes associated with any hazard events in question.

The following is an example of a recommender dialog definition within Python. This uses dictionaries specifying megawidgets (see Section 2.0 Megawidgets Overview) to build up the required user interface elements:

1.3.3 Baseline Recommenders   contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

Recommenders are stored in the AWIPS II Localization File System. This allows Base versions of recommenders to be customized for use at an office and by individual users. They can be viewed and edited using the CAVE Localization Perspective. New Recommenders created via the Localization Perspective can then be run by the Recommender Framework.

1.3.3.1 River Flood Recommender     contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

The River Flood Recommender retrieves information from the Hydro Database about the forecast points within the forecast area for the given site. Hazard Events are created according to the selections made by the user. Criteria for Watches, Warnings, and Advisories are based on the observed and forecast data about each forecast point. These rules can be customized by the site through the override process.

Note that RiverPro functionality has been separated between recommendations and product generation. RiverPro recommends Products which are adjusted by the the forecaster before issuing. In contrast, the River Flood Recommender in Hazard Services recommends Hazard Events. Those Hazard Events can be adjusted by the forecaster before being transformed into Products through Hazard Services Product Generation. The forecaster can also adjust the resulting product pieces, but the ultimate intention is that most information will be captured prior to generating the products.

1.3.3.2 Flash Flood Recommender^[a][b][c][d]    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

The Flash Flood Recommender is dependent on having the FFMP up and running before it will function. The Flash Flood Recommender compares a source of Flash Flood Guidance (most commonly grids from the RFC) to a source of precipitation estimates (most commonly from the local WSR-88D) and identifies those FFMP small basins where the basin average precipitation exceeds the basin average adjusted guidance. “Adjusted guidance” means the raw guidance accounting for whatever modifications are applied in the Flash Flood Recommender GUI. Groups of spatially contiguous small basins so identified are used to create recommendations for Convective Flash Flood Warnings. The Flash Flood Recommender can be challenging to test when the weather does not support recommendations (and therefore challenging to customize). If there are any echoes at all, the most straightforward means of forcing a recommendation to be made is to select the ratio option in the FFMP fields: selector, and then enter an absurdly low number in the Value field, such as 2.00% of guidance.

1.3.3.3 Storm Track Recommender    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

This is described in a document called Storm Track Recommender.

1.3.3.4 DamBreak Flood Recommender    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

The Dam Break Flood Recommender retrieves an impact area boundary from postgres for a chosen dam and creates a Flash Flood hazard event. If the user chooses “Watch” an FF.A event is created. If the user chooses “Warning” an FF.W.NonConvective event is created. See section 1.0.2 for information about how to configure Hazard Service to properly utilize this recommender.

1.3.3.5 BurnScar Flood Recommender   contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

The BurnScar Flood Recommender is similar to the DamBreak Flood Recommender. It retrieves the impact area boundary from postgres for a chosen burn scar and creates a Flash Flood hazard event. If the user chooses “Watch” an FF.A event is created. If the user chooses “Warning” an FF.W.BurnScar event is created. See section 1.0.2 for information about how to configure Hazard Service to properly utilize this recommender.

1.3.3.6 Create RVS Tool.    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

The RVS product is a summary of the state of several river forecast points, none of which necessarily need to be above action or flood stage to issue this product.  In Hazard Services, one must use the Create RVS Tool to issue an RVS.  The user is given the option of issuing a product with forecasts for the next three days (short range) or the next four weeks (long range).

1.4 Product Generation   contents ⇑ ↑  ^  ─  + v  ↓ ⇓

Product Generation integrates the functionality across the legacy applications which included two kinds of template languages (warnGen and RiverPro) and Python module solutions (GHG). The goal is to meet or exceed legacy functionality by preserving the best of these various approaches. Reaching this goal required feedback from and collaboration with operational Focal Points, those who will customize the Hazard Services Product Generation, by translating the current warnGen and RiverPro templates as well as GHG Hazard text formatters into the Hazard Services Product Generation approach. Gathering and translating current legacy customizations has informed the approach.

There are various components to the Hazard Services Product Generation process. The distribution and refinement of logic and function across these components needs to be optimized according to essential considerations such as performance, reliability, usability, and maintainability.

Product Generation components for now only produce the legacy Hydrological products. Some of the areal (non-river) flood hazards now use the Impact Based Warnings (IBW) format. It is expected that in the future more new formats will be added; at least Common Alerting Protocol (CAP), for example. Below is an example legacy product.

Legacy ASCII Product:

WGUS63 KOAX 211939
FFAOAX

URGENT - IMMEDIATE BROADCAST REQUESTED
TEST...FLOOD WATCH...TEST
NATIONAL WEATHER SERVICE OMAHA/VALLEY NE
239 PM CDT THU APR 21 2016

IAZ043-055-069-NEZ034-045-220345-
/T.NEW.KOAX.FF.A.0001.160421T1939Z-160422T0400Z/
/00000.0.ER.000000T0000Z.000000T0000Z.000000T0000Z.OO/
Monona-Harrison-Pottawattamie-Burt-Washington-
Including the cities of Woodbine...Lyons...Mapleton...Onawa...
Blair...Dunlap...Missouri Valley...Logan...Tekamah...Oakland...
Council Bluffs and Decatur
239 PM CDT Thu Apr 21 2016

...TEST FLASH FLOOD WATCH IN EFFECT UNTIL 1100 PM CDT THIS EVENING...

The National Weather Service in Omaha/Valley has issued a

* This is a test message. Test Flash Flood Watch for portions of
Iowa and east central Nebraska...including the following
counties...in Iowa...Harrison...Monona and Pottawattamie. In east
central Nebraska...Burt and Washington.

* This is a test message. Until 1100 PM CDT Thu this evening

* This is a test message. |* current hydrometeorological basis *|

* This is a test message. |* current hydrometeorological impacts *|

PRECAUTIONARY/PREPAREDNESS ACTIONS...

Do not enter or cross flowing water or water of unknown depth.

Stay away or be swept away. River banks and culverts can become
unstable and unsafe.

&&


$$

Please refer to this presentation for more information about Product Generation:

 Documentation for Product Generation.

1.4.1 Product Generator Table    contents ⇑ ↑  ^  v  ↓ ⇓

In the Hazard Services Localization area, there is a Product Generator Table which lists the Product Generators that are called for the various Hazard Types. What follows is the base version of ProductGeneratorTable.py:

ProductGeneratorTable = {
       "FFA_ProductGenerator": {
           "allowedHazards": [
            ('FF.A', "Flood"),
            ('FA.A', "Flood"),
            ('FL.A', "Flood1"),
            ],
           "previewFormatters": ["Legacy_FFA_Formatter"],
           "issueFormatters": ["Legacy_FFA_Formatter"],
           "generatorType": "HAZARD_PRODUCT_GENERATOR",
           "correctAllSegments": False,
           },
       "FLW_FLS_ProductGenerator" : {
           "allowedHazards": [
            ('FA.W', "Flood1"),
            ('FA.Y', "Flood2"),
            ('FL.W', "Flood3"),
            ('FL.Y', "Flood4"),
            ('HY.S', "Flood5"),
           ],
           "previewFormatters": ["Legacy_FLW_FLS_Formatter"],
           "issueFormatters": ["Legacy_FLW_FLS_Formatter"],
           "generatorType": "HAZARD_PRODUCT_GENERATOR",
           "correctAllSegments": False,
           },
       "FFW_FFS_ProductGenerator" : {
           "allowedHazards": [
            ('FF.W.Convective',     "Flood"),
            ('FF.W.NonConvective',  "Flood"),
            ('FF.W.BurnScar',  "Flood"),
            ],
           "previewFormatters": ["Legacy_FFW_FFS_Formatter"],
           "issueFormatters": ["Legacy_FFW_FFS_Formatter"],
           "generatorType": "HAZARD_PRODUCT_GENERATOR",
           "correctAllSegments": True,
           },
       "ESF_ProductGenerator": {
           "allowedHazards": [
            ('HY.O',     "Flood"),
            ],
           "previewFormatters": ["Legacy_ESF_Formatter"],
           "issueFormatters": ["Legacy_ESF_Formatter"],
           "generatorType": "HAZARD_PRODUCT_GENERATOR",
           "correctAllSegments": False,
           },
       "RVS_ProductGenerator": {
           "allowedHazards": [
            ('FL.W', "Flood"),
            ('FL.Y', "Flood"),
            ('FL.A', "Flood"),
            ('HY.S', "Flood"),
            ],
           "changeHazardStatus": False,
           "autoSelect": False,
           "previewFormatters": ["Legacy_RVS_Formatter"],
           "issueFormatters": ["Legacy_RVS_Formatter"],
           "generatorType": "NON_HAZARD_PRODUCT_GENERATOR",
           "correctAllSegments": False,
           },
       "WSW_ProductGenerator" : {
           "allowedHazards": [
           ('BZ.W', 'WinterWx'),     # BLIZZARD WARNING
           ('IS.W', 'WinterWx'),     # ICE STORM WARNING
           ('LE.W', 'WinterWx'),     # LAKE EFFECT SNOW WARNING
           ('WS.W', 'WinterWx'),     # WINTER STORM WARNING
           ('WS.W.LakeEffect', 'WinterWx'),     # WINTER STORM WARNING
           ('ZR.Y', 'WinterWx'),     # FREEZING RAIN ADVISORY
           ('LE.Y', 'WinterWx'),     # LAKE EFFECT SNOW ADVISORY
           ('WW.Y', 'WinterWx'),     # WINTER WEATHER ADVISORY
           ('WW.Y.FreezingRain', 'WinterWx'),     # WINTER WEATHER ADVISORY
           ('WW.Y.LakeEffect', 'WinterWx'),     # WINTER WEATHER ADVISORY
           ('BZ.A', 'WinterWx'),     # BLIZZARD WATCH
           ('LE.A', 'WinterWx'),     # LAKE EFFECT SNOW WATCH
           ('WS.A', 'WinterWx'),     # WINTER STORM WATCH
           ('WS.A.Blizzard', 'WinterWx'),     # WINTER STORM WATCH
           ('WS.A.LakeEffect', 'WinterWx'),     # WINTER STORM WATCH
           ('WC.W', 'WindChill'),    # WIND CHILL WARNING
           ('WC.Y', 'WindChill'),    # WIND CHILL ADVISORY
           ('WC.A', 'WindChill'),    # WIND CHILL WATCH
           ],
           "previewFormatters": ["WWW_WSW_Formatter", "CAP_Formatter"],
           "issueFormatters": ["WWW_WSW_Formatter", "CAP_Formatter"],
           "generatorType": "HAZARD_PRODUCT_GENERATOR",
           },
       }

When the Forecaster chooses to Preview or Issue a set of hazards, the system consults this table to call the appropriate Product Generator(s) and Product Formatters. Note that multiple formats can be supported and this list may grow in the future.

1.4.2 Product Generation Framework     contents ⇑ ↑  ^  v  ↓ ⇓

When the forecaster selects “Preview” from the Hazard Information Dialog (HID), the system calls the Product Generation Framework to execute product generators with a desired set of formats. The framework calls the Product Generators and then calls appropriate Formatters to put information into requested formats.

• Inputs: The Product Generation Framework is given the name of the desired product generator, a list of desired formats, and a set of hazard events.
• Outputs: The Product Generation Framework returns a list of generated products in the desired formats plus a set of updated hazard events. When issued, the hazard events are updated by the Product Generator with product information such as expiration time, issue time, VTEC codes, PILs, and ETNs.

The set of desired formats is configurable via the Product Generator Table as noted in the above section.

1.4.3 Product Generators     contents ⇑ ↑  ^   ─  +  v  ↓ ⇓

The input to a Product Generator is a set of Hazard Events. As with Recommenders, a Product Generator must implement an 'execute' method. Each product generator can generate one or more products. For example, if given a point and an area hazard event, the FFA_ProductGenerator will generate two products, one for the point and one for the area, since points and areas must be in separate products by current policy. Output is one dictionary for each product generated.

Product Generators must also implement a “defineDialog” method similar to that of the recommenders. This method is used to define the whether or not the Staging dialog should be displayed prior to the Product Editor. Specifics on how to implement this method can be found in the Recommender section.

A key function of the Product Generators is to call the VTEC engine and generate the VTEC records. During this time the active table is consulted to ensure the proper lifecycle of the product is maintained.

Another key function of the Product Generators is to access the metadata definitions for the hazards, couple that with the Forecaster input from the HID, and generate a dictionary that contains the required information so that the formatters can correctly report that information in the products. For example, the Hazard Events given as input to the Product Generators contain the list of Calls to Action that were checked off by the Forecaster in the Hazard Information Dialog. Coupling that with the metadata for the hazard tells the Product Generator what text is to be added in the product dictionary.

As an example, here is an excerpt from the metadata for an FF.W hazard:

MetaData_FFW_Convective = [

    ...

          # CALLS TO ACTION

          {

          "fieldType":"CheckList",

          "label":"Calls to Action (1 or more):",

          "fieldName": "cta",

          "choices": [

               ...

               }, {

               "displayString": "Urban flooding",

               "productString": "EXCESSIVE RUNOFF FROM HEAVY RAINFALL WILL CAUSE ELEVATED LEVELS ON SMALL CREEKS AND STREAMS...AND PONDING OF WATER IN URBAN AREAS...HIGHWAYS...STREETS AND UNDERPASSES AS WELL AS OTHER POOR DRAINAGE AREAS AND LOW LYING SPOTS."

               },

...

If the Forecaster chose 'Urban flooding' in the Hazard Information Dialog, the Product Generator would then add the 'productString' to the product dictionary which then can be used by the formatters.

The Product Generators are located under Hazard Services --> Product Generators in the Localization Perspective.

1.4.3.1 Product Template    contents ⇑ ↑  ^   ─  +  v  ↓ ⇓

Most of the required methods for creating a new generator are contained in the Product Template, located under Hazard Services -> Product Generators in the Localization Perspective. This acts a base class and contains common logic across the legacy product generators.

This approach is similar in structure to the AWIPS II Graphical Hazard Generator text formatter infrastructure to allow for code-reuse and familiarity to Focal Points.

1.4.3.2 Product Dictionary     contents ⇑ ↑  ^   ─  +  v  ↓ ⇓

The product dictionary is structured to reflect the legacy segmentation. There are 4 main levels in the product dictionary, each level is a python dictionary embedded inside the previous level. The text below lists each level of the product dictionary from highest to lowest. Also reference slides 28-32 located here, for a consolidated example of a product dictionary.

Product - This is the entire dictionary created by the generator and includes all the available data. This can be accessed anywhere in the formatters by using “self.productDict”.

Segments - The product dictionary will contain one or more segment dictionaries (if the product is segmented). The segment dictionaries can be accessed by using the “segments” key in the product dictionary. Each segment dictionary includes the information needed to create exactly one product segment.

Sections - Each segment dictionary will contain one or more section dictionaries. The section dictionaries can be accessed by using the “sections” key in the segment dictionary. Each section dictionary includes the information needed to create exactly one product section.

Hazards - Each section dictionary will contain one or more hazard event dictionaries. The hazard event dictionaries can be accessed by using the “eventDicts” key in the section dictionary. Each hazard event dictionary will include all the attributes from the hazard event objects (data from the HID).

1.4.4 Formatters     contents ⇑ ↑  ^   ─  +  v  ↓ ⇓

1.4.4.1 NWS_Base_Formatter.py, etc.    contents ⇑ ↑  ^   ─  +  v  ↓ ⇓

The files NWS_Base_Formatter.py, ProductLevelMethods.py, SectionLevelMethods.py, and SegmentLevelMethods.py collectively contribute to formatting the “Legacy” text products; e.g. traditional watch/warnings/advisories as issued by warnGen, GHG, or RiverPro. Overrides to methods in these files can potentially impact the formatting of legacy text products for many different hazard types at once. An example of this is the method named additionalInfoStatement(), which adds a URL where the public can get more information. The actual product part instance of this method is in NWS_Base_Formatter.py, but the text content is provided by the instance in ProductLevelMethods.py. This URL defaults to www.weather.gov, but suppose an office wanted a different URL, such as www.crh.noaa.gov/hazards (this URL is just made up). A site override version of ProductLevelMethods.py would be created with the following:

1.4.4.2 Product Parts    contents ⇑ ↑  ^   ─  +  v  ↓ ⇓

The Legacy format utilizes the concept of product parts. Product parts is a way of breaking a complex format into smaller well defined pieces. The Legacy format then loops over the defined product parts for the specific product type to create the final text. This concept allows for easy modification of the structure of the final output of the format. Pieces of the product can be added or removed easily by adding or removing the corresponding product parts.

For Hydrologic products, the set of product parts included in the text product are defined in HydroProductParts.py (not HazardProductPart.py). When a product part has any non-default characteristics, these characteristics are specified in ProductEditorFields.py. At a minimum, a product part must be entered into ProductEditorFields.py in order for a user to be able to edit that part. Both HydroProductParts.py and ProductEditorFields.py can be found in the Localization Perspective under
   Hazard Services -> Utilities.

You can reference the linked example here of the process needed to add a new product part to an existing product.

The “processProductParts()” method in the NWS_Base_Formatter.py module is the method that handles the looping over each product part and combining the text from each to create the final product. This method also handles providing the correct dictionary to each product part method. Each product part method is passed the same level of the product dictionary as what the product part was defined as. Meaning if the product part was defined as a “segment” level part then only the corresponding segment level dictionary is passed to the method. Ideally that method should need only the data provided in that dictionary.

1.4.5 Text Utilities   contents ⇑ ↑  ^   ─  +  v  ↓ ⇓

These modules located under Hazard Services -> Utilities in the Localization Perspective (under common_static/base/HazardServices/python/textUtilities/ in EDEX) mainly aid in the formating of the products during Product Generation. A brief description of each module is given below:

AttributionFirstBulletText.py - Executive that farms out the generation of the attribution and firstBullet product parts of the legacy format. (See section 4.4 for more information.)

BasisText.py - Generates the Basis Bullet product part for warnGen Hazard Types.

DamMetaData.py and BurnScarMetaData.py - Contains a python dictionary that holds metadata about predefined impact areas for dam breaks and burn scars. (See section 1.0.2.2 for more information.)

EventLevelMethods.py,  ProductLevelMethods.py,  SectionLevelMethods.py, and SegmentLevelMethods.py - Most of the methods that generate text content for individual product parts.

ForecasterInitials.py - Allows sites to override the forecaster’s username that gets put into products that include the “initials” product part. By default this module uses the username from os.getlogin().

ForecastStageText.py - Generates the forecast stage bullet for hazards that were formerly issued by RiverPro.

FourWeekStageForecastTable.py and ThreeDayStageForecastTable.py - Generates text for specific instances of tables of forecast stage data from a river gauge.

IBW_Text.py - Formats the impact based warning bullets.

HydroProductParts.py - Defines the product parts for hydro products.

QueryAfosToAwips.py - Allows access to the afos_to_awips table in the fxatext database.

RiverForecastUtils.py - Utility methods and constants used by the River Flood Recommender and Generators to aid in the gathering and calculation of data for the river forecast points.

TableText.py - Generic module that is used to generate tables in the products. Currently this is utilized to create the Flood Point Table in the FL.* hazards.

TextProductCommon.py - Contains several utility methods to aid in the formatting of products.

The following modules allow river gauge data and river forecasts to be mirrored in python:

 HydrographForecast.py  Hydrograph.py    SHEFObserved.py
HydrographObserved.py  SHEFBase.py      SHEFPrecip.py
HydrographPrecip.py    SHEFForecast.py  StageDischargeUtils.py

There are some additional modules in textUtilities/ which do not apply to the set of hazards IOC is designed to issue, so we won’t bother to describe them now. These are:
HazardCriteria.py DescriptorDictionary.py HazardProductParts.py
HazardServicesHeadlines.py

1.4.6 Product Editor  contents ⇑ ↑  ^   v  ↓ ⇓

The “Hazard Data Editor” tab allows for the editing of the text of various product parts. The product parts that are displayed here are configurable within each formatter. Configuration is described below. Forecasters can configure each of the following options for each product part in the file ProductEditorFields.py. Here we describe the items associated with each part that are most likely to be subject to customization; see the base version of ProductEditorFields.py for a complete list.

displayable - This is a boolean value that will determine if the product part is added to the “Hazard Data Editor” tab on the Product Editor.

editable - This is a boolean value that will determine if the product part is editable while on the “Hazard Data Editor” tab. If set to false, then the text will be added as a non-editable label.

label - This is a string value that will be the label displayed above the product part on the “Hazard Data Editor” tab of the Product Editor.

required - This is a boolean value that will determine if the product part is required. Required parts must contain text before the product can be issued. If a required field is blank then hitting the “Issue All” button on the Product Editor will result in a dialog being presented to the user which notes the problem; it will not be possible to actually issue the product until the product part in question actually has some content. The required fields are denoted by the label (*required field) after the label.

allowPreviousText - Determines whether or not the Product Part allows the use of previous text. If False generated text will always be used. If not defined it will default to allowing previous text for the Product Part.

Below is the product part methods for the “initials” product part. This product part adds the forecaster’s initials to the bottom of FF.W products. Note that for most product parts, there is now a method in a formatter, which is the interface to the product parts executive logic, and a method in one of the ‘LevelMethods’ modules, which actually generates the content.

NWS_Base_Formatter.py:

ProductLevelMethods.py:

1.5 Some miscellaneous best practices.

Chapter 2: Megawidgets -- Customizable User Dialogs    contents ⇑ ↑  ^   v  ↓ ⇓

2.0 Megawidgets Overview     contents ⇑ ↑  ^   v  ↓ ⇓

Megawidgets are a set of graphical user interface (GUI) components found in use throughout Hazard Services. They provide an easy way for those who are modifying or building recommenders requiring user input to create dialog boxes in order to allow said recommenders to gather information from the user. Megawidgets are also used in other areas of Hazard Services that are configurable via localization: they populate the Details (metadata) section of the Hazard Information Dialog, and they may be used to create GUI components in the Product Staging Dialog for product generation as well.

Megawidgets are discussed in depth in a separate document, available here. This document is very good but does have a few minor errors and omissions. Where those have been identified, we attempt to cover them in the next section.

2.1 Megawidget Usage Hints.   contents ⇑ ↑  ↓ ⇓

Most of the time individual megawidgets organize vertically. To have arbitrary widgets organize horizontally, put them in a container widget where you have "fieldType": "Composite", and set the “numColumns” to be the number of widgets.

The megawidget reference currently says that to specify the contents of a Label megawidget, one uses the “label” attribute.  This will cause an error; one actually needs to use the “values” attribute to specify the contents of the Label megawidget.

Assigning a dict directly to the “values” element of a megawidget never does anything useful.  However, one can put any arbitrary dict into a list of length one and assign that to a “values” element, and that will often work fine. Generally the only context in which this consideration matters is for a megawidget of type “HiddenField”.

With many of the megawidgets, you can be imprecise with regard to how you structure the content of the “values” attribute, which is generally used to indicate which things are initially selected. Most megawidgets will allow either a scalar value or a list. The megawidget of type "HierarchicalChoicesTree" is different; the content of the “values” attribute must have exactly the same structure as for the “choices” attribute.  This is neither a simple list nor a scalar, but rather a list of dictionaries with two (or three) very specific keys. Also be aware that the HierarchicalChoicesTree widget has an “identifier” element, which is not mentioned directly in the HierarchicalChoicesTree section of the megawidget reference.  The “identifier” element is at the same level as the “displayString” element, and will default to the same value as the “displayString” element if not specified.  The “displayString” is what is showed to the user, the “identifier” is used for internal accounting. The HierarchicalChoicesTree widget is very challenging to use in hazard MetaData.  If used, it is best to never change either any of the “identifier” values or any of the lists of “children” as the hazard evolves.

There are two valid attributes which can be associated with megawidget elements that are not documented anywhere in the Megawidget Reference. These are “refreshMetadata”  and  'useNewValueOnRefresh'.  Sometimes, as a result of making some choice in the Hazard Information Dialog,  there is a need to either add or delete a megawidget element, change an immutable property, or add a new property to an existing megawidget element. This can only be accomplished if one forces the metadata to regenerate. Adding the “refreshMetadata” attribute with a value of True to a megawidget element will mean that any user selection made from that megawidget element will result in having the metadata regenerate.  Normally, when the metadata regenerates, the values assigned to each element will end up coming from the previous state of the metadata.  This means that regenerating the metadata cannot change the value or values associated with any given metadata element present both before and after the regeneration. If one adds the 'useNewValueOnRefresh' with the value of True to a megawidget element, this means any value assigned to that element during the refresh will be retained and not overridden by the previous metadata content.