Chapter 3: Exercises and Examples    contents ⇑ ↑  ^   v  ↓ ⇓

3.0 Configuration    contents ⇑ ↑  ^   v  ↓ ⇓

3.0.1 Settings     contents ⇑ ↑  ^  ─  +   v  ↓ ⇓

By default initial setting, we mean the setting file that is used on startup in the situation where this had not been specifically defined. The initial setting can be specified by creating a user version of StartUpConfig.py where the item "defaultSettings" is set to the desired initial settings file. This can now also be done using SETUP->Manage Settings->Make My Current Default in the console.

Be aware that there is now a ‘universal generic base’ for all settings files, CommonSettings.py, found in /awips2/edex/data/utility/common_static/base/HazardServices/settings/config/. Also be aware that the way specific settings files interoperate with CommonSettings.py is still in flux.

It is possible to use the localization perspective to manipulate settings. However, for most things one might want to do with settings it makes more sense to use the Edit Default Setting: GUI (at SETUP->Manage Settings->Edit/Filter… in the console). See section 5.0 of the User Guide for more information about this. There are exceptions to this, including promoting custom settings to site level , configuring CommonSettings.py, and removing a base level default setting file.  For completeness, we will show some examples of using the localization perspective to manipulate settings.

3.0.1.1 Modifying an existing Settings file    contents ⇑ ↑  ^  ─  +   v  ↓ ⇓

• Locate the Settings file in the Hazard Services section of the Localization Perspective e.g. Hydrology_All.py
• Open the BASE version in the editor. Let’s suppose for the sake of argument that what you want to do is remove “Stream” from the set of default visible columns for that setting. Here is the part of the base Hydrology_All.py which defines this:

• From the Browser, Click MB3 over the BASE version and select Create Override File-->Site

• Copy and paste the top level Settings dictionary into the Override file e.g.
  Hydrology_All = {
  }
  (if the site file is already there, then for this exercise you are going to add only the "visibleColumns" section to it)
• Copy and paste the section you would like to modify e.g. “visibleColumns” and then modify as desired:

• Save the file in the Localization perspective and your change will be enacted.

Now you can restart Hazard Services and verify that the Stream column is gone, right? Wait a minute, the Stream column is still there...what happened? This is a result of the fact that this file is subject to incremental override. Incremental override is why one does not have to copy the whole base file to the site file. However, for incremental override, the default behavior for two lists at the same namespace is to merge the lists rather than to replace the list in the base file with the list in the site file. You have two options in the site file that can change this default behavior, as follows:

3.0.1.2 Creating a new Settings file   contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

This can be done in two ways; through the Edit Default Settings: dialog or through the localization perspective. Using the Edit Default Settings: dialog has the advantage of it being very hard to shoot yourself in the foot. Using the localization perspective has the advantage that the resulting settings file can be made ‘incremental’ (e.g. you only define the things you want to change.)

Here is how this can be done through the Edit Default Settings: dialog.

• Choose SETUP -> Manage Defaults -> Edit / Filter … from the top of the Hazard Services Console to open the Edit Default Settings: dialog.
• In the Display Name: text field, enter the desired title of your new setting.
• Using the selectors in the various tabs (Hazards Filter, Console, Recommenders, Maps/Overlays), make any desired changes to the settings information.
• Hit the Save As … button at the bottom of the dialog; this will pop a dialog where the new file name for the setting can be entered.
• Enter the new file name for the setting in the text field and then hit OK.

Here is how this can be done through the localization perspective:

• Go into the localization perspective and open Hazard Services -> Settings.
• Identify the existing default setting that is most similar to the new setting you wish to create.
• Open that setting file, and on the BASE level selector, choose Copy To->New File… using a third button popup.
• Enter a unique file name with a .py extension.
• Open the newly created setting file, which should only have a USER level selector.
• Double click on the USER level selector for the new setting file to bring the file into the edit area.
• Change the “displayName”: attribute to what you want the title of that setting to be in the console.
• This is very important; change the variable name in the first line (before the = { ) to be exactly the same as the setting file name without the .py extension.
• Make any other desired changes to the settings information.
• Hit ctrl-S to save the new setting file, and go back to the D-2D or Hydro perspective.
• Choose SETUP->Load… and invoke your newly created setting file.
• Choose SETUP->Manage Defaults->Edit… and make any necessary changes to configure the new setting.
• Hit the Save button.

3.0.1.3 Promoting a settings file to Site level.  contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

It is perfectly reasonable to use the dialog to initially create a new settings file, and then to later go into the localization perspective to further modify the settings information. Also, be advised that both of these means for creating a new settings file will create a USER level settings file. If it is desired to make the new settings file a SITE level settings file, this can be accomplished through the localization perspective as follows:

• Go into the localization perspective and open Hazard Services -> Settings.
• Find the entry for your new settings file and open it.
• Choose Copy To -> Site using a third button popup over the USER entry for your new settings file.
• Choose Delete using a third button popup over the USER entry for your new settings file.

3.0.1.4 Console Time Window.  contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

A recently added feature is to allow a setting file to specify the range of hours offset from the current time for which to display hazards in the Console. This currently defaults to essentially an infinite time range in all default settings files. Some sites or users may find it more convenient now for default settings files to make ALL Status values visible and apply a reduced size console time window. What follows is an override for CommonSettings.py that can accomplish this.

With this example override file, hazards valid in the future will appear in the Console essentially forever, but previous hazards will appear in the Console only for the last 8 hours (e.g. one shift). If there is a desire to have this apply to all users, installing this as a Region level override is a reasonable way to do this.

Note that the Console Time Window can be controlled using the Console tab of the Edit Default Settings: dialog.

3.0.1.5 Remove default base setting file.  contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

New settings files created by the user that are not overrides of an existing default base setting file are straightforward to remove.  If they are implemented at the User level, this can be done either using the localization perspective or by way of SETUP->Manage Settings->Delete.  At the Site level, one can still use the localization perspective, but at the Configured or Region level this needs to be done using the file system.  However, consider the situation where one wants to remove a settings file that was delivered as Base level default settings file.  This cannot be done by overriding or undoing an override of the setting file itself; this must be done using an override of StartUpConfig.py.  Suppose one wanted to remove the Hydrology_ESF settings file.  Here is an override of StartUpConfig.py that can accomplish this:

3.0.2 StartUpConfig      contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

Here we show how to modify an entry in StartUpConfig.py. Note that StartUpConfig.py is subject to incremental override, and so one only need specify the items to change; there is no need to copy the whole base file.

• Locate the StartUpConfig file in the Hazard Services tab of the Localization Perspective and open the BASE version
• Open the BASE version in the editor and locate the section you would like to modify e.g. “Console”
• From the Browser, Click MB3 over the BASE version and select Create Override File-->Site
• Choose the Site version just created in the localization file browser to bring it into the localization file editor. Put in the following content, and it will make ‘Hydrology_All.py’ be the settings file used by Hazard Services upon starting:

• Save the file in the Localization perspective with ctrl-S and your change will be enacted.

Note that the console control SETUP->Manage Settings->Make My Current Default is just like entering your current setting as the default setting for an override of StartUpConfig.py. Also note that this operation results in an incremental version of StartUpConfig.py being written out.

3.0.3 Hazard Types      contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

3.0.3.1 Modifying a Hazard Type entry     contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

Here we show how to modify a Hazard Type entry. These entries occur in the file HazardTypes.py. Note that HazardTypes.py is subject to incremental override, and so one only need specify the items to change; there is no need to copy the whole base file.

• Locate the HazardTypes.py file in the Hazard Services / Hazard Types section of the Localization Perspective
• Open the BASE version in the editor and locate the item you would like to modify e.g. “defaultDuration” for FA.A
• From the Browser, Click MB3 over the BASE version and select Create Override File-->Site
• Choose the Site version just created in the localization file browser to bring it into the localization file editor. Put in the following content; it will change the default duration for FA.A hazards:

• Save the file in the Localization perspective with ctrl-S and your change will be enacted

Here is a related example for a hazard type that has a list of durations to pick from. Suppose that it is desired to allow Flash Flood warnings of a shorter duration (this is often a convenient way to test the behavior of hazards as they approach their end time). Here are two possibilities for what the content of the override file could look like in that case:

Because HazardTypes.py is subject to incremental override, the default behavior for two lists at the same namespace is to merge the lists rather than to replace the list in the base file with the list in the site file. The first example override file tells the system to prepend  '15 min',  

‘30 min’, and  '45 min' to the list of durations, whereas the second example starts by stipulating that the supplied override list should replace the base list.

3.0.3.2 Completely removing an existing Hazard Type.     contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

In this section we describe how to completely remove an existing default hazard type from Hazard Services. We make the assumption that if one wants to remove a user defined hazard type from Hazard Services, one can simply undo the steps described in section 4.2 which were used to add that hazard type in the first place.

In the section entitled disabling the ability to issue a hazard type, we describe how to make a hazard type unissuable. Making a hazard type unissuable retains the ability to view hazards of that type from neighboring offices and the ability to manage that hazard type in the settings. The changes documented in this section result in eliminating all ability for Hazard Services to interoperate with hazards of that type. This means hazards of that type from neighboring offices cannot be viewed, nor can this hazard type be managed in the settings.

Here we will demonstrate specifically how to eliminate the Flood Advisory (FL.Y) hazard type.
We will start by eliminating it from HazardCategories.py. The approach is to create a site override for HazardCategories.py containing the following:

HazardCategories.py is subject to incremental override, and so there is no need to replicate the whole file, we just need to mark FL.Y for deletion from its category, "Hydrology". Were an override for HazardCategories.py already present, its new content would need to be a merge of the immediately preceding example and its current content. If an override file for HazardCategories.py had been used to reassign FL.Y to a different category, updating the override file would need to include removing FL.Y from its new category as well.

Hazard Service’s decisions about which hazard types are important to the console are completely driven by the content of your current setting, and the code that manages the settings files is smart enough to completely throw on the floor any hazard type that does not have a category.  Functionally removing a hazard type from Hazard Services does not require removing the hazard type from HazardTypes.py. In fact, attempting to do this would likely cause more problems than it would solve and therefore is not recommended. The same cannot be said, however, for any recommenders that might generate hazards of that type. Nor is this true for the logic that displays hazards.

For the sake of argument, let us stipulate (not true) that the RiverFloodRecommender recommends hazards only of type FL.Y and no other types. Therefore, all references to that recommender need to be removed from all our settings files. Plus, we want to remove all references to “FL.Y” from the settings files. These four commands allow one to easily identify any setting files for which this is an issue:

> touch /tmp/bbb

> cd /awips2/edex/data/utility/common_static

> find . -name '*.py' -path '*/settings/*' -exec grep 'RiverFloodRecommender' '{}' /tmp/bbb \;

> find . -name '*.py' -path '*/settings/*' -exec grep 'FL.Y' '{}' /tmp/bbb \;

For any base settings file identified where there is no corresponding site override, one needs to create a site override containing what follows. The italicized text is not literal, but rather is the name of the settings file minus the .py extension. Please note that while we are using the file system to identify those settings files that we need to change, creating and/or updating the site override file should be done in the localization perspective.

For any override settings file where there is no corresponding base file, one simply needs to remove any instance of "RiverFloodRecommender" from the "toolbarToolNames" , plus any occurrence of “FL.Y”. The same goes for any override settings file where there is an entry of “_override_replace_”: True, at the top level. Otherwise the modified override settings file needs to be a merge of the immediately preceding example and its current content.

As previously mentioned, it is not actually true that the RiverFloodRecommender recommends hazards only of type FL.Y. Therefore, if one were actually completely removing the hazard type FL.Y from Hazard Services, one needs to go into the recommender itself and disable its ability to interoperate with hazards of type FL.Y. For this particular recommender, this is detailed in section 3.3.3. For other recommenders that recommend multiple hazard types, the details would be different. It is also possible for a hazard type to have no associated recommender. In that case, none of the override settings files would need any entries for "toolbarToolNames".

3.0.4 FilterIcons.py    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

The purpose of FilterIcons.py is to allow icons to be placed on the left side of the console tool bar that indicate when hazard events meeting certain criteria have been filtered out. One can see the default version of FilterIcons.py in the localization perspective under Hazard Services -> Hazard Categories. What follows is a sample override for this file. This file is subject to incremental override, and so to eliminate the default entry the list of entries begins with '_override_replace_' . If there was a desire to only add new entries, those entries could be placed in the override file and the default entry would still be in effect as well. The image referred to with the 'normalIcon' element is what shows up when no events meeting the criteria have been filtered out.; the 'coloredIcon' element indicates the image shown when there ARE events being filtered out that meet the criteria.

3.0.5 Highway Mile Markers in Flash Flood Warnings    contents ⇑ ↑  ^   v  ↓ ⇓

Hazard Services by default supports the ability to list the mile markers of impacted highways in flash flood warnings. Hazard Services leverages much of the same scripting and data structures to do this that warnGen does. For both Hazard Services and warnGen, this functionality is dependent on creating a table in postgres for each route.   These tables appear in the mapdata schema of the maps database.  For warnGen, the name of each table that supports the mile marker functionality is placed in an xml file; for Hazard Services, each table needs to be inserted into an override for the file PointMarkerConfigs.py.  

Here is an example of an entry in PointMarkerConfigs.py that supports the mile marker functionality for one route:

Note that the content in blue is the structure within which the content for each route exists, the content in black is what is specific to the route for this example.  Also note that the site ‘oax’ is embedded in the table name entry. Unlike dam break or burn scar information, there is no formal segregation in postgres according to CWA for this information.  Therefore it is a good idea to always embed the CWA name that the route exists within as part of the table name.

For now lets assume the mile marker functionality is already working in warnGen, and has been implemented for all the markers along all the routes you care about at your site.  If this is the case, there should already be a postgres table for each route, and so one only need create an override for PointMarkerConfigs.py to leverage those.  A python script has been updated to do most of this work for you.  To invoke this script, become user awips on dx3, and run these commands:

> cd /awips2/edex/scripts/HazardServices/

> /awips2/python/bin/python parseWarngenTemplate.py -m

With the -m option, parseWarngenTemplate.py reads the files mileMarkers.vm and mileMarkers.xml from site/LLL/warngen/ and outputs an override for PointMarkerConfigs.py in the following directory (LLL is not literal, but is your primary site):

common_static/configured/LLL/HazardServices/python/events/productgen/geoSpatial/

You can run parseWarngenTemplate.py with no arguments to get usage information.  You can also supply a backup site id on the command line to run it for any arbitrary backup site. Note that if this conversion step fails to create a useable override of PointMarkerConfigs.py, it is always possible to compose these entries for yourself, following the example shown.  If for some reason you want to add mile markers for an additional route not yet implemented in your warnGen, that will also require composing one or more of these entries.

In the case where the mile marker functionality is already working in warnGen, all the postgres tables referred to by the version of PointMarkerConfigs.py created here should already be in postgres.  If mile markers for some or all of the routes are not showing up in FFW text, this means that some or all of the postgres tables are not installed.  If this is the case, here is what to do.  First, identify all the tables referred to in all your overrides of PointMarkerConfigs.py using these commands, best run as user awips on dx3:

> cd /awips2/edex/data/utility/common_static

> grep tableName */LLL/HazardServices/python/events/productgen/geoSpatial/PointMarkerConfigs.py

Next, identify all the tables in the mapdata schema of the maps database.  The command that follows will do this (leave off the -U awips -hdx1f in a dev environment):

> psql -U awips -hdx1f -d maps -c '\dt mapdata.*'

For any table referred to in an override of PointMarkerConfigs.py that is not in postgres, one either needs to remove all entries that refer to it in any overrides of PointMarkerConfigs.py, or add it to postgres.  To add such a table to postgres, one first needs a file that contains a list of mile markers for a single route in a very specific format.  What follows is an example of such a file.  The name of the file does not matter except that it helps keep this information organized in a meaningful way for the person managing this.

highway92_oax.id :

The script used to install this information as a postgres table is available only on dx1, and should be run as user awips. The full path to this script is: /awips2/database/sqlScripts/share/sql/maps/importMarkersInfo.sh

The importMarkersInfo.sh script is very particular about this format, so it is recommended to not deviate from it. If you have already successfully installed mile markers for warnGen, then you will likely already have a set of these ID files that you can reuse for creating mile marker postgres tables for Hazard Services if necessary. The first line in the ID file always contains the number of entries in the file. The ‘p’ and the ‘|1’ are literal and should always be placed in each line exactly as shown. The first and fifth whitespace delimited columns are sequence numbers and should always count up. Just before the vertical bar are the mile marker values; these are arbitrary strings. The latitude and longitude values are self explanatory.

Here is how this script is run to create a postgres table:

> importMarkersInfo.sh /path/to/markerfile.id mapdata tableNameInPointMarkerConfigs.py

mapdata is literal. This results in a new table being created in the mapdata schema of the maps database with the same name as the final argument. Unlike dam break or burn scar information, there is no formal segregation in postgres according to CWA for this information.  Therefore it is a good idea to always embed the CWA name that the route exists within as part of the table name.

The very first time importMarkersInfo.sh is called for a given table name, you might see a message like:
   NOTICE:  table "highway92_oax" does not exist, skipping
This seems bad, but does not necessarily mean the script failed. It may mean simply that it ‘skipped’ the step of deleting the table before reinstalling it.

3.0.6 Other arbitrary Markers in Flash Flood Warnings.    contents ⇑ ↑  ^   v  ↓ ⇓

Hazard Services implements the ability to mention when the location of other arbitrary markers occur within the bounds of a Flash Flood Warning. Here we will present an idealized example of this. It starts with a site override of PointMarkerConfigs.py:

With this in place, the Additional Locations: section of the Hazard Information Dialog for a Flash Flood Warning will have a checkbox labeled "Public Events" (e.g., the value of the “pointSpecificLabel" key). In order for this to have useful functionality, the importMarkersInfo.sh script must be used to create a new table in the mapdata schema of the maps database with the name "publicevents" (e.g. the value of the "tableName" key). Suppose the following invocation of importMarkersInfo.sh was used:
> importMarkersInfo.sh /tmp/publicEvents.id mapdata publicevents

Furthermore, suppose that the contents of publicEvents.id were the following:

Finally, suppose that one created an FFW where the warning box contained both of those latitude/longitude points and the "Public Events" checkbox was activated in the Hazard Information Dialog. The resulting text product would contain the following small paragraph:

This includes the following Public Events...

        Little League Finals and Regional Tractor Pull.

3.1 Recommenders    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

Please see this presentation to become familiar with the Recommender Framework. Also, please see the section on how to configure a Recommender Dialog using megawidgets.

3.1.1 Modifying an existing Recommender     contents ⇑ ↑  ^  v  ↓ ⇓

• Locate the Recommender you want to modify in the Hazard Services / Recommenders section of the Localization Perspective e.g. RiverFloodRecommender.py
• From the Browser, Click MB3 over the BASE version and select Create Override File-->Site.
• A skeleton Recommender class will appear and you can override methods from the BASE class. At the end of the RiverFloodRecommender class is a set of methods designated as typical overrides: “_getWarningThreshold”, “isWatch”, “isWarning”, “isAdvisory”. To change the warning threshold hour from 24 to 20, include the altered method in your override file:

• Save the file in the Localization perspective and your change will be enacted

3.1.2 Setting up a “Modify Call Back” Recommender    contents ⇑ ↑  ^  v  ↓ ⇓

Recommenders by design are allowed to set up a “callback” Recommender to be invoked automatically when a Hazard Event is modified. The line of python that follows is an excerpt from the recommender StormTrackTool.py. This designates that the recommender ModifyStormTrackTool.py should be called when events created by StormTrackTool.py are modfied.

                hazardEvent["modifyCallbackToolName"] = "ModifyStormTrackTool"

When the hazard geometry is modified by the user, the designated tool can be called to update the hazard event.

In actuality, the ability to specify in a recommender the name of a “callback” Recommender is broken. The name of the callback Recommender to use is currently hardcoded in the java for the one default recommender that uses a callback Recommender; specifically the Storm Track recommender. The indefinitely deferred Hazard Services ticket #10414 was written to address this problem.

3.1.3 Creating a new Recommender    contents ⇑ ↑  ^  v  ↓ ⇓

Please note that as of May 2018, the content of this section is not in a very useful state. It is being left in for now as a place holder.

To create a new Recommender, follow these steps:

• Create new Recommender file:

• Click MB3 over the the Recommenders menu item in the Hazard Services section of the Localization perspective and choose “New…”
• Enter the name of the new Recommender in the New Recommender dialog that appears and click OK
• A template Recommender class will appear in the editor where you can now write your code. For the following steps, it will be helpful to copy and paste methods from existing recommenders as a starting point. In particular, an existing recommender will give you an idea of how to construct a dialog, how to extract the input information in the execute method (eventSet, dialogMap, spatialInputMap), and how to construct Hazard Events that are returned to the system.

• Define meta information about the Recommender. The defineScriptMetadata method returns a dictionary which defines basic information about the recommender, such as author, script version, and description.
• Define Dialog. The “defineDialog” method creates a set of megawidgets for constructing an input dialog which will appear when the Recommender is run. The set of hazard events that will be input to the Recommender execute method is also passed to the “defineDialog” method so that your dialog can use the Hazard Event information if needed. See the megawidget section discussing the Recommender Dialog for more details.
• Extract Input Information. The “execute” method receives three arguments which contain information needed to run the Recommender:

• eventSet -- This is an object that contains “attributes” and a list of Hazard Events. To extract the information, you will do something like this:

        # Get the attributes from the EventSet as a Python dictionary

        sessionAttributes = eventSet.getAttributes()  

        # If an event has been sent into the Recommender, then it can be detected and

        # extracted as follows:

        # Pick up the existing event from what is passed in.

        layerEventId = spatialInputMap.get("eventID")

        events = eventSet.getEvents()

        haveEvent = False

        for event in events :

            if event.getEventID() == layerEventId :

                hazardEvent = event

                haveEvent = True

                break

Then the ‘haveEvent’ flag and resulting hazardEvent can be used accordingly.

• dialogInputMap -- This is a Python dictionary containing the fieldname: value user responses from the input dialog.

•  Also, in the case of the RiverFloodRecommender which can be invoked for a particular gage, the dialogInputMap will contain the selected pointID for the gage:

        selectedPointID = None

        try:

            selectedPointID = dialogInputMap.get(SELECTED_POINT_ID)

        except:

            pass        

        if selectedPointID is not None:

              inputMap.put(SELECTED_POINT_ID, selectedPointID)

• spatialInputMap -- besides holding an eventID for some cases noted above, the spatialInputMap may hold information about the “Drag to Hazard Location” and storm track to be used as the basis upon which to update the storm track.

• Write algorithm. Once you have extracted the input information, you can write your algorithm. There is a set of Utilities available to Recommenders (and Product Generators) in the Hazard Services Utilities section of Localization. Here is a set of common operations you may need in your Recommender:

• Access FFMP data through the DataAccessLayer

    def getQPEValues(self):

        '''        

        Strategy method for reading and accumulating data

        from preprocessed FFMP QPE datasets.

        '''

        request = DataAccessLayer.newDataRequest()

        request.setDatatype(GRID_KEY)

        request.setParameters(self._sourceName)

        availableTimes = DataAccessLayer.getAvailableTimes(request)

        # determine correct times

        latestTime = 0

        useTime = availableTimes[0]

        for time in availableTimes :

            tm = time.getRefTime().getTime()

            if tm > latestTime :

                latestTime = tm

           useTime = time

        usedTimes = []

        if latestTime:

            usedTimes.append(useTime)

            grids = DataAccessLayer.getGridData(request, usedTimes)        

            if grids:

                     self.dataGrid = grids[0]

                return True

        return False

Then you can access the dataGrid information as follows:

lonLats = self.dataGrid.getLatLonCoords()

        values = self.dataGrid.getRawData()

• Access GFE Grids through the DataAccessLayer

from ufpy.dataaccess import DataAccessLayer as DAL

self._site = "MFL"

weatherElement = "Wind"

req = DAL.newDataRequest("gfe", parameters = [weatherElement], siteId = self._site)

gridTimes = DAL.getAvailableTimes(req)

grids = DataAccessLayer.getGridData(req, gridTimes)

firstGrid = grids[0].getRawData()

We have found that attempting to fetch more than 20 grids at a time causes a memory error on the server side. Apparently there's some fixed size buffer limit that gets exceeded when you ask for more than 20 or so.

• Hazard Event geometries are illustrated in this example from the RiverFloodRecommender. Note that the Hazard Event “setGeometry” method takes a list of geometries. For example, in this case, we are setting the geometry to a point plus potentially multiple polygons.

import GeometryFactory

...

    def getFloodPolygonForRiverPointHazard(self, hazardEvent):

        """

        Returns a user-defined flood hazard polygon for

        a river forecast point flood hazard. The base version

        of this tool does nothing. It is up to the implementer

        to override this method.

        @param  hazardEvent: A hazard event corresponding to

                           a river forecast point flood

                           hazard

        """

        id = hazardEvent.get(POINT_ID)

        # Always create the point representing the

        # The river forecast point location.

        # Then check to determine if there

        # is an additional inundation map available.

        forecastPointDict = hazardEvent.get(FORECAST_POINT)

        point = forecastPointDict.get(POINT)

        coords = [float(point[0]), float(point[1])]

        pointGeometry = GeometryFactory.createPoint(coords)

        geometryList = [pointGeometry]

        if id in self.hazardPolygonDict:

            hazardPolygon = self.hazardPolygonDict[id]

            geometry = GeometryFactory.createPolygon(hazardPolygon)

            geometryList.append(geometry)     

        geometryCollection = GeometryFactory.createCollection(geometryList)

        hazardEvent.setGeometry(geometryCollection)

• Working with Hazard Event geometries. An experimental FLASHRecommender was written by Zac Flamig. The Recommender can be found here.

• Construct and return Hazard Events. The Recommender can return a single Hazard Event or an EventSet containing a list of hazard events. The objects returned must be Java-backed. Please see examples in the existing Recommenders for constructing and returning your resulting Hazard Events.

3.2 Hazard Metadata    contents ⇑ ↑  ^  v  ↓ ⇓

At the beginning of hazard metadata modules, generally a variable self.hazardEvent is set for the class; this contains a local working copy of the hazard event the metadata is being generated on behalf of. It is very common for metadata logic to inspect the contents of this data structure during the creation of the metadata. However, metadata logic should never change the contents of self.hazardEvent.

Another caution; if one is using the “refreshMetadata” feature to modify an existing megawidget that retains the same "fieldName" attribute, great care must be taken if one is also changing the "fieldType". In general this is not recommended.  More about this in section 3.2.7.

3.2.1 Examples of changes in Calls to Action    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

In this section we will present some customization examples for calls to action. When making modifications to calls to action, it is very important to be mindful of the distinction between the getCTA_Choices() method and the getCTAs() method. getCTA_Choices() is where the list of all calls to action that apply to a hazard is specified. getCTAs() packages those choices as a checkbox menu and controls which are activated by default.  Also be aware that the methods that create the megawidget element for each specific CTA are neither in CommonMetaData.py nor in any of the individual metadata modules; they are all in the module CallsToActionAndImpacts.py.

3.2.1.1 Change the wording of a Call to Action  contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

This example is to change the wording of the “Nightime flooding” call to action. First, one needs to find which python module this call to action is defined in. These commands will accomplish this (see Introduction for more information about searching code for functionality):

> cd /awips2/edex/data/utility/common_static/base/HazardServices

> find . -name '*.py' -exec grep -i 'Nighttime flooding' '{}' \; -print

In this case the file identified will be CallsToActionAndImpacts.py; the base version of all call to action methods are defined here. Search the module CallsToActionAndImpacts.py for the string “Nightime flooding”, and you will see that the method involved is ctaNightTime(). To implement the wording change for all hazard types, create an override file for CallsToActionAndImpacts.py with the following (change from base in green):

Now suppose there was a desire to change the wording of this call to action for only one hazard type; for sake of argument let’s say convective FFWs. Here is what this override would look like in this case:

What is happening here is that the traceback utility is used to determine whether this method is being invoked by the MetaData_FF_W_Convective.py metadata class, and updating the productString accordingly if this is the case.

3.2.1.2 Change list of Calls to Action for a Hazard    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

This customization example will be to change the the default call to action choices for a particular hazard type. Specifically, we will add the "Turn around...don't drown" call to action to the list available for the Areal Flood Watch. Use commands similar to those in the previous example to learn that the default version of this call to action method is implemented CallsToActionAndImpacts.py; inspect CallsToActionAndImpacts.py to learn that the method name is ctaTurnAround(). The specific metadata method for Areal Flood Watch is MetaData_FA_A.py. To make this change, you need to provide an override for the getCTA_Choices() method of MetaData_FA_A.py as follows (change from base in green):

3.2.1.3 Change Calls To Action activated by default    contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

This customization example will be to make the "Stay away" call to action be activated by default for an Areal Flood Watch. First, one needs to locate the call to action method in CallsToActionAndImpacts.py (ctaStayAway) and note the value of the identifier specified in that method, which is "stayAwayCTA". Then we need to create an override for the getCTAs() method of MetaData_FA_A.py. Note that the default instance of the getCTAs() method for MetaData_FA_A.py is actually in CommonMetaData.py. The content of this override follows (change to base instance in green):

Of course, if there were already a site version of MetaData_FA_A.py, the instance of getCTAs() shown would be added to that. The only code added to this is the code in green, which makes the CTA with the identifier of "stayAwayCTA" the one that is turned on by default for a new hazard. The two lines of code immediately preceding that allows an existing set of activated CTAs selected by the forecaster to be propagated to the next product in the life cycle.

3.2.1.4  Remove an existing default Call To Action  contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

In this example we remove any calls to action referring to arroyos from those available for convective Flash Flood warnings. For this example this will be approached differently from other examples where the set of available calls to action were changed; we will show how to implement this by overriding getCTAs(). Note that the base version we are overriding here is originally from CommonMetaData.py.

What follows is new site version of MetaData_FF_W_Convective.py for this purpose. The code added or changed is in green as before. The point here is not to necessarily recommend this as a superior approach to implementing this in getCTA_Choices(); rather to show that this is possible and to point out an advantage of this approach. The main advantage is that any future update of the getCTA_Choices() method in the base MetaData_FF_W_Convective.py will be far less likely to require a manual merge to both pull in any new CTAs but still filter out arroyo related CTAs. Note again that the default instance of getCTAs() for this hazard type is actually in CommonMetaData.py.

3.2.1.5 Add a new Call To Action  contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

In this final customization example related to calls to action, we show how to add a new call to action. First we need to add a new method for the content of the call to action. These methods always go into an override of CallsToActionAndImpacts.py:

To complete this, the new call to action content method needs to be added to the getCTA_Choices() method in the metadata file for the desired hazard type(s). Here is what such an override would look like added in MetaData_FA_A.py:

3.2.2 Changing default Hydrologic Cause     contents ⇑ ↑  ^  ─  +  v  ↓ ⇓

The purpose of this customization example is to change the default hydrologic cause for non-convective flash flood warnings to be “Ice Jam” in winter and “Snow Melt” otherwise. The file being overridden here is MetaData_FF_W_NonConvective.py (changes to base in green).

3.2.3 Adapt the metadata to the VTEC life cycle stage     contents ⇑ ↑  ^  v  ↓ ⇓

Note that as of July 2019, this is being reworked.  The new logic should make this all much less ambiguous.

At times there might be a desire to have the content of the metadata adapt to the stage in the VTEC life cycle (e.g. NEW, CON, etc.) the product being issued will have. As Hazard Services currently functions, it is not possible to do this directly; one has to deduce the VTEC life cycle stage the product will have from the current hazard status. Depending on which stages one cares about, this is often straightforward, but at times can be somewhat ambiguous.

Were it important to make the distinction between CON and EXT (continue and extension in time), one could in theory compare the end time of the hazard to the end time it previously had to resolve this ambiguity. However, when transitioning from issuing a CON to issuing an EXT, the metadata will never be reexecuted and so there will never be an opportunity to actually change the metadata in response to this.

There is some ability to make a distinction between CAN and EXP (cancellation and expiration). Were it important to make the distinction between CAN and EXP, one could compare the end time of the hazard to the current time. For short fused hazards like FFW, if one is more than 10 minutes from the end of the hazard, it is unambiguous that one was issuing a CAN; if less than 5 minutes it would be unambiguous that one was issuing an EXP.  However, in between there would be no way to resolve the ambiguity.

Issuing a correction adds yet another wrinkle to this.  One cannot distinguish between a correction to a NEW and a typical update to an existing hazard resulting in a CON.

Here we show a customization example where the required data structures are examined in order to switch to a code branch for the proper stage in the life cycle. Here we override the execute() method in MetaData_FF_W_Convective.py, and it is true that for most hazard types this would be the method to override for this type of customization. However, be aware that, for example, in MetaData_FF_W_NonConvective.py the buildMetaDataList() method would be the one to override for this purpose. So we point out here that one needs to be on the lookout for different approaches in the metadata for different hazard types. For this example, no actual new code differences are imposed in the new code branches. As described, keep in mind that in actuality one can never distinguish between a CON and an EXT, and that the ability to distinguish between CAN and EXT reliably is limited.

It is hoped that there may be some utility in showing how to access the data structures in question; some users may find other ways to leverage these data structures to do useful things.

3.2.4 Modifying the Product Staging Dialog     contents ⇑ ↑  ^  v  ↓ ⇓

The Product Staging Dialog is a dialog that sometimes comes up after choosing Preview but before the Product Editor appears. The purpose of this dialog is to allow the user to make choices that are for the product as a whole rather than for one particular hazard; this only occurs for hazard types where a single product can possibly contain text for multiple specific hazards.

Files that define metadata for a hazard type have the hazard phensig for the hazard type encoded into the metadata file name (e.g. MetaData_FA_W.py). Files that define metadata for a Product Staging Dialog have the NNN of the product(s) to generate encoded into the metadata file name (e.g. MetaData_FFA_FLW_FLS.py).

Here we have a customization example where all the synopsis options that refer to snow melt are commented out; one might do this if one was at a WFO in a place like Southern Florida. We also make it so the category increase synopsis results in a first guess for the language describing the rivers involved. This is done by overriding the methods execute(), synopsisCategoryIncrease(), and getSynopsisChoices() in MetaData_FFA_FLW_FLS.py (changes from base in green).

3.2.5 Change default search parameters for impacts section of flood warning products    contents ⇑ ↑  ^  v  ↓ ⇓
        
Here we describe how to change the default search parameters for the impacts sections of point flood warning products. First generate any point flood warning recommendation. Click on a recommendation in the Hazard Services console to pop the Hazard Information Dialog. Choose the Impacts Statement tab and then click on Impacts Search Parameters to expand that dialog. Now we want to pick something likely to be somewhat unique from that dialog to search for; for the purpose of this exercise we will pick 'Maximum Depth Below Flood Stage'. So now use the following commands to learn where this dialog is being constructed (see Introduction for more information about searching code for functionality):

> cd /scratch/albatross/awips2/edex/data/utility/common_static/base
> touch /tmp/bbb
> find . -name '*py' -exec grep 'Maximum Depth Below' '{}' /tmp/bbb \;

This tells us that HazardServices/hazardMetaData/CommonMetaData.py is where this dialog is constructed. Inspecting CommonMetaData.py, one learns that the method which constructs this dialog is getStageWindow(). So to change the default search parameters one needs to create a site override of this method. This could be in a site version of CommonMetaData.py, which would impact all hazard types using this dialog, or it could be in MetaData_FL_W.py, which would impact only this particular hazard type. For this exercise, we will implement this in MetaData_FL_W.py. What follows is the user override of MetaData_FL_W.py that implements this (changes from base in green; note that very little is a copy of the base).

There are several things going on here that are unique in comparison to other customization solutions shown in this document. Instead of completely replacing the existing method for obtaining the metadata, we directly call the method in the base class CommonMetaData to get an initial version of the metadata, and then modify that resulting metadata. This has the advantage that if a new base version of CommonMetaData::getStageWindow() appears in a future release, there is a very good chance that no manual merge will be required. We include in the method a simple way for someone to optionally track the behavior of this new version of getStageWindow() in a temporary file. This is set up so that if there is a runtime error in the method updateDefaultValues(), the traceback will also end up in that temporary file. This can be helpful because it then becomes possible for someone not working in eclipse to nonetheless debug the behavior of updateDefaultValues() method. Even if one is working in eclipse, it can sometime be very difficult to identify one traceback of interest from the 'firehose' of information that can appear in the eclipse console.

3.2.6 Change precision for additional rainfall selectors.    contents ⇑ ↑  ^  v  ↓ ⇓

In the Hazard Information Dialog for convective Flash Flood Warnings, there are selectors for specifying additional rain that is expected to fall. The current default behavior of those selectors is for them to only allow rainfall to even whole inches. This customization example explains how to change the precision allowed for these selectors. First it is necessary to identify the module where the default behavior for this is invoked. We use the labeling string for those selectors and search the metadata modules using these commands:

> cd /awips2/edex/data/utility/common_static/base/HazardServices/
> touch /tmp/bbb
> find . -name '*Meta*py' -exec grep -i 'additional rain' '{}' /tmp/bbb \;

This tells us that the default megawidget specification for this menu is in CommonMetaData.py. Inspecting the base version of CommonMetaData.py tells us that the method that puts together this megawidget specification is called additionalRain(). We could override this method for CommonMetaData.py, which would change this behavior for all product types. For this example we choose to implement an override for additionalRain() in MetaData_FF_W_Convective.py, so that the change only applies to that hazard type. What follows is the override for MetaData_FF_W_Convective.py, with changes in green:

With this override, we specify 2 digit precision, which means it is possible to specify rainfall values to the nearest 0.01 inches. This degree of precision, which is unrealistic for any forecast rainfall amount, is nonetheless required to enable the incrementDelta value of a quarter inch. One could make the argument that it makes no sense to increase the precision of the additional rainfall specifier without also doing the same for the ‘rain so far’ specifier; this is left as an exercise for the user.

3.2.7 Interdependency scripts.    contents ⇑ ↑  ^  ─  + v  ↓ ⇓

Interdependency scripts are a means by which a choice made by a user in the Hazard Information Dialog can result in automatically changing the state of other elements in the Hazard Information Dialog. While interdependency scripts are the most efficient way to accomplish this, they are not the only way. One can also define the "refreshMetadata" key to be True for a megawidget element, and this will result in the execute() method for the metadata to be rerun when that element is changed. (To see an example of invoking this refresh logic, examine the getHydrologicCause() method of MetaData_FF_W_NonConvective.py.) Also note that while interdependency scripts can result in changing the state of other elements, invoking the refresh logic is the only way for one choice to result in adding, removing, or changing the label of some other element.  Also, one does not have access to the hazard event inside an interdependency script.

Note that the method applyInterdependencies() is a static method of metadata modules. This particular static module method is overridable, but only because the logic that executes the metadata python is hardwired to do so. Therefore, be advised that it is not possible to introduce any additional static module methods in any of the metadata modules through override. Also be aware that many instances of applyInterdependencies() start with something similar to the following:

The exact name of the module from CommonMetaData may be different than applyInterdependencies.  The thing to keep in mind is if you have a similar construct in an override MetaData module, then at times one must put import CommonMetaData at the top of the override file.

Under the next two headings are examples of using an interdependency script. There is more information about interdependency scripts in the megawidget document.

3.2.7.1 Workaround for problems with floating point spinners.  contents ⇑ ↑  ^  ─  + v  ↓ ⇓

One common feature of the metadata for areal flood hazards is the use of floating point spinners to enter observed or expected rainfall amounts. Ideally these spinners would also allow the user to type in a number, but that feature has never worked well.  This customization example leverages an interdependency script to pair floating point spinners with a text entry field to at least obtain the desired functionality. Here we apply this to the Rain so far: entries for the convective FFW; it is left as an exercise for the reader to apply this to other situations if desired. Note that the extra 30 spaces added to the "inches of rain have fallen" label is to avoid annoying extra whitespace after the Text widgets.

MetaData_FF_W_Convective.py:

3.2.7.2 Changing other default state based on the Source selection.
            contents ⇑ ↑  ^  ─  + v  ↓ ⇓

This example focuses on the state of a checkbox labeled Flash Flooding occurring that appears in the Hazard Information Dialog for convective Flash Flood warnings. Here the goal is to change its default value depending on which value is selected for the set of radio buttons under Source:. If the user actually changes the state of the Flash Flooding occurring checkbox from the Hazard Information Dialog, thereafter the user’s choice for the checkbox will be honored and this defaulting logic will be disabled.

The override that follows for MetaData_FF_W_Convective.py contains an interdependency script that invokes this behavior:

3.2.7.3 Sharing methods between MetaData class and interdependency script.
            contents ⇑ ↑  ^  ─  + v  ↓ ⇓

There may be times when, in order to avoid duplicating code, it would be desirable to share methods between the code in a MetaData class and the code in an interdependency script. This is a bit tricky, because many of the typical classes used for broadly shared methods (such as TextProductCommon.py) are not importable by the MetaData modules. There is one medium weight class with no constructor overhead that is importable by MetaData modules, and that is CallsToActionAndImpacts.py.  Therefore, for now, this is probably as good a place as any to put modules shared between MetaData class code and interdependency script code.  At some point in the future we may try to formalize this.

Here will be one of the rare instances where we will demonstrate a point with an override that does nothing that is actually useful; this is just an example of how to structure the code such that a MetaData class and an interdependency script can share a method.  This override example will function, but will only output the file /tmp/testSharedMethod.txt to demonstrate the method sharing; it changes no actual behavior for the user.

CallsToActionAndImpacts.py:

MetaData_FA_Y.py:

3.2.8 Disclaimers that clarify what is editable.    contents ⇑ ↑  ^  ─  + v  ↓ ⇓

A Hazard Services ticket (#45995) has been written to address some confusing aspects of the appearance of the Impact Statement tab in the Hazard Information Dialog for River hazards. The problem is that the individual impact statements appear to be editable in the Hazard Information Dialog but are not. It may be some time before this ticket is completed, but in the mean time we present this customization that shows disclaimers that make it clear what is editable and what is not. The file being overridden is CommonMetaData.py. This customization is large because it overrides some methods with alot of code, but the changes themselves (in the customary green) are very simple.

3.2.9 Showing Type Source on the Hazard Information Dialog.    contents ⇑ ↑  ^  v  ↓ ⇓

This only applies to river flood hazards. Issuing a river flood hazard depends on having observed and forecast stage or flow data for a gauge. Most gauges have multiple potential sources of both observed and forecast stage and/or flow data. These are referred to as a “Type Source”, and by default these are always shown in the Graphical Time Editor.  This override allows this information to also be shown directly on the Hazard Information Dialog. This involves very simple overrides to four modules; CommonMetaData.py, MetaData_FL_A.py,   MetaData_FL_W.py, and  MetaData_FL_Y.py.  Here we only show the overrides to CommonMetaData.py and MetaData_FL_A.py, with changes from base in the customary green. The overrides for MetaData_FL_W.py and  MetaData_FL_Y.py are so completely analogous to the override for MetaData_FL_A.py that we leave those as an exercise for the reader.

CommonMetaData.py:

MetaData_FL_A.py:

3.2.10 Including gauge data in areal flood hazards.    contents ⇑ ↑  ^  v  ↓ ⇓

Hazard Services has functionality that allows one to include river gauge data in areal flood hazards.  Typically, river gauge data is only shown in river hazard products, so this functionality is deactivated by default.  However, it is very simple to activate through the following override of CommonMetaData.py:

This results in radio buttons labeled River Information and With Impacts being added in the Locations Affected section of the Hazard Information Dialog for areal flood hazards.  Activating the River Information radio button populates the list of Gauges to Include with any for which either the gauge itself or its reach polygon overlaps the area of the hazard. Activating the With Impacts radio button creates a hierarchical choice list below the Locations Affected section; this hierarchical choice list allows one to select gauges and impacts to mention for the gauge.

There are two other methods one can override that are binary switches controlling other aspects of this capability. These are hideMissingDataGauges() in CommonMetaData.py and mentionMissingDataGauges() in SectionLevelMethods.py.  It is left to the reader to read the comments on the Base level instances of these methods to learn what they do.