Chapter 4: Advanced topics. contents ⇑ ↑ ^ ─ + v ↓ ⇓
4.1. Adding new product formats. contents ⇑ ↑ ^ ─ + v ↓ ⇓
4.1.1 Adding a VTEC based product format. contents ⇑ ↑ ^ ─ + v ↓ ⇓
For now this is only a place holder.
4.1.2 Adding a non-VTEC based product format. contents ⇑ ↑ ^ v ↓ ⇓
For now this is only a place holder.
4.2 Adding a new Hazard Type contents ⇑ ↑ ^ v ↓ ⇓
Adding a new Hazard Type to the system is a concept that can have two different specific implications. At times, what one is actually doing is adding a new workflow for an existing product type; this is most often done by adding a Hazard subType. Adding a completely new Product Type is a more complex undertaking.
4.2.1 Adding a new subType. contents ⇑ ↑ ^ v ↓ ⇓
For now (Dec 2018) we have a convenient real world example to help demonstrate this concept. The Dam Break and Burn Scar recommenders create hazards with very detailed predefined boundaries. In Hazard Services there is logic that automatically removes small pieces of counties or zones from a hazard based on configurable parameters; those parameters are called inclusion thresholds. Ideally, Dam Break and Burn Scar hazards would not be subject to this functionality. For Dam Break and Burn Scar warnings this can already be true; they each are associated with a subType, and the inclusion thresholds for those subtypes can be set to zero. For watches this has not been done, which can result in Dam Break and Burn Scar watch hazards being subject to these thresholds when this is not desired.
Here we show a set of overrides that can be used to add a subtype specifically for Dam Break and Burn Scar watch hazards. None of these overrides are complicated, but there are several of them. Except for a one-line change to two Recommenders, they are all additions to various tables. Note that in most cases where this kind of thing is done, one would also add a new class for creating the hazard metadata. However, since here the ONLY thing we want to do differently with the new subType is to have it be associated with its own inclusion thresholds, we can reuse an existing metadata generation class.
First we officially add the new subType, FF.A.BurnDam, in HazardTypes.py:
HOURS = 3600000 MINUTES = 60000 inclusionThresholdWarningDefault = False OVERRIDE_LOCK = ['headline', 'combinableSegments', 'includeAll', 'allowAreaChange', 'allowTimeChange', 'expirationWindow', 'expirationSubWindow', True] HazardTypes = { 'FF.A.BurnDam' : {'headline': 'FLASH FLOOD WATCH', '_override_lock_': OVERRIDE_LOCK, 'combinableSegments': True, 'includeAll': True, 'allowAreaChange': True, 'allowTimeChange': True, 'expirationWindow': (-30, 30), 'endingExpirationDuration': 60 * MINUTES, 'endingExpirationRounding': 15 * MINUTES, 'hazardConflictList': ['FF.A', 'FA.A'], 'warngenHatching': False, 'ugcType': 'zone', 'ugcLabel': 'name', 'hazardClipArea' : 'cwa', 'inclusionAndOr': 'and', 'inclusionFraction': 0, 'inclusionAreaInSqKm' : 0, # MUST be fraction, NOT percent 'inclusionThresholdWarning': inclusionThresholdWarningDefault, 'replacedBy': ['FF.W.Convective', 'FF.W.NonConvective'], 'defaultDuration': 8 * HOURS, 'durationIncrement': 60, 'startDurationIncrement' : 60, 'accurateCities' : False, 'timeRangeWindowHrs' : 48, 'forceSegmentByDefault' : True, }, } |
Next we add that new subType to the Hydrology category; the prepend operation conveniently puts it right before the existing FF.A hazard type. This happens in HazardCategories.py:
import collections HazardCategories = collections.OrderedDict( { "Hydrology": ["_override_prepend_", ("FF", "A" , "BurnDam")], } ) |
Next we specify which generator to use for this subType; when adding a subType you will almost always use an existing generator. This happens in ProductGeneratorTable.py:
ProductGeneratorTable = { "FFA_ProductGenerator": { "allowedHazards": [ ('FF.A.BurnDam', "Flood"), ], }, } |
As mentioned, we will reuse the existing metadata generation class for FF.A hazards (MetaData_FF_A) for this new subType. This is done in HazardMetaData.py:
from HazardCategories import HazardCategories HazardMetaData =[ {"hazardTypes": [("FF", "A","BurnDam")], "classMetaData": "MetaData_FF_A"}, ] |
So that the Time to Expiration column in the console does not always say Overdue Product, we want to associate some alert parameters to this new subType. We add this subType to the list of hazard types associated with the existing alert parameters for FF.A hazards in HazardAlertsConfig.py:
HazardAlertsConfig = { "eventExpiration": { "configuration": [ "_override_by_key_description_", { "description": "Flash Flood Watch", "hazardTypes": ["FF.A.BurnDam"], }, ] } } |
Adding this subType to an existing setting can very easily be done using the Edit Default Settings GUI, but could also be done through an override. There are multiple default settings where one might want to do this; here we just show one example. The assumption here is that there are not already any overrides for this setting. In practice having existing overrides for default settings is very common, which means you would merge this change into the existing settings file override. The example we use here is Hydrology_All.py:
Hydrology_All = { "visibleTypes": [ "FF.A.BurnDam", ], } |
Finally, here is an override that makes a simple one-line change to the Dam Break recommender, which results in using the new subType instead of the existing FF.A hazard type. The change needed to the Burn Scar recommender (in BurnScarFloodRecommender.py) is exactly the same and so is left as an exercise to the reader.
DamBreakFloodRecommender.py:
class Recommender(RecommenderTemplate.Recommender): |
4.2.2 Adding a new a new Product Type. contents ⇑ ↑ ^ v ↓ ⇓
We have a real world example for demonstrating this concept. Namely, changing the Hydrologic Outlook such that it gets issued using an SPS product type instead of an ESF product type. In this example it is not actually necessary to add a new Hazard Type. However, will discuss some alternate scenarios around this change where that would be required and what would be done in that case.
The ESF product currently gets issued on behalf of the HY.O hazard type, and it is not necessary to change that. However, the SPS product is a zone product whereas the ESF is a county product. Therefore it is necessary to override some of the entries for HY.O in HazardTypes.py to reflect this. The left column shows the override that makes this change for HY.O. If for some reason it were necessary to update some of the attributes for HY.O that were locked, it would be necessary to create an entry for a whole new hazard type; the right column shows what this would look like on the assumption that the hazard type HY.Y was used for this purpose.
HazardTypes = { 'HY.O' : {'ugcType': 'zone', 'ugcLabel': 'name', }, } | HOURS = 3600000 MINUTES = 60000 inclusionThresholdWarningDefault = False HazardTypes = { 'HY.Y' : {'headline': 'HYDROLOGIC OUTLOOK', 'combinableSegments': False, 'allowAreaChange': True, 'allowTimeChange': True, 'expirationWindow': (-30, 30), 'endingExpirationDuration': 60 * MINUTES, 'endingExpirationRounding': 15 * MINUTES, 'hazardConflictList': [], 'warngenHatching': False, 'ugcType': 'zone', 'ugcLabel': 'name', 'inclusionAndOr' : 'and', 'inclusionFraction': 0, # MUST be fraction, NOT percent 'inclusionAreaInSqKm': 0, 'inclusionThresholdWarning': inclusionThresholdWarningDefault, 'hazardClipArea' : 'cwa', 'defaultDuration': 8 * HOURS, 'durationIncrement': 60, 'elapseWhenIssued': False, 'elapseWhenExpired': True, }, } |
The next thing needed is a generator for the SPS product type. We create a new file called SPS_ProductGenerator.py, which is mostly a copy of the existing ESF_ProductGenerator.py. It get supplied as an override in the same directory as ESF_ProductGenerator.py. Here we show SPS_ProductGenerator.py with the changes from ESF_ProductGenerator.py in the customary green. (Note that all changes save one are just swapping instances of the string ESF with SPS.):
import os, types, sys, collections from HydroProductParts import HydroProductParts import HydroGenerator class Product(HydroGenerator.Product):
def __init__(self): super(Product, self).__init__() # Used by the VTECEngineWrapper to access the productGeneratorTable self._productGeneratorName = 'SPS_ProductGenerator'
def defineScriptMetadata(self): metadata = collections.OrderedDict() metadata['author'] = "GSD/Raytheon" metadata['description'] = "Product generator for SPS." metadata['version'] = "1.0" return metadata
def defineDialog(self, eventSet): """ @return: dialog definition to solicit user input before running tool """ return {} def _initialize(self): super(Product, self)._initialize() # This is for the VTEC Engine self._productID = "SPS" self._productCategory = "SPS" self._areaName = '' # Number of hours past issuance time for expireTime # If -1, use the end time of the hazard self._purgeHours = 8.0 self._SPS_ProductName = 'Special Weather Statement' self._vtecProduct = False # Polygon-based, so locations listed will be limited to within the polygon rather than county area self._polygonBased = True def execute(self, eventSet, dialogInputMap): ''' Inputs: @param eventSet: a list of hazard events (hazardEvents) plus a map of additional variables @return productDicts, hazardEvents: Each execution of a generator can produce 1 or more products from the set of hazard events For each product, a productID and one dictionary is returned as input for the desired formatters. Also, returned is a set of hazard events, updated with product information. ''' self._initialize() self.logger.info("Start ProductGeneratorTemplate:execute SPS")
# Extract information for execution self._getVariables(eventSet) eventSetAttributes = eventSet.getAttributes()
productDicts, hazardEvents = self._makeProducts_FromHazardEvents(self._inputHazardEvents, eventSetAttributes) return productDicts, hazardEvents
def _groupSegments(self, segments): ''' Group the segments into the products SPS products are not segmented, so make a product from each 'segment' i.e. HY.O event ''' productSegmentGroups = [] for segment in segments: vtecRecords = self.getVtecRecords(segment) productSegmentGroups.append(self.createProductSegmentGroup('SPS', self._SPS_ProductName, 'area', self._vtecEngine, 'counties', False, [self.createProductSegment(segment, vtecRecords)])) return productSegmentGroups def _narrativeForecastInformation(self, segmentDict, productSegmentGroup, productSegment): default = ''' |* Headline defining the type of flooding being addressed (e.g., flash flooding, main stem river flooding, snow melt flooding) Area covered Possible timing of the event
Relevant factors (e.g., synoptic conditions, quantitative precipitation forecasts (QPF), or soil conditions)
Definition of an outlook (tailored to the specific situation)
A closing statement indicating when additional information will be provided. *| ''' productDict['narrativeForecastInformation'] = self._section.hazardEvent.get('narrativeForecastInformation', default) |
The other completely new file needed is a formatter for the SPS product type. The new file needed for this is Legacy_SPS_Formatter.py, which is also mostly a copy of Legacy_ESF_Formatter.py with instances of ESF swapped for SPS. We add the locationsAffected() method so that is is possible to also leverage the recently implemented capability that allows river gauge data to appear in areal hydro hazards.
from com.raytheon.uf.common.hazards.productgen import ProductUtils import NWS_Base_Formatter from HydroProductParts import HydroProductParts class Format(NWS_Base_Formatter.Format): def initialize(self, productDict): self.hydroProductParts = HydroProductParts() super(Format, self).initialize(productDict) def execute(self, productDict): self.initialize(productDict) legacyText = self._createTextProduct() legacyText = ProductUtils.wrapLegacy(legacyText) self.validateText(legacyText) return [legacyText] def determinePartsList(self): ''' Get the list of product parts for this specific product based on the product dictionary. ''' return self.hydroProductParts.productParts_SPS(self.productDict) ###################################################### # Product Part Methods ###################################################### def narrativeForecastInformation(self, sectionDict): narrative = self.processPartValue(sectionDict, 'narrativeForecastInformation', self.getNarrative, sectionDict) return self.getFormattedText(narrative, endText='\n') def getNarrative(self, sectionDict): narrative = '''|* Headline defining the type of flooding being addressed (e.g., flash flooding, main stem river flooding, snow melt flooding) Area covered
Possible timing of the event
Relevant factors (e.g., synoptic conditions, quantitative precipitation forecasts (QPF), or soil conditions)
Definition of an outlook (tailored to the specific situation)
A closing statement indicating when additional information will be provided. *|''' return narrative def locationsAffected(self, sectionDict): location = self.processPartValue(sectionDict, 'locationsAffected', self.sectionMethods.locationsAffected, sectionDict) return self.getFormattedText(location, startText='', endText='\n\n') |
Another difference is that SPS is a segmented product, as opposed to ESF which is a non-segmented product. Therefore new logic is needed in HydroProductParts.py for returning the correct set of product parts for this segmented product. The new methods are mostly a copy of existing methods for ESF; we show the changes from those in the customary green. One very important detail to get right is that the product part method called in Legacy_SPS_Formatter.py is the same as the new product part method (productParts_SPS) introduced here.
class HydroProductParts(object): ###################################################### # Product Parts for SPS ###################################################### def productParts_SPS(self, productDict): segmentParts = [] for segmentDict in productDict.get("segments", []): segmentParts.append(self.segmentParts_SPS(segmentDict)) partsList = [ 'wmoHeader', 'CR', 'productHeader', 'CR', ('segments', segmentParts), 'initials', ] return partsList def segmentParts_SPS(self, segmentDict): sectionParts = [] for section in segmentDict.get("sections", []): sectionParts.append(self.sectionParts_SPS(section)) partsList = [ 'setUp_segment', 'ugcHeader', "areaList", "cityList", "issuanceTimeDate", #'productHeader', 'CR', ('sections', sectionParts), 'endSegment', ] return partsList def sectionParts_SPS(self, sectionDict): partsList = [ 'setUp_section', 'narrativeForecastInformation', 'CR', 'locationsAffected', ] return partsList |
We also need an override of ProductGeneratorTable.py so that the new generator and formatter can be associated with the HY.O hazard type. Note that if for some reason it had been necessary to use the new HY.Y hazard type, the only difference would be that this override would have HY.Y where it currently has HY.O.
ProductGeneratorTable = { "ESF_ProductGenerator": "_override_remove_", "SPS_ProductGenerator": { "allowedHazards": [ ('HY.O', "Flood"), ], "previewFormatters": ["Legacy_SPS_Formatter"], "issueFormatters": ["Legacy_SPS_Formatter"], "correctAllSegments": False, }, } |
Finally, we update the metadata for HY.O so that the proper choices can be made to leverage the ability to include gauge data. The file overridden here is MetaData_HY_O.py. The methods riverInfoInFA() and getLocationsAffected() are actually implemented in CommonMetaData.py by default.
import CommonMetaData import HazardConstants class MetaData(CommonMetaData.MetaData): def execute(self, hazardEvent=None, metaDict=None): self.initialize(hazardEvent, metaDict) metaData = [ self.getLocationsAffected(), self.lidImpactsList(), ] return { HazardConstants.METADATA_KEY: metaData }
def riverInfoInFA(self): return True def getLocationsAffected(self, pathcastDefault=True): eventValue = self.hazardEvent.get("locationsAffectedRadioButton") if eventValue in ["riverInfo", "withImpacts" ] : self.getGaugesInArea(eventValue=="withImpacts") else : self.lidChoices = ["None"] self.lidValues = [] self.impactHierarchy = {} self.cachedStageData = {} defidx = 0 choices = [self.noLocations(), self.riverInfo(), self.withImpacts()] defValue = choices[defidx].get("identifier") if eventValue : for choice in choices : if choice.get("identifier")==eventValue : defValue = eventValue break radioMW = { "fieldType":"RadioButtons", "fieldName": "locationsAffectedRadioButton", "label": "Locations Affected (4th Bullet)", "choices": choices, "values": defValue, "expandHorizontally": False, "expandVertically": False, "refreshMetadata" : self.riverInfoInFA(), } if not self.riverInfoInFA() : return radioMW refreshButton = { "fieldType": "Button", "fieldName": "refreshGaugeData", "label" : "Refresh Gauges" , "refreshMetadata" : True } spacerWidget = {"fieldType":"Label", "fieldName":"placeHolder123", "values":60*" "} if self.impactHierarchy : afterRadio = refreshButton else : afterRadio = self.lidChoiceList() return { "fieldType": "Composite", "fieldName": "locationsAffectedComposite", "numColumns" : 3, "fields" : [radioMW, afterRadio, spacerWidget, self.hiddenStageData() ] } |
This completes everything necessary to change the hazard type HY.O from being issued in an ESF product to being issued in an SPS product. In the hypothetical case where is was necessary to use the new hazard type HY.Y, a different metadata class would be needed, and three additional overrides would also be needed. The new metadata class would be called MetaData_HY_Y.py and would be created with exactly the same content as and installed in the same directory as the MetaData_HY_O.py shown. In order to connect that new metadata class to the HY.Y hazard, the following override would be needed for HazardMetaData.py:
from HazardCategories import HazardCategories HazardMetaData =[ {"hazardTypes": [("HY", "Y")], "classMetaData": "MetaData_HY_Y"}, ] |
Also, this new hazard type would need to be assigned a category by way of an override to HazardCategories.py:
import collections HazardCategories = collections.OrderedDict( "Hydrology": [ ("HY", "Y")], ) |
To finish this up, the new hazard type would be added to a setting. This can very easily be done using the Edit Default Settings GUI, but could also be done through an override. There are multiple default settings where one might want to do this; here we just show one example. The assumption here is that there are not already any overrides for this setting. In practice having existing overrides for default settings is very common, which means you would merge this change into the existing settings file override. The example we use here is Hydrology_All.py:
Hydrology_All = { "visibleTypes": [ "HY.Y", ], } |
4.3. Adding a new Hydrologic Cause. contents ⇑ ↑ ^ v ↓ ⇓
The approach here is a bit different than previous sections. Here we describe some changes that have already been made in order to demonstrate a concept. We will present a fairly substantial series of changes as if they were overrides. Of course, to actually implement this as a baseline change to Hazard Services, whole modified files had to be submitted to the main code set. None of these changes are complicated. However, one should note that logic needed to implement this change involved 11 different methods in 7 different python files. If one wanted to implement their own addition of a hydrologic cause via overrides, it would be very challenging (but not impossible) to get all the details right. Hence, the rationale for placing this in the Advanced Topics chapter.
Up until 17.3.1 there was only one hydrologic cause available for ice jams. It turns out that ice jams have two distinct phases. One phase is when the ice jam first forms and the main thing that is happening is that water is backing up behind the ice jam. Another phase is when the ice jam breaks and the impounded water is rapidly released. The break phase of an ice jam has much in common with a dam break; the impounding phase not so much. We decided to deal with this by introducing separate hydrologic causes for the impounding phase and the break phase of an ice jam (they both still have IJ as the immediate cause).
It was decided to have the existing ice jam related hydrologic cause become cause for the impounding phase and therefore to introduce a new hydrologic cause for the break phase. This decision was arbitrary and this could have been approached a different way. The existing designator for the ice jam hydrologic cause, ‘icejam’ therefore became the designator for the impounding phase. A new hydrologic cause designator, ‘ijbreak’, was introduced for the break phase.
HydroGenerator.py:
class Product(Legacy_Base_Generator.Product): |
AttributionFirstBulletText.py:
class AttributionFirstBulletText(object): |
BasisText.py:
class BasisText(object): |
FFW_FFS_ProductGenerator.py:
class Product(HydroGenerator.Product): |
IBW_Text.py:
class IBW_Text(object): |
SectionLevelMethods.py:
class SectionLevelMethods(object): |
MetaData_FF_W_NonConvective.py:
class MetaData(CommonMetaData.MetaData): |
Finally, the base version of DamMetaData.py was changed, but only to update the comments. What follows is the part of the comments that had a change directly related to this:
Another very important exception is "hydrologicCause" element, which results in a different default initial selection for that selector on the Hazard Information Dialog. The "hydrologicCause" is important because for non-convective FFWs, it functions as a sub-subtype. Current meaningful values for this are 'levee', 'floodgate', 'icejam', 'ijbreak', 'glacier', 'volcano', and 'volcanoLahar'. If not present things default to being for a dam break. |
4.4. The firstBullet and attribution product parts. contents ⇑ ↑ ↓ ⇓
The firstBullet and attribution product parts do not follow the typical paradigm for the structure of the product parts code. The instance of these product part methods found in the formatter works just like the formatter instance for other parts. However, the content instances of firstBullet and attribution found in SectionLevelMethods.py have a much more complicated underlying code structure.
Formatters create and hold an instance of class that is derived from the base class AttributionFirstBulletText.py. The specific derived class created varies from formatter to formatter. As this is being written, the available derived classes are:
AttributionFirstBulletText_Basic.py AttributionFirstBulletText_FFA.py
AttributionFirstBulletText_FFW_FFS.py AttributionFirstBulletText_FLW_FLS.py
You can tell which derived class a given formatter is using by looking at the top of the formatter class and checking which of these derived classes is imported. In AttributionFirstBulletText.py there are two methods that farm out the content generation to a method in the derived class; which method is determined by the VTEC mode of the section being generated. What follows is the code for these two methods:
def getAttributionText(self):
if self.action == 'CAN':
attribution = self.attribution_CAN()
elif self.action == 'EXP':
attribution = self.attribution_EXP()
elif self.action == 'UPG':
attribution = self.attribution_UPG()
elif self.action == 'NEW':
attribution = self.attribution_NEW()
elif self.action == 'EXB':
attribution = self.attribution_EXB()
elif self.action == 'EXA':
attribution = self.attribution_EXA()
elif self.action == 'EXT':
attribution = self.attribution_EXT()
elif self.action == 'CON':
attribution = self.attribution_CON()
elif self.action == 'ROU':
attribution = self.attribution_ROU()
return attribution
def getFirstBulletText(self):
if self.action == 'CAN':
firstBullet = self.firstBullet_CAN()
elif self.action == 'EXP':
firstBullet = self.firstBullet_EXP()
elif self.action == 'UPG':
firstBullet = self.firstBullet_UPG()
elif self.action == 'NEW':
firstBullet = self.firstBullet_NEW()
elif self.action == 'EXB':
firstBullet = self.firstBullet_EXB()
elif self.action == 'EXA':
firstBullet = self.firstBullet_EXA()
elif self.action == 'EXT':
firstBullet = self.firstBullet_EXT()
elif self.action == 'CON':
firstBullet = self.firstBullet_CON()
elif self.action == 'ROU':
firstBullet = self.firstBullet_ROU()
return firstBullet
If for example you knew you wanted to change the content of the firstBullet product part only for a continuance (VTEC mode CON) of a river flood warning (FL.W), you would override only the firstBullet_CON() method of AttributionFirstBulletText_FLW_FLS.py. The member variables self.phenSig and self.subType are always available in an AttributionFirstBulletText instance, and so it would be straightforward to only apply your logic change to the FL.W hazard.
In practice, the scope of these kinds of changes is usually not so narrow. So one approach would be to replicate a change in overrides to several different VTEC mode specific methods. Another approach would be to directly override one of the methods shown, applying a modification to the generated text just before returning the string in the attribution or firstBullet variable. The _init_ method in AttributionFirstBulletText.py caches just about anything you would need to test whether the specific situation that your modification applies to is in effect. Which approach to take of course depends on the details of the change to the text you wish to make.
4.5 Running Hazard Services when Cave is running for a backup site. contents ⇑ ↑ ↓ ⇓
In theory, configuring your Hazard Services to support a backup site should be as simple as gathering all the files from the backup site’s A-II system from under these paths:
/awips2/edex/data/utility/common_static/site/BBB/HazardServices/
/awips2/edex/data/utility/common_static/configured/BBB/HazardServices/
/awips2/edex/data/utility/edex_static/site/BBB/shapefiles/hazardServices/
and staging those files at the same paths on your system. (BBB is not literal, but is the primary site id of some arbitrary backup site.) We assume that if you are backing up a site, you would have already also configured your GFE to backup that site. See section 1.2.6 for more on this; pay particular attention to what is said about having copies of SiteCFG.py, DefaultCityLocation.py, and DefaultAreaDictionary.py installed in backup site directories.
Consider a scenario where taking these steps results in still having some suboptimal behavior that occurs when in service backup. Sometimes such behavior could be fixed by modifying the set of site override files for the backup site. Ideally, one would want to do this using the localization perspective. If such a fix involves tweaking an existing site override for the backup site, then this is straightforward. At the top of the file browser section of the localization perspective, there is a little white downward pointing triangle. If one selects that triangle and picks Show All -> Sites, then it becomes possible to view and edit any existing site override file for your backup site.
However, consider the scenario (probably very rare, but see section 1.1.2 for a counter example) where fixing such suboptimal behavior requires creating a new override file for the backup site. That cannot be done using the localization perspective when Cave is running on behalf of your primary site; this requires running Cave on behalf of your backup site. If you have the proper files in place to allow you to use service backup for that site in Hazard Services, for the most part that should be sufficient to allow Hazard Services to function when running all of Cave for the backup site.
We say for the most part, because when running Cave on behalf of a backup site, files need to be installed in one more directory, or text products will not store properly in practice mode. That directory is:
/awips2/edex/data/utility/common_static/configured/BBB/textdb/config
You can just copy all the files from your primary site’s instance of that directory.
Now if you run Cave itself on behalf of your backup site, you should be able to test Hazard Services for that site AND make any necessary modifications to the set of Site overrides using the localization perspective, including adding new overrides. Note that when Cave itself is being run on behalf of the backup site, issuing products for the backup site will result in the formatting of the MND header not being consistent with backup operations. That is, the MND header will not contain your “ISSUED BY…” line. If one was trying to make a fix to that particular aspect of Hazard Services, then a different approach would be needed. The logic that handles adding the “ISSUED BY…” line for service backup is simple and mature so this weird scenario is unlikely.
One of the upshots of this behavior is that one cannot operationally perform service backup with Hazard Services by running an entire Cave as the backup site; at least not with the default software. However, one can modify this behavior by creating a site override of NWS_Base_Formatter.py for the backup site with the following content (again, LLL is not literal, but is the localization id of your primary site):
class Format(FormatTemplate.Formatter): def setSiteInfo(self): # Primary Site siteEntry = self._siteInfo.get(self._siteID) self._fullStationID = siteEntry.get('fullStationID') # KBOU self._region = siteEntry.get('region') self._wfoCity = siteEntry.get('wfoCity') self._wfoCityState = siteEntry.get('wfoCityState') self._areaName = '' # siteEntry.get('state') # 'GEORGIA' if (self._siteID!="LLL") : self._backupSiteID = "LLL" # Backup Site siteEntry = self._siteInfo.get(self._backupSiteID) self._backupWfoCityState = siteEntry.get('wfoCityState') self._backupFullStationID = siteEntry.get('fullStationID') |
4.6. Allowing Hazard Services to issue products for Sites AER and ALU. contents ⇑ ↑ ↓ ⇓
In order for Hazard Services to issue products for sites AER and ALU, it is assumed that things are already configured such that your GFE is also enabled to do this. For WFOs this is for the most part a given. However, testbeds, regional headquarters, RFCs, national centers, and developers, all of whom may only infrequently (if ever) use GFE, may nonetheless want to test this capability in Hazard Services. To be certain this is enabled, run the script that follows. If lacking these definitions is a problem for using sites AER and ALU, the primary symptom is that no matter which area you select, it is always identified as being “outside of the CWA”.
ak_dual_fix.csh:
#!/bin/csh -f set psqlcmd = ( psql -U awips -d maps -hdx1 -a -c ) if ( -e /awips2/edex/bin/setup.env ) then grep localhost /awips2/edex/bin/setup.env >& /dev/null if ( $status == 0 ) then set psqlcmd = ( psql -d maps -a -c ) endif grep -v '^#' /awips2/edex/bin/setup.env | grep dx >& /dev/null if ( $status != 0 ) then set psqlcmd = ( psql -d maps -a -c ) endif endif # set needed_entries = ( \ "AK135" "AFCAER" "AK171" "AFCAER" "AK125" "AFCAER" "AK181" "AFCALU" \ "AK161" "AFCALU" "AK187" "AFCALU" "AK101" "AFCAER" "AK111" "AFCAER" \ "AK131" "AFCAER" "AK141" "AFCAER" "AK145" "AFCAER" "AK121" "AFCAER" \ "AK195" "AFCALU" "AK152" "AFCALU" "AK155" "AFCALU" "AK185" "AFCALU" \ "AK191" "AFCALU" ) # while ( $#needed_entries > 0 ) set stzone = $needed_entries[1] shift needed_entries set cwa = $needed_entries[1] shift needed_entries $psqlcmd "update mapdata.zone set cwa='$cwa' where state_zone='$stzone' ;" end # |
The AER and ALU sites are only useable in the context of Hazard Services if an entire Cave is run on behalf of those sites. One cannot use Hazard Services service backup for this. See the previous section for how to enable Hazard Services to work if Cave is not running on behalf of your primary site. In addition, one needs the following overrides to be in effect for sites AER, ALU, and AFC. At one time there was some discussion about which localization level is best to implement these overrides at, for now we have settled on using the configured level.
For this scenario, only sites AFG and AJK, if performing backup for AFC, would need to worry about the “ISSUED BY…” line. Note that these overrides only implement a very minimal functionality that allows areal flood watches to be issued on behalf of sites AER and ALU, such that GHG interoperability with work for these hazards. For an operational site, there are a great deal more overrides that will be desired to make the workflows among sites AFC, AER, and ALU be as intuitive as practical.
HazardTypes.py:
HOURS = 3600000 MINUTES = 60000 inclusionThresholdWarningDefault = False OVERRIDE_LOCK = ['headline', 'combinableSegments', 'includeAll', 'allowAreaChange', 'allowTimeChange', 'expirationWindow', 'expirationSubWindow', True] HazardTypes = { # Use these commented out entries only for AFC #'FA.A' : {'_override_lock_': ['ugcType', 'ugcLabel'], # 'ugcType': '', # 'ugcLabel': '', # }, #'FF.A' : {'_override_lock_': ['ugcType', 'ugcLabel'], # 'ugcType': '', # 'ugcLabel': '', #}, #'FA.A.FlashFlood' : {'_override_lock_': ['ugcType', 'ugcLabel'], # 'ugcType': '', # 'ugcLabel': '', #}, 'FA.W' : {'ugcType': 'zone', 'ugcLabel': 'name', }, 'FA.Y' : {'ugcType': 'zone', 'ugcLabel': 'name', }, 'FF.W.Convective' : {'ugcType': 'zone', 'ugcLabel': 'name', }, 'FF.W.NonConvective' : {'ugcType': 'zone', 'ugcLabel': 'name', }, 'FF.W.BurnScar' : {'ugcType': 'zone', 'ugcLabel': 'name', }, } |
Legacy_FFA_Formatter.py, IBW_FFW_FFS_Formatter.py, Legacy_FLW_FLS_Formatter.py:
class Format(NWS_Base_Formatter.Format): def wmoHeader(self, productDict): header = self.processPartValue(productDict.get('productBeginDict', {}), 'wmoHeader', self.plm.wmoHeaderAFC, productDict, True) return self.getFormattedText(header, endText ='\n') |
ProductLevelMethods.py:
class ProductLevelMethods(object): def wmoHeaderAFC(self, productDict): header = self.getWmoId(self.fc._wfo) header += self.fc._productID + self.fc._wfo return header |
Legacy_Base_Generator.py:
class Product(ProductTemplate.Product): def limitGeoZones(self, hazardEvents): ''' Used by the vtecEngine to only run for a selected set of zones. This is needed to support Alaska's dual domain configuration. ''' limitGeoZones = None if hazardEvents: # Only want to limit area based products if hazardEvents[0].get('geoType') == 'area': limitGeoZones = [] for ugc in self._areaDictionary: entry = self._areaDictionary.get(ugc) if isinstance(entry,dict) and "wfo" in entry: wfo = entry.get("wfo") if wfo: # Could be multiple wfos, split every 3 characters wfoList = [wfo[i:i+3] for i in range(0, len(wfo), 3)] if self._siteID in wfoList: limitGeoZones.append(ugc) return limitGeoZones |
Appendix 1: Example Partner XML format contents ⇑ ↑ ↓ ⇓
Removed from baseline
<product> |
Appendix 2: Example Product Generator Output Dictionary contents ⇑ ↑ ↓ ⇓
{ "backupSiteID": "OAX", "correction": false, "generatorName": "FFA_ProductGenerator", "issueFlag": false, "issueTime": 1541793015960, "previousProductLabel": "FFA_area", "productBeginDict": { "overviewSynopsis": "", "productHeader": "Flood Watch\nNational Weather Service Omaha/Valley NE\n150 PM CST Fri Nov 9 2018\n", "wmoHeader": "WGUS63 KOAX 091950\nFFAOAX" }, "productCategory": "FFA", "productEndDict": { "initials": "awips" }, "productID": "FFA", "productLabel": "FFA_area", "productName": "Flood Watch", "purgeHours": 8.0, "runMode": "Operational", "segments": [ { "cityList": [ "Pierce", "Plainview", "Osmond", "Wayne", "Norfolk", "Stanton", "West Point", "Wisner" ], "expirationTime": 1541822400000, "sections": [ { "attribution": "The Flash Flood Watch continues for", "basisBullet": "current hydrometeorological basis.", "callsToAction": "", "callsToAction_sectionLevel": "", "eventDicts": [ { "cityList": [ "Pierce", "Plainview", "Osmond", "Wayne", "Norfolk", "Stanton", "West Point", "Wisner" ], "creationTime": "2018-11-09 19:39:25", "endLowerBoundary": 0, "endTime": "2018-11-10 04:00:00", "endUpperBoundary": 4611686018427387903, "endingSynopsis": null, "etns": [ 25 ], "eventID": "HZ-2018-OAX-OAX-1090", "forceSeg": "1090", "geoType": "area", "geometry": "GEOMETRYCOLLECTION (POLYGON ((-97.64887909523212 42.37097790442211, -96.82320141577584 42.17781256183633, -96.82329559299995 42.17031097400007, -96.82319641099997 42.16321182300004, -96.82329559299995 42.16081237800005, -96.82329559299995 42.13361358600007, -96.82349395799997 42.12771225000006, -96.82349395799997 42.12411117600004, -96.82359313999996 42.11961364700005, -96.82369995099998 42.11291122400007, -96.82379913299997 42.10491180400004, -96.82419586099996 42.09041214700005, -96.80489349399994 42.09041214100006, -96.78749847499994 42.09031295800003, -96.78450012199994 42.09041214000006, -96.78269275865152 42.09040100987558, -96.69925064341432 41.74231338500005, -96.70909881599992 41.74231338500005, -96.71360015999994 41.74221038800005, -96.72049713099995 41.74221038800005, -96.72869872999996 41.74231338500005, -96.73329925499996 41.74231338500005, -96.73819732699997 41.74241256700003, -96.77349853499999 41.74241256700003, -96.77989959699994 41.74251174900007, -96.81939697299998 41.74251174900007, -96.82619476299999 41.74261093100006, -96.84569549599995 41.74261093100006, -96.84869384799998 41.74251174900007, -96.85119628999996 41.74261093100006, -96.85529327399996 41.74261093100006, -96.86629486099996 41.74251174900007, -96.87629699699994 41.74261093100006, -96.89069366499994 41.74261093100006, -96.89709472699995 41.74271392800006, -96.90139770499997 41.74271392800006, -96.90519714399994 41.74261093100006, -96.91320037799994 41.74271392800006, -96.91629791299994 41.74261093100006, -96.92440032999995 41.74271392800006, -96.95449829099992 41.74271392800006, -96.95749664299996 41.74281311000004, -96.97570037799994 41.74281311000004, -96.98139953599997 41.74291229200003, -97.01309966999997 41.74291229200003, -97.01749420199997 41.74301147500006, -97.06169891399998 41.74301147500006, -97.06409347700743 41.74302596047249, -97.7381503068238 41.82971382278279, -97.64887909523212 42.37097790442211)))", "headline": "FLASH FLOOD WATCH", "immediateCause": "ER", "impacts": "", "impactsStringForStageFlowTextArea": null, "issuedEndTime": 1541822400000, "issuedStartTime": 1541792365879, "locationDicts": [ { "entityName": "Pierce", "fullStateName": "Nebraska", "typePlural": "zones", "typeSingular": "zone", "ugc": "NEZ017", "ugcPartsOfState": null, "ugcPortions": "set([])" }, { "entityName": "Wayne", "fullStateName": "Nebraska", "typePlural": "zones", "typeSingular": "zone", "ugc": "NEZ018", "ugcPartsOfState": null, "ugcPortions": "set([])" }, { "entityName": "Madison", "fullStateName": "Nebraska", "typePlural": "zones", "typeSingular": "zone", "ugc": "NEZ031", "ugcPartsOfState": null, "ugcPortions": "set([])" }, { "entityName": "Stanton", "fullStateName": "Nebraska", "typePlural": "zones", "typeSingular": "zone", "ugc": "NEZ032", "ugcPartsOfState": null, "ugcPortions": "set([])" }, { "entityName": "Cuming", "fullStateName": "Nebraska", "typePlural": "zones", "typeSingular": "zone", "ugc": "NEZ033", "ugcPartsOfState": null, "ugcPortions": "set([])" } ], "locationsAffected": [ "Norfolk", "Wayne", "West Point", "Madison", "Pierce", "Stanton", "Battle Creek", "Wisner", "Osmond", "Beemer", "Winside", "Pilger", "Hadar", "Hoskins", "Foster", "8 Miles South Of Stanton", "8 Miles South Of Wayne", "The Highway 15 And 32 Junction", "Willow Creek State Recreation Area", "10 Miles West Of West Point" ], "phen": "FF", "practice": false, "previousFloodSeverity": "0", "productLabel": "FFA_area", "replacedBy": null, "replaces": null, "shapeType": "polygon", "sig": "A", "siteID": "OAX", "siteID4": "KOAX", "startLowerBoundary": 0, "startTime": "2018-11-09 19:39:25", "startUpperBoundary": 4611686018427387903, "subType": null, "timeZones": [ "CST6CDT" ], "ugcs": "set(['NEZ032', 'NEZ033', 'NEZ017', 'NEZ031', 'NEZ018'])", "userName": "awips", "vtecMode": "O" } ], "firstBullet": "a portion of northeast Nebraska, including the following areas, Cuming, Madison, Pierce, Stanton and Wayne.", "impactsBullet": "current hydrometeorological impacts.", "timeBullet": "* Until 10 PM CST this evening", "vtecRecord": { "act": "CON", "endTime": 1541822400000, "etn": 25, "eventID": "set(['HZ-2018-OAX-OAX-1090'])", "forceVTEC": null, "hdln": "FLASH FLOOD WATCH", "hvtec": { "crest": 2147483640, "fallBelow": 2147483640, "floodCategoryForecast": null, "floodCategoryObserved": null, "floodRecord": null, "floodSeverity": "0", "immediateCause": "ER", "pointID": null, "prevFloodCategoryObserved": null, "riseAbove": 2147483640 }, "hvtecstr": "/00000.0.ER.000000T0000Z.000000T0000Z.000000T0000Z.OO/", "id": "set(['NEZ032', 'NEZ033', 'NEZ017', 'NEZ031', 'NEZ018'])", "issueTime": 1541792379000, "key": "FF.A", "officeid": "KOAX", "phen": "FF", "phensig": "FF.A", "pil": "FFA", "seg": 1090, "sig": "A", "startTime": 1541793000000, "status": "ISSUED", "subtype": "", "ufn": 0, "vtecstr": "/O.CON.KOAX.FF.A.0025.000000T0000Z-181110T0400Z/" } } ], "segmentBeginDict": { "areaList": "Pierce-Wayne-Madison-Stanton-Cuming-", "cityListText": "Including the cities of West Point, Pierce, Stanton, Osmond, Wayne, Plainview, Wisner, and Norfolk", "issuanceTimeDate": "150 PM CST Fri Nov 9 2018\n", "setUp_segment": "", "summaryHeadlines": "...FLASH FLOOD WATCH REMAINS IN EFFECT UNTIL 10 PM CST THIS EVENING...", "ugcHeader": "NEZ017-018-031>033-100400-", "vtecString": "/O.CON.KOAX.FF.A.0025.000000T0000Z-181110T0400Z/\n/00000.0.ER.000000T0000Z.000000T0000Z.000000T0000Z.OO/\n" }, "segmentEndDict": {}, "timeZones": [ "CST6CDT" ], "ugcs": [ "NEZ017", "NEZ018", "NEZ031", "NEZ032", "NEZ033" ], "vtecRecords": [ { "act": "CON", "endTime": 1541822400000, "etn": 25, "eventID": "set(['HZ-2018-OAX-OAX-1090'])", "forceVTEC": null, "hdln": "FLASH FLOOD WATCH", "hvtec": { "crest": 2147483640, "fallBelow": 2147483640, "floodCategoryForecast": null, "floodCategoryObserved": null, "floodRecord": null, "floodSeverity": "0", "immediateCause": "ER", "pointID": null, "prevFloodCategoryObserved": null, "riseAbove": 2147483640 }, "hvtecstr": "/00000.0.ER.000000T0000Z.000000T0000Z.000000T0000Z.OO/", "id": "set(['NEZ032', 'NEZ033', 'NEZ017', 'NEZ031', 'NEZ018'])", "issueTime": 1541792379000, "key": "FF.A", "officeid": "KOAX", "phen": "FF", "phensig": "FF.A", "pil": "FFA", "seg": 1090, "sig": "A", "startTime": 1541793000000, "status": "ISSUED", "subtype": "", "ufn": 0, "vtecstr": "/O.CON.KOAX.FF.A.0025.000000T0000Z-181110T0400Z/" } ] } ], "siteID": "OAX", "wfo": "OAX" } |
It is very easy to get access to this kind of output; just make the following override for NWS_Base_Formatter.py:
import json from jsonCombine import jsonCombine class Format(FormatTemplate.Formatter): def recordGeneratedText(self, dictionary, text): ffff = open("/tmp/productDict.txt", "w") try : myJC = jsonCombine() productDictClean = myJC.arbCopyRobust(dictionary) ffff.write(json.dumps(productDictClean, indent=4, sort_keys=True)+"\n") except : ffff.write(str(dictionary)+"\n") ffff.close() |
Appendix 3: Default MetaData_FF_W_Convective.py contents ⇑ ↑ ↓ ⇓
NOTE: This file imports and leverages material from CommonMetaData.py which contains a lot of options common to various hazard types.
''' |
Appendix 4: Example CAP Message contents ⇑ ↑ ↓ ⇓
Removed from baseline
<alert xmlns="urn:oasis:names:tc:emergency:cap:1.2"> |
Appendix 5: RiverStationInfo.java contents ⇑ ↑ ↓ ⇓
/** |