Update #1: In order to ensure a safe restart of GOES after such significant maintenance, we need to stop dataflow to PDA during the PDA OPS 4-6 hour maintenance on May 24. That data will not be recovered after the outage, but GOES data will start again as soon as the database comes back online.
Topic: Planned PDA Partial Failover to CBU and Full Outage of PDA OPS at NSOF
Date/Time Issued: April 27, 2022 1842Z
Product(s) or Data Impacted: All products distributed from PDA. Some products will be available at CBU and some will be unavailable for part of the maintenance window.
Date/Time of the Initial Impact: May 24, 2022 1400Z
Date/Time of Expected End: May 25, 2022 1800Z
Length of the Outage: Failover to CBU ~24 hours; Full outage at NSOF OPS ~4-6 hours
Details/Specifics of Change:
On May 24, a significant database maintenance activity will take place during business hours that will require a period of about 4-6 hours of complete OPS PDA downtime. Delivery of GOES-16/17, Metop, POES and other data provided from the OPS PDA will cease distribution during that affected time. When the OPS PDA system is returned to service, it is anticipated that users will receive products that were ingested/created during the outage time. However, there will be a significant backlog on PDA, and it may take several hours for the OPS PDA at NSOF to receive and distribute all the products produced during the outage period in both ESPC and NCCF. After the OPS PDA outage period, latency will be impacted as the backlog of products are processed and distributed. Some products from externals may be lost, as well as some products NESDIS creates that are missing needed input data.
In order to minimize the outage impact, NESDIS plans to do this maintenance in conjunction with planned Continuity of Operations (COOP) activities. There will be a partial activation of the Consolidated Backup (CBU) site at Fairmont, WV. Only data and products from S-NPP, NOAA-20 and GCOM-W1 will be available. This failover is expected to begin with the first orbit received after 1400Z on May 24 (the day of the outage in OPS) and end with the next orbit received after 1400Z on May 25 (the day after the PDA OPS outage). The exact times of the failover will be announced closer to the event. It is noted that while in this data failover configuration, S-NPP, NOAA-20 and GCOM-W1 data and products will ONLY be at CBU for that period of about 24 hours and then will be returned to the nominal PDA OPS system at NSOF.
At this time we expect little to no duplicate data will be distributed between PDA OPS and CBU; therefore, pull users will be safe to pull data from both sites starting at the time of partial failover and until about 1.5 hours after failback end time while the final NDE products are created at CBU.
PDA staff will be available to help troubleshoot users’ issues with failover from 0930 EDT / 1330 UTC until 1800 EDT / 2200 UTC on Tue May 24 and from 1000 EDT / 1400 UTC until 1400 EDT / 1800 UTC on Wed May 25 for failback. Email PDA_DHS@noaa.gov and someone will provide the google meet details or phone number.
A declaration of Critical Weather (CWD) before or during this event will not postpone or change the date of the COOP. The COOP date is firm and JPSS data will only be available at CBU on those dates. If the database upgrade is not ready to implement on those dates, the PDA OPS database maintenance activity will be postponed to a later date. In this scenario the PDA OPS environment will not be shutdown, allowing non-JPSS data to be processed during the COOP.
Contact Information for Further Information: ESPC Operations at ESPCOperations@noaa.gov and 301-817-3880.
Web Site(s) for applicable information: N/A
This message was sent by ESPC.Notification@noaa.gov. You have been sent this and other notifications because you have opted in to receive it. If for any reason, you wish to unsubscribe, please contact ESPC Help Desk at ESPCOperations@noaa.gov or (301) 817-3880. Please note: it may take up to two business days to process your unsubscribe request.