On Friday 12th August 2022, ANS network engineers were alerted to twelve unexpected crashes and reboots of core network devices, with clients downstream of each device suffering a ten minute loss of service. After investigation, the issue was found to be unique to a particular model of network device, using a specific firmware version.
Following vendor advice, and in order to mitigate against unclean reboots occurring unexpectedly, ANS took the decision to control these reboots by upgrading the firmware of the network device, before manually rebooting.
After successful completion of the upgrade and reload of those network devices most at risk, ANS now needs to upgrade and reboot network devices with the same version/firmware, but a slightly shorter up-time, to the first group.
Of this group, we have seen at least two unexpected reboots occur, indicating that these network devices too are at imminent risk of unexpected reboot.
As a result of this, ANS network engineers are performing a firmware upgrade and reboot of a network device which services your solution tonight between 20:00 16th August and 02:00 on 17th August. This will result in a period of up to ten minutes during which your solution may appear unavailable.
Due to the critical nature of the work ANS are unable to offer an alternative time/date, and due to the volume of devices being updated, ANS are unable to be more specific about the time at which your solution will be affected.
You don't need to take any further action but if you should need any further information or have any questions, please reach out via the usual channels.
Posted Aug 16, 2022 - 15:52 BST
This scheduled maintenance affected: CRM, Letters / Emails / SMS, and Reports.