What? On July 20, 2023, we changed the functions within our AWS Cloudfront service so that we could provide more consistent and precise User Agent data inside Form Submissions.
Why? When we implemented the previous setup for CloudFront several years ago, it was to provide better filtering of traffic to the Formstack sites to improve safety and security. One downside of that is that Cloudfront was also stripping out the User Agent data from the traffic. We implemented a workaround to capture that information from the Form Submission user interface, but the API was still limited. Now, there are new tools that allow us to have the same level of security, and pass through certain key data, like the source IP and Browser information.
What happened? When we added the new filtering setup, we failed to include the API authentication header as allowable information. Therefore, any API connection was refused for a short period of time.
When was it down? The Forms API Authentication service was effectively down from 14:29 to 17:05 Eastern Time on 20 July 2023.
How was it resolved? When the error rate increased on the API service tracking, we recognized the problem and edited the settings to allow the API Authentication to pass through. We also added the “encrypted form” header information to pass through via API, an additional forgotten attribute.
How will this be prevented in the future? Recognizing that the API is a critical service, we added a suite of automated tests around this API Authentication service to ensure that any anomalies are detected well before release to production.