Issue affecting RM Buzz customers
Incident
Report for RM Internet & E-safety
Resolved
The updated SafetyNet Go agent was successfully released on the 15th April and this resolves the issue related to intermittent default filtering profiles. Following close monitoring, the improvements are working as expected with no reported problems. As such we are now closing this incident.
Update
Following the successful release of improvements to the SafetyNet Go back-end infrastructure last week, we are now planning to proceed with the release of the improvements to the SafetyNet Go agent. This release will occur on the morning of Thursday the 15th April. Once complete this will resolve the issue where some users intermittently received default filtering profiles.
Update
As part of improvements to resolve the issue with some users receiving default filtering profiles intermittently, we will be releasing two updates over the coming weeks. The first release will be completed on the 8th April which will deliver improvements to the SafetyNet Go back-end infrastructure. The second release is being targeted for the 15th April which will deliver the improvements to the SafetyNet Go agent on user devices. Both updates will need to be released in order for this issue to be resolved. We will provide a further update later this week following the first release.
Update
We have progressed well with the development of the fix for users receiving intermittent default filtering profiles and are now moving into the quality assurance and testing phases. We anticipate this will be completed over the next couple of weeks and following successful testing, we will then deploy the fix at the end of March or early April. We will provide the next update in the week commencing the 29th March with further details.
Update
The SafetyNet Go platform continues to remain stable. Our development team is continuing to actively work on developing the fix to the intermittent problem where users may receive a default filtering policy. We remain on track to release this in the second half of March following thorough testing.
We will provide a further update at 2pm Wednesday 10th March, or earlier if we have more information to share.
We will provide a further update at 2pm Wednesday 10th March, or earlier if we have more information to share.
Update
The SafetyNet Go platform remains in a stable state. Our engineering team are continuing to work on fixing the outstanding issues, with the priority area being to resolve the intermittent problem where users may receive a default filtering policy. This will require a reasonable amount of development and testing and as such planning indicates we are likely to release this in the second half of March.
We will provide a further update at 2 pm Tuesday 23rd February, or earlier if we have more information to share.
We will provide a further update at 2 pm Tuesday 23rd February, or earlier if we have more information to share.
Update
The SafetyNet Go platform continues to be stable for users. There remains a potential delay to some filtering policy changes, whereby changes to a filter list (e.g. whitelisting a URL) will take effect immediately as normal, however changes to groups to which a policy applies (e.g. a user being moved between policies) may take up to 8 hours to apply. Our engineers are actively working on improvements to address this policy delay plus the intermittent issue causing some users to receive default filtering profiles. We will provide a further update at 2 pm Tuesday 16th February, or earlier if we have more information to share.
Update
Our engineers are continuing to monitor the SafetyNet Go infrastructure closely. An intermittent problem was identified that may have caused some issues with the client filtering policy application and this has since been resolved, however, users may experience a delay in filter policy changes being applied for users (this may be up to 8 hours in some cases). We are continuing to work on developing improvements to return the service to normal, with the aim of deploying these as soon as they have been fully tested. We will provide a further update at 2 pm Friday 12th February, or earlier if we more information to share.
Update
As planned, we successfully released user based filtering capability back into the SafetyNet Go product this morning, restoring full functionality to customers. All indications are that the service remains stable. A small number of users may still randomly receive a default filtering profile which is caused by an authentication failure. As previously advised details on how to identify if this is the case and restore normal user filtering can be found in the SafetyNet Administration help system. Our engineers are continuing to work on the root cause of these intermittent authentication failures and aim to deploy a resolution to this as soon as possible. We will provide a further update at 2 pm, Tuesday 9th February or before if we have more information.
Update
Our engineers have confirmed successful testing of improvements that will allow us to restore full user based filtering functionality for SafetyNet Go customers. We will release these improvements between 4am and 7am tomorrow morning, Friday 5th February. This will restore full functionality for users, however there remains a small chance that an individual user may receive a default filtering profile until this is resolved in the next improvement release. Guidance on identifying whether a user is receiving default filtering and instructions on a workaround to restore user filtering is linked from the SafetyNet Administration help system. We will provide a further update at 2pm, Friday 5th February.
Update
We are continuing to work on developing a set of improvements to restore full functionality for SafetyNet Go customers and these will implemented as soon as we have thoroughly tested them. In the meantime, the per-school filtering functionality continues to be available to customers and is performing well. We will provide the next update at 2 pm Thursday 4th February.
Update
The per-school level filtering functionality remains in place and our monitoring indicates that the service is working well with only a tiny proportion of users experiencing authentication failures. Our engineers continue to focus on specific areas that have been identified as being related to the root cause. We are working on a number of improvements and details of when we will be able to release these will be provided as soon as we can. We will provide the next update at 2 pm Tuesday 2nd February
Update
The improved per-school filtering functionality was successfully released this morning and testing and monitoring indicate this is working as expected. There remains a small chance that authentication could fail randomly for some users and as a result, these users will instead receive a default filtering policy rather than their assigned school filtering policy. Instructions to identify whether the default filtering policy is being applied and how to resolve this are available on the front page of the SafetyNet Administration help documentation. Our engineers also continue to work on restoring full service as a priority and focus has now narrowed down to a set of key application and authentication areas. We will provide a further update at 2 pm Monday 1st February.
Update
Following successful testing, we will be releasing functionality improvements for all SafetyNet Go customers between 4 am and 7 am tomorrow morning, Friday 29th January. This will enable per school filter policies and also re-enable per-user reporting whilst work continues on restoring full functionality. Customers should be aware that there is a low probability that authentication could fail randomly for some users and as a result, these users will instead receive a default filtering policy rather than their assigned school filtering policy. We will be providing instructions on how to identify this and restore the school filtering policy for affected users as soon as we have these available. Our engineers are still focused on restoring full service and are working on multiple avenues of investigation, including continuing to liaise with Google on the identified authentication issues. We will provide a further update at 2 pm Friday 29th January.
Update
Following further investigation by our engineers into potential problems with user authentication, testing has identified an issue with authentication reliability between the SafetyNet Go platform and Google. Our engineers are continuing to analyze the data and engage with Google on this issue as a priority. These authentication issues also affect the functionality that enables the improvements to allow for per school filtering. Out of caution, to ensure we maintain system reliability and ensure online safety for our users, we will be delaying the release of this improvement whilst our engineers focus on a resolution to the user authentication issue.
A further update will be provided by 2 pm on Thursday 28th January.
A further update will be provided by 2 pm on Thursday 28th January.
Update
Our engineers continue to work on the issues with SafetyNet Go via two streams. The first stream is focused on deploying improved filtering to allow per school filtering policies. This functionality is currently undergoing final testing and as soon as this is successful we will be in a position to release this. The second stream involves working on the underlying cause of the problem. Engineers are investigating potential issues with user authentication and are engaged with Google engineers on this. We will provide a further update by 2 pm on Wednesday 27th January at which point we will aim to provide clarity on timescales for the improved per school filtering functionality.
Update
Work has continued by our engineering team over the weekend on analysis of the problem and a number of avenues are being explored in order to restore service. Engineers are still working on providing improved per school filtering and will aim to deploy this and any other improvements as quickly as we can. We thank you for your patience whilst we continue to work on this complex issue. The next update will be at 2pm on Tuesday 26th January.
Update
Our engineers continue to work on restoring full functionality for customers as a priority. Additionally, work to improve the filtering functionality to allow for custom filter lists per school is progressing and we hope to have this released towards the early part of next week. We’re sorry that customers are still unable to use their own user based filtering policies, however our focus has been on ensuring the service remains stable and delivers safe web browsing for users in the short term.
We will provide the next update at 2pm on Monday 25th January.
We will provide the next update at 2pm on Monday 25th January.
Update
Additional detailed monitoring and data analysis continues to be conducted by our engineers to support the identification and implementation of a permanent fix. Customers using SafetyNet Go will still be restricted to a fixed set of filtering policies at this time however we are aiming to implement an improvement next week which will allow schools to have their own individual filter policies allowing flexibility for their own requirements.
We will provide a further update at 2 pm Friday 22nd January.
We will provide a further update at 2 pm Friday 22nd January.
Update
Monitoring and customer feedback indicate that the change deployed to the SafetyNet Go environment has improved the performance and filtering of the service, and we have received no further reports of intermittent issues from customers throughout today. Customers using SafetyNet Go will still be restricted to a fixed set of filtering policies at this time. Our engineers continue to work on understanding the root cause and identifying measures to restore normal operation as soon as possible. We will provide a further update at 2pm Thursday 21st January.
Update
Following significant scaling up of the SafetyNet Go infrastructure, we are aware that customers have still been experiencing intermittent issues. In order to provide a workaround for users whilst further investigation of the root cause can continue, we have deployed a change to the SafetyNet Go environment that will cause all users to receive a fixed set of filtering policies. This will mean that user-based and custom filter policies will not be available whilst a permanent fix is developed and implemented. We apologize for any inconvenience that this may cause to users but this will aim to resolve the existing intermittent performance and filtering issues in the short term. We will provide a further update at 12 pm today
Update
Following further investigation into the issues reporting on Friday 15th January our engineers have worked over the weekend and through today to resolve the problem. Engineers believe the cause of the issue to be related to an unusually high number of users accessing the SafetyNet Go platform at the same time. We have put in place a number of measures to improve the situation and are scaling up the RM SafetyNet Go infrastructure to provide significantly increased capacity for all customers. Our engineers continue to work on this with high priority. We understand that this has impacted a number of our customers, however we thank you for your understanding and apologize for the inconvenience caused. Until that time we ask our customers to be extra vigilant about the type of content your pupils may be accessing. We will provide a further update by midday on Wednesday 20th January 2021, or sooner, if we have new information to share.
Update
We are continuing to work on this issue as our top priority. We will continue to provide updates. Thank you for your patience.
Identified
We have identified the cause of the issue and we are urgently working on a resolution. The next update will be posted tomorrow morning 8:00 am unless significant incident activity occurs. Thank you for your patience.
Update
Our engineers are continuing to investigate this issue as our top priority. Further updates will be provided in the due course.
Update
We are actively working on resolving these issues. Further updates will be provided in due course.
Investigating
We are aware of a performance issue with RM Buzz and some of the customers are impacted. We are actively working on it.
This incident affected: Web Filtering (SafetyNet Plus).
What is RM SafeClix?
