Salesforce, the global customer relationship management platform, is grappling with a major service outage affecting thousands of businesses and government departments worldwide.
The disruption, which began around 05:55 UTC on November 15, has taken down access for users across multiple data centres.
Salesforce, one of the world’s largest enterprise software providers, has yet to identify the exact cause of the outage.
The company’s latest update suggests the issue may involve excessive network traffic and database instability, leading to speculation about a possible cyberattack or distributed denial-of-service (DDoS) attack.
However, Salesforce has not confirmed or denied any malicious activity.
It is also possible that recent changes triggered the database and system load.
"The database team has analyzed affected databases and identified some inconsistencies. The databases are now being restored from the most recent stable backup," Salesforce said in a statement.
Salesforce says it remains unclear when affected customers might see improvements.
The impact of the outage has been substantial, with many global businesses and government departments reliant on the platform for critical operations.
Salesforce customers are awaiting further details, with many bracing for extended delays.
Salesforce has promised to provide updates every 30 minutes as it works toward a resolution.
Service disruption - Salesforce Services
Update
The database team has analyzed affected databases and identified some inconsistencies. The databases are now being restored from the most recent stable backup. The Network team has identified and applied a rate limit to regulate and optimize connections. Further investigations have highlighted a recent change as a potential trigger, and as of 09:00 UTC, teams are working to validate a rollback of this in a canary instance in parallel. We’ll provide an update in 30 minutes or sooner if additional information becomes available.
*****
Update Impacted Services< Core Service
At 05:55 UTC on November 15, 2024, the Salesforce Technology team became aware of a service disruption impacting instances under IT3, IT4, IA7, HN3, and HN4 Data Centers. At approximately 08:15 UTC, teams began taking steps to control excessive traffic, network utilization, and database stability to mitigate the impact quickly. We don’t yet understand when customers may begin experiencing improvement as a result of these actions. Initially, this was posted as an Informational Message due to the widespread impact, but as instances were identified, we have moved to post updates here against impacted instances. We’ll provide an update in 30 minutes or sooner if additional information becomes available.
***** Issue: On November 15, 2024, at 05:55 UTC, the Salesforce Technology team became aware of a service disruption affecting multiple instances.
Impact: During the disruption, customers can’t access the Salesforce services. Multiple teams are actively engaged in investigating the cause and exploring multiple paths to resolution.
*****
Update 3: Posted Fri, 15 Nov 2024 08:11:14 UTC
The Technology team continues to evaluate the increased database connections from the application tier and has identified a suitable instance to allow them to investigate the connections thoroughly
We’ll provide an update in 30 minutes or sooner as we have additional information.
*****
Update 2: Posted Fri, 15 Nov 2024 07:39:35 UTC
The Salesforce Technology team has confirmed that a recent change initially suspected to be related has been ruled out. Teams continue to investigate the source of the high volume of database connections and evaluate the network traffic.
We’ll provide an update in 30 minutes or sooner as we have additional information.
*****
Update 1: Posted Fri, 15 Nov 2024 07:11:09 UTC
The Technology team reports that while the exact cause is still under review, recent changes are suspected contributing factors. High levels of network traffic impacting database stability have been observed, and our teams are actively tracing and mitigating the source.
Additionally, load balancers are being assessed to address any abnormal traffic sources and resolve the issue promptly.
We’ll provide an update in 30 minutes or sooner as we have additional information.
