Post Syndicated from Crosstalk Solutions original https://www.youtube.com/watch?v=gRZ4j-mXbEQ
A Bag of RATs: VenomRAT vs. AsyncRAT
Post Syndicated from Anna Širokova original https://blog.rapid7.com/2024/11/21/a-bag-of-rats-venomrat-vs-asyncrat/
Introduction
Remote access tools (RATs) have long been a favorite tool for cyber attackers, since they enable remote control over compromised systems and facilitate data theft, espionage, and continuous monitoring of victims. Among the well-known RATs are VenomRAT and AsyncRAT. These are open-source RATs and have been making headlines for their frequent use by different threat actors, including Blind Eagle/APT-C-36, Coral Rider, NullBulge, and OPERA1ER. Both RATs have their roots in QuasarRAT, another open-source project, which explains their similarities. However, as both have evolved over time, they have diverged in terms of functionalities and behavior, which affects how attackers use them and how they are detected.
Interestingly, as these RATs evolved, some security vendors have started to blur the line between them, often grouping detections under a single label, such as AsyncRAT or AsyncRAT/VenomRAT. This indicates how closely related the two are, but also suggests that their similarities may cause challenges for detection systems. We took a closer look at recent samples of each RAT to examine how they differ, if at all.
This comparison explores the core technical differences between VenomRAT and AsyncRAT by analyzing their architecture, capabilities, and tactics.
Here’s a comparison table between VenomRAT and AsyncRAT based on the findings
| Capability | VenomRAT | AsyncRAT |
|---|---|---|
| AMSI Bypass | ✔ Patches AmsiScanBuffer in amsi.dll (In-memory patching) T1562.001 | ✘ Not implemented |
| ETW Bypass | ✔ Patches EtwEventWrite in ntdll.dll (In-memory patching) T1562.006 | ✘ Not implemented |
| Keylogging | ✔ Advanced keylogger with filtering and process tracking T1056.001 | ✔ Basic keylogger with clipboard logging T1056.001 |
| Anti-analysis Techniques | ✔ Uses WMI for OS detection, VM check T1497.001 | ✔ VM, sandbox, and debugger detection T1497 |
| Hardware Interaction | ✔ Collects CPU, RAM, GPU, and software data using WMI T1082 | ✔ Collects system data via Win32_ComputerSystem T1082 |
| Process discovery | ✔ This the capability to obtain a listing of running processes T1057 | ✘ Not implemented |
| Anti-process Monitoring | ✔ Terminates system monitoring and security processes T1562.009 | ✘ Not implemented |
| Webcam Access | ✔ Camera detection and access T1125 | ✘ Not implemented |
| Dynamic API Resolution | ✔ DInvokeCore class for dynamic API resolution T1027.007 | ✘ Not implemented |
| Encrypts the configuration | ✔ 16-byte salt ("VenomRATByVenom") T1027.013 | ✔ 32-byte binary salt T1027.013 |
| Error Handling | ✔ Silent failures with basic try-catch | ✔ Sends detailed error reports to C2 T1071 |
Technical analysis
In this technical analysis, we compare two specific RAT samples:
- VenomRAT: 1574d418de3976fc9a2ba0be7bf734b919927d49bd5e74b57553dfc6eee67371AsyncRAT: caf9e2eac1bac6c5e09376c0f01fed66eea96acc000e564c907e8a1fbd594426
Both AsyncRAT and VenomRAT are open-source remote access tools developed in C# and built on the .NET Framework (v4.0.30319). A preliminary analysis based on CAPA results revealed several shared characteristics between the two. For example, both RATs use standard libraries like System.IO, System.Security.Cryptography, and System.Net for file handling, encryption, and networking. They also have common cryptographic components such as HMACSHA256, AES, and SHA256Managed, indicating similar encryption routines. Indeed, upon closer code examination, we found that their encryption classes were identical, with only one minor difference: AsyncRAT uses a 32-byte binary salt, while VenomRAT uses a 16-byte salt derived from the string “VenomRATByVenom.” Additionally, both RATs share similarities in configuration handling, mutex creation, and parts of their anti-analysis class.
However, the CAPA analysis also highlighted distinct differences between the two. Certain features present in one RAT were notably absent in the other. To verify, we manually reviewed code in both samples and described the differences below.
Keylogging and System Hooking
In the samples we analyzed the keylogger was present only in VenomRAT. However, the open-source version of AsyncRAT has a keylogger plugin. We therefore decided to investigate whether the VenomRAT keylogger implementation is the same as AsyncRAT’s implementation. Our findings suggest that the keylogging functionality is different. We summarized a comparative analysis of their keylogging implementations in the table below. Additionally, the VenomRAT keylogger configuration file DataLogs.conf and log files are saved in the user’s %AppData%\MyData folder.
| Feature | VenomRAT | AsyncRAT |
|---|---|---|
| Low-level keyboard hook (WH_KEYBOARD_LL) | ✔ | ✔ |
| Keystroke Processing | ✔ | ✔ |
| Window/Process Tracking | Tracks both process and window title | Tracks window title only |
| Clipboard Logging | ✘ | ✔ |
| Log Transmission | Periodic log sending to C2 | Continuous log sending to C2 |
| Filtering Mechanism | ✔ | ✘ |
| Error Handling | Silent failures with basic try-catch | Sends detailed error reports to C2 |
| Additional Features | Focused on keystrokes | Handles both keystrokes and clipboard |
| Thread Management | ✘ | ✔ |
Anti-Analysis
Both AsyncRAT and Venom RAT have similar implementations of the anti-analysis classes. However, we can see notable differences. AsyncRAT focuses on a broad spectrum of detection techniques, including:
- Virtual Machine Detection: It checks for known system manufacturer names such as VMware,VirtualBox, or Hyper-V.
- Sandbox Detection: It looks for sandbox-related DLLs, such as SbieDll.dll from Sandboxie.
- Debugger Detection: AsyncRAT uses CheckRemoteDebuggerPresent to detect if it’s being monitored by a debugger.
- Disk Size Check: It avoids execution on machines with less than 60GB disk size.
On the other hand, VenomRAT uses a more targeted approach. The virtual machine detection method in VenomRAT relies on querying system memory through WMI (Windows Management Instrumentation) to query system memory via Win32_CacheMemory. The method relies on counting cache memory entries, and if the number is less than 2 cache memories, it assumes the system is a virtual machine (VM). However, modern VMs are more sophisticated, and simply relying on counting cache memories may not be effective.
The other difference is, instead of targeting debuggers or sandboxes, VenomRAT attempts to avoid running on server operating systems by querying the Win32_OperatingSystem WMI class and checking the ProductType, which differentiates between desktop and server environments. We summarized class differences in the table below.
| Feature | AsyncRAT AntiAnalysis Class | Venom RAT Anti_Analysis Class |
|---|---|---|
| VM Detection | ✔ | ✔ |
| Sandbox Detection | ✔ | ✘ |
| Debugger Detection | ✔ | ✘ |
| Operating System Detection | ✔ | ✔ |
| Process Discovery | ✘ | ✔ |
Hardware Interaction
VenomRAT has hardware interaction capabilities, allowing it to gather detailed system information through WMI queries with ManagementObjectSearcher objects. These features are encapsulated in the CGRInfo class, which enables the collection of CPU, RAM, GPU, and software data:
- GetCPUName(): Retrieves the CPU name and the number of cores
- GetRAM(): Fetches the total installed physical memory (RAM)
- GetGPU(): Obtains the GPU name and driver version
- GetInstalledApplications(): Scans the Windows Registry to compile a list of installed applications
- GetUserProcessList(): Collects information on all running processes with visible windows
The collected data is sent back to the command-and-control (C2) server. This class is absent in both the version of AsyncRAT we analyzed and the open-source version.
DcRAT joined the party with AntiProcess and Camera classes
VenomRAT includes two notable classes absent in AsyncRAT: the AntiProcess and Camera classes.
The AntiProcess class is an anti-monitoring and anti-detection component of VenomRAT. Malware uses the Windows API function CreateToolhelp32Snapshot to get a snapshot of all running processes and search for specific processes. We categorized the processes the malware is looking for below.
System Monitoring Tools that can prevent users from identifying or stopping VenomRAT.
- Taskmgr.exe
- ProcessHacker.exe
- procexp.exe
Security & Antivirus Processes: Terminating them reduces the risk of VenomRAT being detected or removed by security software.
- MSASCui.exe
- MsMpEng.exe
- MpUXSrv.exe
- MpCmdRun.exe
- NisSrv.exe
System Configuration Utilities: By targeting these, VenomRAT prevents users from adjusting security settings, inspecting registry changes, or manually removing the malware.
- ConfigSecurityPolicy.exe
- MSConfig.exe
- Regedit.exe
- UserAccountControlSettings.exe
- Taskkill.exe
If a matching process is found, it terminates it by its process ID (PID).
The Camera class is designed to detect webcams on a Windows system by querying the available system devices using COM interfaces. It retrieves a list of devices by category, specifically looking for video input devices. The class uses the ICreateDevEnum and IPropertyBag interfaces to enumerate and extract the device names.
However, both these classes, although absent in AasyncRAT, are not exclusive to VenomRAT only. Apparently they are exact copycats of yet another open-source RAT, DcRAT.
AMSI and ETW Bypass
This class was found only in the VenomRAT sample and is designed to bypass key Windows security mechanisms through in-memory patching. It specifically disables two critical Windows security features: AMSI (Antimalware Scan Interface) and ETW (Event Tracing for Windows), which are often used by antivirus software and monitoring tools to detect malware.
Key Functions:
- AMSI Bypass: The class patches the AmsiScanBuffer function within amsi.dll to prevent AMSI from scanning for malicious content.
- ETW Bypass: The class patches the EtwEventWrite function in ntdll.dll, which stops ETW from logging events related to the malware’s activity.
The patching process is performed in-memory. The class dynamically checks the system’s architecture (32-bit or 64-bit) and loads the appropriate DLLs (amsi.dll and ntdll.dll) to apply the patches based on the platform. The techniques used by VenomRAT closely mirror those found in the SharpSploit project, an open-source tool often used by penetration testers and red teams to test and bypass security features in a controlled environment. SharpSploit contains classes for bypassing both AMSI and ETW using similar in-memory patching methods, which likely served as inspiration for VenomRAT’s implementation.
This security bypass functionality makes VenomRAT more capable of evading modern security defenses.
Dynamic API resolution
VenomRAT has yet another class which is absent in AsyncRAT. The DInvokeCore class is implemented to dynamically resolve and call Windows API functions at runtime; this method bypasses traditional static imports, making it harder for antivirus and endpoint detection and response (EDR) systems to detect malicious activity.
Instead of statically importing Windows APIs, the class resolves function addresses at runtime (e.g., from ntdll.dll or kernel32.dll) using methods like GetLibraryAddress and GetExportAddress. This approach makes it difficult for static analysis tools to flag malicious behavior.
It uses the NtProtectVirtualMemory method to alter memory protection settings, allowing execution of code in memory regions that are normally non-executable—an effective method for in-memory execution of malicious payloads.
Implementation of DInvokeCore closely mirrors the open-source SharpSploit Generic class from the D/Invoke project by TheWover. The DInvokeCore class from VenomRAT appears to be a simplified version, which lacks some features but has core techniques for dynamic API invocation.
Conclusion
Our analysis was sparked by detection vendors grouping VenomRAT and AsyncRAT under the same label, blurring the lines between the two. While they indeed belong to the QuasarRAT family, they are still different RATs.
AsyncRAT appears to closely match the latest open-source release (v0.5.8). However, the VenomRAT seems to have evolved and added other capabilities, although a lot of them seem to be a copy-paste from another open-source RAT (DcRAT) and the SharpSploit project. Despite this, VenomRAT presents more advanced evasion techniques, making it a more sophisticated threat.
Therefore, it’s important for security vendors to treat them as distinct threats, recognizing that VenomRAT brings more advanced evasion capabilities, even if much of it isn’t truly unique. To help to resolve this confusion, we are sharing an updated VenomRAT YARA rule with the community, helping improve detection and response efforts.
Rapid7 customers
InsightIDR and Managed Detection and Response (MDR) customers have existing detection coverage through Rapid7’s expansive library of detection rules. Rapid7 recommends installing the Insight Agent on all applicable hosts to ensure visibility into suspicious processes and proper detection coverage. The following rule will alert on a wide range of malicious hashes tied to behavior in this blog: Suspicious Process – Malicious Hash On Asset
YARA rule
The VenomRAT YARA rule can be found on the Rapid7 Labs GitHub here.
Run high-availability long-running clusters with Amazon EMR instance fleets
Post Syndicated from Garima Arora original https://aws.amazon.com/blogs/big-data/run-high-availability-long-running-clusters-with-amazon-emr-instance-fleets/
AWS now supports high availability Amazon EMR on EC2 clusters with instance fleet configuration. With high availability instance fleet clusters, you now get the enhanced resiliency and fault tolerance of high availability architecture, along with the improved flexibility and intelligence in Amazon Elastic Compute Cloud (Amazon EC2) instance selection of instance fleets. Amazon EMR is a cloud big data platform for petabyte-scale data processing, interactive analysis, streaming, and machine learning (ML) using open source frameworks such as Apache Spark, Presto and Trino, and Apache Flink. Customers love the scalability and flexibility that Amazon EMR on EC2 offers. However, like most distributed systems running mission-critical workloads, high availability is a core requirement, especially for those with long-running workloads.
In this post, we demonstrate how to launch a high availability instance fleet cluster using the newly redesigned Amazon EMR console, as well as using an AWS CloudFormation template. We also go over the basic concepts of Hadoop high availability, EMR instance fleets, the benefits and trade-offs of high availability, and best practices for running resilient EMR clusters.
High availability in Hadoop
High availability (HA) provides continuous uptime and fault tolerance for a Hadoop cluster. The core components of Hadoop, like Hadoop Distributed File System (HDFS) NameNode and YARN ResourceManager, are single points of failure in clusters with a single primary node. In the event that any of them crash, the entire cluster goes down. High Availability removes this single point of failure by introducing redundant standby nodes that can quickly take over if the primary node fails.
In a high availability EMR cluster, one node serves as the active NameNode that handles client operations, and others act as standby NameNodes. The standby NameNodes constantly synchronize their state with the active one, enabling seamless failover to maintain service availability. To learn more, see Supported applications in an Amazon EMR Cluster with multiple primary nodes.
Key instance fleet differentiations
Amazon EMR recommends using the instance fleet configuration option for provisioning EC2 instances in EMR clusters because it offers a flexible and robust approach to cluster provisioning. Some key advantages include:
- Flexible instance provisioning – Instance fleets provide a powerful and simple way to specify up to five EC2 instance types on the Amazon EMR console, or up to 30 when using the AWS Command Line Interface (AWS CLI) or API with an allocation strategy. This enhanced diversity helps optimize for cost and performance while increasing the likelihood of fulfilling capacity requirements.
- Target capacity management – You can specify target capacities for On-Demand and Spot Instances for each fleet. Amazon EMR automatically manages the mix of instances to meet these targets, reducing operational overhead.
- Improved availability – By spanning multiple instance types and purchasing options such as On-Demand and Spot, instance fleets are more resilient to capacity fluctuations in specific EC2 instance pools.
- Enhanced Spot Instance handling – Instance fleets offer superior management of Spot Instances, including the ability to set timeouts and specify actions if Spot capacity can’t be provisioned.
- Reliable cluster launches – You can configure your instance fleet to select multiple subnets for different Availability Zones, allowing Amazon EMR to find the best combination of instances and purchasing options across these zones to launch your cluster in. Amazon EMR will identify the best Availability Zone based on your configuration and available EC2 capacity and launch the cluster.
Prerequisites
Before you launch the high availability EMR instance fleet clusters, make sure you have the following:
- Latest Amazon EMR release – We recommend that you use the latest Amazon EMR release to benefit from the highest level of resiliency and stability for your high availability clusters. High availability for instance fleets is supported with Amazon EMR releases 5.36.1, 6.8.1, 6.9.1, 6.10.1, 6.11.1, 6.12.0, and later.
- Supported applications – High availability for instance fleets is supported for applications such as Apache Spark, Presto, Trino, and Apache Flink. Refer to Supported applications in an Amazon EMR Cluster with multiple primary nodes for the complete list of supported applications and their failover processes.
Launch a high availability instance fleet cluster using the Amazon EMR console
Complete the following steps on the Amazon EMR console to configure and launch a high availability EMR cluster with instance fleets:
- On the Amazon EMR console, create a new cluster.
- For Name, enter a name.
- For Amazon EMR release, choose the Amazon EMR release that supports high availability clusters with instance fleets. The setting will default to the latest available Amazon EMR release.
- Under Cluster configuration, choose the desired instance types for the primary fleet. (You can select up to five when using the Amazon EMR console.)
- Select Use high availability to launch the cluster with three primary nodes.
- Choose the instance types and target On-Demand and Spot size for the core and task fleet according to your requirements.
- Under Allocation strategy, select Apply allocation strategy.
- 1 We recommend that you select Price-capacity optimized for your allocation strategy for your cluster for faster cluster provisioning, more accurate Spot Instance allocation, and fewer Spot Instance interruptions.
- Under Networking, you can choose multiple subnets for different Availability Zones. This allows Amazon EMR to look across those subnets and launch the cluster in an Availability Zone that best suits your instance and purchasing option requirements.
- Review your cluster configuration and choose Create cluster.
Amazon EMR will launch your cluster in a few minutes. You can view the cluster details on the Amazon EMR console.
Launch a high availability cluster with AWS CloudFormation
To launch a high availability cluster using AWS CloudFormation, complete the following steps:
- Create a CloudFormation template with EMR resource type
AWS::EMR::Clusterand JobFlowInstancesConfig property typesMasterInstanceFleet,CoreInstanceFleetand (optional)TaskInstanceFleets. To launch a high availability cluster, configure TargetOnDemandCapacity=3, TargetSpotCapacity=0 for the primary instance fleet and weightedCapacity=1 for each instance type configured for the fleet. See the following code:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"cluster": {
"Type": "AWS::EMR::Cluster",
"Properties": {
"Instances": {
"Ec2SubnetIds": [
"subnet-003c889b8379f42d1",
"subnet-0382aadd4de4f5da9",
"subnet-078fbbb77c92ab099"
],
"MasterInstanceFleet": {
"Name": "HAPrimaryFleet",
"TargetOnDemandCapacity": 3,
"TargetSpotCapacity": 0,
"InstanceTypeConfigs": [
{
"InstanceType": "m5.xlarge",
"WeightedCapacity": 1
},
{
"InstanceType": "m5.2xlarge",
"WeightedCapacity": 1
},
{
"InstanceType": "m5.4xlarge",
"WeightedCapacity": 1
}
]
},
"CoreInstanceFleet": {
"Name": "cfnCore",
"InstanceTypeConfigs": [
{
"InstanceType": "m5.xlarge",
"WeightedCapacity": 1
},
{
"InstanceType": "m5.2xlarge",
"WeightedCapacity": 2
},
{
"InstanceType": "m5.4xlarge",
"WeightedCapacity": 4
}
],
"LaunchSpecifications": {
"SpotSpecification": {
"TimeoutAction": "SWITCH_TO_ON_DEMAND",
"TimeoutDurationMinutes": 20,
"AllocationStrategy": "PRICE_CAPACITY_OPTIMIZED"
}
},
"TargetOnDemandCapacity": "4",
"TargetSpotCapacity": 0
},
"TaskInstanceFleets": [
{
"Name": "cfnTask",
"InstanceTypeConfigs": [
{
"InstanceType": "m5.xlarge",
"WeightedCapacity": 1
},
{
"InstanceType": "m5.2xlarge",
"WeightedCapacity": 2
},
{
"InstanceType": "m5.4xlarge",
"WeightedCapacity": 4
}
],
"LaunchSpecifications": {
"SpotSpecification": {
"TimeoutAction": "SWITCH_TO_ON_DEMAND",
"TimeoutDurationMinutes": 20,
"AllocationStrategy": "PRICE_CAPACITY_OPTIMIZED"
}
},
"TargetOnDemandCapacity": "0",
"TargetSpotCapacity": 4
}
]
},
"Name": "TestHACluster",
"ServiceRole": "EMR_DefaultRole",
"JobFlowRole": "EMR_EC2_DefaultRole",
"ReleaseLabel": "emr-6.15.0",
"PlacementGroupConfigs": [
{
"InstanceRole": "MASTER",
"PlacementStrategy": "SPREAD"
}
]
}
}
}
}
Make sure to use an Amazon EMR release that supports high availability clusters with instance fleets.
- Create a CloudFormation stack with the preceding template:
aws cloudformation create-stack --stack-name HAInstanceFleetCluster --template-body file://cfn-template.json --region us-east-1- Retrieve the cluster ID from the list-clusters response to use in the following steps. You can further filter this list based on filters like cluster status, creation date, and time.
aws emr list-clusters --query "Clusters[?Name=='<YourClusterName>']"- Run the following describe-cluster command:
aws emr describe-cluster --cluster-id j-XXXXXXXXXXX --region us-east-1If the high availability cluster was launched successfully, the describe-cluster response will return the state of the primary fleet as RUNNING and provisionedOnDemandCapacity as 3. By this point, all three primary nodes have been started successfully.
Primary node failover with High Availability clusters
To fetch information on all EC2 instances for an instance fleet, use the list-instances command:
aws emr list-instances --cluster-id j-XXXXXXXXXXX --instance-fleet-type MASTER --region us-east-1
For high availability clusters, it will return three instances in RUNNING state for the primary fleet and other attributes like public and private DNS names.
The following screenshot shows the instance fleet status on the Amazon EMR console.
Let’s examine two cases for primary node failover.
Case 1: One of the three primary instances is accidentally stopped
When an EC2 instance is accidentally stopped by a user, Amazon EMR detects this and performs a failover for the stopped primary node. Amazon EMR also attempts to launch a new primary node with the same private IP and DNS name to recover back the quorum. During this failover, the cluster remains fully operational, providing true resiliency to single primary node failures.
The following screenshots illustrate the instance fleet details.
This automatic recovery for primary nodes is also reflected in the MultiMasterInstanceGroupNodesRunning or MultiMasterInstanceGroupNodesRunningPercentage Amazon CloudWatch metric emitted by Amazon EMR for your cluster. The following screenshot shows an example of these metrics.
Case 2: One of the three primary instances becomes unhealthy
If Amazon EMR continuously receives failures when trying to connect to a primary instance, it is deemed as unhealthy and Amazon EMR will attempt to replace it. Similar to case 1, Amazon EMR will perform a failover for the stopped primary node and also attempt to launch a new primary node with the same private IP and DNS name to recover the quorum.
If you list the instances for the primary fleet, the response will include information for the EC2 instance that was stopped by the user and the new primary instance that replaced it with the same private IP and DNS name.
The following screenshot shows an example of the CloudWatch metrics.
An instance can have connection failures for multiple reasons, including but not limited to disk space unavailable on the instance, critical cluster daemons like instance controller shut down with errors, high CPU utilization, and more. Amazon EMR is continuously improving its health monitoring criteria to better identify unhealthy nodes on an EMR cluster.
Considerations and best practices
The following are some of the key considerations and best practices for using EMR instance fleets to launch a high availability cluster with multiple primary nodes:
- Use the latest EMR release – With the latest EMR releases, you get the highest level of resiliency and stability for your high availability EMR clusters with multiple primary nodes.
- Configure subnets for high availability – Amazon EMR can’t replace a failed primary node if the subnet is oversubscribed (there aren’t any available private IP addresses in the subnet). This results in a cluster failure as soon as the second primary node fails. Limited availability of IP addresses in a subnet can also result in cluster launch or scaling failures. To avoid such scenarios, we recommend that you dedicate an entire subnet to an EMR cluster.
- Configure core nodes for enhanced data availability – To minimize the risk of local HDFS data loss on your production clusters, we recommend that you set the
dfs.replicationparameter to 3 and launch at least four core nodes. Settingdfs.replicationto 1 on clusters with fewer than four core nodes can lead to data loss if a single core node goes down. For clusters with three or fewer core nodes, setdfs.replicationparameter to at least 2 to achieve sufficient HDFS data replication. For more information, see HDFS configuration. - Use an allocation strategy – We recommend enabling an allocation strategy option for your instance fleet cluster to provide faster cluster provisioning, more accurate Spot Instance allocation, and fewer Spot Instance interruptions.
- Set alarms for monitoring primary nodes – You should monitor the health and status of primary nodes of your long-running clusters to maintain smooth operations. Configure alarms using CloudWatch metrics such as
MultiMasterInstanceGroupNodesRunning,MultiMasterInstanceGroupNodesRunningPercentage, orMultiMasterInstanceGroupNodesRequested. - Integrate with EC2 placement groups – You can also choose to protect primary instances against hardware failures by using a placement group strategy for your primary fleet. This will spread the three primary instances across separate underlying hardware to avoid loss of multiple primary nodes at the same time in the event of a hardware failure. See Amazon EMR integration with EC2 placement groups for more details.
When setting up a high availability instance fleet cluster with Amazon EMR on EC2, it’s important to understand that all EMR nodes, including the three primary nodes, are launched within a single Availability Zone. Although this configuration maintains high availability within that Availability Zone, it also means that the entire cluster can’t tolerate an Availability Zone outage. To mitigate the risk of cluster failures due to Spot Instance reclamation, Amazon EMR launches the primary nodes using On-Demand instances, providing an additional layer of reliability for these critical components of the cluster.
Conclusion
This post demonstrated how you can use high availability with EMR on EC2 instance fleets to enhance the resiliency and reliability of your big data workloads. By using instance fleets with multiple primary nodes, EMR clusters can withstand failures and maintain uninterrupted operations, while providing enhanced instance diversity and better Spot capacity management within a single Availability Zone. You can quickly set up these high availability clusters using the Amazon EMR console or AWS CloudFormation, and monitor their health using CloudWatch metrics.
To learn more about the supported applications and their failover process, see Supported applications in an Amazon EMR Cluster with multiple primary nodes. To get started with this feature and launch a high availability EMR on EC2 cluster, refer to Plan and configure primary nodes.
About the Authors
Garima Arora is a Software Development Engineer for Amazon EMR at Amazon Web Services. She specializes in capacity optimization and helps build services that allow customers to run big data applications and petabyte-scale data analytics faster. When not hard at work, she enjoys reading fiction novels and watching anime.
Ravi Kumar is a Senior Product Manager Technical-ES (PMT) at Amazon Web Services, specialized in building exabyte-scale data infrastructure and analytics platforms. With a passion for building innovative tools, he helps customers unlock valuable insights from their structured and unstructured data. Ravi’s expertise lies in creating robust data foundations using open-source technologies and advanced cloud computing, that powers advanced artificial intelligence and machine learning use cases. A recognized thought leader in the field, he advances the data and AI ecosystem through pioneering solutions and collaborative industry initiatives. As a strong advocate for customer-centric solutions, Ravi constantly seeks ways to simplify complex data challenges and enhance user experiences. Outside of work, Ravi is an avid technology enthusiast who enjoys exploring emerging trends in data science, cloud computing, and machine learning.
Tarun Chanana is a Software Development Manager for Amazon EMR at Amazon Web Services.
Comic for 2024.11.21 – This Doesn’t Normally Happen
Post Syndicated from Explosm.net original https://explosm.net/comics/this-doesnt-normally-happen
New Cyanide and Happiness Comic
Should you buy a CHEAP projector in 2024?
Post Syndicated from The Hook Up original https://www.youtube.com/watch?v=p7pcVKvtgCE
[$] The beginning of the 6.13 merge window
Post Syndicated from corbet original https://lwn.net/Articles/998623/
As of this writing, just over 1,800 non-merge changesets have been pulled
into the mainline kernel for the 6.13 release. That number may seem small,
given that a typical merge window brings in at least 12,000 commits, but
the early pulls this time around have focused on significant core changes,
and there are quite a few of them. The time has come to summarize the
changes pulled so far, including lazy preemption, multi-grained timestamps,
new extended-attribute system calls, and more.
Announcing a visual update to the AWS Management Console (preview)
Post Syndicated from Prasad Rao original https://aws.amazon.com/blogs/aws/announcing-a-visual-update-to-the-aws-management-console-preview/
Today, we are announcing a visual update to the AWS Management Console in preview. We are rolling out this update by using the latest version of Cloudscape, the Amazon Web Services (AWS) design system used to build intuitive, inclusive, and meaningful AWS experiences at scale.
In this post, I describe how the visual update makes it easier for you to scan content, focus on the key information, and find what you are looking for more effectively while preserving the familiar and consistent experience of the AWS Management Console.
Improved readability
A revised typography scale and improved treatment of headings result in a stronger visual hierarchy, which helps you to better locate and understand your data. A refined use of color and weight across text elements help you differentiate key pieces of information faster. For example, you’ll see that labels in form fields are now more prominent, which eases scanning. The same applies to keys in key-value pairs and sections across components, such as service navigation, expandable elements, and tabs.
We improved the color palette, made it more vibrant, and simplified the color treatment of interactive elements. For example, secondary buttons, links, tokens, and interactive states for numerous interface elements are now blue, making it easier for you to interact with the content on the screen and contributing to improving task efficiency.
Improved focus in light and dark mode
Reduced visual complexity supports user focus. We replaced drop shadows with a new thinner stroke on main content wrappers, such as cards, panels, and containers, and unified the use of border styles across components. This reduces visual noise and optimizes the space inside the layout. Shadows are now reserved to add emphasis on specific interactive and transient elements, which helps simplify visual depth and improves the overall content hierarchy.
We also released updates to dark mode to address the need for clearer differentiation between elements on the page. These changes include an update to the color ramp and improved contrast between interactive states across components.
Modernized interface
We modernized the interface while retaining familiarity to continue to offer predictable and recognizable experiences across AWS. The user experience is now easier on the eyes, thanks to the use of rounder shapes, brighter colors, and improved layout treatment. These updates create a smoother, more natural appearance, making the interface more visually pleasing.
To deliver a more delightful experience and support visual storytelling, we also introduced a whole new family of illustrations and motion while still offering the highest accessibility standards.
Improved information density
We optimized information density by reducing unused space, leading to more content visible on the screen. Related data is now displayed closer together, reinforcing visual grouping. Space within content wrappers such as cards and containers has been minimized, so you can consume more information at once. The new layout is centered and wider, optimizing the experience to serve larger screen sizes than before. The visual update makes it easier to consume information, which creates a better and friendlier experience within the AWS Management Console.
Additionally, we introduced Toolbar, a new way to navigate and access contextual tools and features. This helps you perform your tasks while maximizing the amount of content available on screen.
Improved consistency
The interface is now more distinctive and consistent. Refreshed colors, iconography, and shapes help deliver a more dynamic and expressive experience while reinforcing a unified and cohesive journey across all AWS experiences.
Available now
You can start experiencing the visual update now in selected consoles across all AWS Regions by visiting the AWS Management Console. We’ll be extending the update across all services. Thanks to the new visual treatment, you can now benefit from an experience that’s more readable and intuitive and that contributes to improved overall task efficiency.
PATREON UNCUT – Join us Behind The Scenes
Post Syndicated from Matt Granger original https://www.youtube.com/watch?v=-PExuy2Wrkw
Security updates for Thursday
Post Syndicated from jake original https://lwn.net/Articles/998949/
Security updates have been issued by AlmaLinux (kernel, NetworkManager-libreswan, and openssl), Fedora (chromium and llvm-test-suite), Mageia (thunderbird), and Ubuntu (linux-aws-6.8, linux-azure, linux-azure-6.8, linux-oracle-6.8,, linux-azure, and ruby2.7).
Secret Service Tracking People’s Locations without Warrant
Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2024/11/secret-service-tracking-peoples-locations-without-warrant.html
This feels important:
The Secret Service has used a technology called Locate X which uses location data harvested from ordinary apps installed on phones. Because users agreed to an opaque terms of service page, the Secret Service believes it doesn’t need a warrant.
TikTok Ban: Last Week Tonight with John Oliver (HBO)
Post Syndicated from LastWeekTonight original https://www.youtube.com/watch?v=5CZNlaeZAtw
Trump’s Vision to Remake the Military
Post Syndicated from The Atlantic original https://www.youtube.com/watch?v=SiCjPqLWnu8
Има нещо гнило в (името на) пуйката
Post Syndicated from original https://www.toest.bg/ima-neshto-gnilo-v-imeto-na-puykata/
След президентските избори в САЩ близо половината американски граждани вероятно се чувстват всякак, но не и благодарни. И все пак това едва ли ще спре огромното мнозинство американци – без значение дали са гласували за Камала Харис, или за Доналд Тръмп – и тази година, както всяка, в четвъртия четвъртък на месец ноември да се съберат със своите семейства, близки, а често и с не толкова близки безпризорни чужденци, за да празнуват.
Точно като такъв безпризорен чужденец през осемте години, които прекарах в Щатите, съм присъствала поне на два пъти по толкова събирания по случай Деня на благодарността. За обкръжаващите ме американци, за които може да се каже всичко, но не и че са негостоприемни, беше немислимо да ме оставят сама на този така важен за тях, макар и незначещ нищо за мен празник. Ето защо аз ежегодно бях канена да присъствам на по две, нерядко на три, а понякога дори и на четири събирания за Деня на благодарността.
При първоначалната ми среща с американската Thanksgiving трапеза няколко месеца след пристигането ми в Америка в края на 90-те се почувствах, както вероятно са се чувствали пилигримите от Плимутската колония през 1621 г., споделяйки празника на реколтата с местното население от индианското племе Уампаноаг. Именно това събитие се смята за първообраза на Деня на благодарността. С други думи,
празничното меню ми се стори изключително екзотично.
Макар че повечето наименования на ястията ми бяха познати поне на теория, много от тях виждах на живо за пръв път, а начинът на приготвяне и комбинирането им надхвърляше и най-развинтеното кулинарно въображение: сос от боровинки и сос „грейви“; печени сладки картофи, покрити с пухкави бонбони маршмелоу; картофено пюре, но приготвено от небелени картофи; плънка от хляб с гъби, печени кестени, бекон и сушени сливи, която се сервира отделно; тиквен пай, пай от пекан или някаква умопомрачаваща комбинация от двете за десерт. Почетното място в центъра на трапезата, разбира се, неизменно беше заето от огромна печена пуйка.
Още тогава, макар и зашеметена от екзотичните гозби и безмерните количества погълната храна¹, усетих, че има нещо гнило – само че не в Дания, а в Турция, или по-точно във факта, че английското название на пуйката по някаква причина е идентично с името на държавата, с която България има граница на югоизток².
Едва сега се заемам да проуча тази мистерия по-внимателно. Оказва се, че макар и летенето изобщо да не ѝ се удава, тази едра, странно изглеждаща птица през изминалите няколко столетия е обиколила цялото земно кълбо – както буквално, така и чрез названията си на различни езици. Проследяването на тези наименования чудесно илюстрира сложните, често водещи до обърквания и недоразумения процеси на глобализация, и то векове преди самият термин да съществува.
Преди да се озове на американската трапеза за Деня на благодарността и да получи англоезичното си наименование, пуйката прекосява Атлантическия океан не веднъж, а цели два пъти, като междувременно се разпространява из цяла Европа, участва в пиеса на Шекспир, минава през Азия, среща се със свои африкански роднини и стига чак до Индия.
Прародителите на пуйката от американската трапеза първоначално се опитомяват и отглеждат от ацтеките, които я наричат huehxolotl, или „голямо чудовище”, на езика науатъл³.
Европейците в лицето на Кортес и неговите конкистадори за пръв път се срещат с въпросното чудовище през 1519 г., а впоследствие вождът на ацтеките Монтесума им подарява 1500 броя от птиците, които те отнасят в Европа. Но испанците тогава объркват пуйката с вид по-скромно украсен паун, като това се отразява и в наименованието pavo – от pavo real ‘паун’) – което те ѝ дават и под което е известна в Испания и досега.
Още през XV век обаче, тоест преди испанците да срещнат чудатата птица в Америка, английски търговци внасят в Европа друг вид птица, известна на български като „токачка“. Тя наподобява пуйката и принадлежи към същия разред (Кокошоподобни, или Galliformes) като нея, но всъщност произлиза от западния бряг на Африка. Тъй като търговията на много стоки през този период минава през Османската империя, въпросната птица (подобно на много други продукти, обозначени като „турски“, без значение какъв е реалният им произход) става известна на английски като turkey-cock.
Ето как в първите стотина години след пристигането на американската пуйка в Европа тя често била бъркана с африканската птица, а понятието turkey-cock се използвало без особена претенция за прецизност и за двете⁴.
През 1620 г. понятието turkey-cock прекосява Атлантическия океан в обратната посока заедно с английските пилигрими на кораба „Мейфлауър“,
които също толкова непридирчиво започват да го прилагат към всякакви едри птици, включително и към северноамериканските наследници на ацтекските пуйки. По този начин, макар да се смята, че пуйката всъщност не е присъствала на първото празнуване на Деня на благодарността през 1621 г., наименованието, впоследствие скъсено до turkey, се установява в Америка⁵.
Жертва на безразборната употреба става и иначе придирчивият Шекспир, който в пиесата си „Хенри V“ от 1599 г. подигравателно представя (в превод на Валери Петров) „оня подъл, низък, долен, въшлив самохвалко и нахалник Пистол“ с думите: Why, here he comes, swelling like a turkey-cock. (Или, отново в същия превод: „Ха, ето го, иде, наперен като пуяк!“). Но определението всъщност се оказва анахронично, тъй като действието в пиесата се развива непосредствено преди и след Битката при Аженкур, състояла се през 1415 г., когато европейците само са могли да мечтаят за Новия свят и неговите пуйки.
Самият Нов свят, както знаем, е резултат от колосално объркване, първоначално географско, а впоследствие и етимологично, вследствие на което в края на XV век карибските острови стават известни като Западни Индии, а коренното население на Америка – като „индианци“. Това объркване вероятно води и до свързаните с Индия наименования на пуйката на няколко езика. Освен в турския, на който англоезичната turkey се нарича hindi, те включват и френски (dinde, от poule d’Inde, или „пиле от Индия“), руски („индейка“ или „индюшка“), полски (indyk), даже и баски (indioilar arrunt), каталански (gall dindi), грузински (ინდაური/indauri) и иврит (הוֹדוּ/hodu).
Към тези имена можем да добавим и наименованията, заети от съвсем реално съществуващо място в самата Индия, а именно град Каликут, където експедицията на португалския мореплавател Вашку да Гама акостира през 1498 г. Точно Каликут служи като основа за наименованията на пуйката на няколко езика, например скандинавските (kalkon на шведски, kalkun на норвежки и датски), откъдето навлиза и във финския (kalkkuna), а също така и на литовски (kalakutas) и нидерландски (kalkoen), откъдето се озовава в африкаанс и даже в синхалски (කළුකුමා/kaḷukumā), макар че би се очаквало неговите ползватели, бидейки жители на близкия до Индия остров Шри Ланка, да са по-добре ориентирани относно индийската фауна.
Тези наименования също се дължат на объркването на пуйката с токачката, която португалските търговци активно внасят в Индия през целия XVI век, преди значително по-впечатляващата ѝ американска роднина тържествено да пристигне на Индийския субконтинент в началото на XVII век.
Когато през 1612 г. американската пуйка се появява заедно с други екзотични животни в двора на моголския император Джахангир в Агра, тя предизвиква истинска сензация. Джахангир е толкова впечатлен от странната птица, че поръчва на придворния художник Устад Мансур да я нарисува, „така че изумлението, което произтича от научаването за [нея], да се увеличи“ още повече. Рисунката, която днес се намира в колекцията на музея „Виктория и Алберт“ в Лондон, представя пуйката като някакво вълшебно създание от приказките и наистина предизвиква изумление.
Името, което пуйката впоследствие получава на хинди – पेरू /piroo, отвежда до място от другата страна на Земята, което е толкова далече от Индия, че на практика би могло да е приказно. На фона на сериозните обърквания на континенти и посоки обаче това наименование – взето назаем от португалската дума peru, появила се в резултат от схващането, че пуйката пристига в Португалия от Перу – се оказва най-близко до истината.
Ако говорим за фактологична точност, българският език поне на пръв поглед, изглежда, се представя доста добре. Етимологичният речник на БАН цитира предполагаемо родство на думата „пуйка“ с „птица“, сочейки съответствия с няколко различни езика, но също така „допуска и ономатопеичен произход“6.
И тъкмо да взема да си помисля, че като по чудо не сме попаднали в капана с обърканите наименования, откривам, че и българският не е застрахован срещу чужди влияния. С други думи, и ние, ако не сме дали, то поне сме взели нещо от света, било то и фактологично неточно. А именно диалектната дума за пуйка „мисирка“ (която през 2020 г. Бойко Борисов направи вайръл, използвайки я като определение на журналисти) всъщност произлиза от арабското наименование на Египет – مِصْر/miṣr, откъдето в миналото (погрешно) се е смятало, че птицата е дошла.
На основата на всички тези разкрития вече ми става трудно да мисля за пуйката като за ястиe, което е типично за Деня на благодарността, или в преносния смисъл на думатa – като синоним на глуповато перчене. Вместо това вече започвам да възприемам птицата като символ на завидна космополитност, както и като особено забавна илюстрация на непредвидимите последствия от човешкия порив да опитоми далечното и непознатото, давайки му отново далечно, макар и по-познато, а често и съвсем грешно име.
1 Един от изразите, които научих покрай празнуването на Деня на благодарността, е food coma, или „хранителна кома“, а сега разбирам, че медицинският термин за това състояние звучи много по-застрашително от обикновена сънливост след хранене – „постпрандиална сомнолентност“.
2 Всъщност пуйката споделя името си с Турция до 2022 г., когато държавата официално сменя изписването на англоезичното си наименование от Turkey на Türkiye – турското име на Турция. Като инициатор на промяната Ердоган твърди, че новото изписване най-добре изразява културата, цивилизацията и ценностите на турския народ, но според някои промяната на практика цели да сложи край на обидните асоциации с пуйката.
3 В Хондурас, Мексико и Ел Салвадор, както и в някои от южните американски щати, пуйката и досега е известна като guajolote – испанизираното наименование на думата от езика науатъл.
4 Езиковите недоразумения се простират и до таксономичното наименование на пуйката, което Карл Линей ѝ дава през 1758 г. Meleagris gallopavo произлиза от старогръцката дума μελεαγρίς (meleagrís, ‘токачка’) и латинската gallus (‘пиле’) + pāvō (‘паун’).
5 Днес менюто за Деня на благодарността чувствително се различава от храната, която според историческите източници е присъствала на първото празненство през 1621 г. Тогава на трапезата е имало еленско месо, разнообразни диви птици, няколко вида риба и местен сорт царевица, известен като „флинт“.
6 За съвременните наименования на пуйката на немски и италиански също може да се каже, че са „фактологично правилни“: немската дума Truthahn е комбинация от trut (смятана за имитация на звука, с която пуйката вика малките си) и Hahn (‘петел’), докато италианската tacchino е формирана чрез добавяне на умалителната наставка -ino към ономатопеичния корен. По-старите наименования и на тези езици обаче са следвали моделите, споменати по-горе в текста: на немски пуйката се е наричала Calecutischerhahn, а на италиански тя е била известна като gallo d’India.
В рубриката „От дума на дума“ Екатерина Петрова търси актуални, интересни или новопоявили се думи от нашето ежедневие и проследява често изненадващия им произход, развитието на значенията им във времето и взаимовръзките им с близки и далечни езици.
На север: Снежни лета в една исландска хижа (първа част)
Post Syndicated from Светла Стоянова original https://www.toest.bg/na-sever-snezhni-leta-v-edna-islandska-hizha-purva-chast/
Ето че плаването на север ни отвежда и до Исландия. Тя е все по-приканваща и загадъчна, но същевременно и все по-посещавана и разгадавана. С население от едва 395 000 души (2024 г.) и поток от туристи от над 2 млн. годишно, Исландия е своеобразен феномен. За територия, малко по-малка от тази на България, разнообразието от природни забележителности е впечатляващо. Този остров по средата на Северния Атлантически океан е създаден от хиляди вулканични изригвания между постоянно раздалечаващите се Евразийска и Северноамериканска тектонска плоча, така че
страната практически се уголемява с по 2 см годишно.
През последните години Исландия попада в новините у нас заради зачестилите вулканични изригвания. Но този остров не е само бълбукаща или изстинала лава, а много повече. В това се уверих през няколкото години, в които живях там, като учих, работих, къмпингувах и правих разнообразни преходи с близките ми хора. Тук ще ви разкажа за едно особено място далеч в исландските планини, което беше мой дом в продължение на две снежни лета. Но най-напред нека си го представим…
Живея и работя в зелена хижа с червен покрив, построена насред блещукаща лава отпреди няколко хилядолетия. Земята тук е съчетание от ситен пясък и едри парчета лъскаво черно стъкло – вълшебния многоъгълен обсидиан. Хижата се намира в средата на огромна калдера с диаметър около 12 км – преди милиони години на това място е имало планина, която след освобождаването на магмата изпод основите ѝ постепенно се вдлъбва в котловина.
Близо до хижата бушуват горещи извори, които я затоплят чрез специална система. Водата от чешмата се носи от ручей топящ се сняг, който препуска по многоцветните камъни. Потокът се слива със себеподобните си в пълноводна река, която все по-стремително си проправя път сред тесни отвесни каньони и вае страните им, докато не се разлее нашироко по заоблените скали в равнината.
В далечината се белеят няколко ледници столетници, отразяват слънчевите лъчи и ослепително искрят. От краищата им зеят ледени пещери и приканват в тъмните си коридори, проблясващи загадъчно от пукнатините. Ледът на входовете обаче непрестанно се топи и капе заплашително. В следващия миг се срутва на огромни късове и се предава пред горещите извори наблизо. Нажежена в земните недра, водата извира и клокочи в яркочервено и сивкавосиньо, оцветява в контрастни багри заобикалящите я камъни и чертае абстрактни окръжности. Земята кипи от живот, изпуска топлина, диша и издиша, докато водата и вятърът дълбаят формите ѝ с нестихваща сила.
Водопади се спускат шумно един подир друг, смесват ледените и жарките извори и вият меандри измежду оранжевите скали. Стръмни чукари се издигат в причудливи форми и отдалеч напомнят същества, застинали във времето. Яркозелени мъхове омекотяват като възглавнички нащърбения релеф на лавата, разстилат се навред и уютно приканват за почивка. Тук-там сред вулканичната пустиня зеленеят оскъдни поляни, на които си почиват овце. Необезпокоявани, те опоскват малкото жълти цветя наоколо, докато черно-бели снежни овесарки любопитстват, чуруликат и отлитат.
Горещ извор / Исландски мъх / Ледена пещера / Меандри / Слоеве лед и вулканичен прах / Релеф от риолитни скали © Светла Стоянова
Такъв спектакъл представяше природата, но често гъстата мъгла спускаше завесите и скриваше всичко от погледите ни. Подобно време е типично за исландските планински земи (исл. Hálendið, англ. Highlands): ненаселени и труднопроходими райони във вътрешната част на острова, с достъп чрез неравни планински пътища, отворени само през лятото. Населените райони на страната са основно по крайбрежието, защото океанът поддържа сравнително устойчива температура през годината. Затова и големите градове, като столицата Рейкявик и Акурейри, са разположени до морския бряг.
Всичко останало отвъд заселената част са яркозелени планини без гори, безкрайни черни пустини без камили и пълноводни ледникови реки без мостове. Неведнъж
тези почти извънземни места са използвани от НАСА за трениране на космонавти за стъпването на Луната през 60-те години, а само преди няколко години беше изпробван и прототип на марсоход за бъдещи мисии на Марс.
Именно там устояват на времето няколко скромни хижи на Исландския туристически съюз, достъпни само от юни до септември. В една от тях работих заедно с приятеля ми в продължение на два сезона. Нашата хижа е част от най-популярния няколкодневен планински преход в Исландия – Лойгавегюр (исл. Laugavegur – словосъчетание от laug ‘горещ извор’ и vegur ‘път’, или в превод „Пътят на горещите извори“).
Подобно на преходите Ел Камино де Сантяго в Испания или Ком – Емине у нас, се върви от хижа на хижа или се нощува в палатка на определените за целта места. Преходът от местността Ландманалойгар до Тоурсмьорк е 55 км, но може да бъде удължен с още 25 км, преминаващи между ледниците Мирдалсьокутъл и Ейяфятлайокутъл¹ до популярния водопад Скоугафос. Макар да не впечатлява с надморската си височина (400–1100 м), теренът се води планински. В по-голямата си част земната маса се състои от различни видове вулканични скали – риолит, обсидиан, базалт и др. Растителността е оскъдна, срещат се предимно мъхове, но и някои видове ниски растения.
Рядко срещани растения / Пейзажът във „Фятлабак“ – Реми Мартинаш © Светла Стоянова
Единствените диви животни са арктическите лисици, с черна козина през лятото и бяла през зимата, както и няколко вида птици, като снежните овесарки, тундровите яребици и гарваните. Други животни, на които човек може да попадне в изобилие, са овцете – те принадлежат на местните фермери, които лете ги пускат да пасат на воля по дивите места, докато не ги приберат отново наесен. Тази практика е типична за исландците, но и за други северни народи, като фарьорците, където по-големи хищници на практика няма.
Околността е природен парк, наречен „Фятлабак“ (исл. Fjallabak, в превод „Зад гърба на планините“) – геотермално активен, богат на горещи извори и често разтърсван от безобидни земетресения. Едно нещо обаче няма – дървета – и така гледката се простира на десетки километри във всички посоки. Освен това през юни и юли дните са дълги и нощем не се стъмва напълно, тъй като Исландия е досами Полярния кръг. Същинските бели нощи, когато слънцето не се скрива зад хоризонта, започват от 66-тия паралел нататък. Нощите са бели не само заради светлината навън.
Макар да е краят на юни, вали сняг на парцали хоризонтално – типично за Исландия. Едрите снежинки се наслояват по прозореца, докато не затулят изцяло „зрелището“ към пристигащите в хижата. Малкото звънче на вратата зазвънява и туристите влизат грохнали и с жвакащи обувки. Въздъхват, съвземат се от топлината и изстискват подгизналите си ръкавици. Били планирали да палаткуват тази вечер, но на бърза ръка решават да платят четворната цена за място в хижата. Приемам ги и си мисля как предния ден семейство с деца на по 4 и 6 години спаха на палатка в снега и бяха дори въодушевени от това приключение. Същевременно обаче си спомням ясно как при първото ми идване тук преди няколко години и аз бях една от тях – брулена от вятъра, капнала от газенето в кишата, мокра до кости и в отчаяна нужда да се приютя някъде на сухо и топло.
На разходка / Снеговалеж през август / Табела на прехода Лойгавегюр / Еднокилометров риолитен ръб, по който вървят туристи, а понякога и овце © Светла Стоянова
Когато предприехме прехода Лойгавегюр през 2020 г., имаше метри сняг. В продължение на шест дни вървяхме по няколко часа и преминавахме от един пейзаж в друг, като от картичка. Бяхме избрали да палаткуваме, така че на гръб носехме раници с всичко нужно: палатка, постелки, спални чували за ниски температури, тенджерка, котлон и газова бутилка, суха храна и сухи дрехи. Това беше първият ми преход, през който пренасях къщичката си на гръб. От ден на ден срещите с всеки хижар ни вълнуваха все повече. Да посрещаш и изпращаш туристите и да оставаш в онези прекрасни места изглеждаше като мечта. И ето че няколко години по-късно тя се сбъдна.
(Следва продължение.)
1 Ейяфятлайокутъл е име на ледник в Южна Исландия, но и на вулкана под него, който изригна през 2010 г. и предизвика отмяната на хиляди полети в Европа.
The Titan II ICBM
Post Syndicated from The History Guy: History Deserves to Be Remembered original https://www.youtube.com/watch?v=iappfoeSXqc
[$] LWN.net Weekly Edition for November 21, 2024
Post Syndicated from corbet original https://lwn.net/Articles/998144/
The LWN.net Weekly Edition for November 21, 2024 is available.
Introducing Amazon CloudFront VPC origins: Enhanced security and streamlined operations for your applications
Post Syndicated from Matheus Guimaraes original https://aws.amazon.com/blogs/aws/introducing-amazon-cloudfront-vpc-origins-enhanced-security-and-streamlined-operations-for-your-applications/
I’m happy to introduce the release of Amazon CloudFront Virtual Private Cloud (VPC) origins, a new feature that enables content delivery from applications hosted in private subnets within their Amazon Virtual Private Cloud (Amazon VPC). This makes it easy to secure web applications, allowing you to focus on growing your businesses while improving security and maintaining high-performance and global scalability with CloudFront.
Customers serving content from Amazon Simple Storage Solution (Amazon S3), AWS Elemental Services and AWS Lambda Function URLs can use Origin Access Control as a managed solution to secure their origins, and make CloudFront the single front-door to your application. However, this was more difficult to achieve for applications that are hosted on Amazon Elastic Compute Cloud (Amazon EC2) or using load balancers, because you had to create your own solution to achieve the same result. You would have to use a combination of methods such as using access control lists (ACLs), managing firewall rules, or using logic such as header validation and a few other techniques to ensure that the endpoint remained exclusive to CloudFront.
CloudFront VPC origins removes the need for this kind of undifferentiated work by offering a managed solution that can be used to point CloudFront distributions directly to Application Load Balancers (ALBs), Network Load Balancers (NLBs), or EC2 instances inside your private subnets. This ensures that CloudFront becomes the sole ingress point for those resources with minimum configuration effort, providing you with improved performance and a cost-saving opportunity because it also eliminates the need for public IP addresses.
Configuring a CloudFront VPC origin
CloudFront VPC origins is available at no additional cost, making it an accessible option for all AWS customers. It can be integrated with new or existing CloudFront distributions using the Amazon CloudFront console or the AWS Command Line Interface (AWS CLI).
Imagine that you have an application hosted privately on an AWS Fargate for Amazon ECS fronted through an ALB. Let’s create a CloudFront distribution that uses the ALB directly inside the private subnet.
Start by navigating to the CloudFront console and select the new menu option: VPC origins.
Creating a new VPC origin is straightforward. You only need to select from a few options. In the Origin ARN, you can search for available resources that are hosted in private subnets or enter it directly. You select the resources that you want, choose a friendly name for your VPC origin alongside some security options, and then confirm. Please note that, at launch, the VPC origin resource must be in the same AWS Account as the CloudFront distribution, although support for resources across all accounts is coming soon.
After the creation process is complete, your VPC origin will be deployed and ready to go! You can check its status on the VPC origins page.
With this, we have created a CloudFront distribution that serves content directly from a resource hosted on a private subnet in a few clicks! After your VPC origin is created, you can navigate to your Distribution window, and add the VPC origin to your Distribution by either selecting the ARN from the dropdown or copy-pasting the ARN manually.
Remember, though, that it’s important to still continue to layer your application’s security by using services such as AWS Web Application Firewall (WAF) to protect from web exploits, or AWS Shield for managed DDos protection, and other services to achieve a full-spectrum protection.
Conclusion
CloudFront VPC Origins offers a new way for organizations to deliver secure, high-performance applications by enabling CloudFront distributions to serve content directly from resources hosted within private subnets. This reduces the complexity and cost of maintaining public-facing origins while ensuring that your application remains secure.
To learn more, see the getting started guide.
Amazon CloudFront now accepts your applications’ gRPC calls
Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/amazon-cloudfront-now-accepts-your-applications-grpc-calls/
Starting today, you can deploy Amazon CloudFront, our global content delivery network (CDN), in front of your gRPC API endpoints.
gRPC is a modern, efficient, and language-agnostic framework for building APIs. It uses Protocol Buffers (protobuf) as its interface definition language (IDL), which enable you to define services and message types in a platform-independent manner. With gRPC, communication between services is achieved through lightweight and high-performance remote procedure calls (RPCs) over HTTP/2. This promotes efficient and low-latency communication across services, making it ideal for microservices architectures.
gRPC offers features such as bidirectional streaming, flow control, and automatic code generation for a variety of programming languages. It’s well-suited for scenarios in which you require high performance, efficient communication, and real-time data streaming. If your application needs to handle a large amount of data or requires low-latency communication between client and server, gRPC can be a good choice. However, gRPC might be more challenging to learn compared to REST. For example, gRPC relies on the protobuf serialization format, which requires developers to define their data structures and service methods in .proto files.
I see two benefits of deploying CloudFront in front of your gRPC API endpoints.
First, it allows the reduction of latency between the client application and your API implementation. CloudFront offers a global network of over 600+ edge locations with intelligent routing to the closest edge. Edge locations provide TLS termination and optional caching for your static content. CloudFront transfers client application requests to your gRPC origin through the fully managed, low-latency, and high-bandwidth private AWS network.
Secondly, your applications benefit from additional security services deployed on edge locations, such as traffic encryption, the validation of the HTTP headers through AWS Web Application Firewall, and AWS Shield Standard protection against distributed denial of service (DDoS) attacks.
Let’s see it in action
To start this demo, I use the gRPC route-guide demo from the official gRPC code repository. I deploy this example application in a container for ease of deployment (but any other deployment option is supported too).
I use this Dockerfile
FROM python:3.7
RUN pip install protobuf grpcio
COPY ./grpc/examples/python/route_guide .
CMD python route_guide_server.py
EXPOSE 50051I also use the AWS Copilot command line to deploy my container on Amazon Elastic Container Service (Amazon ECS). The Copilot command prompts me to collect the information it requires to build and deploy the container. Then, it creates the ECS cluster, the ECS service, and the ECS task automatically. It also creates a TLS certificate and the load balancer for me. I test the client application by modifying line 122 to use the DNS name of the load balancer listener endpoint. I also change the client application code to use grpc.secure_channel instead of grpc.insecure_channel because the load balancer provides the application with an HTTPS endpoint.
When I’m confident my API is correctly deployed and working, I proceed and configure CloudFront.
First, in the CloudFront section of the AWS Management Console, I select Create distribution.
Under Origin, I enter my gRPC endpoint DNS name as Origin domain. I enable HTTPS only as Protocol and leave the HTTPS port as is (443). Then I choose a Name for the distribution.
Under Viewer, I select HTTPS only as Viewer protocol policy. Then, I select GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE as Allowed HTTP methods. I select Enable for Allow gRPC requests over HTTP/2.
Under Cache key and origin requests, I select AllViewer as Origin request policy.
The default cache policy is CacheOptimized, but gRPC isn’t cacheable API traffic. Therefore, I select CachingDisabled as Cache policy.
AWS WAF helps protect you against common web exploits and bots that can affect availability, compromise security, or consume excessive resources. For gRPC traffic, AWS WAF can inspect the HTTP headers of the request and enforce access control. It doesn’t inspect the request body in protobuf format.
For this demo, I choose to not use AWS WAF. Under Web Application Firewall (WAF), I select Do not enable security protections.
I also keep all the other options with their default value. HTTP/2 support is selected by default. Do not disable it because it is required for gRPC.
Finally, I select Create distribution.
There is only one switch to enable gRPC on top of the usual setup. When turned on, with HTTP/2 and HTTP POST enabled, CloudFront detects gRPC client traffic and forwards it to your gRPC origin.
After a few minutes, the distribution is ready. I copy and paste the endpoint URL of the CloudFront distribution, and I change the client-side app to make it point to CloudFront instead of the previously created load balancer.
I test the application again, and it works.
Pricing and Availability
gRPC origins are available on all the more than 600 CloudFront edge locations at no additional cost. The usual requests and data transfer fees apply.
Go and point your CloudFront origin to a gRPC endpoint today.
Bigger and badder: how DDoS attack sizes have evolved over the last decade
Post Syndicated from José Salvador original https://blog.cloudflare.com/bigger-and-badder-how-ddos-attack-sizes-have-evolved-over-the-last-decade
Distributed Denial of Service (DDoS) attacks are cyberattacks that aim to overwhelm and disrupt online services, making them inaccessible to users. By leveraging a network of distributed devices, DDoS attacks flood the target system with excessive requests, consuming its bandwidth or exhausting compute resources to the point of failure. These attacks can be highly effective against unprotected sites and relatively inexpensive for attackers to launch. Despite being one of the oldest types of attacks, DDoS attacks remain a constant threat, often targeting well-known or high traffic websites, services, or critical infrastructure. Cloudflare has mitigated over 14.5 million DDoS attacks since the start of 2024 — an average of 2,200 DDoS attacks per hour. (Our DDoS Threat Report for Q3 2024 contains additional related statistics).
If we look at the metrics associated with large attacks mitigated in the last 10 years, does the graph show a steady increase in an exponential curve that keeps getting steeper, especially over the last few years, or is it closer to linear growth? We found that the growth is not linear, but rather is exponential, with the slope dependent on the metric we are looking at.
Why is this question interesting? Simple. The answer to it provides valuable insights into the evolving strategies of attackers, the sophistication of their tools, and the readiness of defense mechanisms.
As an example, an upward curve of the number of requests per second (rps) suggests that the attackers are changing something on their side that enables them to generate larger volumes of requests. This is an insight that prompts us to investigate more and look at other data to understand if anything new is happening.
For instance, at one of those moments, we looked at the source of the traffic and saw a shift from subscriber/enterprise IP address space (suggesting IoT) to cloud provider IP address space (suggesting VMs), and realized there was a shift in the type and capabilities of devices used by attackers.
As another example: when the HTTP/2 Rapid Reset attack happened, the record number of requests per second seen at that time suggested that a new technique was being employed by attackers, prompting us to swiftly investigate what was being executed and adapt our defenses.
Delimiting an individual attack in time is surprisingly blurry. First of all, an attack analysis can provide inconsistent observations at different layers of the OSI model. The footprint seen at all these different layers may tell different stories for the same attack. There are, however, some variables that together can allow us to create a fingerprint and enable us to group a set of events, establishing that they are part of the same individual attack. Examples include:
-
Do we see the same attack vector(s) being used across this set of events?
-
Are all the attack events focused on the same target(s)?
-
Do the payloads on events share the same signature? (Specific data payloads or request types unique to certain types of attacks or botnets, like Mirai, which may use distinctive HTTP request headers or packet structures).
Before we dive into a growth analysis of DDoS attacks over the last 10 years, let’s take a step back and have a look at the metrics typically used to measure them: requests per second (rps), packets per second (pps), and bits per second (bps). Each metric captures a different aspect of the attack’s scale and impact.
-
Requests per second (rps): Measures the number of HTTP or similar protocol requests made each second. This metric is particularly relevant for application-layer attacks (Layer 7), where the intent is to overwhelm a specific application or service by overloading its request handling, and is useful for measuring attacks targeting web servers, APIs, or applications because it reflects the volume of requests, not just raw data transfer.
-
Packets per second (pps): Represents the number of individual packets sent to the target per second, regardless of their size. This metric is critical for network-layer attacks (Layers 3 and 4), where the goal is to overwhelm network infrastructure by exceeding its packet-processing capacity. pps measurements are useful for volumetric attacks, identifying a quantity of packets that can impact routers, switches, or firewalls.
-
Bits per second (bps): This measures the total data transferred per second and is especially useful in evaluating network-layer attacks that aim to saturate the bandwidth of the target or its upstream provider. bps is widely used measuring Layer 3 and 4 attacks, such as UDP floods, where the attack intends to clog network bandwidth. This metric is often highlighted for DDoS attacks because high bps values (often measured in gigabits or terabits) signal bandwidth saturation, which is a common goal of large-scale DDoS campaigns.
So, how have DDoS attack sizes changed in the last decade? During this period, DDoS attacks have grown bigger and stronger, each year having the potential to be more disruptive.
If we look at the metrics associated with large attacks seen in the last 10 years, does it look like we have a steady increase in an exponential curve that keeps steepening, especially in the last few years, or is it closer to a linear growth? We found that it is exponential, so let’s have a look at the details around why we came to that conclusion.
In this analysis, we used attacks that Google has seen from 2010 until 2022 as a baseline (Figure 1) that we extended with attacks that Cloudflare has seen in 2023 and 2024 (Figure 2).
Going back in time, early in the 2010s, the largest attacks were measured in the Gigabits per second (Gbps) scale, but these days, it’s all about Terabits per second (Tbps). The number of requests per second (rps) and bits per second (bps) are also significantly higher these days, as we will see.
The historical data from Google shown below in Figure 1 reveals a rising trend in requests per second during DDoS attacks observed between 2010 and 2022, peaking at 6 Million requests per second (Mrps) in 2020. The increase highlights a significant escalation in attack volume across the decade.
Figure 1. Largest known DDoS attacks, 2010 – 2022. (Source: Google)
Figure 2 (below) provides a view of trends seen across the different metrics. The escalation seen in Google’s statistics is also visible in Cloudflare’s data regarding large mitigated DDoS attacks observed in 2023 and 2024, reaching 201 Mrps (green line) in September 2024. The rate of packets per second (pps) demonstrates (blue line) a slight exponential growth over time, rising from 230 Mpps in 2015 to 2,100 Mpps in 2024, suggesting that attackers are achieving higher throughput. For bits per second (bps), the trend is also exponential and with a steeper upwards curve (red line), building from a 309 Gbps attack in 2013 to a 5.6 Tbps (5,600 Gbps) attack in 2024.
Over roughly the last decade, attacks driving these metrics have seen significant growth rates:
-
Bits per second increased by 20x between 2013 and 2024
-
Packets per second increased by 10x between 2015 and 2024
-
Requests per second increased by 70x between 2014 and 2024
Figure 2. Data from Figure 1 extended with large attacks observed by Cloudflare in 2023 and 2024.
The blog posts listed in Table 1 highlight some of the attacks that we observed from 2021 to 2024.
|
Month |
Attack size |
Blog post |
|
August 2021 |
17.2 Mrps |
Cloudflare thwarts 17.2M rps DDoS attack — the largest ever reported |
|
April 2022 |
15 Mrps |
|
|
June 2022 |
26 Mrps |
Cloudflare mitigates 26 million request per second DDoS attack |
|
February 2023 |
71 Mrps |
Cloudflare mitigates record-breaking 71 million request-per-second DDoS attack |
|
September 2024 |
3.8 Tbps |
How Cloudflare auto-mitigated world record 3.8 Tbps DDoS attack |
|
October 2024 |
4.2 Tbps |
4.2 Tbps of bad packets and a whole lot more: Cloudflare’s Q3 DDoS report |
|
October 2024 |
5.6 Tbps |
Table 1. Notable DDoS attacks observed by Cloudflare between 2021 – 2024.
An overview of other selected significant high volume DDoS attacks that have occurred over the last decade, including 2018’s Memcached abuse and 2023’s HTTP/2 “Rapid Reset” attacks, can be found on the Cloudflare Learning Center.
Attack duration is not an effective metric to use to qualify attack aggressiveness because establishing a duration of a single attack or campaign is challenging, due to their possible intermittent nature, the potential for a multitude of attack vectors being used at the same time, or how the different defense layers triggered over time.
The attack patterns can differ considerably, with some consisting of a single large spike, while others featuring multiple tightly grouped spikes, or a continuous load maintained over a period of time, along with other changing characteristics.
DDoS attacks are increasingly shifting from IoT-based botnets to more powerful VM-based botnets. This change is primarily due to the higher computational and throughput capabilities of cloud-hosted virtual machines, which allow attackers to launch massive attacks with far fewer devices.
This shift is facilitated by several factors: VM botnets can be easier to establish than IoT botnets, as they don’t necessarily require widespread malware infections, since attackers can deploy them on cloud provider infrastructure anonymously using stolen payment details from data breaches or Magecart attacks.
This trend points to the evolution of DDoS tactics, as attackers exploit both the processing power of VMs and anonymized access to cloud resources, enabling smaller, more efficient botnets capable of launching large-scale attacks without the complexities involved in infecting and managing fleets of IoT devices.
Cloudflare’s Connectivity Cloud, built on our expansive anycast global network, plays a crucial role in defending against DDoS attacks by leveraging automated detection, traffic distribution, and rapid response capabilities. Here’s how it strengthens DDoS protection:
Automated attack detection and mitigation: Cloudflare’s DDoS protection relies heavily on automation, using machine learning algorithms to identify suspicious traffic patterns in real time. By automating the detection process, Cloudflare can quickly recognize and block DDoS attacks without requiring manual intervention, which is critical in high-volume attacks that would overwhelm human responders.
Global traffic distribution with IP anycast: Cloudflare’s network spans over 330 cities worldwide, and DDoS traffic gets distributed across our multiple data centers. IP anycast allows us to distribute traffic across this global network, and this wide distribution helps absorb and mitigate large-scale attacks, as attack traffic is not directed towards a single point, reducing strain on individual servers and networks.
Layered defense: Cloudflare’s Connectivity Cloud offers defense across multiple layers, including network (Layer 3), transport (Layer 4), and application (Layer 7). This layered approach allows for tailored defense strategies depending on the attack type, ensuring that even complex, multi-layered attacks can be mitigated effectively. Learn more about DDoS protection at layers 3, 4, and 7 in our DDoS protection documentation.
Unmetered DDoS mitigation: Pioneering this approach since 2017 to ensure Internet security, Cloudflare provides unmetered DDoS protection, meaning customers are protected without worrying about bandwidth or cost limitations during attacks. This approach helps ensure that businesses, regardless of size or budget, can benefit from robust DDoS protection.
Cloudflare’s distributed cloud infrastructure and advanced technology allows us to detect, absorb, and mitigate DDoS attacks in a way that is both scalable and responsive, avoiding downtime and maintaining service reliability, providing a robust solution to tackle the rising intensity and frequency of DDoS attacks compared to traditional options.
Protecting against DDoS attacks is essential for organizations of every size. Although humans initiate these attacks, they’re carried out by bots, so effective defense requires automated tools to counter bot-driven threats. Real-time detection and mitigation should be as automated as possible, since relying solely on human intervention puts defenders at a disadvantage as attackers adapt to new barriers and can change attack vectors, traffic behavior, payload signatures, among others, creating an unpredicted scenario and thus rendering some manual configurations useless. Cloudflare’s automated systems continuously identify and block DDoS attacks on behalf of our customers, enabling tailored protection that meets individual needs.
Our mission is to help build a better Internet, and providing resilience in the face of DDoS threats is a part of accomplishing that mission.
Read more about Cloudflare DDoS protection in our public technical documentation.
Resilient Internet connectivity in Europe mitigates impact from multiple cable cuts
Post Syndicated from David Belson original https://blog.cloudflare.com/resilient-internet-connectivity-baltic-cable-cuts
When cable cuts occur, whether submarine or terrestrial, they often result in observable disruptions to Internet connectivity, knocking a network, city, or country offline. This is especially true when there is insufficient resilience or alternative paths — that is, when a cable is effectively a single point of failure. Associated observations of traffic loss resulting from these disruptions are frequently covered by Cloudflare Radar in social media and blog posts. However, two recent cable cuts that occurred in the Baltic Sea resulted in little-to-no observable impact to the affected countries, as we discuss below, in large part because of the significant redundancy and resilience of Internet infrastructure in Europe.
On Sunday, November 17 2024, the BCS East-West Interlink submarine cable connecting Sventoji, Lithuania and Katthammarsvik, Sweden was reportedly damaged around 10:00 local (Lithuania) time (08:00 UTC). A Data Center Dynamics article about the cable cut quotes the CTO of Telia Lietuva, the telecommunications provider that operates the cable, and notes “The Lithuanian cable carried about a third of the nation’s Internet capacity, but capacity was carried via other routes.”
As the Cloudflare Radar graphs below show, there was no apparent impact to traffic volumes in either country at the time that the cables were damaged. The NetFlows graphs represent the number of bytes that Cloudflare sends to users and clients in response to their requests.
Internet quality metrics for both countries show changes in measured bandwidth and latency throughout the day on Sunday, but with no sudden anomalous shifts visible around the time of the cable cut. (The loss of connectivity associated with a cable cut potentially manifests itself as an increase in latency and concurrent decrease in bandwidth due to loss of capacity.) The latency graph for Sweden does show an increase in latency, but it began before the cable cut occurred, is similar to a pattern visible several hours earlier, and is matched by an increase in measured bandwidth, so it is unlikely to be related to the cable cut event.
BGP announcements are a way for network providers to communicate routing information to other networks, and announcement activity observed on Telia Lietuva’s autonomous systems around the time of the cable cut may be related to the re-routing referenced in the article. No change in announced IP address space was visible for any of these autonomous systems, suggesting no loss of connectivity as the capacity was re-routed.
Telegeography’s submarinecablemap.com illustrates, at least in part, the resilience in connectivity enjoyed by these two countries. In addition to the damaged cable, it shows that Lithuania is connected to neighboring Latvia as well as to the Swedish mainland. Over 20 submarine cables land in Sweden, connecting it to multiple countries across Europe. In addition to the submarine resilience, network providers in both countries can take advantage of terrestrial fiber connections to neighboring countries, such as those illustrated in a European network map from Arelion (formerly Telia), which is only one of the large European backbone providers.
Less than a day later, the C-Lion1 submarine cable, which connects Helsinki, Finland and Rostock Germany was reportedly damaged during the early morning hours of Monday, November 18. Cinia, the telecommunications company that owns the cable, said that the cable stopped working at about 02:00 UTC.
In this situation as well, as the Cloudflare Radar graphs below show, there was no apparent impact to traffic volumes in either country at the time that the cables were damaged. The Finland graphs, week-on-week, show fewer bytes transferred and fewer HTTP requests, but that difference is present before the cable cut at 02:00 UTC. However, the trend of the current line does not change after the cable cut, so the two events would appear unrelated.
By looking at volume-related metrics, alone, Internet connectivity would appear to be unaffected by the cable cut.
If, however, we change perspective and look at Internet quality, a brief yet interesting change is visible for Finland around the reported time of the cable damage, though it isn’t clear whether it is related in any way. Just after midnight, median measured bandwidth, previously consistent around 50 Mbps begins to grow, peaking just over 200 Mbps around 03:00 UTC. Around that same time, measured median latency also begins to drop, falling from around 30 ms to a low of 13 ms, also around 03:00 UTC. Median bandwidth returned to normal levels around 06:00 UTC, while latency took about two hours longer to return to normal levels. These observed improvements in bandwidth and latency could have been due to traffic being re-routed to along paths with better connectivity to measurement endpoints, but because the shifts began before the cable damage occurred, and recovered shortly thereafter, that is unlikely to be the root cause.
In Germany, a brief minor increase in median bandwidth peaked around 02:45 UTC, while no notable changes were observed in latency.
From a routing perspective, there was no notable BGP announcement activity observed for top autonomous systems in either Finland or Germany around 02:00 on November 18, and total announced IP address space aggregated at a country level also demonstrated no change.
Telegeography’s submarinecablemap.com shows that both Finland and Germany also have significant redundancy and resilience from a submarine cable perspective, with over 10 cables landing in Finland, and nearly 10 landing in Germany, including Atlantic Crossing-1 (AC-1), which connects to the United States over two distinct paths. Terrestrial fiber maps from Arelion and eunetworks (as just two examples) show multiple redundant fiber routes within both countries, as well as cross-border routes to other neighboring countries, enabling more resilient Internet connectivity.
As we have discussed in multiple prior blog posts (Jersey, 2016; AAE-1/SMW5, 2022; WACS/MainOne/SAT3/ACE, 2024; EASSy/Seacom, 2024), cable cuts often cause significant disruptions to Internet connectivity, in many cases because they represent a concentrated point of vulnerability, whether for an individual network provider, city/state, or country. These disruptions are often quite lengthy as well, due to the time needed to marshal repair resources, identify the location of the damage, etc. Although it is not always feasible due to financial or geographic constraints, building redundant and resilient network architecture, at multiple levels, is a best practice. This includes the sending traffic over multiple physical cables (both submarine and terrestrial), connecting to multiple peer and upstream network providers, and even avoiding single points of failure in core Internet resources like DNS servers.
The Cloudflare Radar team continually monitors the status of Internet connectivity in countries/regions around the world, and we share our observations on the Cloudflare Radar Outage Center, via social media, and in posts on blog.cloudflare.com. Follow us on social media at @CloudflareRadar (X), https://noc.social/@cloudflareradar (Mastodon), and radar.cloudflare.com (Bluesky), or contact us via email.