Yearly Archives: 2024

Build a dynamic rules engine with Amazon Managed Service for Apache Flink

Post Syndicated from Steven Carpenter original https://aws.amazon.com/blogs/big-data/build-a-dynamic-rules-engine-with-amazon-managed-service-for-apache-flink/

Imagine you have some streaming data. It could be from an Internet of Things (IoT) sensor, log data ingestion, or even shopper impression data. Regardless of the source, you have been tasked with acting on the data—alerting or triggering when something occurs. Martin Fowler says: “You can build a simple rules engine yourself. All you need is to create a bunch of objects with conditions and actions, store them in a collection, and run through them to evaluate the conditions and execute the actions.”

A business rules engine (or simply rules engine) is a software system that executes many rules based on some input to determine some output. Simplistically, it’s a lot of “if then,” “and,” and “or” statements that are evaluated on some data. There are many different business rule systems, such as Drools, OpenL Tablets, or even RuleBook, and they all share a commonality: they define rules (collection of objects with conditions) that get executed (evaluate the conditions) to derive an output (execute the actions). The following is a simplistic example:

if (office_temperature) < 50 degrees => send an alert

if (office_temperature) < 50 degrees AND (occupancy_sensor) == TRUE => < Trigger action to turn on heat>

When a single condition or a composition of conditions evaluates to true, it is desired to send out an alert to potentially act on that event (trigger the heat to warm the 50 degrees room).

This post demonstrates how to implement a dynamic rules engine using Amazon Managed Service for Apache Flink. Our implementation provides the ability to create dynamic rules that can be created and updated without the need to change or redeploy the underlying code or implementation of the rules engine itself. We discuss the architecture, the key services of the implementation, some implementation details that you can use to build your own rules engine, and an AWS Cloud Development Kit (AWS CDK) project to deploy this in your own account.

Solution overview

The workflow of our solution starts with the ingestion of the data. We assume that we have some source data. It could be from a variety of places, but for this demonstration, we use streaming data (IoT sensor data) as our input data. This is what we will evaluate our rules on. For example purposes, let’s assume we are looking at data from our AnyCompany Home Thermostat. We’ll see attributes like temperature, occupancy, humidity, and more. The thermostat publishes the respective values every 1 minute, so we’ll base our rules around that idea. Because we’re ingesting this data in near real time, we need a service designed specifically for this use case. For this solution, we use Amazon Kinesis Data Streams.

In a traditional rules engine, there may be a finite list of rules. The creation of new rules would likely involve a revision and redeployment of the code base, a replacement of some rules file, or some overwriting process. However, a dynamic rules engine is different. Much like our streaming input data, our rules can also be streamed as well. Here we can use Kinesis Data Streams to stream our rules as they are created.

At this point, we have two streams of data:

  • The raw data from our thermostat
  • The business rules perhaps created through a user interface

The following diagram illustrates we can connect these streams together.Architecture Diagram

Connecting streams

A typical use case for Managed Service for Apache Flink is to interactively query and analyze data in real time and continuously produce insights for time-sensitive use cases. With this in mind, if you have a rule that corresponds to the temperature dropping below a certain value (especially in winter), it might be critical to evaluate and produce a result as timely as possible.

Apache Flink connectors are software components that move data into and out of a Managed Service for Apache Flink application. Connectors are flexible integrations that let you read from files and directories. They consist of complete modules for interacting with AWS services and third-party systems. For more details about connectors, see Use Apache Flink connectors with Managed Service for Apache Flink.

We use two types of connectors (operators) for this solution:

  • Sources – Provide input to your application from a Kinesis data stream, file, or other data source
  • Sinks – Send output from your application to a Kinesis data stream, Amazon Data Firehose stream, or other data destination

Flink applications are streaming dataflows that may be transformed by user-defined operators. These dataflows form directed graphs that start with one or more sources and end in one or more sinks. The following diagram illustrates an example dataflow (source). As previously discussed, we have two Kinesis data streams that can be used as sources for our Flink program.

Flink Data Flow

The following code snippet shows how we have our Kinesis sources set up within our Flink code:

/**
* Creates a DataStream of Rule objects by consuming rule data from a Kinesis
* stream.
*
* @param env The StreamExecutionEnvironment for the Flink job
* @return A DataStream of Rule objects
* @throws IOException if an error occurs while reading Kinesis properties
*/
private DataStream<Rule> createRuleStream(StreamExecutionEnvironment env, Properties sourceProperties)
                throws IOException {
        String RULES_SOURCE = KinesisUtils.getKinesisRuntimeProperty("kinesis", "rulesTopicName");
        FlinkKinesisConsumer<String> kinesisConsumer = new FlinkKinesisConsumer<>(RULES_SOURCE,
                        new SimpleStringSchema(),
                        sourceProperties);
        DataStream<String> rulesStrings = env.addSource(kinesisConsumer)
                        .name("RulesStream")
                        .uid("rules-stream");
        return rulesStrings.flatMap(new RuleDeserializer()).name("Rule Deserialization");
}

/**
* Creates a DataStream of SensorEvent objects by consuming sensor event data
* from a Kinesis stream.
*
* @param env The StreamExecutionEnvironment for the Flink job
* @return A DataStream of SensorEvent objects
* @throws IOException if an error occurs while reading Kinesis properties
*/
private DataStream<SensorEvent> createSensorEventStream(StreamExecutionEnvironment env,
            Properties sourceProperties) throws IOException {
    String DATA_SOURCE = KinesisUtils.getKinesisRuntimeProperty("kinesis", "dataTopicName");
    FlinkKinesisConsumer<String> kinesisConsumer = new FlinkKinesisConsumer<>(DATA_SOURCE,
                    new SimpleStringSchema(),
                    sourceProperties);
    DataStream<String> transactionsStringsStream = env.addSource(kinesisConsumer)
                    .name("EventStream")
                    .uid("sensor-events-stream");

    return transactionsStringsStream.flatMap(new JsonDeserializer<>(SensorEvent.class))
                    .returns(SensorEvent.class)
                    .flatMap(new TimeStamper<>())
                    .returns(SensorEvent.class)
                    .name("Transactions Deserialization");
}

We use a broadcast state, which can be used to combine and jointly process two streams of events in a specific way. A broadcast state is a good fit for applications that need to join a low-throughput stream and a high-throughput stream or need to dynamically update their processing logic. The following diagram illustrates an example how the broadcast state is connected. For more details, see A Practical Guide to Broadcast State in Apache Flink.

Broadcast State

This fits the idea of our dynamic rules engine, where we have a low-throughput rules stream (added to as needed) and a high-throughput transactions stream (coming in at a regular interval, such as one per minute). This broadcast stream allows us to take our transactions stream (or the thermostat data) and connect it to the rules stream as shown in the following code snippet:

// Processing pipeline setup
DataStream<Alert> alerts = sensorEvents
    .connect(rulesStream)
    .process(new DynamicKeyFunction())
    .uid("partition-sensor-data")
    .name("Partition Sensor Data by Equipment and RuleId")
    .keyBy((equipmentSensorHash) -> equipmentSensorHash.getKey())
    .connect(rulesStream)
    .process(new DynamicAlertFunction())
    .uid("rule-evaluator")
    .name("Rule Evaluator");

To learn more about the broadcast state, see The Broadcast State Pattern. When the broadcast stream is connected to the data stream (as in the preceding example), it becomes a BroadcastConnectedStream. The function applied to this stream, which allows us to process the transactions and rules, implements the processBroadcastElement method. The KeyedBroadcastProcessFunction interface provides three methods to process records and emit results:

  • processBroadcastElement() – This is called for each record of the broadcasted stream (our rules stream).
  • processElement() – This is called for each record of the keyed stream. It provides read-only access to the broadcast state to prevent modifications that result in different broadcast states across the parallel instances of the function. The processElement method retrieves the rule from the broadcast state and the previous sensor event of the keyed state. If the expression evaluates to TRUE (discussed in the next section), an alert will be emitted.
  • onTimer() – This is called when a previously registered timer fires. Timers can be registered in the processElement method and are used to perform computations or clean up states in the future. This is used in our code to make sure any old data (as defined by our rule) is evicted as necessary.

We can handle the rule in the broadcast state instance as follows:

@Override
public void processBroadcastElement(Rule rule, Context ctx, Collector<Alert> out) throws Exception {
   BroadcastState<String, Rule> broadcastState = ctx.getBroadcastState(RulesEvaluator.Descriptors.rulesDescriptor);
   Long currentProcessTime = System.currentTimeMillis();
   // If we get a new rule, we'll give it insufficient data rule op status
    if (!broadcastState.contains(rule.getId())) {
        outputRuleOpData(rule, OperationStatus.INSUFFICIENT_DATA, currentProcessTime, ctx);
    }
   ProcessingUtils.handleRuleBroadcast(rule, broadcastState);
}

static void handleRuleBroadcast(FDDRule rule, BroadcastState<String, FDDRule> broadcastState)
        throws Exception {
    switch (rule.getStatus()) {
        case ACTIVE:
            broadcastState.put(rule.getId(), rule);
            break;
        case INACTIVE:
            broadcastState.remove(rule.getId());
            break;
    }
}

Notice what happens in the code when the rule status is INACTIVE. This would remove the rule from the broadcast state, which would then no longer consider the rule to be used. Similarly, handling the broadcast of a rule that is ACTIVE would add or replace the rule within the broadcast state. This is allowing us to dynamically make changes, adding and removing rules as necessary.

Evaluating rules

Rules can be evaluated in a variety of ways. Although it’s not a requirement, our rules were created in a Java Expression Language (JEXL) compatible format. This allows us to evaluate rules by providing a JEXL expression along with the appropriate context (the necessary transactions to reevaluate the rule or key-value pairs), and simply calling the evaluate method:

JexlExpression expression = jexl.createExpression(rule.getRuleExpression());
Boolean isAlertTriggered = (Boolean) expression.evaluate(context);

A powerful feature of JEXL is that not only can it support simple expressions (such as those including comparison and arithmetic), it also has support for user-defined functions. JEXL allows you to call any method on a Java object using the same syntax. If there is a POJO with the name SENSOR_cebb1baf_2df0_4267_b489_28be562fccea that has the method hasNotChanged, you would call that method using the expression. You can find more of these user-defined functions that we used within our SensorMapState class.

Let’s look at an example of how this would work, using a rule expression exists that reads as follows:

"SENSOR_cebb1baf_2df0_4267_b489_28be562fccea.hasNotChanged(5)"

This rule, evaluated by JEXL, would be equivalent to a sensor that hasn’t changed in 5 minutes

The corresponding user-defined function (part of SensorMapState) that is exposed to JEXL (using the context) is as follows:

public Boolean hasNotChanged(Integer time) {
    Long minutesSinceChange = getMinutesSinceChange();
    log.debug("Time: " + time + " | Minutes since change: " + minutesSinceChange);
    return minutesSinceChange >  time;
}

Relevant data, like that below, would go into the context window, which would then be used to evaluate the rule.

{
    "id": "SENSOR_cebb1baf_2df0_4267_b489_28be562fccea",
    "measureValue": 10,
    "eventTimestamp": 1721666423000
}

In this case, the result (or value of isAlertTriggered) is TRUE.

Creating sinks

Much like how we previously created sources, we also can create sinks. These sinks will be used as the end to our stream processing where our analyzed and evaluated results will get emitted for future use. Like our source, our sink is also a Kinesis data stream, where a downstream Lambda consumer will iterate the records and process them to take the appropriate action. There are many applications of downstream processing; for example, we can persist this evaluation result, create a push notification, or update a rule dashboard.

Based on the previous evaluation, we have the following logic within the process function itself:

if (isAlertTriggered) {
    alert = new Alert(rule.getEquipmentName(), rule.getName(), rule.getId(), AlertStatus.START,
            triggeringEvents, currentEvalTime);
    log.info("Pushing {} alert for {}", AlertStatus.START, rule.getName());
}
out.collect(alert);

When the process function emits the alert, the alert response is sent to the sink, which then can be read and used downstream in the architecture:

alerts.flatMap(new JsonSerializer<>(Alert.class))
    .name("Alerts Deserialization").sinkTo(createAlertSink(sinkProperties))
    .uid("alerts-json-sink")
    .name("Alerts JSON Sink");

At this point, we can then process it. We have a Lambda function logging the records where we can see the following:

{
   "equipmentName":"THERMOSTAT_1",
   "ruleName":"RuleTest2",
   "ruleId":"cda160c0-c790-47da-bd65-4abae838af3b",
   "status":"START",
   "triggeringEvents":[
      {
         "equipment":{
            "id":"THERMOSTAT_1",
         },
         "id":"SENSOR_cebb1baf_2df0_4267_b489_28be562fccea",
         "measureValue":20.0,
         "eventTimestamp":1721672715000,
         "ingestionTimestamp":1721741792958
      }
   ],
   "timestamp":1721741792790
}

Although simplified in this example, these code snippets form the basis for taking the evaluation results and sending them elsewhere.

Conclusion

In this post, we demonstrated how to implement a dynamic rules engine using Managed Service for Apache Flink with both the rules and input data streamed through Kinesis Data Streams. You can learn more about it with the e-learning that we have available.

As companies seek to implement near real-time rules engines, this architecture presents a compelling solution. Managed Service for Apache Flink offers powerful capabilities for transforming and analyzing streaming data in real time, while simplifying the management of Flink workloads and seamlessly integrating with other AWS services.

To help you get started with this architecture, we’re excited to announce that we’ll be publishing our complete rules engine code as a sample on GitHub. This comprehensive example will go beyond the code snippets provided in our post, offering a deeper look into the intricacies of building a dynamic rules engine with Flink.

We encourage you to explore this sample code, adapt it to your specific use case, and take advantage of the full potential of real-time data processing in your applications. Check out the GitHub repository, and don’t hesitate to reach out with any questions or feedback as you embark on your journey with Flink and AWS!

About the Authors

Steven Carpenter is a Senior Solution Developer on the AWS Industries Prototyping and Customer Engineering (PACE) team, helping AWS customers bring innovative ideas to life through rapid prototyping on the AWS platform. He holds a master’s degree in Computer Science from Wayne State University in Detroit, Michigan. Connect with Steven on LinkedIn!

Aravindharaj Rajendran is a Senior Solution Developer within the AWS Industries Prototyping and Customer Engineering (PACE) team, based in Herndon, VA. He helps AWS customers materialize their innovative ideas by rapid prototyping using the AWS platform. Outside of work, he loves playing PC games, Badminton and Traveling.

Ransomware Groups Demystified: CyberVolk Ransomware

Post Syndicated from Rapid7 Labs original https://blog.rapid7.com/2024/10/03/ransomware-groups-demystified-cybervolk-ransomware/

Ransomware Groups Demystified: CyberVolk Ransomware

As part of our ongoing efforts to monitor emerging cyber threats, we have analyzed the activities of CyberVolk, a politically motivated hacktivist group that transitioned into using ransomware and has been active since June 2024. Unlike traditional ransomware groups, CyberVolk initially positioned itself as a hacktivist organization, and then started to use ransomware as a tool for retaliation. The group openly declares allegiance to Russia and operates within a broader hacktivist movement, launching attacks in response to geopolitical events. This report offers an in-depth analysis of CyberVolk’s ransomware tactics, underlying motivations, and technical behaviors.

Rapid7 Labs has an ongoing commitment to help organizations understand and mitigate risk from the complex world of ransomware, and this includes highlighting these newer groups. In this post we’re going to focus on CyberVolk’s shift from a hacktivist group to one that now uses ransomware as a key tool in its operations.

Intro to the CyberVolk group

CyberVolk emerged in June 2024 as a hacktivist group associated with pro-Russian activities. Before settling on its current identity it went through several name changes. Initially known as GLORIAMIST India on March 28, 2024, the group rebranded itself as Solntsevskaya Bratva on June 10, 2024. However, this name was short-lived, and on June 23, 2024, the group adopted the name CyberVolk. Their operations escalated after the arrest of members from the hacktivist group NoName57(16), known for targeting NATO-aligned countries. In response, CyberVolk, alongside more than 70 affiliated hacktivist groups, launched coordinated Distributed Denial of Service (DDoS) and ransomware attacks against Spain, which had arrested the NoName57(16) members. These attacks are part of a broader strategy to retaliate against governments opposing Russian interests.

Ransomware Groups Demystified: CyberVolk Ransomware
Figure 1: CyberVolk’s name rebranding form March-June 2024

CyberVolk uses a combination of ransomware and DDoS attacks to undermine their targets. Spanish institutions have been a primary focus, with 27 entities reportedly affected since the campaign began.

This isn’t the first time a hacktivist group has taken a stroll down the dark side. Just last year, we covered the GhostLocker group, which made an attempt to transition from the hacktivist realm to ransomware-as-a-service (RaaS). Side bar: their debut into the ransomware world didn’t exactly go as planned. After realizing that success in the RaaS game wasn’t in their best interest, they swiftly pivoted back to their old hacktivist ways, likely with a sigh of relief. But let’s go back to the CyberVolk (with “Volk” meaning “wolf” in Russian).

Technical analysis of CyberVolk ransomware

We analyzed a sample of the CyberVolk ransomware.

| SHA256 | 102276ae1f518745695fe8f291bf6e69856b91723244881561bb1a2338d54b12 |

CyberVolk follows a standard execution flow typical to ransomware strains. One of the first actions it takes is saving an image file tmp.bmp to C:\Users\USER\AppData\Local\Temp\tmp.bmp and changing the victim’s desktop wallpaper — interestingly, this occurs before any files on the system are encrypted.

The ransomware then creates multiple threads to handle various tasks, including:

  • User interaction: A thread manages the interaction with the victim, displaying dialog boxes for the ransom message, decryption key entry, and cryptocurrency payment options for BTC (Bitcoin) and USDT ERC20. The addresses used are:
  • BTC: bc1q3c9pt084cafxfvyhn8wvh7mq04rq6naew0mk87
  • USDT: TXarMAbSLLmStn4RZj63cTH7tpbodGNGbZ At the time of writing, the BTC wallet had a balance of 0, and the USDT wallet held 34.79 USDT.
  • Task manager monitoring: Another thread checks repeatedly if Task Manager is running by searching for a window with the class name “TaskManagerWindow.” If found, it attempts to kill the process by sending a WM_CLOSE message. This action requires the ransomware to run with escalated privileges.
  • File scanning and encryption: CyberVolk performs a systematic scan of all available drive letters (from a to z) to identify valid drives for encryption. Once the encryption routine is triggered:
  • Files on the infected system are encrypted and given the .cvenc extension.
  • The ransomware methodically scans directories and subdirectories, encrypting files as it proceeds.
  • Decryption key management: After encrypting the files, CyberVolk presents the victim with an interface to input a decryption key following ransom payment. Here’s how the decryption process works:
  • Key validation: The ransomware checks if the entered decryption key is exactly 36 characters long. However, despite the full key being 36 characters, only the first 16 characters are passed to a substitution function that transforms part of the key using a predefined substitution table.
  • Substitution function: The function processes multiple encrypted string arrays and performs character substitution based on a preset character set. It compares each character from the first 16 characters of the entered key with encrypted string arrays and replaces them using the substitution table.
  • Writing the key: The transformed output is written to a file named dec_key.dat, which is then used to complete the decryption process. If the decryption key passes all checks, the ransomware decrypts the files.
  • Cleanup: After successful decryption, it removes files like dec_key.dat and time.dat from C:\Users\USER\AppData\Roaming\ to cover its tracks.
Ransomware Groups Demystified: CyberVolk Ransomware
Figure 2: CyberVolk dialog window

Experiment: Decryption key testing with CyberVolk ransomware

As part of a small experiment, we attempted to execute the CyberVolk ransomware with a pre-created dec_key.dat file placed in C:\Users\USER\AppData\Roaming\. This file contained hardcoded strings we found in the code, such as fc99bb1c28a5ae006e567faf4cfc0d707c1528e and ce12f0967bd216d248cafda3d46ad1368d9f3dee.

Upon running the malware, the presence of the file successfully triggered the decryption routine. However, despite the original file names being restored, the files themselves were empty.

In another experiment, we manually entered 36 random characters into the decryption key dialog box. Again, this triggered the decryption process, and although the file names were restored, the files remained empty.

Additionally, the ransomware claims that it will delete files if an incorrect decryption key is entered. We tested this by entering an invalid key (aaaa). The malware displayed a warning, but when we proceeded, all files remained encrypted, and none were deleted.

Ransomware Groups Demystified: CyberVolk Ransomware
Figure 3: Correct key warning

CyberVolk’s decryption routine seems to have a weakness in its validation process, allowing it to proceed with decryption even with incorrect or random keys. However, without the correct key, the files are rendered unusable, suggesting that the key validation might only partially function or that the ransomware is designed to deceive victims into thinking decryption is occurring, when in reality, the files remain damaged. This could be a design flaw or a deliberate tactic to further frustrate victims.

The fact that files are not deleted as promised when an incorrect key is entered also indicates a discrepancy between the ransomware’s claims and its actual behavior. This could either be a design flaw or a deliberate tactic to further confuse and frustrate victims. Ultimately, even if the ransomware initiates decryption, without the correct key, files remain damaged and unusable.

Ransom note

After encryption, a file named CyberVolk_ReadMe.txt is placed in every affected folder. The ransom note contains the following message:

All your files have been encrypted by CyberVolk ransomware.
Do not attempt to recover your files without the decryption key, which I will provide after you make the payment.
Failure to do so may result in your files being permanently lost.
Follow my instructions carefully.

Payment Details:
Transfer $1000 in Bitcoin to the following address.
You can contact me via Telegram: @hacker7
Our team is available at https:[//]t.me/cubervolk. We look forward to receiving your payment.

The ransom note directs victims to a non-existing channel https:[//]t.me/cubervolk. Looks like the ransomware creators were in such a rush to demand the ransom that they forgot to double-check their own link.

Code reuse from Babuk ransomware

Our comparison of CyberVolk and Babuk ransomware using BinDiff revealed some similarities, particularly in cryptographic routines and system-level interactions. For example, the function CryptAcquireContextW and other cryptographic setups show significant overlap between the two, indicating that CyberVolk’s developers likely reused Babuk’s encryption framework.

However, CyberVolk has added unique functionality, such as:

  • Anti-analysis techniques: Efforts to evade detection through Task Manager termination.
  • AES encryption: Unlike Babuk, CyberVolk incorporates the AES encryption algorithm, enhancing its cryptographic capabilities and further differentiating the two strains.

Conclusion

CyberVolk ransomware shows off the usual ransomware tricks complete with a few bugs for good measure. By reusing some of Babuk’s code — particularly in its cryptographic routines — it’s clear that ransomware authors are getting creative with their remix skills, building on old frameworks to make their threats just a little more polished.CyberVolk also introduces some original features, such as attempting to terminate system processes like Task Manager. It succeeds in this task when run with elevated privileges.

Our decryption tests revealed that ransomware has some flaws. CyberVolk’s key validation is weak enough that even random keys trigger the decryption routine, though files remain unusable without the correct key. Despite its warnings about deleting files if an incorrect key is entered, we found that files remained encrypted but were not deleted, highlighting a gap between what the ransomware claims and what it actually does.

Still, CyberVolk has caused significant disruption, particularly in Spain. With its mix of DDoS and ransomware attacks, it’s becoming a more serious threat. As the group refines its tactics, cybersecurity professionals should keep a close eye on its continued evolution.

Cybersecurity professionals should keep this ransomware on their radar. Despite its bugs, CyberVolk is evolving and has already proven effective, causing significant damage to entities in Spain. It adds enough new tricks to the traditional ransomware formula to evade detection and create serious headaches for its victims.

Read up on additional ransomware groups and get other insights from Rapid7 Labs here.

Customer compliance and security during the post-quantum cryptographic migration

Post Syndicated from Panos Kampanakis original https://aws.amazon.com/blogs/security/customer-compliance-and-security-during-the-post-quantum-cryptographic-migration/

Amazon Web Services (AWS) prioritizes the security, privacy, and performance of its services. AWS is responsible for the security of the cloud and the services it offers, and customers own the security of the hosts, applications, and services they deploy in the cloud. AWS has also been introducing quantum-resistant key exchange in common transport protocols used by our customers in order to provide long-term confidentiality. In this blog post, we elaborate how customer compliance and security configuration responsibility will operate in the post-quantum migration of secure connections to the cloud. We explain how customers are responsible for enabling quantum-resistant algorithms or having these algorithms enabled by default in their applications that connect to AWS. We also discuss how AWS will honor and choose these algorithms (if they are supported on the server side) even if that means the introduction of a small delay to the connection.

Secure connectivity

Security and compliance is a shared responsibility between AWS and the customer. This Shared Responsibility Model can help relieve the customer’s operational burden as AWS operates, manages, and controls the components from the host operating system and virtualization layer down to the physical security of the facilities in which the service operates. The customer assumes responsibility and management of the guest operating system and other associated application software, as well as the configuration of the AWS provided security group firewall. AWS has released Customer Compliance Guides (CCGs) to support customers, partners, and auditors in their understanding of how compliance requirements from leading frameworks map to AWS service security recommendations.

In the context of secure connectivity, AWS makes available secure algorithms in encryption protocols (for example, TLS, SSH, and VPN) for customers that connect to its services. That way AWS is responsible for enabling and prioritizing modern cryptography in connections to the AWS Cloud. Customers, on the other hand, use clients that enable such algorithms and negotiate cryptographic ciphers when connecting to AWS. It is the responsibility of the customer to configure or use clients that only negotiate the algorithms the customer prefers and trusts when connecting.

Prioritizing quantum-resistance or performance?

AWS has been in the process of migrating to post-quantum cryptography in network connections to AWS services. New cryptographic algorithms are designed to protect against a future cryptanalytically relevant quantum computer (CRQC) which could threaten the algorithms we use today. Post-quantum cryptography involves introducing post-quantum (PQ) hybrid key exchanges in protocols like TLS 1.3 or SSH/SFTP. Because both classical and PQ-hybrid exchanges need to be supported for backwards compatibility, AWS will prioritize PQ-hybrid exchanges for clients that support it and classical for clients that have not been upgraded yet. We don’t want to switch a client to classical if it advertises support for PQ.

PQ-hybrid key establishment leverages quantum-resistant key encapsulation mechanisms (KEMs) used in conjunction with classical key exchange. The client and server still do an ECDH key exchange, which gets combined with the KEM shared secret when deriving the symmetric key. For example, clients could perform an ECDH key exchange with curve P256 and post-quantum Kyber-768 from NIST’s PQC Project Round 3 (TLS group identifier X25519Kyber768Draft00) when connecting to AWS Certificate Manager (ACM), AWS Key Management Service (AWS KMS), and AWS Secrets Manager. This strategy combines the high assurance of a classical key exchange with the quantum-resistance of the proposed post-quantum key exchanges, to help ensure that the handshakes are protected as long as the ECDH or the post-quantum shared secret cannot be broken. The introduction of the ML-KEM algorithm adds more data (2.3 KB) to be transferred and slightly more processing overhead. The processing overhead is comparable to the existing ECDH algorithm, which has been used in most TLS connections for years. As shown in the following table, the total overhead of hybrid key exchanges has been shown to be immaterial in typical handshakes over the Internet. (Sources: Blog posts How to tune TLS for hybrid post-quantum cryptography with Kyber and The state of the post-quantum Internet)

Data transferred (bytes) CPU processing (thousand ops/sec)
Client Server
ECDH with P256 128 17 17
X25519 64 31 31
ML-KEM-768 2,272 13 25

The new key exchanges introduce some unique conceptual choices that we didn’t have before, which could lead to the peers negotiating classical-only algorithms. In the past, our cryptographic protocol configurations involved algorithms that were widely trusted to be secure. The client and server configured a priority for their algorithms of choice and they picked the more appropriate ones from their negotiated prioritized order. Now, the industry has two families of algorithms, the “trusted classical” and the “trusted post-quantum” algorithms. Given that a CRQC is not available, both classical and post-quantum algorithms are considered secure. Thus, there is a paradigm shift that calls for a decision in the priority vendors should enforce on the client and server configurations regarding the “secure classical” or “secure post-quantum” algorithms.

Figure 1 shows a typical PQ-hybrid key exchange in TLS.

Figure 1: A typical TLS 1.3 handshake

Figure 1: A typical TLS 1.3 handshake

In the example in Figure 1, the client advertises support for PQ-hybrid algorithms with ECDH curve P256 and quantum-resistant ML-KEM-768, ECDH curve P256 and quantum-resistant Kyber-512 Round 3, and classical ECDH with P256. The client also sends a Keyshare value for classical ECDH with P256 and for PQ-hybrid P256+MLKEM768. The Keyshare values include the client’s public keys. The client does not include a Keyshare for P256+Kyber512, because that would increase the size of the ClientHello unnecessarily and because ML-KEM-768 is the ratified version of Kyber Round 3, and so the client chose to only generate and send a P256+MLKEM768 public key. Now let’s say that the server supports ECDH curve P256 and PQ-hybrid P256+Kyber512, but not P256+MLKEM768. Given the groups and the Keyshare values the client included in the ClientHello, the server has the following two options:

  1. Use the client P256 Keyshare to negotiate a classical key exchange, as shown in Figure 1. Although one might assume that the P256+Kyber512 Keyshare could have been used for a quantum-resistant key exchange, the server can pick to negotiate only classical ECDH key exchange with P256, which is not resistant to a CRQC.
  2. Send a Hello Retry Request (HRR) to tell the client to send a PQ-hybrid Keyshare for P256+Kyber512 in a new ClientHello (Figure 2). This introduces a round trip, but it also forces the peers to negotiate a quantum-resistant symmetric key.

Note: A round-trip could take 30-50 ms in typical Internet connections.

Previously, some servers were using the Keyshare value to pick the key exchange algorithm (option 1 above). This generally allowed for faster TLS 1.3 handshakes that did not require an extra round-trip (HRR), but in the post-quantum scenario described earlier, it would mean the server does not negotiate a quantum-resistant algorithm even though both peers support it.

Such scenarios could arise in cases where the client and server don’t deploy the same version of a new algorithm at the same time. In the example in Figure 1, the server could have been an early adopter of the post-quantum algorithm and added support for P256+Kyber512 Round 3. The client could subsequently have upgraded to the ratified post-quantum algorithm with ML-KEM (P256+MLKEM768). AWS doesn’t always control both the client and the server. Some AWS services have adopted the earlier versions of Kyber and others will deploy ML-KEM-768 from the start. Thus, such scenarios could arise while AWS is in the post-quantum migration phase.

Note: In these cases, there won’t be a connection failure; the side-effect is that the connection will use classical-only algorithms although it could have negotiated PQ-hybrid.

These intricacies are not specific to AWS. Other industry peers have been thinking about these issues, and they have been a topic of discussion in the Internet Engineering Task Force (IETF) TLS Working Group. The issue of potentially negotiating a classical key exchange although the client and server support quantum-resistant ones is discussed in the Security Considerations of the TLS Key Share Prediction draft (draft-davidben-tls-key-share-prediction). To address some of these concerns, the Transport Layer Security (TLS) Protocol Version 1.3 draft (draft-ietf-tls-rfc8446bis), which is the draft update of TLS 1.3 (RFC 8446), introduces text about client and server behavior when choosing key exchange groups and the use of Keyshare values in Section 4.2.8. The TLS Key Share Prediction draft also tries to address the issue by providing DNS as a mechanism for the client to use a proper Keyshare that the server supports.

Prioritizing quantum resistance

In a typical TLS 1.3 handshake, the ClientHello includes the client’s key exchange algorithm order of preferences. Upon receiving the ClientHello, the server responds by picking the algorithms based on its preferences.

Figure 2 shows how a server can send a HelloRetryRequest (HRR) to the client in the previous scenario (Figure 1) in order to request the negotiation of quantum-resistant keys by using P256+Kyber512. This approach introduces an extra round trip to the handshake.

Figure 2: An HRR from the server to request the negotiation of mutually supported quantum-resistant keys with the client

Figure 2: An HRR from the server to request the negotiation of mutually supported quantum-resistant keys with the client

AWS services that terminate TLS 1.3 connections will take this approach. They will prioritize quantum resistance for clients that advertise support for it. If the AWS service has added quantum-resistant algorithms, it will honor a client-supported post-quantum key exchange even if that means that the handshake will take an extra round trip and the PQ-hybrid key exchange will include minor processing overhead (ML-KEM is almost performant as ECDH). A typical round trip in regionalized TLS connections today is usually under 50 ms and won’t have material impact to the connection performance. In the post-quantum transition, we consider clients that advertise support for quantum-resistant key exchange to be clients that take the CRQC risk seriously. Thus, the AWS server will honor that preference if the server supports the algorithm.

Pull Request 4526 introduces this behavior in s2n-tls, the AWS open source, efficient TLS library built over other crypto libraries like OpenSSL libcrypto or AWS libcrypto (AWS-LC). When built with s2n-tls, s2n-quic handshakes will also inherit the same behavior. s2n-quic is the AWS open source Rust implementation of the QUIC protocol.

What AWS customers can do to verify post-quantum key exchanges

AWS services that have already adopted the behavior described in this post include AWS KMS, ACM, and Secrets Manager TLS endpoints, which have been supporting post-quantum hybrid key exchange for a few years already. Other endpoints that will deploy quantum-resistant algorithms will inherit the same behavior.

AWS customers that want to take advantage of new quantum-resistant algorithms introduced in AWS services are expected to enable them on the client side or the server side of a customer-managed endpoint. For example, if you are using the AWS Common Runtime (CRT) HTTP client in the AWS SDK for Java v2, you would need to enable post-quantum hybrid TLS key exchanges with the following.

SdkAsyncHttpClient awsCrtHttpClient = AwsCrtAsyncHttpClient.builder()
            .postQuantumTlsEnabled(true)
            .build();

The AWS KMS and Secrets Manager documentation includes more details for using the AWS SDK to make HTTP API calls over quantum-resistant connections to AWS endpoints that support post-quantum TLS.

To confirm that a server endpoint properly prioritizes and enforces the PQ algorithms, you can use an “old” client that sends a PQ-hybrid Keyshare value that the PQ-enabled server does not support. For example, you could use s2n-tls built with AWS-LC (which supports the quantum-resistant KEMs). You could use a client TLS policy (PQ-TLS-1-3-2023-06-01) that is newer than the server’s policy (PQ-TLS-1-0-2021-05-24). That will lead the server to request the client by means of an HRR to send a new ClientHello that includes P256+MLKEM768, as shown following.

./bin/s2nd -c PQ-TLS-1-0-2021-05-24 localhost 4444
sudo tcpdump port 4444 -w hrr-capture.pcap
./bin/s2nc localhost 4444 -c PQ-TLS-1-3-2023-06-01 -i

The hrr-capture.pcap packet capture will show the negotiation and the HRR from the server.

To confirm that a server endpoint properly implements the post-quantum hybrid key exchanges, you can use a modern client that supports the key exchange and connect against the endpoint. For example, using the s2n-tls client built with AWS-LC (which supports the quantum-resistant KEMs), you could try connecting to a Secrets Manager endpoint by using a post-quantum TLS policy (for example, PQ-TLS-1-2-2023-12-15) and observe the PQ hybrid key exchange used in the output, as shown following.

./bin/s2nc -c PQ-TLS-1-2-2023-12-15 secretsmanager.us-east-1.amazonaws.com 443
CONNECTED:
Handshake: NEGOTIATED|FULL_HANDSHAKE|MIDDLEBOX_COMPAT
Client hello version: 33
Client protocol version: 34
Server protocol version: 34
Actual protocol version: 34
Server name: secretsmanager.us-east-1.amazonaws.com
Curve: NONE
KEM: NONE
KEM Group: SecP256r1Kyber768Draft00
Cipher negotiated: TLS_AES_128_GCM_SHA256
Server signature negotiated: RSA-PSS-RSAE+SHA256
Early Data status: NOT REQUESTED
Wire bytes in: 6699
Wire bytes out: 1674
s2n is ready
Connected to secretsmanager.us-east-1.amazonaws.com:443

An alternative would be using the Open Quantum Safe (OQS) for OpenSSL client to do the same.

As another example, if you want to transfer a file over a quantum-resistant SFTP connection with AWS Transfer Family, you would need to configure a PQ cryptography SSH security policy on your AWS File Transfer SFTP endpoint (for example, TransferSecurityPolicy-2024-01) and enable quantum-resistant SSH key exchange in the SFTP client. Note that in SSH/SFTP, although the AWS server side will advertise the quantum-resistant schemes as higher priority, the client picks the key exchange algorithm. So, a client that supports PQ cryptography would need to have the PQ algorithms configured with higher priority (as described in the Shared Responsibility Model). For more details, refer to the AWS Transfer Family documentation.

Conclusion

Cryptographic migrations can introduce intricacies to cryptographic negotiations between clients and servers. During the migration phase, AWS services will mitigate the risks of these intricacies by prioritizing post-quantum algorithms for customers that advertise support for these algorithms—even if that means a small slowdown in the initial negotiation phase. While in the post-quantum migration phase, customers who choose to enable quantum resistance have made a choice which shows that they consider the CRQC risk as important. To mitigate this risk, AWS will honor the customer’s choice, assuming that quantum resistance is supported on the server side.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on the AWS Security, Identity, & Compliance re:Post or contact AWS Support. For more details regarding AWS PQC efforts, refer to our PQC page.
 

Panos Kampanakis
Panos Kampanakis

Panos is a Principal Security Engineer at AWS. He has experience with cybersecurity, applied cryptography, security automation, and vulnerability management. He has coauthored publications on cybersecurity and participated in various security standards bodies to provide common interoperable protocols and languages for security information sharing, cryptography, and public-key infrastructure. Currently, he works with engineers and industry standards partners to provide cryptographically secure tools, protocols, and standards.
Alex Weibel
Alex Weibel

Alex is a Senior Software Development Engineer on the AWS Crypto Algorithms team. He’s a contributor to Amazon’s TLS Library s2n-tls, the Amazon Corretto Crypto Provider (ACCP), and AWS LibCrypto. Previously, Alex worked on TLS termination and request proxying for S3 and the Elastic Load Balancing Service, developing new features for customers. Alex holds a Bachelor of Science degree in Computer Science from the University of Texas at Austin.

Accelerate application upgrades with Amazon Q Developer agent for code transformation

Post Syndicated from Jonathan Vogel original https://aws.amazon.com/blogs/devops/accelerate-application-upgrades-with-amazon-q-developer-agent-for-code-transformation/

In this blog, we will explore how Amazon Q Developer Agent for code transformation accelerates Java application upgrades. We will examine the benefits of this Generative AI-powered agent and outline strategies to achieve maximal acceleration, drawing from real-world success stories and best practices.

Benefits of using Amazon Q Developer to upgrade your applications

Amazon Q Developer addresses a critical challenge for organizations managing numerous Java applications, particularly as they face the approaching end of Long-Term-Support (LTS) for older Java versions. Upgrading to Java 17 enhances security, resolves vulnerabilities, and improves performance while ensuring long-term compatibility and access to modern features. Currently, Q Developer agent for code transformation supports upgrades from Java 8 and 11 to Java 17. Software developers can utilize Q Developer within their IDE (VS Code and JetBrains) to transform both single-module and multi-module applications. Q Developer will generate a plan that identifies necessary library upgrades and replacements for deprecated code in the application, proposing code changes with the goal of ensuring the transformed code compiles successfully in Java 17. Q Developer can significantly enhance the efficiency of your migration workflow, performing code transformations on applications in hours rather than weeks.

Customer success of using Q Developer to modernize legacy Java applications

Customers have used Q Developer to upgrade their Java applications successfully. Here is how two customers as well as Amazon internal teams use Q Developer to accelerate the migration process.

A large insurance company in North America strategically approached their Java upgrade initiative by identifying applications with dependencies that Q Developer could upgrade effectively. They focused on applications that rely on frameworks like Spring Boot, which can be time-consuming to upgrade manually. After leveraging Q Developer to transform 4 applications in pilot, they estimated a 36% acceleration in their upgrade process, indicating that Q Developer automatically completed over a third of the work that would have been required manually. While the remaining portion still necessitated manual intervention to ensure the code would build and run correctly, the effort acceleration was significant.

A major financial services firm’s experience with Q Developer proved equally compelling. In a focused two-day workshop, 20 developers successfully transformed 20 applications in production using the Amazon Q Developer agent. This results in 42% time savings using Q Developer compared to manual upgrade, saving on average 24 hours per application. They spent about 3 weeks to prepare for the transformation workshop. They identified first-party (1P) dependencies—internal libraries that other production applications rely on. Q developer does not guarantee upgrade of 1P dependencies. With a combination of Q Developer and manual work, the customer upgraded many of these common 1P dependencies leading up to the workshop. This step was crucial to gain maximum acceleration while using Q Developer for the upgrades.

Amazon uses Q Developer internally to upgrade Java applications following company-wide campaigns. The central team who owns the campaigns provides detailed guidance on which Java applications can be upgraded with Q developer most effectively. This team also manages Amazon’s internal build system and provides tooling to automate part of the manual efforts. They are able to achieve significant savings. Amazon was able to upgrade more than 50% of production applications in six months, 79% of the auto-generated code reviews were applied without additional changes.

Use Q Developer to upgrade your applications

To ensure that Q Developer is properly applied to the specific characteristics of their codebases, customers create and follow a transformation approach. Teams and individuals who understand the scope of the upgrade run campaigns across the company to effectively utilize Q Developer. To maximize the acceleration from Q Developer, these teams classify the applications which need to be upgraded, identify which ones can be upgraded using Q Developer, estimate the manual effort required, which provides a baseline to measure the value added by Q Developer agent for code transformation. The preparation phase is crucial before starting the execution phase of the upgrade. Each of the steps in the preparation phase plays an important role in maximizing the acceleration of Amazon Q in their upgrade processes.

  1. Classifying the applications to upgrade: Q Developer supports the upgrade of 30 most common Java libraries. Q Developer’s performance on less common and internal libraries is lower compared to the common libraries. In this case, you can use a combination of Q Developer and manual steps. It’s recommended to include both production applications and internal dependencies in this step. You should also classify your applications and internal libraries based on if/how they are used by other applications, it will help prioritize the applications to upgrade first in campaigns. Classifying applications by libraries used can help you identify the best upgrade approach using Q Developer.
  2. Defining baselines of efficiency: To measure the efficiency of the upgrade effort in your organization, it is crucial to establish baselines. Based on the classification of applications, use Q Developer in a pilot for each class to see which libraries are transformed correctly, and which ones have to be done manually. This helps you operationalize the process of using Q Developer and the manual steps required, and understand how this procedure accelerates the upgrade of a certain class of applications. Some customers use manual effort hours for each upgrade on dependency versions and deprecated code as baseline and compare the manual effort hours with time taken when completing the upgrade using Q Developer. For example, you can classify the applications based on the main frameworks used before upgrading applications using Q Developer. Compare the time taken by Q Developer with manual upgrade hours to understand which applications can be upgraded by Q Developer most effectively.
  3. Identifying applications for migration: Decide which applications to use Q Developer for, and prioritize the applications to upgrade in waves based on expected acceleration and business value. You can prioritize the applications which are most used by other applications and upgrade them in the initial campaign, then upgrade the rest of the applications in the subsequent campaigns. By addressing the foundational components first, the overall upgrade process will be streamlined. In Amazon, a centralized internal team defines migration waves and identifies which packages would be included in the upgrade campaign. Additionally, this team conducted analysis of the apps to determine the likelihood of the upgrade being successful using Q developer, and provides an estimate of the remaining engineering effort needed to complete the upgrade. The team will use this information to select applications and uses an Amazon-internal tool to assign the upgrade tasks to the team owning the applications. While SDEs were free to run the upgrade on their own, following the campaign with a set deadline mobilized the application owner teams to complete the upgrade.

Use Q Developer to automate upgrade tasks

Once the preparation phase is completed, you can start the execution phase. Software developers can use Q Developer to accelerate many of the steps in execution phase.

  1. Assessing the components of an application to upgrade. You can use Q Developer to start a transformation, at the beginning of the transformation, there will be a transformation plan generated for you to view which dependencies and deprecated code will be upgraded.
  2. Research and update dependency versions compatible to the target version. Q Developer will analyze your app and attempt to update the dependencies to the versions compatible with target Java version and in some cases the latest version.
  3. Replace deprecated methods and API calls which are not compatible to the target version. Q Developer will detect the deprecated code and attempt to update to what’s recommended in the compatible Java version.
  4. Reviewing the modified code and address any conflicts or issues that may arise. Q Developer will return code changes to you at the end of the transformation. If the transformation is successful, the app will compile in Java 17. If the transformation is partially successful, Q Developer was able to upgrade library versions and make code changes but could not compile the transformed app successfully in Java 17. Check out this part of our documentation on how to handle partial transformations.
  5. Test the upgraded application thoroughly to ensure correct functionality. Q Developer will run the unit tests and integration tests in your app when compiling in the target version.

Conclusion

As organizations face the pressing need to modernize their Java applications, Amazon Q Developer emerges as a powerful ally in this complex journey. The customer success stories demonstrate the tangible benefits of leveraging AI-assisted code transformation: significant time savings, reduced manual effort, and accelerated upgrade processes.

Q Developer not only addresses the technical challenges of Java upgrades, but also enables organizations to approach these initiatives strategically. By classifying applications, establishing baselines, and prioritizing upgrades, teams can maximize the efficiency of their modernization efforts. While Q Developer streamlines much of the upgrade process, it is important to note that some challenges may still arise. For a comprehensive understanding of potential challenges and detailed guidance on getting started with Q Developer, we encourage you to explore our public documentation.

The journey to Java 17 and beyond doesn’t have to be daunting. With Amazon Q Developer, you have a powerful tool at your disposal to accelerate your upgrade process, reduce costs, and ensure your applications remain secure, performant, and future-ready.

Take the first step towards modernizing your Java ecosystem today. Explore Amazon Q Developer and discover how it can transform your upgrade strategy. See Getting Started with Amazon Q Developer agent for code transformation for a how-to guide on using Q Developer to transform Java applications.

About the authors

Jonathan Vogel

Jonathan is a Developer Advocate at AWS. He was a DevOps Specialist Solutions Architect at AWS for two years prior to taking on the Developer Advocate role. Prior to AWS, he practiced professional software development for over a decade. Jonathan enjoys music, birding and climbing rocks.

Yiyi Guo

Yiyi is a Senior Product Manager at AWS working on Amazon Q developer agent for code transformation, she focuses on leveraging generative AI to accelerate enterprise application modernization.

Backblaze B2 Event Notifications Now Generally Available

Post Syndicated from Bala Krishna Gangisetty original https://www.backblaze.com/blog/using-b2-event-notifications/

A decorative image showing a cloud, gears, and an alarm notification.

No one likes being left out in the cold, least of all your data. With Backblaze B2 Event Notifications—now generally available—you can receive real-time notifications about object changes. That means that you can build more responsive and automated workflows across best-of-breed cloud platforms, saving time and money and improving your end users’ experiences. And, you can be alerted to changes in your data that may speed time to action.

Here’s how it works: With Backblaze B2 Event Notifications, any data changes within B2 Cloud Storage—like uploads, updates, or deletions—can automatically trigger actions in a workflow, including transcoding video files, spooling up data analytics, delivering finished assets to end users, and many others. Importantly, unlike many other solutions currently available, Backblaze’s service doesn’t lock you into one platform or require you to use legacy tools from AWS.

So, to businesses that want to create an automated workflow that combines different compute, content delivery networks (CDN), data analytics, and whatever other cloud service: Now you can, with the bonus of cloud storage at a fifth of the rates of other solutions and free egress.

Key capabilities

  • Flexible implementation: Event Notifications are sent as HTTP POST requests to the desired service or endpoint within your infrastructure or any other cloud service. This flexibility ensures seamless integration with your existing workflows. For instance, your endpoint could be Fastly Compute, AWS Lambda, Azure Functions, or Google Cloud Functions, etc.
  • Event categories: Specify the types of events you want to be notified about, such as when files are uploaded and deleted. This allows you to receive notifications tailored to your specific needs. For instance, you have the flexibility to specify different methods of object creation, such as copying, uploading, or multipart replication, to trigger event notifications. You can also manage Event Notification rules through UI or API.
  • Filter by prefix: Define prefixes to filter events, enabling you to narrow down notifications to specific sets of objects or directories within your storage on Backblaze B2. For instance, if your bucket contains audio, video, and text files organized into separate prefixes, you can specify the prefix for audio files in order to receive Event Notifications exclusively for audio files.
  • Custom headers: Include personalized HTTP headers in your Event Notifications to provide additional authentication or contextual information when communicating with your target endpoint. For example, you can use these headers to add necessary authentication tokens or API keys for your target endpoint, or include any extra metadata related to the payload to offer contextual information to your webhook endpoint, and more.
  • Signed notification messages: You can configure outgoing messages to be signed by the Event Notifications service, allowing you to validate signatures and verify that each message was generated by Backblaze B2 and not tampered with in transit.
  • Test rule functionality: Validate the functionality of your target endpoint by testing Event Notifications before deploying them into production. This allows you to ensure that your integration with your target endpoint is set up correctly and functioning as expected.
  • Retries: Event Notifications are automatically re-sent if the initial delivery attempt fails. This feature increases the reliability of Event Notifications by ensuring that temporary issues do not result in missed events, thus maintaining the integrity of your event-driven workflows.
  • Delivery: Event Notifications are designed for the at-least-once delivery guarantee to ensure Event Notifications are delivered reliably, even in the presence of network or system failures.

Versatile use cases

This past April, we announced Event Notifications in preview, and folks have put Event Notifications to work in some incredible ways. Today, we’re sharing some of the key use cases that came out of the preview to simplify your own workflows so you can focus on extracting insights from your data, rather than managing the logistics of data processing.

A diagram describing how Event Notifications work.

Automated media processing

Video transcoding: Many customers use Event Notifications to automate their video transcoding workflows. When a new video is uploaded to a Backblaze B2 Bucket, an Event Notification can trigger a transcoding process to generate all videos in the desired format. 

Image processing: Similarly, customers also use Event Notifications to set up automated image processing pipelines, such as generating thumbnails or applying filters when new images are added to a Backblaze B2 Bucket.

Media processing is not limited to video transcoding or image processing. It can be extended to any other media processing workflow, minimizing the number of steps in the workflow.

Backup monitoring

Customers can receive notifications when backups are successfully uploaded to a Backblaze B2 Bucket with Event Notifications, providing peace of mind and ensuring data protection. Whether you want to track your nightly or monthly backups, you can get a notification when they are completed.

Presigned URL monitoring

Using a presigned URL is a standard way to share a file without giving the full access to your Backblaze B2 Bucket. Customers are using Event Notifications to know when their clients upload files via presigned URLs to Backblaze B2. They can get a callback to confirm that the upload is complete.

Security and access control

Unauthorized access detection: Customers are using Event Notifications to track access to highly confidential video files and report back to their clients as needed. Event Notifications help them detect any unauthorized access and take immediate action.

Audit trails: Some customers are using Event Notifications to create a detailed audit log of supported bucket activities through Event Notifications, which is useful for their compliance and security purposes.

Anomaly/malware detection: Event Notifications can strengthen security by detecting unusual access patterns, like malware that deletes or overwrites backups, by getting notifications of changes to Backblaze B2 Buckets.

Integration with external systems

Database synchronization: Customers use Event Notifications to keep databases in sync with the state of their Backblaze B2 Buckets. It’s critical to ensure data consistency across systems as their applications run on the databases.

Document management system: Some customers use Event Notifications with a workflow system to track document revisions, uploads, and deletes, or to notify team members when specific documents are uploaded or deleted.

Analytics and reporting

Performance analytics: Some customers use Event Notifications to monitor their backup performance and completion times, helping to optimize their data management strategies.

Usage tracking: Event Notifications can help track storage consumption by individual users or projects, facilitating better resource management and cost allocation.

These are just a few of the use cases our preview customers shared with us, and the sky is truly the limit for ways Event Notifications can empower you to simplify and streamline your workflows. 

Ready to get started?

For existing customers working with a Backblaze account manager, Event Notifications is enabled for you today. If you need assistance, your account manager is happy to help.

For existing customers who are not currently working with an account manager, please contact our Support team to request access.

New to Backblaze? Contact our Sales team to learn more about how Event Notifications can benefit your business and how to get started.

A screenshot of the Backblaze account screen where you can enable Event Notifications.

Once Event Notifications is enabled on your account, log in to your Backblaze B2 account, navigate to the Buckets page, and click on the Event Notifications section. From there, you can set up notification rules for the events you want to track or configure notifications through our API. 

For detailed instructions and best practices, check out the Event Notifications documentation.

What’s next?

Please do share how you’re leveraging Event Notifications to build more efficient, automated, and responsive workflows so that other organizations and developers can benefit from what you find. If you have any questions or feedback, please don’t hesitate to reach out to us.

The post Backblaze B2 Event Notifications Now Generally Available appeared first on Backblaze Blog | Cloud Storage & Cloud Backup

Modernizing Your VM Program with Rapid7 Exposure Command: A Path to Effective Continuous Threat Exposure Management

Post Syndicated from Ryan Blanchard original https://blog.rapid7.com/2024/10/03/modernizing-your-vm-program-with-rapid7-exposure-command-a-path-to-effective-continuous-threat-exposure-management/

Modernizing Your VM Program with Rapid7 Exposure Command: A Path to Effective Continuous Threat Exposure Management

In today’s threat landscape, where cyber-attacks are increasingly sophisticated and pervasive, organizations face the daunting challenge of securing a constantly expanding attack surface. Traditional vulnerability management (VM) programs, while necessary, are no longer sufficient on their own. They often struggle to keep pace with the dynamic nature of threats and the complexity of modern IT environments.

This is where continuous threat exposure management (CTEM) comes into play – an approach that shifts the focus from merely identifying vulnerabilities to understanding and mitigating exposures across the entire attack surface.

Implementing a continuous threat and exposure management process

CTEM is a term originally coined by Gartner, who defined it as, “a five-stage approach that continuously exposes an organization’s networks, systems, and assets to simulated attacks to identify vulnerabilities and weaknesses.”

The five stages of CTEM as defined by Gartner are:

Modernizing Your VM Program with Rapid7 Exposure Command: A Path to Effective Continuous Threat Exposure Management
  • Scoping: This involves understanding the full threat landscape by incorporating tools like external attack surface management (EASM) and network scanning. However, it emphasizes the need to think in terms of business context, focusing on crown jewels, critical applications, and understanding what matters most to the organization.
  • Discovery: This stage focuses on discovering assets and profiling the associated risks. It requires visibility into both cloud and on-premises environments and extends beyond identifying vulnerabilities to include coverage gaps, misconfigurations, and other security risks.
  • Prioritization: Since not all risks can be addressed simultaneously, this phase involves prioritizing issues based on a combination of factors like severity, exploitability, and potential business impact, to determine what should be tackled first.
  • Validation: This stage emphasizes investing in tools that help validate security controls and map potential attack paths. It includes the use of breach and attack simulation (BAS) tools, continuous assessment services, controls monitoring, and attack path mapping to test the effectiveness of existing defenses.
  • Mobilization: The final stage is about taking action. It includes both automation of responses and fostering cross-organizational alignment to ensure that remediation efforts are executed effectively and in sync with business priorities.

This framework helps organizations continuously manage and reduce their exposure to threats in a way that is strategic and aligned to the business.

The role of exposure assessment platforms (EAPs) in CTEM

Exposure assessment platforms (EAPs) are essential to a successful CTEM program. They continuously identify and prioritize exposures, such as vulnerabilities and misconfigurations, across a broad range of asset classes. By consolidating and contextualizing data from various sources, EAPs provide a more comprehensive view of an organization’s risk landscape. This enables security teams to prioritize remediation efforts based on factors such as asset criticality, business impact, and the likelihood of exploitation.

Gartner’s insights into EAPs underscore their importance in modern cybersecurity strategies. By delivering a centralized view of high-risk exposures, EAPs empower organizations to take decisive actions to prevent breaches. They also enhance operational efficiency by offering a unified dashboard that tracks the lifecycle of vulnerabilities and other exposures.

How Rapid7 Exposure Command supports modern vulnerability management programs

Exposure Command is designed to bridge the security-visibility gap many organizations face. By integrating the capabilities of an EAP into a comprehensive security platform, Exposure Command enables organizations to modernize their VM programs and align them with the principles of CTEM.

Exposure Command can help organizations achieve this transformation in a few ways:

  • Consolidated view of exposures from the inside out and outside in: Exposure Command provides a single, consolidated view of all assets and identified exposures from an internal and external perspective, including vulnerabilities, misconfigurations, and other risk signals. This unified view reduces the overhead associated with managing multiple tools and platforms, enabling security teams to focus on what matters most: mitigating the most critical threats.
  • Vendor-agnostic approach: As organizations adopt a CTEM approach, they require tools that can evaluate a wide range of exposure telemetry, including security control configurations. Exposure Command excels in this area by leveraging data from existing endpoint and network investments to create a more accurate, situational picture of the organization’s risk landscape. This holistic view is crucial for making informed decisions about where to focus remediation efforts.
  • Contextualized risk prioritization: Traditional VM programs often rely on CVSS scores to prioritize vulnerabilities, which can lead to misaligned efforts. Exposure Command, however, incorporates threat intelligence, asset criticality, and business impact into its risk-prioritization algorithms. This results in a more accurate and actionable understanding of which exposures pose the greatest risk to the organization.
  • Identify exploitability and potential for lateral movement: Exposure Command provides the contextual asset enrichment that enables effective threat detection, investigation, and response. The platform showcases how an attacker might exploit vulnerabilities and provides guidance on how to prevent such incidents.
  • Automated response workflows and deep ecosystem integration: One of the key benefits of Exposure Command is its ability to automate and streamline workflows. By integrating with existing security tools and platforms, Exposure Command can automatically ingest and analyze exposure data, reducing the manual effort required to maintain a VM program. This automation not only improves efficiency but also ensures security teams have access to the most up-to-date information. More and more we’re running into non-patchable systems too, and this deep integration and ability to provide bi-directional workflows enables more effective mobilization across teams, giving actionable feedback to those around the organization who have the ability to execute the necessary remediation actions.

While the benefits of Exposure Command are clear, it’s important to recognize that its effectiveness is tied to the maturity of the organization’s CTEM processes. If these processes are broken or immature, the value of Exposure Command may be limited. However, by adopting an outcome-driven approach that scopes the most critical aspects of the business and correlates asset context with dynamic risk ratings, organizations can maximize the benefits of Exposure Command.

Furthermore, the platform’s ability to integrate with a wide range of tools ensures it can enhance existing security programs, rather than requiring a complete overhaul. This makes it an ideal solution for organizations looking to modernize their VM programs and adopt a more proactive approach to threat management. As you look to strengthen your organization’s cybersecurity posture, consider how Rapid7 Exposure Command can help you bridge the security visibility gap and take a more proactive approach to managing your threat landscape.

[$] Coping with complex cameras

Post Syndicated from corbet original https://lwn.net/Articles/992411/

Cameras were never the simplest of devices for Linux to support; they have
a wide range of operating parameters and can generate high rates of data.
In recent years, though, they have become increasingly complex, stressing
the ability of the kernel’s media
subsystem to manage them. At the 2024 Linux Plumbers Conference, developers from
that subsystem and beyond gathered to discuss the state of affairs and how
complex camera devices should be supported in the future.

Security updates for Thursday

Post Syndicated from jake original https://lwn.net/Articles/992798/

Security updates have been issued by AlmaLinux (cups-filters), Debian (chromium and php8.2), Fedora (firefox), Oracle (cups-filters, flatpak, kernel, krb5, oVirt 4.5 ovirt-engine, and python-urllib3), Red Hat (cups-filters, firefox, go-toolset:rhel8, golang, and thunderbird), SUSE (postgresql16), and Ubuntu (gnome-shell and linux-azure-fde-5.15).

Introduce the Code Editor into your school

Post Syndicated from Joanne Vincent original https://www.raspberrypi.org/blog/code-editor-for-education/

Since we first launched the Code Editor, a free online tool designed to support young people  as they learn text-based programming, we’ve been excited to hear how educators have been trying it out in their classrooms. 

“I used the Code Editor with my computer science students yesterday and it worked a dream! Students were able to write and run code without any issues.” 

– Head of Computer Science

The Code Editor is designed for learning, rather than for professional use, and is informed by our understanding of pedagogy and computing education. It can be accessed from a web browser without installing any additional software. 

Earlier this year, we announced that we’d be introducing classroom management features and we’re now pleased to confirm that we’ve launched the beta version of Code Editor for Education with school accounts. You can be the first to try out the new features, together with the many schools who have chosen to pre-register their school accounts.

Simple and easy classroom management

We’ve kept the educator interface clean, simple, and easy to use. School owners can invite other teachers to join, add students, organise students into classes, and help students reset their passwords quickly. Educators can create coding projects to share with students and view their work.

Example image of the Raspberry Pi Foundation Code Editor, showcasing its classroom management features.

All features, totally free

We’ve added these classroom management features because one of the key problems we’ve seen educators face over the past months has been the lack of an affordable tool to teach text-based coding in the classroom. We will always provide the Code Editor and all of its features to educators and students for free. 

Safe and private by design

We take safeguarding seriously, providing visibility of student work at all times, as well as features such as the ability to report a concern. In line with best practices protecting children online, we minimise data capture so that we have just enough to keep students safe. 

Future developments 

As the platform is currently in beta, we’d love to hear what you think of the new classroom management features — please send us your feedback. 

We’ll be actively looking to develop new features over the coming months. Such features are set to include an extended set of Python libraries, custom instructions that sit alongside starter code projects and teacher-to-student feedback capabilities. All new developments will be informed by ongoing educator feedback. 

Find out more and register for a free school account.

The post Introduce the Code Editor into your school appeared first on Raspberry Pi Foundation.

Weird Zimbra Vulnerability

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2024/10/weird-zimbra-vulnerability.html

Hackers can execute commands on a remote computer by sending malformed emails to a Zimbra mail server. It’s critical, but difficult to exploit reliably.

In an email sent Wednesday afternoon, Proofpoint researcher Greg Lesnewich seemed to largely concur that the attacks weren’t likely to lead to mass infections that could install ransomware or espionage malware. The researcher provided the following details:

  • While the exploitation attempts we have observed were indiscriminate in targeting, we haven’t seen a large volume of exploitation attempts
  • Based on what we have researched and observed, exploitation of this vulnerability is very easy, but we do not have any information about how reliable the exploitation is
  • Exploitation has remained about the same since we first spotted it on Sept. 28th
  • There is a PoC available, and the exploit attempts appear opportunistic
  • Exploitation is geographically diverse and appears indiscriminate
  • The fact that the attacker is using the same server to send the exploit emails and host second-stage payloads indicates the actor does not have a distributed set of infrastructure to send exploit emails and handle infections after successful exploitation. We would expect the email server and payload servers to be different entities in a more mature operation.
  • Defenders protecting Zimbra appliances should look out for odd CC or To addresses that look malformed or contain suspicious strings, as well as logs from the Zimbra server indicating outbound connections to remote IP addresses.