All posts by Sébastien Stormacq

AWS announces Pixtral Large 25.02 model in Amazon Bedrock serverless

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-announces-pixtral-large-25-02-model-in-amazon-bedrock-serverless/

Today, we announce that the Pixtral Large 25.02 model is now available in Amazon Bedrock as a fully managed, serverless offering. AWS is the first major cloud provider to deliver Pixtral Large as a fully managed, serverless model.

Working with large foundation models (FMs) often requires significant infrastructure planning, specialized expertise, and ongoing optimization to handle the computational demands effectively. Many customers find themselves managing complex environments or making trade-offs between performance and cost when deploying these sophisticated models.

The Pixtral Large model, developed by Mistral AI, represents their first multimodal model that combines advanced vision capabilities with powerful language understanding. A 128K context window makes it ideal for complex visual reasoning tasks. The model delivers exceptional performance on key benchmarks including MathVista, DocVQA, and VQAv2, demonstrating its effectiveness across document analysis, chart interpretation, and natural image understanding.

One of the most powerful aspects of Pixtral Large is its multilingual capability. The model supports dozens of languages including English, French, German, Spanish, Italian, Chinese, Japanese, Korean, Portuguese, Dutch, and Polish, making it accessible to global teams and applications. It’s also trained on more than 80 programming languages including Python, Java, C, C++, JavaScript, Bash, Swift, and Fortran, providing robust code generation and interpretation capabilities.

Developers will appreciate the model’s agent-centric design with built-in function calling and JSON output formatting, which simplifies integration with existing systems. Its strong system prompt adherence improves reliability when working with Retrieval Augmented Generation (RAG) applications and large context scenarios.

With Pixtral Large in Amazon Bedrock, you can now access this advanced model without having to provision or manage any infrastructure. The serverless approach lets you scale usage based on actual demand without upfront commitments or capacity planning. You pay only for what you use, with no idle resources.

Cross-Region inference
Pixtral Large is now available in Amazon Bedrock across multiple AWS Regions through cross-Region inference.

With Amazon Bedrock cross-Region inference, you can access a single FM across multiple geographic Regions while maintaining high availability and low latency for global applications. For example, when a model is deployed in both European and US Regions, you can access it through Region-specific API endpoints using distinct prefixes: eu.model-id for European Regions and us.model-id for US Regions . This approach enables Amazon Bedrock to route inference requests to the geographically closest endpoint, reducing latency while helping to meet regulatory compliance by keeping data processing within desired geographic boundaries. The system automatically handles traffic routing and load balancing across these Regional deployments, providing seamless scalability and redundancy without requiring you to keep track of individual Regions where the model is actually deployed.

See it in action
As a developer advocate, I’m constantly exploring how our newest capabilities can solve real problems. Recently, I had a perfect opportunity to test the new multimodal capabilities in the Amazon Bedrock Converse API when my daughter asked for help with her physics exam preparation.

Last weekend, my kitchen table was covered with practice exams full of complex diagrams, force vectors, and equations. My daughter was struggling with conceptualizing how to approach these problems. That’s when I realized this was the perfect use case for the multimodal capabilities we’d just launched. I snapped photos of a particularly challenging problem sheet containing several graphs and mathematical notation, then used the Converse API to create a simple application that could analyze the images. Together, we uploaded the physics exam materials and asked the model to explain the solution approach.

Physics problem in french

What happened next impressed both of us—the model interpreted the diagrams, recognized the french language and the mathematical notation, and provided a step-by-step explanation of how to solve each problem. As we asked follow-up questions about specific concepts, the model maintained context across our entire conversation, creating a tutoring experience that felt remarkably natural.

# Effet Doppler avec une Source Sonore en Rotation

## Analyse du problème

Ce problème concerne l'effet Doppler produit par une source sonore en rotation. Une source émettant un son à 1500 Hz tourne sur une table tournante dans le sens antihoraire, et nous devons déterminer comment les fréquences sont perçues par un microphone fixe.

## Concepts clés

L'effet Doppler se produit lorsqu'il y a un mouvement relatif entre une source sonore et un observateur. La fréquence perçue dépend de si la source se rapproche ou s'éloigne de l'observateur.

La formule de l'effet Doppler est:
- Pour une source qui se rapproche: $f_{observée} = f_{source} \cdot \frac{v_{son}}{v_{son} - v_{source}}$
- Pour une source qui s'éloigne: $f_{observée} = f_{source} \cdot \frac{v_{son}}{v_{son} + v_{source}}$

## Solution étape par étape

### Étape 1: Analyser le mouvement de la source
La source tourne sur un cercle et le microphone est fixe à 40 cm du centre de rotation. Aux différentes positions (a, b, c), la source a des vitesses radiales différentes par rapport au microphone:

- Position a: La source se déplace perpendiculairement à la ligne source-microphone (vitesse radiale nulle)
- Position b: La source s'éloigne du microphone (vitesse radiale positive)
- Position c: La source se rapproche du microphone (vitesse radiale négative)

### Étape 2: Déterminer l'effet sur les fréquences perçues

- À la position a: Pas d'effet Doppler car la vitesse radiale est nulle, donc $f_a = f_{source} = 1500$ Hz
- À la position b: La source s'éloigne, donc $f_b < f_{source}$, donc $f_b < 1500$ Hz
- À la position c: La source se rapproche, donc $f_c > f_{source}$, donc $f_c > 1500$ Hz

### Étape 3: Comparer les fréquences

D'après notre analyse:
- $f_a = 1500$ Hz
- $f_b < 1500$ Hz
- $f_c > 1500$ Hz

Donc: $f_c > f_a > f_b$

## Conclusion

La proposition correcte est: $f_c > f_a > f_b$

Cette réponse correspond à la troisième proposition dans l'énoncé. Lorsque la source tourne, la fréquence perçue est plus élevée quand la source se rapproche (position c), égale à la fréquence émise quand la source se déplace perpendiculairement (position a), et plus basse quand la source s'éloigne (position b).

The model uses the language of the question to respond. After a thoughtful analysis, it says that the correct answer is f_c > f_a > f_b (and it is right!)

The beauty of this interaction was how seamlessly the Converse API handled the multimodal inputs. As a builder, I didn’t need to worry about the complexity of processing images alongside text—the API managed that complexity and returned structured responses that my simple application could present directly to my daughter.

Here is the code I wrote. I used the Swift programming language, just to show that Python is not the only option you have 😇.

private let modelId = "us.mistral.pixtral-large-2502-v1:0"

// Define the system prompt that instructs Claude how to respond
let systemPrompt = """
You are a math and physics tutor. Your task is to:
1. Read and understand the math or physics problem in the image
2. Provide a clear, step-by-step solution to the problem
3. Briefly explain any relevant concepts used in solving the problem
4. Be precise and accurate in your calculations
5. Use mathematical notation when appropriate

Format your response with clear section headings and numbered steps.
"""
let system: BedrockRuntimeClientTypes.SystemContentBlock = .text(systemPrompt)

// Create the user message with text prompt and image
let userPrompt = "Please solve this math or physics problem. Show all steps and explain the concepts involved."
let prompt: BedrockRuntimeClientTypes.ContentBlock = .text(userPrompt)
let image: BedrockRuntimeClientTypes.ContentBlock = .image(.init(format: .jpeg, source: .bytes(finalImageData)))

// Create the user message with both text and image content
let userMessage = BedrockRuntimeClientTypes.Message(
    content: [prompt, image],
    role: .user
)

// Initialize the messages array with the user message
var messages: [BedrockRuntimeClientTypes.Message] = []
messages.append(userMessage)

// Configure the inference parameters
let inferenceConfig: BedrockRuntimeClientTypes.InferenceConfiguration = .init(maxTokens: 4096, temperature: 0.0)

// Create the input for the Converse API with streaming
let input = ConverseStreamInput(inferenceConfig: inferenceConfig, messages: messages, modelId: modelId, system: [system])

// Make the streaming request
do {
    // Process the stream
    let response = try await bedrockClient.converseStream(input: input)

    // Iterate through the stream events
    for try await event in stream {
        switch event {
        case .messagestart:
            print("AI-assistant started to stream")

        case let .contentblockdelta(deltaEvent):
            // Handle text content as it arrives
            if case let .text(text) = deltaEvent.delta {
                DispatchQueue.main.async {
                    self.streamedResponse += text
                }
            }

        case .messagestop:
            print("Stream ended")
            // Create a complete assistant message from the streamed response
            let assistantMessage = BedrockRuntimeClientTypes.Message(
                content: [.text(self.streamedResponse)],
                role: .assistant
            )
            messages.append(assistantMessage)

        default:
            break
        }
    }

And the result in the app is stunning.

iOS Physics problem resolver

By the time her exam rolled around, she felt confident and prepared—and I had a compelling real-world example of how our multimodal capabilities in Amazon Bedrock can create meaningful experiences for users.

Get started today
The new model is available through these Regional API endpoints: US East (Ohio, N. Virginia), US West (Oregon), and Europe (Frankfurt, Ireland, Paris, Stockholm). This Regional availability helps you meet data residency requirements while minimizing latency.

You can start using the model through either the AWS Management Console or programmatically through the AWS Command Line Interface (AWS CLI) and AWS SDK using the model ID mistral.pixtral-large-2502-v1:0.

This launch represents a significant step forward in making advanced multimodal AI accessible to developers and organizations of all sizes. By combining Mistral AI’s cutting-edge model with AWS serverless infrastructure, you can now focus on building innovative applications without worrying about the underlying complexity.

Visit the Amazon Bedrock console today to start experimenting with Pixtral Large 25.02 and discover how it can enhance your AI-powered applications.

— seb


How is the News Blog doing? Take this 1 minute survey!

(This survey is hosted by an external company. AWS handles your information as described in the AWS Privacy Notice. AWS will own the data gathered via this survey and will not share the information collected with survey respondents.)

AWS Weekly Review: Amazon EKS, Amazon OpenSearch, Amazon API Gateway, and more (April 7, 2025)

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-weekly-review-amazon-eks-amazon-opensearch-amazon-api-gateway-and-more-april-7-2025/

AWS Summit season starts this week! These free events are now rolling out worldwide, bringing our cloud computing community together to connect, collaborate, and learn. Whether you prefer joining us online or in-person, these gatherings offer valuable opportunities to expand your AWS knowledge. I will be attending the Summit in Paris this week, the biggest cloud conference in France, and the London Summit at the end of the month. We will have a small podcast recording studio where I will interview French and British customers to produce new episodes for the AWS Developers Podcast and le podcast 🎙 AWS ☁ en 🇫🇷.

Register today!

But for now, let’s look at last week’s new announcements.

Last week’s launches
At KubeCon London, we introduced the EKS Community Add-Ons Catalog, making it simpler for Kubernetes users to enhance their Amazon EKS clusters with powerful open-source tools. This catalog streamlines the installation of essential add-ons like metrics-serverkube-state-metricsprometheus-node-exportercert-manager, and external-dns. By integrating these community-driven add-ons directly into the EKS console and AWS command line interface (AWS CLI), customers can reduce operational complexity and accelerate deployment while maintaining flexibility and security. This launch reflects AWS’s commitment to the Kubernetes community, providing seamless access to trusted open-source solutions without the overhead of manual installation and maintenance.

Amazon Q Developer now integrates with Amazon OpenSearch Service to enhance operational analytics by enabling natural language exploration and AI-assisted data visualization. This integration simplifies the process of querying and visualizing operational data, reducing the learning curve associated with traditional query languages and tools. During incident responses, Amazon Q Developer offers contextual summaries and insights directly within the alerts interface, facilitating quicker analysis and resolution. This advancement allows engineers to focus more on innovation by streamlining troubleshooting processes and improving monitoring infrastructure.

Amazon API Gateway now supports dual-stack (IPv4 and IPv6) endpoints across all endpoint types, custom domains, and management APIs in both commercial and AWS GovCloud (US) Regions. This enhancement allows REST, HTTP, and WebSocket APIs, as well as custom domains, to handle requests from both IPv4 and IPv6 clients, facilitating a smoother transition to IPv6 and addressing IPv4 address scarcity. Additionally, AWS continues its commitment to IPv6 adoption with recent updates, including AWS Identity and Access Management (IAM) introducing dual-stack public endpoints for seamless connections over IPv4 and IPv6, and AWS Resource Access Manager (RAM) enabling customers to manage resource shares using IPv6 addresses. Amazon Security Lake customers can also now use Internet Protocol version 6 (IPv6) addresses via new dual-stack endpoints to configure and manage the service. These advancements collectively ensure broader compatibility and future-proofing of network infrastructure.

Amazon SES has introduced support for email attachments in its v2 APIs, enabling users to include files like PDFs and images directly in their emails without manually constructing MIME messages. This enhancement simplifies the process of sending rich email content and reduces implementation complexity. Amazon Simple Email Service (Amazon SES) supports attachments in all AWS Regions where the service is available.

Amazon Neptune has updated its Service Level Agreement (SLA) to offer a 99.99% Monthly Uptime Percentage for Multi-AZ DB Instance, Multi-AZ DB Cluster, and Multi-AZ Graph configurations, up from the previous 99.9%. This enhancement demonstrates the commitment AWS has to providing highly available and reliable graph database services for mission-critical applications. The improved SLA is now available in all AWS Regions where Amazon Neptune is offered.

For a full list of AWS announcements, be sure to keep an eye on the What’s New at AWS page.

Other AWS events
Check your calendar and sign up for upcoming AWS events.

AWS GenAI Lofts are collaborative spaces and immersive experiences that showcase AWS expertise in cloud computing and AI. They provide startups and developers with hands-on access to AI products and services, exclusive sessions with industry leaders, and valuable networking opportunities with investors and peers. Find a GenAI Loft location near you and don’t forget to register.

Browse all upcoming AWS led in-person and virtual events here.

That’s all for this week. Check back next Monday for another Weekly Roundup!

— seb

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!


How is the News Blog doing? Take this 1 minute survey!

(This survey is hosted by an external company. AWS handles your information as described in the AWS Privacy Notice. AWS will own the data gathered via this survey and will not share the information collected with survey respondents.)

Accelerating CI with AWS CodeBuild: Parallel test execution now available

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/accelerating-ci-with-aws-codebuild-parallel-test-execution-now-available/

I’m excited to announce that AWS CodeBuild now supports parallel test execution, so you can run your test suites concurrently and reduce build times significantly.

With the demo project I wrote for this post, the total test time went down from 35 minutes to six minutes, including the time to provision the environments. These two screenshots from the AWS Management Console show the difference.

Sequential execution of the test suite

CodeBuild Parallel Test Results

Parallel execution of the test suite

CodeBuild Parallel Test Results

Very long test times pose a significant challenge when running continuous integration (CI) at scale. As projects grow in complexity and team size, the time required to execute comprehensive test suites can increase dramatically, leading to extended pipeline execution times. This not only delays the delivery of new features and bug fixes, but also hampers developer productivity by forcing them to wait for build results before proceeding with their tasks. I have experienced pipelines that took up to 60 minutes to run, only to fail at the last step, requiring a complete rerun and further delays. These lengthy cycles can erode developer trust in the CI process, contribute to frustration, and ultimately slow down the entire software delivery cycle. Moreover, long-running tests can lead to resource contention, increased costs because of wasted computing power, and reduced overall efficiency of the development process.

With parallel test execution in CodeBuild, you can now run your tests concurrently across multiple build compute environments. This feature implements a sharding approach where each build node independently executes a subset of your test suite. CodeBuild provides environment variables that identify the current node number and the total number of nodes, which are used to determine which tests each node should run. There is no control build node or coordination between nodes at build time—each node operates independently to execute its assigned portion of your tests.

To enable test splitting, configure the batch fanout section in your buildspec.xml, specifying the desired parallelism level and other relevant parameters. Additionally, use the codebuild-tests-run utility in your build step, along with the appropriate test commands and the chosen splitting method.

The tests are split based on the sharding strategy you specify. codebuild-tests-run offers two sharding strategies:

  • Equal-distribution. This strategy sorts test files alphabetically and distributes them in chunks equally across parallel test environments. Changes in the names or quantity of test files might reassign files across shards.
  • Stability. This strategy fixes the distribution of tests across shards by using a consistent hashing algorithm. It maintains existing file-to-shard assignments when new files are added or removed.

CodeBuild supports automatic merging of test reports when running tests in parallel. With automatic test report merging, CodeBuild consolidates tests reports into a single test summary, simplifying result analysis. The merged report includes aggregated pass/fail statuses, test durations, and failure details, reducing the need for manual report processing. You can view the merged results in the CodeBuild console, retrieve them using the AWS Command Line Interface (AWS CLI), or integrate them with other reporting tools to streamline test analysis.

Let’s look at how it works
Let me demonstrate how to implement parallel testing in a project. For this demo, I created a very basic Python project with hundreds of tests. To speed things up, I asked Amazon Q Developer on the command line to create a project and 1,800 test cases. Each test case is in a separate file and takes one second to complete. Running all tests in a sequence requires 30 minutes, excluding the time to provision the environment.

In this demo, I run the test suite on ten compute environments in parallel and measure how long it takes to run the suite.

To do so, I added a buildspec.yml file to my project.

version: 0.2

batch:
  fast-fail: false
  build-fanout:
    parallelism: 10 # ten runtime environments 
    ignore-failure: false

phases:
  install:
    commands:
      - echo 'Installing Python dependencies'
      - dnf install -y python3 python3-pip
      - pip3 install --upgrade pip
      - pip3 install pytest
  build:
    commands:
      - echo 'Running Python Tests'
      - |
         codebuild-tests-run \
          --test-command 'python -m pytest --junitxml=report/test_report.xml' \
          --files-search "codebuild-glob-search 'tests/test_*.py'" \
          --sharding-strategy 'equal-distribution'
  post_build:
    commands:
      - echo "Test execution completed"

reports:
  pytest_reports:
    files:
      - "*.xml"
    base-directory: "report"
    file-format: JUNITXML 

There are three parts to highlight in the YAML file.

First, there’s a build-fanout section under batch. The parallelism command tells CodeBuild how many test environments to run in parallel. The ignore-failure command indicates if failure in any of the fanout build tasks can be ignored.

Second, I use the pre-installed codebuild-tests-run command to run my tests.

This command receives the complete list of test files and decides which of the tests must be run on the current node.

  • Use the sharding-strategy argument to choose between equally distributed or stable distribution as I explain above.
  • Use the files-search argument to pass all the files that are candidates for a run. We recommend to use the provided codebuild-glob-search command for performance reasons, but any file search tool, such as find(1), will work.
  • I pass the actual test command to run on the shard with the test-command argument.

Lastly, the reports section instructs CodeBuild to collect and merge the test reports on each node.

Then, I open the CodeBuild console to create a project and a batch build configuration for this project. There’s nothing new here, so I’ll spare you the details. The documentation has all the details to get you startedParallel testing works on batch builds. Make sure to configure your project to run in batch.

CodeBuild : create a batch build

Now, I’m ready to trigger an execution of the test suite. I can commit new code on my GitHub repository or trigger the build in the console.

CodeBuild : trigger a new build

After a few minutes, I see a status report of the different steps of the build; with a status for each test environment or shard.

CodeBuild: status

When the test is complete, I select the Reports tab to access the merged test reports.

CodeBuild: test reports

The Reports section aggregates all test data from all shards and keeps the history for all builds. I select my most recent build in the Report history section to access the detailed report.

CodeBuild: Test Report

As expected, I can see the aggregated and the individual status for each of my 1,800 test cases. In this demo, they’re all passing, and the report is green.

The 1,800 tests of the demo project take one second each to complete. When I run this test suite sequentially, it took 35 minutes to complete. When I run the test suite in parallel on ten compute environments, it took six minutes to complete, including the time to provision the environments. The parallel run took 17.1 percent of the time of the sequential run. Actual numbers will vary with your projects.

Additional things to know
This new capability is compatible with all testing frameworks. The documentation includes examples for Django, Elixir, Go, Java (Maven), Javascript (Jest), Kotlin, PHPUnit, Pytest, Ruby (Cucumber), and Ruby (RSpec).

For test frameworks that don’t accept space-separated lists, the codebuild-tests-run CLI provides a flexible alternative through the CODEBUILD_CURRENT_SHARD_FILES environment variable. This variable contains a newline-separated list of test file paths for the current build shard. You can use it to adapt to different test framework requirements and format test file names.

You can further customize how tests are split across environments by writing your own sharding script and using the CODEBUILD_BATCH_BUILD_IDENTIFIER environment variable, which is automatically set in each build. You can use this technique to implement framework-specific parallelization or optimization.

Pricing and availability
With parallel test execution, you can now complete your test suites in a fraction of the time previously required, accelerating your development cycle and improving your team’s productivity. The demo project I created to illustrate this post consumes 18.7 percent of the time of a sequential build.

Parallel test execution is available on all three compute modes offered by CodeBuild: on-demand, reserved capacity, and AWS Lambda compute.

This capability is available today in all AWS Regions where CodeBuild is offered, with no additional cost beyond the standard CodeBuild pricing for the compute resources used.

I invite you to try parallel test execution in CodeBuild today. Visit the AWS CodeBuild documentation to learn more and get started with parallelizing your tests.

— seb

PS: Here’s the prompt I used to create the demo application and its test suite: “I’m writing a blog post to announce codebuild parallel testing. Write a very simple python app that has hundreds of tests, each test in a separate test file. Each test takes one second to complete.”


How is the News Blog doing? Take this 1 minute survey!

(This survey is hosted by an external company. AWS handles your information as described in the AWS Privacy Notice. AWS will own the data gathered via this survey and will not share the information collected with survey respondents.)

Firewall support for AWS Amplify hosted sites

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/firewall-support-for-aws-amplify-hosted-sites/

Today, we’re announcing the general availability of the AWS WAF integration with AWS Amplify Hosting.

Web application owners are constantly working to protect their applications from a variety of threats. Previously, if you wanted to implement a robust security posture for your Amplify Hosted applications, you needed to create architectures using Amazon CloudFront distributions with AWS WAF protection, which required additional configuration steps, expertise, and management overhead.

With the general availability of AWS WAF in Amplify Hosting, you can now directly attach a web application firewall to your AWS Amplify apps through a one-click integration in the Amplify console or using infrastructure as code (IaC). This integration gives you access to the full range of AWS WAF capabilities including managed rules, which provide protection against common web exploits and vulnerabilities like SQL injection and cross-site scripting (XSS). You can also create your own custom rules based on your specific application needs.

This new capability helps you implement defense-in-depth security strategies for your web applications. You can take advantage of AWS WAF rate-based rules to protect against distributed denial of service (DDoS) attacks by limiting the rate of requests from IP addresses. Additionally, you can implement geo-blocking to restrict access to your applications from specific countries, which is particularly valuable if your service is designed for specific geographic regions.

Let’s see how it works
Setting up AWS WAF protection for your Amplify app is straightforward. From the Amplify console, navigate to your app settings, select the Firewall tab, and choose the predefined rules you want to apply to your configuration. AWS WAF integration in AWS Amplify Hosting

Amplify hosting simplifies configuring firewall rules. You can activate four categories of protection.

  • Amplify-recommended firewall protection – Protect against the most common vulnerabilities found in web applications, block IP addresses from potential threats based on Amazon internal threat intelligence, and protect against malicious actors discovering application vulnerabilities.
  • Restrict access to amplifyapp.com – Restrict access to the default Amplify generated amplifyapp.com domain. This is useful when you add a custom domain to prevent bots and search engines from crawling the domain.
  • Enable IP address protection – Restrict web traffic by allowing or blocking requests from specified IP address ranges.
  • Enable country protection – Restrict access based on specific countries.

Protections enabled through the Amplify console will create an underlying web access control list (ACL) in your AWS account. For fine-grained rulesets, you can use the AWS WAF console rule builder.

After a few minutes, the rules are associated to your app and AWS WAF blocks suspicious requests.

If you want to see AWS WAF in action, you can simulate an attack and monitor it using the AWS WAF request inspection capabilities. For example, you can send a request with an empty User-Agent value. It will trigger a blocking rule in AWS WAF.

Let’s first send a valid request to my app.

curl -v -H "User-Agent: MyUserAgent" https://main.d3sk5bt8rx6f9y.amplifyapp.com/
* Host main.d3sk5bt8rx6f9y.amplifyapp.com:443 was resolved.
...(redacted for brevity)...
> GET / HTTP/2
> Host: main.d3sk5bt8rx6f9y.amplifyapp.com
> Accept: */*
> User-Agent: MyUserAgent
> 
* Request completely sent off
< HTTP/2 200 
< content-type: text/html
< content-length: 0
< date: Mon, 10 Mar 2025 14:45:26 GMT
 

We can observe that the server returned an HTTP 200 (OK) message.

Then, send a request with no value associated to the User-Agent HTTP header.

 curl -v -H "User-Agent: " https://main.d3sk5bt8rx6f9y.amplifyapp.com/ 
* Host main.d3sk5bt8rx6f9y.amplifyapp.com:443 was resolved.
... (redacted for brevity) ...
> GET / HTTP/2
> Host: main.d3sk5bt8rx6f9y.amplifyapp.com
> Accept: */*
> 
* Request completely sent off
< HTTP/2 403 
< server: CloudFront
... (redacted for brevity) ...
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD><BODY>
<H1>403 ERROR</H1>
<H2>The request could not be satisfied.</H2>

We can observe that the server returned an HTTP 403 (Forbidden) message.

AWS WAF provide visibility into request patterns, helping you fine-tune your security settings over time. You can access logs through Amplify Hosting or the AWS WAF console to analyze traffic trends and refine security rules as needed.

AWS WAF integration in AWS Amplify Hosting - Dashboard

Availability and pricing
Firewall support is available in all AWS Regions in which Amplify Hosting operates. This integration falls under an AWS WAF global resource, similar to Amazon CloudFront. Web ACLs can be attached to multiple Amplify Hosting apps, but they must reside in the same Region.

The pricing for this integration follows the standard AWS WAF pricing model, You pay for the AWS WAF resources you use based on the number of web ACLs, rules, and requests. On top of that, AWS Amplify Hosting adds $15/month when you attach a web application firewall to your application. This is prorated by the hour.

This new capability brings enterprise-grade security features to all Amplify Hosting customers, from individual developers to large enterprises. You can now build, host, and protect your web applications within the same service, reducing the complexity of your architecture and streamlining your security management.

To learn more, visit the AWS WAF integration documentation for Amplify or try it directly in the Amplify console.

— seb


How is the News Blog doing? Take this 1 minute survey!

(This survey is hosted by an external company. AWS handles your information as described in the AWS Privacy Notice. AWS will own the data gathered via this survey and will not share the information collected with survey respondents.)

AWS Pi Day 2025: Data foundation for analytics and AI

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-pi-day-data-foundation-for-analytics-and-ai/

Every year on March 14 (3.14), AWS Pi Day highlights AWS innovations that help you manage and work with your data. What started in 2021 as a way to commemorate the fifteenth launch anniversary of Amazon Simple Storage Service (Amazon S3) has now grown into an event that highlights how cloud technologies are transforming data management, analytics, and AI.

This year, AWS Pi Day returns with a focus on accelerating analytics and AI innovation with a unified data foundation on AWS. The data landscape is undergoing a profound transformation as AI emerges in most enterprise strategies, with analytics and AI workloads increasingly converging around a lot of the same data and workflows. You need an easy way to access all your data and use all your preferred analytics and AI tools in a single integrated experience. This AWS Pi Day, we’re introducing a slate of new capabilities that help you build unified and integrated data experiences.

The next generation of Amazon SageMaker: The center of all your data, analytics, and AI
At re:Invent 2024, we introduced the next generation of Amazon SageMaker, the center of all your data, analytics, and AI. SageMaker includes virtually all the components you need for data exploration, preparation and integration, big data processing, fast SQL analytics, machine learning (ML) model development and training, and generative AI application development. With this new generation of Amazon SageMaker, SageMaker Lakehouse provides you with unified access to your data and SageMaker Catalog helps you to meet your governance and security requirements. You can read the launch blog post written by my colleague Antje to learn more details.

Core to the next generation of Amazon SageMaker is SageMaker Unified Studio, a single data and AI development environment where you can use all your data and tools for analytics and AI. SageMaker Unified Studio is now generally available.

SageMaker Unified Studio facilitates collaboration among data scientists, analysts, engineers, and developers as they work on data, analytics, AI workflows, and applications. It provides familiar tools from AWS analytics and artificial intelligence and machine learning (AI/ML) services, including data processing, SQL analytics, ML model development, and generative AI application development, into a single user experience.

SageMaker Unified Studio

SageMaker Unified Studio also brings selected capabilities from Amazon Bedrock into SageMaker. You can now rapidly prototype, customize, and share generative AI applications using foundation models (FMs) and advanced features such as Amazon Bedrock Knowledge BasesAmazon Bedrock Guardrails, Amazon Bedrock Agents, and Amazon Bedrock Flows to create tailored solutions aligned with your requirements and responsible AI guidelines all within SageMaker.

Last but not least, Amazon Q Developer is now generally available in SageMaker Unified Studio. Amazon Q Developer provides generative AI powered assistance for data and AI development. It helps you with tasks like writing SQL queries, building extract, transform, and load (ETL) jobs, and troubleshooting, and is available in the Free tier and Pro tier for existing subscribers.

You can learn more about SageMaker Unified Studio in this recent blog post written by my colleague Donnie.

During re:Invent 2024, we also launched Amazon SageMaker Lakehouse as part of the next generation of SageMaker. SageMaker Lakehouse unifies all your data across Amazon S3 data lakes, Amazon Redshift data warehouses, and third-party and federated data sources. It helps you build powerful analytics and AI/ML applications on a single copy of your data. SageMaker Lakehouse gives you the flexibility to access and query your data in-place with Apache Iceberg–compatible tools and engines. In addition, zero-ETL integrations automate the process of bringing data into SageMaker Lakehouse from AWS data sources such as Amazon Aurora or Amazon DynamoDB and from applications such as Salesforce, Facebook Ads, Instagram Ads, ServiceNow, SAP, Zendesk, and Zoho CRM. The full list of integrations is available in the SageMaker Lakehouse FAQ.

Building a data foundation with Amazon S3
Building a data foundation is the cornerstone of accelerating analytics and AI workloads, enabling organizations to seamlessly manage, discover, and utilize their data assets at any scale. Amazon S3 is the world’s best place to build a data lake, with virtually unlimited scale, and it provides the essential foundation for this transformation.

I’m always astonished to learn about the scale at which we operate Amazon S3: It currently holds over 400 trillion objects, exabytes of data, and processes a mind-blowing 150 million requests per second. Just a decade ago, not even 100 customers were storing more than a petabyte (PB) of data on S3. Today, thousands of customers have surpassed the 1 PB milestone.

Amazon S3 stores exabytes of tabular data, and it averages over 15 million requests to tabular data per second. To help you reduce the undifferentiated heavy lifting when managing your tabular data in S3 buckets, we announced Amazon S3 Tables at AWS re:Invent 2024. S3 Tables are the first cloud object store with built-in support for Apache Iceberg. S3 tables are specifically optimized for analytics workloads, resulting in up to threefold faster query throughput and up to tenfold higher transactions per second compared to self-managed tables.

Today, we’re announcing the general availability of Amazon S3 Tables integration with Amazon SageMaker Lakehouse  Amazon S3 Tables now integrate with Amazon SageMaker Lakehouse, making it easy for you to access S3 Tables from AWS analytics services such as Amazon Redshift, Amazon Athena, Amazon EMR, AWS Glue, and Apache Iceberg–compatible engines such as Apache Spark or PyIceberg. SageMaker Lakehouse enables centralized management of fine-grained data access permissions for S3 Tables and other sources and consistently applies them across all engines.

For those of you who use a third-party catalog, have a custom catalog implementation, or only need basic read and write access to tabular data in a single table bucket, we’ve added new APIs that are compatible with the Iceberg REST Catalog standard. This enables any Iceberg-compatible application to seamlessly create, update, list, and delete tables in an S3 table bucket. For unified data management across all of your tabular data, data governance, and fine-grained access controls, you can also use S3 Tables with SageMaker Lakehouse.

To help you access S3 Tables, we’ve launched updates in the AWS Management Console. You can now create a table, populate it with data, and query it directly from the S3 console using Amazon Athena, making it easier to get started and analyze data in S3 table buckets.

The following screenshot shows how to access Athena directly from the S3 console.

S3 console : create table with AthenaWhen I select Query tables with Athena or Create table with Athena, it opens the Athena console on the correct data source, catalog, and database.

S3 Tables in Athena

Since re:Invent 2024, we’ve continued to add new capabilities to S3 Tables at a rapid pace. For example, we added schema definition support to the CreateTable API and you can now create up to 10,000 tables in an S3 table bucket. We also launched S3 Tables into eight additional AWS Regions, with the most recent being Asia Pacific (Seoul, Singapore, Sydney) on March 4, with more to come. You can refer to the S3 Tables AWS Regions page of the documentation to get the list of the eleven Regions where S3 Tables are available today.

Amazon S3 Metadataannounced during re:Invent 2024— has been generally available since January 27. It’s the fastest and easiest way to help you discover and understand your S3 data with automated, effortlessly-queried metadata that updates in near real time. S3 Metadata works with S3 object tags. Tags help you logically group data for a variety of reasons, such as to apply IAM policies to provide fine-grained access, specify tag-based filters to manage object lifecycle rules, and selectively replicate data to another Region. In Regions where S3 Metadata is available, you can capture and query custom metadata that is stored as object tags. To reduce the cost associated with object tags when using S3 Metadata, Amazon S3 reduced pricing for S3 object tagging by 35 percent in all Regions, making it cheaper to use custom metadata.

AWS Pi Day 2025
Over the years, AWS Pi Day has showcased major milestones in cloud storage and data analytics. This year, the AWS Pi Day virtual event will feature a range of topics designed for developers and technical decision-makers, data engineers, AI/ML practitioners, and IT leaders. Key highlights include deep dives, live demos, and expert sessions on all the services and capabilities I discussed in this post.

By attending this event, you’ll learn how you can accelerate your analytics and AI innovation. You’ll learn how you can use S3 Tables with native Apache Iceberg support and S3 Metadata to build scalable data lakes that serve both traditional analytics and emerging AI/ML workloads. You’ll also discover the next generation of Amazon SageMaker, the center for all your data, analytics, and AI, to help your teams collaborate and build faster from a unified studio, using familiar AWS tools with access to all your data whether it’s stored in data lakes, data warehouses, or third-party or federated data sources.

For those looking to stay ahead of the latest cloud trends, AWS Pi Day 2025 is an event you can’t miss. Whether you’re building data lakehouses, training AI models, building generative AI applications, or optimizing analytics workloads, the insights shared will help you maximize the value of your data.

Tune in today and explore the latest in cloud data innovation. Don’t miss the opportunity to engage with AWS experts, partners, and customers shaping the future of data, analytics, and AI.

If you missed the virtual event on March 14, you can visit the event page at any time—we will keep all the content available on-demand there!

— seb


How is the News Blog doing? Take this 1 minute survey!

(This survey is hosted by an external company. AWS handles your information as described in the AWS Privacy Notice. AWS will own the data gathered via this survey and will not share the information collected with survey respondents.)

AWS CodeBuild for macOS adds support for Fastlane

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/codebuild-for-macos-adds-support-for-fastlane/

I’m pleased to announce the availability of Fastlane in your AWS CodeBuild for macOS environments. AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces ready-to-deploy software packages.

Fastlane is an open source tool suite designed to automate various aspects of mobile application development. It provides mobile application developers with a centralized set of tools to manage tasks such as code signing, screenshot generation, beta distribution, and app store submissions. It integrates with popular continuous integration and continuous deployment (CI/CD) platforms and supports both iOS and Android development workflows. Although Fastlane offers significant automation capabilities, developers may encounter challenges during its setup and maintenance. Configuring Fastlane can be complex, particularly for teams unfamiliar with the syntax and package management system of Ruby. Keeping Fastlane and its dependencies up to date requires ongoing effort, because updates to mobile platforms or third-party services may necessitate adjustments to existing workflows.

When we introduced CodeBuild for macOS in August 2024, we knew that one of your challenges was to install and maintain Fastlane in your build environment. Although it was possible to manually install Fastlane in a custom build environment, at AWS, we remove the undifferentiated heaving lifting from your infrastructure so you can spend more time on the aspects that matter for your business. Starting today, Fastlane is installed by default, and you can use the familiar command fastlane buildin your buildspec.yaml file.

Fastlane and code signing
To distribute an application on the App Store, developers must sign their binary with a private key generated on the Apple Developer portal. This private key, along with the certificate that validates it, must be accessible during the build process. This can be a challenge for development teams because they need to share the development private key (which allows deployment on selected test devices) among team members. Additionally, the distribution private key (which enables publishing on the App Store) must be available during the signing process before uploading the binary to the App Store.

Fastlane is a versatile build system in that it also helps developers with the management of development and distribution keys and certificates. Developers can use fastlane match to share signing materials in a team and make them securely and easily accessible on individual developers’ machines and on the CI environment. match allows the storage of private keys, the certificates, and the mobile provisioning profiles on a secured share storage. It makes sure that the local build environment, whether it’s a developer laptop or a server machine in the cloud, stays in sync with the shared storage. At build time, it securely downloads the required certificates to sign your app and configures the build machine to allow the codesign utility to pick them up.

match allows the sharing of signing secrets through GitHub, GitLab, Google Cloud Storage, Azure DevOps, and Amazon Simple Storage Service (Amazon S3).

If you already use one of these and you’re migrating your projects to CodeBuild, you don’t have much to do. You only need to make sure your CodeBuild build environment has access to the shared storage (see step 3 in the demo).

Let’s see how it works
If you’re new to Fastlane or CodeBuild, let’s see how it works.

For this demo, I start with an existing iOS project. The project is already configured to be built on CodeBuild. You can refer to my previous blog post, Add macOS to your continuous integration pipelines with AWS CodeBuild, to learn more details.

I’ll show you how to get started in three steps:

  • Import your existing signing materials to a shared private GitHub repository
  • Configure fastlane to build and sign your project
  • Use fastlanewith CodeBuild

Step 1: Import your signing materials

Most of the fastlane documentation I read explains how to create a new key pair and a new certificate to get started. Although this is certainly true for new projects, in real life, you probably already have your project and your signing keys. So, the first step is to import these existing signing materials.

Apple App Store uses different keys and certificates for development and distribution (there are also ad hoc and enterprise certificates, but these are outside the scope of this post). You must have three files for each usage (that’s a total of six files):

  • A .mobileprovision file that you can create and download from the Apple developer console. The provisioning profile links your identity, the app identity, and the entitlements the app might have.
  • A .cer file, which is the certificate emitted by Apple to validate your private key. You can download this from the Apple Developer portal. Select the certificate, then select Download.
  • A .p12 file, which contains your private key. You can download the key when you create it in the Apple Developer portal. If you didn’t download it but have it on your machine, you can export it from the Apple Keychain app. Note that the KeyChain.app is hidden in macOS 15.x. You can open it with open /System/Library/CoreServices/Applications/Keychain\ Access.app. Select the key you want to export and right click to select Export.
Export p12 files from Keychain

When you have these files, create a fastlane/Matchfile file with the following content:

git_url("https://github.com/sebsto/secret.git")
storage_mode("git")
type("development")
# or use appstore to use the distribution signing key and certificate
# type("appstore")

Be sure to replace the URL of your GitHub repository and make sure this repository is private. It will serve as a storage for your signing key and certificate.

Then, I import my existing files with the fastlane match import --type appstore command. I repeat the command for each environment: appstore and development.

The very first time, fastlane prompts me for my Apple Id username and password. It connects to App Store Connect to verify the validity of the certificates or to create new ones when necessary. The session cookie is stored in ~/.fastlane/spaceship/<your apple user id>/cookie.

fastlane match also asks for a password. It uses this password to generate a key to crypt the signing materials on the storage. Don’t forget this password because it will be used at build time to import the signing materials on the build machine.

Here is the command and its output in full:

 fastlane match import --type appstore

[✔] 🚀
[16:43:54]: Successfully loaded '~/amplify-ios-getting-started/code/fastlane/Matchfile' 📄

+-----------------------------------------------------+
| Detected Values from './fastlane/Matchfile'         |
+--------------+--------------------------------------+
| git_url.     | https://github.com/sebsto/secret.git |
| storage_mode | git                                  |
| type         | development                          |
+--------------+--------------------------------------+

[16:43:54]: Certificate (.cer) path:
./secrets/sebsto-apple-dist.cer
[16:44:07]: Private key (.p12) path:
./secrets/sebsto-apple-dist.p12
[16:44:12]: Provisioning profile (.mobileprovision or .provisionprofile) path or leave empty to skip
this file:
./secrets/amplifyiosgettingstarteddist.mobileprovision
[16:44:25]: Cloning remote git repo...
[16:44:25]: If cloning the repo takes too long, you can use the `clone_branch_directly` option in match.
[16:44:27]: Checking out branch master...
[16:44:27]: Enter the passphrase that should be used to encrypt/decrypt your certificates
[16:44:27]: This passphrase is specific per repository and will be stored in your local keychain
[16:44:27]: Make sure to remember the password, as you'll need it when you run match on a different machine
[16:44:27]: Passphrase for Match storage: ********
[16:44:30]: Type passphrase again: ********
security: SecKeychainAddInternetPassword <NULL>: The specified item already exists in the keychain.
[16:44:31]: 🔓 Successfully decrypted certificates repo
[16:44:31]: Repo is at: '/var/folders/14/nwpsn4b504gfp02_mrbyd2jr0000gr/T/d20250131-41830-z7b4ic'
[16:44:31]: Login to App Store Connect ([email protected])
[16:44:33]: Enter the passphrase that should be used to encrypt/decrypt your certificates
[16:44:33]: This passphrase is specific per repository and will be stored in your local keychain
[16:44:33]: Make sure to remember the password, as you'll need it when you run match on a different machine
[16:44:33]: Passphrase for Match storage: ********
[16:44:37]: Type passphrase again: ********
security: SecKeychainAddInternetPassword <NULL>: The specified item already exists in the keychain.
[16:44:39]: 🔒 Successfully encrypted certificates repo
[16:44:39]: Pushing changes to remote git repo...
[16:44:40]: Finished uploading files to Git Repo [https://github.com/sebsto/secret.git]

I verify that Fastlane imported my signing material to my Git repository.

Fastlane match - github repo after import

I can also configure my local machine to use these signing materials during the next build:

» fastlane match appstore 

[✔] 🚀 
[17:39:08]: Successfully loaded '~/amplify-ios-getting-started/code/fastlane/Matchfile' 📄

+-----------------------------------------------------+
|   Detected Values from './fastlane/Matchfile'       |
+--------------+--------------------------------------+
| git_url      | https://github.com/sebsto/secret.git |
| storage_mode | git                                  |
| type         | development                          |
+--------------+--------------------------------------+


+-------------------------------------------------------------------------------------------+
|                                 Summary for match 2.226.0                                 |
+----------------------------------------+--------------------------------------------------+
| type                                   | appstore                                         |
| readonly                               | false                                            |
| generate_apple_certs                   | true                                             |
| skip_provisioning_profiles             | false                                            |
| app_identifier                         | ["com.amazonaws.amplify.mobile.getting-started"] |
| username                               | xxxx@xxxxxxxxx                                   |
| team_id                                | XXXXXXXXXX                                       |
| storage_mode                           | git                                              |
| git_url                                | https://github.com/sebsto/secret.git             |
| git_branch                             | master                                           |
| shallow_clone                          | false                                            |
| clone_branch_directly                  | false                                            |
| skip_google_cloud_account_confirmation | false                                            |
| s3_skip_encryption                     | false                                            |
| gitlab_host                            | https://gitlab.com                               |
| keychain_name                          | login.keychain                                   |
| force                                  | false                                            |
| force_for_new_devices                  | false                                            |
| include_mac_in_profiles                | false                                            |
| include_all_certificates               | false                                            |
| force_for_new_certificates             | false                                            |
| skip_confirmation                      | false                                            |
| safe_remove_certs                      | false                                            |
| skip_docs                              | false                                            |
| platform                               | ios                                              |
| derive_catalyst_app_identifier         | false                                            |
| fail_on_name_taken                     | false                                            |
| skip_certificate_matching              | false                                            |
| skip_set_partition_list                | false                                            |
| force_legacy_encryption                | false                                            |
| verbose                                | false                                            |
+----------------------------------------+--------------------------------------------------+

[17:39:08]: Cloning remote git repo...
[17:39:08]: If cloning the repo takes too long, you can use the `clone_branch_directly` option in match.
[17:39:10]: Checking out branch master...
[17:39:10]: Enter the passphrase that should be used to encrypt/decrypt your certificates
[17:39:10]: This passphrase is specific per repository and will be stored in your local keychain
[17:39:10]: Make sure to remember the password, as you'll need it when you run match on a different machine
[17:39:10]: Passphrase for Match storage: ********
[17:39:13]: Type passphrase again: ********
security: SecKeychainAddInternetPassword <NULL>: The specified item already exists in the keychain.
[17:39:15]: 🔓  Successfully decrypted certificates repo
[17:39:15]: Verifying that the certificate and profile are still valid on the Dev Portal...
[17:39:17]: Installing certificate...

+-------------------------------------------------------------------------+
|                          Installed Certificate                          |
+-------------------+-----------------------------------------------------+
| User ID           | XXXXXXXXXX                                          |
| Common Name       | Apple Distribution: Sebastien Stormacq (XXXXXXXXXX) |
| Organisation Unit | XXXXXXXXXX                                          |
| Organisation      | Sebastien Stormacq                                  |
| Country           | US                                                  |
| Start Datetime    | 2024-10-29 09:55:43 UTC                             |
| End Datetime      | 2025-10-29 09:55:42 UTC                             |
+-------------------+-----------------------------------------------------+

[17:39:18]: Installing provisioning profile...

+-------------------------------------------------------------------------------------------------------------------+
|                                          Installed Provisioning Profile                                           |
+---------------------+----------------------------------------------+----------------------------------------------+
| Parameter           | Environment Variable                         | Value                                        |
+---------------------+----------------------------------------------+----------------------------------------------+
| App Identifier      |                                              | com.amazonaws.amplify.mobile.getting-starte  |
|                     |                                              | d                                            |
| Type                |                                              | appstore                                     |
| Platform            |                                              | ios                                          |
| Profile UUID        | sigh_com.amazonaws.amplify.mobile.getting-s  | 4e497882-d80f-4684-945a-8bfec1b310b9         |
|                     | tarted_appstore                              |                                              |
| Profile Name        | sigh_com.amazonaws.amplify.mobile.getting-s  | amplify-ios-getting-started-dist             |
|                     | tarted_appstore_profile-name                 |                                              |
| Profile Path        | sigh_com.amazonaws.amplify.mobile.getting-s  | /Users/stormacq/Library/MobileDevice/Provis  |
|                     | tarted_appstore_profile-path                 | ioning                                       |
|                     |                                              | Profiles/4e497882-d80f-4684-945a-8bfec1b310  |
|                     |                                              | b9.mobileprovision                           |
| Development Team ID | sigh_com.amazonaws.amplify.mobile.getting-s  | XXXXXXXXXX                                   |
|                     | tarted_appstore_team-id                      |                                              |
| Certificate Name    | sigh_com.amazonaws.amplify.mobile.getting-s  | Apple Distribution: Sebastien Stormacq       |
|                     | tarted_appstore_certificate-name             | (XXXXXXXXXX)                                 |
+---------------------+----------------------------------------------+----------------------------------------------+

[17:39:18]: All required keys, certificates and provisioning profiles are installed 🙌

Step 2: Configure Fastlane to sign your project

I create a Fastlane build configuration file in fastlane/Fastfile (you can use fastlane init command to get started):

default_platform(:ios)

platform :ios do
  before_all do
    setup_ci
  end

  desc "Build and Sign the binary"
  lane :build do
    match(type: "appstore", readonly: true)
    gym(
      scheme: "getting started",
      export_method: "app-store"
    )
  end
end

Make sure that the setup_ci action is added to the before_all section of Fastfile for the match action to function correctly. This action creates a temporary Fastlane keychain with correct permissions. Without this step, you may encounter build failures or inconsistent results.

And I test a local build with the command fastlane build. I enter the password I used when importing my keys and certificate, then I let the system build and sign my project. When everything is correctly configured, it produces a similar output.

...
[17:58:33]: Successfully exported and compressed dSYM file
[17:58:33]: Successfully exported and signed the ipa file:
[17:58:33]: ~/amplify-ios-getting-started/code/getting started.ipa

+---------------------------------------+
|           fastlane summary            |
+------+------------------+-------------+
| Step | Action           | Time (in s) |
+------+------------------+-------------+
| 1    | default_platform | 0           |
| 2    | setup_ci         | 0           |
| 3    | match            | 36          |
| 4    | gym              | 151         |
+------+------------------+-------------+

[17:58:33]: fastlane.tools finished successfully 🎉

Step 3: Configure CodeBuild to use Fastlane

Next, I create a project on CodeBuild. I’m not going into the step-by-step guide to help you to do so. You can refer to my previous post or to the CodeBuild documentation.

There is just one Fastlane-specific configuration. To access the signing materials, Fastlane requires access to three secret values that I’ll pass as environment variables:

  • MATCH_PASSWORD, the password I entered when importing the signing material. Fastlane uses this password to decipher the encrypted files in the GitHub repository
  • FASTLANE_SESSION, the value of the Apple Id session cookie, located at ~/.fastlane/spaceship/<your apple user id>/cookie. The session is valid from a couple of hours to multiple days. When the session expires, reauthenticate with the command fastlane spaceauth from your laptop and update the value of FASTLANE_SESSION with the new value of the cookie.
  • MATCH_GIT_BASIC_AUTHORIZATION, a base 64 encoding of your GitHub username, followed by a colon, followed by a personal authentication token (PAT) to access your private GitHub repository. You can generate PAT on the GitHub console in Your Profile > Settings > Developers Settings > Personal Access Token. I use this command to generate the value of this environment variable: echo -n my_git_username:my_git_pat | base64.

Note that for each of these three values, I can enter the Amazon Resource Name (ARN) of the secret on AWS Secrets Manager or the plain text value. We strongly recommend using Secrets Manager to store security-sensitive values.

I’m a security-conscious user, so I store the three secrets in Secrets Manager with these commands:

aws --region $REGION secretsmanager create-secret --name /CodeBuild/MATCH_PASSWORD --secret-string MySuperSecretPassword
aws --region $REGION secretsmanager create-secret --name /CodeBuild/FASTLANE_SESSION --secret-string $(cat ~/.fastlane/spaceship/my_appleid_username/cookie)
aws --region $REGION secretsmanager create-secret --name /CodeBuild/MATCH_GIT_BASIC_AUTHORIZATION --secret-string $(echo -n my_git_username:my_git_pat | base64)

If your build project refers to secrets stored in Secrets Manager, the build project’s service role must allow the secretsmanager:GetSecretValue action. If you chose New service role when you created your project, CodeBuild includes this action in the default service role for your build project. However, if you chose Existing service role, you must include this action to your service role separately.

For this demo, I use this AWS Identity and Access Management (IAM) policy:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Action": [
				"secretsmanager:GetSecretValue"
			],
			"Resource": [
				"arn:aws:secretsmanager:us-east-2:012345678912:secret:/CodeBuild/*"
			]
		}
	]
}

After I created the project in the CodeBuild section of the AWS Management Console, I enter the three environment variables. Notice that the value is the name of the secret in Secrets Manager.

Codebuild - environment variables for Fastlane match

You can also define the environment variables and their Secrets Manager secret name in your buildpsec.yaml file.

Next, I modify the buildspec.yaml file at the root of my project to use fastlane to build and sign the binary. My buildspec.yaml file now looks like this one:

# buildspec.yml
version: 0.2
phases:
  install:
    commands:
      - code/ci_actions/00_install_rosetta.sh
  pre_build:
    commands:
      - code/ci_actions/02_amplify.sh
  build:
    commands:
      - (cd code && fastlane build)
artifacts:
  name: getting-started-$(date +%Y-%m-%d).ipa
  files:
    - 'getting started.ipa'
  base-directory: 'code'

The Rosetta and Amplify scripts are required to receive the Amplify configuration for the backend. If you don’t use AWS Amplify in your project, you don’t need these.

Notice that there is nothing in the build file that downloads the signing key or prepares the keychain in the build environment; fastlane match will do that for me.

I add the new buildspec.yaml file and my ./fastlane directory to Git. I commit and push these files. git commit -m "add fastlane support" && git push

When everything goes well, I can see the build running on CodeBuild and the Succeeded message.

Codebuild - succeed message

Pricing and availability
Fastlane is now pre-installed at no extra cost on all macOS images that CodeBuild uses, in all Regions where CodeBuild for macOS is available. At the time of this writing, these are US East (Ohio, N. Virginia), US West (Oregon), Asia Pacific (Sydney), and Europe (Frankfurt).

In my experience, it takes a bit of time to configure fastlane match correctly. When it’s configured, having it working on CodeBuild is pretty straightforward. Before trying this on CodeBuild, be sure it works on your local machine. When something goes wrong on CodeBuild, triple-check the values of the environment variables and make sure CodeBuild has access to your secrets on AWS Secrets Manager.

Now go build (on macOS)!

AWS Weekly roundup: EventBridge, SNS FIFO, Amazon Corretto, Amazon Connect, Amazon Bedrock, and more

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-eventbridge-sns-fifo-amazon-corretto-amazon-connect-amazon-bedrock-and-more/

I counted about 40 new launches from AWS since last week – back to our normal rhythm of releases. Services teams are listening to your feedback and developing little (or big) changes that makes your life easier when working with our services. The ability to support multiple sessions in the AWS Console is my favorite one so far in 2025.

But our teams didn’t stop there, let’s look at the last week’s new announcements.

Last week’s launches

Beside the usual Regional expansion (new capabilities that are now available in a new Region), here are the launches that got my attention.

Amazon EventBridge announces direct delivery to cross-account targetsAmazon EventBridge is now able to deliver events to targets in another AWS account directly without having to send them to the default bus in the target account first. This will simplify so many architectures out there! It supports any target that supports resource-based policies, including AWS Lambda, Amazon Simple Queue Service (Amazon SQS), Amazon Simple Notification Service (Amazon SNS), Amazon Kinesis, and Amazon API Gateway.

Amazon Corretto quaterly update – We announced quarterly security and critical updates for Amazon Corretto Long-Term Supported (LTS) and Feature Release (FR) versions of OpenJDK. Corretto 23.0.2, 21.0.6, 17.0.14, 11.0.26, 8u442 are now available for download. Amazon Corretto is a no-cost, multi-platform, production-ready distribution of OpenJDK. You can download the updates from the Corretto home page or just type apt-get or yum update.

High-throughput mode for Amazon SNS FIFO Topics – Amazon SNS now supports high-throughput mode for SNS FIFO topics, with default throughput matching SNS standard topics across all Regions. When you enable high-throughput mode, SNS FIFO topics will maintain order within message group, while reducing the deduplication scope to the message-group level. With this change, you can leverage up to 30K messages per second (MPS) per account by default in US East (N. Virginia) Region, and 9K MPS per account in US West (Oregon) and Europe (Ireland) Regions, and request quota increases for additional throughput in any Region.

Amazon Connect agent workspace now supports audio optimization for Citrix and Amazon WorkSpaces virtual desktopsAmazon Connect agent workspace now supports the ability to redirect audio from Citrix and Amazon WorkSpaces Virtual Desktop Infrastructure (VDI) environments to a customer service agent’s local device. Audio redirection improves voice quality and reduces latency for voice calls handled on virtual desktops, providing a better experience for both end customers and agents.

Amazon Redshift announces support for History Mode for zero-ETL integrationsThis new capability enables you to build Type 2 Slowly Changing Dimension (SCD 2) tables on your historical data from databases, out-of-the-box in Amazon Redshift, without writing any code. History mode simplifies the process of tracking and analyzing historical data changes, allowing you to gain valuable insights from your data’s evolution over time.

Finally, Amazon Bedrock has its own set of announcements. First, for anyone investing in retrieval-augmented generation, Bedrock now support multimodal content with Cohere Embed 3 Multilingual and Embed 3 English models. This enables you to create embeddings to not only index text, but also images.

Second, read Luma AI’s Ray2 visual AI model now available in Amazon Bedrock. Luma Ray2 is a large-scale video-generation model capable of creating realistic visuals with fluid, natural movement. With Luma Ray2 in Amazon Bedrock, you can generate production-ready video clips with seamless animations, ultrarealistic details, and logical event sequences with natural language prompts, removing the need for technical prompt engineering. Ray2 currently supports 5- and 9-second video generations with 540p and 720p resolution.

And finally, Amazon Bedrock Flows announces preview of multi-turn conversation support. Amazon Bedrock Flows enables you to link foundation models (FMs), Amazon Bedrock Prompts, Amazon Bedrock Agents, Amazon Bedrock Knowledge Bases, Amazon Bedrock Guardrails and other AWS services together to build and scale pre-defined generative AI workflows. This week, the team announced preview of multi-turn conversation support for agent nodes in Flows. This capability enables dynamic, back-and-forth conversations between users and flows, similar to a natural dialogue.

For a full list of AWS announcements, be sure to keep an eye on the What’s New at AWS page.

Other AWS events
Check your calendar and sign up for upcoming AWS events.

AWS Summits season is starting! I’m already working with the local team to prepare content for the Summits in Paris and London. Summits are free online and in-person events that bring the cloud computing community together to connect, collaborate, and learn about AWS. Stay updated by visiting the official AWS Summit website and sign up for notifications to learn when registration opens for events in your area.

AWS GenAI Lofts are collaborative spaces and immersive experiences that showcase AWS expertise in cloud computing and AI. They provide startups and developers with hands-on access to AI products and services, exclusive sessions with industry leaders, and valuable networking opportunities with investors and peers. Find a GenAI Loft location near you, and don’t forget to register.

Browse all upcoming AWS led in-person and virtual events here.

That’s all for this week. Check back next Monday for another Weekly Roundup!

— seb

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

Introducing default data integrity protections for new objects in Amazon S3

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/introducing-default-data-integrity-protections-for-new-objects-in-amazon-s3/

At Amazon Web Services (AWS), the vast majority of new capabilities are driven by your direct feedback. Two years ago, Jeff announced additional checksum algorithms and the optional client-side computation of checksums to make sure the objects stored on Amazon S3 are exactly what you sent. You told us you love this extra verification because it gives you confidence the object stored is the one you sent. You also told us you would prefer to have this extra verification enabled automatically, freeing you from developing additional code.

Starting today, we’re updating the Amazon Simple Storage Service (Amazon S3) default behavior when you upload objects. To build upon its existing durability posture, Amazon S3 now automatically verifies that your data is correctly transmitted over the network from your applications to your S3 bucket.

Amazon S3 is designed for 99.999999999% data durability (that’s 11 nines). Amazon S3 has always verified the integrity of object uploads by calculating checksums when objects reach our servers, before they are written to multiple storage devices. Once your data is stored in Amazon S3, it continually monitors data durability over time with periodic integrity checks of data at rest. Amazon S3 also actively monitors the redundancy of your data to help verify that your objects can tolerate the concurrent failure of multiple storage devices.

But data can still face integrity risks as it traverses the public internet before reaching our servers. Issues such as faulty hardware on networks we don’t manage or client software bugs could potentially corrupt or drop data before Amazon S3 has a chance to validate it. Previously, you could extend the integrity protection by providing your own precomputed checksums with your PutObject or UploadPart requests. However, this requires configuring tools and applications to generate and track checksums, which can be complex to implement consistently across all your client applications uploading objects to Amazon S3.

The new default behavior builds upon existing data integrity protections without requiring any changes to your applications. Additionally, the new checksums are stored in the object’s metadata, making them accessible for integrity verification at any time.

Automatic client-side integrity protection
Amazon S3 now extends data integrity protection all the way to client-side applications by default. The latest versions of our AWS SDKs automatically calculate a cyclic redundancy check (CRC)-based checksum for each upload and send it to Amazon S3. Amazon S3 independently calculates a checksum on the server side and validates it against the provided value before durably storing the object and its checksum in the object’s metadata.

When your client application doesn’t send a CRC checksum (maybe it uses an old version of our SDK or you haven’t updated your application custom code yet), Amazon S3 computes a CRC-based checksum anyway and stores it in the object metadata for future reference. You can compare at a later stage the stored CRC with a CRC computed on your side and verify the network transmission was correct.

This new capability provides you with an automatic checksum calculation and validation for new uploads from the latest versions of the AWS SDKs, the AWS Command Line Interface (AWS CLI), and the AWS Management Console. You can also verify the checksum stored in the object’s metadata at any time. The new default data integrity protections use the existing CRC32 and CRC32C algorithms or the new CRC64NVME algorithm. Amazon S3 also provides developers with consistent full-object checksums across single-part and multipart uploads.

When uploading files in multiple parts, the SDKs calculate checksums for each part. Amazon S3 uses these checksums to verify the integrity of each part through the UploadPart API. Additionally, S3 validates the entire file’s size and checksum when you call the CompleteMultipartUpload API.

The CreateMultiPartUpload API introduces a new HTTP header, x-amz-checksum-type, which lets you specify the type of checksum to use. You can choose either a full object checksum (calculated by combining the checksums of all individual parts) or a composite checksum.

The full object checksum is stored with the object metadata for future reference. This new protection works seamlessly with server-side encryption. The consistent behavior across uploads, multipart uploads, downloads, and encryption modes simplifies client-side integrity checks. The ability to use full-object checksums to validate integrity and store them for use later can help you streamline your applications.

Let’s see it in action
To start using this additional integrity protection, update to the latest version of the AWS SDK or AWS CLI. No code changes are required to enable the new integrity protections.

Case 1: Amazon S3 now attaches a checksum to objects on the server side when objects are uploaded without a checksum

I wrote a simple Python script to upload and download content to and from an S3 bucket. I enabled maximum logging verbosity to see the actual HTTP headers sent to and from Amazon S3.

import boto3
import logging

BUCKET_NAME="aws-news-blog-20241111"
CONTENT='Hello World!'
OBJECT_NAME='test.txt'

# Enable debug logging for boto3 and botocore to stdout (this is verbose !!!)
logging.basicConfig(level=logging.DEBUG)

# create a s3 client
client = boto3.client('s3')

# put an object
client.put_object(Bucket=BUCKET_NAME, Key=OBJECT_NAME, Body=CONTENT)

# get the object 
response = client.get_object(Bucket=BUCKET_NAME, Key=OBJECT_NAME)
print(response['Body'].read().decode('utf-8'))

In the first step of this demo, I use an old AWS SDK for Python that doesn’t compute the CRC checksum on the client side. Despite this, I can observe that Amazon S3 now responds with a checksum it computed upon receiving the object.

S3 RESPONSE:
{
    ...
    "x-amz-checksum-crc64nvme": "AuUcyF784aU=",
    "x-amz-checksum-type": "FULL_OBJECT",
    ...
}

Case 2: Upload with manually pre-computed CRC64NVME checksum, a new checksum type

When I don’t have the option to use the latest version of the AWS SDK, or when I use my own code to upload objects to S3 buckets, I can compute the checksum and send it in the PutObject API request. Here is how I compute the checksum on my content before sending it to Amazon S3. To keep this code short, I use the checksums package available in the new AWS SDK for Python.

from awscrt import checksums
import base64

checksum = checksums.crc64nvme("Hello World!")
checksum_bytes = checksum.to_bytes(8, byteorder='big')  # CRC64 is 8 bytes
checksum_base64 = base64.b64encode(checksum_bytes)
print(checksum_base64)

And when I run it, I see the CRC64NVME checksum is the same as the one returned by Amazon S3 in the previous step.

$ python crc.py
b'AuUcyF784aU='

I can provide this checksum as part of the PutObject API call.

response = s3.put_object(
    Bucket=BUCKET_NAME,
    Key=OBJECT_NAME,
    Body=b'Hello World!',
    ChecksumAlgorithm='CRC64NVME', 
    ChecksumCRC64NVME=checksum_base64
)

Case 3: The new SDKs compute the checksum on the client-side

Now, I run the upload and download script again. This time, I use the latest version of the AWS SDK for Python. I observe that the SDK now sends the CRC headers in the request. The response also contains the checksum. I can easily compare the versions in the request and in the response to make sure the object received is the one I sent.

REQUEST:
{
    ...
    "x-amz-checksum-crc64nvme": "AuUcyF784aU=",
    "x-amz-checksum-type": "FULL_OBJECT",
    ... 
}

At any time, I can request the object checksum to verify the integrity of my local copy using the HeadObject or GetObject APIs.

 get_response = s3.get_object(
        Bucket=BUCKET_NAME,
        Key=OBJECT_NAME,
        ChecksumMode='ENABLED'
    )

The response object contains the checksum in the HTTPHeaders field.

{
...
    "x-amz-checksum-crc64nvme": "AuUcyF784aU=",
    "x-amz-checksum-type": "FULL_OBJECT",
...
}

Case 4: Multi-part uploads with new CRC-based whole-object checksum

When uploading large objects using the CreateMultipartUpload, UploadPart, and CompleteMultipartUpload APIs, the latest version of the SDK will automatically compute the checksums for you.

If you want to validate the integrity of your data by using a known content checksum, you can pre-compute the CRC-based whole-object checksum for multi-part uploads to simplify your client side tooling. When using full object checksums for multi-part uploads, you no longer have to keep track of part level checksums as you upload objects.


# precomputed CRC64NVME checksum for the full object
full_object_crc64_nvme_checksum = 'Naz0uXkYBPM='

# start multipart upload
create_response = s3.create_multipart_upload(
            Bucket=BUCKET_NAME,
            Key=OBJECT_NAME,
            ChecksumAlgorithm='CRC64NVME',
            ChecksumType='FULL_OBJECT'
        )
upload_id = create_response['UploadId']

# Upload parts
uploaded_parts = []

# part 1
data_part_1 = b'0' * (5 * 1024 * 1024) # minimum part size
upload_part_response_1 = s3.upload_part(
    Body=data_part_1,
    Bucket=BUCKET_NAME,
    Key=OBJECT_NAME,
    PartNumber=1,
    UploadId=upload_id,
    ChecksumAlgorithm='CRC64NVME'
)
uploaded_parts.append({'PartNumber': 1, 'ETag': upload_part_response_1['ETag']})

# part 2
data_part_2 = b'0' * (5 * 1024 * 1024)
upload_part_response_2 = s3.upload_part(
    Body=data_part_2,
    Bucket=BUCKET_NAME,
    Key=OBJECT_NAME,
    PartNumber=2,
    UploadId=upload_id,
    ChecksumAlgorithm='CRC64NVME'
)
uploaded_parts.append({'PartNumber': 2, 'ETag': upload_part_response_2['ETag']})

# Complete the multipart upload with the FULL_OBJECT CRC64NVME checksum to validate the integrity of your entire object. 
complete_response = s3.complete_multipart_upload(
            Bucket=BUCKET_NAME,
            Key=OBJECT_NAME,
            UploadId=upload_id,
            ChecksumCRC64NVME=full_object_crc64_nvme_checksum,
            ChecksumType='FULL_OBJECT',
            MultipartUpload={'Parts': uploaded_parts}
        )
print(complete_response)

Things to know
For your existing objects, the checksum will be added when you copy them. We updated the CopyObject API so you can choose the desired checksum algorithm for the destination object.

This new client-side checksum calculation is implemented in the latest version of the AWS SDKs. When you use an old SDK or custom code that doesn’t pre-compute checksums, Amazon S3 computes the checksum on all new objects it receives and stores it in the object’s metadata, even for multipart uploads.

Pricing and availability
This extended checksum computation and storage is available in all AWS Regions at no additional cost.

Update your AWS SDK and AWS CLI today to automatically benefit from this additional integrity protection for data in transit.

To learn more about data integrity protection on Amazon S3, visit Checking object integrity in the Amazon S3 User Guide.

— seb

AWS Database Migration Service now automates time-intensive schema conversion tasks using generative AI

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-data-migration-service-improves-database-schema-conversion-with-generative-ai/

Starting today, AWS Database Migration Service Schema Conversion (AWS DMS SC) introduces a new capability to improve the database schema conversion experience by automatically converting up to 90 percent of schema objects from commercial databases to PostgreSQL migrations.

AWS DMS is a cloud service that makes it possible to migrate relational databases, data warehouses, NoSQL databases, and other types of data stores. You can use AWS DMS to migrate your data into the Amazon Web Services (AWS) Cloud or between combinations of cloud and on-premises setups.

Today, more than 1 million databases have been migrated using AWS Database Migration Service. AWS DMS helps you migrate your data from one database system to another. And, when migrating between different database engines, AWS DMS SC helps to convert the source database schema and procedures to the target database system.

However, although AWS DMS SC automates many steps in these migrations, certain complex database code elements still require manual intervention, which can extend migration timelines and add cost. This is particularly the case with proprietary system functions or procedures, and data type conversions, which don’t always have direct equivalents in PostgreSQL.

The new generative AI capability in AWS DMS SC is designed to address these challenges by automating some of the most time-intensive schema conversion tasks. Using large language models (LLMs) hosted on Amazon Bedrock, the new capability expands the existing conversion capabilities. It converts code snippets in the source database that were otherwise not supported by traditional rule-based techniques, including complex procedures and functions.

Generative AI–assisted code conversion helps to reduce migration costs and accelerate project timelines. Because AWS DMS SC automates more of the schema conversion process, you can focus on higher value tasks such as refining and optimizing your applications post-migration rather than manually resolving conversion gaps. Our beta customers have already experienced success with these AI-powered features in AWS DMS SC, achieving cost savings and faster migrations.

Let’s find out how it works
To demonstrate the ease of using this new generative AI capability, I’ll walk through the schema conversion process in AWS DMS SC. AWS DMS SC simplifies database migration by automatically converting my source database’s structure, including tables, views, stored procedures, functions, and more, to a format compatible with my target database. Any objects that can’t be automatically converted are flagged for manual attention.

I start with a self-managed commercial database running on Amazon Elastic Compute Cloud (Amazon EC2). I use the AWS Management Console to define the instance profile and the data providers. This is where I configure the replication instance network details, the database engine and its endpoint, the secret where the database password is securely stored, and more. I also create a migration project. These steps aren’t new, and you can refer to Accelerate your database migration journey using AWS DMS Schema Conversion in the AWS Database Blog to learn about the details.

After my project is created, I select it, and on the Schema conversion tab, I choose Launch schema conversion. It takes a couple of minutes to launch the conversion tool the first time.

DMS : Launch migration project

AWS DMS SC with generative AI is an opt-in capability. I first activate the option. On the Settings tab, I turn on Enable Generative AI feature for conversion.DMS : enable GenAI feature

Before diving into the details of the conversion, I would like to get an overall assessment of the migration complexity. I select the schema I want to migrate. Then I select Assess in the menu.

DMS : Assess schema

After a few minutes, a high-level Summary is available. The Action items tab has more details. I choose Export results and choose PDF to receive a report to share with my colleagues. The report is generated and available from an S3 bucket.

The summary screen shows the percentage of Database storage objects and Database code objects that can be converted by the rule-based method. That’s 100% and 57% in this example. Let’s see how the generative AI-based conversion will change that.

DMS : Assess schema summary

The PDF contains an executive summary, various statistics about the number of objects to be migrated, the feasibility of conversion with generative AI, and the complexity of the migration.

DMS : Assess schema PDF page 1 DMS : Assess schema PDF page 2

By reading the report, I learn there is no blocker detected to migrate the stored procedures. I select the stored procedure I want to migrate (PRC_AIML_DEMO6). Then, I select the Actions menu on the source database (the left one) and choose Convert.

After a minute or two, I can read the original procedure code in the left pane and the proposed migrated version on the right panel.

The summary screen has been updated. Now, it shows that 100 percent of the code can be converted automatically.

DNS : view proposed modifications

I can edit the code and make changes as required. When I’m comfortable with the proposed new version, I select the Actions menu on the target database side (the right one) and choose Apply changes.

DMS : Apply changes

With this new generative AI capability, AWS DMS SC can automatically convert up to 90 percent of schema objects from commercial databases to PostgreSQL.

To support your compliance requirements, this capability is initially turned off, and you can enable it as needed. If you choose to use the generative AI features in AWS DMS SC, it will flexibly decide between traditional rule-based methods and generative AI based on the complexity of the objects being converted. Customers with strict policies against generative AI can continue to rely solely on the rule-based approach, with any unconverted or partially converted objects requiring manual adjustments.

Availability and pricing
This new capability is available today in the following AWS Regions: US East (Ohio, N. Virginia), US West (Oregon), and Europe (Frankfurt).

AWS DMS Schema Conversion with generative AI provides you with a faster migration pathway and helps you accelerate your transition to AWS.

To get started, visit the AWS DMS Schema Conversion documentation and learn how this generative AI capability can simplify your next database migration.

— seb

AWS Verified Access now supports secure access to resources over non-HTTP(S) protocols (in preview)

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-verified-access-now-supports-secure-access-to-resources-over-non-https-protocols/

AWS Verified Access provides secure access to your corporate applications and resources without a virtual private network (VPN). We launched Verified Access in preview at re:Invent 2 years ago as a way to provide secure, VPN-less access to corporate applications, enabling organizations to manage network access based on identity and device security instead of IP addresses, which increases control and security over application access.

Today, Verified Access is launching a preview of its secure, VPN-less access capabilities to non-HTTP(S) applications and resources, enabling zero trust access to corporate resources over protocols such as Secure Shell (SSH) and Remote Desktop Protocol (RDP).

Organizations increasingly require secure, remote access to internal resources such as databases, remote desktops, and Amazon Elastic Compute Cloud (Amazon EC2) instances. Traditional VPN solutions, although effective for network access, often grant broad privileges and don’t support granular access controls, which can expose infrastructure with sensitive data. Although some organizations use bastion hosts to mediate access, this approach can create complexity and policy inconsistencies across HTTP(S) and non-HTTP(S) applications. With the rise of zero trust architectures, these gaps highlight the need for a secure access solution that extends consistent access policies across all applications and resources.

Verified Access addresses these needs by providing zero trust access controls for your corporate applications and resources. By supporting protocols such as SSH, RDP, or Java Database Connectivity (JDBC) or Open Database Connectivity (ODBC), Verified Access simplifies your security operations. Now, you can establish uniform, context-aware access policies across your corporate applications and resources. Verified Access evaluates each access request in real time, making sure access is granted only to users who meet specific identity and device security requirements. Additionally, it eliminates the need for separate VPNs or bastion hosts, streamlining operations and reducing the risk of over-privileged access.

One of my favorite capabilities is onboarding a group of resources by specifying their IP Classless Inter-Domain Routing (CIDR) and ports, rather than onboarding one resource at a time. Verified Access automatically creates DNS records for each active resource within the specified CIDR range. This eliminates the need for manual DNS configuration and users can therefore connect to new resources instantly.

Using Verified Access for non-HTTPS access
Configuring Verified Access for non-HTTPS access isn’t very different from what exists today. You can read the blog post I wrote for the launch of the preview 2 years ago or the Get started with Verified Access tutorial to learn how to get started.

Verified Access proposes two new types of endpoint targets: a target for one single resource and a target for multiple resources.

With the network interface, load balancer, or RDS endpoint target you can provide access to an individual resource such as an Amazon Relational Database Service (Amazon RDS) instance or an arbitrary TCP application fronted by a Network Load Balancer or an elastic network interface. This type of target endpoint is defined by a combination of a target type (such as a load balancer or a network interface) and a range of TCP ports. Verified Access will provide a DNS name for each endpoint upon its creation. A Verified Access DNS name is assigned for each target. This is the name end users will use to securely access the resource.

With network CIDR endpoint target, the resources are defined using an IP CIDR and port range. Through this type of endpoint target, you can easily provision secure access to ephemeral resources such as EC2 instances over protocols such as SSH and RDP. This is done without having to perform any actions such as creating or deleting endpoint targets each time a resource is added or removed. As long as these resources are assigned an IP address from the defined CIDR, Verified Access provides a unique public DNS record for each active IP detected in the defined CIDR.

Here is a diagram of the setup for this demo.

AWS Verified Access Demo Setup

Part 1: As a Verified Access administrator

As a Verified Access administrator, I create the Verified Access instance, trust provider, access group, endpoint, and access policies, allowing access by the end user to the SSH server.

For this demo, I configure a Verified Access network CIDR endpoint target. I select TCP as Protocol and Network CIDR as Endpoint type. I make sure the CIDR range is within the one of the VPC where my target resources are. I select the TCP Port ranges and the Subnets within the VPC.

AVA : Create endpoint

This is a good moment to stretch your legs and refill your cup of coffee, it takes a few minutes to create the endpoint.

Once, the status is ✅ Active, I launch an EC2 instance in a private Amazon Virtual Private Cloud (Amazon VPC). I enable SSH and configure the instance’s security group to only access requests coming from the VPC. A few minutes later, I can see the instance IP has been detected and assigned a DNS name to connect to from the Verified Access client application.

I also have the option during the configuration to delegate my own DNS subdomain, such as secure.mycompany.com, and Verified Access will assign DNS names for the resources within that subdomain.

AVA : DNS names

Create an access policy

At this stage, there is no policy defined on the Verified Access endpoint. It will deny every request by default.

On the Verified Access groups page, I select the Policy tab. Then I select the Modify Verified Access endpoint policy button to create an access policy.

Verified Access - group policy tab

I enter a policy allowing anybody who is authenticated and has an email address ending with @amazon.com. This is the email address I used for the user defined in AWS IAM Identity Center. Note that the name after context is the name I entered as Policy reference name when I created the Verified Access trust provider. The documentation page has the details of the policy syntax, the attributes, and the operators I can use.

permit(principal, action, resource)
when {
    context.awsnewsblog.user.email.address like "*@amazon.com"
};

Verified Access - group define policy

After a few minutes, Verified Access updates the policy and becomes Active again.

Distribute the configuration to clients

The last task as a Verified Access administrator is to extract the JSON configuration file of the client applications.

I retrieve the client application configuration file with the AWS Command Line Interface (AWS CLI). As a system administrator, I’ll distribute this configuration to each client machine.

aws ec2 export-verified-access-instance-client-configuration \
     --verified-access-instance-id "vai-0dbf2c4c011083069"

{
    "Version": "1.0",
    "VerifiedAccessInstanceId": "vai-0dbf2c4c011083069",
    "Region": "us-east-1",
    "DeviceTrustProviders": [],
    "UserTrustProvider": {
        "Type": "iam-identity-center",
        "Scopes": "verified_access_test:application:connect",
        "Issuer": "https://identitycenter.amazonaws.com/ssoins-xxxx",
        "PkceEnabled": true
    },
    "OpenVpnConfigurations": [
        {
            "Config": "Y2...bWU=",
            "Routes": [
                {
                    "Cidr": "2600:1f10:4a02:8700::/57"
                }
            ]
        }
    ]
}

Now that I have a resource to connect to and the Verified Access infrastructure in place, let me show you the end user experience to access a network endpoint.

Part 2: As an end user

As the end user, I receive a link to download and install the Verified Access Connectivity Client application. We support Windows and macOS clients at the time of this writing.

I install the configuration file I received from my administrator. I use ClientConfig1.json as the file name and I copy the file to C:\ProgramData\AWSPylon on Windows or /Library/Application Support/com.aws.pylon.client on macOS.

This is the same configuration file for all users, and the system administrator might push the file to all client machines using an endpoint management tool.

I start the Connectivity Client application. I choose Sign in to start the authentication sequence.

AVA Client : Sign inThe authentication opens my web browser on the authentication page of my identity provider. The exact screen and login sequence varies from one provider to the other. After I’m authenticated, the Connectivity Client creates the secure tunnel to access my resource, an EC2 instance for this demo.

AVA Client : Connecting AVA Client : Connected

Once the status is Connected, I can securely connect to the resource, using the DNS name provided by Verified Access. In a terminal application, I type the ssh command to start the connection.

For this demo, I configured a delegated DNS domain secure.mycompany.com for Verified Access. The DNS address I received for the EC2 instance is 10-0-1-199.awsnews.secure.mycompany.com.

$ ssh -i mykey.pem [email protected]

   ,     #_
   ~\_  ####_        Amazon Linux 2023
  ~~  \_#####\
  ~~     \###|
  ~~       \#/ ___   https://aws.amazon.com/linux/amazon-linux-2023
   ~~       V~' '->
    ~~~         /
      ~~._.   _/
         _/ _/
       _/m/'
Last login: Sat Nov 17 20:17:46 2024 from 1.2.3.4

$

Availability and pricing
Verified Access is available as a public preview in 18 AWS Regions: US East (Ohio, N. Virginia), US West (N. California, Oregon), Asia Pacific (Jakarta, Mumbai, Seoul, Singapore, Sydney, Tokyo), Canada (Central), Europe (Frankfurt, Ireland, London, Milan, Stockholm), Israel (Tel Aviv), and South America (São Paulo).

You’re charged for each hour that your non-HTTP(S) Verified Access endpoint remains active and per connection. The first 100 connections per month on each Verified Access endpoint are free. For more information, refer to AWS Verified Access Pricing.

With Verified Access for HTTP(S) and non-HTTP(S) applications you can unify the access controls to your private applications and systems and apply zero trust policies uniformly to all applications, and SSH, RDP, and HTTP(S) resources. It reduces the complexity of your network infrastructure and helps you to implement zero-trust access to your applications and resources. Finally, it adapts to your growing infrastructure, automating DNS setup and supporting large-scale deployments without resource-specific registration.

Go, try Verified Access today, and share your feedback with the team!

— seb

Amazon CloudFront now accepts your applications’ gRPC calls

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/amazon-cloudfront-now-accepts-your-applications-grpc-calls/

Starting today, you can deploy Amazon CloudFront, our global content delivery network (CDN), in front of your gRPC API endpoints.

gRPC is a modern, efficient, and language-agnostic framework for building APIs. It uses Protocol Buffers (protobuf) as its interface definition language (IDL), which enable you to define services and message types in a platform-independent manner. With gRPC, communication between services is achieved through lightweight and high-performance remote procedure calls (RPCs) over HTTP/2. This promotes efficient and low-latency communication across services, making it ideal for microservices architectures.

gRPC offers features such as bidirectional streaming, flow control, and automatic code generation for a variety of programming languages. It’s well-suited for scenarios in which you require high performance, efficient communication, and real-time data streaming. If your application needs to handle a large amount of data or requires low-latency communication between client and server, gRPC can be a good choice. However, gRPC might be more challenging to learn compared to REST. For example, gRPC relies on the protobuf serialization format, which requires developers to define their data structures and service methods in .proto files.

I see two benefits of deploying CloudFront in front of your gRPC API endpoints.

First, it allows the reduction of latency between the client application and your API implementation. CloudFront offers a global network of over 600+ edge locations with intelligent routing to the closest edge. Edge locations provide TLS termination and optional caching for your static content. CloudFront transfers client application requests to your gRPC origin through the fully managed, low-latency, and high-bandwidth private AWS network.

Secondly, your applications benefit from additional security services deployed on edge locations, such as traffic encryption, the validation of the HTTP headers through AWS Web Application Firewall, and AWS Shield Standard protection against distributed denial of service (DDoS) attacks.

Let’s see it in action
To start this demo, I use the gRPC route-guide demo from the official gRPC code repository. I deploy this example application in a container for ease of deployment (but any other deployment option is supported too).

I use this Dockerfile

FROM python:3.7
RUN pip install protobuf grpcio
COPY ./grpc/examples/python/route_guide .
CMD python route_guide_server.py
EXPOSE 50051

I also use the AWS Copilot command line to deploy my container on Amazon Elastic Container Service (Amazon ECS). The Copilot command prompts me to collect the information it requires to build and deploy the container. Then, it creates the ECS cluster, the ECS service, and the ECS task automatically. It also creates a TLS certificate and the load balancer for me. I test the client application by modifying line 122 to use the DNS name of the load balancer listener endpoint. I also change the client application code to use grpc.secure_channel instead of grpc.insecure_channel because the load balancer provides the application with an HTTPS endpoint.

gRPC client application demo - source code with ALB

When I’m confident my API is correctly deployed and working, I proceed and configure CloudFront.

First, in the CloudFront section of the AWS Management Console, I select Create distribution.

Under Origin, I enter my gRPC endpoint DNS name as Origin domain. I enable HTTPS only as Protocol and leave the HTTPS port as is (443). Then I choose a Name for the distribution.

CloudFront - Add origin and name

Under Viewer, I select HTTPS only as Viewer protocol policy. Then, I select GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE as Allowed HTTP methods. I select Enable for Allow gRPC requests over HTTP/2.

CloudFront - Viewer Policy

Under Cache key and origin requests, I select AllViewer as Origin request policy.

The default cache policy is CacheOptimized, but gRPC isn’t cacheable API traffic. Therefore, I select CachingDisabled as Cache policy.

CloudFront - Cache policy

AWS WAF helps protect you against common web exploits and bots that can affect availability, compromise security, or consume excessive resources. For gRPC traffic, AWS WAF can inspect the HTTP headers of the request and enforce access control. It doesn’t inspect the request body in protobuf format.

For this demo, I choose to not use AWS WAF. Under Web Application Firewall (WAF), I select Do not enable security protections.

CloudFront - Security

I also keep all the other options with their default value. HTTP/2 support is selected by default. Do not disable it because it is required for gRPC.

Finally, I select Create distribution.

CloudFront - Create distribution

There is only one switch to enable gRPC on top of the usual setup. When turned on, with HTTP/2 and HTTP POST enabled, CloudFront detects gRPC client traffic and forwards it to your gRPC origin.

After a few minutes, the distribution is ready. I copy and paste the endpoint URL of the CloudFront distribution, and I change the client-side app to make it point to CloudFront instead of the previously created load balancer.

gRPC client application demo - source code

I test the application again, and it works.

gRPC client application demo - execution

Pricing and Availability
gRPC origins are available on all the more than 600 CloudFront edge locations at no additional cost. The usual requests and data transfer fees apply.

Go and point your CloudFront origin to a gRPC endpoint today.

— seb

Replicate changes from databases to Apache Iceberg tables using Amazon Data Firehose (in preview)

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/replicate-changes-from-databases-to-apache-iceberg-tables-using-amazon-data-firehose/

Today, we’re announcing the availability, in preview, of a new capability in Amazon Data Firehose that captures changes made in databases such as PostgreSQL and MySQL and replicates the updates to Apache Iceberg tables on Amazon Simple Storage Service (Amazon S3).

Apache Iceberg is a high-performance open-source table format for performing big data analytics. Apache Iceberg brings the reliability and simplicity of SQL tables to S3 data lakes and makes it possible for open source analytics engines such as Apache Spark, Apache Flink, Trino, Apache Hive, and Apache Impala to concurrently work with the same data.

This new capability provides a simple, end-to-end solution to stream database updates without impacting transaction performance of database applications. You can set up a Data Firehose stream in minutes to deliver change data capture (CDC) updates from your database. Now, you can easily replicate data from different databases into Iceberg tables on Amazon S3 and use up-to-date data for large-scale analytics and machine learning (ML) applications.

Typical Amazon Web Services (AWS) enterprise customers use hundreds of databases for transactional applications. To perform large scale analytics and ML on the latest data, they want to capture changes made in databases, such as when records in a table are inserted, modified, or deleted, and deliver the updates to their data warehouse or Amazon S3 data lake in open source table formats such as Apache Iceberg.

To do so, many customers develop extract, transform, and load (ETL) jobs to periodically read from databases. However, ETL readers impact database transaction performance, and batch jobs can add several hours of delay before data is available for analytics. To mitigate impact on database transaction performance, customers want the ability to stream changes made in the database. This stream is referred to as a change data capture (CDC) stream.

I met multiple customers that use open source distributed systems, such as Debezium, with connectors to popular databases, an Apache Kafka Connect cluster, and Kafka Connect Sink to read the events and deliver them to the destination. The initial configuration and test of such systems involves installing and configuring multiple open source components. It might take days or weeks. After setup, engineers have to monitor and manage clusters, and validate and apply open source updates, which adds to the operational overhead.

With this new data streaming capability, Amazon Data Firehose adds the ability to acquire and continually replicate CDC streams from databases to Apache Iceberg tables on Amazon S3. You set up a Data Firehose stream by specifying the source and destination. Data Firehose captures and continually replicates an initial data snapshot and then all subsequent changes made to the selected database tables as a data stream. To acquire CDC streams, Data Firehose uses the database replication log, which reduces impact on database transaction performance. When the volume of database updates increases or decreases, Data Firehose automatically partitions the data, and persists records until they’re delivered to the destination. You don’t have to provision capacity or manage and fine-tune clusters. In addition to the data itself, Data Firehose can automatically create Apache Iceberg tables using the same schema as the database tables as part of the initial Data Firehose stream creation and automatically evolve the target schema, such as new column addition, based on source schema changes.

Since Data Firehose is a fully managed service, you don’t have to rely on open source components, apply software updates, or incur operational overhead.

The continual replication of database changes to Apache Iceberg tables in Amazon S3 using Amazon Data Firehose provides you with a simple, scalable, end-to-end managed solution to deliver CDC streams into your data lake or data warehouse, where you can run large-scale analysis and ML applications.

Let’ see how to configure a new pipeline
To show you how to create a new CDC pipeline, I setup a Data Firehose stream using the AWS Management Console. As usual, I also have the choice to use the AWS Command Line Interface (AWS CLI), AWS SDKs, AWS CloudFormation, or Terraform.

For this demo, I choose a MySQL database on Amazon Relational Database Service (Amazon RDS) as source. Data Firehose also works with self-managed databases on Amazon Elastic Compute Cloud (Amazon EC2). To establish connectivity between my virtual private cloud (VPC)—where the database is deployed—and the RDS API without exposing the traffic to the internet, I create an AWS PrivateLink VPC service endpoint. You can learn how to create a VPC service endpoint for RDS API by following instructions in the Amazon RDS documentation.

I also have an S3 bucket to host the Iceberg table, and I have an AWS Identity and Access Management (IAM) role setup with correct permissions. You can refer to the list of prerequisites in the Data Firehose documentation.

To get started, I open the console and navigate to the Amazon Data Firehose section. I can see the stream already created. To create a new one, I select Create Firehose stream.

Create Firehose Stream

I select a Source and Destination. In this example: a MySQL database and Apache Iceberg Tables. I also enter a Firehose stream name for my stream.

Create Firehose Stream - screen 1

I enter the fully qualified DNS name of my Database endpoint and the Database VPC endpoint service name. I verify that Enable SSL is checked and, under Secret name, I select the name of the secret in AWS Secrets Manager where the database username and password are securely stored.

Create Firehose Stream - screen 2

Next, I configure Data Firehose to capture specific data by specifying databases, tables, and columns using explicit names or regular expressions.

I must create a watermark table. A watermark, in this context, is a marker used by Data Firehose to track the progress of incremental snapshots of database tables. It helps Data Firehose identify which parts of the table have already been captured and which parts still need to be processed. I can create the watermark table manually or let Data Firehose automatically create it for me. In that case, the database credentials passed to Data Firehose must have permissions to create a table in the source database.

Create Firehose Stream - screen 3

Next, I configure the S3 bucket Region and name to use. Data Firehose can automatically create the Iceberg tables when they don’t exist yet. Similarly, it can update the Iceberg table schema when detecting a change in your database schema.

Create Firehose Stream - screen 4

As a final step, it’s important to enable Amazon CloudWatch error logging to get feedback about the stream progress and the eventual errors. You can configure a short retention period on the CloudWatch log group to reduce the cost of log storage.

After having reviewed my configuration, I select Create Firehose stream.

Create Firehose Stream - screen 5

Once the stream is created, it will start to replicate the data. I can monitor the stream’s status and check for eventual errors.

Create Firehose Stream - screen 6

Now, it’s time to test the stream.

I open a connection to the database and insert a new line in a table.

Firehose - MySQL

Then, I navigate to the S3 bucket configured as the destination and I observe that a file has been created to store the data from the table.

View parquet files on S3 bucket

I download the file and inspect its content with the parq command (you can install that command with pip install parquet-cli)

Parquet file content

Of course, downloading and inspecting Parquet files is something I do only for demos. In real life, you’re going to use AWS Glue and Amazon Athena to manage your data catalog and to run SQL queries on your data.

Things to know
Here are a few additional things to know.

This new capability supports self-managed PostgreSQL and MySQL databases on Amazon EC2 and the following databases on Amazon RDS:

The team will continue to add support for additional databases during the preview period and after general availability. They told me they are already working on supporting SQL Server, Oracle, and MongoDB databases.

Data Firehose uses AWS PrivateLink to connect to databases in your Amazon Virtual Private Cloud (Amazon VPC).

When setting up an Amazon Data Firehose delivery stream, you can either specify specific tables and columns or use wildcards to specify a class of tables and columns. When you use wildcards, if new tables and columns are added to the database after the Data Firehose stream is created and if they match the wildcard, Data Firehose will automatically create those tables and columns in the destination.

Pricing and availability
The new data streaming capability is available today in all AWS Regions except China Regions, AWS GovCloud (US) Regions, and Asia Pacific (Malaysia) Regions. We want you to evaluate this new capability and provide us with feedback. There are no charges for your usage at the beginning of the preview. At some point in the future, it will be priced based on your actual usage, for example, based on the quantity of bytes read and delivered. There are no commitments or upfront investments. Make sure to read the pricing page to get the details.

Now, go configure your first continual database replication to Apache Iceberg tables on Amazon S3 and visit http://aws.amazon.com/firehose.

— seb

Centrally managing root access for customers using AWS Organizations

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/centrally-managing-root-access-for-customers-using-aws-organizations/

AWS Identity and Access Management (IAM) is launching a new capability allowing security teams to centrally manage root access for member accounts in AWS Organizations. You can now easily manage root credentials and perform highly privileged actions.

Managing root user credentials at scale
For a long time, Amazon Web Services (AWS) accounts were provisioned with highly privileged root user credentials, which had unrestricted access to the account. This root access, while powerful, also posed significant security risks. Each AWS account’s root user had to be secured by adding layers of protection like multi-factor authentication (MFA). Security teams were required to manage and secure these root credentials manually. The process involved rotating credentials periodically, storing them securely, and making sure that the credentials complied with security policies.

As our customers expanded their AWS environments, this manual approach became cumbersome and prone to error. For example, large enterprises operating hundreds or thousands of member accounts struggled to secure root access consistently across all accounts. The manual intervention not only added operational overhead but also created a lag in account provisioning, preventing full automation and increasing security risks. Root access, if not properly secured, could lead to account takeovers and unauthorized access to sensitive resources.

Furthermore, whenever specific root actions such as unlocking an Amazon Simple Storage Service (Amazon S3) bucket policy or an Amazon Simple Queue Service (Amazon SQS) resource policy were required, security teams had to retrieve and use root credentials, which only increased the attack surface. Even with rigorous monitoring and strong security policies, maintaining long-term root credentials opened doors to potential mismanagement, compliance risks, and manual errors.

Security teams began seeking a more automated, scalable solution. They needed a way to not only centralize the management of root credentials but also programmatically manage root access without needing long-term credentials in the first place.

Centrally manage root access
With the new ability to centrally manage root access, we address the longstanding challenge of managing root credentials across multiple accounts. This new capability introduces two essential capabilities: the central management of root credentials and root sessions. Together, they offer security teams a secure, scalable, and compliant way to manage root access across AWS Organizations member accounts.

Let’s first discuss the central management of root credentials. With this capability, you can now centrally manage and secure privileged root credentials across all accounts in AWS Organizations. Root credentials management allows you to:

  • Remove long-term root credentials – Security teams can now programmatically remove root user credentials from member accounts, confirming that no long-term privileged credentials are left vulnerable to misuse.
  • Prevent credential recovery – It not only removes the credentials but also prevents their recovery, safeguarding against any unintended or unauthorized root access in the future.
  • Provision secure-by-default accounts – Because you can now create member accounts without root credentials from the start, you no longer need to apply additional security measures like MFA after account provisioning. Accounts are secure by default, which drastically reduces security risks associated with long-term root access and helps simplify the entire provisioning process.
  • Help to stay compliant – Root credentials management allows security teams to demonstrate compliance by centrally discovering and monitoring the status of root credentials across all member accounts. This automated visibility confirms that no long-term root credentials exist, making it easier to meet security policies and regulatory requirements.

But how can we make sure it remains possible to perform selected root actions on the accounts? This is the second capability we launch today: root sessions. It offers a secure alternative to maintaining long-term root access. Instead of manually accessing root credentials whenever privileged actions are required, security teams can now gain short-term, task-scoped root access to member accounts. This capability makes sure that actions such as unlocking S3 bucket policies or SQS queue policies can be performed securely without the need for long-term root credentials.

Root sessions key benefits include:

  • Task-scoped root access – AWS enables short-term root access for specific actions, adhering to the best practices of least privilege. This limits the scope of what can be done and minimizes the duration of access, reducing potential risks.
  • Centralized management – You can now perform privileged root actions from a central account without needing to log in to each member account individually. This streamlines the process and reduces the operational burden on security teams, allowing them to focus on higher-level tasks.
  • Alignment with AWS best practices – By using short-term credentials, organizations align themselves with AWS security best practices, which emphasize the principle of least privilege and the use of short-term, temporary access where possible.

This new capability does not grant full root access. It provides temporary credentials for performing one of these five specific actions. The first three actions are possible with central management of root accounts. The last two come when enabling root sessions.

  • Auditing root user credentials – Read-only access to review root user information
  • Re-enabling account recovery – Reactivating account recovery without root credentials
  • Deleting root user credentials – Removing console passwords, access keys, signing certificates, and MFA devices
  • Unlocking an S3 bucket policy – Editing or deleting an S3 bucket policy that denies all principals
  • Unlocking an SQS queue policy – Editing or deleting an Amazon SQS resource policy that denies all principals

How to obtain root credentials on a member account
In this demo, I show you how to prepare your management account, create a member account without root credentials, and obtain temporary root credentials to make one of the five authorized API call on the member account. I assume you have an organization already created.

First, I create a member account.

aws organizations create-account    \
     --email [email protected] \
     --account-name 'Root Accounts Demo account'
{
    "CreateAccountStatus": {
        "Id": "car-695abd4ee1ca4b85a34e5dcdcd1b944f",
        "AccountName": "Root Accounts Demo account",
        "State": "IN_PROGRESS",
        "RequestedTimestamp": "2024-09-04T20:04:09.960000+00:00"
    }
}

Then, I enable the two new capabilities on my management account. Don’t worry, these commands don’t alter the behavior of the accounts in any way other than enabling use of the new capability.

➜  aws organizations enable-aws-service-access \
        --service-principal iam.amazonaws.com

➜  aws iam enable-organizations-root-credentials-management
{
    "OrganizationId": "o-rlrup7z3ao",
    "EnabledFeatures": [
        "RootCredentialsManagement"
    ]
}

➜  aws iam enable-organizations-root-sessions
{
    "OrganizationId": "o-rlrup7z3ao",
    "EnabledFeatures": [
        "RootSessions",
        "RootCredentialsManagement"
    ]
}

Alternatively, I can also use the console on the management account. Under Access management, I select Account settings.

Root Access Management

Now, I’m ready to make requests to obtain temporary root credentials. I have to pass one of the five managed IAM policies to scope down the credentials to a specific action.

➜  aws sts assume-root \
       --target-principal <my member account id> \
       --task-policy-arn arn=arn:aws:iam::aws:policy/root-task/S3UnlockBucketPolicy 

{
    "Credentials": {
        "AccessKeyId": "AS....XIG",
        "SecretAccessKey": "ao...QxG",
        "SessionToken": "IQ...SS",
        "Expiration": "2024-09-23T17:44:50+00:00"
    }
}

Once I obtain the access key ID, the secret access key, and the session token, I use them as usual with the AWS Command Line Interface (AWS CLI) or an AWS SDKs.

For example, I can pass these three values as environment variables.

$ export AWS_ACCESS_KEY_ID=ASIA356SJWJITG32xxx
$ export AWS_SECRET_ACCESS_KEY=JFZzOAWWLocoq2of5Exxx
$ export AWS_SESSION_TOKEN=IQoJb3JpZ2luX2VjEMb//////////wEaCXVxxxx

Now that I received the temporary credentials, I can make a restricted API call as root on the member account. First, I verify I now have root credentials. The Arn field confirms I’m working with the root account.


# Call get Caller Identity and observe I'm root in the member account
$ aws sts get-caller-identity
{
   "UserId": "012345678901",
   "Account": "012345678901",
   "Arn": "arn:aws:iam::012345678901:root"
}

Then, I use the delete-bucket-policy from S3 to remove an incorrect policy that has been applied to a bucket. The invalid policy removed all bucket access for everybody. Removing such policy requires root credentials.

aws s3api delete-bucket-policy --bucket my_bucket_with_incorrect_policy

When there is no output, it means the operation is successful. I can now apply a correct access policy to this bucket.

Credentials are valid only for 15 minutes. I wrote a short shell script to automate the process of getting the credentials as JSON, exporting the correct environment variables, and issuing the command I want to run as root.

Availability
Central management of root access is available at no additional cost in all AWS Regions except AWS GovCloud (US) and AWS China Regions, where there is no root account. Root sessions are available everywhere.

You can start using it through the IAM console, AWS CLI or AWS SDK. For more information, visit AWS account root user in our documentation and follow best practices for securing your AWS accounts.

— seb

Announcing new APIs for Amazon Location Service Routes, Places, and Maps

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/announcing-new-apis-for-amazon-location-service-routes-places-and-maps/

Today, Amazon Location Service released 17 new and enhanced APIs that expand and improve capabilities for the Routes, Places, and Maps functionalities, creating a more cohesive and streamlined experience for developers. By introducing enhanced features and offering simplified migration, these updates make Amazon Location Service more accessible and useful for a wide range of applications.

You can now access advanced route optimization, toll cost calculations, GPS traces snapping, and a variety of map styles with static and dynamic rendering options, and perform proximity-based search and predictive suggestions, with rich, detailed information on points of interest.

At Amazon, the vast majority of our roadmaps are driven by customer feedback. Many customers building applications with Amazon Location Service have shared that they need purpose-built APIs and more granular details, such as contact information and business hours, when working with location-based data. Although the current API set has provided valuable tools for many customers, developers have expressed a desire for additional capabilities, such as detailed route planning, proximity-based searches, additional places details, and static map images. These new APIs address these requests and provide a more comprehensive, out-of-the-box location solution.

New and enhanced capabilities
Today’s launch introduces 10 updated APIs and seven entirely new APIs, responding directly to your feedback. Each service—Routes, Places, and Maps—receives specific enhancements designed to support a broader range of use cases.

Routes
The Amazon Location Routes API now supports advanced route planning and customization options, allowing users to:

  • CalculateIsolines to identify service areas within specific travel time, or distance
  • OptimizeWaypoints to determine the most efficient sequence of waypoints, helping to minimize either travel time or distance
  • Calculate Toll Costs to provide precise cost estimates for routes involving toll roads
  • SnapToRoads, to enable precise matching of GPS traces by snapping points to the road network

With these capabilities, you can design more accurate and dynamic route experiences for your users. For example, a logistics company could optimize driver routes in real-time, factoring in live traffic and minimizing travel costs for deliveries.

Maps
The updated Amazon Location Maps API includes more purpose-built map styles crafted by skilled cartographers. These map styles offer professional designs that accelerate time to market and eliminate the need for custom map creation. Additionally, the Static Map Image feature allows developers to integrate static maps within applications, reducing the need for continuous data streaming and improving performance in use cases where interactivity isn’t necessary.

Key features of the Maps API include:

  • GetTile, to download a tile from a tileset, with a specified X, Y, and Z axis values
  • GetStyleDescriptor, to return information about the style
  • GetStaticMap, which enables the rendering of non-interactive maps for reporting or visualization purposes

Places
The Amazon Location Places API enhancements allow more detailed search capabilities, addressing requests for increased granularity in location data. The new capabilities include:

  • SearchNearby and Autocomplete, which support proximity-based queries and enable predictive text features for better user experiences
  • Enhanced business details with categories such as Business Hours, Contact Information, and additional attributes for points of interest

These features are especially useful for applications where users need detailed information about nearby locations, such as food delivery services or retail applications. Imagine that a customer opens a food delivery application, searches for nearby restaurants using SearchNearby, and retrieves restaurant details such as business hours and contact information to confirm availability. Once multiple delivery orders are assigned to a driver, the application uses OptimizeWaypoints to suggest the most efficient route for pickups and deliveries. As the driver follows the route, SnaptoRoads provides precise visualization of their location, enhancing the customer’s real-time tracking experience.

Enhanced Location Service in action
Calling the API is straightforward. You can use the AWS Command Line Interface (AWS CLI), one of our AWS SDKs, or the plain REST API. However, displaying the information on a map in a web or mobile app requires some additional setup. Although the process is well documented, it’s too detailed to cover fully here. In this demo, I’ll focus on using the API.

Amazon Location Service allows API calls to be authenticated in two ways: through AWS API authentication (AWS Sigv4 authentication) or through API keys. API keys can be more convenient for developers of mobile applications where the end user is not authenticated or when integrating with Amazon Cognito is not feasible. This is the recommended authentication method for front-end applications.

To demonstrate the versatility of the APIs and how easily you can integrate inside your applications, I use a combination of the AWS CLI, cURL, and a graphical REST API client for each step of the demo.

Step 1: Create an API key

First, I create an API key for my application using the AWS CLI. You can also manage API keys in the AWS Management Console.

REGION=eu-central-1
KEYNAME=geo-key-seb

aws location create-key --region ${REGION} --key-name ${KEYNAME} --restrictions \
AllowActions="geo-routes:*","geo-places:*","geo-maps:*",\
AllowResources="arn:aws:geo-routes:${REGION}::provider/default",\
"arn:aws:geo-places:${REGION}::provider/default",\
"arn:aws:geo-maps:${REGION}::provider/default" \
--no-expiry 

{
    "Key": "v1.public.ey...cy",
    "KeyArn": "arn:aws:geo:eu-central-1:02345678901:api-key/geo-key-seb",
    "KeyName": "geo-key-seb",
    "CreateTime": "2024-09-29T09:35:53.115000+00:00"
}

This command generates the API key, which I can now use to call Amazon Location APIs.

Step 2: Get geographic coordinates

Next, I use cURL to retrieve the geographic coordinates (a longitude and latitude) for the city center of Lille, France, by calling GeoCode and passing an address in the QueryText parameter.

$ curl --silent -X "POST" "https://places.geo.eu-central-1.amazonaws.com/v2/geocode?key=v1.public.ey...cy" \
       -d $'{ "QueryText": "Grand Place, Lille, France" }' 
 
{"ResultItems":[{"PlaceId":"AQ...5U","PlaceType":"Street","Title":"Grand'Place, 59800 Lille, France",
                 "Address":{"Label":"Grand'Place, 59800 Lille, France",
                 "Country":{"Code2":"FR","Code3":"FRA","Name":"France"},
                 "Region":{"Code":"HDF","Name":"Hauts-de-France"},"SubRegion":{"Name":"Nord"},
                 "Locality":"Lille","District":"Centre","PostalCode":"59800",
                 "Street":"Grand'Place","StreetComponents":[{"BaseName":"Grand'Place","Language":"fr"}]},
                 "Position":[3.06361,50.63706],
                 "MapView":[3.0628,50.6367,3.06413,50.63729],
                 "MatchScores":{"Overall":1,"Components":{"Address":{"Country":1,"Locality":1,"Intersection":[1]}}}}]}

This returns several data points, including the GPS coordinates for the city center: [3.06361, 50.63706].

Step 3: Search for nearby places

Using the coordinates retrieved, I use a REST API client tool to call the SearchNearby API to find places of interest around Lille’s city center.

Amazon Location Service - API call for places of interest

On the right side of the screen, I can read the API response: a list of nearby places, such as restaurants, banks, and parking areas. I can further refine the search by specifying categories or restricting the search area.

The SearchNearby API accepts an optional Filter parameter that helps you restrict the search within a bounding box or to include or exclude business chains, categories, countries, or food types.

"Filter": {
   "BoundingBox": [ number ],
   "ExcludeBusinessChains": [ "string" ],
   "ExcludeCategories": [ "string" ],
   "ExcludeFoodTypes": [ "string" ],
   "IncludeBusinessChains": [ "string" ],
   "IncludeCategories": [ "string" ],
   "IncludeCountries": [ "string" ],
   "IncludeFoodTypes": [ "string" ]
},

In my search for nearby points of interest, one of the results returned was a McDonald’s, a well-known international reference 🍔.

Amazon Location Service - SearchNearby result

Step 4: Get driving directions

Finally, I use the AWS CLI to calculate driving directions between two city centers: Brussels, Belgium, and Lille, France.

aws location calculate-routes    \
    --origin 4.35278 50.84687      \
    --destination 3.06361 50.63706 \
    --key "v1.public.ey...cy"

The response includes a polyline for rendering the path on a map and a step-by-step list of driving directions.

...
          "TravelMode": "Car",
          "Type": "Vehicle",
          "VehicleLegDetails": {
            "TravelSteps": [
              {
                "Duration": 15,
                "Distance": 75,
                "ExitNumber": [],
                "GeometryOffset": 0,
                "Type": "Depart"
              },
              {
                "Duration": 10,
                "Distance": 8,
                "ExitNumber": [],
                "GeometryOffset": 2,
                "Type": "Turn",
                "TurnStepDetails": {
                  "Intersection": [],
                  "SteeringDirection": "Right",
                  "TurnIntensity": "Typical"
                }
              },
...

Step 5: Displaying the driving directions on a map

To visualize the route on a map, I use the MapLibre library, which is a rendering engine for displaying maps in web and mobile applications. Following the Amazon Location Service Developer Guide, I built a basic app to display the route.

Amazon Location Service - Map with route

In addition to MapLibre, you can use AWS Amplify to integrate and display Amazon Location data in your applications.

Getting started
With these new and updated APIs, Amazon Location Service offers a more comprehensive suite of mapping and location data for your business needs. These will help to accelerate your development lifecyle by increasing developers’ agility and scalability.

To get started, explore the updated Amazon Location Service Developer Guide and begin integrating these features today. You can also visit the Amazon Location Service page to learn more or try out the APIs with your favorite AWS SDKs to see how they can enhance your applications.

— seb

EC2 Image Builder now supports building and testing macOS images

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/ec2-image-builder-now-supports-building-and-testing-macos-images/

I’m thrilled to announce macOS support in EC2 Image Builder. This new capability allows you to create and manage machine images for your macOS workloads in addition to the existing support for Windows and Linux.

A golden image is a bootable disk image, also called an Amazon Machine Image (AMI), pre-installed with the operating system and all the tools required for your workloads. In the context of a continuous integration and continuous deployment (CI/CD) pipeline, your golden image most probably contains the specific version of your operating system (macOS) and all required development tools and libraries to build and test your applications (Xcode, Fastlane, and so on.)

Developing and manually managing pipelines to build macOS golden images is time-consuming and diverts talented resources from other tasks. And when you have existing pipelines to build Linux or Windows images, you need to use different tools for creating macOS images, leading to a disjointed workflow.

For these reasons, many of you have been asking for the ability to manage your macOS images using EC2 Image Builder. You want to consolidate your image pipelines across operating systems and take advantage of the automation and cloud-centered integrations that EC2 Image Builder provides.

By adding macOS support to EC2 Image Builder, you can now streamline your image management processes and reduce the operational overhead of maintaining macOS images. EC2 Image Builder takes care of testing, versioning, and validating the base images at scale, saving you the costs associated with maintaining your preferred macOS versions.

Let’s see it in action
Let’s create a pipeline to create a macOS AMI with Xcode 16. You can follow a similar process to install Fastlane on your AMIs.

At a high level, there are four main steps.

  1. I define a component for each tool I want to install. A component is a YAML document that tells EC2 Image Builder what application to install and how. In this example, I create a custom component to install Xcode. If you want to install Fastlane, you create a second component. I use the ExecuteBash action to enter the shell commands required to install Xcode.
  2. I define a recipe. A recipe starts from a base image and lists the components I want to install on it.
  3. I define the infrastructure configuration I want to use to build my image. This defines the pool of Amazon Elastic Compute Cloud (Amazon EC2) instances to build the image. In my case, I allocate an EC2 Mac Dedicated Host in my account and reference it in the infrastructure configuration.
  4. I create a pipeline and a schedule to run on the infrastructure with the given recipes and an image workflow. I test the output AMI and deliver it at the chosen destination (my account or another account)

It’s much easier than it sounds. I’ll show you the steps in the AWS Management Console. I can also configure EC2 Image Builder with the AWS Command Line Interface (AWS CLI) or write code using one of our AWS SDKs.

Step 1: Create a component
I open the console and select EC2 Image Builder, then Components, and finally Create component.

Image Builder - Create component

I select a base Image operating system and the Compatible OS Versions. Then, I enter a Component name and Component version. I select Define document content and enter this YAML as Content.

name: InstallXCodeDocument
description: This downloads and installs Xcode. Be sure to run `xcodeinstall authenticate -s us-east-1` from your laptop first.
schemaVersion: 1.0

phases:
  - name: build
    steps:
      - name: InstallXcode
        action: ExecuteBash
        inputs:
          commands:
             - sudo -u ec2-user /opt/homebrew/bin/brew tap sebsto/macos
             - sudo -u ec2-user /opt/homebrew/bin/brew install xcodeinstall
             - sudo -u ec2-user /opt/homebrew/bin/xcodeinstall download -s us-east-1 --name "Xcode 16.xip"
             - sudo -u ec2-user /opt/homebrew/bin/xcodeinstall install --name "Xcode 16.xip"
  
  - name: validate
    steps:
      - name: TestXcode
        action: ExecuteBash
        inputs:
          commands:
            -  xcodebuild -version && xcode-select -p   

I use a tool I wrote to download and install Xcode from the command line. xcodeinstall integrates with AWS Secrets Manager to securely store authentication web tokens. Before running the pipeline, I authenticate from my laptop with the command xcodeinstall authenticate -s us-east-1. This command starts a session with Apple server’s and stores the session token in Secrets Manager. xcodeinstall uses this token during the image creation pipeline to download Xcode.

When you use xcodeinstall with Secrets Manager, you must give permission to your pipeline to access the secrets. Here is the policy document I added to the role attached to the EC2 instance used by EC2 Image Builder (in the following infrastructure configuration).

{
	"Sid": "xcodeinstall",
	"Effect": "Allow",
	"Action": [
            "secretsmanager:GetSecretValue"
            "secretsmanager:PutSecretValue"
        ],
	"Resource": "arn:aws:secretsmanager:us-east-1:<YOUR ACCOUNT ID>:secret:xcodeinstall*"
}

To test and debug these components locally, without having to wait for long cycle to start and recycle the EC2 Mac instance, you can use the AWS Task Orchestrator and Executor (AWSTOE) command.

Step 2: Create a recipe
The next step is to create a recipe. On the console, I select Image recipes and Create image recipe.

I select macOS as the base Image Operating System. I choose macOS Sonoma ARM64 as Image name.

In the Build components section, I select the Xcode 16 component I just created during step 1.

Finally, I make sure the volume is large enough to store the operating system, Xcode, and my builds. I usually select a 500 Gb gp3 volume.

Image Builder - Create a recipe

Steps 3 and 4: Create the pipeline (and the infrastructure configuration)
On the EC2 Image Builder page, I select Image pipelines and Create image pipeline. I give my pipeline a name and select a Build schedule. For this demo, I select a manual trigger.Image Builder - Create Pipeline 1

Then, I select the recipe I just created (Sonoma-Xcode).

Image Builder - Create Pipeline 2

I chose Default workflows for Define image creation process (not shown for brevity).

I create or select an existing infrastructure configuration. In the context of building macOS images, you have to allocate Amazon EC2 Dedicated Hosts first. This is where I choose the instance type that EC2 Image Builder will use to create the AMI. I may also optionally select my virtual private cloud (VPC), security group, AWS Identity and Access Management (IAM) roles with permissions required during the preparation of the image, key pair, and all the parameters I usually select when I start an EC2 instance.

Image Builder - Create Pipeline 4

Finally, I select where I want to distribute the output AMI. By default, it stays on my account. But I can also share or copy it to other accounts.

Image Builder - Create Pipeline 5

Run the pipeline
Now I’m ready to run the pipeline. I select Image pipelines, then I select the pipeline I just created (Sonoma-Xcode). From the Actions menu, I select Run pipeline.

Image Builder - launch pipeline

I can observe the progress and the detailed logs from Amazon CloudWatch.

After a while, the AMI is created and ready to use.

Image Builder - AMI build succeeded

Testing my AMI
To finish the demo, I start an EC2 Mac instance with the AMI I just created (remember to allocate a Dedicated Host first or to reuse the one you used for EC2 Image Builder).

Once the instance is started, I connect to it using secure shell (SSH) and verify that Xcode is correctly installed.

Image Builder - Connect to new AMI

Pricing and availability
EC2 Image Builder for macOS is now available in all AWS Regions where EC2 Mac instances are available: US East (Ohio, N. Virginia), US West (Oregon), Asia Pacific (Mumbai, Seoul, Singapore, Sydney, Tokyo), and Europe (Frankfurt, Ireland, London, Stockholm) (not all Mac instance types are available in all Regions).

It comes at no additional cost, and you’re only charged for the resources in use during the pipeline execution, namely the time your EC2 Mac Dedicated Host is allocated, with a minimum of 24 hours.

The preview of macOS support in EC2 Image Builder allows you to consolidate your image pipelines, automate your golden image creation processes, and use the benefits of cloud-focused integrations on AWS. As the EC2 Mac platform continues to expand with more instance types, this new capability positions EC2 Image Builder as a comprehensive solution for image management across Windows, Linux, and macOS.

Create your first pipeline today! 

— seb

AWS Weekly Roundup: What’s App, AWS Lambda, Load Balancers, AWS Console, and more (Oct 14, 2024).

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-whats-app-aws-lambda-load-balancers-aws-console-and-more-oct-14-2024/

Last week, AWS hosted free half-day conferences in London and Paris. My colleagues and I demonstrated how developers can use generative AI tools to speed up their design, analysis, code writing, debugging, and deployment workflows. These events were held at the GenAI Lofts. These lofts are open until October 25 (London) and November 5 (Paris). They will be packed with events, conferences, workshops, and meetups. If you’re around, be sure to check the agenda (London, Paris).

The AWS team at the NGDE day in London Veliswa live coding on stage at NGDE Day London

Our well-known AWS News blog co-author Veliswa did an amazing demo. She live-coded a Duolingo-like app from scratch, just using suggestions and reviews from Amazon Q Developer.

Now, let’s turn to other exciting news in the AWS universe from last week.

Last week’s launches
Here are some launches that got my attention:

Bring your conversations to WhatsAppAWS has added support for What’sApp to AWS End User Messaging, so developers can reach users on WhatsApp with multimedia and interactive messaging options. This feature integrates with SMS and push notifications already available. Developers can get started quickly using AWS Management Console.

Amazon Redshift data sharing with data lake tables — This offers a secure and convenient way to share live data lake tables across different Amazon Redshift warehouses. Data sharing of data lake tables in AWS Glue Data Catalog provides live access to the data, so you always see the most up-to-date and consistent information as it’s updated in the data lake.

Zonal shift and zonal autoshift for cross zoned Network Load BalancerNetwork Load Balancer (NLB) now supports the Amazon Application Recovery Controller zonal shift and zonal autoshift features on load balancers that are enabled across zones. With Zonal shift, you can quickly shift traffic away from an impaired Availability Zone and recover from events such as bad application deployment and gray failures. Zonal autoshift safely and automatically shifts your traffic away from an Availability Zone when AWS identifies a potential impact to it.

Console to Code to generate infrastructure as a service code — This is by far my favorite launch of the week. Console to Code makes it simple, fast, and cost-effective to move from prototyping in the AWS Management Console to building code for production deployments. You can generate code for their console actions in their preferred format with a single click. The generated code helps you get started and bootstrap your automation pipelines for tasks. Console to Code is powered by Amazon Q Developer.

A new getting started experience for AWS CodePipelineAWS Data Pipeline introduces a simplified and new getting started experience so you can quickly create new pipelines. When you create a new pipeline using the CodePipeline console, you can now select from a list of pipeline templates across build, automation, and deployment use cases. After selecting a pipeline template, you will be prompted to enter values for the action configuration fields in the pipeline definition, and completing the process will render a fully configured pipeline that’s ready to run.

AWS Lambda detects and stops recursive loops between Lambda and Amazon S3 — Lambda recursive loop detection can now automatically detect and stop recursive loops between AWS Lambda and Amazon Simple Storage Service (Amazon S3). Lambda recursive loop detection, which is enabled by default, is a preventative guardrail that automatically detects and stops recursive invocations between Lambda and other supported services, preventing unintended usage and billing from runaway workloads.

Amazon MemoryDB for ValkeyAmazon MemoryDB for Redis is a fully managed, Valkey– and Redis OSS-compatible database service, which provides multi-AZ durability, microsecond read and single-digit millisecond write latency, and high throughput. It is ideal for use cases such as caching, leaderboards, and session stores. With MemoryDB for Valkey, you can benefit from a fully managed experience built on open-source technology while using the security, operational excellence, and reliability that AWS provides. MemoryDB for Valkey also delivers the fastest vector search performance at the highest recall rates among popular vector databases on AWS.

Amazon Polly adds four wew English voices for the generative engine and expands to three RegionsPolly is a managed service that turns text into lifelike speech, so you can create applications that talk and to build speech-enabled products depending on your business needs. The generative engine is the most advanced Amazon Polly text-to-speech (TTS) model. With this launch, we add a variety of new synthetic generative English voices to the Amazon Polly portfolio: an Australian English voice Olivia and three US English voices Joanna, Danielle, and Stephen. These voices have more natural pronunciation and prosody. You can use this high-tier product in various industries and for different purposes such as education, publishing, or marketing.

For a full list of AWS announcements, be sure to keep an eye on the AWS What’s New Feed page.

Upcoming AWS events
Check your calendars and sign up for these AWS events:

AWS Cloud Day Prague — Join us for a free technical conferences in Prague on October 23. I will be there and share with attendees “The Art of Transforming a Foundation Model into a Domain Expert”. Be sure to register today!

Innovate Migrate, Modernize, and Build Whether you are new to the cloud or an experienced user, you will learn something new at AWS Innovate. This is a free online conference. Register for a time and region convenient to North America (October 15), or Europe, Middle East & Africa (October 24).

AWS Community Days Join community-led conferences featuring technical discussions, workshops, and hands-on labs led by expert AWS users and industry leaders from around the world. Don’t miss out on the AWS Community Days happening on October 19 in Vadodara, Spain, and Guatemala.

AWS re:Invent 2024 Registration is now open for the annual tech extravaganza, taking place December 2 – 6 in Las Vegas. Beside recording podcast episodes, I will present three sessions:

  • CMP410 | Accelerate testing cycles of CI/CD pipelines with EC2 Mac instances (with Vishal)
  • DEV301 | The art of transforming foundation models into domain experts (with Gregory)
  • DEV334 | Swift, server-side, serverless

There are just a few seats left for these three sessions, so be sure to book your seat today!

Browse more upcoming AWS led in-person and virtual events and developer-focused events.

That’s all for this week. Check back next Monday for another Weekly Roundup!

— seb

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

NICE DCV is now Amazon DCV with 2024.0 release

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/nice-desktop-cloud-visualization-dcv-is-now-amazon-dcv/

Today, NICE DCV has a new name. So long NICE DCV, welcome Amazon DCV. Today, with the 2024.0 release, along with enhancements and bug fixes, NICE DCV is rebranded to Amazon DCV.

The new name is now also used to consistently refer to the DCV protocol powering AWS managed services such as Amazon AppStream 2.0 and Amazon WorkSpaces.

What is Amazon DCV
Amazon DCV is a high-performance remote display protocol. It lets you securely deliver remote desktops and application streaming from any cloud or data center to any device, over varying network conditions. By using Amazon DCV with Amazon Elastic Compute Cloud (Amazon EC2), you can run graphics-intensive applications remotely on EC2 instances. You can then stream the results to more modest client machines, which eliminates the need for expensive dedicated workstations.

Amazon DCV supports both Windows and major flavors of Linux operating systems on the server side, providing you flexibility to fit your organization’s needs. The client-side that receives the desktops and application streamings could be the native DCV client for Windows, Linux, or macOS or web browsers. The DCV remote server and client transfer only encrypted pixels, not data, so no confidential data is downloaded from the DCV server. When you choose to use Amazon DCV on Amazon Web Services (AWS) with EC2 instances, you can take advantage of the AWS 108 Availability Zones across the 33 geographic Regions and 31 local zones, allowing your remote streaming services to scale globally.

Since Amazon acquired NICE 8 years ago, we’ve witnessed a diverse range of customers adopting DCV. From general-purpose users visualizing business applications to industry-specific professionals, DCV has proven to be versatile. For instance, artists have employed DCV to access powerful cloud workstations for their digital content creation and rendering tasks. In the healthcare sector, medical imaging professionals have used DCV for remote visualization and analysis of patient data. Geoscientists have used DCV to analyze reservoir simulation results, while engineers in manufacturing have used it to visualize computational fluid dynamics experiments. The education and IT support industries have benefited from collaborative sessions in DCV, in which multiple users can share a single desktop.

Notable customers include Quantic Dream, an award-winning game development studio that has harnessed DCV to create high-resolution, low-latency streaming services for their artists and developers. Tally Solutions, an enterprise resource planning (ERP) services provider, has employed DCV to securely stream its ERP software to thousands of customers. Volkswagen has used DCV to provide remote access to computer-aided engineering (CAE) applications for over 1,000 automotive engineers. Amazon Kuiper, an initiative to bring broadband connectivity to underserved communities, has used DCV for designing complex chips.

Within AWS, DCV has been adopted by several services to provide managed solutions to customers. For example, AppStream 2.0 uses DCV to offer secure, reliable, and scalable application streaming. Additionally, since 2020, Amazon WorkSpaces Streaming Protocol (WSP), which is built on DCV and optimized for high performance, is available for Amazon WorkSpaces customers. Today, we’re also phasing out the WSP name and replacing it with DCV. Going forward, you will have DCV as a primary protocol choice in Amazon WorkSpaces.

What’s new with version 2024.0
Amazon DCV 2024.0 introduces several fixes and enhancements for improved performance, security, and ease of use. The 2024.0 release now supports the latest Ubuntu 24.04 LTS, bringing the latest security updates and extended long-term support to simplify system maintenance. The DCV client on Ubuntu 24.04 has built in support for Wayland, offering better graphical rendering efficiency and enhanced application isolation. Additionally, DCV 2024.0 now enables the QUIC UDP protocol by default, allowing clients to benefit from an optimized streaming experience. The release also introduces the capability to blank the Linux host screen when a remote user is connected, preventing local access and interaction with the remote session.

How to get started
The easiest way to test DCV is to spin up a WorkSpaces instance from the WorkSpaces console, selecting one of the DCV-powered bundles, or creating an AppStream session. For this demo however, I want to show you how to install DCV server on an EC2 instance.

I installed DCV server on two servers running on Amazon EC2, one running Windows Server 2022 and one running Ubuntu 24.04. I also installed the client on my macOS laptop. The client and server packages are available to download on our website. For both servers, make sure the security group authorizes inbound connection on UDP or TCP port 8443, the default port DCV uses.

The Windows installation is straightforward: start the msi file, select Next at each step and voilà. It was installed in less time than it took me to write this sentence.

The installation on Linux deserves a bit more care. Amazon Machine Images (AMI) for EC2 servers don’t include any desktop or graphical components. As a prerequisite, I had to install the X Window System and a window manager, and configure X to let users connect and start a graphical user interface session on the server. Fortunately, all these steps are well documented. Here is a summary of the commands I used.

# install desktop packages 
$ sudo apt install ubuntu-desktop

# install a desktop manager 
$ sudo apt install gdm3

# reboot
$ sudo reboot

After the reboot, I installed the DCV server package

# Install the server 
$ sudo apt install ./nice-dcv-server_2024.0.17794-1_amd64.ubuntu2404.deb
$ sudo apt install ./nice-xdcv_2024.0.625-1_amd64.ubuntu2404.deb

# (optional) install the DCV web viewer to allow clients to connect from a web browser
$ sudo apt install ./nice-dcv-web-viewer_2024.0.17794-1_amd64.ubuntu2404.deb

Because my server had no GPU, I also followed these steps to install X11 Dummy driver and configure X11 to use it.

Then, I started the service:

$ sudo systemctl enable dcvserver.service 
$ sudo systemctl start dcvserver.service 
$ sudo systemctl status dcvserver.service 

I created a user at the operating system level and assigned a password and a home directory. Then, I checked my setup on the server before trying to connect from the server.

$ sudo dcv list-sessions
There are no sessions available.

$ sudo dcv create-session console --type virtual --owner seb

$ sudo dcv list-sessions
Session: 'console' (owner:seb type:virtual)

Once my server configuration was ready, I started the DCV client on my laptop. I only had to enter the IP address of the server and the username and password of the user to initiate a session.

DCV Client - enter ip address DCV Client enter username and apssword

On my laptop, I opened a new DCV client window and connected to the other EC2 server. After a few seconds, I was able to remotely work with the Windows and the Ubuntu machine running in the cloud.

DCV two clients from macOS

In this example, I focus on installing Amazon DCV on a single EC2 instance. However, when building your own service infrastructure, you may want to explore the other components that are part of the DCV offering: Amazon DCV Session Manager, Amazon DCV Access Console, and Amazon DCV Connection Gateway.

Pricing and availability
Amazon DCV is free of charges when used on AWS. You only pay for the usage of AWS resources or services, such as EC2 instances, Amazon Workspace desktops, or Amazon App Stream 2.0. If you plan to use DCV with on-premises servers, check the list of license resellers on our website.

Now go build your own servers with DCV.

— seb

Add macOS to your continuous integration pipelines with AWS CodeBuild

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/add-macos-to-your-continuous-integration-pipelines-with-aws-codebuild/

Starting today, you can build applications on macOS with AWS CodeBuild. You can now build artifacts on managed Apple M2 machines that run on macOS 14 Sonoma. AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces ready-to-deploy software packages.

Building, testing, signing, and distributing applications for Apple systems (iOS, iPadOS, watchOS, tvOS, and macOS) requires the use of Xcode, which runs exclusively on macOS. When you build for Apple systems in the AWS Cloud, it is very likely you configured your continuous integration and continuous deployment (CI/CD) pipeline to run on Amazon Elastic Cloud Compute (Amazon EC2) Mac instances.

Since we launched Amazon EC2 Mac in 2020, I have spent a significant amount of time with our customers in various industries and geographies, helping them configure and optimize their pipelines on macOS. In the simplest form, a customer’s pipeline might look like the following diagram.

iOS build pipeline on EC2 Mac

The pipeline starts when there is a new commit or pull request on the source code repository. The repository agent installed on the machine triggers various scripts to configure the environment, build and test the application, and eventually deploy it to App Store Connect.

Amazon EC2 Mac drastically simplifies the management and automation of macOS machines. As I like to describe it, an EC2 Mac instance has all the things I love from Amazon EC2 (Amazon Elastic Block Store (Amazon EBS) volumes, snapshots, virtual private clouds (VPCs), security groups, and more) applied to Mac minis running macOS in the cloud.

However, customers are left with two challenges. The first is to prepare the Amazon Machine Image (AMI) with all the required tools for the build. A minimum build environment requires Xcode, but it is very common to install Fastlane (and Ruby), as well as other build or development tools and libraries. Most organizations require multiple build environments for multiple combinations of macOS and Xcode versions.

The second challenge is to scale your build fleet according to the number and duration of builds. Large organizations typically have hundreds or thousands of builds per day, requiring dozens of build machines. Scaling in and out of that fleet helps to save on costs. EC2 Mac instances are reserved for your dedicated use. One instance is allocated to one dedicated host. Scaling a fleet of dedicated hosts requires a specific configuration.

To address these challenges and simplify the configuration and management of your macOS build machines, today we introduce CodeBuild for macOS.

CodeBuild for macOS is based on the recently introduced reserved capacity fleet, which contains instances powered by Amazon EC2 that are maintained by CodeBuild. With reserved capacity fleets, you configure a set of dedicated instances for your build environment. These machines remain idle, ready to process builds or tests immediately, which reduces build durations. With reserved capacity fleets, your machines are always running and will continue to incur costs as long as they’re provisioned.

CodeBuild provides a standard disk image (AMI) to run your builds. It contains preinstalled versions of Xcode, Fastlane, Ruby, Python, Node.js, and other popular tools for a development and build environment. The full list of tools installed is available in the documentation. Over time, we will provide additional disk images with updated versions of these tools. You can also bring your own custom disk image if you desire.

In addition, CodeBuild makes it easy to configure auto scaling. You tell us how much capacity you want, and we manage everything from there.

Let’s see CodeBuild for macOS in action
To show you how it works, I create a CI/CD pipeline for my pet project: getting started with AWS Amplify on iOS. This tutorial and its accompanying source code explain how to create a simple iOS app with a cloud-based backend. The app uses a GraphQL API (AWS AppSync), a NoSQL database (Amazon DynamoDB), a file-based storage (Amazon Simple Storage Service (Amazon S3)), and user authentication (Amazon Cognito). AWS Amplify for Swift is the piece that glues all these services together.

The tutorial and the source code of the app are available in a Git repository. It includes scripts to automate the build, test, and deployment of the app.

Configuring a new CI/CD pipeline with CodeBuild for macOS involves the following high-level steps:

  1. Create the build project.
  2. Create the dedicated fleet of machines.
  3. Configure one or more build triggers.
  4. Add a pipeline definition file (buildspec.yaml) to the project.

To get started, I open the AWS Management Console, select CodeBuild, and select Create project.

codebuild mac - 1

I enter a Project name and configure the connection to the Source code repository. I use GitHub in this example. CodeBuild also supports GitLab and BitBucket. The documentation has an up-to-date list of supported source code repositories.

codebuild mac - 2

For the Provisioning model, I select Reserved capacity. This is the only model where Amazon EC2 Mac instances are available. I don’t have a fleet defined yet, so I decide to create one on the flight while creating the build project. I select Create fleet.

codebuild mac - 3

On the Compute fleet configuration page, I enter a Compute fleet name and select macOS as Operating system. Under Compute, I select the amount of memory and the quantity of vCPUs needed for my build project, and the number of instances I want under Capacity.

For this example, I am happy to use the Managed image. It includes Xcode 15.4 and the simulator runtime for iOS 17.5, among other packages. You can read the list of packages preinstalled on this image in the documentation.

When finished, I select Create fleet to return to the CodeBuild project creation page.

CodeBuild - create fleet

As a next step, I tell CodeBuild to create a new service role to define the permissions I want for my build environment. In the context of this project, I must include permissions to pull an Amplify configuration and access AWS Secrets Manager. I’m not sharing step-by-step instructions to do so, but the sample project code contains the list of the permissions I added.

codebuild mac - 4

I can choose between providing my set of build commands in the project definition or in a buildspec.yaml file included in my project. I select the latter.

codebuild mac - 5

This is optional, but I want to upload the build artifact to an S3 bucket where I can archive each build. In the Artifact 1 – Primary section, I therefore select Amazon S3 as Type, and I enter a Bucket name and artifact Name. The file name to upload is specified in the buildspec.yaml file.

codebuild mac - 6

Down on the page, I configure the project trigger to add a GitHub WebHook. This will configure CodeBuild to start the build every time a commit or pull request is sent to my project on GitHub.

codebuild - webhook

Finally, I select the orange Create project button at the bottom of the page to create this project.

Testing my builds
My project already includes build scripts to prepare the build, build the project, run the tests, and deploy it to Apple’s TestFlight.

codebuild - project scripts

I add a buildspec.yaml file at the root of my project to orchestrate these existing scripts.

version: 0.2

phases:

  install:
    commands:
      - code/ci_actions/00_install_rosetta.sh
  pre_build:
    commands:
      - code/ci_actions/01_keychain.sh
      - code/ci_actions/02_amplify.sh
  build:
    commands:
      - code/ci_actions/03_build.sh
      - code/ci_actions/04_local_tests.sh
  post_build:
    commands:
      - code/ci_actions/06_deploy_testflight.sh
      - code/ci_actions/07_cleanup.sh
artifacts:
   name: $(date +%Y-%m-%d)-getting-started.ipa
   files:
    - 'getting started.ipa'
  base-directory: 'code/build-release'

I add this file to my Git repository and push it to GitHub with the following command: git commit -am "add buildpsec" buildpec.yaml

On the console, I can observe that the build has started.

codebuild - build history

When I select the build, I can see the log files or select Phase details to receive a high-level status of each phase of the build.

codebuild - phase details

When the build is successful, I can see the iOS application IPA file uploaded to my S3 bucket.

aws s3 ls

The last build script that CodeBuild executes uploads the binary to App Store Connect. I can observe new builds in the TestFlight section of the App Store Connect.

App Store Connect

Things to know
It takes 8-10 minutes to prepare an Amazon EC2 Mac instance and to accept the very first build. This is not specific to CodeBuild. The builds you submit during the machine preparation time are queued and will be run in order as soon as the machine is available.

CodeBuild for macOS works with reserved fleets. Contrary to on-demand fleets, where you pay per minute of build, reserved fleets are charged for the time the build machines are reserved for your exclusive usage, even when no builds are running. The capacity reservation follows the Amazon EC2 Mac 24-hour minimum allocation period, as required by the Software License Agreement for macOS (article 3.A.ii).

A fleet of machines can be shared across CodeBuild projects on your AWS account. The machines in the fleet are reserved for your exclusive use. Only CodeBuild can access the machines.

CodeBuild cleans the working directory between builds, but the machines are reused for other builds. It allows you to use the CodeBuild local cache mechanism to quickly restore selected files after a build. If you build different projects on the same fleet, be sure to reset any global state, such as the macOS keychain, and build artifacts, such as the SwiftPM and Xcode package caches, before starting a new build.

When you work with custom build images, be sure they are built for a 64-bit Mac-Arm architecture. You also must install and start the AWS Systems Manager Agent (SSM Agent). CodeBuild uses the SSM Agent to install its own agent and to manage the machine. Finally, make sure the AMI is available to the CodeBuild organization ARN.

CodeBuild for macOS is available in the following AWS Regions: US East (Ohio, N. Virginia), US West (Oregon), Asia Pacific (Sydney), and Europe (Frankfurt). These are the same Regions that offer Amazon EC2 Mac M2 instances.

Get started today and create your first CodeBuild project on macOS.

— seb

Customize Amazon Q Developer (in your IDE) with your private code base

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/customize-amazon-q-developer-in-your-ide-with-your-private-code-base/

Today, we’re making the Amazon Q Developer (in your IDE) customization capability generally available for inline code completion, and we’re launching a preview of customization for the chat. You can now customize Amazon Q to generate specific code recommendations from private code repositories in the IDE code editor and in the chat.

Amazon Q Developer is an artificial intelligence (AI) coding companion. It helps software developers accelerate application development by offering code recommendations in their integrated development environments (IDE) derived from existing comments and code. Behind the scenes, Amazon Q uses large language models (LLMs) trained on billions of lines of code from Amazon and open source projects.

Amazon Q is available in your IDE, and you can download the extension for JetBrains, Visual Studio Code, and Visual Studio (preview). In the IDE text editor, it suggests code as you type or write entire functions from a comment you enter. You can also chat with Q Developer and ask it to generate code for specific tasks or explain code snippets from a code base you’re discovering.

With the new customization capability, developers can now receive even more relevant code recommendations that are based on their organization’s internal libraries, APIs, packages, classes, and methods.

For example, let’s imagine that a developer working for a financial company is tasked to write a function to compute the total portfolio value for a customer. The developer can now describe the intent in a comment or type a function name such as computePortfolioValue(customerId: String), and Amazon Q will suggest code to implement that function based on the examples it learned from your organization’s private code base.

The developer can also ask questions about their organization’s code in the chat. In the example above, let’s imagine the developer is new to the team and doesn’t know how to retrieve a customer ID. He can ask the question in the chat in plain English: how do I connect to the database to retrieve the customerId for a specific customer? Amazon Q chat could answer: I found a function to retrieve customerId based on customer first and last name which uses the database connection XYZ…

As an administrator, you create customizations built from your internal git repositories (such as GitHub, GitLab, or BitBucket) or an Amazon Simple Storage Service (Amazon S3) bucket. It helps Amazon Q understand the intent, determine which internal and public APIs are best suited to the task, and generate code recommendations.

Amazon Q customization capability meets the strong data privacy and security you expect from AWS. The code base you share with Amazon Q stays private to your organization. We don’t use it to train our foundation model. Once customizations are deployed, the inference endpoint is private for the developers in your organization. Recommendations based on your code won’t pop up in another company’s developer IDE. You decide which developers have access to each individual customization, and you can follow metrics to measure the performance of the customizations you deployed.

We built the Amazon Q customization capability based on leading technical techniques, such as Retrieval Augmented Generation (RAG). This very detailed blog post shares more details about the science behind the Amazon Q customizations capability.

CodeWhisperer RAG diagram

Since we launched the preview on October 17 last year, we’ve added two new capabilities: the ability to update a customization and the ability to customize the chat in the IDE.

Your organization’s code base is constantly evolving, and you want Amazon Q to always suggest up-to-date code snippets. Amazon Q administrator can now start an update process with one step in the AWS Management Console. Administrators can schedule regular updates based on the latest commits on code repositories to ensure developers always receive highly accurate code suggestions.

With the new chat customization (in preview), developers in your organization can select a portion of code in their IDE and send it to the chat to ask for an explanation of what the selected code does. Developers can also ask generic questions relative to their organization’s code base, like “How do I connect to the database to retrieve customerId for a specific customer?”

Let’s see how to use it
In this demo, I decided to focus on the new customization update capability that is generally available today. To quickly learn how to create a customization, activate it, and grant access to developers, read the excellent post from my colleague Donnie.

To update an existing customization, I navigate to the Customizations section of the Amazon Q console page. I select the customization I want to update. Then, I select Actions and Create new version.

Codewhisperer customization - update 1a

Similarly to what I did when I created the customization, I choose the source code repository and select Create.

Codewhisperer customization

Creating a new version of the customization might take a while because depends on the quantity of code to ingest. When ready, a new version appears under the Versions tab. You can compare the Evaluation score of the new version with the previous versions and decide to activate it to make it available to your developers. At any point, you can revert to a previous version.

Codewhisperer customization - update 3a

One of the aspects I like about active customizations is that I can monitor their effectiveness to help increase developer productivity in my organization.

On the Dashboard page, I track the User activity. I can track how many Daily active users there are, how many Lines of code have been generated, how many Security scans were performed, and so on. If, like me, you have used Amazon CodeWhisperer Professional in the past, when you use it now, you might still see the name CodeWhisperer appear on some pages. It will progressively be replaced with the new name: Amazon Q Developer.

Codewhisperer customization dashboard 1

Amazon Q generates more metrics and publishes them on Amazon CloudWatch. I can build CloudWatch dashboards to monitor the performance of the customizations I deployed. For example, here is a custom CloudWatch dashboard that monitors the code suggestions’ Block Accept Rate and Line Accept Rate, broken down per programming language.

Codewhisperer customization dashboard 2

Supported programming languages
Currently, you can customize Amazon Q recommendations on codebases written in Java, JavaScript, TypeScript, and Python. Files written in other languages supported by Amazon Q, such as C#, Go, Rust, PHP, Ruby, Kotlin, C, C++, Shell scripting, SQL, and Scala will not be used when creating the customization or when providing customized recommendations in the IDE.

Pricing and availability
Amazon Q is AWS Region agnostic and available to developers worldwide. Amazon Q is currently hosted in US East (N. Virginia). Amazon Q administrators can configure Amazon Q as an authorized cross-Region application if you have AWS IAM Identity Center in other Regions.

The Amazon Q customization capability is available at no additional charge within the Amazon Q Developer Professional subscription. You can create up to eight customizations per AWS account and keep up to two customizations active.

Now go build, and start to propose Amazon Q customizations to your organization’s developers.

— seb

AWS Weekly Roundup: Amazon S3 Access Grants, AWS Lambda, European Sovereign Cloud Region, and more (July 8, 2024).

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-amazon-s3-access-grants-aws-lambda-european-sovereign-cloud-region-and-more-july-8-2024/

I counted only 21 AWS news since last Monday, most of them being Regional expansions of existing services and capabilities. I hope you enjoyed a relatively quiet week, because this one will be busier.

This week, we’re welcoming our customers and partners at the Jacob Javits Convention Center for the AWS Summit New York on Wednesday, July 10. I can tell you there is a stream of announcements coming, if I judge by the number of AWS News Blog posts ready to be published.

I am writing these lines just before packing my bag to attend the AWS Community Day in Douala, Cameroon next Saturday. I can’t wait to meet our customers and partners, students, and the whole AWS community there.

But for now, let’s look at last week’s new announcements.

Last week’s launches
Here are the launches that got my attention.

Amazon Simple Storage Service (Amazon S3) Access Grants now integrate with Amazon SageMaker and open souce Python frameworksAmazon S3 Access Grants maps identities in directories such as Active Directory or AWS Identity and Access Management (IAM) principals, to datasets in S3. The integration with Amazon SageMaker Studio for machine learning (ML) helps you map identities to your machine learning (ML) datasets in S3. The integration with the AWS SDK for Python (Boto3) plugin replaces any custom code required to manage data permissions, so you can use S3 Access Grants in open source Python frameworks such as Django, TensorFlow, NumPy, Pandas, and more.

AWS Lambda introduces new controls to make it easier to search, filter, and aggregate Lambda function logsYou can now capture your Lambda logs in JSON structured format without bringing your own logging libraries. You can also control the log level (for example, ERROR, DEBUG, or INFO) of your Lambda logs without making any code changes. Lastly, you can choose the Amazon CloudWatch log group to which Lambda sends your logs.

Amazon DataZone introduces fine-grained access controlAmazon DataZone has introduced fine-grained access control, providing data owners granular control over their data at row and column levels. You use Amazon DataZone to catalog, discover, analyze, share, and govern data at scale across organizational boundaries with governance and access controls. Data owners can now restrict access to specific records of data instead of granting access to an entire dataset.

AWS Direct Connect proposes native 400 Gbps dedicated connections at select locationsAWS Direct Connect provides private, high-bandwidth connectivity between AWS and your data center, office, or colocation facility. Native 400 Gbps connections provide higher bandwidth without the operational overhead of managing multiple 100 Gbps connections in a link aggregation group. The increased capacity delivered by 400 Gbps connections is particularly beneficial to applications that transfer large-scale datasets, such as for ML and large language model (LLM) training or advanced driver assistance systems for autonomous vehicles.

For a full list of AWS announcements, be sure to keep an eye on the What’s New at AWS page.

Other AWS news
Here are some additional news items that you might find interesting:

The list of services available at launch in the upcoming AWS Europe Sovereign Cloud Region is available – we shared the list of AWS services that will be initially available at launch in the new AWS European Sovereign Cloud Region. The list has no surprises. Services for security, networking, storage, computing, containers, artificial intelligence (AI), and serverless will be available at launch. We are building the AWS European Sovereign Cloud to offer public sector organizations and customers in highly regulated industries further choice to help them meet their unique digital sovereignty requirements, as well as stringent data residency, operational autonomy, and resiliency requirements. This is an investment of 7.8 billion euros (approximately $8.46 billion). The new Region will be available by the end of 2025.

Upcoming AWS events
Check your calendars and sign up for upcoming AWS events:

AWS Summits – Join free online and in-person events that bring the cloud computing community together to connect, collaborate, and learn about AWS. To learn more about future AWS Summit events, visit the AWS Summit page. Register in your nearest city: New York (July 10), Bogotá (July 18), and Taipei (July 23–24).

AWS Community Days – Join community-led conferences that feature technical discussions, workshops, and hands-on labs led by expert AWS users and industry leaders from around the world. Upcoming AWS Community Days are in Cameroon (July 13), Aotearoa (August 15), and Nigeria (August 24).

Browse all upcoming AWS led in-person and virtual events and developer-focused events.

That’s all for this week. Check back next Monday for another Weekly Roundup!

— seb

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!