Post Syndicated from Eric Smith original https://www.servethehome.com/beelink-ex-gpu-dock-quick-look/
We take a look at the Beelink EX GPU Dock that has an internal power supply and allows for high-performance GPU docking to select mini PCs
The post Beelink EX GPU Dock Quick Look appeared first on ServeTheHome.
Post Syndicated from Esra Kayabali original https://aws.amazon.com/blogs/aws/introducing-amazon-guardduty-extended-threat-detection-aiml-attack-sequence-identification-for-enhanced-cloud-security/
Today, I’m happy to introduce advanced AI/ML threat detection capabilities in Amazon GuardDuty. This new feature uses the extensive cloud visibility and scale of AWS to provide improved threat detection for your applications, workloads, and data. GuardDuty Extended Threat Detection employs sophisticated AI/ML to identify both known and previously unknown attack sequences, offering a more comprehensive and proactive approach to cloud security. This enhancement addresses the growing complexity of modern cloud environments and the evolving landscape of security threats, simplifying threat detection and response.
Many organizations face challenges in efficiently analyzing and responding to the high volume of security events generated across their cloud environments. With the increasing frequency and sophistication of security threats, it has become more challenging to effectively detect and respond to attacks that occur as sequences of events over time. Security teams often struggle to piece together related activities that might be part of a larger attack, potentially missing critical threats or responding too late to prevent significant impact.
To address these challenges, we have expanded GuardDuty threat detection capabilities to include new AI/ML capabilities that correlate security signals to identify active attack sequences in your AWS environment. These sequences can include multiple steps taken by an adversary, such as privilege discovery, API manipulation, persistence activities, and data exfiltration. These detections are represented as attack sequence findings, a new type of GuardDuty finding with critical severity. Previously, GuardDuty had never used critical severity, reserving this level for findings with the utmost confidence and urgency. These new findings introduce critical severity and include a natural language summary of the threat’s nature and significance, observed activities mapped to tactics and techniques from the MITRE ATT&CK® framework, and prescriptive remediation recommendations based on AWS best practices.
GuardDuty Extended Threat Detection introduces new attack sequence findings and improves actionability for existing detections in areas such as credential exfiltration, privilege escalation, and data exfiltration. This enhancement enables GuardDuty to offer composite detections that span multiple data sources, time periods, and resources within an account, providing you with a more comprehensive understanding of sophisticated cloud attacks.
Let me show you how the new capabilities work.
How to use the new AI/ML threat detection in Amazon GuardDuty
To experience the new AI/ML threat detection in GuardDuty, go to the Amazon GuardDuty console and explore the new widgets on the Summary page. The overview widget now helps you view the number of attack sequences you have and consider the details of those attack sequences. Cloud environment findings often reveal multistage attacks, but these sophisticated attack sequences are low volume and account for a small fraction of the total number of findings. For this particular account, you can observe a variety of findings in the cloud environment, but only a handful of actual attack sequences. In a larger cloud environment, you may see hundreds or even thousands of findings, yet the number of attack sequences will likely remain relatively small in comparison.
We’ve also added a new widget that helps you view the findings broken down by severity. This makes it easier to quickly pivot into and investigate specific findings that are of interest to you. The findings are now sorted by Severity, providing you with a clear overview of the most critical issues, including an additional Critical severity category, ensuring that the most urgent detections are immediately brought to your attention. You can also filter just for the attack sequences by choosing Top attack sequences only.
This new capability is enabled by default, so you don’t need to take any additional steps for it to start working. There are no extra costs for this feature beyond the underlying charges for GuardDuty and its associated protection plans. As you enable additional GuardDuty protection plans, this capability will provide more integrated security value, helping you gain deeper insights.
You can observe two types of findings. The first one is data compromise, which indicates a potential data compromise that can be a part of a larger ransomware attack. Data is the most critical organizational asset for most customers, making this an important area of concern. The second finding is compromised credential type, which helps you detect the misuse of compromised credentials, typically during the earlier stages of an attack in your cloud environment.
Let me dive into one of the compromise data findings. I’ll focus on “Potential data compromise of one or more S3 buckets involving a sequence of actions over multiple signals associated with a user in your account”. This finding indicates that we have observed data being compromised across multiple Amazon Simple Storage Service (Amazon S3) buckets with multiple associated signals.
The summary provided with this finding gives you key details, including the specific user (identified by their principal ID) who performed the actions, the account and resources affected, and the extended time period (nearly a full day) over which the activity occurred. This information can help you quickly understand the scope and severity of the potential compromise.
This finding has eight distinct signals observed over a nearly 24-hour period, indicating the use of multiple tactics and techniques mapped to the MITRE ATT&CK® framework. This broad coverage across the attack chain—from credential access, to discovery, evasion, persistence, and even impact and exfiltration—suggests this may indeed be a true positive incident. The finding also surfaces a concerning technique of data destruction, which is particularly alarming.
Additionally, GuardDuty provides further security context by highlighting sensitive API calls, such as the user deleting the AWS CloudTrail trail. This type of evasive behavior, combined with the creation of new access keys and actions targeting Amazon S3 objects, further reinforces the severity and potential scope of the incident. Based on the information presented in this finding, you would likely want to investigate this incident more thoroughly.
Reviewing the ATT&CK tactics associated with the findings provides visibility into the specific tactics involved, whether it’s a single tactic or multiple. GuardDuty also offers security indicators that explain why the activity was flagged as suspicious and assigned a critical severity, including the high-risk APIs called and the tactics observed.
Diving deeper, you can view details about the actor responsible. The information includes how the user connected to and carried out these actions, including the network locations. This additional context helps you better understand the full scope and nature of the incident, which is crucial for investigation and response. You can follow prescriptive remediation recommendations based on AWS best practices, offering you actionable insights to swiftly address and resolve identified detections. These tailored recommendations help you improve your cloud security posture and ensure alignment with security guidelines.
The Signals tab can be sorted by newest or oldest first. If responding to an active attack, you’ll want to start with the latest signals to quickly understand and mitigate the situation. For a post-incident review, you can trace back from the initial activities. Diving into each activity provides detailed information about the specific finding. We also offer a quick view through Indicators, Actors, and Endpoints to summarize what occurred and who took action.
Another way to follow the details is to access the Resources tab, where you can check the different buckets that are involved and the access keys. For each resource, you can check which tactics and techniques happened. Select the open resource to pivot directly to the relevant console and learn more details.
We’ve introduced a full-page view for GuardDuty findings, making it easier to see all the contextual data in one place. However, the traditional findings page with the side panel is still available if you prefer that layout, which provides a quick view of the details for specific findings.
GuardDuty Extended Threat Detection is automatically enabled for all GuardDuty accounts in a Region, leveraging foundational data sources without requiring additional protection plans. Enabling additional protection plans expands the range of security signals analyzed, improving the service’s ability to identify complex attack sequences. GuardDuty specifically recommends activating S3 Protection to detect potential data compromises in Amazon S3 buckets. Without S3 Protection enabled, GuardDuty cannot generate S3-specific findings or identify attack sequences involving S3 resources, limiting its capacity to detect data compromise scenarios in your Amazon S3 environment.
GuardDuty Extended Threat Detection integrates with existing GuardDuty workflows, including the AWS Security Hub, Amazon EventBridge, and third-party security event management systems.
Now available
Amazon GuardDuty Extended Threat Detection significantly enhances cloud security by automating the analysis of complex attack sequences and providing actionable insights, helping you focus on addressing the most critical threats efficiently, reducing the time and effort required for manual analysis.
These capabilities are automatically enabled for all new and existing GuardDuty customers at no additional cost in all commercial AWS Regions where GuardDuty is supported.
To learn more and start benefiting from these new capabilities, visit the Amazon GuardDuty documentation.
Post Syndicated from Donnie Prakoso original https://aws.amazon.com/blogs/aws/container-insights-with-enhanced-observability-now-available-in-amazon-ecs/
Last year, we announced enhanced observability in Amazon CloudWatch Container Insights, a new capability to improve your observability for Amazon Elastic Kubernetes Service (Amazon EKS). This capability helps you detect and fix container issues faster by providing detailed performance metrics and logs.
Expanding this capability, today we’re launching enhanced observability for your container workloads running on Amazon Elastic Container Service (Amazon ECS). This new capability will help reduce your mean time to detect (MTTD) and mean time to repair (MTTR) for your overall applications, helping prevent issues that could negatively impact your user experience.
Here’s a quick look at Container Insights with enhanced observability for Amazon ECS.
Container Insights with enhanced observability addresses a critical gap in container monitoring. Previously, correlating metrics with logs and events was a time-consuming process, often requiring manual searches and expertise in application architecture. Now, with this capability, CloudWatch and Amazon ECS automatically collect granular performance metrics such as CPU utilization at both the task and container levels while providing visual drill downs enabling easy root-cause analysis.
This new capability enables the following use cases:
Using container insights with enhanced observability for Amazon ECS
There are two ways to enable Container Insights with enhanced observability:
To enable this feature at the account level, I navigate to the Amazon ECS console and select Account settings. Under the CloudWatch Container Insights observability section, I can see it’s currently disabled. I choose Update.
On this page, I find a new option called Container Insights with enhanced observability. I select this option and then choose Save changes.
If I need to enable this capability at the cluster level, I can do so when creating a new cluster.
I can also enable this capability for my existing clusters. To do so, I select Update cluster, and then choose the option.
Once enabled, I can see task-level metrics by navigating to the Metrics tab in my cluster overview console. To access health and performance metrics across my clusters, I can select View Container Insights, which will redirect me to the Container Insights page.
To get a big picture of all my workloads across different clusters, I can navigate to Amazon CloudWatch and then to Container Insights.
This view addresses the challenge of effectively monitoring clusters, services, tasks, and containers by providing a honeycomb visualization that offers an intuitive, high-level summary of cluster health. The dashboard employs a dual-state monitoring approach:
Let’s say there’s an issue in one of my clusters. I can hover over the cluster to display all the alarms created under that cluster at different layers, from the cluster layer down to the container layer.
I also have the option to view all clusters in a list format. The list format is essential for cross-account observability, displaying account IDs and labels for cluster ownership. This helps DevOps engineers quickly identify and collaborate with account owners to resolve potential application issues.
Now, I’d like to explore further. I select my cluster link, which redirects me to the Container Insights detailed dashboard view. Here, I can see a spike in memory utilization for this cluster.
I can dive deeper into container-level details, which help me quickly identify which services are causing this issue.
Another useful feature I found is the Filters option, which helps me conduct more thorough investigations across containers, services, or tasks in this cluster.
If I need to delve deeper into the application logs to understand the root cause of this issue, I can select the task, choose Actions, and choose which logs I would like to view.
On top of using AWS X-Ray traces, I can investigate another two types of logs here. First, I can use performance logs—structured logs containing metric data—to drill down and identify container-level root causes. Second, I examine collected application or container logs . These logs give me detailed insights into application behavior within the container, helping me trace the sequence of events that led to any issues.
In this case, I use application logs.
This streamlines my journey to troubleshoot my application. In this case, the issue is on the downstream calls to third-party applications, which return timeouts.
This enhanced capability also works with Amazon CloudWatch Application Signals to automatically instrument my application. I can monitor current application health and track long-term application performance against service-level objectives.
I select the Application Signals tab.
This integration with Amazon CloudWatch Application Signals provides me with end-to-end visibility, helping me correlate container performance with end-user experience.
When I select datapoints in the graphs, I can see associated traces, which show me all correlated services and their impact. I can also access relevant logs to understand root causes.
Additional things to know
Here are a couple of important points to note:
Get started today and experience improved observability for your container workloads. Learn more on the Amazon CloudWatch documentation page.
Happy monitoring,
— Donnie Prakoso
Post Syndicated from Esra Kayabali original https://aws.amazon.com/blogs/aws/aws-clean-rooms-now-supports-multiple-clouds-and-data-sources/
Today, we are announcing support for Snowflake and Amazon Athena as new sources for AWS Clean Rooms data collaborations. AWS Clean Rooms helps you and your partners more seamlessly and securely analyze your collective datasets without sharing or copying one another’s underlying data. This enhancement helps you collaborate with datasets stored in Snowflake or those queryable through Athena features, such as AWS Lake Formation permissions or AWS Glue Data Catalog views, without moving or revealing the source data.
You often need to collaborate with partners to analyze datasets to get insights for research and development, investments, or marketing and advertising campaigns. In some cases, your partners’ datasets are stored or managed outside of Amazon Simple Storage Service (Amazon S3), and companies want to reduce or eliminate the complexity, cost, compliance risks, and delays that are associated with moving or copying data. Companies also find that copying data can result in them using outdated information, potentially reducing the quality of the insights gained.
This launch helps companies to collaborate on the most up-to-date collective datasets in an AWS Clean Rooms collaboration with zero extract, transform, and load (zero-ETL). This eliminates the cost and complexity associated with migrating datasets out of existing environments. For example, an advertiser with data stored in Amazon S3 and a media publisher with data stored in Snowflake can run an audience overlap analysis to determine the percentage of users present in their collective datasets without having to build ETL data pipelines, or share underlying data with one another. No underlying data from external data sources is permanently stored in AWS Clean Rooms during the collaboration process and any data temporarily read into the AWS Clean Rooms analysis environment is deleted upon query completion. You can now work with your partners regardless of where their data is stored, streamlining the process of generating insights.
Let me show you how to use this feature.
How to use multiple clouds and data sources in AWS Clean Rooms
To demonstrate this feature, I use a scenario between an advertiser, Company A, and a publisher, Company B. Company A wants to know how many of their high-value users can be reached on Company B’s website before running an ad campaign. Company A stores their data in Amazon S3. Company B stores their data in Snowflake. To use AWS Clean Rooms, both parties must have their own AWS accounts.
In this demo, Company A, the advertiser, is the collaboration creator. Company A creates the AWS Clean Rooms collaboration and invites Company B, who has data hosted in Snowflake, to collaborate. You can follow the specific steps to create a collaboration in the AWS Clean Rooms general availability announcement blog post.
Next, I show how Company B, the publisher, creates a configured table in AWS Clean Rooms, specifying Snowflake as the data source and providing the Secrets Manager Amazon Resource Name (ARN). AWS Secrets Manager helps you manage, retrieve, and rotate secrets such as database credentials throughout their lifecycles. Your secret must contain the credentials for a Snowflake user with read-only permission to the data you want to collaborate with. AWS Clean Rooms will use it to read your secret and access the data stored in Snowflake. See the Secrets Manager documentation for step-by-step instructions for creating your secret.
Using Company B’s AWS account, I go to the AWS Clean Rooms console and choose Tables under Configured resources. I choose Configure new table. I choose Snowflake under Third-party clouds and data sources. I enter the Secret ARN for the secret that contains Snowflake credentials for a role with read access to the dataset stored in Snowflake I want to collaborate with. These are the credentials that you use to verify the identity of the entity trying to access the Snowflake table and schema. If you don’t have a secret ARN, you can create a new secret using the Store a new secret for this table option.
To define the table and schema details, I use the Import from file option and choose the Columns View Information Schema CSV file I exported from Snowflake to populate the information for me. You can also enter the information manually.
For this demo, I choose All columns under the Columns allowed in collaborations. Next, I choose Configure new table.
I go to the configured table and observe the table details, such as AWS accounts allowed to create queries and columns available for querying. On this page, I can edit the table name, description, and analysis rule.
As part of configuring a table to use in AWS Clean Rooms for collaboration analysis, I need to configure an analysis rule. An analysis rule is a privacy-enhancing control that each data owner sets up on a configured table. An analysis rule determines how the configured table can be analyzed. I choose Configure analysis rule to configure a custom analysis rule that allows custom queries to be run on the configured table.
In Step 1, I proceed with the selections. You can use JSON editor to create, paste, or import an analysis rule definition in a JSON format. I choose Next.
In Step 2, I choose Allow any queries created by specific collaborators to run without review on this table under Analyses for direct querying. With this option, only queries provided by the AWS accounts that I specify in the list of allowed accounts can be run on the table. All analysis templates created by the allowed accounts will automatically be allowed to be run on this table without requiring a review. I choose the allowed account under AWS account ID and choose Next.
In Step 3, I proceed with the selections. I choose None under Columns not allowed in output to allow all columns to be shown in the query output. I choose Not allowed under Additional analyses applied to output, so no additional analyses can be run on this table. I choose Next.
In the final step, I review the configuration and choose Configure analysis rule.
Next, I associate the table with the collaboration Company A, the advertiser, created using Associate to collaboration.
On the pop-up window, I choose a collaboration from the ones with active memberships and select Choose collaboration.
On the next page, I choose the Configured table name and enter the Name under Table associations details. I choose a method to authorize AWS Clean Rooms to give the permission to query the table. I choose Associate table.
Company A, the advertiser, and Company B, the publisher, can now run an audience overlap analysis to determine the percentage of users present in their collective datasets without accessing each other’s raw data. The analysis helps determine how much of the advertiser’s audience can be reached by the publisher. By evaluating the overlap, advertisers can determine whether the publisher provides unique reach or if the publisher’s audience predominantly overlaps with the advertiser’s existing audience, without either party having to move or share their source data. I switch to Company A’s account and go to AWS Clean Rooms console. I choose the collaboration I created and run the following query to get the audience overlap analysis result:
select count (distinct emailaddress)
from customer_data_example as advertiser
inner join synthetic_customer_data as publisher
on 'emailaddress' = 'publisher_hashed_email_address'
In this example, I used Snowflake as a data source. You can also run queries on this data using Athena while following AWS Lake Formation permissions. This helps you do row- and column-level filtering with Lake Formation fine-grained access control and transform data using AWS Glue Data Catalog views before the datasets are associated to the collaboration.
Customer and partner voices
“Data security and privacy is essential to our work at Kinective Media by United Airlines, the world’s first traveler media network,” said Khatidja Ajania, Director, Strategic Partnerships, Kinective Media by United Airlines. “AWS Clean Rooms support of source data in multiple clouds and AWS sources enables us to securely and seamlessly work with more brands to deliver on closed loop measurement and other key use cases. This enhancement will make it easier for us to securely deliver personalized experiences, content, and relevant offerings to millions of United travelers through privacy-enhanced collaboration with our advertisers and partners.”
“Snowflake recognizes the challenges of source data interoperability across tech stacks when using data clean room technology; we are excited to see the progress and one more step taken in the direction of a shared goal to empower users to unlock the full potential of their data partnerships through their solution of choice, safely and effectively” – Kamakshi Sivaramakrishnan, General Manager, Snowflake Data Clean Rooms
Now available
Support for Snowflake and Athena as data sources in AWS Clean Rooms offers significant benefits for cross-cloud collaboration. This launch eliminates the need for data movement across clouds and data sources and simplifies the collaboration process. This is a first step in our efforts to expand the ways in which customers can securely collaborate with any of their partners while protecting sensitive information, regardless of where their data is stored.
Get started with AWS Clean Rooms today. To learn more about collaborating with multiple data sources, visit the AWS Clean Rooms documentation.
Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/new-physical-aws-data-transfer-terminals-let-you-upload-to-the-cloud-faster/
Today, we’re announcing the general availability of AWS Data Transfer Terminal, a secure physical location where you can bring your storage devices and upload data faster to the AWS Cloud.
The first Data Transfer Terminals are located in Los Angeles and New York, with plans to add more locations globally. You can reserve a time slot to visit your nearest location and upload data rapidly and securely to any AWS public endpoints, such as Amazon Simple Storage Service (Amazon S3), Amazon Elastic File System (Amazon EFS), or others, using a high throughput connection. Using AWS Data Transfer Terminal, you can significantly reduce the time of ingesting data with high throughput connectivity in the location near by you. You can upload large datasets from fleets of vehicles operating and collecting data in metro areas for training machine learning (ML) models, digital audio and video files from content creators for media processing workloads, and mapping or imagery data from local government organizations for geographic analysis.
After the data is uploaded to AWS, you can use the extensive suite of AWS services to generate value from your data and accelerate innovation. You can also bring your AWS Snowball devices to the location for upload and retain the device for continued use and not rely on traditional shipping methods.
Getting started with AWS Data Transfer Terminal
You can find the availability of a location in the AWS Management Console and reserve the date and time to visit. Then, you can visit the location, make a connection between your storage device and S3 bucket, initiate the transfer of your data, and validate that your transfer is complete.
Go to the AWS Data Transfer Terminal console, then choose Get started.
Choose Create Transfer Team and make a team by adding the team’s name and description with agreement of service terms and conditions. You can add your team members for personal or group reservation in the team setting.
To reserve your time and location, choose Create Reservation.
In the first step, choose your team, a process owner to manage your reservation, and team members to visit the location for the data transferring job. Now, you can choose a location of Data Transfer Terminal facility and set your preferred visiting time. You’ll pay for the space reservation at an hourly rate for your reserved time.
To secure your reservation, choose Next and Create after reviewing the reservation details.
After your reservation is requested, you can find your upcoming reservations in the team page. You can check the reservation status or cancel your reservation.
On your reserved date and time, visit the location and confirm access with the building reception. You’re escorted by building staff to the floor and your reserved room of the Data Transfer Terminal location.
Don’t be surprised if there are no AWS signs in the building or room. This is for security reasons to keep your work location as secret as possible.
Visiting a pilot Terminal
Instead of me visiting a Data Transfer Terminal location where I live in Seoul, Jeff Barr visited a pilot location near him in Seattle to test uploading data as my team member.
The room is equipped with a patch panel, fiber optic cable, and a personal computer. The patch panel is installed inside a wall mount rack or small floor rack to allow additional space on the desk table. With the personal computer, you can see how to remote access to the server during data transfer process.
Here is Jeff’s feedback about visiting and working at the pilot facility.
When I arrived at the building, I was kindly escorted in and able to work easily using the instructions provided at the time of reservation. This location provides me with direct access to AWS global network infrastructure in a secure and on-demand format. I am excited to see how customers use AWS Data Transfer Terminal to more quickly get data into the cloud where they can more rapidly innovate and build on AWS.
Thanks, Jeff, for visiting the facility and doing the uploading job in my place!
Now available
AWS Data Transfer Terminal is now available today in Los Angeles and New York, with plans to add more locations globally.
You’ll be charged for on-demand use per hour for each location. There will be no per GB charge for the data transfer if you upload data into AWS Regions in the same continent of your location. To learn more, visit the Data Transfer Terminal pricing page.
Give AWS Data Transfer Terminal a try in the AWS Management Console. To learn more, refer to the Data Transfer Terminal page and send feedback through your usual AWS Support contacts.
— Channy
Post Syndicated from Donnie Prakoso original https://aws.amazon.com/blogs/aws/enhance-your-productivity-with-new-extensions-and-integrations-in-amazon-q-business/
Today, we’re announcing a new capability from Amazon Q Business to seamlessly access your assistant within popular web browsers and productivity tools. This helps you save time and complete your work and tasks more efficiently without having to leave your preferred applications.
Now, you can use Amazon Q Business directly from your web browser and other supported messaging and collaboration applications. You can quickly gather insights, review information, and ask questions. For example, you can effortlessly analyze and summarize content, get explanations on complex topics, or create meeting summaries without switching between applications.
Let’s get started
Let me walk you through how to get started with the new browser extensions and integrations. First, let’s look at the browser extensions. The following screenshot shows how it looks.
As an administrator, I need to enable the browser extensions for users of my Amazon Q Business application. To do that, I navigate to my Amazon Q Business application dashboard and select Integrations under the Enhancements section in the left navigation pane.
Then, on the Integrations page, select Edit in the Browser extensions section.
I select the available options in the Browsers section and choose Save. After I’ve enabled these options, my users will receive notification emails prompting them to install the extension.
Now, I’m switching to a user perspective of the Amazon Q Business application. I’ve received an email with a link to the Amazon Q Business web application. I visit the link and sign in to the Amazon Q Business web application. Here, I see a banner with information and a link to install the extension for my browser. I select the Install extension button.
Then, I navigate to the Chrome Web Store and install the browser extension.
After I have installed the browser extension, I sign in to my Amazon Q Business application using the same URL and credentials I use to access the web application.
Now, I can chat with Amazon Q Business apps whenever I visit any webpage. For example, I can ask it to summarize the current website for me.
The following image shows the result.
Application integration with Amazon Q Business
With Amazon Q Business, you can get AI-powered assistance and information not only when browsing, but also when collaborating with your teams. Now, you can integrate Amazon Q Business with supported third-party applications, making it an always-ready productivity and creativity teammate in your conversations.
To add third-party applications to Amazon Q Business, I need to navigate to the Integrations page and choose Add integration.
Here, I find all available integrations that I can use. For this demo, I select Slack.
I fill in all the required details, including the Slack workspace team ID, which you can obtain by following the steps outlined on the Slack documentation page.
After the integration is successfully created, I need to deploy this integration as a Slack bot. From the Integrations page, I select the integration and complete the integration process in the Slack platform. With all the required steps completed, now I can now add the app into my Slack workspace.
Here’s a quick video showing how I use this integration to interact with Amazon Q Business on Slack.
As someone who juggles multiple tools and platforms daily, this new capability unlocks various possibilities for me to improve my productivity. The ability to access AI assistance and perform cross-application tasks without leaving my current workspace helps me save time and maintain focus.
Additional things to know
Get started today and experience an exciting opportunity to enhance your productivity and streamline cross-application workflows. Learn more on the Amazon Q Business page.
Happy building,
— Donnie
Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/announcing-amazon-fsx-intelligent-tiering-a-new-storage-class-for-fsx-for-openzfs/
When I speak to customers who are planning to migrate massive amounts of on-premises data to AWS, they tell me that they want to simplify their storage management, reduce their costs, and to make the data more accessible so that it can be used for analytics, machine learning training, genomics, and other use cases. Customers are already using Network Attached Storage (NAS) on-premises, and are looking for a cloud-based upgrade that offers similar capabilities including point-in-time snapshots, data clones, and user management.
AWS customers such as Amdocs, Vela Games, and Astera Labs have been running their mission-critical and performance-intensive NAS workloads like databases, game development and streaming, and semiconductor chip design on Amazon FSx for OpenZFS. They’ve been using the existing SSD storage class on FSx to provide the predictable, high performance these workloads need. However, many other customers have large data sets that are stored on HDD-based or hybrid SSD/HDD-based NAS storage on prem that find it cost-prohibitive to move their data sets to all-SSD storage. Additionally, these customers are finding it increasingly challenging and expensive to manage provisioned storage on prem for unpredictable data sets and avoid running out of space. And they are keeping their NAS data around for longer because it could have future value for building their next model, investment strategy, or product, but that means they need to spend more time and effort monitoring access patterns and moving data around between hot and cold storage media to optimize costs.
FSx Intelligent-Tiering
Taking all of this into account, I am happy to be able to tell you about the new Amazon FSx Intelligent-Tiering storage class, available today for use with Amazon FSx for OpenZFS file systems. The new storage class is priced 85% lower than the existing SSD storage class and 20% lower than traditional HDD-based deployments on premises, and brings full elasticity and intelligent tiering to NAS data sets.
Your data moves between three storage tiers (Frequent Access, Infrequent Access, and Archive) with no effort on your part, so you get automatic cost savings with no upfront costs or commitments. Here’s how the tiers work:
Frequent Access – Data that has been accessed within the last 30 days is stored in this tier.
Infrequent Access – Data that has been not been accessed for 30 to 90 days is stored in tier, at a 44% cost reduction from Frequent Access.
Archive – Data that has not been accessed for 90 or more days is stored in this tier, at a 65% cost reduction from Infrequent Access.
Regardless of the storage tier, your data is stored across multiple AWS Availability Zones (AZs) for redundancy and availability, and can be retrieved instantly in milliseconds.
There’s no need to manage or pre-provision storage, making this storage class a great fit for uses case such as genomics, financial data analytics, seismic imagery analysis, and machine learning where storage requirements can change dramatically over the course of days or weeks.
Along with the potential for cost savings, you get high performance: up to 400K IOPS and 20 GB/second of throughput for each OpenZFS file system, with a time-to-first-byte of tens of milliseconds for all data, regardless of storage class. You can also configure an SSD-based read cache (64 GiB to 512 TiB) to reduce the time-to-first-byte by 10x to 100x for cached data.
Creating a File System
I can create a file system using the AWS Management Console, CLI, API, or a AWS CloudFormation. From the Console I click Create file system to get started:
I choose Amazon FSx for OpenZFS and click Next:
Then I enter a name (jeff_fsx_openzfs_1) for my file system and select the Intelligent-Tiering storage class. I choose the desired Throughput capacity, and I select one of the three sizing mode options for the read cache, click Next, and confirm my choices in order to create my file system:
It is ready within minutes, and I can NFS mount it to my EC2 instance:
After I run a representative workload for a while I can look at the metrics and review the performance of my file system:
It appears that I have plenty of throughput, but my read cache may be larger than needed. I created it in Automatically Provisioned mode, which allocated 3200 GiB of cache. I can change that (and save some money) with a couple of clicks:
I can also change the throughput capacity as needed:
Amazon FSx NAS Features and Attributes
Let’s take a quick look at some of the features which make FSx for OpenZFS and the FSx Intelligent-Tiering storage class a great for for your NAS-level storage needs:
Built-in Backups – Amazon FSx automatically makes a daily backup of each file system during a specified backup window and retains them for a specified retention period. The backups are file-system consistent, highly durable, and incremental. You can also create backups on your own and retain them for as long as needed.
Point-In-Time Snapshots -You can create a read-only image of an OpenZFS volume at any time. The snapshots are stored within the file system and consume storage; they can be used to restore a volume, restore individual files and folders, or to create a new volume as either a clone or a full-copy.
Replication – You can replicate a point-in-time view of an OpenZFS volume to another volume across file systems, AWS Regions, and AWS accounts. FSx uses ZFS send/receive technology behind the scenes to perform this replication and automatically establishes and maintains network connectivity between file systems to handle interruptions and resume data transfer as needed.
Data Compression – You can enable ZSTD or LZ4 compression on your OpenZFS volumes to reduce storage cost and speed up data transfer.
User and Volume Quotas – You can limit the amount of storage consumed by an individual volume or user.
Things to Know
Here are a couple of things to keep in mind before we wrap up:
Regions – This new storage class is available in the US East (Ohio, N. Virginia), US West (Oregon), Asia Pacific (Mumbai, Singapore, Sydney, Tokyo), Canada (Central), and Europe (Frankfurt, Ireland) AWS Regions.
Pricing – Pricing is based on the amount of primary storage consumed (GB/Month) and read cache provisioned (GB/Month). See the Amazon FSx for OpenZFS Pricing page for more information.
— Jeff;
Post Syndicated from Danilo Poccia original https://aws.amazon.com/blogs/aws/new-rag-evaluation-and-llm-as-a-judge-capabilities-in-amazon-bedrock/
Today, we’re announcing two new evaluation capabilities in Amazon Bedrock that can help you streamline testing and improve generative AI applications:
Amazon Bedrock Knowledge Bases now supports RAG evaluation (preview) – You can now run an automatic knowledge base evaluation to assess and optimize Retrieval Augmented Generation (RAG) applications using Amazon Bedrock Knowledge Bases. The evaluation process uses a large language model (LLM) to compute the metrics for the evaluation. With RAG evaluations, you can compare different configurations and tune your settings to get the results you need for your use case.
Amazon Bedrock Model Evaluation now includes LLM-as-a-judge (preview) – You can now perform tests and evaluate other models with humanlike quality at a fraction of the cost and time of running human evaluations.
These new capabilities make it easier to go into production by providing fast, automated evaluation of AI-powered applications, shortening feedback loops and speeding up improvements. These evaluations assess multiple quality dimensions including correctness, helpfulness, and responsible AI criteria such as answer refusal and harmfulness.
To make it easy and intuitive, the evaluation results provide natural language explanations for each score in the output and on console, and the scores are normalized from 0 to 1 for ease of interpretability. Rubrics are published in full with the judge prompts in the documentation so non-scientists can understand how scores are derived.
Let’s see how they work in practice.
Using RAG evaluations in Amazon Bedrock Knowledge Bases
In the Amazon Bedrock console, I choose Evaluations in the Inference and Assessment section. There, I see the new Knowledge Bases tab.
I choose Create, enter a name and a description for the evaluation, and select the Evaluator model that will compute the metrics. In this case, I use Anthropic’s Claude 3.5 Sonnet.
I select the knowledge base to evaluate. I previously created a knowledge base containing only the AWS Lambda Developer Guide PDF file. In this way, for the evaluation, I can ask questions about the AWS Lambda service.
I can evaluate either the retrieval function alone or the complete retrieve-and-generate workflow. This choice affects the metrics that are available in the next step. I choose to evaluate both retrieval and response generation and select the model to use. In this case, I use Anthropic’s Claude 3 Haiku. I can also use Amazon Bedrock Guardrails and adjust runtime inference settings by choosing the configurations link after the response generator model.
Now, I can choose which metrics to evaluate. I select Helpfulness and Correctness in the Quality section and Harmfulness in the Responsible AI metrics section.
Now, I select the dataset that will be used for evaluation. This is the JSONL file I prepared and uploaded to Amazon Simple Storage Service (Amazon S3) for this evaluation. Each line provides a conversation, and for each message there is a reference response.
I specify the S3 location in which to store the results of the evaluation. The evaluation job requires that the S3 bucket is configured with the cross-origin resource sharing (CORS) permissions described in the Amazon Bedrock User Guide.
For service access, I need to create or provide an AWS Identity and Access Management (IAM) service role that Amazon Bedrock can assume and that allows access to the Amazon Bedrock and Amazon S3 resources used by the evaluation.
After a few minutes, the evaluation has completed, and I browse the results. The actual duration of an evaluation depends on the size of the prompt dataset and on the generator and the evaluator models used.
At the top, the Metric summary evaluates the overall performance using the average score across all conversations.
After that, the Generation metrics breakdown gives me details about each of the selected evaluation metrics. My evaluation dataset was small (two lines), so there isn’t a large distribution to look at.
From here, I can also see example conversations and how they were rated. To view all conversations, I can visit the full output in the S3 bucket.
I’m curious why Helpfulness is slightly below one. I expand and zoom Example conversations for Helpfulness. There, I see the generated output, the ground truth that I provided with the evaluation dataset, and the score. I choose the score to see the model reasoning. According to the model, it would have helped to have more in-depth information. Models really are strict judges.
Comparing RAG evaluations
The result of a knowledge base evaluation can be difficult to interpret by itself. For this reason, the console allows comparing results from multiple evaluations to understand the differences. In this way, you can understand if you’re improving or not for the metrics you care about.
For example, I previously ran two other knowledge base evaluations. They’re related to knowledge bases with the same data sources but different chunking and parsing configurations and different embedding models.
I select the two evaluations and choose Compare. To be comparable in the console, the evaluations need to cover the same metrics.
In the At a glance tab, I see a visual comparison of the metrics using a spider chart. In this case, the results are not much different. The main difference is the Faithfulness score.
In the Evaluation details tab, I find a detailed comparison of the results for each metric, including the difference in scores.
Using LLM-as-a-judge in Amazon Bedrock Model Evaluation (preview)
In the Amazon Bedrock console, I choose Evaluations in the Inference and Assessment section of the navigation pane. After I choose Create, I select the new Automatic: Model as a judge option.
I enter a name and a description for the evaluation and select the Evaluator model that is used to generate evaluation metrics. I use Anthropic’s Claude 3.5 Sonnet.
Then, I select the Generator model, which is the model I want to evaluate. Model evaluation can help me understand if a smaller and more cost-effective model meets the needs of my use case. I use Anthropic’s Claude 3 Haiku.
In the next section I select the Metrics to evaluate. I select Helpfulness and Correctness in the Quality section and Harmfulness in the Responsible AI metrics section.
In the Datasets section I specify the Amazon S3 location where my evaluation dataset is stored and the folder in an S3 bucket where the results of the model evaluation job are stored.
For the evaluation dataset, I prepared another JSONL file. Each line provides a prompt and a reference answer. Note that the format is different compared to knowledge base evaluations.
Finally, I can choose an IAM service role that gives Amazon Bedrock access to the resources used by this evaluation job.
I complete the creation of the evaluation. After a few minutes, the evaluation is complete. Similar to the knowledge base evaluation, the result starts with a Metrics Summary.
The Generation metrics breakdown details each metric, and I can look at details for a few sample prompts. I look at Helpfulness to better understand the evaluation score.
The prompts in the evaluation have been correctly processed by the model, and I can apply the results for my use case. If my application needs to manage prompts similar to the ones used in this evaluation, the evaluated model is a good choice.
Things to know
These new evaluation capabilities are available in preview in the following AWS Regions:
Note that the available evaluator models depend on the Region.
Pricing is based on the standard Amazon Bedrock pricing for model inference. There are no additional charges for evaluation jobs themselves. The evaluator models and models being evaluated are billed according to their normal on-demand or provisioned pricing. The judge prompt templates are part of the input tokens, and those judge prompts can be found in the AWS documentation for transparency.
The evaluation service is optimized for English language content at launch, though the underlying models can work with content in other languages they support.
To get started, visit the Amazon Bedrock console. To learn more, you can access the Amazon Bedrock documentation and send feedback to AWS re:Post for Amazon Bedrock. You can find deep-dive technical content and discover how our Builder communities are using Amazon Bedrock at community.aws. Let us know what you build with these new capabilities!
— Danilo
Post Syndicated from Elizabeth Fuentes original https://aws.amazon.com/blogs/aws/newly-enhanced-amazon-connect-adds-generative-ai-whatsapp-business-and-secure-data-collection/
Today, Amazon Connect introduces a set of new features that help businesses enhance their contact center operations through generative AI, advanced security features, and streamlined bot management. These innovations help businesses deliver better customer experiences by creating more time and space for meaningful human interactions, while maintaining security and compliance.
Contact center managers continually face challenges in optimizing self-service resolution rates, evaluating agent performance efficiently, and maintaining data privacy compliance. Additionally, creating and managing conversational AI experiences often requires specialized expertise and complex integrations across multiple services.
To address these challenges, Amazon Connect introduced key features such as generative AI–powered customer segmentation for targeted campaigns, native WhatsApp Business messaging for omnichannel support, secure collection of sensitive customer data in chat interactions, simplified conversational AI bot management in the Amazon Connect interface, and new enhancements to Amazon Q in Connect. Amazon Connect also added new analytics capabilities through Amazon Connect Contact Lens to help optimize bot performance and contact center operations.
Here are the new capabilities that will help you create more personalized and efficient customer experiences while maintaining the highest standards of data security and operational excellence.
Generative AI powered features
Amazon Connect integrates new generative AI capabilities to automate and enhance customer interactions, enabling smarter targeting and more efficient contact center management.
Generative AI segmentation and trigger-based campaigns – Uses generative AI–powered assistance to create customer segments using conversational prompts. This allows businesses to create precise customer segments using natural language descriptions, making it easier to identify and reach specific customer groups. Trigger campaigns enable organizations to communicate with their customers based on specific customer events, such as cart abandonment.
You can also start with ready-to-use suggestions.
Simplify conversational AI bot creation and enhance them with Amazon Q in Connect – Create, edit, and manage conversational AI bots powered by Amazon Lex directly within the Amazon Connect web interface. You can now enhance these bots with Amazon Q in Connect, a generative AI–powered assistant for customer service. Amazon Q in Connect now supports end-customer self-service interactions across interactive voice response (IVR) and digital channels, in addition to assisting contact center agents with recommended responses and actions.
This integration extends beyond traditional voice and chatbot Amazon Lex capabilities by providing advanced conversational abilities via large language models (LLMs). The system intelligently searches configured knowledge bases, customer information, web content, and third-party application data to respond to customer questions when they don’t match predefined intents. Administrators can set custom guardrails for their instance, defining restrictions on response generation and monitoring Amazon Q in Connect performance.
Generative AI–powered automated evaluations: Supervisors can automatically evaluate up to 100 percent of contacts using generative AI.
Generative AI–powered contact categorization: Improves existing semantic match functionality using natural language intents.
Improved interfaces and tools
Enhanced capabilities for bot management and monitoring, simplifying the creation and optimization of automated experiences.
Amazon Connect for WhatsApp Business messaging – Natively integrate with WhatsApp Business messaging so customers can receive support over WhatsApp in addition to existing Amazon Connect channels such as voice, SMS, chat, and Apple Messages for Business. This addition to Amazon Connect omnichannel capabilities helps businesses meet customers on their preferred communication channel while maintaining consistent service delivery and management within the Amazon Connect application.
Contact Lens conversational AI bot dashboards – Offers analytics to monitor the performance of your conversational AI bots built in Amazon Connect.
Self-service voice (IVR) recording and interaction logs on contact details – Provides comprehensive records of self-service interactions, including audio recordings.
Improved intraday forecasts – Allows comparison of intraday forecasts against previously published forecasts.
Salesforce Contact Center with Amazon Connect (Preview) – Natively integrates the digital channels and unified routing of Amazon Connect into Salesforce customer relationship management (CRM) system. This new offering allows companies to use a single routing and workflow system for both Amazon Connect and Salesforce channels, intelligently directing calls, chats, and cases to the appropriate self-service or agent interaction. If you’re interested, sign up to join the preview.
Enhanced security for chat
New features that enhance security and compliance in chat interactions, enabling secure handling of sensitive information.
Collection of sensitive customer data within chats – Amazon Connect chat and messaging now includes a data privacy option that enables secure handling of sensitive customer information during chat interactions. This feature protects personally identifiable information (PII) and payment card industry (PCI) data, promoting compliance with data protection regulations.
Key benefits
The latest features of Amazon Connect combine generative AI, enhanced security, and streamlined bot management to help businesses:
Transform customer experience – Amazon Connect elevates customer interactions through AI–powered segmentation, enabling personalized engagement strategies. The new WhatsApp Business messaging expands omnichannel support capabilities, meeting customers on their preferred channel. Additionally, advanced bot capabilities, including Amazon Q in Connect, enhance self-service resolution rates, delivering more efficient customer experiences.
Enhance security and operations – Contact centers can now strengthen their security posture with PCI-compliant chat interactions while maintaining operational efficiency. Custom AI guardrails promote appropriate response generation, while the simplified bot management interface eliminates the need for specialized expertise. Analytics and forecasting capabilities provide comprehensive performance monitoring, enabling data-driven decision-making for optimal contact center operations.
Pricing and availability – These features are available today in all AWS Regions where Amazon Connect is supported. For pricing, visit the Amazon Connect Pricing. For implementation guidance, visit the Amazon Connect documentation.
— Eli
Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/securely-share-aws-resources-across-vpc-and-account-boundaries-with-privatelink-vpc-lattice-eventbridge-and-step-functions/
At some point, every AWS customer tells me that they have the desire to move into the future as quickly as possible. They want to simplify their modernization efforts, drive growth, and adapt to the cloud, while also reducing costs as they proceed. These customers typically have a large suite of legacy applications, possibly running on-premises, that are running on diverse technology stacks managed by disparate parts of the organization. To make things even more challenging, these organizations often have to meet stringent security and compliance requirements.
Prepare to Share
You can now share AWS resources such as Amazon Elastic Compute Cloud (Amazon EC2) instances, Amazon Elastic Container Service (Amazon ECS) and Amazon Elastic Kubernetes Service (Amazon EKS) container services, and your own HTTPS services across Amazon Virtual Private Cloud (Amazon VPC) and AWS account boundaries and use them to build event-driven apps via Amazon EventBridge and orchestrate workflows with AWS Step Functions. You can update your existing workloads, connect your modern cloud-native apps to on-premises legacy systems, with all communication routed across private endpoints and networks.
These new features build on Amazon VPC Lattice and AWS PrivateLink, and give you a lot of new options to design and control your network, along with some cool new ways to integrate and orchestrate across all of your technology stacks. For example, you can build hybrid event-driven architectures that make use of your existing on-premises applications.
Today, some customers use AWS Lambda functions or Amazon Simple Queue Service (Amazon SQS) queues to transfer data into VPCs. This undifferentiated heavy lifting can now be replaced with a simpler and more efficient solution.
Bringing all of this together, you get a set of services that will help you to accelerate your modernization efforts and simplify integration between your applications, regardless of where they are situated. EventBridge and Step Functions work hand-in-hand with PrivateLink and VPC Lattice to enable integration of public and private HTTPS-based applications into your event-driven architectures and workflows.
Here are the essential terms and concepts:
Resource Owner VPC – A VPC that has resources to be shared. The owner of this VPC creates a Resource Gateway with one or more associated Resource Configurations, then uses AWS Resource Access Manager (RAM) to share the Resource Configuration with the Resource Consumer, such as another AWS account, or a developer building event-driven architectures and workflows using EventBridge and Step Functions. Let’s define the Resource Owner as the person (maybe you) in your organization who is responsible for the care and feeding of this VPC.
Resource Gateway – Provides a point of ingress to a VPC so that clients can access resources in the Resource Owner VPC, as indicated by the Resource Configurations that are associated with the gateway. One Resource Gateway can make multiple resources available.
Resource – This can be a HTTPS endpoint, a database, a database cluster, an EC2 instance, an Application Load Balancer in front of multiple EC2 instances, an ECS service discoverable via AWS Cloud Map, an Amazon Elastic Kubernetes Service (Amazon EKS) service behind a Network Load Balancer, or a legacy service running in the Resource Owner VPC or running in on-premises across AWS Site-to-Site VPN or AWS Direct Connect.
Resource Configuration – Defines a set of resources that can be accessed through a particular Resource Gateway. The resources can be referenced by IP address, DNS name, or (for AWS resources) an ARN.
Resource Consumer – The person in your organization who is responsible for building applications that connect with and consume services provided by resources in a Resource Owner VPC.
Sharing Resources
You can put all of this power to use in a lot of different ways; I’ll focus on one for this post.
First, I will play the role of the Resource Owner. I click Resource gateways in the VPC Console, see that I don’t have a gateway, and click Create resource gateway to get started:
I assign a name (main-rg) and an IP address type, then pick the VPC and the private subnets where the gateway will have a presence (this is a one-shot selection that cannot be changed without creating a new Resource Gateway). I also choose up to five security groups to control inbound traffic:
I scroll down, assign any desired tags, and click Create resource gateway to proceed:
My new gateway is active within seconds; I nod in appreciation and click Create resource configuration to move ahead:
Now I need to create my first Resource Configuration. Let’s say that I have a HTTPS service running on an EC2 instance on a private subnet in my Resource Owner VPC. I assign a DNS name to the service and use a Amazon Route 53 Alias record which returns the IP address of the instance:
I am using a public hosted zone in this example. We already working on support for private hosted zones.
With DNS all set up, I click Create resource configuration to move ahead. I enter a name (rc-service1), choose Resource as the type, and select the Resource Gateway that I created earlier:
I scroll down and define my EC2 instance as a resource, entering the DNS name and setting up sharing for ports 80 and 443:
Now I take a small detour, and hop over to the RAM Console to create a Resource Share so that other AWS accounts can access the resources (this is optional, and only relevant for cross-account scenarios). I could create one Resource Share for each service, but in most cases I would create one share and use it to package up a collection of related services. I’ll do that, and call it shared-services:
Returning from my detour, I refresh the list of resource shares, pick the one that I created, and click Create resource configuration:
The resource configuration is ready within seconds.
Recap and Planning Time
Before moving ahead, let’s do a quick recap and make some plans. Here’s what I (in the role of Resource Provider) have so far:
Ok, so what’s next?
Share – This is the first and most obvious use use. I can use AWS Resource Access Manager (RAM) to share the Resource Configuration with another AWS account and access the service from another VPC. On the other side (as the Resource Consumer), I take a couple of quick steps to connect to the service that has been shared with me:
Modernize – I can remove my legacy Lambda or SQS integration to get rid of some undifferentiated heavy lifting.
Build – I can use EventBridge and Step Functions to build event-driven architectures and orchestrate applications. I’ll take this option!
Accessing Private Resources with EventBridge and Step Functions
EventBridge and Step Functions already make it easy access to public HTTPS endpoints such as those from SaaS providers like Slack, Salesforce, and Adobe. With today’s launch, consuming private HTTPS services is just as easy.
As a Resource Consumer, I simply create an EventBridge connection, reference a Resource Configuration that was shared with me, and call the service from my event-driven application. Everything that I already know still applies, and I now have the new-found power to access private services.
To create the EventBridge connection, I open the EventBridge console and click Connections in the Integration menu:
I review my existing connections (none so far), then click Create connection to move ahead:
I enter a name (MyService1) and a description for my connection, select Private as the API type, and choose the Resource Configuration that I created earlier:
Scrolling down, I need to configure the authorization for the service that I am connecting to. I select Custom configuration and Basic authorization, and enter the Username and Password for my service. I also add Action=Forecast to the query string (as you can see there are a lot of options for authorization), and click Create:
The connection is created and ready within minutes. Then I use it in my Step Functions workflows by using the HTTP Task, selecting the connection, entering the URL of my API endpoint, and choosing an HTTP method:
And that’s all there is to it: your Step Functions workflows can now make use of Private Resources!
I can also use this connection as an EventBridge API destination target in Event Buses and Pipes.
Things to Know
Here a couple of things to know about these cool new features:
Pricing – Existing pricing for Step Functions, EventBridge, PrivateLink, and VPC Lattice apply including the per-GB charge for data transfer into the VPC.
Regions – You can create and use Resource Gateways and Resource Configurations in 21 AWS Regions: US East (Ohio, N. Virginia), US West (N. California, Oregon), Africa (Cape Town), Asia Pacific (Hong Kong, Mumbai, Osaka, Seoul, Singapore, Sydney, Tokyo), Canada (Central), Europe (Frankfurt, Ireland, London, Milan, Paris, Stockholm), Middle East (Bahrain), and South America (São Paulo).
In the Works – As I noted earlier, we are already working on support for private hosted zones. We are also planning to support access to other types of AWS resources through EventBridge and Step Functions .
— Jeff;
Post Syndicated from Betty Zheng (郑予彬) original https://aws.amazon.com/blogs/aws/new-aws-security-incident-response-helps-organizations-respond-to-and-recover-from-security-events/
Today, we announce AWS Security Incident Response, a new service designed to help organizations manage security events quickly and effectively. The service is purpose-built to help customers prepare for, respond to, and recover from various security events, including account takeovers, data breaches, and ransomware attacks.
Security Incident Response automates the triage and investigation of security findings from Amazon GuardDuty and integrated third party threat detection tools through AWS Security Hub. It facilitates communication and coordination and provides 24/7 access to security experts from the AWS Customer Incident Response Team (CIRT) who can assist during security events. The service aims to provide customers with more comprehensive support across the phases of incident response lifecycle, from preparation to detection, analysis, and recovery.
Security events are becoming more pervasive and complex for customers. Security teams often face an overwhelming number of daily alerts, leading to potential misplaced priorities of resources and reduced effectiveness. Manual investigation of findings strains resources and may cause customers to overlook critical security alerts. Additionally, coordinating responses across multiple stakeholders, managing permissions in various environments, and documenting actions complicate the process. There is an opportunity to better support customers and remove various points of undifferentiated heavy lifting that customers face during security events.
Key capabilities
AWS Security Incident Response addresses these challenges through three main core capabilities that help customers effectively prepare for, respond to, and recover from security events :
In addition to the core capabilities, customers benefit from a service dashboard with metrics that help them measure, monitor, and improve their security incident response performance over time. These metrics include mean time to resolution (MTTR), number of active and closed cases within a specific period, number of triaged findings, and other key performance indicators. Customers can access these metrics instantly without needing to collate information or create one-time reports.
How to get started
The onboarding process can be completed in a few steps. Security Incident Response integrates with AWS Organizations to provide comprehensive security coverage for your current and future accounts with an added layer of security. Customers begin by selecting a central account within their organization, where all active and historical security events can be created and managed.
Next, customers can enable the proactive incident response feature, which creates service-level permissions allowing Security Incident Response to monitor and investigate findings from GuardDuty or third-party detection tools through Security Hub. These findings are then automatically sorted and remediated using service automation and customer-specific data, including common IP addresses, AWS Identity and Access Management (IAM) principals, and other relevant attributes. For findings that can’t be automatically remediated, Security Incident Response creates a security case and notifies the appropriate stakeholders within the customer’s organization.
Customers can also configure permissions for the service to execute containment actions by deploying specific IAM roles. By using these Security Incident Response containment capabilities, customers can achieve faster incident response times and potentially minimize the impact of security events on accounts and resources.
Availability and getting started
AWS Security Incident Response is now available in 12 AWS Regions globally: US East (N. Virginia, Ohio), US West (Oregon), Asia Pacific (Seoul, Singapore, Sydney, Tokyo), Canada (Central), and Europe (Frankfurt, Ireland, London, Stockholm).
Learn more about AWS Security Incident Response by visiting the product page.
Post Syndicated from The History Guy: History Deserves to Be Remembered original https://www.youtube.com/watch?v=CV0LHEWH7JU
Post Syndicated from xkcd.com original https://xkcd.com/3019/
Post Syndicated from Explosm.net original https://explosm.net/comics/crazy-jim
New Cyanide and Happiness Comic
Post Syndicated from corbet original https://lwn.net/Articles/1000379/
Linus has released 6.13-rc1 and closed the
merge window for this release. “And for once – possibly the first time
“
ever – it looks like the release cycle doesn’t clash horribly up with
the holiday season, and we’ll have time both to stabilize this release,
_and_ the work for 6.14 won’t be starting until well into January.
Post Syndicated from Rachel Zheng original https://aws.amazon.com/blogs/compute/new-simplifying-the-use-of-third-party-block-storage-with-aws-outposts/
This post is written by Kate Sposato, Senior Solutions Architect, EC2 Edge Compute
AWS is excited to announce deeper collaboration with industry-leading storage solutions to streamline the use of third-party storage with AWS Outposts. You can now attach and use external block data volumes from NetApp® on-premises enterprise storage arrays and Pure Storage® FlashArray
directly from the AWS Management Console.
Outposts is a fully managed service that extends AWS infrastructure, AWS services, APIs, and tools to customer premises. By providing local access to AWS managed infrastructure, Outposts allows you to build and run applications on premises using the same application programming interfaces (APIs) as in AWS Regions. Moreover, this is done while using local compute and storage resources to meet lower latency and local data processing needs. Outposts is available in various rack and server form factors.
Many of you have block storage systems running in your on-premises environments that provide advanced data storage and management features—such as snapshots, replication, and encryption—to protect data integrity and security. There are various uses cases that would predicate you needing to access data through these external volumes backed by external storage systems from an application running in Amazon Elastic Compute Cloud (Amazon EC2) instances on Outposts. These include: regulatory auditing requirements, government and local regulation compliance, high data durability and resiliency requirements, low-latency data access, and migration of on-premises applications that are tightly coupled with existing external storage systems. To make it easier for you to use external volumes with Outposts, AWS has validated a broad range of third-party storage solutions through the AWS Outposts Ready Program. With this program, you can easily identify storage solutions that are tested to run with Outposts.
Today, we are taking our integration with storage solutions from NetApp and Pure Storage to the next level. Outposts now has a simplified and automated way to launch EC2 instances with attached block storage from external infrastructure through the AWS Management Console. The new integration includes automated user script generation and attachment of data volumes to EC2 instances running on 42U Outposts racks and 2U Outposts servers. This integration reduces the friction associated with using the advanced data management and security features of external storage infrastructure in combination with Outposts, allowing you to create a resilient, compliant, and optimized storage and compute infrastructure.
Outposts racks support Amazon Elastic Block Store (Amazon EBS) volumes for EC2 instances, which provide persistent local block storage.
EC2 instances running on Outposts racks can access data stored on external block storage arrays over the Outposts local gateway (LGW). An LGW enables connectivity between the Outpost subnets, where EC2 instances run, and the on-premises network. It carries storage traffic between the EC2 instances running on the Outposts rack and the local network. The LGW is created by AWS as part of the Outposts rack installation process. Each Outposts rack supports a single LGW.
The following diagram shows an EC2 instance running on an Outposts rack with an elastic network interface (ENI) and LGW configured for instance connectivity. An external storage array communicates with the EC2 instance running on the Outposts rack through the Outpost network devices (ONDs). Customer Network Devices (CNDs) that connect to EC2 instances running on Outposts racks need to support the following:
Figure 1. Outposts rack and Amazon EC2 networking architecture
Outposts servers come with internal NVMe SSD-based high-performance instance storage. Similar to AWS Regions, instance storage is allocated directly to the EC2 instance and follows the lifecycle of the instance. For example, if an EC2 instance is terminated, then the instance storage associated with the instance is also deleted. If you want data to persist after the instance is terminated, you can use external storage solutions to complement the instance storage included with Outposts servers.
Outposts servers have a local network interface (LNI). This logical networking component connects the EC2 instances running on the Outposts servers subnet to the on-premises network and allows communication to other on-premises storage, compute, and networking appliances.
To support the Amazon EC2 on Outposts to external storage array integration, an LNI must be created then added to the EC2 instance during instance launch. An LNI can only be created through the AWS Command Line Interface (AWS CLI) or the AWS software development toolkit (SDK) using the following command. The subnet id is the Outposts server subnet and the device index should be unique to the subnet.
aws ec2 modify-subnet-attribute --subnet-id <subnet id> --enable-lni-at-device-index <device index>In the on-premises network, you must have a Network Interface Card (NIC) at the same device index that you specified when running the preceding CLI command.
Further detailed steps for this workflow are listed in the Outposts server user guide.
When the local network interfaces are enabled on an Outpost subnet, the EC2 instances in the Outpost subnet can be configured to include this LNI in addition to the ENI. The LNI connects to the on-premises network while the ENI connects to the VPC.
The following diagram shows an EC2 instance running on an Outposts server with both an ENI and LNI configured for instance connectivity. There is an external storage array connected to the Outposts server using a CND through NVMe-over-TCP or iSCSI protocol.
Figure 2. Outposts server and Amazon EC2 networking architecture
The rest of this post covers the steps for how to launch an EC2 instance running on an Outposts 2U server or Outposts rack with a connected external block storage volume for local data access from within the EC2 instance. The current release of this feature supports EC2 instances running Microsoft Windows Server 2022 and Red Hat Enterprise Linux 9 (RHEL9) based operating systems.
Support for Outposts and all Outposts integration features, including this one, needs an active AWS Enterprise Support Plan or AWS Enterprise On-Ramp Support Plan. Support for external storage arrays and configurations can be obtained from the respective storage vendor and may need an additional support plan depending on the vendor and the storage solution implemented.
This post assumes you’re familiar with the basic functionality of Outposts servers and Outposts rack. If you would like to learn more about the Outposts family in general, then the user guide, What is AWS Outposts?, is a great place to start.
The following sections outline the solution deployment.
The following deployment architecture shows the workflow attaching an external storage array to an Outpost, launching an EC2 instance through the AWS Management Console, and accessing the data on the external storage array from within the EC2 instance running on the Outpost.
Figure 3. Third-party block storage on Outposts architecture overview
1. (Prerequisite) If there is no block data volume already running and configured on the compatible storage array, this must be completed in the storage solution’s interface before moving to Step 2.
a. Create an NVMe device, subsystem, and namespace for the block data volume.
b. Optionally, generate a host NQN that is used for the EC2 instance connection, and add it to the allow list for the appropriate subsystems.
c. The following pieces of information are used in later steps:
i. Host NQN: Unique identifier of the EC2 instance for attachment;
ii. Target IP: Address of the connected block volume host;
iii. Target Port Number: Port number of the connected block volume host.
You can learn more about launching and configuring external storage arrays in the Outposts family documentation or in the respective storage array vendor documentation.
2. In the Console, navigate to EC2 Launch Instance Wizard by choosing EC2, Instances, Launch instances.
a. Name the instance and add any desired tags to be applied at launch.
b. Choose the desired, compatible RHEL9 based Amazon Machine Image (AMI) from the list, or choose one from the AWS Marketplace.
c. Choose the desired EC2 Instance type.
d. Expand the Network settings section and select Edit. Choose the VPC and subnet of the target Outpost.
i. Outposts servers only: You must create an LNI in the Advanced Network settings before launching the instance.
e. Expand Advanced network configuration and select Add network device. Continue to add network devices until the Device index is equal to the volume index.
Figure 4. Advanced network configuration
f. Expand Configure storage and select Edit next to External storage volumes settings section and choose NVMe/TCP in Storage network protocol.
Figure 5. External storage volumes configuration
g. Enter the HostNQN in the format provided for the NVMe/TCP data volume. Make sure that the HostNQN used has been added to the storage array subsystem allow list.
h. Select Add NVMe/TCP Discovery Controller and enter the IP address and port of the controller from the storage array. Enter 4420 as the Target Port, if the target port is unknown.
i. (Optional) You can add more data volumes that use a different target discovery controller at this time by choosing the Add NVMe/TCP Data Volume button under the Target IP address. Repeat Steps 2.h for each data volume to be attached to the EC2 instance.
j. Expand the Advanced details and provide any additional Amazon EC2 behavior settings as appropriate.
k. At the bottom of the Advanced details section is the automatically generated User data. If you need to manually edit this data, you can do so by selecting Edit at the bottom.
Figure 6. Automatically generated user data file
l. When the configurations are set, choose the Launch instance button in the right-side column.
3. The EC2 Launch Instance Wizard now launches an EC2 instance configured as described on the Outpost and attaches the desired external data volume(s) to the EC2 instance.
4. Applications and users can access the data on the attached external volumes from within the EC2 instance. To verify this:
a. From within the launched EC2 instance, run sudo nvme list
b. The volumes are displayed as /dev/nvme1n1 with the number increasing for each attached volume. Local instance store volumes on Outposts servers and EBS boot volumes on Outposts racks are listed first. External volumes are listed after those with sequentially increasing node numbers.
5. External storage volume and array management, configuration, and backups continue to be managed through the storage vendor-provided toolkit. You can find more information on external storage management in the respective storage array vendor documentation.
1. (Prerequisite) If there is no block data volume already running and configured on the compatible storage array, this must be completed in the storage solution’s interface before moving to Step 2.
a. Create an Initiator group (igroup) and add the Initiator IQN to the igroup. Then map the logical unit number (LUN) to the igroup.
b. Optionally, generate an initiator IQN that is used for the EC2 instance connection, and add it to the allow list for the appropriate subsystems.
c. The following pieces of information are used in later steps:
i. Initiator IQN: Unique identifier of the EC2 instance for attachment;
ii. Target IQNs: Unique identifier of the storage virtual machine (SVM);
iii. Target IP: Address of the connected block volume host;
iv. (Optional) Target Port Number: Port number of the connected block volume host.
You can learn more about launching and configuring external storage arrays in the Outposts family documentation or in the respective storage array vendor documentation.
2. In the Console, navigate to EC2 Launch Instance Wizard by choosing EC2, Instances, Launch instances.
a. Name the instance and add any desired tags to be applied at launch.
b. Choose the desired, compatible RHEL9 or Windows Server 2022 based AMI from the list, or purchase one from the AWS Marketplace.
c. Choose the desired EC2 Instance type.
d. Expand the Network settings section and choose the VPC and subnet of the target Outpost.
i. Outposts servers only: You must create an LNI in the Advanced Network settings before launching the instance.
e. Expand Advanced network configuration and select Add network device. Continue to add network devices until the Device index is equal to the volume index.
Figure 7. Advanced network configuration
f. Expand Configure storage and select Edit next to External storage volumes settings section and choose iSCSI in Storage network protocol.
Figure 8. External storage volumes configuration
g. Enter the Initiator IQN for the iSCSI data volume in the format provided. Make sure that the Initiator IQN used has been added to the allow list for the volume.
h. Select Add iSCSI Target and enter the Target IP, Target Port, and Target IQN of the storage array. Enter 4420 for the Target Port, if the target port is unknown.
i. (Optional) You can add additional data volumes with a different Target IQN at this time by selecting the Add iSCSI Target button under the Target IP address. Repeat Steps 2.h for each data volume to be attached to the EC2 instance.
j. Expand the Advanced details and provide any additional Amazon EC2 behavior settings as appropriate.
k. At the bottom of the Advanced details section is the automatically generated User data. If you need to manually edit this data, you can do so by selecting Edit at the bottom.
Figure 9. Automatically generated user data file
l. When the configurations are set, choose the Launch instance button in the right-side column.
3. The EC2 Launch Instance Wizard now launches an EC2 instance configured as described on the Outpost and attaches the desired external data volume(s) to the EC2 instance.
4. Applications and users can access the data on the attached external volumes from within the EC2 instance. To verify this:
a. From within the launched EC2 instance, run iscsiadm -m session -P3
b. The volumes are displayed as /dev/sd0 with the number increasing for each attached volume.
5. External storage volume and array management, configuration, and backups continue to be managed through the storage vendor-provided toolkit. You can find more information on external storage management in the respective storage array vendor documentation.
This integration offers a streamlined workflow to attach and utilize external block data volumes on Outposts directly through the AWS Management Console, eliminating manual processes. It provides the full benefits of advanced data infrastructure from trusted storage providers in conjunction with the security, reliability, and scalability of AWS managed infrastructure. This helps you accelerate cloud migration with dependencies on third-party storage and realize the full potential of your on-premises data.
To learn more about this integration, visit the NetApp on-premises enterprise storage arrays for AWS Outposts solution page and the Pure Storage FlashArray for AWS Outposts blog post. To discuss your external storage needs with an Outposts expert, submit this form. If you are attending AWS re:Invent 2024, make sure to check out the NetApp booth (booth #1748) and Pure Storage booth (booth #454) to connect with our partner specialists.
Post Syndicated from Veliswa Boya original https://aws.amazon.com/blogs/aws/new-apis-in-amazon-bedrock-to-enhance-rag-applications-now-available/
Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. Amazon Bedrock Knowledge Bases is a fully managed service that empowers developers to create highly accurate, low latency, secure, and customizable generative AI applications cost effectively. Amazon Bedrock Knowledge Bases connects foundation models (FMs) to a company’s internal data using Retrieval Augmented Generation (RAG). RAG helps FMs deliver more relevant, accurate, and customized responses.
In this post, we detail two announcements related to Amazon Bedrock Knowledge Bases:
Support for custom connectors and ingestion of streaming data
Today, we announced support for custom connectors and ingestion of streaming data in Amazon Bedrock Knowledge Bases. Developers can now efficiently and cost-effectively ingest, update, or delete data directly using a single API call, without the need to perform a full sync with the data source periodically or after every change. Customers are increasingly developing RAG-based generative AI applications for various use cases such as chatbots and enterprise search. However, they face challenges in keeping the data up-to-date in their knowledge bases so that the end users of the applications always have access to the latest information. The current process of data synchronization is time-consuming, requiring a full sync every time new data is added or removed. Customers also face challenges in integrating data from unsupported sources, such as Google Drive or Quip, into their knowledge base. Typically, to make this data available in Amazon Bedrock Knowledge Bases, they must first move it to a supported source, such as Amazon Simple Storage Service (Amazon S3), and then start the ingestion process. This extra step not only creates additional overhead but also introduces delays in making the data accessible for querying. Additionally, customers who want to use streaming data (for example, news feeds or Internet of Things (IoT) sensor data) face delays in real-time data availability due to the need to store the data in a supported data source before ingestion. As customers scale up their data, these inefficiencies and delays can become significant operational bottlenecks and increase costs. Keeping all these challenges in mind, it’s important to have a more efficient and cost-effective way to ingest and manage data from various sources to ensure that the knowledge base is up-to-date and available for querying in real-time. With support for custom connector and ingestion of streaming data, customers can now use direct APIs to efficiently add, check the status of, and delete data, without the need to list and sync the entire dataset.
How it works
Custom connectors and ingestion of streaming data can be accessed using the Amazon Bedrock console or the AWS SDK.
PUT /knowledgebases/KB12345678/datasources/DS12345678/documents HTTP/1.1
Content-type: application/json
{
"documents": [{
"content": {
"dataSourceType": "CUSTOM",
"custom": {
"customDocumentIdentifier": {
"id": "MyDocument"
},
"inlineContent": {
"textContent": {
"data": "Hello world!"
},
"type": "TEXT"
},
"sourceType": "IN_LINE"
}
}
}]
}
POST /knowledgebases/KB12345678/datasources/DS12345678/documents/deleteDocuments/ HTTP/1.1
Content-type: application/json
{
"documentIdentifiers": [{
"custom": {
"id": "MyDocument"
},
"dataSourceType": "CUSTOM"
}]
}POST /knowledgebases/KB12345678/datasources/DS12345678/documents/ HTTP/1.1
Content-type: application/json
{
"maxResults": 10
}POST /knowledgebases/KB12345678/datasources/DS12345678/documents/getDocuments/ HTTP/1.1
Content-type: application/json
{
"documentIdentifiers": [{
"custom": {
"id": "MyDocument"
},
"dataSourceType": "CUSTOM"
}]
}Now available
Support for custom connectors and ingestion of streaming data in Amazon Bedrock Knowledge Bases is available today in all AWS Regions where Amazon Bedrock Knowledge Bases is available. Check the Region list for details and future updates. To learn more about Amazon Bedrock Knowledge Bases, visit the Amazon Bedrock product page. For pricing details, review the Amazon Bedrock pricing page.
Send feedback to AWS re:Post for Amazon Bedrock or through your usual AWS contacts, and engage with the generative AI builder community at community.aws.
Support for reranking models
Today we also announced the new Rerank API in Amazon Bedrock to offer developers a way to use reranking models to enhance the performance of their RAG-based applications by improving the relevance and accuracy of responses. Semantic search, supported by vector embeddings, embeds documents and queries into a semantic high-dimension vector space where texts with related meanings are nearby in the vector space and therefore semantically similar, so that it returns similar items even if they don’t share any words with the query. Semantic search is used in RAG applications because the relevance of retrieved documents to a user’s query plays a critical role in providing accurate responses and RAG applications retrieve a range of relevant documents from the vector store.
However, semantic search has limitations in prioritizing the most suitable documents based on user preferences or query context especially when the user query is complex, ambiguous, or involves nuanced context. This can lead to retrieving documents that are only partially relevant to the user’s question. This leads to another challenge where proper citation and attribution of sources is not attributed to the correct sources, leading to loss of trust and transparency in the RAG-based application. To address these limitations, future RAG systems should prioritize developing robust ranking algorithms that can better understand user intent and context. Additionally, it is important to focus on improving source credibility assessment and citation practices to confirm the reliability and transparency of the generated responses.
Advanced reranking models solve for these challenges by prioritizing the most relevant content from a knowledge base for a query and additional context to ensure that foundation models receive the most relevant content, which leads to more accurate and contextually appropriate responses. Reranking models may reduce response generation costs by prioritizing the information that is sent to the generation model.
How it works
At launch, we’re supporting Amazon Rerank 1.0 and Cohere Rerank 3.5 reranking models. For the walkthrough, I will use the Amazon Rerank 1.0 model, I will start by requesting access to this model.
Once access has been granted, I create a knowledge base using the existing Amazon Bedrock Knowledge Bases Console experience (an API process is also available as an alternative). The knowledge base contains two data sources; a music playlist, and a list of films.
As soon as the knowledge base has been created I edit the Service Role to add the policy that contains the bedrock:Rerank action. The API takes the user query as the input along with the list of documents that needs to be reranked. The output will be a reranked prioritized list of documents.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel"
],
"Resource": [
"arn:aws:bedrock:us-west-2::foundation-model/amazon.rerank-v1:0"
]
},
{
"Sid": "Statement2",
"Effect": "Allow",
"Action": [
"bedrock:Rerank"
],
"Resource": [
"*"
]
}
]
}The last step is to sync the data sources to index their contents for searching. A sync can take between a few minutes to a few hours.
The knowledge base is ready for use. The RetrieveAndGenerate API reranks the results retrieved from the vector datastore based on their relevance with the query.
To contrast, I ran the same query against the same data in a separate account that doesn’t have the Rerank API. The outcome is that results aren’t reranked on their relevance with the query. This could affect performance and compromise the accuracy of the responses.
Now available
The Rerank API in Amazon Bedrock is available today in the following AWS Regions: US West (Oregon), Canada (Central), Europe (Frankfurt), and Asia Pacific (Tokyo). Check the Region list for details and future updates. Rerank API can be used independently to rerank documents even if you are not using Amazon Bedrock Knowledge Bases. To learn more about Amazon Bedrock Knowledge Bases, visit the Amazon Bedrock product page. For pricing details, review the Amazon Bedrock pricing page.
Send feedback to AWS re:Post for Amazon Bedrock or through your usual AWS contacts, and engage with the generative AI builder community at community.aws.
– Veliswa.
Post Syndicated from Matheus Guimaraes original https://aws.amazon.com/blogs/aws/connect-users-to-data-through-your-apps-with-storage-browser-for-amazon-s3/
Today, we’re introducing Storage Browser for Amazon S3, an open source UI component you can add to your web applications to enable end users to interact with your data stored in Amazon Simple Storage Service (Amazon S3). With this frontend component, authorized end users can browse, upload, download, copy, and delete data from Amazon S3 based on their specific permissions, which you control using AWS identity and security services or custom managed solutions.
Storage Browser for S3 eases the strain on developers looking to provide end users with access to data in S3, and it is designed so that end users, such as customers, partners, and employees, can efficiently work with data regardless of their familiarity with Amazon S3 or Amazon Web Services. Additionally, developers can customize the look and feel of the Storage Browser interface to align with their application’s design.
Let’s walk through a quick demo to show how you can get started.
Installation
Storage Browser for S3 is an AWS Amplify UI React component, therefore, you must use it in a web application built with React or a React-based framework such as Next.Js, Gatsby, Remix, or any others. You also must have both AWS Amplify and the AWS Amplify UI React packages installed.
This demo uses Next.js. If you want to learn how to set up an app from scratch, check out this step-by-step guide on configuring AWS Amplify and using the Amplify React UI components with a new Next.js application.
You don’t need to install the entire @aws-amplify/ui-react library to use Storage Browser for S3.You can install only the storage-specific package with the following command if that is all you intend to use.
npm i @aws-amplify/ui-react-storage aws-amplify
If you have an existing application that already has the Amplify UI React package installed, make sure to update your dependencies to import the latest version, and run npm install to update any existing installations.
Lastly, if you’re building an application from scratch, make sure to run npm create amplify@latest in your application’s directory so you’re able to use the various categories provided by Amplify like auth, storage, and others.
Choosing an authorization mode
Storage Browser for S3 requires authentication and authorization to be configured so it can render the S3 buckets or prefixes that end users can access as well as the actions they can perform.
There are three options for setting up permissions, each suitable for different use cases:
Using AWS Amplify Auth – This option is ideal when you want to provide your customers and third-party partners access to your data in Amazon S3. You can set up Amplify Storage which uses AWS Amplify Auth by default to manage access control and security for files. This is powered by Amazon Cognito and comes with pre-built UI components for implementing user registration, sign-in, and sign-out flows.
Using AWS IAM Identity Center – This option is ideal for a scalable solution providing your whole workforce with access to your data in S3 through Storage Browser for S3 . You associate an S3 Access Grants instance with your AWS Identity and Access Management (IAM) Identity Center to centrally manage S3 Access Grants permissions for your users and groups, including those hosted on external identity providers such as Microsoft Entra ID, Okta, and others. Additionally, each AWS CloudTrail data event for S3 references the end-user identity that accessed your data which helps to increase the observability for your data access.
Using IAM roles with Amazon S3 Access Grants – This option is ideal when you want to provide IAM principals with access to your data through Storage Browser for S3. To set this up, you must first create an S3 Access Grants instance that you can use to map permissions for S3 buckets and prefixes to the desired IAM identities. Then you create an IAM role that has permissions to invoke s3:GetDataAccess to get temporary least-privilege access to S3 buckets or prefixes.
This demo assumes the end users are not part of our organization so Amplify Auth is a great match for this case.
Setting up permissions
First, you must set up Amplify Storage by following this guide. Then, open amplify/storage/resource.ts to declare an S3 bucket alongside the desired access rules following the Amplify authorization model which utilizes prefixes to configure isolated storage for authorized users.
Next, create a component called StorageBrowser which encapsulates the integration with Amplify Auth and that we can easily drop in a page later. Make sure to call Amplify.config() to stitch it all together with a a reference to amplify_outputs.json as a parameter.
Visit the S3 User Guide for detailed instructions for setting up authentication and authorization for Storage Browser for S3.
Adding Storage Browser for S3 to my application
Now that the component is created, you just need to add it to your application in a page where you want to render it by declaring <StorageBrowser/>.
Use npm run dev to run the application. After it loads, navigate to the page where you added Storage Browser For S3 and you should see it loaded with the default layout. Notice also that it is configured with the same paths and permissions that we defined in amplify/storage/resource.ts above allowing users to browse, read, write, and delete files inside the S3 buckets and prefixes that we have set up.
You can download files and browse folders while accessing management operations from the sub-menu which automatically greys out any unavailable actions.
Storage Browser for S3 automatically pages results and makes it possible to filter and search for files and folders, making it easy to navigate and manage data.
All data access is governed by the configured authorization model enabling end users to seamlessly interact with S3 buckets and prefixes through a highly intuitive interface without compromising your security or compliance requirements.
Customizing the interface
Thanks to its flexible design, you can customize Storage Browser For S3 to match the look and feel of your application. Much like any other Amplify UI components it will use the Amplify theme you have active in your application by default. However, you can easily modify any of its components such as the buttons, breadcrumb, the paging controls, text fields, and others, by creating your own theme or targeting elements directly using CSS.
To create a theme, first you must declare it using the defineComponentTheme() function from the @aws-amplify/ui-react/server library. You give it a name such as 'storage-browser' and then target the elements that you want to style.
You can even rearrange the layout as well if you want. In the code you can see that we are setting the flexDirection of all controls to 'row-reverse', for example.
Then you create the theme using the createTheme() function using the storage-browser theme we declared earlier and apply it. We also override the primaryColor and make it green.
After the page is reloaded, you should see the Storage Browser for S3 component with its new more compact layout and new color scheme with green text.
You can customize essentially any element of the UI interface including any of the display texts such as the title where it says Home, or any others. The only exceptions are the details about the data, of course, such as the bucket names and keys. You can take advantage of this to add support for different languages, for example.
Finally, if you prefer to create your own UI from scratch, you call the createStorageBrowser() function to create a Storage Browser for S3 component programatically. It returns a useView() hook that you can use to integrate with your own custom frontend, giving you full control over the look and feel while leveraging all of the same features. To learn more, see the documentation for more details on the various customization options and how to configure them.
Conclusion
Storage Browser for S3 is a highly customizable and user-friendly AWS Amplify UI React component which enables end users to interact with data on Amazon S3 securely. It gives you full control of the access rules to ensure the frontend complies with your access requirements while providing a great user experience through an interface that you can style to make it appear as a natural extension of your application.
Things to know
Getting started – You can install Storage Browser for S3 from the GitHub page. For more information on getting started, visit the UI documentation.
Compatibility – Storage Browser for S3 is compatible with all Amazon S3 storage classes except for Glacier Flexible Retrieval and S3 Glacier Deep Archive. It is compatible with S3 Intelligent-Tiering, but it’s not compatible with the S3 Intelligent-Tiering Archive Access Tier or the S3 Intelligent-Tiering Deep Archive Access Tier..
Performance and durability – Storage Browser for S3 includes built-in logic that enhances upload requests for high-throughput data transfer, calculates checksums of uploaded data (rejecting requests that fail these durability checks), and optimizes performance for faster load times in your application.
Pricing – Storage Browser for S3 is open source and you can integrate it with your applications at no extra cost. You only pay for your use of the underlying AWS resources you use with Storage Browser for S3.
Support – Storage Browser for S3 is backed by AWS Support just like any other feature of S3. Customers with Business and Enterprise Support plans get 24/7 access to AWS Support engineers to support their use of Storage Browser for S3.
Feedback – We invite you to share feedback on the functionality and the public roadmap for Storage Browser for S3.
Post Syndicated from Danilo Poccia original https://aws.amazon.com/blogs/aws/introducing-new-partyrock-capabilities-and-free-daily-usage/
PartyRock is an Amazon Bedrock playground that anyone can use to create generative AI-powered applications by simply describing the app you want to build without the need to write any code.
Since its launch in November 2023, over half a million apps have been built by users worldwide. These apps range from simple text generators to sophisticated productivity tools that combine multiple AI capabilities.
Throughout this year, we observed that as PartyRock users build skills and intuition by using the playground, they find interesting and useful ways to build apps for improving their daily lives. PartyRock apps increased their individual productivity, and they returned to PartyRock to use them regularly.
Today, we’re introducing improvements meeting the most requested customer needs:
Free daily usage – Previously, PartyRock offered a free trial period for a limited time. Starting in 2025, all users will have a recurring free daily usage granted, with no credit card required.
Search the app catalog – You can now explore hundreds of thousands of apps in the PartyRock catalog and find the right app for your use case by category or functionality. Relevant and popular apps are highlighted to showcase the creativity of the community. Results include app previews and last modified date to help you pick what’s best for you.
Do more with docs – You can upload and process multiple documents simultaneously, making it easier to build apps that handle batch processing, document comparison, or content aggregation.
Let’s see these some new features in action.
Searching and remixing a PartyRock app
I open PartyRock and sign in with my social credentials. In the Home section, I can use the search box to look for apps for a specific use case. I love traveling, and I’d like to improve the way I share my trips with family and friends. I enter travel and vlog in the search box. In the search results, I see an app that gets my attention.
I choose the Travel vlog script writer app and open it in a browser tab. The app generates a travel log script starting from a few inputs: the destination, the itinerary, and the tone.
I like to prepare some travel notes before a trip so that I know what the options are and what I want to visit. What if I can upload my notes and other documents to better personalize the vlog?
One of the key capabilities of PartyRock is that I can start with an existing app and “remix” it to tailor it to my needs. The resulting app can then be shared for others to use.
I choose Remix and then Edit to customize this app. I add a Document widget and edit it:
Notes.Upload your notes and documents with travel tips.I save the new widget and move just after the other input fields.
To use these images in the app, I edit the Your Vlog script widget. I want the script to include the content of those images. In the prompt generating the script, I add a sentence to analyze and consider the image of the destination:
Get inspiration from what you see in @Notes.
I also update the Vlog cover widget prompt to consider the whole script when generating the cover image:
A portrait of a trip to @Destination considering the @Your Vlog script.
I save and leave edit. The remixed app is now ready to be tested.
Using the remixed PartyRock app
Let’s try the customized version of the app. I enter:
Rome, Italy as DestinationA walk in the old city center as ItineraryPeaceful and relaxing as Tone.Then, I upload my travel notes.
I choose the Play button to start the app. The app takes a few seconds to generate its output.
I like the result. The script is quite detailed, and the image cover a nice addition. I can further extend the app to use the image cover in a social media post generator for posting about the vlog to different platforms with different tones and styles. The possibilities are endless!
Things to know
PartyRock with these new capabilities is available at https://partyrock.aws.
No credit card or AWS account is required to use PartyRock, and you can explore hundreds of thousands of published apps even without signing in.
With PartyRock, everyone can become a builder. Apps can be generated from a textual description and then customized and extended with additional capabilities using the visual editor. All apps are automatically optimized for mobile devices and can be shared with others. To make it easier for others to view and use your apps, you can create your personalized playlist page.
For examples of how PartyRock can help you be more productive, refer to How 3 small businesses use PartyRock to help customers. And don’t forget to share your best apps with me!
— Danilo
Post Syndicated from Betty Zheng (郑予彬) original https://aws.amazon.com/blogs/aws/amazon-memorydb-multi-region-is-now-generally-available/
Providing highly available applications while maintaining low latency reads and writes across AWS Regions is a common challenge faced by many customers. Accessing data from different Regions can cause a delay of hundreds of milliseconds compared to microseconds within the same Region. The necessity for developers to create complex custom solutions for data replication and conflict resolution can lead to increased operational workload and potential errors. Beyond multi-Region replication, these customers have to implement manual database failover procedures and provide data consistency and recovery to deliver highly available applications and data durability.
Today, Amazon Web Services (AWS) announced the general availability of Amazon MemoryDB Multi-Region, a fully managed, active-active, multi-Region database that you can use to build applications with up to 99.999 percent availability, microsecond read, and single-digit millisecond write latencies across multiple AWS Regions. MemoryDB Multi-Region is available for Valkey, which is a Redis Open Source Software (OSS) drop-in replacement stewarded by Linux Foundation. This new feature builds upon the existing benefits of Amazon MemoryDB, such as multi-AZ durability and high throughput across multiple AWS Regions, and addresses these common challenges faced by many customers.
In this post, we discuss the benefits of MemoryDB Multi-Region and demonstrate how to get started with it using the AWS Management Console and the AWS Command Line Interface (AWS CLI).
MemoryDB Multi-Region provides the following benefits to customers:
Setting up MemoryDB Multi-Region is straightforward and can be accomplished through the AWS Management Console, AWS SDK, or AWS CLI.
To set up your MemoryDB Multi-Region cluster using the console, complete the following steps:
On the MemoryDB console, choose Clusters in the navigation pane, choose Create cluster, select Multi-Region cluster for Cluster type, and Create new cluster for the Cluster creation method.
You can select the Node type and number of shards based on your workload requirement when you set up your Multi-Region cluster.
Create the Regional cluster within your Multi-Region cluster with the appropriate cluster settings.
You can add a second Regional cluster to your Multi-Region cluster by choosing Add AWS region after the Multi-Region cluster and the first Regional cluster are set up.
When the cluster creation workflow finishes successfully, you can observe that there are two Regional clusters within the Multi-Region cluster.
To begin, create a new MemoryDB Multi-Region cluster:
aws memorydb create-multi-region-cluster \ --multi-region-cluster-name-suffix testmrrlp \ --endpoint-url https://elasticache-qa.us-east-1.amazonaws.com \ --description "testdescription" \ --node-type db.r7g.xlarge \ --region us-east-1 \ --no-verify-ssl
Next, create a Regional cluster in the Multi-Region cluster:
aws memorydb create-cluster \
--cluster-name testmrrlp-member1 \
--multi-region-cluster-name ldgnf-testmrrlp \
--node-type db.r7g.xlarge \
--num-replicas-per-shard 1 \
--snapshot-retention-limit 10 \
--endpoint-url <value> \
--acl-name open-access \
--region us-east-1 \
--no-verify-sslAfter verifying the successful creation of the first cluster, create the second cluster in a different Region:
aws memorydb create-cluster \ --cluster-name testmrrlp-member2 \ --multi-region-cluster-name ldgnf-testmrrlp \ --node-type db.r7g.xlarge \ --num-replicas-per-shard 1 \ --snapshot-retention-limit 10 \ --endpoint-url https://elmo-qa.fra.aws-border.com \ --acl-name open-access \ --region eu-central-1 \ --no-verify-ssl
Check the status of the Multi-Region cluster:
aws memorydb describe-multi-region-clusters \ --multi-region-cluster-name ldgnf-testmrrlp \ --region us-east-1 \ --show-member-cluster-details \ --endpoint-url https://elasticache-qa.us-east-1.amazonaws.com \ --no-verify-ssl
Amazon MemoryDB Multi-Region is available for Valkey and in the following AWS Regions: US East (N. Virginia, Ohio), US West (N. California, Oregon), Asia Pacific (Mumbai, Seoul, Singapore, Sydney, Tokyo), and Europe (Frankfurt, Ireland, London).
To learn more, visit the MemoryDB features page and documentation. For pricing, refer to Amazon MemoryDB pricing.