Set up cross-account AWS Glue Data Catalog access using AWS Lake Formation and AWS IAM Identity Center with Amazon Redshift and Amazon QuickSight

2024-08-05 Poulomi Dasgupta

Post Syndicated from Poulomi Dasgupta original https://aws.amazon.com/blogs/big-data/set-up-cross-account-aws-glue-data-catalog-access-using-aws-lake-formation-and-aws-iam-identity-center-with-amazon-redshift-and-amazon-quicksight/

Most organizations manage their workforce identity centrally in external identity providers (IdPs) and are comprised of multiple business units that produce their own datasets and manage the lifecycle spread across multiple AWS accounts. These business units have varying landscapes, where a data lake is managed by Amazon Simple Storage Service (Amazon S3) and analytics workloads are run on Amazon Redshift, a fast, scalable, and fully managed cloud data warehouse that allows you to process and run your complex SQL analytics workloads on structured and semi-structured data.

Business units that create data products like to share them with others, without copying the data, to promote analysis to derive insights. Also, they want tighter control on user access and the ability to audit access to their data products. To address this, enterprises usually catalog the datasets in the AWS Glue Data Catalog for data discovery and use AWS Lake Formation for fine-grained access control to adhere to the compliance and operating security model for their business units. Given the diverse range of services, fine-grained data sharing, and personas involved, these enterprises often want a streamlined experience for enterprise user identities when accessing their data using AWS Analytics services.

AWS IAM Identity Center enables centralized management of workforce user access to AWS accounts and applications using a local identity store or by connecting corporate directories using IdPs. Amazon Redshift and AWS Lake Formation are integrated with the new trusted identity propagation capability in IAM Identity Center, allowing you to use third-party IdPs such as Microsoft Entra ID (Azure AD), Okta, Ping, and OneLogin.

With trusted identity propagation, Lake Formation enables data administrators to directly provide fine-grained access to corporate users and groups, and simplifies the traceability of end-to-end data access across supported AWS services. Because access is managed based on a user’s corporate identity, end-users don’t need to use database local user credentials or assume an AWS Identity and Access Management (IAM) role to access data. Furthermore, this enables effective user permissions based on collective group membership and supports group hierarchy.

In this post, we cover how to enable trusted identity propagation with AWS IAM Identity Center, Amazon Redshift, and AWS Lake Formation residing on separate AWS accounts and set up cross-account sharing of an S3 data lake for enterprise identities using AWS Lake Formation to enable analytics using Amazon Redshift. Then we use Amazon QuickSight to build insights using Redshift tables as our data source.

Solution overview

This post covers a use case where an organization centrally manages corporate users within their IdP and where the users belong to multiple business units. Their goal is to enable centralized user authentication through IAM Identity Center in the management account, while keeping the business unit that analyzes data using a Redshift cluster and the business unit that produces data cataloged using the Data Catalog in separate member accounts. This allows them to maintain a single authentication mechanism through IAM Identity Center within an organization while retaining access control, resource, and cost separation through the use of separate AWS accounts per business units and enabling cross-account data sharing using Lake Formation.

For this solution, AWS Organizations is enabled in the central management account and IAM Identity Center is configured for managing workforce identities. The organization has two member accounts: one account that manages the S3 data lake using the Data Catalog, and another account that runs analytical workloads on Amazon Redshift and QuickSight, with all the services enabled with trusted identity propagation. Amazon Redshift will access cross-account AWS Glue resources using IAM Identity Center users and groups set up in the central management account using QuickSight in member account 1. In member account 2, permissions on the AWS Glue resources are managed using Lake Formation and are shared with member account 1 using Lake Formation data sharing.

The following diagram illustrates the solution architecture.

The solution consists of the following:

In the centralized management account, we create a permission set and create account assignments for Redshift_Member_Account. We integrate users and groups from the IdP with IAM Identity Center.
Member account 1 (Redshift_Member_Account) is where the Redshift cluster and application exist.
Member account 2 (Glue_Member_Account) is where metadata is cataloged in the Data Catalog and Lake Formation is enabled with IAM Identity Center integration.
We assign permissions to two IAM Identity Center groups to access the Data Catalog resources:
- awssso-sales – We apply column-level filtering for this group so that users belonging to this group will be able to select two columns and read all rows.
- awssso-finance – We apply row-level filtering using data filters for this group so that users belonging to this group will be able to select all columns and see rows after row-level filtering is applied.
We apply different permissions for three IAM Identity Center users:
- User Ethan, part of awssso-sales – Ethan will be able to select two columns and read all rows.
- User Frank, part of awssso-finance – Frank will be able to select all columns and see rows after row-level filtering is applied.
- User Brian, part of awssso-sales and awssso-finance – Brian inherits permissions defined for both groups.
We set up QuickSight in the same account where Amazon Redshift exists, enabling authentication using IAM Identity Center.

Prerequisites

You should have the following prerequisites alreday set up:

A centralized management account where IAM Identity Center is enabled with member accounts added. For more information, see Enabling AWS IAM Identity Center.
Optionally, connect IAM Identity Center with your preferred IdP and sync users and groups. For instructions, refer to Getting started tutorials.
Member account 1 (Redshift_Member_Account) where the Redshift cluster and application exist. To set up a Redshift cluster with IAM Identity Center integration enabled, refer to Integrate Identity Provider (IdP) with Amazon Redshift Query Editor V2 using AWS IAM Identity Center for seamless Single Sign-On.
Member account 2 (Glue_Member_Account) where metadata is cataloged in the Data Catalog.

Member account 2 configuration

Sign in to the Lake Formation console as the data lake administrator. To learn more about setting up permissions for a data lake administrator, see Create a data lake administrator.

In this section, we walk through the steps to set up Lake Formation, enable Lake Formation permissions, and grant database and table permissions to IAM Identity Center groups.

Set up Lake Formation

Complete the steps in this section to set up Lake Formation.

Create AWS Glue resources

You can use an existing AWS Glue database that has a few tables. For this post, we use a database called customerdb and a table called reviews whose data is stored in the S3 bucket lf-datalake-<account-id>-<region>.

Register the S3 bucket location

Complete the following steps to register the S3 bucket location:

On the Lake Formation console, in the navigation pane, under Administration, choose Data lake locations.
Choose Register location.
For Amazon S3 location, enter the S3 bucket location that contains table data.
For IAM role, provide a user-defined IAM role. For instructions to create a user-defined IAM role, refer to Requirements for roles used to register locations.
For Permission mode, select Lake Formation.
Choose Register location.

Set cross-account version

Complete the following steps to set your cross-account version:

Sign in to the Lake Formation console as the data lake admin.
In the navigation pane, under Administration, choose Data Catalog settings.
Under Cross-account version settings, keep the latest version (Version 4) as the current cross-account version.
Choose Save.

Add permissions required for cross-account access

If the AWS Glue Data Catalog resource policy is already enabled in the account, then you can either remove the policy or add the following permissions to the policy that are required for cross-account grants. The provided policy enables AWS Resource Access Manager (AWS RAM) to share a resource policy while cross-account grants are made using Lake Formation. For more information, refer to Prerequisites. Please skip to the following step if your policy is blank under Catalog Settings.

Sign in to the AWS Glue console as an IAM admin.
In the navigation pane, under Data Catalog, choose Catalog settings.
Under Permissions, add the following policy, and provide the account ID where your AWS Glue resources exist:

{ "Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ram.amazonaws.com"
},
"Action": "glue:ShareResource",
"Resource": [
"arn:aws:glue:us-east-1:<account-id>:table/*/*",
"arn:aws:glue:us-east-1:<account-id>:database/*",
"arn:aws:glue:us-east-1:<account-id>:catalog"
]
}
]
}

Choose Save.

For more information, see Granting cross-account access.

Enable IAM Identity Center integration for Lake Formation

To integrate IAM Identity Center with your Lake Formation organization instance of IAM Identity Center, refer to Connecting Lake Formation with IAM Identity Center.

To enable cross-account sharing for IAM Identity Center users and groups, add the target recipient accounts to your Lake Formation IAM Identity Center integration under the AWS account and organization IDs.

Sign in to the Lake Formation console as a data lake admin.
In the navigation pane, under Administration, choose IAM Identity Center integration.
Under AWS account and organization IDs, choose Add.
Enter your target accounts.
Choose Add.

Enable Lake Formation permissions for databases

For Data Catalog databases that contain tables that you might share, you can stop new tables from having the default grant of Super to IAMAllowedPrincipals. Complete the following steps:

Sign in to the Lake Formation console as a data lake admin.
In the navigation pane, under Data Catalog, choose Databases.
Select the database customerdb.
Choose Actions, then choose Edit.
Under Default permissions for newly created tables, deselect Use only IAM access control for new tables in this database.
Choose Save.

For Data Catalog databases, remove IAMAllowedPrincipals.

Under Data Catalog in the navigation pane, choose Databases.
Select the database customerdb.
Choose Actions, then choose View.
Select IAMAllowedPrincipals and choose Revoke.

Repeat the same steps for tables under the customerdb database.

Grant database permissions to IAM Identity Center groups

Complete the following steps to grant database permissions to your IAM Identity Center groups:

On the Lake Formation console, under Data Catalog, choose Databases.
Select the database customerdb.
Choose Actions, then choose Grant.
Select IAM Identity Center.
Choose Add and select Get Started.
Search for and select your IAM Identity Center group names and choose Assign.

Select Named Data Catalog resources.
Under Databases, choose customerdb.
Under Database permissions, select Describe for Database permissions.
Choose Grant.

Grant table permissions to IAM Identity Center groups

In the following section, we will grant different permissions to our two IAM Identity Center groups.

Column filter

We first add permissions to the group awssso-sales. This group will have access to the customerdb database and be able to select only two columns and read all rows.

On the Lake Formation console, under Data Catalog in the navigation pane, choose Databases.
Select the database customerdb.
Choose Actions, then choose Grant.
Select IAM Identity Center.
Choose Add and select Get Started.
Search for and select awssso-sales and choose Assign.

Select Named Data Catalog resources.
Under Databases, choose customerdb.
Under Tables, choose reviews.
Under Table permissions, select Select for Table permissions.
Select Column-based access.
Select Include columns and choose product_title and star_rating.
Choose Grant.

Row filter

Next, we grant permissions to awssso-finance. This group will have access to customerdb and be able to select all columns and apply filters on rows.

We need to first create a data filter by performing the following steps:

On the Lake Formation console, choose Data filters under Data Catalog.
Choose Create data filter.
For Data filter name, provide a name.
For Target database, choose customerdb.
For Target table, choose reviews.
For Column-level access, select Access to all columns.
For Row-level access, choose Filter rows and apply your filter. In this example, we are filtering reviews with star_rating as 5.
Choose Create data filter.

Under Data Catalog in the navigation pane, choose Databases.
Select the database customerdb.
Choose Actions, then choose Grant.
Select IAM Identity Center.
Choose Add and select Get Started.
Search for and select awssso-finance and choose Assign.
Select Named Data Catalog resources.
Under Databases, choose customerdb.
Under Tables, choose reviews.
Under Data Filters, choose the High_Rating
Under Data Filter permissions, select Select.
Choose Grant.

Member account 1 configuration

In this section, we walk through the steps to add Amazon Redshift Spectrum table access in member account 1, where the Redshift cluster and application exist.

Accept Invite from RAM

You should have received a Resource Access Manager (RAM) invite from member account 2 when you added member account 1 under IAM Identity Center integration in Lake Formation at the member account 1.

Navigate to Resource Access Manager(RAM) from admin console.
Under Shared with me, click on resource shares.
Select the resource name and click on Accept resource share.

Please make sure that you have followed this entire blog to establish the Redshift Integration with IAM Identity Center before following the next steps.

Set up Redshift Spectrum table access for the IAM Identity Center group

Complete the following steps to set up Redshift Spectrum table access:

Sign in to the Amazon Redshift console using the admin role.
Navigate to Query Editor v2.
Choose the options menu (three dots) next to the cluster and choose Create connection.
Connect as the admin user and run the following commands to make the shared resource link data in the S3 data lake available to the sales group (use the account ID where the Data Catalog exists):

create external schema if not exists <schema_name> from DATA CATALOG database '<glue_catalog_name>' catalog_id '<accountid>';
grant usage on schema <schema_name> to role "<role_name>";

For example:

create external schema if not exists cross_account_glue_schema from DATA CATALOG database 'customerdb' catalog_id '932880906720';
grant usage on schema cross_account_glue_schema to role "awsidc:awssso-sales";
grant usage on schema cross_account_glue_schema to role "awsidc:awssso-finance";

Validate Redshift Spectrum access as an IAM Identity Center user

Complete the following steps to validate access:

On the Amazon Redshift console, navigate to Query Editor v2.
Choose the options menu (three dots) next to the cluster and choose Create connection.
Select IAM Identity Center.
Enter your Okta user name and password in the browser pop-up.

When you’re connected as a federated user, run the following SQL commands to query the cross_account_glue_schema data lake table.

select * from "dev"."cross_account_glue_schema"."reviews";

The following screenshot shows that user Ethan, who is part of the awssso-sales group, has access to two columns and all rows from the Data Catalog.

The following screenshot shows that user Frank, who is part of the awssso-finance group, has access to all columns for records that have star_rating as 5.

The following screenshot shows that user Brian, who is part of awssso-sales and awssso-finance, has access to all columns for records that have star_rating as 5 and access to only two columns (other columns are returned NULL) for records with star_rating other than 5.

Subscribe to QuickSight with IAM Identity Center

In this post, we set up QuickSight in the same account where the Redshift cluster exists. You can use the same or a different member account for QuickSight setup. To subscribe to QuickSight, complete the following steps:

Sign in to your AWS account and open the QuickSight console.
Choose Sign up for QuickSight.

Enter a notification email address for the QuickSight account owner or group. This email address will receive service and usage notifications.
Select the identity option that you want to subscribe with. For this post, we select Use AWS IAM Identity Center.
Enter a QuickSight account name.
Choose Configure.

Next, assign groups in IAM Identity Center to roles in QuickSight (admin, author, and reader.) This step enables your users to access the QuickSight application. In this post, we choose awssso-sales and awssso-finance for Admin group.
Specify an IAM role to control QuickSight access to your AWS resources. In this post, we select Use QuickSight managed role (default).
For this post, we deselect Add Paginated Reports.
Review the choices that you made, then choose Finish.

Enable trusted identity propagation in QuickSight

Trusted identity propagation authenticates the end-user in Amazon Redshift when they access QuickSight assets that use a trusted identity propagation enabled data source. When an author creates a data source with trusted identity propagation, the identity of the data source consumers in QuickSight is propagated and logged in AWS CloudTrail. This allows database administrators to centrally manage data security in Amazon Redshift and automatically apply data security rules to data consumers in QuickSight.

To configure QuickSight to connect to Amazon Redshift data sources with trusted identity propagation, configure Amazon Redshift OAuth scopes to your QuickSight account:

aws quicksight update-identity-propagation-config --aws-account-id "AWSACCOUNTID" --service "REDSHIFT" --authorized-targets "IAM Identity Center managed application ARN"

For example:

aws quicksight update-identity-propagation-config --aws-account-id "1234123123" --service "REDSHIFT" --authorized-targets "arn:aws:sso::XXXXXXXXXXXX:application/ssoins-XXXXXXXXXXXX/apl-XXXXXXXXXXXX"

After you have added the scope, the following command lists all OAuth scopes that are currently on a QuickSight account:

aws quicksight list-identity-propagation-configs --aws-account-id "AWSACCOUNTID"

The following code is the example with output:

aws quicksight list-identity-propagation-configs --aws-account-id "1234123123"
{
"Status": 200,
"Services": [
{
"Service": "REDSHIFT",
"AuthorizedTargets": [
"arn:aws:sso::1004123000:application/ssoins-1234f1234bb1f123/apl-12a1234e2e391234"
]
}
],
"RequestId": "116ec1b0-1533-4ed2-b5a6-d7577e073b35"
}

For more information, refer to Authorizing connections from Amazon QuickSight to Amazon Redshift clusters.

For QuickSight to connect to a Redshift instance, you must add an appropriate IP address range in the Redshift security group for the specific AWS Region. For more information, see AWS Regions, websites, IP address ranges, and endpoints.

Test your IAM Identity Center and Amazon Redshift integration with QuickSight

Now you’re ready to connect to Amazon Redshift using QuickSight.

In the management account, open the IAM Identity Center console and copy the AWS access portal URL from the dashboard.
Sign out from the management account and enter the AWS access portal URL in a new browser window.
In the pop-up window, enter your IdP credentials.

After successful authentication, you’ll be logged in to the AWS Management Console as a federated user.

On the Applications tab, select the QuickSight app.
After you federate to QuickSight, choose Datasets.
Select New Dataset and then choose Redshift (Auto Discovered).
Enter your data source details. Make sure to select Single sign-on for Authentication method.
Choose Create data source.

Congratulations! You’re signed in using IAM Identity Center integration with Amazon Redshift and are ready to explore and analyze your data using QuickSight.

The following screenshot from QuickSight shows that user Ethan, who is part of the awssso-sales group, has access to two columns and all rows from the Data Catalog.

The following screenshot from QuickSight shows that user Frank, who is part of the awssso-finance group, has access to all columns for records that have star_rating as 5.

The following screenshot from QuickSight shows that user Brian, who is part of awssso-sales and awssso-finance, has access to all columns for records that have star_rating as 5 and access to only two columns (other columns are returned NULL) for records with star_rating other than 5.

Clean up

Complete the following steps to clean up your resources:

Delete the data from the S3 bucket.
Delete the Data Catalog objects that you created as part of this post.
Delete the Lake Formation resources and QuickSight account.
If you created new Redshift cluster for testing this solution, delete the cluster.

Conclusion

In this post, we established cross-account access to enable centralized user authentication through IAM Identity Center in the management account, while keeping the Amazon Redshift and AWS Glue resources isolated by business unit in separate member accounts. We used Query Editor V2 for querying the data from Amazon Redshift. Then we showed how to build user-facing dashboards by integrating with QuickSight. Refer to Integrate Tableau and Okta with Amazon Redshift using AWS IAM Identity Center to learn about integrating Tableau and Okta with Amazon Redshift using IAM Identity Center.

Learn more about IAM Identity Center with Amazon Redshift, QuickSight, and Lake Formation. Leave your questions and feedback in the comments section.

About the Authors

Srividya Parthasarathy is a Senior Big Data Architect on the AWS Lake Formation team. She enjoys building data mesh solutions and sharing them with the community.

Maneesh Sharma is a Senior Database Engineer at AWS with more than a decade of experience designing and implementing large-scale data warehouse and analytics solutions. He collaborates with various Amazon Redshift Partners and customers to drive better integration.

Poulomi Dasgupta is a Senior Analytics Solutions Architect with AWS. She is passionate about helping customers build cloud-based analytics solutions to solve their business problems. Outside of work, she likes travelling and spending time with her family.

AWS Weekly Roundup: Amazon Q Business, AWS CloudFormation, Amazon WorkSpaces update, and more (Aug 5, 2024)

2024-08-05 Matheus Guimaraes

Post Syndicated from Matheus Guimaraes original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-amazon-q-business-aws-cloudformation-amazon-workspaces-update-and-more-aug-5-2024/

Summer is reaching its peak for some of us around the globe, and many are heading out to their favorite holiday destinations to enjoy some time off. I just came back from holidays myself and I couldn’t help thinking about the key role that artificial intelligence (AI) plays in our modern world to help us scale the operation of simple things like traveling. Passport and identity verifications were quick, and thanks to the new airport security system rolling out across the world, so were my bag checks. I watched my backpack with a smile as it rolled along the security check belt with my computer, tablet, and portable game consoles all nicely tucked inside without any fuss.

If it wasn’t for AI, we wouldn’t be able to scale operations to keep up with population growth or the enormous volumes of data we generate on a daily basis. The advent of generative AI took this even further by unlocking the ability to put all this data to use in all kinds of creative ways, driving a new wave of exciting innovations that continues to elevate modern products and services.

This new landscape can be challenging for companies that are learning how generative AI can help them grow or succeed, such as startups. This is why I’m so excited about the AWS GenAI Lofts taking place in the next months around the world.

The AWS GenAI Lofts are collaborative spaces available in different cities around the world for a number of weeks. Startups, developers, investors, and industry experts can meet here while having access to AWS AI experts, and attend talks, workshops, fireside chats, and Q&As with industry leaders. All lofts are free and are carefully curated to offer something for everyone to help you accelerate your journey with AI. There are lofts scheduled in Bengaluru (July 29-Aug 9), San Francisco (Aug 14-Sept 27), Sao Paulo (Sept 2-Nov 20), London (Sept 30-Oct 25), Paris (Oct 8-Nov 25), and Seoul (Nov, pending exact dates). I highly encourage you to have a look at the agendas of a loft near you and drop in to learn more about GenAI and connect with others.

Last week’s launches
Here are some launches that got my attention last week.

Amazon Q Business cross-Region IdC — Amazon Q Business is a generative AI-powered assistant that deeply understands your business by providing connectors that you can easily set up to unify data from various sources such as Amazon S3, Microsoft 365, and more. You can then generate content, answer questions, and even automate tasks that are relevant and specific to your business. Q Business integrates with AWS IAM Identity Center to ensure that data can only be accessed by those who are authorized to do so. Previously, the IAM Identity Center instance had to be located in the same Region as the Q Business application. Now, you can connect to one in a different Region.

Git sync status changes publish to Amazon EventBridge — AWS CloudFormation Git sync is a very handy feature that can help streamline your DevOps operations by allowing you to automatically update your AWS CloudFormation stacks whenever you commit changes to the template or deployment file in source control. As of last week, any sync status change is published in near real-time as an event to EventBridge. This enables you to take your GitOps workflow further and stay on top of your Git repositories or resource sync status changes.

Some AWS Pinpoint’s capabilities are now under AWS End User Messaging — AWS Pinpoint’s SMS, MMS, push, and text to voice capabilities have been shuffled and now are offered through their own service called AWS End User Messaging. There is no impact to existing applications and no changes to APIs, the AWS Command Line Interface (AWS CLI), or IAM policies, however, the new name is now reflected on the AWS Management Console, AWS Billing console dashboard, documentation, and other places.

Amazon WorkSpaces updates — Microsoft Visual Studio Professional and Microsoft Visual Studio Enterprise 2022 are now added to the list of available license included applications on Workspaces Personal. Additionally, Amazon WorkSpaces Thin Client has received Carbon Trust verification. As verified by the Carbon Trust, the total lifecycle carbon emission is 77kg CO2e and 50% of the product is made from recycled materials.

GenAI for the Public Sector — There has been two significant launches that may interest those in the public sector looking into getting started with generative AI. Amazon Bedrock is now a FedRAMP High authorized service in the AWS GovCloud (US-West) Region. Additionally, both Llama 3 8B and Lllama 3 70B are now also available in that Region making this a perfect opportunity to start experimenting with Bedrock and Llama 3 if you have workloads in the AWS GovCloud (US-West) Region.

Customers in Germany can now sign up for AWS using their bank account — That means no debit or credit card is needed to create AWS accounts if you have a billing address in Germany. This can help simplify payment of AWS invoices for some businesses, as well as make it easier for others to get started on AWS.

Learning Materials

These are my recommended learning materials for this week.

AWS Skill Builder — This is more of a broad recommendation, but I’m still surprised that so many people never heard of AWS Skill Builder or have not tried it yet. There is so much learning you can do for free including a lot of hands-on courses. In July alone, AWS Skill Builder has launched 25 new digital training products including AWS SimulLearn and AWS Cloud Quest: Generative AI which are game-based learning experiences. Speaking of that, did you know that if you need to renew your Cloud Practitioner certification you can do it simply by playing the AWS Cloud Quest: Recertify Cloud Practioner game?

Get started with agentic code interpreter — Earlier last month we released a new capability on Agents for Amazon Bedrock which allows agents to dynamically generate and execute code within a secure sandboxed environment. As usual, my colleague Mike Chambers has created a great video and blog post on community.aws showing how you can start using it today.

That’s it for this week. Check back next Monday for another Weekly Roundup!

SaaS authentication: Identity management with Amazon Cognito user pools

2024-08-05 Shubhankar Sumar

Post Syndicated from Shubhankar Sumar original https://aws.amazon.com/blogs/security/saas-authentication-identity-management-with-amazon-cognito-user-pools/

Amazon Cognito is a customer identity and access management (CIAM) service that can scale to millions of users. Although the Cognito documentation details which multi-tenancy models are available, determining when to use each model can sometimes be challenging. In this blog post, we’ll provide guidance on when to use each model and review their pros and cons to help inform your decision.

Cognito overview

Amazon Cognito handles user identity management and access control for web and mobile apps. With Cognito user pools, you can add sign-up, sign-in, and access control to your apps. A Cognito user pool is a user directory within a specific AWS Region where users can authenticate and register for applications. In addition, a Cognito user pool is an OpenID Connect (OIDC) identity provider (IdP). App users can either sign in directly through a user pool or federate through a third-party IdP. Cognito issues a user pool token after successful authentication, which can be used to securely access backend APIs and resources.

Cognito issues three types of tokens:

ID token – Contains user identity claims like name, email, and phone number. This token type authenticates users and enables authorization decisions in apps and API gateways.
Access token – Includes user claims, groups, and authorized scopes. This token type grants access to API operations based on the authenticated user and application permissions. It also enables fine-grained, user-based access control within the application or service.
Refresh token – Retrieves new ID and access tokens when these are expired. Access and ID tokens are short-lived, while the refresh token is long-lived. By default, refresh tokens expire 30 days after the user signs in, but this can be configured to a value between 60 minutes and 10 years.

You can find more information on using tokens and their contents in the Cognito documentation.

Multi-tenancy approaches

Software as a service (SaaS) architectures often use silo, pool, or bridge deployment models, which also apply to CIAM services like Cognito. The silo model isolates tenants in dedicated resources. The pool model shares resources between tenants. The bridge model connects siloed and pooled components. This post compares the Cognito silo and pool models for SaaS identity management.

It’s also possible to combine the silo and pool models by having multiple tiers of resources. For example, you could have a siloed tier for sensitive tenant data along with a pooled tier for shared functionality. This is similar to the silo model but with added routing complexity to connect the tiers. When you have multiple pools or silos, this is a similar approach to the pure silo model but with more components to manage.

More detail on these models are included in the AWS SaaS Lens.

We’ve detailed five possible patterns in the following sections and explored the scenarios where each of the patterns can be used, along with the advantages and disadvantages for each. The rest of the post delves deeper into the details of these different patterns, enabling you to make an informed decision that best aligns with your unique requirements and constraints.

Pattern 1: Representing SaaS identity with custom attributes

To implement multi-tenancy in a SaaS application, tenant context needs to be associated with user identity. This allows implementation of the multi-tenant policies and strategies that comprise our SaaS application. Cognito has user pool attributes, which are pieces of information to represent identity. There are standard attributes, such as name and email, that describe the user identity. Cognito also supports custom attributes that can be used to hold information about the user’s relationship to a tenant, such as tenantId.

By using custom attributes for multi-tenancy in Amazon Cognito, the tenant context for each user can be stored in their user profile.

To enable multi-tenancy, you can add a custom attribute like tenantId to the user profile. When a new user signs up, this tenantId attribute can be set to a value indicating which tenant the user belongs to. For example, users with tenantId “1234” belong to Tenant A, while users with tenantId “5678” belong to Tenant B.

The tenantId attribute value gets returned in the ID token after a successful user authentication. (This value can also be added to the access token through customization by using a pre-token generation Lambda trigger.) The application can then inspect this claim to determine which tenant the user belongs to. The tenantId attribute is typically managed at the SaaS platform level and is read-only to users and the application layer. (Note: SaaS providers need to configure the tenantId attribute to be read-only.)

In addition to storing a tenant ID, you can use custom attributes to model additional tenant context. For instance, attributes like tenantName, tenantTier, or tenantRegion could be defined and set appropriately for each user to provide relevant informational context for the application. However, make sure not to use custom attributes as a database—they are meant to represent identity, not store application data. Custom attributes should only contain information that is relevant for authorization decisions and JSON web token (JWT) compactness and should be relatively static because their values are stored in the Cognito directory. Updating frequently changing data requires modifying the directory, which can be cumbersome.

The custom attributes themselves need to be defined at the time of creating the Amazon Cognito user pool, and there is a maximum of 50 custom attributes that you can create. Once the pool is created, these custom attribute fields will be present on every user profile in that user pool. However, they won’t have values populated yet. The actual tenant attribute values get populated only when a new user is created in the user pool. This can be done in two ways:

During user sign-up, a post confirmation AWS Lambda trigger can be used to set the appropriate tenant attribute values based on the user’s input.
An admin user can provision a new user through the AdminCreateUser API operation and specify the tenant attribute values at that time.

After user creation, the custom tenant attribute values can still be updated by an administrator through the AdminUpdateUserAttributes API operation or by a user with the UpdateUserAttributes API operation, if needed. But the key point is that the custom attributes themselves must be predefined at user pool creation, while the values get set later during user creation and provisioning flows. Figure 1 shows how custom attributes are associated with an ID token and used subsequently in downstream applications.

Figure 1: Associating tenant context with custom attributes

As shown in Figure 1:

The custom tenant attribute values from the user profile are included in the Cognito ID token that is generated after a successful user authentication. These values can be used for access control for other AWS services, such as Amazon API Gateway.
You can configure Amazon API Gateway with a Lambda authorizer function that validates the ID token signature (the aws-jwt-verify library can be used for this purpose) and inspects the tenant ID claim in each request.
Based on the tenant ID value extracted from the ID token, the Lambda authorizer can determine which backend resources and services each authenticated user is authorized to access.

You can use this method to provide fine-grained access control, as described in this blog post, by using tenant claims as context in addition to the user claims embedded within the token. This pattern of embedding information about the user’s identity, along with details on their associated tenant, in a single token is what AWS refers to as SaaS identity.

The multi-tenancy approaches of using siloed user pools, shared pools, or custom attributes rely on embedding tenant context within the user identity. This is accomplished by having Cognito include claims with tenant information in the JWTs issued after authentication.

The JWT encodes user identity information like the username, email address, and so on. By adding custom claims that contain tenant identifiers or metadata, the tenant context gets tightly coupled to the user identity. The embedded tenant context in the JWT allows applications to implement access control and authorization based on the associated tenant for each user.

This combination of user identity information and tenant context in the issued JWT represents the SaaS identity—a unified identity spanning both user and tenant dimensions. The application uses this SaaS identity for implementing multi-tenant logic and policies.

Pattern 2: Shared user pool (pool model)

A single, shared Amazon Cognito user pool simplifies identity management for multi-tenant SaaS applications. With one consolidated pool, changes and configurations apply across tenants in one place, which can reduce overhead.

For example, you can define password complexity rules and other settings once at the user pool level, and then these settings are shared across tenants. Adding new tenants is streamlined by using the settings in the existing shared pool, without duplicating setup per tenant. This avoids deploying isolated pools when onboarding new tenants.

Additionally, the tokens issued from the shared pool are signed by the same issuer. There is no tenant-specific issuer in the tokens when using a shared pool. For SaaS apps with common identity needs, a shared multi-tenant pool minimizes friction for rapid onboarding despite that loss of per-tenant customization.

Advantages of the pool model:

This model uses a single shared user pool for tenants. This simplifies onboarding by setting user attributes rather than configuring multiple user pools.
Tenants authenticate using the same application client and user pool, which keeps the SaaS client configuration simple.

Disadvantages of the pool model:

Sharing one pool means that settings like password policies and MFA apply uniformly, without customization per tenant.
Some resource quotas are managed at a user pool level (for example, the number of application clients or customer attributes), so you need to consider quotas carefully when adopting this model.

Pattern 3: Group-based multi-tenancy (pool model)

Amazon Cognito user pools give an administrator the capability to add groups and associate users with groups. Doing so introduces specific attributes (cognito:groups and cognito:roles) that are managed and maintained by Cognito and available within the ID tokens. (Access tokens only have the cognito:groups attribute.) These groups can be used to enable multi-tenancy by creating a separate group for each tenant. Users can be assigned to the appropriate tenant group based on the value of a custom tenantId attribute. The application can then implement authorization logic to limit access to resources and data based on the user’s tenant group membership that is encoded in the tokens. This provides isolation and access control across tenants, making use of the native group constructs in Cognito rather than relying entirely on custom attributes.

The group information contained in the tokens can then be used by downstream services to make authorization decisions. Groups are often combined with custom attributes for more granular access control. For example, in the SaaS Factory Serverless SaaS – Reference Solution developed by the AWS SaaS Factory team, roles are specified by using Cognito groups, but tenant identity relies on a custom tenantId attribute. The tenant ID attribute provides isolation between tenants, while the groups define individual user roles and access privileges that apply within a tenant.

Figure 2 shows how groups are associated with the user and then the Lambda authorizer can determine which backend resources and services each authenticated user is authorized to access.

Figure 2: Group-based multi-tenancy

In this model, groups can provide role-based controls, while custom attributes like tenant ID provide the contextual information needed to enforce tenant isolation. The authorization decisions are then made by evaluating a user’s group memberships and attribute values in order to provide fine-grained access tailored to each tenant and user. So groups directly enable role-based checks, while custom attributes provide broader context for conditional access across tenants. Together they can provide the data that is needed to implement granular authorization in a multi-tenant application.

Advantages of group-based multi-tenancy:

This model uses a single shared user pool for tenants, so that onboarding requires setting user attributes rather than configuring multiple pools.
Tenants authenticate through the same application client and pool, keeping SaaS client configuration straightforward.

Disadvantages of group-based multi-tenancy:

Sharing one pool means that settings like password policies and MFA apply uniformly without per-tenant customization.
There is a limit of 10,000 groups per user pool.

Pattern 4: Dedicated user pool per tenant (silo model)

Another common approach for multi-tenant identity with Cognito is to provision a separate user pool for each tenant. A Cognito user pool is a user directory, so using distinct pools provides maximum isolation. However, this approach requires that you implement tenant routing logic in the application to determine which user pool a user should authenticate against, based on their tenant.

Tenant routing

With separate user pools per tenant (or application clients, as we’ll discuss later), the application needs logic to route each user to the appropriate pool (or client) for authentication. There are a few options that you can use for this approach:

Use a subdomain in the URL that maps to the tenant—for example, tenant1.myapp.com routes to Tenant 1’s user pool. This requires mapping subdomains to tenant pools.
Rely on unique email domains per tenant—for example, @tenant1.com goes to Tenant 1’s pool. This requires mapping email domains to pools.
Have the user select their tenant from a dropdown list. This requires the tenant choices to be configured.
Prompt the user to enter a tenant ID code that maps to pools. This requires mapping codes to pools.

No matter the approach you chose, the key requirements are the following:

A data point to identify the tenant (such as subdomain, email, selection, or code).
A mapping dataset that takes tenant identifying information from the user and looks up the corresponding user pool to route to for authentication.
Routing logic to redirect to the appropriate user pool.

For example, the AWS SaaS Factory Serverless Reference Architecture uses the approach shown in Figure 3.

Figure 3: Dedicated user pool per tenant

The workflow is as follows:

The user enters their tenant name during sign-in.
The tenant name retrieves tenant-specific information like the user pool ID, application client ID, and API URLs.
Tenant-specific information is passed to the SaaS app to initialize authentication to the correct user pool and app client, and this is used to initialize an authorization code flow.
The app redirects to the Cognito hosted UI for authentication.
User credentials are validated, and Cognito issues an OAuth code.
The OAuth code is exchanged for a JWT token from Cognito.
The JWT token is used to authenticate the user to access microservices.

Advantages of the one pool per tenant model:

Users exist in a single directory with no cross-tenant visibility. Tokens are issued and signed with keys that are unique to that pool.
Each pool can have customized security policies, like password rules or MFA requirements per tenant.
Pools can be hosted in different AWS Regions to meet data residency needs.

Potential disadvantages of the one pool per tenant model:

There are limits on the number of pools per account. (The default is 1,000 pools, and the maximum is 10,000.)
Additional automation is required to create multiple pools, especially with customized configurations.
Applications must implement tenant routing to direct authentication requests to the correct user pool.
Troubleshooting can be more difficult, because configuration of each pool is managed separately and tenant routing functionality is added.

In summary, separate user pools maximize tenant isolation but require more complex provisioning and routing. You might also need to consider limits on the pool count for large multi-tenant deployments.

Pattern 5: Application client per tenant (bridge model)

You can achieve some extra tenant isolation by using separate application clients per tenant in a single user pool, in addition to using groups and custom attributes. Cognito configurations from the application client, such as OAuth scopes, hosted UI customization, and security policies can be specific to each tenant. The application client also enables external IdP federation per tenant. However, user pool–level settings, such as password policy, remain shared.

Figure 4 shows how a single user pool can be configured with multiple application clients. Each of those application clients is assigned to a tenant. However, this approach requires that you implement tenant routing logic in the application to determine which application client a tenant should be mapped to (similar to the approach we discussed for the shared user pool). Once the user is authenticated, you can configure Amazon API Gateway with a Lambda authorizer function that validates the ID token signature. Subsequently, the Lambda authorizer can determine which backend resources and services each authenticated user is authorized to access.

Figure 4: Application client based multi-tenancy

For tenants that want to use their own IdP through SAML or OpenID Connect federation, you can create a dedicated application client that will redirect users to authenticate with the tenant’s federated IdP. This has some key benefits:

If a single external IdP is enabled on the application client, the hosted UI automatically redirects users without presenting Cognito sign-in screens. This provides a familiar sign-in experience for tenants and is frictionless if users have existing sessions with the tenant IdP.
Management of user activities like joining and leaving, passwords, and other tasks are entirely handled by the tenant in their own IdP. The SaaS provider doesn’t need to get involved in these processes.

Importantly, even with federation, Cognito still issues tokens after successful external authentication. So the SaaS provider gets consistent tokens from Cognito to validate during authorization, regardless of the IdP.

Attribute mapping

When federating with an external IdP, Amazon Cognito can dynamically map attributes to populate the tokens it issues. This allows attributes like groups, email addresses, and roles created in the IdP to be passed to Cognito during authentication and added to the tokens.

The mapping occurs upon every sign-in, overwriting the existing mapped attributes to stay in sync with the latest IdP values. Therefore, changes made in the external IdP related to mapped attributes are reflected in Cognito after signing in. If a mapped attribute is required in the Cognito user pool, like email for sign-in, it must have an equivalent in the IdP to map. The target attributes in Cognito must be configured as mutable, since immutable attributes cannot be overwritten after creation, even through mapping.

Important: For SaaS identity, tenant attributes should be defined in Cognito rather than mapped from an external IdP. This helps to prevent tenants from tampering with values and maintains isolation. However, user attributes like groups and roles can be mapped from the tenant’s IdP to manage permissions. This allows tenants to configure application roles by using their own IdP groups.

Advantages of the bridge model:

This model enables tenant-specific configuration like OAuth scopes, UI, and IdPs.
Tenant users access familiar workflows through external IdPs, and when using external IdPs, tenant user management is handled externally.
No custom claim mappings are needed, but can be used optionally.
Cognito still issues tokens for authorization.

Disadvantages of the bridge model:

Requires routing users to the correct app client per tenant.
There is a limit on the number of app clients per user pool.
Some user pool settings remain shared, such as password policy.
There is no dynamic group claim modification.

Conclusion

In this blog post, we explored various ways Amazon Cognito user pools can enable multi-tenant identity for SaaS solutions. A single shared user pool simplifies management but limits the option to customize user pool–level policies, while separate pools maximize isolation and configurability at the cost of complexity. If you use multiple application clients, you can balance tailored options like external IdPs and OAuth scopes with centralized policies in the user pool. Custom claim mappings provide flexibility but require additional logic.

These two approaches can also be combined. For example, you can have dedicated user pools for select high-tier tenants while others share a multi-tenant pool. The optimal choice depends on the specific tenant needs and on the customization that is required.

In this blog post, we have mainly focused on a static approach. You can also use a pre-token generation Lambda trigger to modify tokens by adding, changing, or removing claims dynamically. The trigger can also override the group membership in both the identity and access tokens. Other claim changes only apply to the ID token. A common use case for this trigger is injecting tenant attributes into the token dynamically.

Evaluate the pros and cons of each approach against the requirements of the SaaS architecture and tenants. Often a hybrid model works best. Cognito constructs like user pools, IdPs, and triggers provide various levers that you can use to fine-tune authentication and authorization across tenants.

For further reading on these topics, see the Common Amazon Cognito scenarios topic in the Cognito Developer Guide and the related blog post How to Use Cognito Pre-Token Generation trigger to Customize Claims in ID Tokens.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on Amazon Cognito re:Post

Ferris Jabr | Becoming Earth | Talks at Google

2024-08-05 Talks at Google

Post Syndicated from Talks at Google original https://www.youtube.com/watch?v=f2rhpKKlnyI

[$] The complexity of BUSL transformation

2024-08-05 jzb

Post Syndicated from jzb original https://lwn.net/Articles/984249/

The Business
Source License (BUSL) is a source-available license that “converts”
to an open-source license after a period of time. In theory, this
means that a few years after a version of a product is released under
the BUSL, it becomes open source and is fair game for Linux
distributions to package along with regular open-source projects. In
practice, the license throws a few curveballs that require special
consideration and caution, as the Fedora Project recently discussed.

GNU Binutils 2.43 released

2024-08-05 corbet

Post Syndicated from corbet original https://lwn.net/Articles/984539/

Version 2.43 of the GNU Binutils package is out. Changes include some
improvements to the assembler and the linker, better support for hardware
event counters in the Gprofng profiler, and more.

Security updates for Monday

2024-08-05 jake

Post Syndicated from jake original https://lwn.net/Articles/984552/

Security updates have been issued by Debian (openjdk-11), Fedora (bind, bind-dyndb-ldap, chromium, ffmpeg, hostapd, trafficserver, and wpa_supplicant), and Ubuntu (curl and linux-oem-6.5).

Introducing the Rapid7 Command Platform

2024-08-05 Corey Thomas

Post Syndicated from Corey Thomas original https://blog.rapid7.com/2024/08/05/introducing-the-rapid7-command-platform/

Integrated Security Operations for the Next-Generation Attack Surface

Introducing the Rapid7 Command Platform

As cybercrime and attack surfaces have sprawled, Rapid7 has been able to grow with our customers because we are relentlessly focused on relevance. The way we see it, relevance doesn’t mean aligning to market definitions of categories, but rather deeply understanding how critical decisions need to be made and building solutions that provide the right context when and where customers need it.

From our foundations in vulnerability management 24 years ago, we have developed a comprehensive portfolio of security technologies trusted by over 11,000 customers around the globe. We are also home to a team of experts and researchers and the proud curators of one of the most renowned open source security communities – all of which informs our products and services, so customers can have our expertise at their fingertips.

While other vendors launched SIEMs focused on log aggregation and building mounds of data for customers to parse through, we launched our detections-centric next-gen SIEM focused on keeping SOCs ahead of adversaries. As other MDRs became alert-factories pushing the hard work back on the shoulders of their customers, we focused on building high signal-to-noise full environment coverage and end-to-end partnership. And when other exposure management providers stopped at the endpoint, we pushed integrated coverage to the cloud and beyond.

Today, I’m excited to share the next chapter in our mission to give customers command of their attack surface: the introduction of the Rapid7 Command Platform, our unified threat exposure and detection & response platform.

A Relevant View of Your Attack Surface You Can Trust

You cannot deliver what is relevant for customers without listening to customers. Across our industry, we have focused on building purpose-fit products to solve specific customer needs. As a result, even industry-leading products have become high cost, disparate sources of data which must be manually cobbled together for a low-confidence picture of the attack surface.

How can you secure and manage what you cannot see?

Our team has heard this over and over again and we made the commitment to build a better way. With this launch of the Rapid7 Command Platform, we can now deliver a more comprehensive view of your attack surface, with transparency that you can trust.

Anchored by Surface Command, our new unified asset inventory and attack surface management product, customers can get a more complete, vendor agnostic view of their internal and external attack surface—at a disruptive, all-in value.

We begin with a rapidly growing library of over 100 connectors, which collects and unifies data across your security ecosystem. Our AI-charged correlation engine deduplicates and reconciles telemetry across sources to create a cohesive visualization of your environment. While you get an out-of-the-box view, data transparency and detailed metadata drill downs means you can feel confident that you have both a more complete picture and the context you need to accelerate critical security decisions across teams. From the driver’s seat of your attack surface, you can zero-in on controls and policy gaps, discover shadow IT, and gather relevant context that accelerates prioritization and remediation activities. With the total visibility Surface Command provides, you gain total control of your attack surface.

Unified Threat Exposure, Detection, and Response

Reinforced by data silos, security program fundamentals – exposure management and detection & response – have operated in isolation for too long. The Command Platform’s ability to maximize and integrate your security data from endpoint to cloud will finally bring these functions into a single, integrated program that yields better productivity, efficiency, and efficacy for security teams.

Our second new solution on the Command Platform – Exposure Command – brings together the comprehensive visibility of Surface Command with leading vulnerability management and cloud-native application protection to deliver a true end-to-end attack defense. With Exposure Command, InfoSec teams can defend and monitor their risk landscape through the lens of an adversary.

On top of Surface Command’s total visibility, we add the context and clarity of what is actually being exploited in the wild, the items that we know attackers will have in their sights. Understanding what is truly urgent and important transforms endless to-do lists into actionable hit lists. This transparency also gives security leaders the confidence to socialize and articulate potential business impact if these items are not actioned. Married with policy and vulnerability coverage across hybrid environments, unified reporting, and remediation, Exposure Command is your hub for next-generation risk management.

The Command Platform will drive faster and more efficient Detection & Response, with reduced noise and greater conviction around the right actions to take to extinguish threats. We have already made progress towards this vision today with our Threat Complete product, which delivers the enriched context of exposure management alongside leading next-gen SIEM to stay ahead of attackers. The ability to pinpoint and correlate the events, activity, and alerts that actually matter and then know how to respond is the difference of getting breached or not. Whether you’re leveraging our cloud-first technology or partnering with our MDR SOC as an extension of your team, Rapid7 Detection & Response customers can be confident that they have the ecosystem coverage and context they need to be able to respond like an expert every time.

Confidently Take On the Next Era of Security with Rapid7

I’m so grateful to our customers and partners who have been the North Star in guiding us on this mission, and I am proud of the progress that this launch signifies. We are extremely excited to get the Command Platform, Surface Command, and Exposure Command into more hands and continue to improve how we deliver the most relevant security solutions for customers.

Rapid7 is on a path to deliver 100% attack surface visibility and monitoring that customers can afford and rely on, and we have many milestones that are still in front of us. I look forward to sharing more game-changing updates soon.

For those interested in learning more, we’d love to hear from you and share a demo of our new Exposure Command product – sign up here.

How AWS tracks the cloud’s biggest security threats and helps shut them down

2024-08-05 CJ Moses

Post Syndicated from CJ Moses original https://aws.amazon.com/blogs/security/how-aws-tracks-the-clouds-biggest-security-threats-and-helps-shut-them-down/

Threat intelligence that can fend off security threats before they happen requires not just smarts, but the speed and worldwide scale that only AWS can offer.

Organizations around the world trust Amazon Web Services (AWS) with their most sensitive data. One of the ways we help secure data on AWS is with an industry-leading threat intelligence program where we identify and stop many kinds of malicious online activities that could harm or disrupt our customers or our infrastructure. Producing accurate, timely, actionable, and scalable threat intelligence is a responsibility we take very seriously, and is something we invest significant resources in.

Customers increasingly ask us where our threat intelligence comes from, what types of threats we see, how we act on what we observe, and what they need to do to protect themselves. Questions like these indicate that Chief Information Security Officers (CISOs)—whose roles have evolved from being primarily technical to now being a strategic, business-oriented function—understand that effective threat intelligence is critical to their organizations’ success and resilience. This blog post is the first of a series that begins to answer these questions and provides examples of how AWS threat intelligence protects our customers, partners, and other organizations.

High-fidelity threat intelligence that can only be achieved at the global scale of AWS

Every day across AWS infrastructure, we detect and thwart cyberattacks. With the largest public network footprint of any cloud provider, AWS has unparalleled insight into certain activities on the internet, in real time. For threat intelligence to have meaningful impact on security, large amounts of raw data from across the internet must be gathered and quickly analyzed. In addition, false positives must be purged. For example, threat intelligence findings could erroneously indicate an insider threat when an employee is logged accessing sensitive data after working hours, when in reality, that employee may have been tasked with a last-minute project and had to work overnight. Producing threat intelligence is very time consuming and requires substantial human and digital resources. Artificial intelligence (AI) and machine learning can help analysts sift through and analyze vast amounts of data. However, without the ability to collect and analyze relevant information across the entire internet, threat intelligence is not very useful. Even for organizations that are able to gather actionable threat intelligence on their own, without the reach of global-scale cloud infrastructure, it’s difficult or impossible for time-sensitive information to be collectively shared with others at a meaningful scale.

The AWS infrastructure radically transforms threat intelligence because we can significantly boost threat intelligence accuracy—what we refer to as high fidelity—because of the sheer number of intelligence signals (notifications generated by our security tools) we can observe. And we constantly improve our ability to observe and react to threat actors’ evolving tactics, techniques, and procedures (TTPs) as we discover and monitor potentially harmful activities through MadPot, our sophisticated globally-distributed network of honeypot threat sensors with automated response capabilities.

With our global network and internal tools such as MadPot, we receive and analyze thousands of different kinds of event signals in real time. For example, MadPot observes more than 100 million potential threats every day around the world, with approximately 500,000 of those observed activities classified as malicious. This means high-fidelity findings (pieces of relevant information) produce valuable threat intelligence that can be acted on quickly to protect customers around the world from harmful and malicious online activities. Our high-fidelity intelligence also generates real-time findings that are ingested into our intelligent threat detection security service Amazon GuardDuty, which automatically detects threats for millions of AWS accounts.

AWS’s Mithra ranks domain trustworthiness to help protect customers from threats

Let’s dive deeper. Identification of malicious domains (physical IP addresses on the internet) is crucial to effective threat intelligence. GuardDuty generates various kinds of findings (potential security issues such as anomalous behaviors) when AWS customers interact with domains, with each domain being assigned a reputation score derived from a variety of metrics that rank trustworthiness. Why this ranking? Because maintaining a high-quality list of malicious domain names is crucial to monitoring cybercriminal behavior so that we can protect customers. How do we accomplish the huge task of ranking? First, imagine a graph so large (perhaps one of the largest in existence) that it’s impossible for a human to view and comprehend the entirety of its contents, let alone derive usable insights.

Meet Mithra. Named after a mythological rising sun, Mithra is a massive internal neural network graph model, developed by AWS, that uses algorithms for threat intelligence. With its 3.5 billion nodes and 48 billion edges, Mithra’s reputation scoring system is tailored to identify malicious domains that customers come in contact with, so the domains can be ranked accordingly. We observe a significant number of DNS requests per day—up to 200 trillion in a single AWS Region alone—and Mithra detects an average of 182,000 new malicious domains daily. By assigning a reputation score that ranks every domain name queried within AWS on a daily basis, Mithra’s algorithms help AWS rely less on third parties for detecting emerging threats, and instead generate better knowledge, produced more quickly than would be possible if we used a third party.

Mithra is not only able to detect malicious domains with remarkable accuracy and fewer false positives, but this super graph is also capable of predicting malicious domains days, weeks, and sometimes even months before they show up on threat intel feeds from third parties. This world-class capability means that we can see and act on millions of security events and potential threats every day.

By scoring domain names, Mithra can be used in the following ways:

A high-confidence list of previously unknown malicious domain names can be used in security services like GuardDuty to help protect our customers. GuardDuty also allows customers to block malicious domains and get alerts for potential threats.
Services that use third-party threat feeds can use Mithra’s scores to significantly reduce false positives.
AWS security analysts can use scores for additional context as part of security investigations.

Sharing our high-fidelity threat intelligence with customers so they can protect themselves

Not only is our threat intelligence used to seamlessly enrich security services that AWS and our customers rely on, we also proactively reach out to share critical information with customers and other organizations that we believe may be targeted or potentially compromised by malicious actors. Sharing our threat intelligence enables recipients to assess information we provide, take steps to reduce their risk, and help prevent disruptions to their business.

For example, using our threat intelligence, we notify organizations around the world if we identify that their systems are potentially compromised by threat actors or appear to be running misconfigured systems vulnerable to exploits or abuse, such as open databases. Cybercriminals are constantly scanning the internet for exposed databases and other vulnerabilities, and the longer a database remains exposed, the higher the risk that malicious actors will discover and exploit it. In certain circumstances when we receive signals that suggest a third-party (non-customer) organization may be compromised by a threat actor, we also notify them because doing so can help head off further exploitation, which promotes a safer internet at large.

Often, when we alert customers and others to these kinds of issues, it’s the first time they become aware that they are potentially compromised. After we notify organizations, they can investigate and determine the steps they need to take to protect themselves and help prevent incidents that could cause disruptions to their organization or allow further exploitation. Our notifications often also include recommendations for actions organizations can take, such as to review security logs for specific domains and block them, implement mitigations, change configurations, conduct a forensic investigation, install the latest patches, or move infrastructure behind a network firewall. These proactive actions help organizations to get ahead of potential threats, rather than just reacting after an incident occurs.

Sometimes, the customers and other organizations we notify contribute information that in turn helps us assist others. After an investigation, if an affected organization provides us with related indicators of compromise (IOCs), this information can be used to improve our understanding of how a compromise occurred. This understanding can lead to critical insights we may be able to share with others, who can use it to take action to improve their security posture—a virtuous cycle that helps promote collaboration aimed at improving security. For example, information we receive may help us learn how a social engineering attack or particular phishing campaign was used to compromise an organization’s security to install malware on a victim’s system. Or, we may receive information about a zero-day vulnerability that was used to perpetrate an intrusion, or learn how a remote code execution (RCE) attack was used to run malicious code and other malware to steal an organization’s data. We can then use and share this intelligence to protect customers and other third parties. This type of collaboration and coordinated response is more effective when organizations work together and share resources, intelligence, and expertise.

Three examples of AWS high-fidelity threat intelligence in action

Example 1: We became aware of suspicious activity when our MadPot sensors indicated unusual network traffic known as backscatter (potentially unwanted or unintended network traffic that is often associated with a cyberattack) that contained known IOCs associated with a specific threat attempting to move across our infrastructure. The network traffic appeared to be originating from the IP space of a large multinational food service industry organization and flowing to Eastern Europe, suggesting potential malicious data exfiltration. Our threat intelligence team promptly contacted the security team at the affected organization, which wasn’t an AWS customer. They were already aware of the issue but believed they had successfully addressed and removed the threat from their IT environment. However, our sensors indicated that the threat was continuing and not resolved, showing that a persistent threat was ongoing. We requested an immediate escalation, and during a late-night phone call, the AWS CISO shared real-time security logs with the CISO of the impacted organization to show that large amounts of data were still being suspiciously exfiltrated and that urgent action was necessary. The CISO of the affected company agreed and engaged their Incident Response (IR) team, which we worked with to successfully stop the threat.

Example 2: Earlier this year, Volexity published research detailing two zero-day vulnerabilities in the Ivanti Connect Secure VPN, resulting in the publication of CVE-2023-46805 (an authentication-bypass vulnerability) and CVE-2024-21887 (a command-injection vulnerability found in multiple web components). The U.S. Cybersecurity and Infrastructure Security Agency (CISA) issued a cybersecurity advisory on February 29, 2024 on this issue. Earlier this year, Amazon security teams enhanced our MadPot sensors to detect attempts by malicious actors to exploit these vulnerabilities. Using information obtained by the MadPot sensors, Amazon identified multiple active exploitation campaigns targeting vulnerable Ivanti Connect Secure VPNs. We also published related intelligence in the GuardDuty common vulnerabilities and exposures (CVE) feed, enabling our customers who use this service to detect and stop this activity if it is present in their environment. (For more on CVSS metrics, see the National Institute of Standards and Technology (NIST) Vulnerability Metrics.)

Example 3: Around the time Russia began its invasion of Ukraine in 2022, Amazon proactively identified infrastructure that Russian threat groups were creating to use for phishing campaigns against Ukrainian government services. Our intelligence findings were integrated into GuardDuty to automatically protect AWS customers while also providing the information to the Ukrainian government for their own protection. After the invasion, Amazon identified IOCs and TTPs of Russian cyber threat actors that appeared to target certain technology supply chains that could adversely affect Western businesses opposed to Russia’s actions. We worked with the targeted AWS customers to thwart potentially harmful activities and help prevent supply chain disruption from taking place.

AWS operates the most trusted cloud infrastructure on the planet, which gives us a unique view of the security landscape and the threats our customers face every day. We are encouraged by how our efforts to share our threat intelligence have helped customers and other organizations be more secure, and we are committed to finding even more ways to help. Upcoming posts in this series will include other threat intelligence topics such as mean time to defend, our internal tool Sonaris, and more.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.

Rapid7 Introduces Exposure Command to Eliminate the Security Visibility Gap

2024-08-05 Rapid7

Post Syndicated from Rapid7 original https://blog.rapid7.com/2024/08/05/rapid7-introduces-exposure-command-to-eliminate-the-security-visibility-gap/

Exposure Command provides 360-degree visibility and enables security teams to pinpoint and extinguish your most critical risks.

Rapid7 Introduces Exposure Command to Eliminate the Security Visibility Gap

Security and IT teams are experiencing a significant (changed from “seismic” for clarity) shift in operations as they become more distributed. Development and procurement processes have decentralized, and sensitive data now extends far beyond the network edge. This expansion, coupled with growth and innovation outpacing security investments, has led to a significant “security visibility gap.”

Disparate tools widen this gap, creating data silos and inconsistencies, leading to manual efforts and swivel-charing to manually correlate conflicting findings and dashboards. This situation has been exacerbated by broader industry trends. Gartner estimates that through 2026, ‘unpatchable’ attack surfaces will grow from less than 10% to more than half of the enterprise’s total exposure, reducing the effectiveness of traditional vulnerability management programs.

Security teams need to manage and interpret a broad range of different exposure types – cloud misconfigurations, user entitlements, unmanaged machines, vulnerabilities, etc. with conflicting and duplicate data from various different security and IT management tools with varying levels of data fidelity.

The only way to truly solve this problem is to implement a solution that treats third-party data as a first-class citizen, bringing together telemetry from all of your security tools to build a complete picture of your environment, and thereby your attack surface. To that end, Rapid7 announced the launch of two exciting new product offerings designed to unify your attack surface and deliver effective hybrid risk management: Surface Command and Exposure Command.

Surface Command closes the visibility gap by breaking down data silos, combining internal and external monitoring to build a 360-degree view of your entire environment, combining market leading Cyber Asset Attack Surface Management (CAASM) and External Attack Surface Management (EASM) capabilities into one unified offering.

External scans provide an adversary’s perspective on the attack surface, detecting and validating exposures. Surface Command combines these external scans with a detailed inventory of your internal assets, continuously ingested and updated from a wide range of security and IT tools and automatically correlates the assets to create detailed inventory of your ‘true’ attack surface, highlighting security control gaps.

This process delivers a comprehensive view of your environment that teams across the organization can trust and align on as a ‘single source of truth’ without the risk of blind spots, unprotected assets, and ungoverned access. Understanding how all your interconnected assets are configured enables you to quickly identify and prioritize high-risk vulnerabilities, shadow IT, and compliance issues. With this more comprehensive visibility serving as the foundation of our Command Platform, security teams have a view of their attack surface they can trust and action across their wider organization.

Automatically prioritize exposures across your hybrid environment with Exposure Command

Exposure Command extends the power of Surface Command even further, combining the same unified attack surface visibility with high-fidelity environment detail and risk context to help teams to zero-in on the exposures and vulnerabilities that attackers have in their sights with the threat-aware risk context needed to prioritize more efficiently and effectively.

With Exposure Command, every asset in your environment is enriched with relevant context from all of Rapid7’s exposure management capabilities, including our industry-leading VM, CNAPP and AppSec solutions, which provides teams an understanding of which assets are most critical to the business and those that suffer from toxic combinations that leave the organization vulnerable to a security incident.

This situational awareness allows teams to more effectively prioritize response efforts by honing in on the vulnerabilities that are either being actively exploited in the wild and/or those that present the most risk should a compromise occur.

Prioritization is critical, especially when you consider the massive volume of risk signals produced by modern cloud-native environments on a daily basis. It’s simply not feasible to expect to address everything, so making sure that teams are spending the time they do have on the actions that will have the greatest impact on reducing their overall risk posture and eliminating critical exposures is key.

When it comes to prioritization, there are three primary vectors that we need to consider: Opportunity, Likelihood and Impact of exploitation.

Opportunity – The first step in prioritizing exposures is to understand whether a threat actor could exploit the issue in the first place by analyzing the downstream security controls and mechanisms in place – or not in place for that matter. This includes considering whether or not a resource is publicly accessible, if there are additional mitigating controls like web application or network firewalls, if an at-risk asset has an endpoint protection solution installed, etc.
Likelihood – It’s important to understand how likely it is that an attacker would exploit a given exposure. This can be accomplished in a number of ways, including focusing on CVEs on CISA’s Known Exploited Vulnerabilities (KEV) list, but also involves looking at real-world activity via threat intelligence feeds – like those that feed into Rapid7’s Active Risk score – to get a sense for whether a vulnerability is being exploited elsewhere.
Impact – Taking into account the business criticality of the asset, data or system, what would be the relevant impact should a given risk signal be exploited by a threat actor. This is often accomplished by assigning tags that flag whether or not a given resource is associated with a business critical application or is housing sensitive customer data.

To this end, a new feature coming to the Command Platform with Exposure Command, Remediation Hub, automatically surfaces the top areas teams need to focus on and elevates the mitigation activities that would have the largest impact in reducing the overall risk score of your environment along with any relevant contextual information to assist in validation and remediation efforts.

After 24+ years in exposure management, we are excited to partner with customers through the next era of the attack surface and hybrid risk with our new Exposure Command product. This is just the beginning. Stay tuned here for more updates as we continue to grow our Command Platform.

Learn more about Surface Command and Exposure Command

Attending Black Hat? Come see us at booth #2436 to get a one on one tour! If you can’t make it to the event you can also find additional information on the docs page, or give us a bit of information and we’ll have a member of the team reach out directly.

How To Bypass AT&T Fiber BGW320 w/ WAS-110 SFP+

2024-08-05 digiblur DIY

Post Syndicated from digiblur DIY original https://www.youtube.com/watch?v=3rIsq8tW8js

The Attempted Assassination of President Truman, 1950.

2024-08-05 The History Guy: History Deserves to Be Remembered

Post Syndicated from The History Guy: History Deserves to Be Remembered original https://www.youtube.com/watch?v=JwtQT5Ji_ZQ

New Patent Application for Car-to-Car Surveillance

2024-08-05 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2024/08/new-patent-application-for-car-to-car-surveillance.html

Ford has a new patent application for a system where cars monitor each other’s speeds, and then report then to some central authority.

Slashdot thread.

S11 E19: RFK Jr., The Olympics & Trump at the NABJ: 8/4/24: Last Week Tonight with John Oliver

2024-08-05 LastWeekTonight

Post Syndicated from LastWeekTonight original https://www.youtube.com/watch?v=7TNAyvx-ymI

Kernel prepatch 6.11-rc2

2024-08-05 corbet

Post Syndicated from corbet original https://lwn.net/Articles/984501/

Linus has released 6.11-rc2 for testing.
“Hopefully we’ve gotten rid of the bulk of the silly noise here in rc2,
and not added too much new noise, so that we can get on with the process of
finding more meaningful issues.“

Napoleon’s Invasion of Malta

2024-08-05 The History Guy: History Deserves to Be Remembered

Post Syndicated from The History Guy: History Deserves to Be Remembered original https://www.youtube.com/watch?v=f6YM76WwmXA

How we improved translation experience with cost efficiency

2024-08-05 Grab Tech

Post Syndicated from Grab Tech original https://engineering.grab.com/improved-translation-experience-with-cost-efficiency

Introduction

As COVID restrictions were fully lifted in 2023, the number of tourists grew dramatically. People began to explore the world again, frequently using the Grab app to make bookings outside of their home country. However, we noticed that communication posed a challenge for some users. Despite our efforts to integrate an auto-translation feature in the booking chat, we received feedback about occasional missed or inaccurate translations. You can refer to this blog for a better understanding of Grab’s chat system.

An example of a bad translation. The correct translation is: ‘ok sir’.

In an effort to enhance the user experience for travellers using the Grab app, we formed an engineering squad to tackle this problem. The objectives are as follows:

Ensure translation is provided when it’s needed.
Improve the quality of translation.
Maintain the cost of this service within a reasonable range.

Ensure translation is provided when it’s needed

Originally, we relied on users’ device language settings to determine if translation is needed. For example, if both the passenger and driver’s language setting is set to English, translation is not needed. Interestingly, it turned out that the device language setting did not reliably indicate the language in which a user would send their messages. There were numerous cases where despite having their device language set to English, drivers sent messages in another language.

Therefore, we needed to detect the language of user messages on the fly to make sure we trigger translation when it’s needed.

Language detection

Simple as it may seem, language detection is not that straightforward a task. We were unable to find an open-source language detector library that covered all Southeast Asian languages. We looked for Golang libraries as our service was written in Golang. The closest we could find were the following:

Whatlang: unable to detect Malay
Lingua: unable to detect Burmese and Khmer

We decided to choose Lingua over Whatlang as the base detector due to the following factors:

Overall higher accuracy.
Capability to provide detection confidence level.
We have more users using Malay than those using Burmese or Khmer.

When a translation request comes in, our first step is to use Lingua for language detection. If the detection confidence level falls below a predefined threshold, we fall back to call the third-party translation service as it can detect all Southeast Asian languages.

You may ask, why don’t we simply use the third-party service in the first place. It’s because:

The third-party service only has a translate API that also does language detection, but it does not provide a standalone language detection API.
Using the translate API is costly, so we need to avoid calling it when it’s unnecessary. We will cover more on this in a later section.

Another challenge we’ve encountered is the difficulty of distinguishing between Malay and Indonesian languages due to their strong similarities and shared vocabulary. The identical text might convey different meanings in these two languages, which the third-party translation service struggles to accurately detect and translate.

Differentiating Malay and Indonesian is a tough problem in general. However, in our case, the detection has a very specific context, and we can make use of the context to enhance our detection accuracy.

Making use of translation context

All our translations are for the messages sent in the context of a booking or order, predominantly between passenger and driver. There are two simple facts that can aid in our language detection:

Booking/order happens in one single country.
Drivers are almost always local to that country.

So, for a booking that happens in an Indonesian city, if the driver’s message is detected as Malay, it’s highly likely that the message is actually in Bahasa Indonesia.

Improve quality of translation

Initially, we were entirely dependent on a third-party service for translating our chat messages. While overall powerful, the third-party service is not perfect, and it does generate weird translations from time to time.

An example of a weird translation from a third-party service recorded on 19 Dec 2023.

Then, it came to us that we might be able to build an in-house translation model that could translate chat messages better than the third-party service. The reasons being:

The scope of our chat content is highly specific. All the chats are related to bookings or orders. There would not be conversations about life or work in the chat. Maybe a small Machine Learning (ML) model would suffice to do the job.
The third-party service is a general translation service. It doesn’t know the context of our messages. We, however, know the whole context. Having the right context gives us a great edge on generating the right translation.

Training steps

To create our own translation model, we took the following steps:

Perform topic modelling on Grab chat conversations.
Worked with the localisation team to create a benchmark set of translations.
Measured existing translation solutions against benchmarks.
Used an open source Large Language Model (LLM) to produce synthetic training data.
Used synthetic data to train our lightweight translation model.

Topic modelling

In this step, our aim was to generate a dataset which is both representative of the chat messages sent by our users and diverse enough to capture all of the nuances of the conversations. To achieve this, we took a stratified sampling approach. This involved a random sample of past chat conversation messages stratified by various topics to ensure a comprehensive and balanced representation.

Developing a benchmark

For this step we engaged Grab’s localisation team to create a benchmark for translations. The intention behind this step wasn’t to create enough translation examples to fully train or even finetune a model, but rather, it was to act as a benchmark for translation quality, and also as a set of few-shot learning examples for when we generate our synthetic data.

This second point was critical! Although LLMs can generate good quality translations, LLMs are highly susceptible to their training examples. Thus, by using a set of handcrafted translation examples, we hoped to produce a set of examples that would teach the model the exact style, level of formality, and correct tone for the context in which we plan to deploy the final model.

Benchmarking

From a theoretical perspective there are two ways that one can measure the performance of a machine translation system. The first is through the computation of some sort of translation quality score such as a BLEU or CHRF++ score. The second method is via subjective evaluation. For example, you could give each translation a score from 1 to 5 or pit two translations against each other and ask someone to assess which they prefer.

Both methods have their relative strengths and weaknesses. The advantage of a subjective method is that it corresponds better with what we want, a high quality translation experience for our users. The disadvantage of this method is that it is quite laborious. The opposite is true for the computed translation quality scores, that is to say that they correspond less well to a human’s subjective experience of our translation quality, but that they are easier and faster to compute.

To overcome the inherent limitations of each method, we decided to do the following:

Set a benchmark score for the translation quality of various translation services using a CHRF++ score.
Train our model until its CHRF++ score is significantly better than the benchmark score.
Perform a manual A/B test between the newly trained model and the existing translation service.

Synthetic data generation

To generate the training data needed to create our model, we had to rely on an open source LLM to generate the synthetic translation data. For this task, we spent considerable effort looking for a model which had both a large enough parameter count to ensure high quality outputs, but also a model which had the correct tokenizer to handle the diverse sets of languages which Grab’s customers speak. This is particularly important for languages which use non-standard character sets such as Vietnamese and Thai. We settled on using a public model from Hugging Face for this task.

We then used a subset of the previously mentioned benchmark translations to input as few-shot learning examples to our prompt. After many rounds of iteration, we were able to generate translations which were superior to the benchmark CHRF++ scores which we had attained in the previous section.

Model fine tuning

We now had one last step before we had something that was production ready! Although we had successfully engineered a prompt capable of generating high quality translations from the public Hugging Face model, there was no way we’d be able to deploy such a model. The model was far too big for us to deploy it in a cost efficient manner and within an acceptable latency. Our solution to this was to fine-tune a smaller bespoke model using the synthetic training data which was derived from the larger model.

These models were language specific (e.g. English to Indonesian) and built solely for the purpose of language translation. They are 99% smaller than the public model. With approximately 10 Million synthetic training examples, we were able to achieve performance which was 98% as effective as our larger model.

We deployed our model and ran several A/B tests with it. Our model performed pretty well overall, but we noticed a critical problem: sometimes, numbers got mutated in the translation. These numbers can be part of an address, phone number, price etc. Showing the wrong number in a translation can cause great confusion to the users. Unfortunately, an ML model’s output can never be fully controlled; therefore, we added an additional layer of programmatic check to mitigate this issue.

Post-translation quality check

Our goal is to ensure non-translatable content such as numbers, special symbols, and emojis in the original message doesn’t get mutated in the translation produced by our in-house model. We extract all the non-translatable content from the original message, count the occurrences of each, and then try to match the same in the translation. If it fails to match, we discard the in-house translation and fall back to using the third-party translation service.

Keep cost low

At Grab, we try to be as cost efficient as possible in all aspects. In the case of translation, we tried to minimise cost by avoiding unnecessary on-the-fly translations.

As you would have guessed, the first thing we did was to implement caching. A cache layer is placed before both the in-house translation model and the third-party translation. We try to serve translation from the cache first before hitting the underlying translation service. However, given that translation requests are in free text and can be quite dynamic, the impact of caching is limited. There’s more we need to do.

For context, in a booking chat, other than the users, Grab’s internal services can also send messages to the chat room. These messages are called system messages. For example,our food service always sends a message with information on the food order when an order is confirmed.

System messages are all fairly static in nature, however, we saw a very high amount of translation cost attributed to system messages. Taking a deeper look, we noticed the following:

Many system messages were not sent in the recipient’s language, thus requiring on-the-fly translation.
Many system messages, though having the same static structure, contain quite a few variants such as passenger’s name and food order item name. This makes it challenging to utilise our translation cache effectively as each message is different.

Since all system messages are manually prepared, we should be able to get them all manually translated into all the required languages, and avoid on-the-fly translations altogether.

Therefore, we launched an internal campaign, mandating all internal services that send system messages to chat rooms to get manual translations prepared, and pass in the translated contents. This alone helped us save roughly US$255K a year!

Next steps

At Grab, we firmly believe that our proprietary in-house translation models are not only more cost-effective but cater more accurately to our unique use cases compared to third-party services. We will focus on expanding these models to more languages and countries across our operating regions.

Additionally, we are exploring opportunities to apply learnings of our chat translations to other Grab content. This strategy aims to guarantee a seamless language experience for our rapidly expanding user base, especially travellers. We are enthusiastically looking forward to the opportunities this journey brings!

Join us

Grab is the leading superapp platform in Southeast Asia, providing everyday services that matter to consumers. More than just a ride-hailing and food delivery app, Grab offers a wide range of on-demand services in the region, including mobility, food, package and grocery delivery services, mobile payments, and financial services across 428 cities in eight countries.

Powered by technology and driven by heart, our mission is to drive Southeast Asia forward by creating economic empowerment for everyone. If this mission speaks to you, join our team today!

University Age

2024-08-05 xkcd.com

Post Syndicated from xkcd.com original https://xkcd.com/2968/

This only makes it more urgent that we adopt my roadmap for the next 10 years, which should put us solidly in the lead.

Kioxia FL6 800GB Write Intensive SSD Review

2024-08-04 Eric Smith

Post Syndicated from Eric Smith original https://www.servethehome.com/kioxia-fl6-800gb-write-intensive-ssd-review/

In our Kioxia FL6 review we see how this 800GB NVMe SSD is designed to handle heavy and consistent write workloads with its XL-FLASH NAND

The post Kioxia FL6 800GB Write Intensive SSD Review appeared first on ServeTheHome.

Еретични мисли за професионалния спорт, олимпиадата и какво ни показа Юсуф Дикеч

2024-08-04 VassilKendov

Post Syndicated from VassilKendov original https://kendov.com/%D0%B5%D1%80%D0%B5%D1%82%D0%B8%D1%87%D0%BD%D0%B8-%D0%BC%D0%B8%D1%81%D0%BB%D0%B8-%D0%B7%D0%B0-%D0%BF%D1%80%D0%BE%D1%84%D0%B5%D1%81%D0%B8%D0%BE%D0%BD%D0%B0%D0%BB%D0%BD%D0%B8%D1%8F-%D1%81%D0%BF%D0%BE/

Еретични мисли за професионалния спорт, олимпиадата и какво ни показа Юсуф Дикеч

От доста време се каня да изложа еретичните си мисли за професионалния спорт, който гледаме и ползите от него за спортуващите и обществото. Предпоагам, че повечето хора няма да се съгласят, но то не е и нужно. Други пък изобщо няма да разберат, че говоря предимно за професионалния спорт, а не за масовия, но и това е нормално. Така или наче ще има хора, които ще успеят да хванат тезата, че към днешна дата ПРОФЕСИОНАЛНИЯТ СПОРТ носи позитиви само на спортиста и на организаторите на професионалните спортни събития.

Предполагам повечето от вас са чували за феномена на златната треска. Най-много пари от нея са заработили не залтотърсачите, а хората продаващи кирки и лопати, както и не безизвестния Леви Щраус, който прави дънките световно известно облекло, въпреки че основното им качество е зравината, заради което с апредпочитани от златотърсачите като работно облекло.

Така се получава и с професионалния спорт. На принципа на пирамидата – единици стават шампиони и успяват да осребрят усилията си, а милионите други остават с контузии, загубено време за трупане на други знания, утехата че са сформирали характер, който да им помага в живота като сладкари (примерно) или радостта, че са обиколили 10-тина държави по състезания и турнири.

Не така стои въпросът с всикчи останали, печелещи от техните щастието или нещастието на професионалните спортисти. Те наистина печелят. Ето някои сфери които наистина правят пари:

Собственици на спортни съоражения.

Без съмнение всеки спортист има нужда от спецални условия за да постигне „високо спортно майсторство“. Който се е занимавал със спорт знае, че лагерите и турнирите са неизбежна част от подготовката. Те обаче не са безплатни и отнемат доста време за пътувания. Моите наблюдения са, че последните години спортните клубове до които съм се докосвал, все по-често организират такива школи и турнири. Понякога за тях плащат спонсори, но това е много рядко. Повечето пъти кандидат професионалните състезатели плащат сами. Дори хореографските групи и групите по йога започнаха да правят лагер школи. Не ме разбирайте погрешно, но това са пари, които отиват при собствениците на съоръженията и огранизаторите, което също не е нещо ненормално, но за целта на анализа трябва да бъде отчетено. Все пак говорим за пари и ползи от професионалния спорт.
Преди време се бях зачел за кариерата на Григор Димитров и по-скоро за финансирането на подготовката му. Само да знаете, че човекът и семейството му бяха потънали в дългове до момента, в който не става Топ 3 в световната ранглиста. Това се случва през 2017. А можеше и да не се случи, както при много други.
За да стигне до това 3-то място, той посещава редица тенис академии и лагери, които определено не са безплатни. Преставяте ли си колко може да струва шведската академия „Good to Great“, която посещава през 2012-та. А треньори като Петер Лундгрен, Питър Макнамара и самият Патрик Муратоглу дали им е плащал една минимална работна заплата? Сино се съмнявам.

С две думи за да постигнеш върхови спортни успехи в днешно време, трябва да инвестираш страшно много пари. Преди 40 години не беше точно така и дори имаше изискване на олимпиадите да не участват професионални състезатели. В днешно време с масовизирането на спорта, медиите и пътуванията, парите за подготовка са неразделна част от бизнеса.

Телевизионни права

Професионалният спорт не би се развил толкова без медиите. Ако нямаше телевизии, зрителите не биха били толкова облъчвани. Знаете какво е, когато се предва футболен финал от някой турнир. Освен, че се бяга от работа за такова нещо, всички заведения слагат големи телевизори, та барем някоя агитка дойде да гледа мача при тях и остави някой лев.
Не казвам, че е лошо в никакъв случай, но дали сме наясно, че някъде в Б футболна група някой младеж се надява да стане звезда като Роналдо и влага цалото си време и енергия в това начинане при положение, че статистически това е много малко вероятно да се случи. Но пък финансовия и времеви ресурс са инвестирани и някой е спечелил от това.
Така или иначе телевизионните права са един от най-сигурните приходи на организаторите на спортни събития. За Олимпиадата в Париж, телевизионните права се очаква да донесат 3.3 млрд. долара на организаторите. Добре е нали? На фона на разходи от 10 млр. долара, 33% да са от ТВ права не е зле. Вижте как са се движили през годините приходите от ТВ права.

Атина 2004 – 1,5 млрд. долара
Пекин 2008 – 1,7 млрд. долара
Лондон 2012 – 2,6 млрд. долара
Рио 2016 – 2,9 млрд. долара
Токио 2020 – 3,1 млрд. долара

Имайте предвид, че са продадени над 8 млн. билета с очаквани приходи от едва 1.4 млр. долара. Тоест едва 14% от приходите се очаква да бъдат от зрители на място (и то ако няма печалба за организаторите). С други думи приходите от ТВ права са с около 140% по-високи от приходите от посетители на място.

продължава след бланката за среща
Моля използвайте приложената форма за записване на час за среща
[contact-form-7]

Мисля, че няма смисъл да давам повече примери кой и как печели от спорта. Със сигурност най-малкия % печалба е за състезателите, а рискът да не успеят да пробият, клони към 100%. Разходите им за подготовка обаче са константа и то никак не малка, на фона на личните финанси и бюджети.

Юсуф Дикеч – новият символ на „олимпийския дух“

Харизматичен е Юсуф Дикеч – не може да се отрече. Скромен, без излишни емоции, с ясна полова ориентация… Качества които не са много толерирани или насърчавани от МОК. Поне съдейки по откриването на олимпиадата и всички скандали около нея. Но пък тези качества се харесват на хората.

Тук обаче има една подробност – тези качества се харесват, защото Юсуф стана шампион! Лицемерието в западното бщество е толкова голямо, че ако не беше се класирал на второ място, всички биха неглижирали тези му качества. Нека не забравяме, че в реалния живот той работи в сервиз – професия която не е най-уважаваната. С други думи „с тези качества – толкова“. Ако те не му бяха донесли второ място на олимпиада, никой никога не би им обърнал внимание, защото обществото не им обръща внимание и не ги цени. Цени ги само ако донесат някакви позитиви. Можду другото през 2016 на Олимпиадата в Рио, на същата дисциплина шампион става виетнамецът Hoang Xuan Vinh (не знам как се пише това на български). Същите качества, липса на допълнително оборудване, нооо никакво отразяване по медии и социални мрежи. Ето линк ако не вярвате – https://www.youtube.com/watch?v=d-Fv682oMZg

Мислих да опиша опиша съдбите и живота на няколко олимпийски шампиони, които познавам лично, след спиране на активната им спортна дейност, както и да ги сравня с живота на спртнит еминистри по което време са станали шампиони, но ще стане прекалено дълго. Мисля, че всички се сещате какви биха били изводите и разликите.

Не мога обаче да се сдържа и да спомена Гълъбин Боевски. Освен че лежа в затвор в Бразилия за трафик на наркотици, в момента пак е на топло по същата причина, само че в Португалия. Синът му пък беше задържат за разпространение на наркотици.
По-горе уточнихме колко се инвестира в един шампион и къде отива печалбата от него. Надали би се занимавал с разпространение на наркотици, ако можеше да печели достъчно от вдигане на щанги, след прекратяване на активна спортна дейност. Но как да печелиш от друго, като по времето когато е трябвало да формираш други умения, ти си бил по лагери и състезания?
Верно е, че други олимпийски шампиони се справят доста по-добре, но винаги в сферата на спорта, създавайки надежди в ново поколение, което пък инвестира в професионалния спорт. Тоест стават част от същата система, генерираща все повече приходи за школи, организатори на състезания, телевизии…

Без значение дали го разбираме или не, за всичко това е виновно финансирането. Ако в това общество се правят пари от професионален спорт, то медии, школи, комитети, клубове, стадиони и подготвителни центрове… десетки посредници по веригата ще правят пари от него.
От масовия спорт пари не се правят. Там хората спортуват за здраве, а това не изисква много разходи. Чувал ли е някой за победителя в ученическите първенства на джудо в Швейцария? Няма и да чуем. Ще чуем само ако е нещо „по-нестандартно“, да не кажа извратено. Примерно както в САЩ, ученическите първенства по борба бяха спечелени от момче, определило се като момиче или как във Финландия се е провело първенство по Hobby Horse Competition https://www.youtube.com/watch?v=RGnpWXtI3Q0

Като финал мога да кажа само – спортувайте за здраве! Професионалния спорт е по-скоро бизнес, а не здраве. Бизнес в който Вие сте продукта, в който търговеца обира печалбата от Вас!

Васил Кендов
Специалист по лични финанси
Планирайте за да не страдате

Моля използвайте приложената форма за записване на час за среща
[contact-form-7]

The post Еретични мисли за професионалния спорт, олимпиадата и какво ни показа Юсуф Дикеч appeared first on Kendov.com.

Solution overview

Prerequisites

Member account 2 configuration

Set up Lake Formation

Create AWS Glue resources

Register the S3 bucket location

Set cross-account version

Add permissions required for cross-account access

Enable IAM Identity Center integration for Lake Formation

Enable Lake Formation permissions for databases

Grant database permissions to IAM Identity Center groups

Grant table permissions to IAM Identity Center groups

Column filter

Row filter

Member account 1 configuration

Set up Redshift Spectrum table access for the IAM Identity Center group

Validate Redshift Spectrum access as an IAM Identity Center user

Subscribe to QuickSight with IAM Identity Center

Enable trusted identity propagation in QuickSight

Test your IAM Identity Center and Amazon Redshift integration with QuickSight

Clean up

Conclusion

About the Authors

Cognito overview

Multi-tenancy approaches

Pattern 1: Representing SaaS identity with custom attributes

Pattern 2: Shared user pool (pool model)

Pattern 3: Group-based multi-tenancy (pool model)

Pattern 4: Dedicated user pool per tenant (silo model)

Tenant routing

Pattern 5: Application client per tenant (bridge model)

Attribute mapping

Conclusion

Integrated Security Operations for the Next-Generation Attack Surface

A Relevant View of Your Attack Surface You Can Trust

Unified Threat Exposure, Detection, and Response

Confidently Take On the Next Era of Security with Rapid7

High-fidelity threat intelligence that can only be achieved at the global scale of AWS

AWS’s Mithra ranks domain trustworthiness to help protect customers from threats

Sharing our high-fidelity threat intelligence with customers so they can protect themselves

Three examples of AWS high-fidelity threat intelligence in action

Exposure Command provides 360-degree visibility and enables security teams to pinpoint and extinguish your most critical risks.

Unlock complete attack surface visibility to eliminate blind spots and uncover control gaps with Surface Command

Automatically prioritize exposures across your hybrid environment with Exposure Command

Learn more about Surface Command and Exposure Command

Introduction

Ensure translation is provided when it’s needed

Language detection

Making use of translation context

Improve quality of translation

Training steps

Topic modelling

Developing a benchmark

Benchmarking

Synthetic data generation

Model fine tuning

Post-translation quality check

Keep cost low

Next steps

Join us

The collective thoughts of the interwebz