„Той е моя вяра, мой свят, болест моя, лекар мой.“ Хомосексуалност и ислям (продължение)

Post Syndicated from Атанас Шиников original https://www.toest.bg/toy-e-moya-vyara-moy-svyat-bolest-moya-lekar-moy-homoseksualnost-i-islyam-produlzhenie/

<< Към първа част

… И все пак каква е историческата гледна точка към хомосексуалните практики в мюсюлманския свещен закон?

„Той е моя вяра, мой свят, болест моя, лекар мой.“ Хомосексуалност и ислям (продължение)

Сложна, но не безкрайно противоречива, ако предприема логически непоследователния ход на представяне на извод преди аргумента. Впрочем понятие като „хомосексуалност“ не съществува в мюсюлманската традиция. Основният термин е лиуат, а пък за онзи, който го практикува, се изработва понятието лути, оттук и глаголът талаууата, тоест „постъпвам като народа на Лут“. Те имат корени в текста на Корана, където се говори за греха на „народа на Лут“ и последвалото му унищожение. 

Лут е кораничният еквивалент на библейската фигура на Лот от разказа за Содом и Гомор в Битие 19 в Библията. В мюсюлманското Свещено писание се появява на няколко места, например 7:80, където се говори за народа на Лут и „скверността [фахиша], която преди вас не е сторвал никой народ“. Въпросната „скверност“ откриваме и в 27:54 или пък в 29:28. Без изрично да се указва от какъв характер е, историческата правна и богословска традиция недвусмислено развива възгледа, че става въпрос за практикуване на содомия между мъже. Кораничният разказ задава и тона на предписаното наказание за прегрешението, доколкото именно „ураган от камъни“ (54:34) и „порой камъни от глина“ (15:74) е средството, чрез което Аллах наказва народа на Лут.

Сборниците с преданията на Пророка, т.нар. Сунна, като друг основен източник на правно нормотворчество, също обрисуват практиката със силно негативни краски. Там „народът на Лут“ се появява сред заръките на Пророка в тематичните раздели, посветени на онази особена категория наказания, обозначавани с термина хадд, букв. „граница“ – познатите ни отрязвания на ръката на крадеца, пребиване с камъни за прелюбодейство, екзекуция при метеж срещу управника и пр. Достатъчно красноречиви са опасенията на Пророка: „… нещото, от което най-много се страхувам за общността ми, са делата на народа на Лут.“ Относно прегрешилите пък той заръчва:

… убийте с камъни онзи, който е отгоре, както и онзи, който е отдолу, както онзи, който го върши, така и онзи, комуто го вършат. 

А както знаем, поне от времето на прадядото на мюсюлманската юриспруденция Аш-Шафии от VIII–IX век, основните стълбове на мюсюлманското право са Корана и Сунната. От тях чрез техники като аналогията (кийас) и консенсуса на богословите (иджма‘) се извличат приложими в различни контексти постановки. На този принцип и до днес се конструират и отговорите на богословите и правистите в големите портали за онлайн консултации (фатауа). И тук се случва нещо подобно – чрез аналогия с прегрешението на прелюбодеянието (зина), тоест сексуални отношения извън легитимната рамка на брака или с робини, хомосексуалната практика се разглежда като подлежаща на същото наказание. И то традиционно е пребиване с камъни. По силата на предадени истории около зетя на Пророка Али ибн Аби Талиб и неговия близък съратник и пръв халиф Абу Бакр се допуска и изгаряне или хвърляне от високо. 

Аналогията между прелюбодеянието и хомосексуалния акт е стандартна. Големите мюсюлмански правни школи се придържат основно към нея, при все че съществуват известни нюанси – за някои течения смъртно наказание се полага при всички случаи, други въвеждат разграничаване в зависимост от това дали извършителите са женени, или не. И накрая идват тези, според които наказанието може да бъде смекчено. Вместо санкция от категорията хадд, която не подлежи на оспорване, се предписва наказание като бой с камшик от по-меката категория та‘зир, тоест по преценка на шариатския съдия (кади). Така смята например Ибн Хазм от Кордоба през XI век. А той, въпреки че принадлежи към една от най-буквалистичните школи в мюсюлманското право, си позволява доста фриволни асоциация по темата за любовта между мъжете в най-известното си литературно съчинение с поетичното заглавие „Пръстенът на гълъбицата“. 

По смисъла на така зададената терминологична и нормативна рамка, предмет на регулация са хомосексуалните отношения между мъже, но най-вече самият акт на съвкупление. И те винаги се разглеждат в силно негативна светлина. Отношенията между жени рядко стават предмет на анализ и също касаят самия акт, който бива обозначен с друг термин – сихак или мусахака. Той, подобно на лиуат, впоследствие придобива и по-общо значение и започва да се употребява в смисъла на лесбийство. 

Но как гледат днешните богослови на постъпилите към тях запитвания от мюсюлмани?

Какво е наказанието за хомосексуалност [лиуат] и има ли разлика между онзи, който го върши, и комуто го вършат? –

задава въпроса си един мюсюлманин в портала на влиятелния богослов Мухаммад Салих ал-Мунаджжид,

Подобен тип запитвания и отговорите им са златна мина. Тук отговорът на шейха е подробен, затова започва с кратко резюме. Ако искате, четете само резюмето, както при османските мюфтии, които отговарят единствено с „Да, може“ или „Не, не може“, без да са длъжни да се обясняват. 

Резюмето е кратко и недвусмислено – всички авторитетни съратници на Пророка, твърди богословът, са напълно съгласни, че извършителят трябва да бъде убит. Само че, казва, мненията им се различават по начина на екзекуцията. Някои, измежду които е и Абу Бакр, верният приятел на Пророка, твърдят, че престъпникът трябва да бъде изгорен с огън. Други пък препоръчват, че трябва да бъде хвърлен от високо, а пък трети – че трябва да бъде убит чрез замеряне с камъни. 

Но след времето на Пророка правистите развиват по-детайлни мнения. Някои казват, че при всички случаи трябва да бъде убит, независимо дали е женен, или не, а според други, ако е женен и го извърши, подлежи на убиване с камъни, иначе само го бичуват. Подробният отговор пък се опира до голяма степен на разсъждението на богослова Ибн Кайим ал-Джаузия от XIV век. 

Съществува обаче казус, при който не са те хванали, така че е възможно да не изпиташ цялата строгост на шариата. За такъв случай разбираме от портала, финансиран от катарското Министерство на религиозните дела, където има цял раздел за наказания, свързани с хомосексуалност (лиуат) и „извращение“ (шузуз, един от другите термини).

Именно там мюсюлманин на 27-годишна възраст споделя, че бил изкушен многократно от хомосексуалността, за което се разкайва, и пита какво да стори. Отговорът е очакван: препоръчва се покаяние според редица предания от Пророка и Корана, като отново се цитира съчинението на Ибн Кайим ал-Джаузия. Някои текстове се превръщат в евъргрийн и си остават такива дори след седем столетия. Очертават се обаче и трите степени на подхлъзване към този грях (и престъпление, доколкото по шариата често пъти категориите се припокриват). Първата е само чрез гледане, втората е чрез телесен контакт под формата на прегръдки и други интимности, и накрая, третата е самият акт, който е най-голямата скверност.

Сексуалните практики между жени също не остават встрани от запитванията на мюсюлманите. Ето какво гласи друг въпрос:

Знам, че практикуването на секс между жени е възбранено, но искам да знам какво е наказанието. Сестра във вярата ми каза, че наказанието е точно като това за прелюбодеянието – бой с камшик за неомъжените и пребиване с камъни за омъжените.

Отговорът на шейх Мунаджжид е нюансиран – при все че някои религиозни учени го смятат за голям грях, не се полага наказание като за прелюбодеяние, защото не е такова. Полага се възпитателна наказателна мярка по преценка на съдията. Богословът Ибн Кудама също намира място в отговора – според него Пророкът е казал, че сексуалният контакт между жени все пак се смята за прелюбодейство (зина), но не се полага пълното наказание, защото няма как да се осъществи акт на проникване (джима‘). Оттук и следва отсъждането на наказание по преценка на съдията. Дават се и препоръки за излекуване от порока: обръщане към Аллах в чистота, преклонение и благочестие, свеждане на погледа с цел избягване на изкушенията, спомен за починалите, на които е отсъдено според делата им и не могат да направят нищо, за да изкупят греховете си, или да добавят още добри дела, занимания с полезни неща, а накрая нещо съвсем прагматично – препоръка за женитба колкото може по-скоро. 

Този текст няма претенция за всеобхватност. Със сигурност не отчита цялата палитра от нюанси в отношенията между половете. Не разглежда например възгледите за трансджендър хората, за хора с белези на двата пола или за кросдресинга. Не проблематизира в дълбочина правата на ЛГБТ общностите в Близкия и Средния изток. Няма и за цел да сравнява през границите на традициите в други монотеистични религии, нито пък да търси паралели с античния идеал за любовта между зрял мъж и младеж. За сметка на това открехва пролука, през която да надзърнем към обосновката на точно определено устойчиво отношение към ЛГБТ общността. С него може да сме съгласни или не, но то със сигурност съществува. 

Може да твърдим, подобно на изследователката, цитирана в началото, че тази строгост произтича не от „задълбочено разбиране“ на религиозния закон, а от изначално предпоставена и аксиоматично зададена цисджендър гледна точка. Звучи ми като изказване в духа на „те не са разбрали правилно Корана и Пророка и са си измислил фундаментализми“. Не бих се наел с такъв дързък мисловен експеримент чрез омаловажаване на значението на нормативния текст. Не може да си затворим очите за основанията за такова отношение към практиките на гей общността. 

Но какво можем да направим тогава? Трябва да се „считаме с реалностите“, както казва Тодор Живков в една от речите си.

ЛГБТ общности в Близкия и Средния изток са съществували и ще съществуват независимо от възгледите на „ходжите“,

както пейоративно ги наричат раздразнените им противници.

Може да се опитаме например да разделим постановките на свещения закон от тези на вярата в духа на едно „осветскостяване“ на религията, подобно на историческите процеси на секуларизация в Европа. Според това виждане придържането към свещения закон е историческа отживелица, следователно е незадължително. Коранът не е наръчник по право, твърдят застъпниците на такъв възглед. Оттук и религиозният закон може бъде категорично поставен настрани от основните положения на вярата като несъществен. 

Може да се предприеме и друг ход. Да се опитаме да обосновем иновативни тълкувания – както при съвременните усилия да се предоговорят понятията за пол в исляма по линия на изпълване на думи като фитра (сходно на „природа“) с ново съдържание, включващо и това да си гей. По редица причини тези контранаративни начинания в посока разчупване на основни доктринални положения засега остават в сферата на екзотичната интелектуална спекулация. И в момента, в който нормата започне да определя практиката, крайният резултат – с цялата му строгост – може да бъде доста предсказуем. Без значение дали ни харесва, или не.

Наказанието на народа на Лут, ръкопис от XVI век на „Житията на пророците“ от Исхак ибн Ибрахим ан-Нишапури (XII век), дигитална колекция на Берлинската библиотека

В рубриката „Ориент кафе“ Атанас Шиников поднася любопитни теми, свързани не толкова с горещата политика, колкото с историята и културата на Близкия изток. А той, древен и днешен, е по-близко до нас и съвремието ни, отколкото си представяме.

Announcing systemd v257

Post Syndicated from Lennart Poettering original https://0pointer.net/blog/announcing-systemd-v257.html

Last week we released systemd v257 into the wild.

In the weeks leading up to this release (and the week after) I have
posted a series of serieses of posts to Mastodon about key new
features in this release, under the
#systemd257 hash tag. In
case you aren’t using Mastodon, but would like to read up, here’s a
list of all 37 posts:

I intend to do a similar series of serieses of posts for the next systemd
release (v258), hence if you haven’t left tech Twitter for Mastodon yet, now is
the opportunity.

A sapling matures: meet sq 1.0

Post Syndicated from jzb original https://lwn.net/Articles/1002411/

The Sequoia PGP project has announced
version 1.0 of the sq command-line tool for managing OpenPGP
encryption and signatures. It also provides a decentralized public
key infrastructure
(PKI), and key management facilities. This is
the first stable release since development began on the project in
2017.

sq‘s PKI is probably its most notable feature, and the one we invested
the most time in. The PKI is used to authenticate certificates, and
messages. Authentication is necessary to ensure that you are
encrypting to the person you think you are, and to identify who really
authored a message; without authentication, encryption and
verification are much weaker.

New Amazon EC2 High Memory U7inh instance on HPE Server for large in-memory databases

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/new-amazon-ec2-high-memory-u7inh-instance-on-hpe-server-for-large-in-memory-databases/

Today we’re announcing the general availability of Amazon Elastic Compute Cloud (Amazon EC2) U7inh instance, a new addition to EC2 High Memory family, built in collaboration with Hewlett Packard Enterprise (HPE). Amazon EC2 U7inh instance runs on the 16-socket HPE Compute Scale-up Server 3200, and are built on the AWS Nitro System to deliver a fully integrated and managed experience consistent with other EC2 instances.

Powered by the fourth generation Intel® Xeon® Scalable processors (Sapphire Rapids), U7inh instance supports 32 TB of memory and 1920 vCPUs. This instance offers the highest compute performance, largest compute and memory size in the Amazon Web Services (AWS) Cloud for running large, mission-critical database workloads, like SAP HANA.

In May 2024, we launched U7i instances to support up to 896 vCPUs and up to 32 TB of memory, which our enterprise customers could use to successfully migrate their large mission-critical in-memory databases to AWS and benefit from the flexibility, scalability, reliability, and cost advantages that AWS offers.

As customers continue to scale their business applications, they wanted the performance combined with the additional CPUs and memory along with SAP certification to generate real-time business insights. Other customers that currently run on-premises with HPE servers have also asked how we can help them migrate to AWS to take advantage of cloud benefits while continuing to use HPE hardware.

Here are the detailed specs of new U7inh instance:

Instance name vCPUs Memory (DDR5) EBS bandwidth Network bandwidth
U7inh-32tb.480xlarge 1920 32,768 GiB 160 Gbps 200 Gbps

U7inh instance offers up to two times vCPUs and 1.6 times EBS bandwidth in a single instance, compared with the largest U7i instance. You can run your largest in-memory database workloads like SAP HANA or seamlessly migrate workloads running on HPE hardware to AWS.

U7inh instance supports Amazon Linux, Red Hat Enterprise Linux, and SUSE Enterprise Linux Server. Operating system support for SAP HANA workloads on High Memory instances include: SUSE Linux Enterprise Server 15 SP3 for SAP and above and Red Hat Enterprise Linux 8.6/9.0 for SAP and above.

U7inh instance is SAP certified to run Business Suite on HANA (SoH), Business Suite S/4HANA, Business Warehouse on HANA (BW), and SAP BW/4HANA in production environments. U7inh instance is also certified for scale-out SAP HANA OLTP workloads such as S/4HANA and customers can deploy up to four U7inh instance (128TB) in a cluster for even larger SAP HANA workloads.

To learn more about how to migrate, visit Migrating SAP HANA on AWS to an EC2 High Memory Instance in the SAP HANA on AWS Guides and AWS Launch Wizard for SAP in the AWS Launch Wizard User Guide.

Now available
Amazon EC2 U7inh instance is available in the US East (N. Virginia) and US West (Oregon) AWS Regions.

To learn more, visit the U7i instance product page and send feedback to AWS re:Post for EC2 or through your usual AWS Support contacts.

Channy

Enforce resource configuration to control access to new features with AWS

Post Syndicated from Yossi Cohen original https://aws.amazon.com/blogs/security/enforce-resource-configuration-to-control-access-to-new-features-with-aws/

Establishing and maintaining an effective security and governance posture has never been more important for enterprises. This post explains how you, as a security administrator, can use Amazon Web Services (AWS) to enforce resource configurations in a manner that is designed to be secure, scalable, and primarily focused on feature gating.

In this context, feature gating means that newly supported AWS features and configurations can’t be used unless you explicitly approve them. With feature gating, you maintain control over your AWS environment when new services and capabilities are introduced.

This blog post demonstrates a unique approach to giving users, such as DevOps teams, controlled flexibility within safe boundaries by allowing resource provisioning that uses only approved configurations. This approach also accommodates configurations that will be supported in future versions of the resource, keeping them restricted until explicitly approved, as shown in Figure 1.

Figure 1: Restrict resource provisioning to approved configurations only

Figure 1: Restrict resource provisioning to approved configurations only

Apply your resource configuration enforcement

As shown in Figure 2, our solution for resource configuration enforcement (RCFGE) uses AWS CloudFormation Hooks. By using Hooks, you can run custom logic during the provisioning of resources. These are proactive controls because you inspect and enforce resource configurations before the resource is created, updated, or deleted.

Your Hook will only be effective if CloudFormation supports the AWS resources that you are using and if you implement a service control policy (SCP) that helps prevent users from provisioning resources outside of CloudFormation.

Figure 2: How CloudFormation Hooks work

Figure 2: How CloudFormation Hooks work

The flow shown in Figure 2 consists of the following five steps:

  1. DevSecOps registers and configures a CloudFormation Hook in the account.
  2. DevOps specifies a CloudFormation template that defines the required resources and configurations.
  3. CloudFormation creates a new stack resource, starting the provisioning process based on the template.
  4. The Hook is triggered before provisioning for each resource that’s defined in the template, and runs custom validation logic.
  5. If the validation checks pass, CloudFormation proceeds with provisioning; if not, the process is terminated.

Make your solution scalable

To achieve scalable operations, you should implement a reusable and generic Hook that targets all supported CloudFormation resource types. This Hook enforces resource configuration by loading resource specification files from an external object storage, such as an Amazon Simple Storage Service (Amazon S3) bucket.

These specification files define validation rules in a declarative language. Using this approach, you can add and remove resource configuration validation rules by editing the declarative files. When you externalize custom logic as decoupled validation rules from the Hook, DevSecOps personnel can manage these rules at scale without affecting your infrastructure.

Figure 3: Externalize custom logic as validation rule files in an S3 bucket

Figure 3: Externalize custom logic as validation rule files in an S3 bucket

Figure 3 shows how the solution has been revised to support this approach. Steps 1–3 are the same as in the flow shown in Figure 2:

  1. DevSecOps registers and configures a CloudFormation Hook in the account.
  2. DevOps specifies a CloudFormation template that defines the required resources and configurations.
  3. CloudFormation creates a new stack resource, starting the provisioning process based on the template.
  4. The Hook is triggered before provisioning for each resource that’s defined in the template.
  5. The Hook loads the relevant resource specification file from the S3 bucket and executes the validation rules against the current resource in the CloudFormation template.
  6. If the validation checks pass, CloudFormation proceeds with provisioning; if not, the process is terminated.

You need to configure the Hook schema and the Hook configuration schema to evaluate the configurations of all supported resources across your AWS accounts before changes are provisioned. This setup should cover create, update, and delete operations so that the Hook can help prevent non-approved configurations across stacks.

By using AWS CloudFormation Guard, you can externalize validation rules from the Hook, as described in Extend your pre-commit hooks with AWS CloudFormation Guard. Guard is an open source, general purpose, policy-as-code (PaC) evaluation tool that validates CloudFormation templates against custom rules to help you stay aligned with your organizational policies. For example, the CT.S3.PR.1 rule specification demonstrates a Guard rule that requires an S3 bucket to have its settings configured to block public access. These validation rules apply to currently supported AWS resource configurations and features, but they don’t restrict potential future properties.

Boost your solution with feature gating

Your risk model might lead you to look for mechanisms that further restrict the AWS resource configurations that you allow in your environments. As you will see, the proposed solution restricts authorized workforce users so that they can use new configurations only if you enable them. The proposed approach uses feature gating because it continues to enforce your configurations even when AWS adds new options for your resources.

Guard aims to validate required constraints; but to meet the feature gating objective, you should implement validation rules that check whether resource configurations fulfill structural constraints described by the restricted version of CloudFormation resource schemas. These schemas help you confine the possible resource configurations that can be provisioned in your environment no matter what new configurations AWS introduces.

Figure 4: Enforce resource configuration with restricted resource schema templates

Figure 4: Enforce resource configuration with restricted resource schema templates

Figure 4 shows an updated version of the same flow where validation rules are implemented by using restricted resource schema templates, which are stored in an S3 bucket. These templates are based on the original CloudFormation resource schemas, representing a snapshot of these schemas at a specific point in time. Steps 1–4 are the same as in the flow shown in Figure 3:

  1. DevSecOps registers and configures a CloudFormation Hook in the account.
  2. DevOps specifies a CloudFormation template that defines the required resources and configurations.
  3. CloudFormation creates a new stack resource, starting the provisioning process based on the template.
  4. The Hook is triggered before provisioning for each resource that’s defined in the template.
  5. The Hook loads the relevant restricted resource schema template file from the S3 bucket and uses it to execute schema validation against the current resource in the CloudFormation template.
  6. If the validation checks pass, CloudFormation proceeds with provisioning; if not, the process is terminated.

A restricted resource schema template is a subset of its corresponding original CloudFormation resource schema. It includes additional constraints that limit certain properties to specific values and patterns or exclude certain properties entirely. Furthermore, these templates contain placeholders that you fill in with runtime values, such as the account ID, which your Hook provides as part of the Hook context.

Figure 5: Resource configuration enforcement (RCFGE) CloudFormation Hook flow

Figure 5: Resource configuration enforcement (RCFGE) CloudFormation Hook flow

As shown in Figure 5, the flow within the RCFGE CloudFormation Hook involves the following steps:

  1. The CloudFormation Hook is invoked with the Hook context and the resource’s configuration JSON object.
  2. The Hook loads the restricted resource schema template from the S3 bucket and substitutes placeholders with the Hook context runtime values, producing a valid JSON schema.
  3. The Hook validates the stack’s resource configuration JSON object against the schema. If it returns OperationStatus.SUCCESS, then CloudFormation proceeds with the provisioning process. If it returns OperationStatus.FAILED, then CloudFormation terminates the provisioning process.

If a restricted resource schema template for a CloudFormation resource type isn’t found in the S3 bucket, the schema validation step fails by default.

Sample excerpt of a restricted schema template for an S3 bucket resource

The following is an excerpt from a restricted schema template for an S3 bucket. At runtime, your Hook processes this template, substituting the placeholders with relevant values from the Hook context. In this example, the Hook replaces the <accountID> placeholder in the topic’s pattern with the actual account ID. The resulting JSON schema disallows additional properties beyond those defined by the schema and restricts the Amazon Simple Notification Service (Amazon SNS) topics that can be used for event notifications.

Note: In the code samples that follow, we’ve omitted some code for brevity—we’ve indicated these omissions with three periods: ...

{
  "type": "object",
  "required": [],
  "additionalProperties": false,
  "properties": {
        ...
      "NotificationConfiguration": {
          "$ref": "#/definitions/NotificationConfiguration"
      },
        ...
  },
  "definitions": {
        ...
      "NotificationConfiguration": {
          "type": "object",
          "additionalProperties": false,
          "properties": {
            ...
              "TopicConfigurations": {
                  "type": "array",
                  "uniqueItems": true,
                  "items": {
                      "$ref": "#/definitions/TopicConfiguration"
                  }
              }
          }
      },
        ...
      "TopicConfiguration": {
          "type": "object",
          "additionalProperties": false,
          "properties": {
        ...
              "Topic": {
                  "type": "string",
                  "pattern": "^arn:aws:sns::$<accountID>:.*$"
              },
        ...
            }
      },
  }
}

CloudFormation template for an S3 bucket that adheres to the restricted schema

Let’s assume that your account ID is 111122223333. The account ID is propagated to the Hook through the Hook context.

The following is an excerpt from a CloudFormation template that aligns with the restricted schema for an S3 bucket instantiated from the template shown previously. As a result, your Hook allows the corresponding CloudFormation stack to proceed.

{
   "AWSTemplateFormatVersion": "2010-09-09",
   "Resources": {
     "S3Bucket": {
       "Type": "AWS::S3::Bucket",
       "Properties": {
         "BucketName":
            "valid-bucket-sns-notification-configuration-template",
         "NotificationConfiguration": {
           "TopicConfigurations": [
             {
              "Topic":
                "arn:aws:sns:eu-west-1:111122223333:this-is-my-topic-and-I-trust-it",
              "Event": "s3:ObjectCreated:*"
             }
           ]
         }
       }
    }
  }
}

CloudFormation template for an S3 bucket that diverges from the restricted schema (example 1)

The following is an excerpt from a CloudFormation template that doesn’t align with the restricted schema for an S3 bucket instantiated from the template shown previously because it attempts to configure the Amazon SNS topic for the notification configuration, which uses an Amazon Resource Name (ARN) of another account. As a result, your Hook causes the corresponding CloudFormation stack to fail.

{
   "AWSTemplateFormatVersion": "2010-09-09",
   "Resources": {
     "S3Bucket": {
       "Type": "AWS::S3::Bucket",
       "Properties": {
         "BucketName":
           "invalid-bucket-sns-notification-configuration-template",
         "NotificationConfiguration": {
            "TopicConfigurations": [
              {
               "Topic":
                 "arn:aws:sns:eu-west-1:444455556666:this-is-not-my-topic",
               "Event": "s3:ObjectCreated:*"
              }
            ]
         }
       }
     }
   }
}

CloudFormation template for an S3 bucket that diverges from the restricted schema (example 2)

The following is an excerpt from a CloudFormation template that doesn’t align with the restricted schema for an S3 bucket instantiated from the template shown previously. This time, it violates your feature gating objective by attempting to use a new, imaginary feature of an S3 bucket that isn’t approved for use by your restricted schema for an S3 bucket. As a result, your Hook causes the corresponding CloudFormation stack to fail.

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Resources": {
    "S3Bucket": {
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "BucketName":
           "valid-bucket-sns-notification-configuration-template",
        "NewFeature": {
           "property-1": true,
           "property-2": "public"
        },                
        "NotificationConfiguration": {
          "TopicConfigurations": [
            {
              "Topic":
                 "arn:aws:sns:eu-west-1:111122223333:this-is-my-topic-and-I-trust-it",
              "Event": "s3:ObjectCreated:*"
            }
          ]
        }
      }
    }
  }
}

Protect your controls

If a security control itself isn’t protected adequately, it becomes a weak link in the security chain. For example, a surveillance camera (a physical security control) that isn’t securely mounted can be removed, rendering it useless. This principle also applies to your RCFGE solution.

Next, we will show you how to isolate management activities to a dedicated account and use SCPs as preventative controls.

Isolate RCFGE management in a dedicated account

Organizing your AWS environment by using multiple accounts is a best practice because it enhances security, simplifies management, and allows for better resource isolation and cost tracking. Isolating the operation and management of your RCFGE solution in its own dedicated account is essential for securing the solution’s resources.

With AWS CloudFormation StackSets, you can deploy and manage RCFGE stacks across multiple accounts and AWS Regions from a single central administrator account. This provides consistent and scalable infrastructure while maintaining centralized governance. With this functionality, you can deploy the RCFGE resources to existing accounts and automatically include new accounts as you add them to your organization, simplifying RCFGE management and providing uniformity across your environments. For more information, see Deploy CloudFormation Hooks to an Organization with service-managed StackSets.

Figure 6 shows how to extend that idea so that you can operate the RCFGE solution at scale while maintaining isolation and the separation of duties. The solution operates across three key account types:

  • Management account –use this account to create your organization and designate the CloudFormation StackSets delegated administrator account.
  • Delegated administrator account – this account serves as the centralized management point for the RCFGE solution. It contains a continuous integration and continuous delivery (CI/CD) pipeline that provisions RCFGE resources across the organization by using CloudFormation StackSets with service managed permissions. The account hosts a centralized S3 bucket that stores the RCFGE restricted resource schema templates. The security engineering team uses this account to submit Hook code and restricted resource schema template changes, which trigger the CI/CD pipeline.
  • Member accounts – each member account contains an RCFGE StackSet instance and an AWS Identity and Access Management (IAM) role for provisioning RCFGE resources. It also includes a CloudFormation Hook and an IAM role that allows the Hook to access the centralized S3 bucket with RCFGE restricted resource schema templates.

Figure 6: Securely operate the RCFGE solution

Figure 6: Securely operate the RCFGE solution

Let’s explore how the RCFGE solution architecture enforces resource configuration step by step, as shown in Figure 7.

Figure 7: CloudFormation stack deployment flow with RCFGE validation and enforcement

Figure 7: CloudFormation stack deployment flow with RCFGE validation and enforcement

  1. DevOps initiates the deployment by specifying a CloudFormation template that defines the resources and configurations needed.
  2. CloudFormation creates a new stack resource, initiating the resource provisioning process based on the provided template.
  3. The RCFGE CloudFormation Hook is triggered for each resource defined in the CloudFormation template.
  4. The Hook loads the corresponding restricted resource schema template from the S3 bucket.
  5. The Hook validates a resource configuration:
    • The Hook processes the restricted resource schema template to create a JSON schema.
    • It uses this JSON schema to validate the current resource in the CloudFormation template.
    • If the resource is invalid according to the schema, the provisioning process is terminated.
  6. If the current resource passes validation, CloudFormation proceeds with the resource provisioning process by creating and configuring the resources as specified in the template.

Use SCPs as preventive controls for your organization to help protect RCFGE

The following SCP excerpt accomplishes three objectives:

  • Implements a statement (see AllowedListActions) to explicitly specify the access that is allowed while other access is implicitly blocked.
  • Implements control objectives to help prevent changes to resources set up by the RCFGE solution (see ProtectRCFGEResources and ProtectStackSetExecutionRole).
  • Makes sure that AWS resource provisioning does not occur outside of CloudFormation (see ProvisionResourcesViaCloudFormationOnly).

In this SCP excerpt, the ProvisionResourcesViaCloudFormationOnly statement restricts CloudFormation stacks to being managed only through forward access sessions (FAS) in AWS IAM.

The ProvisionResourcesViaCloudFormationOnly statement explicitly prohibits direct create, update, and delete actions for all supported resources used in your environment. If needed, split this statement into multiple parts so you don’t exceed SCP size limits, while providing comprehensive coverage of your resources to make sure that they are provisioned and managed only through CloudFormation.

The ProtectStackSetExecutionRole statement in this example assumes that CloudFormation trusted access is activated with AWS Organizations, which is required by StackSets to deploy across accounts and Regions by using service managed permissions.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowedListActions",
      "Effect": "Allow",
      "Action": [
        "s3:CreateBucket",
        "s3:DeleteBucket",
        "s3:DeleteBucketPolicy",
        "s3:PutAnalyticsConfiguration",
        "s3:PutBucketLogging",
        "s3:PutBucketNotification",
        "s3:PutBucketObjectLockConfiguration",
        "s3:PutBucketPolicy",
        "s3:PutBucketTagging",
        "s3:PutBucketVersioning",
        "s3:PutLifecycleConfiguration",
        "s3:PutMetricsConfiguration",
        "s3:PutReplicationConfiguration",
        "s3:GetObject",
        ...
      ],
      "Resource": "*"
    },
    {
      "Sid": "ProtectRCFGEResources",
      "Effect": "Deny",
      "Action": "*",
      "Resource": [
        "arn:aws:cloudformation:*:*:stack/RCFGEStackSet",
        "arn:aws:cloudformation:*:*:*/hook/RCFGEHook/*",
        "arn:aws:iam::*:role/RCFGEHookExecutionRole"
      ],
      "Condition": {
        "ArnNotLike": {
          "aws:PrincipalArn": [
            "arn:aws:iam::*:role/stacksets-exec-*"
          ]
        }
      }
    },
    {
      "Sid": "ProtectStackSetExecutionRole",
      "Effect": "Deny",
      "Action": "*",
      "Resource": "arn:aws:iam::*:role/stacksets-exec-*"
    },
    {
      "Sid": "ProvisionResourcesViaCloudFormationOnly",
      "Effect": "Deny",
      "Action": [
        "s3:CreateBucket",
        "s3:DeleteBucket",
        "s3:DeleteBucketPolicy",
        "s3:PutAnalyticsConfiguration",
        "s3:PutBucketLogging",
        "s3:PutBucketNotification",
        "s3:PutBucketObjectLockConfiguration",
        "s3:PutBucketPolicy",
        "s3:PutBucketTagging",
        "s3:PutBucketVersioning",
        "s3:PutLifecycleConfiguration",
        "s3:PutMetricsConfiguration",
        "s3:PutReplicationConfiguration",
        ...
      ],
      "Resource": "*",
      "Condition": {
        "StringNotEquals": {
          "aws:CalledViaFirst": "cloudformation.amazonaws.com"
        }
      }
    }
  ]
}

To allow the Hook to retrieve the necessary restricted resource schema templates, member accounts must be able to access the S3 bucket that contains the RCFGE templates. The following code sample shows the bucket policy for the S3 bucket that contains the RCFGE templates.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowRCFGEHookExecutionRoleGetRCFGETemplates",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Principal": "*",
      "Resource": "arn:aws:s3:::RCFGETemplates/*",
      "Condition": {
        "StringEquals": {
          "aws:PrincipalOrgID": "o-abcdef0123"
        },
        "ArnLike": {
          "aws:PrincipalArn": "arn:aws:iam::*:role/RCFGEHookExecutionRole"
        }
      }
    }
  ]
}

As shown in the following code sample, the RCFGEHookExecutionRole IAM role in member accounts has a policy that grants read-only access to the RCFGE templates that are stored in an S3 bucket in the RCFGE delegated administrator account, where 555555555555 represents the account ID.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowRCFGEHookExecutionRoleGetRCFGETemplates",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::RCFGETemplates/*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceAccount": "555555555555"
        }
      }
    }
  ]
}

In the following code sample, the RCFGEHookExecutionRole IAM role in member accounts has a trust policy that allows it to be assumed only by the relevant CloudFormation service principals, where 444455556666 represents the account ID of the member account.

{
  "Version": "2012-10-17",
  "Statement": {
    "Sid": "AllowRCFGEHookExecutionRoleGetRCFGETemplatesTrust",
    "Effect": "Allow",
    "Principal": {
      "Service": [
        "hooks.cloudformation.amazonaws.com",
        "resources.cloudformation.amazonaws.com"
      ]
    },
    "Action": "sts:AssumeRole",
    "Condition": {
      "ArnLike": {
        "aws:SourceArn": "arn:aws:cloudformation:eu-west-1:444455556666:type/hook/RCFGEHook/*"
      }
    }
  }
}

Define baseline configuration for RCFGE and continuous monitoring with AWS Config

Defense in depth is an effective strategy because if one line of defense fails, additional layers are in place to help stop threats at subsequent points. With AWS Config, you can capture the configuration of RCFGE resources over time. You can set up AWS Config custom rules to automatically assess the compliance of your RCFGE resources against predefined policies. For example, you can use an AWS Config custom rule to make sure that the RCFGE Hook hasn’t been altered or removed.

Conclusion

In this post, you learned how to use CloudFormation Hooks to create a resource configuration enforcement (RCFGE) solution on AWS that is designed to be secure and scalable and that supports feature gating. Using this approach, you, as a security administrator, can maintain strict control over resource configurations and feature adoption across your AWS environments. The solution provides a balanced approach to governance, so that DevOps teams have the flexibility to work within approved boundaries while making sure that new AWS features are only accessible after explicit approval.

If you have feedback about this post, submit comments in the Comments section. For questions, start a new thread on the CloudFormation re:Post or contact AWS Support.
 

Yossi Cohen
Yossi Cohen

Yossi is a Senior Security Specialist Solutions Architect at AWS for the public sector in the EMEA region. Yossi has over two decades of experience in cloud-native architecture development, design, operations, technical due diligence, and governance in highly regulated environments. At AWS, Yossi collaborates closely with defense, intelligence, government, and public sector clients, helping them navigate their unique threat landscapes.
Yaniv Rozenboim
Yaniv Rozenboim

Yaniv is a Senior Solutions Architect at AWS with extensive experience in cloud architecture and security. He specializes in designing and implementing secure, scalable, and efficient cloud infrastructures. Yaniv works closely with clients to help them achieve their business goals through AWS technologies.

AWS KMS: How many keys do I need?

Post Syndicated from Ishva Kanani original https://aws.amazon.com/blogs/security/aws-kms-how-many-keys-do-i-need/

As organizations continue their cloud journeys, effective data security in the cloud is a top priority. Whether it’s protecting customer information, intellectual property, or compliance-mandated data, encryption serves as a fundamental security control. This is where AWS Key Management Service (AWS KMS) steps in, offering a robust foundation for encryption key management on AWS.

One of the first questions that often arises for customers is, “How many keys do I actually need?” This seemingly simple question requires careful consideration of various factors. Although AWS KMS makes encryption straightforward, organizations need to consider several aspects of their key management strategy. These include choosing between AWS managed keys, customer managed keys, and importing your own keys (BYOK), as well as deciding between centralized and decentralized key management approaches. Each option has its own benefits and trade-offs in terms of security, control, and operational overhead. By understanding these choices and how they align with your organization’s needs, you can develop an effective and efficient key management strategy.

In this blog post, we explore the main considerations that drive your AWS KMS key strategy, from organizational structure to compliance requirements. Should you maintain a centralized key management approach with a single team controlling all keys, or adopt a decentralized model where individual teams manage their own keys? These decisions are important because they relate to the AWS shared responsibility model, where AWS maintains the security of the cloud, while customers remain responsible for security in the cloud, including the proper management and use of encryption keys.

Overview – What is AWS Key Management Service?

AWS Key Management Service (AWS KMS) is an AWS managed service that makes it convenient for you to create and control the encryption keys that are used to encrypt your data. The keys that you create in AWS KMS are protected by FIPS 140 Level 3 validated hardware security modules (HSM). The keys never leave AWS KMS unencrypted. To use or manage your KMS keys, you interact with AWS KMS.

Customers are responsible for deciding what data to encrypt, choosing the appropriate encryption keys, and implementing encryption across AWS services with the help of the key policy. Customers are responsible for monitoring and auditing the use of encryption keys through services such as AWS CloudTrail.

A critical aspect of customer responsibility is determining how to manage the keys and how many KMS keys are needed. This decision depends on various factors such as data classification, application architecture, regulatory requirements, and operational needs. We look at these areas in more detail in the next sections.

Guiding principles for key strategy

Following are four guiding engineering principles that, based on our experience, help create a secure and easier-to-maintain system. They will assist you in determining the approximate number of KMS keys for your organization based on your management requirements.

Principle 1 – Data Classification: If a system processes data of different classification levels, employ separate data resources and separate KMS keys to separately govern and audit access to the data. With similarly classified data or a single type of data, the usage of just one KMS key may be justified.

Why it matters: This principle helps to ensure that data that is classified into different sensitivity levels is protected appropriately based on access to encryption keys for that same classification, reducing the risk of unauthorized access and simplifying governance.

Principle 2 – Applications: Multiple applications can run in one AWS account. We recommend that you use distinct KMS keys for each application, because managing access to an individual key can become a complex task when it is delegated to two or more application administrators. Use separate KMS keys for applications running in distinct AWS accounts to further make use of the account boundary, limiting the potential impact in case of a security incident. Use separate keys for distinct application stages (such as development, staging, or production).

Why it matters: This approach isolates access to applications and application access to data. This reduces the potential impact of unintended access to a key.

Principle 3 – AWS Services: When you consider key management across multiple AWS services, focus on both the services and the nature of the data. If you are dealing with one type of data (for example, customer information) that flows through multiple AWS services as part of one application or workflow, consider using a single KMS key. This simplifies key management while maintaining consistent access control. For instance, a customer record that is stored in Amazon Simple Storage Service (Amazon S3), processed by AWS Lambda, and then stored in Amazon DynamoDB could use the same KMS key across these services as mentioned in Principle 1.

However, if you are handling different types of data (such as financial records and user preferences) across various AWS services, even within the same application, consider using separate KMS keys on a per-service basis. This allows for more granular access control and adheres to the principle of least privilege. For example, in an e-commerce application, you might use one KMS key for encrypting payment information in Amazon Relational Database Service (Amazon RDS) and a different key for encrypting user browsing history in Amazon Redshift.

The decision to use one key or multiple keys should be based on your data classification policies and access control requirements. With this approach, you can keep your key management strategy aligned with your data governance policies, regardless of which AWS services you are using.

Why it matters: This principle balances the need for simplicity with the requirement for granular control over data access across different AWS services.

Principle 4 – Separation of Duties: Key policies define who can administer and who can use the key. In the case of distinct encryption use cases and distinct administrators, we recommend that you create separate KMS keys. Another aspect of separation of duties is that, with KMS key policies, two different principals can be made responsible for governing data and data decryption access. However, this does not influence the count of keys.

Why it matters: This principle supports the implementation of least privilege access and helps maintain clear accountability in key management.

By applying these principles, you can develop a key management strategy that describes how many KMS keys you may need, and that balances security, compliance, and operational efficiency. In the following sections, we explore how to apply these principles in various scenarios.

Examples of key management strategy and comparison of centralized and decentralized approaches

In addition to the guiding principles discussed earlier, the structure of your organization and its specific needs play a crucial role in determining the most suitable approach to key management. When implementing key management strategies, organizations generally choose from three main approaches: centralized, decentralized, or a hybrid model. The choice depends on the organization’s structure, needs, and operational context. Each approach offers distinct advantages for specific organizational scenarios.

A decentralized approach is our recommended approach, as most customers fit into the following scenarios:

  • Organizations with autonomous business units or where governance controls provide oversight of key usage
  • Companies where development teams are agile and ownership of keys can be centrally audited
  • Companies that operate in multiple regulatory frameworks
  • Companies that require to operate in a particular AWS Region

A centralized KMS approach is best suited for the following scenarios:

  • Organizations that require strict compliance oversight and centralized management
  • Companies with centralized security or data protection functions

In a hybrid model, there is a blend between centralized and decentralized:

  • Core key policies are managed centrally
  • Day-to-day key operations are handled by teams

    For example, organizations or companies could have independent product teams, but a centralized security team.

Example 1 (Hybrid): A retail website with public product catalog data and confidential customer data should use two KMS keys—one for the public catalog that is encrypted in Amazon S3, and one for customer data that is encrypted in Amazon RDS and other AWS services.

Rationale: This recommendation is based primarily on Principle 1 (Data Classification). The public catalog data and confidential customer data represent different classification levels, justifying the use of separate keys. This approach is further supported by Principle 3 (AWS Services), because the data resides in different AWS services and is of a varied nature.

The benefits of this approach:

  • Implement appropriate access controls for each data type
  • Manage encryption independently for each data classification
  • Enhance overall data security and compliance

Example 2 (Decentralized): A healthcare company with several application teams could use a separate KMS key for each application team, with distinct key policies for each key based on the data and roles of each team.

Rationale: This recommendation is primarily based on Principle 2 (Applications). With multiple application teams operating within the healthcare company, each potentially dealing with distinct types of data and having different access requirements, separate KMS keys provide for independent management of encryption and access for each team. This approach is further supported by Principle 4 (Separation of Duties), allowing for team-specific key policies.

The benefits of this approach:

  • Maintain granular control over data access.
  • Implement team-specific encryption policies.
  • Uphold the principle of least privilege across the organization.
  • Enhance data security: By using separate keys, the company limits the impact of improper access to any given key, enables more precise access control, facilitates independent key rotation schedules, and improves the ability to monitor and audit key usage for each application.
  • Simplify alignment with healthcare regulations: Separate keys support data segregation requirements, enable fine-grained role-based access control, provide clear audit trails for each application’s data access, and allow for tailored data lifecycle management. This functionality is crucial for aligning with various healthcare compliance standards such as HIPAA.
  • Allow for efficient and distributed key management that is tailored to each application team’s needs.

These examples demonstrate how applying the guiding principles can lead to a well-structured key management strategy, tailored to the specific needs of different organizations and use cases.

Considerations for key management

When you implement your key management strategy, several factors need to be considered beyond just the number of keys. This section explores these considerations to help you make informed decisions about your key management approach.

Key types

AWS offers different types of KMS keys, each with its own benefits and use cases.

AWS owned keys are managed by AWS in service accounts, used across multiple customer accounts, and provide no customer visibility or audit capability. Choose AWS owned keys when there are no management or audit requirements for the keys, but encryption of the data at rest is needed.

AWS managed keys are managed entirely by AWS and are used only for your AWS account. Although customers can view these keys in the AWS Management Console and track their usage in AWS CloudTrail logs, they have limited ability to directly control or modify these keys. Choose AWS managed keys when managing keys is not a requirement, but having an audit trail is. It’s worth noting that AWS managed keys are automatically rotated every year, which can be convenient for many use cases.

Customer-managed keys offer the highest level of control and customization, allowing creation of key policies and control over key rotation. However, customer managed keys provide more flexibility, allowing you to set your own rotation schedule or even enable rotation if you are required to do so for regulatory reasons. Choose customer managed keys when you need strict control over key usage and the ability to share keys or control access through key policies, detailed auditing capabilities, alignment with specific compliance requirements, or the ability to integrate key management with your existing processes and tools.

The decision between AWS managed and customer managed keys often comes down to balancing the convenience of automatic management with the need for granular control and customization. As the number of keys increases, so does the complexity of management. More keys mean more policies to create, manage, and audit. Making sure that the right people have access to the right keys becomes more challenging. However, to help audit KMS key access, you can use the IAM Access Analyzer to determine external access to your keys. Managing rotation schedules for multiple keys requires more effort, and more keys mean more policies to analyze and monitor, as well as growing costs.

Cost

Security should be the primary concern, but cost is also a factor. Each customer managed key incurs a monthly storage cost. Both AWS managed and customer managed KMS keys have API usage costs associated with them. Key rotation can increase costs over time, as old key versions are retained.

Manageability

Finding the right balance between security and manageability is crucial. Too few keys might not provide adequate separation of duties or granular access control, while too many keys can lead to increased complexity, higher costs, and potential mismanagement.

Specific requirements

Different industries and regions may have specific requirements for key management. Some regulations might require separation of duties, necessitating multiple keys. Certain compliance standards might dictate specific key rotation or audit trail requirements.

By carefully considering these factors alongside the guiding principles discussed earlier, you can develop a key management strategy that balances security, compliance, cost-effectiveness, and operational efficiency for your specific needs. It is important to approach your KMS strategy holistically, considering not just your immediate security needs, but also the long-term management implications. Regular review and adjustment of your key management strategy will provide assurance that it continues to meet your evolving needs while maintaining robust security and compliance.

Conclusion

As we explored throughout this post, determining the optimal number of AWS KMS keys for your organization is a nuanced decision that balances security, compliance, cost, and operational efficiency. The guiding principles we discussed—data classification, application segregation, AWS service integration, and separation of duties—provide a solid framework for making these decisions. Remember that there’s no one-size-fits-all solution; the right approach depends on your specific needs and circumstances.

As you move forward in implementing or refining your KMS key strategy, consider these next steps: First, conduct a thorough audit of your current data assets, their classifications, and the applications and services that interact with them. Next, map out your ideal key management structure based on the principles we’ve discussed. Then, evaluate the costs and operational overhead of your proposed strategy, adjusting as necessary to find the right balance for your organization. Finally, implement your strategy incrementally, starting with your most sensitive or critical data assets.

Remember that key management is an ongoing process. Regularly review and update your strategy as your data landscape evolves, new compliance requirements emerge, or AWS introduces new features. By thoughtfully applying the principles and considerations we’ve discussed, you can create a robust, scalable, and efficient key management strategy that helps your overall security posture and meets your organization’s unique needs.

 
If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.
 

Ishva Kanani
Ishva Kanani

Ishva is a Security Consultant at AWS. She assists customers with secure cloud migrations and accelerates their cloud journeys by delivering innovative solutions. Passionate about cybersecurity, Ishva provides strategic guidance and best practices for cloud environments. When not safeguarding digital assets, she enjoys exploring local hiking trails and trying new recipes in her kitchen.
Hardik Thakkar
Hardik Thakkar

Hardik is a Prototyping Solutions Architect at AWS Global Financial Services (GFS). He specializes in secure architecture design and foundations on AWS, leveraging his security expertise to serve financial services customers. His focus areas include security-first design patterns, financial services compliance frameworks, and helping institutions build robust cloud infrastructures on AWS.

ASRock Rack 6U8X-EGS2 H200 NVIDIA HGX H200 AI Server Review

Post Syndicated from Patrick Kennedy original https://www.servethehome.com/asrock-rack-6u8x-egs2-h200-nvidia-hgx-h200-ai-server-intel-xeon-review/

We review the ASRock Rack 6U8X-EGS2 H200, an NVIDIA HGX H200 8 GPU design to see how it performs and how GPU servers have evolved since 2015

The post ASRock Rack 6U8X-EGS2 H200 NVIDIA HGX H200 AI Server Review appeared first on ServeTheHome.

And that’s a wrap!

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/and-thats-a-wrap/

After 20 years, and 3283 posts adding up to 1,577,106 words I am wrapping up my time as the lead blogger on the AWS News Blog.

It has been a privilege to be able to “live in the future” and to get to learn and write about so many of our innovations over the last two decades: message queuing, storage, on-demand computing, serverless, and quantum computing to name just a few and to leave many others out. It has also been a privilege to be able to meet and to hear from so many of you that have faithfully read and (hopefully) learned from my content over the years. I treasure those interactions and your kind words, and I keep both in mind when I write.

Next for Jeff
I began my career as a builder. Over the years I have written tens of thousands of lines of assembly code (6502, Z80, and 68000), Visual Basic, and PHP, along with hundreds of thousands of lines of C. However, over the years I’ve progressively spent less time building and more time talking about building. As each new service and feature whizzed past my eyes I would reminiscence about days and decades past, when I could actually use these goodies to create something cool. I went from being a developer who could market, to a marketer who used to be able to develop. There’s absolutely nothing wrong with that, but I like to build. The medium could be code, 3D printing, LEGO bricks, electronics components, or even cardboard –creating and innovating is what motivates and sustains me.

With that as my driving force, my goal for the next step of my career is to invest more time focused on learning and using fewer things, building cool stuff, and creating fresh, developer-focused content as I do so. I’m still working to figure out the form that this will take, so stay tuned. I am also going to continue to make my weekly appearances at AWS OnAir (our Friday Twitch show), and I will continue to speak at AWS community events around the globe.

Next for the Blog
As for the AWS News Blog, it has long been backed by an awesome team, both visible and invisible. Here we are at the recent AWS re:Invent celebration of the blog’s 20th anniversary (photo courtesy of Liz Fuentes with edits by Channy Yun to add those who were otherwise occupied):

During the celebration I told the team that I look forward to celebrating the 30 year anniversary with them at re:Invent 2034.

Going forward, the team will continue to grow and the goal remains the same: to provide our customers with carefully chosen, high-quality information about the latest and most meaningful AWS launches. The blog is in great hands and this team will continue to keep you informed even as the AWS pace of innovation continues to accelerate.

Thanks Again
Once again I need to thank all of you for the very kind words and gestures over the years. Once in your life, if you work hard and get really lucky, you get a unique opportunity to do something that really and truly matters to people. And I have been lucky.

Jeff;

Introducing a new unified data connection experience with Amazon SageMaker Lakehouse unified data connectivity

Post Syndicated from Chiho Sugimoto original https://aws.amazon.com/blogs/big-data/introducing-a-new-unified-data-connection-experience-with-amazon-sagemaker-lakehouse-data-connectivity/

The need to integrate diverse data sources has grown exponentially, but there are several common challenges when integrating and analyzing data from multiple sources, services, and applications. First, you need to create and maintain independent connections to the same data source for different services. Second, the data connectivity experience is inconsistent across different services. For each service, you need to learn the supported authorization and authentication methods, data access APIs, and framework to onboard and test data sources. Third, some services require you to set up and manage compute resources used for federated connectivity, and capabilities like connection testing and data preview aren’t available in all services. This fragmented, repetitive, and error-prone experience for data connectivity is a significant obstacle to data integration, analysis, and machine learning (ML) initiatives.

To solve for these challenges, we launched Amazon SageMaker Lakehouse unified data connectivity. This feature offers the following capabilities and benefits:

  • With SageMaker Lakehouse unified data connectivity, you can set up a connection to a data source using a connection configuration template that is standardized for multiple services. Amazon SageMaker Unified Studio, AWS Glue, and Amazon Athena can share and reuse the same connection with proper permission configuration.
  • SageMaker Lakehouse unified data connectivity supports standard methods for data source connection authorization and authentications, such as basic authorization and OAuth2. This approach simplifies your data journey and helps you meet your security requirements.
  • The SageMaker Lakehouse data connection testing capability boosts your confidence in established connections. With the ability to browse metadata, you can understand the structure and schema of the data source, identify relevant tables and fields, and discover useful data assets you may not be aware of.
  • SageMaker Lakehouse unified data connectivity’s data preview capability helps you map source fields to target schemas, identify needed data transformation, and plan data standardization and normalization steps.
  • SageMaker Lakehouse unified data connectivity provides a set of APIs for you to use without the need to learn different APIs for various data sources, promoting coding efficiency and productivity.

With SageMaker Lakehouse unified data connectivity, you can confidently connect, explore, and unlock the full value of your data across AWS services and achieve your business objectives with agility.

This post demonstrates how SageMaker Lakehouse unified data connectivity helps your data integration workload by streamlining the establishment and management of connections for various data sources.

Solution overview

In this scenario, an e-commerce company sells products on their online platform. The product data is stored on Amazon Aurora PostgreSQL-Compatible Edition. Their existing business intelligence (BI) tool runs queries on Athena. Furthermore, they have a data pipeline to perform extract, transform, and load (ETL) jobs when moving data from the Aurora PostgreSQL database cluster to other data stores.

Now they have a new requirement to allow ad-hoc queries through SageMaker Unified Studio to enable data engineers, data analysts, sales representatives, and others to take advantage of its unified experience.

In the following sections, we demonstrate how to set up this connection and run queries using different AWS services.

Prerequisites

Before you begin, make sure you have the followings:

  • An AWS account.
  • A SageMaker Unified Studio domain.
  • An Aurora PostgreSQL database cluster.
  • A virtual private cloud (VPC) and private subnets required for SageMaker Unified Studio.
  • An Amazon Simple Storage Service (Amazon S3) bucket to store output from the AWS Glue ETL jobs. In the following steps, replace amzn-s3-demo-destination-bucket with the name of the S3 bucket.
  • An AWS Glue Data Catalog database. In the following steps, replace <your_database> with the name of your database.

Create an IAM role for the AWS Glue job

You can either create a new AWS Identity and Access Management (IAM) role or use an existing role that has permission to access the AWS Glue output bucket and AWS Secrets Manager.

If you want to create a new one, complete the following steps:

  1. On the IAM console, in the navigation pane, choose Roles.
  2. Choose Create role.
  3. For Trusted entity type, choose AWS service.
  4. For Service or use case, choose Glue.
  5. Choose Next.
  6. For Add permissions, choose AWSGlueServiceRole, then choose Next.
  7. For Role name, enter a role name (for this post, GlueJobRole-demo).
  8. Choose Create role.
  9. Choose the created IAM role.
  10. Under Permissions policies, choose Add permission and Create inline policy.
  11. For Policy editor, choose JSON, and enter the following policy:
    {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Effect": "Allow",
                 "Action": [
                     "s3:List*",
                     "s3:GetObject",
                     "s3:PutObject",
                     "s3:DeleteObject"
                 ],
                 "Resource": [
                     "arn:aws:s3:::amzn-s3-demo-destination-bucket/*",
                     "arn:aws:s3:::amzn-s3-demo-destination-bucket"
                 ]
             },
            {
                "Effect": "Allow",
                "Action": [
                    "secretsmanager:GetSecretValue"
                ],
                "Resource": [
                    "arn:aws:secretsmanager:<region>:<account-id>:secret:SageMakerUnifiedStudio-Glue-postgresql_source-*"
                ]
            }
         ]
     }

  12. Choose Next.
  13. For Policy name, enter a name for your policy.
  14. Choose Create policy.

Create a SageMaker Lakehouse data connection

Let’s get started with the unified data connection experience. The first step is to create a SageMaker Lakehouse data connection. Complete the following steps:

  1. Sign in to your SageMaker Unified Studio.
  2. Open your project.
  3. On your project, in the navigation pane, choose Data.
  4. Choose the plus sign.
  5. For Add data source, choose Add connection. Choose Next.
  6. Select PostgreSQL, and choose Next.
  7. For Name, enter postgresql_source.
  8. For Host, enter your host name of your Aurora PostgreSQL database cluster.
  9. For Port, enter your port number of your Aurora PostgreSQL database cluster (by default, it’s 5432).
  10. For Database, enter your database name.
  11. For Authentication, select Username and password.
  12. Enter your username and password.
  13. Choose Add data.

After the completion, it will create a new AWS Secrets Manager secret with a name like SageMakerUnifiedStudio-Glue-postgresql_source to securely store the specified username and password. It also creates a Glue connection with the same name postgresql_source.

Now you have a unified connection for Aurora PostgreSQL-Compatible.

Load data into the PostgreSQL database through the notebook

You will use a JupyterLab notebook on SageMaker Unified Studio to load sample data from an S3 bucket into a PostgreSQL database using Apache Spark.

  1. On the top left menu, choose Build, and under IDE & APPLICATIONS, choose JupyterLab.
  2. Choose Python 3 under Notebook.
  3. For the first cell, choose Local Python, python, enter following code, and run the cell:
    %%configure -f -n project.spark
    {
        "glue_version": "4.0"
    }

  4. For the second cell, choose PySpark, spark, enter following code, and run the cell:
    # Read sample data from S3 bucket
    df = spark.read.parquet("s3://aws-bigdata-blog/generated_synthetic_reviews/data/product_category=Apparel/")
    
    # Preview the data
    df.show()

The code snippet reads the sample data Parquet files from the specified S3 bucket location and stores the data in a Spark DataFrame named df. The df.show() command displays the first 20 rows of the DataFrame, allowing you to preview the sample data in a tabular format. Next, you will load this sample data into a PostgreSQL database.

  1. For the third cell, choose PySpark, spark, enter following code, and run the cell (replace <account-id> with your AWS account ID):
    import boto3
    import ast
    
    # replace you account ID before running this cell
    
    # Get secret
    secretsmanager_client = boto3.client('secretsmanager')
    get_secret_value_response = secretsmanager_client.get_secret_value(
        SecretId='SageMakerUnifiedStudio-Glue-postgresql_source' # replace the secret name if needed
    )
    secret = ast.literal_eval(get_secret_value_response["SecretString"])
    
    # Get connection
    glue_client = boto3.client('glue')
    glue_client_response = glue_client.get_connection(
        CatalogId='<account-id>',
        Name='postgresql_source' # replace the connection name if needed
    )
    connection_properties = glue_client_response["Connection"]["ConnectionProperties"]

  2. For the fourth cell, choose PySpark, spark, enter following code, and run the cell:
    # Load data into the DB
    jdbcurl = "jdbc:postgresql://{}:{}/{}".format(connection_properties["HOST"],connection_properties["PORT"],connection_properties["DATABASE"])
    df.write \
        .format("jdbc") \
        .option("url", jdbcurl) \
        .option("dbtable", "public.unified_connection_test") \
        .option("user", secret["username"]) \
        .option("password", secret["password"]) \
        .save()

Let’s see if you could successfully create the new table unified_connection_test. You can navigate to the project’s Data page to visually verify the existence of the newly created table.

  1. On the top left menu, choose your project name, and under CURRENT PROJECT, choose Data.

Within the Lakehouse section, expand the postgresql_source, then the public schema, and you should find the newly created unified_connection_test table listed there. Next, you will query the data in this table using SageMaker Unified Studio’s SQL query book feature.

Run queries on the connection through the query book using Athena

Now you can run queries using the connection you created. In this section, we demonstrate how to use the query book using Athena. Complete the following steps:

  1. In your project on SageMaker Unified Studio, choose the Lakehouse section, expand the postgresql_source, then the public
  2. On the options menu (three vertical dots) of the table unified_connection_test, choose Query with Athena.

This step will open a new SQL query book. The query statement select * from "postgresql_source"."public"."unified_connection_test" limit 10; is automatically filled.

  1. On the Actions menu, choose Save to Project.
  2. For Querybook title, enter the name of your SQL query book.
  3. Choose Save changes.

This will save the current SQL query book, and the status of the notebook will change from Draft to Saved. If you want to revert a draft notebook to its last published state, choose Revert to published version to roll back to the most recently published version. Now, let’s start running queries on your notebook.

  1. Choose Run all.

When a query finishes, results can be viewed in a few formats. The table view displays query results in a tabular format. You can download the results as JSON or CSV files using the download icon at the bottom of the output cell. Additionally, the notebook provides a chart view to visualize query results as graphs.

The sample data includes a column star_rating representing a 5-star rating for products. Let’s try a quick visualization to analyze the rating distribution.

  1. Choose Add SQL to add a new cell.
  2. Enter the following statement:
    SELECT count() as counts, star_rating FROM "postgresql_source"."public"."unified_connection_test"
    GROUP BY star_rating

  3. Choose the run icon of the cell, or you can press Ctrl+Enter or Cmd+Enter to run the query.

This will display the results in the output panel. Now you have learned how the connection works on SageMaker Unified Studio. Next, we show how you can use the connection on AWS Glue consoles.

Run Glue ETL jobs on the connection on the AWS Glue console

Next, we create an AWS Glue ETL job that reads table data from the PostgreSQL connection, converts data types, transforms the data into Parquet files, and outputs them to Amazon S3. It also creates a table in the Glue Data Catalog and add partitions so downstream data engineers can immediately use the table data. Complete the following steps:

  1. On the AWS Glue console, choose Visual ETL in the navigation pane.
  2. Under Create job, choose Visual ETL.
  3. At the top of the job, replace “Untitled job” with a name of your choice.
  4. On the Job Details tab, under Basic properties, specify the IAM role that the job will use (GlueJobRole-demo).
  5. For Glue version, choose Glue version 4.0
  6. Choose Save.
  7. On the Visual tab, choose the plus sign to open the Add nodes
  8. Search for postgresql and add PostgreSQL as Source.
  9. For JDBC source, choose JDBC connection details.
  10. For PostgreSQL connection, choose postgresql_source.
  11. For Table name, enter unified_connection_test
  1. As a child of this source, search in the Add nodes menu for timestamp and choose To Timestamp.
  2. For Column to convert, choose review_date.
  3. For Column type, choose iso.
  4. On the Visual tab, search in the Add nodes menu for s3 and add Amazon S3 as Target.
  5. For Format, choose Parquet.
  6. For Compression Type, choose Snappy.
  7. For S3 Target Location, enter your S3 output location (s3://amzn-s3-demo-destination-bucket).
  8. For Data Catalog update options, choose Create a table in the Data Catalog and on subsequent runs, update the schema and add new partitions.
  9. For Database, enter your Data Catalog database (<your_database>).
  10. For Table name, enter connection_demo_tbl.
  11. Under Partition keys, choose Add a partition key, and choose review_year.
  12. Choose Save, then choose Run to run the job.

When the job is complete, it will output Parquet files to Amazon S3 and create a table named connection_demo_tbl in the Data Catalog. You have now learned that you can use the SageMaker Lakehouse data connection not only in SageMaker Unified Studio, but also directly in AWS Glue console without needing to create separate individual connections.

Clean up

Now to the final step, cleaning up the resources. Complete the following steps:

  1. Delete the connection.
  2. Delete the Glue job.
  3. Delete the AWS Glue output S3 buckets.
  4. Delete the IAM role AWSGlueServiceRole.
  5. Delete the Aurora PostgreSQL cluster.

Conclusion

This post demonstrated how the SageMaker Lakehouse unified data connectivity works end to end, and how you can use the unified connection across different services such as AWS Glue and Athena. This new capability can simplify your data journey.

To learn more, refer to Amazon SageMaker Unified Studio.


About the Authors

Chiho Sugimoto is a Cloud Support Engineer on the AWS Big Data Support team. She is passionate about helping customers build data lakes using ETL workloads. She loves planetary science and enjoys studying the asteroid Ryugu on weekends.

Noritaka Sekiyama is a Principal Big Data Architect on the AWS Glue team. He is responsible for building software artifacts to help customers. In his spare time, he enjoys cycling with his new road bike.

Shubham Agrawal is a Software Development Engineer on the AWS Glue team. He has expertise in designing scalable, high-performance systems for handling large-scale, real-time data processing. Driven by a passion for solving complex engineering problems, he focuses on building seamless integration solutions that enable organizations to maximize the value of their data.

Joju Eruppanal is a Software Development Manager on the AWS Glue team. He strives to delight customers by helping his team build software. He loves exploring different cultures and cuisines.

Julie Zhao is a Senior Product Manager at AWS Glue. She joined AWS in 2021 and brings three years of startup experience leading products in IoT data platforms. Prior to startups, she spent over 10 years in networking with Cisco and Juniper across engineering and product. She is passionate about building products to solve customer problems.

AWS Weekly Roundup: Amazon EC2 F2 instances, Amazon Bedrock Guardrails price reduction, Amazon SES update, and more (December 16, 2024)

Post Syndicated from Danilo Poccia original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-amazon-ec2-f2-instances-amazon-bedrock-guardrails-price-reduction-amazon-ses-update-and-more-december-16-2024/

The week after AWS re:Invent builds on the excitement and energy of the event and is a good time to learn more and understand how the recent announcements can help you solve your challenges. As usual, we have you covered with our top announcements of AWS re:Invent 2024 post.

You can now watch keynotes and sessions on the AWS Event YouTube channel. This year Andy Jassy, now President and CEO at Amazon, returned to re:Invent and shared some thoughts in these videos.

Drawing on experiences Amazon has had building distributed systems at massive scale, Werner Vogels, VP and CTO at Amazon, shared critical lessons and strategies he has learned for managing complex systems in his keynote.

Last week’s launches
Here are the launches that got my attention.

Amazon Elastic Compute Cloud (Amazon EC2) – A new generation of FPGA-powered instances (F2) is now available. In contrast to a purpose-built chip designed with a single function in mind and then hard-wired to implement it, a field programmable gate array (FPGA) can be programmed in the field, after it has been plugged in to a socket on a PC board. We’re also introducing Amazon EC2 High Memory U7i instances with 6TiB and 8TiB of memory. U7i instances are ideal to run large in-memory databases such as SAP HANA, Oracle, and SQL Server. Graviton-based 8th generation instances now support bandwidth configurations for Amazon VPC and Amazon EBS.

Amazon Bedrock Guardrails – We are reducing pricing by up to 85% to help you implement safeguards for your generative AI applications. Also, we’re adding multilingual capabilities with support for Spanish and French languages.

Amazon Simple Email Services (SES) – Now offers Global Endpoints for multi-region sending resilience and announces the availability of Deterministic Easy DKIM (DEED), a new form of global identity which simplifies the use of DomainKeys Identified Mail (DKIM) management.

AWS CloudFormation – An enhanced version of the AWS Secrets Manager transform introducing automatic AWS Lambda upgrades.

Amazon Lex – Launches new multilingual streaming speech recognition models that enhance recognition accuracy through two specialized groupings: a European-based model (for Portuguese, Catalan, French, Italian, German, and Spanish) and a Asia Pacific-based model (for Chinese, Korean, and Japanese).

Amazon Connect – Now supports push notifications for mobile chat on iOS and Android devices. In this way, you can be proactively notified as soon as there is a new message from an agent or chatbot, even when not actively chatting. You can now also configure holidays and other variances to your contact center hours of operation.

AWS Security Hub – Now supports automated security checks aligned to the Payment Card Industry Data Security Standard (PCI DSS) v4.0.1, a compliance framework that provides a set of rules and guidelines for safely handling credit and debit card information.

AWS Resource ExplorerSupports 59 new resource types including Amazon Elastic Kubernetes Service (Amazon EKS), Amazon Kendra, AWS Identity and Access Management (IAM) Access Analyzer, and Amazon SageMaker.

Amazon SageMaker AI – Inference optimized Amazon EC2 G6e instances (powered by NVIDIA L40S Tensor Core GPUs) and P5e (powered by NVIDIA H200 Tensor Core GPUs) are now available on Amazon SageMaker.

Amazon Redshift – Now supports automatically and incrementally refreshable materialized views on tables in a zero-ETL integration. Previously, in this case, you had to run a full refresh.

AWS Toolkit for Visual Studio Code – Now includes Amazon CloudWatch Logs Live Tail, an interactive log streaming and analytics capability that provides real-time visibility into your logs and makes it easier to develop and troubleshoot applications.

Other AWS news
Here are some additional projects, blog posts, and news items that you might find interesting:

Build a managed transactional data lake with Amazon S3 Tables – Just introduced at re:Invent 2024, Amazon S3 Tables is the first cloud object store with built-in Apache Iceberg support and the easiest way to store tabular data at scale. This post on the AWS Storage Blog provides an overview of S3 Tables and an example of how to build a transactional data lake with S3 Tables using Apache Spark on Amazon EMR.

Introducing Cross-Region Connectivity for AWS PrivateLink – More information on this recent launch that can be used to share and access Amazon Virtual Private Cloud (Amazon VPC) endpoint services across different AWS Regions.

Marc Brooker, VP/Distinguished Engineer at AWS, shared on his personal blog a few posts about what Amazon Aurora DSQL is, how it works, and how to make the best use of it:

That’s all for this week. Check back next Monday for another Weekly Roundup!

Danilo

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

The collective thoughts of the interwebz