The updated whitepaper continues our efforts to help AWS customers navigate APRA’s regulatory expectations in a shared responsibility environment. It is intended for APRA-regulated institutions that are looking to run workloads on AWS and is particularly useful for leadership, governance, security, risk, and compliance teams that need to understand APRA requirements and guidance.
The whitepaper summarizes APRA’s requirements and guidance related to operational risk management and information security. It also gives APRA-regulated institutions information they can use to commence their due diligence and assess how to implement the appropriate programs for their use of AWS.
As the regulatory environment continues to evolve, we’ll provide further updates through the AWS Security Blog and the AWS Compliance page. You can find more information on cloud-related regulatory compliance at the AWS Compliance Center. You can also reach out to your AWS account manager for help finding the resources you need.
If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.
Amazon Q generative SQL brings generative AI capabilities to help speed up deriving insights from your Amazon Redshift data warehouses and AWS Glue Data Catalog, generating SQL for Amazon Redshift or Amazon Athena. With Amazon Q, you get SQL commands generated with your context. This means you can focus on deriving insights faster, rather than having to first learn potentially complex schemas. Without generative SQL, your data analysts might have to frequently switch between different types of SQL, which can further slow analysis down. Amazon Q generative SQL can help by generating SQL statements from natural language and speeding up development. This can help onboard analysts faster and improve analyst productivity. The generative SQL experience is available through Amazon SageMaker Unified Studio and Amazon Redshift Query Editor v2.
To scale the use of generative SQL in production scenarios, you need to consider how relevant and accurate SQL is generated. In doing so, it’s important to understand what data is used and how your information is protected. Amazon Q generative SQL is designed to keep your data secure and private. Your queries, data, and database schemas are not used to train generative AI foundation models (FMs). For more information, see Considerations when interacting with Amazon Q generative SQL.
In the post Write queries faster with Amazon Q generative SQL for Amazon Redshift, we provided general advice around getting started with generative SQL. In this post, we discuss the design and security controls in place when using generative SQL and its use in both SageMaker Unified Studio and Amazon Redshift Query Editor v2.
Solution overview
Generating relevant SQL requires context from your data warehouse or data catalog schemas. Your analysts can ask free text or natural language questions in the Amazon Q chat window and have SQL statements returned that reference your tables and columns. It’s important that the generated SQL is consistent with your schema so that it can find the most relevant fields to answer questions and generate queries that accurately reference data. In SageMaker Unified Studio or Amazon Redshift Query Editor v2, when the Amazon Q chat window is open, database metadata that is viewable under the connection context is made available to Amazon Q for SQL generation. This means that only the schema information that the connecting user can access is used. Tables or database objects the user doesn’t have access to are excluded.
When a user submits questions in the Amazon Q chat window, a search algorithm is used to find the most relevant context from the available database schema metadata information. This context is combined with the user’s question and used as a prompt to a large language model (LLM) to generate a SQL statement. The supporting information is cached so that your data source doesn’t need to be queried every time a user initiates SQL generation. Instead, data source metadata will be periodically refreshed if it remains in use, or you can trigger a manual refresh. If the data is not being used, Amazon Q will automatically delete it. Where applicable, the information used to support SQL generation is encrypted with an AWS Key Management Service (AWS KMS) customer managed KMS key where one has been specified in the SageMaker Unified Studio or Amazon Redshift Query Editor v2 settings. Otherwise, an AWS managed key is used. Your information is encrypted in transit and at rest.
The following diagram shows the process flow for SQL generation when using SageMaker Unified Studio or the Amazon Redshift Query Editor and using Amazon Redshift or Data Catalog source data.
The Amazon Q generative SQL process can be summarized as the following steps:
A user interacts with the Amazon Q chat pane through SageMaker Unified Studio or the Amazon Redshift Query Editor.
The SQL chat frontend sends the prompt along with the connection configuration to Amazon Q.
Amazon Q uses the connection context to retrieve information that will support SQL generation if this data is not already available.
Amazon Q encrypts the retrieved information under the appropriate AWS managed or customer managed KMS key. The information is subsequently decrypted on retrieval.
The information is stored along with custom context information, if this has been provided.
Relevant context from the combined information is selected and added to the user’s questions and sent to an LLM to generate a SQL statement, which is returned to the user.
The user can decide whether to run the statement and can provide feedback on usefulness and accuracy.
Additional context to enhance SQL generation
You can provide further context to supplement the database schema information, which can help improve the accuracy and relevancy of the generated SQL.
One option is to provide custom context. Custom context gives the option to specify instructions and extra information, such as descriptions of tables and columns. These descriptions can then be used to help the selection of relevant tables and attributes when generating SQL statements. This is particularly relevant when your schema uses more obscure naming that might not directly relate to business terms or uses non-standard abbreviations. For example, consider a table called sls_r1_2024. With custom context, you can add a table description specifying that, for example, the table includes sales information across stores in the US region for the calendar year 2024. This information can help the LLM generate SQL referencing the correct tables. The same approach can be applied to columns within the table. Your custom context is encrypted using a customer managed KMS key if one has been specified (during Amazon Redshift Query Editor account creation or SageMaker Unified Studio project creation) or an AWS managed key otherwise.
You can also introduce constraints using custom context. For example, you can explicitly include or exclude specific schemas, tables, or columns from SQL query generation. Similarly, specific topics can also be disallowed, such as not generating SQL statements to support financial reporting. For more details about the information that can be supplied, refer to Custom context.
Another option is to grant SQL query history access to the user establishing the connection. This information is then also made available to enhance SQL generation and to provide the LLM with examples of relevant queries. Be aware that granting wider SQL query history access to the connecting user, and therefore also the generative SQL workflow, allows viewing of queries over tables or objects the user might not have access to. Furthermore, string literals might be present in historic statements that might contain sensitive information. To help mitigate this risk, you could instead use the CuratedQueries section of custom context to provide predefined question and answer examples, without exposing all user queries.
Generated statement response
Before a SQL statement is returned to the user, Amazon Q tries to detect syntax issues. This step helps improve the likelihood that only valid SQL syntax is returned. Amazon Q will use the available information for the user to return statements that align with user permissions, to reduce scenarios where users can’t run generated statements. For example, if you have given access to SQL query history information, then the SQL generation step might produce a query statement referencing a table that the user asking the question doesn’t have access to. Amazon Q minimizes the occurrence of this scenario by assessing if the generated SQL aligns with user permissions and updating the statement if not. User permissions are not bypassed through the use of Amazon Q generative SQL. If a statement was returned referencing a table the user doesn’t have access to, the authorization applied to the user will enforce access control when the statement is executed.
Statements generated by Amazon Q that could potentially change your database, such as DML or DDL statements, are returned with a warning. The warning highlights to the user that running the statement could potentially modify the database. Again, these statements are only executable if the user has the required permissions.
Prerequisites
Amazon Q generative SQL works with your Redshift data warehouses and Data Catalog tables. To get started, you should have data available in either or both of these environments. To use Amazon Q generative SQL with your AWS Glue tables, you need a SageMaker Unified Studio domain. Within your domain, you can use the Amazon Q chat integration to ask questions of your data and have SQL generated. This also works for Amazon Redshift data sources available in the domain. You can use Amazon Q generative SQL without a SageMaker Unified Studio domain using the Amazon Redshift Query Editor. Access to the editor enables Amazon Q chat integration against your Amazon Redshift data sources.
Enable Amazon Q generative SQL
You can control access to generative SQL at the account-Region level in the Amazon Redshift Query Editor or at the SageMaker Unified Studio domain level. To enable this feature, an account admin must explicitly turn on Amazon Q generative SQL. By default, the feature is not accessible to your users. Administrators that have permission for the sqlworkbench:UpdateAccountQSqlSettingsAWS Identity and Access Management (IAM) action can turn the Amazon Q generation SQL feature on or off through the admin window, as illustrated in the following sections. When turned off, this will restrict users from opening the Amazon Q chat pane and help prevent interaction with generative SQL.
Enable Amazon Q in your SageMaker domain
To enable Amazon Q in your SageMaker domain, you can navigate to the Amazon Q tab on the domain settings page and choose to enable the service. For more information, see Amazon Q in Amazon SageMaker Unified Studio.
Enable Amazon Q in Amazon Redshift
To enable Amazon Q generative SQL from the Amazon Redshift Query Editor, access the Amazon Q generative SQL settings. This requires the administrator to have the sqlworkbench:UpdateAccountQSqlSettings permission in their IAM policy. For more information, see Updating generative SQL settings as an administrator.
With generative SQL enabled at the account-Region level, you can restrict access to specific users with IAM controls. IAM administrators can build IAM policies that allow or deny access to the action sqlworkbench:GetQSqlRecommendations. For more information, refer to Actions, resources, and condition keys for AWS SQL Workbench. Policies can then be associated with IAM users or roles to control access to SQL generation at a more granular level. An appropriately scoped service control policy (SCP) can be used to limit access to SQL generation to specific accounts within your organization if required.
The following is an example policy denying access to use SQL generation:
Amazon Q Developer uses cross-Region inference to distribute traffic across different AWS Regions, which provides increased throughput and resilience during high demand periods, improved performance, and access to the latest Amazon Q Developer capabilities.
When a request is made from an Amazon Q Developer profile, it is kept within the Regions in the same geography as the original data. Although this doesn’t change where the data is stored, the requests and output results might move across Regions during the inference process. Data is encrypted when transmitted across Amazon’s network. For more information on cross-Region inference, see Cross-region processing in Amazon Q Developer.
Monitoring
To monitor which IAM users or roles are interacting with generative SQL, you can use AWS CloudTrail. CloudTrail monitors API calls and logs which identities have performed particular actions. When a user first asks a question, a CloudTrail event is emitted called IngestQSqlMetadata. This is a result of Amazon Q starting the metadata ingest process. Ingestion is an asynchronous operation, so there might be a series of GetQSqlMetadataStatus events. This is due to the workflow checking the ingestion process status.
After the workflow has completed successfully, each question sees a GetQSqlRecommendation event. This is the result of users submitting questions and triggering generation of SQL statements. The following is an example CloudTrail event for GetQSqlRecommendation. In this example, Amazon Q emits detailed CloudTrail events highlighting the warehouse being queried, IAM principal calling Amazon Q, and the entire response structure from Amazon Q in responseElements:
In this post, we discussed the Amazon Q generative SQL workflow. We highlighted the process around using your schema context alongside metadata such as historic SQL queries and custom context. Using this metadata allows the generation of relevant SQL that helps accelerate your analyst’s productivity. Although it’s important to assist analysts, it’s also imperative to make sure data remains secure and protected. To support this, generative SQL uses only the data the connected user has access to. This helps prevent exposure to information beyond their authorization.When you’re looking to increase the relevance of generated SQL through sharing additional query history, it’s important to consider the trade-off of exposing additional information to the user. Deciding your approach here should take into account the domain context of the data and the possible exposure of metadata the user doesn’t have access to, or potentially sensitive information that might appear in query strings. Keeping these considerations in mind can help you achieve the appropriate security posture for your workloads.
Gregory Knowles is a data and AI specialist solution architect at AWS, focusing on the UK public sector. With extensive experience in cloud-based architectures, Greg guides public sector customers in implementing modern data solutions. His expertise spans governance, analytics, and AI/ML. Greg’s passion lies in accelerating transformation and innovation to improve productivity and outcomes. He has successfully led projects that moved data systems into the cloud, adopted new data architectures, and implemented AI at scale in production.
Abhinav Tripathy is a Software Engineer and Security Guardian at AWS, where he develops Amazon Q generative SQL by combining machine learning, databases, and web systems. Abhinav is passionate about building scalable web systems from scratch that solve real customer challenges. Outside of work, he enjoys traveling, watching soccer, and playing badminton.
Erol Murtezaoglu is a Technical Product Manager at AWS, is an inquisitive and enthusiastic thinker with a drive for self-improvement and learning. He has a strong and proven technical background in software development and architecture, balanced with a drive to deliver commercially successful products. Erol highly values the process of understanding customer needs and problems, in order to deliver solutions that exceed expectations.
И така, през 2016 г. започва проектът за разчитане на ДНК на Леонардо да Винчи, с който вече почти десет години учените целят да създадат пълна „карта“ на неговия генетичен код.
Проектът
Целта на „ДНК на Леонардо да Винчи“ е да се определи дали останките от замъка Амбоаз, за които се твърди, че са именно на художника, в действителност са негови. Това би могло да стане, като се сравни наличната ДНК с тази на негови живи родственици. Проектът има още една амбициозна задача – да секвенира целия геном от останките на Леонардо (в случай че са негови), като така се изясни защо е бил толкова талантлив и дали вниманието му към детайлите се дължи на определени генетични заложби, свързани с неговото зрение. Допълнителните цели на проекта включват изследване на проби от частни и публични колекции от произведения на изкуството от периода на Ренесанса.
Какво може да ни каже ДНК на живите родственици на Леонардо?
Историците са единодушни, че Леонардо да Винчи не е оставил потомство, но пък баща му е имал няколко деца от две различни жени, така че родственици не липсват. По-важни за изследването са тези от мъжки пол. Защо? X и Y хромозомите определят пола. Наличието на една X и една Y хромозома определя мъжки пол. Y хромозомата се предава директно от баща на син и затова е идеално средство за проследяване на родственици по бащина линия.
Всички автозомни хромозоми (тези, които не са свързани с определяне на пола) са в две копия – едното се унаследява от майката, другото – от бащата. Но X и Y хромозомите не се унаследяват „чисти“, а обменят генетичен материал помежду си (т.нар. рекомбинация). Ето защо всеки човек притежава уникална комбинация от генетичния материал на родителите си. Y хромозомата обаче има специфична структура и голяма част от нея не се рекомбинира с X хромозомата по време на репродукция.
Поради този факт Y хромозомата е най-подходяща за проследяване на родствениците на Леонардо. При внимателното проучване са били идентифицирани както починали, така и живи мъже, за които се предполага, че са носители на Y хромозома от родовата линия на Да Винчи. След като се проследят колкото може повече от тях, ще може да се сравнят техните Y хромозоми с Y хромозомата от пробата от останките и да се установи с максимална точност дали те наистина са на Леонардо.
Проследяване на Y хромозомата при роднините на Леонардо
Анализът на Y хромозомите на живите родственици на Леонардо може да допринесе за изясняване на цялата последователност на неговата Y хромозома. За да се подкрепят предположенията, че определени останки или археологически находки принадлежат на художника, е необходимо да се проучат хаплотипите на Y хромозомата, носени от живите родственици. Хаплотипът представлява част от генетичните вариации (алели), която се наследява от единия родител (в случая от бащата, тъй като Y хромозомата се предава от баща на син). Хаплотипите се групират в хаплогрупи. Y хаплогрупите представляват сбор от общи хаплотипи, унаследявани в поколенията.
Смята се, че мутациите в Y хромозомите и в генома на митохондриите се срещат няколко пъти по-често от мутациите в останалите хромозоми. Проектът „ДНК на Леонардо да Винчи“ предлага важна възможност за изясняването на мутациите в Y хромозомата, унаследени в продължение на няколко поколения.
В бъдеще учените биха могли да проучват произведения, създадени в работилницата на Леонардо да Винчи. Единственият останал генетичен материал по тях, който може да се изследва, е от пръстови отпечатъци. Сред най-новите методи за секвениране е този, при който се използва една-единствена клетка. ДНК, съдържаща се в тази клетка, се размножава многократно (амплификация). Цялостното геномно секвениране чрез амплификация на една клетка се използва в доста области на съвременната молекулярна биология, но все още не може да се приложи за изследването на древна ДНК или на ДНК, извлечена от пръстови отпечатъци.
Ролята на единичните разлики в генома при развитието на висок коефициент за интелигентност
Учените изказват и друга хипотеза: че не става въпрос за носителство на множество SNPs, а че отделни единични нуклеотидни полиморфизми са силно обвързани с интелигентността. Тоест не е въпрос толкова на количество на SNPs, а на качество. Тази хипотеза обаче е трудно доказуема, защото има хиляди SNPs и индивидуалният им ефект върху интелигентността е пренебрежимо малък.
Към момента подходът за доказване на гения на Леонардо посредством търсене на SNPs се е изчерпал, тъй като той се дължи на множество други фактори. Затова учените се насочват към изследването на епигенетични белези и малки дупликации. Епигенетиката изучава химични съединения, които маркират участъци от ДНК с цел определяне на момента на активиране на дадени гени. Епигенетичните промени зависят от фактори като околната среда и стреса. За разлика от мутациите в гените, епигенетичните промени са обратими.
Въпреки фокуса върху епигенетичните белези, и те не са идеалният избор, защото не са така стабилни и непроменливи при предаване в поколенията като SNPs. За сравнение, причинените от стрес епигенетични промени са обратими в рамките на едно поколение, а SNPs остават постоянни.
Надеждата да се получи полезна информация от SNPs анализа и да разберем повече за гения на Леонардо е все още малка – дори десет години след началото на проекта. За момента се разглежда възможността за специални алели, кодиращи калиеви канали в ретината, заради които се предполага, че Да Винчи е притежавал по-особено зрение.
Ползите от проекта „ДНК на Леонардо да Винчи“
Те не се ограничават само до изясняването конкретно на неговия генетичен код. Опитите за възстановяването на пълна генетична информация от минимални количества увредени нуклеинови киселини (ДНК и РНК) водят до разработването на нови и още по-прецизни молекулярнобиологични техники.
Един от следващите подходи, към които се насочват учените, участващи в проекта, е възстановяването на ДНК от тетрадките и рисунките на Леонардо. Тази идея отваря врати към изследването на архивите и библиотеките. На теория това е напълно постижимо, тъй като вече е извличана РНК (по-нестабилна от ДНК) от много стари находки. Например преди повече от десет години музейни образци бяха използвани за получаване на транскриптома (цялата РНК в клетката) на изчезнал тасманийски тигър. Това доказва, че при определени обстоятелства ДНК и РНК могат да останат стабилни за продължителен период.
Предстои да научим и за възможно най-детайлното разчитане на Y хромозомата на Леонардо. Всички проучвания, свързани с унаследяването на Y хромозомата в поколенията, ще дадат информация, която ще послужи за разработването на методи за по-точно прослeдяване на родственици от цели популации и за изясняване как Y хромозомите се променят през поколенията.
Manufacturing organizations are racing to digitize their operations through Industry 4.0 initiatives. A key challenge they face is capturing, processing, and analyzing real-time data from industrial equipment to enable data-driven decision making.Modern manufacturing facilities generate massive amounts of real-time data from their production lines. Capturing this valuable data requires a two-tier architecture: first, an edge device that understands industrial protocols collects data directly from the shop floor sensors. Then, these edge gateways securely buffer and transmit the data to AWS Cloud, providing reliability during network interruptions.
In this post, we show how to use AWS service integrations to minimize custom code while providing a robust platform for industrial data ingestion, processing, and analytics. By using Amazon S3 Tables and its built-in optimizations, you can maximize query performance and minimize costs without additional infrastructure setup. Additionally, AWS IoT Greengrass supports VPC endpoints, and you can securely communicate between the edge gateway (hosted on premises) and AWS.
Solution overview
Let’s consider a manufacturing line with and equipment sensors capturing flow rate, temperature, and pressure. To perform analysis on this data, you ingest real-time streaming data from these sensors into the AWS environment using an edge gateway. After data lands in AWS, you can use various analytics services to gain insights.
To demonstrate the data flow from the edge to the cloud, we have assets, machines, and tools publish data using MQTT. Optionally, we use a simulated edge device that publishes data to a local MQTT endpoint. We use an edge gateway with an AWS IoT Greengrass V2 edge runtime to stream data through Amazon Data Firehose in the cloud to S3 Tables.
The following diagram illustrates the solution architecture.
Fig 1 : High Level Architecture
The workflow consists of the following steps:
Collect data from Internet of Things (IoT) sensors and stream real-time data from edge devices to the AWS Cloud using AWS IoT Greengrass.
Store and organize the tabular data using S3 Tables, which provides purpose-built storage for Apache Iceberg format with a simple, performant, and cost-effective querying solution.
Query and analyze the tabular data using Amazon Athena.
The edge data flow consists of the following key components:
IoT device to local MQTT broker – A simulated device used to generate data for the purposes of this post. In a typical production implementation, this would be your equipment or gateway that supports MQTT. IoT devices can publish messages to a local MQTT broker (Moquette) running on AWS IoT Greengrass.
Forwards messages to the kinesisfirehose/message topic.
Uses the IPC interface to subscribe to messages.
Firehose component – The Firehose component subscribes to the kinesisfirehose/message topic. The component then streams the data to Data Firehose in the cloud. It uses QoS 1 for reliable message delivery.
You can scale this solution to multiple edge locations, so you have a seamless view of data across multiple locations of the manufacturing site, as a low-code solution.In the following sections, we walk through the steps to configure the cloud data ingestion flow:
Create an S3 Tables bucket and enable integration with AWS analytics services.
Grant Lake Formation permissions for Athena access.
Query the table to verify data ingestion.
Prerequisites
You must have the following prerequisites:
An AWS account
The required IAM privileges to launch AWS IoT Greengrass on an edge gateway (or another supported device)
An Amazon Elastic Compute Cloud (Amazon EC2) instance with a supported operating system to perform a proof of concept
Install AWS IoT Greengrass on the edge gateway
For instructions to install AWS IoT Greengrass, refer to Install the AWS IoT Greengrass Core software. After you complete the installation, you will have a core device provisioned, as shown in the following screenshot. The status of the device says Healthy, which means that your account is able to communicate with the device successfully.
Because you’re using AWS IoT Greengrass to stream data, you can skip the Kinesis Data Generator steps mentioned in these tutorials. The data will instead flow from your edge devices through the Greengrass components to Data Firehose.After you complete these steps, you will have a Firehose stream and S3 Tables bucket, as shown in the following screenshot. Note the Amazon Resource Name (ARN) of the Firehose stream to use in subsequent steps.
Fig 3: Amazon Data Firehose Stream
Deploy the Greengrass components
Complete the following steps to configure and deploy the Greengrass components. For more details, refer to Create deployments.
Use the following configuration to enable message routing from local MQTT to the AWS IoT Greengrass PubSub component. Note the topic in the code. This is the MQTT topic where the devices will send the data to.
Use the following configuration to deploy the legacy subscription router component (Note that this is a dependent component to the Firehose component):
Create and deploy a custom PubSub component. You can use the following sample code snippet in your preferred language to deploy as a Greengrass component. You can use gdk to create custom components.
After you deploy the components, you will see them on the Components tab of your core device.
Fig 4: AWS IoT Greengrass components
Ingest data
In this step, you ingest the data from your device to AWS IoT Greengrass, which will subsequently land in Data Firehose. Complete the following steps:
From your edge device that is MQTT aware, or your edge gateway, publish the data to the topic defined earlier ( client/#). For example, we publish the data to the client/devices/telemetry MQTT topic.
For additional details on how to publish messages from a sample device, refer to Just-in-time provisioning.
The MQTT bridge component will route the payload from the MQTT topic (client/devices/telemetry) to an IPC topic by the same name. The custom component that you deployed earlier will listen to the IPC topic client/devices/telemetry and publish to the IPC topic kinesisfirehose/message. The message must follow the structure described in Input data.
Validate the data in Athena
You can now query the data published from the edge IoT device using Athena. On the Athena console, find the catalog and database that you set up, and run the following query:SELECT * FROM <<database>>."device_telemetry" limit 10;You should see the data displayed as shown in the following screenshot. Note the database and table name that you had defined as part of the “Provision a Data Firehose” stream step.
Fig 5: Validate Data in Athena
Scale out the solution
In the preceding sections, we showed how multiple equipments can ingest data into the cloud using a single Greengrass edge gateway device. Because manufacturing locations are distributed in a real-world scenario, you might set up Greengrass devices at other sites and publish the data to the same Firehose stream. This makes sure the data from different sites is landed into a single S3 bucket, is partitioned appropriately (Device_Id in our example), and can be queried seamlessly.
Clean up
After you validate the results, you can delete the following resources to avoid incurring additional costs:
In this post, we showed how to set up a scalable edge-to-cloud near real-time data ingestion framework using AWS IoT Greengrass and start performing analytics on the data within AWS services using a low-code approach. We demonstrated how to optimize the data storage into Iceberg format with S3 Tables, and transform the streaming data before it lands on the storage layer using Data Firehose. We also discussed how you can scale this solution horizontally across multiple manufacturing locations (plants or sites) to create a low-code solution to analyze data in near real time.
Joyson Neville Lewis is a Sr. Conversational AI Architect with AWS Professional Services. Joyson worked as a Software/Data engineer before diving into the Conversational AI and Industrial IoT space. He assists AWS customers to materialize their AI visions using Voice Assistant/Chatbot and IoT solutions.
Anil Vure is a Sr. IoT Data Architect with AWS Professional services. Anil has extensive experience building large-scale data platforms and works with manufacturing customers designing high-speed data ingestion systems.
Ashok Padmanabhan is a Sr. IoT Data Architect with AWS Professional Services. Ashok primarily works with manufacturing and automotive customers to design and build Industry 4.0 solutions.
This post was written with John Spencer, Sreeram Thoom, and Dipankar Kushari from Databricks.
Organizations need seamless access to data across multiple platforms and business units. A common scenario involves one team using Amazon EMR for data processing while needing to access data that another team manages in Databricks Unity Catalog. Traditionally, this would require data duplication or complex manual setups.
Although both Amazon EMR and Databricks Unity Catalog are powerful tools on their own, integrating them effectively is crucial for maintaining strong data governance, security, and operational efficiency. In this post, we demonstrate how to achieve this integration using Amazon EMR Serverless, though the approach works well with other Amazon EMR deployment options and Unity Catalog OSS.
EMR Serverless makes running big data analytics frameworks straightforward by offering a serverless option that automatically provisions and manages the infrastructure required to run big data applications. Teams can run Apache Spark and other workloads without the complexity of cluster management, while providing cost-effective scaling based on actual workload demands and seamless integration with AWS services and security controls.
Databricks Unity Catalog serves as a unified governance solution for data and AI assets, providing centralized access control and auditing capabilities. It enables fine-grained permissions across workspaces and cloud platforms, while supporting comprehensive metadata management and data discovery across the organization, and can complement governance tools like AWS Lake Formation.
To enable Amazon EMR to process data maintained in Unity Catalog, the data team traditionally copies data products across the platforms to a location accessible by Amazon EMR. The practice of data duplication not only leads to increased storage costs, but also severely impacts data quality and makes it challenging to effectively enforce same governance policies across different systems, track data lineage, enforce data retention policies, and maintain consistent access controls across the organization.
Now using Unity Catalog’s Open REST APIs, Amazon EMR customers can read from and write to Databricks Unity Catalog and Unity Catalog OSS tables using Spark, enabling cross-platform interoperability while maintaining governance and access controls across Amazon EMR and Unity Catalog.
Solution overview
In this post, we will provide an overview of EMR Spark workload integration with Databricks Unity Catalog and walk through the end-to-end process of reading from and writing to Databricks Unity Catalog tables using Amazon EMR and Spark. We show you how to configure EMR Serverless to interact with Databricks Unity Catalog, run an interactive Spark workload to access the data, and run an analysis to derive insights.
The following diagram illustrates the solution architecture.
Storage for EMR Serverless. We use an Amazon Simple Storage Service (Amazon S3) bucket to store output files and logs from the Spark workload that you will run using an EMR Serverless application. For instructions to create a bucket, see Creating a general purpose bucket.
Set up a principal that will be configured with Amazon EMR for data access.
Grant the principal the privilege to configure the integration of the EXTERNAL USE SCHEMA privilege on the schema containing the objects. For instructions, see Grant a principal EXTERNAL USE SCHEMA.
For Iceberg tables, create an EMR Serverless application called dbx-demo-application-iceberg with version 7.8.0 or higher. Make sure to deselect Use AWS Glue Data Catalog as Metastore under Additional Configurations, Metastore configuration. Add the following Spark configuration (see Configure applications). Provide the name of the catalog in Unity Catalog that contains your tables and the URL of the Databricks workspace.
For Delta tables, create an EMR Serverless application called dbx-demo-application and version 7.8.0 or higher. Make sure to deselect Use AWS Glue Data Catalog as Metastore under Additional Configurations, Metastore configuration. Add the following Spark configuration (see Configure applications). Provide the name of the catalog in Unity Catalog that contains your tables and the URL of the Databricks workspace.
Launch the workspace created in the previous step. Download the notebooks create-delta-table and create-iceberg-table and upload them to the EMR Studio workspace.
The create-delta-table.ipynb notebook configures the metastore properties to work with Delta tables. The create-iceberg-table.ipynb notebook configures the metastore properties to work with Iceberg tables.
Add the generated token to the session.
For a production deployment, store the PAT in Secrets Manager.
For Iceberg tables, connect to the EMR Serverless application dbx-demo-application-iceberg with the runtime role created in earlier steps under compute and run the notebook (create-iceberg-table). Select PySpark as the kernel and execute each cell in the notebook by choosing the run icon. Refer to Submit a job run or interactive workload for further details about how to run an interactive notebook.
We use the following code to create an external Iceberg table in the catalog:
CREATE SCHEMA IF NOT EXISTS customerschema;
USE SCHEMA customerschema;
CREATE TABLE IF NOT EXISTS iceberg_customer (id string, name string, country string) USING iceberg;
insert into iceberg customer values('1','Alice','US');
For Delta tables, connect to the EMR Serverless application dbx-demo-application with the runtime role created in earlier steps and run the notebook (create-delta-table). Select PySpark as the kernel and execute each cell in the notebook by choosing the run icon. Refer to Submit a job run or interactive workload for further details about how to run an interactive notebook.
We use the following code to create an external Delta table in the catalog:
CREATE SCHEMA IF NOT EXISTS customerschema;
USE SCHEMA customerschema;
CREATE TABLE IF NOT EXISTS delta_customer (id int, name string, country string) USING delta LOCATION ‘s3://<bucket_name>/emr-dbx/external/customerschema/delta_customer’;
insert into delta_customer values(1,'Bob','US');
Verify in Databricks for both Iceberg and Delta tables
Now you can run queries in Databricks Unity Catalog to show the records inserted into the Iceberg and Delta tables from EMR Serverless:
Log in to your Databricks workspace.
Choose SQL Editor in the navigation pane.
Run queries for both Iceberg and Delta tables.
Verify the results show the same as what you saw in the Jupyter notebook in EMR Studio.
The following screenshot shows an example of querying the Iceberg table.
The following screenshot shows an example of querying the Delta table.
Clean up
Clean up the resources used in this post to avoid additional charges:
Delete the IAM roles for this post.
Delete the EMR applications and EMR Studio setup created for this post.
Delete the resources created in Unity Catalog.
Empty and then delete the S3 bucket.
Summary
In this post, we demonstrated the powerful interoperability between Amazon EMR and Databricks Unity Catalog by walking through how to enable external access to Unity Catalog, configure EMR Spark to connect seamlessly with Unity Catalog, and perform DML and DDL operations on Unity Catalog tables using EMR Serverless.
Venkatavaradhan (Venkat) Viswanathan is a Global Partner Solutions Architect at Amazon Web Services. Venkat is a Technology Strategy Leader in Data, AI, ML, generative AI, and Advanced Analytics. Venkat is a Global SME for Databricks and helps AWS customers design, build, secure, and optimize Databricks workloads on AWS.
Srividya Parthasarathy is a Senior Big Data Architect on the AWS Lake Formation team. She works with the product team and customers to build robust features and solutions for their analytical data platform. She enjoys building data mesh solutions and sharing them with the community.
Ramkumar Nottath is a Principal Solutions Architect at AWS focusing on Analytics services. He enjoys working with various customers to help them build scalable, reliable big data and analytics solutions. His interests extend to various technologies such as analytics, data warehousing, streaming, data governance, and machine learning. He loves spending time with his family and friends.
John Spencer is a Product Manager at Databricks, dedicated to making Unity Catalog work seamlessly with customers’ ecosystems of tools and platforms so they can easily access, govern, and use their data.
Sreeram Thoom is a Specialist Solutions Architect at Databricks helping customers design secure, scalable applications on the Data Lakehouse.
Dipankar Kushari is a specialist solutions architect at Databricks helping customer architect and build secured applications on Data Lakehouse.
Enterprise customers of Amazon OpenSearch Service require comprehensive security controls with seamless authentication and authorization mechanisms when accessing data in provisioned domains and Amazon OpenSearch Serverless collections. Security teams within these organizations must not only maintain compliance with enterprise policies but also need to make sure that their users can access data securely, with robust identity management. AWS IAM Identity Center is a popular mechanism for identity management that provides single sign-on (SSO) capabilities for these enterprise customers. IAM Identity Center can use Security Assertion Markup Language (SAML) with both OpenSearch Service provisioned domains and OpenSearch Serverless. Now, by using trusted identity propagation, IAM Identity Center provides a new, direct method for accessing data in OpenSearch Service.
In this post, we outline how you can take advantage of this new access method to simplify data access using the OpenSearch UI and still maintain robust role-based access control for your OpenSearch data.
Trusted identity propagation overview
Trusted identity propagation in IAM Identity Center adds the identity context of a user to a role when accessing OpenSearch Service, which in turn uses this context to authorize and scope OpenSearch data access. This simplifies the authentication and authorization flow for customers because the applications access the data on their behalf. Users or user agents need not be present between the application and the backend services for this authorization to happen, unlike methods like SAML where a user agent needs to be present between these entities as a go-between for exchanging assertions. This flexibility helps simplify accessing a wide variety of data sources such as data residing within the Amazon Virtual Private Cloud (Amazon VPC) of an OpenSearch Service domain, or an OpenSearch Serverless collection. By using the OpenSearch UI, you can additionally simplify the backend connections, resulting in seamless access to the data. The following figures shows how the identity propagation works with OpenSearch Service.
Prerequisites
Before starting to use IAM Identity Center with OpenSearch Service, there are a few options that you must enable. To start, set up an organization or account instance of IAM Identity Center following the instruction in this guide. For OpenSearch Service-provisioned domains, you must enable the IAM Identity Center (IDC) Authentication –new option. You can do this though AWS CloudFormation, OpenSearch REST API, AWS SDK, or the AWS Management Console.
To enable Identity Center using the console
To add the capability for an existing provisioned domain, go to the OpenSearch Service console and navigate to the Security configuration tab and choose Edit.
After this step, or if you are creating new domain, select the check box for IAM Identity Center (IDC) Authentication – new.
You have various options to choose for Subject Key and Roles Key depending upon how you want to establish your role-based access control discussed later in this post. For now, select UserName for Subject Key and GroupName for Roles Key.
For OpenSearch Serverless, choose Serverless in the navigation pane, then Security and Authentication. Choose Edit in the IAM Identity Center (IdC) authentication – new section.
Select the checkbox for Authenticate with IAM Identity Center, and then choose Save.
Select the checkbox for Authentication with IAM Identity Center under Single sign-on authentication, when creating an OpenSearch UI application. For step-by-step instructions on how to create an OpenSearch UI application, see Creating an OpenSearch UI application
After these steps, you’re ready to configure IAM Identity Center by creating new users and groups, or by using existing user identities.
Propagating IAM Identity Center identities
Currently, adding single sign-on authentication with IAM Identity Center can be done while setting up a new OpenSearch UI application. Use the following steps to create a new OpenSearch UI application. Note that single sign-on currently cannot be turned on after an application is created. After single sign-on is enabled, you should see an AWS managed application under Applications in the Identity Center console.
Assigning users and groups
After the application is created and the status shows as Active, you need to assign users and groups to the application. This assignment is important and recommended because these assignments determine the scope and permissions for data access within OpenSearch Service. To do this, select the application you created in the previous steps in the OpenSearch Service console. Here, you will see an option for IAM Identity Center user and groups under Single sign-on authentication. Choose Assign users and groups and select the appropriate Identity Center users and groups.
For OpenSearch Serverless, you must create a new data access policy or add a rule to an existing one to grant IAM Identity Center principals appropriate permissions to access the collections. For example, the following figure shows a data access policy that grants specific permissions to a one user with Rule 1 and provide a more restrictive permission to a group with Rule 2.
At this point your OpenSearch Service domains, OpenSearch Serverless collections and OpenSearch UI are set up for identity propagation.
Fine grained access control for IAM Identity Center identities
Fine grained access control is a role-based access control for OpenSearch Service that provides security at index, document, and field levels for provisioned domains. You can choose what aspects of identity context you propagate to OpenSearch Service. You can choose between UserId, UserName, and Email for your Subject keys, and GroupId and GroupName for your Roles key. This configuration is important because the values of the properties in the identity context are used to match exactly with the user and backend role mapping within OpenSearch Service provisioned domains. Note that if IAM Identity Center sign-on isn’t enabled, OpenSearch Service can only evaluate the request signature with AWS signature Version 4. This means that the role your OpenSearch UI will use won’t contain identity context for authorization. To complete authorization, add the values of the identity context fields to the OpenSearch role mapping. See Mapping roles to users under Managing permissions. Role mapping can be done using OpenSearch REST API, AWS SDK, or using OpenSearch Dashboards.
To map roles using OpenSearch Dashboards
From the menu icon on the top left corner or your screen, select Management, Security, Roles, <Your role>.
Choose the Mapped Users tab and select Manage mapping.
When mapping the role, make sure that you enter the values corresponding to the Subject key. This value must be the same as in your identity context. Additionally, use the Roles key to assign access-based IAM Identity Center groups.
With OpenSearch Serverless, the granularity of access control is at the index level so you will need to add additional rules in the Data Access Policy to control principals who can access collections or indices within a collection.
Verifying identity propagation
The final step is to verify identity propagation.
Open the OpenSearch UI application and select IAM Identity Center from the Login drop down.
After you complete the login process with IAM Identity Center, the OpenSearch UI will open. Choose the user icon in the lower left corner of the screen to verify that it’s your correct principal from Identity Center. It should match the Identity Center property you chose earlier.
To verify correct identity propagation, choose the Dev tools icon just above the user profile icon in the bottom left corner of the screen.
Select the correct OpenSearch domain or OpenSearch Serverless collection data source in the top right corner of the screen and run a _search query. You should see results from the data source confirming that the identity is correctly propagated to OpenSearch Service.
Simplified authentication: By eliminating the need for user agents between applications and backend services, the solution streamlines the authentication process compared to traditional SAML-based approaches.
Enhanced security: The integration maintains comprehensive security controls while providing seamless authentication and authorization mechanisms for both OpenSearch Service provisioned domains and Amazon OpenSearch Serverless collections.
Flexible identity management: Organizations can use existing IAM Identity Center implementations to manage user access, making it easier to maintain compliance with enterprise security policies.
Fine-grained access control: The solution supports detailed access control at the index, document, and field level for provisioned domains, allowing organizations to implement precise security measures.
Get started implementing this solution in your environment today!
For more information about identity management and security best practices with OpenSearch Service, we recommend:
Muthu Pitchaimani is a Search Specialist with Amazon OpenSearch Service. He builds large-scale search applications and solutions. Muthu is interested in the topics of networking and security, and is based out of Austin, Texas.
Sohaib Katariwala is a Senior Specialist Solutions Architect at AWS focused on Amazon OpenSearch Service based out of Chicago, IL. His interests are in all things data and analytics. More specifically he loves to help customers use AI in their data strategy to solve modern day challenges.
There is an inherent limit to the privacy of the public
cloud. While Linux can isolate virtual machines (VMs) from each other,
nothing in the system’s memory is ultimately out of reach for the host cloud
provider. To accommodate the most privacy-conscious clients, confidential
computing protects the memory of guests, even from
hypervisors. But the Linux cloud stack needs to be rethought in order to host
confidential VMs, juggling two goals that are often at odds: performance
and security.
Zapier is a leading no-code automation provider whose customers use their solution to automate workflows and move data across over 8,000 applications such as Slack, Salesforce, Asana, and Dropbox. Zapier runs these automations through integrations called Zaps, which are implemented using a serverless architecture running on Amazon Web Services (AWS). Each Zap is powered by an AWS Lambda function.
In this post, you’ll learn how Zapier has built their serverless architecture focusing on three key aspects: using Lambda functions to build isolated Zaps, operating over a hundred thousand Lambda functions through Zapier’s control plane infrastructure, and enhancing security posture while reducing maintenance efforts by introducing automated function upgrades and cleanup workflows into their platform architecture.
Architecting a secure and isolated runtime environment
Zaps created by Zapier’s users implement tenant-specific business logic, hence they require cross-tenant compute isolation. Code implementing one Zap can’t share an execution environment with code implementing another Zap. Moreover, the same Zap type used by two different tenants can’t share execution environments as well.
To achieve the required level of isolation, Zapier’s engineering team adopted AWS Lambda, a serverless compute service that runs code in response to events and automatically manages cloud compute resources. Minimal operational overhead, built-in high availability, automated scaling, high level of isolation, and pay-per-use model made Lambda a great fit for this use case. Currently, Zapier’s architecture is running over a hundred thousand Lambda functions to support their customer’s integration workflows.
Because they’re powered by the open source Firecracker microVMs, each function is completely isolated from the others. Moreover, each execution environment belonging to the same function (sometimes referred to as function instances) is also isolated from other execution environments. The following architecture topology diagram uses red lines to represent isolation boundaries. Each execution environment of every function is isolated from its peers and is getting its own virtual resources such as disk, memory, and CPU. For more details, read Security in AWS Lambda.
Zapier’s control plane is architected using Amazon Elastic Kubernetes Service (Amazon EKS). A designated database is used to maintain the up-to-date function inventory. Whenever a user creates a new Zap, the control plane creates a corresponding Lambda function and stores a reference in the inventory database. When a Zap is triggered, the control plane retrieves information about a relevant Lambda function and invokes it to facilitate the integration workflow, as illustrated in the following diagram.
Understanding the runtime deprecation process
When building architectures using the traditional non-serverless compute, cloud engineers are the ones responsible for keeping operating systems and software on their compute instances up to date and applying security and maintenance patches. With serverless architectures and Lambda functions, security patches and minor runtime upgrades are handled by AWS automatically, which means customers can focus on delivering business value instead of the undifferentiated heavy lifting of infrastructure management.
As Zapier’s user base and architectural complexity – and consequently the number of Zaps – were growing, keeping all functions on the most up-to-date major runtime versions became a laborious task. Top contributing factors were:
High number of functions. At its peak, the Zapier platform was running Zaps using hundreds of thousands of unique Lambda functions. Approximately 35% of these functions were using a runtime that was scheduled for deprecation in the next 12 months.
Zapier architected their data plane environment to be ephemeral – the control plane creates and deletes Lambda functions on demand and manages their lifecycle dynamically. Identifying a specific owner for each affected function wasn’t always straightforward.
Security is paramount at Zapier and upgrading affected functions runtime prior to the deprecation date was an absolute must. At no point could Zapier functions use runtimes after their deprecation date. This was a task which required extra resources.
The upgrade process shouldn’t have had any impact on the end customer experience. At no point should customer experience be affected.
With a short runway, high-volume workload, and the strict requirements of not impacting customer experience, Zapier’s Platform Engineering team took on this challenge of maintaining high security posture in their platform architecture.
Applying the solution
The solution had three work streams:
Reducing the risk by analyzing the architecture and identifying and cleaning up unused functions.
Prioritizing upgrades by identifying the most critical and impactful functions.
Empowering engineering teams with automated tools and knowledge to streamline the upgrade process in future.
Identify and clean up unused functions
The first step in streamlining the upgrade process was identifying and removing unused functions. This reduced the total number of functions in Zapier’s architecture that required upgrades, eliminating unnecessary work for the team.
This meant the team could build a detailed inventory of functions that were running on soon-to-be deprecated runtimes. Using Amazon CloudWatch, Zapier’s platform team started to monitor metrics such as number of invocations. They identified which functions were active, which functions weren’t used for an extended period, and which functions didn’t have an active owner and could be removed.
One of the primary mechanisms for ownership validation within the organization was using resource tags. Functions that were active, but didn’t have clear ownership, were flagged for additional review before removal. Functions that were confirmed as unused or didn’t have an active owner were marked for deletion. Removing such functions allowed Zapier to significantly simplify their architecture and reduce the number of functions that had to be upgraded.
Prioritizing upgrades
With a smaller volume of functions to upgrade, Zapier’s platform team prioritized function upgrades based on usage patterns, criticality, and potential customer impact. Three primary prioritization categories were:
Customer-facing functions – Any functions directly involved in executing user Zaps were marked as high priority. These had to be upgraded first to avoid service disruptions.
Backend infrastructure functions – Internal functions that supported system operations were evaluated based on their importance to platform stability.
High-volume functions – Functions with the highest execution frequency were prioritized because upgrading them would have the greatest impact on reducing operational risk.
Using these factors, Zapier’s platform team has created an upgrade roadmap, ensuring that critical assets were addressed first while minimizing potential disruptions.
Empowering engineering teams with automated tools and knowledge
To ensure a smooth and efficient upgrade process across their serverless architecture, Zapier’s team empowered engineering teams with clear guidelines and automated solutions. The platform incorporated two main approaches: Terraform-managed functions and a custom-built Lambda runtime canary tool. Implementing and adopting these tools and practices resulted in reducing the number of functions using soon-to-be deprecated runtimes by 95%.
For functions managed through infrastructure-as-code (IaC), Zapier’s team developed standardized Terraform modules that specified supported runtime versions. Development teams implemented these modules in their configurations:
After applying the new module version, teams validated changes by testing the new runtime in staging environments and monitoring Terraform plan outputs to ensure proper runtime version updates.
To efficiently manage most Lambda functions in their architecture, Zapier developed the Lambda runtime canary tool suite. Using this solution, they automated the runtime upgrade process for thousands of active Lambda functions with minimal manual intervention. The tool suite implements several key features:
Architected for gradual traffic shifting with the Lambda built-in routing mechanism through function version and aliasing. The tool can gradually shift traffic distribution from an old to a new function version. During this gradual traffic shift, the system monitors CloudWatch metrics for errors and automatically rolls back if error rates exceed acceptable thresholds.
Optimistic upgrade strategy implements direct upgrades for infrequently used functions using a flag value stored in a cache to detect potential issues during the first post-upgrade invocation. If this invocation fails, the control plane retries it using the previous function version. If the retried invocation succeeds, Zapier’s control plane initiates a rollback, assuming the error is most likely due to the runtime upgrade. After rollback, it will log the error and alert relevant stakeholders.
Integration with existing infrastructure uses an administrative interface and task queue for automated traffic shifting. A database ledger maintains tracking of function states and rollback information.
Operational controls provide manual rollback capabilities and implement centralized control switches for process management. After a function was upgraded to a new runtime and no rollback activity was detected within a set time period, an automated pruning task cleans up older versions.
Zapier’s Lambda canary tool, through its integration of gradual traffic shifting, real-time CloudWatch monitoring, and automated rollback mechanisms, established a sustainable framework for managing runtime upgrades across their serverless architecture. This approach not only automated the upgrade process and minimized operational risks but also created a scalable solution that provides continuous runtime upgrades, preventing the use of deprecated runtimes at any point. By allowing continuous function runtime updates with minimal disruption to end user experience, Zapier maintains security and stability while requiring minimal manual intervention. This framework efficiently manages their growing serverless infrastructure, providing both security and operational efficiency for future runtime updates.
Conclusion
In this post, you’ve learned how Zapier architected their software-as-a-service (SaaS) platform to provide secure, isolated execution environments using AWS Lambda and Amazon EKS, enabling their customers to create hundreds of thousands of Zaps. You’ve learned how Zapier’s team implemented the function runtime upgrade process at scale and reduced the number of functions running on soon-to-be deprecated runtimes by 95%. You’ve seen best practices that were established and techniques that helped Zapier to keep high security posture without impacting customer experience.
Use the following links to learn more about Lambda runtimes and upgrading your functions to the latest runtime versions:
This blog was co-authored by Brandon Raabe, Sr. Site Reliability Engineer at HashiCorp.
In cloud-based systems, minutes of downtime can translate to significant business impact and eroded customer trust. HashiCorp, a leader in multicloud infrastructure automation software, faced this critical challenge as their HashiCorp Cloud Platform (HCP) scaled to serve enterprise customers with stringent availability requirements. When Regional outages threatened service continuity, the complex dance of failing over DNS entries, workloads, and databases across AWS Regions had become an error-prone process requiring intense coordination. This post chronicles how HashiCorp’s Site Reliability Engineering (SRE) team transformed their disaster recovery capabilities by implementing Amazon Application Recovery Controller (ARC), creating a solution that not only dramatically simplified cross-Region failovers but also provided a standardized way to signal Regional context to their distributed services.
In this post, we discuss HashiCorp’s journey from manual, stress-inducing failover procedures to a streamlined, confident approach that fundamentally changed how they deliver on their enterprise-grade resilience promises.
Challenges with disaster recovery in a multicloud infrastructure
HashiCorp’s SRE team recognized that as their cloud platform scaled to serve mission-critical enterprise workloads, their disaster recovery approach needed an upgrade. The existing manual processes required precise coordination across multiple systems during already stressful outage scenarios, which could lead to potential complications when speed and accuracy matter most. Regional outages posed particular challenges: if the control planes for critical services became unavailable, the very tools needed to execute recovery might be inaccessible.
ARC emerged as the ideal solution with its unique architecture: a highly available data plane accessible through endpoints in five distinct Regions, so the recovery mechanism remains operational even during significant Regional disruptions. By using the AWS SDK to interface with ARC, HashiCorp gained several critical advantages. They could apply infrastructure as code (IaC) practices to disaster recovery workflows, automate testing of failover procedures, and integrate resilience seamlessly with their existing operational tooling. This solution transformed their disaster recovery from a specialized manual procedure into a codified, repeatable process embedded within their platform operations.
Requirements and architectural considerations
After evaluating multiple disaster recovery approaches, HashiCorp established three core requirements for their solution. First, while maintaining human judgment for initiating failovers, the execution needed to proceed without additional operator interventions after it was triggered. This human-in-the-loop design preserved deliberate decision-making while reducing error-prone manual steps during implementation.
Second, the architecture needed exceptional resilience against the very failures it was designed to mitigate. Traditional DNS failover solutions presented a critical vulnerability: dependency on single-Region control planes that might be unavailable during an outage. ARC solved this problem through its distributed architecture, connecting Amazon Route 53 to a resilient control mechanism, enabled by Route 53 health checks, accessible through multiple Regional endpoints. This means the failover system itself remained available even if the primary Region went offline.
Third, the solution needed to meet or exceed HashiCorp’s existing Recovery Point Objective (RPO) and Recovery Time Objective (RTO) metrics—the maximum acceptable data loss and downtime thresholds. Using ARC, the SRE team planned to not just reach these targets but make substantial improvements, reducing potential customer impact during Regional events and strengthening HashiCorp’s enterprise-grade resilience.
Solution overview
To transform their disaster recovery posture, HashiCorp’s SRE team designed an architecture centered around ARC and complemented by a purpose-built orchestration service. This architecture seamlessly bridges the human decision to initiate failover with the complex technical operations required to shift traffic between Regions with minimal disruption.
At the heart of the solution is a custom failover service that serves as the orchestration layer for Regional transitions. This service maintains configuration details for the ARC cluster and provides a single, controlled interface for initiating Regional switchovers. When activated, the service establishes a secure connection to the ARC API endpoints and executes a two-step workflow: first disabling routing controls for the primary Region, then enabling those for the secondary Region. This sequential approach provides a clean traffic transition without split-brain scenarios or dropped connections.
The DNS architecture underwent a strategic evolution to support this new capability. HashiCorp reconfigured their critical ingress endpoints as Route 53 failover record pairs, with each pair consisting of a primary and secondary record. Each record is linked to a health check that monitors the state of an ARC routing control—effectively connecting AWS’s global DNS service to the ARC routing control. The primary records resolve to endpoints in the primary Region, and secondary records point to corresponding infrastructure in the standby Region. When routing controls change state, the associated health checks automatically trigger Route 53 to adjust DNS resolution patterns, redirecting traffic to the appropriate Regional infrastructure.
HashiCorp maintains their secondary Region in a warm standby configuration, with essential services running but not actively serving client traffic until a failover event occurs. To enable seamless awareness of Region status across their distributed system, the team implemented a signaling mechanism using specially crafted TXT DNS records. These records are tied to the same ARC routing controls as the primary service endpoints, effectively creating a discoverable, global state indicator. Services can query these TXT records to dynamically determine the currently active Region and adjust their internal routing, replication, and operational behaviors accordingly — alleviating the need for a separate configuration distribution system and making sure all components have a consistent view of the current Regional state.
The following diagram illustrates the disaster recovery workflow.
This architecture combines human oversight for initiating critical Regional transitions with fully automated execution after the decision is made. The use of ARC’s globally distributed control plane removes single-Region dependencies that might otherwise compromise the failover mechanism itself during a Regional outage event.
Operational decision framework for Regional failover
HashiCorp’s Regional failover process balances automated monitoring with deliberate human decision-making. Their comprehensive observability platform continuously monitors Regional health, automatically alerting the incident response team when anomalies are detected. When alerts trigger, the incident management protocol activates, with an incident commander quickly assembling experts to assess the situation.
The team follows a structured evaluation framework to determine if failover is warranted: confirming the issue is Region-specific, verifying that redundant intra-Region components can’t mitigate the problem, and assessing whether the projected Regional recovery time exceeds acceptable customer impact thresholds. This approach prevents unnecessary Regional transitions while providing rapid action when genuinely needed.
After the decision to failover is made, an authorized operator initiates the process through a single API call to their orchestration service, which then interfaces with ARC to execute the complex sequence of routing control changes. This design preserves human judgment for the critical decision while using automation for precise execution, so HashiCorp can respond confidently and consistently during high-pressure Regional outage scenarios.
Disaster recovery testing
HashiCorp maintains operational readiness through a disciplined monthly disaster recovery testing program in their integration environment. One week before each scheduled test, the team notifies all stakeholders to confirm organization-wide awareness and participation. On test day, they follow formal incident protocols, creating dedicated communication channels for transparent observation and collaboration.
The test execution mirrors their production failover process: an operator initiates the recovery sequence through their API, activating the ARC routing controls to shift traffic to the secondary Region. What sets HashiCorp’s approach apart is their comprehensive validation methodology. The team verifies critical services in the secondary Region and then fails back to the primary Region with subsequent validation. This bidirectional testing confirms both failover and failback procedures work reliably.
Each exercise concludes with a structured retrospective where the team documents observations and identifies improvement opportunities. By treating these tests as learning experiences rather than compliance activities, HashiCorp has established a continuous improvement cycle for their disaster recovery capabilities. The insights from these regular drills have led to numerous refinements in their ARC implementation and operational procedures, so their team can respond confidently during actual outages with practiced, predictable procedures.
Conclusion
The collaboration between HashiCorp and AWS through ARC has revolutionized HashiCorp’s disaster recovery capabilities. Regional transitions that once required careful DNS record manipulation by specialized operators now execute through a single API call, with traffic shifting within seconds and full propagation completing in approximately 2 minutes. This dramatic simplification, achieved by integrating the resilient ARC architecture with HashiCorp’s custom orchestration service, has not only improved recovery metrics but has also strengthened their enterprise-grade resilience promises.
ARC has solved a fundamental distributed systems challenge by providing a reliable mechanism for services to determine the active Region. By linking ARC routing controls to specialized TXT records, HashiCorp created a consistent global indicator that allows services to automatically adjust their behavior without additional coordination systems—simplifying their architecture and reducing dependencies.
Most significantly, this implementation has democratized disaster recovery within HashiCorp, transforming it from a specialized capability to a standardized procedure executable by their regular on-call rotation. The solution’s highly available endpoints across multiple Regions makes sure the recovery mechanism itself remains operational even during severe outages—addressing a critical vulnerability in their previous approach.
For HashiCorp’s enterprise customers, these improvements translate directly to business value: reduced recovery times during Regional events, increased operational confidence, and assurance that their critical infrastructure management tools will remain available even during major cloud disruptions. As HashiCorp continues to refine their approach through rigorous testing and continuous improvement, their ARC implementation demonstrates how thoughtfully architected disaster recovery can evolve from merely an insurance policy into a strategic competitive advantage.
Security updates have been issued by AlmaLinux (git, kernel, nginx:1.24, and sudo), Fedora (dpkg, java-21-openjdk, java-25-openjdk, java-latest-openjdk, and valkey), Oracle (apache-commons-vfs, sudo, tigervnc, and xorg-x11-server), Red Hat (kernel, krb5, and openssh), SUSE (gnutls, ImageMagick, iputils, kernel-livepatch-MICRO-6-0-RT_Update_10, kubernetes1.18, libarchive, ovmf, python, and salt), and Ubuntu (iputils, linux-aws-6.14, linux-raspi, openjdk-21, and openjdk-24).
Бившият премиер и реформатор Иван Костов каза нещо съществено по bTV, което политиците се направиха, че не са чули. Той смята, че може да се намери такъв кандидат за президент, който да бъде подкрепен и от ГЕРБ, и от ПП–ДБ, ако това ще предотврати „румънски сценарий“ в България.
Има кандидати, които според мен могат да бъдат подкрепени и от ГЕРБ, и от ПП, и от ДБ. Друг е въпросът дали може да се постигне съгласие. Но има такива хора. И не мисля, че това трябва да се изключва като възможност. Коалицията ПП–ДБ трябва да изчислява и този сценарий. Какво ще стане, ако ГЕРБ им предложи кандидатура? Какво ще направят? Защото рискът да се повтори румънският сценарий – всички да стоят на тръни и на нокти, да очакват изборите и да се провалят – е много по-голям в България, отколкото беше в Румъния. Защото България е по-силно поразена от хибридната агресия на Руската федерация.
Казаното от Костов поставя геополитическата ориентация и интереси над вътрешнопартийните вражди. Бившият премиер не призовава към „голяма коалиция“, а към минимален консенсус: ако в предстоящите президентски избори се яви кандидат, който носи риск за демократичния прозападен курс на страната, ГЕРБ и ПП–ДБ трябва да загърбят омразата и да подкрепят приемлива фигура. Не партийна, а именно общоприемлива, за да запазят посоката на България като евроатлантическа демокрация, в която президентът не работи срещу парламента, правителството и националните интереси.
Разбира се, тук има сериозни препятствия – цялата платформа на ПП–ДБ се гради върху антикорупционния наратив и превзетите институции, а Борисов и ГЕРБ са символ на този модел. А и вече зависимостите са ясно дефинирани по оста Борисов–Пеевски, от която лидерът на партията с най-голямо електорално влияние не може да се откъсне. Поне не и безболезнено. Корупцията продължава да е фокус за обществото. Изследване на „Алфа Рисърч“ показва, че 49% от запитаните биха гласували за „нова партия, която е истински борец срещу корупцията“.
ПП–ДБ вече бяха в управленска конфигурация с Борисов и олигарха и лидер на ДПС – Ново начало Пеевски в името на конституционната реформа. Това им коства значителен брой избиратели. Ако решат да издигнат кандидат за президент, без да търсят подкрепа от ГЕРБ, той трябва да е достатъчно ярка фигура, която да събира широко обществено одобрение извън теснопартийния спектър и да стигне до балотаж.
В кампанията за първия си президентски мандат (2017–2022) Румен Радев беше критичен към управлението на ГЕРБ, за да покаже, че прокламираната стабилност е фалшива. Беше твърд, без да е агресивен, умело се позиционира между национално отговорен и антисистемен кандидат. И още преди да се е заклел, даде първите убедителни доказателства за лоялност към Москва. В интервю за френския канал France 24, озаглавено Radev: It Is Positive That Trump Is No Slave to Political Prejudices, се противопостави на санкциите срещу Русия, наложени заради анексирането на Крим.
… Крим е украински; той е на картата на Украйна. Де факто обаче е руски; не можем да пренебрегнем реалността, че над Крим има руски флаг. Бъдещето на Крим трябва да бъде решено от хората. Те трябва да бъдат запитани дали искат да се върнат към Украйна.
От 2017 г. насам геополитическото напрежение се е повишило значително – войната на Русия в Украйна позиционира държавите в ясни лагери, а тече и усилено превъоръжаване „до зъби“, както го определиха някои анализатори. За каквито и да е избори догодина в България – били те президентски и/или предсрочни парламентарни, партиите ще трябва да отчитат и къде и как да се позиционират предвид въвеждането на еврото и ефектите от новата валута. Една по-висока инфлация ще бъде използвана като аргумент от националпопулистите и евроскептиците, не са изключени и организирани протести. Предвид съпротивата и недоверието в част от българското общество, която съвсем не е незначителна, това ще повлияе на резултатите от вота.
Бъдещият кандидат за президент на „демократичната общност“ ще води кампания в трудна среда.
С ГЕРБ или без ГЕРБ?
Хамлетовски, изглежда, е въпросът могат ли ПП–ДБ да включат ГЕРБ в „демократичната общност“, поставила си за цел да излъчи кандидат, за когото липсва единение дори между „Продължаваме промяната“ и „Демократична България“. Макар никой досега да не се е наел да дефинира ясно границите на тази общност – кой има място в нея и кой е неприемлив, – включването на ГЕРБ би означавало преместване на червени линии. Дори въпросът да стане актуален при балотаж, отговорът трябва да е обмислен отрано.
А този отговор ще е изпитание не само за стратегическата зрялост на ПП–ДБ, но и за способността им да балансират между моралната си легитимност и геополитическата необходимост. Още повече че вече го направиха при сглобката, като „прегрешиха“ с ГЕРБ и Пеевски.
Преди месец Бойко Борисов потвърди, че би преговарял с двете политически сили за общ кандидат:
Разбира се, че бих участвал.
Има поне двама политици в държавата, които ще направят всичко по силите си подобно обединение зад обща кандидатура да не се осъществи – Делян Пеевски и Румен Радев.
Първият иска ГЕРБ за себе си, вторият иска да остане главен фактор в българската политика и след края на мандата си – и защо да не го направи чрез структуриране на нова коалиционна ос около антисистемен вот (както спечели втория си мандат)? Президентът разполага с 14 месеца до края на втория си и последен мандат и кръговете зад него нямат интерес от държавен глава, който може да гарантира прозападна последователност – защото би бил коректив на наследството на Радев, а не негов продължител.
Пеевски отдавна не се крие и действа през натиск, зависимости и интриги. Първият удар през Комисията за противодействие на корупцията и през прокуратурата дойде седмица след старта на инициативата на ПП–ДБ за общ кандидат-президент и удари върху Столичната община и фигури, свързани с „Промяната“.
Радев използва антисистемен език и говори за „надпартийност“, за да си осигури подкрепа отвъд традиционния спектър, който обитава с образа си на консервативен евроскептик с проруски симпатии. Последните – вече умело прикрити, да не бият на очи на първи план.
Кого ще донесе вълната?
Призивът на Иван Костов следва да се разбира като предупреждение за стратегически риск пред България, който включването в еврозоната от 1 януари 2026 г. намалява, но не елиминира. „Румънски сценарий“ е напълно реална опция, при която от вълната от антиелитарни, конспиративни и проруски настроения може да изплува кандидат, който с подходяща емоционална реторика и подкрепа от радикализираните периферии да обедини протестния вот и да спечели.
Страховете и напрежението, нагнетявани от правителството с мерките му срещу повишаването на инфлацията, само помагат за изпълнението на такъв сценарий. Един безспорно исторически момент за България, какъвто е приемането на еврото, е зареден в аванс с негативизъм (срив на стандарта на живот и обедняване), който се превръща в идеална платформа за фалшив спасител.
В български условия това би бил кандидат с хибриден профил между Радостин Василев, Костадин Костадинов и Ивелин Михайлов – популист, способен да канализира общественото недоволство в посока, опасна за евроатлантическата ориентация на страната. При това румънският сценарий в България изключва здрави институции, които да се намесят, както направи Конституционният съд в Румъния. Така че всъщност сценарият ще си е 100% български.
„Бухалките“ върху кмета на Варна Благомир Коцев съживиха обществената енергия, също и ПП–ДБ. Ето че коалицията води разговори за общ вот на недоверие с МЕЧ на Радостин Василев. МЕЧ предлага да е за провал на правителството в сферата на обществения ред и сигурност, а от ПП–ДБ – да се разшири със „завладените институции“. Аргументът, представен от съпредседателя на ДБ Божидар Божанов, е, че това правителство е смокинов лист на процесите по завладяване на държавата от Пеевски:
Вотът на недоверие трябва да е изключително сериозен и да удари в сърцето на управляващата коалиция на Пеевски, затова предложихме темата да бъде разширена до „завладените институции“.
Изглежда, че темата за кандидат-президента на демократичната общност е оставена за наесен или когато ѝ дойде времето.
И въпросът остава: някой чу ли Иван Костов?
Защото, ако „демократичната общност“ не може дори да се дефинира, камо ли да действа като политически субект със смелост и отговорност, тогава няма да е страна в предстоящата битка, а наблюдател. Кандидатът трябва да бъде издигнат навреме, за да поведе. Иначе ще е принуден да се бие на терена на популизма.
Когато бях на 15, бях убедена, че секс правят само младите и красивите. Не искате да ви разказвам какво си представях под „млади и красиви“, или може би по-точно е да кажа, че аз не искам да ви разказвам. Но за ориентир ще дам, че майка ми тогава беше на 35 и естествено, това беше възрастта, на която в очите ми хората вече бяха „възрастни“.
От настоящата си гледна точка на „възрастна“ виждам, че някои жени живеят с това убеждение цял живот – че женствеността, „женскостта“, сексапилът са някакви категории със строги и доста ограничаващи параметри и за да бъдеш жена, трябва да се вместваш в изискванията.
За всеобщ късмет това просто не е вярно.
Be a lady, they said
„Аз не правя секс с мъжа си и с мъже изобщо“, „Правих най-много и най-хубав секс след 40 г.“, „Сексът за мен е мозъчна дейност“, „Истински спад в желанието ми имаше около раждането на децата ми“, „Правя секс само на тъмно“, „Научих много за себе си точно заради секса“…
Попитах група жени какво е да сме „секси“ в собствените си очи, как живеем с телата си, променящи се и несъвършени, свикнали ли сме да имаме сексуални потребности и предпочитания, къде се намира сексът в жизненото пространство на жените и нужна ли им е оценката на партньор(ка), за да преживеят себе си?
Oтговорите в голяма степен потвърдиха представите ми. Сексуалността на жените не е статична и се променя с възрастта, опита и личните обстоятелства, което прави темата многопластова и индивидуална. Да си „женствена“ или „секси“ също не е едно нещо – не е свързано с размерите на тялото, вида му или дори с възрастта. Няма „женска енергия“, или поне няма граници, които да затварят значението на това понятие.
И все пак в дискусията ясно изплува изводът, че спокойствието да си такава, каквато си – с големи или малки гърди, руса или брюнетка, слаба или пухкава, стеснителна или с голям сексуален апетит – е ключов елемент в способността ни да водим здрав и удовлетворяващ сексуален живот.
Цял живот.
Хормони и други обстоятелства
Сексуалността на жените е резултат от сложна взаимовръзка между физиологични процеси, които се променят през различните етапи на живота. Разбирането на тези фактори е ключово за осигуряване на здравословен и удовлетворяващ сексуален живот, но липсата на системно здравно образование и културните табута често водят до точно обратното. „Твърде консервативното ми възпитание създаде у мен усещане, че е нещо нередно да искам секс или пък да го правя“, разказва 47-годишна жена. Друга споделя: „Интересът към всичко, свързано със секс и опознаване на собственото тяло и какво му харесва, започна много рано. Всичко това – забулено в табу, притеснение, срам и чувство за нередност и нужда от криене, защото у нас не се говореше за такива работи.“
Всъщност сексът е нещо напълно естествено и при добро познаване на тялото и грижа за себе си той може да остане част от живота ни до много по-късна възраст, отколкото сме склонни да си представяме в младежките си години.
Хормоните играят ключова роля в модулирането на сексуалното желание и възбудата. Най-влиятелните хормони в този контекст са естроген, тестостерон и прогестерон. Естрогенът е основно отговорен за развитието на женските полови характеристики и влияе върху гениталните тъкани, увеличавайки притока на кръв към гениталната област, което може да подобри чувствителността и овлажняването. Нивата на естроген са различни през менструалния цикъл, достигайки пик по време на овулация, което често води до повишено сексуално желание. Тестостеронът, макар че често се свързва с мъжкото сексуално желание, играе важна роля и в женската сексуална възбуда. Въпреки че жените имат по-ниски нива на тестостерон, той влияе върху либидото, възбудата и оргазмения капацитет. По-високите нива на тестостерон са свързани с повишено сексуално желание и при жените. Прогестеронът, от друга страна, би могъл да потиска сексуалното желание и възбудата при много жени, което води до намаляване на либидото.
Сексуалното желание при жените се регулира и от определени невротрансмитери, които влияят на центровете на мозъка, отговарящи за сексуалността. Повишената допаминова активност е нужна среда за високо сексуално желание и мотивация. В същото време този невротрансмитер се активира от сексуални мисли и стимули, засилвайки чувството на вълнение и удоволствие.
Друг важен фактор е серотонинът: въпреки че се свързва с регулирането на настроението, неговият ефект върху сексуалното желание може да бъде сложен. Високите нива на серотонин могат да потиснат сексуалното желание и възбуда, докато по-ниските нива са свързани с повишен сексуален интерес. Третият важен невротрансмитер е окситоцинът. Той е причината да изпитваме удоволствие при докосване, а при достигане на оргазъм тялото ни буквално се къпе в окситоцин.
Отвъд биохимията в женското тяло протичат и други физиологични процеси, които ни помагат или ни пречат да правим секс.
Сексуална дисфункция
Световната здравна организация определя сексуалното здраве като състояние на физическо, емоционално, психическо и социално благополучие във връзка със сексуалността, а не просто липса на болест, дисфункция или физическа аномалия.
Сексуалната дисфункция при жените обхваща редица състояния, например ниско сексуално желание, затруднено постигане на оргазъм, проблеми с възбудата и болка при проникване. Разпространението на тези проблеми варира в зависимост от фактори като възраст, общо здравословно състояние и психологическа нагласа. Макар в повечето случаи да става дума за решими проблеми, все още има жени, за които те са неотменна част от живота:
Либидото ми е било ниско през по-голямата част от живота ми, вероятно причините са физиологични, тъй като е имало промени при сериозни хормонални сътресения (например през бременността правех много секс). След травматично раждане постепенно приключих напълно.
Сериозен фактор за смущения в сексуалния живот играе стресът, който може да е свързан с разнообразна палитра от житейски обстоятелства. Понякога съпровожда дори планирани и щастливи събития, каквото е раждането на дете:
Разпадът в сексуалния ни живот продължи година и четири месеца след раждането на първото бебе, истинското помиряване и прощаване на взаимни грешки отне доста повече време, но се случи. Оттогава сексът става все по-добър,
споделя майка на две вече пораснали деца.
Хроничният стрес и високите нива на хормона на стреса кортизол потискат сексуалното желание. Когато тялото е под стрес, се активира реакцията „бий се или бягай“ и това може да потисне сексуалната възбуда. Повишените нива на кортизол могат да намалят производството на полови хормони, като естроген и тестостерон, което води до по-ниско либидо.
Но не само. „В тази жега не ми е тема. Не искам никой да ме докосва“, категорична е друга събеседничка.
Голяма част от тези проблеми възникват с напредването на възрастта, в периодите на перименопаузата и менопаузата, но невинаги предизвикват усещане за дисфункционалност. Една от жените в цитираната възрастова група разказва:
С времето моето либидо намалява, но осъзнах, че изцяло зависи от месечния цикъл и нищо не може да предизвика у мен интерес в дните преди цикъл. Това е окей, концентрирам възможно най-много секс в първата му половина.
Изследвания сочат, че около 43–44% от жените преживяват някаква форма на сексуална дисфункция в живота си. Макар това да е много висок дял, позитивният му прочит би звучал така:
Повечето жени могат да имат удовлетворителен секс през целия си активен живот
Както науката, така и личният опит на жените показва, че възрастта носи своите особености, без да играе ключова роля в женското сексуално здраве.
В периода на съзряването сексуалното желание става по-силно изразено, естрогенът играе доминираща роля, насърчавайки сексуалната възбуда и отзивчивост, докато нивата на тестостерон, макар и по-ниски, отколкото при момчетата, започват да допринасят за либидото. През този период момичетата започват да изпитват сексуално привличане и желание да изследват своята сексуалност.
Развитието на мозъка също играе роля, като системата за възнаграждение на мозъка става по-чувствителна към сексуални стимули. Тази повишена чувствителност може да доведе до по-силни сексуални импулси и чувства на романтично привличане. Психологическите фактори, включително емоционалната връзка, започват да оформят начина, по който се преживяват тези импулси.
„Започнах на 15, но до 19 не знаех какво е оргазъм“, но също и „Официален старт на 16 години – нямах търпение да ми дойде менструацията, за да започна. Първите години, докъм 20, имах много силно либидо и готовност да правя секс къде ли не, например в кола, тоалетна на дискотека, зад блока и пр.“, разказват събеседничките ми. „Бях безгрижна и имах по-силно либидо като по-млада, което водеше до какви ли не сексуални приключения“, споделя една от тях, докато друга отсича: „Моногамност – от тийн.“
Разбира се, с годините и натрупания опит разбирането на всяка жена за здрав (в смисъл – благосъстоятелен, удовлетворяващ и без патологии) секс се променя. В ранната зряла възраст, обикновено между 20 и 30 години, жените често изпитват най-високите си нива на сексуално желание. Хормоналните нива на естроген и тестостерон достигат пик, а репродуктивната система е напълно зряла. Жените често са в разцвета на плодовитостта си и естественото влечение на тялото към сексуална активност е силно. Желанието за интимност и връзка, водено както от хормонални фактори, така и от емоции, достига своя връх през този период. Сексуалното удовлетворение обикновено е високо, като много жени изследват и научават за своите сексуални предпочитания и желания във връзките си.
Този теоретичен модел напълно се потвърждава от историите на жените, които споделят:
Много интересно за мен беше, че след първото вагинално раждане (30 г.) изведнъж бях способна да си заявя нуждите за предварителна стимулация и съответно нивото на задоволеност за мен се вдигна чувствително. Също така и да кажа, че не съм достигнала до оргазъм и имам нужда от още.
Интересно наблюдение е, че мъжът ми съвсем се е променил – когато първоначално ме е привличал, е имал тяло на тийнейджър, гладки бузи и коса. Абсолютно нищо не е същото, все едно различна обвивка, сега е по-секси, вероятно защото го познавам повече – обичам да го гледам гол и как върши неща, също и как говори по работа…
С годините бях по-готова да експериментирам, защото се появи партньор, комуто да се доверя, и защото по-малко ми пукаше за това как изглеждам (което беше основният ми бъг във всички години, в които всъщност съм изглеждала добре).
Последното признание е на конкретна 50-годишна жена, но обобщава добре множество истории на жени в зряла възраст. „На 45 вече се харесвам и приемам. Колкото повече се харесвам, толкова повече либидото ми изкачва върхове“, споделя друга, а в отговор трета допълва: „Имаше един момент, аз бях на 40, още приспивах дете всяка вечер, беше ковид, не ми се правеше секс и заявих на мъжа ми, че ако си мисли, че цял живот ще правим секс, да си помисли пак. Сега ми се струва абсурдно това. Мисля, че още много секс ни чака.“
И това вероятно е така, ако съдим по историята на друга жена в нашата възрастова група, която дава надежда и за бъдещето:
За мое учудване и радост майка ми на 73 години все още прави секс с приятеля си на 78 години, което ме кара да смятам, че дори и да не става все по-хубав и по-хубав, то очевидно човек може да съхрани удоволствието от секса дори и след определена възраст.
След менопаузата, обикновено около 50-годишна възраст, яйчниците спират да произвеждат яйцеклетки и нивата на естроген спадат значително. Това намаляване на естрогена води до промени в сексуалната функция, включително вагинална сухота и намалено овлажняване, което може да направи секса неприятен за някои жени. Въпреки тези предизвикателства много жени в постменопауза съобщават, че изпитват възраждане на сексуалното удовлетворение поради по-малко страх от забременяване, по-голям фокус върху емоционалната интимност и подобрена комуникация с партньорите си:
Винаги ми е било приятно да правя секс, но най-хубавия секс имах след 40, когато като че ли се отпуснах повече и да експериментирам. Имах едни 7–8 години с ужасно много секс в този период.
Какво искат жените
В същото време според изследване мотивацията на жените за секс не се променя драстично с възрастта. Тя е повлияна от предишните им сексуални преживявания, вида връзка, в която участват, и множество фактори, свързани с начина им на живот. Според изследването жените на възраст 31–45 години имат повече мотиви за секс, отколкото жените на възраст 18–30-годишните.
Основните причини за секс обаче не се различават в целия възрастов диапазон – жените правят секс предимно за да изпитат удоволствие, любов и усещане за свързаност.
„Анатомия на пола: Жена“ разглежда здравето на жените като неразривна част от обществото, историята и културата. В поредицата изследваме как са се променяли нагласите към женското здраве, как медицината е възприемала специфичните потребности на жените и какви процеси са повлияли на достъпа им до качествени здравни грижи. Вглеждаме се в научните открития, но и в културните митове; в официалните политики, но и в личните истории на жени, борещи се за правото си на здраве и достойнство.
We study subliminal learning, a surprising phenomenon where language models learn traits from model-generated data that is semantically unrelated to those traits. For example, a “student” model learns to prefer owls when trained on sequences of numbers generated by a “teacher” model that prefers owls. This same phenomenon can transmit misalignment through data that appears completely benign. This effect only occurs when the teacher and student share the same base model.
Interesting security implications.
I am more convinced than ever that we need serious research into AI integrity if we are ever going to have trustworthy AI.
On July 23, 2025, the White House unveiled its AI Action Plan (Plan), a significant policy document outlining the current administration’s priorities and deliverables in Artificial Intelligence. This plan emerged after the White House received over 10,000 public comments in response to a February 2025 Request for Information (RFI). Cloudflare’s comments urged the White House to foster conditions for U.S. leadership in AI and support open-source AI, among other recommendations.
There is a lot packed into the three pillar, 28-page Plan.
Pillar I: Accelerate AI Innovation. Focuses on removing regulations, enabling AI adoption and developing, and ensuring the availability of open-source and open-weight AI models.
Pillar II: Build American AI Infrastructure. Prioritizes the construction of high-security data centers, bolstering critical infrastructure cybersecurity, and promoting Secure-by-Design AI technologies.
Pillar III: Lead in International AI Diplomacy and Security. Centers on providing America’s allies and partners with access to AI, as well as strengthening AI compute export control enforcement.
Each of these pillars outlines policy recommendations for various federal agencies to advance the plan’s overarching goals. There’s much that the Plan gets right. Below we cover a few parts of the Plan that we think are particularly important.
Encouraging U.S. technology leadership
The Plan takes the position that the U.S. is in a global race to achieve AI dominance, and that it is a national priority for U.S. technology companies to be the gold standard for AI globally. Through the Plan, President Trump commits his Administration to support American workers, technology, and energy to achieve that objective.
We share the view that governments have a helpful role to play in shaping rules and regulations that will enable private-sector innovation to flourish. For Cloudflare’s network to continue to operate globally, we need the U.S. government to shape and influence the right regulatory conditions. They should balance national and economic security concerns, promote consensus industry-led international standards, and support interoperable regulatory regimes.
Far too often in recent years, we’ve observed policy developments that have unnecessarily increased restrictions on U.S. technology providers and have made it challenging to operate. Protectionist mandates, including data sovereignty requirements, customer data retention policies, various supervisory and government access requirements, do little to improve security or innovation and have unintended consequences. Protectionism increases costs for businesses, limits access to world-class technologies, and increases cybersecurity risk.
Implementing policies that guarantee access to global, distributed edge-compute networks and the freedom to choose the best technology for users’ needs will help ensure the right conditions to enable AI to flourish.
The AI ecosystem needed to spur innovation and development
The Plan endorses open-source and open-weight AI models to spur innovation and to benefit commercial and government adoption. The plan recommends ensuring access to computing resources to increase capability in the start up and academic worlds.
Cloudflare shares the view that open-source AI models play a crucial role in driving innovation. As recognized in the Plan, these models offer companies flexibility, freeing them from dependence on closed providers and enabling the use of AI with sensitive data where exporting to closed models might not be possible. That’s why Cloudflare includes access to more than fifty open-source models as part of our Workers AI model catalog.
However, access to open-source models alone is not enough to harness AI’s potential. A complete ecosystem is needed to build and deploy the AI applications and tools that will usher in the new age imagined by the Plan. Cloudflare’s global network, with our GPU-powered inference, can play an essential role. Having a distributed network like ours which allows AI inference at the edge is critical for fast, efficient AI development and for building the next generation of AI applications.
Open ecosystems are deeply embedded in Cloudflare’s DNA. Our developer platform democratizes access, providing powerful tools for anyone to build and deploy applications. We offer global network infrastructure that removes complexities and reduces barriers. This lets AI developers innovate freely, using many different AI models, without relying on gatekeepers. Our commitment to making these tools easy to use mirrors the Plan’s call to foster innovation and support U.S. AI leadership by enabling developers to use open-source AI models to build, deploy, and scale new AI applications globally.
Enhancing cybersecurity with AI
The Plan stresses the importance of cybersecurity for AI in several ways. There are two we want to highlight.
First, it endorses the use of AI technologies for the cybersecurity of critical infrastructure. The use of AI-assisted cyber-defense tools are force multipliers for network defenders, and will be absolutely necessary for all organizations — but particularly critical infrastructure — to protect against cyber threats.
Cloudflare’s network uses predictive AI and machine learning to block 247 billion cyberattacks daily. Under the theory of Defensive AI, Cloudflare uses information to constantly improve the effectiveness of our security solutions. With AI Labyrinth, we’ve even created a new tool that uses AI to trap AI. It is a new, next generation honeypot and cybersecurity defensive tool that leverages AI to confuse crawlers and bots that ignore “no crawl” directives. Instead of blocking these bots, AI Labyrinth directs bots into an endless maze of convincing, AI-generated pages.
Second, to address potential vulnerabilities in AI technologies, the Plan tasks the U.S. government with ensuring that they are secure-by-design.
To secure AI, Cloudflare has been active in shaping the cybersecurity and risk management of AI technologies. We have supported and provided feedback to the U.S. National Institute of Standards and Technology’s efforts to develop a Cybersecurity Profile for Artificial Intelligence. This is critically important and builds on our Secure-by-Design commitment.
We look forward to working with the Administration on the proposed AI information sharing and analysis center and the proposed vulnerability information exchange.
Cloudflare stands ready to accelerate AI adoption in government
The Plan envisions the federal government playing a key role in accelerating AI adoption. Cloudflare can help. As the Plan notes, integrating AI can significantly enhance public service, making government more efficient and effective. Most, if not all, federal agencies now have Chief AI Officers, indicating a clear commitment to this technological shift. The government can further its efforts by fostering information sharing between government agencies, promoting best practices, and training its workforce to maximize AI’s efficiency gains.
Cloudflare can be a key partner in this journey. Our platform provides the secure, reliable, and scalable infrastructure necessary for federal agencies to deploy AI applications with full-stack AI building blocks. Cloudflare is FedRAMP Moderate authorized, and we are committed to FedRAMP High. By leveraging Cloudflare’s global network, federal agencies can ensure their AI initiatives are resilient and accessible, driving greater public benefit.
The need to balance the export of AI with export controls
To lead on AI internationally, the Plan outlines a dual strategy, presenting two approaches in tension with each other: aggressive AI export to allies and partners, and stringent restrictions on exporting AI compute and semiconductors. On one hand, the Plan emphasizes that providing the full U.S. AI technology stack is crucial to prevent allies from turning to rivals. This aims to solidify a global AI alliance and ensure the enduring diffusion of American technology.
Conversely, the plan calls for strengthening export control enforcement and plugging loopholes to prevent export of sensitive technologies. The administration seeks to use export controls — restrictions on what goods a company can export — to deny foreign adversaries access to certain resources for both geostrategic competition and national security concerns. The challenge arises because overly stringent export controls, while aiming to deny access to adversaries, may inadvertently make it harder to export AI even to allies.
This dual approach highlights a critical tightrope walk. Cloudflare, along with many other industry players, will be watching closely to see how the administration balances these competing goals. Providing individuals across the world with access to resources that enable them to innovate and build applications close to their end users aligns with our mission to help build a better, more connected Internet. Having a globally distributed network like ours also enables U.S. AI companies to deploy their services globally. Although we appreciate the need for restricting access to sensitive compute resources, overly broad or imprecise controls could inadvertently stifle innovation and impede the open exchange of ideas crucial for AI development. The implementation of export controls must be meticulously balanced to target adversaries effectively without unwittingly hindering the very innovation and secure global digital ecosystem it seeks to protect.
A reassuring aspect of the Plan is its clear recognition of the private sector’s indispensable role. The document repeatedly emphasizes the need for collaboration with industry and consultation with leading technology companies across various recommended policy actions. For instance, it specifically calls for establishing programs within the Department of Commerce to gather proposals from industry consortia for AI export packages. Furthermore, for strengthening AI compute export control enforcement, it advises exploring new measures “in collaboration with industry.” This commitment to partnership is essential to navigate the complexities of AI development and deployment. This collaboration with industry will ensure that policies are technically feasible, globally effective, and avoid unforeseen negative impacts on the digital economy and cybersecurity.
Shaping the future of AI together
The Plan represents a critical moment for U.S. AI leadership, and Cloudflare stands ready to partner in shaping the future of this critical technology. We applaud the Plan’s focus on accelerating AI development, building robust infrastructure, and leading global diplomacy. The Internet’s global nature means that achieving these goals requires a delicate balance, particularly as the business model for the AI-powered web rapidly evolves. Cloudflare champions an approach that fosters innovation while upholding an open, secure, and interoperable Internet. By prioritizing consensus-driven standards and ensuring that regulations do not inadvertently create barriers to a globally distributed AI infrastructure, we help ensure continued U.S. technological leadership and a sustainable, beneficial AI ecosystem.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Gregory Knowles is a data and AI specialist solution architect at AWS, focusing on the UK public sector. With extensive experience in cloud-based architectures, Greg guides public sector customers in implementing modern data solutions. His expertise spans governance, analytics, and AI/ML. Greg’s passion lies in accelerating transformation and innovation to improve productivity and outcomes. He has successfully led projects that moved data systems into the cloud, adopted new data architectures, and implemented AI at scale in production.
Abhinav Tripathy is a Software Engineer and Security Guardian at AWS, where he develops Amazon Q generative SQL by combining machine learning, databases, and web systems. Abhinav is passionate about building scalable web systems from scratch that solve real customer challenges. Outside of work, he enjoys traveling, watching soccer, and playing badminton.
Erol Murtezaoglu is a Technical Product Manager at AWS, is an inquisitive and enthusiastic thinker with a drive for self-improvement and learning. He has a strong and proven technical background in software development and architecture, balanced with a drive to deliver commercially successful products. Erol highly values the process of understanding customer needs and problems, in order to deliver solutions that exceed expectations.
Ashok Padmanabhan is a Sr. IoT Data Architect with AWS Professional Services. Ashok primarily works with manufacturing and automotive customers to design and build Industry 4.0 solutions.
Venkatavaradhan (Venkat) Viswanathan is a Global Partner Solutions Architect at Amazon Web Services. Venkat is a Technology Strategy Leader in Data, AI, ML, generative AI, and Advanced Analytics. Venkat is a Global SME for Databricks and helps AWS customers design, build, secure, and optimize Databricks workloads on AWS.
Srividya Parthasarathy is a Senior Big Data Architect on the AWS Lake Formation team. She works with the product team and customers to build robust features and solutions for their analytical data platform. She enjoys building data mesh solutions and sharing them with the community.
Ramkumar Nottath is a Principal Solutions Architect at AWS focusing on Analytics services. He enjoys working with various customers to help them build scalable, reliable big data and analytics solutions. His interests extend to various technologies such as analytics, data warehousing, streaming, data governance, and machine learning. He loves spending time with his family and friends.
John Spencer is a Product Manager at Databricks, dedicated to making Unity Catalog work seamlessly with customers’ ecosystems of tools and platforms so they can easily access, govern, and use their data.
Sreeram Thoom is a Specialist Solutions Architect at Databricks helping customers design secure, scalable applications on the Data Lakehouse.
Dipankar Kushari is a specialist solutions architect at Databricks helping customer architect and build secured applications on Data Lakehouse.
