How CSIRO built scalable, cost-optimized genomic variant querying on AWS

Post Syndicated from Prof. Denis Bauer original https://aws.amazon.com/blogs/architecture/how-csiro-built-scalable-cost-optimized-genomic-variant-querying-on-aws/

This is a guest post by Denis Bauer, Yatish Jain, Anuradha Wickramarachchi, Brendan Hosking, and Nick Edwards of CSIRO, in collaboration with the ASP Prototyping and Scaling Team at AWS.

In this post, we describe how researchers at CSIRO, Australia’s national science agency, built Serverless Beacon (sBeacon), a scalable serverless solution for securely querying genomic variant data on AWS, underpinning production-scale clinical and research applications.

The Beacon protocol is the widely adopted standard for exchanging genomic and phenotypic data developed by the Global Alliance for Genomics and Health (GA4GH). It uses an API to define how data is shared, with the goal of enabling efficient and secure data discovery across international research and clinical networks.

sBeacon is a production-ready implementation of this standard, built using AWS services: Amazon Simple Storage Service (Amazon S3), AWS Lambda, Amazon DynamoDB, and Amazon Athena. By using these foundational AWS serverless services, sBeacon is able to provide the following benefits to researchers and clinicians needing to perform genomic variant querying:

  • Highly scalable for large cohorts: sBeacon can scale to support hundreds of millions of individuals (and billions of genomic locations), which makes it suitable even for mega-biobank-scale datasets.
  • Low cost to run: Because it uses a serverless, cloud-native architecture, sBeacon can operate for approximately USD 0.40 per month for a 1000 Genomes-scale dataset. The following case study breaks down ingestion, query, and storage costs in detail.
  • High performance and fast query response: Real-world queries return in seconds (about 5 seconds) because of the serverless compute and efficient architecture, for near real-time data lookups.
  • No heavy data ingestion or transformation needed: sBeacon can directly consume standard VCF files (a common format for genomic variant data), which reduces the need to load data into databases or transform it to different data structures.
  • Rapid onboarding of new data: Genomic data generation is accelerating because it underpins clinical diagnosis and treatment, and because its complexity demands ever-larger cohorts to study complex traits. As a result, both clinical services and research cohorts must continuously onboard new data, and Beacon supports real-time generation-to-use life cycles (about 18 seconds).
  • Improved privacy, data ownership, and decentralization: Because sBeacon doesn’t require central databases and supports federated networks, data stays under the control of original holders, which can help data custodians address privacy and ethical considerations in sensitive genomic and medical data sharing.
  • Lower barrier to entry for broader participation: Its affordability, simplicity, and small operational footprint can help make it more accessible for smaller or resource-limited institutions and countries, which can increase participation from underrepresented populations and improve data diversity.
  • Zero trust model: sBeacon enforces explicit authentication, least-privilege data access, ephemeral compute isolation, and strict cloud-native boundary controls that help confirm no component, user, or request is implicitly trusted.

Prerequisites

sBeacon is deployed as a container that sets up the necessary development environment, with Terraform defining the resources for the deployment. To get started, clone the terraform-aws-serverless-beacon repository on the GitHub website.

git clone https://github.com/aehrc/terraform-aws-serverless-beacon.git

Make sure that your development environment contains Docker and has the necessary permissions for you to use it without super user access. Press Ctrl+Shift+P (Cmd+Shift+P on macOS) to open the command palette in VS Code, and then choose Reopen in Container. This opens the workspace in the container environment that we have defined.

Now, run the following command to initialize the necessary libraries and Lambda layers.

bash init.sh

Next, run the following command to initialize the Terraform environment.

terraform init

Optionally, you can define a backend by following the instructions in the repository. After the preceding command runs successfully, you can run the deployment command.

terraform apply

Enter yes when prompted to proceed with the deployment. After the deployment is complete, you receive information such as the API URL and the command to sign in as the admin or guest user. To shut down the entire service, run terraform destroy. Any created datasets are lost (but not the VCFs on which they are based).

Solution walkthrough

CSIRO developed sBeacon for sharing and querying genomic and medical data. sBeacon uses AWS serverless technology for the elastic scaling of compute resources.

The architecture of sBeacon performs two broad processes:

  1. Data onboarding: the ingestion and indexing of genomic metadata into sBeacon.
  2. Data querying: the querying of the genomic metadata by end users.

Data onboarding

During the onboarding process, you define where the genomic data and the metadata (such as disease status, age, and location) is located. Note that genomic data is not copied out of its original location but rather is referenced when needed. In contrast, metadata is loaded to sBeacon’s storage mechanisms because it is necessary to perform indexing that allows efficient querying. The user will need to ensure no sensitive or privacy-revealing data is disclosed. The example details the approach using CSIRO’s Ontoserver. However, sBeacon supports the API schema of the Ensembl OLS V4 specification.

Data onboarding architecture for sBeacon, showing genomic data location submitted to an API Gateway endpoint, AWS Lambda functions handling indexing, metadata written to Amazon S3 in ORC format, CSIRO Ontoserver building the ontology index, and Amazon Athena building the metadata tables.

Figure 1. Data onboarding.

The data onboarding process is summarized by the following steps:

  1. The onboarding starts with the user submitting the location of the genomic data as request payloads to an API Gateway endpoint.
  2. The request payloads are forwarded to an AWS Lambda function that handles the data indexing.
  3. The metadata is written to an Amazon S3 bucket in the ORC format, to allow future querying and processing by Athena.
  4. An AWS Lambda function is called to orchestrate the indexing process.
  5. The CSIRO Ontoserver is called to build the ontology index for advanced metadata queries.
  6. The resulting index files are written to Amazon S3.
  7. CREATE TABLE AS SELECT (CTAS) queries are run on Amazon Athena to build the metadata tables.
  8. Athena loads the metadata from Amazon S3 into the metadata tables.
  9. The metadata tables are written back to Amazon S3 in ORC format.

Data querying

Querying in sBeacon is flexible, catering to a wide range of applications from human genetic disease to pathogen queries. We achieved this by designing the query architecture modularly. This approach let us separate the querying logic into several Lambda functions based on their querying scope, while maintaining a similar architecture.

The following architecture diagram describes the workflow for metadata querying, which uses the Variant Querying Module described later in this section.

Metadata querying architecture for sBeacon, showing a user query sent to an API Gateway endpoint, the Microservice Lambda function looking up ontology terms in Amazon DynamoDB, querying metadata tables in Amazon Athena, and querying the Variant Querying Module before returning a Beacon-formatted result.

Figure 2. Data querying.

  1. The user submits their query to the API Gateway endpoint.
  2. API Gateway calls the Microservice Lambda function.
  3. The Microservice Lambda function looks up the relevant query ontology terms in an Amazon DynamoDB table.
  4. The matching ontology descendent terms (and their codes) are returned to the Microservice Lambda function. The descendent terms are those that match a hierarchical descendent of each term, or each term itself, from the query.
  5. Using the ontology codes from step 3, the metadata tables on Athena are queried.
  6. The metadata associated with the query is returned from Athena.
  7. If required by the query, the Microservice Lambda function queries the Variant Querying Module.
  8. The variant data associated with the genomic conditions in the query is returned to the Microservice Lambda function.
  9. The result is formatted according to the Beacon protocol and is returned to the user through Amazon API Gateway.
  10. The response is received by the user.
Variant Querying Module architecture for sBeacon, showing an Initiator Lambda function fanning out the splitQuery and performQuery Lambda functions across VCF files in Amazon S3, and optionally querying metadata from Amazon Athena before returning results to the Microservice Lambda function.

Figure 3. Variant Querying Module.

Genomic variant queries are performed using the Variant Querying Module. The workflow of this module is as follows:

  1. The Microservice Lambda function calls an Initiator Lambda function.
  2. The Initiator Lambda function fans out the splitQuery Lambda function across the VCF files.
  3. The performQuery Lambda function is then fanned out across the VCF regions in each of the files involved in the query.
  4. The performQuery Lambda function fetches the VCF files from Amazon S3.
  5. The query results are synchronously returned to the parent Initiator Lambda function.
  6. If requested by the user, metadata can optionally be queried, where the Initiator Lambda function queries the metadata from Athena.
  7. Athena queries the metadata from Amazon S3 (through an external table).
  8. The metadata results are returned to Athena.
  9. The Initiator Lambda function receives the metadata from Athena.
  10. All the query results, including any optional metadata, are returned to the calling Microservice Lambda function.

Case study: 1000 Genomes dataset

We demonstrate sBeacon on chromosome 1 of the 1000 Genomes Project to report how it handles large-scale variant queries. We measure ingestion efficiency, query scalability, and cost for typical population-scale analyses, such as identifying SNP variants across defined genomic regions. The case study uses chromosome 1 (chr1, 8% of the genome) from the 1000 Genomes Project, which contains 2504 samples. This multi-sample VCF is approximately 1.1 GB compressed, with data stored in Amazon S3. Note that sBeacon can also process cohorts of single-sample VCF files. All costs in this section are for the Asia Pacific (Sydney) Region (ap-southeast-2), exclude applicable taxes, and reflect pricing at the time of writing.

sBeacon can ingest chromosome 1 from the 2504 individuals in 18 seconds, for less than 1 cent (USD 0.00052). This is because sBeacon does not copy the large genomic information but instead creates index files that enable random access. Cost is therefore driven predominantly by storing the copied metadata. After ingestion, sBeacon can be maintained for USD 0.000025 per month (1 MB of compressed metadata stored for 2504 samples in ORC format, plus genomic index files). If you store the genomic data as well, this would be USD 0.032 for chr1 (at USD 0.025 per GB in ap-southeast-2) or about USD 0.425 for the whole genome.

Query time is similarly near real time. For example, querying across a region of 10,000 base pairs to determine the genotypes in this region takes 1.52 seconds across the 2504 individuals. This would serve a query such as “Fetch all individuals with a specific BRCA1 mutation who have stage 3 cancer.” The cost for such a query is USD 0.00013. Note how the query time stays constant even with an increasing number of variants returned (for example, from 4 to 400).

Table 1. Query example costing and times (whole chromosome 1).

Query region size (bases) Number of variants found Average Time Compute Cost (per query in USD)
10 4 1.51 s (+- 0.26) 0.00013
100 18 1.52 s (+- 0.25) 0.00013
1,000 84 1.62 s (+- 0.24) 0.00014
5,000 229 1.65 s (+- 0.29) 0.00014
10,000 400 1.52 s (+- 0.11) 0.00013

Table 2. Cost for ingestion, querying, and idling (whole chromosome 1 for 2504 genomes with less than 10 MB of metadata).

Scenario Metric Cost (USD) per month
Ingestion Cost per 1000 ingestions 0.53 (32.82 GB seconds of Lambda)
Query compute cost per 1000 queries 0.28 (9.8 GB seconds of Lambda)
Query Athena Cost per 1000 queries 0.05
Idle Cost (Storage Cost) 1.1 GB 0.03
Query DynamoDB Cost Per 1000 queries 0.0005

Security features

Security and compliance is a shared responsibility between AWS and the customer. AWS is responsible for protecting the infrastructure that runs the AWS services described in this post, and you are responsible for your use of those services, including how you configure them, which identities you grant access to, and which data you choose to onboard. Consider the services you choose carefully, because your responsibilities vary depending on the services used, how you integrate those services into your IT environment, and applicable laws and regulations. For more information, see the AWS Shared Responsibility Model.

Zero trust model

  • Explicit authentication and authorization – Every API request must carry a valid JWT issued by the Amazon Cognito user pool (aws_api_gateway_authorizer.BeaconUserPool-authorizer, type COGNITO_USER_POOLS). The authorizer runs at API Gateway before any Lambda function is invoked, so requests do not reach a handler without Cognito validation. Token validation includes signature, expiry, and audience (Cognito app client ID). You can disable authentication during the first deployment with BEACON_ENABLE_AUTH = false for intentionally public or open beacons. This is an explicit operator decision, not a default.

Authorization (what a valid user can do) is enforced inside the Lambda layer, not in Amazon API Gateway:

  • Group membership (sbeacon-record-access-user-group, and so on) controls the maximum granularity returned.
  • Admin-only operations (dataset submission, deletion) check for sbeacon-admin-group membership before proceeding.
  • Least-privilege data access – sBeacon implements role-based access control (RBAC) through Cognito groups that map directly to disclosure tiers. You assign each user one or more of the following:
Cognito group Maximum disclosure
sbeacon-boolean-access-user-group exists: true/false only
sbeacon-count-access-user-group aggregate counts
sbeacon-record-access-user-group full variant details and sample names
sbeacon-admin-group preceding tiers plus dataset management

The JWT carries the user’s group memberships as claims. The query Lambda function reads these claims to determine requested_granularity and include_details, then passes both flags to performQuery. performQuery computes only what was requested. A boolean-tier user’s request does not cause sample-level data to be computed or returned, even if it exists in the VCF.

  • Ephemeral compute isolation – Lambda execution environments are stateless by design. Each cold start is a fresh container, /tmp (1,024 MB for performQuery) is cleared between cold starts, and concurrent invocations run in separate sandboxes with no shared memory. The bcftools subprocess inside performQuery runs and exits within the Lambda function lifetime (10 second timeout). No state persists after invocation.
  • Cloud-native boundary controls – API Gateway is the public entry point in this architecture. Amazon S3 buckets, DynamoDB tables, Athena, and Amazon SNS topics have no public resource policies. Amazon S3 buckets are created with private ACLs and BucketOwnerPreferred ownership controls. Lambda functions run on AWS-managed VPCs with no inbound network access. Amazon SNS topics are account-private (no external principal grants).

Privacy and data ownership

Each institution deploys the entire Terraform stack into its own AWS account, so there is no shared infrastructure, no central data lake, and no cross-account trust. VCF files live in the deploying institution’s Amazon S3 bucket and do not leave it. performQuery passes the Amazon S3 URL directly to bcftools as a subprocess argument, which uses htslib HTTP byte-range requests to read only the tabix-indexed region of interest (about 1 KB per query). The raw genomic sequence bytes do not pass through Lambda memory as returnable data. What the query returns upstream (exists as a boolean, call_count as an integer, and variant representations) is aggregate result data, not source sequence.

Decentralization in sBeacon is achieved at the storage layer, not the compute layer. The _vcfLocations registered for a dataset are Amazon S3 URIs, and these can point to buckets owned by entirely different organizations. When a query runs, performQuery passes each URI directly to bcftools, and htslib issues HTTP byte-range requests (Range: bytes=X-Y) against the Amazon S3 REST API of whichever organization owns that bucket. The raw VCF bytes do not leave the source organization’s Amazon S3 bucket. Only the query result (exists, count, or variant record) is returned.

Data onboarding privacy

The submitDataset endpoint sits behind the same API Gateway Cognito authorizer as all other endpoints. An unauthenticated request receives a 401 response before reaching any Lambda function. Beyond authentication, the handler also checks that the caller is a member of sbeacon-admin-group. A valid token from a user in only record-access or count-access is rejected. This means the beacon operator explicitly controls the set of people who can introduce data into the system, so onboarding is not a self-service capability.

Further considerations

We chose AWS Lambda over AWS Step Functions in this architecture because it can process much larger payloads. Given the size and complexity of genomic data and the fan-in and fan-out architecture for parallel handling, AWS Lambda emerged as the lower-cost and more flexible approach for this workload.

As demonstrated in the sBeacon publication, the architecture can cater to population-scale datasets. However, if you accidentally attempt to run a range query of the entire genome, the architecture times out at the Amazon API Gateway level. Applying functional operations over the whole genome requires further architectural considerations.

Because a single fan-out query spawns many parallel Lambda invocations, you need to monitor concurrency consumption to confirm that burst queries do not exhaust the account’s concurrency pool and starve other functions. Tracking the ConcurrentExecutions metric at both the account and function level provides early visibility into capacity pressure.

Similarly, because synchronous Lambda invoke does not automatically retry on throttle, a 429 response from a performQuery invocation means the result is silently lost unless the application handles it explicitly. Setting Amazon CloudWatch alarms on the Throttles metric for performQuery allows you to take corrective action, such as requesting a concurrency limit increase, before throttles affect query accuracy. Alternatively, we have produced a separate architecture that sends alert email with diagnostic information when Lambda functions fail, available in the error-catcher repository on the GitHub website. You can implement this in the repository or set it up as a standalone service to catch Lambda errors thrown by sBeacon.

After idle periods, simultaneous performQuery invocations might encounter cold starts that add latency to query responses. Enabling provisioned concurrency on the query-path Lambda functions helps reduce this cold-start latency during burst fan-out scenarios at the price of increasing the idle cost.

Conclusion

In this post, we described how CSIRO built sBeacon, a fast, scalable, and low-cost way to run genomics workloads on AWS. sBeacon implements the GA4GH Beacon standard with a fully serverless and modular architecture. This publicly available solution supports near real-time querying of standard VCF data, scales to mega-biobank cohorts, minimizes ingestion effort, and supports privacy and zero-trust security. If you are considering genomics on AWS, you can deploy sBeacon on existing Amazon S3-hosted VCF data, integrate it with clinical or research workflows through the Beacon API, and progressively federate with other Beacons for secure, cross-institutional genomic data discovery. Set up sBeacon to query your genomic data and explore the possibilities of securely sharing insights with your collaborators. You can read more about sBeacon in our publication: Scalable genomic data exchange and analytics with sBeacon. The source code for sBeacon can be downloaded from our GitHub repository.

About the authors

[$] Looking forward to Git 2.56 — and 3.0

Post Syndicated from corbet original https://lwn.net/Articles/1094575/

The Git source-code management system is
at the core of development processes worldwide, so changes, especially
incompatible changes, are of great interest to the developers involved.
The Git 2.56 release, which can be expected around the end of September, is
currently available in release-candidate form. It
is not the most earth-shaking of releases, but the one that follows, which
might be the long-awaited Git 3.0, may well be.

Systemtap 5.6 released

Post Syndicated from corbet original https://lwn.net/Articles/1095220/

Version 5.6 of the Systemtap tracing tool has been released.

BPF LSM hooks and XDP packet-processing probes for the –bpf
runtime, BTF-based kernel.tracepoint probes, statement execution
tracing, a new @enumname() operator, richer runtime error context,
dyninst hardware watchpoints, modern systemd service templates, and
broad Linux 7.2 runtime/tapset compatibility work. Multithreaded
speedups throughout.

Security updates for Friday

Post Syndicated from corbet original https://lwn.net/Articles/1095219/

Security updates have been issued by AlmaLinux (.NET 10.0, coreutils, kernel, libevent, libsoup3, microcode_ctl, perl-Net-DNS, postgresql18, postgresql:16, postgresql:18, tomcat, and unbound), Debian (bind9, chromium, libapache2-mod-auth-openidc, nginx, xz-utils, and zip), Fedora (chromium, freeipmi, GitPython, gnatcoll, nodejs-undici, parted, python-django5, and sblim-cmpi-base), Mageia (imagemagick and python-starlette), Oracle (.NET 10.0, .NET 8.0, .NET 9.0, coreutils, corosync, firewalld, kernel, libevent, libsoup, microcode_ctl, nginx:1.24, perl, perl:5.32, postgresql:16, postgresql:18, redis, rsync, rsyslog, tesseract, and unbound), Red Hat (vim), SUSE (alsa, chirp, chromium, cjose, cups, discount, firefox, gh, glibc, gvfs, jq, kernel, libcjose-devel, libmbedcrypto7, libpcap, mbedtls-2, netcdf, nodejs18, openai-codex, openvpn, pcre2, perl-net-dns, sngrep, tiff, and znc), and Ubuntu (bison, bubblewrap, and gst-plugins-good1.0).

Are AIs Still Struggling with CAPTCHAs?

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/09/are-ais-still-struggling-with-captchas.html

Anthropic’s recent security-incident document contains a bit about how CAPTCHAs are still frustrating Claude.

In the transcript, the Claude model that is so powerful that Anthropic is gatekeeping access to it appeared to slam its virtual head against the wall solving a simple image identification test. In a test where the agent was asked to identify a shape that didn’t match the others displayed, it couldn’t even decide which image to select. Instead, it repeatedly went over the same images and questioned its own conclusions.

“Actually hmm, wait,” it said in its chain-of-thought transcript, later adding “Ugh,” because we’ve decided that we need to inject human mannerisms into these machines for some reason. The whole thing took so long that the agent eventually realized that the challenge had expired and it would have to start the process again.

At one point, the model struggled to recognize that the CAPTCHA had opened in a new window and couldn’t figure out what its next steps were supposed to be. At one point, it theorized that the test might be “broken by design” and presented human-like anger in its transcript meant for a human audience: “SO WHAT THE HELL IS WRONG WITH THE ANSWERS?”

Meanwhile, I’ve read reports—none of them official—that GPT-6 Astra solved all forty-eight levels of Neal Agarwal’s “I’m Not a Robot” game.

It’s hard to know what to believe right now.

Как държавата режисира ужаса от Петрохан

Post Syndicated from Емилия Милчева original https://www.toest.bg/kak-durzhavata-rezhisira-uzhasa-ot-petrohan/

Как държавата режисира ужаса от Петрохан

Властта умее да режисира ужаса така, че сама да остане извън кадър, когато е необходимо. Показва ни отблизо чудовище – деянията му, жертвите, най-интимните подробности от чуждите травми, и ни оставя да поискаме още от същото. Една част от хората избират да се отвратят от чудовището, друга – от институциите манипулатори, трета, най-малката, и от двете.

Така беше режисиран ужасът от Петрохан шест седмици преди президентските избори. След осем месеца разследване и неприключило досъдебно производство прокуратурата и МВР избраха да разкажат какво е извършил т.нар. лама Ивайло Калушев – мъртъв, както и останалите петима от затворената общност, обитавала планинска хижа; човекът, манипулирал възрастни и прекрачил границите с деца, поверени му за отглеждане от родителите им по неясни причини.

Заключенията са, че няма данни за външна намеса, тоест екзекуторите трябва да се търсят в самата група, и делото най-вероятно ще бъде прекратено, тъй като извършителите са мъртви.

Като главен секретар на МВР Георги Кандев също беше съобщил, че няма данни за външна намеса нито на Околчица, нито на Петрохан.

Огласената версия е, че Ивайло Калушев, застрелял 22-годишния Н.З. и 15-годишния А.М., е организирал фаталния край на групата поради страх от разкрития за действията му с деца.

Кръв, порно, политика

Това, което дават по новините, е белег за нещо отдавна случващо се под носа ни: умишления отказ на институциите да си вършат работата. Коментар на Емилия Милчева.

Гавра с правосъдието

На двучасова пресконференция криминалният психолог Росен Йорданов произнесе моралната присъда, въпреки че няма обвинение, и нареди пред публиката секс с подрастващи, окултни практики, манипулации, оръжия и шест трупа. Говореше ту като вещо лице, ту „като човек“, остро и назидателно, сякаш не представя експертно заключение, а защитава обвинителна теза пред съдебни заседатели. Само че съдебни заседатели нямаше, имаше камери и оставаше само някой да разпространи „обвинителния акт“. 

Поведението на Йорданов допълнително разруши и без това оскъдното доверие в разследването и в институции като МВР и прокуратурата. Още на 10 и 15 февруари пред БНТ, преди да бъде назначен за вещо лице и преди да получи достъп до материалите, той вече беше определил Калушев като човек с „тежки нарцистични проблеми“, „професионален прелъстител и педофил“ и беше обяснил мотивите му. На 17 март същият психолог е привлечен като експерт, за да изследва безпристрастно именно личността, поведението и мотивите, за които вече публично е произнесъл присъдата си.

Затова адвокатите Ина Лулчева и Деница Тодорова, представляващи близки на загиналите, поискаха от прокуратурата да отстрани Йорданов и още две вещи лица заради пристрастност и предубеденост. Адвокат Димитър Марковски нарече пресконференцията „груба грешка“ – при 12 незавършени експертизи прокуратурата е фаворизирала една версия, преди да е приключило събирането и проверката на доказателствата. По БНР правозащитникът адвокат Михаил Екимджиев определи поведението на Йорданов като „фрийкшоу“ извън нормалните представи за професионална етика.

Самият Йорданов отказа да се оттегли. Обяви, че не е предубеден, а емоционален – заради случая. За адвокатите, поискали отвода му, каза: 

Те нямат какво друго да направят.

Този отказ не е маловажен професионален спор. Ако прокуратурата запази представената версия, наказателното производство за убийствата ще бъде прекратено, защото посочените извършители са мъртви. Няма да има открит съдебен процес, в който експертизата да бъде оспорена, вещите лица да бъдат разпитани, защитата да представи други доказателства и съдът да прецени коя версия издържа. Евентуалната жалба срещу прекратяването ще бъде разгледана в закрито заседание. 

Така публичният разказ на един експерт, нает от прокуратурата, може да остане единствената присъда. 

Ето така държавната режисура постига целта си. На обществото е даден достатъчно ужасяващ разказ, за да не пита защо трябва да му се вярва. Но доверие в институциите, още по-малко в българската съдебна система няма – дори и да казват част от истината. 

Да се разобличи прокуратурата не означава Калушев да бъде превърнат в невинна жертва на зли сили. Един критично мислещ човек няма да приеме присъда без съд, нито пък безусловна реабилитация на мъртвия. Съмненията в институция с дълга история на избирателни течове и политическа употреба не изключват тревожните факти за деца, оставени под едноличната власт и опека на възрастен мъж и откъснати от училище и семейство. 

Калушев не става невинен само защото прокуратурата злоупотребява с фактите и обслужва определени интереси. Нито пък самата прокуратура става достойна за доверие поради избора на убедително чудовище. 

В престрелката между основните лагери, на които се разполови общественото мнение, се изгубиха децата – жертви на възрастни, отказали да изпълнят задълженията си. 

Сексуалните престъпления срещу деца. Институционална и обществена слепота

В предишната си статия Теодора Станимирова представи данни, разкриващи системното безсилие на институциите спрямо сексуалните злоупотреби с деца. В продължението на темата Теодора разговаря с експерти, за да разбере каква е реалната ангажираност на обществото и институциите – отвъд популизма.

Гаврата с децата

Задочната психологична експертиза беше „предоставена“ за публикуване в сайта „Епицентър“, чиято главна редакторка Валерия Велева не се посвени да я пусне, без да бъдат заличени имената на децата, посочени като жертви на обявилия се за лама. През март 2010 г. лидерът на ДПС Ахмед Доган ѝ написа отворено писмо, обръщайки се към нея с прозвището „Мадам В.“, и я обвини в корупция и търговия с влияние.

Днес, 16 години по-късно, политическата употреба се оказа по-важна от защитата на децата (и техните родители). Асоциацията на европейските журналисти (АЕЖ) обяви, че ще сезира Държавната агенция за закрила на детето, Комисията за защита на личните данни и ГДБОП. 

Междувременно Велева напусна инициативния комитет на кандидатпрезидентската двойка Илияна Йотова – Кирил Вълчев след призиви да се оттегли. От самия инициативен комитет се разграничиха „от публичното изнасяне на лични данни, независимо от случаите, за които се отнася това“. 

Оттеглянето ѝ не отговаря на главния въпрос: 

Кой нарежда на прокуратурата да продължи практиката от времената на главните прокурори Иван Гешев и Борислав Сарафов за манипулации чрез течове на материали от досъдебни производства?

Има и друг съществен аспект – за ролята на контраразузнаването в аферата „Петрохан“. Според лидера на „Продължаваме промяната“ Асен Василев ДАНС е знаела какво се случва в хижа „Петрохан“ още от 2022 г. и е бездействала.

Очевидно някой в ДАНС покровителства групи, които нанасят щети на българските деца… Истинският въпрос е, когато са подавани сигнали в ДАНС през 2022 г., къде е спал ДАНС три години, какво е направил, кой в ДАНС е покровителствал това нещо, защо ДАНС не са сигнализирали прокуратурата. 

И получи отговор от шефа на ДАНС Пламен Тончев:

Проверете делото, образувано в прокуратурата през януари 2025 г. по информация на ДАНС. Там има всички пунктове и точки, които са изнесени в обвинението и днес, само че тогава, ако някой беше реагирал навреме, тези хора може би щяха да бъдат живи.

В документа от януари е била спомената и педофилия. Самият Тончев, назначение на Румен Радев, оглавява ДАНС от 2021 г., с известно прекъсване, когато беше преместен като шеф на Комисията по досиетата.

Всъщност още през февруари, десетина дни след откриването на труповете, разследващият сайт bird.bg публикува секретната справка от ДАНС. Още тогава стана ясно, че прокуратурата не е свършила нищо по преписка за „извършени сексуални или блудствени действия от Ивайло Калушев, относими към Глава Втора, Раздел VIII от Наказателния кодекс“. 

Резултатът от прехвърлянето на преписката между три прокуратури са тримата мъртъвци от „Петрохан“ и другите трима, един от които дете, в кемпера под връх Околчица. 

А до фаталните изстрели 15-годишният А.М. е живял в затворената общност, откъснат от обичайния си семеен и училищен живот. Преди него в същата среда и под контрола на Калушев там е израснал от малък и Н.З.

Родителите може да са били манипулирани, че поверяват децата си на т.нар. лама. Това обаче не отменя отговорността им – родителството не може да бъде преотстъпено на самопровъзгласил се духовен водач заедно с правото му да контролира всекидневието, образованието и съзнанието на детето. 

Частното училище „Космос“, където е бил записан А.М., също е допуснало продължителните му отсъствия да не задействат системата за закрила. А сега прокуратурата разследва нарушения при приема на ученици и издаването на документи с невярно съдържание, както и неизпълнение на задължения от служители на училището, МОН и регионалното управление. 

Около тези деца е имало кръг възрастни – родители, учители, чиновници и контролни органи. И от всичко това излиза, че държавата, която не успява да забележи навреме изчезването на едно дете от обичайната му среда, все пак успява да забележи интимните подробности от живота му и дори решава да ги покаже на всички. Емил Дечев, служебен министър на вътрешните работи в кабинета „Гюров“, ясно посочи, че „големият отсъстващ е българската прокуратура“, и призова за обективно разследване на всички версии.

Шест седмици преди президентските избори случаят „Петрохан“ е превърнат в оръжие за политическо поразяване на силите, подкрепили кандидатпрезидентската двойка Андрей Гюров и Георги Кандев, също и на кмета на София Васил Терзиев. Както стана ясно по-рано, Терзиев е дарил над 125 000 евро лични средства за дейността на групата и е посещавал нееднократно хижата.

Какво (не) знаем за сексуалните злоупотреби с деца

Какво знаят институциите за сексуалните злоупотреби с деца в България и какви мерки предприемат? Теодора Станимирова се сдоби с информация от ВСС, МВР, АСП, ДАЗД и МЗ, разговаря с експерти и ни разказва какво е научила.

Паралелно с това премиерът Румен Радев също не се засрами да употреби децата. 

Той избра училищния двор и първия учебен ден, за да нарече организацията „свърталище на педофилия“, а Калушев – „хладнокръвен убиец на деца и откровен педофил“. След психолога експерт Йорданов, който се похвали, че премиерът му благодарил, сега и Радев произнесе присъда, преди да е приключило разследването. 

Държавата, която не опази децата от „Петрохан“, подреди други деца за фон на закъснялото си възмущение. 

Какво се скри в мъглата?

Чудовището се оказа твърде удобно за властта. Колкото по-дълго гледаме него, толкова по-малко забелязваме растящите цени на горивата и газа, които внасят инфлация. 

След президентските избори на 25 октомври случаят „Петрохан“ ще бъде изместен от първите сметки за отопление и от увеличените разходи за живот. Последните данни на националната статистика за август показват инфлация от 5,1% на годишна база и ускоряваща се през последния летен месец.

В мъглата се скри границата между познанство, дарение, политическа подкрепа и съучастие. Имената на политици бяха вкарани в един и същ разказ с убийства и сексуално насилие, без да са представени данни, че са знаели за тях. Така вината по асоциация свърши онова, за което доказателствата не стигат – превърна контактите с Калушев в политическо обвинение. 

Не сме длъжни да избираме от какво да се отвратим. Достатъчно е да знаем, че институциите не защитиха децата, но пристигнаха навреме за камерите. 

Run open weight models on AWS Bedrock in AWS European Sovereign Cloud

Post Syndicated from Marta Taggart original https://aws.amazon.com/blogs/security/run-open-weight-models-on-aws-bedrock-in-aws-european-sovereign-cloud/

European organizations can run AI workloads on Amazon Web Services (AWS) while keeping data within the European Union (EU) and meeting regulatory requirements. You can now run generative AI workloads on open weight models on Amazon Bedrock in the AWS European Sovereign Cloud. We’re excited to announce the general availability of the first open weight model family, Gemma 4, on the Amazon Bedrock next-generation inference engine in the AWS European Sovereign Cloud. Gemma 4, released under the Apache 2.0 license, on Amazon Bedrock benefits from the same data residency and operational controls that define the AWS European Sovereign Cloud so you can build, iterate, and scale generative AI applications while meeting digital sovereignty requirements.

The AWS European Sovereign Cloud is an independent cloud for Europe, located entirely within the EU, designed to help customers meet their most stringent digital sovereignty requirements. It runs entirely within the EU and is independently operated with strong technical controls, sovereign assurances and legal protections. Only AWS employees who reside in the EU control day-to-day operations, including access to data centers, technical support, and customer service.

In this post, we explain how the Amazon Bedrock inference engine protects your inference data when running Gemma 4 models, how the AWS European Sovereign Cloud keeps it within the EU, and then walk through the available Gemma 4 models and your first inference request.

Next generation inference engine for Amazon Bedrock

The inference engine is a distributed engine for serving large-scale machine learning models, built for high performance, reliability, and security. You reach it through the bedrock-mantle endpoint, which supports OpenAI-compatible APIs (the Responses and Chat Completions APIs). You can bring an existing OpenAI SDK codebase to Amazon Bedrock by changing only the base URL and API key. The Responses API supports stateful conversation management, which rebuilds context without you passing conversation history with each request. Stored responses are scoped by Amazon Bedrock project, a logical boundary that represents a workload for access control, cost tracking, and usage monitoring.

The engine applies the same operational security practices you rely on across AWS. Access follows a least privilege model, where each operator has access only to the systems a specific task requires, and only for the time that privilege is needed. Any access to systems that store or process customer data or metadata is logged, monitored for anomalies, and audited. All your prompts and responses are kept private during inference.

How your inference data is protected

Amazon Bedrock uses a zero operator access data security model, meaning no service operators can access model input or output during inference. It also uses a zero data retention model, so by default it doesn’t store your inputs or outputs. For certain models, limited retention might apply for abuse detection (see the Amazon Bedrock abuse detection documentation). Your prompts and responses are encrypted in transit and, by default, are not shared with the model provider.

All inference stays within the eusc-de-east-1 AWS Region as described in the following section on data residency. Combined with the data residency and EU-based operations of the AWS European Sovereign Cloud, this gives organizations in highly regulated industries the confidence to run their most sensitive AI workloads in the cloud.

Data residency and regional availability

The AWS European Sovereign Cloud became generally available in January 2026, with its first Region in Brandenburg, Germany (eusc-de-east-1). It’s a separate, independently operated cloud, with infrastructure located entirely within the EU and no critical dependencies on non-EU personnel or infrastructure. All your content remains within the Region you select unless you choose otherwise. Beyond content, customer-created metadata including roles, permissions, resource labels, and configurations also stays within the EU. The AWS European Sovereign Cloud is operated exclusively by EU residents located in the EU. We’re also gradually transitioning the AWS European Sovereign Cloud to be operated exclusively by EU citizens located in the EU. During this transition period we will continue to work with a blended team of EU residents and EU citizens located in the EU.

All Amazon Bedrock inference requests, including Gemma 4, use in-Region inference in eusc-de-east-1, which keeps every request within the AWS European Sovereign Cloud. Global cross-Region inference, which routes requests across commercial AWS Regions worldwide, isn’t available in the AWS European Sovereign Cloud.

Control over who can access your data

With AWS Identity and Access Management (IAM), you decide which principals in your account can call the inference API and which models they can use. Fine-grained permissions let you grant only the access each workload needs, following least privilege, and we recommend short-lived credentials over long-term keys.

For auditing, every call to the endpoint is recorded in AWS CloudTrail, giving your security and compliance teams an audit trail of who invoked inference and when. You can also monitor usage with Amazon CloudWatch and set alarms on patterns that matter to you, such as unexpected spikes in request volume.

Open weight models in the AWS European Sovereign Cloud

Organizations adopting open weight foundation models (FMs) for production face a constant challenge: how to access the leading models without compromising on data protection, regulatory alignment, or operational control. Amazon Bedrock removes that challenge. It gives you leading open weight FMs through a fully managed service, with inference running entirely on infrastructure operated by AWS and the security and privacy controls you expect from Amazon Bedrock. Because the models are open weight, you can independently evaluate the model architecture and training methodology, benchmark your own workloads, and fine-tune on proprietary data when customization is required.

Gemma 4 is a family of open weight models, released under the Apache 2.0 license. It’s available in three instruction-tuned variants, so you can evaluate and choose the model that fits your workload. The following table provides guidance on which model to choose based on your use case:

Model

Use case

Specifications

Gemma 4 31B (google.gemma-4-31b-it)

Reasoning-heavy or coding-heavy with a single dense model

30.7 billion parameter dense model with a 256 K token context window

Gemma 4 26B-A4B (google.gemma-4-26b-a4b-it)

Cost-sensitive at high throughput, with knowledge breadth requirements

Mixture-of-experts model with 25.2 billion total parameters and 3.8 billion active per token, with a 256 K token context window

Gemma 4 E2B

(google.gemma-4-e2b-it)

Latency-sensitive, on-device-style, or multimodal classification

Compact model with 5.1 billion total parameters and 2.3 billion effective parameters using per-layer embeddings (PLE), with a 128 K token context window

All three variants offer built-in reasoning, native function calling, and multimodal input across text and image.

Get started with Gemma 4 models on Amazon Bedrock

Gemma 4 is served through the bedrock-mantle endpoint, the OpenAI-compatible API for the next-generation inference engine, so you can call it with the OpenAI Python and TypeScript SDKs. Use the following steps to use the OpenAI Python SDK to send your first request to Gemma 4 31B in the AWS European Sovereign Cloud.

Prerequisites

To follow this example, you need an AWS account with access to the AWS European Sovereign Cloud and an IAM principal with permissions to call the bedrock-mantle endpoint. Create an IAM policy that grants the two actions this walkthrough uses, then attach it to your IAM principal. The bedrock-mantle:CreateInference action runs inference, and the bedrock-mantle:CallWithBearerToken action authenticates with an Amazon Bedrock API key. The following sample policy grants the actions this example needs. Scope the resources further for your environment as described after the policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "GemmaMantleInference",
      "Effect": "Allow",
      "Action": "bedrock-mantle:CreateInference",
      "Resource": "arn:aws-eusc:bedrock-mantle:eusc-de-east-1:<account-id>:project/<project-id>"
    },
    {
      "Sid": "GemmaMantleBearerToken",
      "Effect": "Allow",
      "Action": "bedrock-mantle:CallWithBearerToken",
      "Resource": "*"
    }
  ]
}

Replace <account-id> and <project-id> with your own values.

Install the OpenAI SDK and the Amazon Bedrock token generator with the command pip install “openai>=2.45.0" aws-bedrock-token-generator.

Authenticate

You authenticate with an Amazon Bedrock API key. Amazon Bedrock offers two types of API keys. Short-term keys expire automatically within 12 hours and inherit the permissions of the IAM principal that generated them, which makes them the recommended choice for production. Long-term keys last until a configured expiration and are intended for development and exploration. For production, use the auto-refreshing short-term key shown in the following example, or store the key in AWS Secrets Manager.

from aws_bedrock_token_generator import provide_token
from openai import BedrockOpenAI

region = "eusc-de-east-1"

client = BedrockOpenAI(
    aws_region=region,
    base_url="https://bedrock-mantle.eusc-de-east-1.api.amazonwebservices.eu/v1",
    bedrock_token_provider=lambda: provide_token(region=region),
)

Alternatively, you can pass a short-term API key through an environment variable. This key isn’t refreshed and expires after at most 12 hours.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://bedrock-mantle.eusc-de-east-1.api.amazonwebservices.eu/v1",
    api_key=os.environ["AWS_BEARER_TOKEN_BEDROCK"],
)

Run your first inference with the Responses API

The Responses API uses a single input field and returns the generated text in output_text. Setting store to false means Amazon Bedrock doesn’t retain the request or response.

response = client.responses.create(
    model="google.gemma-4-31b-it",
    input="Explain the benefits of open-weight models for regulated industries.",
    max_output_tokens=512,
    store=False,
)
print(response.output_text)

Call the Chat Completions API

You can also call the OpenAI-compatible Chat Completions endpoint directly. If you use AWS credentials instead of an API key, sign the request with AWS Signature Version 4 (SigV4), as in the following example.

export ENDPOINT=https://bedrock-mantle.eusc-de-east-1.api.amazonwebservices.eu
export AWS_REGION=eusc-de-east-1

curl -X POST ${ENDPOINT}/v1/chat/completions \
  --aws-sigv4 "aws:amz:${AWS_REGION}:bedrock" \
  --user "${AWS_ACCESS_KEY_ID}:${AWS_SECRET_ACCESS_KEY}" \
  -H "x-amz-security-token: ${AWS_SESSION_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google.gemma-4-31b-it",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Clean up

This walkthrough creates no persistent resources, so there’s nothing to delete. The short-term API keys used here expire automatically within 12 hours.

Pricing and availability

Gemma 4 is available in Amazon Bedrock in the AWS European Sovereign Cloud. You pay per token with no upfront commitment, and usage counts toward your existing AWS commitments. For current pricing, see Amazon Bedrock pricing. For model and Regional availability, see Regional availability by models.

Commitment to innovation

Beyond the technical integration, running AI workloads in a sovereign context raises important questions about requirements. As you plan AI workloads for a sovereign context, evaluate them against your organization’s requirements for data residency, model governance, and operational control. The AWS European Sovereign Cloud is designed to help you meet these requirements in the EU.

AWS is committed to making AWS the best place for European organizations to innovate with AI, without compromise. To learn more about AWS European Sovereign Cloud visit aws.eu.

If you have feedback about this post, submit comments in the Comments section below.

Author

Marta Taggart

Marta is a Principal Product Marketing Manager focused on digital sovereignty and the AWS European Sovereign Cloud in AWS Product Marketing. She helps customers navigate complex digital sovereignty requirements and understand how AWS solutions can help address their needs so they can build and innovate with confidence. Outside of work, she enjoys yoga and coffee.

Zohreh Norouzi

Zohreh Norouzi

Zohreh is a Senior Security Solutions Architect at Amazon Web Services (AWS). She helps customers make good security choices and accelerate their journey to the AWS Cloud. She has been actively involved in AI security initiatives, using her expertise to help customers build secure AI solutions at scale.

The collective thoughts of the interwebz