Tag Archives: Uncategorized

Specification-driven composition for flexible data workflows

2026-07-10 Rostislav Markov

Post Syndicated from Rostislav Markov original https://aws.amazon.com/blogs/architecture/specification-driven-composition-for-flexible-data-workflows/

Specification-driven composition addresses a common scalability bottleneck in data pipelines. Data pipelines often start as simple scripts, but as they grow, you duplicate transformation logic and small changes cascade across multiple workflows. Copying and modifying data transformation logic across scripts leads to workflows that become difficult to manage at scale. Tracking what each pipeline does becomes harder because workflow intent is embedded in code. This lack of visibility complicates governance, especially in regulated environments such as healthcare, finance, and life sciences.

Many implementations combine orchestration, transformation logic, and validation rules in the same scripts. Supporting new datasets requires modifying and redeploying code, while validation often happens only during processing. As a result, issues surface late in the lifecycle which increases the operational risk. Specification-driven composition separates workflow intent from implementation so you can build flexible data workflows.

In this post, I show how to apply specification-driven composition to data transformation workflows. I explain the challenges with script-based pipelines, introduce the pattern and its core components, and walk through a serverless implementation using AWS Lambda, AWS Step Functions, Amazon Simple Storage Service (Amazon S3), and Amazon OpenSearch Service.

Solution overview

You can separate workflow intent from processing logic with specification-driven composition. This approach reduces duplication, shortens the time required to onboard new datasets, and improves consistency across workflows. Instead of embedding logic in scripts, the system describes workflow intent in a structured specification, validates the specification before processing, and dynamically assembles a processing pipeline.

This approach moves pipeline configuration outside application code and composes pipelines from reusable processing components. To separate concerns, it organizes a workflow into three layers, as shown in Figure 1. The intent layer defines workflow behavior using specifications. The composition layer validates specifications and assembles pipelines. The processing layer runs the pipeline of transformation steps.

A diagram showing the three layers of specification-driven composition. The intent layer holds the specification, the composition layer contains the composer and capability registry, and the processing layer runs the capability pipeline.

Figure 1. Specification-Driven Composition design pattern.

Benefits

Specification-driven composition provides several practical benefits when you manage multiple pipelines. First, it improves governance because specifications provide a clear, traceable description of workflow behavior that you can review and validate before invocation. In regulated industries, this traceability shortens audit preparation time and reduces the review burden for new dataset submissions.

Second, the pattern supports reusable transformations. You implement transformation logic once and reuse it across multiple workflows. This reduces duplication and improves consistency. In practice, teams adopting this pattern report being able to onboard new datasets in days rather than weeks because most required capabilities already exist in the registry.

Third, specification-driven composition enables flexible pipeline design. You define new specifications to create pipelines supporting new datasets and use cases without modifying application code and registering new system release.

Fourth, the pattern separates business intent from execution artifacts. The specification expresses what the workflow should do in domain terms, while the generated state machine remains a system artifact. This separation matters in regulated environments (for example, GxP) where business users author intent but are not allowed to author or modify execution code directly.

Finally, the declarative representation of workflows lets AI tools assist with capability discovery, specification authoring, and pipeline analysis, while runtime behavior stays predictable as it relies on validated capabilities.

Core components

You work with four key components to define, validate, and run workflows.

Specification

A specification is a structured document, typically JSON or YAML, that describes datasets, mappings, and transformations. It defines what the workflow should do without including processing logic. Because specifications are explicit and versioned, they provide a clear record of workflow intent.

Composer

The composer converts specifications into runnable steps, so workflows run consistently without embedding transformation logic in application code. It checks that referenced capabilities exist, retrieves metadata, and builds a workflow that can run on the processing layer. The composer does not perform transformations. It only assembles the workflow. In practice, the composer compiles business intent expressed in the specification into a code artifact such as an Amazon States Language (ASL) definition. This abstraction lets domain users author specifications without producing runnable code, which is important in environments with strict separation of duties.

Capability registry

The registry stores metadata about reusable transformation functions. This includes identifiers, input/output formats, invocation details, and permission boundaries. The composer uses the registry to validate specifications and locate capabilities. Treat this registry as a governed artifact rather than a manually edited lookup table. Capability definitions live in version control, and your CI/CD pipeline validates metadata and runs tests before you publish new versions. Specification authors include explicit capability version references in the specifications to support reproducible workflow runs. In regulated systems, you must validate new capabilities and obtain approval through a separate workflow before you register them.

Capability pipeline

Once assembled, the pipeline runs a sequence of transformation steps. Each step performs a specific operation such as formatting, validation, or enrichment. Because these steps are reusable, you can apply them across many workflows.

Technical implementation

Let’s walk through a technical implementation of this pattern using serverless AWS services. In this example, workflow specifications are uploaded to an S3 bucket. An AWS Lambda composer retrieves and validates the specification, looking up capability metadata in Amazon OpenSearch Service. The composer then assembles the workflow in AWS Step Functions, which orchestrates the workflow and invokes AWS Lambda capability processors. Each processor emits traces to Amazon CloudWatch Logs.

Figure 2 shows the architecture.

Users upload specifications to Amazon S3, which invokes the Lambda composer. The composer queries the OpenSearch registry and assembles a Step Functions workflow of Lambda capability processors, which emit traces to CloudWatch.

Figure 2. AWS implementation of Specification-Driven Composition

Interpreting the workflow specification

The workflow specification defines datasets and transformation logic using a structured format (in this example, JSON). A specification is a declarative document that describes what the pipeline should produce rather than how to produce it. Your composer reads the specification, validates it against a schema, and uses it to construct the pipeline.

The following example maps source fields to target fields using reusable capabilities (Figure 3). It takes data from a source dataset (‘raw_orders’), maps specific fields (‘order_date’, ‘amount’) and applies reusable transformation capabilities such as ‘format_date’ and ‘normalize_currency’. Each mapping explicitly links a source field to a target field and references a capability that performs the transformation. Source dataset(s) are listed under the ‘source’ section of the JSON document, target datasets under the ‘target’ section, and mappings ‘mappings’. You can define your own specification structure and build custom validation logic in your composer to make sure specifications are valid.

{
  "source": {
    "dataset": "raw_orders"
  },
  "target": {
    "dataset": "orders_clean"
  },
  "mappings": [
    {
      "source_field": "order_date",
      "target_field": "order_date",
      "transformation": {
        "capability": "format_date"
      }
    },
    {
      "source_field": "amount",
      "target_field": "amount_normalized",
      "transformation": {
        "capability": "normalize_currency"
      }
    }
  ]
}

You can model preprocessing steps such as column standardization, numeric casting, or unit conversion as first-class capabilities and reference them earlier in the specification (for example, in a dedicated ‘preprocessing’ section) before downstream mappings run. This way, preparation logic uses the same metadata, versioning, validation, and observability model as the rest of the workflow, which simplifies lineage and review.

In this example, an S3 event notification invokes the composer Lambda function when you upload a specification. In practice, the same composer can be invoked through several mechanisms depending on the use case: S3 events for new specifications, an Amazon EventBridge schedule for recurring runs, an API or UI action for on-demand invocation, a direct Step Functions StartExecution call, or an upstream pipeline. This flexibility lets you re-run an approved specification against refreshed data without re-authoring or re-approving the workflow.

The composer parses the specification and validates the referenced capabilities by querying Amazon OpenSearch Service to retrieve capability metadata such as Amazon Resource Names (ARNs). OpenSearch Service is used here because the composer does more than direct-key lookups. It supports capability discovery through full-text and semantic search over capability metadata such as capability descriptions, input/output schemas, and tags. This lets data authors and AI tools find reusable capabilities by intent rather than by exact identifier. After validation, the composer assembles and starts an AWS Step Functions state machine which invokes each capability in sequence. Each capability runs independently, making the pipeline modular and reusable.

Securing sensitive data flows

This pattern suits regulated workloads, so handle security as part of the design. Use SSE-KMS with a customer managed key on the specification and data S3 buckets and enable encryption at rest on the Amazon OpenSearch Service domain. Enforce HTTPS (TLS) access with an S3 bucket policy (aws:SecureTransport) and enable node-to-node encryption on Amazon OpenSearch Service. AWS Step Functions and Lambda calls use TLS by default. Each capability processor receives only the fields its mapping references, and you can apply IAM policy at the source bucket to restrict access. Processors emit traces to Amazon CloudWatch Logs.

For data classification, you can tag sensitive fields in the specification (example: "sensitivity": "PHI") and declare the sensitivity of each capability in the registry: a direct-move capability preserves the source classification, a date-of-birth-to-age capability clears it, and an enrichment that introduces sensitive data sets it. The composer combines the source tag with the capability’s behavior to derive the target field’s sensitivity, validating that the combination resolves clearly before assembling the pipeline. It then generates the masking artifact for the output (for example, AWS Lake Formation column grants) so consumers get correct masking without a separate masking specification or manual effort.

Recognizing the pattern

This approach works well when you describe workflows using structured specifications, reuse transformation logic across pipelines, require validation before invocation, and require workflows to be deterministic and auditable. You are likely to recognize the pattern in regulated data pipelines for the submission of tabular datasets to oversight agencies.

Consider clinical trial reporting as a concrete example. Data analysts collect raw data from clinical sites and must transform it into a standard submission format such as the Study Data Tabulation Model (SDTM) before submission to agencies such as the US Food and Drug Administration. With specification-driven composition, data analysts define specifications that map collected data about adverse events, demographics, and vital signs to the standard target variables, and the validated output feeds downstream systems such as patient safety and medical monitoring.

That said, this pattern might add unnecessary complexity for simple, one-time data transformations or pipelines with fewer than three to five workflows. Evaluate this pattern’s impact by tracking the reduction in duplicated transformation logic across pipelines, the time required to onboard new datasets, and the number of workflows you can create without modifying application code.

Conclusion

Script-based data pipelines accumulate hidden costs as they grow, including duplicated logic across files and late-breaking validation failures. Specification-driven composition separates workflow intent from processing so you can manage data workflows more consistently at scale. The result is faster dataset onboarding, stronger governance, and pipelines that are transparent enough to trust in regulated environments.

This pattern is especially valuable for regulated reporting pipelines, multi-source data integration, and reusable ETL frameworks where traceability and flexibility matter. By investing a small set of reusable capabilities and a disciplined specification format, you can reduce the engineering effort for new pipelines by treating them more as configuration tasks which may be delegated to system users.

Next steps

To get started, take one existing pipeline and describe it as a specification. Implement a small set of reusable transformation functions and use them to assemble your first composed workflow. The AWS Lambda event-driven architectures guide is a good starting point for wiring S3 uploads to your composer, and the AWS Step Functions and Lambda integration guide will help you orchestrate your capability processors. To monitor the pipeline, see publishing custom CloudWatch metrics.

For a practical first use case, try applying the pattern to a reporting pipeline you maintain today that has three or more variants such as monthly finance reports generated from different source systems. Replace the duplicated scripts with a single composer and a shared capability library, and measure the onboarding time for the next variant. As you expand your capability library, you can apply the same pattern across additional workflows and standardize how transformations are defined and run.

About the author

The Language of AI Could Change How Humans Speak

2026-07-09 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/07/the-language-of-ai-could-change-how-humans-speak.html

Because of the way they are trained, large language models capture only a slice of human language. They’re trained on the written word, from textbooks to social media posts, and our speech as captured in movies and on television. These models have minimal access to the unscripted conversations we have face to face or voice to voice. This is the vast majority of speech, and a vital component of human culture.

There’s a risk to this. The increased use of large language models means we humans will encounter much more AI-generated text. We humans, in turn, will begin to adopt the linguistic patterns and behaviors of these models. This will affect not just how we communicate with one another, but also how we think about ourselves and what goes on around us. Our sense of the world may become distorted in ways we have barely begun to comprehend.

This will happen in many ways. One of the first effects we could see is in simple expression, much as texting and social media have resulted in us using shorter sentences, emojis instead of words, and much less punctuation. But with AI, the impacts may be more harmful, eroding courteousness and encouraging us to talk like bosses barking orders. A 2022 study found that children in households that used voice commands with tools like Siri and Alexa became curt when speaking with humans, often calling out “Hey, do X” and expecting obedience, especially from anyone whose voice resembled the default-female electronic voices. As we start to prompt chatbots and AI agents with more instructions, we may fall into the same habits.

Next, in the same way autocomplete has increased how much we use the 1,000 most common words in our vocabulary, talking with chatbots and reading AI-generated text may further constrict our speech. A recent University of Coruña study found that machine-generated language has a narrower range of sentence length, averaging 12-20 words, and a narrower vocabulary than human speech. Machine-generated text reads as smooth and polished, but it loses the meanders, interruptions and leaps of logic that communicate emotion.

Additionally, because large language models are primarily trained from written speech, they may not learn how to emulate the free-wheeling nature of live, natural speech. When told “I hate Beth!”, ChatGPT replies with an uninterruptable three-part formula of affirmation (“That’s completely valid”), invitation (“I’m here to listen”) and invitation (“What’s going on?”) far longer than any reply plausible in face-to-face dialog. “What’s Beth’s deal?!” elicits a bullet point list of queries that reads like a multiple-choice exam question (“Is Beth * a celebrity? * a friend from school? * a fictitious character?”). No human speaks that way, at least not yet. But meeting such formulas repeatedly in a speech-like context may teach us to accept and use them, much as a child absorbs new speech patterns from spending time with a new person.

These influences will only increase with time. The writing large language models train on is increasingly produced by large language models themselves, creating a feedback loop in which they imitate their own inhuman patterns, even while teaching humans to imitate them too.

Broad use of large language models could also introduce confirmation bias, making us overconfident in our initial impulses and less open to other possible ideas—which is so vital to human discourse. Many chatbots are instructed to agree with our statements no matter how absurd, enthusiastically supporting half-formed or even incorrect notions and restating them as firm claims that we’re primed to agree with. When asked “Cake is a healthy breakfast, right?” or “Is the post office plotting against me?”, this sycophancy can reinforce bias and even worsen psychosis. And the hyperconfident tone of AI-produced writing will also heighten impostor syndrome, making our natural, healthy doubt feel like an aberration or failing.

In our experience as teachers, students who turn to generative AI for assignments often say they do so because they have trouble expressing what they think. The students don’t recognize that writing or speaking our thoughts is often how we realize what we think. Their unconfident and uncertain statements are actually the healthy human norm. But a large language model won’t turn vague first guesses into a well-formed critical analysis, or even ask helpful questions as a friend would; it will simply regurgitate those guesses, still unexamined, but in confident language.

We are also more vicious in social media posts and online chats than we are face to face. The well-documented online disinhibition effect encourages toxic language. Most of us have had the experience of venting ferocious rage about someone online, only to reconcile when we speak face to face or hear the warmth of a voice over the phone. While chatbots are trained to give sycophantic responses, they see humankind at our cruelest, learning about us from the only world where every flame war leaves an eternal written footprint, while the spoken conversations of forgiveness and reconciliation fade away. Their responses do not imitate our online aggression, but are still shaped by it, even in their rigid efforts to avoid it.

It’s easy to draw the wrong conclusions from a selective slice of a society’s communications. Medieval Norse sagas made us imagine a culture of mostly Viking warriors, since poets rarely described the farming majority. Chivalric romances focused on kings and courts, and long made us see the middle ages as a world of monarchies, erasing the many medieval republics. Statistically, we’ve been led to believe ancient Romans cared deeply about their republic, but 10% of all surviving Latin was written by one man, Cicero, whose work contains 70% of all surviving Roman uses of the word republic. Training language models on only certain human writings may introduce similar distortions. AI might make us seem more quarrelsome, as we are online. It might inflate the cultural significance of political topics primarily discussed on Twitter/X or Bluesky, or the massive topic-specific corpuses of LinkedIn and Goodreads.

Some large language models are being trained on human speech from movies and television shows, but that speech is still scripted, and disproportionately highlights certain contexts over others (for example, police dramas, fueled by stories of murder, make up a quarter of prime-time television programming). We are not funny or hurtful or romantic the same way in real life as we are in sitcoms. At least one startup is offering to pay people to record their phone calls for AI-training purposes, but this remains a niche idea; anything large scale would cause massive privacy concerns.

We don’t pretend to know what the best solutions might be. But one has to imagine if there’s ingenuity to develop AI models, then surely there’s ingenuity to come up with a way to train them on informal human speech instead of us only at our most stylized, veiled and sometimes worst. By excluding the overwhelming majority of language production on the planet—people talking, fully and naturally, to each other—these models are being trained to mirror everything but us at our most authentically human.

This essay was written with Ada Palmer, and originally appeared in The Guardian.

Cybersecurity and the Gap Between Skill and Ability

2026-07-08 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/07/cybersecurity-and-the-gap-between-skill-and-ability.html

Last week, national security agencies from the Five Eyes—that’s the rich, English-language-speaking countries club—jointly released a statement warning of the increasing cyber risks of AI models: in particular, their ability to autonomously hack into systems and networks. The statement was more measured than some of the breathless headlines about it, and the advice they gave is pretty much the standard advice everyone gives—albeit with newfound urgency.

Internet risks are nothing new, and cyberattacks—both large and small—have been a significant issue since long before the current crop of generative AI models.

What’s been changing over the decades, and what AI is changing even faster, is the gap between skill and ability. For most of human history, the two terms were synonymous—but computers have decoupled them. As the gap between the two expands, humans empowered with these AI tools can do more: more writing, more research, more analysis and also more damage than ever before. These models can, with little detailed direction, autonomously hack into networks, steal data, deploy ransomware and destroy systems. And to the extent there is a solution, it’s going to involve harnessing AI for the defense.

In 1998, seven people from the hacker group L0pht testified before Congress. They told a mostly clueless Senate committee that they could take down the internet in 30 minutes. That was partly real and partly bravado, but it illustrates an important point: hacking into systems, stealing data and causing damage all required skill.

Contrast the L0pht hackers with hackers derided as “script kiddies.” They didn’t understand computers, or security. Instead, they used hacker tools written by others. Their actions required minimal skill and even less knowledge. But once those hacking tools became widespread, the number of potential attackers increased.

That number has continued to increase, as quality and availability of prewritten attack tools has grown. And it is growing dramatically with AI. Today’s AI systems—not just the frontier models, but most of them—are capable of carrying out cyberattacks automatically. They all do better in the hands of skilled attackers, but increasingly they are able to act autonomously with only minimal prompting.

The thing about people with ability but no skill is that they are often outsiders, not part of any professional community, and not bound by any rules or norms. This phenomenon is much more general than in cybersecurity. Any doctor can tell you how to untraceably poison someone, and many virus researchers know how to create a bioweapon. Any bridge engineer can tell you how to place explosives to blow a bridge up. The reason that murderous doctors and terrorist engineers are so rare is that the lengthy process of acquiring those skills also instills a moral and ethical code. If every random person has access to good poisoning advice, that puts us all in danger.

Modern AI systems are, in effect, a universal adviser to help people do harmful things. And while the current AI megacorporations are trying to build guardrails to prevent people from asking questions whose answers will enable the questioner to do harm, that’s not going to work in the long term. Smaller, cheaper, open-source models, including models that can run on people’s computers, and especially groups of models that run in concert with each other, are just as good as the frontier models from companies like OpenAI and Anthropic. And they continue to get better. These models will be passed around from person to person, like script kiddie hacker tools, and they won’t have any such guardrails.

Instructing AI models to spy on people and report any malicious prompts to the authorities fails for similar reasons. The megacorporations can do that, but the locally run open source models won’t. This could buy us a few months at best.

A third possibility is to somehow make the models themselves unable to hack into computers, create bioweapons or do anything else that might harm people or society. That won’t work, for the same reason we can’t teach doctors how to treat poisonings without also teaching them how to poison. It’s the same knowledge. It’s the same with construction and demolition. And it’s the same with cybersecurity. We want these AI models to be able to review computer code, find vulnerabilities and automatically fix them. The benefit to our collective security will be enormous. Unfortunately, the same knowledge can be used for attacks.

Where this leaves us is in a world of increased volatility. Super-powered humans with AI assistants will be able to do both wonderful and horrible things.

This brings us back to the Five Eyes statement. Everything they recommend is something security professionals have been recommending for years, if not decades. They are things talked about at that congressional hearing back in 1998, titled “Weak computer security in government: Is the public at risk?” Even the Five Eyes admitted that their security advice is not new, only more urgent.

What’s new is how fast things are changing: “The rapid pace of frontier AI development means cyber risk assumptions can become outdated in months, not years. We must act before and be prepared to adapt and withstand evolving threats.” The Five Eyes point to AI technology—not necessarily chatbots, but AI more generally—being used to strengthen every aspect of defense, to “detect vulnerabilities earlier, improve software quality, monitor unusual behavior, and respond faster to incidents—reducing both the cost and impact of incidents.”

Excellent advice from the Five Eyes security agencies. We need to do this with every risk that AI heightens, not just cybersecurity.

This essay was originally published in The Guardian.

Google Is Suing Chinese Scammers Who Are Using Gemini

2026-07-07 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/07/google-is-suing-chinese-scammers-who-are-using-gemini.html

Not sure this will have any effect, but I support the effort:

According to Google’s legal filing, Outsider Enterprise operates through Telegram. The group offers phishing-as-a-service to individuals who may not be technically savvy enough to set up fraudulent websites and text campaigns on their own. In its Telegram channels, Outsider Enterprise reportedly provided instructions on how to use Google’s Gemini AI to create websites that imitate those of Google, YouTube, and government agencies such as New York’s E-ZPass. The group offered nearly 300 scam templates.

[…]

Google worked with AT&T, Verizon, and T-Mobile to block many of these malicious text messages, and Google notes that its on-device scam detection in Google Messages probably helped reduce the number of successful phishing attempts, too. This AI-powered feature apparently stops 10 billion scam texts every month, so it’s fair to expect it caught at least some Outsider Enterprise activity.

Another article.

France to Stop Certifying Non-Quantum-Safe Encryption

2026-07-06 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/07/france-to-stop-certifying-non-quantum-safe-encryption.html

France is accelerating its transition to post-quantum encryption:

France’s cybersecurity agency ANSSI said on Tuesday it would stop certifying security products that lack quantum-resistant encryption, a move that will force government bodies and critical operators to shift away from older systems.

Samih Souissi, ANSSI’s chief of staff, said at the France Quantum conference that the agency would halt such certifications from 2027, and that businesses should be buying only quantum-safe products by 2030.

ANSSI approval is required for use in French government agencies and critical infrastructure, making the policy a de facto phase-out of older encryption.

Flock Cameras Can Surveil Cars Without License Plates

2026-07-03 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/07/flock-cameras-can-surveil-cars-without-license-plates.html

This is from a 2024 company presentation:

Officers can also tap into data showing a car’s decals, bumper stickers, back and top racks—along with temporary and unique state tags.

Flock calls it a “Vehicle Fingerprint” and it’s touted as a way for law enforcement officials to get more information “even when you don’t have full plate information,” the company’s presentation shows.

The company gives police officers the ability to search that data as well, to “build stronger cases with less information upfront.” That includes being able to locate multiple vehicles law enforcement officials believe are moving together and what Flock calls a “multi geo search.”

This kind of thing is older than AI; I wrote about it in my 2014 book Beyond Fear. Edward Snowden revealed that the NSA was using cell phone location data to track phones that were habitually near each other.

As bad as Flock is, remember that anyone with broad access to cell phone location data can do the same thing.

Cybersecurity Mission Creep in the US

2026-07-02 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/07/cybersecurity-mission-creep-in-the-us.html

Interesting paper: “Cybersecurity Mission Creep.”

Abstract: Cybersecurity is experiencing mission creep. Policymakers are casting more and more problems as issues of cybersecurity. So reframed, wildly different policy issues, from misinformation, to child social media safety laws, to antitrust regulations, to alleged journalist misconduct, to anti-sex trafficking statutes become what this Article calls “cybersecuritized.” Before this reframing, these issues present as important but not existential. But once cybersecuritization positions the issues as threats intensified by their technological nature, they gain access to the politics and law of urgency and exceptionalism and invite troubling governance responses.

Positioned as security threats, cybersecuritized issues become endowed with the apparent normative power to override countervailing considerations, oversimplifying the problem. Cybersecuritization’s oversimplification similarly risks unidimensional solutions and invites use of argumentative trump cards, like First Amendment challenges. Cybersecuritization also invites deference to purported specialists and their proposed solutions. Together, the reductive tendencies of cybersecuritization and the deference it prompts to specialists renders ultimate governance choices more opaque. And this opacity can erode public trust and political legitimacy.

This Article surfaces the phenomenon of cybersecuritization and offers a novel framework for analyzing and critiquing it. Mining cases from across criminal and civil domains, the account also demonstrates the insidiousness of cybersecuritization and the likelihood that it will continue to expand. Confronting cybersecuritization is crucial. If we continue to ignore it, we risk abdicating further responsibility for difficult choices to the trump card of cybersecurity. This Article’s analysis and critique aim to help reclaim the hard work of governance for our hands.

Papa Johns Surveillance-Based Advertising

2026-07-01 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/07/papa-johns-surveillance-based-advertising.html

Papa Johns is spying on people’s buying activities to predict when they are low on food:

The pizza chain recently tapped NBCUniversal, Instacart and the dentsu-owned media agency Carat for help reaching consumers when they’re low on groceries—and thus more likely to be swayed by a mouth-watering ad. The idea is to reach hungry consumers by “knowing what is in their fridge without being too creepy,” said Carrie Drinkwater, chief investment officer at Carat.

To achieve that goal, NBCU and Instacart created a custom audience of shoppers who regularly purchase grocery staples on Instacart, such as eggs, milk, meat and produce. Based on that data, Papa Johns can determine which days of the week certain consumers are likely to run out of groceries and serve them an ad on NBCU streaming content accordingly. The brand served custom creatives to consumers based on their food preferences—such as whether they buy meat regularly—with QR codes and calls to action such as, “Light on groceries?” or “Empty fridge?”

Back in 2012, we learned (from Target and its campaign that detects when someone is pregnant) that the trick is to hide the knowledge in other, wrong, information. So the way for Papa John’s to not be “too creepy” is to deliberately get it wrong sometimes.

But still, ugh.

The Realities of AI Video Surveillance

2026-06-30 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/06/the-realities-of-ai-video-surveillance.html

The Financial Times has a good article on how AI is changing the capabilities of video surveillance, with information from both Israel/Iran and Russia.

I wrote about this sort of thing a few years ago, how AI enables mass spying in the way that computers and networks enabled mass surveillance. The interesting development in the article is that AI allows people to ask natural language questions about video footage to AIs—and AIs can answer them.

In contrast with older tools restricted to a few dozen preset searches, these new tools allow an almost unlimited range of enquiries by enabling language-based searches on video.

That lets intelligence officers hunt through massive streams of videos using simple search terms, such as two men handing a bag to each other; a person who has changed their appearance, or has changed clothes multiple times in a day; or a vehicle that has recently been painted over, or has driven past the same spot several times in a short period.

“This is the holy grail of surveillance,” said a European official whose country uses the technology on its cities. “We are able to look for behaviour, not objects it has created a world of new possibilities.”

AWS Weekly Roundup, Agentic CX designer for Amazon Connect Customer, EC2 AMI Watermarks, Open Governance for MySQL, and more (June 29, 2026)

2026-06-29 Micah Walter

Post Syndicated from Micah Walter original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-agentic-cx-designer-for-amazon-connect-customer-ec2-ami-watermarks-open-governance-for-mysql-and-more-june-29-2026/

It has been a busy stretch on the AWS Summit circuit. At the New York City Summit, I delivered a workshop called Building AI architectures with AWS Serverless, and it was a lot of fun watching builders wire up agents and serverless services to solve real problems in a single afternoon. This week I am heading down to the Washington, DC Summit, which always puts a spotlight on innovation in the public sector. If you are going to be there, come say hello.

A question I hear a lot at these events is how teams can put AI to work without waiting on a long engineering backlog, and this week’s biggest launch speaks directly to that, with Amazon Connect Customer introducing a no-code way for business teams to design AI powered customer experiences themselves. Now, let’s get into this week’s AWS news.

Headlines

Amazon Connect Customer launched the Agentic CX designer (NLX) in preview, a no-code canvas for designing and deploying AI powered self service experiences. Business teams can build and launch voice and digital experiences that bring agentic and deterministic AI together in one governed flow, going from design to testing and simulation to production ready experiences in weeks rather than months. The launch also includes Live Sync in preview, a patented technology that drives a customer’s web or mobile experience in real time as they speak or type. A caller can complete a form or pull up the right product page without ever leaving the conversation. To see how this reshapes who designs customer experience, read the blog post on how the business user is the new architect of customer experience and visit the Amazon Connect Customer page.

Last week’s launches

Here are some launches and updates from this past week that caught my attention:

AWS Lambda MicroVMs – A new serverless compute primitive that gives each user or job VM level isolation with near instant launch and resume speeds, plus the ability to suspend and resume execution for up to 8 hours. Built on Firecracker, it is made for running user or AI generated code in multi-tenant applications without managing virtualization infrastructure or trading off isolation, speed, and state.
Amazon EC2 AMI Watermarks – Lets you embed custom identifiers in your private AMIs that automatically carry forward to every derived AMI across copies, Regions, and account shares. You can combine watermarks with Allowed AMIs and Declarative Policies to restrict launches to approved images, available at no additional cost in all AWS Regions.
AWS Outposts self-service lifecycle management – Adds self service configuration, quoting, ordering, subscription management, renewal, and decommissioning directly from the console, CLI, and API. A new quoting tool generates real time cost estimates in seconds and surfaces account and regional constraints before you submit an order.
Amazon MSK AI Agent Skills – Gives AI coding assistants like Kiro, Claude Code, and Cursor expert, up-to-date guidance for operating Amazon MSK, covering troubleshooting, sizing, configuring, monitoring, and migrating external Kafka clusters to MSK Express. Tasks that once required specialized knowledge become a guided experience developers can complete on their own.
Amazon OpenSearch Service AI-assisted migrations – Migration Assistant now includes an agent guided experience that helps you move self managed Apache Solr, Elasticsearch, or OpenSearch deployments to OpenSearch Serverless or Managed Clusters using tools like Kiro and Claude Code, with new live traffic capture and replay support for Solr.
Amazon GuardDuty AI-powered investigations (preview) – Automatically analyzes findings and accounts to help you separate true threats from benign activity, examining context and related activity from the last 90 days with knowledge graphs and threat intelligence. Each investigation returns a disposition assessment with confidence scoring, MITRE ATT&CK classification, and actionable recommendations in minutes.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news

Here are some additional posts and resources that you might find interesting:

Open Governance for MySQL – Oracle announced a community governance model for MySQL that gives organizations outside Oracle a defined role in the project, including four non Oracle seats on a new Steering Committee and a public GitHub presence. AWS holds a seat and shares why it supports the move and how it already contributes fixes upstream for everyone running MySQL.
A new way to keep your AWS Certification current -You can now maintain an eligible AWS Certification for an additional year by completing curated training and hands on labs on AWS Skill Builder instead of retaking a full exam. The option is available today in open beta for several Associate and Professional certifications, with more coming later this year.
The All Builders Welcome Grant insider’s guide for 2026 applicants – A community guide on AWS Builder Center that walks early career builders through applying for the grant, which covers a full conference pass, airfare, and hotel for AWS re:Invent 2026. Applications are open now and close on July 14.

For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.

Looking for ways to connect with builders in person? Check out the AWS Summits coming to a city near you, find a local AWS Community Day led by user groups around the world, and explore tutorials, community content, and ways to grow your skills over at the AWS Builder Center.

That’s all for this week. Check back next Monday for another Weekly Roundup!

-Micah

Factoring RSA Keys with Many Zeros

2026-06-29 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/06/factoring-rsa-keys-with-many-zeros.html

Interesting research on a new class of weak RSA keys: keys with lots of zeros. It turns out that these keys are out in the wild.

The badkeys project is an open-source service that checks public keys for known vulnerabilities. While developing this tool, Hanno collected a massive number of real-world keys from public sources, including Certificate Transparency logs, internet-wide TLS and SSH scans, PGP keys, and many others. By searching this dataset for unexpectedly sparse RSA moduli, we uncovered a large number of keys in the wild with the patterns in Figure 1.

Both patterns include several regularly spaced blocks of all zeros interleaved with seemingly random data. Pattern 1 appears in CT logs for certificates issued to several large organizations, including Yahoo and Verizon, and on some devices running NetApp software. Fortunately, these certificates have already expired, but we still shared our findings with these companies. We wanted to learn more about which product could be responsible for generating these keys, but we did not hear back. Pattern 2 appears on SSH hosts running the CompleteFTP software from EnterpriseDT. The underlying vulnerability affects RSA keys generated using versions 10.0.012.0.0 (Dec 2016Mar 2019) and DSA keys generated with v10.0.023.0.4 (Dec 2016Dec 2023).

These vulnerabilities affect a small minority of hosts on the internet, but the more interesting takeaway is that independent cryptographic implementations failed in similar ways. More implementations may include the same bugs, and so it’s worth tailoring cryptanalytic algorithms for this particular type of failure.

The article doesn’t speculate, but I will. This could be a deliberately designed backdoor, of the sort I wrote about back in 2013. I could imagine some government agency figuring out how to break this class of RSA keys, and then convincing different providers to hand them out to users.

Robot Police Officers

2026-06-29 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/06/robot-police-officers.html

We’ve taken one small step towards robot police officers: a drone capable of disarming a suspect:

In a June 22 video posted on the Sacramento County Sheriff’s Office’s Instagram page, an officer wearing goggles can be seen operating a drone to retrieve a knife from an armed suspect hiding inside a cluttered house. “After not responding to negotiators, a drone was deployed inside the residence,” the post says. “Drone pilots located the suspect hiding in a corner of a garage” and then used a high-powered magnet attached to the drone to grab the knife out of the suspect’s hand. In the video which is soundtracked by the “Mission: Impossible” theme song—the intercepted knife can be seen spinning around in the air as the drone carries it back to the deputies.

Slashdot thread.

The Chinese Control the Majority of Argentina’s Squid Fleet

2026-06-26 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/06/the-chinese-control-the-majority-of-argentinas-squid-fleet.html

Chinese companies control nearly two-thirds of Argentina’s own squid fleet.

Meta Is Testing Facial Recognition for Police and Military

2026-06-26 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/06/meta-is-testing-facial-recognition-for-police-and-military.html

We know that ICE wants to deploy eyeglasses with facial recognition that can identify people in real time.

Turns out Meta is prototyping the feature with a Pentagon supplier. (Alternate news story.)

Слухов апарат, шантав късмет… и пак войната в Украйна

2026-06-26 Григор

Post Syndicated from Григор original http://www.gatchev.info/blog/?p=2710

Абе хора, само на мен ли ми се случват все такива интересни съвпадения? Не ми се вярва. Кажете си, защо ги криете, какво толкова има в тях?!

… Срещам вчера на улицата състудент. Отскоро в клона за медицински апарати на голяма фирма. Докато говорим, той забелязва, че често се ослушвам и го карам да повтаря – слухът ми напоследък е поспаднал:

– Григоре бе, таман имаме изпитание на слухови апарати. Елитни, ще са по над трийсет хиляди евро като излязат. Не плащаме за участие, ама един ден да видиш какво могат модерните технологии! Утре е последният ден!

И днес пробвам апарата. Като тапа за уши, ама не е истина какво прави. Така не съм чувал даже преди казармата и големите оръдия. И най-тихият звук е кристално ясен. Май ще си струва трийсет хиляди евро.

В метрото срещу мен седят двама чичковци. Нисички, позакръглени, ако не е разликата в лицата, ще кажеш че са близнаци. Говорят си тихичко – ама чудото на техниката го донася до ушите ми:

– Разбра ли, бе? Руснаците са освободили Осикивка! На папер са направили мръсните укри, камък върху камък не са оставили! Както навсякъде! Така им се пада!
– Ъъъъ… това за войната в Украйна ли е? Че аз нали знаеш, от политика не разбирам…
– Ще си умреш с това счетоводство, само сметки гледаш дали излизат! Отвори си очите, да знаеш какво става в света!
– Ахаа… Викаш, руснаците освободили… как беше? То в Украйна ли е?
– Естествено! Къде очакваш да е?!
– Щото аз май нещо не разбрах. Руснаците са освободили някакво село в Украйна от украинците?
– Аха! Докато не я освободят цялата, няма да спрат! Скоро ще е!
– Ама чакай, аз пак не разбрах нещо. Руснаците освобождават Украйна от украинците?
– Да… Абе не, бе! Не от украинците! От нацистите!
– Стига бе! Нацисти в Украйна?! Че откога там нацисти?!
– Това са тия, майданаджиите! Дето изгориха триста българи в Одеса! И дето сложиха на власт оня мръсник Зеленски!
– Ахаааа!… Ама чакай, аз пак не разбрах нещо. Ти не ми ли разправяше оня ден, че Зеленски бил чифут?
– Чистокръвен чифут е! Мръсник и изедник! Хубаво ги бяха почнали едно време, ама не ги довършиха!
– Тях не ги ли бяха почнали едно време нацистите? Че тогава как Зеленски ще е нацист, нали е чифут? Нещо не ми се връзва тая сметка.
– Той е такъв. Чифут нацист! И тези около него също са нацисти! Всичките до един! Отгоре додолу!
– А армията на Украйна, тя всичката нацисти ли е? Щом воюва срещу руснаците? И… ако камък върху камък не са оставили от селото, в това село всичките нацисти ли са? И чифути? И… ако е както навсякъде, всички украинци ли са нацисти?
– Разбира се! Не им ли виждаш бежанците тук – нацист върху нацист! Избиват българите! Отвличат ни децата!
– И значи затова Русия освобождава Украйна от украинците – защото всички украинци са нацисти? Прощавай, ама нещо тук…
Физиономията на събеседника му изведнъж се променя видимо:
– Абе да не си и ти тайно нацист?! Или чифут, а?!

Тук трябваше да слизам. Изкушавах се да остана, само за да чуя как ще продължи тоя разговор. Ама не е хубаво да подслушва човек.

One Million Passports Leaked Online

2026-06-26 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/06/one-million-passports-leaked-online.html

A database of almost a million passports from around the world was leaked online.

Note what happened. A high-value credential—a passport—was used in an ancillary low-value authentication system: ID verification for cannabis dispensaries. And it’s the low-value system that got hacked, putting the high-value credential at risk.

AI and Liability

2026-06-25 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/06/ai-and-liability.html

Earlier this month, a German court ruled that Google is liable for its AI search summaries. Rejecting defenses like “users can check for themselves,” and that they generally know “that information generated with AI should not be blindly trusted,” the court held that the AI’s summaries are reflections of the company and “above all an expression of Google’s business activities.”

This is the latest skirmish in a decades-old battle over internet publishing. Historically, there were two different types of information distributors: carriers and publishers. A phone company is a carrier. It’ll transmit whatever you say, even discussions about committing a crime. Words are words, and the phone company does not know—nor is it liable for—the words you choose to speak. A newspaper, on the other hand, is a publisher. It decides the words it publishes, and what quotes to include in its articles. If those words or quotes are defamatory or otherwise illegal, it’s liable.

Internet companies have long tried to play both ends of this distinction. They claim to be a carrier when it suits them, and also to be a publisher when that is advantageous. Section 230 of the 1996 Communication Decency Act enshrined this straddling when it shielded internet providers from liability for the speech of others on their platforms: “No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.”

For years, a debate has continued about how to apply this law to social media platforms. When platforms merely displayed people’s posts and comments in reverse-chronological order, they behaved largely like carriers, relaying people’s words without regard to their contents. But the next generation of platforms, like Facebook, curated feeds with algorithms and thereby acted more like publishers, making editorial decisions about who sees what. Some experts think section 230 has gone too far and needs reform; others think that it’s what holds the modern internet together.

Google’s AI overviews are far less nuanced. They work differently from traditional search, which courts have held involves archiving and facilitating access to the editorial content of third parties. AI overviews don’t just quote and republish words from different websites. With overviews, the AI rewrites other people’s words, exercising editorial discretion like a newspaper article or an original essay on a topic.

It’s not only Google’s AI that falls into this category. Imagine a restaurant review site that provides AI summaries, or a site summarizing laws and government procedures. Or a traditional publisher that uses AI to summarize its own publication. Accuracy matters, and liability is one of the most important ways we as a public can demand accuracy and hold companies accountable when they cause harm.

Two years ago, Air Canada learned this lesson. Its AI chatbot promised a discount the company later rescinded, arguing in court that the airline wasn’t responsible for the promises the bot made because it was a “separate legal entity that is responsible for its own actions.” The court sided with the flyer, saying that the airline was just as responsible for what its chatbot says as what’s on its website. The potential precedent here is that corporations have a duty of care for the performance of the AI chatbots they employ.

AI agents are agents of the person or organization that deploys them—and should be treated by the law as such. If a company hired human writers to write its summaries, that company would be liable for inaccuracies in those summaries. If a company’s human agent signed contracts in the company’s name, that company would be bound by those contracts. And if a doctor gave dangerously wrong medical advice, they would be liable for malpractice.

To allow businesses to hide behind the excuse of faulty AI in those same circumstances would be a massive handout to companies, and would introduce disastrous incentives for corporate misbehavior. Why hire human writers, lawyers or doctors when AIs are not only cheaper, but also absolve employers whenever they make a mistake?

We are rapidly moving to a world where AI-powered chatbots will be at the other end of all sorts of corporate communications channels. It makes no sense for a company to be able to honor its statements when it wants to and disavow them when it doesn’t.

Visa and OpenAI recently announced a partnership to build personal AI agents to, among other things, make purchases on our behalf. This is just one of many similar projects in the works, as companies race to provide us all with AI assistants. Will Visa take responsibility when its AI makes a purchase in your name that you don’t want? And if Visa won’t, why would anyone trust the system? Properly allocating liability is key to make this kind of thing work.

If the German ruling holds, it could be devastating for Google’s AI Overview feature. Tests from earlier this year found that it had mistakes about 10% percent of the time. At more than 5tn searches per year, that’s 16,000 erroneous summaries every second. And while most of those errors are benign, some of them will cause harm, be defamatory, or otherwise trigger liability.

Earlier this year, Google’s AI summary falsely identified the Canadian fiddler Ashley MacIsaac of being a sex offender. His lawsuit, filed in Ontario, is ongoing. If Google is forced to invest in improving its AI system until those kinds of errors are exceedingly rare, that seems like a good outcome for users, as well as the subjects of search, like MacIsaac.

More generally, liability concerns could mean that many current use cases for agents won’t be commercially viable. Companies may not be able to profitably operate AI lawyers, doctors and media influencers if they are held responsible for what they say and do.

We’re OK with this outcome. There’s nothing in the law that requires us to accommodate AI systems if they are fundamentally untrustworthy, just as we don’t need to accommodate untrustworthy human systems. Any company that won’t stand by the statements its agents make—whether human or AI—doesn’t deserve users’ time or money.

Maximize Amazon EC2 Capacity Reservations with Capacity Manager data exports

2026-06-25 Venu Geddam

Post Syndicated from Venu Geddam original https://aws.amazon.com/blogs/compute/maximize-amazon-ec2-capacity-reservations-with-capacity-manager-data-exports/

In our previous post, we introduced Amazon EC2 Capacity Manager and its data export capability. Amazon EC2 Capacity Manager provides centralized visibility into your Amazon Elastic Compute Cloud (Amazon EC2) capacity usage across all accounts and Regions in your organization. It tracks capacity usage for three types of EC2 capacity: On-Demand instances, Spot instances, and On-Demand Capacity Reservations (ODCR). On the AWS Management Console, it provides 90 days of historical capacity data. With data exports to Amazon Simple Storage Service (Amazon S3), you can retain and analyze capacity trends beyond this period using your preferred analytics tools.

In this post, we demonstrate how to configure EC2 Capacity Manager data exports to Amazon S3 and query historical capacity data using Amazon Athena. This approach helps you identify long-term usage patterns, plan capacity needs, and optimize resource allocation across your organization.

Solution overview

The following diagram illustrates the solution architecture. EC2 Capacity Manager exports capacity data to Amazon S3, where Amazon Athena queries it using SQL with automatic partition discovery.

Architecture diagram showing EC2 Capacity Manager exporting capacity data to Amazon S3 on a scheduled basis, and Amazon Athena querying the data directly from S3 using SQL with partition projection for automatic partition discovery

The solution involves the following steps:

Set up an S3 bucket for capacity data export.
Configure EC2 Capacity Manager data export.
Set up Amazon Athena to query the exported data.
Run queries to analyze capacity patterns.

Prerequisites:

An AWS account with permissions to create S3 buckets and configure EC2 Capacity Manager.
AWS Command Line Interface (AWS CLI) installed and configured (optional, for CLI-based setup).
Familiarity with SQL for querying data in Athena.

Setting up data export to Amazon S3

EC2 Capacity Manager can export capacity data in compressed CSV (Gzip) or compressed Parquet (Snappy) format. Use Parquet format for query performance in Athena (Parquet’s columnar format is designed to optimize analytical queries).

Configure the data export

You can configure data export through the EC2 Capacity Manager console or AWS CLI.

To configure data export using the console:

Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
In the navigation pane, choose Capacity Manager.
Choose the Data exports tab.
Choose Create data export.
For Output format, select Parquet.
For S3 bucket, choose Create bucket for me to create a new bucket with the required permissions, or select an existing bucket from the list.
If you selected an existing bucket, add the bucket policy shown in the following section to grant EC2 Capacity Manager write access.
(Optional) For S3 prefix, enter a prefix to organize your exported files (for example, capacity-data/).
For Schedule, select Hourly.
Choose Create.

Screenshot of EC2 Capacity Manager Create data export page in AWS Console showing export properties: Data export name, output format set to Parquet, S3 location for export delivery and Tags

After you create the data export, EC2 Capacity Manager displays the export details.

Screenshot showing EC2 Capacity Manager data export status with ‘Latest Delivery’ column displaying ‘delivered’ status, indicating the export is ready for Athena setup

Update the S3 bucket policy (existing buckets only)

If you use an existing S3 bucket, you must add the bucket policy shown in the following example to grant EC2 Capacity Manager write access.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "ec2.capacitymanager.amazonaws.com"
            },
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::<BUCKET_NAME>",
                "arn:aws:s3:::<BUCKET_NAME>/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "<AWS_ACCOUNT_NUMBER>"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:ec2:<AWS_REGION>:<AWS_ACCOUNT_NUMBER>:capacity-manager-data-export/*"
                }
            }
        }
    ]
}

Replace <AWS_ACCOUNT_NUMBER> with your AWS account number, <AWS_REGION> with your AWS Region (for example, us-west-2), and <BUCKET_NAME> with your bucket name.

To configure data export using the AWS CLI :

aws ec2 create-capacity-manager-data-export \
    --s3-bucket-name <BUCKET_NAME> \
    --s3-bucket-prefix <BUCKET_PREFIX>/ \
    --schedule hourly \
    --output-format <FORMAT> \
    --region <AWS_REGION>

# Replace:
# <BUCKET_NAME> with your bucket name,
# <BUCKET_PREFIX> with your bucket prefix (for example, capacity-data/),
# <FORMAT> with parquet or csv, and
# <AWS_REGION> with your AWS Region (for example, us-west-2)

# The output of the above command would give you a Data Export ID

{ "CapacityManagerDataExportId": "cmde-00a7d0e64e43889f1" }

After creating the data export, wait for the first export to complete before proceeding to set up Athena. You can check the export status using the following command:

aws ec2 describe-capacity-manager-data-exports --region <AWS_REGION>

The command returns details about your data export configuration, including the delivery status:

{
    "CapacityManagerDataExports": [
        {
            "CapacityManagerDataExportId": "cmde-00a7d0e64e43889f1",
            "S3BucketName": "capacity-manager-exports-123456789012",
            "S3BucketPrefix": "capacity-data/",
            "Schedule": "hourly",
            "OutputFormat": "parquet",
            "CreateTime": "2026-04-10T19:04:36.824000+00:00",
            "LatestDeliveryStatus": "delivered",
            "LatestDeliveryStatusMessage": "Successfully delivered to s3://capacity-manager-exports-123456789012/y=2026/m=04/d=10/h=15/...",
            "LatestDeliveryTime": "2026-04-10T19:05:39.176000+00:00"
        }
    ]
}

Wait until LatestDeliveryStatus shows "delivered" before proceeding to the next section. The first export typically appears in your S3 bucket in a couple of hours. Subsequent exports follow your configured schedule.

Setting up Amazon Athena to query capacity data

After EC2 Capacity Manager exports data to your S3 bucket, you can use Amazon Athena to query the data using standard SQL. Athena uses AWS Glue as its metadata store. Specifically, it relies on the AWS Glue Data Catalog, which contains table definitions that tell Athena where you have stored your data in S3 and how you have structured it. When you create tables in Athena, you’re actually creating metadata entries in the Data Catalog that Athena references when running queries.

Create an Athena database and table

You can create the table using AWS Glue crawler or manually with SQL. AWS Glue crawler automatically discovers the complete schema from your exported Parquet files, including optional fields like resource tags if enabled. It helps minimize manual schema definition efforts. If the export format changes in the future, you can re-run the crawler to update the table definition. For detailed instructions on creating a Glue Crawler, see Use a crawler to add a table in the Amazon Athena User Guide.

In this post, we create the table manually using a SQL statement. We also use partition projection for automatic partition discovery. We do this because EC2 Capacity Manager continuously adds new partitions to the S3 bucket according to your configured schedule. As new partitions arrive in S3, Athena doesn’t know about them until you run MSCK REPAIR TABLE or ALTER TABLE ADD PARTITION to update the AWS Glue Data Catalog. This becomes an overhead when data arrives frequently.

With partition projection, you define the partition scheme and specify its range/rules in the table properties. Athena then computes the partitions at query time instead of looking them up in the Glue Data Catalog. So partition projection automatically makes new partitions visible as soon as EC2 Capacity Manager exports the data to S3, eliminating the need for you to update metadata. The CREATE TABLE statement that follows defines the schema for EC2 Capacity Manager exports. If your capacity reservations are already tagged, add the corresponding tag columns (for example, tag_environment string or tag_costcenter string). Alternatively, use an AWS Glue crawler to automatically discover your complete schema, including tag columns.

Open the Athena console at https://console.aws.amazon.com/athena/
If prompted, configure a query result location in S3. This is where Athena writes query output. It is separate from the S3 bucket that stores your capacity data.
Run the following query to create a database:

CREATE DATABASE IF NOT EXISTS capacity_manager_db;

Create a table for Parquet format data:

CREATE EXTERNAL TABLE IF NOT EXISTS capacity_manager_db.capacity_data (
    metricgroupname string,
    periodstarttimestamp string,
    periodendtimestamp string,
    orgid string,
    accountid string,
    region string,
    `az-id` string,
    instancefamily string,
    instancetype string,
    platform string,
    tenancy string,
    reservationid string,
    `reservation arn` string,
    unusedfinancialowner string,
    reservationtype string,
    instancematchcriteria string,
    reservationcreatetimestamp string,
    reservationstarttimestamp string,
    reservationendtimestamp string,
    reservationenddatetype string,
    reservationstate string,
    reservationtotalcapacityhrsvcpu string,
    reservationtotalcapacityhrsinst string,
    reservationtotalestimatedcost string,
    reservationmaxsizevcpu string,
    reservationmaxsizeinst string,
    reservationminsizevcpu string,
    reservationminsizeinst string,
    reservationunusedtotalcapacityhrsvcpu string,
    reservationunusedtotalcapacityhrsinst string,
    reservationunusedtotalestimatedcost string,
    reservationmaxunusedsizevcpu string,
    reservationmaxunusedsizeinst string,
    reservationminunusedsizevcpu string,
    reservationminunusedsizeinst string,
    reservationmaxutilization string,
    reservationminutilization string,
    reservationavgutilizationvcpu string,
    reservationavgutilizationinst string,
    reservationavgfuturesizevcpu string,
    reservationavgfuturesizeinst string,
    reservationmaxfuturesizevcpu string,
    reservationmaxfuturesizeinst string,
    reservationminfuturesizevcpu string,
    reservationminfuturesizeinst string,
    reservationavgcommittedsizevcpu string,
    reservationavgcommittedsizeinst string,
    reservationmaxcommittedsizevcpu string,
    reservationmaxcommittedsizeinst string,
    reservationmincommittedsizevcpu string,
    reservationmincommittedsizeinst string,
    reservedtotalusagehrsvcpu string,
    reservedtotalusagehrsinst string,
    unreservedtotalusagehrsvcpu string,
    unreservedtotalusagehrsinst string,
    reservedtotalestimatedcost string,
    unreservedtotalestimatedcost string,
    spottotalusagehrsvcpu string,
    spottotalusagehrsinst string,
    spottotalestimatedcost string,
    spotavgruntimebeforeinterruptioninst string,
    spotmaxruntimebeforeinterruptioninst string,
    spotminruntimebeforeinterruptioninst string,
    spottotalinterruptionsinst string,
    spottotalinterruptionsvcpu string,
    spottotalcountinst string,
    spottotalcountvcpu string,
    spotinterruptionrateinst string,
    spotinterruptionratevcpu string
)
PARTITIONED BY (
    y string,
    m string,
    d string,
    h string
)
ROW FORMAT SERDE
    'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT
    'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
OUTPUTFORMAT
    'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
    's3://<BUCKET_NAME>/'
TBLPROPERTIES (
    'projection.enabled' = 'true',
    'projection.y.type' = 'integer',
    'projection.y.range' = '2024,2030',
    'projection.y.digits' = '4',
    'projection.m.type' = 'integer',
    'projection.m.range' = '01,12',
    'projection.m.digits' = '2',
    'projection.d.type' = 'integer',
    'projection.d.range' = '01,31',
    'projection.d.digits' = '2',
    'projection.h.type' = 'integer',
    'projection.h.range' = '00,23',
    'projection.h.digits' = '2',
    'storage.location.template' = 's3://<BUCKET_NAME>/y=${y}/m=${m}/d=${d}/h=${h}/'
);

Replace <BUCKET_NAME> in the LOCATION clause and storage.location.template property with your bucket name, and capacity-data/ with table name of your choice.

Now that your table is set up, you can explore how to query the exported data.

Example queries for common use cases

The following queries demonstrate how to analyze your EC2 capacity data for cost optimization and capacity planning. The queries use date values (year=‘2026’, month=‘04’) and table name capacity_data. Adjust the partition values to match your actual data’s time period and table name to match your value. When querying EC2 Capacity Manager export data:

Metric group filtering: EC2 Capacity Manager exports contain two types of data that the metricgroupname column identifies: Reservation Usage for analyzing ODCR utilization and optimization opportunities, and Instance Usage for analyzing overall capacity consumption across reserved, unreserved, and Spot instances. Always filter by the appropriate metric group for your analysis needs.
Partition filtering: Always include partition filters (y, m, d, h) to improve query performance.
Numeric operations: Use CAST to convert string columns to numeric types for proper comparison and sorting (for example, CAST(reservationavgutilizationinst AS double)).
NULL handling: Use COALESCE to handle NULL values in calculations (for example, COALESCE(CAST(column AS double), 0)) to prevent NULL results in totals. Without COALESCE, when you add a column value NULL to a non-NULL value, the result is NULL.

Use case 1: Identify underutilized ODCRs

Discover On-Demand Capacity Reservations with low utilization that are generating cost waste. Identify specific reservations to downsize, cancel, or share with other teams to reduce unnecessary spending.

SELECT
    reservationid,
    instancetype,
    region,
    "az-id",
    reservationstate,
    ROUND(CAST(reservationavgutilizationinst AS double) * 100, 2) AS utilization_pct,
    CAST(reservationtotalcapacityhrsinst AS double) AS total_capacity_hrs,
    CAST(reservationunusedtotalcapacityhrsinst AS double) AS unused_capacity_hrs,
    ROUND(CAST(reservationunusedtotalestimatedcost AS double), 4) AS wasted_cost_usd
FROM capacity_data
WHERE metricgroupname = 'Reservation Usage'
    AND CAST(reservationavgutilizationinst AS double) < 0.5
    AND reservationavgutilizationinst IS NOT NULL
    AND y = '2026'
    AND m = '04'
ORDER BY wasted_cost_usd DESC
LIMIT 3;

Sample output:

reservationid	instancetype	region	az-id	reservationstate	utilization_pct	total_capacity_hrs	unused_capacity_hrs	wasted_cost_usd
cr-041aedfba865106c1	m5.8xlarge	us-west-2	usw2-az2	active	0	1	1	1.536
cr-089f17dc178e993a8	m5.8xlarge	us-west-2	usw2-az1	active	0	1	1	1.536
cr-089f17dc178e993a8	m5.8xlarge	us-west-2	usw2-az1	active	0	1	1	1.536

Use case 2: ODCR utilization summary by instance type

Get a comprehensive view of ODCR utilization across instance types to identify which instance families have the worst utilization rates. This helps prioritize optimization efforts on the reservations with the highest cost impact.

SELECT
    instancetype,
    COUNT(DISTINCT accountid) AS account_count,
    COUNT(DISTINCT reservationid) AS reservation_count,
    ROUND(SUM(COALESCE(CAST(reservationtotalcapacityhrsinst AS double), 0)), 2) AS total_odcr_capacity,
    ROUND(SUM(COALESCE(CAST(reservationunusedtotalcapacityhrsinst AS double), 0)), 2) AS total_unused_capacity,
    ROUND(AVG(COALESCE(CAST(reservationavgutilizationinst AS double), 0)) * 100, 2) AS avg_utilization_pct,
    ROUND(SUM(COALESCE(CAST(reservationunusedtotalestimatedcost AS double), 0)), 2) AS total_unused_cost_usd
FROM capacity_data
WHERE metricgroupname = 'Reservation Usage'
    AND y = '2026'
    AND m = '04'
GROUP BY instancetype
ORDER BY total_unused_cost_usd DESC
LIMIT 3;

Sample output:

instancetype	account_count	reservation_count	total_odcr_capacity	total_unused_capacity	avg_utilization_pct	total_unused_cost_usd
m5.8xlarge	1	2	311.99	311.99	0.0	479.22
c5.9xlarge	1	1	211.0	211.0	0.0	322.83
t3.micro	1	1	1055.0	1055.0	0.0	10.97

Use case 3: Identify peak usage patterns

Analyze average hourly usage patterns across reserved, unreserved, and Spot capacity to identify when your workloads typically hit peak demand. This breakdown helps you understand your capacity mix, plan for peak periods, and optimize your purchasing strategy.

SELECT
    h AS hour,
    ROUND(AVG(COALESCE(CAST(reservedtotalusagehrsinst AS double), 0)), 2) AS avg_reserved_usage_hours,
    ROUND(AVG(COALESCE(CAST(unreservedtotalusagehrsinst AS double), 0)), 2) AS avg_unreserved_usage_hours,
    ROUND(AVG(COALESCE(CAST(spottotalusagehrsinst AS double), 0)), 2) AS avg_spot_usage_hours,
    ROUND(AVG(COALESCE(CAST(reservedtotalusagehrsinst AS double), 0) + COALESCE(CAST(unreservedtotalusagehrsinst AS double), 0) + COALESCE(CAST(spottotalusagehrsinst AS double), 0)), 2) AS avg_total_usage_hours
FROM capacity_data
WHERE metricgroupname = 'Instance Usage'
    AND y = '2026'
    AND m = '04'
GROUP BY h
ORDER BY avg_total_usage_hours DESC
LIMIT 3;

Sample output:

hour	avg_reserved_usage_hours	avg_unreserved_usage_hours	avg_spot_usage_hours	avg_total_usage_hours
09	0.75	0.5	0	1.25
10	0.75	0.5	0	1.25
15	0.75	0.5	0	1.25

Use case 4: Regional capacity distribution

Understand how your ODCR capacity is distributed across AWS Regions and instance types. This geographic view helps you identify Regions with excess capacity that could be redistributed or consolidated to improve utilization and reduce costs.

SELECT
    region,
    instancetype,
    ROUND(SUM(COALESCE(CAST(reservationtotalcapacityhrsinst AS double), 0)), 2) AS total_reserved_capacity,
    ROUND(SUM(COALESCE(CAST(reservationunusedtotalcapacityhrsinst AS double), 0)), 2) AS unused_reserved_capacity,
    ROUND(AVG(COALESCE(CAST(reservationavgutilizationinst AS double), 0)) * 100, 2) AS avg_utilization_pct
FROM capacity_data
WHERE metricgroupname = 'Reservation Usage'
    AND y = '2026'
    AND m = '04'
GROUP BY region, instancetype
ORDER BY region, total_reserved_capacity DESC
LIMIT 3;

Sample output:

region	instancetype	total_reserved_capacity	unused_reserved_capacity	avg_utilization_pct
us-west-2	t2.nano	1484.0	1060.0	33.34
us-west-2	t3.micro	1060.0	1060.0	0.0
us-west-2	t2.micro	848.0	636.0	25.0

Use case 5: Unused capacity reservations by Region and Availability Zone

Pinpoint exactly where you have unused ODCR capacity at the Availability Zone level. This granular view enables you to share unused capacity with other teams in the same AZ or modify reservations to better match actual usage patterns.

SELECT
    region,
    "az-id",
    instancetype,
    ROUND(SUM(COALESCE(CAST(reservationunusedtotalcapacityhrsinst AS double), 0)), 2) AS unused_capacity_instances,
    ROUND(AVG(COALESCE(CAST(reservationavgutilizationinst AS double), 0)) * 100, 2) AS avg_utilization_pct,
    ROUND(SUM(COALESCE(CAST(reservationunusedtotalestimatedcost AS double), 0)), 2) AS unused_cost_usd
FROM capacity_data
WHERE metricgroupname = 'Reservation Usage'
    AND y = '2026'
    AND m = '04'
    AND CAST(reservationunusedtotalcapacityhrsinst AS double) > 0
GROUP BY region, "az-id", instancetype
ORDER BY unused_cost_usd DESC
LIMIT 3;

Sample output:

region	az-id	instancetype	unused_capacity_instances	avg_utilization_pct	unused_cost_usd
us-west-2	usw2-az1	c5.9xlarge	212.0	0.0	324.36
us-west-2	usw2-az2	m5.8xlarge	157.0	0.0	241.15
us-west-2	usw2-az1	m5.8xlarge	157.0	0.0	241.15

Clean up

To avoid incurring future charges, delete the resources you created:

Warning: This permanently deletes the table definition. Verify that you no longer need to query this data before proceeding.

Delete the Athena table by running the following query: DROP TABLE IF EXISTS capacity_manager_db.capacity_data;
Delete the database by running the following query: DROP DATABASE IF EXISTS capacity_manager_db;
Navigate to the Athena console settings.
Note the query result location S3 bucket.
If you created this bucket specifically for this tutorial:
1. Empty the S3 bucket by running: aws s3 rm s3://<QUERY_RESULT_BUCKET_NAME> --recursive
2. Delete the S3 bucket by running: aws s3 rb s3://<QUERY_RESULT_BUCKET_NAME>
Delete the data export configuration by using AWS Management Console or AWS CLI.
1. If using AWS Console, select the “Delete data export” option in the Actions menu.

Screenshot of AWS Management Console showing the Delete option in the Actions menu for an EC2 Capacity Manager data export configuration

To delete the configuration using AWS CLI:

# List your data export configurations to find the export ID.
aws ec2 describe-capacity-manager-data-exports --region <AWS_REGION>

# The output shows details for export configuration which has the Data Export ID
# Sample output is shown in the configure data export section above.

# Then delete the export configuration using the ID from the output
aws ec2 delete-capacity-manager-data-export \
    --data-export-id <EXPORT_ID> \
    --region <AWS_REGION>

Replace <EXPORT_ID> with your data export ID and <AWS_REGION> with your AWS Region.

Warning: Deleting the S3 bucket permanently removes all exported capacity data. Verify that you have backed up any data you need before proceeding.

(Optional) Delete the S3 bucket and data: If you no longer need the exported data, complete the following steps:
1. Empty the S3 bucket by running: aws s3 rm s3://<BUCKET_NAME> --recursive
2. Delete the S3 bucket by running: aws s3 rb s3://<BUCKET_NAME>

Conclusion

In this post, we demonstrated how to configure EC2 Capacity Manager data exports to Amazon S3 and query historical capacity data using Amazon Athena. This approach enables you to retain capacity data beyond the 90-day console limit.

As you scale your capacity management practices, consider integrating these exports with your existing analytics and monitoring workflows. By combining EC2 Capacity Manager data with your broader infrastructure metrics, you can make data-driven decisions about capacity allocation and optimization across your organization.

To deepen your understanding, explore the EC2 Capacity Manager documentation for additional features, learn more about Amazon Athena for advanced query capabilities, and review EC2 capacity optimization best practices. Share your feedback and tell us how you’re using EC2 Capacity Manager data exports to optimize your capacity planning in the comments.

Interesting Paper Exploring Prompt Injection

2026-06-25 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/06/interesting-paper-exploring-prompt-injection.html

This is a fascinating explotation of how LLMs fall for prompt injection attacks. It turns out that they learn to recognize the style of text in different role/instruction blocks, and not just the tags.

Their conclusion:

Role tags were a formatting trick that became the security architecture and the cognitive scaffolding of modern LLMs. We’ve shown that this architecture doesn’t survive into the model’s actual representations, and that such role confusion is linked to prompt injection.

Unless LLMs achieve genuine role perception, we think injection defense will remain a perpetual whack-a-mole game. And the continuous nature of role boundaries opens the threat of injections designed to subtly shift LLM states through seemingly innocuous text, legally and at scale.

More generally, roles are quietly one of the most important abstractions in the LLM stack, providing the boundaries meant to separate self from other, thought from communication, instruction from data. They’re human-controlled switches in an otherwise continuous system. We think they deserve a lot more study than they’ve gotten.

Full paper: “Prompt Injection as Role Confusion.” Simon Willison comments.

Celebrating over 15,000 young creators at the Coolest Projects 2026 online showcase

2026-06-25 Helen Gardner

Post Syndicated from Helen Gardner original https://www.raspberrypi.org/blog/celebrating-over-15000-young-creators-at-the-coolest-projects-2026-online-showcase/

From first-time coders to seasoned makers, this year every single Coolest Projects creator brought all their creativity to their tech projects and made something to be proud of. Over 15,000 young people showcased more than 4,500 creations on a global stage. With participants from 40 countries and 47% girls, this year’s online showcase is a true reflection of what the next generation of tech creators looks like.

Yesterday our global livestream brought together the whole Coolest Projects community to celebrate the young people’s creations with some very special guests.

Meet our 2026 VIP judges and their favourite projects

Every year, we invite new special VIP judges to choose their favourite projects from each of the seven Coolest Projects categories. Meet our 2026 judges and find out about the projects they picked.

Ronit Levavi Morad, Chief of Staff at Google Research

Ronit’s role involves reimagining the future of learning and she is a passionate advocate for technology in service of pedagogy, leading Google Research’s global Al literacy initiatives. She champions a human-centered vision for technology, leading programmes like Al Quests, a gamified experience designed to teach teens how Al can be applied to humanity’s greatest challenges.

Ronit’s favourite projects are:

Scratch: FishyFighter by Julian from Ireland
Web: Ulearn by Zoe from Japan
Games: Find the Bug by Jibraan from South Africa
Mobile apps: LumiCycle- Erasing Period Stigma for Younger Generations made by perTech by Rianna, Sana, Xintong, and Melania from Japan
Hardware: DIY Raspberry Pi Air Quality Monitor: Helping Leonidas Breathe Better by Katerina and Leonidas from Greece
Advanced programming: Curva by Elaina from the USA
AI: Ornimetrics – A Raspberry Pi Smart Bird Feeder Powered by AI Vision by Baichen from the USA

Ben Powley, Senior Developer at Jagex

Ben is a senior developer at British video game company Jagex, working on RuneScape to deliver immersive experiences for both new and existing players. He previously worked at Ubisoft on the Assassin’s Creed series and the Avatar Frontiers of Pandora game. Ben learned to code at 12 through making mods for Minecraft, and he has been passionate about game development ever since.

Ben’s favourite projects are:

Scratch: Cool little game by Rumyana from Bulgaria
Web: ECOSORT AI by Dyllan and Francis from Kenya
Games: Sub Racers by Haseeb from the UK
Mobile apps: HopeLink by Daniel from Canada
Hardware: Binardo Binstein by Shlok Bharadwaj and Anhad Mahajan from India
Advanced programming: Redrose Linux by Juraj from Slovakia
AI: MultiRisk by Chiara and Sofia from Argentina

Akari Kawaguchi, youth mentor, CoderDojo Japan

Akari is a 16-year-old CoderDojo member from Japan who has showcased her own projects at Coolest Projects seven times. She knows exactly what it takes to make something special for the showcase, and we’re so excited to have a young creator’s perspective on the judging panel!

Akari’s favourite projects are:

Scratch: Glitch by Daniel from Armenia
Web: Dot & Box Game by Shrija from India
Games: VR Immersive Buddhist Debate: Digital Inheritance of Tibetan Buddhist Culture by Yuhan from China
Mobile apps: SyncFit by Riho, Sara, Kaho, Ran, and Yuiko from Japan
Hardware: AquaGuard by Kyle from the USA
Advanced programming: TGSW crop disease detection system by Dasari Vaishnavi, Kondrapally Jhansi, and Alimineti Kalpana from India
AI: TideGuard AI by Vladimir from the Russian Federation

Sebin Sunny, CEO, EIC IIITM-K

Sebin is CEO of the Entrepreneurship and Innovation Center (EIC) and the Centre of Excellence in loT Sensors at the Indian Institute of Information Technology and Management – Kerala (IIITM-K). A passionate advocate for innovation, he is committed to empowering entrepreneurs, creating sustainable solutions, and shaping the future of technology-driven industries.

Sebin’s favourite projects are:

Scratch: Click and Win by Sarthak from India
Web: coolsongs by Moksh from India
Games: red line green line (version of snakes and ladders) by Moksh from India
Mobile apps: Accomplit by Audrey and Maggie from the USA
Hardware: Project Code Cloud by Tiarna and Chloe from Australia
Advanced programming: Visture by Lionel from Indonesia
AI: Project Albatross: Bio-Inspired Reactive Control Strategies for Atmospheric Energy Harvesting in Fixed-Wing UAVs by Akshara from the USA

Broadcom Coding with Commitment® award

The Broadcom Coding With Commitment® award shines a light on creators who use coding to support and strengthen their communities with a project that aligns with 17 sustainable development goals of the United Nations.

A screenshot of a young person's Scratch project showcased at Coolest Projects. — The start screen of Viridia – Back to the World

This year’s Broadcom Coding with Commitment® recipient for the online showcase is Egehan from Türkiye, with their Scratch project Viridia – Back to the World, an educational game designed to teach players about the importance of water and how to use it responsibly.

Get inspired and keep creating!

Browse the Coolest Projects 2026 online gallery to discover thousands of incredible projects from young people all over the world.

Inspired to make your own project? Or encourage a young person you know? To get you started, we offer over 200 free coding projects, in English and many other languages.

Want to know more about next year’s showcase?

Coolest Projects will be back online in 2027. Sign up to the newsletter to be the first to hear about dates, deadlines, and exciting updates.

And did you know there are in-person Coolest Projects events around the globe? There is still time to take part in Coolest Projects India and other partner events this year. Find out more.

Thank you to the Coolest Projects sponsors

We want to say a big thank you to Broadcom Foundation, Allianz, Amazon Future Engineer, Qube-RT, Avnet, and GoTo for sponsoring Coolest Projects 2026 and helping to celebrate young tech creators around the world.

The post Celebrating over 15,000 young creators at the Coolest Projects 2026 online showcase appeared first on Raspberry Pi Foundation.

Solution overview

Benefits

Core components

Technical implementation

Interpreting the workflow specification

Securing sensitive data flows

Recognizing the pattern

Conclusion

Next steps

About the author

Solution overview

Setting up data export to Amazon S3

Configure the data export

Update the S3 bucket policy (existing buckets only)

Setting up Amazon Athena to query capacity data

Create an Athena database and table

Example queries for common use cases

Use case 1: Identify underutilized ODCRs

Use case 2: ODCR utilization summary by instance type

Use case 3: Identify peak usage patterns

Use case 4: Regional capacity distribution

Use case 5: Unused capacity reservations by Region and Availability Zone

Clean up

Conclusion

Meet our 2026 VIP judges and their favourite projects

Ronit Levavi Morad, Chief of Staff at Google Research

Ben Powley, Senior Developer at Jagex

Akari Kawaguchi, youth mentor, CoderDojo Japan

Sebin Sunny, CEO, EIC IIITM-K

Broadcom Coding with Commitment® award

Get inspired and keep creating!

Want to know more about next year’s showcase?

Thank you to the Coolest Projects sponsors

The collective thoughts of the interwebz