Version
6.6.0 of KDE’s Plasma desktop environment has been
released. Notable additions in this release include the ability to
create global themes for Plasma, an “extract text” feature in the Spectacle screenshot
utility, accessibility improvements, and a new on-screen keyboard. See
the changelog
for a full list of new features, enhancements, and bug fixes.
The release is dedicated to the memory of Björn Balazs, a KDE
contributor who passed away in September 2025. “Björn’s drive to
help people achieve the privacy and control over technology that he
believed they deserved is the stuff FLOSS legends are made of.“
In December 2025, Canonical announced a plan to
develop a universal Public Key Infrastructure called upki. Jon Seager has published
an update about the project with instructions on trying it
out.
In the few weeks since we announced upki, the core revocation engine
has been established and is now functional, the CRLite mirroring tool
is working and a production deployment in Canonical’s datacentres is
ongoing. We’re now preparing for an alpha release and remain on track
for an opt-in preview for Ubuntu 26.04 LTS.
Abstract: Scaling up language models has significantly increased their capabilities. But larger models are slower models, and so there is now an extensive body of work (e.g., speculative sampling or parallel decoding) that improves the (average case) efficiency of language model generation. But these techniques introduce data-dependent timing characteristics. We show it is possible to exploit these timing differences to mount a timing attack. By monitoring the (encrypted) network traffic between a victim user and a remote language model, we can learn information about the content of messages by noting when responses are faster or slower. With complete black-box access, on open source systems we show how it is possible to learn the topic of a user’s conversation (e.g., medical advice vs. coding assistance) with 90%+ precision, and on production systems like OpenAI’s ChatGPT and Anthropic’s Claude we can distinguish between specific messages or infer the user’s language. We further show that an active adversary can leverage a boosting attack to recover PII placed in messages (e.g., phone numbers or credit card numbers) for open source systems. We conclude with potential defenses and directions for future work.
Abstract: Deployed large language models (LLMs) often rely on speculative decoding, a technique that generates and verifies multiple candidate tokens in parallel, to improve throughput and latency. In this work, we reveal a new side-channel whereby input-dependent patterns of correct and incorrect speculations can be inferred by monitoring per-iteration token counts or packet sizes. In evaluations using research prototypes and production-grade vLLM serving frameworks, we show that an adversary monitoring these patterns can fingerprint user queries (from a set of 50 prompts) with over 75% accuracy across four speculative-decoding schemes at temperature 0.3: REST (100%), LADE (91.6%), BiLD (95.2%), and EAGLE (77.6%). Even at temperature 1.0, accuracy remains far above the 2% random baseline—REST (99.6%), LADE (61.2%), BiLD (63.6%), and EAGLE (24%). We also show the capability of the attacker to leak confidential datastore contents used for prediction at rates exceeding 25 tokens/sec. To defend against these, we propose and evaluate a suite of mitigations, including packet padding and iteration-wise token aggregation.
Abstract: Large Language Models (LLMs) are increasingly deployed in sensitive domains including healthcare, legal services, and confidential communications, where privacy is paramount. This paper introduces Whisper Leak, a side-channel attack that infers user prompt topics from encrypted LLM traffic by analyzing packet size and timing patterns in streaming responses. Despite TLS encryption protecting content, these metadata patterns leak sufficient information to enable topic classification. We demonstrate the attack across 28 popular LLMs from major providers, achieving near-perfect classification (often >98% AUPRC) and high precision even at extreme class imbalance (10,000:1 noise-to-target ratio). For many models, we achieve 100% precision in identifying sensitive topics like “money laundering” while recovering 5-20% of target conversations. This industry-wide vulnerability poses significant risks for users under network surveillance by ISPs, governments, or local adversaries. We evaluate three mitigation strategies – random padding, token batching, and packet injection – finding that while each reduces attack effectiveness, none provides complete protection. Through responsible disclosure, we have collaborated with providers to implement initial countermeasures. Our findings underscore the need for LLM providers to address metadata leakage as AI systems handle increasingly sensitive information.
Learning Python often starts with the same building blocks: variables, functions, and loops. However, once young people have learnt these essential foundations, they may be eager to grow their skills and start using Python to explore data and create something meaningful to them.
Our free ‘More Python’ project path helps learners move beyond the basics and use data to create impactful projects of their own.
Python as a tool for exploring the world
Python is the most widely used programming language in the world, not just because it’s accessible, but because it’s powerful. It is used to analyse data, build models, create data visualisations, and explore important questions.
For young learners, this means learning Python can become more than a coding exercise. It can be a way to investigate topics they care about, analyse and understand information, and tell powerful stories about real-world issues.
Working with data helps learners see how coding connects to the world around them — and builds confidence along the way.
Why learning with data matters
In our day-to-day lives, data is everywhere: in sports results, maps, and scientific research, to name only a few examples. Learning how to work with data helps young people develop skills that go far beyond programming, including:
Thinking logically and solving problems
Interpreting and questioning information
Making decisions based on evidence
Data also underpins many of the AI systems people use today. For example, large language models, used to build tools such as ChatGPT, are trained on vast amounts of data. Therefore, understanding how data is collected, organised, and used is an important part of AI literacy.
In Python, structures like lists and dictionaries make it possible to organise, analyse, and explore data in creative ways. Using these tools to build projects can help abstract computing concepts start to feel more concrete and meaningful.
What learners create in the ‘More Python’ project path
The ‘More Python’ project path supports learners through three stages: Explore, Design, and Invent. Each stage builds skills while giving learners more ownership over what they create.
In the Explore stage, young people learn new concepts and build confidence in using data and core Python structures, such as lists and dictionaries. Projects include:
Making an interactive chart of Olympic medals
Building a model of the solar system
Creating a frequency graph that learners can analyse to crack a code
These projects help learners develop new skills, while exploring how Python can be used to analyse and explain real-world information.
As learners progress to the Design stage, they start making creative choices about how their projects look and behave. In this stage, they:
Create a project that produces encoded art based on a user’s name
Build an interactive world map that helps users learn interesting facts
Here, Python becomes a creative medium. As well as putting their new skills into practice, learners think about audience, interaction, and presentation to make their projects their own.
In the Invent stage, learners bring everything together. Using the skills they have built, they design and create a data visualisation on a topic they are passionate about. This final project gives learners the freedom to choose their data, shape their idea, and tell a story that matters to them.
By this point, learners are planning and creating their own projects, growing in confidence and independence.
Take the next step with Python
If the young people you support have already learned the basics of Python, ‘More Python’ offers a clear and creative next step. The projects are designed to be accessible, and young people can work through them at their own pace, whether they are learning independently, at a Code Club, or in the classroom.
By working with data, getting creative, and making their own original projects, learners can build confidence and start to see what they can achieve with Python.
Alongside the ‘More Python’ project path, you can access hundreds of free coding projects on our Code Club Projects site. Find more projects to suit your learners’ interests, and support them to build their digital skills through creativity and making.
Когато интернет се появи, много хора бяха обнадеждени. Знанието, което човечеството беше натрупало през вековете, щеше да стане по-достъпно от всякога. Очакването беше, че това ще доведе до по-информирани граждани и по-рационални решения, облагодетелстващи както отделния човек, така и цялото общество.
Днес обаче живеем в свят, в който никога не сме имали повече информация и никога не сме били по-объркани. Причината е не в липсата на данни, а в начина, по който ги възприемаме, подбираме и тълкуваме.
Хората по дефиниция не са рационални, а емоционални същества.
Мозъкът рядко взема разумно решение и едва след това действа – много по-често той намира рационални аргументи, с които да защити позиция, вече заета по емоционални причини. Това е добре изследвано от учени в областта както на когнитивните, така и на политическите науки. В книгата си Democracy for Realists например Кристофър Ейкън и Лари Бартелс защитават тезата, че учебникарската представа за демокрацията, в която гражданите избират на база рационален интерес, е по-скоро мит. В действителност хората се влияят много повече от идентичности, емоции и фактори, които често са извън съзнателния им контрол.
Ако това е вярно, то проблемът с дезинформацията в интернет не може да бъде решен само с повече факти. Нуждаем се и от ориентири – от базов набор инструменти, за да ни помагат да преценяваме каква стойност има информацията, с която се сблъскваме ежедневно. Дори ако в крайна сметка отново вземем емоционално решение, поне ще въоръжим разума с по-добри аргументи, с които да го оправдае.
Йерархия на авторитетите
В съвременния публичен разговор често се сблъскват две крайни гледни точки. Според едната живеем в епоха на сринати авторитети, в която всяко мнение се приема за равностойно, независимо от това дали е подкрепено от знания и опит. Според другата проблемът е тъкмо обратният – прекаленото доверие в авторитети води до интелектуална леност и отказ от самостоятелно мислене. Интернет със своята хоризонтална структура допълнително изостря този конфликт.
И двете позиции съдържат истина. Почитта към някого само по формален признак, защото заема определен пост, или пък заради харизмата на личността му може да доведе до мислене по калъп. Но е вярно и обратното – в социалните мрежи мнението на учен, посветил живота си на изследване на вируси с цел разработване на ваксини, често е приравнено с това на човек, който е прочел няколко публикации в антиваксърски блог, докато си чака кафето.
Затова въпросът е не дали трябва да има авторитети, а защо и при какви условия даден авторитет заслужава доверие. За да отговорим, трябва да направим разграничение не толкова между хората, колкото между начините, по които се добива знанието.
Мисленето на конспиратора обикновено следва една и съща логика:
първо имаме готова теза – например, че извънземните контролират света. След това започваме да търсим доказателства за нея – инцидента в Розуел, кръговете в царевицата, данни за НЛО, заснети от военните, и т.н., след което, доволни, потвърждаваме пред себе си тезата, без всъщност да сме доказали нищо. Първо, защото, както казва Карл Сейгън, грандиозните твърдения изискват грандиозни доказателства. И второ, защото този подход не е научен.
Научният метод тръгва в обратната посока. Първо се набелязва проблем, след това се събира разнообразна информация, тя се анализира и систематизира и едва накрая се формулира хипотеза. Това не гарантира истина, но минимизира риска от самозаблуда.
Давам и личен пример. Преди да кандидатствам за докторант в Софийския университет, имах готова хипотеза за нарастващото поляризиране в САЩ – увеличаващото се етническо многообразие води до напрежение между различните групи. Тя изглеждаше логична, съвпадаше хронологично с процесите, които проследявах, и за нея имаше множество „доказателства“ в интернет, най-често в десни блогове.
Проблемът беше, че бях започнал от края.
Когато добавих Канада към картината, установих, че в някои отношения многообразието там е дори още по-голямо, но подобно напрежение липсва. Нещо повече – предишният път, когато в САЩ е имало вълнения, подобни на днешните, е било по времето на Никсън, когато страната далеч не е била толкова пъстра в демографско отношение.
Всичко това ме принуди да преразгледам подхода си и тепърва да изследвам терена. Години по-късно стигнах до извода, че ключова роля са изиграли медиите. Процесът беше дълъг, но и далеч по-удовлетворителен. Вероятно е индивидуално, но при мен откриването на нова информация е по-вълнуващо от потвърждаването на предварителни убеждения. Големите открития в науката рядко започват с „Еврика!“, а по-често с „Хм, това е интересно“.
Видове източници и тяхната достоверност
Повечето от нас няма да направят научни открития, но всички ежедневно си набавяме информация от различни типове източници – официални институции, научни публикации, публицистика, блогове, социални мрежи и все по-често от съдържание, генерирано от изкуствен интелект.
По принцип официалните институции се ползват с висока степен на доверие, но това зависи от контекста. Русия например е известна с подвеждащата информация, която поднася – за загубите си във войната в Украйна, а и не само за това. В България се наблюдава съмнение в информацията, предоставяна от институциите, което е плод на дългогодишния ни опит с тях. Случаят с мистериозните убийства в района на Петрохан показа колко лесно една официална версия може да бъде отхвърлена от голяма част от обществото – не непременно защото е невярна, а защото хората са убедени, че държавата по принцип лъже, както и заради начина, по който е поднесена.
Научните публикации би следвало да са с най-висока степен на достоверност.
Причината не е в това, че учените са безгрешни, а в самия процес – рецензиране, източници, библиография, минимизиране на личното мнение, когато то не е подкрепено с данни. Минусът е, че академичните текстове често са скучни за четене, а изискването за цитиране в тях, което от външна гледна точка може да изглежда маниакално, е иронизирано още от фантаста Айзък Азимов във „Фондацията“. Там учените имат технологична възможност да проверят дадена теза лично, но предпочитат да сравняват какво са писали колегите им.
Също така трябва да се подчертае, че мнение на представител на науката, изказано в онлайн подкаст или телевизионно предаване, няма същата тежест като публикация в академично издание. Възможно е даден учен да има високи професионални постижения и въпреки това да споделя псевдонаучни тези извън своята експертна област. Такъв пример е д-р Астрид Стукелбергер – лекарка, занимаваща се с последиците от стареенето, която публично твърди, че в Европейската организация за ядрени изследвания (ЦЕРН) се призовават същества от други измерения – нещо, което не се подкрепя от нито едно научно изследване в областта на физиката.
При всичко, казано дотук, когато става дума за здраве, ваксинации или други чувствителни теми, научната литература остава най-сигурният ориентир. Важно е тя да се чете с въпроса „Какво казват учените?“, а не с предварително зададено търсене от типа „ваксините причиняват аутизъм“.
Най-често обаче хората четат публицистика. Тук ориентацията е по-трудна, но не невъзможна. Полезни въпроси са: кой е авторът и експерт ли е по темата? Цитирани ли са източници и какви – официални данни, научни публикации, първоизточници? Прави ли се разлика между факт и мнение? Независима ли е медията и прозрачна ли е по отношение на финансирането си?
Ако тези критерии звучат сложно, може би е по-лесно да започнем от това, което почти сигурно е некачествено. На първо място – изказвания без източници. В България не е рядкост да се публикуват измислени изявления и дори цели интервюта, понякога прикрити с едва виждаща се бележка, че става дума за сатира. Извън страната емблематичен пример за популярната култура беше сайтът Supershadow, станал известен заради фиктивните си интервюта с режисьора Джордж Лукас.
От същия порядък са и твърденията без доказателства. В българската влогосфера е популярен жанрът на интервютата с предполагаемо осведомени хора, които „разкриват“ тайни от световната политика. Заглавия за топ секретни преговори между лидери като Путин, Тръмп или Си Дзинпин пораждат логичен въпрос: присъствал ли е говорещият на тях, че да знае толкова много? Най-често става дума за преразказ на чужди източници или за чиста спекулация, насочена към хора, които лесно биха повярвали в нея.
Винаги трябва да сме особено внимателни с информация, в която силно ни се иска (или силно ни е страх) да повярваме.
Втората група са текстове, които целят да провокират силна емоция на база оскъдна информация. Класически пример са заглавия като „Вземат ни Вазов!“ или „Махат Ботев!“, когато всъщност става дума за технически промени в учебните програми. Подобни похвати, за съжаление, понякога се използват и от официални институции, което допълнително подкопава доверието в тях.
Да се върнем на Петрохан. Изявлението на изпълняващия функциите (вече според няколко съдилища – нелегитимно) на главния прокурор Борислав Сарафов, че случаят напомня на „Туин Пийкс“, е по-подходящо за блогър или за коментар в социална мрежа, отколкото за официално лице. Съвсем отделен е въпросът, че немалка част от скептиците по отношение на официалната версия изпадат в описаната по-горе логическа грешка – да имат готова теза („мафията е замесена“) и след това да търсят доказателства в нейна подкрепа.
Накрая идват кликбейт заглавията от типа „Няма да повярвате…“ или „Еди-кой си разкри истината!“. Те невинаги водят до невярна информация, но твърде често прикриват баналност зад гръмки фрази.
А какво да правим с изкуствения интелект?
Изкуственият интелект най-често обобщава информация от водещи медии и енциклопедии, което го прави по-достоверен от много популярни сайтове. Проблемът е, че когато „не знае“, той халюцинира – създава убедително звучаща, но невярна информация. Понякога прави и странни връзки, резултат от начина, по който функционира. Така например при проверка в Google установих, че според един бот съм любимият чичо на епичното фентъзи – защото имам статия за писателя Тери Брукс със сходно заглавие. Затова ИИ може да бъде полезен помощник, но лош арбитър на истината.
Всичко това може да звучи сложно, но обобщението е сравнително просто. По-голямо доверие заслужават текстове, за които знаем кой е авторът, има ясно посочени източници и съдържанието ни подтиква към размисъл, а не към гняв или страх. Както съветва комикът Бил Махър, не е лошо понякога да излизаме от собствения си балон и да четем и „другия отбор“. Така си поставяме малка интелектуална ваксина срещу манипулация. Важното е, когато го правим, да изискваме качество – и от другите, и от себе си.
Today, we’re announcing the general availability of Amazon Elastic Compute Cloud (Amazon EC2) Hpc8a instances, a new high performance computing (HPC) optimized instance type powered by latest 5th Generation AMD EPYC processors with a maximum frequency of up to 4.5 GHz. These instances are ideal for compute-intensive tightly coupled HPC workloads, including computational fluid dynamics, simulations for faster design iterations, high-resolution weather modeling within tight operational windows, and complex crash simulations that require rapid time-to-results.
The new Hpc8a instances deliver up to 40% higher performance, 42% greater memory bandwidth, and up to 25% better price-performance compared to previous generation Hpc7a instances. Customers benefit from the high core density, memory bandwidth, and low-latency networking that helped them scale efficiently and reduce job completion times for their compute-intensive simulation workloads.
Hpc8a instances Hpc8a instances are available with 192 cores, 768 GiB memory, and 300 Gbps Elastic Fabric Adapter (EFA) networking to run applications requiring high levels of inter node communications at scale.
Instance Name
Physical Cores
Memory (Gib)
EFA Network Bandwidth (Gbps)
Network Bandwidth (Gbps)
Attached Storage
Hpc8a.96xlarge
192
768
Up to 300
75
EBS Only
Hpc8a instances are available in a single 96xlarge size with a 1:4 core-to-memory ratio. You will have the capability to right size based on HPC workload requirements by customizing the number of cores needed at launch instances. These instances also use sixth-generation AWS Nitro cards, which offload CPU virtualization, storage, and networking functions to dedicated hardware and software, enhancing performance and security for your workloads.
You can use Hpc8a instances with AWS ParallelCluster and AWS Parallel Computing Service (AWS PCS) to simplify workload submission and cluster creation and Amazon FSx for Lustre for sub-millisecond latencies and up to hundreds of gigabytes per second of throughput for storage. To achieve the best performance for HPC workloads, these instances have Simultaneous Multithreading (SMT) disabled.
Now available Amazon EC2 Hpc8a instances are now available in US East (Ohio) and Europe (Stockholm) AWS Regions. For Regional availability and a future roadmap, search the instance type in the CloudFormation resources tab of AWS Capabilities by Region.
Since we launched Amazon Nova customization in Amazon SageMaker AI at AWS NY Summit 2025, customers have been asking for the same capabilities with Amazon Nova as they do when they customize open weights models in Amazon SageMaker Inference. They also wanted have more control and flexibility in custom model inference over instance types, auto-scaling policies, context length, and concurrency settings that production workloads demand.
Today, we’re announcing the general availability of custom Nova model support in Amazon SageMaker Inference, a production-grade, configurable, and cost-efficient managed inference service to deploy and scale full-rank customized Nova models. You can now experience an end-to-end customization journey to train Nova Micro, Nova Lite, and Nova 2 Lite models with reasoning capabilities using Amazon SageMaker Training Jobs or Amazon HyperPod and seamlessly deploy them with managed inference infrastructure of Amazon SageMaker AI.
With Amazon SageMaker Inference for custom Nova models, you can reduce inference cost through optimized GPU utilization using Amazon Elastic Compute Cloud (Amazon EC2)G5 and G6 instances over P5 instances, auto-scaling based on 5-minute usage patterns, and configurable inference parameters. This feature enables deployment of customized Nova models with continued pre-training, supervised fine-tuning, or reinforcement fine-tuning for your use cases. You can also set advanced configurations about context length, concurrency, and batch size for optimizing the latency-cost-accuracy tradeoff for your specific workloads.
Let’s see how to deploy customized Nova models on SageMaker AI real-time endpoints, configure inference parameters, and invoke your models for testing.
Deploy custom Nova models in SageMaker Inference At AWS re:Invent 2025, we introduced new serverless customization in Amazon SageMaker AI for popular AI models including Nova models. With a few clicks, you can seamlessly select a model and customization technique, and handle model evaluation and deployment. If you already have a trained custom Nova model artifact, you can deploy the models on SageMaker Inference through the SageMaker Studio or SageMaker AI SDK.
In the SageMaker Studio, choose a trained Nova model in Models in your models in the Models menu. You can deploy the model by choosing Deploy button, SageMaker AI and Create new endpoint.
Choose the endpoint name, instance type, and advanced options such as instance count, max instance count, permission and networking, and Deploy button. At GA launch, you can use g5.12xlarge, g5.24xlarge, g5.48xlarge, g6.12xlarge, g6.24xlarge, g6.48xlarge, and p5.48xlarge instance types for the Nova Micro model, g5.24xlarge, g5.48xlarge, g6.24xlarge, g6.48xlarge, and p5.48xlarge for the Nova Lite model, and p5.48xlarge for the Nova 2 Lite model.
Creating your endpoint requires time to provision the infrastructure, download your model artifacts, and initialize the inference container.
After model deployment completes and the endpoint status shows InService, you can perform real-time inference using the new endpoint. To test the model, choose the Playground tab and input your prompt in the Chat mode.
You can also use the SageMaker AI SDK to create two resources: a SageMaker AI model object that references your Nova model artifacts, and an endpoint configuration that defines how the model will be deployed.
The following code creates a SageMaker AI model that references your Nova model artifacts:
# Create a SageMaker AI model
model_response = sagemaker.create_model(
ModelName= 'Nova-micro-ml-g5-12xlarge',
PrimaryContainer={
'Image': '123456789012.dkr.ecr.us-east-1.amazonaws.com/nova-inference-repo:v1.0.0',
'ModelDataSource': {
'S3DataSource': {
'S3Uri': 's3://your-bucket-name/path/to/model/artifacts/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'
}
},
# Model Parameters
'Environment': {
'CONTEXT_LENGTH': 8000,
'CONCURRENCY': 16,
'DEFAULT_TEMPERATURE': 0.0,
'DEFAULT_TOP_P': 1.0
}
},
ExecutionRoleArn=SAGEMAKER_EXECUTION_ROLE_ARN,
EnableNetworkIsolation=True
)
print("Model created successfully!")
Next, create an endpoint configuration that defines your deployment infrastructure and deploy your Nova model by creating a SageMaker AI real-time endpoint. This endpoint will host your model and provide a secure HTTPS endpoint for making inference requests.
After the endpoint is created, you can send inference requests to generate predictions from your custom Nova model. Amazon SageMaker AI supports synchronous endpoints for real-time with streaming/non-streaming modes and asynchronous endpoints for batch processing.
For example, the following code creates streaming completion format for text generation:
# Streaming chat request with comprehensive parameters
streaming_request = {
"messages": [
{"role": "user", "content": "Compare our Q4 2025 actual spend against budget across all departments and highlight variances exceeding 10%"}
],
"max_tokens": 512,
"stream": True,
"temperature": 0.7,
"top_p": 0.95,
"top_k": 40,
"logprobs": True,
"top_logprobs": 2,
"reasoning_effort": "low", # Options: "low", "high"
"stream_options": {"include_usage": True}
}
invoke_nova_endpoint(streaming_request)
def invoke_nova_endpoint(request_body):
"""
Invoke Nova endpoint with automatic streaming detection.
Args:
request_body (dict): Request payload containing prompt and parameters
Returns:
dict: Response from the model (for non-streaming requests)
None: For streaming requests (prints output directly)
"""
body = json.dumps(request_body)
is_streaming = request_body.get("stream", False)
try:
print(f"Invoking endpoint ({'streaming' if is_streaming else 'non-streaming'})...")
if is_streaming:
response = runtime_client.invoke_endpoint_with_response_stream(
EndpointName=ENDPOINT_NAME,
ContentType='application/json',
Body=body
)
event_stream = response['Body']
for event in event_stream:
if 'PayloadPart' in event:
chunk = event['PayloadPart']
if 'Bytes' in chunk:
data = chunk['Bytes'].decode()
print("Chunk:", data)
else:
# Non-streaming inference
response = runtime_client.invoke_endpoint(
EndpointName=ENDPOINT_NAME,
ContentType='application/json',
Accept='application/json',
Body=body
)
response_body = response['Body'].read().decode('utf-8')
result = json.loads(response_body)
print("✅ Response received successfully")
return result
except ClientError as e:
error_code = e.response['Error']['Code']
error_message = e.response['Error']['Message']
print(f"❌ AWS Error: {error_code} - {error_message}")
except Exception as e:
print(f"❌ Unexpected error: {str(e)}")
Now available Amazon SageMaker Inference for custom Nova models is available today in US East (N. Virginia) and US West (Oregon) AWS Regions. For Regional availability and a future roadmap, visit the AWS Capabilities by Region.
The feature supports Nova Micro, Nova Lite, and Nova 2 Lite models with reasoning capabilities, running on EC2 G5, G6, and P5 instances with auto-scaling support. You pay only for the compute instances you use, with per-hour billing and no minimum commitments. For more information, visit Amazon SageMaker AI Pricing page.
Enterprise customers face an unprecedented security landscape where sophisticated cyber threats use artificial intelligence to identify vulnerabilities, automate attacks, and evade detection at machine speed. Traditional perimeter-based security models are insufficient when adversaries can analyze millions of attack vectors in seconds and exploit zero-day vulnerabilities before patches are available.
The distributed nature of serverless architectures compounds this challenge—while microservices offer agility and scalability, they significantly expand the attack surface where each API endpoint, function invocation, and data store becomes a potential entry point, and a single misconfigured component can provide attackers the foothold needed for lateral movement. Organizations must simultaneously navigate complex regulatory environments where compliance frameworks like GDPR, HIPAA, PCI-DSS, and SOC 2 demand robust security controls and comprehensive audit trails, while the velocity of software development creates tension between security and innovation, requiring architectures that are both comprehensive and automated to enable secure deployment without sacrificing speed.
The challenge is multifaceted:
Expanded attack surface: Multiple entry points across distributed services requiring protection against distributed denial of service (DDoS) attacks, injection vulnerabilities, and unauthorized access
Identity and access complexity: Managing authentication and authorization across numerous microservices and service-to-service communications
Data protection requirements: Encrypting sensitive data in transit and at rest while securely storing and rotating credentials without compromising performance
Compliance and data protection: Meeting regulatory requirements through comprehensive audit trails and continuous monitoring in distributed environments
Network isolation challenges: Implementing controlled communication paths without exposing resources to the public internet
AI-powered threats: Defending against attackers who use AI to automate reconnaissance, adapt attacks in real-time, and identify vulnerabilities at machine speed
The solution lies in defense-in-depth—a layered security approach where multiple independent controls work together to protect your application.
This article demonstrates how to implement a comprehensive AI-powered defense-in-depth security architecture for serverless microservices on Amazon Web Services (AWS). By layering security controls at each tier of your application, this architecture creates a resilient system where no single point of failure compromises your entire infrastructure, designed so that if one layer is compromised, additional controls help limit the impact and contain the incident while incorporating AI and machine learning services throughout to help organizations address and respond to AI-powered threats with AI-powered defenses.
Architecture overview: A journey through security layers
Let’s trace a user request from the public internet through our secured serverless architecture, examining each security layer and the AWS services that protect it. This implementation deploys security controls at seven distinct layers with continuous monitoring and AI-powered threat detection throughout, where each layer provides specific capabilities that work together to create a comprehensive defense-in-depth strategy:
Layer 1 blocks malicious traffic before it reaches your application
Layer 2 verifies user identity and enforces access policies
Layer 3 encrypts communications and manages API access
Layer 4 isolates resources in private networks
Layer 5 secures compute execution environments
Layer 6 protects credentials and sensitive configuration
Layer 7 encrypts data at rest and controls data access
Continuous monitoring detects threats across layers using AI-powered analysis
Figure 1: Architecture diagram
Layer 1: Edge protection
Before requests reach your application, they traverse the public internet where attackers launch volumetric DDoS attacks, SQL injection, cross-site scripting (XSS), and other web exploits. AWS observed and mitigated thousands of distributed denial of service (DDoS) attacks in 2024, with one exceeding 2.3 terabits per second.
DDos protection: AWS Shield provides managed DDoS protection for applications running on AWS and is enabled for customers at no cost. AWS Shield Advanced offers enhanced detection, continuous access to the AWS DDoS Response Team (DRT), cost protection during attacks, and advanced diagnostics for enterprise applications.
Layer 7 protection: AWS WAFprotects against Layer 7 attacks through managed rule groups from AWS and AWS Marketplace sellers that cover OWASP Top 10 vulnerabilities including SQL injection, XSS, and remote file inclusion. Rate-based rules automatically block IPs that exceed request thresholds, protecting against application-layer DDoS and brute force attacks. Geo-blocking capabilities restrict access based on geographic location, while Bot Control uses machine learning to identify and block malicious bots while allowing legitimate traffic.
AI for security: Amazon GuardDuty uses generative AI to enhance native security services, implementing AI capabilities to improve threat detection, investigation, and response through automated analysis.
AI-powered enhancement: Organizations can build autonomous AI security agents using Amazon Bedrock to analyze AWS WAF logs, reason through attack data, and automate incident response. These agents detect novel attack patterns that signature-based systems miss, generate natural language summaries of security incidents, automatically recommend AWS WAF rule updates based on emerging threats, correlate attack indicators across distributed services to identify coordinated campaigns, and trigger appropriate remediation actions based on threat context. This helps enable more proactive threat detection and response capabilities, reducing mean time to detection and response.
Layer 2: Verifying identity
After requests pass edge protection, you must verify user identity and determine resource access. Traditional username/password authentication is vulnerable to credential stuffing, phishing, and brute force attacks, requiring robust identity management that supports multiple authentication methods and adaptive security responding to risk signals in real time.
Amazon Cognito provides comprehensive identity and access management for web and mobile applications through two components:
User pools offer a fully managed user directory handling registration, sign-in, multi-factor authentication (MFA), password policies, social identity provider integration, SAML and OpenID Connect federation for enterprise identity providers, and advanced security features including adaptive authentication and compromised credential detection.
Identity pools grant temporary, limited-privilege AWS credentials to users for secure direct access to AWS services without exposing long-term credentials.
Amazon Cognito adaptive authentication uses machine learning to detect suspicious sign-in attempts by analyzing device fingerprinting, IP address reputation, geographic location anomalies, and sign-in velocity patterns, then allows sign-in, requires additional MFA verification, or blocks attempts based on risk assessment. Compromised credential detection automatically checks credentials against databases of compromised passwords and blocks sign-ins using known compromised credentials. MFA supports both SMS-based and time-based one-time password (TOTP) methods, significantly reducing account takeover risk.
For advanced behavioral analysis, organizations can use Amazon Bedrock to analyze patterns across extended timeframes, detecting account takeover attempts through geographic anomalies, device fingerprint changes, access pattern deviations, and time-of-day anomalies.
Layer 3: The application front door
An API gateway serves as your application’s entry point. It must handle request routing, throttling, API key management, encryption and it needs to integrate seamlessly with your authentication layer and provide detailed logging for security auditing while maintaining high performance and low latency.
Amazon API Gateway is a fully managed service for creating, publishing, and securing APIs at scale, providing critical security capabilities including SSL/TLS encryption withAWS Certificate Manager (ACM) to automatically handle certificate provisioning, renewal, and deployment. Request throttling and quota management protects backend services through configurable burst and rate limits with usage quotas per API key or client to prevent abuse, while API key management controls access from partner systems and third-party integrations. Request/response validation uses JSON Schema to validate data before reaching AWS Lambda functions, preventing malformed requests from consuming compute resources while seamless integration with Amazon Cognito validates JSON Web Tokens (JWTs) and enforces authentication requirements before requests reach application logic.
GuardDuty provides AI-powered intelligent threat detection by analyzing API invocation patterns and identifying suspicious activity including credential exfiltration using machine learning. For advanced analysis, Amazon Bedrock analyzes API Gateway metrics and Amazon CloudWatch logs to identify unusual HTTP 4XX error spikes (for example, 403 Forbidden) that might indicate scanning or probing attempts, geographic distribution anomalies, endpoint access pattern deviations, time-series anomalies in request volume, or suspicious user agent patterns.
Layer 4: Network isolation
Application logic and data must be isolated from direct internet access. Network segmentation is designed to limit lateral movement if a security incident occurs, helping to prevent compromised components from easily accessing sensitive resources.
Amazon Virtual Private Cloud (Amazon VPC) provides isolated network environments implementing a multi-tier architecture with public subnets for NAT gateways and application load balancers with internet gateway routes, private subnets for Lambda functions and application components accessing the internet through NAT Gateways for outbound connections, and data subnets with the most restrictive access controls. Lambda functions run in private subnets to prevent direct internet access, VPC flow logs capture network traffic for security analysis, security groups provide stateful firewalls following least privilege principles, Network ACLs add stateless subnet-level firewalls with explicit deny rules, and VPC endpoints enable private connectivity to Amazon DynamoDB, AWS Secrets Manager, and Amazon S3 without traffic leaving the AWS network.
GuardDuty provides AI-powered network threat detection by continuously monitoring VPC Flow Logs, CloudTrail logs, and DNS logs using machine learning to identify unusual network patterns, unauthorized access attempts, compromised instances, and reconnaissance activity, now including generative AI capabilities for automated analysis and natural language security queries.
Layer 5: Compute security
Lambda functions executing your application code and often requiring access to sensitive resources and credentials must be protected against code injection, unauthorized invocations, and privilege escalation. Additionally, functions must be monitored for unusual behavior that might indicate compromise.
Lambda provides built-in security features including:
Resource-based policies that control which services and accounts can invoke functions to prevent unauthorized invocations
Environment variable encryption using AWS Key Management Services (AWS KMS) for variables at rest while sensitive data should use Secrets Manager function isolation designed so that each execution runs in isolated environments preventing cross-invocation data access
VPC integration enabling functions to benefit from network isolation and security group controls
Runtime security with automatically patched and updated managed runtimes
Code signing with AWS Signer digitally signing deployment packages for code integrity and cryptographic verification against unauthorized modifications
AI-powered code security: Amazon CodeGuru Security combines machine learning and automated reasoning to identify vulnerabilities including OWASP Top 10 and CWE Top 25 issues, log injection, secrets, and insecure AWS API usage. Using deep semantic analysis trained on millions of lines of Amazon code, it employs rule mining and supervised ML models combining logistic regression and neural networks for high true-positive rates.
Vulnerability management: Amazon Inspector provides automated vulnerability management, continuously scanning Lambda functions for software vulnerabilities and network exposure, using machine learning to prioritize findings and provide detailed remediation guidance.
Layer 6: Protecting credentials
Applications require access to sensitive credentials including database passwords, API keys, and encryption keys. Hardcoding secrets in code or storing them in environment variables creates security vulnerabilities, requiring secure storage, regular rotation, authorized-only access, and comprehensive auditing for compliance.
Secrets Manager protects access to applications, services, and IT resources without managing hardware security modules (HSMs). It provides centralized secret storage for database credentials, API keys, and OAuth tokens in an encrypted repository using AWS KMS encryption at rest.
Automatic secret rotation configures rotation for database credentials, automatically updating both the secret store and target database without application downtime.
Fine-grained access control uses IAM policies to control which users and services access specific secrets, implementing least-privilege access.
Audit trails log secret access in AWS CloudTrail for compliance and security investigations. VPC endpoint support is designed so that secret retrieval traffic doesn’t leave the AWS network.
Lambda integration enables functions to retrieve secrets programmatically at runtime, designed so that secrets aren’t stored in code or configuration files and can be rotated without redeployment.
GuardDuty provides AI-powered monitoring, detecting anomalous behavior patterns that could indicate credential compromise or unauthorized access.
Layer 7: Data protection
The data layer stores sensitive business information and customer data requiring protection both at rest and in transit. Data must be encrypted, access tightly controlled, and operations audited, while maintaining resilience against availability attacks and high performance.
Amazon DynamoDB is a fully managed NoSQL database providing built-in security features including:
Encryption at rest (using AWS-owned, AWS managed, or customer managed KMS keys)
Encryption in transit (TLS 1.2 or higher)
Fine-grained access control through IAM policies with item-level and attribute-level permissions
VPC endpoints for private connectivity
Point-in-Time Recovery for continuous backups
Streams for audit trails
Backup and disaster recovery capabilities
Global Tables for multi-AWS Region, multi-active replication designed to provide high availability and low-latency global access
GuarDuty and Amazon Bedrock provide AI-powered data protection:
GuardDuty monitors DynamoDB API activity through CloudTrail logs using machine learning to detect anomalous data access patterns including unusual query volumes, access from unexpected geographic locations, and data exfiltration attempts.
Amazon Bedrock analyzes DynamoDB Streams and CloudTrail logs to identify suspicious access patterns, correlate anomalies across multiple tables and time periods, generate natural language summaries of data access incidents for security teams, and recommend access control policy adjustments based on actual usage patterns versus configured permissions. This helps transform data protection from reactive monitoring to proactive threat hunting that can detect compromised credentials and insider threats.
Continuous monitoring
Even with comprehensive security controls at every layer, continuous monitoring is essential to detect threats that bypass defenses. Security requires ongoing real-time visibility, intelligent threat detection, and rapid response capabilities rather than one-time implementation.
GuardDuty protects your AWS accounts, workloads, and data with intelligent threat detection.
CloudWatch provides comprehensive monitoring and observability, collecting metrics, monitoring log files, setting alarms, and automatically reacting to AWS resource changes.
CloudTrail provides governance, compliance, and operational auditing by logging all API calls in your AWS account, creating comprehensive audit trails for security analysis and compliance reporting.
AI-powered enhancement with Amazon Bedrock provides automated threat analysis; generating natural language summaries of GuardDuty findings and CloudWatch logs, pattern recognition identifying coordinated attacks across multiple security signals, incident response recommendations based on your architecture and compliance requirements, security posture assessment with improvement recommendations, and automated response through Lambda and Amazon EventBridge that isolates compromised resources, revokes suspicious credentials, or notifies security teams through Amazon SNS when threats are detected.
Conclusion
Securing serverless microservices presents significant challenges, but as demonstrated, using AWS services alongside AI-powered capabilities creates a resilient defense-in-depth architecture that protects against current and emerging threats while proving that security and agility are not mutually exclusive.
Security is an ongoing process—continuously monitor your environment, regularly review security controls, stay informed about emerging threats and best practices, and treat security as a fundamental architectural principle rather than an afterthought.
If you have feedback about this blog post, submit them in the Comments section below. If you have questions about using this solution, start a thread in the EventBridge, GuardDuty, or Security Hub forums, or contact AWS Support.
The curl project has found AI-powered tools to be a mixed bag when
it comes to security reports. At FOSDEM 2026, curl creator and
lead developer Daniel Stenberg used his keynote session to discuss his
experience receiving a slew of low-quality reports and, at the same
time, realizing that large language model (LLM) tools can sometimes
find flaws that other tools have missed.
This post is co-written with Srinivasa Are, Principal Cloud Architect, and Karthick Shanmugam, Head of Architecture Verisk EES (Extreme Event Solutions).
Verisk, a catastrophe modeling SaaS provider serving insurance and reinsurance companies worldwide, cut processing time from hours to minutes-level aggregations while reducing storage costs by implementing a lakehouse architecture with Amazon Redshift and Apache Iceberg. If you’re managing billions of catastrophe modeling records across hurricanes, earthquakes, and wildfires, this approach eliminates the traditional compute-versus-cost trade-off by separating storage from processing power.
In this post, we examine Verisk’s lakehouse implementation, focusing on four architectural decisions that delivered measurable improvements:
Execution performance: Sub-hour aggregations across billions of records replaced long batch process
Storage efficiency: Columnar Parquet compression reduced costs without sacrificing response time
Multi-tenant security: Schema-level isolation enforced complete data separation between insurance clients
Schema flexibility: Apache Iceberg support column additions and historical data access without downtime
The architecture separates compute (Amazon Redshift) from storage (Amazon S3), demonstrating how to scale from billions to trillions of records without proportional cost increases.
Current state and challenges
In Verisk’s world of risk analytics, data volumes grow at exponential rates. Every day, risk modeling systems generate billions of rows of structured and semi-structured data. Each record captures a micro-slice of exposure, event probability, or loss correlation. To convert this raw information into actionable insights at scale, experts need a data engine designed for high-volume analytical workloads.
Each Verisk model run produces detailed, high-granularity outputs that include billions of simulated risk factors and event-level results, multi-year loss projections across thousands of perils, and deep relational joins across exposure, policy, and claims datasets.
Running meaningful aggregations (such as, loss by region, peril, or occupancy type) over such high volumes created performance challenges.
Verisk needed to build a SQL service that could aggregate at scale in the fastest time possible and integrate into their broader AWS solutions, requiring a serverless, open, and performant SQL engine capable of handling billions of records efficiently.
Prior to this cloud-based release, Verisk’s risk analytics infrastructure operated on an on-premises architecture centered around relational database clusters. Processing nodes shared access to centralized storage volumes through dedicated interconnect networks. This architecture required capital investment in server hardware, storage arrays, and networking equipment. The deployment model required manual capacity planning and provisioning cycles, limiting the organization’s ability to respond to fluctuating workload demands. Database operations depended on batch-oriented processing windows, with analytical queries competing for shared compute resources.
Amazon Redshift and lakehouse architecture
Lakehouse architecture on AWS combines data lake storage scalability with data warehouse analytical performance in a unified architecture. This architecture stores vast amounts of structured and semi-structured data in cost-effective Amazon S3 storage while maintaining Amazon Redshift’s massively parallel SQL analytics.
Amazon Redshift is a fully managed, petabyte-scale cloud data warehouse service that delivers fast query performance using massively parallel processing (MPP) and columnar storage. Amazon Redshift eliminates the complexity of provisioning hardware, installing software, and managing infrastructure, keeping focus on deriving insights from their data rather than maintaining systems.
To meet their challenge, Verisk designed a hybrid data lakehouse architecture that combines the storage scalability of Amazon S3 with the compute power of Amazon Redshift. The following diagram shows the foundational compute and storage architecture that powers Verisk’s analytical solution.
Architecture Overview
The architecture processes risk and loss data through three distinct stages within the lakehouse architecture, with comprehensive multi-tenant delivery capabilities to maintain isolation between insurance clients.
Amazon Redshift allows retrieving data directly from S3 using standard SQL for background processing. This solution collects detailed result outputs, join them with internal reference data, and executes aggregations over billions of rows. Concurrency scaling guarantees that hundreds of background analyses using multiple serverless clusters can run simultaneous aggregation queries.
The following diagram shows the architecture designed by Verisk
Data ingestion and storage foundation
Verisk stores risk model outputs, location level losses, exposure tables, and model data in columnar Parquet format within Amazon S3. An AWS Glue crawler extracts metadata from S3 and feeds it into the lakehouse processing pipeline.
For versioned datasets like exposure tables, Verisk adopted Apache Iceberg, an open table format that addresses schema evolution and historical versioning requirements. Apache Iceberg provides transactional consistency through atomicity, consistency, isolation, durability ACID-compliant operations that maintain consistent snapshots during concurrent updates. Snapshot-based time travel allows data retrieval at previous points in time for regulatory compliance, audit trails, and model comparison with rollback capabilities. Schema evolution supports adding, dropping, or renaming columns without downtime or dataset rewrites. Incremental processing uses metadata tracking to process only changed data, reducing refresh times. Hidden partitioning and file-level statistics reduce I/O operations, improving aggregation performance. Engine interoperability allows accessing the same tables across Amazon Redshift, Amazon Athena, Spark, and other engines without data duplication.
Verisk built a foundation that combines S3’s cost-effectiveness with data management by adopting Apache Iceberg as open table format for this solution.
Three-stage processing pipeline
This pipeline orchestrates data flow from raw inputs to analytical outputs through three sequential stages. Pre-processing prepares and cleanses data, modeling applies risk calculations and analytics, and post-processing aggregates results for delivery.
Stage 1: Pre-processing transforms raw data into structured formats using Iceberg Tables and Parquet files, then processes it through Amazon Redshift Serverless for initial data cleaning and transformation.
Stage 2: Modeling takes place with a process built on AWS Batch the pre-processed data and applies advanced analytics and feature engineering. Results are stored in Iceberg Tables and Parquet files.
Stage 3: Aggregated Results are obtained during post-processing using Amazon Redshift Serverless, it produces the final analytical outputs in Parquet files, ready for consumption by end users.
Multi-tenant delivery system
The architecture delivers results to multiple insurance clients (tenants) through a secure, isolated delivery system that includes:
Amazon Quick Sight dashboards for visualization and business intelligence
Amazon Redshift as the data warehouse for querying aggregated results
Tenant Roles implementing role-based access control to provide data isolation between clients
Summarized results are exposed through Amazon Quick Sight dashboards or downstream APIs to underwriting teams.
Multi-tenant security architecture
A critical requirement for Verisk’s SaaS solution was supporting comprehensive data and compute isolation between different insurance and reinsurance clients. Verisk implemented a comprehensive multi-tenant security model that provides isolation while maintaining operational efficiency.
Our solution implements an isolation strategy in two layers combining logical and physical separation. At the logical layer, each client’s data resides in dedicated schemas with access controls that prevent cross-tenant operations. Amazon Redshift Metadata security restricts tenants from discovering or accessing other clients’ schemas, tables, or database objects through system catalogs. At the physical layer, for larger deployments, dedicated Amazon Redshift clusters provide workload separation at the compute level, preventing one tenant’s analytical operations from impacting another’s performance. This dual approach meets regulatory requirements for data isolation in the insurance industry through schema-level isolation within clusters for standard deployments and complete compute separation across dedicated clusters for larger-scale implementations.
The implementation uses stored procedures to automate security configuration, maintaining consistent application of access controls across tenants. This defense-in-depth approach combines schema-level isolation, system catalog lockdown, and selective permission grants to create a security model.
Verisk’s architecture reveals three decision points for companies building similar systems.
When to adopt open table formats
Apache Iceberg proved essential for datasets requiring schema evolution and historical versioning. Data engineers should evaluate open table formats when analytical workloads span multiple engines (Amazon Redshift, Amazon Athena, Spark) or when regulatory requirements demand point-in-time data reconstruction.
Multi-tenant isolation strategy
Schema-level separation combined with metadata security prevented cross-tenant data discovery without performance overhead. This approach scales more efficiently than database-per-tenant architectures while meeting insurance industry compliance requirements. Security experts should implement isolation controls during initial deployment rather than retrofitting them later.
Stored procedures or application logic
Redshift stored procedures standardized aggregation calculations across teams and constructed dynamic SQL queries. This approach works best when business logic changes frequently or when multiple teams need different aggregation dimensions on the same datasets.
Conclusion
Verisk’s implementation of Amazon Redshift Serverless with Apache Iceberg and lakehouse architecture shows how separating compute from storage addresses enterprise analytics challenges at billion-record scale. By combining cost-effective Amazon S3 storage with Redshift’s massively parallel SQL compute, Verisk achieved aggregations across billions of catastrophe modeling records, reduced storage costs through efficient parquet compression, and eliminated ingestion delays. Now underwriting teams can run ad-hoc analyses during business hours rather than waiting for long-running batch jobs. The combination of open standards like Apache Iceberg, serverless compute with Amazon Redshift, and multi-tenant security provides the scalability, performance, and cost efficiency needed for modern analytics workloads.
Verisk’s journey has positioned them to scale confidently into the future, processing not just billions, but potentially trillions of records as their model resolution increases.
In e-commerce, delivering fast, relevant search results helps users find products quickly and accurately, improving satisfaction and increasing sales. OpenSearch is a distributed search engine that offers advanced search capabilities including advanced full-text and faceted search, customizable analyzers and tokenizers, and auto-complete to help customers quickly find the products they want. It scales to handle millions of products, catalogs and traffic surge. Amazon OpenSearch Service is a managed service that lets users build search workloads balancing search quality, performance at scale and cost. Designing and sizing an Amazon OpenSearch Service cluster correctly is required to meet these demands.
While general sizing guidelines for OpenSearch Service domains are covered in detail in OpenSearch Service documentation, in this post we specifically focus on T-shirt-sizing OpenSearch Service domains for e-commerce search workloads. T-shirt sizing simplifies complex capacity planning by categorizing workloads into sizes like XS, S, M, L, XL based on key workload parameters such as data volume and query concurrency. For e-commerce search, where data growth is moderate and read-heavy queries predominate, this approach offers a flexible, scalable way to allocate the resources without overprovisioning or underestimating needs.
How OpenSearch Service stores indexes and performs queries
E-commerce search platforms handle vast amounts of data and daily data ingestion is typically relatively small and incremental, reflecting catalog changes, price updates, inventory status and user activities like clicks and reviews. Efficiently managing this data and organizing it per OpenSearch Service best practices is crucial in achieving optimal performance. The workload is read-heavy, consisting of user queries with advanced filtering and faceting, especially during sales or seasonal spikes that require elasticity in compute and storage resources.
You ingest product and catalog updates (inventory, listings, pricing) into OpenSearch using bulk APIs or real-time streaming. You index data into logical indexes. How you create and organize indexes in e-commerce has a significant impact on search, scalability and flexibility. The approach depends on the size, diversity and operational needs of the catalog. Small to medium-sized e-commerce platforms commonly use a single, comprehensive product index that stores all product information with product category. Additional indexes may exist for orders, users, reviews and promotions depending on search requirements and data separation needs. Large, diverse catalogs may split products into category-specific indexes for tailored mappings and scaling. You split each index into primary shards, each storing a portion of the documents. To ensure high availability and enhance query throughput, you configure each primary shard with at least one replica shard stored on different data nodes.
Diagram 1. How primary and replica shards are distributed among nodes
This diagram shows two indexes (Products and Reviews), each split into two primary shards with one replica. OpenSearch distributes these shards across cluster nodes to ensure that primary and replica shards for the same data do not reside on the same node. OpenSearch runs search requests using a scatter-gather mechanism. When an application submits a request, any node in the cluster can receive it. This receiving node becomes the coordinating node for that specific query. The coordinating node determines which indices and shards can serve the query. It forwards the query to either primary or replica shards and orchestrates the different phases of the search operation and returns the response. This process ensures efficient distribution and execution of search requests across the OpenSearch cluster.
Diagram 2. Tracing a Search query: “blue running shoes”This diagram walks through how a search query–for example, “blue running shoes”flows through your OpenSearch Service domain .
Request: The application sends the search for “blue running shoes” to the domain. One data node acts as the coordinating node.
Scatter: The coordinator broadcasts the query to either the primary or replica shard for each of the shards in the ‘Products’ index (Nodes 1, 2, and 3 in this case).
Gather: Each data node searches its local shards(s) for “blue running shoes” and returns its own top results (e.g. Node 1 returns its best matches from P0).
Final results: The coordinator merges these partial lists, sorts them into single definitive list of the most relevant shoes, and sends the result back to the app.
Understanding T-Shirt Sizing for E-commerce OpenSearch Service Cluster
Storage planning
Storage impacts both performance and cost. OpenSearch Service offers two main storage options based on query latency requirements and data persistence needs. Selecting the appropriate storage type in a managed OpenSearch Service improves both performance and optimizes cost of the domain. You can choose between Amazon Elastic Block Store( EBS) storage volumes and instance storage volumes (local storage) for your data nodes.
Amazon EBS gp3 volumes offer high throughput, whereas the local NVMe SSD volumes, for example, on the r8gd, i3, or i4i instance families, offer low latency, fast indexing performance and high-speed storage, making them ideal for scenarios where real time data updates and high search throughput are critical for search operations. For search workloads that require a balance between performance and cost, instances backed with EBS GP3 SSD volumes provide a reliable option. This SSD storage offers input/output operations per second (IOPS) that are well-suited for general-purpose search workloads. It also allows users to provision additional IOPS and storage as needed.
When sizing an OpenSearch cluster, start by estimating total storage needs based on catalog size and expected growth. For example, if the catalog contains 500,000 stock keeping units (SKUs), averaging 50KB each; the raw data sums to about 25GB. The size of the raw data, however, is just one aspect of the storage requirements. Also consider the Replica count, indexing overhead (10%), Linux reserves (5%), and OpenSearch Service reserves (20% up to 20GB) per instance while calculating the required storage.
In summary,if you have 25GB of data at any given time who want one replica, the minimum storage requirement is closer to 25 * 2 * 1.1 / 0.95 / 0.8 = 72.5 GB. This calculation can be generalized as follows:
Storage requirement = Raw data * (1 + number of replicas) * 1.45
This helps ensure disk space headroom on all data nodes, preventing shard failures and maintaining search performance. Provisioning storage slightly beyond this minimum is recommended to accommodate future growth and cluster rebalancing.
Data nodes:
For search workloads, compute-optimized instances (C8g) are well-suited for central processing unit (CPU)-intensive operations like nested queries and joins. However, general-purpose instances like M8g offer a better balance between CPU and memory. Memory-optimized instances (R8g, R8gd) are recommended for memory-intensive operations like KNN search, where larger memory footprint is required. In large, complex deployments, compute-optimized instances like c8g or general-purpose m8g, handle CPU-intensive tasks, providing efficient query processing and balanced resource allocation. The balance between CPU and memory, makes them ideal for managing complex search operations for large-scale data processing. For extremely large search workloads (tens of TB) where latency is not a primary concern, consider using the new Amazon OpenSearch Service Writable warm which supports write operations on warm indices.
Instance Class
Best for users who…
Examples (AWS)
Characteristics
General Purpose
have moderate search traffic and want a well-balanced, entry-level setup
M family (M8g)
Balanced CPU & memory, EBS storage. Good starting point for small to medium-sized catalogs.
Compute Optimized
have high queries per second (QPS) search traffic or queries involve scoring scripts or complex filtering
C family (C8)
High CPU-to-memory ratio. Ideal for CPU-bound workloads like many concurrent queries.
Memory Optimized
work with large catalogs, need fast aggregations, or cache a lot in memory
R family (R8g)
More memory per core. Holds large indices in memory to speed up searches and aggregations.
Storage Optimized
update inventory frequently or have so much data that disk access slows things down
I family (I3, I4g), Im4gn
NVMe SSD and SSD local storage. Best for I/O-heavy operations like constant indexing or large product catalogs hitting disk frequently.
Cluster manager nodes:
For production workloads, it is recommended to add dedicated cluster manager nodes to increase the cluster stability and offload cluster management tasks from the data nodes. To choose the right instance type for your cluster manager nodes, review the service recommendations based on the OpenSearch version and number of shards in the cluster.
Sharding strategy
Once storage requirements are understood, you can investigate the indexing strategy. You create shards in OpenSearch Service to distribute an index evenly across the nodes in a cluster. AWS recommends single product index with category facets for simplicity or partition indexes by category for large or distributed catalogs. The size and number of shards per index play a vital role in OpenSearch Service performance and scalability. The right configuration ensures balanced data distribution, avoids hot spotting, and minimizes coordination overhead on nodes for use cases that prioritizes query speed and data freshness.
For read-heavy workloads like e-commerce, where search latency is the key performance objective, maintain shard sizes between 10-30GB. To achieve this, calculate the number of primary shards by dividing your total index size by your target shard size. For example, if you have a 300GB index and want 20GB shards, configure 15 primary shards (300GB ÷ 20GB = 15 shards). Monitor shard sizes using the _cat/shards API and adjust the shard count during reindexing if shards grow beyond the optimal range.
Add replica shards to improve search query throughput and fault tolerance. The minimum recommendation is to have one replica; you can add more replicas for high query throughput requirements. In OpenSearch Service, a shard processes operations like querying single-threaded, meaning one thread handles a shard’s tasks at a time. Replica shards can serve read requests by distributing them across multiple threads and nodes, enabling parallel processing.
T-shirt sizing for an e-commerce workload
In an OpenSearch T-shirt sizing table, each size label (XSmall, Small, Medium, Large, XLarge) represents a generalized cluster scale category that can help teams translate technical requirements into simple, actionable capacity planning. Each size allows architects to quickly align their catalog size, storage requirements, shard planning, CPU and AWS instance choices to the cluster resources provisioned, making it easier to scale infrastructure as business grows.
By referring to this table, teams can select the category similar to their current workload and use the T-shirt size as a starting point while continuing to refine configuration as they monitor and optimize real-world performance. For example, XSmall is suited for small catalogs with hundreds of thousands of products and minimal search traffic. Small clusters are designed for growing catalogs with millions of SKUs, supporting moderate query volumes and scaling up during busy periods. Medium corresponds to mid-size e-commerce operations handling millions of products and higher search demands, while Large fits large online businesses with tens of millions of SKUs, requiring robust infrastructure for fast, reliable search. XLarge is intended for major marketplaces or global platforms with twenty million or more SKUs, enormous data storage needs, and massive concurrent usage.
T-shirt size
Number of Products
Catalog Size
Storage needed
Primary Shard Count
Active Shard Count
Data Nodes Instance Type
Cluster Manager Node instanceType
XSmall
500K
50 GB
145 GB
2
4
[2] r8g.xlarge
[3] m8g.large
Small
2M
200 GB
580 GB
8
16
[2] c8g.4xlarge
[3] m8g.large
Medium
5M
500 GB
1.45 TB
20
40
[2] c8g.8xlarge
[3] m8g.large
Large
10M
1 TB
2.9 TB
40
80
[4] c8g.8xlarge
[3] m8g.large
XLarge
20M
2 TB
5.8 TB
80
160
[4] c8g.16xlarge
[3] m8g.large
T-shirt size: Represents the scale of the cluster, ranging from XS up to XL for high-volume workloads.
Number of products: The estimated count of SKUs in the e-commerce catalog, which drives the data volume.
Catalog size: The total estimated disk size of all indexed product data, based on typical SKU document size.
Storage needed: The actual storage required after accounting for replicas and overhead, ensuring enough room for safe and efficient operation.
Primary shard count: The number of main index shards chosen to balance parallel processing and resource management.
Active shard count: The total number of live shards (primary with one replica), indicating how many shards need to be distributed for availability and performance.
Data node instance type: The recommended instance type to use for data nodes, selected for memory, CPU, and disk throughput.
Cluster manager node instance type: The recommended instance type for lightweight, dedicated master nodes which manage cluster stability and coordination.
Scaling strategies for e-commerce workloads
E-commerce platforms continually face challenges with unpredictable traffic surges and growing product catalogs. To address these challenges, OpenSearch Service automatically publishes critical performance metrics to Amazon CloudWatch, enabling users to monitor when individual nodes reach resource limits. These metrics include CPU utilization exceeding 80%, JVM memory pressure above 75%, frequent garbage collection pauses, and thread pool rejections.
OpenSearch Service also provides robust scaling solutions that maintain consistent search performance across varying workload demands. Use the vertical scaling strategy to upgrade instance types from smaller to larger configurations, such as m6g.large to m6g.2xlarge. While vertical scaling triggers a blue-green deployment, scheduling these changes during off-peak hours minimizes impact on operations.
Use the horizontal scaling strategy to add more data nodes for distributing indexing and search operations. This approach proves particularly effective when scaling for traffic growth or increasing dataset size. In domains with cluster manager nodes, adding data nodes proceeds smoothly without triggering a blue-green deployment. CloudWatch metrics guide horizontal scaling decisions by monitoring thread pool rejections across nodes, indexing latency, and cluster-wide load patterns. Though the process requires shard rebalancing and may temporarily impact performance, it effectively distributes workload across the cluster.
Temporary replicas provide a flexible solution for managing high-traffic periods. By increasing replica shards through the _settings API, read throughput can be boosted when needed. This approach offers a dynamic response to changing traffic patterns without requiring more substantial infrastructure changes.
Monitoring key performance CloudWatch metrics is essential to ensure a well-optimised OpenSearch service domain. One of the key factors is maintaining CPU utilization on data nodes under 80% to prevent query slowdowns. Another metric is ensuring that JVM memory pressure is maintained below 75% on data nodes to prevent garbage collection (GC) pauses that can affect search response time. OpenSearch service publishes these metrics to CloudWatch at 1 minute interval and users can create alarms on these metrics for alerts on the production workloads. Please refer recommended CloudWatch alarms for OpenSearch Service
P95 query latency should be monitored to identify slow queries and optimize performance. Another important indicator is thread pool rejections. A high number of thread pool rejections can result in failed search requests, and affecting user experience. By continuously monitoring these CloudWatch metrics, users can proactively scale resources, optimise queries, and prevent performance bottlenecks.
Conclusion
In this post, we showed how to right-size Amazon OpenSearch Service domains for e-commerce workloads using a T-shirt sizing approach. We explored key factors including storage optimization, sharding strategies, scaling methods, and essential Amazon CloudWatch metrics for monitoring performance.
To build a performant search experience, start with a smaller deployment and iterate as your business scales. Get started with these five steps:
Evaluate your workload requirements in terms of storage, search throughput, and search performance
Select your initial T-shirt size based on your product catalog size and traffic patterns
Deploy the recommended sharding strategy for your catalog scale
Load test your cluster using OpenSearch benchmark and re-iterate until performance requirements are reached
Configure Amazon CloudWatch monitoring and alarms, then continue to monitor your production domain
I joined AWS in 2021, and since then I’ve watched the Amazon Elastic Compute Cloud (Amazon EC2) instance family grow at a pace that still surprises me. From AWS Graviton-powered instances to specialized accelerated computing options, it feels like every few months there’s a new instance type landing that pushes performance boundaries further. As of February 2026, AWS offers over 1,160 Amazon EC2 instance types, and that number keeps climbing.
This week’s opening news is a good example: The general availability of Amazon EC2 M8azn instances. These are general purpose, high-frequency, high-network instances powered by fifth generation AMD EPYC processors, offering the highest maximum CPU frequency in the cloud at 5 GHz. Compared to the previous generation M5zn instances, M8azn instances deliver up to 2x compute performance, 4.3x higher memory bandwidth, and a 10x larger L3 cache. They also provide up to 2x networking throughput and up to 3x Amazon Elastic Block Store (Amazon EBS) throughput compared with M5zn.
Built on the AWS Nitro System using sixth generation Nitro Cards, M8azn instances target workloads such as real-time financial analytics, high-performance computing, high-frequency trading, CI/CD pipelines, gaming, and simulation modeling across automotive, aerospace, energy, and telecommunications. The instances feature a 4:1 ratio of memory to vCPU and are available in 9 sizes ranging from 2 to 96 vCPUs with up to 384 GiB of memory, including two bare metal variants. For more information visit the Amazon EC2 M8azn instance page.
Last week’s launches Here are some of the other announcements from last week:
Amazon Bedrock adds support for six fully managed open weights models – Amazon Bedrock now supports DeepSeek V3.2, MiniMax M2.1, GLM 4.7, GLM 4.7 Flash, Kimi K2.5, and Qwen3 Coder Next. These models span frontier reasoning and agentic coding workloads. DeepSeek V3.2 and Kimi K2.5 target reasoning and agentic intelligence, GLM 4.7 and MiniMax M2.1 support autonomous coding with large output windows, and Qwen3 Coder Next and GLM 4.7 Flash provide cost-efficient alternatives for production deployment. These models are powered by Project Mantle and provide out-of-the-box compatibility with OpenAI API specifications. With the launch, you can also use new open weight models–DeepSeek v3.2 , MiniMax 2.1, and Qwen3 Coder Next in Kiro, a spec-driven AI development tool.
Amazon Bedrock expands support for AWS PrivateLink – Amazon Bedrock now supports AWS PrivateLink for the bedrock-mantle endpoint, in addition to existing support for the bedrock-runtime endpoint. The bedrock-mantle endpoint is powered by Project Mantle, a distributed inference engine for large-scale machine learning model serving on Amazon Bedrock. Project Mantle provides serverless inference with quality of service controls, higher default customer quotas with automated capacity management, and out-of-the-box compatibility with OpenAI API specifications. AWS PrivateLink support for OpenAI API-compatible endpoints is available in 14 AWS Regions. To get started, visit the Amazon Bedrock console or the OpenAI API compatibility documentation.
Amazon EKS Auto Mode announces enhanced logging for managed Kubernetes capabilities – You can now configure log delivery sources using Amazon CloudWatch Vended Logs in Amazon EKS Auto Mode. This helps you collect logs from Auto Mode’s managed Kubernetes capabilities for compute autoscaling, block storage, load balancing, and pod networking. Each Auto Mode capability can be configured as a CloudWatch Vended Logs delivery source with built-in AWS authentication and authorization at a reduced price compared to standard CloudWatch Logs. You can deliver logs to CloudWatch Logs, Amazon S3, or Amazon Data Firehose destinations. This feature is available in all Regions where EKS Auto Mode is available.
Amazon OpenSearch Serverless now supports Collection Groups – You can use new Collection Groups to share OpenSearch Compute Units (OCUs) across collections with different AWS Key Management Service (AWS KMS) keys. Collection Groups reduce overall OCU costs through a shared compute model while maintaining collection-level security and access controls. They also introduce the ability to specify minimum OCU allocations alongside maximum OCU limits, providing guaranteed baseline capacity at startup for latency-sensitive applications. Collection Groups are available in all Regions where Amazon OpenSearch Serverless is currently available.
Amazon RDS now supports backup configuration when restoring snapshots – You can view and modify the backup retention period and preferred backup window before and during snapshot restore operations. Previously, restored database instances and clusters inherited backup parameter values from snapshot metadata and could only be modified after restore was complete. You can now view backup settings as part of automated backups and snapshots, and specify or modify these values when restoring, eliminating the need for post-restoration modifications. This is available for all Amazon RDS database engines (MySQL, PostgreSQL, MariaDB, Oracle, SQL Server, and Db2) and Amazon Aurora (MySQL-Compatible and PostgreSQL-Compatible editions) in all AWS commercial Regions and AWS GovCloud (US) Regions at no additional cost.
For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
Upcoming AWS events Check your calendar and sign up for upcoming AWS events:
AWS Summits – Join AWS Summits in 2026, free in-person events where you can explore emerging cloud and AI technologies, learn best practices, and network with industry peers and experts. Upcoming Summits include Paris (April 1), London (April 22), and Bengaluru (April 23–24).
AWS AI and Data Conference 2026 – A free, single-day in-person event on March 12 at the Lyrath Convention Centre in Ireland. The conference covers designing, training, and deploying agents with Amazon Bedrock, Amazon SageMaker, and QuickSight, integrating them with AWS data services, and applying governance practices to operate them at scale. The agenda includes strategic guidance and hands-on labs for architects, developers, and business leaders.
AWS Community Days – Community-led conferences where content is planned, sourced, and delivered by community leaders, featuring technical discussions, workshops, and hands-on labs. Upcoming events include Ahmedabad (February 28), Slovakia (March 11), and Pune (March 21).
Greg-Kroah Hartman has released the 6.19.2, 6.18.12, 6.12.73, and 6.6.126 stable kernels. These kernels
each contain a single change; Kroah-Hartman has reverted one
problematic commit that prevents some systems from booting. “If the
last stable release worked just fine, no need to upgrade.“
At the 2025 Linux Plumbers Conference in Tokyo, Stephen Brennan gave a
presentation on the debuginfo
format, which contains the symbols and other information needed for
debugging, along with some alternatives. Debuginfo files are large and, he
believes, are a bit scary to customers because of the “debug” in their name.
By rethinking debuginfo and the tools that use it, he hopes that
free-software developers “can add new, interesting capabilities to tools
that we are already using or build new interesting tools“.
Greg Kroah-Hartman has announced the release of the 6.19.1, 6.18.11, 6.12.72, and 6.6.125 stable kernels. As always, each
contains important fixes throughout the tree; users of these kernels
are advised to upgrade.
The collective thoughts of the interwebz
Manage Consent
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
