Post Syndicated from Crosstalk Solutions original https://www.youtube.com/watch?v=UsJ0MtV99XY
Alarmo: Turn Your Home into a Digital Fortress! (Home Assistant)
Post Syndicated from BeardedTinker original https://www.youtube.com/shorts/vqQhVyYOzKM
[$] On the use of LLM assistants for kernel development
Post Syndicated from corbet original https://lwn.net/Articles/1032612/
By some appearances, at least, the kernel community has been relatively
insulated from the onslaught of AI-driven software-development tools.
There has not been a flood of vibe-coded memory-management patches — yet.
But kernel development is, in the end, software development, and these
tools threaten to change many aspects of how software development is done.
In a world where companies are actively pushing their developers to use
these tools, it is not surprising that the topic is increasingly prominent
in kernel circles as well. There are currently a number of ongoing
discussions about how tools based on large language models (LLMs) fit into
the kernel-development community.
Rust 1.89 released
Post Syndicated from daroc original https://lwn.net/Articles/1032808/
The release of Rust 1.89 has been
announced. Changes this time include
support for inferring the length of certain arrays, lint messages suggesting how to clarify potentially confusing uses of lifetime elision in function signatures, and improvements to the C ABI. The
full changelog is also available.
Security updates for Thursday
Post Syndicated from jake original https://lwn.net/Articles/1032861/
Security updates have been issued by AlmaLinux (glibc, kernel, libxml2, python-requests, and python-setuptools), Debian (chromium), Fedora (chromium, firefox, gdk-pixbuf2, iputils, libsoup3, libssh, perl, perl-Devel-Cover, perl-PAR-Packer, polymake, and poppler), Gentoo (Composer and Spreadsheet-ParseExcel), Oracle (glibc, kernel, libxml2, python-setuptools, sqlite, and virt:rhel and virt-devel:rhel), Red Hat (libxml2), SUSE (grub2, libarchive, libgcrypt, and python311), and Ubuntu (cifs-utils and poppler).
OG Builds & Rides HOVSCO HovGtrs E-Bike
Post Syndicated from digiblur DIY original https://www.youtube.com/shorts/0zIO2sdvoZ0
Comic for 2025.08.07 – Thicker Than Water
Post Syndicated from Explosm.net original https://explosm.net/comics/thicker-than-water
New Cyanide and Happiness Comic
When Generative AI Meets Zabbix
Post Syndicated from Cesar Caceres original https://blog.zabbix.com/when-generative-ai-meets-zabbix/30908/
Zabbix has been the backbone of my infrastructure for over ten years, a journey I’ve been on from version 3.2 to 7.4. It’s a robust and reliable tool. However, in the age of intelligent assistants, I posed a question to myself: Why can’t I interact with my monitoring system as naturally as I talk with Maria, my generative AI assistant?
Table of Contents
What is MCP?
MCP (Model Context Protocol) is a universal protocol that helps generative AI systems interact with global data securely, reliably, and at scale.
Imagine this: It’s 3 AM, and you receive a critical alert on your phone. Instead of opening multiple dashboards and manually correlating data, you simply type: “What’s happening with the production server?”
You get a response like this:
“The web-prod-01 server is experiencing high memory usage (94%). This started 15 minutes ago, coinciding with a traffic spike. I recommend checking the database connection pool and considering a restart of the Apache service. Would you like me to show you the related logs?”
This is no longer science fiction!
Design principle
The main objective is to enhance Zabbix without altering its core. The solution is based on an architecture that adheres to the following principles:
- Zabbix intact: The original installation remains unchanged.
- API-first: All communication is done through Zabbix’s robust JSON-RPC API.
- Intelligent bridge: An intermediary service is created to translate between human language and Zabbix metrics.
- Scalability: The design is prepared to grow alongside the infrastructure.
Proposed architecture:
- Zabbix server: Debian 12, Zabbix 7.4.0, PostgreSQL 15.13
- AI server (MCP): Rocky Linux 9, Gemini AI, Express.js, Winston (Logging), Gemini CLI, Redis, Nginx, PM2
Webhooks
We process Zabbix alerts through a webhook that sends the data to our generative AI service.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import json
import requests
import sys
from datetime import datetime
def send_to_mcp(args):
""" Sends alerts to MCP server"""
# SETTINGS - EDIT ACCORDING TO YOUR ENVIRONMENT
mcp_endpoint = "http://TU_IP_MCP_SERVER:3001/alerts" # Change to the MCP server IP
mcp_token = "TU_MCP_AUTH_TOKEN" # Exchange for your MCP authentication token
zabbix_server_ip = "TU_IP_ZABBIX_SERVER" # Change to the Zabbix server IP
headers = {
'Content-Type': 'application/json',
'Authorization': f'Bearer {mcp_token}'
}
# Extracting arguments from the Zabbix webhook
eventid = args[0] if len(args) > 0 else "unknown"
severity = args[1] if len(args) > 1 else "0"
message = args[2] if len(args) > 2 else "No message"
host = args[3] if len(args) > 3 else "unknown"
value = args[4] if len(args) > 4 else ""
payload = {
"timestamp": datetime.now().isoformat(),
"source": "zabbix",
"eventid": eventid,
"severity": severity,
"message": message,
"host": host,
"value": value,
"zabbix_server": zabbix_server_ip
}
try:
print(f"Sending alert to MCP: {mcp_endpoint}")
print(f"Payload: {json.dumps(payload, indent=2)}")
response = requests.post(mcp_endpoint, json=payload, headers=headers, timeout=15)
response.raise_for_status()
print(f"Alert sent successfully: HTTP {response.status_code}")
print(f"Response: {response.text}")
return True
except requests.exceptions.Timeout:
print("? Error: Timeout connecting to MCP server", file=sys.stderr)
return False
except requests.exceptions.ConnectionError:
print("? Error: Cannot connect to MCP server", file=sys.stderr)
return False
except requests.exceptions.HTTPError as e:
print(f"? HTTP Error: {e}", file=sys.stderr)
print(f"Response: {response.text}", file=sys.stderr)
return False
except Exception as e:
print(f"? Unexpected error: {e}", file=sys.stderr)
return False
if __name__ == "__main__":
# The arguments come from Zabbix
args = sys.argv[1:]
print(f"Script called with args: {args}")
success = send_to_mcp(args)
sys.exit(0 if success else 1)
Use cases
Scenario 1: Monday morning
Cesar: Good morning, how is the infrastructure looking?
Gemini CLI: Good morning! Everything is calm. 14 active hosts, 0 critical problems. There was a scheduled reboot on ‘proxmox-desarrollo’ over the weekend which completed successfully. The average CPU usage is at 23%. Do you need details on any specific system?
Scenario 2: Proactive planning
Cesar: What worrying trends are you seeing this week?
Gemini CLI: I’ve noticed three patterns that require attention:
1. storage-01: Growth of 2GB/day in /var/log.
-
- Projection: 90% capacity in 12 days.
- Recommendation: Configure a more aggressive log rotate.
2. mikrotik: Gradually increasing temperature (+3°C over the last 7 days).
-
- Recommendation: Check ventilation.
3. base-datos-01: Slow queries increasing by 15%.
-
- Recommendation: Review indexes and optimize frequent queries.
MCP implementation
Server (Rocky Linux 9) bash #create the project mkdir /opt/mcp-zabbix cd /opt/mcp-zabbix #Install dependencies sudo dnf install -y nodejs npm redis nginx sudo npm install -g pm2 #Set up the project npm init -y npm install express axios @google/generative-ai winston helmet cors dotenv
Configuration (.env)
bash #Environment variables ZABBIX_URL=http://tu-zabbix-server/zabbix/api_jsonrpc.php ZABBIX_API_TOKEN=tu_token_de_zabbix_aqui GEMINI_API_KEY=tu_api_key_de_gemini MCP_AUTH_TOKEN=genera_un_token_seguro PORT=3001
Webhook in Zabbix
1. Alerts → Media Types → Create
2. Script name: mcp_webhook.py
3. Parameters: {EVENT.ID} {EVENT.NSEVERITY} {ALERT.MESSAGE} {HOST.NAME} {ITEM.VALUE}
Test it
#Start the MCP server
pm2 start ecosystem.config.js
#Test curl
curl -H "Authorization: Bearer TU_TOKEN" \
-H "Content-Type: application/json" \
-d '{"prompt":"¿How many host fo I have?"}' \
http://localhost:3001/ask-zabbix
The future
Dashboard conversations
Cesar: Show me a dashboard of the critical servers.
Gemini CLI: Creating custom dashboard with:
- CPU/memory of your 3 production servers
- Network latency of web services
- Database disk space
- Nightly backup status
Generated dashboard: http://zabbix.local/dashboard/generated-123
Errors to avoid
- Don’t ignore security: Tokens, firewall, rate limiting from day 1
- Don’t forget documentation: Code explains itself, workflows don’t
Resources to get started
- Complete installation: Scripts for Rocky Linux and Debian
- Zabbix configuration: Media types and actions
- API reference: Endpoints and examples
Use cases
Basic monitoring: Hosts, items, triggers
- Intelligent alerts: Automatic analysis
- Ad-hoc queries: Quick investigation
- Automated reports: Periodic summaries
Future integrations
The goal is to develop an application that allows natural interaction with an AI assistant called “Maria.” The idea is that based on what’s happening, Maria suggests actions and executes them proactively.
To achieve this, the assistant will integrate with Gemini’s command-line interface (CLI) and establish an additional secure communication channel. The recommended architecture will consist of several servers capable of understanding each other, including a Zabbix Server, the MCP (Model Context Protocol), and the personal assistant.You can follow the development of the base integration in this repository.
Conclusion
Zabbix will continue to be the reliable engine we all know. The difference is that it now becomes more intuitive and conversational. The goal is not to replace human experience, but to empower it. AI will allow us to create solutions that were previously unthinkable.
To fully leverage this potential, it is essential that we, as experts, continue to train and deepen our knowledge of the tool. This way, we will not only depend on what the AI suggests, but we will be able to validate and authorize its actions with our own judgment.
The post When Generative AI Meets Zabbix appeared first on Zabbix Blog.
China Accuses Nvidia of Putting Backdoors into Their Chips
Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2025/08/china-accuses-nvidia-of-putting-backdoors-into-their-chips.html
The government of China has accused Nvidia of inserting a backdoor into their H20 chips:
China’s cyber regulator on Thursday said it had held a meeting with Nvidia over what it called “serious security issues” with the company’s artificial intelligence chips. It said US AI experts had “revealed that Nvidia’s computing chips have location tracking and can remotely shut down the technology.”
No Easy Fix | 2. Tolerance
Post Syndicated from The Atlantic original https://www.youtube.com/watch?v=HeMIjpcStFg
[$] LWN.net Weekly Edition for August 7, 2025
Post Syndicated from corbet original https://lwn.net/Articles/1032016/
Inside this week’s LWN.net Weekly Edition:
- Front: Don’t fear the TPM; Python performance; Offensive Debian packages; NNCPNET; 6.17 Merge window; Transparent huge pages; SilverBullet.
- Briefs: AUR malware; Secure boot; kbuild and kconfig maintenanec; GPU drivers; NVIDIA on AlmaLinux; Proxmox 9.0; Quotes; …
- Announcements: Newsletters, conferences, security updates, patches, and more.
Mmmm. Barbeque.
Post Syndicated from The History Guy: History Deserves to Be Remembered original https://www.youtube.com/shorts/dVn8K1ahFhg
Спирал ли е Желязков продажбите и какво въобще ще гласуват депутатите?
Post Syndicated from Боян Юруков original https://yurukov.net/blog/2025/jeliazkov-laje/
На 1-ви август публикувах поредния текст от сагата с държавните имоти за разпродажба. В него освен всичко друго описах как списъкът с над 4400 имоти обсъждани и готвени за продажба от кабинета на Желязков е изчезнал от сайта на МРРБ, както и че обявени търгове изчезват от публичния регистър. Списъкът остава единствено наличен в оригиналната статия, с която разкрих него и картата, която го направи истински използваем.
Този текст остана сравнително незабелязан до 4-ти август, когато по покана на радио Хоризонт описах проблемите с хаоса и липсата на прозрачност в този процес. Същия ден Клуб Z са искали коментар от Министерството на регионалното развитие и благоустройството, които им отговарят с прессъобщение. В него потвърждават, че списъкът е свален от сайта, но това било за да се внесат корекции от институциите, както и че местната власт е проявила интерес към имотите и искала да ѝ се предоставят. Повечето от въпросните искания всъщност текат от години без развитие, но все още няма отговор как тези имоти са попаднали в списъка. МРРБ също отричат да има активни търгове за имоти от списъка. За последното Клуб Z цитират пресконференция на Спаси София, които от своя страна изглежда визират отново моята статия и карта свързваща търгове с имоти.
Поредният отговор на кабинета

В следващите дни множество медии писаха за казуса и особено се възроди темата за амфитеатъра в София, който дадох като пример разкривайки данните за продажбите в края на юни. Имаше още няколко подписки и множество сигнали към платформата Uchastvai.bg. Вчера към 17 часа Министерски съвет пусна ново прес-съобщение, което хем доразвива, хем си противоречи с казаното от МРРБ.
…Съгласно Приложение 2, започна подготовка на Индикативен списък на имоти – държавна собственост, нуждата от които е отпаднала. Същият списък е публично достъпен с цел пълна прозрачност и информираност на обществото и е в процес на изработване, попълване и прецизиране.
В него Желязков твърди, че по темата има политически спекулации, лъжи и внушения. Доколкото може би си е негова лична интерпретация, това клише не може да е извинение за бягане от критика и отговорност. Противоречи си обаче като твърди, че списъкът е публичен – към момента на пускане на прес-съобщението, както и до края на деня то все още липсва от страницата на МРРБ и не е налично към документите в решенията на МС. Оригиналът е архивиран единствено на статията, с която започнах темата. Паралелно с това повтаря думите на МРРБ, че списъкът се „прецизира“, та не става ясно дали Желязков сам е сигурен дали нещо е публично или не. Разбиранията ни за прозрачност видимо се разминават.
Има ли продажби на имоти?
За пореден път обръщаме внимание, че към момента продажби по реда на тази програма не се осъществяват, тъй като редица ведомства и общини са заявили за своите нужди желание да придобият имоти с отпаднала необходимост.
Ако разглеждаме строго семантично горното твърдение, то е формално вярно заради две ключови фрази: „към момента“ и „по реда на тази програма“. Ако вземем твърдението му в цялостта си обаче е очевидна лъжа.
В точния момент на публикуването на съобщението няма активни търгове на страницата на АППК. Но само формално. Търговете обикновено траят няколко часа, в които има „продажба“. Семантично е прав, че в 17:00 на 6-ти август няма активна продажба. Ще има обаче в 11:00 на 7-ми август за апартамент в Обзор (търг 1013). Ще тече вероятно докато четете тази статия. Ще има още един на 8-ми август пак от 11:00 за парцел и сграда в с. Голям Желязна, Ловеч. Последният, впрочем, е именно част от списъка на Желязков с 4400-те – имот номер 4205, търг номер 1017. Т.е. ще има продажба от „списъка“ 42 часа след твърдението на Желязков, но не „към момента“.
Интересен момент е, че доста медии съобщиха, че „кабинетът спира продажбите“. Не това се казва в прес-съобщението. Не прави заявка, че ще спира нещо или че няма да продава, а че иска депутатите да решат дали да го спрат. Дневник го описват добре.

Тук обаче идва втората семантична гимнастика – „по реда на тази програма“. Всъщност, заедно със споменатия горе имот има общо шест от списъка скрит от Желязков, които имат активни търгове: този в с. Голям Желязна, Ловеч; в Тръстеник (търг 1018); в Габрово (търг 1009); два в Стара Загора (1015 и 1016) и в Свиленград (търг 1001). Както писах миналата седмица, има още пет търга, които бяха заличени противно на досегашната практика. Вероятно същата съдба очаква и тези. С горната фраза обаче Желязков може да се оправдае, че тези заедно с останалите 22 активни търга са всъщност „по друг ред“. От тях два са публикувани в деня на прес-съобщението на Желязков, три – предходния ден, 10 в деня преди статията ми обявяваща изтритите търгове, а останалите – след първата ми статия, когато се клеха, че ще продават едва след подробен анализ.
Впрочем, до ден днешен няма отговор какво всъщност значи „отпаднала необходимост“ или публикувани анализи или части от тях за който и да е от 4400-те имота. Знаем обаче, че вече има продадени имоти от митичния индикативен списък. Откривам поне 10 такива случая. Девет от тях са преди приемането на „плана“ в края на май. Един в Трявна е още през 2022-ра. Два – в Стара Загора и Плиска – са през 2024-та. Още шест са продадени с търгове между февруари и 7-ми май 2025-та. Един в Драгоево, Велики Преслав (търг 1010) е проведен успешно на 24-ти юли. По същото време Желязков отговаря на депутати от Да, България, че няма подробни анализи за отпаднала необходимост на имоти и настоява, че не се продават. Явно за този не е била нужна. Останалите продадени преди обявяването на списъка все пак намират място в него, което е поредния белег за хаоса в работата на кабинета и бързането по тази тема.
Още за картата
Знам, че в последната статия се зарекох да правя повече промени, а само да се допълват автоматично статуса на търговете. Обаче с последните изяви на Желязков се налага. Добавих два филтъра – предстоящи и „минали и изтрити“ търгове. Активни са само когато се избира опцията за известни или такива от списъка на Желязков. Филтрите не се взимат под внимание когато се търси по текст.

Тук виждате пример за всички предстоящи търгове на АППК. Поради естеството на автоматичното им обработване и хаоса в данните на МРРБ и АППК, на места ще видите, че се засича търг за продажба на апартамент в страда или имот споменат списъка на Желязков. Т.е. е възможно с търг да се продава съседен апартамент или офис сред 4400-те имота или направо там цялата сграда да се продава, а да виждаме търг за отделен апартамент.
Какво всъщност пита Желязков депутатите?
Това не става ясно от прес-съобщението му. Докато не видим официалния документ внесен в парламента може само да гадаем. В деня на прес-съобщението му и заседание на кабинета липсва решение в публичното деловодство на кабинета и нямаше такава точка в дневния ред. Първо казва, че ще иска парламента да реши какво да се прави с този списък, но не предоставя оригиналния списък или ревизиран такъв. Ако не са го публикували още, значи още не е готов. Надали следва да предполагаме, че министър председателят би крил умишлено такава информация от публичността. При такава липса обаче депутатите няма да имат друг избор освен да се ориентират по моята карта.
Казва също, че ще иска депутатите да приемат решение, с което да му забранят да продава имотите докато пак депутатите не одобрят списъка. Тази формулировка е повече от странна. В същото време твърди, че не се провеждат такива продажби, което показах, че е лъжа за всеки здравомислещ човек. Едно обяснение би било, че сам не може да контролира собствените си действия или администрацията си и прехвърля несвойствена отговорност на парламента. Това би било логично само за нездравомислещ човек … или такъв добре запознат с нездравите зависимости в кабинета и новото начало в местния и публичен живот на държавата.
В същото време търговете си текат, обявите за продажба на друго имущество и отдаване на наем в АППК надвишават всичко, което администрацията е виждала до сега. Пряко на първоначалната ми похвала към Желязков, че все пак публикува нещо, видимо прозрачността и отчетността го уплаши и продължи както го води инстинкта. Аз пък мога само да ви призова да подавате сигнали, за да научим повече какво се случва по места и да искаме отговори.
Целият шум около тези имоти обаче сякаш помогнаха на доста национални медии да замълчат изцяло или да премълчат ключови детайли и имена по други теми. Например новата „Лафка“ на Пеевски под формата на държавни магазини; как близък кадър на Пеевски е начело на КЗК и как иска да я закрие вливайки я в другата му бухалка ДАНС; как немските медии отразиха завземането на властта от Пеевски; как Таки ще има достъп до целия трафик на България и ще може да следи всички коли, включително НСО и дипломати и множество други скандални решения на кабинета и парламента преди ваканцията им.
Вие забелязахте ли ги?
The post Спирал ли е Желязков продажбите и какво въобще ще гласуват депутатите? first appeared on Блогът на Юруков.
Improve email deliverability with tenant management in Amazon SES
Post Syndicated from Satya S Tripathy original https://aws.amazon.com/blogs/messaging-and-targeting/improve-email-deliverability-with-tenant-management-in-amazon-ses/
Amazon Simple Email Service (Amazon SES) serves diverse industries—from ecommerce services to financial institutions to marketing technology product providers—helping organizations manage their email communication needs. Many businesses face the challenge of sending emails not just for themselves, but on behalf of their downstream customers or across various business divisions. These scenarios, commonly known as multi-tenant email sending practices, require careful architectural consideration. For example, a marketing service might need to send promotional emails for hundreds of retail clients, or an enterprise IT team might manage email communications across multiple business units (BUs). These clients and BUs are also identified as tenants. To successfully implement multi-tenancy in Amazon SES, customers usually develop an architecture pattern within Amazon SES that accomplishes critical objectives to efficiently handle the email sending needs of thousands of downstream tenants while maintaining isolated email sending reputations for each tenant. This isolation is crucial for protecting each customer’s deliverability metrics and to prevent issues with one tenant from impacting others.
Amazon SES customers can achieve multi-tenancy through isolated configuration sets for sending emails, but traditionally, reputation management and enforcement occur at the account level. To address this, Amazon SES now offers tenant management capabilities that enable tenant isolation and reputation management at the individual tenant level. This new feature provides greater control and flexibility for organizations managing multiple tenants within a single Amazon SES account, allowing each tenant to maintain its own sending reputation independently.
In this post, you will learn about the newly released tenant management feature that helps customers manage individual tenant onboardings and manage their reputations in isolation. This feature helps organizations create and manage up to 10,000 isolated tenants within a single AWS account (which can be increased 300,000 on explicit request), each with independent configurations and reputation metrics. You will discover how these capabilities maintain email deliverability through automated tenant-level controls, real-time monitoring, and customizable sending policies.
Whether you’re a service provider sending emails on behalf of multiple customers or an enterprise coordinating various BUs or lines of business (LOBs), this new feature offers sophisticated workflows to identify reputation-based findings and pause individual tenant sending to protect other tenants’ reputations. These enhancements are available globally across AWS Regions where Amazon SES is offered, representing a significant advancement in email deliverability management at scale.
Use cases
Following use cases can easily achieved though Amazon SES tenant management feature.
- Onboard multiple brands from different BUs with different domains
- Separate marketing and transaction tenants
- Support independent software vendor (ISV) customers’ requirement to segregate email sending reputation of their customers
- Domain management using configuration sets.
- Track individual customers’ email sending reputations and control their email sending processes
Multi-tenant email operation challenges
Businesses rely on email as a critical communication channel. However, managing email operations for multiple tenants (customers or business units) has historically presented significant challenges such as:
- Lack of isolation: Without proper tenant isolation, poor sending practices by one tenant could negatively impact the email deliverability of others, potentially jeopardizing your entire email sending operation.
- Limited visibility: Understanding per-tenant email performance metrics and managing reputation independently has been difficult, if not impossible.
- Scalability constraints: Many organizations struggle to scale their email operations because of account-level limitations on resources such as identities and configuration sets.
- Inadequate control: The inability to set tenant-specific limits and configurations has made it challenging to prevent individual tenants from impacting others or exceeding allocated resources.
- Complex monitoring: Building custom solutions for monitoring tenant activity often leads to inconsistent and inefficient workflows.
Benefits of tenant management capability
The Amazon SES tenant management feature provides a comprehensive solution for organizations managing email sending at scale on behalf of their customers or LOBs (called tenants). This capability is particularly valuable for software as a service (SaaS) providers, email service providers, and enterprises managing email operations across multiple clients or departments while separating tenants from each other.
Through tenant management, organizations can effectively manage email streams and reputation independently and maintain oversight of their various email operations. This new functionality transforms how organizations use Amazon SES, enabling them to handle complex, multi-faceted email operations with greater control and visibility at the tenant level with the following key capabilities.
- Isolate tenant resources and reputation: Tenant management provides dedicated resource isolation that protects your email reputation across different customers and lines of business. Each tenant (customers or LOBs) will have their own dedicated set of resources such as email sending IPs, domain, and identifiers in DomainKeys Identified Mail (DKIM) signed headers, which are observed by mailbox providers within your Amazon SES account. Tenant management delivers granular control over resource allocation. You can assign tenant-specific or shared sending identities based on your organizational requirements. Each tenant can receive dedicated SMTP or API credentials that provide secure access to their allocated resources. You can configure tenant-level IP pools that separate sending traffic and maintain distinct reputation profiles for each tenant. You can use tenant management to manage tenant-specific configuration sets that define how emails are processed and tracked for each tenant. You can associate email templates with specific tenants, confirms that branded communications remain properly segmented and controlled. This isolation helps ensure that one tenant’s actions cannot affect the reputation or performance of other tenants. When a tenant experiences delivery issues or reputation problems, these challenges remain contained within their dedicated resources. This approach maintains fairness across all tenants and establishes clear individual accountability for email practices.
- Monitor tenant-specific metrics in real time: You can access specific reputation metrics for each tenant, including raw bounce rates and complaint rates that directly impact sender reputation. You can use this system to set up tenant-level event destinations though the respective configuration set mapped to the tenant for detailed tracking and analysis. With this, you gain access to detailed tenant-level events that track performance, engagement, and compliance metrics for each tenant individually. You can also establish automated enforcement policies based on configurable thresholds that align with your business requirements. When tenant reputation findings are detected or when tenant status changes occur, you can receive real-time alerts through Amazon EventBridge.
- Scale to hundreds of thousands of tenants: The system is designed to handle massive scale (starting at 10 thousand tenants per account and can increase to as many as 300 thousand tenants), allowing you to grow your business or expand your email operations without worrying about infrastructure limitations. Whether you’re managing dozens or hundreds of thousands of tenants, the system will adapt to your needs.
- Automate tenant management workflows: You can set up automated processes for onboarding new tenants, applying policies, and managing tenant lifecycles. With this system, you can use API and console interfaces to create, modify, and delete tenants and have the flexibility to pause or resume sending capabilities as required. This automation reduces manual overhead for consistent application of your email sending standards across all tenants.
- Take targeted enforcement actions to maintain high deliverability: If issues arise with a specific tenant, you can take precise actions—such as suspending sending privileges or applying stricter reputation policies—without affecting other tenants. This capability helps maintain overall high deliverability rates for your entire operation.
These features collectively represent an advancement in email management capabilities, so organizations can offer more sophisticated, scalable, and reliable email services to their clients or internal departments while maintaining strict control over reputation and compliance.
How tenant management works
You can use the tenant management feature from Amazon SES to segment your email sending operations effectively. You can use the system to create multiple tenants within a single Amazon SES account, with each tenant having its own dedicated resources. These resources include essential components such as sending identities, SMTP credentials, configuration sets, and dedicated IP pools. What makes this architecture particularly flexible is the ability to share common resources across tenants, such as IP pools and configuration sets, enabling optimal resource utilization while maintaining operational separation. The following diagram illustrates the preceding information in detail.

Prerequisites
To get started with tenant management, you need:
- An AWS account
- Verified sending identities within Amazon SES (domains or email addresses)
- Configuration sets for email settings
- A clear understanding of your tenant structure based on your business requirements
How to set up tenant management and its key considerations
Setting up a multi-tenant system in Amazon SES requires careful configuration of three key components: IP pools, domain verification, and configuration sets. By following the set-up procedure, each tenant will have isolated resources, proper tracking, and monitoring capabilities. Using the AWS Management Console for Amazon SES or the Amazon SES APIs, you can create a robust email sending infrastructure that maintains high deliverability while keeping each tenant’s reputation separate.
IP pool configuration
IP pool configuration is a fundamental step to send email communications using Amazon SES. Begin your multi-tenant setup by establishing dedicated IP pools or managed IP pools for each customer though a configuration set. First, access the Amazon SES console and navigate to the Dedicated IP pools section. Create a new Standard dedicated IP pool, giving it a name that clearly identifies the customer. Through AWS Support, request the specific number of IP addresses needed based on your customer’s sending volume—typically one IP per 50,000 daily emails. After the IPs are provisioned, assign them to the appropriate pool. Then, map the IP pool with the configuration set mapped to the tenant. For IP warm-up, you have two options: enable the automatic warm-up schedule, which gradually increases sending volume over 45 days, or disable it to implement your own custom warm-up plan. Monitor the warm-up progress closely to help ensure optimal delivery rates.
Domain verification process
After setting up the IP pool, proceed with domain verification to establish your customer’s sending identity. Navigate to the “verified Identities” (verified identities are the domains or email ids those you have already whitelisted with Amazon SES) section in the Amazon SES console and create a new domain identity using your customer’s domain name. Amazon SES will provide DKIM records that need to be added to the domain’s DNS settings. Work with your customer to implement these records correctly in their DNS configuration. The verification process typically takes 24–72 hours to complete. During this time, regularly check the verification status in the Amazon SES console to make sure the process completes successfully.
Authentication and authorization for tenants
In addition to restricting email sending to specific identities and configurations, you can restrict email sending permissions by tenant using AWS Identity and Access Management (IAM) user or role policies. The following policy demonstrates these restrictions by allowing emails only when the tenant Amazon Resource Name (ARN) is arn:aws:ses:us-east-1:111122223333:tenant/testTenant1/tn-e08a68010000a3e4c67bcd990910, the identity is arn:aws:ses:us-east-1:111122223333:identity/example.com and the configuration-set is arn:aws:ses:us-east-1:111122223333:configuration-set/testTenant1.
Set up a configuration set
The final step involves creating and configuring the configuration set, which manages tracking and monitoring. Start by creating a new configuration set under configuration set section in the Amazon SES console, naming it to match your customer’s identification. Configure the custom tracking domain and enable appropriate tracking settings for opens and clicks. Link this configuration to the previously created IP pool. Next, set up event destinations to monitor email performance—this can include Amazon CloudWatch metrics, Amazon Data Firehose, or Amazon Simple Notification Service (Amazon SNS) topics. In CloudWatch, create alarms for critical metrics such as bounce rates (recommended threshold: 5%) and complaint rates (recommended threshold: 0.1%). Set up notification systems to alert your team when these thresholds are breached, so you can quickly respond to any delivery issues.
Sample CLI commands
To start using tenant management, you can use the console, AWS Command Line Interface (AWS CLI), or AWS SDKs. The following are basic examples of creating and configuring a tenant using the AWS CLI:
Following states a life cycle of the tenant management procedure starting from creating a tenant to deleting it in case you want to remove the tenant. Make sure that you are using AWS CLI version to 2.28.0 or later. See AWS CLI install and update instructions if necessary.
Create a new tenant
Assign a sending identity to the tenant (domain or email ID)
Add a configuration set to the tenant
The assumption here is that the selected configuration set already has an IP-Pool associated.
Get tenant information through get-tenants or list-tenants
You can use get-tenant or List-tenants to get information about a specific tenant, including the tenant’s name, ID, ARN, creation timestamp, tags, and sending status or list-tenants to list all tenants associated with your account
List resources of a tenant
Send email using tenant
To change the reputation policy from standard to strict (Standard policy is applied by default)
Disable sending for a tenant (to temporarily disable or pause a tenant)
Delete the tenant (remove the tenant completely from the Amazon SES account)
Send emails using SMTP
The X-SES-TENANT header is utilized by AWS to manage emails across multiple tenants. You can specify the tenant name by including it in the X-SES-TENANT field. This approach allows for better organization and routing of emails based on tenant information. To implement this, you can add the X-SES-TENANT header when sending emails using SMTP. The following Python code demonstrates how to include this header in your email sending process::
Email event feedback loop management for tenants
Receiving email events or using a feedback loop is important to monitor the email sending practices followed by the tenants. Tenant management provides reputation management capabilities for multi-tenant environments, so organizations can maintain granular control over email sending practices across their tenant base. You can automatically monitor and enforce reputation-based policies at the tenant level, so that problematic email sending behavior from one tenant doesn’t impact the deliverability of others. When reputation issues are detected, Amazon SES can automatically pause sending for the affected tenant while allowing other tenants to continue their email operations unimpeded.Organizations can now implement precise enforcement mechanisms through automated reputation findings that provide early detection of potential deliverability issues. Tenant isolation uses machine learning models and signal-based detection to identify problematic patterns in email sending behaviour. When issues are detected, Amazon SES automatically notifies the parent account and can trigger predetermined actions based on customizable thresholds. This granular control helps maintain strong deliverability rates across the entire email sending infrastructure while isolating and addressing issues at the tenant level.

Enforcement data and patterns
Unlike other communication channels that are governed by a patchwork of national laws, bulk email delivery is subject to requirements dictated by a handful of large inbox providers. Google, Yahoo, Microsoft, and several others set deliverability targets, leaving compliance up to the sender or service providers such as Amazon SES. Amazon SES, in turn, expects its direct customers, including multi-tenant providers, to monitor key signals of enforcement. If any of the tenants send rogue emails, Amazon SES expects the AWS customer to monitor key enforcement signals and take appropriate actions such as pausing or stopping the rogue tenants. Signals for enforcement and trust indicators are essential components of our email reputation management system. These signals are various data points and behaviours we monitor to assess the trustworthiness of email senders. Trust indicators, derived from these signals, provide a measure of a sender’s reputation and adherence to best practices. Amazon SES uses a combination of pre-send signals (such as account vetting and configuration) and post-send signals (including delivery success rates, bounce rates, and recipient engagement) to calculate reputation findings. These findings then inform automated enforcement actions and manual reviews, helping to ensure that our service maintains high deliverability standards while protecting recipients from unwanted or malicious emails. By continuously refining our signal analysis and enforcement processes, we strive to create a reliable and secure email ecosystem for all users.
Administrating tenant isolation and reputation management
When managing multiple tenants sending email through your SES account, you’ll want to monitor sending behaviour and reputation. Amazon SES provides a comprehensive monitoring system through reputation findings, which alert you when tenants exhibit concerning sending patterns. These findings appear in your dashboard and are delivered as events through Amazon EventBridge default event bus, letting you know immediately when issues arise.
As an email deferability administrator, your daily monitoring routine needs to include reviewing the tenant management dashboard where you can see all your tenants’ status at a glance. Pay particular attention to any reputation findings, which come in two levels—low and high severity. These findings indicate when tenants exceed acceptable thresholds for metrics like bounce rates or complaint rates. You can configure reputation policies to automatically pause tenant sending when these thresholds are breached, with options for standard enforcement (pausing on high severity findings) or strict enforcement (pausing on low severity findings).
When a tenant is paused, either automatically or manually, you’ll need to investigate the cause. The reputation findings provide detailed information about what triggered the pause, such as elevated bounce rates or complaint rates. After addressing the underlying issues with the tenant, you can reinstate their sending capabilities. During reinstatement, the tenant can continue sending while you monitor their metrics to verify that they return to healthy levels. After their metrics improve, the tenant will automatically transition back to a normal enabled status.
Available metrics and data points
These core reputation metrics are released by Amazon SES and can be routed to EventBridge. The event feedback loop will contain the tenant name and ID to enable tracking of tenant-specific bounce rates
- Complaint rates per tenant
- Third-party specific complaint rates
- Spamhaus IP listing status
- Email volume pattern
For ongoing management, you have full control over tenant resources and configurations. You can assign or remove sending identities and configuration sets as needed, adjust reputation policies, and manually pause sending if you observe concerning patterns. By using this combination of automated monitoring, clear reputation signals, and flexible management tools, you can maintain control over your tenants while preventing individual tenant issues from affecting your overall account reputation. The key is to stay proactive in monitoring the dashboard and reputation findings, and to act quickly when issues arise.
Conclusion
We’re excited to see how our customers will use the tenant management feature to transform their email operations, boost efficiency, and create better experiences for their users. To get started with tenant isolation simply visit the Amazon SES console or see Tenants in the Amazon SES Developer Guide. You can find details about pricing on the Amazon SES pricing page. We’re committed to improving tenant isolation and management based on your feedback and needs, and we look forward to bringing you even more powerful and flexible email management capabilities in the future. Start exploring multi-tenant management today with Amazon SES.
About the authors
Native NVIDIA support for AlmaLinux OS 9 and 10
Post Syndicated from jzb original https://lwn.net/Articles/1032753/
The AlmaLinux project has announced
the availability of packages to enable native NVIDIA driver support,
including CUDA and Secure Boot, for AlmaLinux 9 and 10.
When AlmaLinux started just 5 years ago, this wouldn’t have been
possible. With NVIDIA’s open source version of their graphics drivers
things have changed. This open source version is slowly becoming the
flagship driver, with new products being added exclusively to it. With
the help of some incredible people in the open source ecosystem and
the AlmaLinux community, we were able to do something that has yet to
be done in the EL ecosystem – ship Secure Boot signed, open source,
NVIDIA kernel modules.
Full documentation is available
on the AlmaLinux wiki.
Improving network observability with new AWS Outposts racks network metrics
Post Syndicated from Adam Duffield original https://aws.amazon.com/blogs/compute/improving-network-observability-with-new-aws-outposts-racks-network-metrics/
With AWS Outposts racks, you can extend AWS infrastructure, services, APIs, and tools to on-premises locations. Providing performant, stable, and resilient network connections to both the parent AWS Region as well as the local network is essential to maintaining uninterrupted service.
The release of two new Amazon CloudWatch metrics, VifConnectionStatus and VifBgpSessionState, gives you greater visibility into the operational status of the Outpost network connections. In this post, we discuss how to use these metrics to quickly identify network disruptions, using additional data points that can help reduce time to resolution.
Outposts network connectivity overview
When connecting an Outposts rack to your chosen data center location, network connections are made between the Outpost Networking Devices (ONDs) and Customer Network Devices (CNDs). These network connections support both the Service Link connectivity back to the chosen anchor Region and connectivity to the on-premises local network through the Local Gateway. First-generation Outposts racks include a minimum of two network devices to provide resilience, with second-generation Outposts racks including four network devices.
Virtual interfaces (VIFs) are used to establish IP network connectivity between the Outpost and CNDs, using Border Gateway Protocol (BGP) for dynamic routing. You can view the details for these VIFs on the Outposts console by choosing Link aggregation groups (LAGs) in the navigation pane and drilling down to find the specific service link and local gateway VIF information. For each connection between an OND and CND, two BGP sessions are established: one to support service link traffic and the other to support local gateway traffic.
The following diagram shows an example of this connectivity for a first-generation Outposts rack.
In this configuration, a total of four VIFs are configured into two link aggregation groups (LAGs): one on each OND for the service link and local gateway VIFs.
Understanding the new CloudWatch metrics for Outposts
Observability into the operational status of Outposts rack, including the status and performance of network connectivity, is important for you to be able to quickly identify and investigate potential issues. With the addition of the VifConnectionStatus and VifBgpSessionState Outposts metrics in CloudWatch, you have greater visibility into the connection status of the Outposts rack to your CNDs. The VifConnectionStatus metric is provided on a per-VIF level, available for both the local gateway and service link VIFs. It provides an indication on the status of the VIF using two possible values:
- A value of 1 indicates that the VIF is successfully connected to the CND with established BGP sessions and able to transmit traffic
- A value of 0 indicates that the VIF is not in an operational state due to an underlying issue
The VifBgpSessionState metric goes deeper into the BGP connectivity status between each Outposts VIF and CND. A BGP session can be in one of multiple states, each providing insight into where a potential issue might be. To reflect this, the CloudWatch metric value shown relates to the following BGP states:
- IDLE – The initial state; the ONDs are waiting for a start event
- Connect – The Outposts rack is waiting for the TCP connection to be complete
- Active – The Outposts rack is trying to initiate a TCP connection
- OpenSent – The router has sent an OPEN message and is waiting for a response
- OpenConfirm – The router has received an OPEN message and is waiting for a KEEPALIVE response
- Established – The BGP connection is fully established and the ONDs and CNDs can exchange routing information
With these metrics now available in CloudWatch, you can configure Amazon CloudWatch alarms to alert when the metric values indicate potential issues. You can combine existing CloudWatch metrics for Outposts racks with these new metrics to give additional context and visibility into network connectivity status.
Using CloudWatch metrics to investigate Outposts network connectivity issues
In the event of network connectivity issues, it’s important to understand how to use these metrics to assist with investigations and understand potential causes when seeing network impairment. To start with, the Configuration state of the VIFs should be checked. For each VIF, there are four possible states:
- Pending – A VIF is in this state from the time that it is created within a VIF group until the VIF becomes active on the OND
- Available – A VIF is active on ONDs
- Deleting – A VIF is in this state immediately after requesting deletion
- Deleted – A VIF is deleted
To check the state of an individual VIF on the Outposts console, choose Networking followed by Link aggregation groups (LAGS) in the navigation pane. The service link and local gateway VIFs associated with a specific LAG are shown, and when you choose a specific LAG, the configuration state of the associated VIFs are visible.
You can also retrieve these details programmatically. For example, use the following AWS Command Line Interface (AWS CLI) command to specifically check the configuration state of a service link VIF with ID sl-vif-087faf21db43ba723:
After confirming the Configuration state, you can use the VifConnectionStatus metric to determine the network connectivity status of individual VIFs. When operating and processing traffic in a healthy state, the value of this metric is 1. If this value changes to 0, it indicates a connectivity problem for that VIF between the Outpost and CNDs.
To further understand the potential cause of the VifConnectionStatus value, you can use the VifBgpSessionState metric. Under normal operational status, this metric value is 6, indicating that the BGP session is established and traffic can be sent and received. However, if this metric value changes to 1–5, then it is indicative of an issue. To start investigating the cause of this, you should review VIF configuration both on the Outposts console and programmatically. This includes the values set on the OND for VLAN, local and peer addresses, and BGP ASN. These values can be validated against the configuration on your on-premises CNDs if required. Furthermore, you can use the VifBgpSessionState metric value to determine the potential cause:
- If the value is 1, validate the values for BGP ASN and peer addresses
- If the value is 2, this might indicate port or IP address issues
- If the value is 3, this might indicate BGP version mismatches
- If the value is 4 or 5, this refers to networking path problems
By using a combination of these metrics, you can gain a clearer understanding of the potential network issue without having to engage with AWS or third-party support teams.
You can view and query these metrics on the CloudWatch console. In the navigation pane, choose All metrics, followed by Outposts under the AWS namespaces section. The Outposts namespace can only be viewed by the Outposts owner account, unless CloudWatch cross-account observability is configured. The new VifConnectionStatus and VifBgpSessionState metrics can be found under the OutpostsID, VirtualInterfaceGroupId, VirtualInterfaceId dimension.
For more information on working with metrics, see Metrics in Amazon CloudWatch. For creating alerts based upon these new metrics and their values, refer to Using Amazon CloudWatch alarms.
The resilient design of using multiple ONDs for both service link and local gateway traffic allows workloads to continue to run in the event of connectivity issues for single VIFs. For example, a single service link VIF might report as being down, but the remaining service link VIFs might be unaffected and remain available. In this scenario, the service link itself would remain functional and connected, albeit with potentially lower resilience and capacity. This can be validated throught the ConnectedStatus metric which would have a value of 1.
Conclusion
This post provided details on the newly released CloudWatch metrics for Outposts racks, VifConnectionStatus and VifBgpSessionState, and how you can use them to investigate potential connectivity issues. For more information on Outposts rack networking patterns, see the Networking section of the Outposts High Availability Design and Architecture Considerations whitepaper. For more information about additional CloudWatch metrics that are available, check out the CloudWatch metrics for AWS Outposts documentation for second-generation Outposts racks and first-generation Outposts racks.
Reach out to your AWS account team, or fill out this form to learn more about observability for Outposts.
Boosting search relevance: Automatic semantic enrichment in Amazon OpenSearch Serverless
Post Syndicated from Jon Handler original https://aws.amazon.com/blogs/big-data/boosting-search-relevance-automatic-semantic-enrichment-in-amazon-opensearch-serverless/
Traditional search engines rely on word-to-word matching (referred to as lexical search) to find results for queries. Although this works well for specific queries such as television model numbers, it struggles with more abstract searches. For example, when searching for “shoes for the beach,” a lexical search merely matches individual words “shoes,” “beach,” “for,” and “the” in catalog items, potentially missing relevant products like “water-resistant sandals” or “surf footwear” that don’t contain the exact search terms.
Large language models (LLMs) create dense vector embeddings for text that expand retrieval beyond individual word boundaries to include the context in which words are used. Dense vector embeddings capture the relationship between shoes and beaches by learning how often they occur together, enabling better retrieval for more abstract queries through what is called semantic search.
Sparse vectors combine the benefits of lexical and semantic search. The process starts with a WordPiece tokenizer to create a limited set of tokens from text. A transformer model then assigns weights to these tokens. During search, the system calculates the dot-product of the weights on the tokens (from the reduced set) from the query with tokens from the target document. You get a blended score from the terms (tokens) whose weights are high for both the query and the target. Sparse vectors encode semantic information, like dense vectors, and supply word-to-word matching through the dot-product, giving you a hybrid lexical-semantic match. For a detailed understanding of sparse and dense vector embeddings, visit Improving document retrieval with sparse semantic encoders in the OpenSearch blog.
Automatic semantic enrichment for Amazon OpenSearch Serverless makes implementing semantic search with sparse vectors effortless. You can now experiment with search relevance improvements and deploy to production with only a few clicks, requiring no long-term commitment or upfront investment. In this post, we show how automatic semantic enrichment removes friction and makes the implementation of semantic search for text data seamless, with step-by-step instructions to enhance your search functionality.
Automatic semantic enrichment
You could already enhance search relevance scoring beyond OpenSearch’s default lexical scoring with the Okapi BM25 algorithm, integrating dense vector and sparse vector models for semantic search using OpenSearch’s connector framework. However, implementing semantic search in OpenSearch Serverless has been complex and costly, requiring model selection, hosting, and integration with an OpenSearch Serverless collection.
Automatic semantic enrichment lets you automatically encode your text fields in your OpenSearch Serverless collections as sparse vectors by just setting the field type. During ingestion, OpenSearch Serverless automatically processes the data through a service-managed machine learning (ML) model, converting text to sparse vectors in native Lucene format.
Automatic semantic enrichment supports both English-only and multilingual options. The multilingual variant supports the following languages: Arabic, Bengali, Chinese, English, Finnish, French, Hindi, Indonesian, Japanese, Korean, Persian, Russian, Spanish, Swahili, and Telugu.
Model details and performance
Automatic semantic enrichment uses a service-managed, pre-trained sparse model that works effectively without requiring custom fine-tuning. The model analyzes the fields you specify, expanding them into sparse vectors based on learned associations from diverse training data. The expanded terms and their significance weights are stored in native Lucene index format for efficient retrieval. We’ve optimized this process using document-only mode, where encoding happens only during data ingestion. Search queries are merely tokenized rather than processed through the sparse model, making the solution both cost-effective and performant.
Our performance validation during feature development used the MS MARCO passage retrieval dataset, featuring passages averaging 334 characters. For relevance scoring, we measured average Normalized discounted cumulative gain (NDCG) for the first 10 search results (ndcg@10) on the BEIR benchmark for English content and average ndcg@10 on MIRACL for multilingual content. We assessed latency through client-side, 90th-percentile (p90) measurements and search response p90 took values. These benchmarks provide baseline performance indicators for both search relevance and response times.
The following table shows the automatic semantic enrichment benchmark.
| Language | Relevance improvement | P90 search latency |
| English | 20.0% over lexical search | 7.7% lower latency over lexical search (bm25 is 26 ms, and automatic semantic enrichment is 24 ms) |
| Multilingual | 105.1% over lexical search | 38.4% higher latency over lexical search (bm25 is 26 ms, and automatic semantic enrichment is 36 ms) |
Given the unique nature of each workload, we encourage you to evaluate this feature in your development environment using your own benchmarking criteria before making implementation decisions.
Pricing
OpenSearch Serverless bills automatic semantic enrichment based on OpenSearch Compute Units (OCUs) consumed during sparse vector generation at indexing time. You’re charged only for actual usage during indexing. You can monitor this consumption using the Amazon CloudWatch metric SemanticSearchOCU. For specific details about model token limits and volume throughput per OCU, visit Amazon OpenSearch Service Pricing.
Prerequisites
Before you create an automatic semantic enrichment index, verify that you’ve been granted the necessary permissions for the task. Contact an account administrator for assistance if required. To work with automatic semantic enrichment in OpenSearch Serverless, you need the account-level AWS Identity and Access Management (IAM) permissions shown in the following policy. The permissions serve the following purposes:
- The
aoss:*IndexIAM permissions is used to create and manage indices. - The
aoss:APIAccessAllIAM permission is used to perform OpenSearch API operations.
You also need an OpenSearch Serverless data access policy to create and manage Indices and associated resources in the collection. For more information, visit Data access control for Amazon OpenSearch Serverless in the OpenSearch Serverless Developer Guide. Use the following policy:
To access private collections, set up the following network policy:
Set up an automatic semantic enrichment index
To set up an automatic semantic enrichment index, follow these steps:
- To create an automatic semantic enrichment index using the AWS Command Line Interface (AWS CLI), use the create-index command:
- To describe the created index, use the following command:
You can also use AWS CloudFormation templates (Type: AWS::OpenSearchServerless::CollectionIndex) or the AWS Management Console to create semantic search during collection provisioning as well as after the collection is created.
Example: Index setup for product catalog search
This section shows how to set up a product catalog search index. You’ll implement semantic search on the title_semantic field (using an English model). For the product_id field, you’ll maintain default lexical search functionality.
In the following index-schema, the title_semantic field has a field type set to text and has parameter semantic_enrichment set to status ENABLED. Setting the semantic_enrichment parameter enables automatic semantic enrichment on the title_semantic field. You can use the language_options field to specify either english or multi-lingual. For this post, we generate a nonsemantic title field named title_non_semantic. Use the following code:
Data ingestion
After the index is created, you can ingest data through standard OpenSearch mechanisms, including client libraries, REST APIs, or directly through OpenSearch Dashboards. Here’s an example of how to add multiple documents using bulk API in OpenSearch Dashboards Dev Tools:
Search against automatic semantic enrichment index
After the data is ingested, you can query the index:
The following is the response:
The search successfully matched the document with Red shoes despite the query using crimson footwear, demonstrating the power of semantic search. The system automatically generated semantic embeddings for the document (truncated here for brevity) which enable these intelligent matches based on meaning rather than exact keywords.
Comparing search results
By running a similar query against the nonsemantic index title_non_semantic, you can confirm that nonsemantic fields can’t search based on context:
The following is the search response:
Limitations of automatic semantic enrichment
Automatic semantic search is most effective when applied to small-to-medium sized fields containing natural language content, such as movie titles, product descriptions, reviews, and summaries. Although semantic search enhances relevance for most use cases, it might not be optimal for certain scenarios:
- Very long documents – The current sparse model processes only the first 8,192 tokens of each document for English. For multilingual documents, it’s 512 tokens. For lengthy articles, consider implementing document chunking to ensure complete content processing.
- Log analysis workloads – Semantic enrichment significantly increases index size, which might be unnecessary for log analysis where exact matching typically suffices. The additional semantic context rarely improves log search effectiveness enough to justify the increased storage requirements.
Consider these limitations when deciding whether to implement automatic semantic enrichment for your specific use case.
Conclusion
Automatic semantic enrichment marks a significant advancement in making sophisticated search capabilities accessible to all OpenSearch Serverless users. By eliminating the traditional complexities of implementing semantic search, search developers can now enhance their search functionality with minimal effort and cost. Our feature supports multiple languages and collection types, with a pay-as-you-use pricing model that makes it economically viable for various use cases. Benchmark results are promising, particularly for English language searches, showing both improved relevance and reduced latency. However, although semantic search enhances most scenarios, certain use cases such as processing extremely long articles or log analysis might benefit from alternative approaches.
We encourage you to experiment with this feature and discover how it can optimize your search implementation so you can deliver better search experiences without the overhead of managing ML infrastructure. Check out the video and tech documentation for additional details.
About the Authors
Jon Handler is Director of Solutions Architecture for Search Services at Amazon Web Services, based in Palo Alto, CA. Jon works closely with OpenSearch and Amazon OpenSearch Service, providing help and guidance to a broad range of customers who have generative AI, search, and log analytics workloads for OpenSearch. Prior to joining AWS, Jon’s career as a software developer included four years of coding a large-scale, eCommerce search engine. Jon holds a Bachelor of the Arts from the University of Pennsylvania, and a Master of Science and a Ph. D. in Computer Science and Artificial Intelligence from Northwestern University.
Arjun Kumar Giri is a Principal Engineer at AWS working on the OpenSearch Project. He primarily works on OpenSearch’s artificial intelligence and machine learning (AI/ML) and semantic search features. He is passionate about AI, ML, and building scalable systems.
Siddhant Gupta is a Senior Product Manager (Technical) at AWS, spearheading AI innovation within the OpenSearch Project from Hyderabad, India. With a deep understanding of artificial intelligence and machine learning, Siddhant architects features that democratize advanced AI capabilities, enabling customers to harness the full potential of AI without requiring extensive technical expertise. His work seamlessly integrates cutting-edge AI technologies into scalable systems, bridging the gap between complex AI models and practical, user-friendly applications.
Almeida: a brief introduction on how GPU drivers work
Post Syndicated from corbet original https://lwn.net/Articles/1032744/
Daniel Almeida continues
his look at graphics drivers on the Collabora blog.
The starting point is to understand that a kernel-mode GPU driver
connects a much larger UMD (user-mode driver) to the actual
GPU. The UMD will actually implement APIs like Vulkan, OpenGL,
OpenCL, and others. These APIs, in turn, will be used by actual
programs to describe their workload to the GPU. This includes
allocating and using not only the geometry and textures, but also
the shaders being used to process said data into the final
result. This means that a key aspect of GPU drivers is actually
allocating GPU memory to house data related to the current scene
being drawn so that it can actually be operated on by the hardware.
How Closely Ukrainians Follow U.S. Politics
Post Syndicated from The Atlantic original https://www.youtube.com/shorts/c37ApaHGRrc
[$] Don’t fear the TPM
Post Syndicated from jzb original https://lwn.net/Articles/1032026/
There is a great deal of misunderstanding, and some misinformation, about the
Trusted
Platform Module (TPM); to combat this, Debian developer Jonathan
McDowell would like to clear the air and help users understand what it
is good for, as well as what it’s not. At DebConf25 in Brest, France,
he delivered a
talk about TPMs that explained what they are, why people might be
interested in using them, and how users might do so on a Debian
system.


