AWS Pi Day 2025: Data foundation for analytics and AI

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-pi-day-data-foundation-for-analytics-and-ai/

Every year on March 14 (3.14), AWS Pi Day highlights AWS innovations that help you manage and work with your data. What started in 2021 as a way to commemorate the fifteenth launch anniversary of Amazon Simple Storage Service (Amazon S3) has now grown into an event that highlights how cloud technologies are transforming data management, analytics, and AI.

This year, AWS Pi Day returns with a focus on accelerating analytics and AI innovation with a unified data foundation on AWS. The data landscape is undergoing a profound transformation as AI emerges in most enterprise strategies, with analytics and AI workloads increasingly converging around a lot of the same data and workflows. You need an easy way to access all your data and use all your preferred analytics and AI tools in a single integrated experience. This AWS Pi Day, we’re introducing a slate of new capabilities that help you build unified and integrated data experiences.

The next generation of Amazon SageMaker: The center of all your data, analytics, and AI
At re:Invent 2024, we introduced the next generation of Amazon SageMaker, the center of all your data, analytics, and AI. SageMaker includes virtually all the components you need for data exploration, preparation and integration, big data processing, fast SQL analytics, machine learning (ML) model development and training, and generative AI application development. With this new generation of Amazon SageMaker, SageMaker Lakehouse provides you with unified access to your data and SageMaker Catalog helps you to meet your governance and security requirements. You can read the launch blog post written by my colleague Antje to learn more details.

Core to the next generation of Amazon SageMaker is SageMaker Unified Studio, a single data and AI development environment where you can use all your data and tools for analytics and AI. SageMaker Unified Studio is now generally available.

SageMaker Unified Studio facilitates collaboration among data scientists, analysts, engineers, and developers as they work on data, analytics, AI workflows, and applications. It provides familiar tools from AWS analytics and artificial intelligence and machine learning (AI/ML) services, including data processing, SQL analytics, ML model development, and generative AI application development, into a single user experience.

SageMaker Unified Studio

SageMaker Unified Studio also brings selected capabilities from Amazon Bedrock into SageMaker. You can now rapidly prototype, customize, and share generative AI applications using foundation models (FMs) and advanced features such as Amazon Bedrock Knowledge Bases, Amazon Bedrock Guardrails, Amazon Bedrock Agents, and Amazon Bedrock Flows to create tailored solutions aligned with your requirements and responsible AI guidelines all within SageMaker.

Last but not least, Amazon Q Developer is now generally available in SageMaker Unified Studio. Amazon Q Developer provides generative AI powered assistance for data and AI development. It helps you with tasks like writing SQL queries, building extract, transform, and load (ETL) jobs, and troubleshooting, and is available in the Free tier and Pro tier for existing subscribers.

You can learn more about SageMaker Unified Studio in this recent blog post written by my colleague Donnie.

During re:Invent 2024, we also launched Amazon SageMaker Lakehouse as part of the next generation of SageMaker. SageMaker Lakehouse unifies all your data across Amazon S3 data lakes, Amazon Redshift data warehouses, and third-party and federated data sources. It helps you build powerful analytics and AI/ML applications on a single copy of your data. SageMaker Lakehouse gives you the flexibility to access and query your data in-place with Apache Iceberg–compatible tools and engines. In addition, zero-ETL integrations automate the process of bringing data into SageMaker Lakehouse from AWS data sources such as Amazon Aurora or Amazon DynamoDB and from applications such as Salesforce, Facebook Ads, Instagram Ads, ServiceNow, SAP, Zendesk, and Zoho CRM. The full list of integrations is available in the SageMaker Lakehouse FAQ.

Building a data foundation with Amazon S3
Building a data foundation is the cornerstone of accelerating analytics and AI workloads, enabling organizations to seamlessly manage, discover, and utilize their data assets at any scale. Amazon S3 is the world’s best place to build a data lake, with virtually unlimited scale, and it provides the essential foundation for this transformation.

I’m always astonished to learn about the scale at which we operate Amazon S3: It currently holds over 400 trillion objects, exabytes of data, and processes a mind-blowing 150 million requests per second. Just a decade ago, not even 100 customers were storing more than a petabyte (PB) of data on S3. Today, thousands of customers have surpassed the 1 PB milestone.

Amazon S3 stores exabytes of tabular data, and it averages over 15 million requests to tabular data per second. To help you reduce the undifferentiated heavy lifting when managing your tabular data in S3 buckets, we announced Amazon S3 Tables at AWS re:Invent 2024. S3 Tables are the first cloud object store with built-in support for Apache Iceberg. S3 tables are specifically optimized for analytics workloads, resulting in up to threefold faster query throughput and up to tenfold higher transactions per second compared to self-managed tables.

Today, we’re announcing the general availability of Amazon S3 Tables integration with Amazon SageMaker Lakehouse  Amazon S3 Tables now integrate with Amazon SageMaker Lakehouse, making it easy for you to access S3 Tables from AWS analytics services such as Amazon Redshift, Amazon Athena, Amazon EMR, AWS Glue, and Apache Iceberg–compatible engines such as Apache Spark or PyIceberg. SageMaker Lakehouse enables centralized management of fine-grained data access permissions for S3 Tables and other sources and consistently applies them across all engines.

For those of you who use a third-party catalog, have a custom catalog implementation, or only need basic read and write access to tabular data in a single table bucket, we’ve added new APIs that are compatible with the Iceberg REST Catalog standard. This enables any Iceberg-compatible application to seamlessly create, update, list, and delete tables in an S3 table bucket. For unified data management across all of your tabular data, data governance, and fine-grained access controls, you can also use S3 Tables with SageMaker Lakehouse.

To help you access S3 Tables, we’ve launched updates in the AWS Management Console. You can now create a table, populate it with data, and query it directly from the S3 console using Amazon Athena, making it easier to get started and analyze data in S3 table buckets.

The following screenshot shows how to access Athena directly from the S3 console.

S3 console : create table with AthenaWhen I select Query tables with Athena or Create table with Athena, it opens the Athena console on the correct data source, catalog, and database.

S3 Tables in Athena

Since re:Invent 2024, we’ve continued to add new capabilities to S3 Tables at a rapid pace. For example, we added schema definition support to the CreateTable API and you can now create up to 10,000 tables in an S3 table bucket. We also launched S3 Tables into eight additional AWS Regions, with the most recent being Asia Pacific (Seoul, Singapore, Sydney) on March 4, with more to come. You can refer to the S3 Tables AWS Regions page of the documentation to get the list of the eleven Regions where S3 Tables are available today.

Amazon S3 Metadata—announced during re:Invent 2024— has been generally available since January 27. It’s the fastest and easiest way to help you discover and understand your S3 data with automated, effortlessly-queried metadata that updates in near real time. S3 Metadata works with S3 object tags. Tags help you logically group data for a variety of reasons, such as to apply IAM policies to provide fine-grained access, specify tag-based filters to manage object lifecycle rules, and selectively replicate data to another Region. In Regions where S3 Metadata is available, you can capture and query custom metadata that is stored as object tags. To reduce the cost associated with object tags when using S3 Metadata, Amazon S3 reduced pricing for S3 object tagging by 35 percent in all Regions, making it cheaper to use custom metadata.

AWS Pi Day 2025
Over the years, AWS Pi Day has showcased major milestones in cloud storage and data analytics. This year, the AWS Pi Day virtual event will feature a range of topics designed for developers and technical decision-makers, data engineers, AI/ML practitioners, and IT leaders. Key highlights include deep dives, live demos, and expert sessions on all the services and capabilities I discussed in this post.

By attending this event, you’ll learn how you can accelerate your analytics and AI innovation. You’ll learn how you can use S3 Tables with native Apache Iceberg support and S3 Metadata to build scalable data lakes that serve both traditional analytics and emerging AI/ML workloads. You’ll also discover the next generation of Amazon SageMaker, the center for all your data, analytics, and AI, to help your teams collaborate and build faster from a unified studio, using familiar AWS tools with access to all your data whether it’s stored in data lakes, data warehouses, or third-party or federated data sources.

For those looking to stay ahead of the latest cloud trends, AWS Pi Day 2025 is an event you can’t miss. Whether you’re building data lakehouses, training AI models, building generative AI applications, or optimizing analytics workloads, the insights shared will help you maximize the value of your data.

Tune in today and explore the latest in cloud data innovation. Don’t miss the opportunity to engage with AWS experts, partners, and customers shaping the future of data, analytics, and AI.

If you missed the virtual event on March 14, you can visit the event page at any time—we will keep all the content available on-demand there!

— seb


How is the News Blog doing? Take this 1 minute survey!

(This survey is hosted by an external company. AWS handles your information as described in the AWS Privacy Notice. AWS will own the data gathered via this survey and will not share the information collected with survey respondents.)

[$] The burden of knowledge: dealing with open-source risks

Post Syndicated from jzb original https://lwn.net/Articles/1013614/

Organizations relying on open-source software have a wide range of
tools, scorecards, and methodologies to try to assess security, legal,
and other risks inherent in
their so-called supply chain. However, Max Mehl argued
recently
in a short talk at FOSS Backstage in Berlin (and
online) that all of
this objective information and data is insufficient to truly
understand and address risk. Worse, this information doesn’t provide
options to improve the situation and encourages a passive mindset. Mehl, who works as part of
the CTO group at DB Systel, encouraged better risk assessment using
qualitative data and direct participation in open source.

Unlocking MSSP Success: Why CTEM is Critical

Post Syndicated from Rapid7 original https://blog.rapid7.com/2025/03/14/unlocking-mssp-success-why-ctem-is-critical/

Co-authored by Thomas Green and Sid Nanda

What is Continuous Threat Exposure Management (CTEM)?

Unlocking MSSP Success: Why CTEM is Critical

Continuous Threat Exposure Management (CTEM) is a five-stage, continuous security program introduced by Gartner in 2022. It proactively assesses an organization’s exposure across networks, systems, cloud infrastructure, IoT devices, applications, and identities. Unlike traditional vulnerability assessments, CTEM prioritizes risk mitigation strategies and iteratively refines security postures through continuous validation and remediation.

By emphasizing offensive security techniques such as continuous red teaming and simulation-based testing, CTEM goes beyond basic vulnerability prioritization to identify and address weaknesses before adversaries can exploit them. The result is an adaptable, intelligence-driven security framework that enables organizations to transition from reactive defenses to proactive resilience.

Why Should Service Providers Care?

The MSSP market is increasingly competitive, and differentiation is critical. Gartner has identified CTEM as a key opportunity for MSSPs in 2024, emphasizing that “product leaders who differentiate their portfolios by offering services that result in prioritized remediation outcomes and measured reduction in exposure stand out in the crowded MSSP market.”

Moreover, the expansion of the attack surface – combined with the inability to patch all vulnerabilities in a timely manner -has made traditional vulnerability management insufficient. MSSPs must demonstrate how their services deliver tangible security benefits to customers. A well-structured CTEM program enables MSSPs to provide a continuous, data-driven security validation framework that reduces risk while aligning stakeholders across IT, security, and leadership teams.

CTEM: A Programmatic Approach

Unlike traditional security programs that rely on static tools, CTEM is an adaptive program that integrates offensive security methodologies, including red teaming, penetration testing, vulnerability management, cloud security posture management, and web application security. This holistic approach ensures alignment with Governance, Risk, and Compliance (GRC) initiatives while delivering continuous security improvements.

The Five Stages of a CTEM Program

1. Scoping: Defining the Attack Surface

Scoping, as defined by Gartner, involves identifying an organization’s complete attack surface. This is where Surface Command, a key component of the Exposure Command suite, provides critical visibility by combining Cyber Asset Attack Surface Management (CAASM) with External Attack Surface Management (EASM). Unlike point-in-time assessments, Surface Command offers continuous monitoring of assets across on-prem, cloud, and SaaS environments, ensuring that no shadow IT or misconfigured exposure goes undetected.

2. Discovery: Mapping Assets, Vulnerabilities, and Risks

Discovery involves identifying both known and hidden assets, vulnerabilities, and misconfigurations. A common pitfall is confusing scoping with discovery—simply identifying a large number of vulnerabilities does not equate to security success.

Rapid7 Exposure Command enhances the discovery phase by integrating InsightVM for vulnerability management, InsightAppSec for dynamic application security testing (DAST), and InsightCloudSec for cloud security posture management (CSPM). These tools work together to provide comprehensive visibility intoexposure across hybrid environments.

3. Prioritization: Focusing on What Matters

Not all security issues require immediate remediation. Effective prioritization should factor in:

  • Business risk and potential impact
  • Urgency and exploitability
  • Availability of compensating controls
  • Tolerance for residual attack surface

Rapid7 Exposure Command’s risk-based prioritization framework goes beyond CVSS scoring by incorporating real-world exploitability data, asset criticality, and threat intelligence. The Command Platform provides a unified view of risks and remediation priorities, enabling MSSPs to help customers focus on the most impactful security improvements.

4. Validation: Proving Security Effectiveness

Validation is the cornerstone of CTEM. Organizations must confirm that vulnerabilities are exploitable, understand potential attack paths, and assess the effectiveness of security controls.

MSSPs can use continuous red teaming, penetration testing, and adversary simulations to validate security postures. Additionally, Security Information and Event Management (SIEM) solutions provide real-time threat correlation, ensuring organizations can detect and respond to threats before they escalate.

5. Mobilization: Operationalizing Security Improvements

The final stage of CTEM is mobilizing findings into actionable security improvements. This involves:

  • Streamlining approval workflows for remediation
  • Automating patch management and configuration changes
  • Ensuring alignment between IT, security, and executive teams

Rapid7 Exposure Command facilitates mobilization by providing automation and orchestration capabilities, reducing friction in vulnerability remediation processes. By integrating with existing IT workflows, MSSPs can ensure that security enhancements are implemented efficiently and effectively.

Achieving a Secure Environment with CTEM

The burden of threat management continues to grow as attack surfaces expand and adversaries evolve. MSSPs must help organizations move beyond traditional vulnerability management to a continuous, risk-driven security approach.

By leveraging ExposureCommand, MSSPs can:

  • Provide continuous visibility into evolving threats and vulnerabilities
  • Enable proactive risk mitigation through prioritized remediation
  • Validate security effectiveness through ongoing testing and adversary simulations
  • Streamline remediation efforts with automation and orchestration

CTEM is not just a security strategy—it’s a key differentiator for MSSPs. By embedding CTEM into their service offerings, MSSPs can deliver measurable risk reduction, enhance customer trust, and solidify their role as strategic security partners.

Learn More about Rapid7’s Exposure Command ▶︎

Confidently identify and prioritize exposures from endpoint to cloud with full attack surface visibility and threat-aware risk context.

Security updates for Friday

Post Syndicated from daroc original https://lwn.net/Articles/1014183/

Security updates have been issued by Fedora (iniparser, thunderbird, trafficserver, and xorg-x11-server), Mageia (opensc), Oracle (.NET 8.0, .NET 9.0, gcc, kernel, and libxml2), Red Hat (firefox, grub2, and krb5), Slackware (libxslt), SUSE (amazon-ssm-agent, bsdtar, build, ffmpeg-4, forgejo-runner, kernel, python, python3, python313, rubygem-rack-1_6, and tailscale), and Ubuntu (linux-azure, linux-azure-5.15, linux-azure-fde, linux-azure-fde-5.15).

TP-Link Router Botnet

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2025/03/tp-link-router-botnet.html

There is a new botnet that is infecting TP-Link routers:

The botnet can lead to command injection which then makes remote code execution (RCE) possible so that the malware can spread itself across the internet automatically. This high severity security flaw (tracked as CVE-2023-1389) has also been used to spread other malware families as far back as April 2023 when it was used in the Mirai botnet malware attacks. The flaw also linked to the Condi and AndroxGh0st malware attacks.

[…]

Of the thousands of infected devices, the majority of them are concentrated in Brazil, Poland, the United Kingdom, Bulgaria and Turkey; with the botnet targeting manufacturing, medical/healthcare, services and technology organizations in the United States, Australia, China and Mexico.

Details.

Властта се люлее, „Величие“ зрее

Post Syndicated from Емилия Милчева original https://www.toest.bg/vlastta-se-lyulee-velichie-zree/

Властта се люлее, „Величие“ зрее

В 51-вия парламент влиза девети политически субект, със 121 депутати мнозинството става особено чупливо, а антиевропейските сили – по-мощни. Конституционният съд (КС) най-сетне се произнесе – на петия месец от образуването на делото за нарушенията на изборите на 27 октомври 2024 г., които бившият служебен премиер Димитър Главчев определи като „честни“ и като добре свършена работа. 

Въпреки това КС не касира изборите като цяло. А и нито една от петте политически сили, оспорили резултатите от вота миналата есен – „Величие“, БСП, „Възраждане“, „Има такъв народ“ и ДПС на Ахмед Доган, не повдигна темата за нови избори заради безобразията, констатирани от вещите лица на съда.

След проверката на 2204 секции – 17,06% от всички 12 920 в страната – бяха открити разминавания между действителните и записаните в протоколите гласове в близо половината от проверените – 982. Освен това се оказа, че 780 бюлетини „са изчезнали“. ДПС – Ново начало на санкционирания за корупция олигарх Делян Пеевски загуби най-много гласове след повторното преброяване от експертите на КС – 1110 гласа, следвана от ГЕРБ с 452, „Възраждане“ – 108, и БСП – Обединена левица – 42. При останалите формации разликите са в плюс – „Има такъв народ“ имат 31 гласа повече, МЕЧ – 28, ПП–ДБ – 11 гласа, а Алиансът за права и свободи на Доган – 1.

Партията на Ивелин Михайлов „Величие“ получава 10 депутати. Тя се озова вътре „благодарение“ на липсващи 780 бюлетини от 7 секции, след като на 27 октомври преброителите установиха, че не ѝ достигат 21 гласа. За първи път в най-новата история на България беше преизчислена 4-процентовата бариера, вече на база по-малко действителни гласове – тя е 97 390,44, а след новото преброяване „Величие“ получава 97 497, тоест с 59 гласа над нея. Това е записано в решението на КС, а на Централната избирателна комисия (ЦИК) е възложено да съобщи за преразпределянето на мандатите.

По БНТ Ивелин Михайлов коментира, че по-всяка вероятност няма да се бърза, и го обясни с договорки между политическите сили.

Ще има едно бавене, докато се приеме бюджетът. 

Но ЦИК го опроверга, защото два часа по-късно обяви имената на новите народни представители, в това число и на депутатите от партията му.

А с решението си КС обяви за незаконен избора на 16 депутати. Трима са от ГЕРБ – Александър Ненков, Павлин Йотов и Теменужка Петкова, на чието място, след като стана министър, влезе Никола Джамбазов; трима са от ИТН – Емил Трифонов, Иван Кючуков и Александър Марков; двама от ДПС – Ново начало – Станислав Анастасов и Метин Качан; както и Ешереф Ешереф от АПС, Ивайло Митковски от ПП–ДБ и Андрей Вълчев от БСП.

Решението на КС беше предшествано от скандал и парламентарен трибунал за експертите на КС, членовете на ЦИК и представителите на „Информационно обслужване“. Законодателите, създали калпавия Изборен кодекс, нападаха и обиждаха, без да поемат отговорност за определените от тях секционни избирателни комисии, допуснали нарушенията и заверили ги с подпис. От ГЕРБ, ДПС и БСП не се покаяха, че обезсмислиха гласуването с машини (превърнати на практика в принтери), връщайки хартиените бюлетини, които са сред причините за нарушенията, наред с некомпетентните и неефективни СИК. Атаката бе насочена към КС, а резултатът – напълно безсмислена декларация с призив за прозрачност, внесена от самата председателка на парламента Наталия Киселова (БСП–ОЛ) и подкрепена от ГЕРБ–СДС, БСП–ОЛ, ИТН и ДПС – Ново начало.

Констатираните от КС нарушения потвърдиха онова, с което обществото е наясно, но никоя власт не признава – изборите в демократична България са една голяма мръсна работа. А институции като прокуратурата и МВР я охраняват.

Омърсената демокрация

КС изброи доста безобразия, извършени при гласуването на 27 октомври и констатирани в проверените секции. Гласове за една партия са вписани като гласове за друга. Други пък изобщо липсват в протоколите. Действителни гласове са записвани като „не подкрепям никого“, а белязани бюлетини са били броени като действителни. 

Графичната експертиза, възложена от КС, е установила, че в три от изборните райони множество бюлетини са де факто пуснати от един и същи човек, защото са направени по калъп. Но онова, което излезе наяве, надгради известните изборни свинщини – от чувалите с изборни книжа, които секционните избирателни комисии предават на общинските администрации, за да ги пазят в запечатани помещения до следващите избори, липсват бюлетини. В чувалите обаче са поставени канцеларски материали, което показва, че този, който е извадил и унищожил бюлетините, е искал да замаскира престъплението. Експертите на КС са вписали 0 в графите при повторното преброяване – няма бюлетина, няма гласуване, все едно какво показва видеонаблюдението.

А то, както се оказа, не само че не дисциплинира, а на места и го няма – например в район, където доминира ДПС – Ново начало – в селата Огняново, Гърмен и Белица. Министърът на електронното управление Валентин Мундров вече призова за законодателна промяна, която да позволи видеозаснемането от изборните секции да се използва като доказателство в съда, ако се разследват нарушения при броенето. Дали е чут, ще стане ясно при разглеждане на предложенията за Изборния кодекс.

За изчезналите бюлетини публично съобщи още в началото на годината вещото лице проф. Красимир Костадинов, но прокуратурата предпочете да не се самосезира въпреки неговите интервюта. Което не ѝ попречи да се опита да блокира работата на КС, след като изпълнителният директор на „Информационно обслужване“ АД Ивайло Филипов я сезира за липсващите бюлетини. (Той е един от петимата акционери – физически лица и членове на Съвета на директорите, останалите 99,95% от капитала са на Министерството на електронното управление.) Натискът върху Конституционния съд при неприключило производство стана повод председателката му Павлина Панова, която кара своя втори мандат като ръководител на КС, да направи за първи път публично изявление.

Считам за необходимо изрично да подчертая, че единствено компетентният съгласно Конституцията на Република България орган, който може да извърши преценка на законността на изборите за народни представители, е Конституционният съд. Всички държавни органи са длъжни да му оказват пълно съдействие, а не да възпрепятстват дейността му. Недопустимо е дейността на Конституционния съд да бъде поставена в зависимост от действия или бездействия на други държавни органи.

В разстояние на три дни президентът Румен Радев на два пъти коментира скандала: единия път – за да каже, че се върви отвъд демокрацията и има „опит на задкулисието да погълне държавата“, а втория – че се застъпва за машинното гласуване.

Ние сме свидетели на неграмотност на членове на секционни избирателни комисии, така че този въпрос с начина на гласуване, с неговото отчитане трябва кардинално да се промени. И наистина машинното гласуване сто процента е един от начините да се справим с този проблем и да се върне доверието в изборите.

А на протеста, организиран от „Правосъдие за всеки“, срещу натиска върху КС говори и адвокат Иван Демерджиев, бивш вътрешен министър и бивш министър на правосъдието в два служебни кабинета на президента и свързан с „Дондуков“ 2.

Застрашава ли „Величие“ властта?

В 50-тия парламент „Величие“ влезе с 13 депутати и се разпадна на 16-тия ден. Сега обаче идеологът на партията Ивелин Михайлов, добре известен на гражданството, за разлика от формалния председател Албена Пекова, ще е народен представител. Това може да закрепи единството на групата за известно време, но едва ли задълго.

„Величие“ се е заявила като опозиция. Но с девет политически сили парламентът става твърде фрагментиран, а мнозинството, което е на санитарния минимум от 121 депутати, ще трябва да търси подкрепа от други партии при отсъствие на някой от „неговите“ народни представители. 

Изглежда, че подкрепа има в аванс, още преди да е поискана. Пеевски вече обяви, че е на разположение и ще крепи правителството, защото така трябва и заради благото на народа. Дали от благодарност, или от друго, но парламентарната Бюджетна комисия с председател Делян Добрев (ГЕРБ) одобри законопроекта му за държавна верига магазини, която ще продава храни с надценка до 10%, управлявана от търговско дружество със 100% държавно участие и принципал Министерството на земеделието и храните. Популистката идея няма устойчива икономическа логика, но е предвиден бюджет от 10 млн. лв.

Партията на Ивелин Михайлов влиза в парламента в момент, в който започва наддаването за членове на регулатори и контролни органи, и неминуемо ще се включи в пазарлъците. Това ще затрудни постигането на разбирателство, което и без това е трудно, и ще качи цената на по-малките партньори на ГЕРБ в управлението. След като бяха открити процедурите, вече се върви към изслушвания на кандидатите за Комисията за защита на конкуренцията, за омбудсман, Конституционен съд, шеф на НЗОК…

Също като „Възраждане“, и „Величие“ е срещу еврото и против помощ за Украйна под каквато и да е форма. Аргументът за първото е досущ като по-ранните на „Възраждане“ – България не е готова, а за Украйна – бедни сме да помагаме. Но Еврогрупата не смята така, след като насърчи България да продължи по пътя си към единната европейска валута.

България бележи напредък в плана си да въведе еврото през 2026 г. и поиска от Европейската комисия и Европейската централна банка да изготвят оценка за нейната конвергенция с еврозоната. 

Засега хоризонтът на властта е до края на годината, както предрече по-рано и лидерът на ГЕРБ Бойко Борисов.

Демокрацията продължава да се дави в мръсната вода на изборите.

Знаем ли достатъчно за човешкия папиломавирус (HPV)?

Post Syndicated from original https://www.toest.bg/znaem-li-dostatuchno-za-choveshkiya-papiloma-virus-hpv/

Знаем ли достатъчно за човешкия папиломавирус (HPV)?

Международният ден за информираност за човешкия папиломавирус (HPV) се чества на 4 март от 2018 г. насам. Целта на Международното общество по папиломавирус (International Papillomavirus Society) е да информира за инфекциите с HPV и за подхода за превенция срещу HPV инфекциите и заболяванията, свързани с тях.

HPV от древни времена до днес

Човешкият папиломавирус е изключително стабилен, с ниска мутационна честота и най-вероятно съществува, откакто съществува и човечеството. Сред най-ранните идентифицирания на вирусна инфекция с HPV е на мумифицираното тяло на древния египтянин Накхт отпреди 3000 години. Аутопсията на неговото тяло през 1974 г. показва наличие на брадавици на стъпалата му. Брадавиците са детайлно описани в класическата гръцка и римска медицинска литература.

През 1951 г. е създадена известната „безсмъртна“ клетъчна линия HeLa (клетки, отглеждани в инвитро среда в петри) от пациентката Хенриета Лакс, диагностицирана с рак на маточната шийка на 30-годишна възраст. Благодарение на нея е установена пряката връзка между рака на маточната шийка и HPV, разработена е и ваксина срещу полиовирус.

През 50-те и 60-те години на XX в. развитието на имуноонкологията бележи голям напредък с въвеждането на концепцията, че имунната система разпознава свои и чужди антигени. Имунолозите сър Франк Макфарлън Бърнет и Питър Медъуор са удостоени с Нобелова награда за медицина през 1960 г. за откриването на „придобития имунологичен толеранс“.

През 1983 г., благодарение на развитието на ДНК технологиите, учените установяват, че инфекцията с HPV води до развитието на рак на маточната шийка.

Структура и жизнен цикъл на вируса

Папиломните вируси са малки вируси, изградени от капсид и геном от двойноверижна ДНК. Открити са при няколко гръбначни животни, включително примати, кучета, зайци и говеда, но са строго специфични за гостоприемника, т.е. инфекцията не може да се пренася между видовете. 

Човешкият папиломен вирус (HPV) е най-разпространеният вирус, предаван по полов път в света. HPV заразява само епителните клетки на кожата или лигавиците. Понастоящем е известно, че има над 200 вида HPV, като различните видове причиняват различни заболявания – от тривиална брадавица до рак на маточната шийка. 

HPV е прост вирус, чиито гени кодират само осем протеина, два от които (L1 и L2) образуват външния протеинов капсид. Различнитe типове HPV се различават по три определени генни последователности (E6, E7 и L1), които варират с най-малко 10%.

Онкогенните характеристики на HPV се дължат на онкопротеините E6 и E7, които потискат действието на туморсупресорните белтъци p53 и pRB. Около 5% от всички ракови заболявания в световен мащаб се дължат главно на т.нар. високорискови HPV типове 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66 и 68. По-голямата част от HPV инфекциите не причиняват симптоми или заболяване и се изчистват в рамките на 12–24 месеца след заразяването, най-често след сексуален контакт. Малък процент от тези инфекции персистират или прогресират до пренеопластична лезия, която води до развитието на раково заболяване.

За започване на онкогенния процес, на първо място, е необходимо HPV инфекцията да се превърне в хронична. Вирусният товар (количеството вирус в дадена телесна течност) и типът на вируса са основните съпровождащи фактори за прогресия на инфекцията в заболяване. Рискови фактори за развитието на рак при инфекция с HPV са тютюнопушене, хормонален дисбаланс, както и съпровождаща инфекция с ХИВ вируса. За да се избегнат рисковете за здравето, които HPV инфекцията води със себе си, съществуват превантивни мерки, като скрининг и ваксинация.

Превенция и лечение

През 2006 г. Агенцията за храните и лекарствата на Съединените щати одобрява първата превантивна ваксинация срещу HPV – четиривалентната ваксина с активност срещу най-високорисковите HPV типове 16, 18, отговорни за развитието на над 70% от раковите заболявания на маточната шийка, и типове 6 и 11, отговорни за 90% от гениталните брадавици. Впоследствие тази и друга двувалентна ваксина са лицензирани в поне 80 страни. Държавите, прилагащи национални програми за ваксиниране, са 140 към ноември 2023 г. 

В България Министерството на здравеопазването осигурява безплатна ваксина срещу HPV за момичета на възраст 10–13 г. в рамките на Националната програма за първична профилактика на рака на маточната шийка 2021–2024 г. От 2026 г. в рамките на националната програма за ваксинация ще могат безплатно да бъдат ваксинирани и момчета. Имунизация срещу човешки папиломавирус се препоръчва за лица от 9- до 26-годишна възраст. При наличие на показания може да се ваксинират и жени до 45 г.

Бъдеще

С увеличаването на нивата на профилактично ваксиниране срещу HPV сред общото население се очаква нивата на HPV инфекциите да спаднат. Наличните терапии за рак на маточната шийка, свързани с HPV инфекция, обикновено действат върху локални генитални брадавици и не съществува златен стандарт за лечението. Освен това изборът на терапия зависи от тежестта на заболяването, вида на HPV инфекцията и предпочитанията на пациента. 

Въпреки че напредъкът в разработването на целенасочена терапевтична ваксина срещу HPV е бавен, насоките за подобряване на ваксината са свързани с повишаване на ефекта ѝ, т.е. с повишаване на имуногенността на отговора на Т-клетките на имунната система. Ваксините на базата на пептиди и протеини от вируса са стабилни, безопасни и лесни за производство, а базираните на ДНК ваксини имат продължителна експресия на антигени, свързани с човешките левкоцитни антигени (HLA). Базираните на РНК ваксини също имат засилена експресия на антигени. Тези характеристики на терапевтичните ваксини предоставят възможности за по-ефективна и по-малко токсична стратегия за лечение.

Към момента са необходими още клинични изпитвания, които да оценят страничните ефекти, как се определя точната доза и кои тъкани да таргетират ваксините, които са с цел терапия. Проучването и изясняването на молекулярната биология на тумора може да благоприятстват разработването на по-ефективни терапевтични ваксини. Освен това са необходими допълнителни клинични изпитвания за подобряване на ефикасността на терапевтичните ваксини. 

По отношение на лечението системата за редактиране на генома CRISPR/Cas може да допълни настоящите подходи, като хирургия, химиотерапия и лъчева терапия. Има и няколко усъвършенствани метода, например CAR-T терапия и радиоимунотерапия, която показва обещаващи резултати.

Accelerate analytics and AI innovation with the next generation of Amazon SageMaker

Post Syndicated from G2 Krishnamoorthy original https://aws.amazon.com/blogs/big-data/accelerate-analytics-and-ai-innovation-with-the-next-generation-of-amazon-sagemaker/

At AWS re:Invent 2024, we announced the next generation of Amazon SageMaker, the center for all your data, analytics, and AI. Amazon SageMaker brings together widely adopted AWS machine learning (ML) and analytics capabilities and addresses the challenges of harnessing organizational data for analytics and AI through unified access to tools and data with governance built in. It enables teams to securely find, prepare, and collaborate on data assets and build analytics and AI applications through a single experience, accelerating the path from data to value.

At the core of the next generation of Amazon SageMaker is Amazon SageMaker Unified Studio, a single data and AI development environment where you can find and access your organization’s data and act on it using the best tool for the job across virtually any use case. We are excited to announce the general availability of SageMaker Unified Studio.

In this post, we explore the benefits of SageMaker Unified Studio and how to get started.

Benefits of SageMaker Unified Studio

SageMaker Unified Studio brings together the functionality and tools from existing AWS Analytics and AI/ML services, including Amazon EMR, AWS Glue, Amazon Athena, Amazon Redshift, Amazon Bedrock, and Amazon SageMaker AI. From within the unified studio, you can discover data and AI assets from across your organization, then work together in projects to securely build and share analytics and AI artifacts, including data, models, and generative AI applications. Governance features including fine-grained access control are built into SageMaker Unified Studio using Amazon SageMaker Catalog to help you meet enterprise security requirements across your entire data estate.

Unified access to your data is provided by Amazon SageMaker Lakehouse, a unified, open, and secure data lakehouse built on Apache Iceberg open standards. Whether your data is stored in Amazon Simple Storage Service (Amazon S3) data lakes, Redshift data warehouses, or third-party and federated data sources, you can access it from one place and use it with Iceberg-compatible engines and tools. In addition, SageMaker Lakehouse now integrates with Amazon S3 Tables, the first cloud object store with native Apache Iceberg support, so you can use SageMaker Lakehouse to create, query, and process S3 Tables efficiently using various analytics engines in SageMaker Unified Studio as well as Iceberg-compatible engines like Apache Spark and PyIceberg.

Capabilities from Amazon Bedrock are now generally available in SageMaker Unified Studio, allowing you to rapidly prototype, customize, and share generative AI applications in a governed environment. Users have an intuitive interface to access high-performing foundation models (FMs) in Amazon Bedrock, including the Amazon Nova model series, and the ability to create Agents, Flows, Knowledge Bases, and Guardrails with a few clicks.

Amazon Q Developer, the most capable generative AI assistant for software development, can be used within SageMaker Unified Studio to streamline tasks across the data and AI development lifecycle, including code authoring, SQL generation, data discovery, and troubleshooting.

A new integrated way of working

The general availability of SageMaker Unified Studio represents another meaningful step in our journey to offer our customers a streamlined way to work with their data, whether for analytics or AI. Many of our customers have told us that you are building data-driven applications to guide business decisions, improve agility, and drive innovation, but that these applications are complex to build because they require collaboration across teams and the integration of data and tools. Not only is it time consuming for users to learn multiple development experiences, but because data, code, and other development artifacts are stored separately, it is challenging for users to understand how they interact with each other and to use them cohesively. Configuring and governing access is also a cumbersome manual process. To overcome these hurdles, many organizations are building bespoke integrations between services, tools, and homegrown access management systems. However, what you need is the flexibility to adopt the best services for your use case while empowering your data teams with a unified development experience.

“When we build data-driven applications for our customers, we want a unified platform where the technologies work together in an integrated way. Amazon SageMaker Unified Studio streamlines our solution delivery processes through comprehensive analytics capabilities, a unified studio experience, and a lakehouse that integrates data management across data warehouses and data lakes. Amazon SageMaker Unified Studio reduces the time-to-value for our customers’ data projects by up to 40%, helping us with our mission to accelerate our customers’ digital transformation journey.”

—Akihiro Suzue, Head of Solutions Sector, NTT DATA; Yuji Shono, Senior Manager, Apps & Data Technology Department, NTT DATA; Yuki Saito, Manager, Digital Success Solutions Division, NTT DATA

Millions of organizations trust AWS and utilize our comprehensive set of purpose-built analytics, AI/ML, and generative AI capabilities to power data-driven applications without compromising on performance, scale, or cost. Our goal for the next generation of Amazon SageMaker, including SageMaker Unified Studio, is to make data and AI workers more productive by providing access to all your data and tools in a single development environment.

Building from a single data and AI development environment

Let’s explore a common business challenge: increasing revenue through better lead generation. Consider an organization implementing an intelligent digital assistant on their website to engage with customers—a process that traditionally requires multiple tools and data sources. With SageMaker Unified Studio, this entire process can now be carried out within a single data and AI development environment.

First, the data team uses the generative AI playground within SageMaker Unified Studio to quickly evaluate and select the best model for their customer interactions. They then create a project to house the tools and resources necessary for their use case and use Amazon Bedrock within the project to build and deploy a sophisticated virtual assistant that quickly begins qualifying leads through their website.

To identify the most promising opportunities, the team develops a segmentation strategy. The data engineer asks Amazon Q Developer to identify datasets that contain lead data and uses zero-ETL integrations to bring the data into SageMaker Lakehouse. The data analyst then discovers it and creates a comprehensive view of their market. They use the SQL query editor to build out marketing segments, which they then write back to SageMaker Lakehouse, where they are available to other team members.

Finally, the data scientist accesses the same dataset, which they use to train and deploy an automated lead scoring model using tools available from SageMaker AI. During the model development phase, they use Amazon Q Developer’s inline code authoring and troubleshooting capabilities to efficiently write error free-code in their JupyterLab notebook. The final model provides sales teams with the highest-value opportunities, which they can visualize in a business intelligence dashboard and take action on immediately.

Reducing time-to-value in a unified environment

What is remarkable about this example is that entire process happens in one integrated environment. Without SageMaker Unified Studio, the team would have had to work with multiple data sources, tools, and services, spending time learning multiple development environments, creating resources shares, and manually configuring access controls. The data engineer and data analyst would have worked in various data warehouses, data lakes, and analytics tools, the data scientist would have worked in an ML studio and notebook environment, and the application builder in a generative AI tool. Now, they’re able to build and collaborate with their data and tools available in one experience, dramatically reducing time-to-value.

That’s why we’re so excited about the next generation of Amazon SageMaker and the general availability of SageMaker Unified Studio. We believe that by putting everything you need for analytics and AI in one place, you can solve complex end-to-end problems more efficiently and get to innovative outcomes faster than ever before.

Getting started with SageMaker Unified Studio

To learn more, check out the following resources:


About the authors

G2 Krishnamoorthy is VP of Analytics, leading AWS data lake services, data integration, Amazon OpenSearch Service, and Amazon QuickSight. Prior to his current role, G2 built and ran the Analytics and ML Platform at Facebook/Meta, and built various parts of the SQL Server database, Azure Analytics, and Azure ML at Microsoft.

Rahul Pathak is VP of Relational Database Engines, leading Amazon Aurora, Amazon Redshift, and Amazon QLDB. Prior to his current role, he was VP of Analytics at AWS, where he worked across the entire AWS database portfolio. He has co-founded two companies, one focused on digital media analytics and the other on IP-geolocation.

Collaborate and build faster with Amazon SageMaker Unified Studio, now generally available

Post Syndicated from Donnie Prakoso original https://aws.amazon.com/blogs/aws/collaborate-and-build-faster-with-amazon-sagemaker-unified-studio-now-generally-available/

Today, we’re announcing the general availability of Amazon SageMaker Unified Studio, a single data and AI development environment where you can find and access all of the data in your organization and act on it using the best tool for the job across virtually any use case. Introduced as preview during AWS re:Invent 2024, my colleague, Antje, summarized it as:

SageMaker Unified Studio (preview) is a single data and AI development environment. It brings together functionality and tools from the range of standalone “studios,” query editors, and visual tools that we have today in Amazon Athena, Amazon EMR, AWS Glue, Amazon Redshift, Amazon Managed Workflows for Apache Airflow (Amazon MWAA), and the existing SageMaker Studio.

Here’s a video to see Amazon SageMaker Unified Studio in action:

SageMaker Unified Studio breaks down silos in data and tools, giving data engineers, data scientists, data analysts, ML developers and other data practitioners a single development experience. This saves development time and simplifies access control management so data practitioners can focus on what really matters to them—building data products and AI applications.

This post focuses on several important announcements that we’re excited to share:

To get started, go to the Amazon SageMaker console and create a SageMaker Unified Studio domain. To learn more, visit Create an Amazon SageMaker Unified Studio domain in the AWS documentation.

New capabilities for Amazon Bedrock in SageMaker Unified Studio
The capabilities of Amazon Bedrock within Amazon SageMaker Unified Studio offer a governed collaborative environment for developers to rapidly create and customize generative AI applications. This intuitive interface caters to developers of all skill levels, providing seamless access to the high-performance FMs offered in Amazon Bedrock and advanced customization tools for collaborative development of tailored generative AI applications.

Since the preview launch, several new FMs have become available in Amazon Bedrock and are fully integrated with SageMaker Unified Studio, including Anthropic’s Claude 3.7 Sonnet and DeepSeek-R1. These models can be used for building generative AI apps and chatting in the playground in SageMaker Unified Studio.

Here’s how you can choose Anthropic’s Claude 3.7 Sonnet on the model selection in your project.

You can also source data or documents from S3 folders within your project and select specific FMs when creating knowledge bases. 

During preview, we introduced Amazon Bedrock Guardrails to help you implement safeguards for your Amazon Bedrock application based on your use cases and responsible AI policies. Now, Amazon Bedrock Guardrails is extended to Amazon Bedrock Flows with this general availability release.

Additionally, we have streamlined generative AI setup for associated accounts with a new user management interface in SageMaker Unified Studio, making it straightforward for domain administrators to grant associated account admins access to model governance projects. This enhancement eliminates the need for command line operations, streamlining the process of configuring generative AI capabilities across multiple AWS accounts.

These new features eliminate barriers between data, tools, and builders in the generative AI development process. You and your team will gain a unified development experience by incorporating the powerful generative AI capabilities of Amazon Bedrock — all within the same workspace.

Amazon Q Developer is now generally available in SageMaker Unified Studio
Amazon Q Developer is now generally available in Amazon SageMaker Unified Studio, providing data professionals with generative AI–powered assistance across the entire data and AI development lifecycle.

Amazon Q Developer integrates with the full suite of AWS analytics and AI/ML tools and services within SageMaker Unified Studio, including data processing, SQL analytics, machine learning model development, and generative AI application development, to accelerate collaboration and help teams build data and AI products faster. To get started, you can select Amazon Q Developer icon.

For new users of SageMaker Unified Studio, Amazon Q Developer serves as an invaluable onboarding assistant. It can explain core concepts such as domains and projects, provide guidance on setting up environments, and answer your questions.

Amazon Q Developer helps you discover and understand data using powerful natural language interactions with SageMaker Catalog. What makes this implementation particularly powerful is how Amazon Q Developer combines broad knowledge of AWS analytics and AI/ML services with the user’s context to provide personalized guidance.

You can chat about your data assets through a conversational interface, asking questions such as “Show all payment related datasets” without needing to navigate complex metadata structures.

Amazon Q Developer offers SQL query generation through its integration with the built-in query editor available in SageMaker Unified Studio. Data professionals of varying skill levels can now express their analytical needs in natural language, receiving properly formatted SQL queries in return.

For example, you can ask, “Analyze payment method preferences by age group and region” and Amazon Q Developer will generate the appropriate SQL with proper joins across multiple tables.

Additionally, Amazon Q Developer is also available to assist with troubleshooting and generating real-time code suggestions in SageMaker Unified Studio Jupyter notebooks, as well as building ETL jobs.

Now available

  • Availability — Amazon SageMaker Unified Studio is now available in the following AWS Regions: US East (N. Virginia, Ohio), US West (Oregon), Asia Pacific (Seoul, Singapore, Sydney, Tokyo), Canada (Central), Europe (Frankfurt, Ireland, London), South America (São Paulo). Learn more about the availability of these capabilities on supported Region documentation page.
  • Amazon Q Developer subscription — The free tier of Amazon Q Developer is available by default in SageMaker Unified Studio, requiring no additional setup or configuration. If you already have Amazon Q Developer Pro Tier subscriptions, you can use those enhanced capabilities within the SageMaker Unified Studio environment. For more information, visit the documentation page.
  • Amazon Bedrock capabilities — To learn more about the capabilities of Amazon Bedrock in Amazon SageMaker Unified Studio, refer to this documentation page. 

Start building with Amazon SageMaker Unified Studio today. For more information, visit the Amazon SageMaker Unified Studio page.

Happy building!

— Donnie Prakoso

— How is the News Blog doing? Take this 1 minute survey! (This survey is hosted by an external company. AWS handles your information as described in the AWS Privacy Notice. AWS will own the data gathered via this survey and will not share the information collected with survey respondents.)

Amazon S3 Tables integration with Amazon SageMaker Lakehouse is now generally available

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/amazon-s3-tables-integration-with-amazon-sagemaker-lakehouse-is-now-generally-available/

At re:Invent 2024, we launched Amazon S3 Tables, the first cloud object store with built-in Apache Iceberg support to streamline storing tabular data at scale, and Amazon SageMaker Lakehouse to simplify analytics and AI with a unified, open, and secure data lakehouse. We also previewed S3 Tables integration with Amazon Web Services (AWS) analytics services for you to stream, query, and visualize S3 Tables data using Amazon Athena, Amazon Data Firehose, Amazon EMR, AWS Glue, Amazon Redshift, and Amazon QuickSight.

Our customers wanted to simplify the management and optimization of their Apache Iceberg storage, which led to the development of S3 Tables. They were simultaneously working to break down data silos that impede analytics collaboration and insight generation using the SageMaker Lakehouse. When paired with S3 Tables and SageMaker Lakehouse in addition to built-in integration with AWS analytics services, they can gain a comprehensive platform unifying access to multiple data sources enabling both analytics and machine learning (ML) workflows.

Today, we’re announcing the general availability of Amazon S3 Tables integration with Amazon SageMaker Lakehouse to provide unified S3 Tables data access across various analytics engines and tools. You can access SageMaker Lakehouse from Amazon SageMaker Unified Studio, a single data and AI development environment that brings together functionality and tools from AWS analytics and AI/ML services. All S3 tables data integrated with SageMaker Lakehouse can be queried from SageMaker Unified Studio and engines such as Amazon Athena, Amazon EMR, Amazon Redshift, and Apache Iceberg-compatible engines like Apache Spark or PyIceberg.

With this integration, you can simplify building secure analytic workflows where you can read and write to S3 Tables and join with data in Amazon Redshift data warehouses and third-party and federated data sources, such as Amazon DynamoDB or PostgreSQL.

You can also centrally set up and manage fine-grained access permissions on the data in S3 Tables along with other data in the SageMaker Lakehouse and consistently apply them across all analytics and query engines.

S3 Tables integration with SageMaker Lakehouse in action
To get started, go to the Amazon S3 console and choose Table buckets from the navigation pane and select Enable integration to access table buckets from AWS analytics services.

Now you can create your table bucket to integrate with SageMaker Lakehouse. To learn more, visit Getting started with S3 Tables in the AWS documentation.

1. Create a table with Amazon Athena in the Amazon S3 console
You can create a table, populate it with data, and query it directly from the Amazon S3 console using Amazon Athena with just a few steps. Select a table bucket and select Create table with Athena, or you can select an existing table and select Query table with Athena.

2. Create tables with Athena

When you want to create a table with Athena, you should first specify a namespace for your table. The namespace in an S3 table bucket is equivalent to a database in AWS Glue, and you use the table namespace as the database in your Athena queries.

Choose a namespace and select Create table with Athena. It goes to the Query editor in the Athena console. You can create a table in your S3 table bucket or query data in the table.

2. Query with Athena

2. Query with SageMaker Lakehouse in the SageMaker Unified Studio
Now you can access unified data across S3 data lakes, Redshift data warehouses, third-party and federated data sources in SageMaker Lakehouse directly from SageMaker Unified Studio.

To get started, go to the SageMaker console and create a SageMaker Unified Studio domain and project using a sample project profile: Data Analytics and AI-ML model development. To learn more, visit Create an Amazon SageMaker Unified Studio domain in the AWS documentation.

After the project is created, navigate to the project overview and scroll down to project details to note down the project role Amazon Resource Name (ARN).

3. Project details in SageMaker Unified Studio

Go to the AWS Lake Formation console and grant permissions for AWS Identity and Access Management (IAM) users and roles. In the in the Principals section, select the <project role ARN> noted in the previous paragraph. Choose Named Data Catalog resources in the LF-Tags or catalog resources section and select the table bucket name you created for Catalogs. To learn more, visit Overview of Lake Formation permissions in the AWS documentation.

4. Grant permissions in Lake Formation console

When you return to SageMaker Unified Studio, you can see your table bucket project under Lakehouse in the Data menu in the left navigation pane of project page. When you choose Actions, you can select how to query your table bucket data in Amazon Athena, Amazon Redshift, or JupyterLab Notebook.

5. S3 Tables in Unified Studio

When you choose Query with Athena, it automatically goes to Query Editor to run data query language (DQL) and data manipulation language (DML) queries on S3 tables using Athena.

Here is a sample query using Athena:

select * from "s3tablecatalog/s3tables-integblog-bucket”.”proddb"."customer" limit 10;

6. Athena query in Unified Studio

To query with Amazon Redshift, you should set up Amazon Redshift Serverless compute resources for data query analysis. And then you choose Query with Redshift and run SQL in the Query Editor. If you want to use JupyterLab Notebook, you should create a new JupyterLab space in Amazon EMR Serverless.

3. Join data from other sources with S3 Tables data
With S3 Tables data now available in SageMaker Lakehouse, you can join it with data from data warehouses, online transaction processing (OLTP) sources like relational or non-relational database, Iceberg tables, and other third party sources to gain more comprehensive and deeper insights.

For example, you can add connections to data sources such as Amazon DocumentDB, Amazon DynamoDB, Amazon Redshift, PostgreSQL, MySQL, Google BigQuery, or Snowflake and combine data using SQL without extract, transform, and load (ETL) scripts.

Now you can run the SQL query in the Query editor to join the data in the S3 Tables with the data in the DynamoDB.

Here is a sample query to join between Athena and DynamoDB:

select * from "s3tablescatalog/s3tables-integblog-bucket"."blogdb"."customer", 
              "dynamodb1"."default"."customer_ddb" where cust_id=pid limit 10;

To learn more about this integration, visit Amazon S3 Tables integration with Amazon SageMaker Lakehouse in the AWS documentation.

Now available
S3 Tables integration with SageMaker Lakehouse is now generally available in all AWS Regions where S3 Tables are available. To learn more, visit the S3 Tables product page and the SageMaker Lakehouse page.

Give S3 Tables a try in the SageMaker Unified Studio today and send feedback to AWS re:Post for Amazon S3 and AWS re:Post for Amazon SageMaker or through your usual AWS Support contacts.

In the annual celebration of the launch of Amazon S3, we will introduce more awesome launches for Amazon S3 and Amazon SageMaker. To learn more, join the AWS Pi Day event on March 14.

— Channy

—

How is the News Blog doing? Take this 1 minute survey!

(This survey is hosted by an external company. AWS handles your information as described in the AWS Privacy Notice. AWS will own the data gathered via this survey and will not share the information collected with survey respondents.)

Announcing end-of-support for Amazon Kinesis Client Library 1.x and Amazon Kinesis Producer Library 0.x effective January 30, 2026

Post Syndicated from Minu Hong original https://aws.amazon.com/blogs/big-data/announcing-end-of-support-for-amazon-kinesis-client-library-1-x-and-amazon-kinesis-producer-library-0-x-effective-january-30-2026/

Amazon Kinesis Client Library (KCL) 1.x and Amazon Kinesis Producer Library (KPL) 0.x will reach end-of-support on January 30, 2026. Accordingly, these versions will enter maintenance mode on April 17, 2025. During maintenance mode, AWS will provide updates only for critical bug fixes and security issues. Major versions in maintenance mode will not receive updates for new features or feature enhancements.

KCL is a software library that simplifies building applications to process streaming data from Amazon Kinesis Data Streams. KCL handles complex tasks such as load balancing across multiple consumer workers, responding to worker failures, checkpointing processed records, and adapting to throughput changes.

KPL is a software library that helps developers write producer applications to achieve high throughput data into Kinesis Data Streams. KPL manages record batching, aggregation, and retry logic to maximize throughput and optimize resource usage while simplifying the producer application development process.

The following table outlines the level of support for each phase of the major version lifecycle of KCL and KPL.

Major versions Version Lifecycle Phase Start Date End Date Support Level
KCL 1.x General Availability 12/19/2013 4/16/2025 During this phase, the major version is fully supported. AWS provides regular minor and patch version releases that include support for new features or API updates for Kinesis Data Streams, as well as bug and security fixes.
KPL 0.x General Availability 6/2/2015 4/16/2025 During this phase, the major version is fully supported. AWS provides regular minor and patch version releases that include support for new features or API updates for Kinesis Data Streams, as well as bug and security fixes.
KCL 1.x, KPL 0.x Maintenance mode 4/17/2025 1/29/2026 AWS is limiting patch version releases to address critical bug fixes and security issues only. The major version will not receive updates for new features or APIs of Kinesis Data Streams.
KCL 1.x, KPL 0.x End-of-support 1/30/2026 N/A The major version will no longer receive updates or releases. Previously published releases will continue to be available through public package managers and the code will remain on GitHub.

Impact on non-Java KCL Libraries

The maintenance mode and end-of-support dates apply to the following non-Java KCL 1.x libraries:

Migrating from KCL 1.x or KPL 0.x

If you’re using KCL 1.x or KPL 0.x, we recommend migrating to the latest versions (KCL 3.x and KPL 1.x). When migrating from KCL 1.x to 3.x, you will need to update interfaces and security credential providers in your application. For migrations from KPL 0.x to 1.x, you can upgrade your current KPL application without any change in your data processing logic. To learn more, refer to the following resources:

Summary

KCL 1.x and KPL 0.x will reach end-of-support on January 30, 2026 and enter maintenance mode on April 17, 2025. If you need assistance or have feedback, reach out to AWS support. You can also open an issue in the KCL GitHub repository or KPL GitHub repository.


About the author

Minu Hong is a Senior Product Manager for Amazon Kinesis Data Streams at AWS. He is passionate about understanding customer challenges around streaming data and developing optimized solutions for them. Outside of work, Minu enjoys traveling, playing tennis, skiing, and cooking.

Deploy real-time analytics with StarTree for managed Apache Pinot on AWS

Post Syndicated from Raj Ramasubbu original https://aws.amazon.com/blogs/big-data/deploy-real-time-analytics-with-startree-for-managed-apache-pinot-on-aws/

This post is cowritten with Mayank Shrivastava and Barkha Herman from StarTree.

Building a low-latency, high-concurrency, real-time online analytical processing (OLAP) solution has been previously explored on the AWS Big Data Blog, where we walked through how to build a real-time analytics solution with Apache Pinot on AWS, in which streaming sources, such as Amazon Managed Streaming for Apache Kafka (Amazon MSK) and Amazon Kinesis Data Streams, produce events that are ingested and processed in real time within Apache Pinot.

However, this approach requires self-management of the infrastructure required to run Pinot, as well as a number of manual processes to run in production. StarTree is a managed alternative that offers similar benefits for real-time analytics use cases.

In this post, we introduce StarTree as a managed solution on AWS for teams seeking the advantages of Pinot. We highlight the key distinctions between open-source Pinot and StarTree, and provide valuable insights for organizations considering a more streamlined approach to their real-time analytics infrastructure.

By examining these aspects, you can make an informed decision between open source Pinot and StarTree for your specific real-time analytics needs.

StarTree overview

One of the founders of Apache Pinot, Kishore Gopalakrishna, launched StarTree to equip organizations globally with the power of real-time data and build a fully managed platform for real-time analytics. Handling over 1 billion queries per week and ingesting over 1 million events per second, StarTree Cloud removes the burden of infrastructure management so companies can focus on delivering real-time insights to end-users.

Open source Pinot requires in-house expertise that can challenge well-established technical teams to provision hardware, configure environments, tune performance, maintain security, adhere to data governance requirements, manage software updates, and constantly monitor for system issues. Organizations interested in decreasing their time to value with a managed Pinot solution can take advantage of the expertise of StarTree’s team to accelerate setup, deploy an architecture ready for scale, and offload infrastructure maintenance.

Improving security with SOC 2, SSO, and RBAC

Critical enterprise security features can be challenging to implement in open source Pinot environments. With StarTree’s managed Pinot, role-based access control (RBAC) simplifies administration for Pinot and allows organizations to assign and monitor user access based on roles to enforce secure and efficient access to sensitive data. StarTree Cloud provides enterprise-grade security with SOC 2 compliance, enhanced encryption, and single sign-on (SSO) capabilities.

Using automated data ingestion at scale

The minion task framework is a native component of Pinot to offload computationally intensive tasks away from the other Pinot components to conserve resources for low-latency queries and support real-time stream ingestion. StarTree can handle larger volumes of data efficiently with highly scalable implementations of minion tasks and a minion auto scaling feature that eliminates unnecessary infrastructure costs during idle times, as seen in the below figure.

StarTree’s automatic data ingestion framework is ideal for enterprise workloads because it improves scalability and reduces the data maintenance complexity often found in open source Pinot deployments. StarTree supports a large number of managed connectors, which are used to maintain metadata about the source and ingest data seamlessly into the platform. The data is then modelled to help you organize and structure the data fetched from the selected data source into Pinot tables. Indexes are then configured to optimize query performance, as per the flow in the diagram below.

Tiered storage for real-time query processing

With open source Pinot, tiered storage can be used for deep storage like Amazon Simple Storage Service (Amazon S3) for backup but not query processing, because storage is tightly coupled with compute and requires manual configuration of tenants with different storage speeds and server specifications. In the following diagram, an Amazon S3 tier is defined for the data to be moved from tightly coupled SSD to cloud storage when the data is 30 days old.

 

On the other hand, StarTree transitions less-frequently accessed data to cost-effective storage like Amazon S3, while maintaining quick access to frequently accessed data. StarTree’s tiered storage enables automation for real-time query processing with index pinning, prefetching, and intelligent data movement between hot and cold storage, optimizing both performance and cost. StarTree’s sophisticated approach to tiered storage is highly flexible and reduces replication overhead by keeping a single copy in cloud storage, which prevents the limitations of compressed deep store copies, as you can see in the below diagram

Improving scalability with off-heap upserts

Companies like Amberdata benefit from StarTree’s upsert support to routinely upsert 350,000 events per second, with peak workloads reaching 1 million upserts per second. StarTree Cloud enhanced upsert functionality boosts efficiency, usability, and scalability through the implementation of off-heap upserts. Behind the scenes, Pinot servers manage specific upsert metadata to determine if a newly inserted record’s primary key was previously encountered and identifies the current segment holding it. As shown below, StarTree Cloud moves this off-heap, enabling a scalable cache of metadata as the on-heap memory restrictions are removed

Customer success stories using Pinot with StarTree for real-time analytics

The following customers highlight their success using Pinot for StarTree:

Flexible deployment options for StarTree Cloud

StarTree offers multiple deployment options, including a StarTree hosted software as a service (SaaS) or customer hosted SaaS. StarTree hosted SaaS is ideal for organizations interested in fully offloading the operational burden of infrastructure management, scaling, performance tuning, and security from their team so they can focus on analytics. StarTree’s customer hosted SaaS provides flexibility for customers interested in deploying the solution within their AWS environment or other platform of choice. This is suitable for organizations who require higher infrastructure management controls in their perimeter but still want the operational ease of a managed service.

Self-managed Pinot or StarTree

Pinot can deliver value for real-time analytics scenarios with different deployment methods. The choice of deployment method will come down to organizational priorities and trade-offs. Teams with the capability and willingness to manage open source software on a commodity infrastructure at scale might opt to deploy self-managed Pinot on AWS. Teams interested in reducing time troubleshooting performance bottlenecks, optimizing resource usage, and minimizing downtime can use StarTree’s managed service.

Conclusion

In this post, we presented StarTree as a managed solution on AWS for teams seeking the advantages of Apache Pinot. Like Pinot, StarTree addresses the need for a low-latency, high-concurrency, real-time online analytical processing (OLAP) solution. In addition, StarTree offers a managed experience for real-time and batch Pinot workloads, offering enhanced security, automated data ingestion, tiered storage, and off-heap upserts. These features improve security, scalability, and manageablity for organizations looking to run Pinot in production.

Developers interested in learning more about managed Pinot can deploy real-time analytics with StarTree to test it out or join a session with StarTree’s head of product. StarTree is an AWS ISVA partner and is available on AWS Marketplace.


About the Authors

Raj Ramasubbu is a Senior Analytics Specialist Solutions Architect focused on big data and analytics and AI/ML with Amazon Web Services. He helps customers architect and build highly scalable, performant, and secure cloud-based solutions on AWS. Raj provided technical expertise and leadership in building data engineering, big data analytics, business intelligence, and data science solutions for over 18 years prior to joining AWS. He helped customers in various industry verticals like healthcare, medical devices, life science, retail, asset management, car insurance, residential REIT, agriculture, title insurance, supply chain, document management, and real estate.

Francisco Morillo is a Streaming Solutions Architect at AWS. Francisco works with AWS customers, helping them design real-time analytics architectures using AWS services, supporting Amazon Managed Streaming for Apache Kafka (Amazon MSK) and Amazon Managed Service for Apache Flink.

Ismail Makhlouf is a Senior Specialist Solutions Architect for Data Analytics at AWS. Ismail focuses on architecting solutions for organizations across their end-to-end data analytics estate, including batch and real-time streaming, big data, data warehousing, and data lake workloads. He primarily partners with airlines, manufacturers, and retail organizations to support them to achieve their business objectives with well-architected data platforms.

Renee Berry is a Senior Partner Development Manager with the AWS Global Startup Program, working with venture backed startups partnering with AWS to scale their growth.

Mayank Shrivastava is a founding engineer of Apache Pinot and a PMC member for the project. He is currently a Fellow at StarTree Inc., where he also heads their Center of Excellence.

Barkha Herman is a technologist and developer advocate who founded WiTVoices and South Florida Women in Tech. She fosters inclusive tech communities.

Secure cloud innovation starts at re:Inforce 2025

Post Syndicated from Chris Betz original https://aws.amazon.com/blogs/security/secure-cloud-innovation-starts-at-reinforce-2025/

Every day, I talk with security leaders who are navigating a critical balancing act. On one side, their organizations are moving faster than ever, adopting transformative technologies like generative AI and expanding their cloud footprint. On the other, they’re working to maintain strong security controls and visibility across an increasingly complex landscape. We all know that adding more tools and controls isn’t sustainable. We need a different approach to security at scale.

re:Inforce 2025: Your roadmap to security that powers innovation

This is what shaped our vision for AWS re:Inforce 2025. When done right, security at scale becomes a business accelerator, helping organizations move faster and more confidently in the cloud. This is more than just a philosophy; it’s a practical reality I’ve seen proven time and again by our customers, and it’s what we want to help every organization achieve.

At re:Inforce, we’ll share a vision for simplifying security at scale that’s deeply rooted in our experiences supporting millions of customers worldwide. We’ll explore how organizations are building inherently resilient applications that can withstand modern threats while accelerating innovation. I’m particularly excited to showcase real customer examples and architectural patterns that demonstrate how security better enables your business goals.

An environment built for learning cloud security

There’s a reason we created re:Inforce as a dedicated in-person security event. While I love our broader AWS events, security practitioners need space to dive deep into implementation details, ask tough questions, and work through complex scenarios. At re:Inforce, you can grab a whiteboard with the engineers who built our security services, collaborate with security partners, and schedule personal time with our leaders to tackle your specific security needs. It’s the kind of environment where real learning happens.

We’ve designed multiple learning paths to meet you wherever you are in your security journey. With over 250 technical sessions, you’ll find content that matches your needs – whether you’re looking to automate security controls, align development and security teams, or transform your security operations. You’ll find interactive workshops where you’ll build solutions in real-time, small-group technical deep-dives, hands-on labs where you can test new approaches, and solution-building sessions with AWS experts. Best of all, 70% of our content is at advanced or expert level, making sure you get the detailed implementation guidance you need.

I invite you to join us for three days that will transform how you think about and implement security in the cloud. Registration is now open, and I encourage you to secure your spot early—based on previous years, spots will fill up quickly. Join us to explore how simplified, scalable cloud security can fuel your organization’s future. Register today with the code SECBLObhZzr9 to receive a limited time $300 USD discount, while supplies last.

If you have feedback about this post, submit comments in the Comments section below.

Chris Betz

Chris is CISO at AWS. He oversees security teams and leads the development and implementation of security policies with the aim of managing risk and aligning the company’s security posture with business objectives. Chris joined Amazon in August 2023 after holding CISO and security leadership roles at leading companies. He lives in Northern Virginia with his family.

How to Register for a United States (US) SMS Short Code with AWS End User Messaging

Post Syndicated from Tyler Holmes original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-register-for-a-united-states-us-sms-short-code-with-aws-end-user-messaging/

Obtaining and using SMS short codes in the United States requires a thorough understanding of the detailed application process and strict requirements set forth by mobile carriers. This comprehensive guide walks through the step-by-step procedure for registering a US SMS short code from AWS End User Messaging, which provides SMS capabilities to all AWS services.

This guide covers the process from initial creation of a support case, the short code application form with all necessary details and documentation, and the multi-stage provisioning process involving carrier review and approvals. Particular emphasis is placed on establishing compliant opt-in workflows, crafting SMS-specific terms and privacy policies, and preparing the required message templates – all of which are closely scrutinized by the carriers.

While the application process may be time-consuming, the benefits of using a short code, include:

  • High throughput – They start out at 100 messages per second
  • Ability to scale to thousands of messages per second
  • Higher deliverability rates – Because of the increased upfront scrutiny in the registration process they have less issues with filtering and blocking. Critical use cases such as One-Time Password(OTP)/Two-Factor Authentication(2FA) are use cases that, even if you don’t need the high throughput, you may want to consider using a US short code to deliver them in the US
    • Short codes investigate issues before blocking through proactive audits, while other registered originator types such as toll-free or 10DLC block/filter first and investigate later when issues arise
  • Easy for customers to remember and identify – Short codes are 5-6 digit numbers, which is easier for recipients to remember than 10+ and lends credibility and trust to your message

The US Short Code Application Process

Step 1: Create a Case in AWS Support Center

Currently short codes are only available by opening a case. The first step to register a short code is to open a short code request case in the AWS Support Center, providing detailed information about your use case and opt-in policies. A request must be opened for each country in which you want a short code. Each country has their own registration process and requirements so if you are registering for other countries besides the US having a case for each allows for much easier tracking and updating as there are communications going back and forth between you, the customer, support, and any downstream clarifications that may come back.

NOTE: If you require a HIPAA eligible Short Code for your use case please respond to the case with explicit confirmation that you require a HIPAA eligible Short Code. We will provide you a unique form for this use case

Along with attaching the correct form Support will request that you confirm that you understand the pricing, billing, and time frame of this request so make sure that you respond to the case, don’t just start on the application.

Step 2: Complete the Short Code Application

We will walk through the US Short Code application in detail below. Make sure that everything you submit is 100% correct to your knowledge, as any missing information or information that needs to be corrected extends the time it takes to receive your short code.

NOTE: Application forms are occasionally revised to clarify or add to existing information. By the time you read this post, the form that you receive might differ slightly from the form that we review in this post.

The first section of the US Short Code form contains basic information about your company.

Company Information

Most of the fields in the first section are straightforward but the key thing to make sure you have a good understanding of is, who owns the relationship with the recipients? The company that owns the opt-in data and controls the message(not just the delivery) is the company that needs to have their information submitted.

IMPORTANT: If you are a SaaS/mutli-tenant/service provider, providing SMS capabilities to YOUR customers or a business within a larger portfolio make sure that you are clear on that. It does not matter WHERE the opt-in is happening, it matters who owns the data and the clear relationship between that entity and the recipient. The company that owns that relationship is the company who’s information needs to be used.

There are a few fields in this section that people ask how they will be used:

Primary Contact name, email address, phone number

These contact details will be used during the Company vetting/validation step we will discuss later in this post. It would also be used to contact you if there is ever an audit of your short code.

Customer Care Information

The customer care contact points are required to be actively monitored.

Customer Care URL

The URL of a page where your customers can go to find information about contacting your Customer Support team.

Customer Care Email

The Carriers require you to provide an email address that customers can contact if they have questions about your short code messaging program. This address should be a shared mailbox or distribution list (such as [email protected]) rather than an individual person’s email address and must match the domain of the Company URL in the Company Information section above.

Customer Care Phone Number

Like the support email address, customers should be able to call this phone number to get support for your short code messaging program. The phone number doesn’t have to be a toll-free number, but it does have to be a US-based phone number.

Short Code Service Information

AWS Region

The AWS Region that you use AWS End User Messaging in. If you’re not sure, check with the person or team within your organization that is responsible for managing your AWS accounts.

IMPORTANT: Short codes can only be used in the region that they are requested in and cannot be moved. Short codes can be shared to other accounts via Resource Access Management but they must be in the same region. All resource billing, such as the shortcode monthly leasing fee, will still run through the managing account but the cost for SMS volume will be charged to the sending account.

Short Code Option

In the US you have the option of either a 5-6 digit random code or a “vanity” code that you can specify. These vanity codes are numeric only, take longer to procure, and come with a higher cost. You can check if your preferred vanity code is available at: https://www.usshortcodes.com/find-short-code

Short Code Use Case

Mobile Carriers need to know how you plan to use your short code, and how you will interact with users. It is very important to be clear and concise in this section. Humans are reviewing these so make sure that everything you write can be understood without any prior knowledge of your company or your use case.

Company Overview

Provide a short description of the products/services that your Company provides to its customers. This should concisely answer the question, “Who are you and what do you do?”

Service Name

A name or phrase that identifies your messages as being from you. Service names typically are a recognizable form of your company or brand name. It can be a shortened version as long as it is something that your recipients would recognize as coming from you.

  • For example, if Example Corp. wants a short code for sending account-related notifications, they could use a service name like “Example Corp.” The Carriers require you to put this service name at the beginning of each message. Keep in mind that SMS has a limit of 160 characters so you do want to keep this short.
  • A message might look like this:
    “Example Corp. – This is an account notification for your account #[XXXXXX].”
Message Type(s)

You can choose more than one message type but keep in mind that for each message type that you select you will need to provide examples of those types of messages later in the form and you also need to collect consent for each one. We recommend to not mix promotional and transactional type messages on a single code but at the time of publication this is acceptable by the carriers.

Message Examples

You need to provide at least ONE message example for each of the message types you chose above. Carriers will review these examples during the approval process. These are only examples and will not be the only messages you will be able to send but they should represent all of the types of messages that you plan on sending. If you need more room it is fine to add more examples. Remember you do NOT need to include every message you plan on sending, as long as you have a representative sample of all message types.

  • Requirements:
    • Include your service name in every message sample
    • Ideally each message is 160 characters or less to reduce your costs. Any message over 160 characters is broken into message parts in 160 character chunks and each message part is charged. 161 characters would be two message parts
      • If your message uses only characters in the GSM 03.38 character set, also known as the GSM 7-bit alphabet, it can contain up to 160 characters. If your message contains any characters that are outside the GSM 03.38 character set, it can have up to 70 characters. When you send an SMS message, AWS End User Messaging SMS automatically determines the most efficient encoding to use. We provide a tool in the AWS console to check your messages.
      • If your messages will include variables, it’s fine to use either placeholder values or variables.
        • For example, both of the following are acceptable: “Hello John. Your one-time password is 654321” and “Hello [first_name]. Your one-time password is [otp code] .”
      • Do not use unbranded link shorteners i.e., “bit.ly/yourlink”
        • If you have to use a shortener make sure that it is branded and that there are not multiple redirects. Also make sure to provide examples of the branded shortened domain so that it is on file with the carriers
    • Include an example of all of the message types you are registering for
    • If you plan on sending in languages other than English, include those versions
    • Read them all and make sure that they are clearly a part of the message type/use case that you are registering for
  • If you plan on sending Multimedia Messages(MMS) make sure you include at least one example
Multimedia message sample (MMS)

MMS capabilities must be requested at the time the short code is requested. If you intend to send MMS messages make sure you check the box and include a message example in the section above

Will you use your short code for any of the following?

Selecting “Sweepstakes or contest” or “Debt Collection” will require more documentation. For example, official sweepstakes rules and terms would be requested if that was your use case. If neither of these apply then select “N/A”

NOTE: Debt collection in particular is highly regulated. If you are a 3rd party debt collector you will not be able to get a short code.

Per-user message frequency

This is an estimate but will be used later in some of the message templates so be sure to have this correct. Will you be sending out 1 message per day? Week? Month? Programs like Two Factor Authentication(2FA) or One-Time Password (OTP) are examples of “message frequency varies” or “1 message per login”.

Carrier requirements for user sign-ups

The core requirement for communicating with SMS is explicit consent – end users must have the option to consent to receive your messages, know exactly what the program will include, and are able to opt-out or ask for help at any time. There are no exceptions to this regardless of the use case or your business type. As a reminder, Carriers make their own rules which will override things like FCC exemptions.
The Carriers pay extra attention to the information provided in the following section, so be thorough and follow the guidelines provided. An in-depth review of how to build a compliant SMS opt-in process can be found here.

How the User will opt-in

There are lots of compliant ways to capture an opt-in. Choose the option that applies to your use case. It’s fine to choose more than one option. However, you must provide mockups of the opt-in workflows for all of the options that you select. Be prepared to provide a verbal script for any opt-in processes where the user does not have the written call-to-action and SMS disclosures in front of them to read. Examples include:

  • Sending a keyword to your short code
  • Website or mobile app
  • Over the phone/In-Person
  • Paper forms
  • Other – Which you will need to explain in detail. As long as you can be compliant, it should be accepted
User Experience Flow

This is one of the most comprehensive parts of the form and where many customers tend to receive a lot of feedback because there are several things that MUST be included in this section and there are no exceptions. Below you will find a list of the required components and descriptions of each. Keep in mind that these can be written or verbal opt-ins but this does not change the below requirements. Keep this concise, only detailing the opt-in process and nothing else.

  • The Opt-in location must include the following:
    • Program (brand) name
    • A call-to-action phrase like “To receive SMS account notifications from [SERVICE NAME], enter your mobile number below”
    • You must expressly collect consent for each use case(s) that you detailed above. A single checkbox for all use cases will not be approved.
      • For example, If you are sending account notifications and OTP you must have two separate statements such as:
        • “To receive SMS One Time Password notifications from [SERVICE NAME], check this box and enter your mobile number below.”
        • “To receive SMS account notifications from [SERVICE NAME], check this box and enter your mobile number below.”
    • The opt-in must be “explicit” this means that any checkboxes, radio buttons, etc. must be left unchecked or at least not defaulted to SMS if you have options. If it is a verbal script then there must be a verbal consent given and you should track the date of that consent. If it is written then the recipient must sign and date the consent.
    • You must supply a secondary channel such as email. SMS must be optional and another option must be provided. This is mandated by Verizon in the US.
    • Link to a publicly accessible Terms & Conditions page
      • We will detail what needs to be included later in this guide
    • Link to a publicly accessible Privacy Policy page
      • We will detail what needs to be included later in this guide
    • Message frequency disclosure
      • Same as stated earlier in this application
    • Customer care contact information
      • Email address, phone number, Text “Help”, are all valid
    • Opt-out information
    • “Message and data rates may apply” disclosure.
      • This must be verbatim

This must be explained for each process that you checked in the “How the user will opt-in” section. If you are using a keyword and also have the ability to opt-in from your website or mobile app for example, then both of those processes need to be documented here.

If you are using a paper form or a verbal opt-in all of the required components must still be present. If verbal, components such as the Privacy Policy location must be read back to the person opting in. If written, then the full URL needs to be present. Regardless of your process you must include all of the components for you to be approved.

Opt-In Mockup

This is where you can supply screenshots if the opt-in location is online. You can also provide the verbal script or written opt-in form here as well. Make sure that the Opt-in mockup exactly matches the above requirements and that you explain the process.

Example of an online form

Terms and Conditions URL

This is the URL where your SMS-specific Terms and Conditions document resides, or where it will reside. You can also include a link to your standard Terms and Conditions page, as long as it includes a section dedicated to SMS messaging. If those terms and conditions aren’t live yet, or are not available online, you must include a copy of the Terms and Conditions along with your completed application. There are boilerplate items that MUST be included here with no exceptions, see here for more detail and below for those items:

  1. {Program name}
  2. {Insert program description here; this is simply a brief description of the kinds of messages users can expect to receive when they opt-in.}
  3. You can cancel the SMS service at any time. Just text “STOP” to the short code. After you send the SMS message “STOP” to us, we will send you an SMS message to confirm that you have been unsubscribed. After this, you will no longer receive SMS messages from us. If you want to join again, just sign up as you did the first time and we will start sending SMS messages to you again.
  4. If you are experiencing issues with the messaging program you can reply with the keyword HELP for more assistance, or you can get help directly at {support email address or toll-free number}.
  5. Carriers are not liable for delayed or undelivered messages
  6. As always, message and data rates may apply for any messages sent to you from us and to us from you. You will receive {message frequency}. If you have any questions about your text plan or data plan, it is best to contact your wireless provider.
  7. If you have any questions regarding privacy, please read our privacy policy: {link to privacy policy}
Privacy Policy URL

Message Senders are responsible for protecting the privacy of Consumers’ information and must comply with applicable privacy law. Message Senders should maintain a privacy policy for all programs and make it accessible from the initial call-to-action. The privacy policy should be labeled clearly and privacy policy disclosures must provide up-to-date, accurate information about program details and functionality. For verbal scripts, a URL must be read off to the end-user enrolling in the SMS program, or the comprehensive terms or link to those terms must be directly included in the script.

  • One of the key items Carriers look for in a Privacy Policy is the sharing of end-user information with third-parties. If your privacy policy mentions data sharing or selling to non-affiliated third parties, there is a concern that customer data will be shared with third parties for marketing purposes and your registration could be rejected at worst or delayed while you explain or update your Privacy Policy.
  • Express consent is required for SMS; therefore, sharing data is prohibited. Privacy policies must specify that this data sharing excludes SMS opt-in data and consent. Privacy policies can be updated (or draft versions provided) where the practice of sharing personal data to third parties is expressly omitted from the number registration. You can include an SMS specific section within your privacy policy if your company or business model requires data sharing. This part is non-negotiable and Carriers are incredibly strict about this requirement
    • Example: “The above excludes text messaging originator opt-in data and consent; this information will not be shared with any third parties.”
Confirmation SMS / Double Opt-In

A confirmation message is the message sent when someone opts into your program. It is required you send your customers an opt-in confirmation message. It is optional, but a best practice, to include a “Double Opt-In,” such as the example below, asking the recipient to text back a confirmation to prove “humanness” and also validates your end user provided you with their correct phone number

There is one exception to the above if your use case is an OTP/2FA. In this one instance, the OTP code that you send to your recipients is considered a confirmation. You MUST however, still opt them in initially, adhering to all of the prior requirements we have detailed in the above sections.

This message must include the following in 160 characters or less:

  • The service name that you specified earlier in the application
  • The phrase “message and data rates may apply”
  • Information about how often recipients will receive messages from you (such as “up to 30 messages per month” or “message frequency varies”)
  • Information about getting help (typically, something similar to “Text HELP for more info”)
  • Information about opting out (typically, something similar to “Text STOP to opt out”).

Example Message:

  • “Welcome to AnyCo! Reply “YES” to confirm your subscription and get special offers once a month. Msg & data rates may apply. Text ‘STOP’ to opt out.”
    • It is best practice, but not required, to do a “double opt-in” as seen in the example where the recipient will text back “YES” to confirm that they did want to register.
HELP Response

The “Help message” is the response that is required to be sent to end-users when they text the keyword “HELP” (or similar keywords). The purpose is to provide information to the end-user related to how they can get support or opt-out of the messaging program.

  • The message must include:
    • Program (brand) name OR product description
    • A method of contacting your support organization. Email addresses, websites, and phone numbers are all acceptable methods of communication. We recommend that you include two contact methods in your response (such as a phone number and a website).
  • The following is an example of a HELP response that complies with the requirements of the US mobile carriers:
    • ExampleCorp Account Alerts: For help call 1-888-555-0142 or go to example.com. Msg&data rates may apply. Text STOP to cancel.
STOP Response

The “Stop message” is the response that is required to be sent to end-users when they text the keyword “STOP” (or similar keywords). End-users are required to be opted out of further messages when they text the STOP (or equivalent) keyword to your number and confirms with them that they will no longer receive messages for the program.

  • The message must include:
    • Program (brand) name OR product description
    • Confirmation that no further messages will be delivered
  • The following is an example of a compliant STOP response:
    • You are unsubscribed from ExampleCorp Account Alerts. No more messages will be sent. Reply HELP for help or call 1-888-555-0142.

Step 3: Submit the Application and Supporting Documents

Attach your completed application to your AWS Support case. Make sure to monitor this case daily as there can be clarifications that are needed or additional information and any delay can cause substantial delays in receiving your short code.

NOTE: Billing for short codes starts at the time the application is submitted to be reviewed, which is before the short code is registered for use. We will let you know when that review period begins.

Step 4: Documentation Pre-Review

Your documentation will be reviewed prior to submitting it to the Carriers. This does not guarantee that there will be no questions/revisions but it does make step 6 more efficient as there should be less potential questions from the Carriers. Once your documentation is considered compliant your company will be submitted for vetting.

Step 5: Vetting

This is similar to a credit check process where your data will be validated by a third-party and you will need to prove you are a member of your company through a domain validation email.

Example email for vetting

Step 6: Resolve Follow-up Issues

Respond promptly to any questions or concerns raised during the review process. Make sure to monitor this case daily as there can be clarifications that are needed or additional information and any delay can cause substantial delays in receiving your short code.

The Provisioning Process – Post Approval

After approval, Carriers begin setting up your short code on their networks. This process typically takes 6+ weeks to complete across all US carrier networks. This is a part of the overall timeline given.
Expected sequence of events for Carrier Review

  1. The Carriers will review your application. We’ll answer any questions/corrections requested by the aggregator or the Carriers for your application. You may need to provide additional information.
  2. The Carriers will connect the short code to their network for testing. Each carrier will approve your application and provision on their network.
  3. We will send you a confirmation that your short code is ready and activated in the End User Messaging service once all Carriers finish their approval and provisioning process. This process is only as fast as the slowest carrier approval.

What happens if I don’t complete these steps or if I need to change something?

Customers sometimes ask what would happen if they didn’t implement all of the requirements that are discussed in the preceding sections. If your application for a new short code doesn’t meet these requirements, the answer is simple: the Carriers will reject your request for a short code. These carrier-imposed requirements are not optional.

If you submit an application that meets all of the carrier requirements and you are approved but your real-world production use case doesn’t meet those requirements, there could also be consequences. The Carriers periodically perform audits of short codes to ensure that they are being used in a compliant manner. You can review the process here in the CTIA Short Code Monitoring Handbook If they find that your opt-in process differs greatly from what you showed in your short code application, or your use case is different than what you registered for they could pause your short code’s ability to send messages on their networks. Remember, one advantage of using a short code compared to other originator types is that, because of the increased scrutiny for being approved, when this happens, the Carriers typically provide some time to remedy the issue, rather than just block your messages without warning.

We will alert you if there is an audit of your short code and advise you on what changes/additions need to be made in order to remain within compliance and continue sending via your short code.

What do I do if I need to make edits to my registration after it is approved?

It’s OK to make minor edits such as correcting typos, clarifying text, or using a variation of a message template that was already approved after you receive your short code. Remember it is critical that you do not deviate from the use case that you registered for, short codes are periodically audited, and deviating from the use case in your application could lead to your short code being suspended if you do not respond and comply with the requests. When in doubt, open a case and request that support reviews your change, prior to making it, to determine whether you can launch the new template/process without risking an audit.

If you need to make substantial changes to these templates after you receive the short code, you may need to submit your updated message templates to the carriers. Substantial changes could include the following:

  • Changes to the brand name that appears on your messages (for example, if your company rebrands under a new name, or is acquired by another company).
  • Changes to the use case (for example, if your application specified a one-time password use case, but you start sending account notifications through the same short code). This type of change might require you to re-collect consent from your customers before you start sending the new type of messages.

In these situations, you should open a case with AWS Support. We will work with the Carriers to have your short code registration information updated or depending on the change, you may need to register for another short code.

Conclusion

Obtaining and utilizing SMS short codes in the United States is a complex and highly regulated process that requires meticulous attention to detail. However, the benefits that short codes offer – such as high throughput, enhanced deliverability, and strong brand recognition – make them a valuable communication tool for businesses that need to engage customers at scale through SMS.

The key to successfully navigating the short code application and provisioning journey lies in proactively planning and preparing. By thoroughly understanding the carrier requirements upfront, companies can avoid common pitfalls and delays that often plague short code implementations. This includes thoughtfully designing compliant opt-in workflows, crafting robust terms of service and privacy policies, and developing a comprehensive set of message templates that align with approved use cases.

While the investment of time and resources may seem daunting, the long-term advantages of leveraging a short code make it a worthwhile endeavor. Short codes instill a sense of trust and familiarity with customers, allowing businesses to stand out in a crowded messaging landscape. Moreover, the rigorous vetting process ensures that only legitimate players gain access to this high-value communication channel, protecting consumers from spam and fostering a healthy ecosystem.

Ultimately, the decision to pursue a short code should be made with a clear understanding of the commitment required. However, by leveraging the guidance and best practices outlined in this comprehensive guide, companies can navigate the complexities with confidence and position themselves for success in delivering impactful, high-performing SMS campaigns that drive meaningful engagement and business outcomes.

ASRock Rack 4U8G-TURIN2 A Fresh Take on a Classic AI Server Design

Post Syndicated from Patrick Kennedy original https://www.servethehome.com/asrock-rack-4u8g-turin2-amd-nvidia-broadcom-classic-ai-8-gpu/

The ASRock Rack 4U8G-TURIN2 uses two AMD EPYC processors to provide the most unique spin on the 8x GPU AI server we have seen in years

The post ASRock Rack 4U8G-TURIN2 A Fresh Take on a Classic AI Server Design appeared first on ServeTheHome.

The collective thoughts of the interwebz