Гласовете на Америка – брой 3

Post Syndicated from Йоанна Елми original https://www.toest.bg/glasovete-na-amerika-broy-3/

Гласовете на Америка – брой 3

Има нещо, което българите авторитетно можем да заявим на американците: когато властимащите изкарат знамената, бутафорните възстановки от стиропор и разпятията, значи държавата със сигурност отива на кино. 

Колкото разпятието на кръговото в Елин Пелин ще предотврати катастрофите при липсата на системно прилагане на законите и на адекватни мерки, толкова и двете чисто нови гигантски знамена около Белия дом успяха да спестят на президента Тръмп въпроси за намесата на САЩ в израелско-иранския конфликт, които ядосаха държавния глава не на шега, но пък поне поразбудиха присъстващите на церемонията. 

Доналд Тръмп, 29 ноември 2011 г.: „За да бъде избран, Барак Обама ще започне война с Иран.“

Президентът заяви, че двете знамена са личен подарък от него и той ще ги плати. От такива подаръци вероятно ще има нужда все повече, след като стана ясно, че проектобюджетът му ще добави 3,4 трилиона долара към дефицита на страната. 

Вече дори не съм сигурна как точно да опиша Републиканската партия, която от години твърди, че намаляването на раздутия бюджет е сред основните ѝ приоритети. Затова ще оставя числата да говорят: Тръмп води със 7,1 трилиона външен дълг, следван от Обама и Байдън със съответно 5,6 и 2,8 трилиона. След това са двамата президенти на републиканците – Буш и Рейгън. Всичко е точно. На когото не му харесва, може да си намери други числа, разбира се. Такива са времената. 

Илън Мъск и Доналд Тръмп си размениха остри реплики в социалните мрежи по този повод. Мъск имплицитно обвини Тръмп в педофилия през вектора на случая с Джефри Епстийн, а Тръмп като истински лидер го нарече „неблагодарник“ с оглед на всичко, което е получил дотук. Защото какво е държавата, ако не група хора, които си правят услуги и трябва да са благодарни на даващите (този урок също можем да го преподадем на американците, ала ББ). 

След като и двамата се успокоиха, Мъск дори се извини – оттогава никой не е чувал за него, освен почитателите му в X, където продължава да се заканва на изкуствения интелект Grok всеки път, когато последният се осмели да му говори с факти. В случая става въпрос за следния обмен на информация между потребител и изкуствен интелект: 

@GrokChecker: Grok, от 2016 г. насам насилието идва повече отляво или отдясно? 
Grok: Данни от 2016 г. насам показват, че идеологически дясното политическо насилие е по-често разпространено и по-смъртоносно, с инциденти като щурма на Капитолия на 6 януари и масовите престрелки (например в Ел Пасо през 2019 г.), които вземат сериозни жертви.
Мъск: Голям провал, това е абсолютно невярно. Grok цитира мейнстрийм медиите. Работим по въпроса. 

Както казах, приятели, на когото не му харесва истината, е добре дошъл да си създаде друга. 

Не такъв е случаят с Тъкър Карлсън, който внезапно отново стана известен, след като миналата година обикаляше руските супермаркети и се впечатляваше от московското метро (което, в интерес на истината, е в по-прилично състояние от хронично лишавания от финансиране щатски градски транспорт; в крайна сметка според настоящата администрация работата на държавата е да залага на крипто и да отнема на щатите правото да регулират изкуствения интелект, а не да субсидира градския транспорт). 

Потенциалното участие на САЩ в поредна война в Близкия изток разцепи MAGA вселената по-сериозно и от сблъсъка между Мъск и Тръмп, а Тъкър Карлсън доказа, че все още може да прави телевизионна журналистика – или че дори и спрелият часовник е верен поне два пъти дневно. Това се случи в интервю със сенатора Тед Крус, в което последният, меко казано, се изложи след поредица от елементарни въпроси и цитиране на Библията като аргумент защо САЩ трябва да подкрепят Израел и Нетаняху. 

Сред критиците на потенциалната намеса на САЩ в конфликта е и друга проруски настроена американска личност – Тълси Габард, понастоящем начело на разузнаването, макар може би не задълго. Дори крайнодясната депутатка от Камарата на представителите Марджъри Тейлър Грийн избухна срещу Тръмп. 

Независимо дали читателят е съгласен, или не с позициите на изредените, трябва да се признае, че поне проявяват повече характер и постоянство от мнозинството в редиците на републиканците, които агитират за оттегляне на финансирането за Украйна, но нямат нищо против да финансират „безкрайните войни“ в Близкия изток, срещу които говорят само когато опозицията е на власт. Не е лошо и да си припомним, че на 17 януари Русия и Иран подписаха споразумение за защита и сътрудничество. Това не е текст по сложната тема за Близкия изток. Но е текст, който напомня, че руските интереси имат своите гласове във вътрешната политика на САЩ. И Иран е добър пример за това. Совите не са това, което са. 

Разбира се, всичко изброено ни води до дежурния въпрос: къде е опозицията? Ами, оплетена в собствения си политически хаос и със сериозни проблеми с финансирането. Вътрешната борба за власт и отказът на демократите да се придвижат извън статуквото на рухващата система могат да им костват и междинните избори, смятат редица експерти. Сенаторът Бърни Сандърс, едно от най-популярните лица не само сред демократите, но и в страната като цяло, призова демократите да скъсат с корпоративните донори и продължава политическото си турне из страната, което привлича огромна публика. Сандърс обаче е на 83 години – част от застаряващия политически елит. Това е един от основните проблеми на Демократическата партия, чиито членове са с по-голяма средна възраст от републиканците. 

Междувременно на 14 юни се проведоха едни от най-мащабните протести в историята на страната – на повече от 2000 места във всички 50 щата, включително и в селища, които се държат от републиканците с добри мнозинства. Следващата дата е 17 юли. 

Протестите съвпаднаха с т.нар. военен парад на Тръмп, който не можа да се похвали с голяма публика и висок дух. Дори руснаците, обичайно подкрепящи Тръмп, си позволиха да му се подиграват в социалните мрежи. В същия ден в домовете си бяха застреляни двама политици от Минесота и партньорите им. Конгресменката Мелиса Хортман и съпругът и загинаха, след като мъж, преоблечен като полицай, влезе в дома им с оръжие. В списъка на 57-годишния Ванс Болтър, заловен след една от най-мащабните полицейски акции в историята на щата Минесота, са били още 45 души – всички от Демократическата партия. Болтър е бял, свръхрелигиозен и обявяващ се срещу правото на аборт. 

Убиецът е бил преоблечен като полицай, както вече споменах. Интересното е, че е нямал идентификация – както напоследък действат и имиграционните служби в страната. Местна тексаска медия например разказва, че мъж без криминални прояви е депортиран, след като на телефона му са открити снимки с друг мъж, който има татуировки [подозренията са, че е член на групировка – б.р.]. Подобни арести в необозначени автомобили и без право на процес са все по-често явление, особено в градовете, където хората гласуват предимно за демократите. 

Със сигурност не това е мотивацията на президента да нареди още по-сериозни акции в Чикаго, Ню Йорк, Лос Анджелис и Филаделфия, след като първо отмени, а после отново поиска арестите на хора, работещи в областта на услугите и земеделието. Това стана след оплаквания на работодатели, че бизнесът им ще пострада. Самият Тръмп продължава да наема чужденци за работа в собствените си обекти.  

Полицаи в САЩ

Нищо не казва „свобода“, „величие“ и „свиване на държавата“ както произволните арести, следенето на социалните мрежи на чуждестранни студенти и натиска над университетите, разпродаването на държавни земи и природни паркове, признанията на Тъкър Карлсън, че най-гледаната в страната телевизия Fox е разпространител на пропаганда (нещо, което допреди това, естествено, не подозирахме), арестите на политици, управлението с укази и извънредни правомощия и най-накрая, но не на последно място – вкарването на хората начело на технологичните компании в армията и събирането на личните данни на всички американци в огромна система за следене. 

Разбира се, всичко това е в името на реда, законността и безопасността. Да не си помислихте нещо друго? Все пак Америка ще бъде велика отново. Не виждате ли как нещата вече се получават? 

Този път гласът на миналото е литературен:

цитат от Чък Паланюк, американски писател и журналист, автор на „Боен клуб“. Откъсът е от романа му „Приспивна песен“, публикуван на български език през 2004 г. в превод на Марин Загорчев. 

Старият Джордж Оруел не е разбрал нищо.
Големият брат не ни гледа. Той пее и танцува. Вади бели зайци от цилиндри. Големият брат се старае да привлече вниманието ти във всеки момент, докато си буден. За да не можеш да се съсредоточиш. За да бъдат сетивата ти винаги притъпени.
Той се стреми да убие въображението ти. Докато закърнее като апендикса ти. Стреми се да отвлича вниманието ти.
Това постоянно разсейване е по-лошо от непрекъснатото наблюдение. Когато сетивата ти са притъпени, никой не се интересува какво можеш да си мислиш. Когато въображението на всички е атрофирало, никой не може да бъде заплаха за властта.


Абонирайте се, за да получавате този бюлетин на електронната си поща в момента, в който излезе!

Вече сте регистриран потребител на Toest.bg? Може директно от настройките на бюлетините в своя профил да изберете „Гласовете на Америка“ или да натиснете бутона по-долу:

Още нямате профил в Toest.bg? Регистрирайте се само с няколко клика:

My a11y journey

Post Syndicated from Matthew Garrett original https://mjg59.dreamwidth.org/72379.html

23 years ago I was in a bad place. I’d quit my first attempt at a PhD for various reasons that were, with hindsight, bad, and I was suddenly entirely aimless. I lucked into picking up a sysadmin role back at TCM where I’d spent a summer a year before, but that’s not really what I wanted in my life. And then Hanna mentioned that her PhD supervisor was looking for someone familiar with Linux to work on making Dasher, one of the group’s research projects, more usable on Linux. I jumped.

The timing was fortuitous. Sun were pumping money and developer effort into accessibility support, and the Inference Group had just received a grant from the Gatsy Foundation that involved working with the ACE Centre to provide additional accessibility support. And I was suddenly hacking on code that was largely ignored by most developers, supporting use cases that were irrelevant to most developers. Being in a relatively green field space sounds refreshing, until you realise that you’re catering to actual humans who are potentially going to rely on your software to be able to communicate. That’s somewhat focusing.

This was, uh, something of an on the job learning experience. I had to catch up with a lot of new technologies very quickly, but that wasn’t the hard bit – what was difficult was realising I had to cater to people who were dealing with use cases that I had no experience of whatsoever. Dasher was extended to allow text entry into applications without needing to cut and paste. We added support for introspection of the current applications UI so menus could be exposed via the Dasher interface, allowing people to fly through menu hierarchies and pop open file dialogs. Text-to-speech was incorporated so people could rapidly enter sentences and have them spoke out loud.

But what sticks with me isn’t the tech, or even the opportunities it gave me to meet other people working on the Linux desktop and forge friendships that still exist. It was the cases where I had the opportunity to work with people who could use Dasher as a tool to increase their ability to communicate with the outside world, whose lives were transformed for the better because of what we’d produced. Watching someone use your code and realising that you could write a three line patch that had a significant impact on the speed they could talk to other people is an incomparable experience. It’s been decades and in many ways that was the most impact I’ve ever had as a developer.

I left after a year to work on fruitflies and get my PhD, and my career since then hasn’t involved a lot of accessibility work. But it’s stuck with me – every improvement in that space is something that has a direct impact on the quality of life of more people than you expect, but is also something that goes almost unrecognised. The people working on accessibility are heroes. They’re making all the technology everyone else produces available to people who would otherwise be blocked from it. They deserve recognition, and they deserve a lot more support than they have.

But when we deal with technology, we deal with transitions. A lot of the Linux accessibility support depended on X11 behaviour that is now widely regarded as a set of misfeatures. It’s not actually good to be able to inject arbitrary input into an arbitrary window, and it’s not good to be able to arbitrarily scrape out its contents. X11 never had a model to permit this for accessibility tooling while blocking it for other code. Wayland does, but suffers from the surrounding infrastructure not being well developed yet. We’re seeing that happen now, though – Gnome has been performing a great deal of work in this respect, and KDE is picking that up as well. There isn’t a full correspondence between X11-based Linux accessibility support and Wayland, but for many users the Wayland accessibility infrastructure is already better than with X11.

That’s going to continue improving, and it’ll improve faster with broader support. We’ve somehow ended up with the bizarre politicisation of Wayland as being some sort of woke thing while X11 represents the Roman Empire or some such bullshit, but the reality is that there is no story for improving accessibility support under X11 and sticking to X11 is going to end up reducing the accessibility of a platform.

When you read anything about Linux accessibility, ask yourself whether you’re reading something written by either a user of the accessibility features, or a developer of them. If they’re neither, ask yourself why they actually care and what they’re doing to make the future better.

comment count unavailable comments

Т.Е. от Е.Т. – епизод 16

Post Syndicated from Тоест original https://www.toest.bg/t-e-ot-e-t-epizod-16/

Т.Е. от Е.Т. – епизод 16

Понякога те се завръщат. В случая Е.Т. се завръща. И как иначе! Сърце юнашко не трай, когато Шиши, Кики, ББ и цялата световна анимация работят всекидневно за съдържание, което плаче да се коментира, защото явно нямаме какво друго да правим.

Един учител – хиляди истории

Post Syndicated from original https://www.toest.bg/edin-uchitel-hilyadi-istorii/

Един учител – хиляди истории

Работата ми като учебен терапевт изисква да изградя добра връзка с живота на учениците в образователен и в личен аспект. Защото всичко накуп – семейството, учителите, съучениците – съставя общата картина, а когато видя всичко в цялост, най-добре мога да разбера как да помогна. Ето така се запознах с мистър Соти, учител по английски език на мой ученик. 

В края на учебната година, в най-жежките дни на оценявания и безпределна умора за учители, ученици и родители, имаме нужда от малко оптимизъм. Защото има и такива учители като Сотион Вело. А щом ги има, значи има и надежда.


Как станахте учител?

Не беше в плановете ми да стана учител. Бях на 18 или на 19 и се наложи да помогна на едно момиче в подготовката му за изпит по английски. Получи ни се много добре. Тя беше във втори или трети клас, нещо такова. След като се записах в университета, имах възможност да стана заместник-учител в Гърция за около година. Обиколих няколко училища, замествах, когато някой учител трябваше да отсъства. Преподавах английски, география, история – всичко, което беше нужно.

Така постепенно натрупах малко опит и видях, че тази професия ми харесва, докато си намеря нещо „по-сериозно“. Да, обаче после попаднах на сериозна работа в една гимназия. Бях голям късметлия, че там имаше много готини колеги с опит, които бяха склонни да ми помогнат и да ми покажат колко харесват професията си. И реших, че искам да правя това, което правят те. 

Не беше да се събудя една сутрин и да си кажа: „Искам да съм учител!“

Имате ли любима история от ученическите Ви години с някой учител?

Да. Обожавах този човек. Е, в началото, когато ми стана класен, не го харесвах много, даже изобщо. Той се казваше г-н Параскевас – на гръцки означава петък, нещо като Mister Friday, господин Петък. Голям, респектиращ мъж с много строги очи. Ако го видиш на улицата и не го познаваш, веднага ще си помислиш, че е учител. 

Беше много различен – викаше ни, наказваше ни, хокаше ни много. Наказанията бяха различни. Едно от тях например беше да застанеш и да стоиш мирно в средата на двора. Минават другите, питат те защо си там, и ти на всеки трябва да обясниш.

С останалите учители си правехме майтапи, с него беше невъзможно – ако му направим номер, автоматично получаваме наказание, това е. Държеше да говорим учтиво и красиво, не можехме да си позволим грубости пред него. Постоянни наказания, наливане на акъл… В началото си мислехме, че ни мрази. Обаче един ден нещата се обърнаха. 

Имахме в класа едно момче – Спирос, много проблемен ученик, още сме приятели с него. В днешно време бих казал, че е хиперактивен. Реактивно момче, двойкаджия. Постоянно правеше бели: беше спукал гумите на учителката по английски, беше хвърлял камъни по прозорците на друг клас и беше счупил стъклата, големи дивотии. Със Спирос постоянно имаше проблеми, а покрай него и с нас, защото искахме да станем като него, разбира се, и го следвахме във всички бели, които правеше. 

Един ден дойде г-н Параскевас при мен и при едно друго момче – Янис, и ни попита: „Има ли нещо, което трябва да знам за Спирос? Какво става в семейството му? Как вървят нещата? Има ли някакви проблеми? Споделял ли е нещо?“. А ние знаехме, че баща му си пийва, бие ги вкъщи, не са добре нещата. Казахме му. 

След няколко дни идва Спирос на училище и ни разказва: г-н Параскевас отишъл у тях и хвърлил един як пердах на баща му. Казал му, че ако още веднъж посегне на децата, направо ще го очисти. А г-н Параскевас беше много едър, хващаше му се вяра. 

Момчета, момичета – всички бяхме от квартала и си тръгвахме заедно от училище, изпращахме се един друг. Обаче Спирос вече не идваше с нас след училище. Започна да остава много често след часовете, за да учи с г-н Параскевас. Той започна да му помага по всички предмети – по история, математика, английски, гръцки, пишеха заедно домашните. 

Само след няколко месеца на мястото на съученика ни Спирос, който нямаше дори раница, се появи съвсем ново момче – винаги с домашно, нови тетрадки, химикалки. Цялото му поведение се промени. Завърши началния етап, отиде в гимназия, после в университет. Г-н Параскевас го подготвяше за кандидатстудентските изпити докрай. А двете семейства празнуваха Коледа заедно.

Г-н Параскевас стана по-късно директор, а ние и до ден днешен си имаме група в Messenger, която се казва „Учениците на г-н Параскевас“, и си правим срещи всяка година. Докато беше жив, всички ние, вече студенти по цял свят, когато се връщахме, минавахме да му се обадим. Естествено, че станахме студенти – той ни беше поставил ултиматум, че ако някой не иска да кандидатства, той лично ще му даде урок. Разпитваше ни за изпитите ни, за постиженията ни… Правеше го, защото това беше мисия за него; защото носеше отговорността, че това е, което може да направи за останалите.

Разбрал е от баща ми, че съм станал учител, и попитал защо. Баща ми му казал: „Защото явно е искал да стане като теб!“ Може би не станах учител заради него, но продължавам да бъда учител заради него. Той беше супергерой – без извънредните фантастични сили на онези от филмите, а истински, натурален супергерой.

На погребението му Спирос и баща му, който беше изял големия пердах от г-н Параскевас, носеха ковчега. Всички бяхме там.

Веднъж някой беше организирал благотворителна акция в училище и бяха донесли играчки, дрехи и други неща за социално слаби ученици. Събрали ги в една стая и им ги раздават. Влиза г-н Параскевас, вдига им скандал и ги гони: „Така ли се прави? Вие не знаете ли, че тези деца имат достойнство!“ 

Винаги пазеше достойнството ни, но за да станем достойни хора, според него първо трябваше да се научим да говорим хубаво, да се държим добре, да поемаме отговорност, да се отнасяме подобаващо един с друг. Е, методите му включваха някой и друг шамар, но тогава беше така.

Един учител – хиляди истории
Сотион Вело е родом от Атина. В момента е учител по английски език в частно училище в София. Завършил е английска литература и творческо писане в Университета в Уелс. Сега учи психология в Софийския университет. Снимка: Личен архив

В днешно време шамари не се раздават и това е добре. А и аз знам, че не го правите. Въпреки това Вашите ученици Ви обожават. Какво „раздавате“ тогава?

Уча децата на ценности. Не на моите ценности, не искам да ме копират. Но искам да имат ценностна система – те да изберат как да я изградят. Трябва да имат отговорност и достойнство, да знаят, че са им необходими умения извън учебните предмети. И трябва да са свободни. Не говоря за буквалната свобода, която хората в затворите нямат, а за свобода на общуване на собствените им идеи и ценности. 

Какво научихте в университета, което прилагате и днес? Знаем, че следването е по-сухо, по-теоретично. И все пак има ли нещо, което практикувате?

Учих литература, не педагогика. Научих се да не се отказвам и винаги разказвам тази история на децата. В началото не искаха да ме приемат в специалността, която си бях избрал – „Английска литература“. По мое време в моя университет тя беше за native speakers – хора, на които английският им е майчин език. Аз обаче много исках да уча точно това и постоянно заставах пред някоя бариера. 

Трябваше да представя портфолио с разкази и поезия. Представих го. После ми казаха, че трябва да имам минимум еди-каква си оценка на изпит по английски – ниво за хора, израснали с езика. Изкарах нужната оценка с много труд. Моят ментор в университета през цялото време ме съветваше да сменя програмата, да избера друго, за да завърша. Аз не се отказах. 

Е, уводът на първата ми книга е написан от него. Когато се разделяхме, той призна, че е искал да види докъде ще стигна и дали няма да се откажа. Според него бях станал истински писател.

Това, което научих, не е толкова теоретично, нали? Наградата е по-сладка, когато знаеш, че не си се отказал. В гимназията не бях блестящ ученик, но знаех, че искам да уча литература, тя е все още най-голямата ми любов. 

Представете си перфектните обстоятелства – можете да бъдете какъвто пожелаете. Бихте ли избрал друга професия? 

Не. Чувствам се добре с това, което правя. Вече знам какво правя. Може да звучи нахално, но съм добър в него и съм наясно че, няма да съм толкова добър в каквото и да било друго. И нищо друго не би ми харесало толкова много.

И все пак по някое време Ви свършват силите, особено сега, в края на учебната година. Представям си, че и Вие си мислите за ваканция като всички деца. Какво Ви мотивира, когато останете без сили?

Ваканцията! (Смее се.) Основното е, че много харесвам това, което правя; много обичам всички деца, които са минали през мен – тези, които не ме обичаха, тези, които ме мразят, и тези, които не ме мразят, тези, които ме обичат. Всички ги помня с имената и историите им. Затова в училище много трудно бих се изморил.

Изморявам се, когато не знам какво да правя с даден случай, но това ме мотивира. Та нали затова съм учител – за да помогна точно на такива трудни деца! Не съм станал учител за лесното. На лесните деца им се вижда бъдещето ясно. Но другите – с трудностите или с тъжните истории, те не знаят още нищо. Заради тях съм учител.

И като казах „трудното“ – и наградата, и удовлетворението са по-сладки, когато си преминал през трудности.

Кое Ви мотивира да сте учител?

Да помагам на деца, които имат нужда от помощ. Не просто да вървя по един учебник. Най-трудната част е подготвителната. Преди да започнеш да преподаваш, децата трябва да ти се доверят. Трябва да видят в един учител какво може да им даде. Не говоря за учебни предмети, а за характер, за личност, за житейски опит. 

Част от моята подготовка, преди да се срещна с нов ученик, е да прочета всичко, което имаме за него – откъде идва, какъв е случаят, дали има обследвания. След това се запознаваме. Всяко дете учи различно и това изисква и различен подход. Важно за мен е да се запознаем – както аз да опозная тях, така и те да опознаят мен – какъв човек съм, какво мога да им дам, дали могат да ми се доверят. И едва след това говорим за времената в английския език, пишем диктовки и всичко останало.

Важни ли са родителите в това? Как поддържате контакт с тях?

На родителите казвам, че ако се нуждая от помощта им, ще ги потърся. Случвало се е броени пъти досега. Има и родители на деца със СОП например, които са изключително притеснени как ще бъде прието детето им. И на тях казвам същото:

ако не ви търся, значи всичко е наред. Но ако се обадя, трябва бързо да вдигнете телефона.

Давам обстойна обратна връзка в края на учебната година, но рядко ми е необходима помощта на родителите. На децата казвам, че в училище могат да намерят цялата подкрепа, която им е нужна. И при мен, при нас, това наистина е така.

Кога Ви е било най трудно?

Може би на първата ми постоянна работа като учител. Преди това бях винаги по заместване. Дотогава някак не ми беше станала ясна огромната отговорност, която нося. Изведнъж ме затрупаха дежурства, пълен работен ден, пълна отговорност за цели класове. Започнах постоянно да се питам правилно ли е това, което правя.

Тогава ми помогнаха колегите с повече опит от мен. Важно ми е да видя как се прави нещо, да слушам, да наблюдавам. Това е чест – да те допуснат да си следващият. Още повече ми помогнаха самите деца с търпението си.

Един учител – хиляди истории
Снимка: Личен архив

Кое е най-хубавото нещо, което Ви се е случило като учител?

Едно от най-хубавите неща се случи миналата година. Обаждат ми се от гръцки номер, непознат за мен. Вдигам и едно момиче казва: „Здравейте, кирие [господине – б.а.]. Аз съм Деспина.“ Деспина беше моя ученичка, на която преподавах още преди официално да стана учител. Подготвях я за един сертификат на Кеймбридж. Покрай тази подготовка говорехме много за литература, за книги. Помня, че ѝ давах да чете различни статии, поезия. Превеждахме ги – това беше част от подготовката ѝ. 

Миналата година Деспина беше намерила баща ми, за да му поиска телефона ми. И тя продължи:

От три дни Ви търся, за да Ви кажа, че Вие сте виновен. Подписах първия си трудов договор като учителка по английски език.

Толкова е хубаво да те помни някой, който при срещата ви е бил на около десет години. А в Гърция е изключително трудоемко да станеш учител по английски – трябва да завършиш филология, а за да влезеш в тази специалност, се изисква може би най-високият бал. 

Има ли нещо, което бихте искал да промените, но нямате право? Някое правило, което Ви слага прът в колелата? 

Има. То ще прозвучи смешно за някои хора, обаче съм много против чиновете в класната стая.

Според мен тя не трябва да е с чинове и с бяла дъска. Трябва да е място, където учениците да имат желание да влязат.

Да е уютно, да има неща, които те харесват и ги карат да се чувстват добре. Да има лаптопи, за да се учат да използват технологии. Да е място, което вдъхновява, на което ти се седи. А не да стоят мирно на чина, да трябва да отворят на страница 25 и да започнат да попълват тестове. 

В моите класове правим това. Седим на земята, не е нужно да вдигаме ръка, за да вземем думата, защото умеем да се изслушваме. Децата могат да отидат до тоалетна по всяко време, без да искат разрешение. За мен би било унизително да ги карам да искат! И те не прекаляват с излизанията от стаята, както биха се опасявали някои родители или учители. Не искат да са вън от стаята, не искат да пропускат нищо, затова и не търсят празни извинения.

Как се справят Вашите ученици според критериите и изискванията на учебните програми, които трябва да изпълнят? 

Знам, че има колеги, които ще се усъмнят колко успешно се преподава по различен начин. Ако тази година трябва да се научат трите основни времена на английски, ние ще ги учим, но няма да ги учим по таблици и учебни тетрадки. Learn from experience [да учиш от опит – б.а.] се нарича това, което правя.

А как изпитвате тогава?

Без учениците да знаят. Често оставям някой от тях да направи обзор на взетите уроци, да пише на дъската, другите да му диктуват или той на тях. Така се изпитват и взаимно. Моите ученици не се притесняват да допуснат грешка – знаят, че и аз, и целият клас ще реагираме с корекция, но не и с наказателна оценка или присмех. Ние се учим един от друг.

На едно от предишните ми работни места дойде един ден директорката при мен и ми каза:

Не може така, не вървите по материала, трябва да сте взели до еди-кой си урок, какво правиш?

Казах ѝ да подготви тя тест на всички уроци, които смята, че трябва да са усвоени до този момент, и да го даде на децата. Без моята намеса. Направиха го. Децата ми изкараха само петици и шестици. И тя ме пита:

Е, как го направи това?

Ами това са тайните на професията, казах ѝ аз.

Доколко училищното ръководство, училищната политика, Министерството на образованието влияят на процеса в класната стая? 

Тук бих направил разграничение между частни и държавни училища. Според мен, ако говорим за частни училища, е малко по-различно. Учителите определено имат повече свобода да правят нещата по свой начин и пак да постигат заложените цели. Не са толкова притиснати да следват точно определен ход, нямат толкова ограничаваща рамка. А вече и резултатите от различните оценявания официално показват, че това води до успех.

Мисля, че във „Фюжън“, където съм сега, имам пълната свобода да преподавам така, както смятам, че е адекватно спрямо времето, в което се намираме. Това беше и условието ми – искам свобода. А резултатите ги обсъждаме в края на всеки етап. 

Това не бих могъл да постигна на място със силно ограничени методи на преподаване, където ми е спуснато отгоре колко и кога да правя. Както и ако не бих имал свободата да избирам учебните материали – учебници, помагала, ресурси.


Светът се променя с бясна скорост. Професиите, в които ще се развиват поколенията, започващи днес образователния си път, все още не са измислени. Подготвена ли е нашата образователна система, за да отговори на тези предизвикателства? Какво може и трябва да се промени? А как?

Веднъж месечно в рубриката „Възможното образование“ ще говорим за промяната – такава, каквато искаме да я видим, за добрите примери и за посоките, в които може би е добре да обърне поглед българската образователна система.

Индуцирани земетресения, застрашават стратегически язовир и региона на Перник Държавата крие 45 земетресения заради корупционни интереси

Post Syndicated from Екип на Биволъ original https://bivol.bg/earthquakes_studena.html

петък 20 юни 2025


ЗЕМЕТРЕСНИЯ В ТИШИНА През последните две години хората от селата Боснек, Студена и Старо село, сигнализират за изключително силни взривове, идващи от кариерите над с. Студена. В непосредствена близост до…

Материалът <span style='color:#ff0000;font-size:12px;'>Индуцирани земетресения, застрашават стратегически язовир и региона на Перник</span> <BR> <H1 class='post-title single-post-title entry-title'>Държавата крие 45 земетресения заради корупционни интереси</H1> е публикуван за пръв път на Bivol!.

Counter Service: How we rewrote it in Rust

Post Syndicated from Grab Tech original https://engineering.grab.com/counter-service-how-we-rewrote-it-in-rust

Abstract

The Integrity Data Platform (IDP) team decided to rewrite one of our heavy Queries Per Second (QPS) Golang microservices in Rust. It resulted in 70% infrastructure savings at a similar performance, but was not without its pitfalls. This article will elaborate on:

  • How we picked what to rewrite in Rust.
  • Approach taken to tackle the rewrite.
  • The pitfalls and speed bumps along the way.
  • Was it worthwhile?

Introduction

Grab is predominantly based on a microservice architecture, with the vast majority of microservices being hosted in a monorepo and written in Golang. It has served the company well so far, as the “simplicity” of Golang allows developers to ramp up and iterate quickly.

However, Rust has seen some gradual adoption across the company. Starting with a few minor CLIs, which then progressed to notable success with a Rust-based reverse proxy in Catwalk for model serving. Additionally, a growing community of Rust enthusiasts within the organisation has expressed interest in advocating for and expanding the adoption of Rust more proactively.

After achieving success with several projects on the ML platform and addressing concerns about Rust’s ability to handle traffic at scale, the next logical step was to assess the Return on Investment (ROI) of rewriting a Golang microservice in Rust.

Background

Rust has the reputation of being highly efficient yet poses a steep learning curve. Rust is often touted to perform close to C, doing away with garbage collection while remaining memory safe through strict compile checks and the borrow checker. It is loved by developers for having rich features like being multi-paradigm (supporting both functional and OOP style), having a rich type system, and doing away with nil pointers and errors.

However, regardless of how well regarded a certain language is in the industry, rewrites of any system should always be considered very carefully. When it comes to “legacy software”, there is a prevalent assumption that rewriting legacy software is a solution to eliminate technical debt and phase out legacy systems. The reality is often more nuanced.

Legacy code occurs when the developers who originally wrote the code are no longer working on the project. There are often business logic and edge-cases baked into complex legacy codebases of which the context has been lost over time. In practice, rewrites frequently take longer than anticipated and tend to reintroduce bugs and edge cases that must be identified and resolved all over again.

Rewriting vs refactoring has been written at length across the internet, you can read more about it here.

The trade-offs of rewriting need to be properly weighed and balanced. It must take into consideration:

  • How much engineering bandwidth goes into the rewrite?
  • What is the complexity of the rewrite?
  • What tangible benefits are brought about by the rewrite?

Rewriting a system solely for the purpose of “rewriting it in Rust” is not a strong enough business justification.

A legitimate concern was the steep learning curve of Rust, coupled with the risk of having only one team member proficient in the language, which would make its adoption unsustainable.

Therefore, we established a set of guidelines to follow when identifying a suitable system for a potential rewrite:

  • The system must be “simple” enough in functionality. For example, it has one or two main functionalities that can be rewritten in a reasonable amount of time and have its complexity constrained.
  • The system targeted should have large enough traffic such that cost savings brought about by adopting Rust is something tangible when balanced against the effort.
  • The members of the team must be comfortable and willing to pick up the language and achieve a certain level of familiarity to make maintaining the service sustainable.

Finding the right service

The ideal service should have a sufficiently large infrastructure footprint to justify the potential cost savings, while also being straightforward in functionality to minimise time spent on handling edge cases and complex business logic.

Looking across the stack of microservices in Integrity, Counter Service stands out. As its name implies, Counter Service is a service that “counts” and serves the counters for ML models and fraud rules. The original service has two primary functionalities:

  • Consuming from streams, counting events and writing to Scylla.
  • Exposing Google Remote Procedure Call (GRPC) endpoints to query from Scylla (and Redis) and return counts of events based on query keys. For example, BatchRead. BatchRead’s functionality of Counter Service serves up to tens of thousands of QPS at peak and is fairly constrained in functionality. Hence, it fulfilled our target criteria of being “simple” in functionality yet serving a large enough amount of traffic that justifies the ROI of a rewrite.
Figure 1: BatchRead flow of Counter Service, reading data from Scylla, DynamoDB, Redis, MySQL and serving the counters through GRPC.

Rewrite approach

There are a few ways to approach a rewrite in another language. One popular way is to convert your code line by line. If the languages are close enough, it might even be possible to programmatically convert your code like C2Rust.

We decided not to use such an approach for our rewrite. The major reason is that idiomatic Golang was not necessarily idiomatic Rust. We wanted to approach this rewrite with a fresh perspective and treat this as a true rewrite.

We treated the application like a black box, with the interfaces well defined, like GRPC endpoints and contracts. Similar to a function, you could call the API and get a deterministic result, and we had the data that was stored in Scylla.

Based on how we understood the application to work based on its specs and contract, we chose to rewrite the application logic from scratch to meet the API contract and to get as close as identical outputs from the new black box.

OSS library support

We started out by mapping out the key external dependencies and checking how well they were supported in the Rust ecosystem and in open source.

Table 1: List of libraries and their star ratings
Functionality Library Stars (as of Nov 24)
Datadog (Statsd Client) https://github.com/56quarters/cadence < 500
Lightstep (OpenTelemetry) https://github.com/56quarters/cadence > 1000
GRPC Server https://github.com/hyperium/tonic > 500
Web Server https://github.com/actix/actix-web > 20,000
Redis Client https://github.com/aembke/fred.rs (Async Redis Library + Client pool) > 5000
Redis Client https://github.com/redis-rs/redis-rs (“Official” redis client, initially picked but discarded) > 3000
Scylla Client https://github.com/scylladb/scylla-rust-driver ~500
Kafka Client https://github.com/kafka-rust/kafka-rust >1000

All the functionality we need is available through libraries in the Rust ecosystem. However, we found that some libraries are not particularly “popular,” as indicated by their relatively low number of GitHub stars.

The practical concern with using less “popular” libraries is the risk of limited community support or potential abandonment over time. That said, if an “unpopular” library is officially maintained by the associated open-source project—for instance, the Scylla driver has only about 500 stars but is officially provided by the Scylla project—we would need to ensure confidence that it will continue to receive active support.

Out of the list of libraries above, the “unpopular” and unofficial libraries can be narrowed down to two libraries:

  • Datadog – Cadence
  • Redis – Fred

For Datadog, there is no “official” Datadog Rust client. Yet, we picked Cadence as the API looked intuitive and the features we needed were already supported.

In regards to Redis, after testing it, we discovered that the support was not up to par with our requirements. We then opted for a newer and less popular library, fred.rs that seemed to be actively being developed by the community.

Company specific internal libraries

With the vast majority of microservices being written in Golang, most internal libraries are also written in Golang. Opting to rewrite a service in Rust means we are not able to use these internal libraries.

Examples include:

  • An internal configuration library that utilises Go Templates to template configurations for different environments (staging and production).
  • The internal configuration library has its own wrappers and injectors to pull and render secrets.

To overcome this gap and re-use Go Templates and configuration language, we decided to write a simple wrapper and parser using the nom parser combinator to parse the templates and render the config.

Nom poses a steep learning curve. But once familiarised, it is flexible and performant enough to build an equivalent to the internal library. Parser combinators are an interesting subset of tooling that allows you to create some fairly elegant parsers.

Road bumps

The borrow checker

One of the most striking paradigm shifts for developers transitioning to Rust is adapting to the strict rules of the borrow checker, which enforces that variables cannot be reused multiple times unless explicitly cloned or borrowed.

Interestingly, the borrow checker was not the biggest hurdle for new developers. The key is to avoid introducing lifetimes too early in the development process, as this can lead to premature code optimisation.

In many cases, adding a few clones (and occasionally Arcs) can help new developers get up to speed and iterate more quickly during development. The resulting code is usually “fast” enough for initial purposes. After that, the code can be revisited to eliminate unnecessary clones for improved performance. An efficient approach to this can be taken by using Flamegraph to profile your code and identify memory allocation bottlenecks.

Async gotchas

When rewriting Golang logic in Rust, there are fundamental differences in how they treat concurrency and parallelism.

One of Golang’s most remarkable strengths is its ability to deliver high-performance concurrency while preserving simplicity.

There are two fundamental approaches to concurrency in programming languages, namely:

  • Preemptive scheduling (stackful coroutines).
  • Cooperative scheduling (stackless coroutines).

Preemptive vs cooperative scheduling is an in-depth topic with the gist of it being, Golang uses preemptive scheduling and each “Goroutine” has a stack that needs a runtime. The Golang scheduler has the power to “preempt” and “freeze” functions and switch to another stack like stackful coroutine. This is a gross oversimplification of the nuances. For more details, this is a good introduction to the topic.

Rust opts for cooperative scheduling whereby it has no runtime and each coroutine does not maintain a stack. Hence, it has no ability to “freeze” a function and swap context. This allows Rust to be more efficient in terms of memory and resources, as it maintains a state machine. However, the consequence is that this moves the complexity up the stack to the programming language itself. Similar to Javascript, functions are “coloured”, and the developer has to explicitly annotate their functions to be async or sync. Await points need to be explicitly called and control needs to be “yielded” (i.e. cooperative and stackless) so the Rust program knows when it is allowed to stop and swap between coroutines. To read more on this, refer to this and this article for the history of async Rust.

Needing to annotate a function is a classic complaint that is addressed in the article “What Colour is Your Function” that highlights developers’ responsibility to explicitly colour their function and consciously think about blocking vs non-blocking code.

Contrast this with Golang, where you simply need to add the go keyword without thinking about which code might block the execution and use channels to communicate across Goroutines. Golang allows the developer to achieve high performance without much cognitive overhead.

This is especially important for developers new to Rust. As the lack of experience in async and blocking code can be somewhat of a footgun. In the initial rewrite of Rust, we made an amateur mistake of using a synchronous Redis function to call the Redis cache. It resulted in the application performing poorly until we corrected it with the non-blocking asynchronous version using the Fred redis library.

Impact

Following the eventful process of rewriting the service from the ground up in Rust, the outcomes proved to be quite intriguing.

Shadowing traffic to both services as seen in Figure 2, the P99 latency is similar (or perhaps even slightly worse) in the Rust service compared to the original Golang one.

Figure 2: P99 latency comparison between the Golang service (purple) and Rust service (blue).

Normalising the QPS and resource consumption, we see from Table 2 that Rust consumes ~20% of the resources of the original Golang application, resulting in 5x savings in terms of resource consumption.

Table 2: Comparison of resource consumption between Rust and Golang service.
Service Indicative QPS Resources
Original Golang Service 1,000 20 Cores
New Rust Service 1,000 4.5 Cores

Learnings and conclusion

The outcomes and insights from this rewrite have been eye-opening, debunking certain myths while also validating others.

Myth 1: Rust is blazingly fast! Faster than Golang!

Verdict: Disproved.
Golang is “fast enough” for most use cases. It’s a mature language built with concurrency at its core, and it performs exceptionally well in its intended domain. While Rust can outperform Golang due to its higher performance ceiling and finer-grained control, rewriting a Golang service in Rust solely for performance improvements is unlikely to yield significant benefits.

Myth 2: Rust is more efficient than Golang

Verdict: True.
Rewriting a Golang service in Rust will probably give you 50% savings in compute. Rust does fulfill its promise of being memory safe without garbage collection, allowing it to be one of the more efficient languages out there. This is in line with other discoveries in the market.

Myth 3: The learning curve of Rust is too high

Verdict: It depends.
Pure synchronous Rust is fine. As long as you don’t overcomplicate the code and only clone what is needed, it is mostly true. The language is easy enough to pick up for most experienced developers. Even with cloning sprinkled in, the code is usually “fast enough”. The compiler is a good teacher, the compiler error messages are amazing, and if your code compiles, it probably works. Also, the Clippy linter is amazing.

However, introducing async can be challenging. Async is something quite different from what you would encounter in other languages like Go. Improper use of blocking code in async code can result in nuanced bugs that can catch inexperienced Rust developers off-guard.

Evaluating the worth of the rewrite

Yes, the effort was worth it for this service. The trade-off between development effort spent and the cost savings were justified.

As a side effect, the service is 80% cheaper and probably more bug free, as Rust eliminates a class of common Golang errors like Null pointers and concurrent map writes by virtue of the design of the language. If your code compiles, you usually have the confidence that it will work as you expect due to the language being more explicit.

Would we encourage choosing Rust over Golang for new microservices? Absolutely, as the resulting service is likely to be at least 50% more efficient than its Go counterpart. However, this decision presents an important and exciting opportunity for management and leaders to invest in empowering their engineers by equipping them with the skills to master Rust’s unique concepts, such as Async and Lifetimes. While the initial development pace might be slower as the team builds proficiency, this investment can unlock long-term benefits. Once the workforce is skilled in Rust, development speed should align with expectations, and the resulting systems are likely to be more stable and secure, thanks to Rust’s inherent safety features.

Join us

Grab is a leading superapp in Southeast Asia, operating across the deliveries, mobility and digital financial services sectors. Serving over 800 cities in eight Southeast Asian countries, Grab enables millions of people everyday to order food or groceries, send packages, hail a ride or taxi, pay for online purchases or access services such as lending and insurance, all through a single app. Grab was founded in 2012 with the mission to drive Southeast Asia forward by creating economic empowerment for everyone. Grab strives to serve a triple bottom line – we aim to simultaneously deliver financial performance for our shareholders and have a positive social impact, which includes economic empowerment for millions of people in the region, while mitigating our environmental footprint.

Powered by technology and driven by heart, our mission is to drive Southeast Asia forward by creating economic empowerment for everyone. If this mission speaks to you, join our team today!

Streamline Operational Troubleshooting with Amazon Q Developer CLI

Post Syndicated from Kirankumar Chandrashekar original https://aws.amazon.com/blogs/devops/streamline-operational-troubleshooting-with-amazon-q-developer-cli/

Amazon Q Developer is the most capable generative AI–powered assistant for software development, helping developers perform complex workflows. Amazon Q Developer command-line interface (CLI) combines conversational AI with direct access to AWS services, helping you understand, build, and operate applications more effectively. The Amazon Q Developer CLI executes commands, analyzes outputs, and provides contextual recommendations based on best practices for troubleshooting tools and platforms available on your local machine.

In today’s cloud-native environments, troubleshooting production issues often involves juggling multiple terminal windows, parsing through extensive log files, and navigating numerous AWS console pages. This constant context-switching delays problem resolution and adds cognitive burden to teams managing cloud infrastructure.

In this blog post, you will explore how Amazon Q Developer CLI transforms the troubleshooting experience by streamlining challenging scenarios through conversational interactions.

The Traditional Troubleshooting Experience

When issues arise, engineers typically spend hours manually examining infrastructure configurations, reviewing logs across services, and analyzing error patterns. The process requires switching between multiple interfaces, correlating information from various sources, and deep AWS knowledge. This complex workflow often extends problem resolution from hours into days and increase the burden on the infrastructure teams.

Solution: Amazon Q Developer CLI

Amazon Q Developer CLI streamlines the entire troubleshooting process, from initial investigation to problem resolution, making complex AWS troubleshooting accessible and efficient through simple conversations.

How Amazon Q Developer CLI works:

  • Natural Language Interface: Execute AWS CLI commands and interact with AWS services using conversational prompts
  • Automated Discovery: Map out infrastructure and analyze configurations
  • Intelligent Log Analysis: Parse, correlate, and analyze logs across services
  • Root Cause Identification: Pinpoint issues through AI-powered reasoning
  • Guided Remediation: Implement fixes with minimal human intervention
  • Validation: Test solutions and explain complex issues simply

One of the built-in tools within the Amazon Q Developer CLI, use_aws, enables natural language interaction with AWS services, as shown in Figure 1. This tool leverages the AWS CLI permissions configured on your local machine, allowing secure and authorized access to your AWS resources.

A command line interface showing a list of tools and their permissions. The display is titled "/tools" and shows several built-in tools including execute_bash, fs_read, fs_write, report_issue, and use_aws. Each tool has an associated permission level indicated by asterisks. The use_aws tool is highlighted with "trust read-only commands" permission. At the bottom, there's a note stating "Trusted tools will run without confirmation" and a tip to "Use /tools help to edit permissions".

Figure 1: Tools selection in Amazon Q Developer CLI

Real-World Troubleshooting Scenario

Demonstration Environment Setup

This demonstration was performed with the following environment configuration:

The environment includes a local development machine with necessary tools, appropriate AWS account permissions, and terminal access. By starting Amazon Q Developer CLI in the project directory, it has immediate access to relevant code and configuration files.

Scenario: Troubleshooting NGINX 5XX Errors

The scenario demonstrates troubleshooting a multi-tier application architecture as shown in figure 2 deployed on Amazon ECS Fargate with:

  • Application Load Balancer (ALB) distributing traffic across availability zones
  • NGINX reverse proxy service handling incoming requests
  • Node.js backend service processing business logic
  • Service discovery enabling internal communication
  • CloudWatch Logs providing centralized logging

An AWS cloud architecture diagram showing the flow of traffic from an Internet user through multiple components. The diagram includes: At the top: An Internet user connecting to an Internet Gateway Within a VPC (Virtual Private Cloud): Two public subnets containing a NAT Gateway and Application Load Balancer Two private subnets within an ECS Cluster containing: An NGINX service (Fargate) A Backend service (Fargate) A 10-second timeout between them A Cloud Map Service Discovery component at the bottom CloudWatch Logs integration on the right side The diagram includes a note about gateway timeouts: "504 Gateway Timeout - Backend takes 15s to respond, NGINX timeout is 10s" All components are connected with arrows showing the flow of traffic and data through the system. The infrastructure follows AWS best practices with public and private subnet separation for security.

Figure 2: AWS Architecture diagram for the app used in this blog post

Traditional Troubleshooting Steps

For the architecture in figure 2, when 502 Gateway Timeout errors occur, traditional troubleshooting requires:

  1. Checking ALB target group health
  2. Examining ECS service status across multiple consoles
  3. Analyzing CloudWatch logs from different log groups
  4. Correlating error patterns between services
  5. Reviewing infrastructure code for configuration issues
  6. Implementing and deploying fixes

Amazon Q Developer CLI Approach

Instead, let’s see how Amazon Q Developer CLI handles this systematically, step by step:

Step1: Initial Problem Report

Amazon Q Developer CLI is provided with the initial prompt as a problem statement within the application project directory as shown in the following screenshot in figure 3. Amazon Q Developer responds back and says it is going investigate the 502 Gateway Timeout errors in the NGINX application.

Prompt:

Our production NGINX application is experiencing 502 Gateway Timeout errors. 
I have checked out the application and infrastructure code locally and the AWS CLI 
profile 'demo-profile' is configured with access to the AWS account where the 
infrastructure and application is deployed to. Can you help investigate and diagnose the issue?

A Visual Studio Code window showing a debugging session for an NGINX application. The interface has three main sections: a file explorer on the left showing project files including 'app.ts' and 'nginx-config-task.json', a terminal tab in the center displaying an "Amazon Q" ASCII art logo, and a conversation where a user is reporting 502 Gateway Timeout errors. The terminal shows AWS CLI command execution using a tool called "use_aws" with parameters including the service name "ecs" and region "us-west-2". The interface has red annotations highlighting key areas like "project files", "User provided initial prompt", and "Q CLI executing AWS CLI calls.

Figure 3: Amazon Q Developer CLI with initial prompt and problem statement

Step2: Systematic Infrastructure Discovery

Amazon Q Developer CLI start to systematically discovering the infrastructure as shown in the following screenshot in figure 4. If you see the initial prompt did not include that the app is hosted on ECS, but Amazon Q Developer CLI understood the context and executes the AWS CLI calls to describe the Cluster and the services within it. It made sure that the ECS tasks are running for both the services within the Cluster. It is a key discovery that both services show healthy status (1/1 desired count), indicating the issue isn’t service availability.

A terminal window showing three sequential AWS CLI commands being executed through a "use_aws" tool: First command: "list-clusters" operation for ECS service in us-west-2 region using demo-profile, completing in 1.244 seconds Second command: "list-services" operation targeting the NginxSimulationCluster, completing in 0.877 seconds with confirmation of finding both nginx-service and backend-service Third command: "describe-services" operation examining both services in detail, completing in 0.968 seconds with confirmation that both services are running as expected (1/1 desired count) Each command includes execution details, parameters, and completion status, with the system preparing to check CloudWatch logs next.

Figure 4: AWS Infrastructure discovery by Amazon Q Developer CLI

Step 3: Intelligent Log Analysis

Amazon Q Developer CLI retrieves and analyzes recent CloudWatch logs from the NGINX container, immediately identifying the critical error pattern as shown in the following screenshot in figure 5, where Amazon Q Developer responds: “Perfect! I found the issue. The NGINX logs show clear 504 gateway timeout with upstream timeout messages.”

A terminal window showing two AWS CloudWatch Logs commands being executed: First command: "describe-log-streams" operation for the "/ecs/nginx-service" log group, limiting to 5 most recent entries, ordered by LastEventTime in descending order Second command: "get-log-events" operation retrieving 50 log entries from a specific NGINX container log stream The output reveals a critical error message highlighted at the bottom showing an upstream timeout (error 110) occurring while reading response headers. The error details include client IP 10.0.0.247, upstream server at http://10.0.3.18:3000/, and host 52.35.62.210.

Figure 5: CloudWatch Log analysis by Amazon Q Developer CLI

Step 4: Amazon Q Developer CLI Analysis and Root Cause Identification

Amazon Q Developer examines backend service logs and discovers a mismatch between the backend service response time and NGINX timeout settings, as seen in the following screenshot in figure 6.

A terminal window showing AWS CloudWatch Logs commands and their output. The first command describes log streams for a backend container, and the second retrieves log events. The output reveals a debugging analysis showing that while health checks work fine, regular requests are being delayed by about 15 seconds, causing NGINX timeout issues. The log group is "/ecs/backend-service" in the us-west-2 region using a demo-profile.

Figure 6: Root cause identification by Amazon Q Developer CLI

Step 5: Amazon Q Developer CLI Root Cause Analysis

Amazon Q Developer CLI examines the ECS task definitions to identify the exact configuration mismatch, as shown in the following screenshot in figure 7. Amazon Q Developer finds that:

  • Backend service is configured with response_delay=15000 (15 secs)
  • NGINX proxy is configured with proxy_read_timeout 10s

This mismatch causes 504 gateway timeout errors when the backend response exceeds NGINX’s timeout threshold.

A terminal window showing two AWS CLI commands to describe ECS task definitions in the us-west-2 region. Below the commands is a highlighted "Root Cause Analysis" section that explains a timeout mismatch: the backend service is configured with a 15-second response delay while NGINX has a 10-second proxy timeout, resulting in 502 Gateway Timeout errors. Both commands use a demo-profile and are labeled as checking timeout and response delay configurations.

Figure 7: Root cause analysis and issue detection by Amazon Q Developer CLI

Step 6: Automated Code Fix

Here’s where Amazon Q Developer CLI truly excels—it doesn’t just diagnose; it implements the fix. Since Amazon Q Developer CLI is started within the project where the CDK code for ECS task definition is defined, it identified the code configuration and also modified it, as shown in the following screenshot in figure 8.

A terminal window showing file operations using fs_read and fs_write tools. The code changes show an NGINX configuration update in ecs-nginx-cdk.ts, where the proxy_read_timeout is being modified from '10s' to '20s'. The file also shows additional timeout configurations being added, including proxy_connect_timeout and proxy_send_timeout. The update is confirmed with a user prompt and completed in 0.2 seconds.

Figure 8: CDK code fix by Amazon Q Developer CLI

Step 7: Deployment

Amazon Q Developer CLI builds and deploys the fix by executing cdk synth and cdk deploy using the ‘demo-profile‘ AWS CLI profile that was initially provided in the prompt, as shown in the following screenshot in figure 9.

A terminal window showing two execute_bash commands running in sequence. The first command builds a CDK project using 'npm run build' in the nginx-app directory, completing in 4.102s. The second command deploys the updated CDK stack using 'cdk deploy' with the demo-profile, showing deployment progress including some warnings about minHealthyPercent configurations and CloudFormation stack updates in us-west-2 region.

Figure 9: CDK code build and deployment by Amazon Q Developer CLI

Step 8: Validation

Amazon Q Developer CLI validates the solution by sending a curl request to the ALB endpoint after the successful deployment, as shown in the following screenshot in figure 10.

A terminal window showing the execution of a curl command to test an NGINX application on AWS. The command targets an Elastic Load Balancer in the us-west-2 region. The response shows a successful HTTP 200 OK status after 14 seconds, with a JSON response containing the message "Hello from backend". The test completes in 15.100 seconds, indicating the fix for previous 502 errors was successful.

Figure 10: Fix validation by Amazon Q Developer CLI

In addition to that, Amazon Q Developer also sends a request to the health check endpoint and validates everything is working after the fix was deployed, as shown in the following screenshot in figure 11.

A terminal screenshot showing the results of a health check on an Nginx server using curl. The command executed shows a successful response with "healthy" status, completing in 0.65 seconds. The output displays various metrics including download speed (386 B/s), 100% completion rate, and timing statistics for real, user, and system processes.

Figure 11: Health endpoint validation by Amazon Q Developer CLI

What Amazon Q Developer CLI Accomplished

Using just conversational commands, Amazon Q Developer CLI performed a complete troubleshooting cycle:

  • Infrastructure Discovery: Automatically mapped ECS clusters, services, and dependencies
  • Log Correlation: Analyzed thousands of log entries across multiple services
  • Root Cause Analysis: Identified exact configuration mismatch between NGINX’s timeout (10s) and the backend’s response delay (15s)
  • Code-Level Diagnosis: Located problematic timeout setting in CDK infrastructure code
  • Automated Implementation: Modified infrastructure code to increase the NGINX timeout
  • End-to-End Deployment: Built, deployed, and validated the complete solution
  • Comprehensive Testing: Verified both fix effectiveness and overall system health

Amazon Q Developer CLI handles troubleshooting tasks through a single, conversational interface, eliminating the need for multiple tools or AWS CLI commands.

Conclusion

Amazon Q Developer CLI represents a significant evolution in how we troubleshoot cloud infrastructure issues. By combining natural language understanding with powerful command execution capabilities, it transforms complex troubleshooting workflows into efficient, action-oriented dialogues. Whether you’re dealing with NGINX 5XX errors or similar issues across other AWS services, Amazon Q Developer CLI can help you diagnose issues, implement fixes, and validate solutions—all through a conversational interface that feels natural and intuitive.

Give Amazon Q Developer CLI a try the next time you encounter a troubleshooting challenge, and experience the difference it can make in your operational workflow.

To learn more about Amazon Q Developer’s features and pricing details, visit the Amazon Q Developer product page.

About the Author

kirankumar.jpeg

Kirankumar Chandrashekar is a Generative AI Specialist Solutions Architect at AWS, focusing on Amazon Q Developer. Bringing deep expertise in AWS cloud services, DevOps, modernization, and infrastructure as code, he helps customers accelerate their development cycles and elevate developer productivity through innovative AI-powered solutions. By leveraging Amazon Q Developer, he enables teams to build applications faster, automate routine tasks, and streamline development workflows. Kirankumar is dedicated to enhancing developer efficiency while solving complex customer challenges, and enjoys music, cooking, and traveling.

Announcing the new AWS CDK EKS v2 L2 Constructs

Post Syndicated from Matteo Luigi Restelli original https://aws.amazon.com/blogs/devops/announcing-the-new-aws-cdk-eks-v2-l2-constructs/

Introduction

Today, we’re announcing the release of aws-eks-v2 construct, a new alpha version of AWS Cloud Development Kit (CDK) L2 construct for Amazon Elastic Kubernetes Service (EKS). This construct represents a significant change in how developers can define and manage their EKS environments using infrastructure as code. While maintaining the powerful capabilities of its predecessor library for creating and managing EKS clusters, this alpha release introduces key architectural improvements that enhance both flexibility and maintainability.

The AWS Cloud Development Kit (AWS CDK) is an open-source software development framework that enables you to define your cloud infrastructure using familiar programming languages and deploy it through AWS CloudFormation.
The CDK uses constructs – a layered abstraction concept where Layer 1 (L1) constructs map directly to CloudFormation resources, while Layer 2 (L2) constructs provide intuitive APIs, helper functions, best-practice defaults, and generate a lot of the boilerplate code and glue logic for you. This layered approach means you can seamlessly move between high-level abstractions for common use cases and low-level resource definitions when you need fine-grained control. The result is an Infrastructure as Code (IaC) experience that helps you maintain productivity while ensuring you have access to the full power of AWS services when you need it.
You can read more about constructs and their benefits in the CDK user guide.

In this post we’ll explore:

  • The reasoning behind the creation of a new L2 construct for EKS and the improvements introduced by this new library
  • How to use the new EKS v2 construct

Background

Amazon EKS is a managed Kubernetes service that makes it easy to run Kubernetes on AWS without needing to manage the control plane or nodes. EKS automatically handles critical tasks like patching, node provisioning, and upgrades. You can run EKS using EC2 instances for worker nodes, AWS Fargate for serverless containers, or a combination of both, providing the flexibility to choose the right compute option for your workloads.

While the existing EKS L2 construct has served customers well, we identified opportunities to further enhance the developer experience and operational efficiency based on their feedback. The new aws-eks-v2 construct delivers significant improvements through native AWS CloudFormation resources, modern Access Entry-based authentication, and enhanced architectural flexibility. Key benefits include reduced deployment overhead, simplified cluster access management, support for multiple EKS clusters within a single stack, and granular control over resource creation with features like the optional kubectl Lambda handler.
These improvements help customers build and manage their EKS infrastructure more efficiently while maintaining the robust functionality they expect from AWS CDK constructs.

Using the L2

Given that this construct is in the alpha stage, you’ll need to install and import the construct using the experimental construct libraries process. During the alpha stage, the CDK team is actively gathering customer feedback and iterating on the implementation. Once the construct meets our bar for general availability, we’ll integrate it directly into the AWS CDK core library, making it as easily accessible as our other L1 and L2 constructs. This approach allows us to rapidly deliver new capabilities while ensuring they meet the high standards our customers expect.

Deploying EKS Cluster with Default Configuration

Let’s explore how to create an Amazon EKS cluster using AWS CDK aws-eks-v2 construct with minimal configuration requirements. The following example demonstrates the most straightforward way to define an EKS cluster, leveraging the power of CDK’s opinionated defaults.
Creating a new cluster is done using the Cluster construct. The only required property is the Kubernetes version.

import * as eksv2 from '@aws-cdk/aws-eks-v2-alpha';

// Creating an EKS Cluster with default properties
const eksCluster = new eksv2.Cluster(this, 'EksCluster', {
    version: eksv2.KubernetesVersion.V1_32
});

This translates in the following Architecture as shown in figure 1:

L2 CDK Construct v2 for EKS - Default Architecture

Figure 1 – L2 CDK construct v2 for EKS, Default Architecture

  • Amazon Virtual Private Cloud (VPC) – A logically isolated section of the AWS Cloud that spans across two Availability Zones, equipped with an Internet Gateway to enable secure communication with the internet. This multi-AZ design helps ensure your applications remain available even if an Availability Zone experiences issues.
  • Amazon EKS Control Plane – A fully managed Kubernetes control plane deployed in an AWS-managed VPC , providing high availability and automatic version management for the Kubernetes control plane components.
  • Public Subnet Infrastructure – Two public subnets, each with its own NAT Gateway Instance, enabling your cluster components to securely access the internet for essential operations like pulling container images and downloading updates. These NAT Gateways provide a secure outbound path while protecting your workloads from direct internet exposure.
  • Private Subnet Configuration – Two private subnets optimized for running your EKS worker nodes, offering enhanced security by isolating your workloads from direct internet access while maintaining the ability to communicate with AWS services and the internet through the NAT Gateways.
  • IAM Security Foundation – A comprehensive set of IAM roles and policies that implement the principle of least privilege:
    • Control plane service role that enables EKS to manage AWS resources on your behalf
    • Node IAM role that allows worker nodes to interact with other AWS services and join the EKS cluster

You can also use FargateCluster to provision a cluster that uses only Fargate workers.

import * as eksv2 from '@aws-cdk/aws-eks-v2-alpha';

// Creating an EKS Cluster with default properties and Fargate workers
const eksFargateCluster = new eksv2.FargateCluster(this, 'EksFargateCluster', {
   version: eksv2.KubernetesVersion.V1_32,
});

To help our customers maintain better control over their cluster access patterns, the Kubectl Handler is not automatically deployed with the default configuration. You can easily enable this functionality by configuring the kubectlProviderOptions property when you need kubectl access management as shown below.

import * as eksv2 from '@aws-cdk/aws-eks-v2-alpha';
import { KubectlV32Layer } from '@aws-cdk/lambda-layer-kubectl-v32'

// Creating an EKS Cluster with default properties and kubectl handler
const eksCluster = new eksv2.Cluster(this, 'EksCluster', {
   version: eksv2.KubernetesVersion.V1_32,
   kubectlProviderOptions: {
      kubectlLayer: new KubectlV32Layer(this, 'KubectlLayer')
   },
});

Deploying EKS Cluster with AutoMode

EKS Auto Mode represents a significant advancement in how Amazon EKS manages compute capacity for Kubernetes clusters. This intelligent capacity management system automatically provisions and scales node groups based on workload demands, removing the need for manual capacity planning.

When you create a new cluster with the aws-eks-v2 construct, EKS Automode is activated by default, by means that DefaultCapacityType.AUTOMODE is automatically set as the default capacity type for the EKS Cluster. If you prefer, you can specify the defaultCapacityType to AutoMode:

import * as eksv2 from '@aws-cdk/aws-eks-v2-alpha';

// Creating an EKS Cluster with AutoMode
const eksCluster = new eksv2.Cluster(this, 'EksCluster', {
   version: eksv2.KubernetesVersion.V1_32,
   defaultCapacityType: eksv2.DefaultCapacityType.AUTOMODE, // default value
});

After deploying the Stack containing the construct instance, in the EKS Console you’ll be able to see that an EKS Cluster has been created with AutoMode enabled:

EKS Cluster Deployed with Automode

Figure 2 – EKS Cluster Deployed with Automode

Auto Mode enhances your Amazon EKS experience by automatically configuring two strategically designed node pools out of the box:

  • A system node pool optimized for running critical cluster system components and add-ons, ensuring reliable cluster operations.
  • A general node pool specifically tuned for your application workloads, providing the flexibility needed for diverse containerized applications.

You can configure which node pools to enable through the compute property:

import * as eksv2 from '@aws-cdk/aws-eks-v2-alpha';

// Creating an EKS Cluster with Automode and selecting nodePools
const eksCluster = new eksv2.Cluster(this, 'EksCluster', {
   version: eksv2.KubernetesVersion.V1_32,
   defaultCapacityType: eksv2.DefaultCapacityType.AUTOMODE,
   compute: {
      nodePools: ['system', 'general-purpose'],
   },
});

Deploying EKS Cluster with Managed Node Groups

Amazon EKS Managed Node Groups deliver a seamless compute management experience for your Kubernetes clusters. This powerful capability eliminates operational complexity by automating the end-to-end lifecycle of Amazon EC2 instances that power your containerized applications.
Behind the scenes, Amazon EKS managed node groups intelligently orchestrate these changes, ensuring zero-disruption to your applications through graceful node draining. The service automatically leverages the latest Amazon EKS-optimized AMIs, providing a secure and optimized foundation for your workloads.

By setting defaultCapacityType to NODEGROUP, customers can leverage the traditional managed node group management approach:

import * as eksv2 from '@aws-cdk/aws-eks-v2-alpha';

// Creating an EKS Cluster with Managed Node Groups and default instance types
const eksCluster = new eksv2.Cluster(this, 'EksCluster', {
   version: eksv2.KubernetesVersion.V1_32,
   defaultCapacityType: eksv2.DefaultCapacityType.NODEGROUP,
});

By default, when using DefaultCapacityType.NODEGROUP, this library will allocate a managed node group with two m5.large instances.
After deploying the above code, you can check the EKS Console to see that an EKS Cluster has been deployed as shown in figure 3:

EKS Cluster Deployed with Managed Node Groups

Figure 3 – EKS Cluster Deployed with Managed Node Groups

You can also check the Compute tab and see the Managed Node Group Configuration as shown in figure 4:

EKS Cluster Managed Node Group Default Configuration

Figure 4 – EKS Cluster Managed Node Group Default Configuration

If you want to have control over instance types of a Managed Node Group, you can specify the default EC2 type as property of the construct:

import * as eksv2 from '@aws-cdk/aws-eks-v2-alpha';
import * as ec2 from 'aws-cdk-lib/aws-ec2'

// Creating an EKS Cluster with Managed Node Groups and specific instance types
const eksCluster = new eksv2.Cluster(this, 'EksCluster', {
   version: eksv2.KubernetesVersion.V1_32,
   defaultCapacityType: eksv2.DefaultCapacityType.NODEGROUP,
   defaultCapacity: 5,
   defaultCapacityInstance: ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.SMALL),
});

You can also specify additional customizations after the EKS cluster declaration, via the addNodegroupCapacity method:

import * as eksv2 from '@aws-cdk/aws-eks-v2-alpha';
import * as ec2 from 'aws-cdk-lib/aws-ec2'

// Creating an EKS Cluster with Managed Node Groups and specific instance types
const eksCluster = new eksv2.Cluster(this, 'EksCluster', {
   version: eksv2.KubernetesVersion.V1_32,
   defaultCapacityType: eksv2.DefaultCapacityType.NODEGROUP,
   defaultCapacity: 0,
});

eksCluster.addNodegroupCapacity('custom-node-group', {
  instanceTypes: [new ec2.InstanceType('m5.large')],
  minSize: 4,
  diskSize: 100,
});

Managing Permissions through Access Entries

The new aws-eks-v2 construct transitions away from the previous ConfigMap-based authentication (which is deprecated in EKS) in favor of the Access Entries Authentication mode. This change introduces Access Entry as the standardized method for managing cluster permissions, offering a more streamlined and secure approach to granting cluster access to IAM users and roles.

You can define Access Policies through the AccessPolicy construct and you can adjust the scope of the Access Policy to the entire EKS cluster or to specific EKS Namespaces:

import * as eksv2 from '@aws-cdk/aws-eks-v2-alpha';

// AmazonEKSClusterAdminPolicy with `cluster` scope
eks.AccessPolicy.fromAccessPolicyName('AmazonEKSClusterAdminPolicy', {
   accessScopeType: eks.AccessScopeType.CLUSTER,
});

// AmazonEKSAdminPolicy with `namespace` scope
eks.AccessPolicy.fromAccessPolicyName('AmazonEKSAdminPolicy', {
   accessScopeType: eks.AccessScopeType.NAMESPACE,
   namespaces: ['foo', 'bar'] 
});

You can then grant access to specific IAM Roles using the grantAccess method:

import * as iam from 'aws-cdk-lib/aws-iam'

// Defining a IAM Role
const clusterAdminRole = new iam.Role(this, 'ClusterAdminRole', {
   assumedBy: new iam.ArnPrincipal('arn_for_trusted_principal'),
});

// Creating an EKS Cluster with AutoMode
const eksCluster = new eksv2.Cluster(this, 'EksCluster', {
   version: eksv2.KubernetesVersion.V1_32,
   defaultCapacityType: eksv2.DefaultCapacityType.AUTOMODE,
});

// Cluster Admin role for this cluster
eksCluster.grantAccess('clusterAdminAccess', clusterAdminRole.roleArn, [
	eks.AccessPolicy.fromAccessPolicyName('AmazonEKSClusterAdminPolicy', {
   	    accessScopeType: eks.AccessScopeType.CLUSTER,
    }),
]);

When the Principal assumes the ClusterAdminRole, it receives seamless access to the EKS cluster through a carefully orchestrated permission chain. This access is governed by the AmazonEKSClusterAdminPolicy, which is automatically attached to the Access Policy linked to the IAM Role.

Conclusion

In this post, we introduced the new AWS CDK L2 construct (aws-eks-v2) for Amazon EKS, demonstrating how it simplifies cluster deployment while offering enhanced flexibility and operational efficiency. Through practical examples, we showcased how customers can leverage the construct’s intelligent defaults and customization options to build production-ready Kubernetes environments on AWS

The new L2 construct for Amazon EKS delivers significant improvements that help customers accelerate their container adoption journey:

  • Enhanced Performance: Eliminates dependency on Custom Resources and AWS Lambda functions by utilizing native AWS CloudFormation resources, resulting in faster and more reliable deployments.
  • Modern Authentication: Implements Access Entry-based authentication, replacing the deprecated ConfigMap approach with a more secure and programmable solution.
  • Improved Scalability: Removes the single-cluster-per-stack limitation and eliminates nested stacks, enabling more flexible architectural patterns.
  • Optimized Resource Creation: Makes the kubectl Lambda handler optional, giving customers fine-grained control over their infrastructure components.
  • Streamlined Operations: Provides automated node group management with intelligent defaults while maintaining full customer control when needed.

To get started with the new EKS L2 construct, visit the AWS CDK documentation. If you have specific features you’d like to see added, we encourage you to submit a feature request in the aws-cdk GitHub repository. Your feedback helps us continue innovating on your behalf.

About the author

Matteo Luigi Restelli

Matteo Luigi Restelli is an AWS Sr. Partner Solutions Architect. He mainly focuses on Italian Consulting AWS Partners and is also specialized in Infrastructure as Code, Cloud Native App Development and DevOps. Outside of work, he enjoys swimming, rock & roll music and learning something new everyday especially in Computer Science space.

Accelerate development with secure access to Amazon Q Developer using PingIdentity

Post Syndicated from Sid Vantair original https://aws.amazon.com/blogs/devops/accelerate-development-with-secure-access-to-amazon-q-developer-using-pingidentity/

 Overview

Customers adopting Amazon Q Developer, a generative AI-powered coding companion, often need authentication through existing identity providers like PingIdentity. By leveraging AWS IAM Identity Center, organizations can enable their developers to access Amazon Q Developer with their existing PingIdentity credentials, streamlining authentication and removing the need for separate login procedures. Amazon Q Developer can chat about code, provide inline code completions, and generate new code. It also scans your code for security vulnerabilities and makes code improvements, including language updates, debugging, and optimizations. Amazon Q Developer comes in two tiers. The Free Tier is available at no cost for individual use. The Pro Tier is a paid version offering enterprise access controls, an analytics dashboard, customization, and higher usage limits. Organizations that enable the Pro tier of Amazon Q Developer for their developers typically authenticate with AWS IAM Identity Center. This approach is popular due to its ability to federate with external identity providers. In this blog, we will show you how to set up PingIdentity as an external IdP for IAM Identity Center and allow developers to access Amazon Q Developer using their existing PingIdentity login credentials.

How it works

AWS authentication flow diagram: Developers interact with Amazon Q Developer and AWS IAM Identity Center, integrating with Ping Identity for SAML-based access.

Figure 1 – Solution Overview

The authentication workflow is as follows:

  1. The developer initiates an access request to Amazon Q Developer.
  2. IAM Identity center checks authentication status.
  3. If not authenticated, redirects to PingIdentity login.
  4. Developer provides PingIdentity Credentials.
  5. PingIdentity validates credentials and sends SAML response.
  6. IAM Identity Center verifies the SAML response.
  7. Upon successful verification, grants Amazon Q Developer access.
  8. Developer begins using Amazon Q Developer.

Prerequisites

  • AWS account
  • PingIdentity environment with users and groups already setup for Amazon Q Developer access
  • IAM identity center
  • Pro Tier subscription of Amazon Q Developer

Walkthrough

In this section, we demonstrate how to create a SAML-based connection between PingIdentity and IAM Identity Center, enabling you to access Amazon Q Developer seamlessly using your PingIdentity credentials.

Note: You will need to switch between PingIdentity portal and IAM Identity Center in your browser. We recommend opening a new browser tab for each console.

Step 1: Enable AWS Single Sign-On in PingIdentity

This step involves enabling AWS Single Sign-On application within PingIdentity.

    1. In the PingIdentity console, Navigate to the Applications Tab > Application Catalog
    2. Browse catalog for AWS Single Sign-On and select + to start the Quick Setup.
Screenshot of the PingIdentity Application Catalog interface. The search term "aws" is entered in the search bar, displaying three results: Amazon Web Services – AWS, AWS Gov-Cloud, and AWS Single Sign-On. The "AWS Single Sign-On" option is outlined with a red box and includes a plus button to add the application

Figure 2 – PingIdentity Application Catalog

Alt Text: Screenshot of the PingIdentity Application Catalog interface. The search term “aws” is entered in the search bar, displaying three results: Amazon Web Services – AWS, AWS Gov-Cloud, and AWS Single Sign-On. The “AWS Single Sign-On” option is outlined with a red box and includes a plus button to add the application

    1. Provide Name, SSO Region and SSO Tenant ID and choose Next
      • Name – Input an appropriate name for the connection
      • SSO Region – Input the appropriate region
      • Tenant ID – Identity Store ID
        You can run the following CLI command to retrieve the value. It’s a 10-digit alphanumeric prefixed by “d-“.
aws sso-admin list-instances –query ‘Instances[0].IdentityStoreId’Output: “d-XXXXXXXXXX”
    1. Navigate to PingOne Mappings and select Email Address from the drop down.
Screenshot of the AWS Single Sign-On configuration in PingIdentity. The screen shows Step 2 of the setup process where the SAML attribute SAML_SUBJECT is mapped to the PingOne attribute "Email Address". A red box highlights the mapping section under "PingOne Mappings".

Figure 3 – AWS Single Sign-On attribute mapping

Alt Text: Screenshot of the AWS Single Sign-On configuration in PingIdentity. The screen shows Step 2 of the setup process where the SAML attribute SAML_SUBJECT is mapped to the PingOne attribute “Email Address”. A red box highlights the mapping section under “PingOne Mappings”.

    1. Search and select the group that you have created earlier for enabling access to Amazon Q Developer and select + to add the group.
    2. Choose Save
Screenshot of Step 3 in the AWS Single Sign-On setup process in PingIdentity. The screen shows the group selection interface where the "Amazon Q" group is listed. A plus icon is shown next to the group to add it, and a blue "Save" button is highlighted in the bottom-right corner to confirm the configuration.

Figure 4 – Select PingIdentity directory Groups for Amazon Q Developer access

Alt Text: Screenshot of Step 3 in the AWS Single Sign-On setup process in PingIdentity. The screen shows the group selection interface where the “Amazon Q” group is listed. A plus icon is shown next to the group to add it, and a blue “Save” button is highlighted in the bottom-right corner to confirm the configuration.

Step 2: Connecting PingIdentity with IAM identity Center

This step involves configuring PingIdentity with the AWS IAM Identity Center sign-on details to complete the authentication setup.

  1. In the PingIdentity console, Navigate to the Applications Tab > Applications and select the application you created earlier in Step 1
  2. Select Enable Advanced Configuration and choose Enable.
Screenshot of the PingIdentity Applications dashboard showing the AWS Single Sign-On application selected. The overview panel displays key configuration sections including protocol (SAML), mapped attributes, selected policies, and access group (Amazon Q). The option "Enable Advanced Configuration" is highlighted near the bottom of the panel.

Figure 5 – Enable Advanced configuration for AWS single Sign-On application

Alt Text: Screenshot of the PingIdentity Applications dashboard showing the AWS Single Sign-On application selected. The overview panel displays key configuration sections including protocol (SAML), mapped attributes, selected policies, and access group (Amazon Q). The option “Enable Advanced Configuration” is highlighted near the bottom of the panel.

  1. Scroll down and select Download Metadata. This will save the Metadata file to your local computer, which you will use later during the configuration process.
  2. In another browser tab login to your AWS IAM Identity Center console and Select Choose your identity source.
  3. Under Identity source, select Change identity source from the Actions drop-down menu.
Screenshot of the IAM Identity Center settings page, focused on the "Identity source" tab. The page displays details such as identity source, authentication method, AWS access portal URL, issuer URL, and identity store ID. A dropdown menu labeled "Actions" is expanded in the top-right corner, showing options to "Customize AWS access portal URL" and "Change identity source," highlighted with a red box.

Figure 6 – Change identity source in IAM Identity Center Console

Alt Text: Screenshot of the IAM Identity Center settings page, focused on the “Identity source” tab. The page displays details such as identity source, authentication method, AWS access portal URL, issuer URL, and identity store ID. A dropdown menu labeled “Actions” is expanded in the top-right corner, showing options to “Customize AWS access portal URL” and “Change identity source,” highlighted with a red box.

  1. On the next page, select External identity provider and choose Next.
  2. Under Service provider metadata copy the IAM Identity Center Assertion Consumer Service (ACS) URL.

    Screenshot of the "Configure external identity provider" step in the AWS IAM Identity Center setup process. The screen displays service provider metadata including the AWS access portal sign-in URL, IAM Identity Center Assertion Consumer Service (ACS) URL (highlighted with a red box), and IAM Identity Center issuer URL. A button labeled "Download metadata file" is shown in the upper right.

    Figure 7 – Copy IAM Identity Center ACS URL

Alt Text: Screenshot of the “Configure external identity provider” step in the AWS IAM Identity Center setup process. The screen displays service provider metadata including the AWS access portal sign-in URL, IAM Identity Center Assertion Consumer Service (ACS) URL (highlighted with a red box), and IAM Identity Center issuer URL. A button labeled “Download metadata file” is shown in the upper right.

  1. Now go back to the PingIdentity browser tab and Navigate to the Configuration tab and select pencil icon to edit the details.
  2. Paste the ACS URL you copied from the IAM identity center console and choose Save.
Screenshots showing the configuration and editing of SAML settings for AWS Single Sign-On in PingIdentity. The first image displays the static configuration view, listing the ACS URL, signing key ("PingOne SSO Certificate for Administrators environment"), signing method ("Response"), and signing algorithm. The second image shows the editable configuration screen with the ACS URL input field highlighted in red, alongside dropdowns for selecting the signing key, options for signing method (Assertion, Response, or both), and the RSA_SHA256 signing algorithm. These screens guide users through setting up secure SAML integration with AWS SSO.

Figure 8 – Configuring AWS Single Sign-On SAML Settings in PingIdentity console

Alt Text: Two screenshots showing the configuration and editing of SAML settings for AWS Single Sign-On in PingIdentity. The first image displays the static configuration view, listing the ACS URL, signing key (“PingOne SSO Certificate for Administrators environment”), signing method (“Response”), and signing algorithm. The second image shows the editable configuration screen with the ACS URL input field highlighted in red, alongside dropdowns for selecting the signing key, options for signing method (Assertion, Response, or both), and the RSA_SHA256 signing algorithm. These screens guide users through setting up secure SAML integration with AWS SSO.

Step 3: Configure PingIdentity as external IdP in IAM identity Center

This step involves setting up PingIdentity as an external IdP in IAM Identity Center to enable federated access.

  1. Navigate back to the previous browser tab where you had IAM Identity Center console open.
  2. Upload the downloaded PingIdentity IdP SAML metadata file from step 3 of previous section and select Next.
Screenshot of the AWS Identity Center configuration screen where the user uploads the IdP SAML metadata XML file. The metadata file is shown as successfully selected. Below are empty fields for optional manual entry of IdP sign-in URL, IdP issuer URL, and IdP certificate. The "Next" button is highlighted in orange at the bottom right, indicating the next step in the setup process.

Figure 9 – AWS IAM Identity Center metadata

Alt Text: Screenshot of the AWS Identity Center configuration screen where the user uploads the IdP SAML metadata XML file. The metadata file is shown as successfully selected. Below are empty fields for optional manual entry of IdP sign-in URL, IdP issuer URL, and IdP certificate. The “Next” button is highlighted in orange at the bottom right, indicating the next step in the setup process.

  1. Review the list of changes. Once you are ready to proceed, type ACCEPT, then select Change identity source.

Step 4: Enable provisioning and identity-aware sessions in IAM identity Center

This step involves configuring user provisioning and enabling identity-aware sessions in AWS IAM Identity Center to support dynamic access control.

  1. In IAM Identity Center Console, Choose Settings in the left navigation pane.
  2. On the Settings page, locate and enable automatic provisioning. This immediately enabled automatic provisioning in IAM Identity Center and displays the necessary SCIM endpoint and access token information.
  3. In the Inbound automatic provisioning dialog box, copy each of the values for the following options. You will need to paste these later when you configure provisioning in PingIdentity.
    • SCIM endpoint
    • Access token
  4. Choose Close.
  5. Next enable identity-aware sessions and automatic provisioning.
Two options are displayed for further configuration: "Enable identity-aware sessions" and "Automatic provisioning." Both options have an "Enable" button on the right-hand side, highlighted in red.

Figure 10 – IAM Identity Center Settings for identity aware sessions and automatic provisioning

Alt Text: Two options are displayed for further configuration: “Enable identity-aware sessions” and “Automatic provisioning.” Both options have an “Enable” button on the right-hand side, highlighted in red.

Step 5: Configure connections provisioning in PingIdentity

This step involves setting up connection provisioning in PingIdentity to enable automatic user and group management.

  1. In the PingIdentity console, Navigate to the Integrations > Provisioning.
  2. Select plus icon > New Connection
  3. Under connection type Select Identity Store.
PingIdentity Provisioning configuration screen. The left sidebar highlights the "Provisioning" tab. The main panel shows the "Create a New Connection" dialog with two connection type options: "Identity Store" and "Gateway." The "Identity Store" option is selected using the "Select" button on the right. A plus (+) icon at the top indicates the option to add a new provisioning connection.

Figure 11 – PingIdentity connection provisioning

Alt Text: PingIdentity Provisioning configuration screen. The left sidebar highlights the “Provisioning” tab. The main panel shows the “Create a New Connection” dialog with two connection type options: “Identity Store” and “Gateway.” The “Identity Store” option is selected using the “Select” button on the right. A plus (+) icon at the top indicates the option to add a new provisioning connection.

  1. Select SCIM outbound from the list of options and select Next.
  2. Provide a name for the connection and select Next.
  3. Paste the SCIM endpoint URL into the SCIM BASE URL field.
  4. Navigate to Authentication Method and select OAuth 2 Bearer Token.
  5. Paste the Access token into the Oauth Access Token field.
  6. Select Test Connection to validate the connectivity and select Next.
PingIdentity interface showing the "Configure Authentication" step in the "Create a New Connection" wizard. Key fields include the SCIM Base URL, SCIM Version (2.0), Authentication Method (OAuth 2 Bearer Token), OAuth Access Token (obscured), and resource paths for Users and Groups. The "Test Connection" and "Next" buttons are visible at the bottom.

Figure 12 – Configure authentication details

Alt Text: PingIdentity interface showing the “Configure Authentication” step in the “Create a New Connection” wizard. Key fields include the SCIM Base URL, SCIM Version (2.0), Authentication Method (OAuth 2 Bearer Token), OAuth Access Token (obscured), and resource paths for Users and Groups. The “Test Connection” and “Next” buttons are visible at the bottom.

  1. Navigate to User Filter Expression and change to userName Eq “%s”.
  2. Choose Save. By default, the connection is created in a Disabled state.
Final step in the PingIdentity "Create a New Connection" wizard showing the "Configure Preferences" screen. The highlighted fields include "User Filter Expression" with the value userName Eq "%s", "User Identifier" set to userName, and group membership handling options ("Merge" and "Overwrite" with "Overwrite" selected). A "Save" button is highlighted at the bottom right.

Figure 13 – Edit UserFilter Expressions for the connection

Alt Text: Final step in the PingIdentity “Create a New Connection” wizard showing the “Configure Preferences” screen. The highlighted fields include “User Filter Expression” with the value userName Eq “%s”, “User Identifier” set to userName, and group membership handling options (“Merge” and “Overwrite” with “Overwrite” selected). A “Save” button is highlighted at the bottom right.

  1. Select the connection you created and select the toggle switch to enable the connection.
PingIdentity configuration screen showing the IAM Identity Store integration. The page displays the identity store name, and tabs for "Overview" and "Configuration." A toggle switch in the top-right corner is highlighted, indicating the integration is currently enabled.

Figure 14 – Enable the connection

Alt Text: PingIdentity configuration screen showing the IAM Identity Store integration. The page displays the identity store name, and tabs for “Overview” and “Configuration.” A toggle switch in the top-right corner is highlighted, indicating the integration is currently enabled.

Step 6: Configure rules provisioning in PingIdentity

This step involves setting up provisioning rules in PingIdentity to define how users and groups are synchronized.

  1. In the PingIdentity console, Navigate to the Integrations > Provisioning.
  2. Select plus icon > New Rule
  3. Provide a Name and Description for the rule.
  4. Choose Create.
  5. Select plus icon to select the Connection you created in the previous step.
  6. Choose Save.
Alt Text: Screenshots showing the final steps in connecting the IAM Identity Center to the IAM identity store using PingIdentity. The first image shows the IAM Identity Store connection listed under "Available Connections" with a plus (+) icon to initiate the link. The second image shows the selected connection from the PingOne Directory (P1) as the source and IAM identity store (SCIM) as the target, with the option to "Save" the configuration.

Figure 15 – Add the IAM identity center connection to the rule

Alt Text: Screenshots showing the final steps in connecting the IAM Identity Center to the IAM identity store using PingIdentity. The first image shows the IAM Identity Store connection listed under “Available Connections” with a plus (+) icon to initiate the link. The second image shows the selected connection from the PingOne Directory (P1) as the source and IAM identity store (SCIM) as the target, with the option to “Save” the configuration.

  1. If you want to sync users from your PingIdentity directory, create a user filter. To do so, navigate to User Filter and select pencil icon to edit the settings.
  2. Choose the appropriate filter from the drop down based on your use case and select Save. I have chosen Group Name which has been designated for Amazon Q Developer access.
Screenshot of the "Edit User Filter" interface in IAM Identity Center. The user filter is configured to provision users who belong to a group with names that contain "Amazon Q Developer." The condition logic is set to match if "Any" of the conditions are true.

Figure 16 – PingIdentity user filter

Alt Text: Screenshot of the “Edit User Filter” interface in IAM Identity Center. The user filter is configured to provision users who belong to a group with names that contain “Amazon Q Developer.” The condition logic is set to match if “Any” of the conditions are true.

  1. If you want to sync a group from your PingIdentity directory, create group provisioning. To do so, navigate to Group Provisioning and select pencil icon to edit the settings.
  2. Select the appropriate group which has been designated for Amazon Q Developer access and choose Save.
Screenshot of the "Edit Group Provisioning" screen in IAM Identity Center. The group "Amazon Q Developer" is selected for outbound provisioning. A "Save" button is highlighted in the bottom-left corner.

Figure 17 – PingIdentity Group Provisioning

Alt Text: Screenshot of the “Edit Group Provisioning” screen in IAM Identity Center. The group “Amazon Q Developer” is selected for outbound provisioning. A “Save” button is highlighted in the bottom-left corner.

  1. Navigate to Attribute Mapping and select the pencil icon to edit the settings.
  2. Delete the PingOne Directory attribute Primary Phone.
  3. Add a new attribute and select Username as PingOne Directory and displayName as IAM identity Store.
  4. Choose Save.
Two screenshots showing the editing of attribute mappings in IAM Identity Center. The first image displays default mappings such as 'Email Address' to 'workEmail' and 'Username' to 'userName', with an option to delete or update each field. The second image shows the addition of a new attribute mapping from 'Username' to 'displayName', along with highlighted 'Add' and 'Save' buttons.

Figure 18 – PingIdentity attribute mapping

Alt Text: Two screenshots showing the editing of attribute mappings in IAM Identity Center. The first image displays default mappings such as ‘Email Address’ to ‘workEmail’ and ‘Username’ to ‘userName’, with an option to delete or update each field. The second image shows the addition of a new attribute mapping from ‘Username’ to ‘displayName’, along with highlighted ‘Add’ and ‘Save’ buttons.

  1. Select the rule you created and select the toggle switch to enable the rule.
  2. This automatically provisions the users/groups from PingIdentity to IAM identity Center using SCIM.
IAM Identity Center sync summary showing successful user and group provisioning. The first image highlights two users impacted and successfully synced. The second image highlights one group impacted and successfully synced. Sync status is marked 'ACTIVE' in both views, confirming successful integration between PingOne and AWS IAM Identity Center.

Figure 19 – PingIdentity Users and Groups Sync status using SCIM

Alt Text: IAM Identity Center sync summary showing successful user and group provisioning. The first image highlights two users impacted and successfully synced. The second image highlights one group impacted and successfully synced. Sync status is marked ‘ACTIVE’ in both views, confirming successful integration between PingOne and AWS IAM Identity Center.

Step 7: Provide access to Amazon Q Developer

This step involves locating and subscribing the groups that need permission to use Amazon Q Developer.

  1. In the Amazon Q Developer console, under Subscriptions add the IAM identity center groups which require access to Amazon Q Developer.
  2. Select Subscribe and search for the group name.
  3. Select Assign.
Screenshot of the Amazon Q Developer Subscriptions page in the AWS Management Console. The "Groups" tab is selected, displaying “Amazon Q Developer,” with a subscription status of “Subscribed.” The “Amazon Q Developer” group is highlighted with a red box.

Figure 20 – Amazon Q Developer subscriptions page

Alt Text: Screenshot of the Amazon Q Developer Subscriptions page in the AWS Management Console. The “Groups” tab is selected, displaying “Amazon Q Developer,” with a subscription status of “Subscribed.” The “Amazon Q Developer” group is highlighted with a red box.

Setup Amazon Q Developer with IAM Identity Center

This section guides you through installing the Amazon Q Developer extension and setting up authentication with IAM Identity Center.

  1. To set up Amazon Q Developer extension in your integrated development environment (IDE), complete the steps in AWS documentation.
  2. Once extension is installed Choose Amazon Q icon in your IDE.
  3. Choose a sign-in option.
  4. Select Use with Pro license and choose
  5. Continue.
  6. Provide the Start URL. You can retrieve this AWS access portal URL from the IAM Identity Center Console.
Screenshot of the IAM Identity Center settings page in the AWS Console, displaying the identity source configuration. It shows that the identity source is set to "External identity provider" with SAML 2.0 authentication and SCIM provisioning. The highlighted section includes the AWS access portal URL and the Identity Store ID. The "Settings" tab is selected in the left navigation pane.

Figure 21 – IAM identity center access portal URL

Alt Text: Screenshot of the IAM Identity Center settings page in the AWS Console, displaying the identity source configuration. It shows that the identity source is set to “External identity provider” with SAML 2.0 authentication and SCIM provisioning. The highlighted section includes the AWS access portal URL and the Identity Store ID. The “Settings” tab is selected in the left navigation pane.

  1. Provide the region that hosts the identity directory and choose Continue
  2. Select Open on the resulting pop up which redirects to your browser.
  3. The browser redirects you to the Pingone URL where you enter your PingIdentity credentials and select Sign On.
  4. Upon successful authentication, select Allow access on the resulting pop up to login successfully.
A screen recording of Visual Studio Code where the user selects the Amazon Q icon from the sidebar. The screen transitions to a login prompt indicating that the user must authenticate using their PingIdentity credentials via IAM Identity Center before accessing Amazon Q Developer features. The message highlights that authentication is required to continue.

Figure 22 – Setup Visual Studio Code Amazon Q Developer extension

Alt Text: A screen recording of Visual Studio Code where the user selects the Amazon Q icon from the sidebar. The screen transitions to a login prompt indicating that the user must authenticate using their PingIdentity credentials via IAM Identity Center before accessing Amazon Q Developer features. The message highlights that authentication is required to continue.

Test Configuration

Upon successfully completing the previous step, you can now leverage the code suggestions by Amazon Q Developer.

A screen recording of Visual Studio Code where Amazon Q Developer generates a sample code inline.

Figure 23 – Amazon Q Developer example

Alt Text: A screen recording of Visual Studio Code where Amazon Q Developer generates a sample code inline.

Clean Up

To avoid ongoing charges after testing this solution, follow these steps to remove all provisioned resources:1. Remove PingIdentity Application Configuration

  • In the PingIdentity console, navigate to Applications.
  • Locate and delete the AWS Single Sign-On application that was configured for IAM Identity Center integration.

2. Reset IAM Identity Center Configuration

  • In the AWS IAM Identity Center console:
    • Navigate to Settings > Identity source.
    • Change the identity source back to the default IAM Identity Center directory if no longer using PingIdentity.
    • Remove any external metadata and configuration uploaded during the setup.

3. Revoke Subscriptions and Access

  • In the Amazon Q Developer console:
    • Go to Subscriptions and remove assigned groups such as Amazon Q Developer or code whisperer trial.
    • This will deactivate access and prevent any future charges tied to those subscriptions.

4. Remove Amazon Q Developer Extension

  • If desired, uninstall the Amazon Q Developer extension from Visual Studio Code to fully revert the development environment.

Conclusion

In this post, we demonstrated how to use existing PingIdentity credentials to access Amazon Q Developer through integration with IAM Identity Center. We provided a step-by-step guide for configuring PingIdentity as an external identity provider (IdP) with IAM Identity Center. Lastly, we demonstrated how to connect Amazon Q Developer extension within your IDE to AWS using your PingIdentity credentials, allowing seamless access to Amazon Q Developer.If you have any comments or questions, share them in the comments section.

To learn more about AWS Services

Amazon Q Developer

IAM Identity Center

AWS Toolkit for Visual Studio Code


About the author

Sid Vantair is a Solutions Architect with AWS covering Strategic accounts. He thrives on resolving complex technical issues to overcome customer hurdles. Outside of work, he cherishes spending time with his family and fostering inquisitiveness in his children.

Secure access to a cross-account Amazon MSK cluster from Amazon MSK Connect using IAM authentication

Post Syndicated from Venkata Sai Mahesh Swargam original https://aws.amazon.com/blogs/big-data/secure-access-to-a-cross-account-amazon-msk-cluster-from-amazon-msk-connect-using-iam-authentication/

Amazon Managed Streaming for Apache Kafka (MSK) Connect is a fully managed, scalable, and highly available service that enables the streaming of data between Apache Kafka and other data systems. Amazon MSK Connect is built on top of Kafka Connect, an open-source framework that provides a standard way to connect Kafka with external data systems. Kafka Connect supports a variety of connectors, which are used to stream data in and out of Kafka. MSK Connect extends the capabilities of Kafka Connect by providing a managed service with added security features, straightforward configuration, and automatic scaling capabilities, enabling businesses to focus on their data streaming needs without the overhead of managing the underlying infrastructure.

In some use cases, you might need to use an MSK cluster in one AWS account, but MSK Connect is located in a separate account. In this post, we demonstrate how to create a connector to achieve this use case. At the time of writing, MSK Connect connectors can be created only for MSK clusters that have AWS Identity and Access Management (IAM) role-based authentication or no authentication. We demonstrate how to implement IAM authentication after establishing network connectivity. IAM provides enhanced security measures, making sure your systems are protected against unauthorized access.

Solution overview

The connector can be configured for a variety of purposes, such as sinking data to an Amazon Simple Storage Service (Amazon S3) bucket, tracking the source database changes, or serving as a migration tool such as MirrorMaker2 on MSK Connect to transfer data from a source cluster to a target cluster this is located in a different account.

The following diagram illustrates a use case using Debezium and Amazon S3 source connectors.

The following diagram illustrates using S3 Sink and migration to a cross-account failover cluster using a MirrorMaker connector deployed on MSK Connect.

Currently MSK Connect connectors can be created only for MSK clusters which have IAM role-based authentication or no authentication. In this blog, I’ll guide you through the essential steps for implementing the industry-recommended IAM (Identity and Access Management) authentication after establishing network connectivity. IAM provides enhanced security measures, ensuring your systems are protected against unauthorized access.

The launch of multi-VPC private connectivity (powered by AWS PrivateLink) and cluster policy support for MSK clusters simplifies the connectivity of Kafka clients to brokers. By enabling this feature on the MSK cluster, you can use the cluster-based policy to manage all access control centrally in one place. In this post, we cover the process of enabling this feature on the source MSK cluster.

We don’t fully utilize the multi-VPC connectivity provided by this new feature because that requires you to use different bootstrap URLs with port numbers (14001:3) that are not supported by MSK Connect as of writing of this post. We explore a secure network connectivity solution that uses private connectivity patterns, as detailed in How Goldman Sachs builds cross-account connectivity to their Amazon MSK clusters with AWS PrivateLink.

Connecting to a cross-account MSK cluster from MSK Connect involves the following steps.

Steps to configure the MSK cluster in Account A:

  1. Enable the multi-VPC private connectivity(Private Link) feature for IAM authentication scheme that is enabled for your MSK cluster.
  2. Configure the cluster policy to allow a cross-account connector.
  3. Implement one of the preceding network connectivity patterns according to your use case to establish the connectivity with the Account B VPC and make network changes accordingly.

Steps to configure the MSK connector in Account B:

  1. Create an MSK connector in private subnets using the AWS Command Line Interface (AWS CLI).
  2. Verify the network connectivity from Account A and make network changes accordingly.
  3. Check the destination service to verify the incoming data.

Prerequisites

To follow along with this post, you should have an MSK cluster in one AWS account and MSK Connect in a separate account.

Set up the MSK cluster setup in Account A:

In this post, we only show the important steps that are required to enable the multi-VPC feature on an MSK cluster:

  1. Create a provisioned MSK cluster in Account A’s VPC with the following considerations, which are required for the multi-VPC feature:
    • Cluster version must be 2.7.1 or higher.
    • Instance type must be m5.large or higher.
    • Authentication should be IAM (you must not enable unauthenticated access for this cluster).
  2. After you create the cluster, go to the Networking settings section of your cluster and choose Edit. Then choose Turn on multi-VPC connectivity.

  1. Select IAM role-based authentication and choose Turn on selection.

It might take around 30 minutes to enable. This step is required to enable the cluster policy feature that allows the cross-account connector to access the MSK cluster.

  1. After it has been enabled, scroll down to Security settings and choose Edit cluster policy.
  2. Define your cluster policy and choose Save changes.

  1. The new cluster policy allows for defining a Basic or Advanced cluster policy. With the Basic option, it only allows CreateVPCConnection, GetBootstrapBrokers, DescribeCluster, and DescribeClusterV2 actions that are required for creating the cross-VPC connectivity to your cluster. However, we have to use Advanced to allow more actions that are required by the MSK Connector. The policy should be as follows:
    {
    
        "Version": "2012-10-17",
        "Statement": [{
            "Effect": "Allow",
            "Principal": {
                "AWS": "Connector-AccountId"
            },
            "Action": [
                "kafka:CreateVpcConnection",
                "kafka:GetBootstrapBrokers",
                "kafka:DescribeCluster",
                "kafka:DescribeClusterV2",
                "kafka-cluster:Connect",
                "kafka-cluster:DescribeCluster",
                "kafka-cluster:ReadData",
                "kafka-cluster:DescribeTopic",
                "kafka-cluster:WriteData",
                "kafka-cluster:CreateTopic",
                "kafka-cluster:AlterGroup",
                "kafka-cluster:DescribeGroup"
            ],
    "Resource": [
                    "arn:aws:kafka:<region>:<Cluster-AccountId>:cluster/<cluster-name>/<uuid>",
                    "arn:aws:kafka:<region>:<Cluster-AccountId>:topic/<cluster-name>/<uuid>/<specific-topic-name>",
                    "arn:aws:kafka:<region>:<Cluster-AccountId>:group/<cluster-name>/<uuid>/<specific-group-name>"
                ]
        }]
    }

You might need to modify the preceding permissions to limit access to your resources (topics, groups). Also, you can restrict access to a specific connector by giving the connector IAM role, or you can mention the account number to allow the connectors in that account.

Now the cluster is ready. However, you need to make sure of the network connectivity between the cross-account connector VPC and the MSK cluster VPC.

If you’re using VPC peering or Transit Gateway while connecting to MSK Connect either from cross-account or the same account, do not configure your connector to reach the peered VPC resources with IPs in the following CIDR ranges (for more details, see Connecting from connectors):

  • 10.99.0.0/16
  • 192.168.0.0/16
  • 172.21.0.0/16

In the MSK cluster security group, make sure you allowed port 9098 from Account B network resources and make changes in the subnets according to your network connectivity pattern.

Set up the MSK connector in Account B:

In this section, we demonstrate how to use the S3 Sink connector. However, you can use a different connector according to your use case and make the changes accordingly.

  1. Create an S3 bucket (or use an existing bucket).
  2. Make sure that the VPC that you’re using in this account has a security group and private subnets. If your connector for MSK Connect needs access to the internet, refer to Enable internet access for Amazon MSK Connect.
  3. Verify the network connectivity between Account A and Account B by using the telnet command to the broker endpoints with port 9098.
  4. Create an S3 VPC endpoint.
  5. Create a connector plugin according to your connector plugin provider (confluent or lenses). Make a note of the custom plugin Amazon Resource Name (ARN) to use in a later step.
  6. Create an IAM role for your connector to allow access to your S3 bucket and the MSK cluster.
    • The IAM role’s trust relationship should be as follows:
      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Principal": {
                      "Service": "kafkaconnect.amazonaws.com"
                  },
                  "Action": "sts:AssumeRole"
              }
          ]
      }

    • Add the following S3 access policy to your IAM role:
      {
          "Version": "2012-10-17",
          "Statement": [{
              "Effect": "Allow",
              "Action": [
                  "s3:ListAllMyBuckets",
                  "s3:ListBucket",
                  "s3:GetBucketLocation",
                  "s3:DeleteObject",
                  "s3:PutObject",
                  "s3:GetObject",
                  "s3:AbortMultipartUpload",
                  "s3:ListMultipartUploadParts",
                  "s3:ListBucketMultipartUploads"
              ],
              "Resource": [
                              "arn:aws:s3:::<destination-bucket>",
                           "arn:aws:s3:::<destination-bucket>/*"
              ],
                 "Condition": {
              "StringEquals": {
                     "aws:SourceVpc": "vpc-xxxx"
                     }
                     }
          }]
      }

    • The following policy contains the required actions by the connector:
      {
      "Version": "2012-10-17",
      "Statement": [
         {
              "Effect": "Allow",
              "Action": [
                  "kafka-cluster:Connect",
                  "kafka-cluster:DescribeCluster",
                  "kafka-cluster:ReadData",
                  "kafka-cluster:DescribeTopic",
                  "kafka-cluster:WriteData",
                  "kafka-cluster:CreateTopic",
                  "kafka-cluster:AlterGroup",
                  "kafka-cluster:DescribeGroup"
              ],
              "Resource": [
                  "arn:aws:kafka:<region>:<Cluster-AccountId>:cluster/<cluster-name>/<uuid>",
                  "arn:aws:kafka:<region>:<Cluster-AccountId>:topic/<cluster-name>/<uuid>/<specific-topic-name>",
                  "arn:aws:kafka:<region>:<Cluster-AccountId>:group/<cluster-name>/<uuid>/<specific-group-name>"
              ]
          }
      ]
      }

You might need to modify the preceding permissions to limit access to your resources (topics, groups)

Finally, it’s time to create the MSK connector. Because the Amazon MSK console doesn’t allow viewing MSK clusters in other accounts, we show you how to use the AWS CLI instead. We also use basic Amazon S3 configuration for testing purposes. You might need to modify the configuration according to your connector’s use case.

  1. Create a connector using the AWS CLI with the following command with the required parameters of the connector, along with Account A’s MSK cluster broker endpoints:
    aws kafkaconnect create-connector \
    --capacity "autoScaling={maxWorkerCount=2,mcuCount=1,minWorkerCount=1,scaleInPolicy={cpuUtilizationPercentage=10},scaleOutPolicy={cpuUtilizationPercentage=80}}" \
    --connector-configuration \
    "connector.class=io.confluent.connect.s3.S3SinkConnector, \
    s3.region=<region>, \
    schema.compatibility=NONE, \
    flush.size=2, \
    tasks.max=1, \
    topics=<MSK-Cluster-topic>, \
    security.protocol=SASL_SSL, \
    s3.compression.type=gzip, \
    format.class=io.confluent.connect.s3.format.json.JsonFormat, \
    sasl.mechanism=AWS_MSK_IAM, \
    sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required, \
    sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler, \
    value.converter=org.apache.kafka.connect.storage.StringConverter, \
    storage.class=io.confluent.connect.s3.storage.S3Storage, \
    s3.bucket.name=<s3-bucket-name>, \
    timestamp.extractor=Record, \
    key.converter=org.apache.kafka.connect.storage.StringConverter" \
    --connector-name "Connector-name" \
    --kafka-cluster '{"apacheKafkaCluster": {"bootstrapServers": "<broker-strings>:9098","vpc": {"securityGroups": ["sg-0b36a015789f859a3"],"subnets": ["subnet-07950da1ebb8be6d8","subnet-026a729668f3f9728"]}}}' \
    --kafka-cluster-client-authentication "authenticationType=IAM" \
    --kafka-cluster-encryption-in-transit "encryptionType=TLS" \
    --kafka-connect-version "2.7.1" \
    --log-delivery workerLogDelivery='{cloudWatchLogs={enabled=true,logGroup="<MSKConnect-log-group-name>"}}' \
    --plugins "customPlugin={customPluginArn=<Custom-Plugin-ARN>,revision=1}" \
    --service-execution-role-arn "<IAM-role-ARN>"

  2. After you create the connector, connect the producer to your topic and insert data into it. In the following code, we use a Kafka client to insert data for testing purposes:
    bin/kafka-console-producer.sh --broker-list <broker-string> --producer.config client.properties --topic <topic-name>

If everything is set up correctly, you should see the data in your destination S3 bucket. If not, check the troubleshooting tips in the following section.

Troubleshooting tips

After deploying the connector, if it’s in the CREATING state on the connector details page, access the Amazon CloudWatch log group specified in your connector creation request. Review the logs for any errors. If no errors are found, wait for the connector to complete its creation process.

Additionally, make sure the IAM roles have their required permissions, and check the security groups and NACLs for proper connectivity between VPCs.

Clean up

When you’re done testing this solution, clean up any unwanted resources to avoid ongoing charges

Conclusion

In this post, we demonstrated how to create an MSK connector when you need to use an MSK cluster in one AWS account, but MSK Connect is located in a separate account. This architecture includes an S3 Sink connector for demonstration purposes, but it can accommodate other types of sink and source connectors. Additionally, this architecture focuses solely on IAM authenticated connectors. If an unauthenticated connector is desired, the multi-VPC connectivity (PrivateLink) and cluster policy components can be ignored. The remaining process, which involves creating a network connection between the account VPCs, remains the same.

Try out the solution for yourself, and let us know your questions and feedback in the comments section.

Check out more AWS Partners or contact an AWS Representative to learn how we can help accelerate your business.


About the Author

Venkata Sai Mahesh Swargam is a Cloud Engineer at AWS in Hyderabad. He specializes in Amazon MSK and Amazon Kinesis services. Mahesh is dedicated to helping customers by providing technical guidance and solving issues related to their Amazon MSK architectures. In his free time, he enjoys being with family and traveling around the world.

Build a multi-Region analytics solution with Amazon Redshift, Amazon S3, and Amazon QuickSight

Post Syndicated from Donatas Kuchalskis original https://aws.amazon.com/blogs/big-data/build-a-multi-region-analytics-solution-with-amazon-redshift-amazon-s3-and-amazon-quicksight/

Organizations increasingly face complex requirements balancing regional data sovereignty with global analytics needs. Regulatory frameworks like GDPR, HIPAA, and local data protection laws often mandate storing data in specific geographic regions, and business operations require global teams to access and analyze this data efficiently.

This post explores how to effectively architect a solution that addresses this specific challenge: enabling comprehensive analytics capabilities for global teams while making sure that your data remains in the AWS Regions required by your compliance framework. We use a variety of AWS services, including Amazon Redshift, Amazon Simple Storage Service (Amazon S3), and Amazon QuickSight.

It’s important to note that this solution focuses primarily on data residency (where data is stored) and not on preventing data from being in transit between Regions. Organizations with strict data transit restrictions might need additional controls beyond what’s covered here. We show how you can configure AWS across Regions to help meet business needs and regulatory requirements simultaneously.

Cross-Region architecture requirements

Before implementing a cross-Region solution, it’s important to understand when this approach is actually necessary. Although single-Region deployments offer simplicity and cost advantages, several specific business and regulatory scenarios warrant a cross-Region approach:

  • Data sovereignty and residency requirements – When regulations like GDPR, HIPAA, or local data sovereignty laws require data to remain in specific geographic boundaries while still enabling global analytics capabilities
  • Global operations with local compliance – When your organization operates globally, but needs to adhere to regional compliance frameworks while maintaining unified analytics
  • Performance optimization for global users – When your organization needs to optimize analytics performance for users in different geographic areas while centralizing data governance
  • Enhanced business continuity – When your analytics capabilities need higher availability and Regional redundancy to support mission-critical business processes

Use case: Financial services analytics with Regional data residency

Consider a financial services company with the following business and regulatory requirements:

  • Data residency requirement – All customer financial data must remain in the Bahrain Region (me-south-1) to comply with local financial regulations.
  • Global analytics capability – The organization’s data science team operates from European offices and needs to access and analyze the financial data without moving it out of its mandated storage Region.
  • Advanced analytics requirements – Business leaders need interactive data exploration and natural language query capabilities to derive insights from financial data.
  • Performance requirement – Specific dashboard queries require subsecond response times for both local executives and the global management team.

This specific combination of requirements can’t be met with a single-Region deployment. Let’s explore how to architect a solution.

Solution overview

The following architecture is designed to address the specific challenge of using QuickSight in one Region while maintaining data in another Region.

As shown in the architecture diagram, data engineers based in Bahrain (me-south-1) work with local data, whereas data engineers in Stockholm (eu-north-1) and analysts in Ireland (eu-west-1) can securely access the same data through Redshift datashares and virtual private cloud (VPC) peering connections. This approach maintains data residency in me-south-1 while enabling global access.

The solution consists of the following key components:

  • Primary data Region (me-south-1):
    • Redshift cluster (primary data repository)
    • S3 buckets for data lake storage
    • Private and public subnets with appropriate security controls
    • Data must remain in this Region for compliance reasons
  • Analytics services Region (eu-west-1):
    • QuickSight deployment
    • Cross-Region VPC peering connection to the primary Region
    • Data access using Redshift datashares (no data replication)
  • Data engineering Region (eu-north-1):
    • Redshift consumer cluster for data engineering workloads
    • Data access using Redshift datashares from me-south-1
    • Makes it possible for data engineering teams in eu-north-1 to access and work with data while maintaining compliance

Before implementing this architecture, evaluate whether:

  • Your requirements actually necessitate a cross-Region approach
  • The performance impact is acceptable for your use case
  • The additional cost is justified by your business requirements

For most analytics workloads, a single-Region architecture remains the recommended approach for simplicity, performance, and cost-effectiveness. Consider cross-Region architectures only when specific business and compliance requirements make them necessary.

Establish cross-Region network connectivity: Amazon Redshift to QuickSight

The foundation of a cross-Region solution is secure, reliable network connectivity. VPC peering provides a straightforward approach for connecting VPCs across Regions. To implement VPC peering in Amazon Virtual Private Cloud (Amazon VPC), complete the following steps:

  1. Create a new VPC in the secondary Region (eu-west-1):
    1. Open the Amazon VPC console in the eu-west-1 Region.
    2. Choose Create VPC.
    3. Set IPv4 CIDR block to 172.32.0.0/16 (verify there is no overlap with the primary Region VPC).
    4. Select Auto-generate to create subnets automatically within this new VPC.
    5. Leave other settings as default and choose Create VPC.

  1. Set up VPC peering:
    1. On the Amazon VPC console, choose Peering connections in the navigation pane and choose Create peering connection.
    2. Select the new eu-west-1 VPC as the requester.
    3. For Select another VPC to peer with, select My account and Another Region.
    4. Choose the primary Region (me-south-1) and enter the VPC ID.
    5. Choose Create peering connection.

  1. Accept the VPC peering connection:
    1. Switch to the primary Region on the Amazon VPC console.
    2. Choose Peering connections in the navigation pane and select the pending connection.
    3. On the Actions dropdown menu, choose Accept request.

  1. Update the route tables:
    1. On the  secondary Region Amazon VPC console, choose Route tables in the navigation pane.
    2. Choose the route table for the new VPC.
    3. Choose Edit routes and add a new route:
      • Destination: Primary Region VPC CIDR (e.g., 172.31.0.0/16).
      • Target: Choose the peering connection.
    4. On the primary Region Amazon VPC console, repeat the process, adding a route to the secondary Region VPC CIDR (172.32.0.0/16) using the peering connection.

  1. Configure security groups:
    1. On the secondary Region Amazon VPC console, choose Security groups in the navigation pane and create a new security group.
    2. Add an outbound rule:
      • Type: Custom TCP
      • Port range: 5439
      • Destination: Primary Region VPC CIDR

    3. On the primary Region Amazon VPC console, locate the Redshift cluster’s security group.
    4. Add an inbound rule:
      • Type: Custom TCP
      • Port range: 5439
      • Source: Secondary Region VPC CIDR

  1. Configure DNS settings:
    1. On the Amazon VPC console for both Regions, choose Your VPCs in the navigation pane.
    2. Select each VPC, and on the Actions dropdown menu, choose Edit DNS hostnames.
    3. Select Enable DNS resolution and Enable DNS hostnames.

Implement cross-Region data sharing

Rather than replicating data, which could create compliance issues, you can use Redshift datashares to provide secure, read-only access to data across Regions. Complete the following steps to set up your datashares:

  1. Create producer datashares in the primary Region:
    1. On the Amazon Redshift console, choose Query editor v2 in the navigation pane to connect to your primary Region Redshift cluster (me-south-1).
    2. Run the following commands:
      -- In Primary Region Redshift
      
      CREATE DATASHARE datashare_1;
      ALTER DATASHARE datashare_1 ADD SCHEMA analytics;
      ALTER DATASHARE datashare_1 ADD TABLE analytics.customers;
      ALTER DATASHARE datashare_1 ADD TABLE analytics.transactions;
      
      -- Grant usage permissions
      	
      GRANT USAGE ON DATASHARE datashare_1 TO ACCOUNT '123456789012';

  1. Create a consumer database in the secondary Region:
  2. Connect to your secondary Region Redshift cluster (eu-west-1) using the query editor and run the following commands:
    -- In Secondary Region Redshift
    
    CREATE DATABASE consumer_db FROM DATASHARE datashare_1 OF ACCOUNT '123456789012'REGION 'me-south-1';
  3. Verify the datashare configuration with the following code:
    -- In Secondary Region Redshift
    
    SELECT * FROM SVV_DATASHARE_CONSUMERS;
    SELECT * FROM SVV_DATASHARE_OBJECTS; 

This approach maintains data residency in the primary Region while enabling analytics access from another Region, addressing the core challenge of Regional service limitations. For our financial services company example, this makes sure that customer financial data remains in Bahrain (me-south-1) while making it securely accessible to the data science team in Europe (eu-west-1).

Configure QuickSight in the analytics Region

With network connectivity and data sharing established, complete the following steps to configure QuickSight to securely access the Redshift data:

  1. Set up a QuickSight VPC connection:
    1. Open the QuickSight console in the secondary Region.
    2. Choose Manage QuickSight, VPC connections, and Add VPC connection.
    3. Configure the connection:
      • Name: Enter a name (for example, Cross-Region-Connection).
      • VPC: Choose the secondary Region VPC.
      • Subnet: Choose the automatically created subnets.
      • Security group: Choose the security group created for cross-Region access.

  1. Add a QuickSight IP range to the data source security group:
    1. Open the Amazon Elastic Compute Cloud (Amazon EC2) console in the primary Region.
    2. Choose Security groups in the navigation pane and find the security group for your data source.
    3. Edit the inbound rules.
    4. Add a new rule:
      • Type: HTTPS (443)
      • Protocol: TCP
      • Port range: 443
      • Source: QuickSight IP range for the secondary Region (for example, 52.210.255.224/27 for eu-west-1).

QuickSight IP ranges can change over time. Refer to AWS Regions, websites, IP address ranges, and endpoints for current IP ranges.

  1. Create a QuickSight data source:
    1. On the QuickSight console, choose Datasets in the navigation pane.
    2. Choose New dataset, then choose Redshift.
    3. Configure the connection:
      • Data source name: Enter a descriptive name.
      • Connection type: Choose the VPC connection.
      • Database server: Enter the Redshift cluster endpoint from the primary Region.
      • Port: 5439
      • Database name: Enter the consumer database name.
      • Username and Password: Enter credentials (consider using AWS Secrets Manager).
    4. Choose Validate connection to test.
    5. Choose Create data source.

  1. Verify the connection and create datasets:
    1. Choose the schema and tables from the consumer database.
    2. Configure appropriate refresh schedules.
    3. Create calculations and visualizations as needed.

Performance considerations for cross-Region analytics

When implementing a cross-Region analytics architecture, be aware of the following performance implications:

  • Query performance impact – Cross-Region queries can experience higher latency than single-Region queries. To mitigate this, consider the following:
    • Use SPICE for QuickSight – Import frequently-used datasets into SPICE (Super-fast, Parallel, In-memory Calculation Engine) to help avoid repeated cross-Region queries. SPICE is the QuickSight in-memory engine that enables fast, interactive visualizations by precomputing and storing datasets locally in the QuickSight Region.
    • Implement efficient query patterns – Minimize the amount of data transferred between Regions.
    • Use appropriate caching – Enable result caching where possible.
    • Monitoring cross-Region performance – Implement monitoring to identify and address performance issues:
      • Set up Amazon CloudWatch metrics to track cross-Region query performance
      • Create dashboards to visualize latency trends
      • Establish performance baselines and alerts for degradation

Security considerations

Maintaining security in a cross-Region architecture requires additional attention:

  • Network security:
    • Limit VPC peering connections to only necessary VPCs
    • Implement restrictive security groups that allow only required traffic
    • Consider using VPC endpoints for service access when possible
  • Data access controls:
    • Use AWS Identity and Access Management (IAM) policies consistently across Regions
    • Implement fine-grained access controls in Redshift datashares
    • Enable audit logging in relevant Regions
  • Compliance monitoring:
    • Implement AWS CloudTrail in all Regions
    • Create centralized logging for cross-Region activities
    • Regularly review cross-Region access patterns

Cost implications

Before implementing a cross-Region architecture, consider these cost factors:

  • Data transfer costs – Data transfer between Regions incurs charges
  • Additional infrastructure – You might need Redshift clusters in multiple Regions
  • VPC peering costs – Data transfer costs are associated with VPC peering
  • Operational overhead – Managing multi-Region deployments requires additional resources
  • Workload-based sizing – You should size each Regional Redshift cluster according to the specific workloads it will handle

Conclusion

The cross-Region architecture described in this post addresses specific challenges related to Regional compliance requirements and global analytics needs, particularly in the following scenarios:

  • Your data must remain in a specific Region for compliance reasons
  • You have teams in different Regions who need to access and analyze this data
  • Different user groups have distinct workload requirements

The datasharing capabilities of Amazon Redshift and Regional storage options in Amazon S3 are key enablers of this solution, allowing data to remain in the required Region while still being accessible for analytics across Regions. However, it’s worth emphasizing that this architecture supports data storage in specific Regions but doesn’t prevent data from traveling between Regions during processing. Organizations concerned about data transit restrictions should evaluate additional controls to address those specific requirements. Combined with secure VPC peering connections and QuickSight visualizations, this architecture creates a complete solution that satisfies both compliance requirements and business needs.

For our financial services example, this architecture successfully enables the company to keep its customer financial data in Bahrain while providing seamless analytics capabilities to the European data science team and delivering interactive dashboards to global business leaders.

For more information, refer to Building a Cloud Security Posture Dashboard with Amazon QuickSight. For hands-on experience, explore the Amazon QuickSight Workshops. Visit the Amazon Redshift console or Amazon QuickSight console to start building your first dashboard, and explore our AWS Big Data Blog for more customer success stories and implementation patterns

Try out this solution for your own use case, and share your thoughts in the comments.


About the Authors

Donatas Kuchalskis is a Cloud Operations Architect at AWS, based in London, focusing on Financial Services customers in the UK. He helps customers optimize their AWS environments for cost, security, and resiliency while providing strategic cloud guidance. Prior to this role, he served as a Prototyping Architect specializing in Big Data and as a Specialist Solutions Architect for Retail. Before joining AWS, Donatas spent 6 years as a technical consultant in the retail sector.

Jumana Nagaria is a Prototyping Architect at AWS. She builds innovative prototypes with customers to solve their business challenges. She is passionate about cloud computing and data analytics. Outside of work, Jumana enjoys travelling, reading, painting, and spending quality time with friends and family.

How to prioritize security risks using AWS Security Hub exposure findings

Post Syndicated from Shahna Campbell original https://aws.amazon.com/blogs/security/how-to-prioritize-security-risks-using-aws-security-hub-exposure-findings/

At re:Inforce 2025, AWS unveiled an enhanced AWS Security Hub that transforms how organizations prioritize their most critical security issues and respond at scale to protect their cloud environments. In this blog post, we discuss how you can use Security Hub to prioritize these issues with exposure findings. The enhanced Security Hub now uses advanced analytics to automatically correlate, enrich, and prioritize security signals across your cloud environment. Security Hub seamlessly integrates with Amazon GuardDuty, Amazon Inspector, Amazon Macie, and AWS Security Hub Cloud Security Posture Management (CSPM), formerly known as AWS Security Hub. Through these integrations, it provides comprehensive threat detection and vulnerability assessment. This intelligent integration helps organizations quickly identify critical security issues, from potential credential compromises to unintended resource exposures, enabling security teams to focus on what matters most.

What is Security Hub?

Security Hub delivers three key security capabilities to help you strengthen your cloud security posture through a unified cloud security solution:

  • Provides visibility across your organization through centralized management and continuous monitoring.
  • Enriches security signals from services such as Amazon Inspector and AWS Security Hub CSPM to surface active risks specific to your environment, so you can prioritize with confidence and streamline response.
  • Delivers integrated risk analysis by correlating findings from Amazon Inspector, AWS Security Hub CSPM, Amazon Macie, and other AWS services to help identify potential attack paths, surface exploitable vulnerabilities and misconfigurations, and provide actionable remediation guidance.

A top concern for customers is: How do I know where to prioritize response first? Managing large volumes of findings across multiple accounts and regions becomes more challenging when security findings are viewed in isolation, making it difficult to determine true priority and impact. Security Hub solves this by providing context-driven analysis. It surfaces the most critical risks by correlating related vulnerabilities, threats, and misconfigurations to reveal exploitable paths. This can help you make informed decisions about which issues to address first.

With exposure findings, you can prioritize critical security issues and respond at scale. Exposures are based on an analysis of findings and traits from Security Hub CSPM, Amazon Inspector (which scans for vulnerabilities), and Amazon Macie (which discovers and protects sensitive data). They are defined as potential security issues, and they are generated by different exposure traits.

Without automated correlation and enriched signals, security teams can struggle to effectively prioritize issues. For example, a vulnerability that Amazon Inspector detects might become critically important when combined with misconfigurations that Security Hub CSPM identifies. However, manually analyzing relationships across thousands of signals is time-consuming and prone to missing critical security context. Teams often build custom solutions to achieve this, but this approach requires significant analyst time and maintenance, which can cause critical security relationships to be overlooked.

Security Hub reduces this complexity by providing native integration across these AWS services in a unified cloud security center, without the operational overhead of log collection and aggregation. For security teams, this means they can help identify and respond to their most critical exposures before the exposures can lead to business impact, rather than spending valuable time manually piecing together individual security signals. Automated correlation and enriched context can help you make faster, more informed decisions about where to focus your efforts. This ultimately helps protect your cloud environment more effectively.

Exposure findings identify security risks in your environment by providing a comprehensive view of your security posture. These findings enable you to understand and address potential risks. Through this consolidated approach, you can efficiently prioritize your remediation efforts by focusing on the most critical exposure findings first,.

Exposure findings are formatted in the Open Cybersecurity Schema Framework (OCSF) schema, an open-source standard that enables security tools to share data seamlessly. The adoption of OCSF by Security Hub has several advantages. As an open, standardized schema that is part of the Linux Foundation, OCSF enables interoperability across multiple security tools and services, both within and outside of the AWS environment. It provides enhanced data normalization with consistent field naming and categorization, making it more straightforward to integrate with third-party security tools.

Partners who already support or intend to support the OCSF schema to receive findings from Security Hub include companies such as Arctic Wolf; CrowdStrike; DataBee, a Comcast company; Datadog; DTEX Systems; Dynatrace; Fortinet; IBM; Netskope; Orca Security; Rapid7; Securonix; SentinelOne; Splunk, a Cisco Company; Sumo Logic; Tines; Trellix; and Wiz. Additionally, service partners such as Accenture, Caylent, Deloitte, IBM, and Optiv can help you adopt Security Hub and the OCSF schema.

Prioritizing security risks

When you navigate to Security Hub, you will see the summary dashboard, which includes an exposure summary widget, as shown in Figure 1. This widget shows your exposures by severity and frequency. Security Hub assigns each exposure finding a default severity of Critical, High, Medium, or Low. Exposure findings with a severity of Informational are not published.

Security Hub calculates exposure finding severity by analyzing and correlating multiple security traits across AWS services. Instead of evaluating these factors in isolation, Security Hub uses a contextual approach, assigning a severity rating based on how these factors are correlated. For example, a resource with an identified vulnerability might receive a higher severity rating if it’s exploitable from the internet or has access to sensitive data.

Security Hub uses several factors to determine the default severity of an exposure finding:

  • Ease of discovery – The availability of automated tools, such as port scans or internet searches to discover the resource at risk.
  • Ease of exploit – The ease with which a threat actor can exploit the risk. For example, if there are open network paths or misconfigured metadata, a threat actor can more quickly exploit the risk.
  • Likelihood of exploit Security Hub uses both external signals, such as the Exploit Prediction Scoring System (EPSS)—a data-driven scoring system that estimates the probability of a vulnerability being exploited—and internal threat intelligence to determine the probability that the risk is exploited. This comprehensive approach applies to exposure findings for Amazon Elastic Compute Cloud (EC2) instances and AWS Lambda functions.
  • Awareness – The extent to which the risk is not merely theoretical but has publicly available or automated exploits. This factor applies to exposure findings for EC2 instances and Lambda functions.
  • Impact – The potential harm if the exploit is carried out. For example, an exposure could lead to loss of confidentiality from data exposure, loss of integrity from data corruption, loss of availability, or loss of accountability.

The list of risks in this widget is limited to the eight highest risks with the greatest number of critical findings. If two or more risks have an equal number of critical findings, the list automatically groups those findings behind more recent critical findings.

Figure 1 : Exposure summary widget

Figure 1 : Exposure summary widget

From the widget, you can pivot to the exposure dashboard to see to a pre-filtered view of your exposures for continued analysis of potential security issues. You can filter by severity by selecting the number associated with each severity, view a specific exposure by selecting from the list, or select View all exposure findings to see a dashboard of new exposures that are currently open, as shown in Figure 2.

Figure 2: Exposure dashboard

Figure 2: Exposure dashboard

The exposure console shows findings by their title and ranked by decreasing severity. It’s organized by the filter criteria and grouped by finding title. On the left-hand side, Quick filters provide a fast way to filter through exposures based on severity, the top 10 attributes based on the most common values across your findings, top 10 accounts, and top 10 resource types, as shown in Figure 2. In addition to using filters, you can use the Group by dropdown to group exposure findings by a specific attribute, such as AWS account ID, resource type, or product name.

To review the exposure, expand the findings, as shown in Figure 3 for the correlation of resources, status, attributes, and traits such as software vulnerabilities, misconfigurations, and reachability. These are also referred to as trait types. For a particular exposure finding, a trait can be associated with one or more signals, and a signal can contain one or more indicators.

Figure 3: Exposure findings

Figure 3: Exposure findings

As shown in Figure 3, the Potential Credential Stealing: Internet reachable EC2 instance with administrative instance profile has network-exploitable software vulnerabilities with a high likelihood of exploitation finding indicates that there are misconfigurations, vulnerabilities, and reachability (indication of an open network path to a resource) associated with the instance. To find out more about the signal, select anywhere in the line associated with the risk, and you will see an overview panel, as shown in Figure 4.

Figure 4: Exposure finding overview

Figure 4: Exposure finding overview

This example highlights a critical-severity finding for an internet-reachable EC2 instance with software vulnerabilities in the us-east-1 Region. This visualization is powerful because the Potential attack path diagram helps you see what matters by mapping out how potential threat actors could exploit these vulnerabilities to access your resources. The finding also includes critical metadata such as the resource identifier, creation time, reachability, vulnerability, and misconfigurations.

Using the finding, you can quickly understand complex security relationships, assess risk context, and determine remediation priorities, so you can better protect your workloads in the cloud and make more informed security decisions. To prioritize your security response efforts, you can also set finding severity levels and update status, and export findings in OCSF format.

To understand why an exposure is present, you can select the Traits tab, as shown in Figure 5. This will list traits such as Misconfiguration or Vulnerability. If you select By signal, in the Traits tab, you have a full list of the signals associated with the exposure finding. These signals are the underlying findings that were created from different services, such as Security Hub CSPM and Amazon Inspector, that were correlated together to determine the risk associated with the exposure finding.

Figure 5: Exposure finding traits

Figure 5: Exposure finding traits

If you select the Resources tab, you will see the resources associated with the exposure finding, as shown in Figure 6.

Figure 6: Exposure finding resources

Figure 6: Exposure finding resources

For this example, we have an EC2 instance, but you might have a combination of resources such as an EC2 instance, Amazon Simple Storage Service (Amazon S3) bucket, and AWS Identity and Access Management (IAM) role. This list of resources will help you determine what needs to be remediated in your environment to mitigate the risk attributed to this finding.

Finally, with the Create ticket option, Security Hub helps streamline the incident management process through its native integrations with popular ticketing systems such as Jira and ServiceNow. This integration minimizes the need for manual ticket creation and reduces the time between finding and fixing security issues. Organizations can use a Security Hub Automation Rule to automatically create and track tickets for security findings directly from the Security Hub console, helping to make sure that no critical security exposure goes unaddressed. Integration with these widely-used ticketing systems helps maintain a consistent workflow, enables better tracking of remediation efforts, and improves collaboration between security and operations teams. This can help you make your security operations more efficient by providing a streamlined path from detection to resolution.

Conclusion

The enhanced exposure findings capabilities in Security Hub represent a significant advancement in how organizations can secure their cloud environments. By automatically correlating and analyzing security signals across multiple AWS services, Security Hub helps you prioritize your most critical security issues confidently and respond at scale. The intuitive visualization of potential attack paths, combined with intelligent severity rankings and comprehensive trait analysis, enables security teams to make data-driven decisions about risk prioritization.

Security Hub exposure findings help organizations move from reactive to proactive security postures by:

  • Automatically discovering and evaluating publicly accessible resources
  • Providing clear visibility into security capabilities and configurations
  • Correlating multiple security signals to identify critical risks
  • Delivering actionable remediation guidance
  • Offering intuitive filtering and grouping options for efficient analysis

As cloud environments continue to grow in complexity, exposure findings provide the automation, intelligence, and context needed to stay ahead of potential security issues. This enables security teams to focus their valuable time on addressing the most critical risks first, ultimately helping organizations maintain a stronger security posture across their cloud environment.

Whether you’re managing a small deployment or a large enterprise environment, exposure findings in Security Hub provide the insights needed to effectively protect your AWS workloads and maintain a robust security position in an ever-evolving landscape.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on AWS Security, Identity, and Compliance re:Post or contact AWS Support.

Shahna Campbell

Shahna Campbell

Shahna is a solutions architect at AWS, working within the specialist organization with a focus on security. Previously, Shahna worked within the healthcare field clinically and as an application specialist. Shahna is passionate about cybersecurity and analytics. In her free time, she enjoys hiking, traveling, and spending time with family.

Author

Marshall Jones

Marshall is a Worldwide Security Specialist Solutions Architect at AWS. His background is in AWS consulting and security architecture and focused on a variety of security domains including edge, threat detection, and compliance. Today, he’s focused on helping enterprise AWS customers adopt and operationalize AWS security services to increase security effectiveness and reduce risk.

Kimberly Dickson

Kimberly is a Security Specialist Solutions Architect at AWS based in Singapore. She is passionate about working with customers on technical security solutions that help them build confidence and operate securely in the cloud.

[$] Asterinas: a new Linux-compatible kernel project

Post Syndicated from corbet original https://lwn.net/Articles/1022920/

Born from research at the Southern University of Science and
Technology
(SUSTech) in Shenzen, China, Asterinas is a new
Linux-ABI-compatible kernel project written in Rust, based on what the
authors call a “framekernel architecture”. The project overlaps somewhat
with the goals of the Rust for Linux
project
, but approaches the problem space from a different direction by
trying to get the best from both monolithic and microkernel designs.

The collective thoughts of the interwebz