Post Syndicated from Cliff Robinson original https://www.servethehome.com/nvidia-mlperf-inference-v4-0-is-out-intel-qualcomm-nvidia-amd-google-tpu-cerebras/
MLPerf Inference v4.0 is out with Qualcomm still trying to get noticed, NVIDIA dominating, Intel submitting and others staying out
The post NVIDIA MLPerf Inference v4.0 is Out appeared first on ServeTheHome.
Post Syndicated from jzb original https://lwn.net/Articles/966835/
Security updates have been issued by Debian (composer and nodejs), Fedora (w3m), Mageia (tomcat), Oracle (expat, firefox, go-toolset:ol8, grafana, grafana-pcp, nodejs:18, and thunderbird), Red Hat (dnsmasq, expat, kernel, kernel-rt, libreoffice, and squid), and SUSE (firefox, krb5, libvirt, and shadow).
Post Syndicated from Home Assistant original https://www.youtube.com/watch?v=Q1yHfqsVidw
Post Syndicated from The History Guy: History Deserves to Be Remembered original https://www.youtube.com/watch?v=yGvXjCJ7Hwg
Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2024/03/security-vulnerability-in-safloks-rfid-based-keycard-locks.html
It’s pretty devastating:
Today, Ian Carroll, Lennert Wouters, and a team of other security researchers are revealing a hotel keycard hacking technique they call Unsaflok. The technique is a collection of security vulnerabilities that would allow a hacker to almost instantly open several models of Saflok-brand RFID-based keycard locks sold by the Swiss lock maker Dormakaba. The Saflok systems are installed on 3 million doors worldwide, inside 13,000 properties in 131 countries. By exploiting weaknesses in both Dormakaba’s encryption and the underlying RFID system Dormakaba uses, known as MIFARE Classic, Carroll and Wouters have demonstrated just how easily they can open a Saflok keycard lock. Their technique starts with obtaining any keycard from a target hotel—say, by booking a room there or grabbing a keycard out of a box of used ones—then reading a certain code from that card with a $300 RFID read-write device, and finally writing two keycards of their own. When they merely tap those two cards on a lock, the first rewrites a certain piece of the lock’s data, and the second opens it.
Dormakaba says that it’s been working since early last year to make hotels that use Saflok aware of their security flaws and to help them fix or replace the vulnerable locks. For many of the Saflok systems sold in the last eight years, there’s no hardware replacement necessary for each individual lock. Instead, hotels will only need to update or replace the front desk management system and have a technician carry out a relatively quick reprogramming of each lock, door by door. Wouters and Carroll say they were nonetheless told by Dormakaba that, as of this month, only 36 percent of installed Safloks have been updated. Given that the locks aren’t connected to the internet and some older locks will still need a hardware upgrade, they say the full fix will still likely take months longer to roll out, at the very least. Some older installations may take years.
If ever. My guess is that for many locks, this is a permanent vulnerability.
Post Syndicated from Rik Cross original https://www.raspberrypi.org/blog/supporting-computing-in-england-with-oak-national-academy/
We are pleased to announce that we are renewing our partnership with Oak National Academy in England to provide an updated high-quality Computing curriculum and lesson materials for Key Stages 1 to 4.
In 2021 we partnered with Oak National Academy to offer content for schools in England that supported young people to learn Computing at home while schools were closed as a result of the coronavirus pandemic.
In our renewed partnership, we will create new and updated materials for primary and secondary teachers to use in the classroom. These classroom units will be available for free on the Oak platform and will include everything a teacher needs to deliver engaging lessons, including slide decks, worksheets, quizzes, and accompanying videos for over 550 lessons. The units will cover both the general national Computing curriculum and the Computer Science GCSE, supporting teachers to provide a high-quality Computing offering to all students aged 5 to 16.
These new resources will update the very successful Computing Curriculum and will be rigorously tested by a Computing subject expert group.
“I am delighted that we are continuing our partnership with Oak National Academy to support all teachers in England with world-leading resources for teaching Computing and Computer Science. This means that all teachers in England will have access to free, rigorous and tested classroom resources that they can adapt to suit their context and students.” – Philip Colligan, CEO
All our materials on the Oak platform will be free and openly available, and can be accessed by educators worldwide.
As we did with The Computing Curriculum, we’ll design these teaching resources to model best practice, and they will be informed by leading research into pedagogy and computing education, as well as by user testing and feedback.
The materials will bring teachers the added benefit of saving valuable time, and schools can choose to adapt and use the resources in the way that works best for their students
We have already started work and will begin releasing units of lessons in autumn 2024. All units across Key Stages 1 to 4 will be available by autumn 2025.
We’re excited to continue our partnership with Oak National Academy to provide support to teachers and students in England.
And as always, our comprehensive classroom resources can be downloaded for free, by anyone in the world, from our website.
The post Supporting Computing in England through our renewed partnership with Oak National Academy appeared first on Raspberry Pi Foundation.
Post Syndicated from Patrik Uytterhoeven original https://blog.zabbix.com/securing-the-zabbix-frontend/27700/
The frontend is what we use to login into our system. The Zabbix frontend will connect to our Zabbix server and our database. But we also send information from our laptop to the frontend. It’s important that when we enter our credentials that we can do this in a safe way. So it makes sense to make use of certificates and one way to do this is by making use of self-signed certificates.
To give you a better understanding of why your browser will warn you when using self-signed certificates, we have to know that when we request an SSL certificate from an official Certificate Authority (CA) that you submit a Certificate Signing Request (CSR) to them. They in return provide you with a Signed SSL certificate. For this, they make use of their root certificate and private key.
Our browser comes with a copy of the root certificate (CA) from various authorities, or it can access it from the OS. This is why our self-signed certificates are not trusted by our browser – we don’t have any CA validation. Our only workaround is to create our own root certificate and private key.
Table of Contents
How to create an SSL certificate:
How SSL works – Client – Server flow:
NOTE: I have borrowed the designs from this video, which does a good job of explaining how SSL works.
In order to configure this, there are a few steps that we need to follow:
The first step is to make a folder named “SSL” so we can create our certificates and save them:
>- mkdir ~/ssl
>- cd ~/ssl
>- openssl ecparam -out myCA.key -name prime256v1 -genkey
Let’s explain all the options:
openssl req -x509 -new -nodes -key myCA.key -sha256 -days 1825 -out myCA.pema
Let’s explain all the options:
The information you enter is not so important, but it’s best to fill it in as comprehensively as possible. Just make sure you enter for CN your IP or DNS.
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:BE
State or Province Name (full name) []:vlaams-brabant
Locality Name (eg, city) [Default City]:leuven
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:192.168.0.134
Email Address []:
It’s probably good practice to use the dns name of your webiste in the name for the private key. As we use in this case an IP address rather than a dns, I will use the fictive dns zabbix.mycompany.internal.
openssl genrsa -out zabbix.mycompany.internal.key 2048
openssl req -new -key zabbix.mycompany.internal.key -out zabbix.mycompany.internal.csr
You will be asked the same set of questions as above. Once again, your answers hold minimal significance and in our case no one will inspect the certificate, so they matter even less.
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:BE
State or Province Name (full name) []:vlaams-brabant
Locality Name (eg, city) [Default City]:leuven
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:192.168.0.134
Email Address []:
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
# vi zabbix.mycompany.internal.ext
Add the following lines in your certificate extension file. Replace IP or DNS with your own values.
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = @alt_names
[alt_names]
IP.1 = 192.168.0.133
#DNS.1 = MYDNS (You can use DNS if you have a dns name if you use IP then use the above line)
openssl x509 -req -in zabbix.mycompany.internal.csr -CA myCA.pem -CAkey myCA.key \
-CAcreateserial -out zabbix.mycompany.internal.crt -days 825 -sha256 -extfile zabbix.mycompany.internal.ext
cp zabbix.mycompany.internal.crt /etc/pki/tls/certs/.
cp zabbix.mycompany.internal.key /etc/pki/tls/private/.
Import the CA in Linux (RHEL)
We need to update the CA certificates, so run the below command to update the CA certs.
cp myCA.pem /etc/pki/ca-trust/source/anchors/myCA.crt
update-ca-trust extract
Import the CA in OSX
Import the CA in Windows
Warning! You also need to import the myCA.crt file in your OS. We are not an official CA, so we have to import it in our OS and tell it to trust this Certificate. This action depends on the OS you use.
As you are using OpenSSL, you should also create a strong Diffie-Hellman group, which is used in negotiating Perfect Forward Secrecy with clients. You can do this by typing:
openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048
Add the following lines to your Nginx configuration, modifying the file paths as needed. Replace the existing lines with port 80 with this configuration. This will enable SSL and HTTP2.
# vi /etc/nginx/conf.d/zabbix.conf
server {
listen 443 http2 ssl;
listen [::]:443 http2 ssl;
server_name <ip qddress>;
ssl_certificate /etc/ssl/certs/zabbix.mycompany.internal.crt;
ssl_certificate_key /etc/pki/tls/private/zabbix.mycompany.internal.key;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
To redirect traffic from port 80 to 443 we can add the following lines above our https block:
server {
listen 80;
server_name _; #dns or ip is also possible
return 301 https://$host$request_uri;
}
Restart all services and allow https traffic
systemctl restart php-fpm.service
systemctl restart nginx
firewall-cmd --add-service=https --permanent
firewall-cmd —reload
When we go to our url http://<IP or DNS>/ we get redirected to our https:// page and when we check we can see that our site is secure:
You can check out this article in its original form (and keep an eye out for more of Patrik’s helpful tips) at https://trikke76.github.io/Zabbix-Book/security/securing-zabbix/.
The post Securing the Zabbix Frontend appeared first on Zabbix Blog.
Post Syndicated from Светла Енчева original https://www.toest.bg/vuoruzhena-bditelnost-po-nikoe-vreme/
На 6 март 2024 г. кабинетът „Денков“ подаде оставка, а на 25 март, след оттеглянето на ГЕРБ от преговорите с ПП–ДБ, номинираната за премиер Мария Габриел се отказа от кандидатурата си. Какво очаквате да върши един вътрешен министър в оставка в периода, когато още не е ясно ще има ли правителство и ако да, ще бъде ли той министър в него? Най-логично е просто да се грижи подопечното му министерство да осигурява реда. Вместо това министърът в оставка Калин Стоянов демонстрира свръхактивност, свързана със сериозен човешки (и потенциално – финансов) ресурс. И бутафорно присъствие.
Макар официално всички министри в кабинета на Николай Денков с изключение на Мария Габриел да бяха номинации на ПП–ДБ, с времето стана ясно, че това не е точно така. Един от министрите, за които се разбра, че са по-скоро предложения на ГЕРБ, беше именно Калин Стоянов. Кой всъщност стои зад него, излезе наяве след протеста против тогавашното ръководство на Българския футболен съюз на 16 ноември 2023 г. Този протест прерасна в масови безредици, а полицията приложи неоправдано насилие в опит да го овладее. Срещу протестиращите беше използвано водно оръдие. Бяха бити и невинни хора, които дори не са участвали в протеста, просто са минавали в района или са пиели бира в заведение наблизо.
Заради този случай на масово полицейско насилие, както и заради това, че МВР не е създало организация, която да предотврати ескалацията на напрежението, от ПП–ДБ поискаха оставката на Калин Стоянов. Премиерът Николай Денков дори заплаши, че ще си подаде оставката, ако министърът не осъди полицейското насилие.
Стоянов обаче отказа да сдаде поста с аргумента, че не иска да е „изкупителна жертва на дългогодишните проблеми в МВР“. Тогава от ПП–ДБ обявиха Стоянов за министър на ГЕРБ и ДПС, а Мария Габриел го защити, като заяви, че той „отстоява принципите за нулева толерантност към насилието“.
От този момент нататък от ГЕРБ ревностно бранят Калин Стоянов, макар да не признават, че той е бил тяхна номинация. В брифинга, на който Мария Габриел заяви, че се отказва от премиерския пост, тя изтъкна като един от аргументите за решението си, че от ПП–ДБ са искали смяната на вътрешния министър.
В този контекст, в междувремието от оставката на Денков до безславния край на мандата на Мария Габриел, Калин Стоянов на два пъти активизира силите на реда по „поводи“, които всъщност не бяха поводи.
Първият не-повод бяха четири инцидента между 6 и 8 март, които не бяха свързани с насилие от страна на чужденци към българи. Жители на родопското село Храбрино взеха чуждестранни студенти по медицина за бежанци и ги подложиха на граждански арест. Други чуждестранни студенти пък бяха нападнати от български тийнейджъри на 8 март насред столичния булевард „Витоша“. На същата улица през нощта на 6 срещу 7 март имаше бой между две групи младежи, едната от които съставена от млади мъже с арабски произход. Както стана ясно впоследствие, двете групи са се познавали и конфликтът вероятно е бил заради момиче. А кандидати за убежище от общежитието на Държавната агенция за бежанците в „Овча купел“ бяха заснети на 8 март вечерта как се карат и се гонят помежду си.
Тези дребни инциденти, в два от които насилниците са българи, но няма сериозно пострадал български гражданин, станаха повод за катализиране на поредната ксенофобска истерия и протести пред бежански центрове. Реакцията на вътрешния министър беше двойствена. От една страна, той подчерта, че няма увеличение на случаите на нарушение на реда с участието на чужденци, за разлика от закононарушенията, извършени от непълнолетни. От друга страна, Калин Стоянов използва случая да въведе засилени мерки за сигурност… заради чужденците, които определи като „хора с различен произход“:
„Засилваме полицейското присъствие в централната градска част и в райони, където има събиране на хора с различен произход, както и около центровете за настаняване на Държавната агенция за бежанците. В централната част на столицата екипите ще са съвместни от СДВР и на общината, а край центровете за бежанци ще патрулират полицейски служители.“
Стоянов също така повтори заявката си, дадена на 8 март, да инициира законодателна промяна, в резултат на която да се назначат още 1260 гранични полицаи. Ако това стане, ще бъдат отделени немалко средства за МВР, което е известно с раздутия си щат. Предложението за откриване на работни места за такъв брой полицейски служители по границите впрочем не е оригинално негово – преди година същото число споменава тогавашният служебен вътрешен министър Иван Демерджиев.
Вторият не-повод беше атентатът в Москва от вечерта на 22 март, при който бяха убити близо 140 души, отишли на рок концерт, а повече от 180 – ранени. Отговорност за него пое фракцията на „Ислямска държава“ – „Хорасан“. Русия обаче обвинява Украйна, която категорично отрича връзка с терористичния акт.
Засега няма публично известни данни България да е в опасност след този атентат. На 23 март обаче МВР излезе с прессъобщение, в което се казва, че министър Калин Стоянов е разпоредил „засилени мерки за обезпечаване на сигурността и спокойствието на гражданите на територията на страната. Полицейски патрули и жандармеристи, оборудвани с дългоцевно оръжие, ще патрулират в градските зони и ще съблюдават за реда и за гарантиране на безопасността“.
Не се споменава поводът за изкарването по улиците на сили на МВР, въоръжени с пушки. Вместо това просто се съобщава, че то е „с оглед предотвратяване на нарушения на обществения ред и повишаване на бдителността“.
След двете решения на Калин Стоянов за увеличаване на полицейското присъствие могат да се видят служители на реда най-вече в района около бул. „Витоша“ и НДК, край молове, както и около бежански центрове. Жандармеристи видимо се шляят из центъра на София, някои от тях си пушат, но пък са с тежки униформи и с големи, страшни автомати.
Като се изключи тази показност, на много места в страната и дори в столицата полицията на практика отсъства там, където е нужна. Има столични квартали, в които хората може с години да не видят полицай, като изключим някой и друг катаджия, скрит в храстите. Но пък служители на реда традиционно се навъртат по паркове и градинки и обискират хора, които им приличат на такива, които биха пушили трева. Понякога стоят на изходите на метростанциите и спират за легитимация минувачи, чиято външност им изглежда съмнителна – например мъже с дълги коси и бради или с вид на чужденци.
Присъствието на патрулиращи полицейски служители е необходимо – както в една европейска столица, каквато е София, така и в по-малките населени места. Както за предотвратяване на престъпления, така и в помощ на гражданите – например да бъде упътен някой, да се реагира бързо и адекватно, ако друг се почувства зле или е в безпомощно състояние, и т.н. За всекидневната дейност на един патрулиращ полицай пушка не само не е нужна, а и би могла да затрудни навременната му реакция.
На 13 ноември 2015 г. в Париж е извършена серия от терористични атаки, в резултат на които са убити близо 130 души. Като реакция тогавашният президент Франсоа Оланд обявява извънредно положение. Временно са затворени границите и е обявен вечерен час, а по улиците излизат въоръжени военни. Скоро границите се отварят, а вечерният час се отменя, но военните по улиците остават.
Присъствието на военни с автомати обаче не предотвратява атентата в Ница на следващата година, навръх 14 юли – националния празник на Франция. Тогава камион се врязва в множество, събрало се на крайбрежната алея да наблюдава фойерверките, и над 80 души са убити.
Когато настоящият президент Еманюел Макрон встъпи в длъжност, той остави патрулиращите въоръжени военни. И до днес те може да се видят най-вече в групи по четирима край гари, молове и другаде, където има масово струпване на хора.
Въпреки продължаващото на практика 9 години извънредно положение Франция продължава да е в постоянен риск от терористични атаки. Един от последните случаи беше през есента на 2023 г. в училище, в което бивш ученик от чеченски произход уби учител и двама ученици.
След всеки подобен случай Франция повишава предупреждението за опасност и мерките за сигурност. Повиши ги и след атентата в Москва от 22 март.
Нека впрочем видим и самата Русия. Атентатът в Москва беше допуснат, макар в страната вече почти всеки да е под подозрение, че е „екстремист“, и човек не може да протестира дори с бял лист на улицата. В Русия, където арестуват момичета заради това, че са били с обеци в цветовете на дъгата, терористи с големи пушки и запалителни вещества безпрепятствено могат да влязат в концертна зала и да извършат масово убийство.
Съвсем различен е подходът на Германия. В края на 2016 г. в Берлин камион се вряза в коледен базар. Дванайсет души бяха убити. В страната обаче не беше въведено извънредно положение, а канцлерката Ангела Меркел заяви, че не желае хората в Германия да живеят „парализирани от страха от злото“, а иска да бъдат „свободни заедно и отворени към света“.
Животът в Германия видимо си продължи като преди. Сериозната работа за сигурността на гражданите се вършеше в голяма степен от разузнаването, което успяваше да предотврати някои планове за терористични актове в зародиш. В страната все така се провеждат коледни базари, но често се слагат (така, че да не се набиват твърде много на очи) препятствия, които да попречат на превозно средство да се вреже в хората.
Като се има предвид колко е пробита системата ни за национална сигурност, наивно е да се мисли, че с изкарването на жандармеристи с автомати по улиците може да се предотврати терористичен акт. А никой не казва, че има непосредствена опасност от такъв. Тогава защо вътрешният министър в оставка Калин Стоянов предприема показни акции?
Като се има предвид, че се задават избори, този, който владее МВР, в голяма степен има влияние върху районите с контролиран вот, както и върху ромските махали. Присъствието на въоръжени сили на реда може да сплаши когото трябва. Изборите обаче ще се проведат при служебно правителство. Няма гаранция, че Калин Стоянов ще е министър в него.
По-реалистично предположение е с тези акции да се цели насаждане в обществото на страх от някаква надвиснала неясна опасност. Изкарването на тежковъоръжени патрули по улиците не толкова вдъхва сигурност, колкото усещането, че щом са там, значи има от какво да бъдем пазени. Този страх може да се канализира по отношение на чужденците, както и стана в началото на март, или да се нарочи (и) друг враг. И да се насочи в желаната електорална посока.
Канализирайки обществените страхове, подобни акции имат функцията да отвличат вниманието от действително сериозните проблеми. Като „изгубената“ пътна карта за един газопровод и трите милиарда, които България е харизала на Русия за изграждането му. Газопровод, който заобикаля Украйна, което пък после развързва ръцете на Русия да я нападне. Или сериозните проблеми в прокуратурата. Или „Магнитски“. Теми, от които да се отвлича вниманието – бол.
А може би на министъра в оставка Калин Стоянов просто му харесва да демонстрира груба сила – това го кара да се чувства значим. При всички хипотези накрая отговорът може да се окаже обидно прост.
Post Syndicated from Matt Granger original https://www.youtube.com/watch?v=ow3QzhSIex8
Post Syndicated from Explosm.net original https://explosm.net/comics/brain-injury
New Cyanide and Happiness Comic
Post Syndicated from xkcd.com original https://xkcd.com/2912/
Post Syndicated from Max Peterson original https://aws.amazon.com/blogs/security/how-aws-is-helping-customers-achieve-their-digital-sovereignty-and-resilience-goals/
As we’ve innovated and expanded the Amazon Web Services (AWS) Cloud, we continue to prioritize making sure customers are in control and able to meet regulatory requirements anywhere they operate. With the AWS Digital Sovereignty Pledge, which is our commitment to offering all AWS customers the most advanced set of sovereignty controls and features available in the cloud, we are investing in an ambitious roadmap of capabilities for data residency, granular access restriction, encryption, and resilience. Today, I’ll focus on the resilience pillar of our pledge and share how customers are able to improve their resilience posture while meeting their digital sovereignty and resilience goals with AWS.
Resilience is the ability for any organization or government agency to respond to and recover from crises, disasters, or other disruptive events while maintaining its core functions and services. Resilience is a core component of sovereignty and it’s not possible to achieve digital sovereignty without it. Customers need to know that their workloads in the cloud will continue to operate in the face of natural disasters, network disruptions, and disruptions due to geopolitical crises. Public sector organizations and customers in highly regulated industries rely on AWS to provide the highest level of resilience and security to help meet their needs. AWS protects millions of active customers worldwide across diverse industries and use cases, including large enterprises, startups, schools, and government agencies. For example, the Swiss public transport organization BERNMOBIL improved its ability to protect data against ransomware attacks by using AWS.
AWS has made significant investments in building and running the world’s most resilient cloud by building safeguards into our service design and deployment mechanisms and instilling resilience into our operational culture. We build to guard against outages and incidents, and account for them in the design of AWS services—so when disruptions do occur, their impact on customers and the continuity of services is as minimal as possible. To avoid single points of failure, we minimize interconnectedness within our global infrastructure. The AWS global infrastructure is geographically dispersed, spanning 105 Availability Zones (AZs) within 33 AWS Regions around the world. Each Region is comprised of multiple Availability Zones, and each AZ includes one or more discrete data centers with independent and redundant power infrastructure, networking, and connectivity. Availability Zones in a Region are meaningfully distant from each other, up to 60 miles (approximately 100 km) to help prevent correlated failures, but close enough to use synchronous replication with single-digit millisecond latency. AWS is the only cloud provider to offer three or more Availability Zones within each of its Regions, providing more redundancy and better isolation to contain issues. Common points of failure, such as generators and cooling equipment, aren’t shared across Availability Zones and are designed to be supplied by independent power substations. To better isolate issues and achieve high availability, customers can partition applications across multiple Availability Zones in the same Region. Learn more about how AWS maintains operational resilience and continuity of service.
Resilience is deeply ingrained in how we design services. At AWS, the services we build must meet extremely high availability targets. We think carefully about the dependencies that our systems take. Our systems are designed to stay resilient even when those dependencies are impaired; we use what is called static stability to achieve this level of resilience. This means that systems operate in a static state and continue to operate as normal without needing to make changes during a failure or when dependencies are unavailable. For example, in Amazon Elastic Compute Cloud (Amazon EC2), after an instance is launched, it’s just as available as a physical server in a data center. The same property holds for other AWS resources such as virtual private clouds (VPCs), Amazon Simple Storage Service (Amazon S3) buckets and objects, and Amazon Elastic Block Store (Amazon EBS) volumes. Learn more in our Fault Isolation Boundaries whitepaper.
Information Services Group (ISG) cited strengthened resilience when naming AWS a Leader in their recent report, Provider Lens for Multi Public Cloud Services – Sovereign Cloud Infrastructure Services (EU), “AWS delivers its services through multiple Availability Zones (AZs). Clients can partition applications across multiple AZs in the same AWS region to enhance the range of sovereign and resilient options. AWS enables its customers to seamlessly transport their encrypted data between regions. This ensures data sovereignty even during geopolitical instabilities.”
AWS empowers governments of all sizes to safeguard digital assets in the face of disruptions. We proudly worked with the Ukrainian government to securely migrate data and workloads to the cloud immediately following Russia’s invasion, preserving vital government services that will be critical as the country rebuilds. We supported the migration of over 10 petabytes of data. For context, that means we migrated data from 42 Ukraine government authorities, 24 Ukrainian universities, a remote learning K–12 school serving hundreds of thousands of displaced children, and dozens of other private sector companies.
For customers who are running workloads on-premises or for remote use cases, we offer solutions such as AWS Local Zones, AWS Dedicated Local Zones, and AWS Outposts. Customers deploy these solutions to help meet their needs in highly regulated industries. For example, to help meet the rigorous performance, resilience, and regulatory demands for the capital markets, Nasdaq used AWS Outposts to provide market operators and participants with added agility to rapidly adjust operational systems and strategies to keep pace with evolving industry dynamics.
Millions of customers trust that AWS is the right place to build and run their business-critical and mission-critical applications. We provide a comprehensive set of purpose-built resilience services, strategies, and architectural best practices that you can use to improve your resilience posture and meet your sovereignty goals. These services, strategies, and best practices are outlined in the AWS Resilience Lifecycle Framework across five stages—Set Objectives, Design and Implement, Evaluate and Test, Operate, and Respond and Learn. The Resilience Lifecycle Framework is modeled after a standard software development lifecycle, so you can easily incorporate resilience into your existing processes.
For example, you can use the AWS Resilience Hub to set your resilience objectives, evaluate your resilience posture against those objectives, and implement recommendations for improvement based on the AWS Well-Architected Framework and AWS Trusted Advisor. Within Resilience Hub, you can create and run AWS Fault Injection Service experiments, which allow you to test how your application will respond to certain types of disruptions. Recently, Pearson, a global provider of educational content, assessment, and digital services to learners and enterprises, used Resilience Hub to improve their application resilience.
Other AWS resilience services such as AWS Backup, AWS Elastic Disaster Recovery (AWS DRS), and Amazon Route53 Application Recovery Controller (Route 53 ARC) can help you quickly respond and recover from disruptions. When Thomson Reuters, an international media company that provides solutions for tax, law, media, and government to clients in over 100 countries, wanted to improve data protection and application recovery for one of its business units, they adopted AWS DRS. AWS DRS provides Thomson Reuters continuous replication, so changes they made in the source environment were updated in the disaster recovery site within seconds.
AWS offers multiple ways for you to achieve your resilience goals, including assistance from AWS Partners and AWS Professional Services. AWS Resilience Competency Partners specialize in improving customers’ critical workloads’ availability and resilience in the cloud. AWS Professional Services offers Resilience Architecture Readiness Assessments, which assess customer capabilities in eight critical domains—change management, disaster recovery, durability, observability, operations, redundancy, scalability, and testing—to identify gaps and areas for improvement.
We remain committed to continuing to enhance our range of sovereign and resilient options, allowing customers to sustain operations through disruption or disconnection. AWS will continue to innovate based on customer needs to help you build and run resilient applications in the cloud to keep up with the changing world.
If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.
Post Syndicated from jzb original https://lwn.net/Articles/966187/
The GNOME project announced
GNOME 46 (code-named “Kathmandu”) on March 20. The release has quite a few updates and improvements
across user applications, developer tools, and under the hood. One
thing stood out while looking over this release—a major emphasis on
Flatpaks as the way to acquire and update GNOME software.
Post Syndicated from Abhishek Panday original https://aws.amazon.com/blogs/security/use-amazon-verified-permissions-for-fine-grained-authorization-at-scale/
Implementing user authentication and authorization for custom applications requires significant effort. For authentication, customers often use an external identity provider (IdP) such as Amazon Cognito. Yet, authorization logic is typically implemented in code. This code can be prone to errors, especially as permissions models become complex, and presents significant challenges when auditing permissions and deciding who has access to what. As a result, within Common Weakness Enumeration’s (CWE’s) list of the Top 25 Most Dangerous Software Weaknesses for 2023, four are related to incorrect authorization.
At re:Inforce 2023, we launched Amazon Verified Permissions, a fine-grained permissions management service for the applications you build. Verified Permissions centralizes permissions in a policy store and lets developers use those permissions to authorize user actions within their applications. Permissions are expressed as Cedar policies. You can learn more about the benefits of moving your permissions centrally and expressing them as policies in Policy-based access control in application development with Amazon Verified Permissions.
In this post, we explore how you can provide a faster and richer user experience while still authorizing all requests in the application. You will learn two techniques—bulk authorization and response caching—to improve the efficiency of your applications. We describe how you can apply these techniques when listing authorized resources and actions and loading multiple components on webpages.
You can use Verified Permissions to enforce permissions that determine what the user is able to see at the level of the user interface (UI), and what the user is permitted to do at the level of the API.
Cedar provides consistent and readable policies that can be used at both the level of the UI and the API. For example, a single policy can be checked at the level of the UI to determine whether to show the transfer funds button and checked at the level of the API to determine authority to initiate the funds transfer.
Verified Permissions can be used for implementing fine-grained API permissions. Customer applications can use Verified Permissions to authorize API requests, based on centrally managed Cedar policies, with low latency. Applications authorize such requests by calling the IsAuthorized API of the service, and the response contains whether the request is allowed or denied. Customers are happy with the latency of individual authorization requests, but have asked us to help them improve performance for use cases that require multiple authorization requests. They typically mention two use cases:
In this post, we show you two techniques to optimize the application’s latency based on API permissions and UI permissions.
You can use UI permissions to authorize what resources and actions a user can view in an application. We see developers implementing these controls by first generating a small set of resources based on database filters and then further reducing the set down to authorized resources by checking permissions on each resource using Verified Permissions. For example, when a user of a business banking system tries to view balances on company bank accounts, the application first filters the list to the set of bank accounts for that company. The application then filters the list further to only include the accounts that the user is authorized to view by making an API request to Verified Permissions for each account in the list. With batch authorization, the application can make a single API call to Verified Permissions to filter the list down to the authorized accounts.
Similarly, you can use UI permissions to determine what components of a page or actions should be visible to users of the application. For example, in a banking application, the application wants to control the sub-products (such as credit card, bank account, or stock trading) visible to a user or only display authorized actions (such as transfer or change address) when displaying an account overview page. Customers want to use Verified Permissions to determine which components of the page to display, but that can adversely impact the user experience (UX) if they make multiple API calls to build the page. With batch authorization, you can make one call to Verified Permissions to determine permissions for all components of the page. This enables you to provide a richer experience in your applications by displaying only the components that the user is allowed to access while maintaining low page load latency.
Compound authorization is where a single user action results in a sequence of multiple authorization calls. You can use bulk authorization combined with response caching to improve efficiency. The application makes a single bulk authorization request to Verified Permissions to determine whether each of the component API calls are permitted and the response is cached. This cache is then referenced for each component’s API call in the sequence.
We’re using an online order management application for a toy store to demonstrate how you can apply batch authorization and response caching to improve UX and application performance.
One function of the application is to enable employees in a store to process online orders.
The application is used by two types of users:
The application supports these use cases:
Figure 1 shows orders for Julian, who is a pack associate in the Soft Toy department
Figure 1: Orders for Julian in the Soft Toy department
Figure 2: Actions available to Julian as a pack associate
Figure 3: Order Details for Julian, showing permitted actions
The application uses Verified Permissions as a centralized policy store. These policies are expressed in Cedar. The application uses the Role management using policy templates approach for implementing role-based access controls. We encourage you to read best practices for using role-based access control in Cedar to understand if the approach fits your use case.
In the sample application, the policy template for the store owner role looks like the following:
When a user is assigned a role, the application creates a policy from the corresponding template by passing the user and store. For example, the policy created for the store owner is as follows:
To learn more about the policy design of this application, see the readme file of the application.
In this section, we discuss high level design, challenges with the barebones integration, and how you can use the preceding techniques to reduce latency and costs.
Figure 4: Architecture for listing orders
As shown in Figure 4, the process to list orders is:
Figure 5 shows that the application called IsAuthorized multiple times, sequentially. Multiple sequential calls cause the page to be slow to load and increase infrastructure costs.
Figure 5: Graph showing repeated calls to IsAuthorized
If you transition to using batch authorization, the application can receive 30 authorization decisions with a single API call to Verified Permissions. As you can see in Figure 6, the time to authorize has reduced from close to 800 ms to 79 ms, delivering a better overall user experience.
Figure 6: Reduced latency by using batch authorization
Figure 7: Order actions architecture
As shown in Figure 7, the process to get authorized actions for an order is:
As you saw with listing orders, Figure 8 shows how the application is still calling IsAuthorized multiple times, sequentially. This means the page remains slow to load and has increased impacts on infrastructure costs.
Figure 8: Graph showing repeated calls to IsAuthorized
If you add another layer by transitioning to using batch authorization once again, the application can receive all decisions with a single API call to Verified Permissions. As you can see from Figure 9, the time to authorize has reduced from close to 500 ms to 150 ms, delivering an improved user experience.
Figure 9: Graph showing results of layering batch authorization
Figure 10: Order viewing architecture
The process to view an order, shown in Figure 10, is:
In using the standard authorization pattern for this use case, the application calls Verified Permissions three times. This is because the user action to view an order requires compound authorization because each API call made by the console is authorized. While this enforces least privilege, it impacts the page load and reload latency of the application.
You can use batch authorization and decision caching to reduce latency. In the sample application, the cache is maintained by API Gateway. As shown in Figure 11, applying these techniques to the console application results in only one call to Verified Permissions, reducing latency.
Figure 11: Batch authorization with decision caching architecture
The decision caching processshown in Figure 11, is:
You’ve seen how you can use caching to implement fine-grained authorization at scale in your applications. This technique works well when your application has high cache hit rates, where authorization results are frequently loaded from the cache. Applications where the users initiate the same action multiple times or have a predictable sequence of actions will observe high cache hit rates. Another consideration is that employing caching can delay the time between policy updates and policy enforcement. We don’t recommend using caching for authorization decisions if your application requires policies to take effect quickly or your policies are time dependent (for example, a policy that gives access between 10:00 AM and 2:00 PM).
In this post, we showed you how to implement fine grained permissions in application at scale using Verified Permissions. We covered how you can use batch authorization and decision caching to improve performance and ensure Verified Permissions remains a cost-effective solution for large-scale applications. We applied these techniques to a demo application, avp-toy-store-sample, that is available to you for hands-on testing. For more information about Verified Permissions, see the Amazon Verified Permissions product details and Resources.
If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.
Post Syndicated from Brent Nowak original https://backblazeprod.wpenginepowered.com/blog/backblaze-network-stats-real-world-measurements-from-the-backblaze-us-west-region/
As you’re sitting down to watch yet another episode of “Mystery Science Theater 3000” (Anyone else? Just me?), you might be thinking about how you’re receiving more than 16 billion signals over your home internet connection that need to be interpreted and sent to decoding software in order to be viewed on a screen as images. (Anyone else? Just me?)
In telecommunications, we study how signals (electrical and optical) propagate in a medium (copper wires, optical fiber, or through the air). Signals bounce off objects, degrade as they travel, and arrive at the receiving end (hopefully) intact enough to be interpreted as a binary 0 or 1, eventually to be decoded to form images that we see on our screens.
Today in the latest installment of Network Stats, I’m going to talk about one of the most important things that I’ve learned about the application of telecommunication to computer networks: the study of how long operations take. It’s a complicated process, but that doesn’t make buffering any less annoying, am I right? And if you’re using cloud storage in any way, you rely on signals to run applications, back up your data, and maybe even stream out those movies yourself.
So, let’s get into how we measure data transmission, what things we can do to improve transmission time, and some real-world measurements from the Backblaze US-West region.
At the risk of being too basic, when we study network communication, we’re measuring how long signals take to get from point A to point B, which implies that there is some amount of distance between them. This is also known as latency. As networks have evolved, the time it takes to get from point A to point B has gone from being measured in hours to being measured in fractions of a second. Since we live in more human relatable terms like minutes, hours, and days, it can be hard for us to understand concepts like nanoseconds (a billionth of a second).
Here’s a breakdown of how many milli, micro, and nanoseconds are in one second.
| Time | Symbol | Number in 1 Second |
|---|---|---|
| 1 second | 1 | |
| 1 millisecond | ms | 1,000 |
| 1 microsecond | μs | 1,000,000 |
| 1 nanosecond | ns | 1,000,000,000 |
For reference, taking a deep breath takes approximately one second. When you’re watching TV, you start to notice audio delays at approximately 10–20ms, and if your cat pushes your pen off your desk, it will hit the ground in approximately 400ms.
In the 1960s, Grace Murray Hopper (1906–1992) explained computer operations and what a nanosecond means in a succinct, tangible fashion. An early computer scientist who also served in the U.S. Navy, Hopper was often asked by folks like generals and admirals why it took so long for signals to reach satellites. So, Hopper had pieces of wire cut to 30cm (11.8in) in length, which is the distance that it takes light to travel in perfect conditions—that is, a vacuum—in one nanosecond. (Remember: we humans express speed as distance over time.)
You could touch the wire, feel the distance from point A to point B, and see what a nanosecond in light-time meant. Put a bunch of them together, and you’d see what that looks like on a human scale. In literal terms, she was forcing people to measure the distance to a satellite (anywhere between 160–35,786 km) in terms of the long side of a piece of printer paper. (Rough math: it’s about 741,058,823 to 1,657,529,411,764 pieces of paper end-to-end.) That’ll definitely make you realize that there are a lot of nanoseconds between you and the next city over or a satellite in space.
A fun side note: if we go up a factor from a nanosecond to a microsecond, the wire would be almost 300 meters (984 feet)—which is the height of the Eiffel Tower, minus the broadcast antennae. It gets even more fun to think about because we still have to scale up to two more orders of magnitude to get to a millisecond, and then again to get a second.
I love when a difficult topic can be grasped with an elegant explanation. It’s one of the skills that I strive to develop as an engineer—how can I relate a complicated concept to a larger audience that doesn’t have years of study in my field, and make it easily digestible and memorable?
We don’t live in an ideal, perfect vacuum where signals can propagate in a line-of-sight fashion and travel in an optimal path. We have wires in buildings and in the ground that wind around metro regions, follow long-haul railroad tracks as they curve, and have to pass over hills and along mountainous terrain that add elevation to the wire length, all adding light-time.
These physical factors are not the only component in the way of receiving your data. There are computer operations that add time to our transactions. We have to send a “hello” message to the server and wait for an acknowledgement, negotiate our security protocols so that we can transmit data without anyone snooping in on the conversation, spend time receiving the bytes that make up our file, and acknowledge chunks of information as received so the server can send more.
How much do geography and software operations add to the time it takes to get a file? That’s what we’re going to explore here. So, if we’re requesting a file that’s stored on Backblaze’s servers in the US-West region, what does that look like if we are delivering the file to different locations across the continental U.S.? How long does it take?
Today we’re going to focus on network statistics for our US-West location and how the performance profile looks as we travel east and the various components that contribute to the change in translation time.
Below we’re going to compare theoretical best case numbers to the real world. There are three categories in our analysis that contribute to the total time it takes to get a request from a person in Los Angeles, Chicago, or New York, to our servers in the US-West region. Let’s take a look at each one:
Fun fact: You don’t need any monitoring infrastructure in order to take a deeper dive—every Chrome web browser has the ability to show load times for all the elements that are needed to present a website.
Do note that test results may vary based on your ISP connectivity, hardware capabilities, software stack, or improvements Backblaze makes over time.
To show more detailed information, open Chrome:
A deeper dive into this can be found on Google’s developer.chrome.com website.
If you wish to run agent based tests, you can start with Google’s Chromium Project, as it offers a free and open source method to simulate and perform profiling.
Here are the results from just one test we ran:
It’s important, at this stage, to caveat these numbers with a few things. First, they include a decent amount of overhead from being within our (or any) infrastructure, which can be affected by things like your browser, security, and lookup time needed to connect to a server infrastructure. And, if a user is running a different browser, has different layers of security, and so on, those things can affect RTT results.
Second, they don’t accurately talk about performance without context. Every type of application has its own requirements for what is a “good” or “bad” RTT time, and these numbers can change based on how you store your data; where you store, access, and serve your data; if you use a content delivery network (CDN); and more. As with anything related to performance in cloud storage, your use case determines your cloud storage strategy, not the other way around.
In addition to the Chrome tools we talk about above, we have access to agents running in various geographical locations across the world that run periodic tests to our services. These agents can simulate client activity and record metrics for us that we use for alerting and trend analysis. Simulating client operations helps alert our operations teams to potential issues, helping us to better support our clients and be more proactive.
With this type of agent based testing, we have greater insight into the network that lets us break down the Software step and observe if any one step in the transfer pipeline is underperforming. We’re not only looking at the entire round trip time of downloading a file, but also including all the browser, security, and lookup time needed to connect to our server infrastructure. And, as always, the biggest addition in time it takes to deliver files is often distance-based latency, or the fact that even with ideal conditions, the further away an end user is, the longer it takes to transport data across networks.
The below chart shows how long in milliseconds it takes to get a sample file from our US-West cluster from agents running in different locations across the U.S. and all the Software steps involved.
You can find definitions for all these terms in the Chromium dev docs, but here’s a cheat sheet for our purposes:
Let’s zoom in on the Los Angeles and New York tests and group just the Download (Content Download) and all the other Setup items (Queueing, DNS Lookup, SSL, Initial Connection, TTFB) and see if they differ drastically.
In the Los Angeles test, which is the closest geographical test to the US-West cluster, the total transaction time is 71ms. It takes our storage system 23.8ms to start to send the file, and we’re spending 47ms (66%) of the total time in setup.
If we go further east to New York, we can see how much more time it takes to retrieve our test file from the West (71ms vs 470ms), but the ratio between download and setup doesn’t differ all that drastically. This is because all of our operations are the same, but we’re spending more sending each file over the network, so it all scales up.
Note that no matter where the client request is coming from, the data center is doing the same amount of work to serve up the file.
Latency is certainly a factor to consider when you choose where to store your data, especially if you are running extremely time sensitive processes like content delivery—as we’ve noted here and elsewhere, the closer you are to your end user, the greater the speed of delivery. Content delivery networks (CDNs) can offer one way to layer your network capabilities for greater speed of delivery, and Backblaze offers completely free egress through many CDN partners. (This is in addition to our normal amount of free egress, which is up to 3x the data stored and includes the vast majority of use cases.)
There are other reasons to consider different regions for storage as well, such as compliance and disaster resilience. Our EU datacenter, for instance, helps to support GDPR compliance. Also, if you’re storing data in only one location, you’re more vulnerable to natural disasters. Redundancy is key to a good disaster recovery or business continuity plan, and you want to make sure to consider power regions in that analysis. In short, as with all things in storage optimization, considering your use case is key to balancing performance and cost.
I started this article talking about Grace Murray Hopper demonstrating nanoseconds with pieces of wire, and we’re concluding what can be considered light years (ha) away from that point. The biggest thing to remember, as a network engineering team, is that even though approximately 600ms from the US-West to the US-East regions seems miniscule, the amount of times we travel that distance very quickly takes us up those orders of magnitude from milliseconds to seconds. And, when you, the user, are choosing where to store your data—knowing that we register audio lag at as little as 10ms—those inconsequential numbers start to get to human relative terms very, very quickly.
So, when we find that peering shaves a few milliseconds off of a file delivery time, that’s a big, human-sized win. You can see some of the ways we’re optimizing our network in the first article of this series, and the types of tests we’re running above give us good insights—and inspiration—for more, better, and ongoing improvements. We’ll keep sharing what we’re measuring and how we’re improving, and we’re looking forward to seeing you all in the comments.
The post Backblaze Network Stats: Real-World Measurements from the Backblaze US-West Region appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.
Post Syndicated from BeardedTinker original https://www.youtube.com/watch?v=6fCZk85Ex3I
Post Syndicated from Patrick Kennedy original https://www.servethehome.com/asrock-rack-sienad8-2l2t-amd-epyc-8004-siena-motherboard-review/
In our ASRock Rack SIENAD8-2L2T review, we see how this ATX size AMD EPYC 8004 motherboard delivers utility with a little added flair
The post ASRock Rack SIENAD8-2L2T AMD EPYC 8004 Siena Motherboard Review appeared first on ServeTheHome.
Post Syndicated from Ismail Makhlouf original https://aws.amazon.com/blogs/big-data/create-an-end-to-end-data-strategy-for-customer-360-on-aws/
Customer 360 (C360) provides a complete and unified view of a customer’s interactions and behavior across all touchpoints and channels. This view is used to identify patterns and trends in customer behavior, which can inform data-driven decisions to improve business outcomes. For example, you can use C360 to segment and create marketing campaigns that are more likely to resonate with specific groups of customers.
In 2022, AWS commissioned a study conducted by the American Productivity and Quality Center (APQC) to quantify the Business Value of Customer 360. The following figure shows some of the metrics derived from the study. Organizations using C360 achieved 43.9% reduction in sales cycle duration, 22.8% increase in customer lifetime value, 25.3% faster time to market, and 19.1% improvement in net promoter score (NPS) rating.
Without C360, businesses face missed opportunities, inaccurate reports, and disjointed customer experiences, leading to customer churn. However, building a C360 solution can be complicated. A Gartner Marketing survey found only 14% of organizations have successfully implemented a C360 solution, due to lack of consensus on what a 360-degree view means, challenges with data quality, and lack of cross-functional governance structure for customer data.
In this post, we discuss how you can use purpose-built AWS services to create an end-to-end data strategy for C360 to unify and govern customer data that address these challenges. We structure it in five pillars that power C360: data collection, unification, analytics, activation, and data governance, along with a solution architecture that you can use for your implementation.
When you embark on creating a C360, you work with multiple use cases, types of customer data, and users and applications that require different tools. Building a C360 on the right datasets, adding new datasets over time while maintaining the quality of data, and keeping it secure needs an end-to-end data strategy for your customer data. You also need to provide tools that make it straightforward for your teams to build products that mature your C360.
We recommend building your data strategy around five pillars of C360, as shown in the following figure. This starts with basic data collection, unifying and linking data from various channels related to unique customers, and progresses towards basic to advanced analytics for decision-making, and personalized engagement through various channels. As you mature in each of these pillars, you progress towards responding to real-time customer signals.
The following diagram illustrates the functional architecture that combines the building blocks of a Customer Data Platform on AWS with additional components used to design an end-to-end C360 solution. This is aligned to the five pillars we discuss in this post.
As you start building your customer data platform, you have to collect data from various systems and touchpoints, such as your sales systems, customer support, web and social media, and data marketplaces. Think of the data collection pillar as a combination of ingestion, storage, and processing capabilities.
You have to build ingestion pipelines based on factors like types of data sources (on-premises data stores, files, SaaS applications, third-party data), and flow of data (unbounded streams or batch data). AWS provides different services for building data ingestion pipelines:
The following diagram illustrates the different pipelines to ingest data from various source systems using AWS services.
Structured, semi-structured, or unstructured batch data is stored in an object storage because these are cost-efficient and durable. Amazon Simple Storage Service (Amazon S3) is a managed storage service with archiving features that can store petabytes of data with eleven 9’s of durability. Streaming data with low latency needs is stored in Amazon Kinesis Data Streams for real-time consumption. This allows immediate analytics and actions for various downstream consumers—as seen with Riot Games’ central Riot Event Bus.
Raw data is often cluttered with duplicates and irregular formats. You need to process this to make it ready for analysis. If you are consuming batch data and streaming data, consider using a framework that can handle both. A pattern such as the Kappa architecture views everything as a stream, simplifying the processing pipelines. Consider using Amazon Managed Service for Apache Flink to handle the processing work. With Managed Service for Apache Flink, you can clean and transform the streaming data and direct it to the appropriate destination based on latency requirements. You can also implement batch data processing using Amazon EMR on open source frameworks such as Apache Spark at 3.5 times better performance than the self-managed version. The architecture decision of using a batch or streaming processing system will depend on various factors; however, if you want to enable real-time analytics on your customer data, we recommend using a Kappa architecture pattern.
To link the diverse data arriving from various touchpoints to a unique customer, you need to build an identity processing solution that identifies anonymous logins, stores useful customer information, links them to external data for better insights, and groups customers in domains of interest. Although the identity processing solution helps build the unified customer profile, we recommend considering this as part of your data processing capabilities. The following diagram illustrates the components of such a solution.
The key components are as follows:
After you have done the identity processing and segmentation, you need a storage capability to store the unique customer profile and provide search and query capabilities on top of it for downstream consumers to use the enriched customer data.
The following diagram illustrates the unification pillar for a unified customer profile and single view of the customer for downstream applications.
Graph databases excel in modeling customer interactions and relationships, offering a comprehensive view of the customer journey. If you are dealing with billions of profiles and interactions, you can consider using Neptune, a managed graph database service on AWS. Organizations such as Zeta and Activision have successfully used Neptune to store and query billions of unique identifiers per month and millions of queries per second at millisecond response time.
Although graph databases provide in-depth insights, yet they can be complex for regular applications. It is prudent to consolidate this data into a single customer view, serving as a primary reference for downstream applications, ranging from ecommerce platforms to CRM systems. This consolidated view acts as a liaison between the data platform and customer-centric applications. For such purposes, we recommend using Amazon DynamoDB for its adaptability, scalability, and performance, resulting in an up-to-date and efficient customer database. This database will accept a lot of write queries back from the activation systems that learn new information about the customers and feed them back.
The analytics pillar defines capabilities that help you generate insights on top of your customer data. Your analytics strategy applies to the wider organizational needs, not just C360. You can use the same capabilities to serve financial reporting, measure operational performance, or even monetize data assets. Strategize based on how your teams explore data, run analyses, wrangle data for downstream requirements, and visualize data at different levels. Plan on how you can enable your teams to use ML to move from descriptive to prescriptive analytics.
The AWS modern data architecture shows a way to build a purpose-built, secure, and scalable data platform in the cloud. Learn from this to build querying capabilities across your data lake and the data warehouse.
The following diagram breaks down the analytics capability into data exploration, visualization, data warehousing, and data collaboration. Let’s find out what role each of these components play in the context of C360.
Data exploration helps unearth inconsistencies, outliers, or errors. By spotting these early on, your teams can have cleaner data integration for C360, which in turn leads to more accurate analytics and predictions. Consider the personas exploring the data, their technical skills, and the time to insight. For instance, data analysts who know to write SQL can directly query the data residing in Amazon S3 using Amazon Athena. Users interested in visual exploration can do so using AWS Glue DataBrew. Data scientists or engineers can use Amazon EMR Studio or Amazon SageMaker Studio to explore data from the notebook, and for a low-code experience, you can use Amazon SageMaker Data Wrangler. Because these services directly query S3 buckets, you can explore the data as it lands in the data lake, reducing time to insight.
Turning complex datasets into intuitive visuals unravels hidden patterns in the data, and is crucial for C360 use cases. With this capability, you can design reports for different levels catering to varying needs: executive reports offering strategic overviews, management reports highlighting operational metrics, and detailed reports diving into the specifics. Such visual clarity helps your organization make informed decisions across all tiers, centralizing the customer’s perspective.
The following diagram shows a sample C360 dashboard built on Amazon QuickSight. QuickSight offers scalable, serverless visualization capabilities. You can benefit from its ML integrations for automated insights like forecasting and anomaly detection or natural language querying with Amazon Q in QuickSight, direct data connectivity from various sources, and pay-per-session pricing. With QuickSight, you can embed dashboards to external websites and applications, and the SPICE engine enables rapid, interactive data visualization at scale. The following screenshot shows an example C360 dashboard built on QuickSight.
Data warehouses are efficient in consolidating structured data from multifarious sources and serving analytics queries from a large number of concurrent users. Data warehouses can provide a unified, consistent view of a vast amount of customer data for C360 use cases. Amazon Redshift addresses this need by adeptly handling large volumes of data and diverse workloads. It provides strong consistency across datasets, allowing organizations to derive reliable, comprehensive insights about their customers, which is essential for informed decision-making. Amazon Redshift offers real-time insights and predictive analytics capabilities for analyzing data from terabytes to petabytes. With Amazon Redshift ML, you can embed ML on top of the data stored in the data warehouse with minimum development overhead. Amazon Redshift Serverless simplifies application building and makes it straightforward for companies to embed rich data analytics capabilities.
You can securely collaborate and analyze collective datasets from your partners without sharing or copying one another’s underlying data using AWS Clean Rooms. You can bring together disparate data from across engagement channels and partner datasets to form a 360-degree view of your customers. AWS Clean Rooms can enhance C360 by enabling use cases like cross-channel marketing optimization, advanced customer segmentation, and privacy-compliant personalization. By safely merging datasets, it offers richer insights and robust data privacy, meeting business needs and regulatory standards.
The value of data diminishes the older it gets, leading to higher opportunity costs over time. In a survey conducted by Intersystems, 75% of the organizations surveyed believe untimely data inhibited business opportunities. In another survey, 58% of organizations (out of 560 respondents of HBR Advisory council and readers) stated they saw an increase in customer retention and loyalty using real-time customer analytics.
You can achieve a maturity in C360 when you build the ability to act on all the insights acquired from the previous pillars we discussed in real time. For example, at this maturity level, you can act on customer sentiment based on the context you automatically derived with an enriched customer profile and integrated channels. For this you need to implement prescriptive decision-making on how to address the customer’s sentiment. To do this at scale, you have to use AI/ML services for decision-making. The following diagram illustrates the architecture to activate insights using ML for prescriptive analytics and AI services for targeting and segmentation.
With ML, you can improve the overall customer experience—you can create predictive customer behavior models, design hyper-personalized offers, and target the right customer with the right incentive. You can build them using Amazon SageMaker, which features a suite of managed services mapped to the data science lifecycle, including data wrangling, model training, model hosting, model inference, model drift detection, and feature storage. SageMaker enables you to build and operationalize your ML models, infusing them back into your applications to produce the right insight to the right person at the right time.
Amazon Personalize supports contextual recommendations, through which you can improve the relevance of recommendations by generating them within a context—for instance, device type, location, or time of day. Your team can get started without any prior ML experience using APIs to build sophisticated personalization capabilities in a few clicks. For more information, see Customize your recommendations by promoting specific items using business rules with Amazon Personalize.
Now that you know who your customers are and who to reach out to, you can build solutions to run targeting campaigns at scale. With Amazon Pinpoint, you can personalize and segment communications to engage customers across multiple channels. For example, you can use Amazon Pinpoint to build engaging customer experiences through various communication channels like email, SMS, push notifications, and in-app notifications.
Establishing the right governance that balances control and access gives users trust and confidence in data. Imagine offering promotions on products that a customer doesn’t need, or bombarding the wrong customers with notifications. Poor data quality can lead to such situations, and ultimately results in customer churn. You have to build processes that validate data quality and take corrective actions. AWS Glue Data Quality can help you build solutions that validate the quality of data at rest and in transit, based on predefined rules.
To set up a cross-functional governance structure for customer data, you need a capability for governing and sharing data across your organization. With Amazon DataZone, admins and data stewards can manage and govern access to data, and consumers such as data engineers, data scientists, product managers, analysts, and other business users can discover, use, and collaborate with that data to drive insights. It streamlines data access, letting you find and use customer data, promotes team collaboration with shared data assets, and provides personalized analytics either via a web app or API on a portal. AWS Lake Formation makes sure data is accessed securely, guaranteeing the right people see the right data for the right reasons, which is crucial for effective cross-functional governance in any organization. Business metadata is stored and managed by Amazon DataZone, which is underpinned by technical metadata and schema information, which is registered in the AWS Glue Data Catalog. This technical metadata is also used both by other governance services such as Lake Formation and Amazon DataZone, and analytics services such as Amazon Redshift, Athena, and AWS Glue.
Using the following diagram as a reference, you can create projects and teams for building and operating different capabilities. For example, you can have a data integration team focus on the data collection pillar—you can then align functional roles, like data architects and data engineers. You can build your analytics and data science practices to focus on the analytics and activation pillars, respectively. Then you can create a specialized team for customer identity processing and for building the unified view of the customer. You can establish a data governance team with data stewards from different functions, security admins, and data governance policymakers to design and automate policies.
Building a robust C360 capability is fundamental for your organization to gain insights into your customer base. AWS Databases, Analytics, and AI/ML services can help streamline this process, providing scalability and efficiency. Following the five pillars to guide your thinking, you can build an end-to-end data strategy that defines the C360 view across the organization, makes sure data is accurate, and establishes cross-functional governance for customer data. You can categorize and prioritize the products and features you have to build within each pillar, select the right tool for the job, and build the skills you need in your teams.
Visit AWS for Data Customer Stories to learn how AWS is transforming customer journeys, from the world’s largest enterprises to growing startups.
Ismail Makhlouf is a Senior Specialist Solutions Architect for Data Analytics at AWS. Ismail focuses on architecting solutions for organizations across their end-to-end data analytics estate, including batch and real-time streaming, big data, data warehousing, and data lake workloads. He primarily works with organizations in retail, ecommerce, FinTech, HealthTech, and travel to achieve their business objectives with well architected data platforms.
Sandipan Bhaumik (Sandi) is a Senior Analytics Specialist Solutions Architect at AWS. He helps customers modernize their data platforms in the cloud to perform analytics securely at scale, reduce operational overhead, and optimize usage for cost-effectiveness and sustainability.
Post Syndicated from jake original https://lwn.net/Articles/966678/
Security updates have been issued by CentOS (kernel), Debian (firefox-esr), Fedora (webkitgtk), Mageia (curaengine & blender and gnutls), Red Hat (firefox, grafana, grafana-pcp, libreoffice, nodejs:18, and thunderbird), SUSE (glade), and Ubuntu (crmsh, debian-goodies, linux-aws, linux-aws-6.5, linux-aws-hwe, linux-azure, linux-azure-4.15, linux-oracle, linux-azure, linux-azure-5.4, linux-oracle, linux-oracle-5.15, pam, and thunderbird).
