What’s New in Rapid7 Products & Services: 2023 Year in Review

2023-12-21 Margaret Wei

Post Syndicated from Margaret Wei original https://blog.rapid7.com/2023/12/21/whats-new-in-rapid7-products-services-2023-year-in-review/

Throughout 2023 Rapid7 has made investments across the Insight Platform to further our mission of providing security teams with the tools to proactively anticipate imminent risk, prevent breaches earlier, and respond faster to threats. In this blog you’ll find a review of our top releases from this past year, all of which were purpose-built to bring your team a holistic, unified approach to security operations and command of your attack surface.

Proactively secure your environment

Endpoint protection with next-gen antivirus in Managed Threat Complete

To provide protection against both known and unknown threats, we released multilayered prevention with Next-Gen Antivirus in Managed Threat Complete. Available through the Insight Agent, you’re immediately able to:

Block known and unknown threats early in the kill chain
Halt malware that’s built to bypass existing security controls
Maximize your security stack and ROI with existing Insight Agent
Leverage the expertise of our MDR team to triage and investigate these alerts

New capabilities to help prioritize risk in your cloud and on-premise environments and effectively communicate risk posture

As the attack surface expands, we know it’s critical for you to have visibility into vulnerabilities across your hybrid environments and communicate it with your executive and remediation stakeholders. This year we made a series of investments in this area to help customers better visualize, prioritize, and communicate risk.

What’s New in Rapid7 Products & Services: 2023 Year in Review

Executive Risk View, available as a part of Cloud Risk Complete, provides security leaders with the visibility and context needed to track total risk across cloud and on-premises assets to better understand organizational risk posture and trends.
Active Risk, our new vulnerability risk-scoring methodology, helps security teams prioritize vulnerabilities that are actively exploited or most likely to be exploited in the wild. Our approach enriches the latest version of the Common Vulnerability Scoring System (CVSS) with multiple threat intelligence feeds, including intelligence from proprietary Rapid7 Labs research. Active Risk normalizes risk scores across cloud and on-premises environments within InsightVM, InsightCloudSec, and Executive Risk View.
The new risk score in InsightCloudSec’s Layered Context makes it easier for you to understand the riskiest resources within your cloud environment. Much like Layered Context, the new risk score combines a variety of risk signals – including Active Risk – and assigns a higher risk score to resources that suffer from toxic combinations or multiple risk vectors that present an increased likelihood or impact of compromise.
Two new dashboard cards in InsightVM to help security teams communicate risk posture cross-functionally and provide context on asset and vulnerability prioritization:
Vulnerability Findings by Active Risk Score Severity – ideal for executive reporting, this dashboard card indicates total number of vulnerabilities across the Active Risk severity levels and number of affected assets and instances.
Vulnerability Findings by Active Risk Score Severity and Publish Age – ideal for sharing with remediation stakeholders to assist with prioritizing vulnerabilities for the next patch cycle, or identifying critical vulnerabilities that may have been missed.

Coverage and expert analysis for critical vulnerabilities with Rapid7 Labs

Rapid7 Labs provides easy-to-use threat intelligence and guidance, curated by our industry-leading attack experts, to the security teams.

Emergent Threat Response (ETR) program, part of Rapid7 Labs, provides teams with accelerated visibility, alerting, and guidance on high-priority threats. Over this past year we provided coverage and expert analysis within 24 hours for over 30 emergent threats, including Progress Software’s MOVEit Transfer solution where our security research team was one of the first to detect exploitation—four days before the vendor issued public advisory. Keep up with future ETRs on our blog here.

Detect and prioritize threats anywhere, from the endpoint to the cloud

Enhanced alert details in InsightIDR Investigations

An updated evidence panel for attacker behavior analytics (ABA) alerts gives you a description of the alert and recommendations for triage, rule logic that generated the alert and associated data, and a process tree (for MDR customers) to show details about what occurred before, during, and after the alert was generated.

AI-driven detection of anomalous activity with Cloud Anomaly Detection

Cloud Anomaly Detection provides AI-driven detection of anomalous activity occurring across your cloud environments, with automated prioritization to assess the likelihood that activity is malicious. With Cloud Anomaly Detection, your team will benefit from:

A consolidated view that aggregates threat detections from CSP-native detection engines and Rapid7’s AI-driven proprietary detections.
Automated prioritization to focus on the activity that is most likely to be malicious.
The ability to detect and respond to cloud threats using the same processes and tools your SOC teams are using today with easy API-based ingestion into XDR/SIEM tools for threat investigations and prioritizing remediation efforts.

Detailed views into risks across your cloud environment with Identity Analysis and Attack Path Analysis

We’re constantly working to improve the ways with which we provide a real-time and comprehensive view of your current cloud risk posture. This year, we made some major strides in this area, headlined by two exciting new features:

Identity Analysis provides a unified view into identity-related risk across your cloud environments, allowing you to achieve least privileged access (LPA) at scale. By utilizing machine learning (ML), Identity Analysis builds a baseline of access patterns and permissions usage, and then correlates the baseline against assigned permissions and privileges. This enables your team to identify overly-permissive roles or unused access so you can automatically right-size permissions in accordance with LPA.
Attack Path Analysis enables you to analyze relationships between resources and quickly identify potential avenues bad actors could navigate within your cloud environment to exploit a vulnerable resource and/or access sensitive information. This visualization helps teams communicate risk across the organization, particularly for non-technical stakeholders that may find it difficult to understand why a compromised resource presents a potentially larger risk to the business.

More flexible alerting with Custom Detection Rules

Every environment, industry, and organization can have differing needs when it comes to detections. With custom detection rules in InsightIDR, you can detect threats specific to your needs while take advantage of the same capabilities that are available for out-of-the-box detection rules, including:

The ability to set a rule action and rule priority to choose how you are alerted when your rule detects suspicious activity.
The ability to add exceptions to your rule for specific key-value pairs.

A growing library of actionable detections in InsightIDR

In 2023 we added over 3,000 new detection rules. See them in-product or visit the Detection Library for descriptions and recommendations.

Agent-Based Policy supports custom policy assessment in InsightVM

Guidelines from Center for Internet Security (CIS) and Security Technical Implementation Guides (STIG) are widely used industry benchmarks for configuration assessment. However, a benchmark or guideline as-is may not meet the unique needs of every business.

Agent-Based Policy assessment now supports Custom Policies. Global Administrators can customize built-in policies, upload policies, or enable a copy of existing custom policies for agent-based assessments. Learn more here.

Investigate and respond with confidence

Faster containment and remediation of threats with expansion of Active Response for Managed Detection and Response customers

Attackers work quickly and every second you wait to take action can have detrimental impacts on your environment. Enter automation—Active Response enables Rapid7 SOC analysts to immediately quarantine assets and users in a customer’s environment with response actions powered by InsightConnect, Rapid7’s SOAR solution.

Active Response has you covered to quarantine via our Insight Agent, as well as a variety of third-party providers—including Crowdstrike and SentinelOne. And with MDR analyst actions logged directly in InsightIDR, you have more expansive, collaborative detection and response faster than ever before. Read what Active Response can do for your organization—and how it stopped malware in a recent MDR Investigation—here.

Velociraptor integrates with InsightIDR for broader DFIR coverage

The attack surface is continually expanding, and so should your visibility into potential threats across it. This year we integrated Velociraptor, Rapid7’s open-source DFIR framework, with our Insight Platform to bring the data you need for daily threat monitoring and hunting into InsightIDR for investigation via our Insight Agent.

This integration brings you faster identification and remediation, always-on monitoring for threat activity across your endpoint fleet, and expanded threat detection capabilities. Read more about what this integration unlocks here.

Stay tuned!

As always, we’re continuing to work on exciting product enhancements and releases throughout the year. Keep an eye on our blog and release notes as we continue to highlight the latest in product and service investments at Rapid7. See you in 2024!

The Backblaze Team Recommends: Tech That Saves the Day

2023-12-21 Molly Clancy

Post Syndicated from Molly Clancy original https://www.backblaze.com/blog/the-backblaze-team-recommends-tech-that-saves-the-day/

Everyone has their arsenal of indispensable gadgets and apps they absolutely couldn’t live without, and we had a feeling the folks here at Backblaze would have a lot to say about the subject. We tapped the smart, savvy minds that keep our storage cloud up and running, and discovered a treasure trove of insights into the tech essentials that power their daily lives.

From budgeting apps to text editors to humble charging jacks, our staff share the tools they can’t live without. So, without further ado, let’s dig into the gear that keeps our collective gears turning:

Tech for Staying Connected

Solutions Engineering Director, Troy Liljedahl

As a Mac guy, I love my Airpod Pros and the way they work seamlessly with my iPhone, Macbook Pro, and iPad. But things get a little wonky when you try to use them outside of the Apple ecosystem. I tried many different wireless earbuds and settled on the Anker Soundcore Space A40 Earbuds. I’m a big fan of The Wirecutter by the New York Times (and they’re big fans of us) and they had these at the top of their list. I love the sound quality, noise canceling, and excellent battery life. My Airpod Pros are still my go to when I’m out of the house, but when I need a good headset at home for my PC and other non-Apple devices, these have become my go to earbuds.

Senior Director, Marketing, Yev Pusin

You know what I hate? Getting my phone out at inopportune times. You know what I don’t mind so much? Glancing down at my Google Pixel Watch to see that the cold call I’m receiving is being answered by my phone’s call screening and I don’t have to pick it up. Whether it’s the first version or the second, I have grown accustomed to having something on my wrist that acts as an extension of my phone. True digital bliss.

Chief Technical Evangelist, Pat Patterson

I use my webcam a LOT: Zoom and Google Meet with coworkers, FaceTime with family and friends, webinars with the Backblaze community of developers and admins, and quick-start videos for the Backblaze YouTube channel. Ever since I got my PlexiCam Pro mount about a year ago, it’s been my secret weapon in every one of those interactions. It’s a transparent plexiglass webcam mount that hangs from the top edge of my monitor, allowing me to position my webcam in my eyeline, just above my focus. To anyone on the call, I appear to be looking directly into the camera.

At $85, it’s not cheap, but it’s well designed and constructed, and feels like it will last forever. Highly recommended for anyone who spends a lot of time flicking their eyes between the screen and the webcam!

Tech for Devs

Senior Site Reliability Engineer, nathaniel wagner

Ah, the age old debate of Emacs versus Vim: the two most widely used editors for Linux operating systems. I solidly planted my flag on team Vim once I learned how to save and exit the program. 😉

I do aspire to one day having a computer that only runs Emacs because byte compiled Emacs is cool to me. Until I graduate to that level of wizardry, I stick to Neovim when I need to quickly edit something from a terminal or want to appear cool in front of my coworkers. I mostly use the Vim extension in Intellij for day-to-day modifying of code and configs. If you would like to also learn Vim, I really enjoyed playing through Vim Adventures, which is a free game that teaches you a lot of the shortcuts and movements in Vim.

Lead Software Engineer, Application Security, Ola Nordstrom

Continuing the discussion in favor of Vim, specifically Neovim. You only have to learn the keybindings once. There is a fantastic set of plugins to customize it to your heart’s content—Visual Studio Code has VSCodeVim, Intellij has IdeaVim, for example. Then you don’t have to relearn keybindings while switching between languages, projects, and code.

Tmux the terminal multiplexer: like Vim, it may have a steep learning curve but once you learn it you can’t live without it. The tmux wiki has some great getting started guides. I strongly recommend remapping the leader key (mine is Caps + A, or Caps Lock + A on Windows keyboards). You can set up customized tmux scripts to re-create all your environments (one session for server code, another for a different codebase, another for your notes, and so on). Each session then has multiple windows which you can create, split, and close quickly, no need to leave your keyboard.

For web technologies, learn the toolset available in the browser developer tools. Remember to preserve logs filter to specific responses so you won’t be overwhelmed looking at messages.

Stepping away from the browser and back into the terminal, learn Curl and ag or rg. For every “old” unix command there’s likely a modern replacement that’s 100s of times faster with much more customization available.

But there’s always the middle ground for situations where you may need to initiate a complex series of browser–webapp interactions and you need to modify or test something quickly. To do this, learn how to use Burp. In the long run it’s well worth it. It makes it a breeze to modify data between the browser and your app.

Last tip: for native code, just learn how to use the debugger.

Tusen Tack!

Tech That Makes Home Homier

Principle Site Reliability Engineer, Elliott Sims

One power adapter to rule them all: the Anker 715 Charger (Nano II 65W). This one little power adapter can power my personal laptop, work laptop, headphones, and more. With one cable, one small charger cube, and a few small USB end adapters, I can charge everything.

Senior Product Marketing Manager, James Flores

A Roku. I use it at home for streaming apps, but I also carry one in my travel bag. There’s nothing worse than flipping through basic cable channels in a hotel when you’re traveling. Wait, yes there is—signing in to Netflix on the hotel TV and forgetting to sign out. If I travel with it, I just plug it in and I’m already signed in to all my apps.

Senior Technical Editing Manager, Alison McClelland

I got tired of locking myself out of my own house and forgetting who I gave spare keys to, so I really appreciate this Yale Lock with Nest Connect. It works with the Nest cameras that I mostly use to see whether the UPS or DoorDash delivery person has the nerve to ring my doorbell. (Drop it and run, people!)

It’s secure and easy-to-use; no more locking myself out of my own house in the middle of winter. I can give a code to a friend so they can feed my cats while I’m away, or create temporary passcodes so I don’t have to wait around for the cable guy.

Senior Content Editor, Molly Clancy

I have a hard time turning my brain off at night, so I used to pop in earbuds to listen to something soothing (not comfortable at all!). Then I got this Cozyband as a gift and became 100% addicted to it. I CAN fall asleep without it, but I don’t do it willingly. It’s also good for working out if you hate sweaty earbuds slipping out all the time.

Chief Executive Officer, Gleb Budman

It gets cold in my home, and I don’t necessarily want to heat the whole place when it’s just me. A good old fashioned heated blanket does the trick. All the tech in the world won’t help you when you’re shivering.

The Apps Have Entered the Chat

Partner Marketing Director, Jen Newman

As a working mom with two boys, I am always on the go. Both of my boys are now playing for AAA travel teams. I try my best to keep up with all their games, and LiveBarn is how I stay connected to them. It allows me to pull up a live feed or on-demand video of the game. Last weekend was a great example: they were playing at two different rinks across town. I was able to pull up one game on LiveBarn on my phone, and watch both games literally at the same time. When my older son came home and asked me if I saw his goal, I was able to say, “I heard Coach yell, ‘Nice shot, Newy!’”

Associate Editor and Writer, Stephanie Doyle

My life changed when I adopted a password manager years ago. Before I went back to full-time corporate life in 2020, I freelanced quite a bit—which means an endless series of logins and passwords, depending on how you’re engaging with your clients. And, while I enjoy making up 13–15 character passphrases with a mix of upper and lowercase letters, at least one symbol with some outlawed symbols (but different ones on each site), and then remembering which ones I’ve used for which accounts without reusing them… Oh wait, I actually really don’t like that. I’d rather have a password manager like Bitwarden that can generate passwords, follows me device to device, and allows me to enable biometric controls. And, spoiler alert for any of my family members who diligently read my work (I’m sure): This year the whole family is getting a subscription as a gift, and I can centrally manage it for my non-tech-inclined family members.

Senior Director of Publishing, Patrick Thomas

I’m not going into the story of how I got hooked on this app, but I will tell you that CENTR’s meal planning tool is a life saver. You can set a crazy variety of dietary needs, select your meals and portions for a week, and it spits out a perfectly organized shopping list. Then, when you want to make a meal, you just pop into the app and it tells you exactly how to look like you know what you’re doing in the kitchen. It’s sort of pricey, but the amount of money I save by not ordering in or wasting food that I bought without a plan more than makes up for it.

And, oh yeah, Chris Hemsworth is one of its founders.

Senior Product Marketing Manager, Kari Rivas

Hoping on the app train here. The one I can’t live without is definitely YNAB. I’m trying to get better at budgeting ahead (and get my husband and I on the same page—haha!) and I like their philosophical approach to a typically boring subject.

The “Tech Is a Tool” Answer

Principal Cloud Storage Storyteller, Andy Klein

I don’t have a favorite tech thing. I certainly use lots of them, but I would not be lost without them because I was raised without any of the current tech. I know how to tie my shoes without watching a Youtube video. Just sayin’.

But, you know, just in case.

Thanks, Andy

Leave it to Andy to send us off with a reminder to put the tech down sometimes, as we hope you all get a chance to do this holiday season. But, we also want to know: what’s the tech that you can’t live without? Let us know in the comments.

The post The Backblaze Team Recommends: Tech That Saves the Day appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

От „нова визия за децата“ към демонополизиране на образованието. Разговор с Нора Гавазова

2023-12-21 Надежда Цекулова

Post Syndicated from Надежда Цекулова original https://www.toest.bg/ot-nova-viziya-za-decata-kum-demonopolizirane-na-obrazovanieto-razgovor-s-nora-gavazova/

От „нова визия за децата“ към демонополизиране на образованието. Разговор с Нора Гавазова

Нора Гавазова е детски психолог, специализира в работата с деца на възраст между 2 и 7 години, с техните семейства и учители. В различни периоди от живота си е работила в образованието и сферата на социалните дейности в Германия, Турция и България. Председател е на Фондация „Калейдоскоп“, която свързва България с Reggio Children – Loris Malaguzzi Center, Италия – една от значимите обучителни организации за качествено и включващо ранно образование.

В началото на декември Фондация „Калейдоскоп“ представи на кръгла маса в Народното събрание своята „Визия за децата“ – едно от малкото ценностно ориентирани изследвания на концепцията за ранното образование у нас.

От „нова визия за децата“ към демонополизиране на образованието. Разговор с Нора Гавазова — Нора Гавазова. Снимка: Личен архив

Докладът Ви „Визия за децата“ представя резултатите от национално изследване на демократичните нагласи по отношение на детството. По време на заключителното събитие обаче остана усещането, че дори самите понятия, с които боравите, звучат революционно в контекста на българската образователна система. Нека започнем с опит да ги обясним. Какво означава „демократична нагласа към детството и образованието“?

Образованието не е случайна система, то буквално оформя тъканта на общество. Ако знанието се налага чрез пасивност, заучаване и послушание, с натрупването на тези ценности в поколенията образованието доказано може да започне да накланя обществото към тоталитарен режим. От една страна, хората свикват да приемат знанието като нещо независещо от тях, което се налага от външни авторитети; от друга – свикват и да бъдат пасивни граждани, да се подчиняват безкритично на авторитети. При определени съчетания на социални, културни и политически обстоятелства знаем, че тази комбинация може да има фатални последствия за човечеството.

Това е причината днес да имаме поредица от международни споразумения, които демократичните държави следват при структурирането на националните образователни системи. Те започват да се публикуват точно около периода на двете световни войни, когато постепенно кристализира тази осъзнатост. В началото са били пожелателни, а след Втората световна война стават задължителни за държавите членки на ООН, впоследствие за ЕС.

Следвоенният период играе много важна роля за общественото и политическо отхвърляне на традиционния образователен модел в Западна Европа, САЩ и в други части от света (за съжаление, България не е сред тях) и преизграждането на националните образователни системи около ценности като избор, активност, критично мислене. Идеята е, че тази система не може да бъде оставена да се развива по традиционния модел, защото в дългосрочен план това застрашава не само местната демократична култура, но и международния мир. За съжаление, както наблягаме и в доклада, България се отнася към международното право изцяло формално: условията се изпълняват на хартия, но структурата остава непроменена, тоест тоталитарна.

Може ли да обвържете този проблем по-конкретно с образователната система? Със съжаление трябва да признаем, че почти 35 години след началото на прехода към демокрация това понятие все още остава абстрактно за много хора у нас и вероятно ще е трудно да се асоциира това, за което говорите, с конкретни дефицити на образователната система.

Нещо доста конкретно е например кой и как определя какво означава „образование“. У нас това е само държавата и за да си образован, трябва да си минал задължително по начертания от нея път.

Образованието, което наричаме „традиционно“, е свързано с индустриалния период. Неговата цел е била да създаде множество хора с унифицирани умения – като поточна линия. Това обаче поставя човека в подчинена позиция и ограничава потенциала му. С развиване на идеите за демократичното общество, според които свободата на личността е ценност, се появяват много други концепции за образованието, които се наричат реформаторски. Тези концепции са на повече от век, не са толкова съвременни, колкото си мислим. За да могат да се осъществят обаче, държавата трябва да пусне контрола върху образованието.

Затова едно от ключовите неща, които се казват в документите, споменати по-рано, е, че не може да има държавен образователен монопол. В България положението е много особено – частното образование не е забранено, но върви с редица рестрикции и съществуването му на практика не води до свобода. Друг елемент за „демонополизиране“ на образованието е възможността на родителите и обществото да участват. В момента у нас държавата е оставила на свободната инициатива в образованието възможността да създаде още една услуга от същия тип, за която обаче да се плаща, вместо да остави възможност за алтернативна образователна услуга.

Ако правилно Ви разбирам, според Вас частната инициатива в образованието трябва да предлага алтернативни концепции и подходи, а не само алтернативен начин на финансиране?

Точно така. Частната инициатива е ценна, когато е източник на многообразие, и това трябва да се защити ясно в закона. Нейната роля се обезсмисля, когато е затворена в унифицирания държавен стандарт. В момента имаме един държавен монопол, който се разпростира и върху частната инициатива.

И оттук произтичат редица проблеми. Ако на ценностно ниво образованието работи срещу демократичните възгледи, се получава следното: тези семейства, които са ценностно ориентирани и могат да си го позволят, започват да се оттичат от държавния в частния сектор или даже извън системата. С това качеството в общинските институции обаче спада до минимума, който наблюдаваме в момента, и се получава феноменът на две паралелно развиващи се системи – „лошо държавно образование за бедните“ (както го наричат в Препоръките към образование, базирано на човешките права) и платено за останалите, които могат да си го позволят. Ние сме точно в тази ситуация и трябва спешно да осъзнаем, че ножицата започва да се отваря още в ранното детство и е много важно да я затворим пак там.

Държавата е длъжна, и то, подчертавам, не само на думи – тези задължения са юридически и произлизат от членството ни в ЕС, – да гарантира превенция на проблема, като направи така, че образованието да отговаря на няколко характеристики: да не е зависимо от покупателната способност на семействата; да достига реално до всички деца; да гарантира многообразие от услуги и подходи, а не да налага унифициран модел; да предвижда активна роля на самите родители и граждани в самоорганизирането и доставянето на такива услуги и да предоставя на практика правото на всяко дете да учи чрез собствената си активност, личен избор и глас още от най-ранна възраст.

Нашата държава обаче продължава да мисли в тоталитарните модели на контрол и управление на масите – тя се държи, все едно въобще не е чувала за тези неща и е достатъчно да направи институциите задължителни и безплатни, ако може и за тригодишните. Пропуска се фактът, че докато не повишим качеството и не се погрижим то да достига до всички, най-вече до рисковите групи; докато не гарантираме свобода и демократични ценности; докато продължаваме да потискаме правата на децата и техните родители, ние реално все повече вредим на собствените си деца, на обществото и на самата крайна цел, която е образованост на масите. Всичко това също е ясно описано в международните документи: ако висококачествените услуги доказано са от полза за децата, то услугите с ниско качество имат значително отрицателно влияние върху децата и обществото като цяло.

В този контекст какво и как измерихте в проекта „Визия за децата“?

Проектът продължи повече от половин година. Проведохме срещи в десет български града и поканихме представители на детски градини, училища, занимални, родители и деца. Поставихме им три ключови въпроса: 1) Какви са децата?; 2) Как учат?; 3) Каква среда им осигуряваме, за да се развиват?

Тъй като изследването е качествено, искахме да видим, ако в България нямаме свързване на ценностно ниво между демократичното общество и образованието, как може да бъде създадено то. От анализа на отговорите на тези три въпроса извлякохме изводи кои са ценностите, които хората най-често посочват. Идеята беше да формулираме заявка за какво искаме да работи образователната система. Ние изхождаме от убеждението, че в България това още не се е случило, и затова е много трудно да продължим нататък.

При представянето на доклада Ваша колежка каза, че в момента очакваме 18 години да отглеждаме деца, които да бъдат послушни, да изпълняват безропотно каквото им се казва, да не си позволяват самоинициатива, но на 19 да се превърнат в критично мислещи и активни граждани. Изследването дава ли някакво обяснение на този парадокс?

Парадоксът се дължи на незавършилия преход в умовете на хората. За да се осъществи преход от индустриална към демократична образователна система, ние първо трябва да преобърнем гледната си точка към детето като личност. Това е бавен процес, ние самите сме плод на тази система и сме естествено склонни да я възпроизвеждаме. Ако няма достатъчно силно осъзнаване, възпроизвеждането върви поколение след поколение, независимо какви са промените около нас.

Една от целите ни беше да провокираме това осъзнаване и да наблюдаваме какви конфликти ще се породят във всяка среща, защото най-вече там се ражда новото учене.

Например може да се обединим около идеята, че детето трябва да е активно, да може да играе свободно. Но се оказва, че в тази идея хората може да вложат много различни тълкувания. Едно от нещата, които видяхме, е, че за хората, работещи с деца, играта може да е едно, а за родителя – нещо коренно различно.

Още на първата среща една детска учителка стана и каза: „Ние това всичкото го правим, не разбирам защо не ви харесва. При нас децата играят, извеждаме ги на двора, какъв е проблемът?“ Един академичен преподавател беше провокиран и ѝ отговори: „Колега, това, което Вие наричате игра, изобщо не е игра.“ Имаше предвид, че когато на децата се каже, че вече може да станат от столчетата си и сега ще скачат или ще играят на зайчета или на жабки, у тях се възпитават съвсем различни разбирания от това, което би възпитала свободната игра.

Тоест ключовият въпрос е къде учителят смята, че е правото на игра – у детето, или у възрастния. В нашите детски градини то често се оказва у възрастния, с което ние отново упражняваме контрол над личността на детето и не насърчаваме и не развиваме умения за свободен избор, нито за вземане на решения и т.н. Все умения, които после ще очакваме да видим при PISA и у едни пораснали граждани.

Може ли от „Визия за децата“ да се извлекат идеи как да се излезе от този капан?

За съжаление, по-скоро не. За целта ще е нужна следваща стъпка. Няма точна рецепта „правим това, получаваме онова като резултат“, тъй като става дума за обществени процеси и влияят много фактори.

Но има няколко стъпки, които според нас може да се изпълняват едновременно, за да се насърчи едновременна промяна в системата и в обществото. Първата стъпка е да приведем нормативната регулация в съответствие с международните документи – без това не може. То означава не само да напишеш в закона, че се насърчава многообразието, както е при нас, а да създадеш среда, в която може да покълва. Ето тук ние се проваляме тотално, защото сега действащият финансов механизъм в образованието цели запазване на държавния образователен монопол. В случая с малките деца това е институционален монопол. Така е и при по-големите, но през първите седем години е особено проблемно за личността на детето.

Второто е да се разделим със задължителните педагогически ситуации, които се налагат на детето, независимо от неговата готовност и интереси. Нашата система не дава никаква възможност за индивидуален подход, въпреки че непрекъснато говорим за него.

Подготовката на учителите е друг аспект, на който следва да обърнем специално внимание. Трябва да научим учителите да работят през отношенчески процеси, което е изцяло нов подход. Например при малките деца, защото нашият фокус е върху тях, това означава учителят да може да види в играта, която детето само̀ организира с другите деца, какви процеси на учене протичат. Това е обръщане на образователната парадигма – не възрастният да „налива“ знания в детето, а на основата на убеждението ни, че детето има в себе си заложбите, да осигурим условия и среда, в които те да могат да се проявят и развиват.

Необходимо е възрастният да наблюдава, да документира, да познава добре теориите на ученето и развитието, за да може да ги „вижда“ в детето. Това обаче са все умения, които в българския университет не присъстват. Тук се работи в дидактичен стил и е важно да се чуят самите университети, доколко те са отворени за нещо различно.

Учителите, които в момента са в системата, как гледат на идеята за подобна промяна?

Изследването ни носи усещането, че сякаш след няколко години в системата самите учители се разделят със свободата си. Тук дори още не говорим за свободата на детето, а за свободата на учителя. Учителят в България се възприема като някой, на когото му казват какво да прави. Така че ако искаме промяна в системата, тя трябва да започне от еманципиране на самия учител – да се почувства свободен, да се почувства овластен в комуникацията си с децата и с родителите, за да може отношенията да започнат да водят до учене.

На прага на 2024 г. е крайно време да се разделим с идеята, че ученето е процес, в който възрастният казва на детето какво да прави, и то го изпълнява автоматизирано. Ученето е в самото дете и колкото по-малко го потискаш, толкова повече то самò иска да развива знанията и уменията си.

Живеем във време, в което да научиш това, което поискаш, в момент, в който го искаш, е по-лесно от когато и да било преди в човешката история. Въпреки това – или може би имено заради това – формалното образование преживява криза на идентичността. В рубриката „Разговори за образованието“ Надежда Цекулова и нейните събеседници търсят философията, смисъла и формите на онова, което наричаме „образование“ в третата декада на ХХI в.

[$] Data-type profiling for perf

2023-12-21 corbet

Post Syndicated from corbet original https://lwn.net/Articles/955709/

Tooling for profiling the effects of memory usage and layout has always
lagged behind that for profiling processor activity, so Namhyung Kim’s patch set for data-type profiling
in perf is a welcome addition. It provides aggregated breakdowns of
memory accesses by data type that can inform structure layout and access
pattern changes. Existing tools have either, like heaptrack, focused on
profiling allocations, or, like perf mem, on accounting memory
accesses only at the address level. This new work builds on the latter,
using DWARF debugging information to correlate memory operations with their
source-level types.

How smava makes loans transparent and affordable using Amazon Redshift Serverless

2023-12-21 Alex Naumov

Post Syndicated from Alex Naumov original https://aws.amazon.com/blogs/big-data/how-smava-makes-loans-transparent-and-affordable-using-amazon-redshift-serverless/

This is a guest post co-written by Alex Naumov, Principal Data Architect at smava.

smava GmbH is one of the leading financial services companies in Germany, making personal loans transparent, fair, and affordable for consumers. Based on digital processes, smava compares loan offers from more than 20 banks. In this way, borrowers can choose the deals that are most favorable to them in a fast, digitalized, and efficient way.

smava believes in and takes advantage of data-driven decisions in order to become the market leader. The Data Platform team is responsible for supporting data-driven decisions at smava by providing data products across all departments and branches of the company. The departments include teams from engineering to sales and marketing. Branches range by products, namely B2C loans, B2B loans, and formerly also B2C mortgages. The data products used inside the company include insights from user journeys, operational reports, and marketing campaign results, among others. The data platform serves on average 60 thousand queries per day. The data volume is in double-digit TBs with steady growth as business and data sources evolve.

smava’s Data Platform team faced the challenge to deliver data to stakeholders with different SLAs, while maintaining the flexibility to scale up and down while staying cost-efficient. It took up to 3 hours to generate daily reporting, which impacted business decision-making when re-calculations needed to happen during the day. To speed up the self-service analytics and foster innovation based on data, a solution was needed to provide ways to allow any team to create data products on their own in a decentralized manner. To create and manage the data products, smava uses Amazon Redshift, a cloud data warehouse.

In this post, we show how smava optimized their data platform by using Amazon Redshift Serverless and Amazon Redshift data sharing to overcome right-sizing challenges for unpredictable workloads and further improve price-performance. Through the optimizations, smava achieved up to 50% cost savings and up to three times faster report generation compared to the previous analytics infrastructure.

Overview of solution

As a data-driven company, smava relies on the AWS Cloud to power their analytics use cases. To bring their customers the best deals and user experience, smava follows the modern data architecture principles with a data lake as a scalable, durable data store and purpose-built data stores for analytical processing and data consumption.

smava ingests data from various external and internal data sources into a landing stage on the data lake based on Amazon Simple Storage Service (Amazon S3). To ingest the data, smava uses a set of popular third-party customer data platforms complemented by custom scripts.

After the data lands in Amazon S3, smava uses the AWS Glue Data Catalog and crawlers to automatically catalog the available data, capture the metadata, and provide an interface that allows querying all data assets.

Data analysts who require access to the raw assets on the data lake use Amazon Athena, a serverless, interactive analytics service for exploration with ad hoc queries. For the downstream consumption by all departments across the organization, smava’s Data Platform team prepares curated data products following the extract, load, and transform (ELT) pattern. smava uses Amazon Redshift as their cloud data warehouse to transform, store, and analyze data, and uses Amazon Redshift Spectrum to efficiently query and retrieve structured and semi-structured data from the data lake using SQL.

smava follows the data vault modeling methodology with the Raw Vault, Business Vault, and Data Mart stages to prepare the data products for end consumers. The Raw Vault describes objects loaded directly from the data sources and represents a copy of the landing stage in the data lake. The Business Vault is populated with data sourced from the Raw Vault and transformed according to the business rules. Finally, the data is aggregated into specific data products oriented to a specific business line. This is the Data Mart stage. The data products from the Business Vault and Data Mart stages are now available for consumers. smava decided to use Tableau for business intelligence, data visualization, and further analytics. The data transformations are managed with dbt to simplify the workflow governance and team collaboration.

The following diagram shows the high-level data platform architecture before the optimizations.

High-level Data Platform architecture before the optimizations

Evolution of the data platform requirements

smava started with a single Redshift cluster to host all three data stages. They chose provisioned cluster nodes of the RA3 type with Reserved Instances (RIs) for cost optimization. As data volumes grew 53% year over year, so did the complexity and requirements from various analytic workloads.

smava quickly addressed the growing data volumes by right-sizing the cluster and using Amazon Redshift Concurrency Scaling for peak workloads. Furthermore, smava wanted to give all teams the option to create their own data products in a self-service manner to increase the pace of innovation. To avoid any interference with the centrally managed data products, the decentralized product development environments needed to be strictly isolated. The same requirement was also applied for the isolation of different product stages curated by the Data Platform team.

Optimizing the architecture with data sharing and Redshift Serverless

To meet the evolved requirements, smava decided to separate the workload by splitting the single provisioned Redshift cluster into multiple data warehouses, with each warehouse serving a different stage. In addition, smava added new staging environments in the Business Vault to develop new data products without the risk of interfering with existing product pipelines. To avoid any interference with the centrally managed data products of the Data Platform team, smava introduced an additional Redshift cluster, isolating the decentralized workloads.

smava was looking for an out-of-the-box solution to achieve workload isolation without managing a complex data replication pipeline.

Right after the launch of Redshift data sharing capabilities in 2021, the Data Platform team recognized that this was the solution they had been looking for. smava adopted the data sharing feature to have the data from producer clusters available for read access on different consumer clusters, with each of those consumer clusters serving a different stage.

Redshift data sharing enables instant, granular, and fast data access across Redshift clusters without the need to copy data. It provides live access to data so that users always see the most up-to-date and consistent information as it’s updated in the data warehouse. With data sharing, you can securely share live data with Redshift clusters in the same or different AWS accounts and across Regions.

With Redshift data sharing, smava was able to optimize the data architecture by separating the data workloads to individual consumer clusters without having to replicate the data. The following diagram illustrates the high-level data platform architecture after splitting the single Redshift cluster into multiple clusters.

High-level Data Platform architecture after splitting the single Redshift cluster in multiple clusters

By providing a self-service data mart, smava increased data democratization by providing users with access to all aspects of the data. They also provided teams with a set of custom tools for data discovery, ad hoc analysis, prototyping, and operating the full lifecycle of mature data products.

After collecting operational data from the individual clusters, the Data Platform team identified further potential optimizations: the Raw Vault cluster was under steady load 24/7, but the Business Vault clusters were only updated nightly. To optimize for costs, smava used the pause and resume capabilities of Redshift provisioned clusters. These capabilities are useful for clusters that need to be available at specific times. While the cluster is paused, on-demand billing is suspended. Only the cluster’s storage incurs charges.

The pause and resume feature helped smava optimize for cost, but it required additional operational overhead to trigger the cluster operations. Additionally, the development clusters remained subject to idle times during working hours. These challenges were finally solved by adopting Redshift Serverless in 2022. The Data Platform team decided to move the Business Data Vault stage clusters to Redshift Serverless, which allows them to pay for the data warehouse only when in use, reliably and efficiently.

Redshift Serverless is ideal for cases when it’s difficult to predict compute needs such as variable workloads, periodic workloads with idle time, and steady-state workloads with spikes. Additionally, as usage demand evolves with new workloads and more concurrent users, Redshift Serverless automatically provisions the right compute resources, and the data warehouse scales seamlessly and automatically, without the need for manual intervention. Data sharing is supported in both directions between Redshift Serverless and provisioned Redshift clusters with RA3 nodes, so no changes to the smava architecture were needed. The following diagram shows the high-level architecture setup after the move to Redshift Serverless.

High-level Data Platform architecture after introducing Redshift Serverless for Business Vault clusters

smava combined the benefits of Redshift Serverless and dbt through a seamless CI/CD pipeline, adopting a trunk-based development methodology. Changes on the Git repository are automatically deployed to a test stage and validated using automated integration tests. This approach increased the efficiency of developers and decreased the average time to production from days to minutes.

smava adopted an architecture that utilizes both provisioned and serverless Redshift data warehouses, together with the data sharing capability to isolate the workloads. By choosing the right architectural patterns for their needs, smava was able to accomplish the following:

Simplify the data pipelines and reduce operational overhead
Reduce the feature release time from days to minutes
Increase price-performance by reducing idle times and right-sizing the workload
Achieve up to three times faster report generation (faster calculations and higher parallelization) at 50% of the original setup costs
Increase agility of all departments and support data-driven decision-making by democratizing access to data
Increase the speed of innovation by exposing self-service data capabilities for teams across all departments and strengthening the A/B test capabilities to cover the complete customer journey

Now, all departments at smava are using the available data products to make data-driven, accurate, and agile decisions.

Future vision

For the future, smava plans to continue to optimize the Data Platform based on operational metrics. They’re considering switching more provisioned clusters like the Self-Service Data Mart cluster to serverless. Additionally, smava is optimizing the ELT orchestration toolchain to increase the number of parallel data pipelines to be run. This will increase the utilization of provisioned Redshift resources and allow for cost reductions.

With the introduction of the decentralized, self-service for data product creation, smava made a step forward towards a data mesh architecture. In the future, the Data Platform team plans to further evaluate the needs of their service users and establish further data mesh principles like federated data governance.

Conclusion

In this post, we showed how smava optimized their data platform by isolating environments and workloads using Redshift Serverless and data sharing features. Those Redshift environments are well integrated with their infrastructure, flexible in scaling on demand, and highly available, and they require minimum administration efforts. Overall, smava has increased performance by three times while reducing the total platform costs by 50%. Additionally, they reduced operational overhead to a minimum while maintaining the existing SLAs for report generation times. Moreover, smava has strengthened the culture of innovation by providing self-service data product capabilities to speed up their time to market.

If you’re interested in learning more about Amazon Redshift capabilities, we recommend watching the most recent What’s new with Amazon Redshift session in the AWS Events channel to get an overview of the features recently added to the service. You can also explore the self-service, hands-on Amazon Redshift labs to experiment with key Amazon Redshift functionalities in a guided manner.

You can also dive deeper into Redshift Serverless use cases and data sharing use cases. Additionally, check out the data sharing best practices and discover how other customers optimized for cost and performance with Redshift data sharing to get inspired for your own workloads.

If you prefer books, check out Amazon Redshift: The Definitive Guide by O’Reilly, where the authors detail the capabilities of Amazon Redshift and provide you with insights on corresponding patterns and techniques.

About the Authors

Blog author: Alex Naumov Alex Naumov is a Principal Data Architect at smava GmbH, and leads the transformation projects at the Data department. Alex previously worked 10 years as a consultant and data/solution architect in a wide variety of domains, such as telecommunications, banking, energy, and finance, using various tech stacks, and in many different countries. He has a great passion for data and transforming organizations to become data-driven and the best in what they do.

Blog author: Lingli Zheng Lingli Zheng works as a Business Development Manager in the AWS worldwide specialist organization, supporting customers in the DACH region to get the best value out of Amazon analytics services. With over 12 years of experience in energy, automation, and the software industry with a focus on data analytics, AI, and ML, she is dedicated to helping customers achieve tangible business results through digital transformation.

Blog author: Alexander Spivak Alexander Spivak is a Senior Startup Solutions Architect at AWS, focusing on B2B ISV customers across EMEA North. Prior to AWS, Alexander worked as a consultant in financial services engagements, including various roles in software development and architecture. He is passionate about data analytics, serverless architectures, and creating efficient organizations.

This post was reviewed for technical accuracy by David Greenshtein, Senior Analytics Solutions Architect.

Accelerate analytics on Amazon OpenSearch Service with AWS Glue through its native connector

2023-12-21 Basheer Sheriff

Post Syndicated from Basheer Sheriff original https://aws.amazon.com/blogs/big-data/accelerate-analytics-on-amazon-opensearch-service-with-aws-glue-through-its-native-connector/

As the volume and complexity of analytics workloads continue to grow, customers are looking for more efficient and cost-effective ways to ingest and analyse data. Data is stored from online systems such as the databases, CRMs, and marketing systems to data stores such as data lakes on Amazon Simple Storage Service (Amazon S3), data warehouses in Amazon Redshift, and purpose-built stores such as Amazon OpenSearch Service, Amazon Neptune, and Amazon Timestream.

OpenSearch Service is used for multiple purposes, such as observability, search analytics, consolidation, cost savings, compliance, and integration. OpenSearch Service also has vector database capabilities that let you implement semantic search and Retrieval Augmented Generation (RAG) with large language models (LLMs) to build recommendation and media search engines. Previously, to integrate with OpenSearch Service, you could use open source clients for specific programming languages such as Java, Python, or JavaScript or use REST APIs provided by OpenSearch Service.

Movement of data across data lakes, data warehouses, and purpose-built stores is achieved by extract, transform, and load (ETL) processes using data integration services such as AWS Glue. AWS Glue is a serverless data integration service that makes it straightforward to discover, prepare, and combine data for analytics, machine learning (ML), and application development. AWS Glue provides both visual and code-based interfaces to make data integration effortless. Using a native AWS Glue connector increases agility, simplifies data movement, and improves data quality.

In this post, we explore the AWS Glue native connector to OpenSearch Service and discover how it eliminates the need to build and maintain custom code or third-party tools to integrate with OpenSearch Service. This accelerates analytics pipelines and search use cases, providing instant access to your data in OpenSearch Service. You can now use data stored in OpenSearch Service indexes as a source or target within the AWS Glue Studio no-code, drag-and-drop visual interface or directly in an AWS Glue ETL job script. When combined with AWS Glue ETL capabilities, this new connector simplifies the creation of ETL pipelines, enabling ETL developers to save time building and maintaining data pipelines.

Solution overview

The new native OpenSearch Service connector is a powerful tool that can help organizations unlock the full potential of their data. It enables you to efficiently read and write data from OpenSearch Service without needing to install or manage OpenSearch Service connector libraries.

In this post, we demonstrate exporting the New York City Taxi and Limousine Commission (TLC) Trip Record Data dataset into OpenSearch Service using the AWS Glue native connector. The following diagram illustrates the solution architecture.

By the end of this post, your visual ETL job will resemble the following screenshot.

Prerequisites

To follow along with this post, you need a running OpenSearch Service domain. For setup instructions, refer to Getting started with Amazon OpenSearch Service. Ensure it is public, for simplicity, and note the primary user and password for later use.

Note that as of this writing, the AWS Glue OpenSearch Service connector doesn’t support Amazon OpenSearch Serverless, so you need to set up a provisioned domain.

Create an S3 bucket

We use an AWS CloudFormation template to create an S3 bucket to store the sample data. Complete the following steps:

Choose Launch Stack.
On the Specify stack details page, enter a name for the stack.
Choose Next.
On the Configure stack options page, choose Next.
On the Review page, select I acknowledge that AWS CloudFormation might create IAM resources.
Choose Submit.

The stack takes about 2 minutes to deploy.

Create an index in the OpenSearch Service domain

To create an index in the OpenSearch service domain, complete the following steps:

On the OpenSearch Service console, choose Domains in the navigation pane.
Open the domain you created as a prerequisite.
Choose the link under OpenSearch Dashboards URL.
On the navigation menu, choose Dev Tools.
Enter the following code to create the index:

PUT /yellow-taxi-index
{
  "mappings": {
    "properties": {
      "VendorID": {
        "type": "integer"
      },
      "tpep_pickup_datetime": {
        "type": "date",
        "format": "epoch_millis"
      },
      "tpep_dropoff_datetime": {
        "type": "date",
        "format": "epoch_millis"
      },
      "passenger_count": {
        "type": "integer"
      },
      "trip_distance": {
        "type": "float"
      },
      "RatecodeID": {
        "type": "integer"
      },
      "store_and_fwd_flag": {
        "type": "keyword"
      },
      "PULocationID": {
        "type": "integer"
      },
      "DOLocationID": {
        "type": "integer"
      },
      "payment_type": {
        "type": "integer"
      },
      "fare_amount": {
        "type": "float"
      },
      "extra": {
        "type": "float"
      },
      "mta_tax": {
        "type": "float"
      },
      "tip_amount": {
        "type": "float"
      },
      "tolls_amount": {
        "type": "float"
      },
      "improvement_surcharge": {
        "type": "float"
      },
      "total_amount": {
        "type": "float"
      },
      "congestion_surcharge": {
        "type": "float"
      },
      "airport_fee": {
        "type": "integer"
      }
    }
  }
}

Create a secret for OpenSearch Service credentials

In this post, we use basic authentication and store our authentication credentials securely using AWS Secrets Manager. Complete the following steps to create a Secrets Manager secret:

On the Secrets Manager console, choose Secrets in the navigation pane.
Choose Store a new secret.
For Secret type, select Other type of secret.
For Key/value pairs, enter the user name opensearch.net.http.auth.user and the password opensearch.net.http.auth.pass.
Choose Next.
Complete the remaining steps to create your secret.

Create an IAM role for the AWS Glue job

Complete the following steps to configure an AWS Identity and Access Management (IAM) role for the AWS Glue job:

On the IAM console, create a new role.
Attach the AWS managed policy GlueServiceRole.
Attach the following policy to the role. Replace each ARN with the corresponding ARN of the OpenSearch Service domain, Secrets Manager secret, and S3 bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "OpenSearchPolicy",
            "Effect": "Allow",
            "Action": [
                "es:ESHttpPost",
                "es:ESHttpPut"
            ],
            "Resource": [
                "arn:aws:es:<region>:<aws-account-id>:domain/<amazon-opensearch-domain-name>"
            ]
        },
        {
            "Sid": "GetDescribeSecret",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetResourcePolicy",
                "secretsmanager:GetSecretValue",
                "secretsmanager:DescribeSecret",
                "secretsmanager:ListSecretVersionIds"
            ],
            "Resource": "arn:aws:secretsmanager:<region>:<aws-account-id>:secret:<secret-name>"
        },
        {
            "Sid": "S3Policy",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation",
                "s3:ListBucket",
                "s3:GetBucketAcl",
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::<bucket-name>",
                "arn:aws:s3:::<bucket-name>/*"
            ]
        }
    ]
}

Create an AWS Glue connection

Before you can use the OpenSearch Service connector, you need to create an AWS Glue connection for connecting to OpenSearch Service. Complete the following steps:

On the AWS Glue console, choose Connections in the navigation pane.
Choose Create connection.
For Name, enter opensearch-connection.
For Connection type, choose Amazon OpenSearch.
For Domain endpoint, enter the domain endpoint of OpenSearch Service.
For Port, enter HTTPS port 443.
For Resource, enter yellow-taxi-index.

In this context, resource means the index of OpenSearch Service where the data is read from or written to.

Select Wan only enabled.
For AWS Secret, choose the secret you created earlier.
Optionally, if you’re connecting to an OpenSearch Service domain in a VPC, specify a VPC, subnet, and security group to run AWS Glue jobs inside the VPC. For security groups, a self-referencing inbound rule is required. For more information, see Setting up networking for development for AWS Glue.
Choose Create connection.

Create an ETL job using AWS Glue Studio

Complete the following steps to create your AWS Glue ETL job:

On the AWS Glue console, choose Visual ETL in the navigation pane.
Choose Create job and Visual ETL.
On the AWS Glue Studio console, change the job name to opensearch-etl.
Choose Amazon S3 for the data source and Amazon OpenSearch for the data target.

Between the source and target, you can optionally insert transform nodes. In this solution, we create a job that has only source and target nodes for simplicity.

In the Data source properties section, specify the S3 bucket where the sample data is located, and choose Parquet as the data format.
In the Data sink properties section, specify the connection you created in the previous section (opensearch-connection).
Choose the Job details tab, and in the Basic properties section, specify the IAM role you created earlier.
Choose Save to save your job, and choose Run to run the job.
Navigate to the Runs tab to check the status of the job. When it is successful, the run status should be Succeeded.
After the job runs successfully, navigate to OpenSearch Dashboards, and log in to the dashboard.
Choose Dashboards Management on the navigation menu.
Choose Index patterns, and choose Create index pattern.
Enter yellow-taxi-index for Index pattern name.
Choose tpep_pickup_datetime for Time.
Choose Create index pattern. This index pattern will be used to visualize the index.
Choose Discover on the navigation menu, and choose yellow-taxi-index.

You have now created an index in OpenSearch Service and loaded data into it from Amazon S3 in just a few steps using the AWS Glue OpenSearch Service native connector.

Clean up

To avoid incurring charges, clean up the resources in your AWS account by completing the following steps:

On the AWS Glue console, choose ETL jobs in the navigation pane.
From the list of jobs, select the job opensearch-etl, and on the Actions menu, choose Delete.
On the AWS Glue console, choose Data connections in the navigation pane.
Select opensearch-connection from the list of connectors, and on the Actions menu, choose Delete.
On the IAM console, choose Roles in the navigation page.
Select the role you created for the AWS Glue job and delete it.
On the CloudFormation console, choose Stacks in the navigation pane.
Select the stack you created for the S3 bucket and sample data and delete it.
On the Secrets Manager console, choose Secrets in the navigation pane.
Select the secret you created, and on the Actions menu, choose Delete.
Reduce the waiting period to 7 days and schedule the deletion.

Conclusion

The integration of AWS Glue with OpenSearch Service adds the powerful ability to perform data transformation when integrating with OpenSearch Service for analytics use cases. This enables organizations to streamline data integration and analytics with OpenSearch Service. The serverless nature of AWS Glue means no infrastructure management, and you pay only for the resources consumed while your jobs are running. As organizations increasingly rely on data for decision-making, this native Spark connector provides an efficient, cost-effective, and agile solution to swiftly meet data analytics needs.

About the authors

Basheer Sheriff is a Senior Solutions Architect at AWS. He loves to help customers solve interesting problems leveraging new technology. He is based in Melbourne, Australia, and likes to play sports such as football and cricket.

Shunsuke Goto is a Prototyping Engineer working at AWS. He works closely with customers to build their prototypes and also helps customers build analytics systems.

WOW STH YouTube Now at 400K Subscribers

2023-12-21 Patrick Kennedy

Post Syndicated from Patrick Kennedy original https://www.servethehome.com/wow-sth-youtube-now-at-400k-subscribers/

We hit another milestone as the STH YouTube channel crossed the 400K subscriber mark. A little bit of studio behind-the-scenes is in order

The post WOW STH YouTube Now at 400K Subscribers appeared first on ServeTheHome.

A return to US net neutrality rules?

2023-12-21 Zaid Zaid http://blog.cloudflare.com/author/zaid-zaid/

Post Syndicated from Zaid Zaid http://blog.cloudflare.com/author/zaid-zaid/ original https://blog.cloudflare.com/a-return-to-us-net-neutrality-rules

For nearly 15 years, the Federal Communications Commission (FCC) in the United States has gone back and forth on open Internet rules – promulgating and then repealing, with some court battles thrown in for good measure. Last week was the deadline for Internet stakeholders to submit comments to the FCC about their recently proposed net neutrality rules for Internet Service Providers (ISPs), which would introduce considerable protections for consumers and codify the responsibility held by ISPs.

For anyone who has worked to help to build a better Internet, as Cloudflare has for the past 13 years, the reemergence of net neutrality is déjà vu all over again. Cloudflare has long supported the open Internet principles that are behind net neutrality, and we still do today. That’s why we filed comments with the FCC expressing our support for these principles, and concurring with many of the technical definitions and proposals that largely would reinstitute the net neutrality rules that were previously in place.

But let’s back up and talk about net neutrality. Net neutrality is the principle that ISPs should not discriminate against the traffic that flows through them. Specifically, when these rules were adopted by the FCC in 2015, there were three bright line rules: (1) that ISPs cannot block subscribers from reaching legal content, applications or services, (2) that ISPs cannot throttle subscribers’ access to content, putting some content in a “slow lane”, and (3) that ISPs can’t engage in “paid prioritization” which means charging websites and services for special access to their subscribers.

Net neutrality has a long history. In 2010, the FCC passed the first set of open Internet rules which were: (1) no blocking; (2) no unreasonable discrimination; and (3) transparency rules. In 2014, after a lawsuit from Verizon, the D.C. Circuit Court invalidated some of the 2010 rules, saying that if the FCC wanted to have these rules, it needed to treat ISPs as “common carriers.” (A common carrier is an entity that offers its services to the general public and will provide its services to anyone willing to pay the fee.) In 2015, the FCC did exactly that: it reclassified ISPs as common carriers, and instituted rules which we now know as net neutrality protections. In 2017, the FCC reversed course and repealed the rules. Now, the FCC again wants to reinstate them. It’s a dizzying chain of events.

And all the while, the Internet has carried on. For most Americans, net neutrality principles are reasonably uncontroversial — surveys show that more than 80% of Americans support them. And for all the lawsuits and regulatory ping-pong, in our view ISPs have largely followed these principles. The Internet has worked and is working.

What is broadband Internet?

In the same way that the delivery of Internet service hasn’t changed much, the underlying rationale for the net neutrality rules hasn’t changed. Broadband Internet is more critical than ever for our day-to-day lives, with more of our healthcare, work, education and entertainment happening over the Internet. ISPs still now, as then, are likely to have a monopoly on how subscribers reach the Internet – there’s only one path in and out of most people’s homes over the Internet, and even where consumers have a choice, they often face onerous switching costs. The FCC is ensuring there are rules for that road by defining the requirements that ISPs are obliged to fulfill.

In late September, the FCC released a public draft of its Notice of Proposed Rulemaking (NPRM) on net neutrality and gave the public about 3 months to review it and submit comments to the agency. The current NPRM asks what has changed about the Internet since 2015, whether the original principles are still the right ones, what should be the definition of an ISP, and many other things. The net neutrality principles proposed by the FCC will be familiar to net neutrality advocates, who have campaigned for similar ideas for years. As always, at Cloudflare we want consumers to have full access to legal content and services on the Internet.

What has changed – or at least become more complicated – is all of the various services that consumers and businesses use on the Internet. At Cloudflare, we know this well because we offer many of these services. We offer a content delivery network that protects and accelerates website delivery to consumers. We have a developer platform that developers use to deploy their code all across the world. And we have a platform that offers large businesses the ability to securely connect their offices and employees. Of course, we’re not alone. The ability of the Internet to foster permissionless innovation is unmatched.

For all the innovation (and some quite complicated services) flowing across the Internet, the ISPs that would be subject to these rules are, in our view, easy to define. In FCC terminology, an ISP is a provider of Broadband Internet Access Services (“BIAS”). As the FCC proposes to define it, a BIAS service is a mass-market Internet service which consumers purchase with the expectation they can reach the whole Internet. One of the main things we said to the FCC in our comments boils down to “you know a BIAS service when you see one.” Once we have a simple definition of BIAS service, we’ve also established that everything else is not BIAS.

As we said in our comments to the FCC:

[The FCC’s] historic definition identifies two primary characteristics of BIAS: (1) “a mass-market retail service” that (2) “provides the capability to transmit data to and receive data from all or substantially all internet endpoints.” The proposed definition of BIAS places the focus where it belongs: the ability of Internet end users to reach and interact with all Internet destinations without interference from their BIAS provider.

Interconnection and traffic exchange between networks

The interconnection section of the FCC’s proposed rules is also worthy of attention. Interconnection is how networks send data to one another on the Internet. Cloudflare is one of the best connected networks in the world (we’re directly connected to over 13,000 other networks, and are present at nearly as many Internet exchanges as any other network) so we know this topic well.

To give a very brief overview of the way interconnection works, assume a user on the network of ISP A requests cloudflare.com in their browser. That request goes out from the subscriber’s home through the ISP’s network. At some point it will reach an interconnection point, which is a data center where lots of networks connect together. If the ISP network and the content network (in this example it’s Cloudflare, since they are requesting cloudflare.com) directly connect (called “peering”) then the request will pass to Cloudflare and Cloudflare will respond, delivering back the HTML, JavaScript, images, and everything else that makes up a website.

Maybe the ISP and Cloudflare aren’t peered directly, but if they are both members of the same Internet Exchange, traffic could be exchanged there. Or, if neither of those are an option, the ISP and Cloudflare might exchange data through an IP transit provider, a 3rd party network that gets paid to deliver traffic on their behalf.

Interconnection is relevant to the FCC’s net neutrality proceeding because an ISP makes a representation to their subscriber that the subscriber can access the whole Internet, and the ISP needs to make interconnection arrangements to make good on that representation.

What the FCC is proposing is that ISPs would be required to make interconnection arrangements as part of their responsibility to deliver the whole Internet to their subscribers without blocking, throttling, or paid prioritization.

Beyond the representation that ISP’s make to their subscribers, the FCC is not proposing to directly impose rules on interconnection. Instead, the FCC is proposing to adopt a “watch, learn, and act as required” case-by-case approach to interconnection challenges. Interconnection disputes between ISPs and content and service providers have happened. Famously, in 2014, Comcast and Netflix didn’t have enough interconnection capacity and thus Comcast subscribers trying to watch Netflix were subject to lots of buffering and a generally bad experience. But they worked it out between themselves. Similar disputes in the United States have been rare since.

Both from the Comcast-Netflix instance, and other issues we see internationally, we know interconnection disputes can arise, and they can affect users. For example, we’re currently monitoring interconnection in Germany, where users on one of the largest networks have had trouble reaching normal websites like GitHub, or just browsing the Internet. It’s likely those issues are caused by insufficient interconnection capacity.

While we don’t have this type of interconnection issue in the United States currently, under the proposed rules the FCC would be set up as an arbiter of last resort for disputes in the United States. With this approach, hopefully we would be able to avoid the type of issues we’re seeing in Germany. And if ever consumers’ Internet experience was being harmed by the interconnection policy of any network, the FCC could adjudicate the matter.

It has been eight years since net neutrality rules were passed in the United States, and six years since they were repealed. During that time the Internet has kept growing. If the FCC does reinstate net neutrality rules, we’re hopeful they will be common sense rules of the road for ISPs, making official the already-widely-followed principles of a free and open Internet.

Ultimate Battery Bank Comparison – AMAZING Results!

2023-12-21 The Hook Up

Post Syndicated from The Hook Up original https://www.youtube.com/watch?v=gMIZixr7UFU

Announcing `async fn` and return-position `impl Trait` in traits (Rust Blog)

2023-12-21 corbet

Post Syndicated from corbet original https://lwn.net/Articles/955925/

The Rust Blog announces
the stabilization of a couple of trait features aimed at improving support
for async code:

Ever since the stabilization of RFC #1522 in Rust 1.26, Rust has
allowed users to write impl Trait as the return type of
functions (often called “RPIT”). This means that the function
returns “some type that implements Trait“. This is
commonly used to return closures, iterators, and other types that
are complex or impossible to write explicitly. […]

Starting in Rust 1.75, you can use return-position impl Trait in trait (RPITIT) definitions and in trait impls. For
example, you could use this to write a trait method that returns an
iterator: […]

So what does all of this have to do with async functions? Well,
async functions are “just sugar” for functions that return
-> impl Future. Since these are now permitted in
traits, we also permit you to write traits that use async fn.

The Bhola Cyclone: The Storm that Almost Started a Nuclear War

2023-12-21 Geographics

Post Syndicated from Geographics original https://www.youtube.com/watch?v=HXw5M2yDOas

LATEST Sonoff Zigbee Motion Sensor – SNZB-03P

2023-12-21 BeardedTinker

Post Syndicated from BeardedTinker original https://www.youtube.com/watch?v=9oScFcsV340

Три отговора за Конституцията

2023-12-21 Bozho

Post Syndicated from Bozho original https://blog.bozho.net/blog/4187

Три въпроса/критики, които виждам най-често във връзка с изменения в Конституцията.

1. Защо парламентарната квота в прокурорския съвет е по-голяма от професионалната – така не се ли поставя прокуратурата под политически контрол?

Проблемът с прокуратурата към момента е, че главният прокурор (който е началник на всички прокурори и може да осъществява надзор за законност по делата) или не носеше отговорност пред никого („над мен е само Бог“), или получаваше задкулисни указания от различни центрове на власт. Съответно отговорност за действията на прокуратурата не се носеха – „те си го избират прокурорите“.

С измененията, освен че му премахваме правата за намеса, слагаме превес на обществената квота. Това, от една страна, предотвратява капсулирането (прокурори, които зависят кариерно от главния, го поддържат в съвета), а от друга дава отговорност и отчетност. Трябва да отбележим и че обществената квота не е партийна и това ще е водещ принцип при нейното излъчване.

С друго изменение подчертаваме, че водещата роля в съдебната власт е на съда. Прокуратурата и следствието са страна по делата, а съдът е арбитър. Именно независимостта на съда е в центъра на съдебната власт, поради което квотите там са наобратно – 8 съдии избрани от съдии (от общо 15 члена, като само 5 се избират от парламента). Водещото при съда е, че трябва да е независим. Водещото при прокуратурата, която осъществява наказателната политика на държавата, е да бъде отчетна. Именно за това са такива квотите в двата съвета.

2. Защо двойно гражданство? Така не се ли отваря вратата за чуждо влияние?

В рамките на дебата бяха изнесени данни от г-жа Цонева, че България е последната държава и в ЕС, и в Съвета на Европа, която позволява двойно гражданство по принцип, но го забранява за изборни длъжности. С премахването на това ограничение правим по-лесно за българската диаспора (стотиците хиляди емигранти за последните 30 години) да се включи в политическия без бюрократични пречки и усложнения.

Изискването да нямаш двойно гражданство, за да си депутат/министър, не е пречка за чужди агенти на влияние да заемат позиции, както някои се надяват. Пример е Георги Димитров, който преди да се върне и да превърне България в съветска колония, се отказва от съветското си гражданство. Борбата с чуждото влияние е работа на службите, а не на формално конституционно ограничение.

Има няколко вида българско гражданство – по рождение (ако си се родил с българско гражданство или си дете на български гражданин), по произход (от историческите територии, на които е имало българско население) и по натурализация (ако сиsigna живял поне 5 години в България и отговаряш на допълнителни изисквания по закон). Няма хипотеза, в която някой чужденец получава българско гражданство без да е чувал за България, така че тези притеснения са неоснователни.

3. Защо с обикновено Народно събрание приемате изменения, за които се изисква Велико Народно Събрание, съгласно решение на Конституционния съд?

Решение 3 от 2003 г. на Конституционния съд, по искане на тогавашния главен прокурор (който публично се е хвалел с влиянието си в Конституционния съд във връзка с това решение), определя как се тълкува разпоредбата на чл. 158, т. 3 от Констиуцията, а именно, какво значи „формата на държавно устройство и на държавно управление“. Тезата, че обикновено Народно събрание не може да разделя Висшия съдебен съвет на два съвета и да премахва правомощия на главния прокурор, се базира на това решение.

Има обаче и друго решение – номер 8 от 2005 г., което казва, че може да има повече от едно тълкувателно решение по една норма, и че „Промени в Конституцията, концентрирани в рамките на съдебната власт, [..] които са насочени към преструктуриране, оптимизиране от съдържателна гледна точка и прецизиране на отделни функции на нейни органи, поставяне на акценти или уточняване на техни правомощия или наименования, както и взаимодействието им с институции на другите власти, не представляват промяна във формата на държавно управление“

И двете решения са тълкувателни – по искане на главния прокурор и по искане на ВКС. Те дават рамка, но както посочва Решение 8, трябва да се извърши преценка на конкретни изменения, за да се прецени нюанса и баланса. Иначе казано, не може да се твърди, че Решение 3 прави настоящите изменения противоконституционни.

Всички тези аргументи са изказвани в комисия и в пленарна зала. Аз лично смятам, че измененията в Конституцията са добри, водят до по-независим съд и по-отчетна прокуратура. Критиките към проекта бяха отчетени с измененията между първо и второ четене, като бяха обсъждани с професионалните общности в двата месеца между двете първи гласувания. Подхождаме към Конституцията с огромно внимание и с разбиране за отговорността, която носим. Ако някой е прочел някоя критична статия или е чул изказване на някой от опозицията или е извадил едно-две критични изречения от три страници становище на някоя институция, призовавам към по-задълбочен прочит. И на контекста, и на дебатите в комисия и в зала.

Материалът Три отговора за Конституцията е публикуван за пръв път на БЛОГодаря.

How to implement client certificate revocation list checks at scale with API Gateway

2023-12-21 Arthur Mnev

Post Syndicated from Arthur Mnev original https://aws.amazon.com/blogs/security/how-to-implement-client-certificate-revocation-list-checks-at-scale-with-api-gateway/

ityAs you design your Amazon API Gateway applications to rely on mutual certificate authentication (mTLS), you need to consider how your application will verify the revocation status of a client certificate. In your design, you should account for the performance and availability of your verification mechanism to make sure that your application endpoints perform reliably.

In this blog post, I demonstrate an architecture that will help you on your journey to implement custom revocation checks against your certificate revocation list (CRL) for API Gateway. You will also learn advanced Amazon Simple Storage Service (Amazon S3) and AWS Lambda techniques to achieve higher performance and scalability.

Choosing the right certificate verification method

One of your first considerations is whether to use a CRL or the Online Certificate Status Protocol (OCSP), if your certificate authority (CA) offers this option. For an in-depth analysis of these two options, see my earlier blog post, Choosing the right certificate revocation method in ACM Private CA. In that post, I demonstrated that OCSP is a good choice when your application can tolerate high latency or a failure for certificate verification due to TLS service-to-OCSP connectivity. When you rely on mutual TLS authentication in a high-rate transactional environment, increased latency or OCSP reachability failures may affect your application. We strongly recommend that you validate the revocation status of your mutual TLS certificates. Verifying your client certificate status against the CRL is the correct approach for certificate verification if you require reliability and lower, predictable latency. A potential exception to this approach is the use case of AWS Certificate Manager Private Certificate Authority (AWS Private CA) with an OCSP responder hosted on AWS CloudFront.

With an AWS Private CA OCSP responder hosted on CloudFront, you can reduce the risks of network and latency challenges by relying on communication between AWS native services. While this post focuses on the solution that targets CRLs originating from any CA, if you use AWS Private CA with an OCSP responder, you should consider generating an OCSP request in your Lambda authorizer.

Mutual authentication with API Gateway

API Gateway mutual TLS authentication (mTLS) requires you to define a root of trust that will contain your certificate authority public key. During the mutual TLS authentication process, API Gateway performs the undifferentiated heavy lifting by offloading the certificate authentication and negotiation process. During the authentication process, API Gateway validates that your certificate is trusted, has valid dates, and uses a supported algorithm. Additionally, you can refer to the API Gateway documentation and related blog post for details about the mutual TLS authentication process on API Gateway.

Implementing mTLS certificate verification for API Gateway

In the remainder of this blog post, I’ll describe the architecture for a scalable implementation of a client certificate verification mechanism against a CRL on your API Gateway.

The certificate CRL verification process presented here relies on a custom Lambda authorizer that validates the certificate revocation status against the CRL. The Lambda authorizer caches CRL data to optimize the query time for subsequent requests and allows you to define custom business logic that could go beyond CRL verification. For example, you could include other, just-in-time authorization decisions as a part of your evaluation logic.

Implementation mechanisms

This section describes the implementation mechanisms that help you create a high-performing extension to the API Gateway mutual TLS authentication process.

Data repository for your certificate revocation list

API Gateway mutual TLS configuration uses Amazon S3 as a repository for your root of trust. The design for this sample implementation extends the use of S3 buckets to store your CRL and the public key for the certificate authority that signed the CRL.

We strongly recommend that you maintain an updated CRL and verify its signature before data processing. This process is automatic if you use AWS Private CA, because AWS Private CA will update your CRL automatically on revocation. AWS Private CA also allows you to retrieve the CA’s public key by using an API call.

Certificate validation

My sample implementation architecture uses the API Gateway Lambda authorizer to validate the serial number of the client certificate used in the mutual TLS authentication session against the list of serial numbers present in the CRL you publish to the S3 bucket. In the process, the API Gateway custom authorizer will read the client certificate serial number, read and validate the CRL’s digital signature, search for the client’s certificate serial number within the CRL, and return the authorization policy based on the findings.

Optimizing for performance

The mechanisms that enable a predictable, low-latency performance are CRL preprocessing and caching. Your CRL is an ASN.1 data structure that requires a relatively high computing time for processing. Preprocessing your CRL into a simple-to-parse data structure reduces the computational cost you would otherwise incur for every validation; caching the CRL will help you reduce the validation latency and improve predictability further.

Performance optimizations

The process of parsing and validating CRLs is computationally expensive. In the case of large CRL files, parsing the CRL in the Lambda authorizer on every request can result in high latency and timeouts. To improve latency and reduce compute costs, this solution optimizes for performance by preprocessing the CRL and implementing function-level caching.

Preprocessing and generation of a cached CRL file

The first optimization happens when S3 receives a new CRL object. As shown in Figure 1, the S3 PutObject event invokes a preprocessing Lambda that validates the signature of your uploaded CRL and decodes its ASN.1 format. The output of the preprocessing Lambda function is the list of the revoked certificate serial numbers from the CRL, in a data structure that is simpler to read by your programming language of choice, and that won’t require extensive parsing by your Lambda authorizer. The asynchronous approach mitigates the impact of CRL processing on your API Gateway workload.

Figure 1: Sample implementation flow of the pre-processing component

Client certificate lookup in a CRL

The optimization happens as part of your Lambda authorizer that retrieves the preprocessed CRL data generated from the first step and searches through the data structure for your client certificate serial number. If the Lambda authorizer finds your client’s certificate serial number in the CRL, the authorization request fails, and the Lambda authorizer generates a “Deny” policy. Searching through a read-optimized data structure prepared by your preprocessing step is the second optimization that reduces the lookup time and the compute requirements.

Function-level caching

Because of the preprocessing, the Lambda authorizer code no longer needs to perform the expensive operation of decoding the ASN.1 data structures of the original CRL; however, network transfer latency will remain and may impact your application.

To improve performance, and as a third optimization, the Lambda service retains the runtime environment for a recently-run function for a non-deterministic period of time. If the function is invoked again during this time period, the Lambda function doesn’t have to initialize and can start running immediately. This is called a warm start. Function-level caching takes advantage of this warm start to hold the CRL data structure in memory persistently between function invocations so the Lambda function doesn’t have to download the preprocessed CRL data structure from S3 on every request.

The duration of the Lambda container’s warm state depends on multiple factors, such as usage patterns and parallel requests processed by your function. If, in your case, API use is infrequent or its usage pattern is spiky, pre-provisioned concurrency is another technique that can further reduce your Lambda startup times and the duration of your warm cache. Although provisioned concurrency does have additional costs, I recommend you evaluate its benefits for your specific environment. You can also check out the blog dedicated to this topic, Scheduling AWS Lambda Provisioned Concurrency for recurring peak usage.

To validate that the Lambda authorizer has the latest copy of the CRL data structure, the S3 ETag value is used to determine if the object has changed. The preprocessed CRL object’s ETag value is stored as a Lambda global variable, so its value is retained between invocations in the same runtime environment. When API Gateway invokes the Lambda authorizer, the function checks for existing global preprocessed CRL data structure and ETag variables. The process will only retrieve a read-optimized CRL when the ETag is absent, or its value differs from the ETag of the preprocessed CRL object in S3.

Figure 2 demonstrates this process flow.

Figure 2: Sample implementation flow for the Lambda authorizer component

In summary, you will have a Lambda container with a persistent in-memory lookup data structure for your CRL by doing the following:

Asynchronously start your preprocessing workflow by using the S3 PutObject event so you can generate and store your preprocessed CRL data structure in a separate S3 object.
Read the preprocessed CRL from S3 and its ETag value and store both values in global variables.
Compare the value of the ETag stored in your global variables to the current ETag value of the preprocessed CRL S3 object, to reduce unnecessary downloads if the current ETag value of your S3 object is the same as the previous value.
We recommend that you avoid using built-in API Gateway Lambda authorizer result caching, because the status of your certificate might change, and your authorization decision would rest on out-of-date verification results.
Consider setting a reserved concurrency for your CRL verification function so that API Gateway can invoke your function even if the overall capacity for your account in your AWS Region is exhausted.

The sample implementation flow diagram in Figure 3 demonstrates the overall architecture of the solution.

Figure 3: Sample implementation flow for the overall CRL verification architecture

The workflow for the solution overall is as follows:

An administrator publishes a CRL and its signing CA’s certificate to their non-public S3 bucket, which is accessible by the Lambda authorizer and preprocessor roles.
An S3 event invokes the Lambda preprocessor to run upon CRL upload. The function retrieves the CRL from S3, validates its signature against the issuing certificate, and parses the CRL.
The preprocessor Lambda stores the results in an S3 bucket with a name in the form <crlname>.cache.json.
A TLS client requests an mTLS connection and supplies its certificate.
API Gateway completes mTLS negotiation and invokes the Lambda authorizer.
The Lambda authorizer function parses the client’s mTLS certificate, retrieves the cached CRL object, and searches the object for the serial number of the client’s certificate.
The authorizer function returns a deny policy if the certificate is revoked or in error.
API Gateway, if authorized, proceeds with the integrated function or denies the client’s request.

Conclusion

In this post, I presented a design for validating your API Gateway mutual TLS client certificates against a CRL, with support for extra-large certificate revocation files. This approach will help you align with the best security practices for validating client certificates and use advanced S3 access and Lambda caching techniques to minimize time and latency for validation.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on the AWS Security, Identity, and Compliance re:Post or contact AWS Support.

Starlink Gen 3 Dish – Early Access Review

2023-12-21 digiblurDIY

Post Syndicated from digiblurDIY original https://www.youtube.com/watch?v=oMmFSE9y_DE

Security updates for Thursday

2023-12-21 jake

Post Syndicated from jake original https://lwn.net/Articles/955914/

Security updates have been issued by Debian (firefox-esr), Fedora (kernel), Mageia (bluez), Oracle (fence-agents, gstreamer1-plugins-bad-free, opensc, openssl, postgresql:10, and postgresql:12), Red Hat (postgresql:15 and tigervnc), Slackware (proftpd), and SUSE (docker, rootlesskit, firefox, go1.20-openssl, go1.21-openssl, gstreamer-plugins-bad, libreoffice, libssh2_org, poppler, putty, rabbitmq-server, wireshark, xen, xorg-x11-server, and xwayland).

Cyberattack on Ukraine’s Kyivstar Seems to Be Russian Hacktivists

2023-12-21 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/12/cyberattack-on-ukraines-kyivstar-seems-to-be-russian-hacktivists.html

The Solntsepek group has taken credit for the attack. They’re linked to the Russian military, so it’s unclear whether the attack was government directed or freelance.

This is one of the most significant cyberattacks since Russia invaded in February 2022.

Celebrating young Coolest Projects creators at a London museum

2023-12-21 Sophie Ashford

Post Syndicated from Sophie Ashford original https://www.raspberrypi.org/blog/coolest-projects-creators-young-v-a-london/

Each year, young people all over the world share and celebrate their amazing tech creations by taking part in Coolest Projects, our digital technology showcase. Our global online showcase and local in-person events give kids a wonderful opportunity to celebrate their creativity with their communities, explore other young creators’ tech projects, and gain inspiration and encouragement for their future projects.

The Coolest Projects exhibit at the Young V&A in London.

Now, visitors to the Young V&A museum in London can also be inspired by some of the incredible creations showcased at Coolest Projects. The museum has recently reopened after a large reimagining, and some of the inspiring projects by Coolest Projects 2022 participants are now on display in the Design Gallery, ready to spark digital creativity among more young people.

Projects to solve problems

Many Coolest Projects participants showcase projects that they created to make an impact and solve a real-world problem that’s important to them, for example to help members of their local community, or to protect the environment.

A Coolest Projects entry at the Young V&A in London. — At Coolest Projects, Donal (age 9) showcased his creation to send notifications about coronavirus test results via email.

One example on display in the Young V&A gallery is EleVoc, by 15-year-old Chinmayi from India. Chinmayi was inspired to create her project after she and her family faced a frightening encounter:

“My family and I are involved in wildlife conservation. One time we were charged by elephants even though we were only passing by in a Jeep. This was my first introduction to human–animal conflict, and I wanted to find a way to solve it!” – Chinmayi

The experience prompted Chinmayi to create EleVoc, an early-warning device designed to reduce human–elephant conflict by detecting and classifying different elephant sounds and alerting nearby villages to the elephants’ proximity and behaviour.

Also exhibited at the Young V&A is the hardware project Gas Leak Detector by Sashrika, aged 11, from the USA. Gas Leak Detector is a device that detects if a fuel tank for a diesel-powered heating system is leaking and notifies householders through an app in a matter of second.

A Coolest Projects entry at the Young V&A.

A young person and their home-made gas leak detector.

Sashrika knew this invention could really make a difference to people’s lives. She explained, “Typically, diesel gas tanks for heating are in the basement where people don’t visit every day. Leakage may be unnoticed and lead to fire or major repair cost.”

Projects to have fun

As well as projects designed to solve problems, Coolest Projects also welcomes young people who create things to entertain or have fun.

At the Young V&A, visitors can enjoy the fun, fast-paced game project Runaway Nose, by 10-year-old Harshit from Ireland. Runaway Nose uses facial recognition, and players have to use their nose to interact with the prompts on the screen.

Harshit shared the motivation behind his project:

“I wanted to make a fun game to get you thinking fast and that would get you active, even on a rainy day.” – Harshit

We can confirm Runaway Nose is a lot of fun, and a must-do activity for people of all ages on a visit to the museum.

Join in the celebration!

If you are in London, make sure to head to the Young V&A to see Chinmayi’s, Sashrika’s, and Harshit’s projects, and many more. We love seeing the ingenuity of the global community of young tech creators celebrated, and hope it inspires you and your young people.

With that in mind, we are excited that Coolest Projects will be back in 2024. Registrations for the global Coolest Projects online showcase will be open from 14 February to 22 May 2024, and any young creator up to age 18 anywhere in the world can get involved. We’ll also be holding in-person Coolest Projects events for young people in Ireland and the UK. Head to the Coolest Projects website to find out more.

The exhibition hall at Coolest Projects Ireland 2023. — Coolest Projects Ireland 2023.

Coolest Projects is for all young people, no matter their level of coding experience. Kids who are just getting started and would like to take part can check out the free project guides on our projects site. These offer step-by-step guidance to help everyone make a tech project they feel proud of.

To always get the latest news about all things Coolest Projects, from event updates to the fun swag coming for 2024, sign up for the Coolest Projects newsletter.

The post Celebrating young Coolest Projects creators at a London museum appeared first on Raspberry Pi Foundation.

[$] LWN.net Weekly Edition for December 21, 2023

2023-12-21 corbet

Post Syndicated from corbet original https://lwn.net/Articles/955132/

The LWN.net Weekly Edition for December 21, 2023 is available.

TSA 2023

2023-12-21 Oglaf! -- Comics. Often dirty.

Post Syndicated from Oglaf! -- Comics. Often dirty. original https://www.oglaf.com/tsa-2023/

Proactively secure your environment

Detect and prioritize threats anywhere, from the endpoint to the cloud

Investigate and respond with confidence

Tech for Staying Connected

Tech for Devs

Tech That Makes Home Homier

The Apps Have Entered the Chat

The “Tech Is a Tool” Answer

Thanks, Andy

Overview of solution

Evolution of the data platform requirements

Optimizing the architecture with data sharing and Redshift Serverless

Future vision

Conclusion

About the Authors

Solution overview

Prerequisites

Create an S3 bucket

Create an index in the OpenSearch Service domain

Create a secret for OpenSearch Service credentials

Create an IAM role for the AWS Glue job

Create an AWS Glue connection

Create an ETL job using AWS Glue Studio

Clean up

Conclusion

About the authors

What is broadband Internet?

Interconnection and traffic exchange between networks

Choosing the right certificate verification method

Mutual authentication with API Gateway

Implementing mTLS certificate verification for API Gateway

Implementation mechanisms

Data repository for your certificate revocation list

Certificate validation

Optimizing for performance

Performance optimizations

Preprocessing and generation of a cached CRL file

Client certificate lookup in a CRL

Function-level caching

Conclusion

Projects to solve problems

Projects to have fun

Join in the celebration!

The collective thoughts of the interwebz