Post Syndicated from The Hook Up original https://www.youtube.com/watch?v=cuLpDo-TVK8
[$] splice() and the ghost of set_fs()
Post Syndicated from original https://lwn.net/Articles/896267/
The normal rule of kernel development is that the creation of user-space
regressions is not allowed; a patch that breaks a previously working
application must be either fixed or reverted. There are exceptions,
though, including a
5.10 patch that has been turning up regressions ever since. The story
that emerges here shows what can happen when the goals of stability,
avoiding security problems, and code cleanup run into conflict.
What happened to Perl 7?
Post Syndicated from original https://lwn.net/Articles/896353/
The Perl Steering Council has posted a
blog entry on its plans for the language and when Perl 7 might be
released.
For now, our plan is to continue introducing new features and to
resolve all existing experimental features, so they’re either
dropped, or become non-experimental features (and so are included
in the version bundle).The downside with this is that people often can’t remember which
version of Perl introduced which feature(s).At some point in the future, the PSC may decide that the set of
features, taken together, represent a big enough step forward to
justify a new baseline for Perl. If that happens, then the version
will be bumped to 7.0.
Handy Tips #30: Detect continuous increase or decrease of values with monotonic history functions
Post Syndicated from Arturs Lontons original https://blog.zabbix.com/handy-tips-30-detect-continuous-increase-or-decrease-of-values-with-monotonic-history-functions-2/20867/
The post Handy Tips #30: Detect continuous increase or decrease of values with monotonic history functions appeared first on Zabbix Blog.
Security updates for Thursday
Post Syndicated from original https://lwn.net/Articles/896346/
Security updates have been issued by Debian (chromium, dpkg, filezilla, irssi, puma, and python-django), Fedora (firefox, ignition, and pcre2), Mageia (cockpit, firefox/thunderbird, openldap, supertux, unrar, and vim), Oracle (firefox and thunderbird), Red Hat (rh-varnish6-varnish), SUSE (cups, fribidi, kernel-firmware, redis, and wpa_supplicant), and Ubuntu (dpkg, logrotate, and subversion).
Cloudflare’s approach to handling BMC vulnerabilities
Post Syndicated from Derek Chamorro original https://blog.cloudflare.com/bmc-vuln/


In recent years, management interfaces on servers like a Baseboard Management Controller (BMC) have been the target of cyber attacks including ransomware, implants, and disruptive operations. Common BMC vulnerabilities like Pantsdown and USBAnywhere, combined with infrequent firmware updates, have left servers vulnerable.
We were recently informed from a trusted vendor of new, critical vulnerabilities in popular BMC software that we use in our fleet. Below is a summary of what was discovered, how we mitigated the impact, and how we look to prevent these types of vulnerabilities from having an impact on Cloudflare and our customers.
Background
A baseboard management controller is a small, specialized processor used for remote monitoring and management of a host system. This processor has multiple connections to the host system, giving it the ability to monitor hardware, update BIOS firmware, power cycle the host, and many more things.

Access to the BMC can be local or, in some cases, remote. With remote vectors open, there is potential for malware to be installed on the BMC from the local host via PCI Express or the Low Pin Count (LPC) interface. With compromised software on the BMC, malware or spyware could maintain persistence on the server.

According to the National Vulnerability Database, the two BMC chips (ASPEED AST2400 and AST2500) have implemented Advanced High-Performance Bus (AHB) bridges, which allow arbitrary read and write access to the physical address space of the BMC from the host. This means that malware running on the server can also access the RAM of the BMC.
These BMC vulnerabilities are sufficient to enable ransomware propagation, server bricking, and data theft.
Impacted versions
Numerous vulnerabilities were found to affect the QuantaGrid D52B cloud server due to vulnerable software found in the BMC. These vulnerabilities are associated with specific interfaces that are exposed on AST2400 and AST2500 and explained in CVE-2019-6260. The vulnerable interfaces in question are:
- iLPC2AHB bridge Pt I
- iLPC2AHB bridge Pt II
- PCIe VGA P2A bridge
- DMA from/to arbitrary BMC memory via X-DMA
- UART-based SoC Debug interface
- LPC2AHB bridge
- PCIe BMC P2A bridge
- Watchdog setup
An attacker might be able to update the BMC directly using SoCFlash through inband LPC or BMC debug universal async receiver-transmitter (UART) serial console. While this might be thought of as a usual path in case of total corruption, this is actually an abuse within SoCFlash by using any open interface for flashing.
Mitigations and response
Updated firmware
We reached out to one of our manufacturers, Quanta, to validate that existing firmware within a subset of systems was in fact patched against these vulnerabilities. While some versions of our firmware were not vulnerable, others were. A patch was released, tested, and deployed on the affected BMCs within our fleet.
Cloudflare Security and Infrastructure teams also proactively worked with additional manufacturers to validate their own BMC patches were not explicitly vulnerable to these firmware vulnerabilities and interfaces.
Reduced exposure of BMC remote interfaces
It is a standard practice within our data centers to implement network segmentation to separate different planes of traffic. Our out-of-band networks are not exposed to the outside world and only accessible within their respective data centers. Access to any management network goes through a defense in depth approach, restricting connectivity to jumphosts and authentication/authorization through our zero trust Cloudflare One service.
Reduced exposure of BMC local interfaces
Applications within a host are limited in what can call out to the BMC. This is done to restrict what can be done from the host to the BMC and allow for secure in-band updating and userspace logging and monitoring.
Do not use default passwords
This sounds like common knowledge for most companies, but we still follow a standard process of changing not just the default username and passwords that come with BMC software, but disabling the default accounts to prevent them from ever being used. Any static accounts follow a regular password rotation.
BMC logging and auditing
We log all activity by default on our BMCs. Logs that are captured include the following:
- Authentication (Successful, Unsuccessful)
- Authorization (user/service)
- Interfaces (SOL, CLI, UI)
- System status (Power on/off, reboots)
- System changes (firmware updates, flashing methods)
We were able to validate that there was no malicious activity detected.
What’s next for the BMC
Cloudflare regularly works with several original design manufacturers (ODMs) to produce the highest performing, efficient, and secure computing systems according to our own specifications. The standard processors used for our baseboard management controller often ship with proprietary firmware which is less transparent and more cumbersome to maintain for us and our ODMs. We believe in improving on every component of the systems we operate in over 270 cities around the world.
OpenBMC
We are moving forward with OpenBMC, an open-source firmware for our supported baseboard management controllers. Based on the Yocto Project, a toolchain for Linux on embedded systems, OpenBMC will enable us to specify, build, and configure our own firmware based on the latest Linux kernel featureset per our specification, similar to the physical hardware and ODMs.
OpenBMC firmware will enable:
- Latest stable and patched Linux kernel
- Internally-managed TLS certificates for secure, trusted communication across our isolated management network
- Fine-grained credentials management
- Faster response time for patching and critical updates
While many of these features are community-driven, vulnerabilities like Pantsdown are patched quickly.
Extending secure boot
You may have read about our recent work securing the boot process with a hardware root-of-trust, but the BMC has its own boot process that often starts as soon as the system gets power. Newer versions of the BMC chips we use, as well as leveraging cutting edge security co-processors, will allow us to extend our secure boot capabilities prior to loading our UEFI firmware by validating cryptographic signatures on our BMC/OpenBMC firmware. By extending our security boot chain to the very first device that has power to our systems, we greatly reduce the impact of malicious implants that can be used to take down a server.
Conclusion
While this vulnerability ended up being one we could quickly resolve through firmware updates with Quanta and quick action by our teams to validate and patch our fleet, we are continuing to innovate through OpenBMC, and secure root of trust to ensure that our fleet is as secure as possible. We are grateful to our partners for their quick action and are always glad to report any risks and our mitigations to ensure that you can trust how seriously we take your security.
How we treat content as a product
Post Syndicated from Kim Jeske original https://blog.cloudflare.com/content-as-a-product/


At Cloudflare, we talk a lot about how to help build a better Internet. On the Product Content Experience (PCX) team, we treat content like a product that represents and fulfills this mission. Our vision is to create world-class content that anticipates user needs and helps build accessible Cloudflare products. We believe we can impact the Cloudflare product experience and make it as wonderful as possible by intentionally designing, packaging, and testing the content.
What is “content like a product”?
I like taking on projects. A singular goal is met, and I clearly know I’m successful because the meaning of “done” is normally very clear. For example, I volunteer some of my time editing academic papers about technology. My role as an editor is temporary and there is a defined beginning and end to the work. I send my feedback and my task is largely complete.
“Content like a product” is when you shift your mindset from completing projects to maintaining a product, taking into consideration the user and their feedback. Product content at Cloudflare is an iterative, living, breathing thing. Inspired by the success of teams that adopt an agile mindset, along with some strategic functions you might find in a product management organization, treating content like a product means we treat content much like how a software project is created and maintained. This strategy allows for content development behaviors that closely align with the release of actual products, while also allowing technical writers and content designers to be laser-focused on doing what’s best for the user.
Adopting a product development mindset
When the content team was new, we initially adopted many traditional agile methodologies. Why agile? Before I joined Cloudflare I was a product owner and was a huge advocate for sprint planning, retrospectives, and daily stand-ups. I liked Agile — I could easily keep up with a technical team, focus on priorities, and get things done quickly and efficiently. However, the rigidity of agile was just a bit too much for a content team. Over time, we modified and chose our favorite parts of the methodology while letting the rest go.
Shifting to a product development process created a lot of flexibility, but we didn’t want to abandon all process. Situationally, we take a process-focused mindset. For writing tasks that need to be predictable and consistent, like choosing inclusive terminology throughout our documentation, we have automated and manual processes to ensure we’re following our best practices.
Aligning content to the product development process means that when a new product is shipping, we have developer documentation ready to publish. Whenever the UI of the product changes, screenshots in the docs are updated accordingly. When new features are launched, we provide how-to guides and configuration content. Better alignment with the product team not only means the content team maintains accuracy of staying on top of all changes, it allows us to be user-focused. Above all, writers are aligned to the most important priority — shipping fast and often.

Shipping content — fast!
As you know, Cloudflare ships fast. You can see just a small sample of what I mean by fast here, here, and here. That speed was driven home within my first few weeks. I started just before Birthday Week 2020, and was super excited because I just wanted to jump in and create a lot of great content. But wow. What an intense start time. After Birthday Week, my main concerns were how to balance quality while meeting demand. I also wanted to create a quality environment for a team.
In retrospect, Birthday Week was a great time to start because it highlighted that keeping pace with products was going to be a big priority. Here’s how the content team met the demand.
First, the writers and I established that our focus was creating the most important content for the user, which allowed us to establish a product development mindset. We were now aligned with the product team.
Second, we moved content to an open source platform. This helped writers ship content fast because our authoring tools were consolidated to fewer platforms, and we were now in the same environment as our users.
We actually started publishing content as fast as products shipped within a few months! The content team began chipping away at the backlog once we understood the product team’s release cadence, and within less than six months we were ahead of the backlog and focusing on bigger initiatives including how to make content accessible, more consistent, and approachable to a wider group of users. It happened fast and was thrilling as a content creator.
The open source authoring tools on developers.cloudflare.com have evolved since 2020, continuing to help writers and contributors publish content faster by improving the review and build processes. We moved the docs platform to Cloudflare Pages earlier this year, allowing the writers to help build a more robust open source docs community while also providing valuable feedback to the Cloudflare Pages team.
Conclusion
Adopting a “content as a product” strategy requires buy-in from product managers and engineers, but it scales really well once established because everyone is focused on supporting the user versus the specifics of a content strategy itself. We go through the same planning, research, and analytics tasks you might find for a product to identify if we are creating the right content for folks who read the docs or use Cloudflare products. While everything we do with content is done so that we can create better content for our users, we also intentionally communicate that the content strategy is just a tool that enables a great user experience.
Over the next few weeks expect to see more about how Cloudflare writers have embraced the “content as a product” methodology as part of their own specific roles. In addition to learning more about how the developers.cloudflare.com site was moved to Cloudflare Pages, writers will share how they leaned into content creation for an open source community, their journey from technical writer to UX writing and content design, and share more specifics about our content strategy including the customer journey and success metrics.
I’ll admit. I’m fortunate to work for a company with overwhelming support regarding content. Great documentation is important to so many folks, and we’ve created the type of writing environment I always wanted to be a part of. It’s an exciting time to be helping build a better Internet through excellent product content.
Malware-Infested Smart Card Reader
Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2022/05/malware-infested-smart-card-reader.html
Brian Krebs has an interesting story of a smart ID card reader with a malware-infested Windows driver, and US government employees who inadvertently buy and use them.
But by all accounts, the potential attack surface here is enormous, as many federal employees clearly will purchase these readers from a myriad of online vendors when the need arises. Saicoo’s product listings, for example, are replete with comments from customers who self-state that they work at a federal agency (and several who reported problems installing drivers).
Германският билет за 9 евро – що е то?
Post Syndicated from Светла Енчева original https://toest.bg/germanskiyat-bilet-za-9-evro/
Повишените цени на горивата са проблем не само в България, а войната на Русия срещу Украйна го изостря още повече. Различните държави имат различни стратегии за справяне с него. Една от идеите за антикризисни мерки в България например е държавата да поема по 25 стотинки от цената на всеки литър гориво (от по-евтините му варианти), който хората си купуват на бензиностанциите. Така обаче, вместо да се стимулира ограничаването на използването на изкопаеми горива, гражданите се поощряват да продължат да се придвижват с личните си автомобили. На този фон
Германия предприема радикален, макар и ограничен във времето експеримент.
От 1 юни до края на август т.г. всички хора в Германия ще могат да използват обществения транспорт в цялата страна за 9 евро на месец. Това включва метрото, автобусите, трамваите, градските и регионалните влакове (вагони втора класа), също и тези фериботи и корабчета, които спадат към обществения транспорт. Изключение правят експресните влакове, както и някои автобусни и влакови линии, които не са свързани със системата на „Дойче Бан“, и разбира се, самолетите и туристическите кораби. Билетът е поименен (тоест не може да се преотстъпва) и е валиден в рамките на календарния месец, за който е купен. Всеки може да се сдобие с такъв билет, включително ако не е германски гражданин и дори ако няма адресна регистрация в страната.
Целта на мярката е да се стимулират хората да се придвижват с обществен транспорт вместо с личните си автомобили и по този начин да се ограничи потреблението на горива. От нея се очаква да се облагодетелстват най-вече т.нар. Pendler – хора, на които се налага непрекъснато да пътуват между две населени места, например защото в единия град живеят, а в другия работят. Немалка част от често пътуващите между две населени места предпочитат да го правят с личните си автомобили – не на последно място, защото им излиза по-евтино, отколкото да използват влака.
Експериментът е радикален, тъй като общественият транспорт в Германия е… скъпичък.
Карта само за един ден за двете най-популярни транспортни зони в Берлин струва 8,80 евро – почти колкото експерименталния месечен билет. Ако искате да пътувате и в трите зони на германската столица, дневният билет е 10 евро. Придвижването в рамките на 24 часа из цялата транспортна област, включваща федералните провинции Берлин и Бранденбург, струва 23 евро. Еднодневна карта за по-голяма провинция ще ви излезе още по-скъпо, за Северен Рейн-Вестфалия например ще трябва да извадите 30,60 евро.
Повечето хора в Германия рядко плащат толкова скъпи билети – купуват ги основно туристите, както и хората, извършващи пътувания, които не се покриват (или поне не изцяло) от абонаментните им карти. В страната има всевъзможни абонаментни карти, билети за пътуващи по двойки и в групи, облекчения за ученици, студенти, социално слаби, хора с увреждания и пр. Ако платите например наведнъж годишна карта за двете вътрешни зони на Берлин от 720 евро, това прави по 60,67 евро на месец. Ако пък искате да пътувате неограничено из цяла Германия, включително и с експресни влакове, най-изгодно ще ви е да си купите годишна карта за 4144 евро (за втора класа), което ще ви излезе 345,33 евро на месец. На този фон 9 евро си е направо без пари.
Сигурно вече се питате кой плаща за това удоволствие.
Билетът от 9 евро е приет с решение на германското правителство и се финансира от държавата, която отделя за целта 2,5 млрд. евро от бюджета си. Само че тук има уловка – ако тази сума се надвиши по време на експеримента, разликата ще трябва да се поеме от федералните провинции и общините, според някои от които пък парите няма да стигнат. Те се опасяват, че още непреодолели загубите от пандемията, ще трябва да влизат в нови дългове заради евтиния билет. Сред най-гласовитите критици са Бавария и Баден-Вюртемберг, известни впрочем не на последно място с производството на автомобили.
Последствието от въвеждането на евтиния билет не е само в това, че хората ще си купуват него вместо по-скъпи абонаментни карти и билети. Трябва да се компенсират и онези, които вече са си купили локални абонаментни карти и които впрочем дори няма нужда да си купуват новия билет, защото картата им важи като такъв, независимо къде е издадена. Ако става дума за дългосрочен абонамент, е по-лесно – платената сума може да се прехвърли към следващ период. Но за вече купени временни карти и билети се налага да се възстановяват пари. Освен това от 1 юни много хора в Германия
ще се втурнат да пътуват с обществения транспорт не защото им се налага, а защото е достъпно.
Мярката неслучайно е планирана за летните месеци – тогава голяма част от германците излизат в отпуск или ваканция, а много от тях традиционно почиват в чужбина. Но пък останалите в страната ще имат еднократната възможност да се напътуват на воля. А е твърде вероятно в Германия да пристигнат и туристи, привлечени именно от евтиния билет. Освен това германските отпуски не са по три месеца, а ученическите ваканции са по-кратки от българските, така че през поне два от тези три месеца голяма част от хората ще продължават да ходят на работа, на училище и в университета.
Всичко това създава риска превозните средства да се окажат толкова претъпкани, че да не може да се ползват нормално от онези, на които действително са нужни. Обсъжда се дали да се ограничи превозването на велосипеди, за да има повече място за хората с увреждания. „Дойче Бан“ отпуска допълнително по 50 регионални влака дневно, които да изпълняват общо 250 пътувания, осигурявайки 60 000 седящи места на денонощие (без да броим правостоящите). Дали това ще е достатъчно, ще се разбере след 1 юни. Но всички тези допълнителни влакове имат машинисти, контрольори и прочее, на които ще се плаща.
Билетът от 9 евро не е привлекателен във всички случаи.
Основното му ограничение не е за пренебрегване, а именно че не покрива пътуването с експресни влакове. Теоретично е възможно из цяла Германия да се придвижвате и само с регионалните. На практика обаче при по-дългите дистанции това означава поне два пъти повече време и множество прекачвания. Ако имате достатъчно свободно време и сте в добро здраве, може да превърнете дефекта в ефект и да си организирате пътешествия със спирки за похапване, разходки и преспиване. Ако ви се налага обаче спешно да отидете от Кьолн в Берлин например, трябва да избирате дали да се кламбучкате минимум десетина часа, сменяйки няколко влака, или да платите стотина евро за експрес. А има и хора, които редовно пътуват на дълги разстояния – поради работа или лични причини.
Въпреки че общественият транспорт в Германия е далеч по-развит от нашенския, и там не всички биха заменили удобството и престижа на личния си автомобил. Особено в Южна Германия, където е концентрирана автомобилната индустрия на страната – в Бавария и Баден-Вюртемберг са централите на концерни като „Мерцедес“, „Порше“, БМВ и „Ауди“. В тази част на страната мрежата на междуградския обществен транспорт не е толкова гъста, колкото на север (макар да е несравнимо по-развита от българската), а личният автомобил е на почит, особено ако е от гореспоменатите марки. Освен това има хора, които поради спецификата си на живот просто не пътуват много.
И все пак пускането на новия билет в продажба на 23 май се превърна в събитие.
За броени часове чрез онлайн системата на „Дойче Бан“ се продадоха над 200 000 билета. Това доведе до сривове в сайта и мобилното приложение на компанията и множество съобщения за грешка, въпреки че новите билети нито са в ограничено количество, нито може да се купят единствено чрез „Дойче Бан“ – те са налични и в автоматите за продажба на билети, както и в някои мобилни приложения на местни транспортни организации. Трудно е обаче човек да устои на изкушението да си подсигури евтино пътуване в първия възможен момент.
Тепърва ще се преценява дали иновативната антикризисна мярка е успех, или провал. Ще се ограничи ли чувствително използването на горива в Германия? Федералните провинции ще затънат ли в дългове? Ще спечели ли поради облекчената възможност за пътуване местната туристическа индустрия, сериозно пострадала поради пандемията от COVID-19?
Споровете за и против билета от 9 евро са не на последно място политически –
апокалипсис вещаят предимно икономически десни гласове, без да забравяме и силните лобита на автомобилната индустрия в страната, пуснали дълбоки корени в политическата система. Мярката се харесва най-много на „Зелените“. За загрижените за екологията германци експериментът може да се превърне в основа на устойчиво решение за по-достъпен обществен транспорт, макар и не за 9 евро на месец. Защото не може хем страната да си поставя за цел да премине към чисти и възобновяеми енергийни източници, хем пътуването с кола да излиза по-евтино от това с влак.
В този контекст войната в Украйна може да се окаже катализатор за преосмислянето на енергийната политика в Германия и еманципирането от лобитата на автомобилната индустрия. Тя извади на показ огромните измерения на зависимостта от руските горива, експлоатирана от режима на Путин по твърде конфузен за германското общество начин. Ясно е, че независимостта има висока цена, която ще трябва да се плаща от всички – разбира се, ако Германия се реши да го направи, тъй като информацията, че страната е готова да купува газ според условията на Русия, дава индикации в обратната посока. И все пак високата цена може да се плати не само с по-високи сметки за енергия и по-скъпи стоки и услуги. Парите на германските данъкоплатци могат да се инвестират и в по-достъпен обществен транспорт и по-чиста околна среда.
Заглавна снимка: © Светла Енчева
[$] LWN.net Weekly Edition for May 26, 2022
Post Syndicated from original https://lwn.net/Articles/895776/
The LWN.net Weekly Edition for May 26, 2022 is available.
[Security Nation] Omer Akgul and Richard Roberts on YouTube VPN Ads
Post Syndicated from Rapid7 original https://blog.rapid7.com/2022/05/25/security-nation-omer-akgul-and-richard-roberts-on-youtube-vpn-ads/
![[Security Nation] Omer Akgul and Richard Roberts on YouTube VPN Ads](https://blog.rapid7.com/content/images/2022/05/security_nation_logo.jpg)
In this episode of Security Nation, Jen and Tod chat with academics Omer Akgul and Richard Roberts about their recent paper, “Investigating Influencer VPN Ads on YouTube.” They talk about the over-promising and obfuscation that’s commonplace in advertisements for commercial VPN services on the video streaming platform and what these tactics reveal about communication around security tools and ideas to laypeople.
Stick around for our Rapid Rundown, where our hosts talk with Rapid7’s public policy guru Harley Geiger about the recent news that the US Department of Justice will stop prosecuting ethical hackers.
Omer Akgul
![[Security Nation] Omer Akgul and Richard Roberts on YouTube VPN Ads](https://blog.rapid7.com/content/images/2022/05/Omer.png)
Omer Akgul is a fifth-year Computer Science Ph.D. student at the University of Maryland, College Park. Advised by Michelle Mazurek, Omer works on several human factors in security and privacy problems. Most recently, he has been investigating harmful mental models of secure communication tools. His research regularly appears in prominent security and privacy venues and can be found here.
Richard Roberts
![[Security Nation] Omer Akgul and Richard Roberts on YouTube VPN Ads](https://blog.rapid7.com/content/images/2022/05/19477697_10207172397147623_66495529147866075488_o.jpg)
Richard Roberts is a Ph.D. student at the University of Maryland studying computer science with Dr. Dave Levin. There is often a disconnect between technical specification and lay user perception. Richard is interested in how those cracks form, how they are leveraged by malicious actors, and how to design technical solutions that meet users where they are. Richard’s other research interests include authentication and impersonation on the internet, measurements and unintended consequences of the web’s PKI, and how security is depicted in media.
You can find links to his publications and more information about his work here.
Show notes
Interview links
- Check out Omer and Richard’s paper.
- Learn more about Omer’s work and Richard’s work.
Rapid Rundown links
- Read the news about the change in DOJ policy toward ethical hackers.
- Visit the Rapid7 blog on the same topic.
- Dive into Harley’s great Twitter thread on the topic.
- Read up on the HiQ and Missouri cases mentioned.
Like the show? Want to keep Jen and Tod in the podcasting business? Feel free to rate and review with your favorite podcast purveyor, like Apple Podcasts.
Want More Inspiring Stories From the Security Community?
[$] Making O_TMPFILE atomic (and statx() additions)
Post Syndicated from original https://lwn.net/Articles/896153/
Right on the heels of his previous filesystem
session at the
2022 Linux Storage,
Filesystem, Memory-management and BPF Summit (LSFMM), Steve French led
a session on temporary files and their interaction with network
filesystems.
The problem is that creating temporary files is not always atomic, so he was
proposing changing that, which would eliminate a possible race condition
and be more efficient for network filesystems.
Since the temporary-file discussion did not fill the 30-minute slot, however, French took
the opportunity to discuss some attributes he would like to see get added for the
statx()
system call.
Visualize MongoDB data from Amazon QuickSight using Amazon Athena Federated Query
Post Syndicated from Soujanya Konka original https://aws.amazon.com/blogs/big-data/visualize-mongodb-data-from-amazon-quicksight-using-amazon-athena-federated-query/
In this post, you will learn how to use Amazon Athena Federated Query to connect a MongoDB database to Amazon QuickSight in order to build dashboards and visualizations.
Amazon Athena is a serverless interactive query service, based on Presto, that provides full ANSI SQL support to query a variety of standard data formats, including CSV, JSON, ORC, Avro, and Parquet, that are stored on Amazon Simple Storage Service (Amazon S3). For data that isn’t stored on Amazon S3, you can use Athena Federated Query to query the data in place or build pipelines that extract data from multiple data sources and store it in Amazon S3. With Athena Federated Query, you can run SQL queries across data that is stored in relational, non-relational, object, and custom data sources.
MongoDB is a popular NoSQL database option for websites and API endpoints. You can choose to deploy MongoDB as a self-hosted or fully-managed database. Databases are a popular choice for UI applications for managing user profiles, product catalogs, profile views, clickstream events, events from a connected device, and so on. QuickSight is a serverless business analytics service with built-in machine learning (ML) capabilities that can automatically look for patterns and outliers, and has the flexibility to embed dashboards in applications for a data-driven experience. You can also use QuickSight Q to allow users to ask questions using natural language and find answers to business questions immediately.
Overview of Athena Federated Query
Athena Federated Query uses data source connectors that run on AWS Lambda to run federated queries to other data sources. Prebuilt data source connectors are available for native stores, like Amazon Timestream, Amazon CloudWatch Logs, Amazon DynamoDB, and external sources like Vertica and SAP Hana. You can also write a connector by using the Athena Query Federation SDK. You can customize Athena’s prebuilt connectors for your own use, or modify a copy of the source code to create your own AWS Serverless Application Repository package.
Solution overview
The following architecture diagram shows the components of the Athena Federated Query MongoDB connector. It contains the following components:
- A virtual private cloud (VPC) configured with public and private subnets across three Availability Zones.
- A MongoDB cluster with customizable Amazon Elastic Block Store (Amazon EBS) storage deployed in private subnets and NAT gateways in a public subnet for outbound internet connectivity for MongoDB instances.
- Bastion hosts in an auto scaling group with Elastic IP addresses to allow inbound SSH access.
- An AWS Identity and Access Management (IAM)
MongoDBnoderole with Amazon Elastic Compute Cloud (Amazon EC2) and Amazon S3 permissions. - Security groups to enable communication within the VPC.
- Lambda functions deployed in a private subnet accessing S3 buckets. Athena invokes the Lambda function, which in turn fetches the data from MongoDB and maps the response back to Athena.
- AWS Secrets Manager through a VPC endpoint.

Prerequisites
To implement the solution, you need the following:
- An AWS account to access AWS services.
- An IAM user with permission to
CreateRole,ListRoles,GetPolicy, andAttachRolePolicy. - An IAM user with an access key and secret key to configure an integrated development environment (IDE).
- A MongoDB database. You can deploy a hosted MongoDB on Amazon EC2 or MongoDB Atlas in a VPC.
- If you don’t have a QuickSight subscription configured, sign up for one. You can access the QuickSight free trial as part of the AWS Free Tier option.
- A new secret in Secrets Manager to store your MongoDB user name and password.
- Data loaded into your MongoDB database. For this example, we used an airline dataset. Load the sample data either from the MongoDB command line or the MongoDB Atlas user interface, if using MongoDB Atlas.
Configure a Lambda connector
The first step in the deployment is to set up the connector environment. Athena uses data source connectors that run on Lambda to run federated queries. To connect with MongoDB, use the Amazon Athena DocumentDB Connector, which also works with any endpoint that is compatible with MongoDB.
To configure a Lambda connector, complete the following steps:
- On the Athena console, choose Data sources in the navigation pane.
- To view a published list of data sources for Athena, select Amazon DocumentDB.
- Choose Next.

- In the Data source details section, give your data source a unique name; for example,
ds_mongo.
This will be the connection name that appears under Data sources for Athena.

- Choose Create Lambda function.
This launches the Create function page in Lambda. The connector is deployed by using AWS Serverless Application Repository.

- For SecretNameOrPrefix, enter
mongo. - For SpillBucket, enter
spl-mongo-athena-test. - For AthenaCatalogName, enter
us-west-mongo-cat. - For DocDBConnectionString (the MongoDB connection), enter the following:
- For SecurityGroupIds, choose the security group that you want to associate with the function. Make sure that the security group of the MongoDB instance allows traffic from the Lambda function.
- For SpillPrefix, enter
athena-spill. - For Subnetids, enter the subnet IDs of subnets with MongoDB instances.
In this case, LambdaMemory and LambdaTimeout have been set to the maximum values, but these can vary depending on the query run and memory requirements. SpillBucket is an S3 bucket in your account to store data that exceeds the Lambda function response size limits. - Keep the rest as defaults.
- Select the acknowledgement check box choose Deploy.
The connection function is launched based on the given parameters.
- Create a VPC endpoint to allow the Lambda function to access Amazon S3 through an endpoint.
This is for the spill bucket. The spill bucket is a staging area for copying the results of the queries that are performed on MongoDB via Athena federation. This is so that the Lambda function in the VPC can access Amazon S3. - Go back to the Athena console.
- Under Connection details, for Lambda function, choose the newly created Lambda function.
- Choose Next.

- To verify the connection, on the Athena console, choose Data sources, then choose
ds_mongo.
Associated databases from the connection should be listed.

You should now be able to query the datasets from the Athena query editor by using SQL. - In the query editor, for Data Source, choose
ds_mongo.
Athena federates the query using the connector, which invokes the Lambda function. Then the query is performed by the function on MongoDB, and the query results are translated back to Athena. The following is a sample query that was performed on the airlines dataset.

Create a dataset on QuickSight to read the data from MongoDB
Before you launch QuickSight for the first time in an AWS account, you must set up an account. For instructions, see Signing in to Amazon QuickSight.
After the initial setup, you can create a dataset with Athena as the source. The QuickSight service role needs permission to invoke the Lambda function that connects MongoDB. The aws-quicksight-service-role-v0 service role is automatically created with the QuickSight account.
To create a dataset in QuickSight, complete the following steps:
- On the IAM console, in the navigation pane, choose Roles.
- Search for the role
aws-quicksight-service-role-v0and add the permissionLambda _fullaccess.
In an organization, there could be different data stores based on data load and consumption patterns. Examples include catalog or manual data that is associated with products in a MongoDB or key-value index store, transactions or sales data in a SQL database, and images or video clips that are associated with the product in an object store.
In this case, an airlinestable from MongoDB is joined with a flat file that contains information on the airports. - Use the QuickSight cross-data store feature to join data from different sources on common fields.

- We then update the data types for our geographic fields like fields like city, country, latitude, and longitude so we can build maps later.
- You can also create calculated fields while preparing your dataset, which allows you to reuse them in other QuickSight analyses.

With a few clicks, you should be able to create a dashboard with the published dataset. For instance, you can plot your data on a map, show trends in a line chart, and add autonarratives from the list of Suggested Insights to create the analysis shown in the following screenshot.
Clean up
Make sure to clean up your resources to avoid resource spend and associated costs. You need to delete the EC2 instances with MongoDB. In the case of MongoDB Atlas, you can delete the databases and tables. Delete the Athena data source ds_mongo and unsubscribe your QuickSight account from the Manage QuickSight admin page.
Conclusion
With QuickSight and Athena Federated Query, organizations can access additional data sources beyond those already supported by QuickSight. If you have data in sources other than Amazon S3, you can use Athena Federated Query to analyze the data in place or build pipelines that extract and store data in Amazon S3. Athena now also supports cross-account federated queries to enable teams of analysts, data scientists, and data engineers to query data stored in other AWS accounts. Try connecting to proprietary data formats and sources, or build new user-defined functions, with the Athena Query Federation SDK.
About the Author
Soujanya Konka is a Solutions Architect and Analytics specialist at AWS, focused on helping customers build their ideas on cloud. Expertise in design and implementation of business information systems and Data warehousing solutions. Before joining AWS, Soujanya has had stints with companies such as HSBC, Cognizant.
Nilesh Parekh is a Partner Solution Architect with ISV India segment. Nilesh help assist partner to review and remediate their workload running on AWS based on the AWS Well-Architected and Foundational Technical Review best practices. He also helps assist partners on Application Modernizations and delivering POCs.
[$] Change notifications for network filesystems
Post Syndicated from original https://lwn.net/Articles/896055/
Steve French led a discussion on change notifications for network
filesystems in a session at the
2022 Linux Storage,
Filesystem, Memory-management and BPF Summit (LSFMM). He is part of
the Samba team and noted that both Windows and macOS clients get notified
of new and changed files in
a shared directory immediately, while on Linux that does not happen. He
wanted to explore what it would take to add that functionality.
Kristina Cho | Mooncakes and Milk Bread | Talks at Google
Post Syndicated from Talks at Google original https://www.youtube.com/watch?v=UCoD2iccIWA
Phil Rosenthal | Somebody Feed Phil | Talks at Google
Post Syndicated from Talks at Google original https://www.youtube.com/watch?v=JB9qUbaDPvI
The Linux Foundation’s “security mobilization plan”
Post Syndicated from original https://lwn.net/Articles/896244/
The Linux Foundation has posted an “Open Source
Software Security Mobilization Plan” that aims to address a number of
perceived security problems with the expenditure of nearly
$140 million over two years.
While there are considerable ongoing efforts to secure the OSS
supply chain, to achieve acceptable levels of resilience and risk,
a more comprehensive series of investments to shift security from a
largely reactive exercise to a proactive approach is required. Our
objective is to evolve the systems and processes used to ensure a
higher degree of security assurance and trust in the OSS supply
chain.This paper suggests a comprehensive portfolio of 10 initiatives
which can start immediately to address three fundamental goals for
hardening the software supply chain. Vulnerabilities and weaknesses
in widely deployed software present systemic threats to the
security and stability of modern society as government services,
infrastructure providers, nonprofits and the vast majority of
private businesses rely on software in order to function.
Manipulating Machine-Learning Systems through the Order of the Training Data
Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2022/05/manipulating-machine-learning-systems-through-the-order-of-the-training-data.html
Yet another adversarial ML attack:
Most deep neural networks are trained by stochastic gradient descent. Now “stochastic” is a fancy Greek word for “random”; it means that the training data are fed into the model in random order.
So what happens if the bad guys can cause the order to be not random? You guessed it—all bets are off. Suppose for example a company or a country wanted to have a credit-scoring system that’s secretly sexist, but still be able to pretend that its training was actually fair. Well, they could assemble a set of financial data that was representative of the whole population, but start the model’s training on ten rich men and ten poor women drawn from that set then let initialisation bias do the rest of the work.
Does this generalise? Indeed it does. Previously, people had assumed that in order to poison a model or introduce backdoors, you needed to add adversarial samples to the training data. Our latest paper shows that’s not necessary at all. If an adversary can manipulate the order in which batches of training data are presented to the model, they can undermine both its integrity (by poisoning it) and its availability (by causing training to be less effective, or take longer). This is quite general across models that use stochastic gradient descent.
Research paper.
My TOP 5 HACS components – integrations and front-end improvements
Post Syndicated from BeardedTinker original https://www.youtube.com/watch?v=jedZw1aIrbc
F-Droid: Our build and release infrastructure, and upcoming updates
Post Syndicated from original https://lwn.net/Articles/896240/
Here’s an
update from F-Droid regarding upcoming changes to its build and
distribution infrastructure.
If you have an app on f-droid.org, you might have noticed that all
builds happen on a 5 year old Debian release: stretch. We are in
the midst of a big effort to upgrade to the latest bullseye release
right now. This is not just a simple apt-get upgrade, we
are also
taking this opportunity to overhaul the build process so that app
builds work with a relatively plain Debian install as the base
OS. We have to provide a platform to build thousands of apps, so we
cannot just upgrade the base image as often as we like.










