Greg Kroah-Hartman has announced another round of stable kernel
updates: 6.9.4, 6.6.33, and 6.1.93 have been released. Each contains
another set of important fixes, users of these kernels are advised to
upgrade right away.
One of the great benefits of the Internet has been its ability to empower activists and journalists in repressive societies to organize, communicate, and simply find each other. Ten years ago today, Cloudflare launched Project Galileo, a program which today provides security services, at no cost, to more than 2,600 independent journalists and nonprofit organizations around the world supporting human rights, democracy, and local communities. You can read last week’s blog and Radar dashboard that provide a snapshot of what public interest organizations experience on a daily basis when it comes to keeping their websites online.
Origins of Project Galileo
We’ve admitted before that Project Galileo was born out of a mistake, but it’s worth reminding ourselves. In 2014, when Cloudflare was a much smaller company with a smaller network, our free service did not include DDoS mitigation. If a free customer came under a withering attack, we would stop proxying traffic to protect our own network. It just made sense.
One evening, a site that was using us came under a significant DDoS attack, exhausting Cloudflare resources. After pulling up the site and seeing Cyrillic writing and pictures of men with guns, the young engineer on call followed the playbook. He pushed a button and sent all the attack traffic to the site’s origin, effectively kicking it off the Internet.
This was in 2014, during Russia’s first invasion into Ukraine, when Russia invaded Crimea. What the engineer did not know was that he had just kicked off an independent Ukrainian newspaper that was covering the attack and the invasions. The newspaper had tried to pay for services with a credit card but failed because Russia had targeted Ukraine’s financial infrastructure, taking banking institutions offline. It wasn’t the engineer’s fault. He had no reason to know that the site was important, and no alternative playbook to follow.
After that incident, we vowed to never let an organization that was serving such an important purpose go offline simply because they couldn’t pay for services. And so the idea for Project Galileo was born.
Although the idea of providing free security services was straightforward, figuring out which organizations are important enough to deserve such services was not. We know we can’t build a better Internet alone – it’s why Cloudflare’s mission is to help build a better Internet. So with Project Galileo, we sought the assistance of a group of civil society organizations to partner with us and help identify the organizations that need our protection.
Repression of ideas that were threatening to authority hardly started with DDoS attacks or the invention of the Internet. We named the effort Project Galileo after the story of Galileo Galilei. Galileo was persecuted in the 1600s for publishing a book concluding that the Earth was not at the center of the universe, but that the Earth orbits the sun. After Galileo was labeled a heretic, his book was banned and his ideas were suppressed for more than 100 years.
Four hundred years after Galileo, we see attempts to suppress the online voices of journalists and human rights workers who might challenge the status quo. We’re proud of the fact that through Project Galileo, we keep so many of those voices online.
Growth of Project Galileo
Ten years after the launch of Project Galileo, Cloudflare has changed a lot. Our network has grown from data centers in fewer than 30 cities in 2014 to a network that runs in 320 cities and more than 120 countries. We’ve massively expanded our product suite to include whole new lines of products, including a full set of Zero Trust services and a developer suite that enables developers to build a wide range of applications, including AI applications, on our network.
As Cloudflare has grown, so has Project Galileo. We have more than quadrupled the number of entities we protect in the last five years, from 600 at Project Galileo’s five-year anniversary to more than 2,600 today, located in 111 different countries. We’ve expanded from our original 14 civil society partners to 54 today. Our partners span countries, continents, and subject matter areas, sharing their expertise on organizations that would benefit from cybersecurity assistance.
When we expand our product offerings, we routinely ask whether new services would be valuable to the journalists, humanitarian groups, and nonprofits that benefit from Project Galileo. After Cloudflare launched our Zero Trust offering, we announced that we would offer those services for free to participants in Project Galileo to protect themselves against threats like data loss and malware. After Cloudflare acquired Area 1, we announced that we would offer Cloudflare’s email security products for free to the same participants.
We’ve tried to make our products easy for a small organization to use, building a Social Impact Portal and a Zero Trust roadmap for civil society and at-risk communities. Cloudflare’s teams also help participants onboard and troubleshoot when they face challenges.
What Project Galileo means for civil society groups now
On June 6, we celebrated Project Galileo’s 10-year anniversary with partners from government, civil society, and industry at an event in Washington, DC. We used the opportunity to talk about the future of the Internet, and how we can all work together to protect and advance the free and open Internet.
For humanitarian organizations with few resources, the types of services offered under Project Galileo can be life changing. At our Project Galileo event, we heard the story of a small French nonprofit that lost 17 years of data after being targeted by ransomware. Our resources help organizations defend themselves not only against nation states determined to take them offline, but also against common ransomware and phishing attacks.
During our event, the President of the National Endowment for Democracy (NED) told the story of traveling in the Western Balkans where the struggle for an independent media is palpable. NED is a strong supporter of media outlets across the region. But those media outlets come under frequent cyber attacks that have incapacitated their websites. As described by Damon Wilson:
Those attacks prevent news from reaching the public, where information is very much something that is used and weaponized against communities across Bosnia. And this was precisely the case with one of our partners, Buka. It’s a news outlet that’s based in Banja Luka and Republika Srpska. And while I was there, I met with some of our partners from Banja Luka who had been physically beaten up and intimidated. There’s a crackdown on civil society, new restrictions and laws against them. But for Buka, it was a little bit of a different scenario because earlier this year they suffered a DDoS attack, during which their server servers were overwhelmed by up to 700 million page requests. And the sheer volume suggests the attackers had significant resources, making it a particularly severe threat.
But by onboarding Buka into Project Galileo, we were able to help them restore their site’s functionality, and now Buka’s website is equipped to withstand even the most sophisticated attacks, ensuring that their critical reporting continues uninterrupted, exactly at the time when the Republic gets Covid, Republika Srpska government is looking to close and restrict independent civic voices in that part of Bosnia.
And this is just one example. Last week, traveling in Bosnia, of the numerous NED partners who’ve benefited from Cloudflare’s Project Galileo since NED became a partner in 2019, it’s profound to the efficacy of our partners’ work. It effectively ensures that bad actors can’t silence the voices and the work of democracy advocates and independent media around the world.
The importance of collaboration
Our work with Project Galileo highlights the power of the partnerships that we’ve built, not only with civil society, but with government and industry partners as well. By working together, we can expand protections for the many at-risk organizations that need cybersecurity assistance. Cybersecurity is a team sport.
In 2023, one of our Project Galileo partners, the CyberPeace Institute, approached us about doing even more to help protect nonprofit organizations against phishing attacks. The CyberPeace Institute collaborates with its partners to reduce the harms from cyberattacks on people’s lives worldwide and provide them assistance. CyberPeace also analyzes cyberattacks to expose their societal impact, to demonstrate how international laws and norms are being violated, and to advance responsible behavior in cyberspace.
CyberPeace realized that there was an opportunity to document attacks against civil society groups and improve the ecosystem for everyone. Many development and humanitarian organizations are small, with limited staff and little cybersecurity experience. They can easily fall prey to common cyber attacks – like phishing – designed to access their systems or steal their data. If they manage to use tools effectively to defend themselves, they do not typically report on the information about the attacks they see.
CyberPeace proposed to help onboard development and humanitarian organizations to Cloudflare services through their CyberPeace Builders program and analyze the phishing campaigns targeting those organizations. The substantive insights and information gained from that work could then be fed to other civil society organizations as real time security alerts. Cloudflare worked with CyberPeace to develop the new approach, enabling their volunteers to onboard organizations in their network to Area 1 tools and their analysts to access threat indicators from the collective organizations onboarded.
Government can play an important role in helping protect civil society from cyberattacks as well. Since the Summit for Democracy last year, Cloudflare has been working closely with the Joint Cyber Defense Collaborative (JCDC), which is run by the U.S. Cybersecurity and Infrastructure Security Agency (CISA), on their High-Risk Communities initiative. Earlier this year, JCDC launched a web page outlining cybersecurity resources for civil society communities facing digital security threats because of their work. The effort includes tools and services that nonprofits can use to secure themselves online, including those offered under Project Galileo.
Expanding Cloudflare’s Impact
In many ways, the creation of Project Galileo altered the trajectory of the company. Project Galileo cemented the idea that protecting and keeping important organizations online, regardless of whether they could pay us, was part of Cloudflare’s DNA. It pushed us to innovate to improve security not only for the large enterprises that pay us, but for the small organizations doing good for the world that cannot afford to pay for the latest technological innovation. It gave us our mission – to help build a better Internet – and a standard to live up to and measure ourselves against.
To meet that standard, we routinely reach out to offer our services to important organizations in need. In 2022, after Russia’s invasion of Ukraine, Cloudflare jumped in to offer services to Ukrainian critical infrastructure facing a barrage of cyberattacks and have continued providing them services ever since. At our Project Galileo event, the State Department’s Special Envoy and Coordinator for Digital FreedomEnvoy read an email she’d received from Ukraine’s Deputy Foreign Minister and Chief Digital Transformation officer of Ukraine the night before:
It is absolutely definite that Cloudflare services provide a vital layer of cybersecurity within the Ukrainian segment of cyberspace. Numerous DDoS attacks are directed at state electronic services, fintech, official information sources. So if there was no Cloudflare as a proven protection against DDoS attacks, it would have serious consequences causing chaos, especially when these attacks are synchronized by the enemy in parallel with kinetic attacks.
We’ve launched sections of Cloudflare Radar designed to use Cloudflare’s network to help civil society monitor Internet outages and disruptions, as well as route hijacks and other traffic anomalies. We’ve participated in the Freedom Online Coalition’s Task Force on Internet Shutdowns.
Project Galileo also helped pave the way for a variety of Cloudflare projects to provide other at-risk populations free services. These programs include:
Athenian Project: Launched in 2017, the Athenian Project is Cloudflare’s program to protect election-related domains for state and local governments so that citizens have reliable access to information on voter registration, polling places, and the reporting of election results.
Cloudflare for Campaigns: Launched in 2020, Cloudflare for Campaigns helps secure US political candidates’ election websites and internal data while also ensuring site reliability during peak traffic periods. The program is run in partnership with Defending Digital Campaigns.
Project Pangea: Launched in 2021, Project Pangea is a program to provide secure, performant and reliable access to the Internet for community networks that support underserved communities.
Project Safekeeping: Launched in 2022, Project Safekeeping supports at-risk critical infrastructure entities in Australia, Japan, Germany, Portugal, and the UK by providing Zero Trust and application security solutions.
Project Cybersafe Schools: Launched in 2023, Project Cybersafe Schools equips small public school districts in the US with Zero Trust services, including email protection and DNS filtering.
Project Secure Health: Launched on June 10, 2024, Project Secure Health provides security tools to Australia’s general practitioner clinics to safeguard patient data and counter challenges such as data breaches, ransomware attacks, phishing scams, and insider threats.
Looking forward
The world has only gotten more complicated since we first launched Project Galileo in 2014. We face real challenges ranging from malicious cyber actors targeting critical infrastructure, to election interference, to data theft. Governments have responded with increasingly aggressive attempts to control aspects of the Internet. At our recent celebration of Project Galileo, we lamented the thirteenth consecutive year of decline of global Internet freedom, as documented by our Project Galileo partner Freedom House.
But one thing has not changed. We continue to believe the single, global Internet is a miracle that we should all be fighting for. We sometimes forget that the Internet is an incredibly radical concept. The world somehow came together over the last 40 years, agreed on a set of standards, and then made it so that a collection of networks could all exchange data. And that miracle that is the Internet has brought incredible opportunities for the voices of civil society to be heard, to help extend their impact, to spread their message, and to keep them connected.
Connecting everyone online in a permissionless way comes with real harms and real risks. But we need to be surgical as we address those challenges. We need to partner to find solutions that preserve the open Internet, much as we do with projects like Project Galileo. Even if we are at a moment of democratic decline, continuing to defend the open, interoperable Internet preserves space and capacity for a future in which the Internet can also fuel greater freedom.
The openSUSE
Leap 15.6 release is available; this is intended to be the last
Leap 15.x release before Leap 16 comes out.
“Leap 15.6 is projected to receive maintenance and security updates
until the end of 2025 to ensure sufficient overlap with the next
release“. Changes include the addition of the Cockpit server-management tool, a
6.4 kernel, GNOME 45, and many other upgrades. This release also
removes a long list of unmaintained Python packages. See the
release notes for details.
Public polling is a critical function of modern political campaigns and movements, but it isn’t what it once was. Recent US election cycles have producedcopiouspostmortems explaining both the successes and the flaws of public polling. There are two main reasons polling fails.
First, nonresponse has skyrocketed. It’s radically harder to reach people than it used to be. Few people fill out surveys that come in the mail anymore. Few people answer their phone when a stranger calls. Pew Research reported that 36% of the people they called in 1997 would talk to them, but only 6% by 2018. Pollsters worldwide have faced similar challenges.
Second, people don’t always tell pollsters what they really think. Some hide their true thoughts because they are embarrassed about them. Others behave as a partisan, telling the pollster what they think their party wants them to say—or what they know the other party doesn’t want to hear.
Despite these frailties, obsessive interest in polling nonetheless consumes our politics. Headlines more likely tout the latest changes in polling numbers than the policy issues at stake in the campaign. This is a tragedy for a democracy. We should treat elections like choices that have consequences for our lives and well-being, not contests to decide who gets which cushy job.
Polling Machines?
AI could change polling. AI can offer the ability to instantaneously survey and summarize the expressed opinions of individuals and groups across the web, understand trends by demographic, and offer extrapolations to new circumstances and policy issues on par with human experts. The politicians of the (near) future won’t anxiously pester their pollsters for information about the results of a survey fielded last week: they’ll just ask a chatbot what people think. This will supercharge our access to realtime, granular information about public opinion, but at the same time it might also exacerbate concerns about the quality of this information.
I know it sounds impossible, but stick with us.
Large language models, the AI foundations behind tools like ChatGPT, are built on top of huge corpuses of data culled from the Internet. These are models trained to recapitulate what millions of real people have written in response to endless topics, contexts, and scenarios. For a decade or more, campaigns have trawled social media, looking for hints and glimmers of how people are reacting to the latest political news. This makes asking questions of an AI chatbot similar in spirit to doing analytics on social media, except that they are generative: you can ask them new questions that no one has ever posted about before, you can generate more data from populations too small to measure robustly, and you can immediately ask clarifying questions of your simulated constituents to better understand their reasoning
Researchers and firms are already using LLMs to simulate polling results. Current techniques are based on the ideas of AI agents. An AI agent is an instance of an AI model that has been conditioned to behave in a certain way. For example, it may be primed to respond as if it is a person with certain demographic characteristics and can access news articles from certain outlets. Researchers have set up populations of thousands of AI agents that respond as if they are individual members of a survey population, like humans on a panel that get called periodically to answer questions.
The big difference between humans and AI agents is that the AI agents always pick up the phone, so to speak, no matter how many times you contact them. A political candidate or strategist can ask an AI agent whether voters will support them if they take position A versus B, or tweaks of those options, like policy A-1 versus A-2. They can ask that question of male voters versus female voters. They can further limit the query to married male voters of retirement age in rural districts of Illinois without college degrees who lost a job during the last recession; the AI will integrate as much context as you ask.
What’s so powerful about this system is that it can generalize to new scenarios and survey topics, and spit out a plausible answer, even if its accuracy is not guaranteed. In many cases, it will anticipate those responses at least as well as a human political expert. And if the results don’t make sense, the human can immediately prompt the AI with a dozen follow-up questions.
Making AI agents better polling subjects
When we ran our own experiments in this kind of AI use case with the earliest versions of the model behind ChatGPT (GPT-3.5), we found that it did a fairly good job at replicating human survey responses. The ChatGPT agents tended to match the responses of their human counterparts fairly well across a variety of survey questions, such as support for abortion and approval of the US Supreme Court. The AI polling results had average responses, and distributions across demographic properties such as age and gender, similar to real human survey panels.
Our major systemic failure happened on a question about US intervention in the Ukraine war. In our experiments, the AI agents conditioned to be liberal were predominantly opposed to US intervention in Ukraine and likened it to the Iraq war. Conservative AI agents gave hawkish responses supportive of US intervention. This is pretty much what most political experts would have expected of the political equilibrium in US foreign policy at the start of the decade but was exactly wrong in the politics of today.
This mistake has everything to do with timing. The humans were asked the question after Russia’s full-scale invasion in 2022, whereas the AI model was trained using data that only covered events through September 2021. The AI got it wrong because it didn’t know how the politics had changed. The model lacked sufficient context on crucially relevant recent events.
We believe AI agents can overcome these shortcomings. While AI models are dependent on the data they are trained with, and all the limitations inherent in that, what makes AI agents special is that they can automatically source and incorporate new data at the time they are asked a question. AI models can update the context in which they generate opinions by learning from the same sources that humans do. Each AI agent in a simulated panel can be exposed to the same social and media news sources as humans from that same demographic before they respond to a survey question. This works because AI agents can follow multi-step processes, such as reading a question, querying a defined database of information (such as Google, or the New York Times, or Fox News, or Reddit), and then answering a question.
In this way, AI polling tools can simulate exposing their synthetic survey panel to whatever news is most relevant to a topic and likely to emerge in each AI agent’s own echo chamber. And they can query for other relevant contextual information, such as demographic trends and historical data. Like human pollsters, they can try to refine their expectations on the basis of factors like how expensive homes are in a respondent’s neighborhood, or how many people in that district turned out to vote last cycle.
Likely use cases for AI polling
AI polling will be irresistible to campaigns, and to the media. But research is already revealing when and where this tool will fail. While AI polling will always have limitations in accuracy, that makes them similar to, not different from, traditional polling. Today’s pollsters are challenged to reach sample sizes large enough to measure statistically significant differences between similar populations, and the issues of nonresponse and inauthentic response can make them systematically wrong. Yet for all those shortcomings, both traditional and AI-based polls will still be useful. For all the hand-wringing and consternation over the accuracy of US political polling, national issue surveys still tend to be accurate to within a few percentage points. If you’re running for a town council seat or in a neck-and-neck national election, or just trying to make the right policy decision within a local government, you might care a lot about those small and localized differences. But if you’re looking to track directional changes over time, or differences between demographic groups, or to uncover insights about who responds best to what message, then these imperfect signals are sufficient to help campaigns and policymakers.
Where AI will work best is as an augmentation of more traditional human polls. Over time, AI tools will get better at anticipating human responses, and also at knowing when they will be most wrong or uncertain. They will recognize which issues and human communities are in the most flux, where the model’s training data is liable to steer it in the wrong direction. In those cases, AI models can send up a white flag and indicate that they need to engage human respondents to calibrate to real people’s perspectives. The AI agents can even be programmed to automate this. They can use existing survey tools—with all their limitations and latency—to query for authentic human responses when they need them.
This kind of human-AI polling chimera lands us, funnily enough, not too distant from where survey research is today. Decades of social science research has led to substantial innovations in statistical methodologies for analyzing survey data. Current polling methods already do substantial modeling and projecting to predictively model properties of a general population based on sparse survey samples. Today, humans fill out the surveys and computers fill in the gaps. In the future, it will be the opposite: AI will fill out the survey and, when the AI isn’t sure what box to check, humans will fill the gaps. So if you’re not comfortable with the idea that political leaders will turn to a machine to get intelligence about which candidates and policies you want, then you should have about as many misgivings about the present as you will the future.
And while the AI results could improve quickly, they probably won’t be seen as credible for some time. Directly asking people what they think feels more reliable than asking a computer what people think. We expect these AI-assisted polls will be initially used internally by campaigns, with news organizations relying on more traditional techniques. It will take a major election where AI is right and humans are wrong to change that.
Today, we’re announcing that a new AWS Region will be coming to Taiwan by early 2025. The new AWS Asia Pacific (Taipei) Region will consist of three Availability Zones at launch, and will give AWS customers in Taiwan the ability to run workloads and store data that must remain in Taiwan.
Each of the Availability Zones will be physically independent of the others in the Region – close enough to support applications that need low latency, yet sufficiently distant to significantly reduce the risk that an event at an Availability Zone level might impact business continuity.
The Availability Zones in this Region will be connected together through high-bandwidth, low-latency network connections over dedicated, fully redundant fiber. This connectivity supports applications that need synchronous replication between Availability Zones for availability or redundancy. You can take a peek at the AWS Global Infrastructure page to learn more about how we design and build Regions and Availability Zones.
AWS in Taiwan AWS has been investing and supporting customers and partners in Taiwan for more than 10 years. To support our customers in Taiwan, we have business development teams, solutions architects, partner managers, professional services consultants, support staff, and various other roles working in our Taipei office.
Other AWS infrastructure includes two Amazon CloudFront edge locations along with access to the AWS global backbone through multiple redundant submarine cables. You can access any other AWS Region (except Beijing and Ningxia) from AWS Direct Connect locations in Taipei, operated by Chief Telecom and Chunghwa Telecom. With AWS Direct Connect, your data that would have previously been transported over the internet is delivered through a private network connection between your facilities and AWS.
You can also use AWS Outposts in Taiwan, a family of fully managed solutions delivering AWS infrastructure and services to virtually any on-premises or edge location for a truly consistent hybrid experience. With AWS Local Zones in Taipei, you can deliver applications that require single-digit millisecond latency to end users.
AWS continues to invest in upskilling students, local developers and technical professionals, nontechnical professionals, and the next generation of IT leaders in Taiwan through offerings like AWS Academy, AWS Educate, and AWS Skill Builder. Since 2017, AWS has trained more than eight million people across the Asia Pacific-Japan region on cloud skills, including more than 100,000 people in Taiwan.
To learn more, join AWS Summit 2024 Taiwan in July; in-person events that bring the cloud computing community together to connect, collaborate, and learn about AWS.
AWS customers in Taiwan AWS customers in Taiwan have been increasingly moving their applications to AWS and running their technology infrastructure in other AWS Regions around the world. With the addition of this new AWS Region, customers will be able to provide even lower latency to end users and use advanced technologies such as generative artificial intelligence (generative AI), Internet of Things (IoT), mobile services, and more, to drive innovation. This Region will give AWS customers the ability to run their workloads and store their content in Taiwan.
Here are some examples of customers using AWS to drive innovation:
Chunghwa Telecom is the largest integrated telecom provider in Taiwan. To improve AI data security and governance, they use Amazon Bedrock for a variety of generative AI applications, including automatically generating specifications documents for the software development lifecycle and crafting custom marketing campaigns. With Amazon Bedrock, Chunghwa Telecom is saving developer hours and has also developed an immersive, interactive virtual English teacher for the first time.
Gamania Group is a leader in the development and publication of online games in Taiwan. To maximize the value of running on AWS, they worked with AWS Training and Certification to enhance AWS skills across all of its departments, such as AWS Classroom training, AWS Well-Architected Framework, and AWS GameDay events. As a result, they reduced the time needed to make critical operational decisions by 50 percent, lowered its time-to-market by up to 70 percent, and accelerated the launch of new games.
KKCompany Technologies is a multimedia technology group building a music streaming platform, AI-powered streaming solution, and cloud intelligence service in Taiwan. The company specializes in generative AI, multimedia technology, and digital transformation consulting services for enterprises in Taiwan and Japan. You can find BlendVision, a cloud-based streaming solution in AWS Marketplace.
Stay Tuned We’ll announce the opening of this and the other Regions in future blog posts, so be sure to stay tuned! To learn more, visit the AWS Region in Taiwan page in Traditional Chinese.
A while back, I wrote about using the SSH agent protocol to satisfy WebAuthn requests. The main problem with this approach is that it required starting the SSH agent with a special argument and also involved being a little too friendly with the implementation – things worked because I could provide an arbitrary public key and the implementation never validated that, but it would be legitimate for it to start doing so and then break everything. And it also only worked for keys stored on tokens that ssh supports – there was no way to extend this to other keystores on the client (such as the Secure Enclave on Macs, or TPM-backed keys on PCs). I wanted a better solution.
It turns out that it was far easier than I expected. The ssh agent protocol is documented here, and the interesting part is the extension support extension mechanism. Basically, you can declare an extension and then just tunnel whatever you want over it. As before, my goto was the go ssh agent package which conveniently implements both the client and server side of this. Implementing the local agent is trivial – look up SSH_AUTH_SOCK, connect to it, create a new agent client that can communicate with that by calling NewClient, and then implement the ExtendedAgent interface, create a new socket, and call ServeAgent against that. Most of the ExtendedAgent functions should simply call through to the original agent, with the exception of Extension(). Just add a case statement against extensionType, define some reasonably namespaced extension, and you’re done.
Now you need to use this agent. You probably don’t want to use this for arbitrary hosts (agent forwarding should only be enabled for remote systems you trust, not arbitrary machines you connect to – if you enabled agent forwarding for github and github got compromised, github would be able to use any private keys loaded into your agent, and you probably don’t want that). So the right approach is to add a Host entry to the ssh config with a ForwardAgent stanza pointing at the socket you created in your new agent. This way the configured subset of remote hosts will automatically talk to this new custom agent, while forwarding for anything else will still be at the user’s discretion.
For the remote end things are even easier. Look up SSH_AUTH_SOCK and call NewClient as before, and then simply call client.Extension(). Whatever you stick in the contents argument will simply end up being received at the client end. You now have a communication channel between a the remote system and the local client, and what you do with that is up to you. I’m using it to allow a remote system to obtain auth tokens from Okta and forward WebAuthn challenges that can either be satisfied via a local WebAuthn token or by passing the query off to Mac TouchID, but there’s fundamentally no constraints whatsoever on what can be done here.
(If you want to do this on Windows and still have everything work with existing clients you’ll need to take this into account – Windows didn’t really do Unix sockets until recently so everything there is awful)
Yesterday evening we released systemd v256 into the wild. While other projects, such as Firefox
are just about to leave the 7bit world and enter 8bit territory, we already
entered 9bit version territory! For details about the release, see our
announcement
mail.
In the weeks leading up to this release I have posted a series of serieses of
posts to Mastodon about key new features in this release. Mastodon has its
goods and its bads. Among the latter is probably that it isn’t that great for
posting listings of serieses of posts. Hence let me provide you with a list of
the relevant first post in the series of posts here:
I intend to do a similar series of serieses of posts for the next systemd
release (v257), hence if you haven’t left tech Twitter for Mastodon yet, now is
the opportunity.
The extensible scheduler class
(“sched_ext”) framework allows the writing of CPU schedulers as a set of
BPF programs. It has been somewhat
controversial, and its merging into the kernel has been blocked despite
a clear level of interest from users.
Linus Torvalds has now let
it be known that he has made a decision and, overriding the scheduler
maintainer, will merge sched_ext for the 6.11 release.
I honestly see no reason to delay this any more. This whole
patchset was the major (private) discussion at last year’s kernel
maintainer summit, and I don’t find any value in having the same
discussion (whether off-list or as an actual event) at the upcoming
maintainer summit one year later, so to make any kind of sane
progress, my current plan is to merge this for 6.11.
It’s June 2024 Patch Tuesday. Microsoft is addressing 51 vulnerabilities today, and has evidence of public disclosure for just a single one of those. At time of writing, none of the vulnerabilities published today are listed on CISA KEV, although this is always subject to change. Microsoft is patching a single critical remote code execution (RCE) vulnerability today. Seven browser vulnerabilities were published separately this month, and are not included in the total.
MSMQ: critical RCE
The sole critical RCE patched today is CVE-2024-30080 for all current versions of Windows. Exploitation requires that an attacker send a specially crafted malicious packet to an MSMQ server, which Patch Tuesday watchers will know as a perennial source of vulnerabilities. As usual, Microsoft points out that the Windows message queuing service is not enabled by default; as usual, Rapid7 notes that a number of applications – including Microsoft Exchange – quietly introduce MSMQ as part of their own installation routine. As is typical of MSMQ RCE vulnerabilities, CVE-2024-30080 receives a high CVSSv3 base score due to the network attack vector, low attack complexity, and lack of required privileges. Code execution is presumably in a SYSTEM context, although the advisory does not specify.
Office: malicious file RCEs
Microsoft Office receives patches for a pair of RCE-via-malicious-file vulnerabilities. CVE-2024-30101 is a vulnerability in Outlook; although the Preview Pane is a vector, the user must subsequently perform unspecified specific actions to trigger the vulnerability and the attacker must win a race condition. On the other hand, CVE-2024-30104 does not have the Preview Pane as a vector, but nevertheless ends up with a slightly higher CVSS base score of 7.8, since exploitation relies solely on the user opening a malicious file.
SharePoint: RCE
This month also brings a patch for SharePoint RCE CVE-2024-30100. The advisory is sparing on details, and the context of code exploitation is not clear. The weakness is described as CWE-426: Untrusted Search Path; many (but not all) vulnerabilities associated with CWE-426 lead to elevation of privilege.
DNSSEC NSEC3: CPU exhaustion DoS
And now for something completely different: CVE-2023-50868, which describes a denial of service vulnerability in DNSSEC. This vulnerability is present in the DNSSEC spec itself, and the CVE was assigned by MITRE on behalf of DNSSEC. Microsoft’s implementation of DNSSEC is thus subject to the same attack as other implementations. An attacker can exhaust CPU resources on a DNSSEC-validating DNS resolver by demanding responses from a DNSSEC-signed zone, if the resolver uses NSEC3 to respond to the request. NSEC3 is designed to provide a safe way for a DNSSEC-validating DNS resolver to indicate that a requested resource does not exist. Under certain circumstances, the DNS resolver must perform thousands of iterations of a hash function to calculate an NSEC3 response, and this is the foundation on which this DoS exploit rests. All current versions of Windows Server receive a patch today.
Typically, when Microsoft publishes a security advisory and describes the vulnerability as publicly disclosed, that public disclosure will have been recent. However, in the case of CVE-2023-50868, the flaw in DNSSEC was first publicly disclosed on 2024-02-13. The advisory acknowledges four academics from the German National Research Centre for Applied Cybersecurity (ATHENE), which is perhaps of interest since these same researchers are authors on a March 2024 academic paper that downplays the DoS potential of CVE-2024-50868. Those same researchers published another DNSSEC flaw CVE-2023-50387 (also known as KeyTrap) in January 2024, which they describe as having potentially serious implications; Microsoft patched that one at the next scheduled opportunity in February. The CVE-2023-50868 advisory published today does not provide further insight as to why this vulnerability wasn’t patched sooner; a reasonable assumption might be that Microsoft assesses CVE-2023-50868 as less urgent/critical than CVE-2023-50387, although both receive a rating of Important on Microsoft’s proprietary severity ranking scale. It’s also possible that Microsoft does not wish to be the only major server OS vendor without a patch.
Lifecycle update
There are no significant changes to the lifecycle phase of Microsoft products this month. In July, Microsoft SQL Server 2014 will move past the end of extended support. From August onwards, Microsoft only guarantees to provide SQL Server 2014 security updates to customers who choose to participate in the paid Extended Security Updates program.
Summary Charts
What goes up must come down and/or is an attacker’s privilege level.No spoofing. No security feature bypass. Plenty of elevation of privilege though.
We’re excited to announce that our Europe (London) Region has renewed our accreditation for United Kingdom (UK) Police-Assured Secure Facilities (PASF) for Official-Sensitive data. Since 2017, the Amazon Web Services (AWS) Europe (London) Region has been assured under the PASF program. This demonstrates our continuous commitment to adhere to the heightened expectations of customers with UK law enforcement workloads. Our UK law enforcement customers who require PASF can continue to run their applications in the PASF-assured Europe (London) Region in confidence.
The PASF is a long-established assurance process, used by UK law enforcement, as a method for assuring the security of facilities such as data centers or other locations that house critical business applications that process or hold police data. PASF consists of a control set of security requirements, an on-site inspection, and an audit interview with representatives of the facility.
The Police Digital Service (PDS) confirmed the renewal for AWS on May 24, 2024. A letter confirming PASF status from the Police Digital Service (PDS) can be found on AWS Artifact. The UK police force and law enforcement organizations can also obtain confirmation of the compliance status of AWS through the Police Digital Service.
To learn more about our compliance and security programs, see AWS Compliance Programs. As always, we value your feedback and questions; reach out to the AWS Compliance team through the Contact Us page.
Please reach out to your AWS account team if you have questions or feedback about PASF compliance.
If you have feedback about this post, submit comments in the Comments section below.
BPF is in a unique position in terms of security. It runs in a privileged
context, within the kernel, and can have access to many sensitive details of the
kernel’s operation. At the same time, unlike kernel modules, BPF programs aren’t signed.
Additionally, the mechanisms behind BPF present challenges to implementing
signing or other security features. Three nearly back-to-back sessions at the
2024 Linux Storage,
Filesystem, Memory Management, and BPF Summit
addressed some of the potential security problems.
Amazon Web Services (AWS) provides tools that simplify automation and monitoring for compliance with security standards, such as the NIST SP 800-53 Rev. 5 Operational Best Practices. Organizations can set preventative and proactive controls to help ensure that noncompliant resources aren’t deployed. Detective and responsive controls notify stakeholders of misconfigurations immediately and automate fixes, thus minimizing the time to resolution (TTR).
By layering the solutions outlined in this blog post, you can increase the probability that your deployments stay continuously compliant with the National Institute of Standards and Technology (NIST) SP 800-53 security standard, and you can simplify reporting on that compliance. In this post, we walk you through the following tools to get started on your continuous compliance journey:
This post covers quite a few solutions, and these solutions operate in different parts of the security pillar of the AWS Well-Architected Framework. It might take some iterations to get your desired results, but we encourage you to start small, find your focus areas, and implement layered iterative changes to address them.
For example, if your organization has experienced events involving public Amazon Simple Storage Service (Amazon S3) buckets that can lead to data exposure, focus your efforts across the different control types to address that issue first. Then move on to other areas. Those steps might look similar to the following:
Use Security Hub and Prowler to find your public buckets and monitor patterns over a predetermined time period to discover trends and perhaps an organizational root cause.
Apply IAM policies and SCPs to specific organizational units (OUs) and principals to help prevent the creation of public buckets and the changing of AWS account-level controls.
Set up Automated Security Response (ASR) on AWS and then test and implement the automatic remediation feature for only S3 findings.
Remove direct human access to production accounts and OUs. Require infrastructure as code (IaC) to pass through a pipeline where CloudFormation Guard scans IaC for misconfigurations before deployment into production environments.
Detective controls
Implement your detective controls first. Use them to identify misconfigurations and your priority areas to address. Detective controls are security controls that are designed to detect, log, and alert after an event has occurred. Detective controls are a foundational part of governance frameworks. These guardrails are a second line of defense, notifying you of security issues that bypassed the preventative controls.
Security Hub NIST SP 800-53 security standard
Security Hub consumes, aggregates, and analyzes security findings from various supported AWS and third-party products. It functions as a dashboard for security and compliance in your AWS environment. Security Hub also generates its own findings by running automated and continuous security checks against rules. The rules are represented by security controls. The controls might, in turn, be enabled in one or more security standards. The controls help you determine whether the requirements in a standard are being met. Security Hub provides controls that support specific NIST SP 800-53 requirements. Unlike other frameworks, NIST SP 800-53 isn’t prescriptive about how its requirements should be evaluated. Instead, the framework provides guidelines, and the Security Hub NIST SP 800-53 controls represent the service’s understanding of them.
Using this step-by-step guide, enable Security Hub for your organization in AWS Organizations. Configure the NIST SP 800-53 security standard for all accounts, in all AWS Regions that are required to be monitored for compliance, in your organization by using the new centralized configuration feature; or if your organization uses AWS GovCloud (US), by using this multi-account script. Use the findings from the NIST SP 800-53 security standard in your delegated administrator account to monitor NIST SP 800-53 compliance across your entire organization, or a list of specific accounts.
Figure 1 shows the Security Standard console page, where users of the Security Hub Security Standard feature can see an overview of their security score against a selected security standard.
Figure 1: Security Hub security standard console
On this console page, you can select each control that is checked by a Security Hub Security Standard, such as the NIST 800-53 Rev. 5 standard, to find detailed information about the check and which NIST controls it maps to, as shown in Figure 2.
Figure 2: Security standard check detail
After you enable Security Hub with the NIST SP 800-53 security standard, you can link responsive controls such as the Automated Security Response (ASR), which is covered later in this blog post, to Amazon EventBridge rules to listen for Security Hub findings as they come in.
Prowler
Prowler is an open source security tool that you can use to perform assessments against AWS Cloud security recommendations, along with audits, incident response, continuous monitoring, hardening, and forensics readiness. The tool is a Python script that you can run anywhere that an up-to-date Python installation is located—this could be a workstation, an Amazon Elastic Compute Cloud (Amazon EC2) instance, AWS Fargate or another container, AWS CodeBuild, AWS CloudShell, AWS Cloud9, or another compute option.
Figure 3 shows Prowler being used to perform a scan.
Figure 3: Prowler CLI in action
Prowler works well as a complement to the Security Hub NIST SP 800-53 Rev. 5 security standard. The tool has a native Security Hub integration and can send its findings to your Security Hub findings dashboard. You can also use Prowler as a standalone compliance scanning tool in partitions where Security Hub or the security standards aren’t yet available.
At the time of writing, Prowler has over 300 checks across 64 AWS services.
In addition to integrations with Security Hub and computer-based outputs, Prowler can produce fully interactive HTML reports that you can use to sort, filter, and dive deeper into findings. You can then share these compliance status reports with compliance personnel. Some organizations run automatically recurring Prowler reports and use Amazon Simple Notification Service (Amazon SNS) to email the results directly to their compliance personnel.
Get started with Prowler by reviewing the Prowler Open Source documentation that contains tutorials for specific providers and commands that you can copy and paste.
Preventative controls
Preventative controls are security controls that are designed to prevent an event from occurring in the first place. These guardrails are a first line of defense to help prevent unauthorized access or unwanted changes to your network. Service control policies (SCPs) and IAM controls are the best way to help prevent principals in your AWS environment (whether they are human or nonhuman) from creating noncompliant or misconfigured resources.
IAM
In the ideal environment, principals (both human and nonhuman) have the least amount of privilege that they need to reach operational objectives. Ideally, humans would at the most only have read-only access to production environments. AWS resources would be created through IaC that runs through a DevSecOps pipeline where policy-as-code checks review resources for compliance against your policies before deployment. DevSecOps pipeline roles should have IAM policies that prevent the deployment of resources that don’t conform to your organization’s compliance strategy. Use IAM conditions wherever possible to help ensure that only requests that match specific, predefined parameters are allowed.
The following policy is a simple example of a Deny policy that uses Amazon Relational Database Service (Amazon RDS) condition keys to help prevent the creation of unencrypted RDS instances and clusters. Most AWS services support condition keys that allow for evaluating the presence of specific service settings. Use these condition keys to help ensure that key security features, such as encryption, are set during a resource creation call.
You can use an SCP to specify the maximum permissions for member accounts in your organization. You can restrict which AWS services, resources, and individual API actions the users and roles in each member account can access. You can also define conditions for when to restrict access to AWS services, resources, and API actions. If you haven’t used SCPs before and want to learn more, see How to use service control policies to set permission guardrails across accounts in your AWS Organization.
Use SCPs to help prevent common misconfigurations mapped to NIST SP 800-53 controls, such as the following:
Prevent governed accounts from leaving the organization or turning off security monitoring services.
Build protections and contextual access controls around privileged principals.
Mitigate the risk of data mishandling by enforcing data perimeters and requiring encryption on data at rest.
Although SCPs aren’t the optimal choice for preventing every misconfiguration, they can help prevent many of them. As a feature of AWS Organizations, SCPs provide inheritable controls to member accounts of the OUs that they are applied to. For deployments in Regions where AWS Organizations isn’t available, you can use IAM policies and permissions boundaries to achieve preventative functionality that is similar to what SCPs provide.
The following is an example of policy mapping statements to NIST controls or control families. Note the placeholder values, which you will need to replace with your own information before use. Note that the SIDs map to Security Hub NIST 800-53 Security Standard control numbers or NIST control families.
For a collection of SCP examples that are ready for your testing, modification, and adoption, see the service-control-policy-examples GitHub repository, which includes examples of Region and service restrictions.
You should thoroughly test SCPs against development OUs and accounts before you deploy them against production OUs and accounts.
Proactive controls
Proactive controls are security controls that are designed to prevent the creation of noncompliant resources. These controls can reduce the number of security events that responsive and detective controls handle. These controls help ensure that deployed resources are compliant before they are deployed; therefore, there is no detection event that requires response or remediation.
CloudFormation Guard
CloudFormation Guard (cfn-guard) is an open source, general-purpose, policy-as-code evaluation tool. Use cfn-guard to scan Information as Code (IaC) against a collection of policies, defined as JSON, before deployment of resources into an environment.
Cfn-guard can scan CloudFormation templates, Terraform plans, Kubernetes configurations, and AWS Cloud Development Kit (AWS CDK) output. Cfn-guard is fully extensible, so your teams can choose the rules that they want to enforce, and even write their own declarative rules in a YAML-based format. Ideally, the resources deployed into a production environment on AWS flow through a DevSecOps pipeline. Use cfn_guard in your pipeline to define what is and is not acceptable for deployment, and help prevent misconfigured resources from deploying. Developers can also use cfn_guard on their local command line, or as a pre-commit hook to move the feedback timeline even further “left” in the development cycle.
Use policy as code to help prevent the deployment of noncompliant resources. When you implement policy as code in the DevOps cycle, you can help shorten the development and feedback cycle and reduce the burden on security teams. The CloudFormation team maintains a GitHub repo of cfn-guard rules and mappings, ready for rapid testing and adoption by your teams.
Figure 4 shows how you can use Guard with the NIST 800-53 cfn_guard Rule Mapping to scan infrastructure as code against NIST 800-53 mapped rules.
Figure 4: CloudFormation Guard scan results
You should implement policy as code as pre-commit checks so that developers get prompt feedback, and in DevSecOps pipelines to help prevent deployment of noncompliant resources. These checks typically run as Bash scripts in a continuous integration and continuous delivery (CI/CD) pipeline such as AWS CodeBuild or GitLab CI. To learn more, see Integrating AWS CloudFormation Guard into CI/CD pipelines.
Many other third-party policy-as-code tools are available and include NIST SP 800-53 compliance policies. If cfn-guard doesn’t meet your needs, or if you are looking for a more native integration with the AWS CDK, for example, see the NIST-800-53 rev 5 rules pack in cdk-nag.
Responsive controls
Responsive controls are designed to drive remediation of adverse events or deviations from your security baseline. Examples of technical responsive controls include setting more stringent security group rules after a security group is created, setting a public access block on a bucket automatically if it’s removed, patching a system, quarantining a resource exhibiting anomalous behavior, shutting down a process, or rebooting a system.
Automated Security Response on AWS
The Automated Security Response on AWS (ASR) is an add-on that works with Security Hub and provides predefined response and remediation actions based on industry compliance standards and current recommendations for security threats. This AWS solution creates playbooks so you can choose what you want to deploy in your Security Hub administrator account (which is typically your Security Tooling account, in our recommended multi-account architecture). Each playbook contains the necessary actions to start the remediation workflow within the account holding the affected resource. Using ASR, you can resolve common security findings and improve your security posture on AWS. Rather than having to review findings and search for noncompliant resources across many accounts, security teams can view and mitigate findings from the Security Hub console of the delegated administrator.
The architecture diagram in Figure 5 shows the different portions of the solution, deployed into both the Administrator account and member accounts.
Figure 5: ASR architecture diagram
The high-level process flow for the solution components deployed with the AWS CloudFormation template is as follows:
Detect – AWS Security Hub provides customers with a comprehensive view of their AWS security state. This service helps them to measure their environment against security industry standards and best practices. It works by collecting events and data from other AWS services, such as AWS Config, Amazon GuardDuty, and AWS Firewall Manager. These events and data are analyzed against security standards, such as the CIS AWS Foundations Benchmark. Exceptions are asserted as findings in the Security Hub console. New findings are sent as Amazon EventBridge events.
Initiate – You can initiate events against findings by using custom actions, which result in Amazon EventBridge events. Security Hub Custom Actions and EventBridge rules initiate Automated Security Response on AWS playbooks to address findings. One EventBridge rule is deployed to match the custom action event, and one EventBridge event rule is deployed for each supported control (deactivated by default) to match the real-time finding event. Automated remediation can be initiated through the Security Hub Custom Action menu, or, after careful testing in a non-production environment, automated remediations can be activated. This can be activated per remediation—it isn’t necessary to activate automatic initiations on all remediations.
Orchestrate – Using cross-account IAM roles, Step Functions in the admin account invokes the remediation in the member account that contains the resource that produced the security finding.
Log – The playbook logs the results to an Amazon CloudWatch Logs group, sends a notification to an Amazon SNS topic, and updates the Security Hub finding. An audit trail of actions taken is maintained in the finding notes. On the Security Hub dashboard, the finding workflow status is changed from NEW to either NOTIFIED or RESOLVED. The security finding notes are updated to reflect the remediation that was performed.
The NIST SP 800-53 Playbook contains 52 remediations to help security and compliance teams respond to misconfigured resources. Security teams have a choice between launching these remediations manually, or enabling the associated EventBridge rules to allow the automations to bring resources back into a compliant state until further action can be taken on them. When a resource doesn’t align with the Security Hub NIST SP 800-53 security standard automated checks and the finding appears in Security Hub, you can use ASR to move the resource back into a compliant state. Remediations are available for 17 of the common core services for most AWS workloads.
Figure 6 shows how you can remediate a finding with ASR by selecting the finding in Security Hub and sending it to the created custom action.
Figure 6: ASR Security Hub custom action
Findings generated from the Security Hub NIST SP 800-53 security standard are displayed in the Security Hub findings or security standard dashboards. Security teams can review the findings and choose which ones to send to ASR for remediation. The general architecture of ASR consists of EventBridge rules to listen for the Security Hub custom action, an AWS Step Functions workflow to control the process and implementation, and several AWS Systems Manager documents (SSM documents) and AWS Lambda functions to perform the remediation. This serverless, step-based approach is a non-brittle, low-maintenance way to keep persistent remediation resources in an account, and to pay for their use only as needed. Although you can choose to fork and customize ASR, it’s a fully developed AWS solution that receives regular bug fixes and feature updates.
To get started, see the ASR Implementation Guide, which will walk you through configuration and deployment.
Several options are available to concisely gather results into digestible reports that compliance professionals can use as artifacts during the Risk Management Framework (RMF) process when seeking an Authorization to Operate (ATO). By automating reporting and delegating least-privilege access to compliance personnel, security teams may be able to reduce time spent reporting compliance status to auditors or oversight personnel.
Let your compliance folks in
Remove some of the burden of reporting from your security engineers, and give compliance teams read-only access to your Security Hub dashboard in your Security Tooling account. Enabling compliance teams with read-only access through AWS IAM Identity Center (or another sign-on solution) simplifies governance while still maintaining the principle of least privilege. By adding compliance personnel to the AWSSecurityAudit managed permission set in IAM Identity Center, or granting this policy to IAM principals, these users gain visibility into operational accounts without the ability to make configuration changes. Compliance teams can self-serve the security posture details and audit trails that they need for reporting purposes.
Meanwhile, administrative teams are freed from regularly gathering and preparing security reports, so they can focus on operating compliant workloads across their organization. The AWSSecurityAudit permission set grants read-only access to security services such as Security Hub, AWS Config, Amazon GuardDuty, and AWS IAM Access Analyzer. This provides compliance teams with wide observability into policies, configuration history, threat detection, and access patterns—without the privilege to impact resources or alter configurations. This ultimately helps to strengthen your overall security posture.
For more information about AWS managed policies, such as the AWSSecurityAudit managed policy, see the AWS managed policies.
To learn more about permission sets in IAM Identity Center, see Permission sets.
AWS Audit Manager
AWS Audit Manager helps you continually audit your AWS usage to simplify how you manage risk and compliance with regulations and industry standards. Audit Manager automates evidence collection so you can more easily assess whether your policies, procedures, and activities—also known as controls—are operating effectively. When it’s time for an audit, Audit Manager helps you manage stakeholder reviews of your controls. This means that you can build audit-ready reports with much less manual effort.
Audit Manager provides prebuilt frameworks that structure and automate assessments for a given compliance standard or regulation, including NIST 800-53 Rev. 5. Frameworks include a prebuilt collection of controls with descriptions and testing procedures. These controls are grouped according to the requirements of the specified compliance standard or regulation. You can also customize frameworks and controls to support internal audits according to your specific requirements.
For more information about using Audit Manager to generate automated compliance reports, see the AWS Audit Manager User Guide.
Security Hub Compliance Analyzer (SHCA)
Security Hub is the premier security information aggregating tool on AWS, offering automated security checks that align with NIST SP 800-53 Rev. 5. This alignment is particularly critical for organizations that use the Security Hub NIST SP 800-53 Rev. 5 framework. Each control within this framework is pivotal for documenting the compliance status of cloud environments, focusing on key aspects such as:
Related requirements – For example, NIST.800-53.r5 CM-2 and NIST.800-53.r5 CM-2(2)
Severity – Assessment of potential impact
Description – Detailed control explanation
Remediation – Strategies for addressing and mitigating issues
Such comprehensive information is crucial in the accreditation and continuous monitoring of cloud environments.
To further augment the utility of this data for customers seeking to compile artifacts and articulate compliance status, the AWS ProServe team has introduced the Security Hub Compliance Analyzer (SHCA).
SHCA is engineered to streamline the RMF process. It reduces manual effort, delivers extensive reports for informed decision making, and helps assure continuous adherence to NIST SP 800-53 standards. This is achieved through a four-step methodology:
Active findings collection – Compiles ACTIVE findings from Security Hub that are assessed using NIST SP 800-53 Rev. 5 standards.
Results transformation – Transforms these findings into formats that are both user-friendly and compatible with RMF tools, facilitating understanding and utilization by customers.
Data analysis and compliance documentation – Performs an in-depth analysis of these findings to pinpoint compliance and security shortfalls. Produces comprehensive compliance reports, summaries, and narratives that accurately represent the status of compliance for each NIST SP 800-53 Rev. 5 control.
Findings archival – Assembles and archives the current findings for downloading and review by customers.
The diagram in Figure 7 shows the SHCA steps in action.
Figure 7: SHCA steps
By integrating these steps, SHCA simplifies compliance management and helps enhance the overall security posture of AWS environments, aligning with the rigorous standards set by NIST SP 800-53 Rev. 5.
The following is a list of the artifacts that SHCA provides:
RMF-ready controls – Controls in full compliance (as per AWS Config) with AWS Operational Recommendations for NIST SP 800-53 Rev. 5, ready for direct import into RMF tools.
Controls needing attention – Controls not fully compliant with AWS Operational Recommendations for NIST SP 800-53 Rev. 5, indicating areas that require improvement.
Control compliance summary (CSV) – A detailed summary, in CSV format, of NIST SP 800-53 controls, including their compliance percentages and comprehensive narratives for each control.
Security Hub NIST 800-53 Analysis Summary – This automated report provides an executive summary of the current compliance posture, tailored for leadership reviews. It emphasizes urgent compliance concerns that require immediate action and guides the creation of a targeted remediation strategy for operational teams.
Original Security Hub findings – The raw JSON file from Security Hub, captured at the last time that the SHCA state machine ran.
User-friendly findings summary –A simplified, flattened version of the original findings, formatted for accessibility in common productivity tools.
As shown in Figure 8, the Security Hub NIST 800-53 Analysis Summary adopts an OpenSCAP-style format akin to Security Technical Implementation Guides (STIGs), which are grounded in the Department of Defense’s (DoD) policy and security protocols.
Organizations can use AWS security and compliance services to help maintain compliance with the NIST SP 800-53 standard. By implementing preventative IAM and SCP policies, organizations can restrict users from creating noncompliant resources. Detective controls such as Security Hub and Prowler can help identify misconfigurations, while proactive tools such as CloudFormation Guard can scan IaC to help prevent deployment of noncompliant resources. Finally, the Automated Security Response on AWS can automatically remediate findings to help resolve issues quickly. With this layered security approach across the organization, companies can verify that AWS deployments align to the NIST framework, simplify compliance reporting, and enable security teams to focus on critical issues. Get started on your continuous compliance journey today. Using AWS solutions, you can align deployments with the NIST 800-53 standard. Implement the tips in this post to help maintain continuous compliance.
What happens when you push to GitHub? The answer, “My repository gets my changes” or maybe, “The refs on my remote get updated” is pretty much right—and that is a really important thing that happens, but there’s a whole lot more that goes on after that. To name a few examples:
Pull requests are synchronized, meaning the diff and commits in your pull request reflect your newly pushed changes.
Push webhooks are dispatched.
Workflows are triggered.
If you push an app configuration file (like for Dependabot or GitHub Actions), the app is automatically installed on your repository.
GitHub Pages are published.
Codespaces configuration is updated.
And much, much more.
Those are some pretty important things, and this is just a sample of what goes on for every push. In fact, in the GitHub monolith, there are over 60 different pieces of logic owned by 20 different services that run in direct response to a push. That’s actually really cool—we should be doing a bunch of interesting things when code gets pushed to GitHub. In some sense, that’s a big part of what GitHub is, the place you push code1 and then cool stuff happens.
The problem
What’s not so cool is that, up until recently, all of these things were the responsibility of a single, enormous background job. Whenever GitHub’s Ruby on Rails monolith was notified of a push, it enqueued a massive job called the RepositoryPushJob. This job was the home for all push processing logic, and its size and complexity led to many problems. The job triggered one thing after another in a long, sequential series of steps, kind of like this:
There are a few things wrong with this picture. Let’s highlight some of them:
This job was huge, and hard to retry. The size of the RepositoryPushJob made it very difficult for different push processing tasks to be retried correctly. On a retry, all the logic of the job is repeated from the beginning, which is not always appropriate for individual tasks. For example:
Writing Push records to the database can be retried liberally on errors and reattempted any amount of time after the push, and will gracefully handle duplicate data.
Sending push webhooks, on the other hand, is much more time-sensitive and should not be reattempted too long after the push has occurred. It is also not desirable to dispatch multiples of the same webhook.
Most of these steps were never retried at all. The above difficulties with conflicting retry concerns ultimately led to retries of RepositoryPushJob being avoided in most cases. To prevent one step from killing the entire job, however, much of the push handling logic was wrapped in code catching any and all errors. This lack of retries led to issues where crucial pieces of push processing never occurred.
Tight coupling of many concerns created a huge blast radius for problems. While most of the dozens of tasks in this job rescued all errors, for historical reasons, a few pieces of work in the beginning of the job did not. This meant that all of the later steps had an implicit dependency on the initial parts of the job. As more concerns are combined within the same job, the likelihood of errors impacting the entire job increases.
For example, writing data to our Pushes MySQL cluster occurred in the beginning of the RepositoryPushJob. This meant that everything occurring after that had an implicit dependency on this cluster. This structure led to incidents where errors from this database cluster meant that user pull requests were not synchronized, even though pull requests have no explicit need to connect to this cluster.
A super long sequential process is bad for latency. It’s fine for the first few steps, but what about the things that happen last? They have to wait for every other piece of logic to run before they get a chance. In some cases, this structure led to a second or more of unnecessary latency for user-facing push tasks, including pull request synchronization.
What did we do about this?
At a high level, we took this very long sequential process and decoupled it into many isolated, parallel processes. We used the following approach:
We added a new Kafka topic that we publish an event to for each push.
We examined each of the many push processing tasks and grouped them by owning service and/or logical relationships (for example, order dependency, retry-ability).
For each coherent group of tasks, we placed them into a new background job with a clear owner and appropriate retry configuration.
Finally, we configured these jobs to be enqueued for each publish of the new Kafka event.
To do this, we used an internal system at GitHub that facilitates enqueueing background jobs in response to Kafka events via independent consumers.
We had to make investments in several areas to support this architecture, including:
Creating a reliable publisher for our Kafka event–one that would retry until broker acknowledgement.
Setting up a dedicated pool of job workers to handle the new job queues we’d need for this level of fan out.
Improving observability to ensure we could carefully monitor the flow of push events throughout this pipeline and detect any bottlenecks or problems.
Devising a system for consistent per-event feature flagging, to ensure that we could gradually roll out (and roll back if needed) the new system without risk of data loss or double processing of events between the old and new pipelines.
Now, things look like this:
A push triggers a Kafka event, which is fanned out via independent consumers to many isolated jobs that can process the event without worrying about any other consumers.
Results
A smaller blast radius for problems.
This can be clearly seen from the diagram. Previously, an issue with a single step in the very long push handling process could impact everything downstream. Now, issues with one piece of push handling logic don’t have the ability to take down much else.
Structurally, this decreases the risk of dependencies. For example, there are around 300 million push processing operations executed per day in the new pipeline that previously implicitly depended on the Pushes MySQL cluster and now have no such dependency, simply as a product of being moved into isolated processes.
Decoupling also means better ownership. In splitting up these jobs, we distributed ownership of the push processing code from one owning team to 15+ more appropriate service owners. New push functionality in our monolith can be added and iterated on by the owning team without unintentional impact to other teams.
Pushes are processed with lower latency.
By running these jobs in parallel, no push processing task has to wait for others to complete. This means better latency for just about everything that happens on push.
For example, we can see a notable decrease in pull request sync time:
Improved observability.
By breaking things up into smaller jobs, we get a much clearer picture of what’s going on with each job. This lets us set up observability and monitoring that is much more finely scoped than anything we had before, and helps us to quickly pinpoint any problems with pushes.
Pushes are more reliably processed.
By reducing the size and complexity of the jobs that process pushes, we are able to retry more things than in the previous system. Each job can have retry configuration that’s appropriate for its own small set of concerns, without having to worry about re-executing other, unrelated logic on retry.
If we define a “fully processed” push as a push event for which all the desired operations are completed with no failures, the old RepositoryPushJob system fully processed about 99.897% of pushes.
In the worst-case estimate, the new pipeline fully processes 99.999% of pushes.
Conclusion
Pushing code to GitHub is one of the most fundamental interactions that developers have with GitHub every day. It’s important that our system handles everyone’s pushes reliably and efficiently, and over the past several months we have significantly improved the ability of our monolith to correctly and fully process pushes from our users. Through platform level investments like this one, we strive to make GitHub the home for all developers (and their many pushes!) far into the future.
Notes
People push to GitHub a whole lot, as you can imagine. In the last 30 days, we’ve received around 500 million pushes from 8.5 million users. ↩
By: Nicholas Handy, Director of Technical Alliances & Partnerships at Cado Security
Velociraptor is a robust open-source tool designed for collecting and querying forensic and incident response artifacts across various endpoints. This powerful tool allows incident responders to effortlessly gather data from remote systems, regardless of their location.
Advanced data analysis with the Cado Security Platform
The Cado Security platform is a complementary technology that enables analysis and process of captured data at scale and from multiple sources. In conjunction with Velociraptor data, Cado analyzes data captured from cloud VMs, container-based, serverless, and SaaS environments. The platform automatically scales up and down to provide fast, parallel data processing. This means that it can process hundreds of systems simultaneously.
The Cado Security Platform integrates seamlessly with Velociraptor, creating a comprehensive suite for end-to-end data capture and analysis. In fact, Cado’s existing customers routinely analyze data collected by Velociraptor during investigations using this platform, making the most of its powerful capabilities
Optimized data processing and analysis
A common use case involves users performing offline triage to create an agent to collect Windows.KapeFiles from endpoints, to then upload these to cloud storage where Cado can import, process, and analyze them. This capability leverages Cado’s cloud-based parallel processing to quickly normalize collected artifacts. Cado creates a timeline of what happened on the systems, runs analysis against the files and enables an analyst to search and browse the captured data.
Enhanced threat visibility
The Cado Security Platform creates detailed timelines of system events, conducts thorough file analysis, and enables analysts to search and browse captured data efficiently. This detailed insight is invaluable for understanding the full impact of threats.
With Velociraptor and The Cado Security Platform working together, incident response teams can achieve a better understanding of the impact of threats with complete visibility across their entire ecosystem, enhancing the overall efficiency of forensic investigations and incident response.
The collective thoughts of the interwebz
Manage Consent
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.