Post Syndicated from Explosm.net original https://explosm.net/comics/three-ghosts
New Cyanide and Happiness Comic
Post Syndicated from Explosm.net original https://explosm.net/comics/three-ghosts
New Cyanide and Happiness Comic
Post Syndicated from Екип на Биволъ original https://bivol.bg/nezakonna-sech-sofia-oblast.html

Липсата на реален контрол в горите и кражби за над 1 млн. лв. в Софийска област разкрива първото от серията нови разследвания на “Биволъ” за незаконния дърводобив в партньорство с…
Post Syndicated from Let's Encrypt original https://letsencrypt.org/2023/12/13/ngos/
For more than ten years, we at the nonprofit Internet Security Research Group (ISRG) have been focused on our mission of building a more secure and privacy-respecting Internet for everyone, everywhere. As we touch on in our 2023 Annual Report, we now serve more than 360 million domains with free TLS certificates.
Beyond being a big number, what does that signify? What’s the importance of having TLS being widely adopted anyways? We’ll take a closer look at these questions through the lens of one group of Subscribers we can relate to particularly well: nonprofits.
Let’s Encrypt serves 57% of all websites using the .org top level domain (TLD), which is commonly used by nonprofits. In the US alone there are 1.8M registered nonprofit organizations. And while the focus of these organizations are varied, all of them rely on the Internet in some capacity.
When a nonprofit uses a TLS certificate on their website, it protects their visitors and stakeholders from snoopers, MITM attacks, and surveillance. Without TLS, nonprofits’ content could be changed without their knowledge or their visitors’ private information could be compromised. Access to free and automated TLS via Let’s Encrypt means these nonprofits face as few barriers as possible to adopting TLS.
In short, something as fundamental as security and privacy should be as easy to access as possible. For nonprofits both large and small, Let’s Encrypt makes it easy to provide security and privacy for users of their websites, enabling them to remain focused on their missions.
The American Civil Liberties Union (ACLU) uses Let’s Encrypt as it works to realize its focus of being a “guardian of liberty” for US citizens. Using Let’s Encrypt protects ACLU’s constituents when they’re trying to know their rights or take action. With more than 4 million page views per month, ACLU’s website is a critical part of their mission.
Human Rights Watch (HRW) is an international nonprofit organization. With more than 500 individuals on staff around the world, HRW’s website is a trove of information empowering individuals and organizations alike to be informed and take action with a global perspective. Nearly 70% of HRW’s web traffic comes from people outside of the United States; that’s millions of page views per month secured by Let’s Encrypt—and by extension, millions of people around the world benefitting from a more secure and privacy-respecting Web.
The Center for Democracy & Technology (CDT) uses Let’s Encrypt to advance its mission to promote democratic values by shaping technology policy and architecture, with a focus on the rights of the individual. The CDT website offers updated and insightful information into the ways policy and innovation impact the digital space. Without a TLS certificate, the content of these pages could be intercepted and changed. What’s more, for those looking to financially support CDT, using TLS on their donation page encrypts the transaction protecting user details such as credit card and other personal information. Mallory Knodel, CTO at CDT and longtime digital rights defender and advocate commented, “Billions of people in over 60 countries access the internet with less censorship and surveillance because Let’s Encrypt hastened the adoption of web security measures by making certificates easy to obtain.”
In the United States, the work of nonprofits is made possible in large part through philanthropic foundations and organizations. When it comes to philanthropy’s web presence, Let’s Encrypt is there, too.
We provide TLS to billion dollar philanthropic organizations like the Hewlett Foundation, the Silicon Valley Community Foundation, Yield Giving, and many others. Taking a look at the top 50 philanthropic organizations around the world, Let’s Encrypt serves 36% of them. For large philanthropies, their website is the primary tool they have to communicate their focus areas for future funding as well as the impact they’ve made with past giving.
One of the leading philanthropists in the US, Craig Newmark, uses Let’s Encrypt and Digital Ocean for his website, craig newmark philanthropies. Commenting on our work, Craig recently shared, “The people at ISRG have been helping protect the Internet for over ten years, and continue to protect us all. They’re a necessary part of Cyber Civil Defense and national security.”
Overall, while Let’s Encrypt aims to build a better Internet, we’re particularly proud that our impact protects those seeking to build a better world.
Internet Security Research Group (ISRG) is the parent organization of Prossimo, Let’s Encrypt, and Divvi Up. ISRG is a 501(c)(3) nonprofit. If you’d like to support our work, please consider getting involved, donating, or encouraging your company to become a sponsor.
Post Syndicated from xkcd.com original https://xkcd.com/2867/
![It's not just time zones and leap seconds. SI seconds on Earth are slower because of relativity, so there are time standards for space stuff (TCB, TGC) that use faster SI seconds than UTC/Unix time. T2 - T1 = [God doesn't know and the Devil isn't telling.] It's not just time zones and leap seconds. SI seconds on Earth are slower because of relativity, so there are time standards for space stuff (TCB, TGC) that use faster SI seconds than UTC/Unix time. T2 - T1 = [God doesn't know and the Devil isn't telling.]](https://imgs.xkcd.com/comics/datetime.png)
Post Syndicated from Let's Encrypt original https://letsencrypt.org/2023/12/13/ngos.html
For more than ten years, we at the nonprofit Internet Security Research Group (ISRG) have been focused on our mission of building a more secure and privacy-respecting Internet for everyone, everywhere. As we touch on in our 2023 Annual Report, we now serve more than 360 million domains with free TLS certificates.
Beyond being a big number, what does that signify? What’s the importance of having TLS being widely adopted anyways? We’ll take a closer look at these questions through the lens of one group of Subscribers we can relate to particularly well: nonprofits.
Let’s Encrypt serves 51% of all websites using the .org top level domain (TLD), which is commonly used by nonprofits. In the US alone there are 1.8M registered nonprofit organizations. And while the focus of these organizations are varied, all of them rely on the Internet in some capacity.
When a nonprofit uses a TLS certificate on their website, it protects their visitors and stakeholders from snoopers, MITM attacks, and surveillance. Without TLS, nonprofits’ content could be changed without their knowledge or their visitors’ private information could be compromised. Access to free and automated TLS via Let’s Encrypt means these nonprofits face as few barriers as possible to adopting TLS.
In short, something as fundamental as security and privacy should be as easy to access as possible. For nonprofits both large and small, Let’s Encrypt makes it easy to provide security and privacy for users of their websites, enabling them to remain focused on their missions.
The American Civil Liberties Union (ACLU) uses Let’s Encrypt as it works to realize its focus of being a “guardian of liberty” for US citizens. Using Let’s Encrypt protects ACLU’s constituents when they’re trying to know their rights or take action. With more than 4 million page views per month, ACLU’s website is a critical part of their mission.
Human Rights Watch (HRW) is an international nonprofit organization. With more than 500 individuals on staff around the world, HRW’s website is a trove of information empowering individuals and organizations alike to be informed and take action with a global perspective. Nearly 70% of HRW’s web traffic comes from people outside of the United States; that’s millions of page views per month secured by Let’s Encrypt—and by extension, millions of people around the world benefitting from a more secure and privacy-respecting Web.
The Center for Democracy & Technology (CDT) uses Let’s Encrypt to advance its mission to promote democratic values by shaping technology policy and architecture, with a focus on the rights of the individual. The CDT website offers updated and insightful information into the ways policy and innovation impact the digital space. Without a TLS certificate, the content of these pages could be intercepted and changed. What’s more, for those looking to financially support CDT, using TLS on their donation page encrypts the transaction protecting user details such as credit card and other personal information. Mallory Knodel, CTO at CDT and longtime digital rights defender and advocate commented, “Billions of people in over 60 countries access the internet with less censorship and surveillance because Let’s Encrypt hastened the adoption of web security measures by making certificates easy to obtain.”
In the United States, the work of nonprofits is made possible in large part through philanthropic foundations and organizations. When it comes to philanthropy’s web presence, Let’s Encrypt is there, too.
We provide TLS to billion dollar philanthropic organizations like the Hewlett Foundation, the Silicon Valley Community Foundation, and many others. Taking a look at the top 50 philanthropic organizations around the world, Let’s Encrypt serves 36% of them. For large philanthropies, their website is the primary tool they have to communicate their focus areas for future funding as well as the impact they’ve made with past giving.
One of the leading philanthropists in the US, Craig Newmark, uses Let’s Encrypt and Digital Ocean for his website, craig newmark philanthropies. Commenting on our work, Craig recently shared, “The people at ISRG have been helping protect the Internet for over ten years, and continue to protect us all. They’re a necessary part of Cyber Civil Defense and national security.”
Overall, while Let’s Encrypt aims to build a better Internet, we’re particularly proud that our impact protects those seeking to build a better world.
Internet Security Research Group (ISRG) is the parent organization of Prossimo, Let’s Encrypt, and Divvi Up. ISRG is a 501(c)(3) nonprofit. If you’d like to support our work, please consider getting involved, donating, or encouraging your company to become a sponsor.
Post Syndicated from corbet original https://lwn.net/Articles/954783/
Konstantin Ryabitsev has announced
that the movement of kernel mailing lists away from the venerable
vger.kernel.org system is nearly complete:
Over the past few months we’ve migrated all of the vger.kernel.org
mailing lists, with the exception of the Big One (linux-kernel, aka
LKML). This list alone is responsible for about 80% of all vger
mailing list traffic, so we left it for the last.This Thursday, December 14, at 11AM Pacific (19:00 UTC), we will
switch the MX record for vger to point to the new location
(subspace.kernel.org), which will complete the mailing list
migration from the legacy vger server to the new infrastructure.
Post Syndicated from corbet original https://lwn.net/Articles/954777/
The story of Canonical’s takeover of the LXD container manager, and the
subsequent creation of the Incus fork, has been
simmering for a while. Now Incus developer Stéphane Graber reports
that Canonical has changed the license and contribution terms for LXD:
Per the commit message performing the re-licensing, all further
contributions will be under the AGPLv3 license and all
contributions from Canonical employees have been re-licensed to
AGPLv3.However, Canonical does not own the copyright on any contribution
from non-employees, such as the many changes they have imported
from Incus over the past few months. Those therefore remain under
the Apache2 license that they were contributed under.As a result, Canonical cannot release LXD under the AGPLv3 license
and likely never will be able to. LXD is now under a weird mix of
Apache2 and AGPLv3 with no clear metadata indicating what file or
what part of each file is under one license or the other.
He also notes that this change will put an end to the flow of patches — in
either direction — between the two projects.
Post Syndicated from The Atlantic original https://www.youtube.com/watch?v=f-tbomtUtrI
Post Syndicated from Adam Barnett original https://blog.rapid7.com/2023/12/12/patch-tuesday-december-2023/

Microsoft is addressing 34 vulnerabilities this December Patch Tuesday, including a single zero-day vulnerability and three critical remote code execution (RCE) vulnerabilities. December Patch Tuesday has historically seen fewer patches than a typical month, and this trend continues in 2023. This total does not include eight browser vulnerabilities published earlier this month. At time of writing, none of the vulnerabilities patched today are yet added to the CISA KEV list.
This month’s lone zero-day vulnerability is CVE-2023-20588, which describes a potential information disclosure due to a flaw in certain AMD processor models as listed on the AMD advisory. AMD states that a divide-by-zero on these processor models could potentially return speculative data. AMD believes the potential impact of the vulnerability is low since local access is required; however, Microsoft ranks severity as important under its own proprietary severity scale. The vulnerability is patched at the OS level in all supported versions of Windows, even as far back as Windows Server 2008 for Azure-hosted assets participating in the Extended Security Update (ESU) program.
CVE-2023-35628 describes a critical RCE vulnerability in the MSHTML proprietary browser engine still used by Outlook, among others, to render HTML content. Of particular note: the most concerning exploitation scenario leads to exploitation as soon as Outlook retrieves and processes the specially crafted malicious email. This means that exploitation could occur before the user interacts with the email in any way; not even the Preview Pane is required in this scenario. Other attack vectors exist: the user could also click a malicious link received via email, instant message, or other medium. Assets where Internet Explorer 11 has been fully disabled are still vulnerable until patched; the MSHTML engine remains installed within Windows regardless of the status of IE11.
This month also brings patches for a pair of critical RCE vulnerabilities in Internet Connection Sharing. CVE-2023-35630 and CVE-2023-35641 share a number of similarities: a base CVSS v3.1 score of 8.8, Microsoft critical severity ranking, low attack complexity, and presumably execution in SYSTEM context on the target machine, although the advisories do not specify execution context. Description of the exploitation method does differ between the two, however. CVE-2023-35630 requires the attacker to modify an option->length field in a DHCPv6 DHCPV6_MESSAGE_INFORMATION_REQUEST input message. Exploitation of CVE-2023-35641 is also via a maliciously crafted DHCP message to an ICS server, but the advisory gives no further clues. A broadly similar ICS vulnerability in September 2023 led to RCE in a SYSTEM context on the ICS server. In all three cases, a mitigating factor is the requirement for the attack to be launched from the same network segment as the ICS server. It seems improbable that either of this month’s ICS vulnerabilities are exploitable against a target on which ICS is not running, although Microsoft does not explicitly deny the possibility.
Notable by their absence this month: no security patches for Exchange, SharePoint, Visual Studio/.NET, or SQL Server. There are also no lifecycle transitions for Microsoft products this month, although a number of Windows Server 2019 editions and Office components will transition out of mainstream support and into extended support from January 2024.



| CVE | Title | Exploited? | Publicly disclosed? | CVSSv3 base score |
|---|---|---|---|---|
| CVE-2023-35624 | Azure Connected Machine Agent Elevation of Privilege Vulnerability | No | No | 7.3 |
| CVE-2023-35625 | Azure Machine Learning Compute Instance for SDK Users Information Disclosure Vulnerability | No | No | 4.7 |
| CVE | Title | Exploited? | Publicly disclosed? | CVSSv3 base score |
|---|---|---|---|---|
| CVE-2023-35618 | Microsoft Edge (Chromium-based) Elevation of Privilege Vulnerability | No | No | 9.6 |
| CVE-2023-36880 | Microsoft Edge (Chromium-based) Information Disclosure Vulnerability | No | No | 4.8 |
| CVE-2023-38174 | Microsoft Edge (Chromium-based) Information Disclosure Vulnerability | No | No | 4.3 |
| CVE-2023-6512 | Chromium: CVE-2023-6512 Inappropriate implementation in Web Browser UI | No | No | N/A |
| CVE-2023-6511 | Chromium: CVE-2023-6511 Inappropriate implementation in Autofill | No | No | N/A |
| CVE-2023-6510 | Chromium: CVE-2023-6510 Use after free in Media Capture | No | No | N/A |
| CVE-2023-6509 | Chromium: CVE-2023-6509 Use after free in Side Panel Search | No | No | N/A |
| CVE-2023-6508 | Chromium: CVE-2023-6508 Use after free in Media Stream | No | No | N/A |
| CVE | Title | Exploited? | Publicly disclosed? | CVSSv3 base score |
|---|---|---|---|---|
| CVE-2023-36006 | Microsoft WDAC OLE DB provider for SQL Server Remote Code Execution Vulnerability | No | No | 8.8 |
| CVE-2023-35639 | Microsoft ODBC Driver Remote Code Execution Vulnerability | No | No | 8.8 |
| CVE-2023-35641 | Internet Connection Sharing (ICS) Remote Code Execution Vulnerability | No | No | 8.8 |
| CVE-2023-35630 | Internet Connection Sharing (ICS) Remote Code Execution Vulnerability | No | No | 8.8 |
| CVE-2023-35628 | Windows MSHTML Platform Remote Code Execution Vulnerability | No | No | 8.1 |
| CVE-2023-21740 | Windows Media Remote Code Execution Vulnerability | No | No | 7.8 |
| CVE-2023-35633 | Windows Kernel Elevation of Privilege Vulnerability | No | No | 7.8 |
| CVE-2023-35632 | Windows Ancillary Function Driver for WinSock Elevation of Privilege Vulnerability | No | No | 7.8 |
| CVE-2023-36011 | Win32k Elevation of Privilege Vulnerability | No | No | 7.8 |
| CVE-2023-36005 | Windows Telephony Server Elevation of Privilege Vulnerability | No | No | 7.5 |
| CVE-2023-36004 | Windows DPAPI (Data Protection Application Programming Interface) Spoofing Vulnerability | No | No | 7.5 |
| CVE-2023-35622 | Windows DNS Spoofing Vulnerability | No | No | 7.5 |
| CVE-2023-35643 | DHCP Server Service Information Disclosure Vulnerability | No | No | 7.5 |
| CVE-2023-35638 | DHCP Server Service Denial of Service Vulnerability | No | No | 7.5 |
| CVE-2023-35629 | Microsoft USBHUB 3.0 Device Driver Remote Code Execution Vulnerability | No | No | 6.8 |
| CVE-2023-35642 | Internet Connection Sharing (ICS) Denial of Service Vulnerability | No | No | 6.5 |
| CVE-2023-36012 | DHCP Server Service Information Disclosure Vulnerability | No | No | 5.3 |
| CVE-2023-20588 | AMD: CVE-2023-20588 AMD Speculative Leaks Security Notice | No | Yes | N/A |
| CVE | Title | Exploited? | Publicly disclosed? | CVSSv3 base score |
|---|---|---|---|---|
| CVE-2023-36020 | Microsoft Dynamics 365 (on-premises) Cross-site Scripting Vulnerability | No | No | 7.6 |
| CVE-2023-35621 | Microsoft Dynamics 365 Finance and Operations Denial of Service Vulnerability | No | No | 7.5 |
| CVE | Title | Exploited? | Publicly disclosed? | CVSSv3 base score |
|---|---|---|---|---|
| CVE-2023-36019 | Microsoft Power Platform Connector Spoofing Vulnerability | No | No | 9.6 |
| CVE | Title | Exploited? | Publicly disclosed? | CVSSv3 base score |
|---|---|---|---|---|
| CVE-2023-35636 | Microsoft Outlook Information Disclosure Vulnerability | No | No | 6.5 |
| CVE-2023-36009 | Microsoft Word Information Disclosure Vulnerability | No | No | 5.5 |
| CVE-2023-35619 | Microsoft Outlook for Mac Spoofing Vulnerability | No | No | 5.3 |
| CVE | Title | Exploited? | Publicly disclosed? | CVSSv3 base score |
|---|---|---|---|---|
| CVE-2023-36010 | Microsoft Defender Denial of Service Vulnerability | No | No | 7.5 |
| CVE | Title | Exploited? | Publicly disclosed? | CVSSv3 base score |
|---|---|---|---|---|
| CVE-2023-35634 | Windows Bluetooth Driver Remote Code Execution Vulnerability | No | No | 8 |
| CVE-2023-35644 | Windows Sysmain Service Elevation of Privilege | No | No | 7.8 |
| CVE-2023-36696 | Windows Cloud Files Mini Filter Driver Elevation of Privilege Vulnerability | No | No | 7.8 |
| CVE-2023-35631 | Win32k Elevation of Privilege Vulnerability | No | No | 7.8 |
| CVE-2023-36391 | Local Security Authority Subsystem Service Elevation of Privilege Vulnerability | No | No | 7.8 |
| CVE-2023-36003 | XAML Diagnostics Elevation of Privilege Vulnerability | No | No | 6.7 |
| CVE-2023-35635 | Windows Kernel Denial of Service Vulnerability | No | No | 5.5 |
Post Syndicated from Debu Panda original https://aws.amazon.com/blogs/big-data/federate-iam-based-single-sign-on-to-amazon-redshift-role-based-access-control-with-okta/
Amazon Redshift accelerates your time to insights with fast, easy, and secure cloud data warehousing at scale. Tens of thousands of customers rely on Amazon Redshift to analyze exabytes of data and run complex analytical queries.
You can use your preferred SQL clients to analyze your data in an Amazon Redshift data warehouse. Connect seamlessly by leveraging an identity provider (IdP) or single sign-on (SSO) credentials to connect to the Redshift data warehouse to reuse existing user credentials and avoid additional user setup and configuration. Using role-based access control (RBAC), you can simplify managing user privileges, database permissions and management of security permissions in Amazon Redshift. You can also use redshift database roles to define a set of elevated permissions, such as for a system monitor or database administrator.
Using AWS Identity and Access Management (IAM) with RBAC, organizations can simplify user management because you no longer need to create users and map them to database roles manually. You can define the mapped database roles as a principal tag for the IdP groups or IAM role, so Redshift database roles and users who are members of those IdP groups are granted to the database roles automatically.
Earlier in 2023, we launched support for Okta integration with Amazon Redshift Serverless using database roles. In this post, we focus on Okta as the IdP and provide step-by-step guidance to integrate a Redshift provisioned cluster with Okta using the Redshift Query Editor v2 and with SQL clients like SQL Workbench/J. You can use this mechanism with other IdP providers such as Azure Active Directory or Ping with any applications or tools using Amazon’s JDBC, ODBC, or Python driver.
Recently we also announced Amazon Redshift integration with AWS IAM Identity Center, supporting trusted identity propagation, allowing you to use third-party Identity Providers (IdP) such as Microsoft Entra ID (Azure AD), Okta, Ping, and OneLogin. This integration simplifies the authentication and authorization process for Amazon Redshift users using Query Editor V2 or Amazon Quicksight, making it easier for them to securely access your data warehouse. AWS IAM Identity Center offers automatic user and group provisioning from Okta to itself by utilizing the System for Cross-domain Identity Management (SCIM) 2.0 protocol. This integration allows for seamless synchronization of information between two services, ensuring accurate and up-to-date information in AWS IAM Identity Center. Refer to Integrate Okta with Amazon Redshift Query Editor V2 using AWS IAM Identity Center for seamless Single Sign-On blog post to learn more about setting up single sign-on (SSO) to Amazon Redshift using integration with IdC and Okta as the Identity Provider.
If you are interested in using IAM-based single sign-on with Amazon Redshift database roles then you can continue reading this blog.
The following diagram illustrates the authentication flow of Okta with a Redshift provisioned cluster using federated IAM roles and automatic database role mapping.
The workflow contains the following steps:
AssumeRoleWithSAML. The SAML assertion contains the IdP user and group information that is stored in the RedshiftDbUser and RedshiftDbRoles principal tags, respectively. Temporary IAM credentials are returned to the SQL client or, if using the Query Editor v2, the user’s browser is redirected to the Query Editor v2 console using the temporary IAM credentials.GetClusterCredentialsWithIAM. This API uses the principal tags to determine the user and database roles that the user belongs to. An associated database user is created if the user is signing in for the first time and is granted the matching database roles automatically. A temporary password is returned to the SQL client.You need the following prerequisites to set up this solution:
To connect using Query Editor v2, complete the following steps:
GetClusterCredentialsWithIAM API:
Now you’re ready to connect to your Redshift provisioned cluster using Query Editor v2 and federated login.
Because Ethan is part of the sales group and has been granted permissions to access tables in the sales_schema, he should be able to access those tables without any issues. However, if he tries to access tables in the finance_schema, he would receive a permission denied error because Ethan is not part of the finance group in Okta.
To connect as a federated user via a third-party client, complete the following steps:
jdbc:redshift:iam://<cluster endpoint>:<port>:<databasename>?groupfederation=true. For example, jdbc:redshift:iam://redshift-cluster-1.abdef0abc0ab.us-west-2.redshift.amazonaws.com:5439/dev?groupfederation=trueIn the preceding URL, groupfederation is a mandatory parameter that allows you to authenticate with the IAM credentials for the Redshift provisioned cluster. Without the groupfederation parameter, it will not use Redshift database roles.
User Ethan will be able to access the sales_schema tables. If Ethan tries to access the tables in the finance_schema, he will get a permission denied error.
If your connection didn’t work, consider the following:
When you’re done testing the solution, clean up the resources to avoid incurring future charges:
In this post, we provided step-by-step instructions to integrate a Redshift provisioned cluster with Okta using the Redshift Query Editor v2 and SQL Workbench/J with the help of federated IAM roles and automatic database role mapping. You can use a similar setup with other SQL clients (such as DBeaver or DataGrip). We also showed how Okta group membership is mapped automatically with Redshift provisioned cluster roles to use role-based authentication seamlessly.
If you have any feedback or questions, please leave them in the comments.
Debu Panda is a Senior Manager, Product Management at AWS. He is an industry leader in analytics, application platform, and database technologies, and has more than 25 years of experience in the IT world.
Ranjan Burman is an Analytics Specialist Solutions Architect at AWS. He specializes in Amazon Redshift and helps customers build scalable analytical solutions. He has more than 16 years of experience in different database and data warehousing technologies. He is passionate about automating and solving customer problems with cloud solutions.
Maneesh Sharma is a Senior Database Engineer at AWS with more than a decade of experience designing and implementing large-scale data warehouse and analytics solutions. He collaborates with various Amazon Redshift Partners and customers to drive better integration.
Post Syndicated from Gaurav Parekh original https://aws.amazon.com/blogs/big-data/orchestrate-amazon-emr-serverless-spark-jobs-with-amazon-mwaa-and-data-validation-using-amazon-athena/
As data engineering becomes increasingly complex, organizations are looking for new ways to streamline their data processing workflows. Many data engineers today use Apache Airflow to build, schedule, and monitor their data pipelines.
However, as the volume of data grows, managing and scaling these pipelines can become a daunting task. Amazon Managed Workflows for Apache Airflow (Amazon MWAA) can help simplify the process of building, running, and managing data pipelines. By providing Apache Airflow as a fully managed platform, Amazon MWAA allows data engineers to focus on building data workflows instead of worrying about infrastructure.
Today, businesses and organizations require cost-effective and efficient ways to process large amounts of data. Amazon EMR Serverless is a cost-effective and scalable solution for big data processing that can handle large volumes of data. The Amazon Provider in Apache Airflow comes with EMR Serverless operators and is already included in Amazon MWAA, making it easy for data engineers to build scalable and reliable data processing pipelines. You can use EMR Serverless to run Spark jobs on the data, and use Amazon MWAA to manage the workflows and dependencies between these jobs. This integration can also help reduce costs by automatically scaling the resources needed to process data.
Amazon Athena is a serverless, interactive analytics service built on open-source frameworks, supporting open-table and file formats. You can use standard SQL to interact with data. Athena, a serverless and interactive analytics service, makes this possible without the need to manage complex infrastructure.
In this post, we use Amazon MWAA, EMR Serverless, and Athena to build a complete end-to-end data processing pipeline.
The following diagram illustrates the solution architecture.

The workflow includes the following steps:
You should have the following prerequisites:
emr_serverless_execution_role job execution role with permissions to run an EMR Serverless application and read from S3 buckets, and iam:PassRole permission to the Amazon MWAA cluster execution role. For more information, refer to EMR Serverless Samples, Job runtime roles, and Adding and removing IAM identity permissions.To illustrate using EMR Serverless jobs with Apache Spark via Amazon MWAA and data validation using Athena, we use the publicly available NYC taxi dataset. Download the following datasets to your local machine:
In later steps, we upload these datasets to Amazon S3.
This section outlines the steps for setting up data processing and transformation.
You can create one or more EMR Serverless applications that use open source analytics frameworks like Apache Spark or Apache Hive. Unlike EMR on EC2, you do not need to delete or terminate EMR Serverless applications. EMR Serverless application is only a definition and once created, can be re-used as long as needed. This makes the MWAA pipeline simpler as now you just have to submit jobs to a pre-created EMR Serverless application.
By default, EMR Serverless application will auto-start on job submission and auto-stop when idle for 15 minutes by default to ensure cost efficiency. You can modify the amount of idle time or choose to turn the feature off.
To create an application using EMR Serverless console, follow the instructions in “Create an EMR Serverless application”. Note down the application ID as we will use it in following steps.
![]()
Complete the following steps to set up your S3 bucket and folders:
input_data folder for storing input data.green, yellow, and zone_lookup.You can download and work with the latest datasets available. For our testing, we use the following files:
green/ folder has the file green_tripdata_2022-06.parquetyellow/ folder has the file yellow_tripdata_2022-06.parquetzone_lookup/ folder has the file taxi_zone_lookup.csv
Complete the following steps to set up your DAG scripts:
version >=1.23.9.yellow_taxi_zone_lookup, green_taxi_zone_lookup, and ny_taxi_summary,. These tasks involve running Spark jobs to lookup taxi zones, and generating a data summary .df_green and df_zone data frames, performs column-based joins, and aggregates data like passenger count, trip distance, and fare amount. Lastly, it creates the output_data folder in the specified S3 bucket to write the resulting data frame, df_green_zone, as Parquet files.yellow_zone. It reads the yellow taxi files and zone lookup file from the specified S3 bucket, creates temporary views, performs a join based on location ID, and calculates statistics such as passenger count, trip distance, and fare amount. Lastly, it creates the output_data folder in the specified S3 bucket to write the resulting data frame, df_yellow_zone, as Parquet files.green_zone and yellow_zone files to aggregate statistics on taxi rides, grouping data by service zones and location IDs. It requires an S3 bucket name as a command line argument, creates a SparkSession named ny_taxi_summary, reads the files from S3, performs a join, and generates a new data frame named ny_taxi_summary. It creates an output_data folder in the specified S3 bucket to write the resulting data frame to new Parquet files.blog_dag_mwaa_emrs_ny_taxi.py script with the following information:
requirements.txt file to the S3 bucket created earlier
dags and upload the updated blog_dag_mwaa_emrs_ny_taxi.py file from your local machine.
scripts inside the S3 bucket and upload the scripts to this folder from your local machine.
To create an Airflow environment, complete the following steps:

mwaa_emrs_athena_pipeline.dags folder.requirements.txt file.
This will populate two of the private subnets in your VPC.
This allows the Apache Airflow UI to be accessed over the internet by users granted access to the IAM policy for your environment.



It may take about 20–30 minutes to create your Amazon MWAA environment.
emr_serverless_execution_role.To trigger the DAG, complete the following steps:

blog_dag_mwaa_emr_ny_taxi, choose the play icon, and choose Trigger DAG.
blog_dag_mwaa_emrs_ny_taxi and choose Graph to locate your DAG run workflow.
The DAG will take approximately 4–6 minutes to run all the scripts. You will see all the complete tasks and the overall status of the DAG will show as success.

To rerun the DAG, remove s3://<<your_s3_bucket here >>/output_data/.
Optionally, to understand how Amazon MWAA runs these tasks, choose the task you want to inspect.

Choose Run to view the task run details.

The following screenshot shows an example of the task logs.

If you like to dive deep in the execution logs, then on the EMR Serverless console, navigate to “Applications”. The Apache Spark driver logs will indicate the initiation of your job along with the details for executors, stages and tasks that were created by EMR Serverless. These logs can be helpful to monitor your job progress and troubleshoot failures.
By default, EMR Serverless will store application logs securely in Amazon EMR managed storage for a period of 30 days. However, you can also specify Amazon S3 or Amazon CloudWatch as your log delivery options during job submission.

Let’s validate the data loaded by the process using Athena SQL queries.
<S3_BUCKET_NAME>/athena), then choose Save.

Run the following query on the recently created ny_taxi_summary table to retrieve the first 10 rows to validate the data:

To prevent future charges, complete the following steps:

To delete the application, navigate to the List applications page. Select the application that you created and choose Actions → Stop to stop the application. After the application is in the STOPPED state, select the same application and choose Actions → Delete.

Data engineering is a critical component of many organizations, and as data volumes continue to grow, it’s essential to find ways to streamline data processing workflows. The combination of Amazon MWAA, EMR Serverless, and Athena provides a powerful solution to build, run, and manage data pipelines efficiently. With this end-to-end data processing pipeline, data engineers can easily process and analyze large amounts of data quickly and cost-effectively without the need to manage complex infrastructure. The integration of these AWS services provides a robust and scalable solution for data processing, helping organizations make informed decisions based on their data insights.
Now that you’ve seen how to submit Spark jobs on EMR Serverless via Amazon MWAA, we encourage you to use Amazon MWAA to create a workflow that will run PySpark jobs via EMR Serverless.
We welcome your feedback and inquiries. Please feel free to reach out to us if you have any questions or comments.
Rahul Sonawane is a Principal Analytics Solutions Architect at AWS with AI/ML and Analytics as his area of specialty.
Gaurav Parekh is a Solutions Architect helping AWS customers build large scale modern architecture. He specializes in data analytics and networking. Outside of work, Gaurav enjoys playing cricket, soccer and volleyball.
December 2023: This post was reviewed for technical accuracy by Santosh Gantaram, Sr. Technical Account Manager.
Post Syndicated from Channy Yun original https://aws.amazon.com/blogs/aws/new-for-aws-amplify-query-mysql-and-postgresql-database-for-aws-cdk/
Today we are announcing the general availability to connect and query your existing MySQL and PostgreSQL databases with support for AWS Cloud Development Kit (AWS CDK), a new feature to create a real-time, secure GraphQL API for your relational database within or outside Amazon Web Services (AWS). You can now generate the entire API for all relational database operations with just your database endpoint and credentials. When your database schema changes, you can run a command to apply the latest table schema changes.
In 2021, we announced AWS Amplify GraphQL Transformer version 2, enabling developers to develop more feature-rich, flexible, and extensible GraphQL-based app backends even with minimal cloud expertise. This new GraphQL Transformer was redesigned from the ground up to generate extensible pipeline resolvers to route a GraphQL API request, apply business logic, such as authorization, and communicate with the underlying data source, such as Amazon DynamoDB.
However, customers wanted to use relational database sources for their GraphQL APIs such as their Amazon RDS or Amazon Aurora databases in addition to Amazon DynamoDB. You can now use @model types of Amplify GraphQL APIs for both relational database and DynamoDB data sources. Relational database information is generated to a separate schema.sql.graphql file. You can continue to use the regular schema.graphql files to create and manage DynamoDB-backed types.
When you simply provide any MySQL or PostgreSQL database information, whether behind a virtual private cloud (VPC) or publicly accessible on the internet, AWS Amplify automatically generates a modifiable GraphQL API that securely connects to your database tables and exposes create, read, update, or delete (CRUD) queries and mutations. You can also rename your data models to be more idiomatic for the frontend. For example, a database table is called “todos” (plural, lowercase) but is exposed as “ToDo” (singular, PascalCase) to the client.
With one line of code, you can add any of the existing Amplify GraphQL authorization rules to your API, making it seamless to build use cases such as owner-based authorization or public read-only patterns. Because the generated API is built on AWS AppSync‘ GraphQL capabilities, secure real-time subscriptions are available out of the box. You can subscribe to any CRUD events from any data model with a few lines of code.
Getting started with your MySQL database in AWS CDK
The AWS CDK lets you build reliable, scalable, cost-effective applications in the cloud with the considerable expressive power of a programming language. To get started, install the AWS CDK on your local machine.
$ npm install -g aws-cdk
Run the following command to verify the installation is correct and print the version number of the AWS CDK.
$ cdk –version
Next, create a new directory for your app:
$ mkdir amplify-api-cdk
$ cd amplify-api-cdk
Initialize a CDK app by using the cdk init command.
$ cdk init app --language typescript
Install Amplify’s GraphQL API construct in the new CDK project:
$ npm install @aws-amplify/graphql-api-construct
Open the main stack file in your CDK project (usually located in lib/<your-project-name>-stack.ts). Import the necessary constructs at the top of the file:
import {
AmplifyGraphqlApi,
AmplifyGraphqlDefinition
} from '@aws-amplify/graphql-api-construct';
Generate a GraphQL schema for a new relational database API by executing the following SQL statement on your MySQL database. Make sure to output the results to a .csv file, including column headers, and replace <database-name> with the name of your database, schema, or both.
SELECT
INFORMATION_SCHEMA.COLUMNS.TABLE_NAME,
INFORMATION_SCHEMA.COLUMNS.COLUMN_NAME,
INFORMATION_SCHEMA.COLUMNS.COLUMN_DEFAULT,
INFORMATION_SCHEMA.COLUMNS.ORDINAL_POSITION,
INFORMATION_SCHEMA.COLUMNS.DATA_TYPE,
INFORMATION_SCHEMA.COLUMNS.COLUMN_TYPE,
INFORMATION_SCHEMA.COLUMNS.IS_NULLABLE,
INFORMATION_SCHEMA.COLUMNS.CHARACTER_MAXIMUM_LENGTH,
INFORMATION_SCHEMA.STATISTICS.INDEX_NAME,
INFORMATION_SCHEMA.STATISTICS.NON_UNIQUE,
INFORMATION_SCHEMA.STATISTICS.SEQ_IN_INDEX,
INFORMATION_SCHEMA.STATISTICS.NULLABLE
FROM INFORMATION_SCHEMA.COLUMNS
LEFT JOIN INFORMATION_SCHEMA.STATISTICS ON INFORMATION_SCHEMA.COLUMNS.TABLE_NAME=INFORMATION_SCHEMA.STATISTICS.TABLE_NAME AND INFORMATION_SCHEMA.COLUMNS.COLUMN_NAME=INFORMATION_SCHEMA.STATISTICS.COLUMN_NAME
WHERE INFORMATION_SCHEMA.COLUMNS.TABLE_SCHEMA = '<database-name>';
Run the following command, replacing <path-schema.csv> with the path to the .csv file created in the previous step.
$ npx @aws-amplify/cli api generate-schema \
--sql-schema <path-to-schema.csv> \
--engine-type mysql –out lib/schema.sql.graphql
You can open schema.sql.graphql file to see the imported data model from your MySQL database schema.
input AMPLIFY {
engine: String = "mysql"
globalAuthRule: AuthRule = {allow: public}
}
type Meals @model {
id: Int! @primaryKey
name: String!
}
type Restaurants @model {
restaurant_id: Int! @primaryKey
address: String!
city: String!
name: String!
phone_number: String!
postal_code: String!
...
}
If you haven’t already done so, go to the Parameter Store in the AWS Systems Manager console and create a parameter for the connection details of your database, such as hostname/url, database name, port, username, and password. These will be required in the next step for Amplify to successfully connect to your database and perform GraphQL queries or mutations against it.

In the main stack class, add the following code to define a new GraphQL API. Replace the dbConnectionConfg options with the parameter paths created in the previous step.
new AmplifyGraphqlApi(this, "MyAmplifyGraphQLApi", {
apiName: "MySQLApi",
definition: AmplifyGraphqlDefinition.fromFilesAndStrategy(
[path.join(__dirname, "schema.sql.graphql")],
{
name: "MyAmplifyGraphQLSchema",
dbType: "MYSQL",
dbConnectionConfig: {
hostnameSsmPath: "/amplify-cdk-app/hostname",
portSsmPath: "/amplify-cdk-app/port",
databaseNameSsmPath: "/amplify-cdk-app/database",
usernameSsmPath: "/amplify-cdk-app/username",
passwordSsmPath: "/amplify-cdk-app/password",
},
}
),
authorizationModes: { apiKeyConfig: { expires: cdk.Duration.days(7) } },
translationBehavior: { sandboxModeEnabled: true },
});
This configuration assums that your database is accessible from the internet. Also, the default authorization mode is set to Api Key for AWS AppSync and the sandbox mode is enabled to allow public access on all models. This is useful for testing your API before adding more fine-grained authorization rules.
Finally, deploy your GraphQL API to AWS Cloud.
$ cdk deploy
You can now go to the AWS AppSync console and find your created GraphQL API.

Choose your project and the Queries menu. You can see newly created GraphQL APIs compatible with your tables of MySQL database, such as getMeals to get one item or listRestaurants to list all items.

For example, when you select items with fields of address, city, name, phone_number, and so on, you can see a new GraphQL query. Choose the Run button and you can see the query results from your MySQL database.

When you query your MySQL database, you can see the same results.

How to customize your GraphQL schema for your database
To add a custom query or mutation in your SQL, open the generated schema.sql.graphql file and use the @sql(statement: "") pass in parameters using the :<variable> notation.
type Query {
listRestaurantsInState(state: String): Restaurants @sql("SELECT * FROM Restaurants WHERE state = :state;”)
}
For longer, more complex SQL queries, you can reference SQL statements in the customSqlStatements config option. The reference value must match the name of a property mapped to a SQL statement. In the following example, a searchPosts property on customSqlStatements is being referenced:
type Query {
searchPosts(searchTerm: String): [Post]
@sql(reference: "searchPosts")
}
Here is how the SQL statement is mapped in the API definition.
new AmplifyGraphqlApi(this, "MyAmplifyGraphQLApi", {
apiName: "MySQLApi",
definition: AmplifyGraphqlDefinition.fromFilesAndStrategy( [path.join(__dirname, "schema.sql.graphql")],
{
name: "MyAmplifyGraphQLSchema",
dbType: "MYSQL",
dbConnectionConfig: {
// ...ssmPaths,
}, customSqlStatements: {
searchPosts: // property name matches the reference value in schema.sql.graphql
"SELECT * FROM posts WHERE content LIKE CONCAT('%', :searchTerm, '%');",
},
}
),
//...
});
The SQL statement will be executed as if it were defined inline in the schema. The same rules apply in terms of using parameters, ensuring valid SQL syntax, and matching return types. Using a reference file keeps your schema clean and allows the reuse of SQL statements across fields. It is best practice for longer, more complicated SQL queries.
Or you can change a field and model name using the @refersTo directive. If you don’t provide the @refersTo directive, AWS Amplify assumes that the model name and field name exactly match the database table and column names.
type Todo @model @refersTo(name: "todos") {
content: String
done: Boolean
}
When you want to create relationships between two database tables, use the @hasOne and @hasMany directives to establish a 1:1 or 1:M relationship. Use the @belongsTo directive to create a bidirectional relationship back to the relationship parent. For example, you can make a 1:M relationship between a restaurant and its meals menus.
type Meals @model {
id: Int! @primaryKey
name: String!
menus: [Restaurants] @hasMany(references: ["restaurant_id"])
}
type Restaurants @model {
restaurant_id: Int! @primaryKey
address: String!
city: String!
name: String!
phone_number: String!
postal_code: String!
meals: Meals @belongsTo(references: ["restaurant_id"])
...
}
Whenever you make any change to your GraphQL schema or database schema in your DB instances, you should deploy your changes to the cloud:
Whenever you make any change to your GraphQL schema or database schema in your DB instances, you should re-run the SQL script and export to .csv step mentioned earlier in this guide to re-generate your schema.sql.graphql file and then deploy your changes to the cloud:
$ cdk deploy
To learn more, see Connect API to existing MySQL or PostgreSQL database in the AWS Amplify documentation.
Now available
The relational database support for AWS Amplify now works with any MySQL and PostgreSQL databases hosted anywhere within Amazon VPC or even outside of AWS Cloud.
Give it a try and send feedback to AWS re:Post for AWS Amplify, the GitHub repository of Amplify GraphQL API, or through your usual AWS Support contacts.
— Channy
P.S. Specially thanks to René Huangtian Brandel, a principal product manager at AWS for his contribution to write sample codes.
Post Syndicated from jake original https://lwn.net/Articles/954059/
So-called “immutable” Linux distributions have been in development for
some time, but (unless you count Chrome OS) haven’t gained much traction. Project Bluefin, is a heavily
customized set of Fedora
Silverblue images coming from the Universal Blue community; they are
designed to deliver a reliable Linux desktop that’s as easy to use as a
Chromebook but more customizable. Bluefin’s mission is to change up
the desktop experience and attract a new generation of open-source
contributors with a “cloud-native”
take on developing and delivering the operating system.
Post Syndicated from Patrick Kennedy original https://www.servethehome.com/asrock-industrial-4x4-box-7840u-review-amd-powered-nuc-form-factor/
In our ASRock Industrial 4×4 BOX-7840U review, we see how this AMD Ryzen 7040 series mini PC compares to others we have tested
The post ASRock Industrial 4×4 BOX-7840U Review AMD Powered NUC Form Factor appeared first on ServeTheHome.
Post Syndicated from Talks at Google original https://www.youtube.com/watch?v=ogIn1l7JXd0
Post Syndicated from BeardedTinker original https://www.youtube.com/watch?v=7GAQxomFQ5k
Post Syndicated from sabeen malik original https://blog.rapid7.com/2023/12/12/peeking-into-the-crystal-ball-what-2023-cyber-threats-told-us-about-2024/

Stepping into 2024 feels like opening the latest best-selling mystery novel – you know there’s adventure ahead, but the plot is still up in the air.
In the twist-riddled world of cybersecurity, we can’t help but ask, What’s next on the digital horizon? Sure, every business would like 2024 to be “less surprising” in terms of cyber threats, but let’s be honest, that’s not going to happen.
Even though we’re surely in for more than a few surprises in the coming year, there are ways we can be better prepared. So sit back and relax as we venture through some insights we’ve gained in 2023 and offer ways you can put them into practice in the coming year.
RDP (remote desktop protocol) has long been the initial entry vector of choice for ransomware groups, closely followed by the less-so sophisticated email. However the MOVEit and SysAid campaigns show change is brewing.
Rapid7 has observed an increasing number of zero-day vulnerabilities being exploited by ransomware groups, and it’s unlikely this trend will abate. Forget the mindset that ransomware actors just go after “the low hanging fruit”; they are now exploiting zero-day vulnerabilities at mass scale.
This trend is seeing criminal groups that to date have not demonstrated any real capable skills in gaining access to previously unidentified vulnerabilities, exploit them and gain a foothold into victim networks. This demonstrates that potentially something is afoot in the ransomware ecosystem. For organizations, the message is simple: get your vulnerability management and patching procedures in place and do it now. Being proactive when it comes to dealing with vulnerabilities that are being exploited in the wild is imperative.
With the growing number of regulatory disclosures for cyber risk management practices and incidents, the emergence of GenAI as a potent tool for cyber attacks, more ransomware hijacks, and the lack of common lexicon around cyber risk, businesses are truly going to have to spend more time than ever determining their risk profile, and subsequently thinking about the tools and services that they will need to address the risks.
This means that more leaders will be deciding between whether to deal with compliance risk mitigation and/or creating agile cyber risk management strategies. The leaders that understand this moment as a rallying call to uplevel the conversation about systemic risks will set their business up for success by not getting sidetracked by playing compliance whack-a-mole, but by investing in a strategic vision for dealing with cyber business risks. Those businesses will also be able to withstand the scrutiny related to more global requirements for disclosure of both cyber risk management and cyber incident response and procedures. More disclosure may not necessarily lead to clarity in the short term on what are best practices, but over the long term we will see more consolidation on best practices on cyber disclosures and risk management practices.
At the same time, governments will also be struggling to find the right balance on how to incentivize risk management rather than compliance risk mitigation whack-a-mole if they continue to introduce regulations that are not driven by harmonization around best practices and product security instead of first to market on regulations.
Here’s the catch: as regulations become more comprehensive, they may inadvertently nudge the industry towards a more consolidated structure – a double edged sword.
The regulatory dance floor will definitely become more crowded in 2024, especially with AI cutting in.
This new dance partner will be adding to the complexity of tools needed to deal with cyber risk mitigation and will lead to more robust and global public-private partnerships. We might see something like a global cybersecurity flash mob in 2024. Instead of just sharing the usual threat intelligence of cyber threats and cyber risks, governments and businesses will join hands to share threat intel, resources and bolster defenses in concentrated ways to deal with specific threats. Ultimately, moving beyond the historical PPP’s of quarterly meetings, to a more real-time sharing approach in order to deal with the diminishing timelines between initial entry vectors to final stage payloads.
It will be interesting to see if more action oriented partnerships bolster capacity and cyber defenses. In order for such an approach to be successful, a mentality of “information sharing” and an open door of communication must be developed.
The cloud will continue to be a critical cyber battleground. And in the coming year, an emerging concern will likely be the misuse of commercial cloud service providers (CSPs). That’s because cybercriminals are no longer relying on known command-and-control servers; instead, they’re turning to commercial CSPs for cover to host malicious content.
It’s a clever trend, and it comes back to the game of hide-and-seek, with attackers exploiting the cloud’s anonymity and legitimacy, and blending their activities with legitimate services. Combatting this threat requires more innovative solutions, such as those leveraging AI and advanced automation techniques — as well as heightened vigilance — in the cloud. Organizations need advanced risk scoring across cloud environments, so security teams get complete visibility that eliminates blind spots and enables them to effectively prioritize remediation actions.
As mentioned in prediction 4, innovations in AI and automation promise to effectively address an ever-increasing volume of attacks. Seeing threat intelligence is one thing, but it’s a completely different ball game to be doing something about it. This is where more automated responses come into play. With AI coming and more advanced automation techniques, the majority of detection and remediation or prevention work will occur automatically.
But, let’s not get ahead of ourselves. The inevitable rush to market for some solutions means that some AI capabilities will miss the mark. Therefore, organizations that adopt AI solutions must ensure that they truly improve cyber resilience without presenting new cyber risks.
Over the next year, a growing AI use case will be the use of AI synthetic media (i.e., deep fakes) and identity management. Governments will have the challenge of navigating the tricky space between the problematic use case of biometric technology and synthetic media, while businesses will have to understand how to manage the risks with identity and access management.
So, there’s our bird’s eye view of what the cyber landscape could look like in 2024. But as always, there will be many shifts, evolutions, and transformations in the new year, some unprecedented and some expected. Regardless, practitioners must stay on their toes, remain vigilant, and aim for resilience. Here’s to a more prepared, secure — and less stressful — 2024.
For more thoughts from our team on what 2024 could bring, watch the Top Cybersecurity Predictions webinar on-demand.
Post Syndicated from corbet original https://lwn.net/Articles/954706/
Security updates have been issued by Debian (libreoffice and webkit2gtk), Fedora (java-1.8.0-openjdk and seamonkey), Oracle (apr, edk2, kernel, and squid:4), Red Hat (postgresql:12, tracker-miners, and webkit2gtk3), SUSE (curl, go1.20, go1.21, hplip, openvswitch, opera, squid, and xerces-c), and Ubuntu (binutils, ghostscript, libreoffice, linux, linux-aws, linux-aws-5.15, linux-azure, linux-azure-5.15,
linux-azure-fde, linux-azure-fde-5.15, linux-gcp, linux-gke,
linux-hwe-5.15, linux-ibm, linux-ibm-5.15, linux-kvm, linux-nvidia,
linux-oracle, linux-oracle-5.15, linux-raspi, linux, linux-aws, linux-aws-5.4, linux-azure, linux-azure-5.4,
linux-bluefield, linux-gcp, linux-gcp-5.4, linux-hwe-5.4, linux-ibm,
linux-ibm-5.4, linux-kvm, linux-xilinx-zynqmp, postfixadmin, python3.11, and webkit2gtk).
Post Syndicated from João Tomé original http://blog.cloudflare.com/radar-2023-year-in-review-internet-services/

Ask nearly any Internet user, and they are bound to have their own personal list of favorite sites, applications, and Internet services for news, messaging, video, AI chatbots, music, and more. Sum that question up across a lot of users in a lot of different countries, and you end up with a sense of the most popular websites and services in the world. In a nutshell, that’s what this blog post is about: how humans interacted with the online world in 2023 from what Cloudflare observed.
Building on similar reports we’ve done over the past two years, we have compiled a ranking of the top Internet properties of 2023. In addition to our overall ranking, we chose 9 categories to focus on. One of these is a new addition in 2023: Generative AI. Here are the 9 categories we’ll be digging into:
1. Generative AI
2. Social Media
3. E-commerce
4. Video Streaming
5. News
6. Messaging
7. Metaverse & Gaming
8. Financial Services
9. Cryptocurrency Services
Our method for calculating the results is the same as in 2022: we analyze anonymized DNS query data from our 1.1.1.1 public DNS resolver, used by millions of people around the world. To build the lists of Internet services, we use two additional methods. First, we aggregate domains that belong to one online service. For instance, for Twitter/X, we include twitter.com, t.co, and x.com among others. Second, we reference our large source list of domains and identify the sites that provide services to humans. That means that our rankings do not include every domain seen in the data (for example, we exclude domains such as root-servers.net and cloudflare-dns.com). A site’s overall ranking is relative to other sites that meet these criteria; its rank within a category is relative to other sites in the same category. That’s important to note: just because a site has gone down in the rankings, it doesn’t necessarily mean its traffic has declined — it could just be that other sites’ traffic increased. Similarly, the inverse is true. What we’re doing here isn’t tracking absolute traffic, but rather, relative popularity.
With that, we can begin our analysis. Following the success of OpenAI’s ChatGPT launch on November 30, 2022, Generative AI has captivated both the world and the news. OpenAI is now close to ranking among our top 100 most popular Internet services overall, rising from #200 in January.
In e-commerce, low price and fast fashion marketplace Temu experienced similar growth, becoming the year’s surprise by surpassing Shein and other major players, reaching #7 in its category. And Black Friday was the best day for several other e-commerce services, as well as payment services such as PayPal, Stripe and Klarna.
In social media, X/Twitter lost some ground in our Overall ranking, but maintained its status as a significant online discussion platform. New additions like Threads by Instagram, launched in July, are still gaining their footing in the category. Additionally, we observed notable news-related trends linked to events such as the March 2023 United States banking crisis, the Titan submersible implosion, the Wagner Group rebellion, and the October 7 Hamas attack on Israel.
We close our report out with a discussion of a range of trends that don’t fit neatly into our other categories, including a look at Taylor Swift’s and Beyoncé websites — both of which appeared in our overall traffic rankings.
Keep reading for a detailed look at the evolution of trends throughout the year. For more, visit our 2023 Cloudflare Radar Year in Review microsite. Along with the lists of most popular Internet services, the Year in Review site and its associated blog post explore a number of additional metrics.
Since we began reporting these rankings in 2021, we always start with an Overall Top 10 list. These are the services that are the top Internet properties globally in 2023, based on DNS traffic through our 1.1.1.1 resolver. Unsurprisingly, Google (we’re including here services like Google Maps and Google Flights that use google.com, for example) remained the #1 most popular Internet service in our Overall ranking. Since we implemented our new ranking method last year, no other service has challenged Google’s position as the #1 leader in our rankings.
Beyond Google, Facebook was consistently #2, while Apple (that uses, for example, apple.com for several use cases, including on iPhone related services) was generally #3, except when TikTok took the spot in April. Microsoft mostly held the #5 ranking, although it sometimes traded places with YouTube at #6.
AWS — we’re separating it from Amazon by using domains like amazonaws.com — held a firm position at #7, and Instagram was clearly #8 through the year, with a few exceptions involving Amazon. The #10 position changed more frequently, alternating between iCloud (for which we use domains such as icloud.com, distinct from apple.com), Netflix (mainly on weekends), and Microsoft Office. In the chart below, you can follow the evolution of the top Internet services in our Overall ranking throughout the year.

In 2022, X/Twitter ranked as high as #10 in our Overall ranking, but never reached this spot in 2023. We’ll talk more about X/Twitter’s performance in the Social Media category below.
The Generative AI category became a global phenomenon in 2023, though it started gaining attention in late November 2022 with the launch of ChatGPT from OpenAI. We tested it on our ‘This Week in NET’ show on December 2, 2022, and were impressed. It’s not surprising that OpenAI was popular early in 2023 and topped this list. Other generative AI services also emerged during the year.
Significant changes occurred below Open AI’s #1 spot throughout the year. After mid-year, we noticed a shift, when character.ai took the #2 spot from Quillbot. Quora’s Poe AI, which combines several AI chatbots including ChatGPT, entered the top 10 in late March, holding the #3 spot until late April before settling at #4.
This figure shows movement among the Generative AI services that were more popular later in the year:

AI model platform Hugging Face was typically #5, but Google’s Bard also reached #5 in November. Bard, launched in limited regions in March, made our top 10 within the category after its broader release in Europe and Brazil. It was #7 from July to September before peaking later in the year.
Other Generative AI services that became important in our list later in 2023 include Anthropic’s Claude (whose new model, Claude 2, launched in July, though it is not yet available in the European Union). It was #5 in August before dropping to #6 after September. Perplexity AI, a ChatGPT competitor, was #8 after September.
Midjourney, which relies on Discord bot commands to create artwork, peaked at #3 in late March but started declining, dropping to #5 in April and May, and then to #10 in September.
Other AI-inclusive services like Prowritingaid, Voicemod, and Wordtune were more popular earlier in the year but became less so later on. Bardeen was in the top 10 in May and June, while Descript appeared at #9 between March and May.
Notable trends that we observed when looking at trends for Generative AI services in our larger Overall ranking include:

An analysis from Kepios estimates that there are 4.95 billion social media users around the world in 2023, comprising 61.4% of the world population, so this category plays a big role in our everyday life as a central stage for communication, information, and general attention.
Unsurprisingly, social media platforms such as Facebook, TikTok, and Instagram rank high in our most popular Internet properties and are featured in our Top 10 Overall Internet services list.
Within our Social Media category list, the top five remained the same as last year. Facebook was #1, followed by TikTok (#2), Instagram (#3), X/Twitter (#4), and Snapchat (#5).
In contrast to 2022, when Twitter (renamed X in July 2023) and Instagram often swapped places at #3, this year X/Twitter never challenged Instagram in our rankings.
LinkedIn held a strong #6, with Discord challenging it during a few days in January and April.
Reddit, usually at #8, competed with Discord for the #7 spot in February and March but fell back to #8 in April, concurrent with the controversy around Reddit API changes. Reddit often challenged Discord on weekdays, while Discord was more popular on weekends. Following these are Pinterest (#9) and Kwai, the Chinese video app popular in Brazil, Indonesia, and other countries (#10). Here’s the top 10 chart across 2023:

Let’s continue on social media, but leave its specific category and examine how social media services fared in our Overall ranking where bigger shifts between services are seen.
The graph below depicts the position of Instagram and Twitter in the Overall ranking. Instagram consistently ranked in the top 10, typically holding positions #8 or #9 with brief dips to #10 around May 2023.
X/Twitter, which reached as high as #8 overall last year following the start of the war in Ukraine, ended 2022 at #12 and continued its downward trend in 2023. It started between #12 and #16, lower than last year, and ended between #13 and #19. Here’s how these two evolved in 2023:

We noticed that X/Twitter was particularly popular on weekends, peaking at #11 or #12 between April 15 and June 10. Specifically, it ranked at #11 in the weekends before May 14 and then at #12 from May 14 to June 10. This coincides with key moments in European football competitions. However, after the European Champions League final on June 10, X/Twitter never reached those heights again.
Let’s explore this possible football (or soccer) Twitter trend. X/Twitter’s rankings peaked around significant moments in the English Premier League (arguably the most-watched sports league in the world), particularly when Manchester City made crucial advances to their title after April 15. The trend lasted until the European Champions League final on June 10, where City’s victory and historic treble coincided with X/Twitter’s final peak ranking of the year — the weekend of June 3, when there were already no Premier League games, and X/Twitter dropped to #13.
Tumblr, a more established platform than other recent alternatives, fluctuated between #125 and #153 in our Overall ranking, showing a downward trend. Close behind was an aggregation of several hundred Mastodon servers, ranking between #160 and #200.
Threads from Instagram/Meta (seen in the next chart), peaked at #227 on July 6 in our Overall ranking, then dropped but recovered to around #300 after late August, and did not make it into the top 10 social media services ranking for 2023. For context, Kwai (#10 in the Social Media ranking) usually ranks around our Overall top 50.

Bluesky, a newer entrant, first appeared in our Top 500 in late August, with a first spike at #432 on September 19 and reaching the top 400 in November, peaking at #397 on November 19. Truth Social had a peak at #318 on August 23 but then dropped, averaging around #450 in November.
Other social apps including Hive Social, Counter Social, Post.News, T2/Pebble, Parler, etc., didn’t appear in our Overall 500 ranking.
Here are some other trends we observed among social media apps, and how they did in our Overall ranking:
E-commerce remains as relevant as ever, something that is especially evident in our recent Cyber Week blog post. Amazon consistently tops the category, with Taobao as a solid #2. In 2022, eBay was mostly #2 but has now dropped to #3, although it had several days in early 2023 in the #2 spot, as well as on June 19.
Throughout the year, Shopify was a solid #4, overtaking eBay at #3 on Black Friday. Alibaba was #5, peaking on Singles’ Day, November 11. AliExpress followed at #6.
Rakuten and Temu battled for the #7 ranking in 2023. Temu, the Chinese-owned and Boston-based low price and fast fashion marketplace (launched in the US in September 2022) was definitely one of the surprises of the year. It expanded to Canada, Australia and New Zealand in February, and ended 2023 as #7 in front of Rakuten. It ranked #6 at times during the year, including Black Friday. Temu wasn’t in our top 10 in 2022.
The South American e-commerce platform Mercado Libre, the American retail giant Walmart, and the fast fashion brand Shein, ‘battled’ during the year for the 9th and 10th place rankings in the E-commerce category.

Looking at how e-commerce sites did in our Overall ranking, we observed the following trends:

Video streaming platforms continue to play a central role in entertaining us. YouTube remains the top service for all video streaming, with Netflix as a close second. However, within our rankings, Netflix is the clear leader among paid streaming services, followed by Disney Plus, Amazon Prime Video, Hulu, and HBO/HBO Max.
Twitch, which has a significant amount of video game live-streaming content, secured the #3 spot, as it did in 2022. Roku, a digital media player that also offers streaming services, ranked #4. Following are Disney Plus, Prime Video, Hulu, and HBO/Max (now known as Max), with Vimeo surpassing Pluto TV later in the year. Dailymotion also made a top 10 appearance in late February for a few days.

Throughout the year, Disney+ occasionally challenged Roku, especially on weekends following the premiere of the Star Wars-related miniseries Ahsoka on August 23.
Looking at how video streaming services performed in our Overall ranking, we found:
News organizations worldwide play a crucial role in keeping the public informed, especially during times of crisis like pandemics, wars, or financial uncertainty. With that in mind, the ranking of news organizations also highlights a few newsworthy trends.
Last year, BBC and Globo tied for #1 in this category. In 2023, Globo — a Brazilian media conglomerate popular in South America, was consistently #1, followed by the BBC, the UK’s national broadcaster. The only exception was on June 20, when the BBC was #1. This coincided with the Titan submersible implosion on June 18, which remained in global news for most of the following week.

Fox News and CNN also played a key role in our list. Fox News was overtaken by CNN for the #3 spot from July onwards, though Fox led again in November. The New York Times held a steady #5, followed by the Daily Mail and NPR, with the Washington Post overtaking NPR in June.
The ranking lower in the top 10 fluctuated more, with the Wall Street Journal mostly at #9, reaching #8 in September. The Guardian was #8 in October, coinciding with the start of the Hamas-Israel conflict on October 7.
Notable news trends we identified in our larger Overall ranking include:


Messaging is seen as a type of social media and remains as relevant as ever including for specific communication purposes. Apple’s iMessage is not included in this category, because it doesn’t have a unique domain name whose traffic can be analyzed. Keeping that in mind, WhatsApp remained the top messaging service in 2023, consistent with its position in 2022.
Following WhatsApp is the Chinese service QQ, also known as Tencent QQ, which includes games and mobile payments and is popular in Asia, at #2. Viber, popular in Eastern Europe, Asia, and the Middle East, comes next. The top three are unchanged from 2022. Telegram, widely used in Eastern Europe and Asia, holds the #4 spot. LINE from Japan briefly contested this position early in 2023. Signal follows, and the Chinese app WeChat is at #7, ahead of Facebook’s Messenger.
The list concludes with Microsoft’s GroupMe and the Canadian service Kik Messenger. The standings are similar to 2022, but Telegram, WeChat, and Signal have shown improvements.

Here are other messaging trends from our Overall ranking:
Is gaming part of the metaverse? In a sense, one could argue that it depends on the game, given that it is all about being immersed in another world. Concepts aside, we’ve included both in the same category since last year. Roblox was the uncontested winner of this category in 2023, followed by two services that are now much more than just popular gaming consoles, but also popular online gaming services: Microsoft’s Xbox and Sony’s PlayStation.
Xbox and PlayStation were at the podium of the list for most of the year. The top spots were similar to 2022, but PlayStation, which was #3 last year, was surpassed by Epic Games (known for Fortnite) during the last part of 2023.
Oculus, a VR headset and also metaverse experience service owned by Meta (we’re considering domains related to those two aspects provided by Meta’s Reality Labs), rose in the rankings to #5, continuing a trend from late 2022. It reached as high as #2 on June 29, and again on October 3-4 after the announcement of Quest 3 at Meta Connect 2023.
Steam was more popular than Oculus from July to September, reaching #5. Electronic Arts was a solid #7 throughout the year, and Blizzard (famous for World of Warcraft and an Activision Blizzard subsidiary), was mostly in #8. The top 10 list ends with Nintendo and Riot Games/League of Legends ‘battling’ for the #9 and #10 spots, with the latter overtaking Nintendo since September.
Here’s the top 10 chart across 2023:

The financial services sector is diverse, ranging from traditional banking to cryptocurrency-only services to tax-related services. This year, Stripe, an Irish-American payment platform, dethroned PayPal as the top service in this category.
PayPal started the year at #1 but was overtaken by Stripe on most days since March. PayPal still led on some weekends during the summer and on Black Friday. TradingView lost its #3 spot to Alipay in late July. Intuit was more popular early in the year, peaking on April 18, the Tax Day in the United States, but ended mostly at #5 on weekdays (see our 2022 blog post on how Tax Day impacts related sites for more on this trend).

The Brazilian Nubank, an online-only bank or neobank, and the largest of its kind in Latin America, was a surprise at #6, often reaching higher ranks on weekends, especially on Saturdays, as it peaked on June 3 and July 1.
Focusing on specific crypto services in the Financial category (more on crypto below), we can see that Binance lost ground throughout 2023, especially after August moving between #8 and #9. This is a change from its trend in 2022, when Binance was on a growth trajectory and ranked #6. CoinGecko, a cryptocurrency data aggregation and tracking website, made a late appearance in 2023 with several days as #6 in November, and Coinbase appeared at #10 after late October. Here’s the crypto perspective in this Financial services category:

Here are other financial services trends from our Overall ranking

In addition to our Financial Services category, we also evaluated cryptocurrency-related services in particular. Despite the disappearance of Sam Bankman-Fried’s FTX from our rankings after its bankruptcy in November 2022, the crypto sector continues to show several changes this year. Binance, Coinbase, and CoinGecko (a provider of crypto data tools) remain at the top of the list, a trend similar to 2022. While Binance held on to its #1 position across most of 2023, in the last month it was overtaken by CoinGecko.
Throughout the year, CoinGecko gained momentum, surpassing the cryptocurrency exchange platforms Binance and Coinbase in November. CoinMarketCap and NiceHash were also prominent, with CoinMarketCap reaching #4 by October. OKX, MEXC, and Crypto.com were already in the top 10 in 2022, and CryptoCompare, Kraken, and Trust: Crypto & Bitcoin, which also competed for the #10 position, were new inclusions in 2023. Kucoin and Etherscan fell out of the 2023 top 10 after being there in 2022.

What happened to Binance? In both the Financial Services and Cryptocurrency categories, we noticed Binance, a key cryptocurrency player, lost its leading position in 2023, dropping to #2 in November. The company faced challenges in July, with several top executives leaving. Then, in November, US authorities filed a lawsuit against Binance, resulting in multiple charges and fines. In our Overall ranking, Binance’s highest point was on April 19, the day after US Tax Day, ranking at #122. However, its rank fell later, though it slightly improved to around #140 by November.

Outside the categories we reviewed in the Year in Review, several notable trends emerged in our Overall ranking:
Surprise!! 1989 (Taylor’s Version) is on its way to you 🔜! The 1989 album changed my life in countless ways, and it fills me with such excitement to announce that my version of it will be out October 27th. To be perfectly honest, this is my most FAVORITE re-record I’ve ever done… pic.twitter.com/JFYOWhBxhj
— Taylor Swift (@taylorswift13) August 10, 2023

The Internet plays a role in socializing, entertaining, working, communicating, learning, and staying informed when you most need it. In our popular Internet services rankings, the dominance of giants like Google and Facebook, and the relevance of TikTok and others, underscore the continued influence of established players in shaping online interactions and content consumption. However, the rise of generative AI services, notably OpenAI’s ChatGPT, signals an exciting sector that is rapidly gaining traction. Let’s see where generative AI services can go in 2024.
In the social media realm, X/Twitter seems to be losing some influence in our ranking but continues to be highly influential, and much higher than the direct competition. Mastodon, Threads, and others still have a long way to go to compete. Although not seen as direct microblogging competition, Discord and Reddit continue to show growth.
An emerging player, Temu, made significant strides in the E-commerce realm. In the cryptocurrency space, Binance lost momentum as CoinGecko gained traction. In the gaming and metaverse sectors, the highlights included Roblox’s consistency and Oculus’s growth.
Looking ahead, the trends observed in 2023 set the stage for an even more interconnected and technologically advanced future. The growing importance of AI, the steadfast popularity of social media, and the evolving dynamics in e-commerce and financial services suggest a future where humans will have to continue to adapt to the opportunities and challenges that lie ahead.
Creating rankings is a team effort that comes with its own challenges and requires careful attention and frequent updates. If you want to help us make these categorical rankings better, you can. Feedback is appreciated, including regarding other categories to include in the 2024 Year in Review.
(Our data scientist, Sabina Zejnilovic, played a crucial role in accurately gathering the Internet services data and contributed to this blog post, as did David Belson with his guidance, along with many others.)
Post Syndicated from David Belson original http://blog.cloudflare.com/radar-2023-year-in-review/

The 2023 Cloudflare Radar Year in Review is our fourth annual review of Internet trends and patterns observed throughout the year at both a global and country/region level across a variety of metrics. Below, we present a summary of key findings, and then explore them in more detail in subsequent sections.
Cloudflare Radar launched in September 2020, and in the blog post that announced its availability, we talked about how its intent was to “shine a light on the Internet’s patterns”. Cloudflare’s network currently spans more than 310 cities in over 120 countries/regions, serving an average of over 50 million HTTP(S) requests per second for millions of Internet properties, in addition to handling over 70 million DNS requests per second on average. The data generated by this massive global footprint and scale, combined with data from complementary Cloudflare tools, enables Radar to provide unique near-real time perspectives on the patterns and trends we observe across the Internet. For the last several years (2020, 2021, 2022), we’ve been aggregating these insights into an annual Year In Review, shining a light on the Internet’s patterns over the course of that year. The new Cloudflare Radar 2023 Year In Review continues that tradition, featuring interactive charts, graphs, and maps you can use to explore notable Internet trends observed throughout this past year.
The 2023 Year In Review is organized into three sections: Traffic Insights & Trends, Connectivity & Speed, and Security. We have incorporated several new metrics this year, and have endeavored to keep underlying methodologies consistent with last year wherever possible. Website visualizations shown at a weekly granularity cover the period from January 2 through November 26, 2023. Trends for over 180 countries/regions are available on the website, with some smaller or less populated locations excluded due to insufficient data. Note that some of the metrics are presented only as a worldwide view, and will not be shown if a country/region is selected. Because of the control plane and analytics outage that occurred November 2-4, traffic data for relevant metrics has been interpolated for that three-day period.
Below, we provide an overview of the content contained within the major Year In Review sections (Traffic Insights & Trends, Connectivity & Speed, and Security), along with notable observations and key findings. In addition, we have also published a companion blog post that specifically explores trends seen across Top Internet Services.
However, the notable observations and key findings contained within this post only skim the surface of the unique insights that can be found in the Year in Review website, which we strongly encourage you to visit to explore the data in more detail and look at trends for your country/region. As you do so, we encourage you to consider how the trends presented within these blog posts and the website’s various sections impact your business or organization, and to think about how these insights can inform actions that you can take to improve user experience or enhance your security posture in the future.

Twenty-five years ago, Worldcom executives claimed that Internet traffic was doubling every 100 days (3.5 months). A quarter-century later, we know that these claims were unrealistically aggressive, but it is clear that the Internet is growing quickly as more and more devices are connected, consuming content from a growing universe of websites, applications, and services.
To determine the traffic trends over time, we first established a baseline, calculated as the average daily traffic volume (excluding bot traffic) over the second full calendar week (January 8-14) of 2023. We chose the second calendar week to allow time for people to get back into their “normal” routines (school, work, etc.) after the winter holidays and New Year’s Day. The percent change shown in our traffic trends chart is calculated relative to the baseline value, and represents a seven-day trailing average — it does not represent absolute traffic volume for a country/region. The seven-day averaging is done to smooth the sharp changes seen with a daily granularity. A trend line for 2022 is shown for comparison purposes.
Our data shows that globally, Internet traffic grew 25% in 2023, with nominal initial growth accelerating during the second half of the year. Overall, the pattern is similar to that observed in 2022 (excepting last year’s late February spike), and peak growth for the year is just slightly above the peak growth level seen in 2022. Traffic patterns in Canada were also rather consistent year-over-year, exhibiting similar seasonality, and peak growth above 30% in both 2022 and 2023. In many countries, the 2022 trend line shows a clear drop in traffic heading into the Christmas holiday, with a slight rebound ahead of New Year’s Day. It will be interesting to see if traffic follows this pattern in 2023 as well.


Comparisons with 2022 traffic trends helps make the impact of major holidays on Internet traffic more visible. For example, in Muslim countries including Indonesia, Turkey, and the United Arab Emirates, the celebration of Eid-Ul-Fitr, the festival marking the end of the fast of Ramadan, is visible as a noticeable drop in traffic around April 21-23, 2023, just before a similar drop visible in the 2022 trend line during last year’s celebration on May 2-3. In Italy, a drop in traffic is clearly visible around Pasqua di Resurrezione and Lunedì dell’Angelo (Easter Sunday and Monday) on April 9-10, one week ahead of a similar drop in traffic in 2022

In addition, extended disruptions to Internet connectivity are also clearly visible within the traffic trend charts. Examples include Mauritania, where government-directed shutdowns occurred from March 6-12 and May 30 – June 6, and Gabon, where a shutdown was in place from August 26-30, as well as Guam, where Super Typhoon Mawar caused a multi-week drop in traffic starting on May 24.


One of the most popular sections of the Year In Review over the last several years has been the exploration of the most popular Internet services, both generally and across a number of categories. These rankings of service popularity are based on analysis of anonymized query data of traffic to our 1.1.1.1 public DNS resolver from millions of users around the world. Although DNS resolution operates at a domain level, domains that belong to a single Internet service are grouped together for the purposes of these rankings.
In the overall category, Google once again held the top spot, owing in part to its broad portfolio of services as well as the popularity of the Android mobile operating system. In addition to perennial categories like e-commerce, video streaming, and messaging, this year we also looked at Generative AI, which has been on a meteoric rise in 2023. In this category, OpenAI held the top spot, building on the success and popularity of ChatGPT, which it launched only a year ago. And despite the turmoil seen in the cryptocurrency space this year, Binance remained the most popular service in that category.
We explore these categorical rankings, as well as trends seen by specific services, in more detail in a separate blog post.

Apple’s iOS and Google’s Android are the two leading operating systems used on mobile devices, and analysis of information in the user agent reported with each request allows us to gain insight into the distribution of traffic by client operating system throughout the year. Given the wide range of both devices and price points for Android devices, it is not surprising that Android is responsible for the majority of mobile device traffic when aggregated globally.
Globally, over two-thirds of mobile device traffic was from Android devices. The split is in line with Android/iOS usage observed in 2022. When looking at the countries/regions with the highest levels of Android usage, we find Bangladesh and Papua New Guinea at the top of the list, both with over 95% of mobile device traffic coming from Android devices. Looking more closely at other countries that see particularly high levels of Android usage, it is interesting to note that they are largely in Africa, Oceania/Asia, and South America, and that many have lower levels of gross national income per capita. This is presumably where the availability of lower priced “budget” phones plays to Android’s advantage from an adoption perspective.
In contrast, while the share of mobile device traffic from iOS at a country/region level never tops 70%, many of the countries with an iOS share over 50%, including Denmark, Australia, Japan, and Canada, have comparatively higher gross national income per capita, which likely speaks to a greater ability to afford higher priced devices.

SpaceX’s Starlink high-speed satellite Internet service has continued to rapidly grow its footprint since launching in 2019, making high performance Internet connections available in many countries/regions that were previously unserved or underserved by traditional wired or wireless broadband. The current leader in the space, in the future it will be joined by Amazon’s Project Kuiper service, which launched its first two test satellites this year, as well as Eutelsat OneWeb, which grew its satellite constellation in 2023 as well.
To track the growth in usage and availability of Starlink’s service, we analyzed aggregate Cloudflare traffic volumes associated with the service’s autonomous system (AS14593) throughout 2023. Although Starlink is not yet available globally, we did see traffic growth across a number of countries/regions. The request volume shown on the trend line in the chart represents a seven-day trailing average. A trend line for 2022 is shown for comparison purposes, and is scaled to the maximum value across 2022 and 2023.
Globally, we saw Starlink traffic more than triple this year. In the United States, traffic from Starlink was up over 2.5x, and grew over 17x in Brazil. In countries where Starlink turned up service in 2023, including Kenya, the Philippines, and Zambia, we saw traffic grow rapidly once the service became available.

Modern websites are complex productions, relying on a mix of frameworks, platforms, services, and tools, and the developer community is responsible for making them coexist with one another to deliver a seamless experience. Using the Cloudflare Radar URL Scanner, which we launched in March 2023, we scanned websites associated with the top 5000 domains to identify the most popular technologies and services used across a dozen different categories, including (but not limited to) Analytics, where Google Analytics was by far the most widely used; JavaScript Frameworks, where React had a commanding lead; and Marketing Automation providers, where leader HubSpot was closely followed by several competitors.

HTTP (HyperText Transfer Protocol) is the core protocol that the web relies upon. HTTP/1.0 was first standardized in 1996, HTTP/1.1 in 1999, and HTTP/2 in 2015. The most recent version, HTTP/3, was completed in 2022, and runs on top of QUIC, a new transport protocol. On the client side, HTTP/3 support is enabled by default in the latest versions of desktop and mobile Google Chrome and Mozilla Firefox, and for a portion of Apple Safari users. HTTP/3 is available for free for all Cloudflare customers, though not every customer chooses to enable it.
Using QUIC allows HTTP/3 to deliver improved performance by mitigating the effects of packet loss and network changes, as well as establishing connections more quickly. It also provides encryption by default, mitigating the risk of attacks. Websites and applications that remain on older versions of HTTP miss out on these benefits.
Analysis of the HTTP version negotiated for each request allows us to gain insight into the distribution of traffic by the various versions of the protocol aggregated throughout the year. (“HTTP/1.x” aggregates requests made over HTTP/1.0 and HTTP/1.1.) At a global level, 20% of requests were made over the latest version, HTTP/3. Another third of requests were made over the comparatively ancient HTTP/1.x versions, while HTTP/2 remained dominant, and accounted for the 47% balance.

Looking at the version distribution geographically, we found a number of Asian countries, including Nepal, Thailand, Malaysia, and Sri Lanka among those with highest rates of HTTP/3 usage, although these rates did not exceed 35%. In contrast, more than half of the requests from ten countries, including Ireland, Albania, Finland, and China, were made over HTTP/1.x during 2023.
In addition, as developers increasingly use automated API calls to power dynamic websites and applications, we can use our unique visibility into Web traffic to identify the top languages these API clients are written in. Looking at API-related requests determined to not be coming from a person using a browser or native mobile application, we applied heuristics to help identify the language used to build the client.
Our analysis found that almost 15% of automated API requests are made by NodeJS clients, with Go, Java, Python, and .NET holding smaller shares.

Cloudflare Radar enables users to see Internet traffic trends at a country/region or network level over a selected period of time. However, we wanted to zoom out a bit, and look at the traffic Cloudflare saw from the entire IPv4 Internet over the course of the entire year. Hilbert curves, as “continuous space-filling curves”, have properties that are useful for visualizing the Internet’s IPv4 address space.
Using a Hilbert curve visualization, we can visualize aggregated request traffic (over IPv4) to Cloudflare from January 1st through November 26th, 2023. In order to make the amount of data used for the visualization manageable, IP addresses are aggregated at a /20 level, meaning that at the highest zoom level, each cell represents traffic from 4096 IPv4 addresses. (The sheer size of the IPv6 address space would make associated traffic very hard to see in such a visualization, especially as such a small amount has been allocated for assignment by the Regional Internet Registries.)
Within the visualization, IP addresses are grouped by ownership, and for much of the IP address space shown there, a mouseover at the default zoom level will show the Regional Internet Registry (RIR) that the address block belongs to. However, there are also a number of blocks that were assigned prior to the existence of the RIR system, and for these, they are labeled with the name of the organization that owns them. Progressive zooming ultimately shows the autonomous system and country/region that the IP address block is associated with, as well as its share of traffic relative to the maximum. (If a country/region is selected, only the IP address blocks associated with that location are visible.) Overall traffic shares are indicated by shading based on a color scale, and although a number of large unshaded blocks are visible, this does not necessarily mean that the associated address space is unused, but rather that it may be used in a way that does not generate traffic to Cloudflare.

Areas of higher request volume, indicated by warmer orange/red shading, are visibly scattered throughout the plot, but the IP address block that had the maximum request volume to Cloudflare during 2023 was 66.249.64.0/20, which belongs to Google. This IP address block is one of several used by the Googlebot web crawler, which is a likely explanation for the high request volume, given the number of web properties on Cloudflare’s network.

It is hard to do this visualization justice with a short summary and static screenshot. To explore it in more detail, we encourage you to go to the Year in Review website and explore it by dragging and zooming to move around the IPv4 Internet.

During 2023, we have written frequently about Internet outages, whether due to technical issues, government-directed shutdowns, or geopolitical conflict, as well as infrastructure resilience issues (including fiber cuts, power outages, and severe weather) highlighted in our quarterly summaries. The impacts of these outages can be significant, including significant economic losses and severely limited communications. The Cloudflare Radar Outage Center tracks these Internet outages, and uses Cloudflare traffic data for insights into their scope and duration.
Some of these outages seen through the year were short-lived, lasting just a couple of hours, while others have stretched on for multiple months. In the latter category, localized government-directed shutdowns in Manipur, India and Amhara, Ethiopia have lasted over seven and four months respectively (as of early December). In the former category, Iraq frequently experienced multi-hour nationwide Internet shutdowns intended to prevent cheating on academic exams — these contribute to the clustering visible in the timeline during June, July, and August.
Within the timeline on the Year in Review website, mousing over a dot will display metadata about that outage, and clicking on it will open a page with additional information. If a country/region is selected, only outages for that country will be displayed.

IPv6 has been around in some fashion since 1998, with an expanded address space that better supports the universe of Internet-connected devices that has grown exponentially over the last quarter-century. And over that time, available IPv4 space has been exhausted, leading connectivity providers to resort to solutions like Network Address Translation, and cloud and hosting providers to acquire blocks of IPv4 address space for as much as $50 per address. IPv6 also brings a number of other benefits to network providers, and if implemented correctly, adoption should be transparent from an end user perspective.
Cloudflare has been a vocal and active advocate for IPv6 stretching all the way back to our first birthday in 2011, when we announced our Automatic IPv6 Gateway, which enabled free IPv6 support for all of our customers. Just a few years later, we enabled IPv6 support by default for all of our customers. (Although it is enabled by default, not all customers choose to keep it enabled for a variety of reasons.) However, this support is only half of the equation for driving IPv6 adoption, as end user connections need to support it as well. (Technically, it is a bit more complex than that, but those are the two foundational requirements.) Analysis of the IP version used for each request made to Cloudflare allows us to gain insight into the distribution of traffic by the various versions of the protocol, aggregated throughout the year.
Thanks to near-complete IPv6 adoption by Indian telecommunications provider Reliance Jio, 70% of dual-stacked requests from Indian users were made via IPv6. India was followed closely by Malaysia, where 66% of dual-stacked requests were made over IPv6 during 2023, thanks to strong IPv6 adoption rates across leading Internet providers within the country. Other countries that saw more than half of dual-stacked requests, on average, made over IPv6 include Saudi Arabia, Vietnam, Greece, France, Uruguay, and Thailand. In contrast, there were on the order of 40 countries/regions where less than 1% of dual-stacked requests were made over IPv6 during 2023. Lagging adoption across such a large cohort of countries/regions 25 years after IPv6 was first published as a draft standard is quite surprising.

Even when they are not facing Internet outages, users around the world are often contending with poor performance on their Internet connections, whether due to low speeds, high latency, or a combination of these factors. Although Internet providers continue to evolve their service portfolios to offer increased connection speeds and reduced latency in order to support growth in use cases like online gaming and videoconferencing, consumer adoption is often mixed due to cost, availability, or other issues. By aggregating the results of speed.cloudflare.com tests taken during 2023, we can get a geographic perspective on connection quality metrics including average download and upload speeds, and average idle and loaded latencies, as well as the distribution of the measurements.
In Iceland, over 85% of all Internet connections are over fiber, and this is reflected in its ranking as the country with the best overall Internet quality metrics, as speed test results show that providers there deliver the highest average speeds (282.5 Mbps download, 179.9 Mbps upload) and lowest average latencies (9.6 ms idle, 77.1 ms loaded). The histogram below shows that while there is a large cluster of download speeds between 0–100 Mbps, there were also a significant number of tests that measured even higher speeds, including some in excess of 1 Gbps.
Western European countries including Spain, Portugal, and Denmark also ranked among the top 10 across multiple Internet quality metrics.

Over the last 15 years or so, mobile devices have become increasingly ubiquitous, becoming indispensable in both our personal and professional lives, thanks in large part to their ability to enable us to access the Internet from nearly anywhere at any time. In some countries/regions, mobile devices primarily connect to the Internet via Wi-Fi, while others are “mobile first”, where Internet access is primarily through 4G/5G services.
Analysis of information contained with the user agent reported with each request to Cloudflare enables us to categorize it as coming from a mobile, desktop, or other type of device. Aggregating this categorization throughout the year at a global level, we found that 42% of traffic came from mobile devices, with 58% coming from desktop devices such as laptops and “classic” PCs. These traffic shares were in line with those measured in 2022. 79% of traffic came from mobile devices in Zambia, making it the country with the largest mobile device traffic share in 2023. Other countries/regions that had more than 50% of traffic come from mobile devices were concentrated in the Middle East/Africa, the Asia Pacific Region, and South/Central America. In contrast, Finland had one of the highest shares of desktop device traffic, at 80%.


Malicious bots are often used to attack websites and applications. To protect customers from these threats, Cloudflare mitigates (blocks) this attack traffic using DDoS mitigation techniques or Web Application Firewall (WAF) Managed Rules. However, customers may also choose to have Cloudflare mitigate traffic using other techniques for a variety of other reasons, such as rate-limiting requests, or blocking all traffic from a given location, even if it isn’t malicious. Analyzing traffic to Cloudflare’s network seen throughout 2023, we looked at the overall share that was mitigated (for any reason), as well as the share that was mitigated as a DDoS attack or by WAF Managed Rules.
Overall, just under 6% of global traffic was mitigated by Cloudflare’s systems as being potentially malicious or for customer-defined reasons, while only around 2% of it saw DDoS/Managed WAF mitigations. Some countries, such as Bermuda, saw the percentages for the two metrics track very closely, while other countries, like Pakistan and South Africa showed much larger gaps between their trend lines.

Bot traffic describes any non-human Internet traffic, and monitoring bot traffic levels can help site and application owners spot potentially malicious activity. Of course, bots can be helpful too, and Cloudflare maintains a list of verified bots to help keep the Internet healthy. Verified bots include those used for things like search engine indexing, performance testing, and availability monitoring. Regardless of intent, we wanted to look at where bot traffic was coming from, and we can use the IP address of a request to identify the network (autonomous system) and country/region associated with the bot making the request. Perhaps unsurprisingly, we found that cloud platforms were among the leading sources of bot traffic. This is likely due to the ease of automating the provisioning/teardown of compute resources and the relatively low cost of doing so, the distributed geographic footprint of cloud platforms, and the availability of high-bandwidth connections.
Globally, nearly 12% of bot traffic comes from Amazon Web Services, and over 7% from Google. Some of it comes from consumer ISPs as well, with U.S. broadband provider Comcast originating over 1.5% of global bot traffic. A disproportionate amount of bot traffic originates from the United States, responsible for nearly a third of global bot traffic, four times that of Germany, which originates just 8%. Within the United States, Amazon’s total share of bot traffic just edges out Google’s.


The industries targeted by attacks often shift over time, depending on the intent of the attackers. They may be trying to cause financial harm by attacking ecommerce sites during a busy shopping period, or they may be trying to make a political statement by attacking government-related sites. To identify industry-targeted attack activity during 2023, we analyzed mitigated traffic for customers that had an associated industry and vertical within their customer record. Mitigated traffic was aggregated weekly by source country/region across 18 target industries.
At a global level, Finance organizations were the most attacked over the course of the year, though we saw a significant amount of volatility from week-to-week. Interestingly, some clustering was evident, as Finance, which includes organizations that provide websites and applications for mobile payments, investments/trading, and cryptocurrency, was also a top target for a number of European countries, including Austria, Switzerland, France, the United Kingdom, Ireland, Italy, and the Netherlands, as well as in North America, for Canada, the United States, and Mexico. The Health industry, which includes companies that make exercise equipment, as well medical testing device manufacturers, was a top target across multiple African countries, including Benin, Côte d’Ivoire, Cameroon, Ethiopia, Senegal, and Somalia.
Overall, however, the year started slowly, with no industry seeing more than 8% of traffic being mitigated. As the first quarter progressed, Professional Services and News/Media/Publications organizations saw spikes in the share of mitigated traffic later in January, with Health jumping in mid-February and Law & Government organizations seeing a sharp increase in mitigated traffic in early March. Customers in the Arts/Entertainment/Recreation industry classification were apparently targeted by a multi-week attack campaign, with more than 20% of traffic mitigated during the weeks of March 26, April 2, and April 9. The overall peak during the year was experienced by the Professional Services industry, which saw a mitigated traffic share of 38.4% for the week of August 6, nearly twice its January spike. The timing of spikes and the industries experiencing those spikes varied widely across countries/regions.

In August 2023, we published a blog post that explored traffic seen by Cloudflare for the most commonly exploited vulnerabilities of 2022, as listed in a joint Cybersecurity Advisory. These included vulnerabilities in the Log4j Java-based logging utility, Microsoft Exchange, Atlassian’s Confluence platform, VMWare, and F5’s BIG-IP traffic management system. Although these are older vulnerabilities, attackers continued to actively target and exploit them throughout 2023, in part because organizations are frequently slow to follow the recommendations outlined in the Cybersecurity Advisory. We updated the analysis done for our blog post to include just the attack activity seen in 2023.
Attack activity by vulnerability varied by location, and in some, attacks targeted only a subset of the vulnerabilities. Aggregated worldwide, attack volume targeting Log4j consistently dwarfed that seen for the other vulnerabilities, and saw spikes during the last week of October and mid-late November; attack activity targeting Atlassian vulnerabilities increased in late July and trended slowly higher through the rest of the year. At a country/region level, Log4j was generally the most targeted vulnerability. In countries including France, Germany, India, and the United States, associated attack volume remained at a significant level throughout the year, while in other countries/regions, these attacks are most visible as infrequent, short-lived spikes within a country/region’s graphs, punctuating otherwise low levels of attack volume.

We also expect that through 2024, attackers will continue to target the HTTP/2 Rapid Reset vulnerability disclosed in October. The vulnerability (see CVE-2023-44487 for details) abuses an underlying weakness in the request cancellation feature of the HTTP/2 protocol, leading to resource exhaustion on the target web/proxy server. Between the end of August and the beginning of October, we saw a number of attacks targeting this vulnerability. Across this set of attacks, the average attack rate was 30M requests per second (rps), with nearly 90 peaking above 100M rps, and the largest one hitting 201M rps. This largest attack was nearly 3x bigger than our previous biggest attack on record.
One notable concern about this vulnerability is that the attacker was able to generate such a large attack with a botnet consisting of just 20,000 compromised systems. This is much smaller than some of the largest botnets today, which comprise hundreds of thousands or millions of hosts. With average web traffic estimated to be between 1–3 billion requests per second, attacks using this method could conceivably focus an entire web’s worth of requests on a few unsuspecting targets.

Post-quantum refers to a new set of cryptographic techniques that can protect data from adversaries with the ability to capture and store today’s data for decryption by sufficiently powerful quantum computers in the future. The Cloudflare Research team has been exploring post-quantum cryptography since 2017.
In October 2022, we enabled post-quantum key agreement at our edge by default, but use of it requires that the browser support it as well. Google’s Chrome browser started to slowly enable support in August 2023, and we expect its support will continue to grow in 2024, and that other browsers will add support over time as well. In September 2023, we announced general availability of post-quantum cryptography for both inbound and outbound connections and for many internal services, and expect to finish upgrading all internal services by the end of 2024.
After first enabling support in August, Chrome began ramping the number of browsers (version 116 and later) that use post-quantum cryptography, resulting in gradual growth leading to the significant increase seen on November 8. These actions helped push the share of TLS 1.3 traffic using post-quantum encryption to 1.7% at the end of November. As this ramp continues with future Chrome updates, and as other browsers add support for post-quantum encryption, we expect this share to continue to grow rapidly in 2024.

As the #1 business application, email represents a very attractive entry point into enterprise networks for attackers. Targeted malicious emails may attempt to impersonate an otherwise legitimate sender, try to get the user to click on a deceptive link, or contain a dangerous attachment, among other types of threats. Cloudflare Area 1 Email Security protects customers from email-based attacks, including those carried out through targeted malicious email messages. Over the course of 2023, an average of 2.65% of emails analyzed by Cloudflare Area 1 were found to be malicious. Aggregated at a weekly level, spikes to over 3.5%, 4.5%, and over 5% were seen in early February, early September, and late October respectively.

When carrying out attacks using malicious email messages, attackers use a variety of techniques, which we refer to as threat categories. These categories are defined and explored in detail in Cloudflare’s 2023 phishing threats report. Analysis of malicious emails shows that messages may contain multiple types of threats, highlighting the need for a comprehensive email security solution. Exploring threat activity trends for these categories, aggregated weekly across the year, we found that as much as 80% of them contained deceptive links.
However, it appears that attackers may have started to shift strategies in August, as the percentage of emails containing deceptive links began to fall while the share proposing to extort the recipient began to increase. By the end of October, and into November, the two threat categories had traded places, with nearly 80% of analyzed malicious emails containing an extortion threat, while only 20% contained deceptive links, as seen towards the right side of the graph below. However, this extortion campaign may have been short-lived, as its percentage fell almost as quickly as it rose. Identity deception and credential harvesting were also commonly identified threats, though the share of emails they were found in gradually declined over the course of the year.

Border Gateway Protocol (BGP) is the routing protocol for the Internet, communicating routes between networks, enabling traffic to flow between source and destination. However, because it relies on trust between networks, incorrect information shared between peers, whether done so intentionally or not, can send traffic to the wrong place, potentially with malicious results. Resource Public Key Infrastructure (RPKI) is a cryptographic method of signing records that associate a BGP route announcement with the correct originating AS number. In simple terms, it provides a way of ensuring that the information being shared originally came from a network that is allowed to do so. (Note that this is only half of the challenge of implementing routing security, as network providers also need to validate these signatures and filter out invalid announcements.) In the United States, the federal government recognizes the importance of routing security, with the Federal Communications Commission holding a “Border Gateway Protocol Security Workshop” on July 31.
Cloudflare has been a strong proponent of routing security, from being a founding participant in the MANRS CDN and Cloud Programme, to releasing an RPKI toolkit for network operators, to providing a public tool that enables users to test whether their Internet provider has implemented BGP safely, to presenting at this summer’s FCC workshop.
Building on the July release of the new Routing page on Cloudflare Radar, we analyzed data from RIPE NCC’s RPKI daily archive to determine the share of RPKI valid routes (as opposed to those route announcements that are invalid or whose status is unknown) and how that share has changed over the course of 2023. Since the start of the year, the global share of RPKI valid routes grew to nearly 45%, up six percentage points from the end of 2022. At a country/region level, we are looking at routes announced by autonomous systems associated with the given country/region. In the United States, the increased FCC attention on routing security is arguably warranted, as less than a third of the routes are RPKI valid. Although this is significantly better than South Korea, where less than 1% of announced routes are RPKI valid, it trails Vietnam significantly, where the share increased 35 percentage points during the first half of the year to 90%.

In the Cloudflare Radar 2023 Year In Review, we have attempted to provide a snapshot of the Internet, as dynamic as it is, through trend graphs and summary statistics, providing unique perspectives on Internet traffic, Internet quality, and Internet security, and how key metrics across these areas vary around the world.
As we said in the introduction, we strongly encourage you to visit the Cloudflare Radar 2023 Year In Review website and explore the trends relevant to metrics, countries/regions, and industries of interest, and to consider how they impact your organization so that you are appropriately prepared for 2024.
If you have any questions, you can contact the Cloudflare Radar team at [email protected] or on social media at @CloudflareRadar (X/Twitter), cloudflare.social/@radar (Mastodon), and radar.cloudflare.com (Bluesky).
As we noted last year, it truly is a team effort to produce the data, website, and content for our annual Year in Review, and I’d like to acknowledge those team members that contributed to this year’s effort. Thank you to: Sabina Zejnilovic, Jorge Pacheco, Carlos Azevedo (Data Science); Arun Chintalapati, Reza Mohammady (Design); Vasco Asturiano, Nuno Pereira, Tiago Dias (Front End Development); João Tomé (Most popular Internet services); and Davide Marquês, Paula Tavares, Celso Martinho (Project/Engineering Management) as well as countless other colleagues for their answers, edits, and ideas.