Post Syndicated from original https://lwn.net/Articles/894836/
Over on the Python Software Foundation (PSF) blog, Alex Waygood has a report from this year’s Python Language Summit. There are reports from each of the nine sessions, including “Python without the GIL“, The ‘Faster CPython’ project: 3.12 and beyond“, “F-Strings in the grammar“, lightning talks, and more.
Post Syndicated from VassilKendov original http://kendov.com/%D0%B7%D0%B0%D0%BA%D0%BE%D0%BD-%D0%B7%D0%B0-%D0%BA%D0%BE%D0%BB%D0%B5%D0%BA%D1%82%D0%BE%D1%80%D1%81%D0%BA%D0%B8%D1%82%D0%B5-%D1%84%D0%B8%D1%80%D0%BC%D0%B8/
В Обединеното кралство се води статистика и се следи строго, броят на новите закони да съвпада с броя на отпадналите такива.
У нас явно тенденцията е да се натрупват един върху друг и да създават противоречия.
Последният закон за колекторските фирми, лобиран от Корнелия Нинова, е точно такъв – излишен и отчитащ дейност. При наличие на ЧСИ и Публични изпълнители с много права по отношение на събирането на вземания, е напълно излишно да се регламентира дейността на нови структури, занимаващи се със същата дейност.
За срещи и консултации по банкови неволи, моля използвайте посочената форма.
[contact-form-7]
The post Законът за колекторските фирми на Корнелия Нинова – полезен или буря в чаша вода? appeared first on Kendov.com.
Post Syndicated from Göksel SARIKAYA original https://aws.amazon.com/blogs/big-data/orchestrate-big-data-jobs-on-on-premises-clusters-with-aws-step-functions/
Customers with specific needs to run big data compute jobs on an on-premises infrastructure often require a scalable orchestration solution. For large-scale distributed compute clusters, the orchestration of jobs must be scalable to maximize their utilization, while at the same time remain resilient to any failures to prevent blocking the ever-growing influx of data and jobs. Moreover, on-premises compute resources can’t be extended on demand, therefore, the jobs may be competing for the same resources with different priorities.
This post showcases serverless building blocks for orchestrating big data jobs using AWS Step Functions, AWS Lambda, and Amazon DynamoDB with a focus on reliability, maintainability, and monitoring. In this solution, Step Functions enables thousands of workflows to run parallel. Additionally, Lambda provides flexibility implementing arbitrary interfaces to the on-premises infrastructure and its compute resources. With additional steps in the orchestration, the solution also allows operations to monitor thousands of parallel jobs in a visual interface for better debugging.
The proposed serverless solution consists of the following main components:
The following diagram illustrates the solution architecture.
The main process of the solution consists of seven steps:
In this post, we assume the on-premises compute cluster offers interfaces to interact with the compute resources. For example, customers could run a Spark compute cluster on premises that allows the following basic interactions through an API:
In addition, we assume the cluster can provide metrics on its current utilization. For example, the cluster could provide Prometheus metrics as aggregates over all resources within a compute cluster:
We use the terminology introduced here for the example in this post. Depending on the capabilities of the on-premises cluster, you can adjust these concepts. For example, the compute cluster could use Kubernetes or SLURM instead of Spark.
The job manager is responsible for communicating with on-premises clusters to trigger big data jobs and query their status. It’s a Step Functions state machine that consists of three steps, as illustrated in the following figure.
The first step is JobQueueRequest, which makes a request to the job queue manager component and waits for the callback. When the job queue manager sends OK to the waiting step with a callback pattern, the second step StartJobRun runs.
The StartJobRun step communicates with the on-premises environment (for example, via HTTP post to a REST API endpoint) to trigger an on-premises job.
The third step GetJobStatus queries the job status from the on-premises cluster. If the job status is InProgress, the state machine waits for a configured time. When the Wait state is over, it returns to the GetJobStatus step to query the job status again in a loop. When the job returns a successful state from the on-premises cluster, the state machine completes its cycle with a Success state. If the job fails with a timeout or with an error, the state machine completes its cycle with a Fail state.
The following screenshot shows the details of the state machine on the Step Functions console.
The job queue manager is responsible for managing job queues based on job priorities and cluster utilization. It consists of DynamoDB, Lambda, and EventBridge.
The JobQueue table keeps data of waiting jobs, including jobId as the primary key, priority as the sort key, needed memory and CPU consumptions, callbackId, and timestamp information. You can add further information to the table dynamically if required by the scheduling algorithm.
The following screenshot shows the attribute details of the JobQueue table.
EventBridge triggers the job scheduler Lambda function on a regular bases in a configured interval. First, the job scheduler function gets waiting jobs data from the JobQueue table in DynamoDB. Then it establishes a connection with the on-premises cluster to fetch cluster metrics such as memory and CPU utilization. Based on this information, the function decides which jobs are ready to be triggered on the on-premises cluster.
The scheduling algorithm proposed here follows a simple concept to maximize resource utilization, while respecting the job priority. Essentially, for an on-premises cluster (we could potentially have multiple in different geographies), the job scheduler Lambda function builds a queue of jobs according to their priority, while allocating the first job in the queue to compute resources on the cluster. If enough resources are available, the scheduler moves to the next job in the queue and repeats.
Due to the flexibility of Lambda functions, you can tailor the scheduling algorithm for a specific use case. Cluster scheduling algorithms are still an open research topic with different optimization goals, such as throughput, data location, fairness, deadlines, and more.
In this section, we provide a starting point for the solution described in this post. The steps walk you through creating a Step Functions state machine with the appropriate template, and the necessary Lambda and DynamoDB interactions to create the job manager and job queue manager building blocks. Example code for the Lambda functions is excluded from this post, because the communication with the on-premises cluster to trigger jobs can vary depending on your on-premises interface.
This solution uses Step Functions to track all jobs until completion, and therefore the Step Functions quotas must be considered for potential use cases. Mainly, a workflow can run for a maximum of 1 year (cannot be increased) and by default 1 million parallel runs can run in a single account (can be increased to millions). See Quotas for further details.
This post described how to orchestrate big data jobs running in parallel on on-premises clusters with a Step Functions workflow. To learn more about how to use Step Functions workflows for serverless orchestration, visit Serverless Land.
Göksel Sarikaya is a Senior Cloud Application Architect at AWS Professional Services. He enables customers to design scalable, high-performance, and cost effective applications using the AWS Cloud. He helps them to be more flexible and competitive during their digital transformation journey.
Nicolas Jacob Baer is a Senior Cloud Application Architect with a strong focus on data engineering and machine learning, based in Switzerland. He works closely with enterprise customers to design data platforms and build advanced analytics/ml use-cases.
Shukhrat Khodjaev is a Senior Engagement Manager at AWS ProServe, based out of Berlin. He focuses on delivering engagements in the field of Big Data and AI/ML that enable AWS customers to uncover and to maximize their value through efficient use of data.
Post Syndicated from Crosstalk Solutions original https://www.youtube.com/watch?v=HkBrIS8QLYU
Post Syndicated from Greg Hamer original https://www.backblaze.com/blog/announcing-developer-quick-starts-access-open-source-code-you-can-build-on/
Developing finished applications always requires coding custom functionality, but, as a developer, isn’t it great when you have pre-built, working code you can use as scaffolding for your applications? That way, you can get right to the custom components.
To help you finish building applications faster, we are launching our Developer Quick Start series. This series provides developers with free, open-source code available for download from GitHub. We also built pre-staged buckets with a browsable media application and sample data. For read-only API calls against those buckets, we are sharing API key pairs for programmatic access to these pre-staged buckets. That means you can download the code, run it, and see the results, all without even having to create a Backblaze account!
Today, we’re debuting the first Quick Start in the series—using Python with the Backblaze S3 Compatible API. Read on to get access to all of the resources, including the code on GitHub, sample data to run it against, a video walkthrough, and guided instructions.
All of the resources you need to use Python with the Backblaze S3 Compatible API are linked below:
Depending on your skill level, the open-source code may be all that you need. If you’re new to the cloud, or just want a deeper, guided walk-through on the source code, check out the written code walk-throughs and video-guided code walk-throughs, too. Whatever works best for you, please feel free to mix and match as you see fit.
The Quick Start walks you through how to perform create and delete API operations inside your own account, all of which can be completed using Backblaze B2 Cloud Storage—and the first 10GB of storage per month are on us.
With the Quick Start code we are sharing, you can get basic functionality working and interacting with B2 Cloud Storage in minutes.
Know someone who might be interested in leveraging the power and ease of cloud storage? Feel free to share these resources at will. Also, we welcome your participation in the projects on GitHub via pull requests. If you are satisfied, feel free to star the project on GitHub or like the videos on YouTube.
Finally, please explore our other Backblaze B2 Sample Code Repositories up on GitHub.
The initial launch of the Developer Quick Start series logic is available in Python. We will be rolling out Developer Quick Starts for other languages in the months ahead.
Which programming languages (or scripting environments) are of most interest for you? Please let us know in the comments down below. We are continually adding more working examples in GitHub projects, both in Python and in additional languages. Your feedback in the comments below can help guide what gets priority.
We look forward to hearing from you about how these Developer Quick Starts work for you!
The post Announcing Developer Quick Starts: Open-source Code You Can Build On appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.
Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/amazon-ec2-now-supports-nitrotpm-and-uefi-secure-boot/
In computing, Trusted Platform Module (TPM) technology is designed to provide hardware-based, security-related functions. A TPM chip is a secure crypto-processor that is designed to carry out cryptographic operations. There are three key advantages of using TPM technology. First, you can generate, store, and control access to encryption keys outside of the operating system. Second, you can use a TPM module to perform platform device authentication by using the TPM’s unique RSA key, which is burned into it. And third, it may help to ensure platform integrity by taking and storing security measurements.
During re:Invent 2021, we announced the future availability of NitroTPM, a virtual TPM 2.0-compliant TPM module for your Amazon Elastic Compute Cloud (Amazon EC2) instances, based on AWS Nitro System. We also announced Unified Extensible Firmware Interface (UEFI) Secure Boot availability for EC2.
I am happy to announce you can start to use both NitroTPM and Secure Boot today in all AWS Regions outside of China, including the AWS GovCloud (US) Regions.
You can use NitroTPM to store secrets, such as disk encryption keys or SSH keys, outside of the EC2 instance memory, protecting them from applications running on the instance. NitroTPM leverages the isolation and security properties of the Nitro System to ensure only the instance can access these secrets. It provides the same functions as a physical or discrete TPM. NitroTPM follows the ISO TPM 2.0 specification, allowing you to migrate existing on-premises workloads that leverage TPMs to EC2.
The availability of NitroTPM unlocks a couple of use cases to strengthen the security posture of your EC2 instances, such as secured key storage and access for OS-level volume encryption or platform attestation for measured boot or identity access.
Secured Key Storage and Access
NitroTPM can create and store keys that are wrapped and tied to certain platform measurements (known as Platform Configuration Registers – PCR). NitroTPM unwraps the key only when those platform measurements have the same value as they had at the moment the key was created. This process is referred to as “sealing the key to the TPM.” Decrypting the key is called unsealing. NitroTPM only unseals keys when the instance and the OS are in a known good state. Operating systems compliant with TPM 2.0 specifications use this mechanism to securely unseal volume encryption keys. You can use NitroTPM to store encryption keys for BitLocker on Microsoft Windows. Linux Unified Key Setup (LUKS) or dm-verity on Linux are examples of OS-level applications that can leverage NitroTPM too.
Platform Attestation
Another key feature that NitroTPM provides is “measured boot” a process where the bootloader and operating system extend PCRs with measurements of the software or configuration that they load during the boot process. This improves security in the event that, for example, a malicious program overwrites part of your kernel with malware. With measured boot, you can also obtain signed PCR values from the TPM and use them to prove to remote servers that the boot state is valid, enabling remote attestation support.
How to Use NitroTPM
There are three prerequisites to start using NitroTPM:
How to Create an AMI with TPM Enabled
AWS provides AMIs for multiple versions of Windows with TPM enabled. I can verify if an AMI supports NitroTPM using the DescribeImagesAPI call. For example:
aws ec2 describe-images --image-ids ami-0123456789
When NitroTPM is enabled for the AMI, “TpmSupport”: “v2.0” appears in the output, such as in the following example.
{
"Images": [
{
...
"BootMode": "uefi",
"TpmSupport": "v2.0"
}
]
}I may also query for tpmSupport using the DescribeImageAttribute API call.
When creating my own AMI, I may enable TPM support using the RegisterImage API call, by setting boot-mode to uefi and tpm-support to v2.0.
aws ec2 register-image \
--region us-east-1 \
--name my-image \
--boot-mode uefi \
--architecture x86_64 \
--root-device-name /dev/xvda \
--block-device-mappings DeviceName=/dev/xvda,Ebs={SnapshotId=snap-0123456789example} DeviceName=/dev/xvdf,Ebs={VolumeSize=10} \
--tpm-support v2.0Now that you know how to create an AMI with TPM enabled, let’s create a Windows instance and configure BitLocker to encrypt the root volume.
A Walk Through: Using NitroTPM with BitLocker
BitLocker automatically detects and uses NitroTPM when available. There is no extra configuration step beyond what you do today to install and configure BitLocker. Upon installation, BitLocker recognizes the TPM module and starts to use it automatically.
Let’s go through the installation steps. I start the instance as usual, using an AMI that has both uefi and TPM v2.0 enabled. I make sure I use a supported version of Windows. Here I am using Windows Server 2022 04.13.
Once connected to the instance, I verify that Windows recognizes the TPM module. To do so, I launch the tpm.msc application, and the Trusted Platform Module (TPM) Management window opens. When everything goes well, it shows Manufacturer Name: AMZN under TPM Manufacturer Information.
Next, I install BitLocker.
I open the servermanager.exe application and select Manage at the top right of the screen. In the dropdown menu, I select Add Roles and Features.
I select Role-based or feature-based installation from the wizard.
I select Next multiple times until I reach the Features section. I select BitLocker Drive Encryption, and I select Install.
I wait a bit for the installation and then restart the server at the end of the installation.
After reboot, I reconnect to the server and open the control panel. I select BitLocker Drive Encryption under the System and Security section.
I select Turn on BitLocker, and then I select Next and wait for the verification of the system and the time it takes to encrypt my volume’s data.
Just for extra safety, I decide to reboot at the end of the encryption. It is not strictly necessary. But I encrypted the root volume of the machine (C:) so I am wondering if the machine can still boot.
After the reboot, I reconnect to the instance, and I verify the encryption status.
I also verify BitLocker’s status and key protection method enabled on the volume. To do so, I open PowerShell and type
manage-bde -protectors -get C:
I can see on the resulting screen that the C: volume encryption key is coming from the NitroTPM module and the instance used Secure Boot for integrity validation. I can also view the recovery key.
I left the recovery key in plain text in the previous screenshot because the instance and volume I used for this demo will not exist anymore by the time you will read this. Do not share your recovery keys publicly otherwise.
Important Considerations
Now that I have shown how to use NitroTPM to protect BitLocker’s volume encryption key, I’ll go through a couple of additional considerations:
RegisterImage API via the AWS CLI and not via the Amazon EC2 console.ModifyInstanceAttribute API is not supported on running or stopped instances.ImportImage API, will omit NitroTPM data.At the moment, we support all Intel and AMD instance types that supports UEFI boot mode. Graviton1, Graviton2, Xen-based, Mac, and bare-metal instances are not supported. Some additional instance types are not supported at launch (I shared the exact list previously). We will add support for these soon after launch.
There is no additional cost for using NitroTPM. It is available today in all AWS Regions, including the AWS GovCloud (US) Regions, except in China.
And now, go build 😉
Post Syndicated from BeardedTinker original https://www.youtube.com/watch?v=sfAn3OzMpQs
Post Syndicated from Rapid7 original https://blog.rapid7.com/2022/05/11/security-nation-jim-ogorman-and-g0tmi1k-on-kali-linux/
![[Security Nation] Jim O’Gorman and g0tmi1k on Kali Linux](https://blog.rapid7.com/content/images/2022/05/security_nation_logo--1-.jpg){kind=logo}
In this episode of Security Nation, Jen and Tod sit down with Jim O’Gorman and Ben “g0tmi1k” Wilson of Offensive Security to chat about Kali Linux. They walk our hosts through the vision behind Kali and how they understand the uses, advantages, and challenges of open-source security tools.
Stick around for our Rapid Rundown, where producer Jesse joins Tod to talk about an upcoming change in security protocols across the internet that might make passwords obsolete (eventually).
![[Security Nation] Jim O’Gorman and g0tmi1k on Kali Linux](https://blog.rapid7.com/content/images/2022/05/Jim-800x800.jpg)
Jim O’Gorman (Elwood) began his tech career as a network administrator with a particular talent for network intrusion simulation, digital investigations, and malware analysis. Jim started teaching for OffSec in 2009 as an instructor for the Penetration Testing with Kali (PWK) course — a role he still enjoys. He went on to co-author Metasploit: The Penetration Tester’s Guide and Kali Linux: Revealed, and has developed and curated a number of OffSec courses. As the Chief Content and Strategy officer, he currently oversees the open source Kali Linux development project and participates with OffSec’s Penetration Testing Team.
![[Security Nation] Jim O’Gorman and g0tmi1k on Kali Linux](https://blog.rapid7.com/content/images/2022/05/image1.jpg)
Ben “g0tmi1k” Wilson has been in the information security world for nearly two decades. Since joining Offensive Security nine years ago, he has applied his experience in a number of roles including live instructor, content developer, and security administrator. He is currently managing the day-to-day activity as well as developing Kali Linux, pushing it forward. He has worked on various vulnerabilities, which are published on Exploit-DB that he also works on. Furthermore he created and still runs VulnHub, allowing for hands-on experience.
Interview links
Rapid Rundown links
Like the show? Want to keep Jen and Tod in the podcasting business? Feel free to rate and review with your favorite podcast purveyor, like Apple Podcasts.
More episodes:
Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2022/05/ice-is-a-domestic-surveillance-agency.html
Georgetown has a new report on the highly secretive bulk surveillance activities of ICE in the US:
When you think about government surveillance in the United States, you likely think of the National Security Agency or the FBI. You might even think of a powerful police agency, such as the New York Police Department. But unless you or someone you love has been targeted for deportation, you probably don’t immediately think of Immigration and Customs Enforcement (ICE).
This report argues that you should. Our two-year investigation, including hundreds of Freedom of Information Act requests and a comprehensive review of ICE’s contracting and procurement records, reveals that ICE now operates as a domestic surveillance agency. Since its founding in 2003, ICE has not only been building its own capacity to use surveillance to carry out deportations but has also played a key role in the federal government’s larger push to amass as much information as possible about all of our lives. By reaching into the digital records of state and local governments and buying databases with billions of data points from private companies, ICE has created a surveillance infrastructure that enables it to pull detailed dossiers on nearly anyone, seemingly at any time. In its efforts to arrest and deport, ICE has — without any judicial, legislative or public oversight — reached into datasets containing personal information about the vast majority of people living in the U.S., whose records can end up in the hands of immigration enforcement simply because they apply for driver’s licenses; drive on the roads; or sign up with their local utilities to get access to heat, water and electricity.
ICE has built its dragnet surveillance system by crossing legal and ethical lines, leveraging the trust that people place in state agencies and essential service providers, and exploiting the vulnerability of people who volunteer their information to reunite with their families. Despite the incredible scope and evident civil rights implications of ICE’s surveillance practices, the agency has managed to shroud those practices in near-total secrecy, evading enforcement of even the handful of laws and policies that could be invoked to impose limitations. Federal and state lawmakers, for the most part, have yet to confront this reality.
EDITED TO ADD (5/13): A news article.
Post Syndicated from original https://lwn.net/Articles/894531/
Memory protection keys are a CPU feature that allows additional access
restrictions to be imposed on regions of memory and changed in a fast and
efficient way. Support for protection keys in user space has been in the
kernel for some time, but kernel-side protection (often called “protection
keys supervisor” or PKS) remains unsupported — on x86, at least. At the
2022 Linux
Storage, Filesystem, Memory-management and BPF Summit (LSFMM), Ira
Weiny provided an update on the state of PKS and led a discussion on what
the proper in-kernel API for PKS should be.
Post Syndicated from original https://lwn.net/Articles/894808/
The Rust Blog warns
developers of a malicious crate named rustdecimal, which was
evidently targeted at GitLab users who mistype rust_decimal.
The crate contained identical source code and functionality as the
legit rust_decimal crate, except for the Decimal::new function.When the function was called, it checked whether the GITLAB_CI
environment variable was set, and if so it downloaded a binary
payload into /tmp/git-updater.bin and executed it. The binary
payload supported both Linux and macOS, but not Windows.
Post Syndicated from original https://lwn.net/Articles/894802/
Security updates have been issued by Debian (mutt), Fedora (blender, freerdp, kernel, kernel-headers, kernel-tools, mingw-freetype, and vim), Oracle (kernel and kernel-container), Red Hat (aspell, bind, bluez, c-ares, cairo and pixman, cockpit, compat-exiv2-026, container-tools:3.0, container-tools:rhel8, cpio, dovecot, exiv2, fapolicyd, fetchmail, flatpak, gfbgraph, gnome-shell, go-toolset:rhel8, grafana, grub2, httpd:2.4, keepalived, kernel, kernel-rt, libpq, libreoffice, libsndfile, libssh, libtiff, lynx, maven:3.5, maven:3.6, mod_auth_mellon, mod_auth_openidc:2.3, openssh, php:7.4, pki-core:10.6, postgresql:10, python-lxml, python27:2.7, python3, python38:3.8 python38-devel:3.8, python39:3.9 python39-devel:3.9, qt5-qtbase, qt5-qtsvg, rust-toolset:rhel8, samba, squid:4, udisks2, virt:rhel virt-devel:rhel, webkit2gtk3, xorg-x11-server xorg-x11-server-Xwayland, and zsh), SUSE (gzip and php-composer), and Ubuntu (busybox, cairo, cron, dnsmasq, libsndfile, and nss).
Post Syndicated from Rita Kozlov original https://blog.cloudflare.com/introducing-d1/
We announced Cloudflare Workers in 2017, giving developers access to compute on our network. We were excited about the possibilities this unlocked, but we quickly realized — most real world applications are stateful. Since then, we’ve delivered KV, Durable Objects, and R2, giving developers access to various types of storage.
Today, we’re excited to announce D1, our first SQL database.
While the wait on beta access shouldn’t be long — we’ll start letting folks in as early as June (sign up here), we’re excited to share some details of what’s to come.
D1 is built on SQLite. Not only is SQLite the most ubiquitous database in the world, used by billions of devices a day, it’s also the first ever serverless database. Surprised? SQLite was so ahead of its time, it dubbed itself “serverless” before the term gained connotation with cloud services, and originally meant literally “not involving a server”.
Since Workers itself runs between the server and the client, and was inspired by technology built for the client, SQLite seemed like the perfect fit for our first entry into databases.
So what can you build with D1? The true answer is “almost anything!”, that might not be very helpful in triggering the imagination, so how about a live demo?
You can check out an example of D1 in action by trying out our demo running here: northwind.d1sql.com.
If you’re wondering “Who are Northwind Traders?”, Northwind Traders is the “Hello, World!” of databases, if you will. A sample database that Microsoft would provide alongside Microsoft Access to use as their own tutorial. It first appeared 25 years ago in 1997, and you’ll find many examples of its use on the Internet.
It’s a typical business application, with a realistic schema, with many foreign keys, across many different tables — a truly timeless representation of data.
When was the recent order of Queso Cabrales shipped, and what ship was it on? You can quickly find out. Someone calling in about ordering some Chai? Good thing Exotic Liquids still has 39 units in stock, for just \$18 each.
We welcome you to play and poke around, and answer any questions you have about Northwind Trading’s business.
The Northwind Traders demo also features a dashboard where you can find details and metrics about the D1 SQL queries happening behind the scenes.
Going back to our original question before the demo, however, what can you build with D1?
While you may not be running Northwind Traders yourself, you’re likely running a very similar piece of software somewhere. Even at the very core of Cloudflare’s service is a database. A SQL database filled with tables, materialized views and a plethora of stored procedures. Every time a customer interacts with our dashboard they end up changing state in that database.
The reality is that databases are everywhere. They are inside the web browser you’re reading this on, inside every app on your phone, and the storage for your bank transaction, travel reservations, business applications, and on and on. Our goal with D1 is to help you build anything from APIs to rich and powerful applications, including eCommerce sites, accounting software, SaaS solutions, and CRMs.
You can even combine D1 with Cloudflare Access and create internal dashboards and admin tools that are securely locked to only the people in your organization. The world, truly, is your oyster.
We’ll talk about the capabilities, and upcoming features further down in the post, but at the core of it, the strength of D1 is the developer experience: allowing you to go from nothing to a full stack application in an instant. Think back to a tool you’ve used that made development feel magical — that’s exactly what we want developing with Workers and D1 to feel like.
To give you a sense of it, here’s what getting started with D1 will look like.
With D1, you will be able to create a database, in just a few clicks — define the tables, insert or upload some data, no need to memorize any commands unless you need to.
Of course, if the command-line is your jam, earlier this week, we announced the new and improved Wrangler 2, the best tool for wrangling and deploying your Workers, and soon also your tool for deploying D1. Wrangler will also come with native D1 support, so you can create & manage databases with a few simple commands:
Attaching D1 to your Worker is as easy as creating a new binding. Each D1 database that you attach to your Worker gets attached with its own binding on the env parameter:
export default {
async fetch(request, env, ctx) {
const { pathname } = new URL(request.url)
if (pathname === '/num-products') {
const { result } = await env.DB.get(`SELECT count(*) AS num_products FROM Product;`)
return new Response(`There are ${result.num_products} products in the D1 database!`)
}
}
}
Or, for a slightly more complex example, you can safely pass parameters from the URL to the database using a Router and parameterised queries:
import { Router } from 'itty-router';
const router = Router();
router.get('/product/:id', async ({ params }, env) => {
const { result } = await env.DB.get(
`SELECT * FROM Product WHERE ID = $id;`,
{ $id: params.id }
)
return new Response(JSON.stringify(result), {
headers: {
'content-type': 'application/json'
}
})
})
export default {
fetch: router.handle,
}
First and foremost, we want you to be able to develop with D1, without having to worry about cost.
At Cloudflare, we don’t believe in keeping your data hostage, so D1, like R2, will be free of egress charges. Our plan is to price D1 like we price our storage products by charging for the base storage plus database operations performed.
But, again, we don’t want our customers worrying about the cost or what happens if their business takes off, and they need more storage or have more activity. We want you to be able to build applications as simple or complex as you can dream up. We will ensure that D1 costs less and performs better than comparable centralized solutions. The promise of serverless and a global network like Cloudflare’s is performance and lower cost driven by our architecture.
Here’s a small preview of the features in D1.
With D1, we want to make it easy to store your whole application’s state in the one place, so you can perform arbitrary queries across the full data set. That’s what makes relational databases so powerful.
However, we don’t think powerful should be synonymous with cumbersome. Most relational databases are huge, monolithic things and configuring replication isn’t trivial, so in general, most systems are designed so that all reads and writes flow back to a single instance. D1 takes a different approach.
With D1, we want to take configuration off your hands, and take advantage of Cloudflare’s global network. D1 will create read-only clones of your data, close to where your users are, and constantly keep them up-to-date with changes.
Many operations in an application don’t just generate a single query. If your logic is running in a Worker near your user, but each of these queries needs to execute on the database, then sending them across the wire one-by-one is extremely inefficient.
D1’s API includes batching: anywhere you can send a single SQL statement you can also provide an array of them, meaning you only need a single HTTP round-trip to perform multiple operations. This is perfect for transactions that need to execute and commit atomically:
async function recordPurchase(userId, productId, amount) {
const result = await env.DB.exec([
[
`UPDATE users SET balance = balance - $amount WHERE user_id = $user_id`,
{ $amount: amount, $user_id: userId },
],
[
'UPDATE product SET total_sales = total_sales + $amount WHERE product_id = $product_id',
{ $amount: amount, $product_id: productId },
],
])
return result
}
But we’re going further. With D1, it will be possible to define a chunk of your Worker code that runs directly next to the database, giving you total control and maximum performance—each request first hits your Worker near your users, but depending on the operation, can hand off to another Worker deployed alongside a replica or your primary D1 instance to complete its work.
There are few things as critical as the data stored in your main application’s database, so D1 will automatically save snapshots of your database to Cloudflare’s cloud storage service, R2, at regular intervals, with a one-click restoration process. And, since we’re building on the redundant storage of Durable Objects, your database can physically move locations as needed, resulting in self-healing from even the most catastrophic problems in seconds.
While D1 already supports the SQLite API, making it easy for you to write your queries, you might also need data to run them on. If you’re not creating a brand-new application, you may want to import an existing dataset from another source or database, which is why we’ll be working on allowing you to bring your own data to D1.
Likewise, one of SQLite’s advantages is its portability. If your application has a dedicated staging environment, say, you’ll be able to clone a snapshot of that data down to your local machine to develop against. And we’ll be adding more flexibility, such as the ability to create a new database with a set of test data for each new pull request on your Pages project.
This wouldn’t be a Cloudflare announcement if we didn’t conclude on “we’re just getting started!” — and it’s true! We are really excited about all the powerful possibilities our database on our global network opens up.
Are you already thinking about what you’re going to build with D1 and Workers? Same. Give us your details, and we’ll give you access as soon as we can — look out for a beta invite from us starting as early as June 2022!
Post Syndicated from Tanushree Sharma original https://blog.cloudflare.com/logs-r2/
Hot on the heels of the R2 open beta announcement, we’re excited that Cloudflare enterprise customers can now use Logpush to store logs on R2!
Raw logs from our products are used by our customers for debugging performance issues, to investigate security incidents, to keep up security standards for compliance and much more. You shouldn’t have to make tradeoffs between keeping logs that you need and managing tight budgets. With R2’s low costs, we’re making this decision easier for our customers!
Cloudflare helps customers at different levels of scale — from a few requests per day, up to a million requests per second. Because of this, the cost of log storage also varies widely. For customers with higher-traffic websites, log storage costs can grow large, quickly.
As an example, imagine a website that gets 100,000 requests per second. This site would generate about 9.2 TB of HTTP request logs per day, or 850 GB/day after gzip compression. Over a month, you’ll be storing about 26 TB (compressed) of HTTP logs.
For a typical use case, imagine that you write and read the data exactly once – for example, you might write the data to object storage before ingesting it into an alerting system. Compare the costs of R2 and S3 (note that this excludes costs per operation to read/write data).
| Provider | Storage price | Data transfer price | Total cost assuming data is read once |
|---|---|---|---|
| R2 | $0.015/GB | $0 | $390/month |
| S3 (Standard, US East | $0.023/GB | $0.09/GB for first 10 TB; then $0.085/GB | $2,858/month |
In this example, R2 leads to 86% savings! It’s worth noting that querying logs is where another hefty price tag comes in because Amazon Athena charges based on the amount of data scanned. If your team is looking back through historical data, each query can be hundreds of dollars.
Many of our customers have tens to hundreds of domains behind Cloudflare and the majority of our Enterprise customers also use multiple Cloudflare products. Imagine how costs will scale if you need to store HTTP, WAF and Spectrum logs for all of your Internet properties behind Cloudflare.
For SaaS customers that are building the next big thing on Cloudflare, logs are important to get visibility into customer usage and performance. Your customer’s developers may also want access to raw logs to understand errors during development and to troubleshoot production issues. Costs for storing logs multiply and add up quickly!
When designing products, one of Cloudflare’s core principles is ease of use. We take on the complexity, so you don’t have to. Storing logs is only half the battle, you also need to be able to access relevant logs when you need them – in the heat of an incident or when doing an in depth analysis.
Our product, Logpull, offers seven days of log retention and an easy to use API to access. Our customers love that Logpull doesn’t need any setup on third parties since it’s completely managed by Cloudflare. However, Logpull is limited in the retention of logs, the type of logs that we store (only HTTP request logs) and the amount of data that can be queried at one time.
We’re building tools for log retrieval that make it super easy to get your data out of R2 from any of our datasets. Similar to Logpull, we’ll start by supporting lookups by time period and rayId. From there, we’ll tackle more complex functions like returning logs within time X and Y that have 500 errors or where WAF action = block.
We’re looking for customers to join a closed beta for our Log Retrieval API. If you’re interested in testing it out, giving feedback and ultimately helping us shape the product sign up here.
Enterprise customers first need to get R2 added to their contract. Reach out to your account team if this is something you’re interested in! Once enabled, create an R2 bucket for your logs and follow the Logpush setup flow to create your job.
It’s that simple! If you have questions, our Logpush to R2 developer docs go into more detail.
We’re continuing to build out more advanced Logpush features with a focus on customization. Here’s a preview of what’s next on the roadmap:
We also have exciting plans to build out log analysis and forensics capabilities on top of R2. We want to make log storage tightly coupled to the Cloudflare dash so you can see high level analytics and drill down into individual log lines all in one view. Stay tuned to the blog for more!
Post Syndicated from Alex Krivit original https://blog.cloudflare.com/introducing-cache-reserve/
One hundred percent. 100%. One-zero-zero. That’s the cache ratio we’re all chasing. Having a high cache ratio means that more of a website’s content is served from a Cloudflare data center close to where a visitor is requesting the website. Serving content from Cloudflare’s cache means it loads faster for visitors, saves website operators money on egress fees from origins, and provides multiple layers of resiliency and protection to make sure that content is always available to be served.
Today, I’m delighted to announce a massive extension of the benefits of caching with Cache Reserve: a new way to persistently serve all static content from Cloudflare’s global cache. By using Cache Reserve, customers can see higher cache hit ratios and lower egress bills.
Every second, Cloudflare serves tens-of-millions of requests from our cache which equates to multiple terabytes-per-second of cached data being delivered to website visitors around the world. With this massive scale, we must ensure that the most requested content is cached in the areas where it is most popular. Otherwise, visitors might wait too long for content to be delivered from farther away and our network would be running inefficiently. If cache storage in a certain region is full, our network avoids imposing these inefficiencies on our customers by evicting less-popular content from the data center and replacing it with more-requested content.
This works well for the majority of use cases, but all customers have long tail content that is rarely requested and may be evicted from cache. This can be a cause of concern for customers, as this unpopular content can be a major cost driver if it is evicted repeatedly and needs to be served from an origin. This concern can be especially significant for customers with massive content libraries. So how can we make sure to keep this less popular content in cache to shield the customer from origin egress?
Cache Reserve removes customer content from this popularity contest and ensures that even if the specific content hasn’t been requested in months, it can still be served from Cloudflare’s cache – avoiding the need to pull it from the origin and saving the customer money on egress. Cache Reserve helps get customers closer to that 100% cache ratio and helps serve all of their content from our global CDN, forever.
Most content served from our cache starts its journey from an origin server – where content is hosted. In order to be admitted to Cloudflare’s cache the content sent from the origin must meet certain eligibility criteria that ensures it can be reused to respond to other requests for a website (content that doesn’t change based on who is visiting the site).
After content is admitted to cache, the next question to consider is how long it should remain in cache. Since cache ratios are calculated by taking the number of requests for content and identifying the portion that are answered from a cache server instead of an origin server, ensuring content remains cached in an area it is highly requested is paramount to achieving a high cache ratio.
Some CDNs use a pay-to-play model that allows customers to pay more money to ensure content is cached in certain areas for some length of time. At Cloudflare, we don’t charge customers based on where or for how long something is cached. This means that we have to use signals other than a customer’s willingness to pay to make sure that the right content is cached for the right amount of time and in the right areas.
Where to cache a piece of content is pretty straightforward (where it’s being requested), how long content should remain in cache can be highly variable.
Beyond headers like cache-control or cdn-cache-control, which help determine how long a customer wants something to be served from cache, the other element that CDNs must consider is whether they need to evict content early to optimize storage of more popular assets. We do eviction based on an algorithm called “least recently used” or LRU. This means that the least-requested content can be evicted from cache first to make space for more popular content when storage space is full.
This caching strategy requires keeping track of a lot of information about when requests come in and constantly updating the cache to make sure that the hottest content is kept in cache and the least popular content is evicted. This works well and is fair for the wide-array of customers our CDN supports.
However, if a customer has a large library of content that might go through cycles of popularity and which they’d like to serve from cache regardless, then LRU might mean additional origin egress as assets that are requested sparingly over a long time frame are pulled more from the origin.
That’s where Cache Reserve comes in. Cache Reserve is not an alternative to our popularity-based cache but a complement to it. By backstopping all cacheable content in Cache Reserve customers don’t have to worry about cache eviction or ephemerality any longer.
Cache Reserve is a large, persistent data store that is implemented on top of R2. By pushing a single button in the dashboard, all of your website’s cacheable content will be written to Cache Reserve. In the same way that Tiered Cache builds a hierarchy of caches between your visitors and your origin, Cache Reserve serves as the ultimate upper-tier cache that will reserve storage space for your assets for as long as you want. This ensures that your content is always served from cache, shielding your origin from unneeded egress fees, and improving response performance.
Cache Reserve sits between our edge data centers and your origin and provides guaranteed SLAs for how long your content can remain in cache.
As content is pulled from the origin, it will be written to Cache Reserve, followed by upper-tier data centers, and lower-tier data centers until it reaches the client to fulfill the request. Subsequent requests for the same content will not need to go all the way back to the origin for the response and can, instead, be served from a cache closer to the visitor. Improving both performance and costs of serving the assets. As content gets evicted from lower-tiers and upper-tiers, it will be backstopped by Cache Reserve.
Cache Reserve voids the request-based eviction that’s implemented in LRU and ensures that assets will remain in cache as long as they are needed. Cache Reserve extends the benefits of Tiered Cache by reducing the number of times Cloudflare’s network needs to ask an origin for content we should have in cache, while simultaneously limiting the number of connections and requests that our data centers need to open to your origin to ask for missing content. Using Cache Reserve with tiered cache helps collapse the number of requests that result from multiple concurrent cache misses from lower-tiers for the same content.
As an example, let’s assume a cold request for example.com, something our network has never seen before. If a client request comes into the closest lower-tier data center and it is a miss, that lower-tier is mapped to an upper-tier data center. When the lower-tier asks the upper-tier for the content and it is also a miss, the upper-tier will ask Cache Reserve for the content. Now, being the ultimate upper-tier, it will be the only data center that can ask the origin for content if it is not stored on our network. This will help limit the origin resources you need to devote to serving this content as once it’s written to Cache Reserve, your origin doesn’t need to fan out the content to any other part of Cloudflare’s network.
When your content does need updating, Cache Reserve will respect cache-control headers and purge requests. This means that if you want to control how long something remains fresh in Cache Reserve, before Cloudflare goes back to your origin to revalidate the content, set it as a cache-control header and it will be respected without risk of early eviction. Or if you want to update content on the fly, you can send a purge request which will be respected in both Cloudflare’s cache and in Cache Reserve.
Currently, Cache Reserve is in closed beta, meaning that it’s available to anyone who wants to sign up but we will be slowly rolling it out to customers over the coming weeks to make sure that we are quickly triaging edge cases and making fundamental improvements before we make it generally available to everyone.
To sign up for the Cache Reserve beta:
The Cache Reserve Plan will mimic the low cost of R2. Storage will be \$0.015 per GB per month and operations will be \$0.36 per million reads, and \$4.50 per million writes. For more information about pricing, please refer to the R2 page to get a general idea (Cache Reserve pricing page will be out soon).
Cache Reserve holds tremendous promise to increase cache hit ratios — which will improve the economics of running any website while speeding up visitors’ experiences. We’re excited to begin letting people use Cache Reserve soon. Be sure to check out the beta and let us know what you think.
Post Syndicated from Matt Alonso original https://blog.cloudflare.com/durable-objects-alarms/
Since we launched Durable Objects, developers have leveraged them as a novel building block for distributed applications.
Durable Objects provide globally unique instances of a JavaScript class a developer writes, accessed via a unique ID. The Durable Object associated with each ID implements some fundamental component of an application — a banking application might have a Durable Object representing each bank account, for example. The bank account object would then expose methods for incrementing a balance, transferring money or any other actions that the application needs to do on the bank account.
Durable Objects work well as a stateful backend for applications — while Workers can instantiate a new instance of your code in any of Cloudflare’s data centers in response to a request, Durable Objects guarantee that all requests for a given Durable Object will reach the same instance on Cloudflare’s network.
Each Durable Object is single-threaded and has access to a stateful storage API, making it easy to build consistent and highly-available distributed applications on top of them.
This system makes distributed systems’ development easier — we’ve seen some impressive applications launched atop Durable Objects, from collaborative whiteboarding tools to conflict-free replicated data type (CRDT) systems for coordinating distributed state launch.
However, up until now, there’s been a piece missing — how do you invoke a Durable Object when a client Worker is not making requests to it?
As with any distributed system, Durable Objects can become unavailable and stop running. Perhaps the machine you were running on was unplugged, or the datacenter burned down and is never coming back, or an individual object exceeded its memory limit and was reset. Before today, a subsequent request would reinitialize the Durable Object on another machine, but there was no way to programmatically wake up an Object.
Durable Objects Alarms are here to change that, unlocking new use cases for Durable Objects like queues and deferred processing.
Durable Object Alarms allow you, from within your Durable Object, to schedule the object to be woken up at a time in the future. When the alarm’s scheduled time comes, the Durable Object’s alarm() handler will be called. If this handler throws an exception, the alarm will be automatically retried using exponential backoff until it succeeds — alarms have guaranteed at-least-once execution.
Alarms are more fine-grained than Cron Triggers. While a Workers service can have up to three Cron Triggers configured at once, it can have an unlimited amount of Durable Objects, each of which can have a single alarm active at a time.
Alarms are directly scheduled from and invoke a function within your Durable Object. Cron Triggers, on the other hand, are not programmatic — they execute based on their schedules, which have to be configured via the Cloudflare Dashboard or centralized configuration APIs.
First, you’ll need to add the durable_object_alarms compatibility flag to your wrangler.toml.
compatibility_flags = ["durable_object_alarms"]
Next, implement an alarm() handler in your Durable Object that will be called when the alarm executes. From anywhere else in your Durable Object, call state.storage.setAlarm() and pass in a time for the alarm to run at. You can use state.storage.getAlarm() to retrieve the currently set alarm time.
In this example, we implemented an alarm handler that wakes the Durable Object up once every 10 seconds to batch requests to a single Durable Object, deferring processing until there is enough work in the queue for it to be worthwhile to process them.
export default {
async fetch(request, env) {
let id = env.BATCHER.idFromName("foo");
return await env.BATCHER.get(id).fetch(request);
},
};
const SECONDS = 1000;
export class Batcher {
constructor(state, env) {
this.state = state;
this.storage = state.storage;
this.state.blockConcurrencyWhile(async () => {
let vals = await this.storage.list({ reverse: true, limit: 1 });
this.count = vals.size == 0 ? 0 : parseInt(vals.keys().next().value);
});
}
async fetch(request) {
this.count++;
// If there is no alarm currently set, set one for 10 seconds from now
// Any further POSTs in the next 10 seconds will be part of this kh.
let currentAlarm = await this.storage.getAlarm();
if (currentAlarm == null) {
this.storage.setAlarm(Date.now() + 10 * SECONDS);
}
// Add the request to the batch.
await this.storage.put(this.count, await request.text());
return new Response(JSON.stringify({ queued: this.count }), {
headers: {
"content-type": "application/json;charset=UTF-8",
},
});
}
async alarm() {
let vals = await this.storage.list();
await fetch("http://example.com/some-upstream-service", {
method: "POST",
body: Array.from(vals.values()),
});
await this.storage.deleteAll();
this.count = 0;
}
}
Once every 10 seconds, the alarm() handler will be called. In the event an unexpected error terminates the Durable Object, it will be re-instantiated on another machine, following a short delay, after which it can continue processing.
Under the hood, Alarms are implemented by making reads and writes to the storage layer. This means Alarm get and set operations follow the same rules as any other storage operation – writes are coalesced with other writes, and reads have a defined ordering. See our blog post on the caching layer we implemented for Durable Objects for more information.
Alarms are designed to have no single point of failure and to run entirely on our edge – every Cloudflare data center running Durable Objects is capable of running alarms, including migrating Durable Objects from unhealthy data centers to healthy ones as necessary to ensure that their Alarm executes. Single failures should resolve in under 30 seconds, while multiple failures may take slightly longer.
We achieve this by storing alarms in the same distributed datastore that backs the Durable Object storage API. This allows alarm reads and writes to behave identically to storage reads and writes and to be performed atomically with them, and ensures that alarms are replicated across multiple datacenters.
Within each data center capable of running Durable Objects, there are multiple processes responsible for tracking upcoming alarms and triggering them, providing fault tolerance and scalability within the data center. A single elected leader in each data center is responsible for detecting failure of other data centers and assigning responsibility of those alarms to healthy local processes in its own data center. In the event of leader failure, another leader will be elected and become responsible for executing Alarms in the data center. This allows us to guarantee at-least-once execution for all Alarms.
Alarms are a great way to build new distributed primitives, like queues, atop Durable Objects. They also provide a method for guaranteeing work within a Durable Object will complete, without relying on a client request to “kick” the Object.
You can get started with Alarms now by enabling Durable Objects in the Cloudflare dashboard. For more info, check the developer docs or jump in our Discord.
Post Syndicated from Greg McKeon original https://blog.cloudflare.com/r2-open-beta/
In September, we announced that we were building our own object storage solution: Cloudflare R2. R2 is our answer to egregious egress charges from incumbent cloud providers, letting developers store as much data as they want without worrying about the cost of accessing that data.
The response has been overwhelming.
Cloudflare exists to help build a better Internet. Today, the Internet gets what it deserves: R2 is now in open beta.
Self-serve customers can enable R2 in the Cloudflare dashboard. Enterprise accounts can reach out to their CSM for onboarding.
R2 has two APIs: an API accessible only from within Workers, which we call the In-Worker API, and an S3-compatible API, which exposes your bucket on a URL of the form bucket.account.r2storage.com. Before you can make requests to R2, you’ll need to be authenticated — R2 buckets are private by default.
With the in-Worker API, a bucket is “bound” to a specific Worker, which can then perform PUT, GET, DELETE and LIST operations against the bucket.
For the S3-compatible API, authentication is done the same way as on S3: SigV4 against an R2 URL. SigV4 signs requests using a secret key to authenticate them to R2. This means public access to R2 over the Internet is only possible today by hosting a Worker, connecting it to R2, and routing requests through it.
The easiest way to test the S3-compatible API is to use an S3 client. One of the most popular S3 clients is the boto3 SDK.
In Python, copy the following script and fill in the account_id, access_key, and secret_access_key fields with your R2 account credentials.
#!/usr/bin/env python
import boto3
import pprint
from botocore.client import Config
account_id = ''
access_key_id = ''
secret_access_key = ''
endpoint = f'https://{account_id}.r2.cloudflarestorage.com'
cl = boto3.client(
's3',
aws_access_key_id=access_key_id,
aws_secret_access_key=secret_access_key,
endpoint_url=endpoint,
config=Config(
region_name = endpoints[endpoint_name].get('region', 'auto'),
s3={'addressing_style': 'path'},
retries=dict( max_attempts=0 ),
),
)
printer = pprint.PrettyPrinter().pprint
printer(cl.head_bucket(Bucket='some bucket'))
printer(cl.create_bucket(Bucket='some other bucket'))
printer(cl.put_object(Bucket='some bucket', Key='my object', Body='some payload'))
R2 comes with support for all basic create/read/update/delete S3 features through both of its APIs.
During the open beta period, we’re targeting R2 to sustain 1,000 GET operations per second and 100 PUT operations per second, per bucket. R2 supports objects up to approximately 5 TB in size, with individual parts limited to 5 GB of data.
R2 provides strongly consistent access to data. Once a PUT is confirmed by R2, future GET operations will always reflect the new key/value pair. The only exception to this is when deleting a bucket. For a short period of time following deletion, the bucket may still exist and continue to allow reads/writes.
When we initially announced R2, we included preliminary pricing numbers. One of our main goals with R2 has been to serve the developers who can’t negotiate large discounts with cloud vendors. To that end, we’re also announcing a forever-free tier that lets developers start building on R2 with no charges at all.
R2 charges depend on the total volume of data stored and the type of operation performed on the data:
Class A operations tend to mutate state, such as creating a bucket, listing objects in a bucket, or writing an object. Class B operations tend to read existing state, for example reading an object from a bucket. You can find more information on pricing and a full list of operation types in the docs.
Of course, there is no charge for egress bandwidth from R2. You can access your bucket to your heart’s content.
R2’s forever-free tier includes:
Free usage resets each month. While in the open beta phase, R2 usage over the free tier will be billed.
We’ve spent the past six months in closed beta with a number of design partners, building out our storage solution. Backed by Durable Objects, R2’s novel architecture delivers both high availability and consistent performance.
While we’ve made great progress on R2, we still have plenty left to build in the coming months.
Our first priority is to improve performance and reliability. While we’ve thrown internal usage and our design partner’s demands at R2, there’s no substitute for live production traffic.
During the open beta period, R2 can sustain a maximum of 1,000 GET operations per second and 100 PUT operations per second, per bucket. We’ll look to raise these limits as we get comfortable operating the system. If you have higher needs, reach out to us!
When you create a bucket, you won’t see a region selector. Our vision for R2 includes automatically globally distributed storage, where R2 seamlessly places each object into the storage region closest to where the request comes from. Today, R2 primarily stores data in North America, which can lead to higher latencies when accessing content from other regions. We’ll first look to address this by adding additional regions where objects can be created, before adding automatic migration of existing objects across regions. Similar to what we’ve built with jurisdictional restrictions for Durable Objects, we’ll also enable restricting where an R2 bucket places data to comply with privacy regulations.
We’ll then focus on expanding R2 capabilities beyond the basic S3 API. In the near term, we’re focused on delivering:
If you have additional feature requests that aren’t listed above, we want to hear from you! Reach out and let us know what you need to make R2 your new, zero-cost egress object store.
Post Syndicated from The History Guy: History Deserves to Be Remembered original https://www.youtube.com/watch?v=095t1w6Iuno
Post Syndicated from Adam Bradley original https://ibms360.co.uk/?p=902
Dear Reader,
Wow, it’s been a while since our last post here, almost 2 years! Time has totally flown by. I checked the traffic this morning and was pleasantly surprised to see that we’re still getting 2,000+ hits per month which is just incredible given that we haven’t published any updates.
So, I’m guessing you probably want to know whats going on and why we’re not posting here. Let me summarily answer your most important questions below:
So, the reason we haven’t been posting here is mainly because, well, nothing has changed. Chris & I have both been insanely busy with regular life, work is non-stop and personal commitments on top mean that currently we have little time to focus on the project. I’m additionally moving to Southampton for a new job which will put me further away from the project and will likely just add to the delay.
The small updates we do have for you are mainly administrative. Back in June 2020 Peter Vaughan purchased and donated some shelving units to the project to allow us to better store our parts, media, etc.
And in September of 2020 we had some members of the CCS (British Computer Conservation Society) visit us in a socially distanced fashion (remember that?!) to ask us some questions about the project following our application to join their projects register.
We have now successfully joined the CCS, and look forward to working with them in the future.
So, what of the project now? Well, for now we’ve basically decided to park the project for a while until one or both of us has more time to spend on it. It sucks because we really want to see the project move forward and succeed, but right now neither of us are particularly in a position to make that happen; and whilst we do have fantastic support from the rest of the team, realistically we need to be somewhat involved in order to be able to progress things in the direction we’d like them to go in.
So, basically we’re on pause for the moment. When will we be off pause? I don’t know. It depends on a lot of factors. Trust me though, if anything changes you will all be the first to hear about it!
All the best,
Adam
P.S. If you’ve sent us an email and we haven’t replied, I can only apologise. A lot of them seem to have disappeared into a black hole of our old email server, and so if you’d like to get in touch please send us another email and we’ll do our best to get back to you.
Post Syndicated from Rebecca Franks original https://www.raspberrypi.org/blog/kids-coding-electronics-raspberry-pi-pico-free-learning-resource/
Since the release of the Raspberry Pi Pico microcontroller in 2021, we have seen people all over the world come up with creative Pico-based inventions.
Now, thanks to our brand-new and free ‘Introduction to Raspberry Pi Pico’ learning path, young coders can easily join in and make their own cool Pico projects! This free learning path has six guided projects to help kids to independently develop their coding skills, and their skills in physical computing and electronics.
In this post, I’ll tell you about Raspberry Pi Pico, what kids can make by following our free ‘Intro to Pico’ path, and what skills they will be learning.
Raspberry Pi Pico is a physical computing device that is low-cost and easy to use. It’s much smaller than any Raspberry Pi computer, and it needs much less power. That’s because it’s not a full computer but instead a microcontroller. That means Pico is a device that you program by writing code on any computer, and then sending that code to Pico via a USB cable.
In the ‘Intro to Raspberry Pi Pico’ path, we’ve designed new digital making projects specifically using Pico. By following the projects in the path, young people learn to make things with different electronic components. They’ll bring to life their own LED fireflies; they’ll make music with a sound machine and dial (a potentiometer); they’ll look after themselves and people around them by making a mood indicator and a heart rate visualiser. To find out more, visit the path, or scroll to the bottom of this post and click on ‘Details about the projects’.
The specially designed structure of our learning paths helps kids become confident and independent coders and digital makers. Through this project path, we want to show young people what is possible with Raspberry Pi Pico and inspire them to continue their digital making journey beyond the six projects. Seeing tech creations from our amazing community is super special to us, and we would love to hear about what your young coders have made with Pico. Kids can share their projects in the path gallery, or you can tag us on social media if you post photos!
While young people make all these Raspberry Pi Pico projects, they will learn the skills and independence to make and code their very own, unique creations with a Pico. We have designed our new project paths to help kids become independent digital makers. As they progress through a path, kids gain new skills, practise what they have learnt, and finally write and follow their own project brief.
Our learning paths help kids develop many of the skills that are important to all coders and digital makers, no matter how much experience they have:
The learning paths also encourage kids to make projects about the things that matter to them.
We have written the projects in this path with young people around the age of 9 to 13 in mind.
Programs for Raspberry Pi Pico are written in a text-based language called MicroPython. That means a young person who wants to start the ‘Intro to Pico’ path needs to be familiar with typing on a keyboard.
If your kid has never coded in a text-based language before, they could complete our free ‘Introduction to Python‘ project path first, but this is not a prerequisite.
To help with the programming aspects of the projects, the instructions in the path tell young people about:
One of the great things about this project path is that it helps young people explore physical computing and electronics. In the ‘Intro to Pico’ path, they’ll use:
We’ve designed the path to be completed in around six one-hour sessions, with one hour per project. However, the project instructions encourage kids to upgrade their projects and go further if they wish. This means that they might want to spend a little more time getting their projects exactly as they imagine.
Young people need a web browser so they can follow the project instructions. The first two projects in the path provide detailed instructions for how to install the free software needed for the projects.
The first step of each project lists what components are needed to create the project. You can purchase a kit from Kitronik or from Pimoroni that includes all of the components used in the path:
If your young coders enjoy MicroPython, they’ll also love our Python learning paths: ‘Introduction to Python‘ and ‘More Python‘. Both are structured in the same way as our Pico path, and will help young people learn Python while creating their own visual designs.
The ‘Intro to Raspberry Pi Pico’ path is structured according to our Digital Making Framework, with three Explore projects, two Design projects, and a final Invent project. You can also check out our learning graph to see the progression of skills and knowledge throughout the path.
The ‘LED firefly’ project introduces creators to Raspberry Pi Pico while they make their first project with a blinking LED. They program the LED with a blink pattern that is common to fireflies in the wild. To upgrade their projects, creators can place their LED firefly into a glass jar to create a twinkling effect.
‘Party popper’ introduces creators to the RGB LED and a buzzer. To form the popper, they craft a pull switch out of kitchen foil and cardboard. When the popper is activated, the RGB LED flashes in their chosen colour, and a ‘tada’ sound plays on the buzzer.
‘Beating heart’ uses a potentiometer (dial) to control the pulsing speed of an LED. Creators craft their own hearts using red paper and origami before placing the pulsing LED inside. In this way, they create a model of a heart they can use to learn about medicine or to bring to life a favourite toy.
In the ‘Mood indicator’ project, kids use switches and an RGB LED to create a device that can communicate a need or a mood to another person. This Design project gives young creators lots of opportunities to use their new skills to create something personal to them.
‘Sound machine’ is a project for kids to work with the different tones that a buzzer can make. They can use the buzzer to create sound effects, or to recreate their favourite songs. Once they have decided on their sounds, they can think about how a user of their project might choose to play them.
This project gives creators that chance to pick their favourite elements of the path to create something totally unique to them. They could make all sorts of sensory gadgets, from a Picosaber to a candle that can be blown out. Creators are encouraged to showcase their creations in the path gallery to give other young makers inspiration.
The post Get kids coding and learning electronics with Raspberry Pi Pico appeared first on Raspberry Pi.
By continuing to use the site, you agree to the use of cookies. more information
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.