The story of web framework Hono, from the creator of Hono

Post Syndicated from Yusuke Wada original https://blog.cloudflare.com/the-story-of-web-framework-hono-from-the-creator-of-hono

Hono is a fast, lightweight web framework that runs anywhere JavaScript does, built with Web Standards. Of course, it runs on Cloudflare Workers.

It was three years ago, in December 2021. At that time, I wanted to create applications for Cloudflare Workers, but the code became verbose without using a framework, and couldn’t find a framework that suited my needs. Itty-router was very nice but too simple. Worktop and Sunder did the same things I wanted to do, but their APIs weren’t quite to my liking. I was also interested in creating a router — a program that determines which action is executed based on the HTTP method and URL path of the Request — made of a Trie tree structure because it’s fast. So, I started building a web framework with a Trie tree-based router.


 “While trying to create my applications, I ended up creating my framework for them.” — a classic example of yak shaving. However, Hono is now used by many developers, including Cloudflare, which uses Hono in core products. So, this journey into the depths of yak shaving was ultimately meaningful.

Write once, run anywhere

Hono truly runs anywhere — not just on Cloudflare Workers. I’ll discuss why later in the post, but Hono also runs on Deno, Bun, and Node.js. This is because Hono does not depend on external libraries, but uses only the Web Standards API, and each runtime supports Web Standards.

It’s a delight for developers to know that the same code can run across different runtimes. For instance, the following src/index.ts code will run on Cloudflare Workers, Deno, and Bun.

import { Hono } from 'hono'

const app = new Hono()
app.get('/hello', (c) => c.text('Hello Hono!'))

export default app

To run it on Cloudflare Workers, you execute the Wrangler command:

wrangler dev src/index.ts

The same code works on Deno:

deno serve src/index.ts

And it works on Bun too:

bun run src/index.ts

This is only a simple “Hello World” example, but more complex applications with middleware and helpers that are discussed below can be run on Cloudflare Workers or the other runtimes. As proof of this, almost all our test code for Hono itself can run the same way on these runtimes. This is a genuine “write once, run anywhere” experience.

Who is using Hono?

Hono is now used by many developers and companies. For example, Unkey deploys their application built with Hono’s OpenAPI feature to Cloudflare Workers. The following is a list of companies using Hono, based on my survey “Who is using Hono in production?”.

There are many, many more companies not listed here. And major web services or libraries, such as Prisma, Resend, Vercel AI SDK, Supabase, and Upstash, use Hono in their examples. There are also several influencers who like Hono and use it as an alternative to Express.

Of course, at Cloudflare, we also use Hono. D1 uses Hono for the internal Web API running on Workers. Workers Logs is based on code from Baselime (acquired by Cloudflare) and uses Hono to migrate the applications from their original infrastructure to Cloudflare Workers. All Workers Logs internal or customer-facing APIs are run on Workers using Hono. We also use Hono as part of the internals of many other products, such as KV and Queues.

Why are you making a “multi-runtime” framework?

You might wonder “Why is an employee of Cloudflare creating a framework that runs everywhere?” Initially, Hono was designed to work exclusively with Cloudflare Workers. However, starting with version 2, I added support for Deno and Bun. This was a very wise decision. If Hono had been targeted only at Cloudflare Workers, it might not have attracted as many users. By running on more runtimes, it gains more users, leading to the discovery of bugs and receiving more feedback, which ultimately leads to higher quality software.

Hono and Cloudflare are a perfect combo

The combination of Hono and Cloudflare offers a delightful developer experience.

Many websites, including our Cloudflare Docs, introduce the following “vanilla” JavaScript as a “Hello World” for Cloudflare Workers:

export default {
  fetch: () => {
    return new Response('Hello World!')
  }
}

This is primitive and good for understanding the Workers principle. However, if you want to create an endpoint that “returns a JSON response for GET requests that come to /books“, you need to write something like this:

export default {
  fetch: (req) => {
    const url = new URL(req.url)
    if (req.method === 'GET' && url.pathname === '/books') {
      return Response.json({
        ok: true
      })
    }
    return Response.json(
      {
        ok: false
      },
      {
        status: 404
      }
    )
  }
}

If you use Hono, you can write it like the following:

import { Hono } from 'hono'

const app = new Hono()

app.get('/books', (c) => {
  return c.json({
    ok: true
  })
})

export default app

It is short. And you can understand that “it handles GET accesses to /books” intuitively.

If you want to handle GET requests to /authors/yusuke and get “yusuke” from the path —  “yusuke” is variable, you have to add something more complicated. The below is “vanilla” JavaScript example:

if (req.method === 'GET') {
  const match = url.pathname.match(/^\/authors\/([^\/]+)/)
  if (match) {
    const author = match[1]
    return Response.json({
      Author: author
    })
  }
}

If you use Hono, you don’t need if statements. Just add the endpoint definition to the app. Also, you don’t need to write a regular expression to get “yusuke”. You can get it with the function c.req.param():

app.get('/authors/:name', (c) => {
  const author = c.req.param('name')
  return c.json({
    Author: author
  })
})

One or two routes may be fine, but any more than that and maintenance becomes tricky. Code becomes more complex and bugs are harder to find. Using Hono, the code is very neat.

It is also easy to handle bindings to Cloudflare products, such as KV, R2, D1, etc. as Hono uses a “context model”. A context is a container that holds the application’s state until a request is received, and a response is returned. You can use a context to retrieve a request object, set response headers, and create custom variables. It also holds Cloudflare bindings. For example, if you set up a Cloudflare KV namespace with the name MY_KV, you can access it as follows, with TypeScript type completion.

import { Hono } from 'hono'

type Env = {
  Bindings: {
    MY_KV: KVNamespace
  }
}

const app = new Hono<Env>()

app.post('/message', async (c) => {
  const message = c.req.query('message') ?? 'Hi'
  await c.env.MY_KV.put('message', message)
  return c.text(`message is set`, 201)
})

Hono lets you write code in a simple and intuitive way, but that doesn’t mean there are limitations. You can do everything possible with Cloudflare Workers using Hono.

Add it when you want to use it

Hono is tiny. With the smallest preset, hono/tiny, you can write a “Hello World” application in just 12 KB. This is because it uses only the Web Standards API built into the runtime and has minimal functions. In comparison, the bundle size of Express is 579 KB.


However, there is much that you can do.

You can easily add functions using middleware. For example, it is a bit tedious to implement Basic Authentication from scratch, but with the built-in Basic Auth middleware, you can apply Basic Authentication to the path /auth/page with just this:

import { Hono } from 'hono'
import { basicAuth } from 'hono/basic-auth'

const app = new Hono()

app.use(
  '/auth/*',
  basicAuth({
    username: 'hono',
    password: 'acoolproject',
  })
)

app.get('/auth/page', (c) => {
  return c.text('You are authorized')
})

Hono’s package also includes built-in middleware that allows Bearer and JWT authentication, and easy configuration of CORS, etc. These built-in middleware components do not depend on external libraries, but there is also many 3rd-party middleware that allow the use of external libraries, such as authentication middleware using Clerk and Auth.js, and validators using Zod and Valibot.

There are also a number of built-in helpers, including the Streaming helper, which is useful for implementing AI. These can be added when you want to use them, and the file size increases only when they are added.

In Cloudflare Workers, there is a limit to a file size of a Worker. Keeping the core small and extending functions with middleware and helpers makes a lot of sense.

Onion structure

The important concepts of Hono are ”handler” and “middleware”.

A handler is a place to write a function that receives a request and returns a response, as specified by the user. For example, you can write a handler that gets a value of a query parameter, retrieves data from a database, and returns the result in JSON. Middleware can handle the requests that come to the handler and the responses that the handler returns. You can combine middleware with other middleware to build more large and complex applications. It is structured like an onion.


In a remarkably simple way, you can create middleware. For example, a custom logger that logs the request can be written as follows:

app.use(async (c, next) => {
  console.log(`[${c.req.method}] ${c.req.path}`)
  await next()
})

If you want to add a custom header to the response, write the following:

app.use(async (c, next) => {
  await next()
  c.header('X-Message', 'Hi, this is Hono!')
})

It would be interesting to combine this with HTMLRewriter. If an endpoint returns HTML, the middleware that modifies the HTML tags in it can be written as follows:

app.get('/pages/*', async (c, next) => {
  await next()

  class AttributeRewriter {
    constructor(attributeName) {
      this.attributeName = attributeName
    }
    element(element) {
      const attribute = element.getAttribute(this.attributeName)
      if (attribute) {
        element.setAttribute(this.attributeName, attribute.replace('oldhost', 'newhost'))
      }
    }
  }
  const rewriter = new HTMLRewriter().on('a', new AttributeRewriter('href'))

  const contentType = c.res.headers.get('Content-Type')

  if (contentType!.startsWith('text/html')) {
    c.res = rewriter.transform(c.res)
  }
})

There is very little to remember to create middleware. All you have to do is to work with the context, which you should already know.

The RPC is like magic

Hono has a strong type system. One feature that uses this is RPC (Remote Procedure Call). With RPC, you can express server-side API specifications as TypeScript types. When these types are loaded as generics in a client, the paths, arguments, and return types of each API endpoint are inferred. It’s like magic.

For example, imagine an endpoint for creating a blog post. This endpoint takes a number type id and a string type title. Using Zod, one of the validator libraries that support TypeScript inference, you can define the schema like this:

import { z } from 'zod'

const schema = z.object({
  id: z.number(),
  title: z.string()
})

You create a handler that receives this object in JSON format via a POST request to the path /posts. Using Zod Validator, you check if it matches the schema. The response will have a property called message of type string.

import { zValidator } from '@hono/zod-validator'

const app = new Hono().basePath('/v1')

// ...

const routes = app.post('/posts', zValidator('json', schema), (c) => {
  const data = c.req.valid('json')
  return c.json({
    message: `${data.id.toString()} is ${data.title}`
  })
})

This is a “typical” Hono handler. However, the TypeScript type you can get from the typeof for the routes will contain the information about its Web API specification. In this case, it includes the endpoint for creating blog posts — sending a POST request to the path /posts returns a JSON object.

export type AppType = typeof routes

Now, let’s create a client. You pass the earlier AppType as generics to a Hono client object.

import { hc } from 'hono/client'
import { AppType } from '.'

const client = hc<AppType>('http://localhost:8787')

With this setup, you’re ready. It’s magic time.

Code completion works perfectly. When you write client-side code, you no longer need to know the API specifications completely, which also helps eliminate mistakes.

Server-side JSX is fun

Hono provides built-in JSX, a syntax that allows you to write code in JavaScript that looks like HTML tags. When you hear the term JSX, you may think of React, a front-end UI library. However, Hono’s JSX was initially developed to run only on the server side. When we first started developing Hono, we were looking for template engines to render HTML. Most template engines, such as Handlebars and EJS, use eval internally and are incompatible with Cloudflare Workers, which does not support it. Then we came up with the idea of using JSX.

Hono’s JSX is unique in that it treats the tags as a string. So the following strange code actually works.

console.log((<h1>Hello!</h1>).toString())

There is no need to do renderToString() as in React. If you want to render HTML, just return this as is.

app.get('/', (c) => c.html(<h1>Hello</h1>))

Very interesting is the creation of Suspense — a feature in React that allows you to display a fallback UI while waiting for an asynchronous component to load — without any client implementation. The asynchronous components are running in a server-only implementation.

Server-side JSX is a better developer experience than you might imagine. You can use the toolchains for React’s JSX in the same way for Hono’s JSX, including the ability to complete tags in the editor. They bring mature front-end technology to the server side.

Testing is important

Testing is important. Fortunately, you can write tests easily when using Hono.

For example, let’s write a test for an endpoint. To test for a 200 response status of a request coming to / with the GET method, you can write the following:

it('should return 200 response', async () => {
  const res = await app.request('/')
  expect(res.status).toBe(200)
})

Simple, right? The beauty of this test is that you don’t have to bring up the server. The Web Standard API black boxes the server layer. The internal tests of Hono have 20,000 lines of code, but most of them are written in the same style as above, without the server up and running. 

Going to full-stack

We released a new major version 4 in February 2024. There are three main features that stand out:

  1. Static site generation

  2. Client components

  3. File-based routing

With these features, you can create full-stack applications with a user interface in Hono.

The introduction of client components allows JSX to work in the client. Now you can add interactions to your pages. Static site generation allows you to create blogs, etc. without having to bundle them into a single JavaScript file. We have also started an experimental project called HonoX. This is a meta-framework using Hono and Vite that provides file-based routing and a mechanism to hydrate client-side components to server-side generated HTML. It is easier to create larger applications that are a great match for Cloudflare Pages or Workers.

In addition to that, plans are underway to run it as a base server for existing full-stack frameworks such as Remix and Qwik.

In contrast to the Next.js framework, which started from the client-side with React, Hono is trying to become a full-stack framework starting from the server-side.

Hono Conference

On June 22, 2024, I held the “Hono Conference” in Tokyo, the first event to consist entirely of Hono-focused talks. One hundred people attended, and the event was a great success.

It was my dream to do this event. Now, there are 200 contributors to the honojs/hono repository on GitHub. If you include other Hono related repositories, there are many more. Creating “the most invincible framework we could think of” is a lot of fun for contributors and users.

Below is a group photo taken at the end of the event. This is my treasure. I want to make the 2nd event a global event.


Hono is 炎

I haven’t mentioned the origin of the name Hono yet. The name Hono is from the Japanese word for “炎“. It is similar to the word “flare“. Hono now runs on a variety of runtimes, but I said that it was first created to create Cloud”flare” Workers applications. It is an honor for Cloudflare that it has remained in its name.

That is all that the creator of Hono has to say about Hono.

Just try it

Everyone who has experienced application development with Hono and Cloudflare Workers says “the developer experience is a great experience“. If you haven’t experienced it yet, just try it.

See the Hono website for how to get started. If you are interested in reporting issues or contributing, please see the GitHub project. Plus, you can watch my interview about Hono on the YouTube Cloudflare Developers channel.

Understanding your Attack Surface: Different Approaches to Asset discovery

Post Syndicated from Jon Schipp original https://blog.rapid7.com/2024/10/17/understanding-your-attack-surface-different-approaches-to-asset-discovery/

Understanding your Attack Surface: Different Approaches to Asset discovery

Over the past two blogs (Help, I can’t see! A Primer for Attack Surface Management Blog Series and The Main Components of an Attack Surface Management (ASM) Strategy) in our series on Attack Surface Management, we’ve focused on the drivers and core elements of an Attack Surface Management solution. In this post, we’ll delve intoprocess of discovering assets. We cannot secure what we cannot see so getting this piece right is foundational to the success of your ASM program. This blog will explore four different methods of asset discovery starting with the most basic, deployed software agents.

Software Agents

Deployed agents are how most asset inventory and asset management systems work. A software agent is deployed on a workstation or server, phoning home to the management system about the asset. The benefits of this approach are that you get a very high fidelity dataset on this particular asset, including up-to-date information on installed software, location, etc. However, this approach is only as good as the reach of the software agent. From an asset discovery perspective, the problem that is encountered is that you cannot discover assets that do not have that software agent installed. If we consider the funnel diagram below, this is effectively having visibility from one row in the funnel. In reality, most organizations do not have 100% deployment coverage of agents and many assets that will not be able to deploy agents, so they have many different tools that provide asset visibility, all with different perspectives.

Understanding your Attack Surface: Different Approaches to Asset discovery

Also, If the software agent is an IT management agent, and not an endpoint security agent, it will typically lack security controls, vulnerability and exposure context which means key information to best understand the attack surface may be missing.

In sum, software agents should be treated as pieces of the attack surface puzzle.

Data Aggregation & Correlation

The more comprehensive way to discover assets is through the ingestion of asset data across a variety of tools the organization uses. This is the primary way assets are discovered with a CAASM solution. By ingesting data from your IT, business applications, and security tools via API connectors we get the broadest visibility and can see across the data gaps from any individual tools.

A CAASM solution asks each connected tool for the latest list of assets on a recurring basis. Security and identity data related to an asset is then stored and mapped to build a relationship in a database (ideally a graph) that is easily discoverable and queryable. Note that some solutions exist, but not many, that enable asset history tracking to perform data trending to view how an asset and organization changes over time. In this case, more than a single asset record is stored per asset, retained for a configurable length of time.

Due to the correlation engine provided by CAASM solutions, the more data you ingest from your tools, the better your attack surface visibility and accuracy. Remember the funnel illustrated at the beginning of this article? Since your tools might not agree on the fundamental aspects like the number of assets, it’s necessary to ingest data across them to get closer to a truer picture of your attack surface.

Organization’s start putting together the pieces by ingesting data typically from 5 primary sources

  • Directory Services (Active Directory, Azure AD, LDAP, etc.)
  • Endpoint Security (insightIDR, Crowdstrike, SentinelOne, etc.)
  • Vulnerability Management (insightVM, Qualys, Tenable etc.)
  • Identity & Access Management (Okta, Duo, etc.)
  • Cloud Service Provider or Cloud Security Posture Management (AWS, Azure, insightCloudSec, Wiz, etc.)

Full deployments typically have between 10-20 data sources depending on the size of the organization. These will also include integrations into CMDB, IT Asset Management systems, Digital Risk Protection Service (DRPS), and more.

For the external attack surface (EASM), assets are discovered using one of two methods, or a combination of both. The first method is again, data ingestion from sources: like Shodan, Bitsight, etc. The second method is through active internet scanning that occurs on a recurring basis to discover the latest public facing assets and services running on them. We have covered data ingestion in detail already, we will then take a look at active network scanning shortly but let’s start with passive network scanning first.

Passive Network Scanning

Not every asset in the organization will be linked to a pre-existing data source. For complete attack surface coverage, you also need to consider methods that go further that address visibility gaps from your data sources. The first of these is passive network scanning.

In one scenario, attackers could gain access to your internal network through a malicious insider. A disgruntled employee could plug an unapproved workstation or malicious device into an ethernet port, or attackers might gain a foothold through WiFi attacks to gain entry on the network with a static IP address. In both of these cases, the malicious device would effectively be invisible to your teams and tools, with the exception of the perspective of network switches, firewalls, and network traffic analysis.

Support for data sources of passive network traffic therefore can give teams visibility into new assets that come online that are not correlated with any other data source. This can provide visibility into rogue devices that are circumventing security policy and protocol. Most CAASM solutions today do not ingest network data such as Netflow, or support NTA data ingestion, although some can use data from agents that process ARP or DHCP broadcasts to discover new assets. However, these agents need to be deployed on a specific network segment otherwise they won’t be able to discover unknown assets. In these cases, Active Network Scanning is a potential alternative to increase visibility of assets that are circumventing normal controls and monitoring.

Active Network Scanning

The most difficult-to-discover Shadow IT assets can also be the most vulnerable, because they won’t have the necessary security controls enabled. These assets are not discoverable through network data alone, as they provide no telemetry.  Even with network data, security teams often miss fingerprinting and fail to identify the services running on these devices. Active scanning offers a way to capture information from these assets that are otherwise missed. Active network scanning is a necessary feature in environments where full visibility is extremely important.

A fully deployed vulnerability scanner is superior to native active network scanning because it uses the same network discovery techniques but also understands vulnerabilities and exposures. Using a CAASM solution to understand which assets and networks are not being continuously assessed for vulnerabilities is a great way to also increase your ability to discover new assets by active network scanning.

For the final blog in this series, we will look at how we can drive greater insights through the context we can acquire with effective Attack Surface Management (ASM).

Introducing picamzero: Simplifying Raspberry Pi Camera projects for beginners

Post Syndicated from Laura Sach original https://www.raspberrypi.org/blog/picamzero-raspberry-pi-camera-projects-for-beginners/

Thousands of learners worldwide take their first steps into text-based programming using the Python programming language. Python is not only beginner-friendly, but is also used extensively in industry.

An educator helps two young learners with a coding project in a classroom.

In 2015, Python developer Daniel Pope, who has a keen interest in education, noticed that beginners often have great ideas for creating projects but struggle because the software libraries they need to use are aimed at more confident programmers. To address this, he created Pygame Zero — a simplified version of the popular PyGame software. Since then, various developers have expanded the range of ‘zero’ libraries for Python.

How Python zero libraries help beginner programmers

The Raspberry Pi Foundation has a long history of supporting Python zero libraries. GPIO Zero was launched back in 2015, followed by guizero and then picozero. The goal of all ‘zero’ libraries is the same: to help beginner programmers create amazing projects using simple, understandable code, supported by useful documentation. 

The Picamera2 library is a powerful tool for advanced users, but beginners — such as Astro Pi: Mission Space Lab programme participants — would benefit from a zero library to allow them to use the Raspberry Pi Camera module. 

The Astro Pi Mark II units.
The Astro Pi Mark II units
Image taken by Astro Pi: Mission Space Lab programme participants

Picamzero: how to get started

The Code Club Projects and Youth Programmes teams at the Raspberry Pi Foundation have joined forces to create picamzero: a new library that makes it simple for beginners to use the Raspberry Pi Camera board.

As with the other ‘zero’ libraries, it’s straightforward to get started. You can install picamzero by typing two commands in your Raspberry Pi’s terminal:

sudo apt update

sudo apt install python3-picamzero

Once it’s installed, setting up your program to communicate with your camera is easy:

from picamzero import Camera

cam = Camera()

You can ask picamzero to take a time-lapse sequence and make a video of your images using a single line of code.

cam.capture_sequence("mysequence.jpg", make_video=True)

Picamzero also makes it easy to add text and image overlays to your images.

A Lego scene captured using picamzero.
A Lego scene captured using picamzero

We’ve written beginner-friendly documentation for the new library so that you can explore what you can create using just a few lines of code. We’ve also updated our resources so that you can start making exciting projects using picamzero straight away:

We hope you enjoy using picamzero. Please get in touch if you have any feedback or suggestions. Happy coding!

The post Introducing picamzero: Simplifying Raspberry Pi Camera projects for beginners appeared first on Raspberry Pi Foundation.

Automating multi-AZ high availability for WebLogic administration server with DNS: Part 2

Post Syndicated from Robin Geddes original https://aws.amazon.com/blogs/architecture/automating-multi-az-high-availability-for-weblogic-administration-server-with-dns-part-2/

In Part 1 of this series, we used a floating virtual IP (VIP) to achieve hands-off high availability (HA) of WebLogic Admin Server. In Part 2, we’ll achieve an arguably superior solution using Domain Name System (DNS) resolution.

Using a DNS to resolve the address for WebLogic admin server

Let’s look at the reference WebLogic deployment architecture on AWS shown in Figure 1.

Reference WebLogic deployment with multi-AZ admin HA capability

Figure 1. Reference WebLogic deployment with multi-AZ admin HA capability

This solution comes in two parts:

  • Configure the environment to use DNS to locate the admin server.
  • Create a mechanism to automatically update the DNS entry when the admin server is launched.

Environment configuration

A WebLogic domain resides in private subnets of a Virtual Private Cloud (VPC). The admin server resides in one of the private subnets on its own Amazon Elastic Compute Cloud (Amazon EC2) instance. In this scenario, the admin server is bound to the private IP address of the EC2 host associated with a hostname/DNS record (configured in Amazon Route53).

We deploy WebLogic in multi-Availability Zone (multi-AZ) active-active stretch architecture. For this simple example, there is only one WebLogic domain and one admin server. To meet this requirement, we:

  1. create an EC2 launch template for the admin server, and then
  2. associate the launch template to an Amazon EC2 Auto Scaling group named wlsadmin-asg with min, max, and desired capacity of 1. Note we will need the group name later.

The Auto Scaling group detects EC2 and Availability Zone degradation and launches a new instance – in a different AZ if the current one becomes unavailable.

To enable access, we create two route tables: one for the private subnets, and the other for public subnets.

Next, we use the Amazon Route 53 DNS service to abstract the IPv4 address of the WebLogic admin server:

  • Create a private hosted zone in Amazon Route 53; in this example, we use example.com.
  • Create an A record for the admin server; in this example, example.com, pointing to the IP address of the EC2 instance hosting the admin server. Set the TTL to 60 seconds so the managed servers’ DNS records will be propagated before the admin server has finished starting.
  • Note the ID of the hosted zone, it will be required later in two places: to create an IAM role with permissions to update the DNS A record, and as an environment variable for an AWS Lambda function to perform the update.

We then update the WebLogic domain configuration and set the WebLogic Admin server listen address to the DNS name we chose. In this example, we set the line of WebLogic Admin server configuration to <listen-address>wlsadmin.example.com</listen-address> in WebLogic domain configuration file $DOMAIN_HOME/config/config.xml.

Automatically updating the DNS A record upon admin server launch

On-premises, it would often be a cultural anathema to update a DNS record as part of a server’s lifecycle. Operations that cut across team boundaries and responsibilities can be difficult to orchestrate. In the cloud, we have tools and a security model to enable such operations.

There are several approaches for this, and it is important to understand the patterns we prototyped and why they were rejected before we describe our recommended implementation pattern:

  • Rejected Option 1 – Simple: The user data script makes an API call to update the A record (with suitable IAM instance policy). However, a compromised server could update that A record for nefarious means; hence, we reject this option.
  • Rejected Option 2 – Better: The user data script calls a Lambda function to update the A record and include suitable checks to prevent misuse of the A record, such as setting it to a public address. This still requires granting permission for instance to call the lambda function and determining the correct logic to validate the IP address.
  • Accepted Option 3 – Best: We do not grant the EC2 instance any additional permission to update the DNS A Record. We rely on the event lifecycle of the Auto Scaling group as shown in Figure 2.
Triggering the DNS A record update from EventBridge using Lambda

Figure 2. Triggering the DNS A record update from EventBridge using Lambda

  1. When the Auto Scaling group successfully launches a new admin server through a scale-out action, an “EC2 Instance Launch Successful” event is created in Amazon EventBridge.
  2. An EventBridge rule calls an AWS Lambda function, passing the event data as a JSON object.
  3. The Lambda function:
    1. parses the event data to determine the EC2 Instance ID,
    2. obtains the IP address of new server using the Instance ID, then
    3. updates the DNS A Record for the admin server in Hosted Zone we created above with the IP address.
  4. The Lambda function needs permissions to:
    • describe EC2 instances within the account (to get the IP address).
    • update the A-record in (only) the Hosted Zone we created earlier.

Working backwards, first we create the IAM Policy; second, we create the Lambda function (which references the policy); finally, we create the EventBridge rule (which references the Lambda function).

Policy

Create a policy “AllowWeblogicAdminServerUpdateDNS“ with the following JSON. Replace <MY_HOSTED_ZONE_ID> with the ID you recorded earlier.

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Action": [
				"route53:ChangeResourceRecordSets"
			],
			"Resource": "arn:aws:route53:::hostedzone/<MY_HOSTED_ZONE_ID>",
			"Condition": {
				"ForAllValues:StringLike": {
					"route53:ChangeResourceRecordSetsNormalizedRecordNames": [
						"wlsadmin.example.com"
					]
				},
				"ForAnyValue:StringEquals": {
					"route53:ChangeResourceRecordSetsRecordTypes": "A"
				}
			}
		},
		{
			"Effect": "Allow",
			"Action": [
				"ec2:DescribeInstances"
			],
			"Resource": "*"
		}
	]
}

Lambda function

We create a Lambda function named “wlsAdminARecordUpdater” with the default settings for runtime (Node.js), architecture (x86_64) and permissions.

Add an environment variable named WLSHostedZoneID and value of the Hosted Zone ID created earlier.

A role will have been created for the Lambda function with a name beginning with “wlsAdminARecordUpdater-role-“. Add the policy AllowWeblogicAdminServerUpdateDNS to this role.

Finally, add the following code then save and deploy the Lambda function.

import { EC2Client, DescribeInstancesCommand } from "@aws-sdk/client-ec2"; 
import { Route53Client, ChangeResourceRecordSetsCommand } from "@aws-sdk/client-route-53"; 
				
export const handler = async (event, context, callback) => {
  				  
  const ec2input = {
    "InstanceIds": [
      event.detail.EC2InstanceId 
    ]
  };
				
  const ec2client = new EC2Client({region: event.region});
  const route53Client = new Route53Client({region: event.region});
				  
  const ec2command = new DescribeInstancesCommand(ec2input);
  const ec2data = await ec2client.send(ec2command);
  const ec2privateip = ec2data.Reservations[0].Instances[0].PrivateIpAddress;
				    
  const r53input = {
  "ChangeBatch": {
    "Changes": [
      {
        "Action": "UPSERT",
        "ResourceRecordSet": {
          "Name": "wlsadmin.weblogic.com",
          "ResourceRecords": [
            {
              "Value": ec2privateip
            }
          ],
          "TTL": 60,
          "Type": "A"
        }
      }
    ],
    "Comment": "weblogic admin server"
    },
    "HostedZoneId": process.env.WLSHostedZoneID
  };
 const r53command = new ChangeResourceRecordSetsCommand(r53input);
 
 return await route53Client.send(r53command);
 
};

EventBridge rule

We create an EventBridge rule, “wlsAdminASG-ScaleOut”, enabled on the default event bus.

  • Rule type: “Rule with an event pattern”
  • Event Source: AWS Events or EventBridge partner events
  • Creation Method – Use pattern Form
  • Event Pattern
    • Event Source: AWS Services
    • AWS Service: Auto Scaling
    • Event Type: Instance Launch and Terminate
    • Event Type Specification 1: Specific instance event(s)
    • Event Type Specification 2: wlsadmin-asg
      The event definition should look like the following example, scoped only to the Auto Scaling group wlsadmin-asg we created earlier.

      {
        "source": ["aws.autoscaling"],
        "detail-type": ["EC2 Instance Launch Successful"],
        "detail": {
          "AutoScalingGroupName": ["wlsadmin-asg"]
        }
      }
  • Target 1: AWS Service
    • Select a target: Lambda Service
    • Function: wlsAdminARecordUpdater

Review and create the rule. Note that “EventBridge (CloudWatch Events): wlsAdminASG-ScaleOut” will be added as a trigger to the Lambda function.

If you cycle the Auto Scaling group (set min and desired to 0, let the admin server terminate, then set min and desired to 1), you will observe that after the new server is successfully launched, the value of the DNS A record wlsadmin.example.com matches the IP of the new WebLogic Admin server.

Enabling internet access to the admin server

If we want to enable internet access to the admin server, we need to create an internet-facing Application Load Balancer (ALB) attached to the public subnets. With the route to the admin server, the ALB can forward traffic to it.

  1. Create an IP-based target group that points to the wlsadmin.example.com.
  2. Add a forwarding rule in the ALB to route WebLogic admin traffic to the admin server.

Conclusion

AWS has a successful track record of running Oracle applications, Oracle EBS, PeopleSoft, and mission critical JEE workloads. In this post, we delved into leveraging DNS for the WebLogic admin server location, and using Auto Scaling groups to ensure an available and singular admin server. We showed how to automate the DNS A record update for the admin server. We also covered enabling public access to the admin server. This solution showcases multi-AZ resilience for WebLogic admin server with automated recovery.

Root Access for Data Control: A DEF CON IoT Village Story

Post Syndicated from Rapid7 original https://blog.rapid7.com/2024/10/16/root-access-for-data-control-a-def-con-iot-village-story/

Root Access for Data Control: A DEF CON IoT Village Story

Every year, Rapid7 is a presenter at DEF CON’s IoT Village, sharing in-depth insight and expertise into the hacking of all things Internet of Things. This year, our perennial IoT hacking presenter, Principal Security Researcher, IoT, Deral Heiland, along with Rapid7 pentest team members, showed attendees many methods of extracting firmware from IoT devices and manipulating the systems in the name of control and operations.

Extracting firmware without the use of destructive means can be difficult and in some cases impossible. However, Deral went deep with IoT Village attendees, presenting a live hands-on exercise each attendee in the room could interact with. It was an enlightening and productive presentation. But we are aware not everyone could make it to DEF CON 32 this year.

Which is why we’ve transformed the presentation into a handy whitepaper. Deral has gone step-by-step through the exercise, and even improved upon it in some cases (so even if you were in the room, there’s likely even more for you to get from it). While DEF CON 32 may be firmly in the rear-view mirror, the hacking carries on. And if you missed DEF CON, or Deral’s presentation, you have another chance to learn and take part in the exercise.

To check out the whitepaper, please click here. And if you’d like to learn more about Deral’s previous IoT Village presentations (he’s done a lot of them), many live right here on the blog.

Enhance release control with AWS CodePipeline stage-level conditions

Post Syndicated from Ryan Bachman original https://aws.amazon.com/blogs/devops/enhance-release-control-with-aws-codepipeline-stage-level-conditions/

It’s an established practice for development teams to build deployment pipelines, with services such as AWS CodePipeline, to increase the quality of application and infrastructure releases through reliable, repeatable and consistent automation.

Automating the deployment process helps build quality into our products by introducing continuous integration to build and test code, however enterprises may sometimes wish to implement certain conditions such as an approved deployment window to ensure more fine-grained control over pipeline executions.

AWS CodePipeline recently added stage-level conditions to implement pipeline gates. In this blog post, we will cover how you can implement stage-level conditions to improve your governance, code quality, and security by following a simple pipeline scenario detailed in the sections below.

AWS CodePipeline

AWS CodePipeline is a fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates.

AWS CodePipeline orchestration stages

Figure 1. AWS CodePipeline orchestration

CodePipeline has three core constructs:

  • Pipelines – A pipeline is a workflow construct that describes how software changes go through a release process. Each pipeline is made up of a series of stages.
  • Stages – A stage is a logical unit you can use to isolate an environment and to limit the number of concurrent changes in that environment. Each stage contains actions that are performed on the application artifacts.
  • Actions – An action is a set of operations performed on application code and configured so that the actions run in the pipeline at a specified point.

A pipeline execution is a set of changes released by a pipeline. Each pipeline execution is unique and has its own ID. An execution corresponds to a set of changes, such as a merged commit or a manual release of the latest commit.

You can now configure stage-level conditions to gate a pipeline execution before entering the stage, as well as when exiting a stage for both success and failure state. A condition consists of one or more rules, and a result (such as rolling back) to apply when the condition fails. Conditions are also referred to as gates because they allow you to specify when an execution will enter and run through a stage or exit the stage after running through it.

You can configure a stage-level condition from the console, API, CLI, AWS CloudFormation, or SDK. For the purposes of this blog post, we will showcase how you can configure the two gate scenarios using the console. You can see how you can create conditions using the CLI in the documentation.

Scenario

For this blog, we started with a four stage Pipeline based off the Amazon ECS deployment example from this tutorial. While this tutorial provides a getting started example on how to build a CI/CD pipeline for an ECS application, it lacks the quality gates we want to enforce to ensure what we deploy to production is safe and tested. For this we will turn to CodePipeline stage-level conditions to verify certain criteria is met throughout the pipeline execution.

In our environment, we started with a source repository in a Github organization and used AWS CodeConnections to connect the source stage to our pipeline. Inside our repository are files for a sample application, a Dockerfile to build our application, and a buildspec.yaml file to define the steps that will be executed in the build stage of our pipeline. We added a fourth stage to our pipeline which separated production deployments after a successful deployment to the staging environment. We released a change using this pipeline to ensure everything deployed as expected.

Example of a 4-Stage CodePipeline

Figure 2: Example of a 4-Stage CodePipeline

Applying Stage Conditions

One of the primary concerns in any development process is maintaining high code quality standards. With stage-level conditions, you can now include specific checks to verify that your organization’s security and governance standards are met throughout the entire execution of your pipelines. These checks can include restricting deployments to certain days of the week, monitoring Amazon CloudWatch alarms prior to progressing, and verifying the results of security testing before promoting code as examples.

The AWS CodePipeline documentation details what conditions are available and how they can be used.

You can add conditions to stages on the pipelines detail page by choosing Edit and then selecting Edit stage from the relevant stage.

Adding a stage condition in the AWS console

Figure 3: Adding a Stage Condition in the AWS console

In our pipeline, we added rules that address three scenarios:

  • Scenario 1 – “On Success” condition that fails if the Amazon Elastic Container Registry (Amazon ECR) image scanning detects a critical severity in the Docker image in the build stage.
  • Scenario 2 – “On Failure” condition that rolls back a deployment stage in the event a CloudWatch alarm triggers after a successful deployment.
  • Scenario 3 – “Entry” condition to prevent deploying to production on Fridays, Saturdays and Sundays

Scenario 1 – “On Success” condition

For the first gate we implemented an OnSuccess exit condition that would fail the execution using the AWS Lambda Invoke rule provider. If you would like to dive deeper into this implementation, you can take a look at Logging image scan findings from Amazon ECR in CloudWatch using an AWS Lambda function.

Lambda invoke rule in the console

Figure 4: Lambda invoke rule in the console

Before we added this condition rule, we created a Lambda function in the same region that was responsible for getting the findings from an image scan that Amazon ECR executed. If a critical finding was detected in the image, the Lambda function will fail the condition. You can read more about utilizing Lambda with CodePipeline in the documentation. The LambdaInvoke rule is able to use input artifacts to implement additional logic in the function. Here we passed in the artifact created during the build stage which contains the ECR image name and tag. This pattern allows you to re-use Lambda functions as conditions across many pipelines.

After configuring the rule, we reviewed our stage configuration and saved the pipeline.

Adding a stage condition to the build stage

Figure 5: Adding a stage condition to the build stage

Scenario 2 – “On Failure” condition

Next, we configured a condition to fail and rollback a deployment in our staging environment. If you would like to read more about implementing rollback in your CodePipeline pipelines, you can refer to De-risk releases with AWS CodePipeline rollbacks. Oftentimes a deployment will appear successful, but regressions in the code, or bugs in a new feature, may surface only when users interact with the application. To mitigate this risk, we configured a CloudWatch alarm to alert when error rates exceeded 3% of overall traffic.

CloudWatch alarm definition

Figure 6: Amazon CloudWatch alarm definition

We then added an OnSuccess condition to the DeployToStage stage with a rollback rule that tracks the CloudWatch alarm configured for the above error rate metric. A wait time was configured in the rule to allow time for CloudWatch to detect any increase in errors related to the new deployment.

AWS CodePipeline condition rule using Amazon CloudWatch

Figure 7: AWS CodePipeline condition rule using Amazon CloudWatch

Scenario 3 – “Entry” condition

Finally, we added a third condition as an entry condition to the DeployToProduction stage. We added an AWS DeploymentWindow rule provider with a cron expression to fail the condition if a deployment to production was executed outside of our defined window.

Cron rule for AWS CodePipeline stage conditions

Figure 8: Cron rule for AWS CodePipeline stage conditions

Testing Stage Conditions

Testing Scenario 1

To test our CodePipeline with these stage-level conditions, we executed a git push to the source repository, which triggered a new pipeline execution. Our build stage successfully built a Docker image and pushed that image to the ECR repository, triggering the OnSuccess condition to check for critical vulnerabilities. The Lambda function reported that the scan exceeded our vulnerability threshold and stopped our pipeline with a reason of “Critical Findings Detected”

ECR scan condition failure

Figure 9: Amazon ECR scan condition failure

Navigating to the Amazon Elastic Container Registry service console, we were able to identify the critical vulnerability.

Amazon ECR vulnerability report

Figure 10: Amazon ECR vulnerability report

By reading the CVE we learned that the finding was related to a system package present in the source image defined in our Dockerfile. To remediate this issue, we added a new line to the Dockerfile in our source repository. The full Dockerfile is shown below for example with the change highlighted.

FROM public.ecr.aws/docker/library/python:3.12

WORKDIR /qchess
COPY . /qchess/
RUN apt-get remove --purge git git-man -y
RUN cd /qchess && pip install -r requirements.txt
RUN apt update && apt upgrade -y

CMD ["flask","run","--host=0.0.0.0","--port=80"]

With this change, we ensured that our image’s system packages would be upgraded on every build along with the version of the vulnerable package discovered by our condition check. We committed and pushed to our source repository to trigger a new pipeline execution.

Testing Scenario 2

During the next run, our Pipeline made it to the DeployToStage stage and successfully deployed our code changes to our staging environment. However, when the OnSuccess condition was executed, it detected that the error rate alarm had transitioned to alarm status, resulting in the stage condition to issue a rollback. We had a bug in our new feature!

CloudWatch Rule triggering a stage rollback

Figure 11: CloudWatch Rule triggering a stage rollback

We reviewed the CloudWatch alarm and noticed the elevated error rates related to the deployment. The error rates returned to normal after the stage completed its rollback.

Amazon CloudWatch alarm details

Figure 12: Amazon CloudWatch alarm details

After this rollback we reviewed our code changes and discovered a bug in the code. We made the appropriate changes and pushed to our main branch.

Testing Scenario 3

In the final pipeline execution, the pipeline made it past our DeployToStage condition and entered the entry condition in our DeployToProduction stage. This condition verified that the deployment was being executed within the defined deployment window and successfully deployed our change to production.

Deployment window allowed by condition rule

Figure 13: Deployment window allowed by stage condition rule

Clean up

If you followed along you should remove any resources you created related to this scenario. These resources include:

  • ECS Services
  • ECR Repository
  • CodeBuild Project
  • CodePipeline Pipeline
  • Lambda Function
  • CloudWatch Alarm
  • Associated IAM roles

Conclusion

Adding conditions to your AWS CodePipeline pipelines allows you to have fine-grained control over your pipeline. We have seen how you can safely automate deployments based on variables relevant to your organization. To learn more about CodePipeline conditions, visit How do stage conditions work.

Further reading

About the Authors:
Ryan Bachman profile image

Ryan Bachman

Ryan Bachman is a Sr. Specialist Solutions Architect with Amazon Web Services (AWS) with a focus on DevOps. Ryan is passionate about helping customers adopt process and services that increase their efficiency developing applications for the cloud. He has over 20 years professional experience as a technologist, including roles in development, network architecture, and technical product management.

Mirabela Dan profile image

Mirabela Dan

Mirabela Dan is a Solutions Architect for AWS with a focus on Next Generation Developer Experience and building Generative AI applications. She is passionate about DevOps and making developers lives easier with automation and productivity solutions. Outside of work, Mirabela loves travelling (70+ countries) and is a keen museum-goer and reader of history books.

Test Driving a New Benefit Programme in Belfast

Post Syndicated from Rapid7 original https://blog.rapid7.com/2024/10/16/test-driving-a-new-benefit-programme-in-belfast/

Test Driving a New Benefit Programme in Belfast

When most people think about benefits packages at work, what typically comes to mind are things like healthcare programmes, financial stipends, or wellbeing incentives. For Stephen, one benefit he uses on a daily basis comes on four wheels.

Rapid7’s electric vehicle scheme was rolled out in late 2023 for Belfast employees. The programme enables employees to lease an electric car via their employer and pay for it on a salary sacrifice basis, offering substantial tax and national insurance savings.

“I kept reading about the program and thinking – is it really this simple? What’s the catch?” said Stephen Gallagher, a Lead Product Manager who received his new electric BMW this past May. “The more I learned about the process and understood what that pre-tax payment would be vs. paying for a vehicle on my own, it was really a no brainer.”

The unique offering also contributes to the company’s sustainability goals by making electric vehicles more accessible, thanks to the pre-tax salary sacrifice. “I’ve worked for some other big tech companies in Belfast, but I’ve never seen this as a company offering. It definitely gives me a great sense of pride to work for Rapid7, and I feel motivated to do well for a company that takes care of employees in such a unique way.”

Test Driving a New Benefit Programme in Belfast

“The program provides employees with make and model options based on different salary levels to ensure the monthly payment is reasonable. Once an employee enrolls and selects a vehicle, our vendor sources it and coordinates delivery. Employees don’t pay anything until the vehicle is delivered.” Says Karen Hendry, Senior Benefits Manager. “I’ve watched employees go through the process, and I’m excited to have just taken ownership of a car myself through the programme!”

In addition to a competitive monthly payment, the program also eliminates the need for down payments, dealer fees, maintenance, or separate insurance fees as the offering is all inclusive. Stephen shared more on his recent experience by adding “I recently got a scratch on the car, so it’s been in the shop to get repaired. All I had to do was reach out to our vendor, and they got me in touch with a repair shop and coordinated everything for me”.

“As a benefits team, we are always evaluating our offerings and looking for ways to bring value to our employees through unique programmes. It’s exciting to see something new like this take off successfully in Belfast”

Learn more about Rapid7 in Belfast here.

[$] Using LKMM atomics in Rust

Post Syndicated from daroc original https://lwn.net/Articles/993785/

Rust, like C, has its own memory model describing how concurrent access to the
same data by multiple threads can behave.
The Linux kernel, however, has its own
ideas. The

Linux kernel memory model
(LKMM) is subtly different from both the
standard C memory model and Rust’s model.
At Kangrejos, Boqun Feng gave a presentation about the
need to reconcile the memory models used by Rust and the kernel,
including a few potential avenues for doing so. While
no consensus was reached, it is an area of active discussion.

[$] Two pidfd tweaks: PIDFD_GET_INFO and PIDFD_SELF

Post Syndicated from corbet original https://lwn.net/Articles/992991/

The pidfd mechanism, which uses file descriptors to refer to processes in
an unambiguous and race-free way, was first
introduced
in 2018. Since then, the interface has gained a number of new features, but
development has slowed over time as the interface has matured. There are,
however, a couple of patches in circulation that are meant to make working
with pidfds simpler in some situations.

The collective thoughts of the interwebz