Post Syndicated from Kedasha Kerr original https://github.blog/2023-03-06-how-to-automate-your-dev-environment-with-dev-containers-and-github-codespaces/

When I started my first role as a software engineer, I remember taking about four days to set up my local development environment. I had so many issues with missing dependencies, incorrect versions, and failed installations. When I finally finished setting up all the tools and software I needed to be a productive member of the team, I cloned one of our repositories to my machine, set up my environment variables, ran npm run dev and received so many errors because I forgot to install the dependencies (and read the README) or switch to the right node version. Ugh! I can’t tell you how many times this happened to me in my first year!

Back then, I wished I had a way that was streamlined—something I set up only once, that just worked every time I accessed the repository. Although I did learn how to automate my computer setup with a Brewfile, I wish I could just get to coding in a repository without thinking about configuration.

source

When I think about how we work on projects in a repository, I realize that many of the processes we need to get started on that project can be automated with the help of dev containers, in this case, by using a devcontainer.json file and Codespaces.

Let’s take a look at how we can automate our dev environment by adding a dev container to this open source project—Tech is Hiring in GitHub Codespaces.

For a TLDR of this post, GitHub Codespaces enables you to start coding faster when coupled with dev containers. See image below for a summary of how:

Now, let’s get some definitions out of the way.

What is GitHub Codespaces?

GitHub Codespaces is a development environment in the cloud. It is hosted by GitHub in an isolated environment (Docker container) that runs on a virtual machine. If you’re not familiar with virtual machines or Docker containers, take a look at these videos: what is a virtual machine? and what are Docker containers?.

Currently, individual developers have 60 hours of free codespaces usage per month, so definitely take advantage of this awesomeness to build from anywhere.

What are dev containers?

Dev containers allow us to run our code in a preconfigured, isolated environment (container). It gives us the ability to predefine our dev environment in our repositories and use a consistent, reliable environment across the board without worrying about configuration files—since it’s all set up for us from the beginning with a devcontainer.json file.

What is the devcontainer.json file?

The devcontainer.json file is a document that defines how your project will build, display, and run. Simply put, it allows us to specify the behavior we want in our dev environment. For example, if we have a devcontainer.json file that includes installing the ESLint extension in VS Code, once we open up a workspace, ESLint will be automatically installed for us.

Automating your workflow with dev containers and GitHub Codespaces

To start using GitHub Codespaces, we don’t need to set up a devcontainer.json file. By default, GitHub Codespaces uses the universal dev container image, which caters to a vast array of languages and tools. This means, whenever you open up a new codespace without a devcontainer.json file your codespace will automagically load so you can code instantly. However, adding a devcontainer.json file gives us the ability to automate a lot of our dev environment workflows to our liking.

Okay, okay, that was a lot of chatter—let’s now get into what you really came here for!

source

Using the open source project, Tech is Hiring, let’s walk through how we typically work with a repository using our local dev environment.

What work typically looks like

At first glance, we see that this project uses Nextjs, Tailwind CSS, Chakra UI, TypeScript, Storybook, Vite, Cypress, Axios, and Reactjs as some of its dependencies. We’d need to install all these dependencies to our local machine to get this project running.

1. Let’s clone the repository, and cd into the project.

   

2. Then, let’s install dependencies to get the project running locally.

   

3. This project uses storybook, so let’s run both storybook and spin up the actual app locally.

   

The process is not so bad, but it took a bit of time. We also need to check to make sure we’re using the correct node version, check if we need any environment variables, and if there are any runtime errors to resolve. Thankfully, I didn’t encounter any errors while working on this, but it still took a bit of time.

Going faster with GitHub Codespaces and dev containers

Let’s make the process better by adding a devcontainer.json file to this project and opening it in GitHub Codespaces to see what happens.

We can either use the VS Code command palette to add a pre-existing dev container or we can write the configuration file ourselves (which we’ll do below).

1. Let’s first create a .devcontainer folder in the root of the project and a devcontainer.json file in the new folder.

   

2. Now, let’s automate installing dependencies, starting the dev server, opening a preview of our app on localhost:3000, and installing vscode extensions. Once we get everything configured, your json file should look like this:

   {
     // image being used
      "image": "mcr.microsoft.com/devcontainers/universal:2",
     // set minimum cpu
      "hostRequirements": {
          "cpus": 4
      },
      // install dependencies and start app
      "updateContentCommand": "npm install",
      "postAttachCommand": "npm run dev",
      // open app.tsx once container is built
      "customizations": {
          "codespaces": {
              "openFiles": [
                  "src/pages/_app.tsx"
              ]
          },
          // install some vscode extensions
          "vscode": {
              "extensions": [
                  "dbaeumer.vscode-eslint",
                  "github.vscode-pull-request-github",
                  "eamodio.gitlens",
                  "christian-kohler.npm-intellisense"
              ]
          }
      },
      // connect to remote server
      "forwardPorts": [3000],
      // give port a label and open a preview of the app
      "portsAttributes": {
         "3000": {
            "label": "Application",
            "onAutoForward": "openPreview"
          }
        }
   }
   

   Sidenote: I’ve broken down the purpose of the properties in this file for you by adding comments before each. To learn more about each property, continue reading at container.dev. I also installed a few extensions that are not needed, but I wanted to show you that you could automate installing extensions, too!

3. Let’s commit the file and merge it into the main branch, then open up the project in GitHub Codespaces.

   

   When we open up the application in GitHub Codespaces, our dependencies will be installed, the server will start, and a preview will automatically open for us. If we needed environment variables to work on this repository, those would have already been configured for us as a repository secret on GitHub. We also didn’t need to install hefty node_modules to our machine.

I call this a win!

Comparing both ways

There’s plenty more that we can do with dev containers and GitHub Codespaces to automate our dev environment. But let’s summarize what we just did in GitHub Codespaces and with the help of dev containers:

• We clicked the GitHub Codespaces button on the GitHub repository.
• Everything was setup/installed for us (thanks json file!).
• We got to work and started coding.

Now, isn’t that better?

Wrapping up

So, what’s the point of GitHub Codespaces and why should you care as a developer? Well, for one, you can automate most of the startup processes you need to access a repository. You can also do a lot more customizations to your dev environment with dev containers. Take a look at all the options you have—and watch out for my next blog post where I’ll go through the anatomy of a devcontainer.json file.

Secondly, you can code from anywhere. I hate it when I’m not able to access one of my side projects on a different machine because that one machine is configured perfectly to suit the project. With GitHub Codespaces, you can start coding at the click of a button and from any machine that supports a modern browser.

I encourage you to get started with GitHub Codespaces today and try adding a devcontainer.json file to one of your projects! I promise you won’t regret it.

Until next time, happy coding!

Post Syndicated from Rizel Scarlett original https://github.blog/2023-02-28-10-things-you-didnt-know-you-could-do-with-github-codespaces/

Ever feel like you’re coding on a plane mid-flight? When I first learned to code about five years ago, my laptop was painstakingly slow, but I couldn’t afford a better one. That’s why I relied on browser-based IDEs like jsbin.com to run my code.

Now fast forward to today, where GitHub Codespaces provides a fully-fledged, browser-based Integrated Development Environment (IDE) on a virtual machine. This means you can code without draining your local machine’s resources. Cloud-powered development is a game-changer for folks with less powerful machines, but that barely scratches the surface of GitHub Codespaces’ versatility.

In this blog post, we’ll discuss a few ways that you can get the most out of GitHub Codespaces!

Generate AI images

You can run Stable Diffusion with GitHub Codespaces. Like DALL-E and Midjourney, Stable Diffusion is one of many machine-learning models using deep learning to convert text into art.

Stable Diffusion takes the following steps to convert text into art:

• AI receives an image
• AI adds noise to the image until the image becomes completely unrecognizable. (Noise is another way of saying pixelated dots.)
• AI removes the noise until it produces a clear, high-quality image
• AI learns from a database called LAION-Aesthetics. The database has image-text pairs to learn to convert text into images.

This entire process is resource-intensive! Experts recommend using a computer with a powerful graphics processing unit (GPU) to run data-heavy tasks like Stable Diffusion. However, not everyone has a computer with that type of computing power (including me), so instead, I use GitHub Codespaces.

Since your codespace is hosted on a virtual machine, you can set the machine type from 2-core to 32-core. You can request access to a GPU-powered codespace if you need a more powerful machine. This means a machine learning engineer can use an iPad or Chromebook to perform data-heavy deep learning computations via GitHub Codespaces.

Check out my DEV post and repository to learn more about how I generated art inside my codespace.

Below you can see the code and AI image that GitHub Codespaces helped me produce:

from torch import autocast

# Change prompt for image here!
prompt = "a cartoon black girl with cotton candy hair and a pink dress standing in front of a pink sky with cotton candy clouds"
with autocast(device):
  image = pipe(prompt, height=768, width=768).images[0]
image

Manage GitHub Codespaces from the command line

Back in the 2000s, I would inadvertently close my documents, or my computer would suddenly shut down, and my work would be lost forever. Fortunately, it’s 2023, and many editors protect against that, including GitHub Codespaces. If you close a codespace, it will save your work even if you forget to commit your changes.

However, if you’re done with a codespace because you pushed your code to a repository, you can feel free to delete it. GitHub Codespace management is important for the following reasons:

• You don’t want to get confused with all your GitHub Codespaces and accidentally delete the wrong one.
• By default, inactive GitHub Codespaces automatically delete every 30 days.

You can manage your GitHub Codespaces through the GitHub UI or GitHub CLI. The GitHub CLI allows you to do things like:

Create a codespace

gh codespace create -r OWNER/REPO_NAME [-b BRANCH]

List all your codespaces

gh codespace list

Delete a codespace:

gh codespace delete -c CODESPACE-NAME

Rename a codespace

gh codespace edit -c CODESPACE-NAME -d DISPLAY-NAME

Change the machine type of a codepace

gh codespace edit -m MACHINE-TYPE-NAME

This is not an exhaustive list of all ways you can manage GitHub Codespaces via the CLI. You learn more about managing your codespaces from the command line here!

Pair program with a teammate

Pair programming when you’re working remotely is challenging. It’s harder to share your screen and your code when you’re not sitting next to your teammate. However, it’s worthwhile when it works because it helps both parties improve their communication and technical skills. Installing the Live Share extension and forward ports can make remote pair programming easier with GitHub Codespaces.

Share your forwarded ports

You can share the URL of your locally hosted web app and make it available to your teammates or collaborators. If you right-click on the port you’d like to share, you’ll see the option to switch the port visibility from “Private” to “Public.” If you copy the value in the local address and share it with necessary collaborators, they can view, test, and debug your locally hosted web app even though they’re in a different room than you.

Live Share

The Live Share extension allows you and your teammate(s) to type in the same project simultaneously. It highlights your teammate’s cursor versus your cursor, so you can easily identify who is typing, and it’s completely secure!

Pair program with AI

Visual Studio Code’s marketplace offers a wide variety of extensions compatible with GitHub Codespaces, including GitHub Copilot. If you need to exchange ideas while you code, but there’s no one around, try installing the GitHub Copilot extension to pair program with AI. GitHub Copilot is an AI-powered code completion tool that helps you write code faster by suggesting code snippets and completing code for you.

Beyond productivity through code completion, GitHub Copilot empowers developers of varied backgrounds. One of my favorite GitHub Copilot tools is “Hey, GitHub,” a hands-free, voice-activated AI programmer. Check out the video below to learn how “Hey, GitHub” improves the developer experience for folks with limited physical dexterity or visual impairments.

---

---

Learn about more use cases for GitHub Copilot here.

Teach people to code

While educating people about coding via bootcamps, classrooms, meetups, and conferences, I’ve learned that teaching people how to code is hard. Sometimes in workshops, attendees spend most of the time trying to set up their environment so that they can follow along. However, teaching people to code or running a workshop in GitHub Codespaces creates a better experience for all participants. Instead of expecting beginner developers to understand how to clone repositories to work with a template, they can open up a codespace and work in a development environment you’ve configured. Now, you can have the peace of mind that everyone has the same environment as you and can easily follow along. This reduces tons of time, stress, and chaos.

We made GitHub Codespaces more accessible for teachers and students by offering 180 free hours of usage (equivalent to five assignments per month for a class of 50). Check out my tutorial on configuring GitHub Codespaces and leveraging extensions like CodeTour to run self-guided workshops.

Learn how CS50, Harvard’s largest class, uses GitHub Codespaces to teach students to code.

Learn a new framework

When you’re learning to code, it’s easy to over-invest your time into following tutorials. Learning is often more effective when you balance tutorial consumption with building projects. GitHub Codespaces’ quickstart templates are a speedy and efficient way to learn a framework.

Quickstart templates include boilerplate code, forwarded ports, and a configured development container for some of the most common application frameworks, including Next.js, React.js, Django, Express, Ruby on Rails, Preact, Flask, and Jupyter Notebook. Templates provide developers with a sandbox to build, test, and debug applications in a codespace. It only takes one click to open a template and experiment with a new framework.

Experimenting with a framework in a codespace can help developers better understand a framework’s structure and functionalities, leading to better retention and mastery. For example, I’m not familiar with Jupyter Notebook, but I’ve used a Jupyter Notebook template in GitHub Codespaces. The boilerplate code helped me to experiment with and better understand the structure of a Jupyter Notebook. I even built a small project with it!

Check out this blog post and template, where we use GitHub Codespaces to guide developers through writing their first lines of React.

Store environment variables

In the past, I’ve had multiple moments where I’ve accidentally shared or mishandled my environment variables. Here are a few cringe-worthy moments where I’ve mishandled environment variables:

• I’ve shown them to an audience during a live demo.
• I’ve pushed them to GitHub.
• I couldn’t figure out a great way to share the values with teammates.

To avoid these moments, I could’ve used GitHub Codespaces secrets. You can store API Keys, environment variables, and secret credentials in your GitHub Codespaces secrets for each of your GitHub Codespaces.

After you store them in your GitHub Codespaces secrets, you can refer to the environment variables in your code as: process.env.{SUPER_SECRET_API_KEY}.

Check out this tutorial for more information on securely storing your environment variables with GitHub Codespaces

Onboard developers faster

Typically, software engineers are responsible for setting up their local environment when they join a team. Local environment setup can take a day or a week, depending on the quality of the documentation. Fortunately, organizations can automate the onboarding process using GitHub Codespaces to configure a custom environment. When a new engineer joins the team, they can open a codespace and skip the local environment setup because the required extensions, dependencies, and environment variables exist within the codespace.

In 2021, the GitHub Engineering Team migrated from our macOS model to GitHub Codespaces for the development of GitHub.com. After over 14 years, our repository accrued almost 13 GB on disk. It took approximately 45 minutes to clone, set up dependencies, and bootstrap our repository. After fully migrating to GitHub Codespaces, it only takes 10 seconds to launch GitHub.com in a preconfigured, reliable codespace.

However, you don’t have to take our word for it. Ketan Shah, Manager of Information Services at TELUS, a telecommunications company, attests, “GitHub shaves off entire days from the new employee onboarding process. New developers are up and running within minutes because everything is all in one place.”

Learn more about GitHub Codespaces can quickly onboard developers:

• How TELUS uses GitHub Codespaces
• How Elanco uses GitHub Codespaces
• How your team can use GitHub Codespace to configure a positive onboarding experience with development containers

Conduct technical interviews

Take-home projects, technical screens, and live coding are often painful, but necessary parts of the interview experience for software developers, but using GitHub Codespaces can help ease some of the difficulties. By providing candidates with a well-configured and reliable environment, interviewers can help reduce anxiety and improve performance. With GitHub Codespaces, candidates no longer have to worry about setting up their environment. Interviewers can use GitHub codespaces to provide real-time feedback and collaborate using built-in tools like Visual Studio Code Live Share. Furthermore, GitHub Codespaces offers a secure and isolated environment for the interview process, ensuring that sensitive information and data remain protected.

Learn how various engineering teams at GitHub use GitHub Codespaces to conduct interviews.

Code in the cloud with your preferred editor

I love Visual Studio Code; it’s my primary editor. However, depending on your role and other factors, you might have a different preference. GitHub Codespaces supports the following editors, in addition to Visual Studio Code:

• Jupyter Notebook
• IntelliJ IDEA
• RubyMine
• GoLand
• PyCharm
• PhpStorm
• WebStorm

But, wait, there’s more!

GitHub Codespaces’ superpower is that you can code from any device and get a standardized environment as long as you have internet. However, it also includes features that make software development workflows easier. With 60 hours of free access per month for GitHub users and 90 hours per month for GitHub Pro users, there’s no reason not to try it out and experience its benefits for yourself. So why not give it a try today?

Post Syndicated from Tanmayee Kamath original https://github.blog/2022-06-15-prebuilding-codespaces-is-generally-available/

Prebuilding codespaces is generally available

We’re excited to announce that the ability to prebuild codespaces is now generally available. As a quick recap, a prebuilt codespace serves as a “ready-to-go” template where your source code, editor extensions, project dependencies, commands, and configurations have already been downloaded, installed, and applied, so that you don’t have to wait for these tasks to finish each time you create a new codespace. This helps significantly speed up codespace creations–especially for complex or large codebases.

Codespaces prebuilds entered public beta earlier this year, and we received a ton of feedback around experiences you loved, as well as areas we could improve on. We’re excited to share those with you today.

How Vanta doubled its engineering team with Codespaces

With Codespaces prebuilds, Vanta was able to significantly reduce the time it takes for a developer to onboard. This was important, because Vanta’s Engineering Team doubled in size in the last few months. When a new developer joined the company, they would need to manually set up their dev environment; and once it was stable, it would diverge within weeks, often making testing difficult.

“Before Codespaces, the onboarding process was tedious. Instead of taking two days, now it only takes a minute for a developer to access a pristine, steady-state environment, thanks to prebuilds,” said Robbie Ostrow, Software Engineering Manager at Vanta. “Now, our dev environments are ephemeral, always updated and ready to go.”

Scheduled prebuilds to manage GitHub Actions usage

Repository admins can now decide how and when they want to update prebuild configurations based on their team’s needs. While creating or updating prebuilds for a given repository and branch, admins can choose from three available triggers to initiate a prebuild refresh:

• Every push (default): Prebuild configurations are updated on every push made to the given branch. This ensures that new Codespaces always contain the latest configuration, including any recently added or updated dependencies.
• On configuration change: Prebuild configurations are updated every time configuration files change. This ensures that the latest configuration changes appear in new Codespaces. The Actions workflow that generates the prebuild template will run less often, so this option will use fewer Actions minutes.
• Scheduled: With this setting, you can have your prebuild configurations update on a custom schedule. This can help further reduce the consumption of Actions minutes.

With increased control, repository admins can make more nuanced trade-offs between “environment freshness” and Actions usage. For example, an admin working in a large organization may decide to update their prebuild configuration every hour rather than on every push to get the most economy and efficiency out of their Actions usage.

Failure notifications for efficient monitoring

Many of you shared with us the need to be notified when a prebuild workflow fails, primarily to be able to watch and fix issues if and when they arise. We heard you loud and clear and have added support for failure notifications within prebuilds. With this, repository admins can specify a set of individuals or teams to be informed via email in case a workflow associated with that prebuild configuration fails. This will enable team leads or developers in charge of managing prebuilds for their repository to stay up to date on any failures without having to manually monitor them. This will also enable them to make fixes faster, thus ensuring developers working on the project continue getting prebuilt codespaces.

To help with investigating failures, we’ve also added the ability to disable a prebuild configuration in the instance repository admins would like to temporarily pause the update of a prebuild template while fixing an underlying issue.

Improved ‘prebuild readiness’ indicators

Lastly, to help you identify prebuild-enabled machine types to avail fast creations, we have introduced a ‘prebuild in progress’ label in addition to the ‘prebuild ready’ label in cases where a prebuild template creation for a given branch is in progress.

Billing for prebuilds

With general availability, organizations will be billed for Actions minutes required to run prebuild associated workflows and storage of templates associated with each prebuild configuration for a given repository and region. As an admin, you can download the usage report for your organization to get a detailed view of prebuild-associated Actions and storage costs for your organization-owned repositories to help you manage usage.

Alongside enabling billing, we’ve also added a functionality to help manage prebuild-associated storage costs based on the valuable feedback that you shared with us.

Template retention to manage storage costs

Repository administrators can now specify the number of prebuild template versions to be retained with a default template retention setting of two. A default of two means that the codespace service will retain the latest and one previous prebuild template version by default, thus helping you save on storage for older versions.

How to get started

Prebuilds are generally available for the GitHub Enterprise Cloud and GitHub Team plans as of today.

As an organization or repository admin, you can head over to your repository’s settings page and create prebuild configurations under the “Codespaces” tab. As a developer, you can create a prebuilt codespace by heading over to a prebuild-enabled branch in your repository and selecting a machine type that has the “prebuild ready” label on it.

Here’s a link to the prebuilds documentation to help you get started!

Post general availability, we’ll continue working on functionalities to enable prebuilds on monorepos and multi-repository scenarios based on your feedback. If you have any feedback to help improve this experience, be sure to post it on our GitHub Discussions forum.

Post Syndicated from Tanmayee Kamath original https://github.blog/2022-02-23-codespaces-largest-repositories-faster/

Today, the ability to prebuild codespaces is entering public beta. Prebuilding a codespace enables fast environment creation times, regardless of the size or complexity of your repositories. A prebuilt codespace will serve as a “ready-to-go” template where your source code, editor extensions, project dependencies, commands, and configurations have already been downloaded, installed, and applied so that you don’t have to wait for these tasks to finish each time you create a new codespace.

Getting to public beta

Our primary goal with Codespaces is to provide a one-click onboarding solution that enables developers to get started on a project quickly without performing any manual setup. However, because a codespace needs to clone your repository and (optionally) build a custom Dockerfile, install project dependencies and editor extensions, initialize scripts, and so on in order to bootstrap the development environment, there can be significant variability in the startup times that developers actually experience. A lot of this depends on the repository size and the complexity of a configuration.

As some of you might be aware, migrating to Codespaces transformed how we develop at GitHub.

Prebuilds were a huge part of how we meaningfully reduced the time-to-bootstrap in Codespaces for our core GitHub.com codebase. With that, our next mission was to replicate this success and enable the experience for our customers. Over the past few months, we ran a private preview for prebuilds with approximately 50 organizations. Overall, we received positive feedback on the ability of prebuilds to improve productivity for teams working on complex projects. At the same time, we also received a ton of valuable feedback around the configuration and management of prebuilds, and we’re excited to share those improvements with you today:

• You can now identify and quickly get started with a fast create experience by selecting machine types that have a “prebuild ready” tag.
• A seamless configuration experience helps repository admins easily set up and manage prebuild configurations for different branches and regions.
• To reduce the burden on repository admins around managing Action version updates for each prebuilt branch, we introduced support for GitHub Actions workflows that will be managed by the Codespaces service.
• Prebuild configurations are now built on GitHub Actions virtual machines. This enables faster prebuild template creations for each push made to your repository, and also provides repository admins with access to a rich set of logs to help with efficient debugging in case failures occur.

Our goal is to keep iterating on this experience based on the feedback captured during public beta and to continue our mission of enabling a seamless developer onboarding experience.

So how do prebuilds work?

During public beta, repository admins will be able to create prebuild configurations for specific branches and region(s) in their repository.

Prebuild configurations will automatically trigger an associated GitHub Actions workflow, managed by the Codespaces service, that will take care of prebuilding the devcontainer configuration and any subsequent commits for that branch. Associated prebuild templates will be stored in blob storage for each of the selected regions.

Each workflow will provide a rich set of logs to help with debugging in case failures occur.

Every time you request a prebuilt codespace, the service will fetch a prebuilt template and attach it to an existing virtual machine, thus significantly reducing your codespace creation time. To request changes to the prebuild configuration for your branch as per your needs, you can always update its associated devcontainer configuration with a pull request, specifically using the onCreateCommand or updateContentCommand lifecycle scripts.

How to get started

Prebuilds are available to try in public beta for all organizations that are a part of GitHub Enterprise Cloud and Team plans. As an organization or repository admin, you can head over to your repository’s settings page and create prebuild configurations under the “Codespaces” tab. As a developer, you can create a prebuilt codespace by heading over to a prebuild-enabled branch in your repository and selecting a machine type that has the “prebuild ready” label on it.

Here’s a link to the prebuilds documentation to help you get started!

If you have any feedback to help improve this experience, be sure to post it on our discussions forum.

Post Syndicated from Cory Wilkerson original https://github.blog/2021-08-11-githubs-engineering-team-moved-codespaces/

Today, GitHub is making Codespaces available to Team and Enterprise Cloud plans on github.com. Codespaces provides software teams a faster, more collaborative development environment in the cloud. Read more on our Codespaces page.

---

The GitHub.com codebase is almost 14 years old. When the first commit for GitHub.com was pushed, Rails was only two years old. AWS was one. Azure and GCP did not yet exist. This might not be long in COBOL time, but in internet time it’s quite a lot.

Over those 14 years, the core repository powering GitHub.com (github/github) has seen over a million commits. The vast majority of those commits come from developers building and testing on macOS.

But our development platform is evolving. Over the past months, we’ve left our macOS model behind and moved to Codespaces for the majority of GitHub.com development. This has been a fundamental shift for our day-to-day development flow. As a result, the Codespaces product is stronger and we’re well-positioned for the future of GitHub.com development.

The status quo

Over the years, we’ve invested significant time and effort in making local development work well out of the box. Our scripts-to-rule-them-all approach has presented a familiar interface to engineers for some time now—new hires could clone github/github, run setup and bootstrap scripts, and have a local instance of GitHub.com running in a half-day’s time. In most cases things just worked, and when they didn’t, our bootstrap script would open a GitHub issue connecting the new hire with internal support. Our #friction Slack channel—staffed by helpful, kind engineers—could debug nearly any system configuration under the sun.

Yet for all our efforts, local development remained brittle. Any number of seemingly innocuous changes could render a local environment useless and, worse still, require hours of valuable development time to recover. Mysterious breakage was so common and catastrophic that we’d codified an option for our bootstrap script: --nuke-from-orbit. When invoked, the script deletes as much as it responsibly can in an attempt to restore the local environment to a known good state.

And of course, this is a classic story that anyone in the software engineering profession will instantly recognize. Local development environments are fragile. And even when functioning perfectly, a single-context, bespoke local development environment felt increasingly out of step with the instant-on, access-from-anywhere world in which we now operate.

Collaborating on multiple branches across multiple projects was painful. We’d often find ourselves staring down a 45-minute bootstrap when a branch introduced new dependencies, shipped schema changes, or branched from a different SHA. Given how quickly our codebase changes (we’re deploying hundreds of changes per day), this was a regular source of engineering friction.

And we weren’t the only ones to take notice—in building Codespaces, we engaged with several best-in-class engineering organizations who had built Codespaces-like platforms to solve these same types of problems. At any significant scale, removing this type of productivity loss becomes a very clear productivity opportunity, very quickly.

Development infrastructure

In the infrastructure world, industry best practices have continued to position servers as a commodity. The idea is that no single server is unique, indispensable, or irreplaceable. Any piece could be taken out and replaced by a comparable piece without fanfare. If a server goes down, that’s ok! Tear it down and replace it with another one.

Our local development environments, however, are each unique, with their own special quirks. As a consequence, they require near constant vigilance to maintain. The next git pull or bootstrap can degrade your environment quickly, requiring an expensive context shift to a recovery effort when you’d rather be building software. There’s no convention of a warm laptop standing by.

But there’s a lot to be said for treating development environments as our own—they’re the context in which we spend the majority of our day! We tweak and tune our workbench in service of productivity but also as an expression of ourselves.

With Codespaces, we saw an opportunity to treat our dev environments much like we do infrastructure—a commodity we can churn—but still maintain the ability to curate our workbench. Visual Studio Code extensions, settings sync, and dotfiles repos bring our environment to our compute. In this context, a broken workbench is a minor inconvenience—now we can provision a new codespace at a known good state and get back to work.

Adopting Codespaces

Migrating to Codespaces addressed the shortcomings in our existing developer environments, motivated us to push the product further, and provided leverage to improve our overall development experience.

And while our migration story has a happy ending, the first stages of our transition were… challenging. The GitHub.com repository is almost 13 GB on disk; simply cloning the repository takes 20 minutes. Combined with dependency setup, bootstrapping a GitHub.com codespace would take upwards of 45 minutes. And once we had a repository successfully mounted into a codespace, the application wouldn’t run.

Those 14 years of macOS-centric assumptions baked into our bootstrapping process were going to have to be undone.

Working through these challenges brought out the best of GitHub. Contributors came from across the company to help us revisit past decisions, question long-held assumptions, and work at the source-level to decouple GitHub development from macOS. Finally, we could (albeit very slowly) provision working GitHub.com codespaces on Linux hosts, connect from Visual Studio Code, and ship some work. Now we had to figure out how to make the thing hum.

45 minutes to 5 minutes

Our goal with Codespaces is to embrace a model where development environments are provisioned on-demand for the task at hand (roughly a 1:1 mapping between branches and codespaces.) To support task-based workflows, we need to get as close to instant-on as possible. 45 minutes wasn’t going to meet our task-based bar, but we could see low-hanging fruit, ripe with potential optimizations.

Up first: changing how Codespaces cloned github/github. Instead of performing a full clone when provisioned, Codespaces would now execute a shallow clone and then, after a codespace was created with the most recent commits, unshallow repository history in the background. Doing so reduced clone time from 20 minutes to 90 seconds.

Our next opportunity: caching the network of software and services that support GitHub.com, inclusive of traditional Gemfile-based dependencies as well as services written in C, Go, and a custom build of Ruby. The solution was a GitHub Action that would run nightly, clone the repository, bootstrap dependencies, and build and push a Docker image of the result. The published image was then used as the base image in github/github’s devcontainer—config-as-code for Codespaces environments. Our codespaces would now be created at 95%+ bootstrapped.

These two changes, along with a handful of app and service level optimizations, took GitHub.com codespace creation time from 45 minutes to five minutes. But five minutes is still quite a distance from “instant-on.” Well-known studies have shown people can sustain roughly ten seconds of wait time before falling out of flow. So while we’d made tremendous strides, we still had a way to go.

5 minutes to 10 seconds

While five minutes represented a significant improvement, these changes involved tradeoffs and hinted at a more general product need.

Our shallow clone approach—useful for quickly launching into Codespaces—still required that we pay the cost of a full clone at some point. Unshallowing post-create generated load with distracting side effects. Any large, complex project would face a similar class of problems during which cloning and bootstrapping created contention for available resources.

What if we could clone and bootstrap the repository ahead of time so that by the time an engineer asked for a codespace we’d already done most of the work?

Enter prebuilds: pools of codespaces, fully cloned and bootstrapped, waiting to be connected with a developer who wants to get to work. The engineering investment we’ve made in prebuilds has returned its value many times over: we can now create reliable, preconfigured codespaces, primed and ready for GitHub.com development in 10 seconds.

New hires can go from zero to a functioning development environment in less time than it takes to install Slack. Engineers can spin off new codespaces for parallel workstreams with no overhead. When an environment falls apart—maybe it’s too far behind, or the test data broke something—our engineers can quickly create a new environment and move on with their day.

Increased leverage

The switch to Codespaces solved some very real problems for us: it eliminated the fragility and single-track model of local development environments, but it also gave us a powerful new point of leverage for improving GitHub’s developer experience.

We now have a wedge for performing additional setup and optimization work that we’d never consider in local environments, where the cost of these optimizations (in both time and patience) is too high. For instance, with prebuilds we now prime our language server cache and gem documentation, run pending database migrations, and enable both GitHub.com and GitHub Enterprise development modes—a task that would typically require yet another loop through bootstrap and setup.

With Codespaces, we can upgrade every engineer’s machine specs with a single configuration change. In the early stages of our Codespaces migration, we used 8 core, 16 GB RAM VMs. Those machines were sufficient, but GitHub.com runs a network of different services and will gladly consume every core and nibble of RAM we’re willing to provide. So we moved to 32 core, 64 GB RAM VMs. By changing a single line of configuration, we upgraded every engineer’s machine.

Codespaces has also started to steal business from our internal “review lab” platform—a production-like environment where we preview changes with internal collaborators. Before Codespaces, GitHub engineers would need to commit and deploy to a review lab instance (which often required peer review) in order to share their work with colleagues. Friction. Now we ctrl+click, grab a preview URL, and send it on to a colleague. No commit, no push, no review, no deploy — just a live look at port 80 on my codespace.

Command line

Visual Studio Code is great. It’s the primary tool GitHub.com engineers use to interface with codespaces. But asking our Vim and Emacs users to commit to a graphical editor is less great. If Codespaces was our future, we had to bring everyone along.

Happily, we could support our shell-based colleagues through a simple update to our prebuilt image which initializes sshd with our GitHub public keys, opens port 22, and forwards the port out of the codespace.

From there, GitHub engineers can run Vim, Emacs, or even ed if they so desire.

This has worked exceedingly well! And, much like how Docker image caching led to prebuilds, the obvious next step is taking what we’ve done for the GitHub.com codespace and making it a first-class experience for every codespace.

Reception

Change is hard, doubly so when it comes to development environments. Thankfully, GitHub engineers are curious and kind—and quickly becoming Codespaces superfans.

I used codespaces yesterday while my dev environment was a little broken and I finished the entire features on codespaces before my dev env was done building lol ~@lindseyb My friends, I’m here to tell you I was a Codespaces skeptic before this started and now I am not. This is the way. ~@iolsen I really was more productive with respect to the Rails part of my work this week than I think I ever have been before. Everything was just so fast and reliable. ~@jclem Whomever has worked on getting codespaces up and running, you enabled me to have an awesome first week! ~@bestra I do solemnly swear that never again will my CPU have to compile ruby from source. ~@latentflip

Codespaces are now the default development environment for GitHub.com. That #friction Slack channel that we mentioned earlier to help debug local development environment problems? We’re planning to archive it.

We’re onboarding more services and more engineers throughout GitHub every day, and we’re discovering new stories about the value Codespaces can generate along the way. But at the core of each story, you’ll discover a consistent theme that resonates with every engineer: I found a better tool, I’m more productive now, and I’m not going back.