Post Syndicated from Brendan Coll original https://blog.cloudflare.com/automatically-generated-types/

Historically, keeping our Rust and TypeScript type repos up to date has been hard. They were manually generated, which means they ran the risk of being inaccurate or out of date. Until recently, the workers-types repository needed to be manually updated whenever the types changed. We also used to add type information for mostly complete browser APIs. This led to confusion when people would try to use browser APIs that aren’t supported by the Workers runtime they would compile but throw errors.

That all changed this summer when Brendan Coll, whilst he was interning with us, built an automated pipeline for generating them. It runs every time we build the Workers runtime, generating types for our TypeScript and Rust repositories. Now everything is up-to-date and accurate.

A quick overview

Every time the Workers runtime code is built, a script runs over the public APIs and generates the Rust and TypeScript types as well as a JSON file containing an intermediate representation of the static types. The types are sent to the appropriate repositories and the JSON file is uploaded as well in case people want to create their own types packages. More on that later.

This means the static types will always be accurate and up to date. It also allows projects running Workers in other, statically-typed languages to generate their own types from our intermediate representation. Here is an example PR from our Cloudflare bot. It’s detected a change in the runtime types and is updating the TypeScript files as well as the intermediate representation.

Using the auto-generated types

To get started, use wrangler to generate a new TypeScript project:

$ wrangler generate my-typescript-worker https://github.com/cloudflare/worker-typescript-template

If you already have a TypeScript project, you can install the latest version of workers-types with:

$ npm install --save-dev @cloudflare/workers-types

And then add @cloudflare/workers-types to your project’s tsconfig.json file.

{
"compilerOptions": {
"target": "ES2020",
"module": "CommonJS",
"lib": ["ES2020"],
"types": ["@cloudflare/workers-types"]
}
}

After that, you should get automatic type completion in your IDE of choice.

How it works

Here is some example code from the Workers runtime codebase.

class Blob: public js::Object {
public:
typedef kj::Array<kj::OneOf<kj::Array<const byte>, kj::String, js::Ref<Blob>>> Bits;
struct Options {
js::Optional<kj::String> type;
JS_STRUCT(type);
};
static js::Ref<Blob> constructor(js::Optional<Bits> bits, js::Optional<Options> options);
int getSize();
js::Ref<Blob> slice(js::Optional<int> start, js::Optional<int> end);
JS_RESOURCE_TYPE(Blob) {
JS_READONLY_PROPERTY(size, getSize);
JS_METHOD(slice);
}
};

A Python script runs over this code during each build and generates an Abstract Syntax Tree containing information about the function including an identifier, any argument types and any return types.

{
  "name": "Blob",
  "kind": "class",
  "members": [
    {
      "name": "size",
      "type": {
        "name": "integer"
      },
      "readonly": true
    },
    {
      "name": "slice",
      "type": {
        "params": [
          {
            "name": "start",
            "type": {
              "name": "integer",
              "optional": true
            }
          },
          {
            "name": "end",
            "type": {
              "name": "integer",
              "optional": true
            }
          }
        ],
        "returns": {
          "name": "Blob"
        }
      }
    }
  ]
}

Finally, the TypeScript types repositories are automatically sent PRs with the updated types.

declare type BlobBits = (ArrayBuffer | string | Blob)[];

interface BlobOptions {
  type?: string;
}

declare class Blob {
  constructor(bits?: BlobBits, options?: BlobOptions);
  readonly size: number;
  slice(start?: number, end?: number, type?: string): Blob;
}

Overrides

In some cases, TypeScript supports concepts that our C++ runtime does not. Namely, generics and function overloads. In these cases, we override the generated types with partial declarations. For example, DurableObjectStorage makes heavy use of generics for its getter and setter functions.

declare abstract class DurableObjectStorage {
	 get<T = unknown>(key: string, options?: DurableObjectStorageOperationsGetOptions): Promise<T | undefined>;
	 get<T = unknown>(keys: string[], options?: DurableObjectStorageOperationsGetOptions): Promise<Map<string, T>>;
	 
	 list<T = unknown>(options?: DurableObjectStorageOperationsListOptions): Promise<Map<string, T>>;
	 
	 put<T>(key: string, value: T, options?: DurableObjectStorageOperationsPutOptions): Promise<void>;
	 put<T>(entries: Record<string, T>, options?: DurableObjectStorageOperationsPutOptions): Promise<void>;
	 
	 delete(key: string, options?: DurableObjectStorageOperationsPutOptions): Promise<boolean>;
	 delete(keys: string[], options?: DurableObjectStorageOperationsPutOptions): Promise<number>;
	 
	 transaction<T>(closure: (txn: DurableObjectTransaction) => Promise<T>): Promise<T>;
	}

You can also write type overrides using Markdown. Here is an example of overriding types of KVNamespace.

Creating your own types

The JSON IR (intermediate representation) has been open sourced alongside the TypeScript types and can be found in this GitHub repository. We’ve also open sourced the type schema itself, which describes the format of the IR. If you’re interested in generating Workers types for your own language, you can take the IR, which describes the declaration in a “normalized” data structure, and generate types from it.

The declarations inside `workers.json` contain the elements to derive function signatures and other elements needed for code generation such as identifiers, argument types, return types and error management. A concrete use-case would be to generate external function declarations for a language that compiles to WebAssembly, to import precisely the set of available function calls available from the Workers runtime.

Conclusion

Cloudflare cares deeply about supporting the TypeScript and Rust ecosystems. Brendan created a tool which will ensure the type information for both languages is always up-to-date and accurate. We also are open-sourcing the type information itself in JSON format, so that anyone interested can create type data for any language they’d like!