Piping streams

A common usecase for streams is downloading files without buffering the whole file in memory. First, we open a file we want to write to

const download = await Ende.open("example.html", { create: true, write: true });

Now let’s make a request to a web page

const req = await fetch("https://examples.ende.sh");

We can pipe this response straight into the file. In a more realistic example, we would handle a bad request, but here we’ll just use the nullish coalescing operator. Instead of opening the file and manually piping the fetch body to it, we could also just use `Ende.writeFile`.

req.body?.pipeTo(download.writable);

Pipes can connect a lot of things. For example, we could pipe the file we just downloaded into stdout. We could also pipe our streams through transformers to get a more interesting result. In this case, we will pipe our file through a stream which will highlight all "<" characters in the terminal.

First we will import a utility from the standard library to help us with this.

import { bgBrightYellow } from "jsr:@ende/fmt/colors";

Then we will create a transform stream utility class

class HighlightTransformStream extends TransformStream<string, string> {
  constructor() {
    super({
      transform: (chunk, controller) => {
        controller.enqueue(chunk.replaceAll("<", bgBrightYellow("<")));
      }
    });
  }
}

Let’s open our file for reading

const example = await Ende.open("example.html", { read: true });

Now we can pipe the result from the file, into a TextDecoderStream, into our custom transform class, back through a TextEncoderStream, and finally into stdout

await example.readable
  .pipeThrough(new TextDecoderStream())
  .pipeThrough(new HighlightTransformStream())
  .pipeThrough(new TextEncoderStream())
  .pipeTo(Ende.stdout.writable);

Run this example locally using the Ende CLI:

ende run -N -R -W https://docs.ende.sh/examples/scripts/piping_streams.ts

Additional resources

• MDN: ReadableStream
• Example: TCP Listener