Part VI — Networking & the web

Sockets and listeners, the stream layer over them, the pure-Quoin HTTP client and server, and the [Web] application framework — whose core is a function you can call without any network at all.

Nav: Foundations · Blocks & control · Objects · Patterns & errors · Concurrency & iteration · Networking & the web · Types · Tooling · Library & reference · Appendices

22. The I/O model

Everything in this chapter rides Part V's concurrency system: all I/O is asynchronous by construction. A read, write, accept, or sleep that cannot proceed parks the current task, and the cooperative scheduler runs other tasks meanwhile — one API, no blocking/non-blocking split, so thousands of idle connections multiplex over one program. Tasks and their handles, Async.gather:, Async.timeout:do:, and cancellation are Part V's subject (§18–19); a CPU-bound handler round-robins with its siblings at scheduler boundaries rather than starving them (§18), but real multi-core parallelism is worker isolates (§21, or serve:workers: in §27). Sockets, listeners, and streams are native built-ins — always available, no use. The HTTP layers are pure Quoin in the stdlib: use std:net/http loads the [HTTP] client, use std:net/http_server the [HTTP]Server transport, use std:web/* the [Web] framework, and use std:net/* both net layers at once.

Every runnable example in this chapter uses one shape, borrowed from the VM's own test suite: bind 127.0.0.1:0 (the OS picks a free port), run server and client as concurrent tasks under Async.gather:, and wrap the lot in Async.timeout: as a hang guard — nothing below touches a non-loopback network:

var results = Async.timeout:5000 do:{
    Async.gather:#(
        { Async.sleep:10; 'slow' }
        { 'fast' }
    )
};
results        "* -> #(slow fast)

The two blocks run as concurrent tasks: the total wait is roughly the slowest task, not the sum, and results come back in input order.

23. Sockets & streams

Rules

A complete echo round-trip — server and client are just two tasks on the same scheduler:

var listener = TcpListener.listen:'127.0.0.1:0';       "* port 0: the OS picks
var target = '127.0.0.1:' + listener.port.s;
var results = Async.timeout:5000 do:{
    Async.gather:#(
        { listener.acceptOnce:{ |conn| conn.writeAll:(conn.read:5) }; 'served' }
        { TcpSocket.connect:target do:{ |c| c.writeAll:'hello'.asBytes; (c.read:5).asString } }
    )
};
listener.close;
results         "* -> #(served hello)

The server task parks in acceptOnce:; the client task connects, writes, and parks in read:; the scheduler interleaves them until both finish. For anything line- or delimiter-oriented, wrap the socket in a stream instead of framing by hand:

var listener = TcpListener.listen:'127.0.0.1:0';
var target = '127.0.0.1:' + listener.port.s;
var reply = Async.timeout:5000 do:{
    (Async.gather:#(
        { listener.acceptOnce:{ |conn|
            var st = conn.stringStream;        "* consumes conn: text framing on top
            st.writeln:('you said ' + st.readLine) } }
        { var c = TcpSocket.connect:target;
          var st = c.stringStream;
          st.writeln:'ping';
          var line = st.readLine;
          st.close;
          line }
    )).at:1
};
listener.close;
reply           "* -> 'you said ping'

⚠ Gotcha — wrapping consumes the handle below. socket.byteStream, stream.stringStream, and TlsSocket.wrap:host: all transfer the connection upward: the lower handle is left closed and further operations on it throw. Keep the topmost wrapper and talk only to it. (acceptOnce: closing the original socket afterwards is still safe — close is idempotent.)

The write-side split matters once files enter the picture — a socket write goes straight to the peer, but a file write stream holds bytes back until they're worth a syscall:

var path = '/tmp/qn-book-streams.txt';
var out = ([IO]File.create:path).stringStream;   "* a BUFFERED write stream
out.writeln:'alpha';
out.writeln:'beta';                              "* still in the 16 KiB buffer
out.close;                                       "* close (or flush!) drains it
var lines = #();
([IO]File.open:path).stringStream.eachLine:{ |l| lines.add:l };
[IO]File.delete:path;
lines           "* -> #(alpha beta)

⚠ Gotcha — readAll on a socket returns only at EOF. It keeps reading until the peer closes. In a request/response protocol where the other side keeps the connection open, readAll parks forever — frame reads with read:, readExactly:, or readUntil: instead (that is what the HTTP layers do).

24. TcpServer

Rules

var server = TcpServer.new:{ var address = '127.0.0.1:0' };
server.start:{ |conn| conn.writeAll:(conn.read:4) };
var target = '127.0.0.1:' + server.port.s;
var replies = Async.timeout:5000 do:{
    Async.gather:#(
        { TcpSocket.connect:target do:{ |c| c.writeAll:'AAAA'.asBytes; (c.read:4).asString } }
        { TcpSocket.connect:target do:{ |c| c.writeAll:'BBBB'.asBytes; (c.read:4).asString } }
    )
};
server.stop;
server.join;
server.close;
replies         "* -> #(AAAA BBBB)

Both clients are served concurrently by one TcpServer — the stop / join / close tail is the standard wind-down and reappears unchanged on [HTTP]Server in §27.

25. The [HTTP] client

Rules

The everyday surface — shown, not run, because these lines touch a real network:

use std:net/http;
var resp = [HTTP]Client.get:'https://example.org/';
resp.status;                      "* e.g. 200
resp.header:'content-type';      "* case-insensitive header lookup
resp.body.text;                   "* drain as a String — .json / .bytes likewise

"* POST a Map: it JSON-encodes itself, Content-Type included
[HTTP]Client.post:'https://api.example.org/users' body:#{ 'name':'Quoin' };

"* the builder covers custom verbs, headers, and redirect policy; .send fires
var deleted = (((([HTTP]Client.request:'https://api.example.org/users/7')
    .method:'DELETE')
    .header:'Authorization' value:'Bearer t0ken')
    .followRedirects:false)
    .send;

"* stream a large body chunk by chunk instead of draining it into memory
var out = [IO]File.create:'/tmp/big.bin';
([HTTP]Client.get:'https://example.org/big.bin').body.each:{ |chunk|
    out.writeAll:(chunk.bytes)
};
out.close

The client is pure Quoin over TcpSocket / TlsSocket — only the head parser is native — so it happily talks to any server, including a four-line one faked out of §23's parts:

use std:net/http;
var listener = TcpListener.listen:'127.0.0.1:0';
var wire = 'HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: 11\r\n\r\n{"ok":true}';
var served = Task.spawn:{ listener.acceptOnce:{ |conn| conn.read:1024; conn.writeAll:wire.asBytes } };
var resp = Async.timeout:5000 do:{ [HTTP]Client.get:('http://127.0.0.1:' + listener.port.s + '/') };
served.join;
listener.close;
#( resp.status (resp.body.json.at:'ok') )       "* -> #(200 true)

⚠ Gotcha — an undrained body owns the connection. send returns as soon as the head is parsed; the socket stays open, owned by resp.body, until you drain it (.text / .json / .bytes), iterate it fully, or .close it. Always do one of those — a body left dangling holds its connection until GC reaps it.

26. WebSocket

Rules

use std:net/websocket;
var ws = WebSocket.connect:'wss://stream.example.org/live';
ws.sendText:(JSON.generate:#{ 'subscribe': 'trades' });
ws.eachMessage:{ |m| (JSON.parse:m).print };    "* until the server closes
ws.closeCode                                    "* why it ended

27. Serving HTTP: [HTTP]Server and [Web]App

Serving splits into two layers, mirroring the client's philosophy: [HTTP]Server (use std:net/http_server) is the HTTP/1.1 protocol machine — no routing, no policy — and the [Web] framework (use std:web/*) layers routing, middleware, and render conventions on top of a pure request→response pipeline.

The transport: [HTTP]Server

Rules

use std:net/http_server;
var server = [HTTP]Server.new:{
    var address = ':8080';
    var maxBodyBytes = 1048576;              "* non-default limits go here
    var handler = { |req|
        ([HTTP]ServerResponse.new:{ var status = 200 }).body:('hello ' + req.target)
    }
};
server.start        "* accept in the background; stop / join / close to wind down

Most programs never write that: the [Web] framework supplies the handler.

The framework: [Web]App

Rules

Routing and the render conventions, driven entirely in-process — the requests are plain objects, fabricated with new:{}:

use std:net/http_server;
use std:web/*;
var app = [Web]App.new;
app.get:'/hello/:name' do:{ |req| 'hi ' + (req.param:'name') }   "* String -> text/plain
app.get:'/files/*rest' do:{ .param:'rest' }                      "* the request is self here
app.get:'/data' do:{ #{ 'n':1 } }                                "* Map -> JSON
app.get:'/teapot' do:{ 418 }                                     "* Integer -> a bare status

var get = { |t| [HTTP]ServerRequest.new:{ var method = 'GET'; var target = t } };
(app.handle:(get.value:'/hello/qn')).body.asString     "* -> 'hi qn'
(app.handle:(get.value:'/files/a/b')).body.asString    "* -> 'a/b'
(app.handle:(get.value:'/data')).body.asString         "* -> '{"n":1}'
(app.handle:(get.value:'/teapot')).status              "* -> 418
(app.handle:(get.value:'/nope')).status                "* -> 404

Routes are first-class values implementing the ~ match protocol (Part IV), so they also compose with case: outside any app:

use std:net/http_server;
use std:web/*;
var route = [Web]Route.of:'/users/:id/files/*rest';
route ~ '/users/7/files/a/b'         "* -> true
route.bind:'/users/7/files/a/b'      "* -> #{'id': '7' 'rest': 'a/b'}

Middleware wraps the pipeline as an onion — request work on the way in, response work on the way out:

use std:net/http_server;
use std:web/*;
var log = #();
var app = [Web]App.new;
app.get:'/' do:{ log.add:'handler'; 'ok' }
app.use:{ |req next| log.add:'auth>'; var r = next.value:req; log.add:'<auth'; r }
app.use:{ |req next| log.add:'trace>'; var r = next.value:req; log.add:'<trace'; r }
app.handle:([HTTP]ServerRequest.new:{ var method = 'GET'; var target = '/' })
log             "* -> #(auth> trace> handler <trace <auth)

And HttpError short-circuits from any depth — thrown in a handler (or anything it calls), mapped by the dispatcher:

use std:net/http_server;
use std:web/*;
var app = [Web]App.new;
app.get:'/admin' do:{ |req|
    (req.header:'authorization').defined?.else:{ HttpError.throw:401 };
    'welcome'
}
app.get:'/signup' do:{ HttpError.throw:422 body:#{ 'error':'bad email' } }
var get = { |t| [HTTP]ServerRequest.new:{ var method = 'GET'; var target = t } };
(app.handle:(get.value:'/admin')).status               "* -> 401
(app.handle:(get.value:'/signup')).body.asString       "* -> '{"error":"bad email"}'

Streaming responses and the workers pool round out the surface — illustrative, since one runs forever and the other wants real cores:

"* server-sent events: a Generator return streams one chunked frame per yield
app.get:'/events' do:{
    Generator.from:{
        (1..5).each:{ |n|
            ^> ('data: tick ' + n + '\n\n');
            Async.sleep:1000
        }
    }
}

app.serve:':8080'                               "* single VM: blocks until stopped
app.serve:':8080' workers:4                     "* + a pool of 4 worker isolates
app.serve:':8080' workers:8 backing:'process'   "* child processes: real multicore

⚠ Gotcha — pool-mode handlers cannot capture main-VM mutable state. With workers:n, the transport VM keeps the sockets and ships each request as data to a worker isolate that re-runs your program's unit and executes handle: there. Isolates share nothing: a captured counter increments per-worker, not globally. Keep shared state in an external store — or serve single-VM, where handlers may close over anything. (The pure handle: core is exactly what makes requests shippable.)

28. End to end: a JSON service, tested in-process

The framework's testing story is handle: — build the app, then call it like a function. No listener, no ports, no concurrency; requests are constructed and responses inspected directly:

use std:net/http_server;
use std:web/*;

var users = #{ '7': #{ 'id':'7' 'name':'Ada' } };
var app = [Web]App.new;
app.get:'/users/:id' do:{ |req|
    var user = users.at:(req.param:'id');
    user.defined?.if:{ user } else:{ 404 }
}
app.post:'/users' do:{ |req|
    var u = req.json;
    users.at:(u.at:'id') put:u;
    [Web]Response.json:u status:201
}

var get = { |t| [HTTP]ServerRequest.new:{ var method = 'GET'; var target = t } };
(app.handle:(get.value:'/users/7')).body.asString      "* -> '{"id":"7","name":"Ada"}'
(app.handle:(get.value:'/users/9')).status             "* -> 404

var post = [HTTP]ServerRequest.new:{
    var method = 'POST';
    var target = '/users';
    var body = [HTTP]Body.of:('{"id":"8","name":"Grace"}'.asBytes) contentType:'application/json'
};
(app.handle:post).status                               "* -> 201
(app.handle:(get.value:'/users/8')).body.asString      "* -> '{"id":"8","name":"Grace"}'

Serving the same app over real sockets is one line — start: — and the round trip composes with everything from this chapter: the [HTTP]Client from §25, the ephemeral-port-and-timeout pattern from §22:

use std:net/*;
use std:web/*;
var app = [Web]App.new;
app.get:'/users/:id' do:{ |req| #{ 'id':(req.param:'id') } }

var server = app.start:'127.0.0.1:0';        "* bind + accept in the background
var base = 'http://127.0.0.1:' + server.port;
var got = Async.timeout:5000 do:{ [HTTP]Client.get:(base + '/users/7') };
server.stop;
server.join;
server.close;
#( got.status (got.body.json.at:'id') )      "* -> #(200 7)

This split — pure tests against handle:, a few loopback round-trips for the transport — is how the stdlib tests itself (qnlib/tests/47–49 are the pure half; 46 and the live half of 49 drive real sockets).

Next: Part VII — The gradual type system.