← Home Codex CLI Skills System Stripe Agents

FreeBSD Jails Networking Plan

Start with a plain native jail for Clawdie. Add VNET and Tailscale only if the simpler model stops being enough.

Phase 1 Plain native FreeBSD jail with outbound internet for PI agent execution.
Phase 2 Optional VNET, bridge, pf NAT, and Tailscale per jail.
Key correction vmm is not required for jails.

1. Recommendation

Use the simplest architecture that proves the runtime

For Clawdie, the first milestone is not mesh networking. It is one real thick control-plane jail that can run the worker, reach Telegram and provider APIs, and execute pi successfully.

Stage	What it proves	Needed now
Thick control-plane jail	`jexec`, mounts, outbound network, PI runtime, stable Warden base	Yes
Thin worker jail	Short-lived worker execution with lower storage overhead	Later
Thin VNET worker jail	Per-worker network stack and future tailnet exposure	Later
Linux VM via `bhyve`	Real browser/GUI workloads and full Linux desktop automation	Separate architecture

2. What You Do Not Need First

vmm is not needed for jails.
linuxulator is not needed for native FreeBSD jails.
Per-jail Tailscale is not needed just to let Clawdie call Telegram and LLM APIs.
A Linux jail or VM is not the first step for the PI-based FreeBSD-native runtime.

3. Phase 1: Minimal Clawdie Jail

Bastille configuration

Keep the stock /usr/local/etc/bastille/bastille.conf structure intact. This block shows the recommended Clawdie-specific values for a ZFS-backed plain-jail setup on this host.

#####################
## [ BastilleBSD ] ##
#####################

## Default paths
bastille_prefix="/usr/local/bastille"
bastille_backupsdir="${bastille_prefix}/backups"
bastille_cachedir="${bastille_prefix}/cache"
bastille_jailsdir="${bastille_prefix}/jails"
bastille_releasesdir="${bastille_prefix}/releases"
bastille_templatesdir="${bastille_prefix}/templates"
bastille_logsdir="/var/log/bastille"

## pf configuration path
bastille_pf_conf="/etc/pf.conf"

## Bastille commands directory
bastille_sharedir="/usr/local/share/bastille"

## Bootstrap archives
bastille_bootstrap_archives="base"

## Pkgbase package sets
bastille_pkgbase_packages="base-jail"

## Default timezone
bastille_tzdata="Europe/Ljubljana"

## Default jail resolv.conf
bastille_resolv_conf="/etc/resolv.conf"

## Bootstrap URLs
bastille_url_freebsd="http://ftp.freebsd.org/pub/FreeBSD/releases/"
bastille_url_hardenedbsd="https://installers.hardenedbsd.org/pub/"
bastille_url_midnightbsd="https://www.midnightbsd.org/ftp/MidnightBSD/releases/"

## ZFS options
bastille_zfs_enable="YES"
bastille_zfs_zpool="zroot"
bastille_zfs_prefix="clawdie-runtime"
bastille_zfs_options="-o compress=on -o atime=off"

## Export/Import options
bastille_compress_xz_options="-0 -v"
bastille_decompress_xz_options="-c -d -v"
bastille_compress_gz_options="-1 -v"
bastille_decompress_gz_options="-k -d -c -v"
bastille_compress_zst_options="-3 -v"
bastille_decompress_zst_options="-k -d -c -v"
bastille_export_options=""

## Networking
bastille_network_vnet_type="if_bridge"
bastille_network_loopback="clawdie0"
bastille_network_pf_ext_if="ext_if"
bastille_network_pf_table="clawdie_jails"

bastille_network_shared=""
bastille_network_gateway=""
bastille_network_gateway6=""

## Default Templates
bastille_template_base="default/base"
bastille_template_empty=""
bastille_template_thick="default/thick"
bastille_template_clone="default/clone"
bastille_template_thin="default/thin"
bastille_template_vnet="default/vnet"
bastille_template_vlan="default/vlan"

## Monitoring
bastille_monitor_cron_path="/usr/local/etc/cron.d/bastille-monitor"
bastille_monitor_cron="*/5 * * * * root /usr/local/bin/bastille monitor ALL >/dev/null 2>&1"
bastille_monitor_logfile="${bastille_logsdir}/monitor.log"
bastille_monitor_healthchecks=""

Keep the stock Bastille structure intact. Only the ZFS, timezone, and naming lines are intentionally customized for this host.

bastille_sharedir is required. Without it, Bastille cannot find bootstrap.sh.

Copy-paste-safe Bastille config update

If your terminal mangles spaces during copy/paste, use this exact heredoc block as root:

cp /usr/local/etc/bastille/bastille.conf.bak /usr/local/etc/bastille/bastille.conf
cat > /usr/local/etc/bastille/bastille.conf << 'EOF'
#####################
## [ BastilleBSD ] ##
#####################

## Default paths
bastille_prefix="/usr/local/bastille"
bastille_backupsdir="${bastille_prefix}/backups"
bastille_cachedir="${bastille_prefix}/cache"
bastille_jailsdir="${bastille_prefix}/jails"
bastille_releasesdir="${bastille_prefix}/releases"
bastille_templatesdir="${bastille_prefix}/templates"
bastille_logsdir="/var/log/bastille"

## pf configuration path
bastille_pf_conf="/etc/pf.conf"

## Bastille commands directory (assumed by bastille pkg)
bastille_sharedir="/usr/local/share/bastille"

## Bootstrap archives, which components of the OS to install.
## base  - The base OS, kernel + userland
## lib32 - Libraries for compatibility with 32 bit binaries
## ports - The FreeBSD ports (3rd party applications) tree
## src   - The source code to the kernel + userland
## test  - The FreeBSD test suite
## Whitespace-separated list:
## bastille_bootstrap_archives="base lib32 ports src test"
bastille_bootstrap_archives="base"

## Pkgbase package sets
## Any set with [-dbg] can be installed with debugging
## symbols by adding '-dbg' to the package set
## base[-dbg]          - Base system
## base-jail[-dbg]     - Base system for jails
## devel[-dbg]         - Development tools
## kernels[-dbg]       - Base system kernels
## lib32[-dbg]         - 32-bit compatability libraries
## minimal[-dbg]       - Basic multi-user system
## minimal-jail[-dbg]  - Basic multi-user jail system

## optional[-dbg]      - Optional base system software
## optional-jail[-dbg] - Optional base system software for jails
## src                 - System source code
## tests               - System test suite
## Whitespace-separated list:
## bastille_pkgbase_packages="base-jail lib32-dbg src"
bastille_pkgbase_packages="base-jail"

## Default timezone
bastille_tzdata="Europe/Ljubljana"

## Default jail resolv.conf
bastille_resolv_conf="/etc/resolv.conf"

## Bootstrap URLs
bastille_url_freebsd="http://ftp.freebsd.org/pub/FreeBSD/releases/"
bastille_url_hardenedbsd="https://installers.hardenedbsd.org/pub/"
bastille_url_midnightbsd="https://www.midnightbsd.org/ftp/MidnightBSD/releases/"

## ZFS options
bastille_zfs_enable="YES"
bastille_zfs_zpool="zroot"
bastille_zfs_prefix="clawdie-runtime"
bastille_zfs_options="-o compress=on -o atime=off"

## Export/Import options
bastille_compress_xz_options="-0 -v"
bastille_decompress_xz_options="-c -d -v"
bastille_compress_gz_options="-1 -v"
bastille_decompress_gz_options="-k -d -c -v"
bastille_compress_zst_options="-3 -v"
bastille_decompress_zst_options="-k -d -c -v"
bastille_export_options=""

## Networking
bastille_network_vnet_type="if_bridge"
bastille_network_loopback="clawdie0"
bastille_network_pf_ext_if="ext_if"
bastille_network_pf_table="clawdie_jails"

bastille_network_shared=""
bastille_network_gateway=""
bastille_network_gateway6=""

## Default Templates
bastille_template_base="default/base"
bastille_template_empty=""
bastille_template_thick="default/thick"
bastille_template_clone="default/clone"
bastille_template_thin="default/thin"
bastille_template_vnet="default/vnet"
bastille_template_vlan="default/vlan"

## Monitoring
bastille_monitor_cron_path="/usr/local/etc/cron.d/bastille-monitor"
bastille_monitor_cron="*/5 * * * * root /usr/local/bin/bastille monitor ALL >/dev/null 2>&1"
bastille_monitor_logfile="${bastille_logsdir}/monitor.log"
bastille_monitor_healthchecks=""
EOF

Goal

Prove the canonical thick Warden control-plane jail on warden0 can host the runtime cleanly.

# Install Bastille
pkg install bastille

# Bootstrap FreeBSD 15 jail base with pkgbase
bastille bootstrap -p 15.0-RELEASE

# Create one native FreeBSD thick control-plane jail
bastille create -T -B clawdie-cp 15.0-RELEASE 10.0.0.100/24 warden0

# Set canonical hostname and restart
bastille config clawdie-cp set host.hostname ai.clawdie.si
bastille restart clawdie-cp

# Enter the jail
bastille console clawdie-cp

# Verify outbound access
fetch -qo- https://api.telegram.org >/dev/null && echo ok

# Install runtime dependencies inside the jail as needed
pkg install -y node npm git python312 gmake
npm install -g @mariozechner/pi-coding-agent

ZFS snapshots and rollback

With Bastille on ZFS, jail datasets can be snapshotted before risky changes and rolled back if needed.

# Inspect Bastille datasets
zfs list | grep clawdie-runtime

# Snapshot before a risky change
zfs snapshot zroot/clawdie-runtime/jails/clawdie-cp@fresh

# Roll back if needed
zfs rollback zroot/clawdie-runtime/jails/clawdie-cp@fresh

Success criteria

You can enter the jail reliably.
The jail has outbound internet access.
pi runs inside the jail.
Clawdie can invoke the jailed worker once end to end.

Warden profile policy

Profile	Runtime	Provisioning	Networking	Use
`controlPlane`	`freebsd-jail`	`thick`	`vnet`	Main `clawdie-cp` jail on `warden0`
`worker`	`freebsd-jail`	`thin`	`shared`	Default short-lived worker
`networkedWorker`	`freebsd-jail`	`thin`	`vnet`	Future worker with its own network stack
`browserVm`	`linux-vm`	`image`	`vm-bridged`	Future Linux desktop/browser executor via `bhyve`

4. Phase 2: Optional VNET + Tailscale

Only add this if you actually need per-jail networking

The control-plane already uses the canonical Warden VNET shape. This phase is about extending that model to worker jails and optional per-jail Tailscale later.

# Example later-stage VNET jail
bastille create -V warden-worker 15.0-RELEASE 10.0.0.101 warden0

# Jail config additions when Tailscale lives inside the jail
vnet;
vnet.interface = e0b_warden_worker;
allow.tun;

When VNET/Tailscale is justified

You need each jail to be reachable as its own node.
You want stricter network isolation between jails.
You want services inside jails to participate directly in your tailnet.

5. Host Networking for VNET Later

Bridge and NAT are phase-2 concerns

# Create the Warden bridge
sysrc cloned_interfaces="bridge0"
ifconfig bridge0 name warden0
sysrc ifconfig_warden0="inet 10.0.0.1/24"
service netif cloneup

# Enable IPv4 forwarding
sysrc gateway_enable="YES"
sysctl net.inet.ip.forwarding=1

# Then add pf NAT rules for 10.0.0.0/24

This is a valid design. It is just more moving parts than you currently need to prove Clawdie.

6. Tailscale on FreeBSD

Reality check

Tailscale on FreeBSD runs in userspace mode. That is usually fine for SSH, HTTP, HTTPS, Telegram, and API traffic, but it is not the same as Linux kernel-mode networking.

Use case	FreeBSD userspace mode
SSH / HTTP / HTTPS	Good
Telegram / LLM API calls	Good
Heavy routing / high throughput	Less ideal
Non-TCP/UDP protocols	Not a good fit

7. Clawdie-Specific Order

# 1. Keep Clawdie working on host first
# 2. Create the thick VNET control-plane jail
# 3. Prove PI runs in that jail
# 4. Prove Clawdie can execute one jailed turn
# 5. Expand to workers on 10.0.0.101+ once control-plane is stable
# 6. Only then decide whether Tailscale belongs inside the jail

8. Longer-Term Architecture

Separate jails from VM use cases

If you later want an agent to control Chrome or Firefox in a full Linux desktop, that is a different execution tier. Use bhyve for that, not the plain jail worker path.

Executor	Best use
FreeBSD jail	PI agent, coding tasks, CLI automation, low-overhead worker isolation
Linux VM via `bhyve`	Browser automation, GUI apps, full Linux desktop workloads

9. Future Linux Browser VM

Do not model the browser desktop as a jail

If you later need real Chrome or Firefox with a Linux desktop session, treat that as a separate Warden runtime class.

Executor	Best use
FreeBSD jail	PI agent, coding, CLI automation, low-overhead isolated workers
`bhyve` Linux VM	Full Linux desktop, browser automation, GUI apps, VNC/RDP workflows

No extra bastille.conf settings are needed now for that future VM path. Bastille manages jails; the Linux desktop should be a separate image-backed VM lifecycle.