# Health Monitoring for Private Locations

See [Environment Variables](<https://docs.escape.tech/documentation/platform/private-location/environment-variables/index.md>) for the complete Private Location configuration reference.

## Verbosity Control

The Private Location supports configurable log verbosity levels through the `ESCAPE_VERBOSITY` environment variable. This setting controls the amount of diagnostic information output by the Private Location service.

The effective level is the higher of `ESCAPE_VERBOSITY` and the number of `-v` flags. The Helm chart includes `-v`, so it runs at debug level even when `ESCAPE_VERBOSITY=0`.

### Available Verbosity Levels

| Level | Description |
| --- | --- |
| `0` | Default level: minimal output with essential information only |
| `1` | Debug level: detailed diagnostic information for troubleshooting |
| `2` | Trace level: internal operations and proxy connection diagnostics |
| `3` | Trace plus raw HTTP request and response output for Escape API calls |

### Configuration Example

The verbosity level can be set using the `ESCAPE_VERBOSITY` environment variable:

**Docker Compose:**

```yaml
services:
  private-location:
    image: escapetech/cli:latest
    restart: always
    command: locations start -v location-name
    environment:
      - ESCAPE_API_KEY=<ESCAPE_API_KEY>
      - ESCAPE_VERBOSITY=1
```

**Helm:**

```yaml
container:
  env:
    - name: ESCAPE_VERBOSITY
      value: "1"
```

> [!TIP]
>
> **Debugging Connection Issues**
>
> Set `ESCAPE_VERBOSITY=1` or `ESCAPE_VERBOSITY=2` for connection diagnostics. To return to level `0`, set `ESCAPE_VERBOSITY=0` and remove `-v` from the startup command. The Helm chart hardcodes `-v`: level `0` requires a customized chart or Deployment. Level `3` logs Escape API traffic, not tunneled target HTTP traffic.

## How Does Escape Determine if a Private Location Is Alive and Operational?

Escape monitors Private Location health through a multi-layered approach:

- **Regular heartbeats**: The Private Location sends periodic health checks to the Escape platform to signal operational status. Heartbeats update **Last seen** separately from location logs.
- **Connection status tracking**: The platform records connect and disconnect on the location logs page when the SSH tunnel opens or closes.
- **Usage counts**: Info-level usage logs report proxy connection and DNS query counts. They don't measure response times or success rates.

Open a location on the [Private Locations dashboard](<https://app.escape.tech/organization/private-locations/>) to see last-seen time, connect/disconnect events, and forwarded Info-level and higher-severity logs. Debug and trace stay on the agent so scan traffic doesn't flood that page.

DNS queries for internal targets use the resolver available to the Private Location host or container. Check that it can resolve your internal zones. See [DNS Resolution](<https://docs.escape.tech/documentation/platform/private-location/prerequisites/#dns-resolution>) for connectivity requirements.

## How Do Restarts Work?

The agent retries an SSH connection when it disconnects. It exits if startup registration fails or a later registration attempt rejects its API key. Docker Compose's `restart: always` only restarts a container after its process exits, not when `/health` returns `503`.

The Helm chart configures a scheduled `24h` restart with `ESCAPE_CLI_RESTART_INTERVAL`. It also checks `/health` every 60 seconds and restarts the pod after 60 failed checks. The chart doesn't expose liveness probe timing as a Helm value.

For Docker Compose or a custom deployment, set `HEALTH_CHECK_PORT`, monitor `/health`, and restart the container when it reports `503`. See [Deployment Methods](<https://docs.escape.tech/documentation/platform/private-location/deployment/#health-check>).

# Advanced Request Logging and Monitoring for Private Locations

To read HTTP headers in tunneled HTTPS traffic, configure a proxy to intercept TLS. The CLI forwards the encrypted traffic, and the proxy provides HTTP inspection. Below is an example using [mitmproxy](<https://mitmproxy.org/>)

## Example: Using `mitmproxy` to Extract the `X-Escape-Request-Id` Header

> [!WARNING]
>
> Set `ESCAPE_ENABLE_LOGS_ENDPOINT=true` and `HEALTH_CHECK_PORT` to enable the `/log` endpoint. For HTTPS targets, the scanner must accept mitmproxy's interception certificate. The agent's TLS environment variables don't configure target trust. See [mTLS Authentication](<https://docs.escape.tech/documentation/platform/private-location/mtls/index.md>).

The addon posts the request ID to the agent's `/log` endpoint. The agent writes it to its own debug output (`ESCAPE_VERBOSITY=1` or `-v`); it isn't forwarded to the Escape platform. Create a file called `./mitmproxy/extract_escape_request_id.py` with the following content:

```python
import requests
import os


class Addon:
    def __init__(self):
        port = os.getenv("HEALTH_CHECK_PORT", "8080")
        self.log_url = f"http://private-location:{port}/log"

    def request(self, flow):
        request_id = flow.request.headers.get("X-Escape-Request-Id", "")
        if request_id:
            requests.post(self.log_url, data=f'Forwarding X-Escape-Request-Id: {request_id}')

addons = [Addon()]
```

> [!NOTE]
>
> See the [mitmproxy addons documentation](<https://docs.mitmproxy.org/stable/addons-overview/>) for more information.

Then to configure a Private Location to use this addon, you can use the following docker-compose file:

```yaml
services:
  private-location:
    image: escapetech/cli:latest
    restart: always
    command: locations start -v location-name
    environment:
      - ESCAPE_API_KEY=<ESCAPE_API_KEY>
      - HEALTH_CHECK_PORT=8080
      - ESCAPE_BACKEND_PROXY_URL=http://mitm-proxy:8080
      - ESCAPE_ENABLE_LOGS_ENDPOINT=true
  mitm-proxy:
    image: mitmproxy/mitmproxy:latest
    restart: always
    ports:
      - "8080:8080"
    command: "mitmdump -s /mitmproxy/extract_escape_request_id.py"
    volumes:
      - ./mitmproxy:/mitmproxy
```