Service: Logflare - EyevinnOSC/community GitHub Wiki
Logflare (serviceId logflare-logflare) is a log ingestion and querying engine that stores and queries log events in a columnar database. In Eyevinn Open Source Cloud (OSC) it is a long-running service instance that uses a PostgreSQL database as its backend. License: Apache-2.0.
- Ingest log events from applications into named sources.
- Query and browse log events in the Logflare web UI.
- Receive logs from Node.js applications with the
pinoandpino-logflarelibraries (example below).
- Create a PostgreSQL instance (
serviceId: "birme-osc-postgresql") withcreate-service-instance. It must be initialized with the SQLALTER SYSTEM SET wal_level = 'logical';, set through thePostgresInitDbSqloption when the instance is created, because it cannot be configured once the database is running. Wait until it is running and read its IP and port withdescribe-service-instance. The URI ispostgres://postgres:<password>@<IP>:<PORT>/postgres. - Store the URI with
create-service-secret(serviceIdlogflare-logflare), for example namedburl. - Create the instance with
create-service-instance,serviceId: "logflare-logflare":
{ "name": "mylogs", "PostgresBackendUrl": "{{secrets.dburl}}", "SecretKeyBase": "{{secrets.logflaresecretbase}}" }SecretKeyBase is required; the platform can generate a value for it. Wait for it to be running (wait-for-service-ready or get-instance-status) and get the URL with describe-service-instance.
| Option | Required | Type | Meaning |
|---|---|---|---|
| name | Yes | string | Name of the instance. Letters, digits and underscore only (^\w+$), at most 20 characters. |
| PostgresBackendUrl | Yes | string, sensitive | PostgreSQL connection URI of the backend database, for example postgres://postgres:<password>@<IP>:<PORT>/postgres. The database user needs sufficient permissions to manage the schema. Corresponds to upstream POSTGRES_BACKEND_URL. |
| SecretKeyBase | Yes | string, sensitive | Secret key base of the Logflare application. The platform can generate it. |
| DbSchema | No | string | Database schema that scopes Logflare operations. The public schema is used when unset. Corresponds to upstream DB_SCHEMA. |
| DbEncryptionKey | No | string, sensitive | Base64 encryption key used to encrypt sensitive data in the database. Corresponds to upstream LOGFLARE_DB_ENCRYPTION_KEY; upstream uses a default hardcoded key when none is set, so set your own. |
| ApiKey | No | string, sensitive | Deprecated upstream alias (LOGFLARE_API_KEY) for the public access token. |
| PublicAccessToken | No | string | If set, creates a public access token for the provisioned user, for ingestion or querying (LOGFLARE_PUBLIC_ACCESS_TOKEN, single-tenant mode). |
| PrivateAccessToken | No | string, sensitive | If set, creates a private access token for the provisioned user, for management API use (LOGFLARE_PRIVATE_ACCESS_TOKEN, single-tenant mode). |
Store sensitive options as service secrets and pass them as {{secrets.NAME}} (whole field value only). Overwriting a secret does not restart a running instance.
Open the instance URL and create a source with "New source" (for example myapp). Note the Source ID and the generated API key; both are shown once when the source is created. The API key is not recoverable afterwards, and anyone holding it can write log events into your source, so treat it as a secret.
Example Node.js application that sends logs:
mkdir myapp && cd myapp
npm init
npm install --save pino pino-logflaremyapp.js, where apiKey is the source API key, sourceToken the Source ID and apiBaseUrl your instance URL:
import pino from "pino"
import { createWriteStream } from "pino-logflare"
const stream = createWriteStream({
apiKey: "<your-api-key>",
apiBaseUrl: "https://<your-instance>.logflare-logflare.auto.prod-se.osaas.io",
sourceToken: "<your-source-id>",
});
const logger = pino({}, stream)
logger.info("Informational message")
logger.error(new Error("things got bad"), "error message")
const child = logger.child({ property: "value" })
child.info("hello child!")Run it with node myapp.js and the events appear in the source in the Logflare UI.
- The PostgreSQL
wal_levelmust be set tologicalat database creation time; it cannot be changed on a running instance.