Configuration¶

BSPump uses a layered configuration system supporting configuration files, environment variables, and programmatic configuration.

Configuration Sources¶

BSPump loads configuration from multiple sources in order of precedence:

Command-line arguments: Highest priority
Environment variables: BSPUMP_* prefix
Configuration files: pipelines.conf, *.conf
Default values: Built into components

Configuration File Format¶

BSPump uses INI-style configuration files:

[section:name]
key=value
another_key=another value

# Comments start with #
; Or with semicolon

Section Naming¶

Sections follow naming patterns:

[connection:ConnectionId] - Connection configuration
[pipeline:PipelineName] - Pipeline-level configuration
[pipeline:PipelineName:ComponentId] - Component configuration
[lookup:LookupId] - Lookup configuration

Example:

[connection:KafkaConnection]
bootstrap_servers=kafka:9092

[pipeline:MyPipeline]
max_concurrent=10

[pipeline:MyPipeline:KafkaSource]
topic=input-events

[lookup:UserLookup]
path=/data/users.json

Environment Variable Substitution¶

Use ${VAR} syntax for environment variables:

[connection:PostgreSQLConnection]
host=${POSTGRES_HOST}
password=${POSTGRES_PASSWORD}

Default values:

[connection:PostgreSQLConnection]
host=${POSTGRES_HOST:-localhost}
port=${POSTGRES_PORT:-5432}

Loading Configuration¶

Specify configuration files at startup:

python app.py -c pipelines.conf -c secrets.conf

Or in code:

app = bspump.BSPumpApplication()
app.Config.read("pipelines.conf")

Accessing Configuration¶

Access configuration in components:

class MyProcessor(bspump.Processor):
    def __init__(self, app, pipeline, id=None, config=None):
        super().__init__(app, pipeline, id, config)

        # Access configuration
        self.threshold = self.Config.getint("threshold", 100)
        self.enabled = self.Config.getboolean("enabled", True)
        self.rate = self.Config.getfloat("rate", 1.0)

Configuration Methods¶

Config.get(key, default=None)¶: Get a string value.

Config.getint(key, default=0)¶: Get an integer value.

Config.getfloat(key, default=0.0)¶: Get a float value.

Config.getboolean(key, default=False)¶: Get a boolean value (true/false, yes/no, 1/0).