Configuration

On this page

Configuration is an essential aspect of any cloud-native application. Effect simplifies the process of managing configuration by offering a convenient interface for configuration providers.

The configuration front-end in Effect enables ecosystem libraries and applications to specify their configuration requirements in a declarative manner. It offloads the complex tasks to a ConfigProvider, which can be supplied by third-party libraries.

Effect comes bundled with a straightforward default ConfigProvider that retrieves configuration data from environment variables. This default provider can be used during development or as a starting point before transitioning to more advanced configuration providers.

To make our application configurable, we need to understand three essential elements:

  • Config Description: We describe the configuration data using an instance of Config<A>. If the configuration data is simple, such as a string, number, or boolean, we can use the built-in functions provided by the Config module. For more complex data types like HostPort, we can combine primitive configs to create a custom configuration description.

  • Config Frontend: We utilize the instance of Config<A> to load the configuration data described by the instance (a Config is, in itself, an effect). This process leverages the current ConfigProvider to retrieve the configuration.

  • Config Backend: The ConfigProvider serves as the underlying engine that manages the configuration loading process. Effect comes with a default config provider as part of its default services. This default provider reads the configuration data from environment variables. If we want to use a custom config provider, we can utilize the Layer.setConfigProvider layer to configure the Effect runtime accordingly.

Getting Started

Effect provides a set of primitives for the most common types like string, number, boolean, integer, etc.

Let's start with a simple example of how to read configuration from environment variables:

primitives.ts
ts
import { Effect, Config } from "effect"
 
const program = Effect.gen(function* (_) {
  const host = yield* _(Config.string("HOST"))
  const port = yield* _(Config.number("PORT"))
  console.log(`Application started: ${host}:${port}`)
})
 
Effect.runSync(program)
primitives.ts
ts
import { Effect, Config } from "effect"
 
const program = Effect.gen(function* (_) {
  const host = yield* _(Config.string("HOST"))
  const port = yield* _(Config.number("PORT"))
  console.log(`Application started: ${host}:${port}`)
})
 
Effect.runSync(program)

If we run this application we will get the following output:

bash
(Missing data at HOST: "Expected HOST to exist in the process context")
bash
(Missing data at HOST: "Expected HOST to exist in the process context")

This is because we have not provided any configuration. Let's try running it with the following environment variables:

Terminal
bash
HOST=localhost PORT=8080 ts-node primitives.ts
Terminal
bash
HOST=localhost PORT=8080 ts-node primitives.ts

Now we get the following output:

bash
Application started: localhost:8080
bash
Application started: localhost:8080

Primitives

Effect offers these basic types out of the box:

  • string: Constructs a config for a string value.
  • number: Constructs a config for a float value.
  • boolean: Constructs a config for a boolean value.
  • integer: Constructs a config for a integer value.
  • date: Constructs a config for a date value.
  • literal: Constructs a config for a literal (*) value.
  • logLevel: Constructs a config for a LogLevel value.
  • duration: Constructs a config for a duration value.
  • secret: Constructs a config for a secret value.

(*) string | number | boolean | null | bigint

Default Values

In some cases, you may encounter situations where an environment variable is not set, leading to a missing value in the configuration. To handle such scenarios, Effect provides the Config.withDefault function. This function allows you to specify a fallback or default value to use when an environment variable is not present.

Here's how you can use Config.withDefault to handle fallback values:

withDefault.ts
ts
import { Effect, Config } from "effect"
 
const program = Effect.gen(function* (_) {
  const host = yield* _(Config.string("HOST"))
  const port = yield* _(Config.number("PORT").pipe(Config.withDefault(8080)))
  console.log(`Application started: ${host}:${port}`)
})
 
Effect.runSync(program)
withDefault.ts
ts
import { Effect, Config } from "effect"
 
const program = Effect.gen(function* (_) {
  const host = yield* _(Config.string("HOST"))
  const port = yield* _(Config.number("PORT").pipe(Config.withDefault(8080)))
  console.log(`Application started: ${host}:${port}`)
})
 
Effect.runSync(program)

When running the program with the command:

Terminal
bash
HOST=localhost ts-node withDefault.ts
Terminal
bash
HOST=localhost ts-node withDefault.ts

you will see the following output:

bash
Application started: localhost:8080
bash
Application started: localhost:8080

Even though the PORT environment variable is not set, the fallback value of 8080 is used, ensuring that the program continues to run smoothly with a default value.

Constructors

Effect provides several built-in constructors. These are functions that take a Config as input and produce another Config.

  • array: Constructs a config for an array of values.
  • chunk: Constructs a config for a sequence of values.
  • option: Returns an optional version of this config, which will be None if the data is missing from configuration, and Some otherwise.
  • repeat: Returns a config that describes a sequence of values, each of which has the structure of this config.
  • hashSet: Constructs a config for a sequence of values.
  • hashMap: Constructs a config for a sequence of values.

In addition to the basic ones, there are three special constructors you might find useful:

  • succeed: Constructs a config which contains the specified value.
  • fail: Constructs a config that fails with the specified message.
  • all: Constructs a config from a tuple / struct / arguments of configs.

Example

array.ts
ts
import { Effect, Config } from "effect"
 
const program = Effect.gen(function* (_) {
  const config = yield* _(Config.array(Config.string(), "MY_ARRAY"))
  console.log(config)
})
 
Effect.runSync(program)
array.ts
ts
import { Effect, Config } from "effect"
 
const program = Effect.gen(function* (_) {
  const config = yield* _(Config.array(Config.string(), "MY_ARRAY"))
  console.log(config)
})
 
Effect.runSync(program)
Terminal
bash
MY_ARRAY=a,b,c ts-node array.ts
[ 'a', 'b', 'c' ]
Terminal
bash
MY_ARRAY=a,b,c ts-node array.ts
[ 'a', 'b', 'c' ]

Operators

Effect comes with a set of built-in operators to help you manipulate and handle configurations.

Transforming Operators

These operators allow you to transform a config into a new one:

  • validate: Returns a config that describes the same structure as this one, but which performs validation during loading.
  • map: Creates a new config with the same structure as the original but with values transformed using a given function.
  • mapAttempt: Similar to map, but if the function throws an error, it's caught and turned into a validation error.
  • mapOrFail: Like map, but allows for functions that might fail. If the function fails, it results in a validation error.

Fallback Operators

These operators help you set up fallbacks in case of errors or missing data:

  • orElse: Sets up a config that tries to use this config first. If there's an issue, it falls back to another specified config.
  • orElseIf: This one also tries to use the main config first but switches to a fallback config if there's an error that matches a specific condition.

Example

validate.ts
ts
import { Effect, Config } from "effect"
 
const program = Effect.gen(function* (_) {
  const config = yield* _(
    Config.string("NAME").pipe(
      Config.validate({
        message: "Expected a string at least 4 characters long",
        validation: (s) => s.length >= 4
      })
    )
  )
  console.log(config)
})
 
Effect.runSync(program)
validate.ts
ts
import { Effect, Config } from "effect"
 
const program = Effect.gen(function* (_) {
  const config = yield* _(
    Config.string("NAME").pipe(
      Config.validate({
        message: "Expected a string at least 4 characters long",
        validation: (s) => s.length >= 4
      })
    )
  )
  console.log(config)
})
 
Effect.runSync(program)
Terminal
bash
NAME=foo ts-node validate.ts
[(Invalid data at NAME: "Expected a string at least 4 characters long")]
Terminal
bash
NAME=foo ts-node validate.ts
[(Invalid data at NAME: "Expected a string at least 4 characters long")]

Custom Configurations

In addition to primitive types, we can also define configurations for custom types. To achieve this, we use primitive configs and combine them using Config operators (zip, orElse, map, etc.) and constructors (array, hashSet, etc.).

Let's consider the HostPort data type, which consists of two fields: host and port.

ts
class HostPort {
  constructor(
    readonly host: string,
    readonly port: number
  ) {}
}
ts
class HostPort {
  constructor(
    readonly host: string,
    readonly port: number
  ) {}
}

We can define a configuration for this data type by combining primitive configs for string and number:

HostPort.ts
ts
import { Config } from "effect"
 
export class HostPort {
  constructor(
    readonly host: string,
    readonly port: number
  ) {}
 
  get url() {
    return `${this.host}:${this.port}`
  }
}
 
const both = Config.all([Config.string("HOST"), Config.number("PORT")])
 
export const config = Config.map(
  both,
  ([host, port]) => new HostPort(host, port)
)
HostPort.ts
ts
import { Config } from "effect"
 
export class HostPort {
  constructor(
    readonly host: string,
    readonly port: number
  ) {}
 
  get url() {
    return `${this.host}:${this.port}`
  }
}
 
const both = Config.all([Config.string("HOST"), Config.number("PORT")])
 
export const config = Config.map(
  both,
  ([host, port]) => new HostPort(host, port)
)

In the above example, we use the Config.all(configs) operator to combine two primitive configs Config<string> and Config<number> into a Config<[string, number]>.

If we use this customized configuration in our application:

App.ts
ts
import { Effect } from "effect"
import * as HostPort from "./HostPort"
 
export const program = Effect.gen(function* (_) {
  const hostPort = yield* _(HostPort.config)
  console.log(`Application started: ${hostPort.url}`)
})
App.ts
ts
import { Effect } from "effect"
import * as HostPort from "./HostPort"
 
export const program = Effect.gen(function* (_) {
  const hostPort = yield* _(HostPort.config)
  console.log(`Application started: ${hostPort.url}`)
})

when you run the program using Effect.runSync(program), it will attempt to read the corresponding values from environment variables (HOST and PORT):

Terminal
bash
HOST=localhost PORT=8080 ts-node HostPort.ts
Application started: localhost:8080
Terminal
bash
HOST=localhost PORT=8080 ts-node HostPort.ts
Application started: localhost:8080

Top-level and Nested Configurations

So far, we have learned how to define configurations in a top-level manner, whether they are for primitive or custom types. However, we can also define nested configurations.

Let's assume we have a ServiceConfig data type that consists of two fields: hostPort and timeout.

ServiceConfig.ts
ts
import * as HostPort from "./HostPort"
import { Config } from "effect"
 
class ServiceConfig {
  constructor(
    readonly hostPort: HostPort.HostPort,
    readonly timeout: number
  ) {}
}
 
const config = Config.map(
  Config.all([HostPort.config, Config.number("TIMEOUT")]),
  ([hostPort, timeout]) => new ServiceConfig(hostPort, timeout)
)
ServiceConfig.ts
ts
import * as HostPort from "./HostPort"
import { Config } from "effect"
 
class ServiceConfig {
  constructor(
    readonly hostPort: HostPort.HostPort,
    readonly timeout: number
  ) {}
}
 
const config = Config.map(
  Config.all([HostPort.config, Config.number("TIMEOUT")]),
  ([hostPort, timeout]) => new ServiceConfig(hostPort, timeout)
)

If we use this customized config in our application, it tries to read corresponding values from environment variables: HOST, PORT, and TIMEOUT.

However, in many cases, we don't want to read all configurations from the top-level namespace. Instead, we may want to nest them under a common namespace. For example, we want to read both HOST and PORT from the HOSTPORT namespace, and TIMEOUT from the root namespace.

To achieve this, we can use the Config.nested combinator. It allows us to nest configs under a specific namespace. Here's how we can update our configuration:

ts
const config = Config.map(
  Config.all([
    Config.nested(HostPort.config, "HOSTPORT"),
    Config.number("TIMEOUT")
  ]),
  ([hostPort, timeout]) => new ServiceConfig(hostPort, timeout)
)
ts
const config = Config.map(
  Config.all([
    Config.nested(HostPort.config, "HOSTPORT"),
    Config.number("TIMEOUT")
  ]),
  ([hostPort, timeout]) => new ServiceConfig(hostPort, timeout)
)

Now, if we run our application, it will attempt to read the corresponding values from the environment variables: HOSTPORT_HOST, HOSTPORT_PORT, and TIMEOUT.

Testing Services

When testing services, there are scenarios where we need to provide specific configurations to them. In such cases, we should be able to mock the backend that reads the configuration data.

To accomplish this, we can use the ConfigProvider.fromMap constructor. This constructor takes a Map<string, string> that represents the configuration data, and it returns a config provider that reads the configuration from that map.

Once we have the mock config provider, we can use Layer.setConfigProvider function. This function allows us to override the default config provider and provide our own custom config provider. It returns a Layer that can be used to configure the Effect runtime for our test specs.

Here's an example of how we can mock a config provider for testing purposes:

mockConfigProvider.ts
ts
import { ConfigProvider, Layer, Effect } from "effect"
import * as App from "./App"
 
// Create a mock config provider using ConfigProvider.fromMap
const mockConfigProvider = ConfigProvider.fromMap(
  new Map([
    ["HOST", "localhost"],
    ["PORT", "8080"]
  ])
)
 
// Create a layer using Layer.setConfigProvider to override the default config provider
const layer = Layer.setConfigProvider(mockConfigProvider)
 
// Run the program using the provided layer
Effect.runSync(Effect.provide(App.program, layer))
// Output: Application started: localhost:8080
mockConfigProvider.ts
ts
import { ConfigProvider, Layer, Effect } from "effect"
import * as App from "./App"
 
// Create a mock config provider using ConfigProvider.fromMap
const mockConfigProvider = ConfigProvider.fromMap(
  new Map([
    ["HOST", "localhost"],
    ["PORT", "8080"]
  ])
)
 
// Create a layer using Layer.setConfigProvider to override the default config provider
const layer = Layer.setConfigProvider(mockConfigProvider)
 
// Run the program using the provided layer
Effect.runSync(Effect.provide(App.program, layer))
// Output: Application started: localhost:8080

By using this approach, we can easily mock the configuration data and test our services with different configurations in a controlled manner.

Secret

What sets Config.secret apart from Config.string is its handling of sensitive information. It parses the Config value and wraps it in a Secret, a data type designed for holding secrets.

When you use console.log on a secret, the actual value remains hidden, providing an added layer of security. The only way to access the value is by using Secret.value(secret).

Here's a simple example to illustrate:

ts
import {
  Effect,
  Config,
  ConfigProvider,
  Layer,
  Console,
  Secret
} from "effect"
 
const program = Config.secret("API_KEY").pipe(
  Effect.tap((secret) => Console.log(`console.log: ${secret}`)),
  Effect.tap((secret) => Console.log(`Secret.value: ${Secret.value(secret)}`))
)
 
Effect.runSync(
  program.pipe(
    Effect.provide(
      Layer.setConfigProvider(
        ConfigProvider.fromMap(new Map([["API_KEY", "my-api-key"]]))
      )
    )
  )
)
/*
Output:
console.log: Secret(<redacted>)
Secret.value: my-api-key
*/
ts
import {
  Effect,
  Config,
  ConfigProvider,
  Layer,
  Console,
  Secret
} from "effect"
 
const program = Config.secret("API_KEY").pipe(
  Effect.tap((secret) => Console.log(`console.log: ${secret}`)),
  Effect.tap((secret) => Console.log(`Secret.value: ${Secret.value(secret)}`))
)
 
Effect.runSync(
  program.pipe(
    Effect.provide(
      Layer.setConfigProvider(
        ConfigProvider.fromMap(new Map([["API_KEY", "my-api-key"]]))
      )
    )
  )
)
/*
Output:
console.log: Secret(<redacted>)
Secret.value: my-api-key
*/

In this example, you can see that when logging the secret using console.log, the actual value is replaced with <redacted>, ensuring that sensitive information is not exposed. The Secret.value function, on the other hand, provides a controlled way to retrieve the original secret value.