Skip to main content

pod

Pods for responding to tasks.

Classes​

Pod​

class Pod(    name: str,    datasources: Iterable[DatasourceContainerConfig],    username: str | None = None,    hub: BitfountHub | None = None,    message_service: MessageServiceConfig | None = None,    access_manager: BitfountAM | None = None,    pod_keys: AsymmetricEncryptionKeyPair[RSAPrivateKey, RSAPublicKey] | None = None,    approved_pods: list[str] | None = None,    differential_privacy: DPPodConfig | None = None,    pod_db: bool | PodDbConfig = False,    update_schema: bool = False,    register_datasets: bool = True,    secrets: APIKeys | RefreshableJWT | dict[SecretsUse, APIKeys | RefreshableJWT] | None = None,    ehr_config: EHRConfig | None = None,    config_reload_settings: ConfigReloadSettings | None = None,    orchestrator_url: str | None = None,    manage_compute_cluster: bool = True,):

Makes data and computation available remotely and responds to tasks.

The basic component of the Bitfount network is the Pod (Processor of Data). Pods are co-located with data, check users are authorized to do given operations on the data and then do any approved computation. Creating a Pod will register the pod with Bitfount Hub.

Example usage:
import bitfount as bf

datasource = bf.CSVSource("/path/to/data.csv")
pod = bf.Pod(
    name="really_cool_data",
    datasources=[
        bf.DatasourceContainerConfig(
            name="really_cool_data",
            datasource=datasource,
        )
    ],
)
pod.start()
tip

Once you start a Pod, you can just leave it running in the background. It will automatically respond to any tasks without any intervention required.

Arguments

  • name: Name of the pod. This will appear on Bitfount Hub and Bitfount AM. This is also used for the name of the table in a single-table BaseSource.
  • datasources: The list of datasources to be associated and registered with this pod. Each will have their own data config and schema (although not necessarily present at this point).
  • username: Username of the user who is registering the pod. Defaults to None.
  • hub: Bitfount Hub to register the pod with. Defaults to None.
  • message_service: Configuration for the message service. Defaults to None.
  • access_manager: Access manager to use for checking access. Defaults to None.
  • pod_keys: Keys for the pod. Defaults to None.
  • approved_pods: list of other pod identifiers this pod is happy to share a task with. Required if the protocol uses the SecureAggregator aggregator.
  • differential_privacy: Differential privacy configuration for the pod. Defaults to None.
  • pod_db: Whether the results should be stored in a database. Defaults to False. If argument is set to True, then a SQLite database will be created for the pod in order to enable results storage for protocols that return them. It also keeps track of the pod datapoints so any repeat task is ran only on new datapoints.
  • update_schema: Whether the schema needs to be re-generated even if provided. Defaults to False.
  • register_datasets: Whether to register the datasets and upload their schemas to the Hub during construction. Defaults to True (the live-pod path). Set to False for ephemeral, read-only task-runner pods (e.g. run_background_task / run_interactive_task) so they consume the schema generated by run_pod without overwriting it on the Hub.
  • secrets: Secrets for authenticating with Bitfount services. If not provided then an interactive flow will trigger for authentication.
  • manage_compute_cluster: Whether this pod owns its datasets and should therefore create and link a compute cluster for them on the Hub. Defaults to True, which is correct for pods instantiated with locally-provided datasources. It is ignored under API-key authentication, where the compute cluster is inferred from the key scope. Set to False for compute-node pods that fetch their datasets from an existing Hub cluster rather than owning them.

Attributes

  • datasources: The set of datasources associated with this pod.
  • name: Name of the pod.
  • pod_identifier: Identifier of the pod.
  • private_key: Private key of the pod.

Raises

  • PodRegistrationError: If the pod could not be registered for any reason.
  • DataSourceError: If the BaseSource for the provided datasource has not been initialised properly. This can be done by calling super().__init__(**kwargs) in the __init__ of the DataSource.

Variables​

  • base_datasources : dict[str, DatasourceContainer] - Dictionary of base datasources, delegated to DatasourceManager.

  • configured_datasource_names : set[str] - Every datasource name this pod's config currently carries.

    Wider than base_datasources, which holds only the servable ones: a datasource whose hub registration failed is dropped from there but is still in the config, and the operator has removed nothing. Anything that acts on a removal -- the configured_datasources snapshot, and cancelling in-flight indexing -- must read this, or a transient PodRegistrationError is indistinguishable from a config change.

  • datasource : DatasourceContainer | None - If there is only a single datasource, this is a shorthand for retrieving it.

    If there is more than one datasource (or no datasources) this will log a warning and return None.

  • ehr_secrets : RefreshableJWT | None - The pod's externally-supplied EHR JWT secrets, if any.

    SMARTBackendEHRConfig deliberately returns None: it carries its own auth on the config object, and its consumers read that instead.

    Returns: The EHR RefreshableJWT, or None when none is configured, the configured secrets are of another type, or the config is SMART Backend.

  • is_ehr_configured : bool - Check if EHR configuration is present.

    Returns: True if ehr_config is set, False otherwise.

  • is_ehr_connected : bool - Check if EHR is connected and accessible.

    This performs a multi-level check for EHRs requiring authentication:

  1. Verifies EHR secrets are configured (must be RefreshableJWT)
  2. Attempts to reach the FHIR endpoint (if ehr_config is available)
  3. Falls back to checking token expiry if FHIR check fails

For unauthenticated EHRs (such as FHIR Candle without SMART Launcher, or other internal/testing EHRs): the path for EHRs without credentials is handled separately from the main authenticated flow and the allow_no_ehr_secrets configuration must be True to enable this. If _secrets is None, or EHR secrets are missing or not RefreshableJWT, this method calls _check_fhir_endpoint_reachable_without_auth() (GET to the FHIR /metadata endpoint) and returns its result.

Returns: True if EHR connection is verified, False otherwise.

  • name : str - Pod name property.

  • pod_key_path : pathlib.Path - Absolute path to this pod's RSA private key file.

    Reflects the directory the keys were actually loaded from in __init__. Used by open_background_cache to wire the same key into the cache-encryption layer.

Methods​

plan_standalone_metadata_indexing​

def plan_standalone_metadata_indexing(    self,) ‑> MetadataKickoffPlan | None:

Decide the file metadata indexing order for a pod with no orchestrator.

For run_pod, which indexes alongside start rather than after it, so runs its own health check instead of reading the cached one. The decision is reported like the orchestrator path's.

Blocking (health checks and Hub calls). Never raises.

Returns The plan, or None if it could not be built.

start​

def start(self) ‑> None:

Starts a pod instance, listening for tasks.

The blocking entry point; see start_async for what the pod does once it is up, and for how the two kinds of work it accepts differ in whether they run concurrently.

start_async​

async def start_async(self) ‑> None:

Starts a pod instance, listening for tasks.

Runs continuously, serving two kinds of work that differ in how they are scheduled:

  • Federated worker tasks (v8 JOB_REQUESTs). Each spawns a worker. Nothing serialises them: the transport dispatches one task per message, so two overlapping requests do overlap. _is_busy tracks only this lane, and tracks it as a single flag, so it is a rough signal rather than an exact one.
  • Background and interactive DAG runs. Concurrent by design. A background run executes in a subprocess the pod's Prefect runner launches from that project's deployment; the runner is stopped and its runs' rows closed on shutdown.