MoveData uses a metadata-driven pipeline framework to transform notifications into Salesforce records. Each pipeline defines the sequence of operations required to process a specific type of data, such as donations or commerce transactions.

Pipelines ensure that records are created in the correct order, business logic is applied consistently, and every processing step is logged for troubleshooting.

Why pipelines matter#

Different data types require different processing sequences. A donation notification needs accounts, contacts, campaigns, recurring donations, and opportunities created in that specific order. A commerce notification needs accounts, contacts, campaigns, products, orders, and order items created in that specific order.

Pipelines provide this structure. They guarantee that dependent records exist before child records are created, that duplicate checking occurs at every stage, and that your organisation's business rules are applied automatically.

How pipeline registration works#

When a notification arrives in Salesforce, MoveData needs to determine which pipeline should process it. This routing is controlled by the movedata__Movedata_Schema_Map__mdt custom metadata type.

Schema mapping process#

Each MoveData extension registers its processing capabilities by creating entries in the schema map metadata. These entries define two things:

• Schema type — the standardised notification schema, such as "donation" or "commerce".
• Pipeline class — the Apex class responsible for processing notifications of that schema type.

Pipeline discovery#

When a notification arrives, MoveData follows four steps to find and start the correct pipeline:

1. Schema identification — the notification's schema type is extracted from the incoming data.
2. Metadata lookup — MoveData queries movedata__Movedata_Schema_Map__mdt to find the active pipeline for this schema.
3. Pipeline selection — the appropriate pipeline class is identified and instantiated.
4. Processing initiation — the pipeline begins processing the notification through its defined phases.

Pipeline architecture#

Every processing pipeline follows a consistent, multi-phase architecture.

Phase-based processing#

Pipelines are organised into logical phases, where each phase handles a different aspect of the notification.

For example, the donation pipeline has five phases:

1. Account — creates or matches the organisation (if present).
2. Contact — creates or matches the supporter(s).
3. Campaign — creates or matches the campaign(s).
4. Recurring — creates or updates the recurring donation (if applicable).
5. Donation — creates the donation record.

The commerce pipeline uses a different set of phases suited to transactional data, including product catalogue and order item phases.

Phase orchestration#

The pipeline engine coordinates phase execution using four mechanisms:

• Sequential processing — phases execute in a defined order to maintain data dependencies. An account must exist before a contact can be linked to it. A campaign must exist before a donation can reference it.
• Conditional execution — phases only execute if relevant data is present in the notification. If a donation notification contains no account information, the account phase is skipped.
• Error handling — phases can halt processing based on configuration, preventing downstream errors when earlier phases fail.
• Audit logging — every phase execution is logged to the execution log for troubleshooting and compliance.

Pipeline configuration#

Individual pipeline phases are configured through the movedata__MoveData_Pipeline__mdt custom metadata type. This metadata controls how each phase behaves without requiring code changes.

Phase control settings#

Each phase supports four categories of configuration:

Extension-provided pipelines#

MoveData extensions provide pre-built pipelines optimised for their respective schemas:

• Fundraising and Donations — processes donation notifications through account, contact, campaign, recurring, and donation phases. See Donation pipeline.
• Commerce — processes commerce notifications through account, contact, campaign, product, order, and order item phases. See Commerce pipeline.

Example: notification processing flow#

The following walkthrough illustrates how a donation notification moves through the pipeline, using the NPSP Fundraising and Donations extension as an example.

Step-by-step processing#

1. Notification dispatch A notification is dispatched to the MoveData engine hosted within Salesforce, triggered by an event from an integrated fundraising platform.

2. Schema identification The engine interrogates the notification to identify the schema type. In this example, the notification uses the donation schema.

3. Pipeline registration lookup A lookup into movedata__Movedata_Schema_Map__mdt identifies that the donation schema has a pipeline registered by the NPSP Fundraising and Donations extension.

4. Phase initialisation The pipeline loads and begins processing the notification through its defined phases: Account, Contact, Campaign, Recurring, and Donation. In this example, the notification contains no account data but includes the contact who made the donation. The pipeline triggers the contact phase.

5. Enable/disable check The pipeline checks movedata__MoveData_Pipeline__mdt to see if the contact phase is disabled. It is not, so processing continues.

6. Determine SObject type The pipeline checks the metadata to see if the SObject for the contact phase is overridden. This can be set directly in the metadata or determined dynamically via a flow. In this example, there is no override, so the pipeline defaults to the Contact SObject type.

7. Fieldset loading The pipeline reads the metadata to identify any fieldsets that need to be loaded. Fieldset references instruct the pipeline which fields to retrieve from Salesforce when matching existing records. This ensures all fields referenced in flows are preloaded to prevent execution failures.

8. Duplicate detection MoveData performs duplicate checking using Lightning Flows and Salesforce Duplicate Rules. It first attempts to match on existing platform keys. If no match is found, it executes the organisation's Salesforce Duplicate Rules.

9. Populate record Mapping rules execute to apply additions and changes to the contact record, regardless of whether an existing record was found. This stage contains the majority of business logic and field transformation rules.

10. Record persistence The processed record is returned from the mapping action and the pipeline performs an upsert operation to persist the contact data to Salesforce.

11. Post-processing Following the upsert, additional actions handle post-processing requirements such as linking the contact record to other objects and creating related child records.

12. Phase progression The pipeline advances to the next phase once all contact entries in the notification have been processed successfully. In this scenario, the pipeline proceeds to address campaign information.

13. Completion and logging Once all phases have been processed, MoveData persists the execution logs and results. The notification is marked as successful, providing a comprehensive audit trail.

Metadata reference#

MoveData's pipeline framework relies on two Salesforce custom metadata types.

The keys for pipeline metadata entries are documented in the Donation pipeline and Commerce pipeline articles, which list every metadata key and its purpose for each phase.

Other resources#

• Donation pipeline — detailed reference for the donation processing pipeline.
• Commerce pipeline — detailed reference for the commerce processing pipeline.
• Architecture overview — how pipelines fit into MoveData's overall architecture.
• Salesforce architecture — the managed package structure that hosts pipelines.
• Understanding extensions — how extensions provide pipeline implementations.