sf-industry-commoncore-datamapper

$2c

Key Insights

Insight

Details

Extract vs Turbo Extract

Extract uses standard SOQL with relationship queries. Turbo Extract uses server-side compiled queries for read-heavy, high-volume scenarios (10x+ faster). Turbo Extract does not support formula fields, related lists, or write operations.

Transform is in-memory

Transform Data Mappers operate entirely in memory with no DML or SOQL. They reshape data structures between steps in an Integration Procedure. Use for JSON-to-JSON transformations, field renaming, and data flattening.

Load = DML

Load Data Mappers perform insert, update, upsert, or delete operations. They require proper FLS checks and error handling. Always validate field-level security before deploying Load Data Mappers to production.

OmniDataTransform metadata

Data Mappers are stored as OmniDataTransform and OmniDataTransformItem records. Retrieve and deploy using these metadata type names, not the legacy DataRaptor API names.

Workflow (5-Phase Pattern)

Phase 1: Requirements Gathering

Ask the user to gather:

• Data Mapper type (Extract, Transform, Load, Turbo Extract)

• Target Salesforce object(s) and fields

• Target org alias

• Consuming component (Integration Procedure, OmniScript, or FlexCard name)

• Data volume expectations (record counts, frequency)

Then:

• Check existing Data Mappers: Glob: **/OmniDataTransform*

• Check existing OmniStudio metadata: Glob: **/omnistudio/**

• Create a task list

Phase 2: Design & Type Selection

Type

Use Case

Naming Prefix

Supports DML

Supports SOQL

Extract

Read data from one or more objects with relationship queries

DR_Extract_

No

Yes

Turbo Extract

High-volume read-only queries, server-side compiled

DR_TurboExtract_

No

Yes (compiled)

Transform

In-memory data reshaping between procedure steps

DR_Transform_

No

No

Load

Write data (insert, update, upsert, delete)

DR_Load_

Yes

No

Naming Format: [Prefix][Object]_[Purpose] using PascalCase

Examples:

• DR_Extract_Account_Details -- Extract Account with related Contacts

• DR_TurboExtract_Case_List -- High-volume Case list for FlexCard

• DR_Transform_Lead_Flatten -- Flatten nested Lead data structure

• DR_Load_Opportunity_Create -- Insert Opportunity records

Phase 3: Generation & Validation

For Generation:

• Define the OmniDataTransform record (Name, Type, Active status)

• Define OmniDataTransformItem records (field mappings, input/output paths)

• Configure query filters, sort order, and limits for Extract types

• Set up lookup mappings and default values for Load types

• Validate field-level security for all mapped fields

For Review:

• Read existing Data Mapper configuration

• Run validation against best practices

• Generate improvement report with specific fixes

Run Validation:

Score: XX/100 Rating

|- Design & Naming: XX/20

|- Field Mapping: XX/25

|- Data Integrity: XX/25

|- Performance: XX/15

|- Documentation: XX/15

Generation Guardrails (MANDATORY)

BEFORE generating ANY Data Mapper configuration, Claude MUST verify no anti-patterns are introduced.

If ANY of these patterns would be generated, STOP and ask the user:

"I noticed [pattern]. This will cause [problem]. Should I:

A) Refactor to use [correct pattern]

B) Proceed anyway (not recommended)"

Anti-Pattern

Detection

Impact

Extracting all fields

No field list specified, wildcard selection

Performance degradation, excessive data transfer

Missing lookup mappings

Load references lookup field without resolution

DML failure, null foreign key

Writing without FLS check

Load Data Mapper with no security validation

Security violation, data corruption in restricted profiles

Unbounded Extract query

No LIMIT or filter on Extract

Governor limit failure, timeout on large objects

Transform with side effects

Transform attempting DML or callout

Runtime error, Transform is in-memory only

Hardcoded record IDs

15/18-char ID literal in filter or mapping

Deployment failure across environments

Nested relationship depth >3

Extract with deeply nested parent traversal

Query performance degradation, SOQL complexity limits

Load without error handling

No upsert key or duplicate rule consideration

Silent data corruption, duplicate records

DO NOT generate anti-patterns even if explicitly requested. Ask user to confirm the exception with documented justification.

See: references/best-practices.md for detailed patterns

See: references/naming-conventions.md for naming rules

Phase 4: Deployment

Step 1: Validation

Use the sf-deploy skill: "Deploy OmniDataTransform [Name] to [target-org] with --dry-run"

Step 2: Deploy (only if validation succeeds)

Use the sf-deploy skill: "Proceed with actual deployment to [target-org]"

Post-Deploy: Activate the Data Mapper in the target org. Verify it appears in OmniStudio Designer.

Phase 5: Testing & Documentation

Completion Summary:

Data Mapper Complete: [Name]

  Type: [Extract|Transform|Load|Turbo Extract]

  Target Object(s): [Object1, Object2]

  Field Count: [N mapped fields]

  Validation: PASSED (Score: XX/100)

Next Steps: Test in Integration Procedure, verify data output, monitor performance

Testing Checklist:

• Preview data output in OmniStudio Designer

• Verify field mappings produce expected JSON structure

• Test with representative data volume (not just 1 record)

• Validate FLS enforcement with restricted profile user

• Confirm consuming Integration Procedure/OmniScript receives correct data shape

Best Practices (100-Point Scoring)

Category

Points

Key Rules

Design & Naming

20

Correct type selection; naming follows DR_[Type]_[Object]_[Purpose] convention; single responsibility per Data Mapper

Field Mapping

25

Explicit field list (no wildcards); correct input/output paths; proper type conversions; null-safe default values

Data Integrity

25

FLS validation on all fields; lookup resolution for Load types; upsert keys defined; duplicate handling configured

Performance

15

Bounded queries with LIMIT/filters; Turbo Extract for read-heavy scenarios; minimal relationship depth; indexed filter fields

Documentation

15

Description on OmniDataTransform record; field mapping rationale documented; consuming components identified

Thresholds: ✅ 90+ (Deploy) | ⚠️ 67-89 (Review) | ❌ <67 (Block - fix required)

CLI Commands

Query Existing Data Mappers

sf data query -q "SELECT Id,Name,Type FROM OmniDataTransform" -o <org>

Query Data Mapper Field Mappings

sf data query -q "SELECT Id,Name,InputObjectName,OutputObjectName,LookupObjectName FROM OmniDataTransformItem WHERE OmniDataTransformationId='<id>'" -o <org>

Retrieve Data Mapper Metadata

sf project retrieve start -m OmniDataTransform:<Name> -o <org>

Deploy Data Mapper Metadata

sf project deploy start -m OmniDataTransform:<Name> -o <org>

Cross-Skill Integration

From Skill

To sf-industry-commoncore-datamapper

When

sf-industry-commoncore-omnistudio-analyze

-> sf-industry-commoncore-datamapper

"Analyze dependencies before creating Data Mapper"

sf-metadata

-> sf-industry-commoncore-datamapper

"Describe target object fields before mapping"

sf-soql

-> sf-industry-commoncore-datamapper

"Validate Extract query logic"

From sf-industry-commoncore-datamapper

To Skill

When

sf-industry-commoncore-datamapper

-> sf-industry-commoncore-integration-procedure

"Create Integration Procedure that calls this Data Mapper"

sf-industry-commoncore-datamapper

-> sf-deploy

"Deploy Data Mapper to target org"

sf-industry-commoncore-datamapper

-> sf-industry-commoncore-omniscript

"Wire Data Mapper output into OmniScript"

sf-industry-commoncore-datamapper

-> sf-industry-commoncore-flexcard

"Display Data Mapper Extract results in FlexCard"

Edge Cases

Scenario

Solution

Large data volume (>10K records)

Use Turbo Extract; add pagination via Integration Procedure; warn about heap limits

Polymorphic lookup fields

Specify the concrete object type in the mapping; test each type separately

Formula fields in Extract

Standard Extract supports formula fields; Turbo Extract does not -- fall back to standard Extract

Cross-object Load (master-detail)

Insert parent records first, then child records in a separate Load step; use Integration Procedure to orchestrate sequence

Namespace-prefixed fields

Include namespace prefix in field paths (e.g., ns__Field__c); verify prefix matches target org

Multi-currency orgs

Map CurrencyIsoCode explicitly; do not rely on default currency assumption

RecordType-dependent mappings

Filter by RecordType in Extract; set RecordTypeId in Load; document which RecordTypes are supported

Notes

• Metadata Type: OmniDataTransform (not DataRaptor -- legacy name deprecated)

• API Version: Requires OmniStudio managed package or Industries Cloud

• Scoring: Block deployment if score < 67

• Dependencies (optional): sf-deploy, sf-metadata, sf-industry-commoncore-omnistudio-analyze, sf-industry-commoncore-integration-procedure

• Turbo Extract Limitations: No formula fields, no related lists, no aggregate queries, no polymorphic fields

• Activation: Data Mappers must be activated after deployment to be callable from Integration Procedures

• Draft DMs can't be retrieved: sf project retrieve start -m OmniDataTransform:<Name> only works for active Data Mappers. Draft DMs return "Entity cannot be found".

• Creating via Data API: Use sf api request rest --method POST --body @file.json to create OmniDataTransform and OmniDataTransformItem records. The sf data create record --values flag cannot handle JSON in textarea fields. Write the JSON body to a temp file first.

• Foreign key field name: The parent lookup on OmniDataTransformItem is OmniDataTransformationId (full word "Transformation"), not OmniDataTransformId.

License

MIT License.

Copyright (c) 2026 David Ryan (weytani)