The data migration environment
What a fit-for-purpose data migration environment consists of, and why it is separate from both the source and the target.
Data remediation and migration at scale is performed in a dedicated environment that sits between the source systems and the target Salesforce org: a database server in which source data is replicated, transformed and validated, from which it is loaded into the org's sandboxes and then production, and in which every result is stored. The environment is temporary, provisioned by the client under its own security controls, and typically withdrawn at the end of the stabilization period.
What the data migration environment consists of
| Element | Specification | Reason |
|---|---|---|
| Compute | Processor, memory and storage sized to the data volume, attachment load and number of concurrent developers; separate operating-system and data volumes on SSD; disk encryption; endpoint protection. | Source replicas, working sets and attachments occupy substantial space, and concurrent developer sessions compete for processing. |
| Database | SQL Server; edition and hosting are confirmed against the data volume and the selected ETL tooling. | Staged tables, transformation scripts and load results are held in one database for validation and audit. Some ETL tooling requires the database engine and the tooling on the same server. |
| ETL tooling | ETL tooling between the migration database and each source system, whether Salesforce or another platform, and the target Salesforce org (e.g., DBAmp, SQL-Sales, Salesforce Data Loader, Python, etc.). | Stages source data, loads it to the target Salesforce org, and stages the results of every load back for validation. |
| Other software | Database management tooling; a desktop Excel installation with shared-computer activation. | Validation reports and legacy Excel sources are produced and read on the server. |
| Source and target access | Named logins for the data leads in every Salesforce source and target org, through the client's single sign-on where used; an API-enabled integration user in each Salesforce org, and in other source systems where data is pulled directly, restricted by IP range or permissions because the ETL tooling cannot complete multi-factor authentication. | Data leads inspect the data they work with and, where needed, the configuration behind it. |
Hosting and security review
The server may be hosted on premises or as a cloud virtual machine, sized to the data volume and the number of developers working concurrently. Where a security review is required before ETL tooling is approved, our consultants supply the initial documentation.
The migration database and its tooling
A source-controlled repository holds the foundation: proven patterns, including ones for uncommon scenarios; the details are tailored to each client. The tooling makes the essential steps routine: replicating and loading objects, mapping fields, checking load tables against Salesforce before any load, and storing the results of every load, from which the validation reports are produced.
Assess the data before the agents depend upon it
The first call is with the practitioners who would do the work, not a sales intermediary. The measures come first and the estimate follows.