Seeding production with People from a CSV file - RodrigoMattosoSilveira/EnterpriseRemoteSystems GitHub Wiki
ERS provides a controlled workflow for moving a Tenant's People from Test to Production.
The supported workflow is:
Test Tenant
β People β Export CSV
β securely handle and verify the exported CSV
β identify the Production Tenant
β back up Production
β copy the CSV to Production
β Production dry-run
β Production import
β verify through the Production UI
β remove temporary CSV copies
Do not insert People directly with SQL.
The CSV importer uses the ERS People service/repository path and therefore applies the application's normal validation and duplicate-detection rules.
The importer is transactional and all-or-nothing:
- if any row fails validation, no People are inserted;
-
-dry-runexecutes the same validation/create path but rolls back all inserts; - the real import commits only when the complete file succeeds.
People CSV files may contain highly sensitive information, including:
- CPF;
- RG;
- cellular number;
- email;
- address;
- bank information;
- checking-account information;
- PIX key;
- emergency-contact information.
Therefore:
- do not commit an exported People CSV to Git;
- do not attach it to GitHub issues or pull requests;
- keep temporary copies only as long as necessary;
- transfer it only using the approved server-access mechanism;
- delete temporary server/container copies after successful verification;
- securely archive or delete the local copy according to the applicable data-handling policy.
Do not casually open and re-save the exported CSV in spreadsheet software. Spreadsheet applications can modify values such as CPF, CEP, account numbers, or other text-like numeric fields.
The ERS-exported file can be consumed directly by import-people without spreadsheet modification.
Sign in to the Test environment as the Tenant Administrator for the Tenant whose People will be moved to Production.
Test:
https://tst.enterpriseremotesystems.com
- Sign in through the normal ERS login page.
- Select the correct prospect/customer Tenant.
- Confirm the Tenant shown by ERS before continuing.
- Navigate through the UI to:
People
- Review the People list and confirm that this is the Tenant whose People are intended for Production.
- Click:
Export CSV
or, when using Brazilian Portuguese:
Exportar CSV
- The browser downloads:
ers-people-export.csv
The export contains only People having a Membership in the selected/acting Tenant.
Do not use an export from another Tenant.
The exported file uses the exact canonical CSV structure consumed by import-people.
The header is:
firstName,lastName,nickname,cpf,rg,cellular,email,statusId,notes,street1,street2,city,state,cep,country,bankName,bankNumber,checkingAccount,pixKey,emergencyName,emergencyCellular,emergencyEmailThe export includes portable Person and Membership information such as:
- name;
- CPF;
- RG;
- cellular number;
- email;
- address;
- bank information;
- PIX key;
- emergency contact;
- Membership status;
- Membership notes.
The export deliberately excludes environment-specific or operational identities such as:
Tenant ID
Membership ID
Person database ID
Actor ID
Authentication Account ID
roles
photos
Journeys
Current Account / ledger records
expenses
other operational evidence
Although the column is still named:
statusId
an ERS Bite 32.7 export normally places the stable status code in that column, for example:
ACTIVE
INACTIVE
DISCONTINUED
It does not carry the source Test Tenant's reference-data ID.
During import, ERS resolves that stable status code against the target Production Tenant's own reference data.
Existing manually prepared CSV files that contain an already-valid target-Tenant Person-status ID remain supported, but new ERS exports should use the portable status code.
Do not use:
MineraΓ§Γ£o Serra Dourada β DEMO
as the source of a Test β Production importer validation.
The Brazilian demo fixture intentionally uses CPF values with invalid check digits so that demo values cannot be mistaken for real identity-verification data.
The Brazilian demo may still be used for:
- UI testing;
- pt-BR testing;
- testing the Export CSV action;
- testing CSV structure;
- authorization testing.
It must not be used to prove that a Production import passes CPF validation.
Use the real prospect/customer Test Tenant whose People were created through normal ERS validation.
On the local Mac:
export PEOPLE_CSV="$HOME/Downloads/ers-people-export.csv"
ls -lh "$PEOPLE_CSV"Confirm the canonical header:
head -1 "$PEOPLE_CSV"Expected:
firstName,lastName,nickname,cpf,rg,cellular,email,statusId,notes,street1,street2,city,state,cep,country,bankName,bankNumber,checkingAccount,pixKey,emergencyName,emergencyCellular,emergencyEmail
Count data rows using a CSV parser rather than wc -l:
python3 - "$PEOPLE_CSV" <<'PY'
import csv
import sys
with open(sys.argv[1], newline="", encoding="utf-8-sig") as f:
rows = list(csv.DictReader(f))
print("People rows:", len(rows))
print("Status values:", sorted({row["statusId"] for row in rows}))
PYRecord:
Expected People rows: __________
Status values: __________
Typical status output should contain values such as:
ACTIVE
not Test-environment reference-data IDs.
Compute a checksum before transfer:
shasum -a 256 "$PEOPLE_CSV"Record:
Local CSV SHA-256: ________________________________
Never assume that the Production Tenant ID is:
default
Use the explicit Production Tenant ID.
On the Production server, identify the backend container:
docker psFor the normal Production deployment it may be:
ers-prd-backend
but use the actual running container name.
Set:
export BACKEND_CONTAINER="ers-prd-backend"List Production Tenants:
docker exec -it "$BACKEND_CONTAINER" \
sqlite3 -header -column /app/data/app.db \
"SELECT id, code, name, active FROM tenants ORDER BY name;"Identify the intended Production Tenant.
Set its exact ID:
export PRODUCTION_TENANT_ID='<PRODUCTION-TENANT-ID>'Record:
Production Tenant name: __________________
Production Tenant ID: __________________
Do not continue unless the target Tenant is unambiguous.
Before importing, verify the expected canonical tables exist:
docker exec -it "$BACKEND_CONTAINER" \
sqlite3 /app/data/app.db "
SELECT name
FROM sqlite_master
WHERE type='table'
AND name IN (
'tenants',
'reference_data',
'global_people',
'person_tenant_memberships'
)
ORDER BY name;
"Expected:
global_people
person_tenant_memberships
reference_data
tenants
Then:
docker exec -it "$BACKEND_CONTAINER" \
sqlite3 /app/data/app.db "PRAGMA integrity_check;"Expected:
ok
And:
docker exec -it "$BACKEND_CONTAINER" \
sqlite3 /app/data/app.db "PRAGMA foreign_key_check;"Expected:
no rows
Do not run import-people against an empty or unmigrated database.
The importer does not perform the ERS migration chain itself.
Before import:
docker exec -it "$BACKEND_CONTAINER" \
sqlite3 /app/data/app.db "
SELECT COUNT(*)
FROM person_tenant_memberships
WHERE tenant_id = '$PRODUCTION_TENANT_ID';
"Record:
Production Tenant People count before import: __________
This provides a useful independent verification after the real import.
Take a verified Production database backup before the import.
Using the ERS repository/deployment checkout:
make server-backup ENV=productionThe ERS backup target:
- uses SQLite
.backup; - performs
PRAGMA integrity_check; - performs
PRAGMA foreign_key_check; - copies the verified backup out of the backend container;
- prints the backup SHA-256.
Do not proceed if the backup operation fails.
Record:
Production backup file: __________________
Production backup SHA-256: __________________
From the local Mac:
scp "$PEOPLE_CSV" <production-user>@<production-server>:/tmp/ers-people-export.csvSSH to the Production server:
ssh <production-user>@<production-server>Protect the temporary file:
chmod 600 /tmp/ers-people-export.csvVerify its checksum:
sha256sum /tmp/ers-people-export.csvConfirm that the SHA-256 matches the checksum recorded locally.
Do not continue if the checksums differ.
On the Production server:
docker cp \
/tmp/ers-people-export.csv \
"$BACKEND_CONTAINER":/tmp/ers-people-export.csvProtect it:
docker exec "$BACKEND_CONTAINER" \
chmod 600 /tmp/ers-people-export.csvConfirm that the importer binary exists:
docker exec -it "$BACKEND_CONTAINER" \
ls -l /app/import-peopleThe command must exist at:
/app/import-people
Run the exact exported CSV against the exact target Production Tenant:
docker exec -it "$BACKEND_CONTAINER" /app/import-people \
-db /app/data/app.db \
-file /tmp/ers-people-export.csv \
-tenant-id "$PRODUCTION_TENANT_ID" \
-dry-runExpected:
People CSV dry-run report
Rows read: <expected>
Rows validated: <expected>
Rows inserted: 0
Errors: 0
The following must all be true:
Rows read == expected exported People count
Rows validated == Rows read
Rows inserted == 0
Errors == 0
Do not continue if any error is reported.
Examples of promotion/import blockers include:
CPF is invalid
duplicate CPF
duplicate email
invalid cellular
invalid CEP
invalid UF
invalid country
invalid statusId/status code
wrong CSV header
wrong column count
A dry-run failure does not partially import the successful rows. The transaction is rolled back.
Correct the underlying source data or CSV problem and begin again with a new successful dry-run.
Only after the exact same file passes the Production dry-run with:
Errors: 0
run the real import:
docker exec -it "$BACKEND_CONTAINER" /app/import-people \
-db /app/data/app.db \
-file /tmp/ers-people-export.csv \
-tenant-id "$PRODUCTION_TENANT_ID"Expected:
People CSV import report
Rows read: <expected>
Rows validated: <expected>
Rows inserted: <expected>
Errors: 0
For a new set of People:
Rows read == Rows validated == Rows inserted
Errors == 0
If any row fails, the entire transaction is rolled back.
Run:
docker exec -it "$BACKEND_CONTAINER" \
sqlite3 /app/data/app.db "
SELECT COUNT(*)
FROM person_tenant_memberships
WHERE tenant_id = '$PRODUCTION_TENANT_ID';
"Compare with the pre-import count.
For a file containing only new People:
new count = old count + Rows inserted
Record:
Production Tenant People count after import: __________
Open:
https://app.enterpriseremotesystems.com
Sign in with an authorized Production Tenant Administrator identity.
Select the same Tenant used for:
-tenant-id "$PRODUCTION_TENANT_ID"
Navigate to:
People
Verify representative imported records.
For several People, confirm:
- name;
- CPF;
- RG;
- cellular;
- email;
- status;
- notes, where present;
- address, where present;
- bank information, where present;
- PIX key, where present;
- emergency contact, where present.
Confirm that the imported People belong to the intended Production Tenant.
Do not consider the operation complete solely because the CLI returned Errors: 0; perform the UI verification.
If a database-level verification is required, use the current canonical identity tables.
Do not query the removed legacy people table.
For example:
docker exec -it "$BACKEND_CONTAINER" \
sqlite3 -header -column /app/data/app.db "
SELECT
gp.first_name,
gp.last_name,
gp.email,
m.tenant_id,
rd.code AS status
FROM global_people gp
JOIN person_tenant_memberships m
ON m.person_id = gp.id
JOIN reference_data rd
ON rd.id = m.status_id
WHERE m.tenant_id = '$PRODUCTION_TENANT_ID'
ORDER BY gp.last_name, gp.first_name;
"Use this only when database inspection is actually necessary; normal verification should primarily use the ERS UI.
After the real import:
docker exec -it "$BACKEND_CONTAINER" \
sqlite3 /app/data/app.db "PRAGMA integrity_check;"Expected:
ok
Then:
docker exec -it "$BACKEND_CONTAINER" \
sqlite3 /app/data/app.db "PRAGMA foreign_key_check;"Expected:
no rows
After the Production UI and integrity checks succeed, remove the sensitive CSV from the backend container:
docker exec -it "$BACKEND_CONTAINER" \
rm -f /tmp/ers-people-export.csvRemove the temporary server copy:
rm -f /tmp/ers-people-export.csvVerify:
docker exec -it "$BACKEND_CONTAINER" \
test ! -e /tmp/ers-people-export.csv \
&& echo "Container CSV removed"
test ! -e /tmp/ers-people-export.csv \
&& echo "Server CSV removed"Handle the local copy:
~/Downloads/ers-people-export.csv
according to the approved data-retention policy.
Do not leave unnecessary copies in Downloads, /tmp, container filesystems, Git working trees, or shared locations.
Record the following for the operational record:
Source environment: Test
Source Tenant:
Export date/time:
Exported row count:
Local CSV SHA-256:
Production Tenant name:
Production Tenant ID:
Production backup filename:
Production backup SHA-256:
Production dry-run:
Rows read:
Rows validated:
Rows inserted:
Errors:
Production import:
Rows read:
Rows validated:
Rows inserted:
Errors:
Production People count before:
Production People count after:
Production UI verification: PASS / FAIL
Post-import integrity_check: PASS / FAIL
Post-import foreign_key_check: PASS / FAIL
Temporary CSV cleanup: COMPLETE / INCOMPLETE
Assuming these have already been set correctly:
export BACKEND_CONTAINER="ers-prd-backend"
export PRODUCTION_TENANT_ID="<PRODUCTION-TENANT-ID>"docker exec -it "$BACKEND_CONTAINER" /app/import-people \
-db /app/data/app.db \
-file /tmp/ers-people-export.csv \
-tenant-id "$PRODUCTION_TENANT_ID" \
-dry-runProceed only when:
Rows validated == Rows read
Rows inserted == 0
Errors == 0
docker exec -it "$BACKEND_CONTAINER" /app/import-people \
-db /app/data/app.db \
-file /tmp/ers-people-export.csv \
-tenant-id "$PRODUCTION_TENANT_ID"Proceed to verification only when:
Rows validated == Rows read
Rows inserted == Rows read
Errors == 0
docker exec -it "$BACKEND_CONTAINER" \
rm -f /tmp/ers-people-export.csv
rm -f /tmp/ers-people-export.csvThe supported Production People portability process is:
1. Sign in to Test as the source Tenant Administrator.
2. Select the correct Test Tenant.
3. People β Export CSV.
4. Use the downloaded ers-people-export.csv unchanged.
5. Confirm canonical header, row count, and portable status codes.
6. Calculate and record the CSV SHA-256.
7. Identify the exact Production Tenant ID.
8. Verify Production is using a valid migrated ERS database.
9. Record the Production Tenant's pre-import People count.
10. Take and verify a Production database backup.
11. Securely copy the CSV to the Production host/container.
12. Verify the transferred CSV checksum.
13. Run import-people with explicit -tenant-id and -dry-run.
14. Continue only if Errors: 0 and all rows validate.
15. Run the real import using the same file and Tenant ID.
16. Verify the new People count.
17. Verify representative People through the Production UI.
18. Run database integrity and foreign-key checks.
19. Delete temporary CSV copies.
20. Record the import evidence.
This is an import operation against the preserved Production database, not a Production database seed/reset operation.