SFTP Configuration Guide

Set up automated file transfer data synchronization for seamless, hands-off importing of student and billing data. This guide covers all three connection modes available on the Data Management page.

Overview

Automated file transfer eliminates manual CSV uploads by syncing data on a schedule. Arbol supports three connection modes, selected via radio buttons on the Data Management page:

Institution Managed SFTP

Default selection. Your institution hosts the SFTP server.

You manage the server - Arbol connects to your SFTP server to pull files
SSH key authentication - Arbol generates a keypair; you add the public key to your server
Your directories - You create and maintain the folder structure on your server

Arbol Managed SFTP

Recommended for institutions without existing SFTP infrastructure.

Arbol hosts the server - You connect to Arbol's SFTP server to push files
Self-service setup - Provision directly from the Data Management page
Firewall protection - Access restricted to your institution's IP address
Ready-made directories - Default folder structure created automatically

Institution Managed FTP

For legacy systems that do not support SFTP.

Standard FTP - Arbol connects to your FTP server with username and password
No encryption - FTP transmits data in plain text

Protocol Recommendation

SFTP is strongly recommended over FTP. FTP transmits credentials and data in plain text, while SFTP provides encrypted file transfer over SSH. Only use Institution Managed FTP if your server does not support SFTP.

Arbol Managed SFTP

Arbol hosts a dedicated SFTP server. Your institution connects to it to upload CSV files. Arbol's scheduled import job then reads those files and processes them.

1 Provision Your SFTP Access

Navigate to Data Management in the staff portal and select Arbol Managed SFTP.

The provisioning panel will appear with these fields:

Field	Description	Example
Primary Client IP / CIDR	Your institution's primary public IP address (or CIDR range) that will connect to the SFTP server. Required.	`203.0.113.50/32`
Secondary Client IP / CIDR	An optional second IP address. Use this if your institution connects from two locations (e.g., primary and backup network). Must differ from the primary IP.	`198.51.100.10/32`
SSH Public Key	Your institution's public SSH key. This is how you authenticate when connecting.	`ssh-rsa AAAAB3NzaC1yc2EAAAA...`

After entering both fields, click Set Up SFTP. Arbol will:

Create a dedicated directory on the SFTP server for your institution
Create the default folder structure (incoming/users/, incoming/school_bills/, processed/, results/)
Register your SSH public key so your institution can connect
Generate a separate internal keypair so Arbol's import job can read your uploaded files
Add your IP address(es) to the server's firewall allowlist

Once provisioning completes, a Provisioned status badge appears along with your connection details:

Field	Value
Host	Provided by Arbol (read-only)
Port	`22` (read-only)
Username	Auto-derived from your school name (read-only)

How Authentication Works

Your institution's key authenticates your SFTP client when uploading files.

Arbol's internal key (generated during provisioning) authenticates the scheduled import job when it reads files from your directory. You do not need to manage this key.

Both keys are registered on the same SFTP user and serve independent purposes. Revoking access removes the SFTP user and both keys together.

SSH Key Security

Keep your private SSH key secure and never share it
Only provide the public key to Arbol
If your private key is compromised, generate a new key pair and re-provision

2 Connect and Upload Files

Use any SFTP client (FileZilla, WinSCP, sftp command line, etc.) with the connection details shown after provisioning.

Your directory structure is ready to use:

/ (root)
├── incoming/
│   ├── users/        # Upload student user CSV files here
│   └── school_bills/ # Upload school bill CSV files here
├── processed/        # Files moved here after processing
└── results/          # JSON result files with import details

Upload your CSV files to the appropriate incoming/ subdirectory. They will be picked up and processed on the next scheduled run.

Directories Are Read-Only in the Portal

For Arbol Managed SFTP, the directory paths are pre-configured and shown as read-only fields on the Data Management page. The default structure matches the directories created during provisioning.

3 Configure Schedules

See Import Scheduling below. Both User and School Bill import schedules are configured on the same page.

4 Revoking Access

If you need to fully disconnect your institution from the Arbol Managed SFTP server, click Revoke Access on the Data Management page. A confirmation dialog will list everything that will be removed:

Your SFTP user account and all registered SSH keys
Your firewall allowlist entry
All files in your S3 directory (incoming, processed, and results)
Arbol's internal sync keypair for your institution

Revocation Is Permanent

Revoking access deletes all uploaded and processed files on the server. Download anything you need before revoking. You can re-provision later, but previous files will not be recoverable.

Institution Managed SFTP

Your institution hosts the SFTP server. Arbol connects to it on a schedule to pull files.

1 Configure Connection

Select Institution Managed SFTP (the default) on the Data Management page, then fill in:

Parameter	Description	Example
Host	Your SFTP server hostname or IP	`sftp.university.edu`
Port	Connection port (typically `22`)	`22`
Username	The SFTP user account Arbol will use	`arbol_service`

2 Generate Arbol's SSH Key

Click Generate Key. Arbol creates an SSH keypair:

The private key is stored securely by Arbol (per-institution, isolated)
The public key is displayed for you to copy

Add the public key to the authorized_keys file for the SFTP user on your server. This allows Arbol to authenticate when connecting to pull files.

How Institution Managed SFTP Works

Arbol generates an SSH key pair when you click "Generate Key"
Arbol securely stores the private key per institution
You copy the public key and add it to your SFTP server
Arbol connects with the private key on schedule to download and process files

This approach avoids sharing passwords. Only Arbol (using your institution's dedicated private key) can connect.

To remove the keypair, click Revoke Key. You should also remove the public key from your server's authorized_keys.

3 Server Requirements

Your SFTP server must have:

SSH access on the configured port
A dedicated user account with read/write permissions
SSH key authentication enabled for that user
All configured directories must already exist on your server (incoming, processed, results). Arbol will not create directories on your server.

Directories Must Pre-Exist

Unlike Arbol Managed SFTP (where directories are created during provisioning), Institution Managed SFTP requires that all configured directories already exist on your server. If a directory is missing when the scheduled import runs, that import type will be skipped and a warning will be logged.

4 Network Requirements

Ensure Arbol's IP addresses can reach your SFTP server:

Firewall rules: Allow inbound connections from Arbol's IPs (provided during setup)
Port access: The configured port must be reachable

Institution Managed FTP

Your institution hosts a standard FTP server. Arbol connects with username and password.

1 Configure Connection

Select Institution Managed FTP on the Data Management page, then fill in:

Parameter	Description	Example
Host	Your FTP server hostname or IP	`ftp.university.edu`
Port	Connection port (typically `21`)	`21`
Username	FTP account username	`arbol_service`
Password	FTP account password	`[secure password]`

FTP Security

FTP transmits credentials and data in plain text. Use Institution Managed SFTP whenever possible. FTP is only available for legacy systems where SFTP is not an option.

2 Server Requirements

FTP access on the configured port
A dedicated user account with read/write permissions
All configured directories must already exist on your server

Directories Must Pre-Exist

Arbol will not create directories on your FTP server. All configured directories (incoming, processed, results) must exist before the scheduled import runs. Missing directories cause that import type to be skipped.

Directory Configuration

All three connection modes use the same directory concept. Two directories are configured at the server level and one directory is configured per import type.

Server-Wide Directories

Directory	Default	Purpose
Processed Directory	`/processed`	Files are moved here after processing to prevent reprocessing
Results Directory	`/results`	JSON result files are written here with import details

Per-Import-Type Directories

Each import type (User and School Bill) has its own Location (remote directory) where Arbol looks for CSV files:

Import Type	Default Location	Purpose
User	`/incoming/users`	Upload student user CSV files here
School Bill	`/incoming/school_bills`	Upload school bill CSV files here

For Arbol Managed SFTP, these directories are created during provisioning and shown as read-only in the portal.

For Institution Managed modes, you must create these directories on your server before enabling the schedule.

Import Scheduling

Both User and School Bill import schedules are visible and configurable simultaneously on the Data Management page (no toggling between them).

Timezone

A timezone dropdown appears above the schedule settings. All schedule times are evaluated in this timezone. The default is Eastern (America/New_York). Available options:

Eastern (America/New_York)
Central (America/Chicago)
Mountain (America/Denver)
Pacific (America/Los_Angeles)

Schedule Options Per Import Type

Each import type is configured independently with:

Setting	Description
Enable schedule	Checkbox to activate scheduled imports for this type
Frequency	Every 6 hours, Daily, or Weekly
Start Hour	Hour of day (0-23) when the import window opens
Start Day of Week	For weekly: which day the import runs
Start Day of Month	Optional: import only activates on or after this day of the month
Send notifications	Whether students receive notifications when this import processes

Student Notifications

The notification checkbox controls whether students receive notifications about their account or billing updates when the import completes. This is the same behavior as checking "Send notifications" during manual uploads.

This does not send notifications to administrators. See Processing Monitoring for how to track import status as staff.

Click Save Configuration after setting up both import types.

Connection Testing

The Data Management page provides two types of testing. Both test against your saved configuration, so save any changes first.

Test Connection

Click Test Connection to verify:

Server connectivity and authentication
Whether each configured directory exists

The results dialog shows a checklist with a green checkmark or red X for each directory:

Processed directory
Results directory
User source directory (if configured)
School Bill source directory (if configured)

This helps diagnose configuration issues before enabling a schedule.

Test Import (Per Type)

Two additional buttons let you run a real import as a one-off test:

Test User Import - Picks up 1 file from the User source directory and runs the full import pipeline
Test Bill Import - Picks up 1 file from the School Bill source directory and runs the full import pipeline

These run the exact same processing that the schedule would, limited to a single file. The first file alphabetically in the configured folder is selected. The file will be moved to processed and a result file will be written to results, just like a scheduled run. Check Import History on the same page for the results.

File Processing

How Files Are Processed

When a scheduled import runs (or a test import is triggered), Arbol performs these steps for each CSV file found in the configured source directory:

Download the file from the remote server
Create an import record in the database
Store a copy in Arbol's internal storage
Queue the import job (User Import or School Bill Import)
Move the file from the source directory to the processed directory
Create a tracking record that links the processed file to its result file

The import job then parses the CSV, validates each row, creates or updates records, and writes a JSON result file to the results directory.

Processed File Naming

When a file is moved from incoming to processed, a UTC timestamp is always appended to the filename to ensure uniqueness:

Original filename	Processed filename
`students.csv`	`students_20260330T140000Z.csv`
`bills_2026-03-30.csv`	`bills_2026-03-30_20260330T140000Z.csv`

This means you can safely upload a file with the same name every day. Each processed copy will have a unique name.

In the rare case of a race condition where the timestamped name already exists, an additional counter is appended: students_20260330T140000Z(1).csv.

Result Files

A JSON result file is written to the results directory for every file processed, including files that fail entirely. The result filename is derived from the processed filename:

Processed file	Result file
`students_20260330T140000Z.csv`	`students_20260330T140000Z_<id>.json`

The <id> suffix is a unique numeric identifier that links the result to the specific import.

Result File Structure

{
  "metadata": {
    "generated_at": "2026-03-30T14:05:00-04:00",
    "import_type": "User",
    "source_file_name": "students.csv",
    "processed_path": "./processed/students_20260330T140000Z.csv"
  },
  "results": {
    "status": "Completed with Warnings",
    "message": null,
    "total_records": 150,
    "success_records": 148,
    "warning_records": 2,
    "error_records": 0,
    "success_rate": 98.67
  },
  "errors": [],
  "warnings": [
    {
      "header": "Row 12 (jsmith@university.edu)",
      "messages": ["SSO Identifier differs from existing value"]
    }
  ]
}

Possible statuses:

Status	Meaning
`Complete`	All rows processed without errors or warnings
`Completed with Warnings`	All rows processed, some had non-blocking warnings
`Completed with Errors`	Some rows failed validation
`Completed with Errors & Warnings`	Some rows failed, others had warnings
`Failed to Upload File`	The file could not be processed at all (download failed, corrupt file, etc.)

Results Are Always Generated

A result file is written even when the import fails completely (for example, if the file cannot be downloaded or parsed). In that case, the result file will have a Failed to Upload File status with an error message in the results.message field and zero record counts.

File Movement

Files are always moved from incoming to processed after processing, regardless of whether rows had errors
If the file cannot be moved (e.g., processed directory missing), the import still runs but the file remains in incoming. A warning is logged.
Files in the processed directory are never processed again

Row-Level Errors Don't Prevent File Movement

A file with 100 rows where 5 rows have validation errors will still be moved to processed. The result file will show which rows succeeded and which failed. This prevents the file from being reprocessed on the next schedule run.

File Lifecycle Example

Scenario

Your institution uploads student data daily and billing data weekly. The timezone is set to Eastern.

Configuration:

User Import: Location /incoming/users, Daily at 2 AM, Notifications enabled
School Bill Import: Location /incoming/school_bills, Weekly on Monday at 6 AM, Notifications enabled
Processed Directory: /processed
Results Directory: /results

Day 1: Upload files

/ (root)
├── incoming/
│   ├── users/
│   │   └── students.csv                        ← Uploaded by institution
│   └── school_bills/
│       └── billing_spring_2026.csv             ← Uploaded by institution
├── processed/                                   (empty)
└── results/                                     (empty)

Day 1, 2:00 AM: User import runs

/ (root)
├── incoming/
│   ├── users/                                   (empty - file picked up)
│   └── school_bills/
│       └── billing_spring_2026.csv             (waiting for Monday schedule)
├── processed/
│   └── students_20260330T060000Z.csv           ← Moved with timestamp
└── results/
    └── students_20260330T060000Z_17119....json  ← JSON result file

Day 2: Upload another file with the same name

/ (root)
├── incoming/
│   └── users/
│       └── students.csv                        ← Same filename, new data
├── processed/
│   └── students_20260330T060000Z.csv           (yesterday's file)
└── ...

Day 2, 2:00 AM: User import runs again

/ (root)
├── incoming/
│   └── users/                                   (empty)
├── processed/
│   ├── students_20260330T060000Z.csv           (Day 1)
│   └── students_20260331T060000Z.csv           ← Day 2 - unique timestamp
└── results/
    ├── students_20260330T060000Z_17119....json  (Day 1 result)
    └── students_20260331T060000Z_17120....json  ← Day 2 result

Each day's file gets its own unique name in processed and its own result file. No conflicts.

Monday, 6:00 AM: School bill import runs

/ (root)
├── incoming/
│   └── school_bills/                            (empty)
├── processed/
│   ├── students_20260330T060000Z.csv
│   ├── students_20260331T060000Z.csv
│   └── billing_spring_2026_20260401T100000Z.csv  ← Moved with timestamp
└── results/
    ├── students_20260330T060000Z_17119....json
    ├── students_20260331T060000Z_17120....json
    └── billing_spring_2026_20260401T100000Z_17121....json  ← Result

Processing Monitoring

Import History

The Data Management page shows an Import History table with all imports (both manual uploads and scheduled FTP imports). Each row shows:

Import type (User or School Bill)
Source (Manual or FTP)
Status
Record counts (total, processed)
Timestamp

Click View Details on any import to see the full error and warning breakdown.

FTP Sync Log

When an active FTP configuration exists, a Sync Log button appears next to the Save Configuration button. This opens a dialog showing a log of all scheduled and test sync runs with:

Date and time of processing
Import type
Source file name
Import status (color-coded)
Record counts (total, success, warnings, errors)
Summary generation status (Written, Pending, or Failed)

Use the Sync Log to confirm your schedule is firing as expected and to trace which files were picked up by each run. This is especially useful for monitoring Arbol Managed SFTP where you push files and want to verify they were processed.

File-Based Monitoring

Monitor your SFTP/FTP server directories directly:

New files in /processed/ indicate the import ran and the file was picked up
New files in /results/ contain the detailed JSON result for each import
Files still in /incoming/ either haven't been picked up yet (schedule hasn't fired) or the directory check failed

No Email Notifications for FTP Imports

Email notifications for FTP import completion are not currently sent to administrators. Monitor import status through:

Staff Portal: Import History on the Data Management page
Server: Check /processed/ and /results/ directories

Email notifications for administrators are planned for a future release.

Automated Monitoring

If you need real-time awareness of import completion, configure your own monitoring to check for new files in /processed/ or /results/.

File Format Requirements

CSV files must follow the format specifications for each import type:

Student Users: See User Format
School Bills: See School Bill Format

File Naming

Arbol processes any .csv file found in the configured source directory. File naming is flexible, but we recommend:

students_YYYY-MM-DD.csv or users_YYYY-MM-DD.csv for user imports
bills_YYYY-MM-DD.csv or billing_YYYY-MM-DD.csv for school bill imports
Avoid spaces in filenames (use underscores or hyphens)

Since Arbol appends a timestamp when moving files to processed, you do not need to worry about filename collisions. You can safely reuse the same filename across uploads.

Manual Intervention

When automated imports encounter issues:

Check the result file in /results/ for the specific error details
Check Import History on the Data Management page and click "View Details"
Fix the source data based on the error messages
Re-upload the corrected file to the incoming directory (it will be picked up on the next schedule run)
Use manual upload as a fallback - the same Data Management page supports direct CSV upload with the same processing

Support

Need help with SFTP/FTP configuration or data imports?

Email: support@growarbol.com
Documentation: Use the navigation menu to explore detailed guides
Data Management Issues: Include the import ID (found under "View Details" in Import History) if one was produced