CSV Spark connector - CSharplie/ploosh GitHub Wiki
This connector is used to read CSV files using Spark.
See Spark documentation for more information.
No connection is required by this connector
Name | Mandatory | Default | Description |
---|---|---|---|
path | yes | Path to the CSV | |
delimiter | no | , | Column delimiter |
header | no | true | Use the first row as header |
Example CSV Spark:
source:
type: csv_spark
path: data/employees/*.csv
expected:
type: sql_spark
query: |
select *
from employees
where hire_date < "2000-01-01"