load_csv_to_postgres_via_copy considers _id and _full_text

[hannelita]

The task load_csv_to_postgres_via_copy considers these two fields when trying to fit the CSV into the table. Find a solution that allows a clean CSV load

[hannelita]

Xloader will have other methods to handle these two fields https://github.com/ckan/ckanext-xloader/blob/master/ckanext/xloader/loader.py , see https://github.com/ckan/ckanext-xloader/blob/master/ckanext/xloader/loader.py#L496.

We will probably need something similar to tackle this issue @rufuspollock

[hannelita]

I tried a workaround which did not work: create a temporary table, export csv to temporary table and insert the contents of the temp. table into the resource table. It did not work because there is a trigger for the _full_text field:

create temporary table t (x1 integer, ... , x10 text)
COPY t (x1, ... , x10) from '/path/to/my_file' with (format csv)


insert into my_table (x2, x5, x7, x10)
select x2, x5, x7, x10
from t



drop table t

(source: https://stackoverflow.com/questions/12618232/copy-a-few-of-the-columns-of-a-csv-file-into-a-table)

[hannelita]

Freezing that for now