PythonSDK mashes columns into one

I’m having trouble using the Python SDK with a new datasource. You can see that all these keys got collapsed into one column on Panoply. Can you recommend a format to use when converting a dataframe to a list of dicts so that this won’t happen?

to_dict

I converted a csv to a dict_list as in:

with open(os.path.join(zipfiledir,file)) as csvfile:
    reader = csv.DictReader(csvfile,delimiter=",")
    for line in reader:
        dict_list.append(line)

When I first tried:

for dict in dict_list:
conn.write( targettables, dict );

I got this mashed-up column in Panoply, so I converted the list of dicts to a DataFrame and back, but I still got the same thing.

I have tried converting the DataFrame to_dict as ‘records’, but that didn’t help.

Alon Brody

February 24, 2019 23:55

This process seems fine. I’ve also checked it with my own CSV (a = 'gcs.txt'):

As you can see in my example I’ve also used json.dumps() on it.

May I ask you to contact support at support@panoply.io with a sample of the CSV you are trying to load in order for us to help with the specific file and the entire script you are using?

Also, if it’s a CSV that you are trying to load why won’t you use one of the file system data sources that we have?

moranbuying

February 27, 2019 10:59

Thanks, I’ll share a sample with support. The reason I need to move the csv to Python before Panoply is that I need to unzip it automatically. I’m trying to read the csv, that I’ve already unzipped and saved locally, back into a dict before using the SDK.

PythonSDK mashes columns into one

Comments

Didn't find what you were looking for?