• Register
0 votes

Problem :

I have a big dataframe and I am trying to split that and after concat that but facing following error 
first argument must be an iterable of pandas objects, you passed an object of type "dataframe"
6 5 3
6,930 points

1 Answer

0 votes

Solution :

I found soution for you as follows:

df2 = pd.read_csv('et_users.csv', header=None, names=names2, chunksize=100000)
for chunk in df2:
    chunk['ID'] = chunk.ID.map(rep.set_index('member_id')['panel_mm_id'])

df2 = pd.concat(chunks, ignore_index=True)

Please note here you need to append each chunk to a list and after that you need to use concat to concatenate them all, also the ignore_index is not necessary.

9 7 4
38,600 points

Related questions

0 votes
1 answer 1.9K views
Problem : I have the two DataFrames which I would want to merge. I have referred many documents and also tried to perform many operations but I am not sure what to do now. Please find my two DataFrames as below: DataFrame1: id name type currency 0 BTTA.S Apple ... here I met with the exception as below : ValueError: can not merge DataFrame with instance of type <class 'pandas.core.series.Series'>
asked Dec 24, 2019 alecxe 7.5k points
0 votes
1 answer 43 views
Here's what I've got. I have two data frames. One is a set of financial data that already exists in the system and another set that has some that may or may not exist in the system. I need to find the difference and add the ... pandas\core\frame.py", line 3571, in _compare_frame raise ValueError('Can only compare identically-labeled ' ValueError: Can only compare identically-labeled DataFrame
asked Oct 28 psandprop 2.4k points
0 votes
1 answer 3.9K views
Problem : I have the pandas data frame with some of the categorical predictors or variables as 0 & 1, and some of the numeric variables. When I fit that to a stasmodel like below : est = sm.OLS(y, X).fit() It throws the below error : Pandas data ... hundreds of variables. For that I have concatenated the 3 pandas DataFrames to come up with the final DataFrame to be used in the model building.
asked Dec 18, 2019 alecxe 7.5k points
0 votes
1 answer 19 views