• Register
0 votes

Problem :

I have a big dataframe and I am trying to split that and after concat that but facing following error 
first argument must be an iterable of pandas objects, you passed an object of type "dataframe"
6.9k points

1 Answer

0 votes

Solution :

I found soution for you as follows:

df2 = pd.read_csv('et_users.csv', header=None, names=names2, chunksize=100000)
for chunk in df2:
    chunk['ID'] = chunk.ID.map(rep.set_index('member_id')['panel_mm_id'])

df2 = pd.concat(chunks, ignore_index=True)

Please note here you need to append each chunk to a list and after that you need to use concat to concatenate them all, also the ignore_index is not necessary.

36.1k points

Related questions

0 votes
1 answer 851 views
Problem : I have the two DataFrames which I would want to merge. I have referred many documents and also tried to perform many operations but I am not sure what to do now. Please find my two DataFrames as below: DataFrame1: id name type currency 0 BTTA.S Apple ... here I met with the exception as below : ValueError: can not merge DataFrame with instance of type <class 'pandas.core.series.Series'>
asked Dec 24, 2019 alecxe 7.5k points
0 votes
1 answer 2K views
Problem : I have the pandas data frame with some of the categorical predictors or variables as 0 & 1, and some of the numeric variables. When I fit that to a stasmodel like below : est = sm.OLS(y, X).fit() It throws the below error : Pandas data ... hundreds of variables. For that I have concatenated the 3 pandas DataFrames to come up with the final DataFrame to be used in the model building.
asked Dec 18, 2019 alecxe 7.5k points
1 vote
1 answer 37 views
Problem : I am very new to Python. While trying to execute my code I am facing below warning C:\Python27\lib\site-packages\pandas\core\indexing.py:411: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer ... '] = mydf.review_text.map(lambda x: tokenizer.tokenize(x)) print mydf[:3] I am looking for expert help to fix above error.
asked Apr 27 stewart 4k points
1 vote
2 answers 346 views
Problem : I am receiveing error as the truth value of a dataframe is ambiguous. use a.empty, a.bool(), a.item(), a.any() or a.all().
asked Nov 11, 2019 peterlaw 6.9k points
0 votes
1 answer 821 views
Problem : I am getting bellow error attributeerror: can only use .str accessor with string values, which use np.object_ dtype in pandas
asked Nov 7, 2019 peterlaw 6.9k points