• Register
0 votes
493 views

Problem :

I wanted to read all the csv files from my folder and after that concatenate them to create a big csv file. The structure of all my files was the same, After concatenating I want to save it and read it again. All this was done by me using Pandas. But I am facing the Error while reading.

Below is my csv reader code :-

import csv
with open("C:\mywork\DATA\Raw_data\\store.csv",'rb') as csv:
csvreader = csv. csvreader (csv)
l = list(csvreader)

My Error:-

Error                                     Traceback (most recent call last)
<ipython-input-36-9249469f31a6> in <module>()
1 with open('C:\ mywork \DATA\Raw_data\\store.csv', 'rb') as csv:
2     csvreader = csv. csvreader (csv)
----> 3     l = list(csvreader)
Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?
6 5 3
7,540 points

1 Answer

0 votes

Solution :

I also faced this error in recent past, the root cause I found was that there were some carriage returns "\r" in my data that pandas was using as the line terminator similar to "\n". I guess I want to share it here as that may be a common reason behind this error

The solution which I found was to add the lineterminator='\n' into my read_csv function as follows:

df_clean = pd.read_csv('test_myerror.csv',lineterminator='\n')

 

9 7 4
38,600 points

Related questions

0 votes
1 answer 3.1K views
3.1K views
Problem : I have the pandas data frame with some of the categorical predictors or variables as 0 & 1, and some of the numeric variables. When I fit that to a stasmodel like below : est = sm.OLS(y, X).fit() It throws the below error : Pandas data ... hundreds of variables. For that I have concatenated the 3 pandas DataFrames to come up with the final DataFrame to be used in the model building.
asked Dec 18, 2019 alecxe 7.5k points
0 votes
2 answers 497 views
497 views
Problem : I am new to RandomForest model. While predicting my test data using the RandomForest model I am often facing below ValueError. &ldquo;Input contains nan, infinity or a value too large for dtype('float64')&rdquo; I have spent more than two days on the above error but I am unable to fix above error. Can somebody help me in fixing above error?
asked Feb 24 mphil 2.3k points
0 votes
1 answer 1.3K views
1.3K views
Problem : MySample.csv contains the below details : NAME Id No Dept Tommy 1 12 CS Jimmy 2 35 EC Bonny 3 21 IT Franky 4 61 EE And my Python file contains the below code : import csv myifile = open('mysample.csv', "rb") read = csv.reader(myifile) for row in read : ... in for row in read : _csv.Error: iterator should return strings, not bytes (did you open the file in text mode?) How should I fix it?
asked Jan 2 alecxe 7.5k points
0 votes
1 answer 355 views
355 views
Probem : I am getting bellow error new-line character seen in unquoted field - do you need to open the file in universal-newline mode?
asked Nov 7, 2019 peterlaw 6.9k points
0 votes
1 answer 810 views
810 views
Problem : When I run my Python code I get the following exception: _csv.error: iterator should return strings, not bytes (did you open the file in text mode?)
asked Nov 18, 2019 peterlaw 6.9k points