• Register
0 votes
918 views

Problem :

I wanted to read all the csv files from my folder and after that concatenate them to create a big csv file. The structure of all my files was the same, After concatenating I want to save it and read it again. All this was done by me using Pandas. But I am facing the Error while reading.

Below is my csv reader code :-

import csv
with open("C:\mywork\DATA\Raw_data\\store.csv",'rb') as csv:
csvreader = csv. csvreader (csv)
l = list(csvreader)

My Error:-

Error                                     Traceback (most recent call last)
<ipython-input-36-9249469f31a6> in <module>()
1 with open('C:\ mywork \DATA\Raw_data\\store.csv', 'rb') as csv:
2     csvreader = csv. csvreader (csv)
----> 3     l = list(csvreader)
Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?
6 5 3
7,540 points

Please log in or register to answer this question.

1 Answer

0 votes

Solution :

I also faced this error in recent past, the root cause I found was that there were some carriage returns "\r" in my data that pandas was using as the line terminator similar to "\n". I guess I want to share it here as that may be a common reason behind this error

The solution which I found was to add the lineterminator='\n' into my read_csv function as follows:

df_clean = pd.read_csv('test_myerror.csv',lineterminator='\n')

 

9 7 4
38,600 points

Related questions

0 votes
1 answer 66 views
66 views
Problem: attributeerror: 'dataframe' object has no attribute 'data'
asked Feb 18 charles mathews 5.5k points
0 votes
1 answer 44 views
44 views
Problem : Any clue: Pandas data cast to numpy dtype of object. check input data with np.asarray(data)..
asked Mar 6 Wafa Abu Yousef 6.1k points
0 votes
1 answer 5.6K views
5.6K views
Problem : I have the pandas data frame with some of the categorical predictors or variables as 0 & 1, and some of the numeric variables. When I fit that to a stasmodel like below : est = sm.OLS(y, X).fit() It throws the below error : Pandas data ... hundreds of variables. For that I have concatenated the 3 pandas DataFrames to come up with the final DataFrame to be used in the model building.
asked Dec 18, 2019 alecxe 7.5k points
0 votes
1 answer 11 views
11 views
Problem: how i can move on in dealing with this situation ? Iterator should return strings, not bytes (did you open the file in text mode?.
asked Mar 6 Wafa Abu Yousef 6.1k points
0 votes
1 answer 37 views
37 views
Problem: I am a beginner to python and am at the moment having trouble using the command line. I have a script test.py (which only contains print("Hello.")), and it is located in the map C:\Python27. In my system variables, I have specified python to be C:\Python27 (I ... test.py in the command line, but when I do so I get this: File "<stdin>", line 1 python test.py ^ SyntaxError: invalid syntax
asked Jan 9 Mashhoodch 13k points
0 votes
1 answer 2.9K views
2.9K views
Problem : MySample.csv contains the below details : NAME Id No Dept Tommy 1 12 CS Jimmy 2 35 EC Bonny 3 21 IT Franky 4 61 EE And my Python file contains the below code : import csv myifile = open('mysample.csv', "rb") read = csv.reader(myifile) for row in read : ... in for row in read : _csv.Error: iterator should return strings, not bytes (did you open the file in text mode?) How should I fix it?
asked Jan 2, 2020 alecxe 7.5k points
0 votes
1 answer 720 views
720 views
Probem : I am getting bellow error new-line character seen in unquoted field - do you need to open the file in universal-newline mode?
asked Nov 7, 2019 peterlaw 6.9k points
0 votes
1 answer 2 views
2 views
Problem: I am stuck during learning as I have already mentioned in my question someone please cooperate me to continue my project-Thanks
asked Mar 29 padma1 10.4k points
0 votes
1 answer 3 views
3 views
Problem: I'm using Oracle 10g , SQL Developer I want to know if there is any way to import data from CSV file , which the fields separated by ',' and the lines terminated by '\n' using sql query i tried this query LOAD DATA INFILE 'C:/tmp.csv' ... FIRSTNAME,LASTNAME,BIRTHDATE); But it didn't work and i always get error message from sql Developer telling me Unknow command. Please help me to fix?
asked Mar 26 Shovo210 13.2k points
0 votes
1 answer 28 views
28 views
Problem: I need a quick PHP code that will export my records to a csv file. The fields in the record includes the item_id, item_name, item_description, item_price, item_qty. I needed these headers included in the csv.
asked Oct 31, 2020 RJ Lam 650 points